Compare commits

...
92 Commits
Author SHA1 Message Date
Matthew Jackson e99cb95bd4 v1.7.3: bump version (freemkv-unlock git-pinned for the tag) 2026-09-19 05:31:10 +00:00
Matthew Jackson 1794f5c406 merge: record v1.7.2 release into dev (-s ours; keep path deps + dev's next-cycle work) 2026-09-18 21:39:15 -07:00
Matthew Jackson 9379507c15 docs: backtick item[10]/item[0..5] in mpls module doc so rustdoc doesn't read them as intra-doc links (doc-links CI) 2026-09-18 21:30:25 -07:00
Matthew Jackson 8457efb5da changelog: 1.7.3 — multi-angle MPLS STN fix (#45) + read_structure_files for --share 2026-09-18 21:11:59 -07:00
Matthew Jackson dbc135bee0 disc: add read_structure_files() to capture BDMV/DVD index metadata for --share diagnostics 2026-09-18 21:08:28 -07:00
Matthew Jackson 48fb9b2cf6 mpls: locate STN table past the multi-angle angle block (issue #45)
The PlayItem STN (stream-number) table offset was hard-coded to 32, which
is only correct for a single-angle PlayItem. A multi-angle PlayItem inserts
number_of_angles(1) + flags(1) + (number_of_angles-1) angle references (10
bytes each) right after still_time, so its STN begins at 32 + 2 +
(number_of_angles-1)*10. The fixed offset lands inside that angle block and
misreads the stream counts as garbage (e.g. n_video=51 on Spider-Man 3 UHD
00245/00246.mpls, number_of_angles=2 => real offset 44), dropping every
stream including the primary video. That makes DiscTitle::has_video() return
false, which drops a multi-angle/seamless-branch feature from the disc's nav
candidate set and loses it in main-title selection.

Compute the STN offset from is_multi_angle (bit 4 of item[10]) + number_of
_angles. Add a regression test building a 2-angle PlayItem and asserting the
video stream is found past the angle block.

Validated on a 92-disc real UHD/BD corpus (offline scan): 0 main-feature
pick changes vs baseline, 3 multi-angle titles (Spider-Man 3 244/245/246)
recover has_video. Adds examples/corpus_titledump.rs as the regression tool.
2026-09-18 16:38:45 -07:00
Matthew Jackson 013f27eede fix(disc): SAFETY — spare seamless-branch feature from wrapper-composite demotion (#45)
The fallback composite gate in rank_titles flagged title P as a "play-all
wrapper" whenever some other title Q's clip set was a proper subset of P's,
Q had video, and Q.size >= 0.5 * P.size. main_feature_order then ranks
standalone-before-composite, demoting P below the decoy Q.

On a seamless-branch feature this inverts the pick: the real 00800 (2h03,
4 clips = body + intro/credits/seamless segments) is a SUPERSET of the
single-clip body decoy 00700 (1h50), and 700 >= 50% of 800's size, so 800
was mis-flagged composite and lost to the shorter 700 — the wrong title
gets ripped.

Add a duration-first wrapper-shape gate: P is only a composite when the
subset Q is EITHER a much shorter cut (Q.duration < 0.85 * P.duration, a
concat/play-all padding Q's runtime) OR nearly size-equal (Q.size >= 0.90 *
P.size, a bumper/outro wrapper). A near-full-length but markedly smaller
body subset (Q runs ~0.89 of P's duration yet holds far fewer bytes,
because P's branch segments add bytes not runtime) is a real seamless
branch, so P is kept. Uses per-title duration_secs/size_bytes, stays
capacity-independent and feature-gate-safe.

Preserves the real play-all avoidance: the 00245 bumper-wrapper decoy
(Q ~0.96 of both duration and size) stays demoted via the size limb, and
a concat play-all (subset < half the runtime) via the duration limb.

Add regression tests: seamless_branch_feature_not_demoted_issue_45 and
true_play_all_concat_wrapper_still_demoted. All existing selection tests
pass.
2026-09-18 13:34:50 -07:00
Matthew Jackson 9f0e10a524 restore freemkv-unlock path dep for local dev (post-v1.7.2) 2026-09-18 18:37:24 +00:00
Matthew Jackson b65124728c v1.7.2: bump version (freemkv-unlock git-pinned for the tag) 2026-09-18 18:37:21 +00:00
Matthew Jackson 10d073286c changelog: 1.7.2 — record the SET data-phase unlock fix + READ CAPACITY/empty-ISO hardening 2026-09-18 06:55:00 -07:00
Matthew Jackson 89fd1a6e4d merge: record v1.7.1 main into dev (dev already at 1.7.1; keep path deps) 2026-09-17 23:08:09 -07:00
Matthew Jackson 5c9e2fefe5 comment-guard: trim over-cap comment blocks to caps (comments only, no code change)
Bring every flagged comment block within its cap (inline // <=3 lines,
/// item-doc prose <=8): tighten wording, or convert an over-long inline
block that documents the item below it into a /// doc, preserving all
technical facts. No code, string literals, or logic changed.
2026-09-17 22:47:32 -07:00
Matthew Jackson 041ac5876f test: make case-collision test host-FS-independent
Probe dir_is_case_insensitive against the real target volume by creating
the dir first. Previously the test probed a not-yet-created dir, which
returns a conservative true; that disagreed with extract_tree's own
post-mkdir probe on a case-sensitive host (Linux CI), panicking the test.
Both branches (collision on insensitive, clean coexist on sensitive) still
verified.
2026-09-17 22:37:17 -07:00
Matthew Jackson e0875b630c docs: fix broken intra-doc links
- decrypt.rs: three refs to the #[cfg(test)]-only decrypt_sectors_mapped
  wrapper can't resolve in a non-test doc build; de-link to inline code
  (keeps the name, no broken link).
- Fix DecryptingSectorSource::with_content_ranges link to its full
  crate::sector path so it resolves.
2026-09-17 22:37:17 -07:00
Matthew Jackson d5902ff9f9 deps: sha1 0.10→0.11 (collapse RustCrypto digest/block-buffer/crypto-common dup); drop unused rand 2026-09-17 22:18:07 -07:00
Matthew Jackson 01cd85f514 changelog: open 1.7.2 — UNRELEASED (unified release; firmware ABI v2 in freemkv-unlock) 2026-09-17 14:35:17 -07:00
Matthew Jackson 6de61484a3 docs: point the freemkv-unlock path-dep note at release-orchestrate.yml (release.sh is gone) 2026-09-17 14:24:49 -07:00
Matthew Jackson 764519de43 aacs/disc: test coverage for VID key-derivation + handshake route selection; diag: print unknown mkb_version 2026-09-15 20:37:18 -07:00
Matthew Jackson 2fb7d78b29 libfreemkv: retry READ CAPACITY for ISO sizing; hard-error if unavailable
READ CAPACITY is the only authoritative whole-disc size for an image
(the UDF partition is a subset — measured 288 sectors / ~576 KB short on
a real BD, which would truncate the backup anchor). Its field failures
are sporadic (5/5 reliable on the target drive), so retry it a few times
rather than substituting a short value. On persistent failure the shared
scan/identify path stays lenient (capacity 0; MKV reads by extent) and
image_read_sectors() returns EmptyImage so an ISO fails cleanly instead
of writing a truncated or empty file.
2026-09-15 16:45:48 -07:00
Matthew Jackson 18024ea5f5 scsi/macos: fix io_service_t leak, honor bsd_name in fallback, fix FFI doc
shim_list_drives: reorder the drain loop condition so IOIteratorNext is
not called on the exit iteration, releasing every obtained io_service_t.

shim_open_exclusive: the fallback matcher ignored the requested bsd_name
and could open a different drive; iterate and accept only the candidate
whose IOMedia BSD name matches, else fail. Correct the misleading da_hold
TOCTOU comment.
2026-09-15 14:40:54 -07:00
Matthew Jackson 2287bb4449 labels/pes/disc: overflow + robustness fixes
class_reader: checked pc + size add (guards 32-bit overflow) and remove
the dead BadInstruction variant.

criterion: dedup duplicate stream numbers instead of emitting a number
twice. deluxe: continue past a bad studio attribute instead of aborting
the whole parse. bdmt: treat mixed-content XML (embedded markup) as XML
so it is dropped, without over-rejecting a bare '<' in prose.

pes: grow the frame buffer incrementally instead of an eager
MAX_FRAME_SIZE allocation. extract: give the case-probe a unique random
suffix so concurrent probes cannot collide (TOCTOU-safe).
2026-09-15 14:40:47 -07:00
Matthew Jackson 013cc3ce8d net/error/hex: NAT64 SSRF arm, Dir Display arms, reject leading +
network: add the NAT64 (64:ff9b::/96) arm to the IP classifier -
extract the embedded IPv4 from the last 32 bits and re-check it, matching
the keysources canonical copy, so a NAT64-wrapped private/special
address is blocked like the bare IPv4.

error: add Display arms for DirInsufficientSpace (E9027), DirNameCollision
(E9028), DirWriteFailed (E9029) so their fields render (codes-only, like
the sibling Dir variants).

hex: reject a leading + in the hex int parser.
2026-09-15 14:40:40 -07:00
Matthew Jackson ab2416d07a aacs: propagate MKB walk errors, add bounds/truncation guards, fix FMTS carry
inf: read_mkb_from_drive used .is_ok() after the first pack, silently
truncating the MKB on a mid-walk transport error; propagate the error
instead. Add checked offset guards (94/117) and reject a truncated title
table (161) like the key list already does.

segment: the FMTS key-range carry assumed 2048-aligned segment starts;
use the floor-difference of the actual LBAs so a non-2048-aligned
segment carries correctly.

Annotate css::keyless::recover_title_key as a test-only helper and
correct the clpi_audit dedup-by-PID doc.
2026-09-15 14:40:33 -07:00
Matthew Jackson b735fb3730 mux/codec: PTS carry, buffer cap, EOS core drop, PPS-by-id, dead code
lpcm: carry last PTS + DTS fallback (add last_pts_ns) and propagate
pes.discontinuity instead of hardcoding false, matching adts/flac/
mpegaudio.

pesbuf: add an internal MAX_BUFFERED_BYTES cap (defense-in-depth like
reorder.rs's MAX_GOP) so a misbehaving caller can't grow it unbounded.

dts: drop a truncated trailing core at end-of-stream instead of emitting
it, while keeping genuine trailing extensions.

hevc: resolve num_extra_slice_header_bits from the PPS the slice
references (slice_pic_parameter_set_id), not the last-active PPS.

crc: remove the dead, misnamed crc8_atm (impl was CRC-8/SMBUS, no
callers). Add the missing drop-gate mismatch test; fix the startcode
read_bits contract doc and the h264 SEI comment.
2026-09-15 14:40:26 -07:00
Matthew Jackson f1b7a73b93 mux: harden MKV child loops, read real mp4 sample rate, fix pre-video PTS origin
mkvstream: add the consumed > remaining => MkvSourceInvalid guard (as
BLOCK_GROUP already had) to the INFO/TRACKS/TrackEntry/Video/Audio child
loops, and route the DURATION/SAMPLING_FREQUENCY float reads through the
checked_size cap instead of truncating cs as usize.

mp4/read: audio sample_rate was taken from the mdhd timescale, ignoring
the AudioSampleEntry 16.16 samplerate at the stsd body; read the real
field and fall back to timescale only when it is 0. Drop the unreachable
MAX_TRACKS break (the box scan is already capped) for a debug_assert.

tsmux + m2ts_mux: PTS origin was seeded on video only, so audio
preceding the first video (or an audio-only stream) rebased onto its own
PTS and lost A/V offset; seed the origin from the first frame of any
kind.
2026-09-15 14:40:18 -07:00
Matthew Jackson fbe91895c4 mux: fix filtered-export drop gate, log skipped key-map, restore json fields
demux_sink: dropped_total() folded drops across ALL tracks while
frames_mapped now counts only persisted frames, so a filtered
(sub://audio://) multi-clip export could false-trip SeamPlanDroppedMost /
SinkWroteNothing. Compare drops and mapped frames over the same
persisted-track set via TimelineContinuity::dropped_for; update the
frames_mapped doc to the persisted-only semantics.

resolve: resolve_mux_key_map(...).ok() swallowed the error silently
(TrueHD channel correction skipped with no trace); log it like the
sibling reopen() fix before skipping.

meta_sink: TitleProfile::from_title dropped json:// stream fields
(pid, color/aspect/dimensions/interlaced/mvc, audio secondary/
sample_rate/purpose, subtitle pid/descriptive_service); restore them at
the json document level.
2026-09-15 14:40:09 -07:00
Matthew Jackson 679894e5ea aacs/disc/drive: fix bus-key gate credit, shift guard, tray-unlock log
disc/encrypt: on the disc-keyed cert route, drive_unlocked was derived
from is_drive_unlocker(matched), which is provably always false -
run_bus only ever dispatches the disc unlockers (AACS/DVD), never a
firmware/drive unlocker. Hardcode false with an honest comment; the
genuine drive-unlock case (drive_unlocked, no read_data_key) is the
OEM-VID short-circuit and the bus-key gate already credits it. Gate the
now test-only is_drive_unlocker behind cfg(test) and add a regression
test through resolve_vid_only.

aacs/derive: resolve_dk_node looped 0..u_mask_shift shifting 1u32 << b,
which panics/wraps when the disc/keydb-controlled u_mask_shift is >= 32.
Cap the search at the 32 real bit positions.

drive: unlock_tray dropped its SCSI result with let _; warn! on failure
so a stuck medium-removal PREVENT lock is visible. Fix the stale
probe_disc doc (now a no-op; calibration moved into the unlocker).
2026-09-15 14:39:53 -07:00
Matthew Jackson b7914c5995 decrypt: honor with_content_ranges in the mapped AACS read path
DecryptingSectorSource::with_content_ranges was a silent no-op: the
proactive AACS key_map path early-returned before consulting the content
extents, and decrypt_span ignored its _content argument. A whole-disc
reader (sweep/patch) that installed both a key map and content ranges
therefore decrypted/verified clear UDF/BDMV-nav units, and its
misaligned filesystem reads tripped the AACS unit-alignment gate.

Thread the content extents through decrypt_sectors_mapped_in_content ->
decrypt_span -> apply_aacs_map, skipping any unit whose LBA falls outside
every content range (never decrypted, verified, or counted as loss), and
gate the alignment check on span_touches_content so out-of-content
filesystem reads pass through untouched.

Also drop the redundant is_multiple_of clause in the prefetched
unit-tail guard (remaining > 0 is already guaranteed).
2026-09-15 14:39:31 -07:00
Matthew Jackson 747aeba4a4 error: drop premature E7031 KeyServiceConfig variant
Reverts the cross-repo E7031 addition: it forced enumerating the code across
freemkv test_support + freemkv-i18n's 29 locales for a LOW config-fault nicety.
keysources config faults keep borrowing the transient key-service code until a
coordinated i18n release; decrypt_bus-pub and the SSRF-parity additions stay.
2026-09-15 13:46:06 -07:00
Matthew Jackson 034a8b145b fix: cross-repo integration — widen decrypt_bus, SSRF parity, KeyServiceConfig
- aacs/content: widen decrypt_bus to pub so freemkv-drivetest can invert
  AACS 2.0 bus encryption out-of-tree (was pub(crate), broke that build).
- mux/network: port the 6to4 (2002::/16) and Teredo (2001:0000::/32)
  tunnel-decode arms and the 198.18.0.0/15 + 192.0.0.0/24 blocks from the
  canonical keysources SSRF classifier so both cover the same set.
- error: add distinct KeyServiceConfig variant (E_KEY_SERVICE_CONFIG 7031)
  for the standing key-service misconfiguration fault, separate from the
  transient KeyServiceUnavailable; keysources' online.rs adopts it later.
2026-09-15 13:36:47 -07:00
Matthew Jackson 6720ccdf8b style: cargo fmt the audit changes 2026-09-15 11:02:53 -07:00
Matthew Jackson 24ab408559 test/labels: cover dirimage nesting cap, long_ad dir, paramount feature hint
- dirimage: add a MAX_DEPTH nesting-cap test that walks a real
  deeper-than-cap directory chain and expects DirImageTooLarge.
- udf: add a long_ad (ICB flags = 1) directory read test — the LBA sits
  at AD offset 4, not 12 like extended_ad.
- labels/paramount: derive the feature hint's id and filename from one
  parsed number (canonical 5-digit NNNNN.mpls) so they can't disagree,
  the same fix already applied to fox, and add a hint-parse test.
2026-09-15 11:02:04 -07:00
Matthew Jackson 6e427e2eac test(scsi/linux): cover the fd-reopen path helpers
Add unit tests for to_c_path (single trailing NUL, path bytes preserved
for libc::open) and resolve_to_sg (sg passthrough plus the no-filename
and non-sr/sg fallbacks that need no sysfs) — the helpers the open /
drive_has_disc / recovery reopen path relies on. Verified compiling for
x86_64-unknown-linux-gnu (this module is Linux-only).
2026-09-15 10:58:38 -07:00
Matthew Jackson e130528fea test: cover drive tray/speed rejection, bdnav Call ops, mkv interlace patch
- drive: SET CD SPEED and PREVENT/ALLOW MEDIUM REMOVAL are best-effort; a
  drive rejection must warn, not fail the rip. New test drives a failing
  transport through set_speed/lock_tray/unlock_tray.
- bdnav: CallObject/CallTitle follow the same play path as JumpObject/
  JumpTitle for feature resolution; add tests for both.
- mkv: FlagInterlaced is corrected at finish() to the whole-stream scan
  majority; add a test that a progressive leader on an interlaced feature
  is patched from progressive to interlaced.
2026-09-15 10:56:26 -07:00
Matthew Jackson 0de2e598ef dvdnav(test): cover Vm set-ops (incl divide/mod by zero) and First-Play SetSystem
Add a direct Vm::set test exercising mov/add/sub/mul/div/mod/and/or/xor
and the divide- and modulo-by-zero guards (register left intact, no
panic) plus an unmodelled-op no-op, and a First-Play test that a
SetSystem instruction keeps resolution running so a following JumpTT
still resolves.
2026-09-15 10:51:23 -07:00
Matthew Jackson d0a5a90261 cleanup: remove dead MOBJ cmd cap, add early-break and doc clarifications
- bdnav/mobj: drop the unreachable MAX_CMDS cap (num_cmds is a u16 and the
  span is already bounded against the input) and its flagged NOTE.
- drive/wait_ready: break out immediately on a dead bus (transport
  failure) instead of polling a phantom for the full ~30 s; add a test.
- css/is_scrambled_pack: document that the 0x11 stream_id offset assumes
  zero pack stuffing (true for DVD-Video VOB packs) and why a stuffed pack
  degrades safely.
- scsi/macos_shim: document why releasing g_handle_lock before da_hold is
  TOCTOU-safe (da_hold re-validates the handle under the lock).
2026-09-15 10:49:16 -07:00
Matthew Jackson be296b3901 mux(timeline): tag provisionals with a monotonic epoch counter
passive_offset tagged a provisional offset with epoch_offsets.len() as
the epoch identity, but that length is capped at MAX_EPOCHS. Past the cap
the ring's remove+push leaves len() constant, so a provisional captured
then compared equal to the current identity forever and never retired,
drifting that track off every other one for the rest of the title. Add a
monotonic epoch_seq bumped once per open_epoch and tag provisionals with
it. Add a past-the-cap retirement test.
2026-09-15 10:45:35 -07:00
Matthew Jackson b30b06e04e mpls/ifo: fix STN gate off-by-one and pin the PGC palette chroma order
mpls: the STN table gate required item.len() > STN_OFFSET + 16, one byte
more than the 16-byte header actually spans, skipping a first play item
whose STN header ends exactly at the item boundary. Use >=, with a
boundary test.

ifo: the subtitle palette layout [padding, Y, Cr, Cb] (Cr BEFORE Cb) was
correct but under-documented; harden the comment against a future
byte-swap and add a test that cross-checks the parse against the DvdSub
ycbcr_to_rgb consumer so the two read Cr/Cb the same way.
2026-09-15 10:42:37 -07:00
Matthew Jackson e6d86b3582 labels: fix fox hint field disagreement and deluxe bare studio anchor
fox: playlist_id and filename were derived independently from the raw id,
so a 5-digit id above u16::MAX left playlist_id None while the filename
stayed Some — a half-hint naming different things. Derive both from one
parsed number via a feature_hint() helper with the canonical NNNNN.mpls
filename, and test that the fields agree.

deluxe: parse_studio_attr anchored on a bare find("studio"), which also
matched substudio=/studioId= and the word 'studio' inside another
attribute's value, then read a later attribute's quotes as the studio.
Require a word boundary before it and '=' after, and add anchoring tests.
2026-09-15 10:38:40 -07:00
Matthew Jackson 2a701a217a dirimage: refuse a read that begins past the image's capacity
read_sectors fabricated zeros and reported success for any LBA, including
one at or beyond the image's own capacity — an oversized/crafted offset
that a real file/ISO source rejects. Refuse a read whose START is past
capacity (as read_exact does past EOF) while still zero-padding a batch
that starts in range and overshoots the end, which the mux's final batch
and the FS scan's fixed-LBA probes rely on. Split the read-range test to
pin both behaviours.
2026-09-15 10:33:26 -07:00
Matthew Jackson 90e8e99152 mux(resolve): surface the reopen error instead of swallowing it
The TrueHD channel-correction probe's reopen closure returned Option<S>
via .ok(), collapsing a real io::Error into an indistinguishable None.
Make reopen return io::Result<S> and log the actual error on the failure
arm so a failed re-open is diagnosable, not silent.
2026-09-15 10:33:26 -07:00
Matthew Jackson 1edacf294f unlock: derive unlocker names/classification from the real unlockers
unlocker_names() was a hand-maintained list that could drift from the
unlockers that actually dispatch, and the cert route hardcoded
drive_unlocked:false. Factor the firmware and disc unlocker sets into
single constructors, derive unlocker_names() and a new is_drive_unlocker()
from their real .name(), and set HandshakeResult.drive_unlocked from
whichever unlocker claimed the drive so the bus-key gate credits a drive
unlock even on the cert dispatch path. Add derivation and classification
tests.
2026-09-15 10:27:09 -07:00
Matthew Jackson 7350577d91 disc(extract): fold host-path collisions by the target volume's case rule
The extract collision check always folded names to lowercase, so two
disc files differing only by case (Movie/movie) aborted the extract even
on a case-sensitive volume where they are distinct, coexisting files.
Probe the real target directory's case-sensitivity and fold only when the
host would (APFS/NTFS), defaulting to case-insensitive when the probe
can't run. Update the case-collision test to track the real volume, and
drop a dead duplicate pack-start write in the CSS cross-contamination test.
2026-09-15 10:24:37 -07:00
Matthew Jackson b985618fb5 mux(dts): continue the timeline for a PTS-less PES instead of resetting
When a PTS-less PES arrived with pending_pts invalidated (e.g. after a
forced flush), the fallback fell straight to 0, snapping the timeline
backwards. Add continuation_base_ns() to reach for the projected next-AU
start, then the last PES front, before conceding 0, and stop stashing a
drained buffer's stale front mark as pending_pts. Add a fallback test.
2026-09-15 10:22:04 -07:00
Matthew Jackson d3c4ce4a60 udf: read every FID extent of a directory, honouring the AD chain
read_directory read only the FIRST allocation descriptor of an out-of-
line directory, so a FID list spanning multiple extents was truncated at
the first boundary, and a type-3 continuation descriptor was mis-read as
FID data. Walk the whole descriptor list (types 0/1/2 as allocated FID
extents with the 2-bit type masked out of the length; type 3 followed as
a continuation block, bounded by MAX_AD_BLOCKS), gathering every extent
into one buffer while tracking the declared FID-list length separately
from the sector-rounded buffer. Bound the FID walk and the name read by
that declared length, and read a FID whose 38-byte header ends exactly at
the boundary (<=, not <) while still rejecting one whose name spills past
it. Add a multi-extent directory test.
2026-09-15 10:18:51 -07:00
Matthew Jackson f4a48480df mux(h264): gate open-GOP promotion on all-intra slices and base view
The open-GOP keyframe promotion trusted only the first slice's coding
type and wasn't gated on the base view. A picture whose first slice is I
but whose later slices are P/B is not a clean random-access point, and an
intra MVC dependent view is never an independent anchor. Track every VCL
slice's coding type (all_vcl_intra) and require a real VCL slice, then
promote only when all slices are intra AND !mvc. Add mixed-slice and MVC
non-promotion tests.
2026-09-15 10:06:20 -07:00
Matthew Jackson aad821c8a2 dirimage: count FID-list sectors in the metadata ceiling
The MAX_META_BYTES guard multiplied (dir_count + file_count) by the
sector size, counting one File Entry per node but omitting each
directory's FID-list data sectors. A wide, shallow tree's directory data
can dwarf the per-node entries, so an oversized folder could slip past
the guard and be materialized in full. Gate on metadata_block_count()
instead, which already sums File Entries and FID-list sectors, and add a
test proving the FID-list sectors are counted.
2026-09-15 10:04:12 -07:00
Matthew Jackson 959bba919d mux(demux): count only persisted frames toward frames_mapped
frames_mapped incremented for every mapped frame, including kind-filtered
ones that write to no track file (e.g. video during an audio:// export).
That inflated the denominator so a seam plan dropping every persisted
frame could ship zero-byte files at exit 0. Move the increment inside the
write block so it counts persisted frames only, and add a test where more
frames are seam-dropped than persisted -> SeamPlanDroppedMost fires.
2026-09-15 10:02:53 -07:00
Matthew Jackson ff3e46a560 error: give ImageTruncated a Display arm carrying have/want
ImageTruncated fell through to the bare-code catch-all, dropping the
have/want byte lengths a truncated-resume report needs. Add an explicit
arm emitting them in the identifier-free E{code} have/want form.
2026-09-15 10:01:14 -07:00
Matthew Jackson 14f97aacd6 aacs: fall through a primary DiscRead to the DUPLICATE copy
read_first propagated every non-UdfNotFound error immediately, so a
DiscRead on the primary Unit_Key_RO.inf aborted instead of trying the
/AACS/DUPLICATE/ backup that AACS ships for exactly this case. Treat a
DiscRead as retriable like an absent candidate: continue to the next
path, remembering the error, and only propagate it once every candidate
has been tried (still more informative than a bare AacsNoKeys). Add
fall-through and propagate-after-exhaustion tests.
2026-09-15 10:01:14 -07:00
Matthew Jackson 5c911d1573 mp4: reject hostile 64-bit largesize in find_boxes_capped
find_boxes_capped used plain `pos + box_size` on an attacker-controlled
64-bit largesize cast to usize. A value near usize::MAX at a nonzero pos
wraps past the length guard, producing a release panic on the payload
slice or an infinite loop on the advance. Mirror read_moov: compute the
end once with checked_add, reject on overflow or overrun, and reuse it
for the guard, slice, and advance. Add the hostile-largesize test.
2026-09-15 09:59:35 -07:00
Matthew Jackson cdc1d17993 v1.7.1: bump version (freemkv-unlock git-pinned for the tag) 2026-09-14 03:34:47 +00:00
Matthew Jackson 7b4a0b31d5 docs(changelog): 1.7.1 release notes (LibreDrive unlock fix) 2026-09-13 20:02:14 -07:00
Matthew Jackson a85764da38 ci: bump actions/checkout v4->v7 in identity-guard (dependabot) 2026-09-14 01:30:58 +00:00
Matthew Jackson 5af336776c chore: bump version to 1.7.1 (unified release lockstep) 2026-09-14 01:22:18 +00:00
Matthew Jackson 86c2c3c501 Add Sponsor badge to README 2026-09-06 23:29:33 +00:00
Matthew Jackson c280478ef9 Add FUNDING.yml 2026-09-06 23:25:41 +00:00
Matthew Jackson e7bcef0c45 docs(changelog): record the scsi-only Windows build fix in 1.7.0
The scsi::windows::list_drives fix (build under default-features=false,
features=["scsi"]) ships in 1.7.0; add it to the changelog's Fixed list.
2026-09-05 18:14:36 -07:00
Matthew Jackson 695569d29a scsi(windows): make list_drives self-contained (build under scsi-only)
scsi::windows::list_drives delegated to crate::drive::windows::find_drives(),
but the drive and identity modules are #[cfg(feature = "rip")]. A scsi-only
build (default-features = false, features = ["scsi"]) — which is exactly how
freemkv-firmware consumes libfreemkv — therefore failed to compile: 'could not
find drive in the crate root'. The --all-features CI never saw it (rip is on).

Enumerate directly at the SCSI layer via scsi::open + scsi::inquiry, mirroring
the Linux and macOS backends, which already build their DriveInfo without the
rip-gated modules. The rip-side drive::windows::find_drives() is untouched.

Verified: cargo check --target x86_64-pc-windows-msvc passes both scsi-only and
with default features.
2026-09-05 15:54:45 -07:00
Matthew Jackson 77e4bca234 docs: reconcile unlock-bridge with the current single-unlock dispatch 2026-09-05 12:35:03 -07:00
Matthew Jackson ae8c20f1ca scsi/macos: assert the presence-probe leaves the lock unchanged, not the global bit 2026-09-05 12:35:03 -07:00
Matthew Jackson 086547b78b scsi/linux: cap detached fd-recovery threads on a wedged bridge 2026-09-05 12:35:03 -07:00
Matthew Jackson b09ed299fc drive: log SCSI failures on lock_tray/set_speed instead of dropping them 2026-09-05 12:35:03 -07:00
Matthew Jackson 3f262ba6cb aacs: surface real disc-read errors instead of a blanket AacsNoKeys 2026-09-05 12:35:02 -07:00
Matthew Jackson 3c8fce3031 disc/encrypt: don't fall through to the AACS cert unlocker on a firmware-claimed drive
A drive claimed by a firmware unlocker (freemkv/LD/Renesas) at init() is that
unlocker's sole VID source. When it stashed no VID, do_handshake_cert now returns
(None, None) instead of running the AACS cert route (run_bus's AacsUnlocker) whose
REPORT KEY AGID cycle poisons the drive's AGID selector so every later bare 0xAD
VID read aborts until the medium is reloaded. A key source can still supply the key.

Adds a red-before-green test proving the guard issues zero SCSI on that path.
2026-09-03 07:41:43 -07:00
Matthew Jackson 5578e42961 Cargo: restore dev path-dep for freemkv-unlock
The dev branch tip must path-dep ../freemkv-unlock so CI's sibling checkout
(and local dev) resolves the branch tip; release.sh swaps it to a git tag
only in the tagged commit. dev had been left in the released git-tag
(v1.6.14) form after the last release, so the new 1.7.0 unlock API couldn't
resolve. Restore the path form the comment already describes.
2026-09-02 18:02:26 -07:00
Matthew Jackson a7db00c0a1 unlock_bridge: drive unlockers through the unified single-unlock contract
Adopt freemkv-unlock's new Unlocker::unlock(). The bridge assembles the
firmware set (freemkv / LD / Renesas) for drive-prep and the disc set
(AACS cert + DVD) for bus removal, injecting host certs into the AACS
unlocker at construction — the loop no longer knows any unlocker's guts.

- run_features / run_bus now loop unlock() and return Ok(Option<Unlocked>);
  drive init and do_handshake_cert adapt to the Option.
- unlocker_matrix reports the LD name and credits Renesas as a drive-level
  (raw-read) route, so AACS shows "no" when any firmware route took the bus.
- Cert route sets HandshakeResult.drive_unlocked=false: the bus is removed
  via read_data_key (AKE), not at the drive; the gate credits it that way.
2026-09-02 17:53:33 -07:00
Matthew Jackson 09957e066b ci: identity-guard — block unrelated-corp / non-canonical committer emails 2026-09-02 14:46:16 -07:00
Matthew Jackson a88b7486ac release: 1.7.0
Fixed
- mux/mp4: honour 64-bit largesize (size==1) and size==0 (run-to-end) in the
  capped child-box scan; a largesize sibling previously ended the walk early and
  later boxes vanished from a MOV/MP4.

Version 1.6.15 (unreleased) -> 1.7.0; changelog folded under 1.7.0.

Earlier 1.7.0 work remains as its own commits below this one: the
GHSA-j8ww-f5fg-9pmh sector-recycle soundness fix, CI/lint hardening (#15/#17),
docs, and Rob Ekl's scsi raw_command / fd-handoff change (#16).
2026-09-02 12:40:55 -07:00
Rob Ekl 6957c7251c scsi: route raw_command through checked_cdb_len; add the fd hand-off release edge (#16)
raw_command clamped CDBs with cdb.len().min(16) instead of routing through
the shared checked_cdb_len guard, so an over-length or empty CDB became a
different command rather than a rejected one; it now returns the crate Result
so a rejected CDB is distinguishable from an I/O failure.

The recovery/Drop fd hand-off lacked a release edge: the teardown swap was not
a release op and the recovery CAS was not an acquire load, so a recovery thread
publishing its fd after Drop drained the slot could miss dead==true and leak the
descriptor. Both sides are now AcqRel, with the execute() drain left at Acquire
(sound via the release-sequence rule). A loom model (cfg(loom) dev-dep, its own
CI job) provides the regression coverage cargo test cannot.

Module docs for fd_handoff and timeline now resolve in their own scope, and item
docs stay within comment-guard's caps.

Fixes #16.
2026-09-02 12:40:55 -07:00
Matthew Jackson 6e14fc0c0c libfreemkv: 1.7.0 groundwork — sector-recycle soundness + CI hardening
Security
- Remove the unsafe Vec::set_len over recycled sector buffers — UB reachable
  from safe code (GHSA-j8ww-f5fg-9pmh, low). Uses Vec::resize; pooling unchanged.

CI / tooling
- clippy on stable with the MSRV single-sourced from Cargo.toml; cargo-deny
  dependency-audit gate with first-party git sources allowed (#15); audience-based
  comment-guard (#17).

Docs
- README (keydb ownership, BD-J parser count), drive-access transport/reset
  notes, mux timeline module-doc links.
2026-09-02 12:40:55 -07:00
Matthew Jackson 962aa5d852 v1.6.14: bump version (freemkv-unlock git-pinned for the tag) 2026-08-31 23:04:25 +00:00
Matthew Jackson 719b63e376 chore: reconcile v1.6.13 release commit from main into dev 2026-08-31 15:48:53 -07:00
Matthew Jackson 3cf7922574 docs(changelog): 1.6.14 entry 2026-08-31 15:32:40 -07:00
Matthew Jackson 0c953e9340 chore(msrv): floor to 1.88, pin CI on the floor, allow uninlined_format_args
Declared rust-version and the ci.yml/qa.yml toolchain pins both sit at 1.88 (the
edition-2024 floor once zip 8 and let-chains are accounted for), so CI builds on
exactly the MSRV: an accidental bump to a newer toolchain fails CI by design and
moving the floor takes a deliberate pin edit. The uninlined_format_args allow
reconciles 1.88's clippy with the allow-by-default it became in 1.97+.
2026-08-31 11:06:35 -07:00
Matthew Jackson 2e90a7585d drive: move --share capture orchestration to freemkv-engine
libfreemkv keeps only generic SCSI primitives; the app-specific drive-info
capture (INQUIRY / GET_CONFIG / vendor buffers) now lives in its consumer.
2026-08-31 10:08:36 -07:00
Matthew Jackson 3e11225389 audit: resolve remaining v1.6.10..HEAD findings (parallel fan-out)
Correctness/robustness:
- scsi/macos_shim: guard g_handle / g_da_bsd read-modify-write with a
  process-global mutex (scoped OUT of the 5 s async-claim wait) — closes the
  DiskArbitration global-state TOCTOU on concurrent/re-entrant open+close.
- disc/profile: the public DiscProfile now surfaces `encrypted` and a numeric
  `key_error` (Error::code of aacs/css failure) so a disc with failed key
  resolution no longer serializes identically to a rippable one.

Performance:
- disc/dvd: read VIDEO_TS.IFO once and reuse it for both parse_vmg and
  resolve_main_title (parse_vmg_with / resolve_main_title gain an optional
  vmg_bytes) — eliminates a duplicate VMGI read on every DVD scan.

Tests:
- bdnav: MAX_OBJECTS count-cap rejection, MAX_STEPS non-convergence guard, and
  an end-to-end index.bdmv + MovieObject.bdmv resolver test (all red-green).
- drive/capture: cover the Renesas "SAT"-signature capture gate (match + miss).

Craft/docs:
- css/keyless: factor the duplicated LFSR0 output-tap into one helper.
- drive/capture: name the Renesas vendor CDB constants.
- labels/deluxe: use the named ANEWARRAY/NEWARRAY opcodes, not bare hex.
- dvdnav/vmcmd: tighten pub -> pub(crate). dvdnav/mod: mark the parked stub.
- disc/bluray: document that whole-clip extents are intentional (trimming is
  PTS-level in SeamPlan). disc/mod + error: doc/comment drift corrected.
2026-08-31 08:36:08 -07:00
Matthew Jackson 82511d84b6 audit: resolve macOS shim lifetime bug + close coverage/DRY gaps
- scsi/macos_shim: the DA claim used a stack DAClaimResult passed to an async
  callback and never released its semaphore — on the 5 s timeout that is both a
  use-after-free (the callback outlives da_hold) and a semaphore leak per drive
  open. Heap-allocate it, reference-count it (rc=2), and let whichever side
  finishes last release the semaphore and free. (Compiles; needs on-hardware
  DA validation.)
- bdnav/index: add a regression test that a title count over MAX_TITLES is
  rejected before any record read (untrusted index.bdmv count).
- dvdnav/nav: share ifo::MAX_TT_SRPT_TITLES instead of a second private copy so
  the two 99-title caps can never diverge.
2026-08-31 08:11:26 -07:00
Matthew Jackson 20afbfdccb audit: fixes from the v1.6.10..HEAD codeaudit sweep
Correctness:
- dvd: join the nav main-feature target on the REAL vts_title_num (carried on
  DvdTitle) instead of the title's position in the vec, which desyncs when a
  sibling PGC is dropped as unparseable or TT_SRPT is non-monotonic.
- diag: the "demoted: no-video" dump now gates on has_probable_video (the gate
  main_feature_order actually uses), not the stricter has_video.

Robustness:
- labels/deluxe: clinit_enum_field_names now caps retained bytes
  (MAX_CLINIT_LDC_BYTES), like its sibling, so a malicious BD-J .class with a
  huge CONSTANT_Utf8 field name can't grow the output unbounded.

Mux (FlagInterlaced correction cleanup):
- factor the PGS/FlagInterlaced deferred byte patches through one patch_bytes
  helper; name the InterlacedRewrite record; add ebml::void_element (validated)
  for the demotion Void; bound the muxed_* test scans to the pre-Cluster region.

Tests:
- add the FlagInterlaced majority tie-break test (tie stays progressive).
- de-vacuum the profile subtitle-count and meta_sink channel-count assertions.

Docs:
- clpi module doc described an EP-map/sector-range parser that was never
  implemented; corrected to what the code actually reads.
2026-08-31 07:51:05 -07:00
Matthew Jackson 6d78a36d22 lib: trim feature-gate comment blocks to satisfy comment-guard 2026-08-30 22:41:40 -07:00
Matthew Jackson a9d41f53f8 mux: correct FlagInterlaced from the whole-stream scan majority
FlagInterlaced was written from only the FIRST coded picture, but MPEG-2
progressive_frame is a per-picture flag: a progressive leader on an
interlaced feature (or an interlaced first picture on progressive film)
mislabeled the whole track, so players either deinterlaced progressive
frames or left interlaced frames combed.

Tally every picture's measured scan and correct the reserved FlagInterlaced
byte to the dominant scan at finish() -- the same reserve-then-patch idiom as
PgsForcedFixup and the AC-3/duration fixups. On a demotion to progressive the
now-stale FieldOrder element is Void'd. Also pins the interlaced direction at
the muxed-byte level and adds promote/demote regression tests.
2026-08-30 22:38:51 -07:00
Matthew Jackson c7c53989f7 Feature-gate the crate: scsi (transport-only) vs rip (full)
Split the crate into two additive features so a transport-only consumer
(freemkv-flash) can depend on the SCSI passthrough layer without pulling the
ripping dependency tree (serde, zip, rayon, aes, freemkv-unlock, …):

- scsi = the SCSI passthrough transport (src/scsi/ + error): ScsiTransport,
  CDB builders, per-OS backends + the macOS cc shim, sense parsing. Pulls only
  tracing + the always-on libc target dep.
- rip  = the full ripping tree (drive/disc/UDF/MPLS/CLPI/AACS/CSS/mux/keydb +
  the freemkv-unlock seam) and all its heavy deps. rip implies scsi.
- default = ["rip", "scsi"], so the full build stays byte-for-byte identical for
  the existing consumers; transport-only users set default-features = false,
  features = ["scsi"].

All ripping modules are gated #[cfg(feature = "rip")]; error + scsi stay always
compiled. Two read-path timeout consts are rip-gated so a transport-only build
does not warn on unused.
2026-08-30 22:10:05 -07:00
dependabot[bot] 7ec2dbfbcb build(deps): bump codecov/codecov-action from 5 to 7
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 7.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-30 21:53:05 -07:00
avon662 16e5b5ee38 Write FlagInterlaced from the measured picture, not the declared resolution
MkvTrack::video sets `interlaced` from the source scan's DECLARED
resolution. An NTSC DVD's IFO declares 480i whether or not its pictures
are coded progressive -- film and animation routinely are -- so
progressive content ships as FlagInterlaced=1, and every player that
honours the flag runs a deinterlacer over progressive frames, softening
every frame of an otherwise bit-exact remux.

The measurement was already there and already trusted for the sibling
property. `apply_coding_to_track` receives the first coded picture's
PictureInfo and derives FieldOrder::Progressive from it
(progressive_sequence / progressive_frame, ISO/IEC 13818-2 6.3.5 /
6.3.10) -- it used that verdict only to clear FieldOrder and then
discarded it. It now also corrects the scan type, exactly as a measured
TFF/BFF already overrides the declaration.

Measured on a real NTSC DVD: ffmpeg's `idet` reports 100% progressive
(multi-frame detection: 0 TFF, 0 BFF, 0 repeated fields) on titles that
shipped FlagInterlaced=1. progressive_frame is set on every picture
coding extension in the stream while progressive_sequence stays 0, which
is the ordinary shape for a film or animation DVD.

Video payload is unaffected -- byte-identical before and after; only the
track header changes.
2026-08-30 21:47:19 -07:00
Matthew Jackson d40540ce9e v1.6.13: bump version (freemkv-unlock git-pinned for the tag) 2026-08-28 15:48:58 +00:00
Matthew Jackson ba14d66c18 Release 1.6.13: capture additional drive buffers during info --share 2026-08-27 21:22:11 -07:00
Matthew Jackson f16abe1cf4 restore freemkv-unlock path dep for local dev (post-v1.6.12) 2026-08-27 22:57:33 +00:00
Matthew Jackson 477c0da6dd v1.6.12: bump version (freemkv-unlock git-pinned for the tag) 2026-08-27 22:57:29 +00:00
Matthew Jackson 41c5b8b785 changelog: 1.6.12 (unreleased) 2026-08-27 13:11:27 -07:00
Matthew Jackson 9f43ae94d3 Comment and Documentation Cleanup
- Cap every inline `//` comment block at 3 lines; distil the rationale
  rather than delete it. Enforced by a new comment-guard CI check.
- Fix stale and orphaned rustdoc: broken intra-doc links (renamed or
  removed items), doc blocks that had merged onto the wrong item (leaving
  the intended item undocumented), and doc-contradicts-code drift.
- Add a doc-links CI check (cargo doc, deny broken intra-doc links) so
  both classes of comment rot are caught going forward.
2026-08-27 12:07:53 -07:00
Matthew Jackson 9ed6f86255 Fix macOS optical E1000: hold a DiskArbitration claim for the session
The macOS SCSI transport took exclusive SCSI access plus a one-shot
`diskutil unmountDisk`, but never claimed the disc from DiskArbitration.
diskarbitrationd was therefore free to remount the volume mid-rip (a
Spotlight re-probe is the usual trigger), yanking the device out from
under an in-progress read; the failure surfaced to the user as E1000.

Hold a DADiskClaim plus a mount-approval dissenter scoped to our disc for
the transport lifetime, so the OS cannot remount it until shim_close().
Link the DiskArbitration framework in build.rs.
2026-08-27 11:17:01 -07:00
Matthew Jackson 70ae7b6c10 restore freemkv-unlock path dep for local dev (post-v1.6.11) 2026-08-25 23:17:11 -07:00
336 changed files with 36729 additions and 30226 deletions
+1
View File
@@ -0,0 +1 @@
github: MattJackson
+132 -20
View File
@@ -11,6 +11,25 @@ on:
pull_request:
jobs:
# ── Single source of truth for the MSRV ─────────────────────────────────────
#
# `rust-version` in Cargo.toml is the ONE place the MSRV is declared. This job
# reads it; every job that must build/test AT the MSRV consumes
# `needs.resolve-msrv.outputs.v` instead of a hardcoded pin, so the floor
# cannot drift out of sync across workflow files. The `test` job building
# green at this value IS the verification that the declared MSRV is real.
# clippy and the `consumers` job deliberately do NOT use it — they run on
# `stable` (lint debt is caught against current Rust; dependents float highest).
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
@@ -38,6 +57,7 @@ jobs:
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
@@ -57,25 +77,37 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
# fmt runs on the pinned MSRV toolchain; clippy runs on stable. Third
# parties compile our source on their own toolchain and never run our
# clippy, so the lint toolchain need not match the MSRV — only build/test
# must (see the test/cross jobs). Linting on stable makes new-lint debt a
# BLOCKING gate the pinned version would otherwise hide. The MSRV pin is
# installed second so it stays the default toolchain (fmt uses it
# implicitly); clippy is invoked explicitly with `+stable`.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy, rustfmt
components: clippy
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
# Cargo.lock is now committed (issue #15) and the `deny` job audits the
# resolved graph, but --locked is still omitted: CI's sibling path-patch
# layout resolves the freemkv crates as path packages, which --locked
# can't satisfy — the same reason the binary crates omit it in ci.yml.
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
- run: cargo +stable clippy --all-targets -- -D warnings
working-directory: libfreemkv
test:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
@@ -95,13 +127,84 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
# The memory orderings in `scsi::fd_handoff` — the fd hand-off between a
# recovery thread and `SgIoTransport::drop` — cannot be checked by `cargo
# test`: they compile to the same instructions on x86_64, so the WRONG ones
# pass every run. loom enumerates the executions the C++20 model permits and
# reports the leaked descriptor directly, which makes it the only regression
# coverage that part of the module has. If this job stops running, that
# coverage disappears silently behind a green build.
#
# Its own job rather than a step in `test` because `RUSTFLAGS` is part of the
# rust-cache key: folding it into `test` costs that job a full cache miss on
# every run. It takes the same sibling checkout as the jobs above even though
# `--no-default-features` drops freemkv-unlock from the BUILD — `--features`
# does not exempt an optional path dependency from manifest RESOLUTION, so
# without the sibling this job would fail the moment the branch tip carries
# the path-dep form of that dependency rather than a git tag.
loom:
needs: resolve-msrv
runs-on: ubuntu-latest
env:
RUSTFLAGS: --cfg loom
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --lib --no-default-features --features scsi fd_handoff
working-directory: libfreemkv
# `lint` runs clippy without `--cfg loom`, so this is the only place the
# model code is linted at all.
- run: cargo clippy --lib --no-default-features --features scsi --profile test -- -D warnings
working-directory: libfreemkv
deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@stable
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check advisories bans licenses sources
manifest-path: libfreemkv/Cargo.toml
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
@@ -115,6 +218,7 @@ jobs:
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
@@ -135,7 +239,9 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -155,6 +261,7 @@ jobs:
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
@@ -175,7 +282,9 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -230,10 +339,11 @@ jobs:
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
# The consumers span a higher MSRV than libfreemkv itself (the bins are
# 1.98, keysources 1.97), so build them with a toolchain that satisfies the
# HIGHEST dependent floor — libfreemkv's own 1.94 cannot compile them.
- uses: dtolnay/rust-toolchain@1.98.0
# The dependents span a higher MSRV than libfreemkv itself — the binaries
# (freemkv/autorip/bdemu) always track the highest supported Rust. Build
# them on `stable` so this job satisfies the highest dependent floor; the
# library's own lower MSRV (verified by the `test` job) cannot compile them.
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
@@ -261,14 +371,15 @@ jobs:
#
# Reports line coverage to Codecov. Deliberately NOT wired into any release
# gate: it is informational only (see codecov.yml) and no other job `needs:`
# it. Mirrors the `test` job's layout exactly — same pinned 1.94 toolchain,
# same dual checkout so the `../freemkv-unlock` path dep resolves, same cache
# it. Mirrors the `test` job's layout exactly — same MSRV toolchain (resolved
# from Cargo.toml), same dual checkout so `../freemkv-unlock` resolves, cache
# workspace and working-directory. cargo-llvm-cov comes as a prebuilt binary
# from taiki-e/install-action, so it is toolchain-independent.
#
# No `--locked`: like the fmt/clippy/test jobs above, this crate's Cargo.lock
# is gitignored, so there is nothing to lock against on a fresh runner.
# No `--locked`: like the fmt/clippy/test jobs above, the sibling path-patch
# layout resolves the freemkv crates as path packages, which --locked can't satisfy.
coverage:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
@@ -279,8 +390,9 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
@@ -288,7 +400,7 @@ jobs:
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v5
- uses: codecov/codecov-action@v7
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
+17
View File
@@ -0,0 +1,17 @@
name: comment-guard
# Caps explanatory prose by COMMENT INTENT (not visibility):
# //! / /*! module docs — unbounded (module-scoped, can't hide per-item text)
# /// / /** */ item docs — 8 prose lines (fenced ``` examples excluded)
# // / /* */ asides — 3 lines (single-blank-separated runs count as one)
# Overflow belongs in the module //! or docs/<topic>.md with a one-line // pointer.
on: [push, pull_request]
jobs:
comment-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Run comment-guard
run: python3 ci/comment-guard.py
+32
View File
@@ -0,0 +1,32 @@
name: doc-links
# Fails on broken intra-doc links in `///`/`//!` doc comments (e.g. a `[Type]`
# link to a renamed/removed item). Catches doc drift that `cargo build` misses.
#
# libfreemkv path-deps ../freemkv-unlock on the branch tip, so — like ci.yml —
# both repos are checked out into sibling subdirectories for the path dep to
# resolve.
on: [push, pull_request]
jobs:
doc-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- name: cargo doc (deny broken intra-doc links)
working-directory: libfreemkv
env:
RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links"
run: cargo doc --no-deps --document-private-items --lib --bins
+41
View File
@@ -0,0 +1,41 @@
name: identity-guard
# Prevention: no commit may carry an unrelated-business or non-canonical author/
# committer email. dev@example.com (a dead corp address) once slipped onto the
# release cascade + a batch of local commits; this fails the build if it recurs.
on:
push:
branches: [dev, qa, main]
pull_request:
jobs:
identity:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Reject banned author/committer emails
env:
BEFORE: ${{ github.event.before }}
AFTER: ${{ github.sha }}
EVENT: ${{ github.event_name }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
PR_HEAD: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
# Unrelated business domains + a personal address normalized to the
# GitHub noreply. Real external contributors are unaffected.
BANNED='@example-a\.invalid|@example-b\.invalid|owner@example\.invalid'
if [ "$EVENT" = "pull_request" ]; then
range="$PR_BASE..$PR_HEAD"
elif [ -n "${BEFORE:-}" ] && [ "$BEFORE" != "0000000000000000000000000000000000000000" ] && git cat-file -e "$BEFORE" 2>/dev/null; then
range="$BEFORE..$AFTER"
else
range="$AFTER~1..$AFTER"
fi
bad=$(git log --format='%an <%ae>%n%cn <%ce>' "$range" 2>/dev/null | sort -u | grep -iE "$BANNED" || true)
if [ -n "$bad" ]; then
echo "::error::commit(s) carry a banned author/committer identity:"; echo "$bad" | sed 's/^/ /'
echo "::error::Commit as your GitHub noreply (1085847+MattJackson@users.noreply.github.com). Fix: git config user.email <noreply>; git commit --amend --reset-author."
exit 1
fi
echo "identity-guard: clean"
+24 -3
View File
@@ -24,6 +24,19 @@ on:
workflow_dispatch:
jobs:
# Single source of truth for the MSRV — see ci.yml. Jobs that build/test AT
# the MSRV read `needs.resolve-msrv.outputs.v` from Cargo.toml's rust-version;
# clippy (cross-lint) runs on stable instead.
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
@@ -65,6 +78,7 @@ jobs:
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
needs: resolve-msrv
strategy:
fail-fast: false
matrix:
@@ -79,7 +93,9 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -100,7 +116,9 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
# clippy runs on stable (matches ci.yml's lint job); build/test stay on
# the MSRV pin. See ci.yml for why the lint toolchain need not be the MSRV.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
@@ -115,6 +133,7 @@ jobs:
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
needs: resolve-msrv
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
@@ -125,7 +144,9 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
+10 -4
View File
@@ -29,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.94, same commit). Binary consumers
# gated by the local precommit (same MSRV, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -38,10 +38,16 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.94.0
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
# Cargo.lock is now committed (issue #15), but --locked is still omitted:
# the sibling path-patch layout resolves the freemkv crates as path
# packages, which --locked can't satisfy.
- run: cargo test
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
-1
View File
@@ -1,5 +1,4 @@
/target
Cargo.lock
*.swp
*.swo
.DS_Store
+66 -1
View File
@@ -1,6 +1,71 @@
# Changelog
## [Unreleased]
## [1.7.3] — 2026-09-19
### Fixed
- MPLS title enumeration: the STN (stream number) table of the first PlayItem is now located past the multi-angle *angle block* instead of at the fixed offset 32. A multi-angle first PlayItem (e.g. a seamless-branch UHD title) carries `number_of_angles` extra 10-byte angle entries before the STN table, so the old fixed offset read the wrong bytes and misparsed the primary title's streams — the root cause behind autorip/CLI picking the wrong (or no) main feature on some UHD discs. Validated against a 92-disc real-media corpus with zero regressions (issue #45).
### Added
- `disc::read_structure_files()` — captures a disc's **structure metadata** (`BDMV/index.bdmv`, `MovieObject.bdmv`, `PLAYLIST/*.mpls`, `CLIPINF/*.clpi`, `BDJO/*.bdjo`, `META/DL/*.xml`, DVD `VIDEO_TS/*.IFO`) over any `SectorSource` (drive, ISO, `dir://`). No audio/video essence and no AACS keys are read, so the result is safe to attach to a bug report (a few hundred KB). Powers `freemkv info … --share` so a reporter can reproduce a title-selection issue without shipping the full ISO.
## [1.7.2] — UNRELEASED
### Fixed
- ISO imaging: the READ CAPACITY (disc-size) query is now retried, with a UDF partition-size fallback, and `image_read_sectors()` hard-errors (`EmptyImage`) when the size is unavailable instead of writing a silent 0-byte ISO reported as success. Fixes an intermittent empty ISO caused by a transient capacity-query transport failure (shared CLI/autorip engine).
### Changed
- Unified release with freemkv-unlock 1.7.2 (firmware ABI v2).
## [1.7.1] — 2026-09-14
### Changed
- Unified release with freemkv-unlock 1.7.1 (LibreDrive profile-match fix). No functional changes to this crate.
## [1.7.0] — 2026-09-02
### Fixed
- `mux/mp4`: the capped child-box scan treated any `size < 8` as a stop, which dropped a 64-bit `largesize` child box (`size == 1`, real length in the following 8 bytes) and every sibling after it; a `size == 0` (run-to-end) box was mishandled too. Both are now honoured, so an MP4/MOV whose boxes use 64-bit sizes is walked fully instead of truncated.
- Linux: `SgIoTransport::raw_command` clamped an over-length CDB with `cdb.len().min(16)` instead of routing it through the shared `checked_cdb_len` guard that `execute()` uses. Under SPC-4 a CDB's length is fixed by its opcode, so truncating one does not shorten the command — it sends a descriptor the drive will read as something else. An empty CDB was likewise unguarded, and went to the sg driver as a zero-length command descriptor. Unreachable — `raw_command` is private and its one caller passes a fixed 6-byte ALLOW MEDIUM REMOVAL — but it was the one transport path that did the thing the guard exists to prevent.
- Linux: the fd hand-off between a background recovery thread and `SgIoTransport::drop` had no release edge, so a recovery thread that published its fd after teardown had drained the slot was not guaranteed to observe `dead == true` and could leave the descriptor unclosed. `Drop`'s claiming `swap` was `Acquire` (a relaxed store, heading no release sequence) and the recovery thread's `compare_exchange` was `Release` (a relaxed load, acquiring nothing) — neither side alone was enough, and upgrading only one still leaks. Both are now `AcqRel`. Worst case was a leaked file descriptor on a transport already being torn down.
- Windows: `scsi::windows::list_drives` reached into the `rip`-gated `drive`/`identity` modules, so a scsi-only consumer (`default-features = false, features = ["scsi"]`) failed to compile — `--all-features` CI never exercised that combination. It now enumerates directly via `scsi::open` + `scsi::inquiry`, like the Linux and macOS backends. Surfaced by freemkv-firmware, which links libfreemkv scsi-only.
### Changed
- The Linux recovery-thread fd hand-off moved to `scsi::fd_handoff`, where it is compiled and unit-tested on every platform (`linux.rs` is built on one host only) and model-checked under `loom`. No public API change.
### Security
- Fixed a soundness bug (GHSA-j8ww-f5fg-9pmh, low severity): `PrefetchedSectorSource::into_channels` handed out a `Sender<Vec<u8>>` recycle channel whose buffers the producer re-exposed with `unsafe { Vec::set_len }` guarded only by capacity, so a downstream caller recycling a `Vec::with_capacity(n)` (len 0) could drive `set_len` over uninitialized memory — undefined behaviour reachable from safe code. The producer now uses `Vec::resize`, removing the `unsafe`; buffer pooling (the real cross-thread alloc/free win) is unchanged. No in-tree caller triggered it.
## [1.6.14] — 2026-08-31
### Changed
- Version aligned to 1.6.14 for the unified release.
## [1.6.13] — 2026-08-28
### Added
- Capture a few additional drive buffers during `info --share` collection on supported drives.
## [1.6.12] — 2026-08-27
### Fixed
- macOS: hold a DiskArbitration claim (plus a mount-approval dissenter) for the SCSI transport's lifetime so `diskarbitrationd` can't remount the disc mid-rip — the cause of the E1000 "disc stolen mid-rip" failure on macOS.
### Changed
- Comment and documentation cleanup.
## [1.6.11] — 2026-08-26
Generated
+850
View File
@@ -0,0 +1,850 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "aes"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8eb277bec05f56a0e0591f155a484cbd0f4f07ff2905051a48c72f004f7ed58"
dependencies = [
"cipher",
"cpubits",
"cpufeatures",
]
[[package]]
name = "aho-corasick"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
dependencies = [
"memchr",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "base64"
version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "bitflags"
version = "2.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da"
[[package]]
name = "block-buffer"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
dependencies = [
"hybrid-array",
]
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cc"
version = "1.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ad534f4357a5264cce5019c989cf66a4f0dc4e0d1b1d15f8aacec0ff7360273"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "chacha20"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
dependencies = [
"cfg-if",
"cpufeatures",
"rand_core",
]
[[package]]
name = "cipher"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c"
dependencies = [
"crypto-common",
"inout",
]
[[package]]
name = "const-oid"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c"
[[package]]
name = "cpubits"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae"
[[package]]
name = "cpufeatures"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566"
dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8498c871161e1742aaa9d52551b2d6ebdd4c3d45a3be423e3728f33b955be550"
dependencies = [
"cfg-if",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-epoch"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "crypto-common"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
dependencies = [
"hybrid-array",
]
[[package]]
name = "digest"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [
"block-buffer",
"const-oid",
"crypto-common",
]
[[package]]
name = "either"
version = "1.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "errno"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "fastrand"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
[[package]]
name = "find-msvc-tools"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d45db016d36b838f563236e9193d0ee6ce38f3f68b6c94e914b4929c96bbb890"
[[package]]
name = "flate2"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"zlib-rs",
]
[[package]]
name = "freemkv-unlock"
version = "1.7.1"
dependencies = [
"aes",
"base64",
"num-bigint",
"num-traits",
"rand",
"serde",
"serde_json",
"sha1",
"sha2",
"tracing",
"zeroize",
]
[[package]]
name = "generator"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3b854b0e584ead1a33f18b2fcad7cf7be18b3875c78816b753639aa501513ae"
dependencies = [
"cc",
"cfg-if",
"libc",
"log",
"rustversion",
"windows-link",
"windows-result",
]
[[package]]
name = "getrandom"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"rand_core",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "hybrid-array"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "707114b52a152fa7bdb290cd7cd5912d9467273b6d74e21b8d81aca1f8533f6b"
dependencies = [
"typenum",
]
[[package]]
name = "indexmap"
version = "2.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07aa2048142242915a31d35844fb311e0e53fcca590c3a0a40dcf1b841fa09eb"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "inout"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4250ce6452e92010fdf7268ccc5d14faa80bb12fc741938534c58f16804e03c7"
dependencies = [
"hybrid-array",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "lazy_static"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "libc"
version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libfreemkv"
version = "1.7.1"
dependencies = [
"aes",
"base64",
"crossbeam-channel",
"freemkv-unlock",
"libc",
"loom",
"memchr",
"rayon",
"roxmltree",
"serde",
"serde_json",
"sha1",
"tempfile",
"tracing",
"zip",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "log"
version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "loom"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca"
dependencies = [
"cfg-if",
"generator",
"scoped-tls",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "matchers"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9"
dependencies = [
"regex-automata",
]
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "nu-ansi-term"
version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys",
]
[[package]]
name = "num-bigint"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93e7820bc0a80a0238e650327316f929ba18d5be054b647490a3a6a339f3e7c0"
dependencies = [
"num-integer",
"num-traits",
]
[[package]]
name = "num-integer"
version = "0.1.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b"
dependencies = [
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "rand"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80"
dependencies = [
"chacha20",
"getrandom",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
[[package]]
name = "rayon"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
dependencies = [
"either",
"rayon-core",
]
[[package]]
name = "rayon-core"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
dependencies = [
"crossbeam-deque",
"crossbeam-utils",
]
[[package]]
name = "regex-automata"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "roxmltree"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"
[[package]]
name = "rustix"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
dependencies = [
"bitflags",
"errno",
"libc",
"linux-raw-sys",
"windows-sys",
]
[[package]]
name = "rustversion"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.4",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "sha1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aacc4cc499359472b4abe1bf11d0b12e688af9a805fa5e3016f9a386dc2d0214"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "sha2"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "sharded-slab"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6"
dependencies = [
"lazy_static",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "smallvec"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f"
[[package]]
name = "syn"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "tempfile"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom",
"once_cell",
"rustix",
"windows-sys",
]
[[package]]
name = "thread_local"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ad99c4c6d32803332c548b1af0540b357b3f5fc0be8f6c6bfe8b2e6ae784070"
dependencies = [
"cfg-if",
]
[[package]]
name = "tracing"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"pin-project-lite",
"tracing-attributes",
"tracing-core",
]
[[package]]
name = "tracing-attributes"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "tracing-core"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
"valuable",
]
[[package]]
name = "tracing-log"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3"
dependencies = [
"log",
"once_cell",
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.3.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
dependencies = [
"matchers",
"nu-ansi-term",
"once_cell",
"regex-automata",
"sharded-slab",
"smallvec",
"thread_local",
"tracing",
"tracing-core",
"tracing-log",
]
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "valuable"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "zeroize"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
dependencies = [
"zeroize_derive",
]
[[package]]
name = "zeroize_derive"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "zip"
version = "8.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b"
dependencies = [
"crc32fast",
"flate2",
"indexmap",
"memchr",
"typed-path",
"zopfli",
]
[[package]]
name = "zlib-rs"
version = "0.6.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34b31d188d9d685a4f9c7b46d6e36631b07058d2cfe190267adce54dc230bf12"
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[patch.unused]]
name = "freemkv-engine"
version = "1.7.1"
[[patch.unused]]
name = "freemkv-i18n"
version = "1.7.1"
[[patch.unused]]
name = "freemkv-keysources"
version = "1.7.1"
+80 -19
View File
@@ -1,8 +1,8 @@
[package]
name = "libfreemkv"
version = "1.6.11"
version = "1.7.3"
edition = "2024"
rust-version = "1.94"
rust-version = "1.88"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@@ -18,45 +18,78 @@ publish = false
lto = "thin"
codegen-units = 1
# ── Feature layout ───────────────────────────────────────────────────────────
# `scsi` — the SCSI passthrough transport layer ONLY (`src/scsi/` + `error`):
# ScsiTransport/execute, ScsiResult/ScsiSense/parse_sense, DriveInfo,
# list_drives, drive_has_disc, CDB builders, the per-OS backends and
# the raw-`cc` macOS shim (built by build.rs). Pulls only `tracing`
# (transport instrumentation) plus the always-on `libc` target dep —
# none of the ripping dependency tree.
# `rip` — the full ripping tree (drive lifecycle, disc/UDF/MPLS/CLPI parsing,
# AACS/CSS decrypt, PES/mux pipeline, keydb, freemkv-unlock seam). All
# heavy deps (serde, zip, rayon, roxmltree, rand, crossbeam, memchr,
# sha1, aes, base64, freemkv-unlock, serde_json) live here.
# `default = ["rip", "scsi"]` preserves today's full build byte-for-byte;
# consumers wanting transport only use `default-features = false, features =
# ["scsi"]`. `rip` implies `scsi` because the ripping tree reads through it.
[features]
default = ["rip", "scsi"]
scsi = ["dep:tracing"]
rip = [
"scsi",
"dep:serde",
"dep:serde_json",
"dep:sha1",
"dep:aes",
"dep:freemkv-unlock",
"dep:zip",
"dep:base64",
"dep:roxmltree",
"dep:crossbeam-channel",
"dep:rayon",
"dep:memchr",
]
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.11" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
serde = { version = "1", features = ["derive"], optional = true }
serde_json = { version = "1", optional = true }
sha1 = { version = "0.11", optional = true }
aes = { version = "0.9", optional = true }
# Interim path dep for local cross-repo dev; `release-orchestrate.yml` re-pins
# this to `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` in the tagged commit
# (so the released tag resolves freemkv-unlock from git, not a sibling path) and
# restores the path dep on the branch tip.
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.7.3", optional = true }
zip = { version = "8", default-features = false, features = ["deflate"], optional = true }
base64 = { version = "0.23", optional = true }
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
roxmltree = { version = "0.20", optional = true }
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Permitted
# and friends). Part of the `scsi` feature — the transport layer traces here.
# Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
tracing = { version = "0.1", optional = true }
# Bounded MPSC channel with kernel-wakeup send_timeout. Used by `io::pipeline`
# so the halt-aware send/finish loops can BLOCK on consumer drain instead of
# polling — the 50 ms poll cadence of the previous mpsc-based impl capped mux
# throughput at ~1 MB/s (0.21.7).
crossbeam-channel = "0.5"
crossbeam-channel = { version = "0.5", optional = true }
# Persistent work-stealing thread pool for parallel AACS unit
# decryption. Per-call std::thread::scope spawned fresh OS threads
# and that overhead dominated for typical batch sizes (60 units).
# rayon's global pool initialises once on first use.
rayon = "1"
rayon = { version = "1", optional = true }
# SIMD-accelerated bytestring search. Drives the HEVC/H.264 start-code
# scan in `mux::codec::h264::find_start_code` — naive byte-by-byte
# walk is ~500 MB/s single-thread on x86_64; memchr's vectorised
# `memmem::find` for the 3-byte `00 00 01` needle hits ~5 GB/s on
# AVX2-capable hosts.
memchr = "2"
memchr = { version = "2", optional = true }
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
@@ -67,7 +100,35 @@ libc = "0.2"
[dev-dependencies]
tempfile = "3"
# Permutation model checker for the `scsi::fd_handoff` atomics — the fd
# hand-off between a Linux recovery thread and `SgIoTransport::drop`. A normal
# test cannot see a missing release edge there: the orderings compile to the
# same instructions on x86_64, so the wrong ones pass every time. loom
# enumerates the executions the C++20 model permits and reports the leaked
# descriptor directly.
#
# Behind `cfg(loom)`, which nothing sets unless you ask for it, so this is out
# of the graph for every ordinary build and for every CI job but the dedicated
# `loom` one. Run it with:
#
# RUSTFLAGS="--cfg loom" cargo test --lib --no-default-features \
# --features scsi fd_handoff
#
# `cargo test --lib` specifically: as a DEV-dependency loom is only in scope for
# test targets, which is why `fd_handoff` gates its import on `all(loom, test)`
# and falls back to std elsewhere. Without that, `--cfg loom` would break plain
# `cargo build`/`clippy` with an unresolved-crate error.
[target.'cfg(loom)'.dev-dependencies]
loom = "0.7"
[[bench]]
name = "sgio_read"
harness = false
[lints.rust]
# `loom` is set by RUSTFLAGS for the model checker above, never by a feature.
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(loom)'] }
[lints.clippy]
uninlined_format_args = "allow"
+8 -2
View File
@@ -1,7 +1,7 @@
# libfreemkv — local dev helper.
# Mirrors the workspace-wide CI checks but scoped to this single crate.
.PHONY: test build check ci clean
.PHONY: test build check loom ci clean
test:
cargo test --tests
@@ -13,7 +13,13 @@ check:
cargo fmt --check
cargo clippy --all-targets -- -D warnings
ci: check build test
# Model-check the scsi::fd_handoff atomics. `cargo test` cannot fail on those
# orderings — they compile to the same instructions on x86_64 — so this is the
# only check on them. Mirrors the `loom` CI job.
loom:
RUSTFLAGS="--cfg loom" cargo test --lib --no-default-features --features scsi fd_handoff
ci: check build test loom
clean:
cargo clean
+8 -8
View File
@@ -1,12 +1,14 @@
[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-ea4aaa?logo=github-sponsors)](https://github.com/sponsors/MattJackson)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/libfreemkv/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/libfreemkv)
# libfreemkv
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.
DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require a `keydb.cfg` (default `~/.config/freemkv/keydb.cfg`) supplying disc-specific volume unique keys; no AACS key material is compiled in.
DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require disc-specific volume unique keys, supplied by the consumer (e.g. via [freemkv-keysources](https://github.com/freemkv/freemkv-keysources), which owns `keydb.cfg` lookup/download); libfreemkv itself never reads `keydb.cfg` or downloads keys, and no AACS key material is compiled in.
**12+ MB/s** sustained read speeds on BD. Drive prep (`init()`) handles unlocking internally via the `freemkv-unlock` crate — clients never see it; when no drive unlock applies, the library rips via the host-certificate AACS handshake.
@@ -72,9 +74,8 @@ cannot call into it; front-ends get recovery from the engine directly. See
- **Drive access** — open, identify, internal unlock + prep, speed control, eject
- **12+ MB/s reads** — auto-detects kernel transfer limits, sustained full speed
- **Disc scanning** — UDF 2.50 filesystem, MPLS playlists, CLPI clip info
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- **Stream labels** — 7 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, DBP, Deluxe, Fox)
- **AACS decryption** — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
- **KEYDB updates** — download, verify, save from any HTTP URL (zero deps, raw TCP)
- **Content reading** — adaptive batch reads with automatic decryption
- **Stream I/O** — unified stream pipeline for reading and writing any format
@@ -96,7 +97,7 @@ Streams implement a single unified `pes::Stream` trait (re-exported as `PesStrea
DVDs (CSS) decrypt out of the box, with no external key file needed.
Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). No AACS key material is compiled into the binary.
Blu-rays and UHD (AACS) require Unit Keys passed in via `ScanOptions`/`KeySpec`. libfreemkv does not read `keydb.cfg` or fetch keys itself — that's the consumer's job (see [freemkv-keysources](https://github.com/freemkv/freemkv-keysources)). No AACS key material is compiled into the binary.
## Architecture
@@ -111,10 +112,9 @@ Disc — scan titles, streams, AACS/CSS state
├── MPLS parser — playlists → titles + clips + streams
├── CLPI parser — clip info → EP map → sector extents
├── IFO parser — DVD title sets, PGC chains, cell addresses
├── Labels — 5 BD-J format parsers (detect + parse)
├── Labels — 7 BD-J format parsers (detect + parse)
├── AACS — key resolution + content decryption
├── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
└── KEYDB — download + verify + save
└── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
Streams — unified PES pipeline
├── PesStream — pes::Stream: one trait, read()/write() PES frames
+1
View File
@@ -5,6 +5,7 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
println!("cargo:rustc-link-lib=framework=DiskArbitration");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
+252
View File
@@ -0,0 +1,252 @@
#!/usr/bin/env python3
"""comment-guard: cap explanatory prose by COMMENT INTENT, not visibility.
Three tiers, keyed on the syntax the author reached for:
* `//!` / `/*!` — a module (or crate) design doc. UNBOUNDED. There is one per
file, at the top, documenting the module as a whole, so it can't be used to
smuggle per-item rationale the way item docs can. Long correctness / design
essays (e.g. a memory-ordering proof) legitimately live here.
* `///` / `/** */` — item documentation. Up to DOC_CAP *prose* lines. Fenced
``` code blocks do NOT count, so real examples aren't punished. This holds
whether or not the item is `pub`: a `pub(crate)` fn deserves a real doc
paragraph just like a `pub` one. Capping `///` here (rather than exempting
it, as the old guard did — issue #17) is what stops rationale being hidden
behind a doc sigil, while still leaving room for a genuine paragraph.
* `//` / `/* */` — a quick internal aside. Up to INLINE_CAP lines. This is the
shape that sprawls into walls, so it's kept tight: stanzas separated by a
single blank line count CUMULATIVELY, so one essay can't be split into runs.
Overflow belongs in the module `//!`, or in docs/<topic>.md with a one-line `//`
pointer — both show up in review.
Scans src/ and tests/ under the given root (default cwd). `--selftest` runs
built-in cases. Exit non-zero if any block exceeds its cap.
"""
import re
import sys
from pathlib import Path
INLINE_CAP = 3
DOC_CAP = 8
_RAW_OPEN = re.compile(r'r(#*)"') # raw string opener: r"…", r#"…"#, br##"…"## …
def _strip_line_comment(s):
"""Return (kind, body) for a whole-line comment, else (None, None).
kind is 'doc' for `///`/`//!`, 'plain' for other `//`. A trailing comment on
a code line is not a whole-line comment and is ignored (single-line, cheap).
"""
if s.startswith("//!"):
return "doc", s[3:]
if s.startswith("///") and not s.startswith("////"):
return "doc", s[3:]
if s.startswith("//"):
return "plain", s[2:]
return None, None
def _fence_toggle(body):
"""True if this doc body line opens/closes a ``` fenced code block."""
return body.strip().startswith("```")
def _classify(lines):
"""Yield per-line dicts: comment kind ('doc'/'plain'/None), doc-ness of
block comments, and code info. Block comments are tracked across lines.
"""
info = []
in_block = False
block_is_doc = False
raw_hashes = None # None = not in a raw string; int N = open, needs '"' + N*'#'
for raw in lines:
s = raw.lstrip()
rec = {"raw": raw, "s": s, "comment": None, "is_doc": False, "code": False}
# Inside a raw string literal (e.g. an embedded HTML/CSS/JS asset): the
# bytes are string data, not Rust comments, so the guard is blind to them
# until the string closes. Prevents `/* */` or `//` lines in an embedded
# page from being counted as comment blocks.
if raw_hashes is not None:
if ('"' + "#" * raw_hashes) in raw:
raw_hashes = None
info.append(rec)
continue
if in_block:
rec["comment"] = "doc" if block_is_doc else "plain"
rec["is_doc"] = block_is_doc
if "*/" in raw:
in_block = False
info.append(rec)
continue
# A block comment only starts a comment-line if it's the first token.
if s.startswith("/*"):
rec["is_doc"] = s.startswith("/**") or s.startswith("/*!")
rec["comment"] = "doc" if rec["is_doc"] else "plain"
if "*/" not in s[2:]:
in_block = True
block_is_doc = rec["is_doc"]
info.append(rec)
continue
kind, _ = _strip_line_comment(s)
if kind is not None:
rec["comment"] = kind
rec["is_doc"] = kind == "doc"
info.append(rec)
continue
# Code (or blank). Blank = empty after strip.
rec["code"] = s != ""
# A code line may OPEN a raw string that spans following lines. Detected
# only on code (comments were handled above), so `// see r"x"` is safe.
m = _RAW_OPEN.search(raw)
if m and ('"' + "#" * len(m.group(1))) not in raw[m.end():]:
raw_hashes = len(m.group(1))
info.append(rec)
return info
def violations(path):
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
info = _classify(lines)
n = len(info)
out = []
i = 0
while i < n:
rec = info[i]
if rec["comment"] is None:
i += 1
continue
start = i
if rec["is_doc"]:
# Doc run: contiguous doc lines. `//!` / `/*!` module docs are
# UNBOUNDED; any other doc run caps PROSE (fenced ``` excluded) at
# DOC_CAP, regardless of the documented item's visibility.
j = i
prose = 0
in_fence = False
module_doc = False
while j < n and info[j]["comment"] == "doc":
s = info[j]["s"]
if s.startswith("//!") or s.startswith("/*!"):
module_doc = True
body = (
s[3:] if (s.startswith("///") or s.startswith("//!")) else s
)
if _fence_toggle(body):
in_fence = not in_fence
elif not in_fence and body.strip() != "":
prose += 1
j += 1
if not module_doc and prose > DOC_CAP:
out.append((start + 1, prose, DOC_CAP, "doc prose"))
i = j
continue
# Internal (plain) run, with single-blank bridging (cumulative).
j = i
count = 0
while j < n:
r = info[j]
if r["comment"] == "plain":
count += 1
j += 1
continue
if r["comment"] == "doc":
break
if r["comment"] is None and not r["code"]:
# blank: bridge only a SINGLE blank, else stop
if j + 1 < n and info[j + 1]["comment"] == "plain":
j += 1
continue
break
break # code
if count > INLINE_CAP:
out.append((start + 1, count, INLINE_CAP, "inline comment"))
i = j
return out
def main(argv):
if "--selftest" in argv:
return _selftest()
root = Path(argv[1]) if len(argv) > 1 else Path.cwd()
files = sorted(p for d in ("src", "tests") for p in (root / d).rglob("*.rs"))
total = 0
for f in files:
for start, length, cap, kind in violations(f):
rel = f.relative_to(root)
print(f"{rel}:{start}: {kind} block is {length} lines (max {cap})")
total += 1
if total:
print(f"\ncomment-guard: {total} comment block(s) exceed their cap.")
print(
f"Inline // asides cap at {INLINE_CAP}; /// item-doc prose caps at "
f"{DOC_CAP} (fenced examples excluded); //! module docs are unbounded. "
"Trim, move the rationale into the module //!, or docs/<topic>.md "
"with a // pointer."
)
return 1
return 0
def _selftest():
import tempfile
cases = [
# (rust source, expected number of violations)
# short /// doc -> ok
("/// short doc\npub fn a() {}\n", 0),
# /// on a PRIVATE item now gets the doc cap (8), so 4 lines is fine
("/// l1\n/// l2\n/// l3\n/// l4\nfn priv_a() {}\n", 0),
# ...but /// is NOT unlimited: 9 prose lines on a private item fails
("".join(f"/// l{k}\n" for k in range(9)) + "fn priv_b() {}\n", 1),
# 8 prose /// on a pub item -> ok (boundary)
("".join(f"/// l{k}\n" for k in range(8)) + "pub fn c() {}\n", 0),
# 9 prose /// on a pub item -> violation
("".join(f"/// l{k}\n" for k in range(9)) + "pub fn d() {}\n", 1),
# /// with a long fenced example -> example excluded, ok
(
"/// summary\n/// # Examples\n/// ```\n"
+ "".join("/// code\n" for _ in range(12))
+ "/// ```\npub fn e() {}\n",
0,
),
# pub(crate) gets the SAME doc cap as pub (no pub/private split anymore)
("".join(f"/// l{k}\n" for k in range(6)) + "pub(crate) fn f() {}\n", 0),
# bare inline 4 lines -> violation
("// a\n// b\n// c\n// d\nlet x = 1;\n", 1),
# blank-split dodge: 3 + blank + 3 inline -> cumulative -> violation
("// a\n// b\n// c\n\n// d\n// e\n// f\nfn z() {}\n", 1),
# plain /* */ block, 5 lines -> inline cap -> violation
("/* one\n two\n three\n four\n five */\nfn q() {}\n", 1),
# comment-like lines INSIDE a raw string are string data, not comments
(
'const H: &str = r##"<style>\n'
+ "".join("/* css note */\n" for _ in range(8))
+ '</style>"##;\nfn h() {}\n',
0,
),
# //! module doc may run long -> EXEMPT (no cap)
("".join(f"//! l{k}\n" for k in range(40)) + "\npub fn m() {}\n", 0),
# /*! block module doc may run long -> EXEMPT
("/*! l0\n" + "".join(f" l{k}\n" for k in range(30)) + "*/\nfn n() {}\n", 0),
]
ok = True
with tempfile.TemporaryDirectory() as d:
for idx, (src, want) in enumerate(cases):
p = Path(d) / f"case{idx}.rs"
p.write_text(src)
got = len(violations(p))
flag = "ok" if got == want else "FAIL"
if got != want:
ok = False
print(f" selftest {idx}: want {want} got {got} [{flag}]")
print("selftest PASSED" if ok else "selftest FAILED")
return 0 if ok else 1
if __name__ == "__main__":
sys.exit(main(sys.argv))
+42
View File
@@ -0,0 +1,42 @@
# cargo-deny — gates the RESOLVED dependency graph (issue #15).
# Runs in CI against the committed Cargo.lock: RUSTSEC advisories, a permissive
# licence allow-list, and source/ban hygiene. Update the lock (cargo update)
# to clear an advisory rather than adding an `ignore` entry without cause.
[advisories]
yanked = "deny"
# ignore = ["RUSTSEC-0000-0000"] # only with a written justification
[licenses]
# The whole tree is permissive. r-efi's LGPL arm is satisfied by Apache-2.0
# (its expression is "Apache-2.0 OR LGPL-2.1-or-later"), so no copyleft is
# allowed here. Add a licence only when a real dependency introduces it.
allow = [
"MIT",
"Apache-2.0",
"Apache-2.0 WITH LLVM-exception",
"0BSD",
"Unicode-3.0",
"Unlicense",
"Zlib",
"BSD-2-Clause",
"BSD-3-Clause",
"ISC",
"CDLA-Permissive-2.0",
]
confidence-threshold = 0.9
[bans]
multiple-versions = "warn" # transitive dupes (digest 0.10/0.11 etc.) are noise, not a gate
wildcards = "deny"
# The freemkv siblings resolve as path deps under CI's path-patch layout, which
# reads as a wildcard version req. That is internal, not a supply-chain wildcard.
allow-wildcard-paths = true
[sources]
unknown-registry = "deny"
unknown-git = "deny"
# The freemkv siblings are first-party git-tag deps; allow git sources from
# our own org only (CI resolves them as git, not the local path override).
[sources.allow-org]
github = ["freemkv"]
+190
View File
@@ -0,0 +1,190 @@
# AACS Content Decryption — Implementation Notes
Rationale and background trimmed out of `src/aacs/content.rs` doc comments to
satisfy `ci/comment-guard.py`'s per-audience prose caps. See
[AACS Encryption](aacs.md) for the public-facing key resolution story and
[AACS Crypto Primitives](aacs-crypto.md) for the low-level AES primitives.
## `is_unit_aligned`
AACS aligned units (6144 B / 3 sectors) are anchored at the start of each
clip's encrypted region, so a read must begin a whole number of units past
that base for `decrypt_sectors` (which anchors units at buffer offset 0) to
align the CBC correctly. This is the single source of truth for the
decrypt-on-read gate, the inline and highway mux read paths, and the
key-validation sample reader — all key off this, never absolute `lba % 3`. A
disc whose clip `start_lba` is not itself 3-aligned would otherwise mis-gate
(reject readable units, then report "Decryption failed") on exactly the
titles whose clips land off a 3-boundary.
`saturating_sub` makes the `lba < unit_base` case well-defined — it clamps
the offset to 0, a unit boundary — rather than the latent `wrapping_sub` trap
where an underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`,
mis-reports the alignment (e.g. `lba == unit_base - 1` would falsely read as
aligned).
## `aacs_unit_encrypted`
The mux is the principal consumer of this signal for `BdTs` discs (via
[`is_clean`], see below). It is readable WITHOUT a key. CRITICAL: only
meaningful when `unit` is read at the correct clip-FILE-anchored boundary — a
disc-absolute / mis-aligned read makes the flag byte arbitrary mid-stream
data, which is why per-unit checks run clip-anchored, not in the whole-disc
sweep.
## `aacs_unit_needs_decrypt`
Buffer-iterating sites that may run twice over the same `buf` (the post-fetch
re-decrypt, sample collection, failure diagnosis) need an IDEMPOTENT "does
this still need work?" test, so `aacs_unit_encrypted` (the authoritative flag,
which decryption never rewrites) is composed with a "structure restored?"
check that flips once decrypted: TS syncs come back for `BdTs`; valid
`00 00 01 BA` packs come back for `MpegPs`.
## HD-DVD scramble flag (`PS_SCRAMBLE_OFF` / `PS_SCRAMBLE_MASK`)
BD/UHD/FMTS flag encryption uses the Copy Permission Indicator in the top 2
bits of byte 0 (the M2TS `TP_extra_header`). HD-DVD `.evo` is Program Stream —
byte 0 is a `00 00 01 BA` pack_start_code, NOT a CPI — so AACS reuses the
MPEG-2 `PES_scrambling_control` field instead: pack_header (14 bytes) + PES
start-code/`stream_id` (4) + `PES_packet_length` (2) puts the PES flags byte at
offset 20, with `PES_scrambling_control` in bits 5-4 (`& 0x30`). Non-zero =
encrypted. Derived from BackupHDDVD (`Header[20] & 0x30`) cross-checked
against MPEG-2 systems (ISO/IEC 13818-1).
UNVERIFIED against a real ENCRYPTED HD-DVD disc — none available to confirm
byte-exactly. Two open questions a real disc must settle:
1. A pack carrying an MPEG `system_header` (`00 00 01 BB`) before the PES
packet shifts this offset past 20.
2. Whether offset 20 is even readable pre-decrypt. BD keeps only the first 16
bytes clear (the seed) and AES-CBC-encrypts 16..6144 — under that model
offset 20 is ciphertext. BackupHDDVD reading `Header[20]` pre-decrypt
implies HD-DVD instead encrypts PES *payloads* with *clear* pack/PES
headers (per-PES model). If so, `decrypt_unit`'s 16-byte-seed model also
would not fit HD-DVD and needs its own path. TS is unaffected either way.
## `KEY_PROOF_PACKETS`
Four `0x47` syncs are 32 bits of MPEG-TS structure, but the per-UNIT
false-pass risk is NOT 2^-32: `is_clean_ts` accepts ANY four of the ~31
encrypted packets in a 6144-byte aligned unit, so for a wrong key (uniform
AES noise, `0x47` at 1/256 per packet) it is ≈ C(31,4)·256^-4 ≈ 7e-6, i.e.
~1e-5. 1-in-4-billion is the probability for four SPECIFIC packets and
overstates the margin by ~4000x; at `KEY_PROOF_PACKETS = 3` the per-unit rate
is ≈ C(31,3)·256^-3 ≈ 2.6e-4, so do NOT lower it on the strength of slack
that is not there. It is an ABSOLUTE proof floor, NOT a proportion — a unit
the key opened but whose content is bad-encoded (many non-conforming
packets) is proven by ANY four good packets, not rejected for the bad ones.
## `is_clean_ts`
The mux is its principal consumer for `BdTs` discs, reaching it through
`is_clean`: `mux::resolve`'s multi-CPS `pick` closure selects a unit key by
it, `probe_index_phase` reports each FMTS index's interleave parity by it,
and `decrypt::decrypt_sectors_mapped` uses it as the forensic-range verify
net. (The doc used to say "the mux never calls this", which invited a
maintainer to tighten or loosen the proof rule believing only whole-disc read
verification was affected — while it in fact changes which unit key a
multi-CPS disc muxes with and which phase an FMTS index is muxed at.)
Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the
ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed,
so it reads `0x47` for ANY key and is never evidence), let `E` = non-padding
content packets and `synced` = those carrying `0x47`. The key opened the unit
iff `E == 0` (nothing encrypted to prove) OR `synced >= min(E, KEY_PROOF_PACKETS)`.
* WRONG key → ~0 synced → fails (reaching 4 by chance ≈ 1e-5/unit, and every
unit of a clip would have to fluke — astronomically safe).
* RIGHT key, bad-encoded content → any 4 good packets pass; the bad ones are
the muxer's problem. (This is the false-negative the old 75% PROPORTION
caused — a mostly-bad unit the key opened was wrongly rejected.)
* `min(E, 4)` handles the end-of-clip fragment TAIL: a unit with only E=1
real packet (then source-zero padding) needs just that one to sync, so a
sparse-but-valid tail is never false-rejected. Padding (all-zero payload)
is excluded throughout.
## `is_clean_ps`
The Program-Stream arm of `is_clean` (HD-DVD `.evo`): every 2048-byte pack
begins with the pack_start_code `00 00 01 BA`; a 6144-byte AACS unit spans
three packs. Pack 0's start sits in the clear 16-byte seed (present
regardless of the key — mirrors `is_clean_ts` skipping packet 0); packs 1
and 2 (offsets 2048 and 4096) are in the encrypted region, so a wrong key
garbles them and this returns false (64 bits of discrimination). Validated
against real decrypted HD-DVD `.evo` (two real retail titles): pack starts
are exactly 2048-aligned, three per aligned unit, at offsets 0 / 2048 / 4096.
UNVERIFIED-HDDVD-DECRYPT: the pack structure here is confirmed on decrypted
rips, but that a real ENCRYPTED `.evo` decrypts to it via the same
6144-byte aligned unit (16-byte seed + AES-CBC over 16..6144) as BD is the
unverified assumption — we have no encrypted HD DVD. If HD-DVD decryption
yields garbage with a known-good key, the unit granularity is the suspect.
## `decrypt_unit` / `encrypt_unit`
Block Key = AES-128E(Kcu, seed) ⊕ seed (`[BD]` §3.10.1 Fig 3-8: encrypt the
clear 16-byte seed under the CPS Unit Key, XOR the seed back in — the
trailing ⊕ seed is load-bearing); then AES-128-CBC decrypt/encrypt bytes
16..6144 under the AACS IV. `encrypt_unit` is the exact inverse of
`decrypt_unit`, for authoring an encrypted disc image (and for building
genuinely-encrypted read-path fixtures); it does NOT set the encrypted flag,
because where that flag lives is container-specific, and keeping it out
keeps the crypto container-agnostic.
Source-zero padding packets (all 192 bytes zero on disc) are restored to
zero by `decrypt_unit`: their decrypted bytes are AES-noise from decrypting
zeros, but the source WAS zero, so writing the true source back is faithful
and gives the demux a tidy gap. Content packets are left EXACTLY as
decrypted — the decrypt path never rewrites content, so an authored-bad
packet passes through verbatim. `encrypt_unit` deliberately does NOT mirror
this: an all-zero plaintext packet encrypts to ciphertext that is not
all-zero, so it is not mistaken for padding on the way back and the round
trip is still exact. Authoring that wants true source-zero padding leaves
those packets unencrypted instead.
`encrypt_unit` returns `false` — encrypting nothing — when `unit` is shorter
than `ALIGNED_UNIT_LEN`. That case MUST be checked: the caller has already
set the container's encrypted flag by then (the contract requires
flag-before-crypto, since the header is the key seed), so ignoring the
result leaves a unit marked encrypted while still carrying plaintext, which
is the worst possible outcome for an authoring tool.
## `decrypt_bus` key-schedule regression (test: `decrypt_bus_expands_the_read_data_key_once_per_unit`)
MEASURED, not reasoned: `decrypt_bus` used to call `aes_cbc_decrypt` once per
2048-byte sector, and each call built its own AES-128 key schedule, so a
6144-byte aligned unit performed THREE key expansions under the same
loop-invariant `read_data_key`. On a 90 GB UHD read on a stock
(non-firmware-unlocked) drive — ~14.6 million aligned units — that is ~29
million redundant expansions on the per-unit decrypt hot path, for a key
that is constant for the whole disc. The counter is incremented inside
`crypto::new_cipher`, the single construction site.
## `IV0_PUBLISHED` (test fixture)
`[C]` §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation. Both
IV tests used to compute their expected value from `crypto::AACS_IV` itself,
so the constant was asserted against itself and NOTHING in the suite pinned
its bytes: swapping `AACS_IV` for `[0u8; 16]` left both tests passing (one
builds its ciphertext with the same value and the other cancels the change
in a triple XOR) while every real AACS disc decrypted to noise — block 0 of
every 6144-byte aligned unit and of every bus-encrypted sector XORed with the
wrong IV. `IV0_PUBLISHED` is the independent witness the tests assert
against instead.
## Encrypted-flag readers (`aacs_unit_seed_encrypted`, `aacs_unit_encrypted`)
`aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the guard
that stops a truncated encrypted fragment from being emitted as clear
content. It reads ONLY the two Copy Permission Indicator bits (`[BD]`
§3.10.2, byte 0 bits 6-7); the remaining six bits are `TP_extra_header`
arrival-timestamp bits and carry no encryption meaning. Both failure
directions are damaging and silent: a reader that answers "encrypted" for a
clear fragment discards good content, and one that answers "clear" for an
encrypted fragment writes ciphertext into the output as if it were video.
`aacs_unit_encrypted` is the AUTHORITATIVE gate and requires a WHOLE
6144-byte aligned unit: on anything shorter the flag byte is not guaranteed
to be the unit's, so it must answer `false` and leave the partial-unit case
to `aacs_unit_seed_encrypted`. A reversed length guard would both classify
fragments off arbitrary mid-stream bytes and, on an empty slice, index out
of bounds.
+89
View File
@@ -0,0 +1,89 @@
# AACS Crypto Primitives — Implementation Notes
Rationale and background for the internal (non-public) primitives in
`src/aacs/crypto.rs`. See [AACS Encryption](aacs.md) for the public-facing
key resolution and decryption story.
## `aes_cbc_encrypt`
Forward direction of `aes_cbc_decrypt`, and its exact inverse (`[C]` §2.1.2,
`AES-128CBCE`). `data.len()` must be a multiple of 16; the `debug_assert!`
documents/enforces that contract.
Constructs the cipher ONCE for the whole slice. Driving this from the
single-block `aes_ecb_encrypt` instead rebuilds the AES key schedule per
16-byte block, which for a 6144-byte aligned unit is 383 redundant key
expansions.
## `aes_cbc_decrypt`
AES-128-CBC decrypt in-place with the fixed AACS IV (`[C]` §2.1.2,
`AES-128CBCD`). `data.len()` must be a multiple of 16; any trailing partial
block is silently ignored. All callers pass aligned regions (6128 and 2032
bytes), and the `debug_assert!` documents/enforces that contract.
This doc was once orphaned onto `aes_cbc_encrypt` above it, when that
function was inserted directly after it with no separating blank line:
rustdoc rendered the crate's only forward-direction AACS primitive as
"decrypt" and cited the spec's DECRYPT clause for it, while this function
had no doc at all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in
`content.rs` pins the directions behaviourally, so a maintainer "fixing" the
contradiction by swapping the two bodies fails the suite instead of shipping
a second decryptor behind an already-set encrypted flag.
## `cbc_decrypt_blocks`
Split out of `aes_cbc_decrypt` so a caller that decrypts several regions
under one loop-invariant key expands the schedule once. `decrypt_bus`
(`super::content::decrypt_bus`) is that caller: bus encryption (`[C]` §4.2 /
the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY 2048-byte
sector, so a 6144-byte aligned unit is three regions under one
`read_data_key` — three key schedules where one suffices, on the per-unit
decrypt hot path of a whole 90 GB read.
## `aes_g`
AES-G(x1, x2) = AES-128D(x1, x2) XOR x2 (`[C]` §2.1.3, note: uses
AES-128**D**). The Media Key Variant chain uses AES-G to derive both the
variant number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
(`Kvu = AES-G(Km, VID)`). See `super::derive::derive_vuk` for the classical
VUK form — the math is identical, this exposes it as a neutral primitive
for the variant chain.
## `aesg3`
AACS-G3 (`[C]` §3.2.2, Triple AES Generator): `left = D(k,s0)⊕s0` inc 0,
`pk = D(k,s0+1)⊕(s0+1)` inc 1, `right = D(k,s0+2)⊕(s0+2)` inc 2. Shared
with `super::variant` (its variant chain runs the same SD tree); a single
definition keeps the two walks byte-identical.
## Test: `S0`
The AACS-G3 seed `s0`, transcribed independently from `[C]` §3.2.2 rather
than read from `AESG3_SEED` — a test that sourced the seed from the
production constant would assert that constant against itself and would
still pass if it were edited.
## Test: `aesg3_inverts_to_the_spec_seed_under_aes_encrypt`
`aesg3` is the node function of the AACS subset-difference tree: every
Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every descent
step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that returned a
fixed block would make every device key in the crate derive the SAME
Processing Key, and a `^` that became `|` or `&` would derive a
wrong-but-plausible one — in both cases the MKB walk simply stops finding
Media Keys, with no error to say why.
Pinned through the spec relation rather than a re-implementation: `[C]`
§3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for `s = s0 + inc` (added
into the last seed byte), so applying the FORWARD primitive
`aes_ecb_encrypt` — a different function from the one under test — to
`aesg3(k, inc) XOR s` must reproduce `s` exactly.
## Test: `aesg3_yields_three_distinct_subkeys_for_the_three_increments`
The Triple Generator's three outputs (`[C]` §3.2.2: left = inc 0, the
Processing Key = inc 1, right = inc 2) are the two child node keys and the
Processing Key of ONE tree node. They must be three different keys — if
`inc` were ignored, a descent would revisit its own parent and the walk
would derive the same key at every level of the tree.
+68
View File
@@ -0,0 +1,68 @@
# `src/aacs/derive.rs` — key derivation notes
Rationale and background relocated here from doc comments in `derive.rs` to
keep those comments within the repo's comment-length limits.
## `derive_media_key_from_pk` — why the direct PK × cvalue scan
A Processing Key is **terminal**: it is the key at its Subset-Difference
node, one `AES-G` from the Media Key. So the fast path tries each PK
*directly* against the MKB cvalue tables (no tree descent). On a large
AACS 2.x UHD MKB (~181k cvalues) this is ~15x faster than treating a PK as
a device-node label and walking the tree.
MKB record types relevant to this path:
| Type | Meaning |
|------|---------|
| `0x10` | Type and Version Record (has MKB version) |
| `0x81` | Verify Media Key Record, AACS 1.0 (has `mk_dv`) |
| `0x86` | Verify Media Key Record, AACS 2.0/2.1 (has `mk_dv`) |
| `0x04` | Subset-Difference Index (has UVS entries) |
| `0x05` | Media Key Data Record (cvalues, 1:1 with `0x04`) |
| `0x07` | Explicit Subset-Difference Record (NOT cvalues) |
## `recover_dk_position` — search strategy and cost
This finds a device key's tree node empirically: for each MKB
subset-difference record, it tries the device at the record's node AND at
every ancestor v-position (the device may sit one or more levels ABOVE the
record, descending via AES-G3 to reach it), deriving the candidate
Processing Key DIRECTLY (one `calc_pk_from_dk` per candidate, no full
re-walk) and checking it validates against that record's cvalue.
Cost is `O(slots × tree_depth)` — linear in the MKB's subset-difference
index, not the quartic cost of re-deriving per candidate. Slots are
independent so the scan parallelizes (~181k slots, ~26s serial on a UHD
MKB); `find_map_any` returns the first match and cancels the rest.
## `resolve_candidate` — composed primitives and `None` conditions
Composes the raw derivation primitives (`derive_media_key_from_pk`,
`derive_media_key_and_pk_from_dk`, `derive_vuk`, `derive_unit_keys`) and
parses `Unit_Key_RO.inf` at the version the disc's MKB declares, so a
multi-CPS disc yields all its unit keys from the one candidate.
Validate `unit_keys` against a real encrypted unit with `decrypt_unit` +
`is_clean_ts` to prove the candidate actually opens the disc — this function
is pure derivation, no sampling or validation.
Returns `None` only when derivation itself cannot proceed: a PK its MKB
rejects, a `Dk` the MKB can't process, a missing VID on a path that needs
one, or an unparseable/empty `Unit_Key_RO.inf`.
## Test fixtures — inverting the AACS relations
There are no published AACS test vectors, but none are needed: the AACS
relations (`[C]` §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
key can be constructed with `aes_ecb_encrypt` and the same `aesg3` the walk
uses as its node function. `plant_mkb` and its siblings in
`position_recovery_tests` do exactly this — no real key material, and the
assertions check the DERIVED Media Key, not any intermediate the code under
test also produces.
`plant_mkb`'s single-slot fixture picks `uv = 0x0400` (lowest set bit 10)
with `u_mask_shift = 12` so a gating device node exists: `resolve_dk_node`
flips one bit `b < 12`, and the walk's gate (`[C]` §3.2.4) needs that bit
inside `v_mask` (`0xFFFF_FFFF << 11`) and outside `u_mask`
(`0xFFFF_FFFF << 12`) — i.e. `b == 11`.
+78
View File
@@ -0,0 +1,78 @@
# `src/aacs/resolve.rs` — internal rationale
Overflow detail relocated here by the comment-guard pass; each section is
pointed to by a short `//` comment at the named site in `resolve.rs`.
## `resolve_keys_classical`
Resolve all AACS keys for a disc using the classical (single-stage Media Key
derivation) paths. Used by both V10 and V20.
Paths run in root-of-trust -> per-disc-leaf order. A match at any path
returns immediately:
1. MKB + device keys -> processing key -> media key -> VUK
2. MKB + processing keys -> media key -> VUK
3. KEYDB MK + matching VID -> derived VUK
4. KEYDB disc-hash -> VUK
5. KEYDB disc-hash -> pre-decrypted unit keys (no VUK)
## `unique_verifying_mk`
The MK-pool selection rule of path 2.5, split out of `resolve_keys_v1` so the
ambiguity guard has a reachable test.
`verifies` is the MKB check -- in production
`AES-D(mk, mk_dv)[..8] == MK_VERIFY_MAGIC`. Returns a Media Key only when
EXACTLY ONE DISTINCT candidate passes. Duplicates of the same key are one
candidate (a pool aggregated across providers routinely repeats a key), but
two DIFFERENT keys that both verify mean the pool cannot say which is this
disc's Km: picking either derives a wrong VUK, and a wrong VUK decrypts to
plausible-looking garbage rather than failing loudly. Bail and let the later
hash/VID paths answer instead.
The predicate is a parameter rather than the inlined AES check because a
genuine two-key multi-hit cannot be synthesised: it needs one ciphertext
that decrypts under two distinct AES-128 keys to plaintexts sharing a 64-bit
prefix -- a 2^64 search. Injecting the verifier is the only way the
ambiguity branch is reachable from a test at all.
## Test: `mk_pool_ambiguity_bails_rather_than_picking_a_media_key`
Path 2.5's ambiguity guard: when MORE THAN ONE DISTINCT pooled Media Key
verifies against the MKB, the resolver must return no key at all rather than
pick one. A wrong Km derives a wrong VUK, and a wrong VUK does not fail
loudly -- it decrypts the title to garbage that muxes and plays as a corrupt
rip.
The real MKB check cannot be forced into a multi-hit: two distinct AES-128
keys decrypting one `mk_dv` to plaintexts that share the 64-bit verify magic
is a 2^64 search, not a fixture. So the rule is tested through
`unique_verifying_mk`, whose verifier is a parameter -- the same function
`resolve_keys_v1` calls, with the same pool semantics.
## Test: `resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid`
`resolve_keys_v21` gates paths 1 and 3 on `has_vid`, and an all-zero Volume
ID is the crate's "the VID was never read" sentinel -- the SCSI handshake
leaves the buffer zeroed when it does not run or fails.
Both directions matter and both fail silently:
- treating the zero sentinel as a real VID runs path 3 and derives
`Kvu = AES-G(Km, 0...0)`, a perfectly well-formed but WRONG VUK. It
unwraps the title keys to garbage, and nothing downstream errors -- the
rip just decodes to noise.
- treating a real VID as absent skips paths 1 and 3 entirely, so a disc
that could have been resolved from its Media Key reports no key.
Asserted through the final VUK, not through the flag.
## Test: `classify_vid_present_with_material_is_no_material_not_vid`
A NON-zero VID present, but resolution still fails (the providers carry
material that doesn't resolve this disc). The VID is available, so the
failure is NOT "VID unavailable" -- it must classify NoMaterial regardless
of how much derivation material is present, because re-acquiring the VID is
not the fix. This is the `has_vid == true` short-circuit, which no existing
test covers (the gate test only uses the zero-VID sentinel).
+14
View File
@@ -0,0 +1,14 @@
# AACS resolution trace shape
`src/aacs/trace.rs` defines the structured trace of a resolution attempt (the
unlock phase, then the key-resolution phase).
- No user-facing English: every step's STATE is a typed enum variant.
Applications render these into localized text — the library never does.
This module only defines the shape and is wired through the
resolve/handshake return path far enough to compile.
- The `who` of each step is the source's `label()` / unlocker's `name()` — a
stable identifier string (a NAME, like a codec id, NOT user-facing prose),
carried verbatim so an app renderer never has to match an enum back to a
name it already has. Only the OUTCOME / path enums are structured states
the app maps to i18n English.
+97
View File
@@ -117,3 +117,100 @@ starting with `;` or `#` are comments) holding the host credentials and per-disc
entries the library uses to resolve a disc. autorip can auto-download and
refresh it from a configured URL. The library does not ship any AACS keys
compiled into the binary.
## `UnitKey::is_default_index` test rationale
`is_default_index` is the public predicate that separates ordinary (index-0)
content keys from FMTS forensic index keys (see `UnitKey` docs; AACS 2.1
`IndividualSegment.tbl` tagging). A body answering `true` for everything would
present a forensic index key as an ordinary content key -- the caller would
decrypt the bulk of the title with a key that only opens 1/32nd of it;
answering `false` for everything would hide every ordinary key.
`is_default_index_separates_the_two_constructors` is pinned against the two
named constructors, which are the contract: `UnitKey::new` builds the ordinary
key, `UnitKey::forensic` builds an index key for `1..=32`.
`is_default_index_agrees_with_the_forensic_index_resolver` checks the
predicate agrees with the one consumer of `index_number` in the crate:
`crate::aacs::index_select::resolve_disc_index` resolves the disc's forensic
index from exactly the keys that are NOT default. If the two disagree, a disc
resolves an index whose key the rest of the pipeline treats as ordinary (or
vice versa).
## `src/aacs/mod.rs` module notes
### KEYDB.cfg line format
```
| DK | DEVICE_KEY 0x... | DEVICE_NODE 0x... | KEY_UV 0x... | KEY_U_MASK_SHIFT 0x...
| PK | 0x...
| HC | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
| HC2 | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
0x<disc_hash> = <title> | D | <date> | M | 0x<media_key> | I | 0x<disc_id> | V | 0x<vuk> | U | <unit_keys>
```
### Spec provenance tags
The crypto in `src/aacs/` carries `[TAG] §x.y` citations back to the published
AACS specification (Final Rev 0.953), so each primitive links to the section it
implements:
- `[C]` -- AACS Introduction and Common Cryptographic Elements Book (primitives,
MKB/key-management).
- `[PR]` -- AACS Pre-recorded Video Book (Volume/Title Key layer).
- `[BD]` -- AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit,
Block Key).
- `[RE]` -- reverse-engineered from real discs, cited only where the public
spec is silent (the `0x86` verify record and the Category-C MKB type values).
### AACS key-input path discovery (`AacsRole`, `role_paths`, `find_hddvd_aacs_dir`)
BD and UHD keep their key material under a fixed `/AACS/...` tree, so those
paths are constants. HD DVD keeps the equivalents in a reserved root directory
whose NAME is authoring-house-specific -- observed `ANY!` (Dukes of Hazzard)
and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK` mirror -- and
whose title-key file is NOT always `VTKF000.AACS` (Freedom ships
`VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED from the
parsed UDF tree (`find_hddvd_aacs_dir` + `role_paths`), never hardcoded.
Each key role (`AacsRole`) resolves to an ordered candidate list -- the BD/UHD
constants first, then whatever the HD DVD directory actually holds -- which
every reader walks with `read_first`, first-that-reads. No reader ever
branches on disc type: a BD/UHD disc has the `/AACS/` files so those win; an
HD DVD has none of them, so it falls through to the discovered entries.
Centralised so `resolve_vid_only`, `read_aacs_inputs`, `read_mkb_content`, and
`read_aacs_version` can never silently diverge the disc_hash / MKB / VID that
another reader feeds a key service.
`find_hddvd_aacs_dir` identifies the HD DVD AACS directory structurally, NOT by
a hardcoded name: the root child directory whose name ends in `!` (so the
`<name>!_BAK` backup mirror, which also ends in a non-`!` char, is not
mistaken for it) and which contains `MKBROM.AACS`. Observed real names:
`ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD disc has no such
directory -> `None`.
`role_paths` builds the ordered candidate list: fixed BD/UHD paths first, then
the discovered HD DVD files. For `AacsRole::UnitKey` every `VTKF*.AACS` in the
directory is appended in sorted name order -- a disc may carry more than one
variant (Freedom: `VTKF090` + `VTKF100`), not just `VTKF000`.
`read_first` walks an AACS role's candidate paths and returns the first that
reads; `read` performs the actual per-path read (full file or bounded prefix),
so callers share the same first-present walk regardless of read style. Returns
`Error::AacsNoKeys` if no candidate is present. Generic over the path element
(`&str` or owned `String`) so it accepts the `Vec<String>` that `role_paths`
builds from the discovered HD DVD directory.
### Test: `a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory`
The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required -- the
discovery is a conjunction, not a disjunction. The existing fixtures only ever
present a directory that satisfies both (`AAC!` with `MKBROM.AACS`) alongside
one that satisfies neither (`AAC!_BAK` -- which contains `MKBROM.AACS` but is
ALSO reached only after the real dir), so either half of the conjunction could
be dropped and the same directory would still be found. This test's fixture
has a directory that satisfies the name half and NOT the contents half: it
must not be picked. If it were, the HD DVD path would resolve `MKBROM.AACS`,
`CONTENT_CERT.AACS` and the title-key file under a directory that holds none
of them -- the disc reports "no AACS key files" and never rips.
+220
View File
@@ -0,0 +1,220 @@
# AC3 / E-AC3 parser notes (`src/mux/codec/ac3.rs`)
## Why a frame set is one sample, never split into separate tracks
A legacy AC-3 syncframe is a complete access unit on its own, but an E-AC-3
access unit (ETSI TS 102 366 / ATSC A/52 Annex E) is a whole FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream frame that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7) and their own dependents. A decoder needs the whole set
to reconstruct the programme (a 5.1 independent substream plus a dependent
substream carrying the extra channels of a 7.1 programme, and the AC-3-core +
E-AC-3-dependent arrangement used for backwards-compatible Dolby Digital
Plus), and every substream of the frame set covers the SAME time period.
WHY the whole frame set is ONE sample, rather than splitting an additional
independent substream (an associated / commentary service) into a track of
its own: Annex E orders the substreams of a frame set inside a single
elementary stream, all covering one time period, and a decoder handed a frame
set renders the programme it is asked for from it — that is exactly what the
container's E-AC-3 sample and its `dec3`/`num_ind_sub` description denote.
Splitting would mean rewriting the bitstream (re-numbering `substreamid` so
the extracted service becomes substream 0, and rebuilding its frame sets),
because a substream numbered 1..7 with no substream 0 is not a conforming
stream. That is a transcode, not a remux; this parser is lossless, so the
frame set stays whole and player-side programme selection decides what is
heard.
## `SAMPLE_RATES` / fscod2
Sample rates indexed by fscod (0=48kHz, 1=44.1kHz, 2=32kHz). fscod=3 is
reserved in AC-3; in E-AC-3 it signals "fscod2" (reduced rates: 24/22.05/16
kHz, selected by byte-4 bits `[5:4]`). `frame_sample_rate` decodes fscod2 in
the E-AC-3 case; the table's index-3 entry (48 kHz) is only the fallback when
the header is too short to read fscod2.
## `MAX_AC3_BUF` derivation
An AC-3/E-AC-3 syncframe is at most 8192 bytes (the `frame_size > 8192`
reject) and an access unit is a whole frame set: up to 8 independent
substreams, each with up to 8 dependent substreams (ETSI TS 102 366 Annex E)
— 72 syncframes worst case. A cap of one straddling frame set plus slack
therefore has to exceed 72 × 8192 = 576 KiB. If the buffer grows past the cap
without yielding a frame (pathological / never-syncing input) it is dropped
and resync begins, rather than accumulating one PES worth of data per call
for the whole title.
## `scan_access_units`
Per ETSI TS 102 366 (ATSC A/52) Annex E an access unit is one FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7 — associated/commentary services) with their own
dependents. The access unit therefore closes only when the next
`substreamid`-0 independent substream (or, with `at_eos`, the end of the
stream) is seen, and it carries THAT substream's PTS and duration: every
other substream of the frame set describes the SAME time period and adds no
duration of its own.
`base_pts_ns` times the access unit that begins at `data[0]` — i.e. the
running cadence carried over from the previous call. `anchor` re-anchors the
running PTS to this PES's own timestamp at the first access unit that STARTS
in the newly-appended bytes: a PES timestamp applies to the first access unit
beginning in that PES, never to one that began in an earlier PES and is only
being completed (or was held) here.
Returns the emitted access units, the offset in `data` from which bytes must
be carried over to the next call, and the PTS to stamp on the access unit
that begins that carry-over.
## `substream_role`
Byte 2 of an E-AC-3 syncframe is `strmtyp(2) | substreamid(3) | frmsiz[10:8]`
(ETSI TS 102 366 Annex E BSI). `strmtyp` 0 and 2 are independent substreams
(type 2 being an independent substream that is not the first of the bit
stream); `strmtyp` 1 is the dependent substream.
Annex E defines a FRAME SET as independent substream 0 — mandatory, always
first — with its dependent substreams, followed by the OPTIONAL additional
independent substreams 1..7, each with their own dependents; all of them
carry the same time period. So the access-unit boundary is an independent
substream with `substreamid` == 0, and NOT merely "an independent substream":
an additional independent substream (an associated or commentary service)
sits INSIDE the frame set already open. Keying the boundary on `strmtyp`
alone made every such substream close the access unit and advance the
running PTS a second time over the same ~32 ms, doubling the timeline (about
a second of A/V drift per second of audio).
`strmtyp` 3 is reserved: its BSI layout is not defined, so its `substreamid`
bits cannot be trusted and it is treated as starting a fresh access unit — an
unknown frame is never merged into an unrelated programme, and never
discarded as an orphan either.
Legacy AC-3 (`bsid < 11`) has no substream structure — byte 2 there is the
crc1 field, never `strmtyp` — so it always starts an access unit.
A frame set MAY carry a substream as several consecutive syncframes of fewer
than six blocks each (Annex E allows numblkscod < 3). Those extra syncframes
are `substreamid` 0 too, so each starts its own access unit here rather than
merging into one frame set. That is deliberate: each carries its OWN
numblkscod-derived duration (see `frame_duration_ns`), so the timeline total
stays exact, and the substreams are still delivered in bitstream order.
## `frame_crc_ok`
Per ETSI TS 102 366 (ATSC A/52) the frame carries a CRC-16/ANSI (poly 0x8005,
init 0, non-reflected) over the bytes after the 2-byte syncword — i.e.
`crc16_ansi(&buf[2..]) == 0` covers `frame_size - 2` bytes; the trailing crc
word makes a clean frame's residue zero. A nonzero residue is a
~1-in-65536-certain sign of payload corruption, so the frame is dropped
(silence gap) rather than shipped as a glitch.
## `acmod_channels` bit layout
This is the AUTHORITATIVE channel count for the track header: the DVD IFO
`audio_attr_t.channels` nibble is a well-known unreliable/stale field, so the
muxer prefers this over the IFO-claimed count (the bitstream acmod is
authoritative; the IFO audio nibble is not trusted). LFE adds one channel
(e.g. acmod=7 + lfeon → 6 = 5.1).
Bit layout from the syncword (A/52 §5.3.2 BSI):
```text
byte 5: bsid(5) | bsmod(3)
byte 6: acmod(3) | [cmixlev(2) if acmod has a centre and acmod!=1]
| [surmixlev(2) if acmod has surround]
| [dsurmod(2) if acmod==2] | lfeon(1) | ...
```
`acmod` therefore always occupies byte-6 bits 7-5; `lfeon` follows a variable
number of optional 2-bit fields, so the decoder tracks the bit cursor.
## `ACMOD_CHANNELS` table
Base channel count per AC-3 `acmod` (A/52 Table 5.8), BEFORE the LFE. Index is
the 3-bit acmod value; add 1 when `lfeon` is set.
```text
0 = 1+1 (Ch1, Ch2) -> 2 4 = 2/1 (L,R,S) -> 3
1 = 1/0 (C, mono) -> 1 5 = 3/1 (L,C,R,S) -> 4
2 = 2/0 (L, R) -> 2 6 = 2/2 (L,R,SL,SR) -> 4
3 = 3/0 (L,C,R) -> 3 7 = 3/2 (L,C,R,SL,SR) -> 5
```
## Test helpers and coverage notes
### `finalize_ac3_crc`
Relies on the CRC-16/ANSI residue property: appending `crc16([2..n-2])`
(big-endian) zeroes the register over `[2..n]`.
### `the_working_buffer_is_reused_across_packets_not_reallocated`
`parse` runs once per PES packet — of the order of 10^5 times on a feature's
audio track — and used to build a fresh `Vec` every call. The copy itself
cannot go without restructuring the borrow relationship, but the allocation
can: the buffer is taken out of `self`, filled, and put back with its
capacity intact. Asserting on capacity is deliberately white-box, because
that is exactly what regresses if someone reverts to `to_vec()` — the
behaviour would be identical and no other test would notice.
### `make_eac3_frame` byte layout
ETSI TS 102 366 Annex E: byte 2 is strmtyp(2) | substreamid(3) | frmsiz[10:8],
byte 3 is frmsiz[7:0], and the frame is `(frmsiz + 1) * 2` bytes. byte 4 =
fscod 0 (48 kHz), numblkscod 3 (6 blocks → 1536 samples → 32 ms), acmod 7 +
lfeon (5.1); byte 5 = bsid 16 so the E-AC-3 paths are taken.
### `a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet`
An access unit closes only at the next `substreamid`-0 independent substream,
so one that keeps gaining dependent substreams stays OPEN across PES
boundaries and its bytes stay in the carry-over. The carry-over must not be
re-scanned — and re-CRCed — from the access unit's first byte on every
packet: the buffer only stops growing at `MAX_AC3_BUF` (1 MiB), and a PES on
a DVD is about 2 KiB, so re-deriving the held access unit costs work
quadratic in the packets fed.
Measured directly, because a work bound has no frame-level shadow:
`frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
Mutation: pass `None` for `held` in `parse` (or drop the `if let Some(h) =
held` resume) — the count returns to the quadratic figure.
### `a_discontinuity_drops_the_held_access_unit_with_its_bytes`
`parse` clears `self.acc` on a discontinuity because the buffered bytes are a
truncated frame. The held access unit is described by OFFSETS into exactly
those bytes, so it has to go with them. Without `self.held = None`, the next
packet resumes a HeldAu whose `start`/`end` were computed against the
pre-gap buffer but are applied to the unrelated post-gap bytes — splicing
audio across the gap at best, and indexing past the end of the new, shorter
buffer at worst.
The two existing discontinuity tests use plain AC-3 (bsid < 11), which never
holds an access unit open, so neither of them reaches this reset.
### `a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes`
Mutation this catches: deleting (or inverting) the resume-path re-check `if
drop_reason.is_none() && self.tally.is_poisoned()` at the top of
`scan_access_units`. The verdict on a held access unit is frozen at the
moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
previous access unit is closed — so the very drop that crosses the poison
threshold lands after the next unit's verdict was already taken as `None`.
Without the re-check that unit is emitted as an ordinary frame after the
track has been judged too damaged to mux: corrupt audio passed through as
success, and worse, it is the ONE frame that escapes a whole-track fallback
whose entire point is that nothing after the verdict ships.
Neither existing held-AU test reaches this: both keep a pristine tally.
The fixture drives the exact interleaving above. `DropTally` poisons once
`verified_dropped * 2 > kept + dropped` past the 200-AU gate, so one PES
carries 200 CRC-failing `substreamid`-0 syncframes followed by a clean one.
Closing corrupt unit #200 (which happens only when the clean frame is
reached) latches the poison — after that clean frame's own verdict was
computed. Being an E-AC-3 unit that can still gain substreams, it is then
HELD across the boundary with `drop_reason: None`, which is precisely the
state the re-check exists for.
+37
View File
@@ -0,0 +1,37 @@
# ADTS decodability gate
## Module rationale (`src/mux/codec/adts.rs`)
Per the ADTS framing defined in ISO/IEC 13818-7 / ISO/IEC 14496-3, a header
is structurally invalid in exactly three ways this gate treats as hard
rejects: syncword != 0xFFF, a reserved `sampling_frequency_index` (the sample
rate table has 13 valid entries, so index >= 13 is reserved), and
`aac_frame_length < 7` (shorter than the fixed+variable header itself). The
optional 16-bit ADTS CRC is not verified here — it is simply skipped. So the
gate enforces those three rejects: a packet that begins with the ADTS sync
but is otherwise malformed is dropped; a packet with no ADTS sync is raw AAC
(e.g. from an MP4 container, which carries no ADTS header) or a continuation
and passes through unchanged — never false-dropped. Raw AAC has no per-frame
integrity data, so like LPCM it cannot be gated.
## Test: `a_crc_present_header_shorter_than_its_own_crc_is_invalid`
A header that CLAIMS a CRC (protection_absent = 0) but declares a frame
length too short to contain one.
`aac_frame_length` counts the header and the CRC, not just the payload,
so with a CRC present the smallest structurally possible frame is 9
bytes: the 7-byte fixed+variable header plus the 16-bit crc_check.
The gate compared against a flat 7 and never read protection_absent at
all, so a frame whose own header says it is impossible was classified
Valid and forwarded to the muxer.
## Test: `dropped_frames_are_counted_but_their_duration_is_not_invented`
A dropped ADTS frame is dropped BECAUSE its header failed validation, so
the very fields a duration would come from (sampling_frequency_index, and
the 1024-samples-per-AAC-frame constant applied to it) are the ones known
to be untrustworthy. This gate therefore reports the drop's duration as
zero rather than deriving a number from a header it has just rejected —
the honest answer, and the one the count alongside it must be read with.
A nonzero constant here would report silence that was never measured.
+217
View File
@@ -0,0 +1,217 @@
# `src/mux/au_assembly.rs` — access-unit assembly
## Why this exists
The contract a codec parser converts is `PES → access units (Frames)`. A
*transport* stream hands the parser one AU per PES for free (BD aligns one
access unit per PES; the TS demuxer reassembles to the
`payload_unit_start_indicator`). A *program* stream does not — the PS muxer
chops the elementary stream into fixed-size PES fragments with no AU
alignment, and only the first fragment of an AU carries a PTS. So a parser
that assumes one-AU-per-PES (h264/hevc/vc1, written against TS) mis-frames a
program stream, while `mpeg2` — the DVD/PS codec — must reassemble across PES.
`AuAssembler` is that reassembly, factored out so EVERY program-stream video
parser shares one implementation instead of hand-rolling the buffer. The
h264/hevc/vc1 parsers (`Mode::StartCode` / `Mode::Vc1`) and the MPEG-2 parser
(`Mode::Mpeg2`, via `AuAssembler::mpeg2`) all drive it. It buffers PES-fragment
bytes and emits one AU per codec AU boundary, carrying the AU-start
timing/source forward. Since the boundary is a codec start code, it lives with
the codec parser (which picks the marker); only the generic buffering +
timing-carry is shared here.
This is *inside* the parser, not a pipeline stage: the pipeline stays
`Demuxer → PES → Parser → Frames`, and the demuxer stays codec-agnostic. Every
stream a parser sees runs through one of these — self-framing codecs (MPEG-2,
audio) use `Mode::Passthrough` so the parser code path is uniform.
## `MAX_AU_BUFFER`
Safety cap on a single in-progress access unit. A real coded picture is far
below this; a stream that never yields a second AU boundary is force-flushed
at the cap rather than buffering without bound on hostile/corrupt input.
## `MAX_MARKS`
Cap on buffered timing/discontinuity marks. A real access unit spans a few
hundred PES fragments at most; this bounds the mark deques so a run of
zero-length (or start-code-free) timed fragments — which grow no buffer bytes
and so never trip the `MAX_AU_BUFFER` mark-prune — cannot accumulate marks
without bound on hostile/corrupt disc input.
## `AuAssembler::for_codec`
Video codecs whose parsers assume AU-complete PES (H.264 / HEVC / VC-1) get a
`Mode::StartCode` assembler; MPEG-2 (self-reassembles) and audio/subtitle
codecs (self-framing) get `Mode::Passthrough` so callers can run every stream
through this uniformly.
## `AuAssembler::mpeg2`
The MPEG-2 parser owns one of these directly (rather than hand-rolling the
buffer): the demux layer runs MPEG-2 through `Mode::Passthrough` and hands
each fragment to the parser, which feeds them here to be reframed on picture
boundaries.
## `AuAssembler::push`/`push_owned`
For a self-framing (`Passthrough`) stream the payload is MOVED straight into
the emitted unit with no copy — the common DVD/HD-DVD case (MPEG-2 video, all
audio). A buffering mode copies into `buf` exactly as `push`.
## `take_front`
Detaches `buf[..end]` as the emitted AU's own `Vec` and leaves `buf` holding
the tail.
The AU's bytes are HANDED OVER — `buf`'s allocation becomes the returned `Vec`
and a fresh buffer (pre-sized to the same capacity, so the next AU accumulates
without re-growing) takes its place holding only the short tail.
`buf[..end].to_vec()` + `drain(..end)` instead copied every AU out in full: on
a UHD HEVC title that is a whole-frame memcpy (hundreds of KB) per coded
picture, ~200k times, for bytes that are about to be discarded from `buf`
anyway.
The allocation COUNT is unchanged (one per AU either way — the frame `Vec`
before, the replacement buffer now), so the only difference is the copy that
no longer happens. Nothing depends on `buf` keeping its identity: the only
state tied to `buf[0]`'s position is `base`/`scan_pos`/`opener_pos`, which the
caller updates immediately after.
Falls back to a copy when the buffer's capacity is far larger than the AU (a
small AU after a multi-MB one): handing over would otherwise attach an
oversized idle allocation to a small frame for as long as the frame queues
downstream, trading a copy for resident memory.
That fallback must not become permanent. `buf`'s capacity used to be a one-way
high-water mark — the replacement buffer was created with `cap.max(tail_len)`,
and the copy path's `drain` also preserves `cap` — so once ONE large AU had
been assembled, every later smaller AU satisfied `cap > 2*end` and took the
copy path forever. On a UHD HEVC title the first IDR grows `buf` to ~4-8 MB,
after which each ~200-400 KB P/B AU paid a whole-AU allocation plus a
whole-AU memcpy plus a tail memmove for ~99% of the ~200,000 coded pictures —
tens of GB of exactly the memcpy this handover exists to remove. So the copy
path now also RELEASES the high-water capacity, which re-arms the handover
for the next AU: one copy after a size step down, not one per frame forever.
## `drop_marks_before`
This is the STREAM-START case: bytes ahead of the first AU boundary are the
tail of an access unit that began before we had sync, and there is no prior
AU for them to be discontinuous *from*. Carrying a mark forward here would arm
the resync gate at the head of every title and drop its first GOP.
## `discard_gap_before`
This is the BACKSTOP case: `MAX_AU_BUFFER` bytes accumulated with no AU start
code in them, so the run is unusable and gets thrown away. Unlike the
stream-start trim above, there IS a prior AU here, and whatever follows
definitively does not continue it — a decoder handed the next picture would
resolve its references against frames separated from it by megabytes of
discarded data.
So the discard is itself a discontinuity, whether or not the source signalled
one. It is recorded as a sticky flag rather than an offset mark because a
mark placed at the new base would be retired moments later by the pre-sync
trim that follows resync — the gap has to outlive the bytes that caused it.
It arms the resync gate, which drops to the next keyframe instead of emitting
a picture with dangling references.
Timing marks before `off` are still retired — they describe bytes that no
longer exist, and the AU that eventually emits takes its PTS from the
fragment that actually opened it.
## Tests
### `a_backstop_discard_marks_the_next_au_discontinuous`
The 8 MiB backstop throws away a start-code-free run as unusable. The AU that
eventually emits after that discard MUST be marked discontinuous, whether or
not the source ever signalled a discontinuity: megabytes of the stream are
simply gone, so the next picture cannot resolve its references against the
last one that was emitted.
`discontinuity` is what arms the resync gate downstream (`resync.rs`, driven
from `mux/disc.rs`), which drops to the next keyframe rather than emitting a
picture with dangling references. If the flag is retired with the discarded
bytes, the gate never arms and the broken picture goes out — a silent
corruption, which is the one class of loss this crate refuses to have.
### `a_stream_start_trim_does_not_mark_the_first_au_discontinuous`
The opposite case, and the reason the two call sites are separate. Bytes
ahead of the FIRST access-unit delimiter are the tail of an AU that began
before we had sync. There is no prior AU for them to be discontinuous from, so
retiring the marks there is right — and necessary: marking the first AU of
every title discontinuous would arm the resync gate at the head of each one
and drop its opening GOP.
### `a_source_signalled_discontinuity_reaches_the_au_it_opens`
A source-signalled discontinuity reaches the AU it opens. This is the
`disc_marks` path — the ORIGINAL mechanism, distinct from the sticky
`pending_gap` the backstop sets. Nothing else pins it: the two tests above
drive `pending_gap`, and a mark placed on a fragment that is later discarded
is retired by design.
Deliberately NOT combined with the backstop. A previous version of this test
signalled the discontinuity on the first over-cap push and asserted the flag
on the AU after the discard — but that first run still has the next AU's
delimiter at `buf[0]`, so it force-flushes as an over-long AU, and THAT AU
consumes the mark. The assertion was then satisfied entirely by
`pending_gap`, making the test a duplicate of the one above it under a name
promising something else. The two mechanisms cannot be isolated in one
fixture, so they get one test each.
### `drained_au_takes_over_the_buffer_allocation_without_copying`
MEASURED: a drained AU must be HANDED the accumulation buffer's allocation,
not copied out of it. The emitted `Vec`'s data pointer is the buffer's own
pointer — which is only true if no full-frame copy happened.
(`buf[..end].to_vec()` allocates fresh, so the pointers differ.) One
whole-AU memcpy per coded picture is ~200k memcpys of a few hundred KB each
on a UHD feature.
### `handover_survives_a_large_au_instead_of_copying_every_later_one`
MEASURED: `take_front`'s copy fallback must not become permanent.
`buf`'s capacity used to be a one-way high-water mark, and the copy path's
`drain` preserves it, so after ONE large AU every later smaller AU satisfied
`cap > 2*end` and copied forever. On a UHD HEVC title the first IDR grows
`buf` to multiple MB, after which ~99% of the ~200,000 coded pictures each
paid a whole-AU allocation + whole-AU memcpy + tail memmove — tens of GB of
exactly the copy the handover exists to remove. Counted at the copy path
itself: one copy is expected right after the size step down; a per-frame
copy is the bug.
### `discarded_pre_sync_marks_do_not_time_the_first_access_unit`
After the pre-sync bytes are discarded, the emitted AU must take the timing
of the fragment that ACTUALLY opened it. `drop_marks_before` is what retires
the discarded fragment's marks; a no-op there stamps the first real access
unit with the PTS and source of bytes that were thrown away — a whole-title
A/V sync offset, since every later frame is timed relative to it.
### AU-opener detection: the per-mode start-code rule
`au_opener_from` duplicates a rule each codec parser also encodes; the tests
below pin it to the normative byte values so a drift between copies shows up.
### `au_opener_from_locates_the_real_start_code_per_codec`
The opener offset must be the position of the real start code, never a fixed
0. A constant `Some(0)` makes every pre-sync run of junk bytes look like the
head of an access unit, so the first AU of every stream that does not begin
exactly on a start code is emitted with junk glued to its front.
### `for_codec_routes_each_video_codec_to_its_reassembly_mode`
`for_codec` is the dispatch that decides whether a stream is REASSEMBLED
across PES fragments or passed straight through. Getting it wrong is silent:
an H.264/HEVC/VC-1 stream routed to `Passthrough` on a program source emits
one "frame" per PES fragment — a few hundred bytes of a coded picture, framed
as a whole access unit — and the output plays as corruption, not as an error.
Each mode is identified BEHAVIOURALLY (feed a two-AU stream in two halves and
see whether it reassembles), so the case cannot pass by matching a constant.
+42
View File
@@ -0,0 +1,42 @@
# BDMV disc-library metadata (`bdmt.rs`)
Every commercial Blu-ray carries a disc-library metadata directory with one
XML file per shipped language. The schema is the Blu-ray "disc library
metadata" namespace (`urn:BDA:bdmv;disclibmeta`), conventionally prefixed
`di:`. Fields commonly present:
- `<di:title>` or `<di:name>` — the title string. Vendor practice varies
(Paramount discs tend to use `<di:name>`).
- `<di:description>` — optional synopsis (often absent on retail discs;
common on box sets and special editions).
- `<di:discNumber>` / `<di:numSets>` (or `<di:numberOfSets>`) — set position
for multi-disc releases.
This module is intentionally separate from the BD-J `StreamLabel` parsers
under `labels/*.rs`. The XML here is disc-level (title, description, set
position), not per-stream.
Real-world XML is irregular: missing description elements, multiple title
elements (first one wins), and occasional malformed content. Extraction is
best-effort — a malformed file is treated as "no metadata" (returns `None`
from the helper), and the caller can still get metadata from
sibling-language XML files.
## `MAX_BDMT_BYTES`
The size comes from attacker-controlled UDF metadata; real files are a few
KB, so 1 MiB is generous while preventing a crafted huge-size entry from
triggering an oversized allocation in `read_file`.
## `parse_bdmt_xml`
Title-element preference: `<di:name>` → `<di:title>` →
`<di:tableOfContents>/<di:titleName>` (first match wins, per the
authoring-tool conventions documented above).
## `looks_like_xml`
Rejects candidate description strings that are themselves XML fragments —
observed on a captured disc, where `<di:description>` contained
`<di:thumbnail href="…"/>` child elements and no actual prose. Surfacing
that raw to the JSON output is worse than dropping the field entirely.
+21
View File
@@ -0,0 +1,21 @@
# bdnav — HDMV navigation resolver
Notes relocated from `src/bdnav/mod.rs` doc comments (kept out of the source
to respect the comment-guard's prose caps).
## No per-disc special-casing
There is no per-disc special-casing: one correctly-booted VM resolves
densely-branched dispatchers as well as simple ones. The honest boundary is
BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
and selection falls back to the structural/heuristic order.
## `resolve_feature`
Resolves the playlist id the disc's First-Play navigation plays as the
feature, restricted to ids the caller marks as feature candidates
(`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J
discs, missing/malformed nav data, or when navigation does not converge on
a candidate.
+239
View File
@@ -0,0 +1,239 @@
# src/disc/bluray.rs — extended notes
Overflow prose for comments that were trimmed to the internal 3-line cap in
`src/disc/bluray.rs`. Each section is pointed to by a `// See docs/bluray.md
— <topic>` comment at the corresponding site.
## CLIP_STREAM_EXTS
Stream-file extensions probed for a BD-family playlist clip, in priority
order. A clip is normally `.m2ts`; AACS 2.1 (FMTS) discs name the main
feature `.fmts` (an M2TS transport stream plus forensic variant segments)
and 3D discs use `.ssif`. `.m2ts` is tried first, so a normal clip is
unaffected — the fallback only runs when `.m2ts` is absent (exactly when
`file_extents` errors).
Scope: these are all variants that live in `BDMV/STREAM/` and are reached
through an MPLS playlist. HD-DVD's `.evo` does NOT belong here — HD-DVD is
a different tree (`HVDVD_TS/`) with `.XPL` playlists and needs its own
enumerator (a peer to `parse_playlist`), not another extension in this
list.
## scan_bluray_titles
Cancellation: `halt` is polled between playlists and once more after the
loop, and a read that fails with `Error::Halted` — how a live drive
reports a Stop, since `Drive::checked_exec` fails every command once its
flag is set and `Drive::read` preserves the variant — is propagated rather
than swallowed. `Halted` is the only error this returns from the
enumeration itself; an unreadable or unparseable playlist keeps its
best-effort skip.
It has to be an error and not a short title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it. Before this, a Stop
pressed mid-scan failed every remaining `.mpls` read in turn, each one
silently skipped, and the scan returned `Ok(truncated)` at rc=0.
## parse_playlist
Sums PlayItem durations; returns `Ok(None)` if the playlist is under 30
seconds (skips menu / clip-info stub playlists), fails to parse, or names a
clip that cannot be resolved. Physical sector extents are pulled from the
UDF allocation descriptors of each referenced `.m2ts` (deduplicated by
clip_id).
`Ok(None)` keeps its two benign meanings (unparseable MPLS, sub-30s
playlist) plus the deliberate "drop this title" outcomes below. `Err` means
the scan is over: today that is only `Error::Halted`, the operator's Stop,
which must not be reported as a disc that simply holds fewer titles.
## Test: parse_playlist_keeps_exactly_30_seconds
At exactly 30s the playlist is kept (`< 30.0` is strict). The fixture is a
fully wired BDMV (`make_bdmv_fs`), not the bare `make_min_fs` this used to
use. `make_min_fs` lays no STREAM/CLIPINF, so the play item's `.clpi` read
now fails and the title drops — which is correct behaviour for an
unresolvable clip but has nothing to do with the 30-second boundary this
test exists to pin. Wiring the clip keeps the test measuring exactly one
thing.
## make_bdmv_fs_ssif
Full BDMV with a real Blu-ray 3D layout: `.ssif` files under
`BDMV/STREAM/SSIF/<clip>.ssif` (note the SSIF subdirectory, unlike
`make_bdmv_fs_ext`) plus a matching `.clpi` in CLIPINF. Resolving the SSIF
is what latches `is_3d = true` in `parse_playlist`.
## Test: parse_playlist_fmts_clip_resolves_extent
AACS 2.1: the feature clip is `00001.fmts`, NOT `.m2ts`. The
`CLIP_STREAM_EXTS` fallback in `parse_playlist` must still resolve the
physical extent — before the fix the hard-coded `.m2ts` path errored,
yielding empty extents (a silent empty rip and 0 encrypted samples for key
resolution). Size still comes from the `.clpi`, which parses regardless.
## Test: parse_playlist_dedups_repeated_clip_extents_and_size
THE 0.31.0 DEDUP PATH. A playlist that references the SAME clip_id from
multiple PlayItems (seamless split / looped segment) must count the
physical extents and packet bytes exactly once — mux reads extents in
order, so a duplicate would mux the A/V twice and inflate `size_bytes`
(bluray.rs: `first_ref = seen_clips.insert(...)` gates both `total_size +=`
and the `extents.push`). Per-PlayItem Clip entries are still recorded for
both.
## Test: parse_playlist_missing_clpi_yields_no_title
A clip whose `.clpi` is missing must yield no title.
This test used to assert the opposite — a title with `size_bytes == 0` and
empty extents — and its docstring quoted the buggy control flow ("bluray.rs
only fetches extents inside the `if let Ok(clpi_data)`") as if it were the
specification. It was blessing the defect: the title's `duration_secs` is
summed from the PlayItems before the clip is resolved, so the returned
title advertised the movie's full runtime while carrying none of its
bytes, and the discarded error meant not one log line said so. That is the
flagship failure class of this crate — a failure that looks like success —
reached through an ordinary missing or scratched CLIPINF file.
The correct behaviour is the same as for a clip whose extents cannot be
resolved (see `parse_playlist_unreadable_clip_icb_yields_no_title`): drop
the title and log the read's own error code.
That last clause is now asserted, not merely asked for. The site's comment
insists on `e.code()` because a missing `.clpi` (E6003), a scratched one
(E6000) and a malformed one (E6002) are different populations, but nothing
checked it: putting a literal back compiled and passed. Mutation: `"E6017"`
(or any fixed code) in place of `"E{}", e.code()` fails here, and so does
dropping the warn entirely — a silent drop is the same invisible title
loss this test was written for, one step later.
## Test: parse_playlist_unrecorded_extent_yields_no_title
A clip stream whose ICB declares an UNRECORDED (ECMA-167 4/14.14.1.1
type-1) extent must not yield a title at all.
The extent is allocated to the file but was never written, so the file's
content there is zeros while the media holds whatever was left at those
sectors. Neither answer a `(lba, sector_count)` read plan can give is true
— reading it splices undefined sectors into the rip as content, dropping
it slides every later extent's byte space — so the title is refused rather
than mis-ripped. This fixture is the shape a crafted disc uses to get such
a range into a title's extent list.
(The `sectors > 0 && lba > 0` filter below the resolver stays as defence in
depth; a zero-length AD is only reachable as an unrecorded descriptor,
since a zero-length TYPE 0 one terminates the AD list.)
## Test: parse_playlist_unreadable_clip_icb_yields_no_title
An unrecorded extent was never the only way a clip fails to resolve.
RED BEFORE GREEN: this fixture gives the .m2ts an ICB whose descriptor tag
is neither 261 nor 266, so `file_extents` returns `DiscRead` — the same
variant a scratched sector under a real clip's ICB produces, which is the
ordinary way this happens on real media. Before the fix only
`UdfUnrecordedExtent` set the drop flag, so this fell through to the "file
absent" path and `parse_playlist` returned a title: full declared duration
from the play item, `total_size` already counted from the .clpi, and zero
extents — a movie advertising its runtime with the content missing, and
not one log line. The clip must drop the title exactly as an unrecorded
extent does.
## Test: parse_playlist_logs_a_non_absence_ssif_failure_the_m2ts_fallback_hid
A non-absence SSIF failure that the `.m2ts` fallback then papers over must
be logged, with its own code.
`unresolved` had exactly one reader — `if let (None, Some(code))` — so a
code recorded for `/BDMV/STREAM/SSIF/<clip>.ssif` was thrown away whenever
the fallback succeeded. The disc here IS a 3D disc: the SSIF is present and
carries both eyes, and only its ICB is unreadable (a scratched sector, the
ordinary way this happens). The title shipped base-view 2D at rc=0 with
`is_3d` false and not one log line, so the operator's rip is silently
missing the dependent view and the journal cannot tell this disc apart
from one that was only ever 2D.
The title is deliberately still returned — the base view resolved, so
refusing would trade a degraded rip for no rip. The defect being fixed is
the silence, not the fallback.
Mutations this catches: deleting the new `(Some(_), Some(code))` arm, or
restoring the `if let (None, Some(code))` shape, leaves no event to find;
logging a fixed code instead of `e.code()` fails the code assertion;
making the arm `return Ok(None)` fails the title assertion.
## Test: parse_playlist_id_falls_back_to_zero_when_suffix_is_not_mpls
A filename that is long enough (>= 5 bytes) but does NOT end in ".mpls"
must NOT have its last 5 bytes stripped — the whole string is handed to
the numeric parse instead, which fails and falls back to playlist_id 0
(bluray.rs `filename.len() >= 5 && filename[len-5..].eq_ignore_ascii_case(".mpls")`).
## Test: scan_bluray_titles_skips_non_mpls_extension_file
A non-directory PLAYLIST entry whose name does NOT end in ".mpls" must be
skipped even though its content parses as a perfectly good (long) MPLS
playlist — extension gating, not content sniffing, decides eligibility
(bluray.rs `!entry.is_dir && entry.name...ends_with(".mpls")`).
## HaltingReader
A `SectorSource` that fails every read in `halt_range` with
`Error::Halted` — exactly how a live drive behaves once the operator
presses Stop: `Drive::checked_exec` fails every SCSI command with `Halted`
from then on, and `Drive::read` deliberately preserves the variant. Reads
outside the range still succeed, so a test can aim the cancel at one
structure and leave the scan far enough along to have something to
truncate.
## Test: halted_playlist_read_is_not_reported_as_a_shorter_disc
A Stop on a live drive never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The Blu-ray enumerator must not swallow that into a
successful scan.
RED BEFORE GREEN: with the propagation reverted this returned
`Ok([00800])` — the `if let Ok(mpls_data)` skipped the cancelled read of
00801.mpls, the loop ended, and a half-enumerated disc came back at
success. One title from a two-title disc is indistinguishable from a disc
that genuinely holds one title, and the caller caches and rips from it.
The halt lands on the second playlist deliberately: it is the last
iteration, so nothing after it would poll a flag — only propagating the
read's own error catches it.
## Test: halted_clpi_read_is_not_accounted_as_an_unresolvable_clip
The same cancel landing on a `.clpi` read must not be classified as an
unresolvable clip either.
RED BEFORE GREEN: with the `Err(Error::Halted)` arm removed from the
CLIPINF match, the cancel fell into the generic "clip could not be
resolved" arm, which logs a disc-defect code and drops the title —
accounting an operator Stop as a scratched disc, and (in the scan loop)
dropping every remaining playlist in turn for a truncated `Ok`. With the
fix the cancel is propagated with its own variant intact.
## Test: halted_extent_resolve_is_not_a_title_missing_its_clip
And the same cancel landing on `file_extents` — the clip's ICB, not its
CLIPINF — must propagate too.
RED BEFORE GREEN: `note` used to exempt `Halted` from the unresolved
classification (correctly — a cancel is not an authoring hole) but had no
way to propagate it, so the resolver simply produced no extents and
`parse_playlist` returned a title with the clip's runtime counted, its
`size_bytes` counted from the .clpi, and zero bytes behind it. Measured
with the propagation reverted: `Ok(Some((768000, [])))` — the flagship
defect shape, wearing a cancel.
## Test: read_meta_title_ignores_non_xml_file_regardless_of_content
A non-.xml file must be ignored even if its content looks like a valid
meta XML (contains a `<di:name>`) — extension gating, not content
sniffing, decides eligibility (bluray.rs `!e.is_dir &&
e.name...ends_with(".xml")`).
+61
View File
@@ -0,0 +1,61 @@
# `src/io/bounded.rs` — bounded-syscall primitive
## Why this exists
[`crate::halt::Halt`] is cooperative: callers poll `is_cancelled()`. It
cannot reach inside a syscall the kernel currently owns the thread for —
`libc::sync_file_range`, `libc::fsync`, `File::write` on NFS, and so on.
`/api/stop` from autorip therefore can't unstick a thread sitting in
such a syscall.
`bounded_syscall` is the escape hatch: it runs `op` on a fresh worker
thread, then `recv_timeout`s on a rendezvous channel for the result.
The wait is broken into ~250 ms slices so the calling thread can poll
the supplied `Halt` in between. If the deadline elapses or the halt
fires, the worker is intentionally leaked — the syscall will unwind
whenever the kernel decides, or at process exit, but the caller is
free to fall back to a degraded code path (skip the sync, log loudly,
etc.).
## Trade-offs
- **Thread per call.** Cheap (`std::thread::spawn` is < 100 µs on
Linux/macOS), but not free. Use on coarse-grained finalisation
syscalls (`sync_all`, `sync_file_range(WAIT_AFTER)`), not on hot
inner-loop writes.
- **Leak on timeout.** A wedged syscall keeps a kernel slot and a
user-space thread around for the rest of the process's life. Bounded
by the number of independent rip/mux sessions, which is one per
disc. The alternative — trapping the caller forever — defeats the
entire purpose of `/api/stop`.
- **Halt granularity ~250 ms.** Halt observation is not instant; it's
the worst-case latency of the `recv_timeout` slice. Good enough for
human-driven stop requests; not suitable for hard real-time
deadlines.
## Single source of truth
Do NOT inline this pattern. Every blocking-syscall wrapper in the rip
+ mux pipeline calls this helper, so changes (e.g. swapping the
channel impl, adjusting the poll slice, adding metrics) land in one
place.
## Platform
Pure `std::thread` + `std::sync::mpsc`. No `cfg(target_os)` needed
here — the helper itself is platform-agnostic. Callers that wrap
Linux-only syscalls (`sync_file_range`) still need their own
`#[cfg(target_os = "linux")]` gates; this helper does not.
## `bounded_syscall` contract details
- `halt` is polled at `POLL_INTERVAL` granularity. Pass `None` for
callers that don't (yet) have a halt token plumbed through —
behaviour degrades to deadline-only, matching the 0.20.5
`wait_after_with_timeout` shape this helper generalises.
- `op` returns `R: Send + 'static`. The closure must own everything it
touches because it may outlive this call (timeout / halt cases).
- On `Halted` / `Timeout` the worker thread is intentionally leaked:
the syscall will unwind whenever the kernel decides, or when the
process exits. The calling thread is never trapped inside a kernel
call.
+111
View File
@@ -0,0 +1,111 @@
# `byte_prefetcher` — design notes and test rationale
`BytePrefetcher` (`src/io/byte_prefetcher.rs`) is the `std::io::Read`
analogue of `crate::sector::PrefetchedSectorSource`. It spawns a
producer thread that fills a bounded pool of `Vec<u8>` chunks from the
underlying reader and ships them through a channel; the consumer pulls
filled chunks, uses them, and sends the empty `Vec<u8>` back through a
recycle channel so the producer can re-fill in place. Result: zero
allocations and zero cross-thread frees in the steady-state hot loop.
This is the byte-stream half of the freemkv mux highway —
`BytePrefetcher` feeds `crate::mux::demux_thread::DemuxThread` for
`m2ts://` (the only in-tree caller today, via `crate::mux::resolve`),
and works for any stream whose source is an `io::Read` rather than a
`SectorSource`.
## `PrefetchShell`
Drop blocks the calling thread until the producer exits. To guarantee
a prompt exit, drop the forward receiver and the recycle sender first
so the producer observes channel disconnection (or cancel the `Halt`
passed to `BytePrefetcher::new`, which the producer polls at
`POLL_INTERVAL` granularity even while parked on a channel op).
## Test coverage rationale
- **`recycle_depth_is_forward_depth_plus_one`** — `RECYCLE_DEPTH` must
be one MORE than `FORWARD_DEPTH` per its own doc comment: the
producer needs at least one buffer to fill while the consumer holds
the other `FORWARD_DEPTH`-worth in flight. A `+` -> `*`/`-` mutation
on `FORWARD_DEPTH + 1` would under-size the recycle channel (e.g.
`FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one short), which starves the
producer of a spare buffer.
- **`into_channels_drop_releases_producer`** — the CRITICAL
regression: after `into_channels`, dropping the returned forward
receiver + recycle sender must let the producer observe
disconnection and exit, so dropping the `PrefetchShell` (which
joins the producer) returns promptly. With the old clone+forget the
leaked endpoints kept the producer blocked and this join hung
forever.
- **`delivers_all_bytes_in_order_across_chunks`** — CORE CONTRACT: the
prefetcher must deliver every source byte, in order, exactly once —
never silently truncate or duplicate. Source is 5000 bytes; chunk
size 1024 forces multiple chunks (4 full + 1 short of 904). The
reassembled stream must equal the source. Mutation: replacing
`buf.truncate(n)` with a no-op would over-report bytes on the final
short read and this fails.
- **`short_read_truncates_to_actual_length`** — a reader that returns
fewer bytes than requested per call must NOT leave stale tail bytes
in the delivered chunk. Cursor over 10 bytes with a 4096 chunk
yields a single 10-byte chunk; the consumer must see exactly 10
bytes, not 4096. Grounds `buf.truncate(n)`. Mutation: delete the
truncate and the chunk would carry 4086 zero bytes of padding,
failing the length assert.
- **`empty_source_yields_clean_eof_no_batches`** — EOF semantics: an
empty source (Cursor over `[]`) yields `read() == Ok(0)` on the
first call, which the producer treats as EOF and returns, dropping
tx. The consumer sees RecvError (zero batches), NOT an Err batch and
NOT a zero-length Ok batch. Grounds the `Ok(0) => return` arm.
Mutation: changing `Ok(0) => return` to `Ok(0) => continue` would
spin forever (test times out).
- **`read_error_is_propagated_as_err_batch`** — a reader that fails
mid-stream must surface the `io::Error` as an `Err` batch on the
forward channel, not swallow it. We deliver one good chunk then an
error. The consumer must see the good bytes followed by the error.
Mutation: changing `let _ = tx.send(Err(e)); return;` to a plain
`return` would drop the error silently and this fails.
- **`read_panic_surfaces_as_err_batch_not_clean_eof`** — PANIC
propagation: a reader that PANICS mid-stream must NOT be read as a
clean EOF at the demux boundary. The producer's `catch_unwind` sends
an explicit `Err` sentinel before the thread unwinds, so the
consumer sees the good bytes followed by an error batch — never a
silent truncation. Without the `catch_unwind` the panic would just
drop `tx`, the consumer would see RecvError (== clean EOF) and the
partial output would be finalized as if complete.
- **`recycled_buffer_carries_no_stale_tail`** — recycle-buffer reuse
must NOT leak stale bytes between chunks of different lengths. After
a full chunk, a short read reuses the same recycled buffer; the
regrow-then-truncate sequence must yield only fresh bytes. Verified
by reassembling the full stream. Source: 8 bytes of 0xAA + 3 bytes
of 0xBB, with chunk_bytes=8 → chunk0 = 8×0xAA, chunk1 = 3×0xBB.
- **`exact_multiple_length_no_trailing_empty_batch`** — exact-multiple
boundary: when the source length is an exact multiple of
chunk_bytes, the final non-empty chunk is followed by an `Ok(0)` EOF
read, NOT a spurious empty Ok batch. 12 bytes with chunk_bytes=4 →
three 4-byte chunks then clean EOF. Total bytes must equal 12 and no
zero-length batch may appear.
- **`drop_finite_prefetcher_joins_cleanly`** — dropping the
`BytePrefetcher` directly (without `into_channels`) must join the
producer cleanly when the source is finite. The producer reaches
EOF, drops tx, and exits; Drop's join returns. Grounds the
`BytePrefetcher` `Drop` impl. Mutation: removing the `Ok(0) =>
return` EOF exit would hang this join.
- **`drop_endless_prefetcher_joins_cleanly`** — regression: dropping a
`BytePrefetcher` directly (without `into_channels`) with an ENDLESS
source must not deadlock. Before the fix, `Drop` joined the producer
while `rx`/`recycle_tx` were still alive (sibling field drop order),
so the producer filled the depth-2 forward channel and then spun in
`send_timeout` forever (rx never drained, halt=None). The fix drops
`rx`+`recycle_tx` BEFORE the join so the producer sees
`SendTimeoutError::Disconnected` and exits.
+48
View File
@@ -0,0 +1,48 @@
# `labels::class_reader`
Hand-rolled JVM `.class` file reader, tailored to the subset needed for BD-J
label extraction (Deluxe / dbp / similar frameworks). Implements JVMS §4
(class file format) and §6 (bytecode) only as far as: constant pool,
methods, the `Code` attribute, and a non-allocating bytecode iterator.
No external deps beyond `std`. No `unsafe`. No panics on malformed input —
every parse fault is a typed `Error`. This is shared infrastructure for any
label parser that needs structured access to `.class` files inside a
`/BDMV/JAR/<x>.jar`.
## `ConstantPool::from_entries`
Test-only constructor — builds a constant pool directly from a vector of
entries. Real callers go through `ClassFile::parse`, which builds this from
class-file bytes. Used by parser unit tests (e.g. `labels::deluxe`) that
need to exercise bytecode walkers against synthetic class fixtures without
hand-rolling valid `.class` byte buffers.
Caller is responsible for: prepending a `CpInfo::Empty` at index 0 (the
spec-reserved slot), and inserting a `CpInfo::Empty` after each Long/Double
entry (the 2-slot quirk).
## `decode_modified_utf8`
Decodes JVM "modified UTF-8" (JVMS §4.4.7). Practically identical to
standard UTF-8 for the BMP-printable subset seen in label strings, but with
two notable deviations:
- U+0000 is encoded as the two-byte sequence `0xC0 0x80`, not as `0x00`.
- Supplementary characters (U+10000..) are encoded as a UTF-16 surrogate
pair, each surrogate emitted as 3-byte modified UTF-8.
For label data (mostly ASCII / Latin-1 / CJK in BMP), the simple
implementation here covers everything we'll encounter. The `0xC0 0x80` →
U+0000 case is tolerated explicitly; supplementary characters would need
surrogate-pair stitching, but no label-relevant string uses them.
## `Reader::slice` overflow test
`Reader::slice` takes an attacker-supplied length: a JVMS `u4`
`attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8 length
(§4.4.7). Adding it to `pos` without a wrap check panics on overflow in
debug and, in release, wraps to a small end offset that slips past the
bounds check and then panics inside the slice index. Both are panics
escaping a parser whose whole input is untrusted disc bytes; the contract
is an EOF error.
+18
View File
@@ -0,0 +1,18 @@
# CLPI vs MPLS cross-validation diagnostic
`src/labels/clpi_audit.rs` walks both per-clip CLPI program info and
per-playlist MPLS STN tables, normalizes their stream lists by `(PID,
language, coding_type)`, and classifies each PID into one of four
buckets:
1. **CLPI only** — a stream present in a `.clpi` ProgramInfo that no
playlist STN table references (orphan on disc).
2. **MPLS only** — a stream a playlist references that no `.clpi`
ProgramInfo lists (indicates a parser disagreement).
3. **Match** — both sources agree on coding_type and language.
4. **Divergent** — both sources see the PID but disagree on
coding_type or language.
`audit` returns a structured `ClpiVsMplsAudit` report. This is a
diagnostic surface only; it does not feed the label-selection
pipeline.
+36
View File
@@ -201,6 +201,42 @@ The full ripping pipeline chains three parsers:
The `Disc::scan()` method in `src/disc/mod.rs` orchestrates this: for each play item in each playlist, it loads the corresponding CLPI, calls `get_extents()` with the play item's in/out times, and collects the resulting sector ranges into the title's extent list.
## ProgramInfo Section (Per-Stream Metadata)
`parse_program_info` in `src/clpi.rs` reads per-stream (pid, coding_type,
language, codec sub-fields) from the ProgramInfo section, per the Blu-ray
Disc Read-Only Format Part 3 CLIPINF (CLPI) specification:
```text
ProgramInfo:
length: 4 bytes
reserved: 1 byte
num_programs: 1 byte
for each program:
spn_program_sequence_start: 4 bytes
program_map_pid: 2 bytes
num_streams: 1 byte
num_groups: 1 byte
for each stream:
pid: 2 bytes
stream_coding_info_length: 1 byte
stream_coding_info: (varies by coding_type)
coding_type: 1 byte
per-type bytes (see match arms in parse_program_info)
```
It returns `Vec::new()` on any structural mismatch — errors aren't
propagated because the EP map is the primary CLPI output, and a corrupt
program_info shouldn't break sector-range lookups.
### Test fixture notes
- `prog_info_start_zero_does_not_parse_header_as_program_info`: the
fixture is crafted so that parsing from offset 0 WOULD yield a stream
(num_programs at `[5]`, a second program header whose num_streams
byte at `[20]` is 1, then a well-formed stream record), so an empty
result can only come from the `prog_info_start > 0` gate in `parse`.
## References
- BD-ROM Part 3, Section 5.5: Clip Information file format
+79
View File
@@ -0,0 +1,79 @@
# `src/mux/codec/mod.rs` — extended notes
Relocated prose from doc comments in `mod.rs`, trimmed there to stay within
the comment-guard's prose caps. Each `//`/`///` pointer in the source names
the section here it corresponds to.
## `Frame::discontinuity`
Carried per-FRAME (not per-PES) because buffering parsers decouple the frame
from the PES that carried the gap signal: MPEG-2 emits whole GOPs, H.264/HEVC
lag one access unit behind the PES that produced them. A clean rip leaves
every frame `false`; the flag is only ever set on the degraded/conceal path
(P3/B1), when a frame's data begins after packets the demuxer never received
— an undecryptable unit concealed as NULL-TS upstream, or a continuity break
in a damaged source.
## Drop-on-undecodable policy across codecs ("clean muxes always")
- **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
through a per-codec corruption check and drops the ones that fail, keeping
A/V sync (a drop is a silence gap, never a shift) and logging every drop
via the shared `dropgate::DropTally`. DTS validates via its core-frame
header (ETSI TS 102 114); AC-3 uses its native frame CRC.
- **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
data, so a corrupt sample is indistinguishable from a quiet one — there is
nothing to detect, so nothing can be honestly dropped.
- **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
predicted, so dropping one frame corrupts every frame that references it
until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
ResyncGate) and lets the decoder conceal — a fundamentally different model
than per-frame audio dropping.
- TrueHD/MLP, FLAC, MP2/MP3 and AAC-ADTS also gate undecodable frames via a
`DropTally` (poison/drop-forward for MLP's inter-AU restart state on a
major-sync boundary; CRC/sync-verdict drops for the passthrough codecs).
## `codec_private()` — why "absent" is the only truthful answer for some parsers
A codec parser's `codec_private()` feeds `DiscStream::codec_private`, which
the MKV muxer turns directly into the track's `CodecPrivate` element (RFC
9559 §5.1.4.1.24): `Some(bytes)` writes an element holding exactly those
bytes, `None` omits the element entirely. The two are NOT interchangeable —
a zero-length `CodecPrivate` asserts that the codec's initialisation data IS
empty, which is not true of any codec, and a one-byte one asserts a config no
decoder can parse.
The gating parsers (ADTS, MPEG audio, FLAC) and the passthrough parser
extract no configuration at all: they validate and forward frames whose
configuration is carried in band (ADTS headers, MPEG-1 audio frame headers,
FLAC frame headers) or supplied by the source container. Having derived
nothing, the only truthful answer they can give is "absent" — any `Some`
would be a value they invented. Nothing else in the test suite distinguished
the two, so each of these impls could have returned a fabricated `Some` and
produced a malformed track header unnoticed.
## `provenance_guard` test module
Every emitted frame must carry the source byte offset of the packet it came
from. That invariant held only for video for as long as it existed: `dts`,
`ac3`, `adts`, `truehd`, `pgs`, `dvdsub`, `flac`, `lpcm`, `mpegaudio` and the
passthrough parser all built frames with `source: None`, so a multi-clip
title could not place audio or subtitles by byte and fell back to inferring
the clip from timestamps — which is what made branched titles run minutes
long.
Nothing asserted it, so nothing caught it. Finding it took a brace-balanced
scan of the tree by hand, which also turned up five sites a regex had
missed. `provenance_guard` is that scan, as a test.
### `code_only` / `frame_literals` design
`frame_literals` walks `Frame { .. }` literals with balanced braces. A regex
cannot do this — `Frame` blocks contain nested braces (`coding: Some(..)`,
closures) and a non-greedy match stops at the first `}`, which is how five
sites survived the first pass.
`code_only` strips line comments before scanning. A comment that MENTIONS
`source: None` is prose, not a construction — this guard's own doc comment
tripped it on the first run, which is the same mistake as grepping a file
and matching its commentary.
+17
View File
@@ -0,0 +1,17 @@
# `src/mux/codec/coding.rs`
## Spec references
ITU-T H.273 (CICP code points, shared elsewhere), ISO/IEC 13818-2 §6.3.10
(MPEG-2 picture coding extension: `top_field_first`, `repeat_first_field`,
`progressive_frame`), RFC 9559 §5.1.4.1.28 (Matroska `FieldOrder` element
0x9D).
## HDR10 unit scaling
`Hdr10Metadata` values are stored in their raw SEI integer units (not yet
scaled to the Matroska float domain); the muxer applies the H.265 → Matroska
unit conversion at emit time so the scaling lives in exactly one place.
Spec: Rec. ITU-T H.265 D.2.28 (Mastering Display Colour Volume, payloadType
137) and D.2.35 (Content Light Level Info, payloadType 144).
+11
View File
@@ -0,0 +1,11 @@
# `consts::coding_type` — spec citations
`coding_type` is one registry used in two places that share the same value
space: the MPEG-TS PMT `stream_type` (ISO/IEC 13818-1 Table 2-34) and the
Blu-ray STN/CLPI `stream_coding_type` (BD-ROM Part 3).
The standardized video codes (`0x02`, `0x1B`, `0x24`) are ISO assignments
(ISO/IEC 13818-1 Table 2-34); `0xEA` (VC-1) is a BD-ROM convention in the ISO
user-private range. The `0x80..=0xA2` audio/graphics codes also sit in the
user-private range and follow the Blu-ray Disc Association / ATSC A/52
convention.
+14
View File
@@ -0,0 +1,14 @@
# CRC helpers (`src/mux/codec/crc.rs`)
## `crc16_mlp` reversed byte order
MLP's checksum is the "reversed" scheme. `crc16_mlp` emits its two bytes in
the OPPOSITE order to a standard little-endian CRC readout, so the caller
swaps them back and compares against the stored trailer word read
LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
Comparing big-endian instead is precisely the bug that function was fixed
for: it could never validate a real extended major sync, so whole TrueHD
tracks were dropped silently. This comment used to prescribe exactly that,
and to point at a `truehd::mlp_major_sync_ok` that does not exist.
Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
+87
View File
@@ -0,0 +1,87 @@
# `src/labels/criterion.rs` — internal notes
## `assign_stream_numbers`
Assign a 1-based stream number per `StreamInfo`, parallel to `infos`.
A stream mapped in `playbackconfig.xml` (`stream_map`) keeps its
mapped number. Streams with no mapping (absent or incomplete
`playbackconfig.xml`, or an unmatched `StreamInfo_ID`) are numbered
1-based per type — but the fallback counter SKIPS any number already
claimed via the map, so a synthesized number can never collide with a
map-assigned one. (Both numbering domains are 1-based per type, and
`apply_labels` matches on `(type, stream_number)`, so a collision
would mislabel tracks.)
Returns `None` when the 1-based stream-number space is exhausted —
every number in `1..=u16::MAX` for that type is either already
claimed by the map or already synthesized. That is unreachable on
real media: the BD STN_table carries at most 32 primary audio and 32
PG streams per playlist, so the 65535-wide space leaves >2000x
headroom. It IS reachable from a crafted `streamproperties.xml`
listing >65535 stream entries, and the only correct answers there are
"fail the parse" or "emit colliding numbers"; we fail.
The skip search is bounded by the numbering space itself: a `u16`
`saturating_add` here parked the counter at `u16::MAX` forever
whenever the map also claimed `u16::MAX`, turning an overflow guard
into a hang that `apply()`'s `catch_unwind` cannot interrupt. The
counters are therefore widened to `u32` so the skip loop strictly
increases toward a fixed ceiling (guaranteeing termination) and
exhaustion is reported rather than absorbed.
## Test: immunity pin, section-boundary half
Covers `an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it`.
`parse_stream_infos` emits one `StreamInfo` per `*StreamInfos` element
unconditionally — no filter, no `continue` — so an element whose
fields are missing or unrecognized still occupies its position, and
`assign_stream_numbers` still spends a number on it.
That is the property that keeps this parser out of the failure mode
where a skipped entry pulls every later label one stream forward. It
is load-bearing for the fallback path specifically: with no
`playbackconfig.xml` the numbers come purely from position in this
list, so dropping an element there would shift the rest.
Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
real audio streams renumber to 1 and 2.
Each stream here is one closed XML element, and every field is read
out of `&text[start..end]` — the range `xml::find_element` returned —
so one element can never absorb the next one's fields, however the
document is malformed around it. Contrast the flat-string walk in
pixelogic, where a section whose end marker is missing keeps
consuming entries as STN slots.
The missing-boundary case fails closed. An element with no close tag
of its own ends at the NEXT close tag, so it absorbs the element
behind it — the list comes back SHORTER. It cannot come back longer:
nothing outside a returned range is ever read as a stream, and
`find_element` yields `None` rather than a range running to EOF when
no close tag exists at all. A malformed document can cost this parser
a slot; it can never invent one.
Mutation: read fields from the document rather than the element's
range, or let a close-less element run to EOF → the trailing elements
re-enter the list as extra streams.
## Test: `exhausted_numbering_terminates_instead_of_looping`
A crafted `streamproperties.xml` can drive the fallback counter to
the top of the 1-based u16 stream-number space and then present one
more unmapped stream whose successor number is also claimed by the
map.
This must TERMINATE. The bound is the numbering space itself, so the
assertion is on the spec-derived exhaustion behaviour (`None`), not
on any tunable constant. Run on a worker thread with a deadline so a
non-terminating loop fails the test in 20 s instead of hanging CI.
## Test: `full_u16_numbering_space_is_usable_and_unique`
The whole 1-based u16 space must remain usable: 65535 unmapped audio
streams get 65535 distinct numbers with no panic and no wrap. The
literals here are the JVMS-independent, spec-derived size of a u16
1-based numbering domain, not a tunable cap.
+245
View File
@@ -0,0 +1,245 @@
# CSS keyless title-key recovery (`src/css/keyless.rs`)
## The cipher this attacks
The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
`sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
title-key mangling on the content path, so the recovery is a single inversion
of the sector cipher.
## The attack
1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
forward; for the first four steps deduce the LFSR0 output bytes from the
keystream (carry-tracked), reconstructing LFSR0's state. For the next six
steps clock LFSR0 normally and check it reproduces the keystream — a wrong
LFSR1 seed fails fast.
3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
the byte shifted in) to reach the initial state, then undo the
`r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
Known plaintext for step 1 comes from the longest periodic run in the
cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
0x80.
## `recover_title_key_from_plain`
Core of the recovery. `crypted` is the ciphertext starting at sector byte
0x80; `decrypted` is the matching known plaintext; `seed` is
`sector[0x54..0x59]`. On success returns the recovered 5-byte title key;
`None` if no LFSR seed reproduces the keystream. At least 10 bytes of
`crypted`/`decrypted` are required (the cipher is iterated 10 times: 4 to
reconstruct LFSR0, 6 to validate).
## `attack_crib`
Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
run that repeats with a cycle length in 2..0x2F. If the run is long enough
(`plen > 3` and at least two full cycles), the plaintext at 0x80 is taken to
be that periodic run continuing forward. Returns `None` for an unscrambled
sector or one with no usable run — such a sector can be neither cracked nor
key-validated, only descrambled with an externally-cached key.
The header is untouched by `descramble_sector`, so the crib is identical
before and after descramble: the decrypt path uses it as a per-sector
"did the cached key descramble correctly?" oracle (the predicted plaintext
must reappear at 0x80), and the cracker uses it as its known plaintext.
## Tests
### `synth_sector`
Builds a synthetic scrambled sector for a given title key and seed, with
`plain` placed as the plaintext at byte 0x80, scrambled with EXACTLY the
cipher `descramble_sector` inverts. Returns
`(scrambled_sector, full_plaintext_body)`.
### `synth_periodic_sector`
Builds a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends in a
periodic run that continues into the encrypted region — the case
`crack_title_key` is designed to crack.
### `recover_round_trips_known_keys`
MANDATORY round-trip (Task C.1): synthesize a scrambled sector for a known
(title_key, seed), then assert `recover_title_key` returns a key that
descrambles the body back to plaintext. CSS title-key recovery is
well-defined up to keys that scramble identically; we assert the full body
round-trips (the true correctness property), and additionally that the EXACT
key is returned for the common case.
### `descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with`
`descramble_matches` is the ONLY gate between the LFSR search and a key
handed back to the caller: both `recover_title_key` and the crib-driven
`crack_title_key_inner` return a candidate only if this says the key really
descrambles the sector to the known plaintext. A body that always answered
`true` would let the first spurious LFSR-seed match through as the title
key — the ripper would then descramble the whole title with a key that opens
nothing, producing garbage rather than a "no key" error.
Pinned both directions: the genuine key is accepted, and EVERY key one bit
away from it is rejected. The one-bit neighbours are the strongest form of
wrong key — a gate that only rejects wildly different keys would still pass
a near-miss out of the 2^16 seed search.
### `descramble_matches_does_not_disturb_the_caller_s_sector`
The gate is applied to a COPY: verifying a candidate must not modify the
caller's sector. `recover_title_key` runs the gate and then hands the sector
on to be descrambled for real — if verification descrambled in place, that
second descramble would run over already-transformed bytes (and, worse, a
rejected candidate would leave the sector corrupted).
### `recover_rejects_a_sector_that_ends_inside_the_encrypted_region`
A sector buffer that ENDS inside the encrypted region must be refused, not
sliced. `recover_title_key` slices `sector[0x80..0x8A]` unconditionally
after its length guard. The existing short-sector test uses
`SECTOR_BYTES - 1`, which is still long enough for that slice to succeed —
so the guard was never the thing producing the `None`, and dropping it (or
weakening the `||` to `&&`, which a full-length crib satisfies) changed
nothing observable. On a real short read this is an out-of-bounds panic on
the rip thread.
### `a_buffer_longer_than_one_sector_still_yields_its_key`
A buffer LONGER than one sector is still one sector: both entry points read
the first `SECTOR_BYTES` and must recover the key from it. Callers read DVD
data in multi-sector blocks, so an over-long slice is the normal case, not
an exotic one. A length guard that rejected it (`len > SECTOR_BYTES` instead
of `<`) would make every block-read caller silently unable to crack
anything.
### `recover_accepts_more_than_ten_bytes_of_known_plaintext`
`recover_title_key` accepts MORE than ten bytes of known plaintext, and uses
all of it: the extra bytes tighten the `descramble_matches` gate. The
ten-byte figure is a MINIMUM (the cipher is iterated ten times), not an
exact requirement — a guard reading it as an upper bound would reject every
caller that knows a longer crib.
### `a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused`
The scramble-flag gate on a sector whose BODY really is ciphertext. Both
entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
unscrambled sector has no title key to recover, and its bytes at 0x80 are
already plaintext. Every prior test of this gate used an all-zero or
all-`0x11` sector, where the recovery would have found nothing anyway — so
widening the mask test (`&` to `|`, which makes it true for EVERY flag byte)
produced the same `None` and went unseen.
Here the sector is genuinely scrambled and its key IS recoverable; only the
cleared flag stands in the way. If the gate stops working, both functions
start returning keys for sectors the disc says are in the clear.
`attack_crib` carries its own copy of the same gate and is the one that
actually stops the crack (`crack_title_key`'s is defensive duplication); it
also doubles as the decrypt path's cached-key oracle.
### `descramble_matches_forces_the_scramble_flag_on_its_own_copy`
The gate must verify a candidate against the sector's CIPHERTEXT regardless
of what the sector's own flag byte says. `descramble_matches` forces `0x10`
on its copy precisely because `super::lfsr::descramble_sector` is a no-op
when the scramble bits are clear — without that, verifying a
scrambled-but-unflagged sector compares raw ciphertext against the crib, and
every candidate key is rejected. Nothing exercised it: every fixture already
had the flag set, where forcing the bit is a no-op.
### `descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector`
The gate compares the WHOLE supplied plaintext, clamped to the encrypted
region. Two properties in one, because they are the two halves of
`plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
- it must compare beyond the first sixteen bytes, or a key that opens only
the head of the crib is accepted; and
- it must never compare past the end of the sector — a caller that knows
more plaintext than the 1920-byte encrypted region holds otherwise
indexes off the end of the buffer and panics.
### `sector_with_trailing_run`
Builds a sector whose clear header ends in a `period`-length repeating run
of exactly `run_len` bytes immediately before 0x80. The run is anchored to
ABSOLUTE sector offset (`sec[x] = pat[x % period]`), which is what makes
"the run continues past 0x80" a statement independent of the code under
test: the byte at `0x80 + i` of the underlying plaintext is
`pat[(0x80 + i) % period]`. Everything before the run is `0x00` (the pattern
bytes are all >= 0xD0, so the run cannot be extended backwards by accident),
and the encrypted region is filled with `0xFF` — so a crib that reads past
0x80 into "ciphertext" is immediately visible.
### `attack_crib_predicts_the_periodic_run_continuing_past_0x80`
KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly at
0x80, the crib is the run continued forward — the same ten bytes for every
run length, because the prediction depends only on the pattern and the
phase, never on how many cycles happened to be visible. The short lengths
are the load-bearing ones: at `run_len = 11` the crib window starts at 0x76
and is only 10 bytes from the end of the header, so any drift in
`plain_start`, in `cycles * best_p`, or in the `i % best_p` wrap reads the
0xFF "ciphertext" instead of the run.
### `attack_crib_refuses_a_run_shorter_than_two_cycles`
A run of exactly ONE cycle (plus the trivial tail the detector counts) is
not enough to predict forward: `attack_crib` requires at least two full
cycles. Weakening that guard would let a one-off byte sequence be declared
periodic and produce a confidently wrong crib — which the decrypt path uses
as its "is my cached key still right?" oracle.
### `attack_crib_refuses_a_buffer_shorter_than_a_sector`
`attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
own length guard is the only thing between a short buffer and an
out-of-bounds read. Nothing reached it: every caller-level test used a full
sector, and the entry points' guards fire first.
### `attack_crib_survives_a_header_that_is_periodic_to_offset_zero`
A header that is periodic ALL THE WAY to offset 0 must not walk the
backward scan off the front of the sector. The detector counts backwards
from 0x7f while `j < 0x80`. On a fully periodic header the run never
breaks, so `j` reaches 0x7f and the bound is the ONLY thing that stops it —
one step further and `0x7f - j` underflows a `usize` and panics. A constant
or fully-patterned 128-byte header is ordinary DVD data (padding, a run of
zeros), not a crafted input, and every existing fixture had a
filler/run boundary well before offset 0 that stopped the scan early.
### `attack_crib_is_independent_of_the_encrypted_region`
The crib is read from the CLEAR header only. A run that reaches 0x80 must
predict from the header bytes, never from the encrypted region — the
previously-fixed bug this function's doc comment records. Pinned by
rewriting the encrypted region and requiring the crib not to move.
### `recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input`
`recover_title_key_from_plain` unconditionally builds a 10-byte keystream
buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard is
the only thing standing between a short slice and an index-out-of-bounds
PANIC. Nothing reached that guard before: `recover_title_key` rejects
`plain.len() < 10` at its own door and always hands on exactly ten
ciphertext bytes, and `crack_title_key_inner` always passes a fixed
`[u8; 10]` crib. The guard is a live contract for any future caller and was
executed by no test at either boundary.
### `recover_title_key_from_plain_xors_the_sector_seed_back_out`
The seed XOR-back (`recover_title_key_from_plain`'s last step) is what turns
the recovered LFSR key into the TITLE key: `key ^= sector_seed`. Pinned as a
known answer across seeds that differ only in one byte — the same
ciphertext/plaintext pair therefore must yield title keys differing in
exactly that byte. Without this, a body that ORed the seed in (or dropped
the step) still round-trips on any fixture whose seed is zero, and on the
non-zero ones the failure looks like "no key found" rather than a wrong
step.
+311
View File
@@ -0,0 +1,311 @@
# `src/css/mod.rs` — extended notes
Overflow detail relocated from doc comments in `src/css/mod.rs` (kept short
there per the comment-guard caps). Function/item names below match the code.
## `CSS_LOCKED_BAIL`
The bus-auth read gate is global (all-or-nothing), so a run this long of
`05/6F/03` (CSS-locked) reads means the gate is shut and nothing here is
crackable — bail instead of grinding the full 50_000-sector budget. That
grind is what made rc5 appear to hang on a wedged USB bridge.
## `crack_key`
Recovers the CSS title key with no keys, by scanning scrambled sectors and
running the known-plaintext attack (see the `keyless` module). The crib comes
from the periodic-run detector: a scrambled sector's cleartext region
(bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
constant fill); the attack assumes that run continues across the 0x80
boundary into the encrypted region, giving the known plaintext the 2^16 LFSR
recovery needs. Scans up to 50000 sectors across the extents and returns the
first sector that yields a key — no player keys, no disc-key crack. Works on
a live drive (after bus-auth unlocks reads) and on disc images alike.
This convenience form runs to completion (no cancellation) and returns just
the key; callers needing an operator-Stop / watchdog cancel, or the
three-way `CrackOutcome` (to distinguish "unencrypted" from
"encrypted-but-uncracked"), use `crack_key_outcome`, which takes a `halt`
token.
## `CrackOutcome`
Distinguishes the three cases the bare `Option<CssState>` conflated (and
which caused a silent-failure bug: scrambled-but-uncracked content was
treated as "unencrypted" and muxed as plaintext garbage at exit 0):
- `Cracked` — a scrambled sector yielded a title key.
- `Unencrypted` — NO scrambled sector was seen across the scanned extents
(`is_scrambled_pack` never true): the content is genuinely plaintext, so
proceeding without a key is correct.
- `ScrambledUncracked` — scrambled sectors WERE seen but no key could be
recovered (the recovery found no crackable crib, or the scrambled region
was unreadable). The content is encrypted; muxing it as plaintext would
emit garbage, so callers MUST surface a hard error instead of falling
through to "unencrypted" — the per-title `crate::error::Error::CssKeyMissing`
when it is ONE title's own re-crack that failed (skippable: a sibling VTS
may still crack), or the disc-level `crate::error::Error::CssNoDiscKey`
when it is the disc-wide scan (`Disc::css_error`, every title fails
identically).
## `resolve_dvd_title_key`
The SINGLE place every DVD read path obtains a title key, so the file-backed
mux highway (`crate::build_iso_pipeline`) and the live-drive single-pass
`crate::DiscStream` descramble a DVD identically ("reading is reading"). CSS
keys are per-VTS and crackable from the scrambled data itself, so a
`None`/MPEG-PS title cracks its own key here, in playback order over
`extents`. Everything else is left untouched:
- AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
- a title that already carries a key — nothing to resolve.
- a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
A scrambled-but-uncrackable title is a hard `crate::error::Error::CssKeyMissing`,
never a silent scrambled-passthrough mux. That code is the PER-TITLE one
(`error::is_skippable_title_stub`), which is correct here: this function
cracks ONE title's own extents, and another VTS on the same disc may still
yield its key, so an all-titles rip skips this title and finishes the rest.
The whole-disc failure is `crate::error::Error::CssNoDiscKey`, raised by
`Disc::ensure_decryptable_keys` from the scan's `css_error`.
## `crack_key_scan` (private)
Tracks a `saw_scrambled` flag so a scrambled-but-uncracked disc is
distinguished from a genuinely-unencrypted one (the `crack_key` `Option`
wrapper collapses both to `None` via `CrackOutcome::into_state`).
## `descramble_sector`
A no-op unless the sector is a scrambled MPEG-2 PS PACK. The pack start code
is checked, not just the byte 0x14 flag bits, for the same reason
`descramble_region` checks it: 0x14 only means "scrambling control" inside a
pack, and in an IFO it is whatever that format stores there. A real
`VIDEO_TS.IFO` sector holds 0x15 there while starting `00 26 00 00`;
descrambling it destroyed 1912 of its 2048 bytes, and because that sector
carries TT_SRPT the disc enumerated 38 titles while an image decrypted from
it enumerated 10, silently, at exit 0.
This function has no callers inside the crate, but the module-level example
prescribes it — so the crate's own documented guidance led straight into
that defect. Making the guard part of the function, rather than something
each caller must remember, is what keeps the safe path the easy one.
## `descramble_region`
Descrambles a whole CSS buffer in place, re-cracking the title key on a VOB
region boundary. `title_key` is a CACHE of the last crack, not a fixed disc
key: it changes per VTS/VOB region, so it is validated on every scrambled
sector and re-cracked on a miss (the standard on-demand per-region rekey).
This CSS key acquisition is intrinsic to the cipher — CSS has no external key
source, the ONLY way to a title key is cracking the data — so it lives with
the CSS primitives and runs inside `decrypt::decrypt_sectors` (a public,
self-contained CSS decrypt), NOT at the post-decrypt recovery seam that AACS
key-fetch and FMTS segment-skip use (those consume external inputs).
The clear header (`<0x80`) is never scrambled, so its periodic crib predicts
the plaintext at `0x80`. Descramble with the cached key; if the crib fails to
reappear the key region changed (or the primed key was wrong) — restore the
ciphertext, re-crack from this very sector, and descramble again. A crib-less
sector (no periodic run) can be neither validated nor cracked, so it rides
the cached key — correct, because it lives in the same region as the nearby
crib sector that set the cache.
This section used to document an `Error::DecryptFailed` for the case where a
sector's crib rejects the cached key and the re-crack from that sector also
fails. That behaviour was tried and REVERTED: crib mismatch plus crack
failure is the signature of a crib FALSE POSITIVE, not of a stale key, and
failing there made real discs unrippable. The code now descrambles with the
cached key and returns `Ok`.
## `has_scramble_flag_bits`
Deliberately NOT called `is_scrambled`. Byte 0x14 only means "scrambling
control" inside an MPEG-2 Program Stream pack; in an IFO, UDF or ISO 9660
sector it is whatever that format stores there. Treating the flag alone as
proof of scrambling is what destroyed 1912 bytes of a real disc's
`VIDEO_TS.IFO` — the sector carrying TT_SRPT — so the disc enumerated 38
titles and an image decrypted from it enumerated 10, silently, at exit 0.
Callers want `is_scrambled_pack`. It asks the same question and also
requires the pack start code, which every genuinely scrambled VOB sector
carries and no IFO sector does.
It stays public only because an integration test asserts the flag
extraction directly. It has no production callers, and its previous doc
comment claimed one (`decrypt::decrypt_sectors`) that did not exist — so the
name was an invitation and the documentation was an argument for accepting
it.
## `is_scrambled_pack`
The HARDENED test the crack scan uses to set its `saw_scrambled` evidence
flag (Fix 3).
`has_scramble_flag_bits` keys solely on bits 4-5 of byte 0x14. That byte is
only meaningful inside a real DVD sector — an MPEG-2 Program Stream pack,
which ALWAYS begins with the 32-bit pack-start code `00 00 01 BA` at offset
0x00. A tiny clear / nav-only stub (a 0.5 s menu loop, an FBI-warning title)
can carry arbitrary bytes that happen to set bits 4-5 of byte 0x14; trusting
byte 0x14 alone there would flip the scan's `saw_scrambled` gate and make a
genuinely-UNENCRYPTED title report `ScrambledUncracked` — a false E7023.
Requiring the pack-start signature FIRST means only a sector that is
structurally a DVD video pack can be counted as scramble evidence. This does
NOT weaken the genuine "encrypted but uncrackable" hard-fail: a real
scrambled feature is made of valid PS packs, so its scrambled sectors still
pass this check and still drive `ScrambledUncracked` when no key cracks.
The pack-start check alone is not enough, and the sector's PES `stream_id`
(offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
(`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
stream_id even on a scrambled sector. The MPEG-PS structural packets that
are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
because on those the byte at 0x14 is NOT a PES scrambling-control field but
raw payload/structure whose bits 4-5 land set by chance.
This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
set. With no `0x11` gate those nav packs flipped the crack scan's
`saw_scrambled` flag on a disc that carries no CSS at all; the crack then
found no key (there is none) and the scan returned `ScrambledUncracked`,
hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
`0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
can act on (the recovery only recovers a key from a scrambled ES pack), so a
decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
private_stream_1 (`0xBD`) packs, none of which are excluded, so its
scrambled sectors still set `saw_scrambled` and still drive
`ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
The DESCRAMBLE path gates on this same function — `descramble_sector` and
`descramble_region` both call it, not the raw flag test — because the raw
test does not merely mis-skip a sector there: it descrambles one that was
never scrambled and destroys it. The measured case is written up above under
`descramble_region`: a `VIDEO_TS.IFO` sector holding 0x15 at offset 0x14 lost
1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38 titles
became 10 — silently, at exit 0. One gate, both paths.
## Tests (`mod tests`)
Detail relocated from test doc comments:
- `a_crib_false_positive_keeps_the_cached_key_rather_than_failing`:
`attack_crib` is a heuristic — it finds a periodic run in the unscrambled
header and predicts the run continues past 0x80. When that prediction does
not hold, the crib reports a mismatch even though the cached key is
correct, and the re-crack then fails BECAUSE the crib was never valid. So
this combination is the signature of a crib false positive, not of a stale
key, and the cached key remains the best available evidence. This test
exists because a prior round read the same code as "descrambling with a
key we just proved stale" and made it `DecryptFailed` to match the AACS
path; real DVDs hit this constantly, and the change made a real disc
unrippable (no unit test caught it — the real-media acceptance gate did).
CSS is not AACS: an AACS unit key either opens a unit or does not, whereas
a CSS title key is recovered from data whose recoverability varies sector
by sector.
- `has_scramble_flag_bits_short_buffer_is_false_no_panic`: grounding —
`sector.len() >= 2048 && (sector[0x14] >> 4) & 0x03 != 0` short-circuits so
a short buffer never reads index 0x14. Mutation: swapping operand order
would panic indexing a 20-byte slice; this test catches it.
- `has_scramble_flag_bits_uses_bits_4_5_only`: grounding —
`(sector[0x14] >> 4) & 0x03`. Mutation: widening the mask to `& 0x0F` makes
0x40 report scrambled; the 0x40 assert catches it.
- `has_scramble_flag_bits_exact_sector_length_accepted`: grounding —
`sector.len() >= 2048`. Mutation: `>= 2048` → `> 2048` makes an exact
2048-byte scrambled sector report false; this catches it.
- `is_scrambled_pack_requires_pack_start_signature`: Fix 3 hardening —
`is_scrambled_pack` requires BOTH the pack-start code and the 0x14 bits, so
a clear/nav-only stub with stray 0x14 bits isn't scramble evidence (else a
genuinely unencrypted title reports the false E7023).
- `is_scrambled_pack_excludes_nav_and_structural_stream_ids`: detection fix —
`is_scrambled_pack` must exclude system_header/padding/private_stream_2 by
`stream_id` at 0x11, since CSS never scrambles those and byte 0x14 there is
raw payload, not a scrambling-control field. This is the exact
decrypted-HD-DVD defect (an `.evo` RDI pack is 0xBF with pack-start and
0x14 bits set); the video/private_stream_1 cases prove the exclusion
doesn't weaken real-DVD detection.
- `MockSource` fields `lock_all`, `crackable`, `short_read`, `stream_id`: model
a CSS-locked drive (`05/6F/03` on every read), a designated crackable LBA
serving a full synthetic scrambled sector, a short-read count (the
`recovery: true` short-read contract, `Some(0)` being the degenerate
spin-guard case), and the PES `stream_id` written at 0x11 (used to model an
HD-DVD `.evo` RDI nav pack via `PRIVATE_STREAM_2`).
- `crackable_sector`: builds a crackable scrambled sector whose cleartext
header (0x59..0x80) carries a periodic run continuing across the 0x80
boundary, mirroring the `synth_periodic_sector` fixture in the keyless
tests but built from the crate-internal `scramble_sector`.
- `a_source_that_returns_zero_sectors_still_obeys_the_scan_budget`: `tried`
increments only per inspected sector inside `for s in 0..usable`; an
`Ok(0)` never runs that loop, so `advance` is forced to 1 to keep the
cursor moving and the extent's full declared length is walked. Mutation:
deleting the `tried` charge in the `usable == 0` arm goes red at 60,000
reads for a misbehaving/adversarial source.
- `crack_key_caps_total_tries_at_50000`: grounding — `max_tries = 50_000`,
`tried += 1` before the read, loop guard `tried < max_tries`. Mutation:
widening the cap, or removing the increment, both go red.
- `crack_key_budget_is_shared_across_extents`: grounding — `tried` is
declared outside the extent loop with a shared `if tried >= max_tries`
break. Mutation: moving the declaration inside the loop would give each
extent its own budget (80,000 total instead of 50,000).
- `crack_key_scans_from_extent_start_lba`: grounding —
`reader.read_sectors(ext.start_lba + i, ...)`. Mutation: dropping
`ext.start_lba +` would start LBAs at 0 instead of the extent's start.
- `crack_key_continues_past_read_errors`: grounding — a read `Err` falls
through to the next sector rather than aborting. Mutation: propagating the
error with `?` would stop the scan after the first failure.
- `crack_key_empty_extent_reads_nothing` / `crack_key_no_extents_is_none`:
grounding — `while i < ext.sector_count` / `for ext in extents` are no-ops
on empty input; an off-by-one (`<=`) or a pre-loop read would break these.
- `crack_outcome_reaches_cracked_with_span`: audit gap "MockSource never
yields a crackable sector" — drives the full `crack_key_scan` over a
synthetic ISO to exercise the `Cracked` branch and `crack_span` recording
end-to-end, previously untested.
- `is_scrambled_uncracked_is_true_for_that_case_and_false_for_the_other_two`:
a prior round introduced the three-way split precisely because conflating
the cases made an uncrackable disc exit 0 with garbage output; earlier
tests only asserted the TRUE direction, so a predicate answering "yes"
unconditionally was indistinguishable from a correct one. All three
outcomes here come from real `crack_key_outcome` scans, not hand-built enum
values.
- `resolve_dvd_title_key_*` tests: `resolve_dvd_title_key` is the SINGLE
shared per-title CSS step both read paths (`build_iso_pipeline` multi-pass
and `DiscStream::new` single-pass) call, so these pin its full contract at
the shared boundary — crack success, hard-fail on scrambled-uncrackable
(never leaving `keys` as `None`, which would mux scrambled bytes as
plaintext), the decrypted-HD-DVD RDI-pack regression (end to end), `--raw`
passthrough, the AACS-keys-untouched gate, the clear-DVD no-op, and the
halt-cancellation-surfaces-as-Halted fix (not a truncated-scan verdict).
- `all_locked_synthetic_iso_yields_css_key_missing_signal`: audit §2/§5 #7 —
pins that an all-locked synthetic ISO across MULTIPLE extents produces
`ScrambledUncracked`, the exact signal `disc/mod.rs` converts into
`disc.css_error = Some(Error::CssKeyMissing)`.
- `recrack_succeeds_on_other_vts_extents`: audit gap "success path missing"
— the prior re-crack test only covered the locked→None path; this proves a
key cracked for one VTS is genuinely re-derived (not reused) for another.
+32
View File
@@ -0,0 +1,32 @@
# Warner CTRM label parser — relocated rationale
Referenced from `src/labels/ctrm.rs` unit tests via `// See docs/ctrm.md` pointers.
## Immunity pin: no analog of the `paramount` forced_sub false positive
Immunity pin against the defect measured in the `paramount` parser, where a
vendor `forced_sub` cell hung off a FULL dialogue track's own slot to say
"this track also contains forced signs", and reading that cell as "this
track is forced" flagged 30 MB dialogue tracks forced.
This format cannot express that. The forced signal is not a flag beside a
track's entry — it IS the entry's stream-kind token, drawn from a closed
vocabulary in which `subtitle_production` (the full dialogue track) and
`subtitle_narrative` (the forced-narrative track) are mutually exclusive
alternatives in the same position. A row is one or the other; there is no
cell a full track can carry to acquire the qualifier, so the paramount
failure mode has no encoding here.
Mutation: give `subtitle_production` a `Forced` qualifier, or add a forced
side-flag that both kinds may carry.
## Immunity pin: stream numbers come from the row, not a counter
`language_streams.txt` states each stream's number in field 3, so a row the
parser cannot use is simply dropped — it can never renumber the rows behind
it. This is the property that keeps this parser out of the STN-slot-shifting
failure mode that bites parsers which count positionally: there, a skipped
entry silently pulls every later label one stream forward.
Mutation: replace `parts[2]` with a running per-type counter → the three
unusable rows in the corresponding test collapse the survivors onto 1/2 and 1.
+39
View File
@@ -0,0 +1,39 @@
# `labels::dbp`
## Implementation
Iterates `CpInfo::Utf8` constant-pool entries rather than raw
byte-scanning each class file. Equivalent label coverage (the literal
`TextField,...` strings live in the CP as Utf8 entries) with no
false-positive risk from method bytecode or attribute names that
happen to contain `TextField,`. Language / purpose / qualifier
classification lives in `super::vocab` so all Java-parser families
share one source of truth.
The parser ignores any prefix before the first `TextField,` occurrence
in a constant string — whatever string-pool ordering placed ahead of
it is irrelevant.
## `MAX_LABEL_BYTES` rationale
The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
constant contributes up to 65535 bytes, and the `u16` stream-number
keyspace admits 65536 of them per type.
Headroom: real dbp menu labels are short display names — "English Dolby
Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
bytes. 256 leaves >5x headroom over that, and any string past it is menu
geometry or padding, never a language name — `vocab::lang` would not
resolve it anyway.
## `MAX_LABELS_PER_TYPE` rationale
The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
type are reachable; paired with `MAX_LABEL_BYTES` this bounds the whole
scan at 2 x 512 x 256 bytes.
Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
streams per playlist, and dbp emits one menu TextField per stream. 512
leaves 16x headroom over the spec maximum.
+500
View File
@@ -0,0 +1,500 @@
# decrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/decrypt.md — <topic>` pointer
left at the original site.
## Parallel AACS decrypt
Each AACS aligned unit (6144 bytes) is decrypted INDEPENDENTLY of every
other unit — per-unit key derivation from the unit_key plus the unit's
own first-16-byte header. There is no cross-unit dependency, so a
buffer of N units can be decrypted on N threads in parallel via a
persistent rayon thread pool.
Small buffers (< `PARALLEL_MIN_UNITS` units) fall through to the serial
path to avoid pool dispatch overhead beating the per-unit AES work.
## Thread-count configuration — three layers
Resolution order (highest wins):
1. The most recent `set_decrypt_threads` call with `n > 0`. Calling
this *replaces* the live thread pool — useful for a settings-page
slider in a long-running daemon.
2. `FREEMKV_THREADS` env var, if set and `> 0`. Single knob covering
decrypt today, intended to also drive any future input-side /
output-side worker pools.
3. Default: all available cores. Algorithm optimisation comes first —
we measure single-thread performance to find serial bottlenecks
before throwing parallelism at it — but once a pool is engaged we
use the whole box. Hard cap at `MAX_THREADS` (rayon stack memory).
## DECRYPT_POOL
Current rayon pool. `RwLock<Option<Arc<...>>>` so that
`set_decrypt_threads` can swap the pool out without leaking the old
one and without blocking ongoing decrypt work (in-flight calls hold an
`Arc` clone via `decrypt_pool` and finish on the old pool; new calls
pick up the new pool).
## decrypt_pool()
Get (or lazily build) the active rayon thread pool. Returns an `Arc`
so in-flight work survives a concurrent `set_decrypt_threads` swap.
Returns `None` if the pool cannot be built (e.g. the OS refuses the
worker threads under a pid/thread limit). The caller falls back to the
serial decrypt path — library code never panics here.
## unit_is_our_phase
Does this unit belong to the phase we hold the key for?
An FMTS forensic segment interleaves two variants at the unit level:
the disc carries both, and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving
it as ciphertext is correct, because the muxer drops untouched
ciphertext cleanly.
`Phase::All` means the whole range is ours (the non-forensic case).
This lived inline inside `apply_aacs_map`'s per-unit closure, where
nothing could reach it: a mutation run flipped the `-` to `+` and the
`/` to `*` in the index arithmetic and every test still passed.
Getting either wrong silently decrypts the wrong half of a forensic
segment.
## AacsKeyMap (proactive AACS key-selection map)
Which held unit key decrypts each LBA of a title's encrypted content,
decided ONCE before mux from the disc's CPS-unit (and, later, FMTS
segment) structure — never by trial-decrypt-and-check per unit at mux
time.
This is the pivot that ends the mux "key-server storm": the old path
decrypts a unit, checks whether the plaintext looks like clean MPEG-TS,
and — because authored-bad content never reaches that bar — concludes
"wrong key, fetch a fresh one" and re-asks the key service for units it
already holds the correct key for. There is NO per-unit byte pattern
that separates "correctly decrypted but authored-bad" from "still
encrypted", so that check is unanswerable. The map removes the
question: we resolve one key per CPS unit / segment up front (see
`resolve_mux_key_map`), record which LBA ranges each covers, and at mux
time simply "decrypt this LBA with key K" and trust it — bad TS is the
muxer's concern, exactly as for a physically-read clear disc.
Ranges are `[start_lba, end_lba)` → index into the `Aacs { unit_keys }`
pool, sorted and disjoint. The map is a POSITIVE list: an LBA in no
range is passed through untouched (no default key). How a single-CPS
disc is mapped depends on the caller: the whole-disc EXTRACT path uses
one blanket range `(0, u32::MAX, 0)` so every encrypted unit — parsed
title or orphan clip — resolves to key 0; the per-title MUX/sweep path
(`resolve_mux_key_map` → `content_map`) maps only the title's own
extents, so an orphan clip outside them is left as pass-through. Either
way, clear nav/filesystem sectors (encrypted-flag off) pass through.
## AacsKeyMap::read_plan
Build the FMTS **read plan**: the title's aligned units filtered down
to only the units this rip must actually read — every default / CPS
unit, plus, inside each forensic segment, ONLY our-phase (`Phase::Even`
/ `Phase::Odd`) units. The alternate-phase units are a different device
group's variant: a licensed player never reads them, and neither do we.
They are omitted from the plan entirely, so they are never fetched,
decrypted, or handed to the demux — the demux therefore sees one
gapless our-variant stream, with no ciphertext to trip a
concealed-gap resync (the old behaviour that dropped good frames
around every segment).
`extents` are the title's clip extents (unit-aligned in the interior; a
shorter tail is ordinary content and always kept). `unit_sectors` is
the AACS aligned-unit size in sectors (3). Contiguous kept units
coalesce into as few extents as possible so the producer still issues
large sequential reads across default content; only inside a ~480 KB
forensic segment do reads become unit-granular (every other unit). A
map with no forensic (Even/Odd) range returns `extents` unchanged — the
common disc is not touched.
The parity test is byte-identical to the decrypt hot loop
(`(unit_lba - range_start) / unit_sectors`), so a unit kept here is
exactly a unit `decrypt_sectors_mapped` would open, and vice-versa.
## apply_aacs_map
AACS scheme step: apply `map`'s per-unit keys to `buf`, in-place.
`base_lba` is the absolute LBA of `buf`'s first sector; each aligned
unit (3 sectors) is decrypted with the key the map assigns to its LBA.
There is NO key trial and NO `is_clean` verdict: the map already
decided the key from disc structure, so this applies it and moves on —
a unit that decrypts to authored-bad TS passes through for the muxer to
drop, never re-fetched.
A SCHEME, not a policy. It reports what it could not open by returning
`Err(DecryptFailed)`; the decision that an unopenable unit must never
be emitted belongs to `decrypt_span`, which is the one place that
decides it for every scheme. A map index outside the held pool is
likewise a fail-loud `Error::DecryptFailed`: the resolver's job is to
guarantee every selectable index is present, so a gap here is a
resolver bug, not silent loss.
## decrypt_sectors — CSS / clear detail
For CSS: descrambles per 2048-byte sector, self-cracking the title key
from the data (no external input). For `None`: a no-op. For AACS:
**always** returns `Err(DecryptFailed)` — AACS decrypts exclusively
through the resolved key map (`decrypt_sectors_mapped`), which keys
every content unit up front and fails at RESOLVE time when a key is
missing. Reaching this arm with AACS keys means a reader was built
without installing its map (a bug), so it fails loud rather than apply
a guessed key.
`unit_key_idx` is a legacy parameter kept so the CSS / `None` wrapper
signature stays stable; it is ignored (the CSS arm self-gates on its
per-sector scramble flag). Returns `Err` if decryption was expected but
impossible; never produces silently corrupted output. The `usize`
return is a legacy unverified-byte count that is always `0` for the CSS
/ `None` arms.
## decrypt_span — orchestrator rationale
THE decrypt orchestrator. Every path into this crate's decryption goes
through here.
How a disc decrypts is one process — resolve a key for this span,
apply it, and refuse if no key can be proven. Only the resolve-and-apply
step is scheme-specific. This function owns the loop and the refusal;
the schemes below supply only what genuinely differs between AACS, CSS
and clear media.
That split exists because its absence caused six separate defects in
one release. There used to be TWO top-level paths — this one for CSS
and clear, and a wholly separate `decrypt_sectors_mapped` for AACS
whose arm here was a bare `return Err` stub — so each scheme decided
its own answer to "there is no key for these bytes" and nothing held
them to the same one. CSS drifted to descrambling with a key it had
just proven stale; the mapped path drifted to passing an unkeyable
encrypted unit through as ciphertext. Both looked like success to the
caller.
Adding a scheme means adding an arm here, which means answering the
refusal question. That is the point.
## Tests — relocated rationale
### a_forensic_read_plan_drops_units_the_full_extents_include
`DecryptKeys::None` is a no-op even with a content map + scrambled
bytes. A forensic map's read plan is NOT the extents it was given —
and that is the precondition the mux's provenance guard keys on.
A clip's feed span is measured over the FULL extents at scan time,
while the mux reads this reduced plan, so the byte offsets stamped on
frames and the offsets recorded in the spans describe different
streams. The deficit accumulates, so every frame after the first
segment resolves to an earlier clip than it came from. The spans still
tile each other, so the tiling check cannot see it; the mux compares
the plan against the full extents instead and stops trusting
provenance when they differ.
### content_gate_css_keys_is_noop
CSS ignores the content gate (it lives in the AACS arm) and always
reports `0` — confirming the gate is a no-op for CSS and the read
stays scheme-agnostic (the litmus test: adding CSS verify touches only
the CSS arm, never the read).
### content_gate_css_actually_descrambles_the_buffer
`decrypt_sectors_in_content` is the entry point `DecryptingSectorSource`
dispatches to whenever a content map is installed
(`sector/decrypting.rs` line ~211), so it is on the live read path for
every mapped rip. It must actually DECRYPT. The two `_is_noop` tests
above only assert its `usize` return is `0` — which is what a body
replaced by `Ok(0)` also returns, so neither one constrains it at all.
Here a genuinely scrambled CSS sector goes in and the CONSTRUCTED
plaintext must come out. Anything that skips `css::descramble_region` —
including a body that just reports `Ok(0)` — leaves ciphertext in the
buffer and the caller muxes scrambled MPEG at exit 0.
Expected bytes come from the plaintext this test built BEFORE
scrambling (CSS scrambles only 0x80..2048; the header stays clear), not
from re-running any descramble routine.
### content_gate_aacs_keys_fail_loud_not_ok_zero
The AACS arm of the same entry point must fail LOUD. Under the
keymap-only model AACS decrypts exclusively through
`decrypt_sectors_mapped`; reaching this wrapper with AACS keys means a
reader was built without installing its key map, and continuing would
hand the caller ciphertext under an `Ok`. `DecryptFailed` is the
correct verdict per the function's own contract — it must not be
softened into a success with a zero count.
### css_region_change_recracks_the_title_key
CHARACTERIZATION (recovery refactor safety net): the CSS arm's
per-region re-crack (the `title_key` cache is stale for a new VOB
region → restore ciphertext, `crack_title_key` this sector,
re-descramble). Two crackable sectors scrambled under DIFFERENT keys
sit back-to-back; the cache is primed to the FIRST key. Sector 0 rides
the cache (crib matches); sector 1 must trip the crib mismatch and
re-crack to its own key. Both must land correct plaintext, and the
cache must end on region 1's key.
This behaviour currently lives inline in `decrypt_sectors` (the `Css`
arm). It is the delicate logic the recovery refactor will move to the
input-stream seam, so it must stay green byte-for-byte across that
move.
### aacs_scrambled_trailing_partial_is_rejected
Whole leading unit plus a SCRAMBLED trailing partial that is FLAGGED
encrypted in its clear seed (the malformed danger case): an encrypted
unit split across an extent boundary cannot be CBC-decrypted
standalone. The mapped decrypt must fail loud with `DecryptFailed`
rather than emit the ciphertext partial as clear. Exercises the real
shipping path (`decrypt_sectors_mapped`) and its trailing-partial
guard.
### aacs_clear_trailing_partial_passes_through
A CLEAR trailing partial (encrypted flag NOT set) is a legitimate
content tail and must pass through, never trip the guard above.
"Passes through" means byte-for-byte unchanged, not merely `Ok`.
Asserting only `is_ok()` let a mutant that corrupts the clear partial
while still returning `Ok` pass — which is the whole failure this test
names. Mutation: XOR any byte of the tail before returning -> the
snapshot comparison fails.
### none_keys_is_noop
`DecryptKeys::None` is a pure no-op: the buffer must be returned
byte-for-byte unchanged with Ok, regardless of content (even content
that looks scrambled).
Grounding: the `DecryptKeys::None => {}` match arm does nothing.
Mutation: replace the empty arm with a call that mutates buf -> the
unchanged assert fails.
### is_encrypted_matches_variant
`is_encrypted` reflects the variant: None -> false, Css/Aacs -> true.
Grounding: `!matches!(self, DecryptKeys::None)`.
Mutation: invert the `!` -> None reports true, this fails.
### css_descrambles_with_title_key
The CSS path descrambles each 2048-byte sector with the title key. A
scrambled sector run through decrypt_sectors must come back to its
plaintext body (keystream XOR is involutive), proving the title key is
actually applied.
Grounding: `DecryptKeys::Css { title_key } => for chunk in
buf.chunks_mut(2048) { descramble_sector(title_key, chunk) }`.
Mutation: change `chunks_mut(2048)` to `chunks_mut(2049)` or pass a
fixed wrong key -> the body no longer matches the plaintext.
### css_processes_every_sector_in_buffer
The CSS path processes EACH 2048-byte sector independently in a
multi-sector buffer. Two scrambled sectors (with different seeds) in
one buffer must both round-trip — pinning that the loop steps by 2048
and applies the key to every sector, not just the first.
Grounding: `for chunk in buf.chunks_mut(2048)`.
Mutation: change the loop to descramble only the first chunk (e.g.
`.next()`) -> the second sector stays scrambled, assert fails.
### css_rekeys_when_title_key_region_changes
CSS title keys are per-VTS/VOB region: a real disc holds DIFFERENT keys
for different regions and the only way to get each is to crack it. The
decrypt path must re-crack when the cached key stops descrambling (its
crib no longer reappears at 0x80) instead of blindly applying one key
across a region boundary — the bug that pixelated every freemkv DVD
rip.
Two sectors scrambled under DIFFERENT keys, cache primed to ONLY the
first (exactly what the one-shot scan crack leaves). Sector 0 validates
+ descrambles with the cached key; sector 1's cached-key descramble
fails the crib, so the path re-cracks sector 1's own key and recovers
its plaintext. Before the fix (blind single-key apply) sector 1 was
garbage.
Grounding: the CSS arm's `attack_crib` → `chunk[0x80..] != crib` →
`crack_title_key` → `*title_key = fresh` rekey.
Mutation: drop the rekey branch (apply the cached key always) →
sector 1's body no longer matches its plaintext; this fails.
### css_leaves_clear_sector_unchanged
The CSS path leaves UNSCRAMBLED sectors (flag clear) byte-for-byte
untouched — descramble_sector early-returns on a zero flag. A clear
sector mixed into the buffer must not be corrupted.
Grounding: descramble_sector returns immediately when
`(sector[0x14] >> 4) & 0x03 == 0`.
Mutation: remove that early return in lfsr.rs -> a clear sector would
be XORed with a keystream and change; this fails.
### css_empty_buffer_is_ok
CSS decrypt always returns Ok (it cannot fail — descrambling is XOR,
no key validity check), even for an empty buffer.
Grounding: the CSS arm has no `return Err` path; `chunks_mut` over an
empty slice is a no-op; the function ends `Ok(())`.
Mutation: make the CSS arm return Err -> this fails.
### aacs_mapped_out_of_range_key_idx_errors
A map that selects a key index OUTSIDE the held pool must fail loud
with DecryptFailed — never silently apply a wrong key or pass
ciphertext through. This validates `decrypt_sectors_mapped`'s up-front
`key_indices()` bounds check (the real shipping AACS decrypt path).
Mutation: drop the `unit_keys.get(idx).is_none()` guard → the
out-of-range index would not error; this fails.
### aacs_via_unmapped_decrypt_sectors_fails_loud
SAFETY NET: reaching the CSS/`None` wrapper (`decrypt_sectors`) with
AACS keys means a reader was built with no map — a bug. It must fail
loud, never apply a guessed key. (AACS decrypts exclusively via
`decrypt_sectors_mapped`.)
### read_plan_forensic_reads_only_our_phase_units
FMTS: a forensic Even segment drops exactly its alternate (odd) units
from the read plan — they are never fetched — while default content on
either side stays one coalesced sequential run. The kept units are
byte-identical to the ones the decrypt hot loop opens.
### read_plan_phase_parity_is_measured_from_an_unaligned_range_start
A forensic range does NOT start on an aligned-unit boundary. Its start
LBA comes from a source-packet number — `start_spn * 192` put through
`clip_byte_to_lba` (`mux/resolve.rs`) — and 192-byte packets have no
relationship to the 3-sector aligned unit, so `range_start % 3` is
whatever the disc says.
That makes `unit_ix = (lba - range_start) / us` load-bearing in both of
its operations, and the existing coverage used a range starting exactly
on the extent's first unit, where several wrong formulas agree with the
right one by arithmetic accident.
Getting the parity wrong is not a crash. It reads and decrypts the
ALTERNATE variant's half of a forensic segment: the units this disc's
key does not open decrypt to garbage, and the units it does open are
skipped. AACS 2.1 forensic marking is exactly the mechanism that makes
the two halves different, so a phase inversion is silent — it produces
a full-length rip carrying the wrong variant.
### read_plan_gates_the_last_whole_unit_of_an_extent_not_just_the_remnant
An extent whose last whole unit is an alternate-phase unit must still
drop it. The tail guard exists for a REMNANT shorter than a unit —
bytes with no following unit to desync — and an extent ending exactly
on a unit boundary has no remnant at all.
With the guard widened to `remaining <= us`, the final unit of every
extent bypasses the phase gate and is read unconditionally. On a
forensic segment that lands at an extent end, that is one
alternate-variant unit pulled into the rip and decrypted with a key
that does not open it.
### every_scheme_gives_the_same_verdict_when_no_key_can_be_proven
Every scheme that CANNOT prove a key answers the same way.
This is the property `decrypt_span` exists to hold. There used to be
two top-level decrypt paths — one for CSS and clear media, one for
AACS — and each decided its own answer, so they drifted apart in
opposite directions within a single release: CSS descrambled with a
key it had just proven stale, and the AACS path passed an unkeyable
encrypted unit through as ciphertext. Both reported success.
Asserting one verdict across the schemes is what makes a future
divergence a test failure rather than a silent corruption. A
per-scheme test cannot do that: each would still pass while the two
disagreed.
CSS is deliberately NOT in this list. Its title key is recovered from
the data, sector by sector, by a heuristic that false-positives — a
crib mismatch whose re-crack fails means the crib was wrong, not that
the key is stale, so the cached key is kept and used. Round 9 folded
CSS in here on the reasoning that "no key" should mean one thing
everywhere; that made real DVDs unrippable, and the real-media gate
caught it. Uniform policy is right for schemes that can PROVE a key
wrong. CSS cannot.
### an_encrypted_unit_outside_every_key_range_fails_instead_of_passing_through
An ENCRYPTED unit that falls outside every key-map range must fail,
not pass through as ciphertext.
"The map has no key here" and "there is nothing to decrypt here" are
different statements, and only the second makes passing the unit
through correct. On a multi-CPS disc an orphan clip — referenced by no
playlist, so in no title extent and therefore in no range — hits the
first and was treated as the second. `extract_tree` then counted those
bytes as GOOD, dropped the `.partial` suffix, and reported `complete:
true`, exit 0: a scrambled file on disk with a clean bill of health.
A CLEAR unit outside every range is the ordinary case (filesystem and
nav on a whole-disc read) and must still pass through untouched — so
this asserts both directions.
### mapped_key_selection_is_positional_so_pool_order_matters
The mapped descramble indexes the committed key pool POSITIONALLY
(`unit_keys[key_idx].1`), so the ORDER of the `Vec<UnitKey>` a
`KeySource` returns is load-bearing — it is NOT "cosmetic, the decrypt
path strips it and tries every key", as
`keysource::resolve_and_apply_traced`'s doc used to claim. Trial-decrypt
was deliberately deleted; nothing here searches the pool. Reordering
the same two keys therefore sends each range to the WRONG key: the
range that decrypted clean now fails the correct-phase `is_clean` net
loudly (or, off a forensic phase, would decrypt a whole span under a
neighbour's key). Pins the corrected doc.
### decrypt_threads_within_valid_pool_range
The default (auto) decrypt thread count is always a usable pool size:
at least 1 (a 0-thread rayon pool is invalid) and never above
MAX_THREADS (rayon stack-memory cap). This test reads the resolved
value without mutating the process-global override, so it is safe to
run in parallel with other tests.
Grounding: `cores.clamp(1, MAX_THREADS)` in the default branch;
`env.min(MAX_THREADS)` in the env branch.
Mutation: change `.clamp(1, MAX_THREADS)` to `.clamp(0, MAX_THREADS)`
on a 0-core probe (unlikely) — more robustly, change the cap to
`MAX_THREADS * 2` -> on a many-core CI box the upper-bound assert can
fail. The lower-bound (>=1) guard is the load-bearing invariant.
### phase_gate_selects_only_our_parity_of_a_forensic_segment
The FMTS phase gate picks which half of an interleaved forensic
segment we decrypt. Both the `-` and the `/` in its index arithmetic
survived a mutation run, and getting either wrong silently decrypts the
alternate variant into garbage while reporting success.
### phase_gate_does_not_panic_on_a_malformed_map
A malformed key map must not take down a long-running service. A unit
below its own range start, or a zero unit size, are both map bugs —
they must return a defined answer rather than panicking on debug
overflow or dividing by zero.
Every case here asserts the DEFINED answer, not merely the absence of a
panic. The zero-unit-size case used to be written
`assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
accepts both answers and so pinned nothing at all: the guards could
invert and it would still pass. The answer is knowable —
`saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
is even — so pin it.
</content>
+34
View File
@@ -0,0 +1,34 @@
# `DecryptingSectorSource`
## Why this decorator is the single source of truth for decrypt-on-read
Every decrypt-on-read caller (e.g. `DiscStream`) wraps its source in this
decorator rather than calling the cipher helpers directly. The actual cipher
code lives in `crate::aacs` and `crate::css`; the decorator just calls the
existing `crate::decrypt::decrypt_sectors` helper that drives both of them
in-place after each read. Centralizing the call site means callers can wire
the decorator unconditionally and keep their pipeline shape uniform
regardless of encryption state — for `DecryptKeys::None` discs it is simply
a pass-through, so there is no branch in caller code for "is this disc
encrypted?".
## `KeyFetch`: why two explicit operations
`KeyFetch` exposes `unit_keys` and `fmts_indexes` as two separate operations
rather than one generic callback, so the "one base key vs. a whole forensic
set" contract lives in the type instead of a caller guessing at the return
length:
* `unit_keys` resolves the base Unit Key(s) for a CPS unit from real
encrypted samples drawn from it. The non-forensic path: one key per CPS
unit (the pool grows by whatever it returns). Used by the mux's base /
multi-CPS map resolution and by the sweep/patch recovery decorator.
* `fmts_indexes` resolves the disc's AACS 2.1 forensic index keys from an
index-1 single-phase anchor batch. The source hands back the COMPLETE set
(ordered index 1..N); the caller sizes the forensic map to `len()` and
never assumes a fixed N (32 is all we've seen, but the contract is
"whatever the source returns, >= 1, is all of them").
`KeyFetch` is a stateless, shared pair of `Arc<Fn>`, so one instance is built
once and cloned cheaply (two `Arc` bumps) into every read path; it is
`Send + Sync` so it can ride the mux highway's producer thread.
+227
View File
@@ -0,0 +1,227 @@
# Deluxe BD-J framework parser (`src/labels/deluxe.rs`)
Detected on discs whose `/BDMV/JAR/<x>.jar` contains a `com/bydeluxe/`
directory entry.
## What this parser reads
Deluxe-authored discs store stream labels as ordinal references into
enum classes whose names are obfuscated per-disc, so a name-based
match won't work. The label data is instead recovered by matching on
the **shape of each enum's `<clinit>`**, which is framework-stable:
| Enum | Signature |
|---|---|
| Language | 70 `ldc` operations in `<clinit>`, sequence starts `English, French, Spanish, Dutch, ...` |
| Purpose | 8 ldcs starting `Normal, Commentary, PiP, Trivia, ...` |
| VideoFormat | 7 ldcs starting `HD, HDR10 Plus, HD Dolby, ...` |
| Region | 22 ldcs starting `USA_D1, LIC1, LIC2, LIC3, ...` |
| Studio | 6 ldcs in `<clinit>` |
Matching on the shape rather than the class name keeps the parser
working across obfuscation variants. Codec strings come from the
standard BD-J `org/bluray/ti/CodingType` enum referenced directly by
the binding constructors (see `StackVal::CodingType`), not from a
Deluxe-internal enum.
## Implementation phases
- **Phase A** — master enum identification (`identify_master_enums`).
Walks every `.class`'s `<clinit>` ldc sequence and matches against
the framework-stable fingerprints. Output: `Vec<(label, MasterEnum)>`
with full ordinal → string-value tables.
- **Phase C** — binding-class identification (`find_binding_classes`).
The per-stream table is built by some class via repeated
`getstatic` references to the master enums identified in A.
That class has the highest such `getstatic` count in the jar.
Heuristic shape; precise threshold may need tuning.
- **Phase D** — binding-class bytecode decoder (`decode_binding`).
Walks the binding class's `<clinit>` with a tiny symbolic stack
machine. For each `new X / dup / ... / invokespecial X.<init>`
sequence, collects the int values and enum-reference operands
between the `dup` and the constructor call, then emits a
`DecodedStream`. The signal-to-StreamLabel mapping (which arg is
stream index? which is language? audio vs subtitle?) uses a
heuristic — see `interpret_streams` for the mapping rules.
## Confidence
`parse` returns `Some(ParseResult::high(labels))` when Phases A
through D produce at least one stream: the master enums matched their
framework-stable fingerprints (a strong ordered-prefix signature), the
binding class decoded, and at least one per-stream binding resolved to a
real (language, purpose, codec) tuple — the schema was fully recovered,
not guessed. `None` when the disc isn't Deluxe-authored or when decoding
produces zero streams (a recognized-but-broken state that the analyzer
still surfaces via `parsers_detected`).
## Size caps (Phase A candidate pool)
`MAX_CLINIT_LDC_STRINGS` (cap on `ldc` operands retained per class by
`clinit_ldc_strings`): unlike every other count cap in this crate, the
paired byte cap here cannot be the disc-file size — a `.class` entry
gated only by a `com/bydeluxe/` path prefix deflates from ~100 KB up to
the 64 MiB `MAX_CLASS_BYTES` read ceiling, so ~33M two-byte `ldc`
instructions — one retained `String` each — are reachable from a small
crafted disc. The bound has to be on the decompressed work, so it is
applied here. Headroom: the largest framework-stable enum is `Language`
at 70 values (`FINGERPRINTS`), and no fingerprint matches a count more
than `LDC_COUNT_TOLERANCE` away from its expected size, so anything past
~74 can never identify a master enum. 4096 leaves ~55x headroom over the
largest real enum for framework drift.
`MAX_CLINIT_LDC_BYTES` (companion byte cap): the count cap alone still
admits 4096 x 64 KiB of `Utf8` (a JVMS `CONSTANT_Utf8_info` length is a
u16), i.e. ~268 MB per class from repeated `ldc` of one huge constant.
Headroom: master-enum values are short display names ("English",
"HDR10 Plus", "USA_D1") well under 32 bytes, so a real Language enum
retains ~1 KB. 256 KiB admits 4096 values averaging 64 bytes each.
`MAX_CANDIDATE_TOTAL_BYTES` (aggregate companion, bounds retention across
ALL classes at once): `identify_master_enums` holds every class's
retained strings in one map simultaneously, so the per-class cap alone
still admits `classes x 256 KiB`: a 64 MiB jar of minimal `.class`
entries reaches tens of GiB. This is the same bound one level up.
Headroom: the five `FINGERPRINTS` enums together hold ~113 short values
(~1.2 KB). Every other class in a real BD-J jar contributes only
whatever string constants its own `<clinit>` loads — resource paths,
config keys — so a large authored jar lands in the low hundreds of KB.
16 MiB leaves ~40x headroom over a deliberately generous 400 KB estimate
for real media.
`MAX_CANDIDATE_CLASSES` (entry-count companion to
`MAX_CANDIDATE_TOTAL_BYTES`): the byte budget alone still admits ~16M
map entries when every class retains a single one-byte `ldc`, and the
per-entry `HashMap` + `String` overhead is not counted by that budget.
Headroom: a large retail BD-J title ships on the order of 1-3k classes,
and only those with a non-empty `<clinit>` ldc sequence become
candidates. 65536 leaves >20x headroom over the class count of any real
jar.
## Internal helpers
- `clinit_enum_field_names`: collects the static-field names an enum
class's `<clinit>` stores its own instances into, in declaration order
— the ordinal -> field-name mapping that mirrors `clinit_ldc_strings`'s
ordinal -> value mapping. An obfuscated Deluxe enum constant compiles
to `new E; dup; ldc "Value"; invokespecial E.<init>(…); putstatic
E.<field>`, so the `putstatic` whose owning class AND field descriptor
are both this class's own type names the field for that ordinal. The
binding class then references that constant as `getstatic E.<field>`;
without this mapping `MasterEnumTable` cannot turn the obfuscated field
name back into an ordinal. Bounded by `MAX_CLINIT_LDC_STRINGS` like the
value walk.
- `clinit_ldc_strings`: walks `<clinit>` and collects every `ldc`/`ldc_w`
operand that resolves to a `String` or `Utf8` constant, in declaration
order; `None` if the class has no `<clinit>`. Collection stops at
`MAX_CLINIT_LDC_STRINGS` operands or `MAX_CLINIT_LDC_BYTES` of retained
text, whichever comes first: the walk is driven by the decompressed
class, so it isn't bounded by the disc-file size cap the way the rest
of this module's counts are. Truncation cannot lose a real match — a
truncated sequence is far longer than any `FINGERPRINTS` entry's
`expected_count + LDC_COUNT_TOLERANCE`, so it would have been rejected
on count anyway.
- `find_binding_classes`: identifies all binding-class candidates by
getstatic-count to the master enums, using `MIN_GETSTATIC = 4` — the
binding class on a typical disc has 50+ such getstatic references (one
per slot x arity); the low floor lets a small disc with few streams
still qualify, while still filtering out classes that just reference
the language enum once for a config string. Some Deluxe discs split
the per-stream table across two binding classes (one for audio, one
for subtitle — the audio class commonly has the most getstatic refs,
the subtitle class somewhat fewer, both sharing the master Language +
Purpose enums), so this returns top-K candidates ordered by descending
getstatic count, filtered to a minimum concentration of references.
`MAX_CONSTRUCTIONS` (Phase D, cap on `Construction`s retained from a
binding `<clinit>` walk): one entry is appended per matched `new X / dup
/ invokespecial X.<init>` with no other bound — a `.class` gated only by
a `com/bydeluxe/` path prefix deflates to the 64 MiB `MAX_CLASS_BYTES`
ceiling, and the shortest matching sequence is a handful of bytes, so
millions of `Construction`s (each a `String` plus an arg `Vec`) are
reachable from a small crafted disc (~1 GiB). The same cap bounds the
per-class union in `decode_binding_class` (a crafted class may repeat
`<clinit>`, which JVMS §4.6 forbids but this reader tolerates) and the
cross-class union in `parse`, so the whole phase retains at most this
many. Headroom: the BD `STN_table` (BDAV stream-entry counts) admits at
most 32 primary audio and 32 PG streams per playlist, and a Deluxe
binding table covers the disc's playlists — low hundreds of entries on
the largest retail titles. 4096 leaves >20x headroom.
`Decoder::push` (symbolic-stack push during binding `<clinit>` walking):
honours the Code attribute's declared `max_stack`. JVMS 4.7.3 requires
that a method's operand stack never exceed `max_stack` at any point, so
a push past it can only come from bytecode that would fail JVM
verification. Dropping it costs nothing on real bytecode and bounds the
decoder: a `.class` gated only by a `com/bydeluxe/` path prefix deflates
from ~100 KB to the 64 MiB `MAX_CLASS_BYTES` ceiling, i.e. ~67M
single-byte `iconst_0` (~2 GiB of `StackVal`) on an unbounded `Vec`.
Headroom: `max_stack` is exactly what javac computed for the real
binding `<clinit>`, so no real construction can be clipped. The symbolic
stack counts `long`/`double` as one slot where the JVM counts two, so
its depth is never greater than the verified depth.
## interpret_streams: Constructions -> StreamLabels
Converts the per-construction tuples from Phase D into `StreamLabel`s.
Two binding-constructor shapes are handled:
- 5-arg: `BindingType.<init>(I, Lang;, Lpurpose;, I, LCodingType;)V`
- 4-arg: `BindingType.<init>(I, Lang;, Lpurpose;, LCodingType;)V`
Args are identified by **TYPE**, not position:
- First `EnumRef{kind: "Language"}` -> audio/subtitle language
- First `EnumRef{kind: "Purpose"}` -> Deluxe purpose ordinal
- First `CodingType(name)` -> codec field name (translated via
`coding_type_to_codec_hint`)
- First `Int(n)` -> stream index (preserved as ordering hint;
per-type sequential stream_number is what actually goes into the
StreamLabel, since BD spec stream-numbering is anchored on MPLS
data, not the binding code)
Stream type inference:
- Construction has a `CodingType` arg -> audio stream (subtitles on
Deluxe don't carry a CodingType; their codec is implicit PGS via
the BD spec).
- Construction has Language but no CodingType -> subtitle stream.
- Neither, and its binding type never yielded a stream -> not a
stream (skip). See `slot_kind` for why the binding type is
consulted rather than the language alone.
`slot_kinds`: which stream list each binding type enumerates, learned
from the constructions that DID resolve a language. A `<clinit>` walk
emits a `Construction` for every `new X; ...; invokespecial X.<init>`
it sees, so the list mixes real stream bindings with whatever else the
class initializer builds. `binding_type` is the constructed class name,
which is how the two are told apart: the stream bindings all share one
class (Deluxe splits audio and subtitle across two), and that class is
identifiable from the slots that resolved. A binding type that resolved
as both kinds is left out — with no consistent answer, guessing a list
to advance would be worse than not advancing.
`slot_kind`: the stream list an unresolved construction occupies a slot
in, or `None` when it is not a stream binding. A
`org.bluray.ti.CodingType` argument is decisive on its own: nothing but
an audio stream binding is handed one. Otherwise falls back to what the
binding type's resolved siblings showed (`slot_kinds`).
## Confirmed studio variants
- **Universal** (`studio="uni"`): Language enum `pd` (65 values,
`English, French, Spanish, Dutch, …`), Purpose enum `lp`
(`Normal, Commentary, PiP, Trivia, Descriptive, Score`), audio binding
`np.<init>(I, Lpd;, Llp;, Lorg/bluray/ti/CodingType;)`, subtitle binding
`wb.<init>(I, Lpd;, Llp;, Lmi;)`, all built in one binding class's
`<clinit>` alongside a title-wrapper object whose constructor takes the
per-stream arrays (filtered out by the array-parameter guard in the
decoder). SDH/RNIB is encoded as a distinct Language VALUE
("English SDH", "English RNIB"), recovered into the qualifier from the
name. Grounded on fixtures captured from a real Universal Blu-ray release.
- **Disney/Warner**: 70-value Language enum, same binding shape; the
original corpus this parser was written against.
+74
View File
@@ -0,0 +1,74 @@
# `demux://` sink — per-codec ES output
The `demux://` sink is a write-only `crate::pes::Stream`. Instead of muxing
the per-track `PesFrame` stream into a single container (as `MkvStream`
does), it routes each frame's payload to the file for `frame.track`,
post-processing where the internal codec `Frame` form differs from the
standalone on-disk ES form:
- **HEVC / H.264**: the codec parsers emit hvcC/avcC-style 4-byte
length-prefixed NALs (start codes stripped) and carry the parameter sets
out-of-band in `codec_private`. A standalone `.hevc`/`.h264` needs Annex-B
framing with the parameter sets prepended — see `AnnexBWriter`.
- **PGS**: the parser collapses a display/clear PCS pair into one
duration-bearing `Frame` with the raw segment payload but no `PG` magic /
timestamp header. A `.sup` needs the HDMV segment framing rebuilt — see
`PgsSupWriter`.
- **VobSub**: the `.sub` is the raw SPU stream; the `.idx` sidecar is
synthesized from per-SPU PTS + byte offsets — see `VobSubWriter`.
Every other codec writes `frame.data` verbatim (`PassthroughWriter`).
The sink does NOT touch the MKV mux path; it is purely additive.
## `PgsSupWriter`: rebuilding HDMV segment framing
The parser hands the demux sink the concatenated PGS segments of a display
set in `frame.data` (segment_type + segment_size + payload, repeated), with
no `PG` magic and no PTS/DTS. A `.sup` prefixes each segment with a 13-byte
header: `0x50 0x47` ("PG") | PTS u32 BE | DTS u32 BE | (segment_type|size
already present in the payload). When the parser folded a trailing clear
(`duration_ns` set), the writer re-emits it as an empty composition at
`pts + duration` so players time the subtitle out.
## `synthetic_clear_display_set`: synthesizing the clear PCS
Builds a synthetic "clear" display set: an empty PCS (0 composition objects)
followed by an END segment. The parser folds the original clear/end PCS
pair's wipe time into the display frame's `duration_ns` and drops the clear
bytes, so a faithful `.sup` re-emits one here at `display_pts + duration`;
without it every subtitle lingers to EOF. `width`/`height` are carried from
the display set's PCS so the clear PCS advertises the same video geometry —
they don't affect the wipe but keep the segment well-formed. Returned bytes
are concatenated `type(1)+size(2 BE)+payload` segments, the same shape
`emit_segments` consumes.
## Test: `epoch_driver_follows_ref_video_not_track_zero`
Regression for the hardcoded `frame.track == 0` epoch driver: on an M2TS/PMT
title the PMT can list an AUDIO ES before the VIDEO ES, so the video lands
at stream index 1. The sink already resolves the video reference
dynamically (`ref_video_track`); the epoch driver must use that SAME
reference, not the literal 0. In the test, track 0 is audio and track 1 is
video; a video-only clip boundary (track 1) must open a new epoch (bump
`offset_ns`). With the bug (only `frame.track == 0` drives epochs) the
video back-jump would be treated as a passive rider and `offset_ns` would
stay 0 — corrupting every track's rebased timeline.
## Test: `a_demux_export_the_seam_plan_emptied_fails`
Audit finding: this sink builds its timeline with
`TimelineContinuity::with_clips` but no test ever gave it a title with
usable PlayItem marks, so `frames_mapped` and `dropped_total()` were always
zero and BOTH gates in `finish` were unreachable. Deleting either left the
whole suite green while a `demux://` export of a seamless-branching title
wrote zero-byte track files beside a populated chapters document, at exit 0.
## `sanitize`: path-hostile filename characters
Control characters are replaced, NUL above all. Every string this touches is
disc bytes, and a language code is three raw STN bytes run through
`from_utf8_lossy` with no validation — `00 00 00` is the ordinary
"undefined" encoding on real Blu-rays. A NUL in a path aborts
`File::create` with `InvalidInput`, which took the whole demux export down
before a single track file was opened.
+65
View File
@@ -0,0 +1,65 @@
# `DemuxThread`
## Why a second worker thread
With `crate::sector::PrefetchedSectorSource` alone, read+decrypt already
runs on a producer thread; the *consumer* (main) thread still serialises
`ts_demuxer.feed` (M2TS parsing) with the codec parsers. Profiling showed
feed at ~37% and codec parse at ~44% of consumer wall time — i.e. feed is
heavy enough that pipelining it with parse pays for itself.
Splitting them: feed runs in `DemuxThread`; the consumer thread receives
`Vec<PesPacket>` batches and runs codec parse + frame emission only. Total
throughput becomes `1/max(feed, parse)` instead of `1/(feed + parse)`.
## Lifecycle
`DemuxThread::spawn_zero_copy` consumes the prefetch channels and the
demuxer state, returning a handle plus a `Receiver<DemuxBatch>`. Dropping
the handle closes the channel which signals the worker to exit; the join
in `Drop::drop` blocks until the worker observes channel closure and
returns (no timeout — a wedged downstream would block the drop until it
releases the channel).
## `spawn_zero_copy` rationale
Instead of taking a `SectorSource` and memcpy-ing through its
`read_sectors` API, this constructor consumes the prefetch channels
directly: filled buffers come in via `prefetch_rx`, the demux thread
feeds them, then returns them to `recycle_tx` for the producer to
re-fill. Eliminates the 16 MiB memcpy per batch that the SectorSource
adapter incurred (and, with the producer-side recycling pool, also
eliminates the per-batch heap alloc / cross-thread free that was costing
40%+ of demux-thread time before).
## Test: empty-batch disconnect fix (`worker_exits_promptly_on_consumer_drop_during_empty_batches`)
Regression: worker must detect consumer disconnect even when every demux
batch is empty (no matching PIDs / null packets).
Before the fix, `tx.send()` was never called for empty batches so the
worker never observed the consumer drop — it would spin through ALL
remaining extents before exiting, causing `DemuxThread::drop`'s `join()`
to block for minutes on a mostly-untracked disc region.
The watchdog: if the worker doesn't exit within 1s of the consumer drop
the test fails (rather than hanging forever as the bug would).
## Test: spawn-failure drop-order (`channels_disconnected_before_producer_join_on_spawn_failure`)
Regression: on thread-spawn failure the channels must be dropped BEFORE
the producer shell so the upstream producer observes disconnection and
exits, allowing `join()` to complete without hanging.
A true EAGAIN/pids-limit spawn failure cannot be reliably forced in a
unit test without root or ulimit co-operation, so the test exercises the
drop-order contract directly: a mock shell that panics if `join()` is
called while either channel end is still open.
The test constructs a `(prefetch_tx, prefetch_rx)` pair where the tx side
is held by a sentinel that stays alive as long as either channel end is
open, then asserts that the sentinel is gone by the time
`producer_shell`'s join logic would run. Because a real spawn failure
can't be forced, the test instead verifies the helper logic in isolation:
drop `prefetch_rx` and `recycle_tx` first, then observe the producer-side
sender is disconnected, which is the property the fix relies on.
+65
View File
@@ -0,0 +1,65 @@
# diag.rs — scan diagnostics rationale
## Why this module exists
A bug report log must be self-diagnosing: everything needed to explain *why*
freemkv made the choices it did at scan must be in the log, in a compact,
machine-parseable form. `diag.rs` emits one terse line per row (title, cell,
stream, decision) under the `tracing` target `freemkv::diag`, which the CLI
routes to `log.txt` when `--log-level 3` (debug) is set.
## Format conventions (stable, greppable)
- Every line is prefixed by a `tag=` so a log scraper can filter (`disc`,
`title`, `dvd.cell`, `dvd.vattr`, `dvd.aattr`, `bd.clip`, `bd.mark`, `aacs`,
`stream`, `decision`).
- Raw bytes are shown as `0xNN` next to their decode so a wrong decode is
obvious against the raw value.
- This module only READS already-parsed scan state — it never re-reads the
disc and never mutates anything.
## DVD per-cell table vs. Disc-level dump
The DVD per-cell table (with the raw cell-category byte) is emitted from the
IFO scan itself (`dump_dvd_cells`), because the per-cell `ifo::DvdCell` detail
is lowered away before the `Disc` is built. The `Disc`-level dump
(`dump_disc`) covers everything that survives lowering: titles, streams, the
picked main feature, and AACS state.
## `dump_dvd_substream_probe`
Emits the ACTUAL per-physical-sub-stream AC-3 channel counts read off the VOB
during the mux-time sub-stream probe (the Silence-of-the-Lambs wrong-stream
fix). This is the ground truth the IFO nibble is compared against: each row is
`sub_id=0x8x channels=N` for a physical `private_stream_1` AC-3 sub-stream
whose first frame was decoded. An empty probe (scrambled / unreadable / short
VOB) logs a single `probed=0` line so the absence is explicit in a bug log.
Self-sufficiency: with `tag=dvd.aattr` (the IFO's declared sub_id + claimed
channels) and these `tag=dvd.substream` rows (the physical reality), a bug log
alone shows whether the ordinal `0x80` actually carries the declared channel
layout — no disc needed to diagnose a wrong-substream rip.
## `dump_mkv_track`
Emits the MKV `TrackEntry` elements the muxer is about to WRITE for one track
— the Windows-fps-class metadata (FlagInterlaced, FieldOrder,
DefaultDuration, DefaultDecodedFieldDuration, Display dims) plus the
codecPrivate as hex. With this row a bug log alone is enough to verify why
Windows Explorer reports a given frame rate for an interlaced SD track: the
container values that drive its fps derivation are all present, no disc and
no MediaInfo needed.
`track_number` is the 1-based MKV track number; `track` is the built
`crate::mux::mkv::MkvTrack` whose fields map one-to-one onto the emitted
elements (see `MkvMuxer::new`).
## `main_feature_reason`
The `reason=` token on the main-feature decision row is DERIVED from
`Disc::CANONICAL_TITLE_ORDER_KEYS`, which lives beside the comparator that
actually implements them — never restated here. The previous hand-written
copy drifted (it advertised a `fewest-clips` key the comparator had replaced
with largest-physical-size), which made the self-diagnosing log explain the
pick with a rule the code does not apply. A diagnostic that disagrees with the
decision it documents is worse than no diagnostic.
+132
View File
@@ -0,0 +1,132 @@
# Rationale for `src/dirimage/tests.rs`
Longer-form notes for individual tests, referenced from `// See docs/dirimage-tests.md`
pointers in the source so the inline comments stay within the comment-guard caps.
## `Scratch::unique_path`
The path a scratch dir takes is kept separate from directory creation so its
uniqueness is testable without a syscall between draws. A monotonic counter,
NOT a timestamp, is what guarantees it: `SystemTime::now()` resolves to only
a MICROSECOND on macOS, so two of the seven parallel tests that share the
"bdmv" tag routinely read the same value within one microsecond, collide on
the same directory, and the first to finish `remove_dir_all`s it out from
under the others' reads — an intermittent `read_sectors` ENOENT. The counter
is unique regardless of clock granularity; pairing it with the pid keeps it
unique across test-binary processes.
## `scratch_paths_are_unique_even_at_clock_resolution`
Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
tag and run in parallel; if two land the same path, the first to drop
`remove_dir_all`s it out from under the other's reads, which surfaced as an
intermittent `read_sectors` failure. Regression: with the old
`SystemTime::now()` name this is RED, because macOS's clock advances only
per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
between them to nudge the clock, mirroring the real cross-thread collision —
hands back the same value many times over. The counter-based name is unique
regardless of clock granularity.
## `a_directory_spanning_many_blocks_reads_back_whole`
A directory with enough children that its FID list spans several 2048-byte
blocks, read back through the production parser.
This is the case the block-alignment question turns on: FIDs are packed
contiguously and a descriptor may straddle a block boundary, because
`read_directory` (`udf.rs:1312`) walks the directory extent as one flat byte
run and BREAKS at the first non-257 tag. Padding each block would truncate
the directory at the first pad. 200 entries also exceeds the 16-handle LRU
several times over, so it exercises handle eviction on the read path.
## `a_file_below_video_ts_is_placed_not_just_declared`
A file inside a SUBDIRECTORY of `VIDEO_TS` must still have its data placed.
Audit finding. The DVD branch places only the files directly inside
`VIDEO_TS`, because only those carry the IFO-relative constraints — and the
follow-up loop skipped that directory entirely, so anything one level deeper
got a File Entry declaring the file's real size with no extents behind it.
It appeared in the tree at full length and read back as nothing, at exit 0.
The identical folder under `BDMV/` was always placed correctly, which is
what made the gap easy to miss.
## `a_vob_offset_past_the_image_cap_is_refused`
A VOBS offset far past the content must be REFUSED, not honoured.
Audit finding. The planner honours a title set's declared VOBS offset
because honouring it is what makes a real backup readable, and nothing
bounded it: a regenerated `.BUP`, a rewritten IFO or a hand-assembled folder
naming a huge offset grew the image to wherever it pointed. A `u32` sector
count reaches roughly 8.8 TB, and writing that to an `iso://` destination
would fill a disk with zeros before anything noticed.
The companion (`an_oversized_vob_offset_leaves_a_gap_rather_than_failing`)
pins that a MODEST oversize is still honoured as a gap, so this cap refuses
only what it must.
## `playable_bdmv`
A BD folder that really enumerates a title, so the AACS content probe has
an extent to sample.
The `.m2ts` is built as real 192-byte BD source packets, because that is
what the probe judges: byte 0 carries the AACS Copy Permission Indicator in
its top two bits, and byte 4 is the MPEG-TS sync. `scrambled` sets the CPI
and withholds the sync — "flagged and not structurally clean", which is
exactly `aacs_unit_needs_decrypt`. An all-zero payload would prove nothing
either way: `is_clean_ts` skips zero payloads as padding.
## `both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory`
The two doors must AGREE. Same folder, same verdict, same extents.
Stated as a differential rather than a bare `is_ok()` so it cannot go
vacuous: if the fixture ever stops producing an AACS state, a one-sided
`Ok` assertion would still pass while guarding nothing, whereas "both doors
see the same title" is the invariant the shared function actually exists to
hold.
## `a_scrambled_folder_is_refused_through_the_dir_url_door_too`
The other verdict, through the same door: a folder whose content units are
really scrambled is refused with the TYPED code, not muxed into garbage.
This is the load-bearing half. `E_DIR_IMAGE_ENCRYPTED` is produced at
exactly one site in the crate (`session::apply_folder_encryption_verdict`),
reachable from here only through the `is_folder` argument — so this test
cannot pass if that argument is dropped, whatever else changes.
## `write_and_mount_externally`
Write a synthesized image to a real file and ask the OS to mount it.
This is the only check here that is not circular: `read_filesystem` shares
every assumption with the encoder, an operating system's UDF driver shares
none of them. Ignored by default because it shells out to `hdiutil` and
needs a host that can attach an image; run with
`cargo test -- --ignored write_and_mount_externally --nocapture`.
## `dvd_placement_invariant_on_a_real_folder`
Diagnostic (opt-in): verify the DVD placement invariant for every title set
in a REAL folder. `ifo.rs` derives a title's extents as
`file_start_lba(VTS_nn_0.IFO) + vtstt_vobs`, so that sum must land exactly on
`VTS_nn_1.VOB` or the mux reads the wrong sectors for that title.
Run with: `FMKV_DVD_FOLDER=/path/to/tree cargo test --lib
dvd_placement_invariant_on_a_real_folder -- --ignored --nocapture`
## `a_multi_clip_playlist_produces_feed_spans_that_tile_the_feed`
`Clip::feed_span` is the INPUT to the whole provenance feature — it is what
tells the muxer which clip a byte offset belongs to — and it is produced in
exactly one place, `disc/bluray.rs`. Every `SeamPlan` test synthesizes spans
by hand, so the consumer is thoroughly tested against fixtures written from
the same assumptions as the producer, and nothing checks the producer at all.
That is the shape of the original defect: the placement logic was tested and
correct, and the thing feeding it was wrong. `SeamPlan` only trusts spans
that TILE the feed contiguously from zero, so this asserts exactly that,
against the real scanner reading a real synthesized filesystem.
+185
View File
@@ -0,0 +1,185 @@
# `dirimage` — `dir://` as an image-level source
`dirimage` synthesizes a real, minimal, valid UDF 1.02 volume over a host
folder (a DVD `VIDEO_TS/` or Blu-ray `BDMV/` extracted by a MakeMKV-style
backup). Nothing is emulated:
* **Metadata sectors** (anchors, the volume descriptor sequences, the File
Set Descriptor, every File Entry, every directory's FID list) are encoded
into RAM by `encode` — a few MiB even for a large Blu-ray.
* **Data sectors** are not materialized at all. Each one maps to a byte
range of a real file, read on demand.
So `udf::read_filesystem` parses this image by exactly the same code path it
parses a real disc with, and every consumer above it is unchanged. The cost
is that a single-partition synthetic volume never exercises the UDF 2.50
Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
this module's tests do not cover that block and must not be read as if they
did.
## What this module deliberately does NOT do
* **3D / SSIF** — rejected up front (`Error::DirImageSsifUnsupported`). An
SSIF aliases the same sectors as its base and dependent `.m2ts`; the
planner allocates disjoint extents, so a 3D folder would produce silently
wrong output.
* **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
* **Encrypted folders** — a folder whose content is still AACS-scrambled is
rejected by the caller-side probe, not decrypted here.
## `layout.rs` — rationale detail
Full detail for the comments trimmed in `src/dirimage/layout.rs`, in file order.
**Module overview.** Two phases, kept apart because they fail for different
reasons: walk rejects what the image model cannot represent (3D SSIF, an
unrecognized tree, a case collision inside one directory); assign places
metadata first (File Set Descriptor, one File Entry per node, then the
directory FID lists), then file data. Data placement is the part with real
constraints. For a Blu-ray there are none — `.mpls`/`.clpi` address clips by
name, never by LBA — so files are packed sequentially. For a DVD the IFOs
record VOB positions as sector offsets from the IFO's own start, and `ifo.rs`
re-derives every title extent from them, so the placement must reproduce
those offsets exactly or the rip reads the wrong sectors (`place_video_ts`).
**`DATA_FLOOR`.** Well clear of the volume-space descriptors, and — the
load-bearing part — far from zero: `disc/bluray.rs:137` drops any clip extent
whose LBA is 0, so a file that landed at block 0 would vanish from the title
with no error.
**`MAX_AD_BYTES`.** The AD length field is 30 bits (the top two are the
ECMA-167 4/14.14.1.1 extent type), so 0x3FFF_FFFF is the arithmetic ceiling —
but a non-final extent must be a whole number of blocks, so the usable
ceiling is the largest multiple of 2048 below it. Files larger than this are
split across several ADs; `udf.rs` reads them back as a multi-extent file,
the same shape a dual-layer disc produces.
**`MAX_DEPTH`.** Matches `udf.rs`'s `MAX_DIR_DEPTH`: anything deeper would be
recorded but never descended into, so the files under it would be invisible
to every consumer. Refuse instead of silently dropping.
**`MAX_CS0_NAME_BYTES`.** The FID's name-length field is one byte, so 255 is
the ceiling; 254 leaves the encoder no way to produce a value that wraps to
zero.
**`MAX_SUBDIRS`.** A directory File Entry's link count is `u16` and counts
one per child directory plus one for its own entry in the parent, so the
last usable value is `u16::MAX - 1`. Lowered under `cfg(test)` ONLY so the
guard can actually be executed: building a folder with 65534 subdirectories
to reach the real cap is not a test anyone can run, so an earlier test
asserted arithmetic about the constant instead and would have passed with
the guard deleted. The production value is unchanged.
**Module-scope link-count assert.** Must live at module scope, not inside
`mod tests`: it previously sat inside `#[cfg(test)] mod tests` while also
carrying its own `#[cfg(not(test))]`, so it was compiled in NO configuration
and could never fire — exactly the dead gate the test above it was written
to replace.
**`MAX_IMAGE_SECTORS`.** A DVD title set records where its VOBS begins as an
offset in its own IFO, read verbatim out of a file in the folder. A
regenerated `.BUP`, a tool that rewrote an IFO, or a hand-assembled folder
can therefore name an offset far beyond the content — and the planner
honours it, because honouring it is what makes a real backup readable.
Without a ceiling the image grows to wherever that offset points: a `u32`
sector count reaches ~8.8 TB, and writing one to an `iso://` destination
would fill a disk with zeros before anything noticed. 128 GiB clears the
largest real medium (BD-100) with room to spare, so a genuine disc folder
never meets it.
**`MAX_META_BYTES`.** Every node costs a 2 KiB File Entry sector held for the
life of the image, so the entry cap alone permits ~205 MB of metadata for
content of no size at all — and the mux holds two of these at once while
probing. The module documents a budget of "a few MiB even for a large
Blu-ray"; this is what enforces it rather than merely asserting it.
**`is_excluded`.** Dot-files are host artefacts, never disc content: macOS
sprays `.DS_Store` and `._*` resource forks through any folder a Finder
window has touched, and including them would put files on the "disc" that
were never on the disc. `.partial` is freemkv's own in-flight extraction
suffix (see `disc/extract.rs`) — picking one up would mean planning an
extent over a file that is still being written.
**`read_head`.** Reads the first `n` bytes of an IFO so its placement
offsets can be resolved. Errors propagate: returning an empty buffer instead
would send every offset through `unwrap_or(0)`, recording NO placement
constraint, and a VOB placed without its constraint yields an image that
reads at the wrong offset with nothing reported. An IFO that cannot be read
is a folder that cannot be planned.
**`place_video_ts`.** DVD-Video records VOB positions INSIDE the IFOs, as
sector offsets from the IFO file's own first sector: `VIDEO_TS.IFO` + 0xC0
(`VMGM_VOBS`) → `VIDEO_TS.VOB`; `VTS_nn_0.IFO` + 0xC0 (`vtsm_vobs`) →
`VTS_nn_0.VOB` (the VTS menu); `VTS_nn_0.IFO` + 0xC4 (`vtstt_vobs`) →
`VTS_nn_1.VOB` (the title stream). `ifo.rs:554-556` re-derives every title
extent as `file_start_lba(IFO) + vtstt_vobs + cell.first_sector`, so only the
third of those is load-bearing for freemkv itself — but the other two are
load-bearing for DVD PLAYERS, which read the menus freemkv ignores. All
three are honoured. `VTS_nn_1.VOB … VTS_nn_9.VOB` are one logical stream
split at the 1 GB file-size limit, and the cell sector addresses run
continuously across the split, so they are placed back-to-back with no gap;
laying the group out in its canonical on-disc order gives exactly that for
free, so the constraint becomes a CHECK, not a search. The check can fail —
a regenerated `.BUP`, a tool that rewrote an IFO, a folder assembled by hand
— and when it does the required position lies below the end of the file
that must precede it. There is no placement that satisfies it, so it is a
typed error naming the file rather than a silent misplacement.
**`reject_ssif`.** `disc/bluray.rs:124-130` probes
`/BDMV/STREAM/SSIF/{clip}.ssif` and sets `is_3d` the moment one resolves —
unconditionally, before any capability check. On a real 3D disc the `.ssif`
and the base/dependent `.m2ts` files ALIAS the same sectors (the SSIF
interleave IS the two m2ts streams), which this planner cannot express: it
would allocate three disjoint copies, so the clip's extents and the m2ts's
extents would disagree and the rip would read the wrong bytes at exit 0.
Reject the folder instead.
**`plan` — capacity, and what it changes.** `total_sectors` becomes the
`Disc`'s `capacity_sectors` / `capacity_bytes`, and `capacity_bytes` is not
inert: `canonical_title_order`'s oversize gate (`disc/mod.rs:2039-2040`, body
`:2209-2210`) demotes any title whose `size_bytes` exceeds it, which decides
which title sorts first and therefore what `-t 1` selects. The capacity
reported here is **the synthesized image's own size** — what an ISO built
from this folder would report — and nothing else. It is NOT padded up to a
media tier (BD25/BD50/…): that was considered and does not work, since a
folder does not record the tier of the disc it came from, so a 22 GB folder
off a BD50 would pad to BD25 and diverge anyway, and the padding would
inflate any pre-sized output written from the image. The consequence, stated
plainly: **for a folder that is missing files the source disc had — a
selective MakeMKV backup being the normal case — the capacity is smaller
than the source disc's, the oversize gate is therefore stricter, and a
borderline "play-all" composite title that the ISO kept can be demoted here.
`-t 1` can select a different title from `dir://FOLDER` than from an
`iso://` of the same disc.** For a complete folder the two agree, because
the capacities agree. This is chosen over the alternatives because it is the
only one that is self-consistent: `dir://X` and an `iso://` built from `X`
describe the same image and must answer the same way.
**Tests.**
* `a_file_past_the_ad_ceiling_splits_on_a_block_boundary` — a file above the
30-bit AD ceiling must split into MULTIPLE descriptors, and every
non-final one must be a whole number of blocks, or the next extent's bytes
start mid-sector and the file reassembles wrong. Kills a mutant that emits
one oversized AD (whose length would collide with the extent-TYPE bits
`udf.rs:642` reads).
* `two_names_the_reader_cannot_tell_apart_are_refused` — audit finding: the
uniqueness check compared raw host names, while `parse_udf_name` trims
whitespace and drops code units `char::from_u32` rejects. So
" 00000.m2ts" and "00000.m2ts" were two distinct entries at plan time and
ONE name at read time; `find`/`read_file` take the first match, so a title
resolved to the wrong file's extents and muxed the wrong bytes at exit 0.
Calls `plan` on a real folder and fails if the round-trip key is reverted
to comparing host names.
* `an_over_long_name_is_refused_by_the_planner` — audit finding: the length
was narrowed with `as u8`, so a 255-byte ASCII name (POSIX NAME_MAX, legal
on ext4/APFS/NTFS) encoded to 256 bytes with the CS0 compression byte and
wrote a length of ZERO, making every later entry in that directory read
from the wrong offset. An earlier version of this test asserted arithmetic
about the constants and never called `plan`, so it would have passed with
the guard deleted.
* `the_subdir_cap_refuses_a_folder_with_too_many_subdirectories` — pins the
arithmetic relationship the guard relies on and executes the guard rather
than restating the constant: link count (child dirs + 1) is 16 bits, so
exceeding it wraps and the image lies about its own directory structure.
Creating 65,535 directories in a test is not reasonable, so this does NOT
exercise `walk`.
+492
View File
@@ -0,0 +1,492 @@
# src/disc/mod.rs — internal notes
Overflow rationale for internal (non-`pub`) items in `src/disc/mod.rs`,
relocated here by the comment-guard so the in-file comment can stay short.
Each section is pointed to by a one-line `// See docs/disc-mod.md — <topic>`
comment at the corresponding call site.
## aacs_dir_present
THE ONE definition of "is this disc structurally AACS-encrypted". Both
`Disc::identify` (fast path, filesystem only) and `Disc::scan_with` (full
scan) need this, and both used to spell out the same two `find_dir` calls by
hand. They agreed today; nothing made them agree tomorrow — adding a third
AACS location, or excluding an empty placeholder directory, to one copy and
not the other would silently desync the fast identify from the full scan (the
same disc reported encrypted by one and clear by the other).
This is STRUCTURAL, not cryptographic: it says the tree looks like an
encrypted disc, not that any sector actually is. A folder copied verbatim
from a decrypted disc keeps its `AACS/` and answers true here (see
`session::scan_dir`, which corrects for exactly that).
## correct_truehd_channels
Corrects a title's TrueHD audio-stream metadata by probing the first
decrypted access units — channel count, real sample rate, and Atmos detection
in a single major-sync read. The MPLS descriptors declare the BASE layout
(often 5.1, or a container-guessed rate) even for a 7.1/Atmos TrueHD track;
the truth is in the MLP major sync. `reader` must yield DECRYPTED sectors
(the m2ts is AACS-encrypted, so this can only run at mux time, not scan).
Reads a bounded window of the title's first extent.
Corrections, each individually guarded so a malformed field never writes a
wrong header:
- **Channels**: from the presentation channel masks (as before).
- **Sample rate**: from the whitelisted rate nibble; left untouched on an
unknown rate or no major sync.
- **Atmos**: when a 4th substream is detected AND the stream still carries
the basic descriptor label, the label is promoted to the Atmos form;
richer editorial labels (e.g. an existing "Dolby Atmos") are left intact.
## Resolution::pixels
`Unknown` deliberately does NOT fabricate a plausible 1920x1080. This is the
same trap already closed on `AudioChannels::count` (which used to return 6)
and `SampleRate::hz` (48000.0): a plausible wrong answer is indistinguishable
from a real one at every call site, so a caller can forget to check the
variant first — the `json://` sink had already walked into exactly that,
reporting confident dimensions for a stream whose neighbouring `resolution`
field said "unknown".
It also does not return `(0, 0)`: that shape looks like a usable pair, so the
MP4 sink stored it and serialised a 0x0 video track — a structurally complete
file no player can render, written with no error. Before that it returned a
fabricated 1920x1080, which was wrong but at least playable, so the sink had
never needed a guard and the absence of one was invisible. A doc comment
listing which callers are safe cannot hold this: one did not check, and a
third kept its own duplicate `Unknown` test long after the accessor took the
job over. `Option` forces every caller to make its OWN decision: Matroska
omits the optional PixelWidth/PixelHeight elements, the metadata sinks report
the dimensions as absent, and MP4 must refuse per ISO/IEC 14496-12 (width and
height are mandatory in both `tkhd` 8.3.2 and VisualSampleEntry 12.1.3).
## has_probable_video
The exposed `streams` come from the title's FIRST PlayItem only, so a
feature whose first PlayItem is a non-video bumper reports
`has_video() == false` and would be wrongly disqualified (audit R6). This
predicate is deliberately MORE permissive than `has_video` — it only ever
ADMITS a title the stricter gate would reject — so it cannot reopen the
decoy hole: a size-inflated streamless decoy is demoted STRUCTURALLY by
composite detection in `Disc::rank_titles`, not by this gate, while it still
rejects tiny streamless menu/metadata playlists. The size fallback requires
at least one clip, so a clip-less pure-metadata decoy (which cannot be a
real bumper-lead feature — that has the feature clip) is still rejected.
## image_crack_extents
The extents an image-time CSS crack scans, in the crate's CANONICAL order:
the main feature's own extents, in natural playback order. Both halves are
deferred to logic the crate already owns rather than re-derived here:
- WHICH title — `scan_with` has already sorted `titles` with
`Self::canonical_title_order`, so the canonical main feature is the first
title that actually has extents. No local pick.
- WHAT ORDER — playback order, i.e. the extent vector untouched, exactly as
`Self::decrypt_keys_for_title` hands `&title.extents` to
`css::crack_key_outcome`.
**Why this function exists (do not let the copy grow back):** `Disc::scan_image`
used to re-implement both halves inline — it picked the title with the
largest total sector count, then re-sorted that title's extents
LARGEST-CELL-FIRST before handing them to `crack_key_outcome`. Both had
drifted from the canonical rules:
- Largest-cell-first is the 1.5.1 garbage bug. A CSS DVD's biggest cell opens
with a long CLEAR run, and the crack's 50_000-sector budget is shared
across the whole extent list, so starting there can exhaust the budget
without ever reaching a scrambled sector — the crack reports `Unencrypted`
and the mux emits scrambled MPEG as plaintext. Playback order reaches the
scrambled feature body after only the small clear front matter that
precedes it on a real disc (CSS: the title key is recovered from the
scrambled data itself, so the scan must actually MEET scrambled data).
- The sector-count pick ignores `canonical_title_order`'s capacity gate, so
it selects the oversize "play-all" composite (whose declared cells
double-count data shared with other playlists) instead of the real feature
— i.e. a DIFFERENT title, and on a multi-VTS disc a different VTS, whose
CSS title key does not descramble the feature at all.
The result: the same disc cracked as an ISO could disagree with the same
disc cracked from the drive, which is precisely what the duplicate was free
to do. Keep the derivation here, shared, so it cannot recur.
## read_aacs_inputs_from_reader
Reads a disc's AACS key-input files from a sector source: returns
`(Unit_Key_RO.inf, MKB)` raw bytes. Shared body for `Disc::read_aacs_inputs`
(ISO) and `Disc::read_aacs_inputs_from_drive` (live drive). Prefers MKB_RO,
falls back to MKB_RW, then TRIMS to the real record length. Both files are
allocated to a fixed ~128 MiB and zero-padded, so reading either ships up to
~124 MiB of nothing — trim to the record stream so callers send/store a few
MB, not 128 MiB.
## read_aacs_version
AACS major version (`crate::aacs::mkb::AACS_MAJOR_BD` /
`crate::aacs::mkb::AACS_MAJOR_UHD`) from the content certificate. Drives the
`Unit_Key_RO.inf` parse stride (48-byte V10 vs 64-byte V20/V21), so the
out-of-band key-fetch path parses `enc_title_keys` at the right stride (a
server VUK then derives the correct unit keys).
When no content certificate is readable/parseable, defaults to **UHD (V20,
64-byte stride)** — the conservative choice the pre-1.2.0 fetch path
hardcoded — and logs it: a wrong stride here folds a server VUK against
mis-strided title keys (silent wrong unit keys), so a missing cert must not
quietly pick the V10 stride for a UHD disc.
## read_mkb_content
Reads the AACS MKB's real record stream — NOT its zero padding. `MKB_RO.inf`
/ `MKB_RW.inf` are allocated to a fixed ~128 MiB and zero-padded; the actual
record stream is a few MiB. Reads a bounded prefix, finds the record-stream
length via `crate::aacs::mkb::mkb_content_len` and returns exactly that,
growing the prefix if the records run past it. This avoids reading 100+ MiB
of padding on every scan AND avoids the `read_file` `MAX_FILE_BYTES` cap that
(since 0.31.0) rejected the padded 128 MiB MKB outright — which made
`read_aacs_inputs` fail and autorip report "could not read this disc's key
files" without ever contacting the keyserver.
## CANONICAL_TITLE_ORDER_KEYS
**Why not just sort by duration descending?** Branching UHDs (and some BD
authoring) ship a "play-all" virtual playlist that references the same
source clips multiple times for seamless alternate-angle / alternate-ending
playback. Those playlists report an inflated `duration_secs` (often 4+
hours) and an inflated `size_bytes` greater than the disc's physical
capacity. Example seen in the wild — *The Amateur (2025)* UHD, 58.5 GB
BD-100 disc:
| Title | Playlist | Duration | Size | Clips |
|-------|------------|----------|---------|-------|
| 1 | 00020.mpls | 4h 13m | 92.4 GB | 253 |
| 2 | 00800.mpls | 2h 02m | 57.2 GB | 1 |
Title 1's 92.4 GB cannot fit on a 58.5 GB disc unless the same clip data is
referenced multiple times — proof it's a virtual composite. A duration-only
sort would put it at `titles[0]`, so `freemkv -t 1`, `disc.titles.first()`,
and autorip's main-feature picker all grab the 4-hour composite instead of
the 2-hour movie that actually matches TMDB.
**Effect on non-branching discs:** unchanged — the main movie is already the
longest 1-clip title. **Effect on branching UHDs:** the virtual play-all
playlist is pushed to the back, the actual movie surfaces at index 0.
Defined immediately beside the comparator, so a diagnostic can NAME the
ordering instead of restating it. The `freemkv::diag` main-feature decision
row used to carry its own hand-written copy of this list, and it drifted: it
still advertised a `fewest-clips` key long after the comparator replaced
clip-count with largest-physical-size, so the `--log-level 3` bug-report log
explained freemkv's top-level pick with a rule freemkv does not apply. Any
change to the keys must change this list in the same edit.
## probe_forced_subtitles_for_bdts_titles
Pulled out of `scan_with` as its own callable predicate: the decision is
otherwise reachable only by driving the full scan pipeline (tree dispatch ->
title parsing -> this loop), so a test could not pin the gate without also
authoring an entire synthetic disc image. Running it against an HD-DVD/DVD
(`MpegPs`) title's extents would demux the wrong container, since only
`BdTs` titles carry PES-wrapped PGS the shared classifier
(`pgs_forced_probe`) understands.
## MAIN_FEATURE_ORDER_KEYS
- `nav-feature`: the title the disc's own HDMV navigation plays as the
feature, resolved by running First-Play in the `crate::bdnav` VM the way a
real player would. This is the authoritative, size-independent pick; it is
video-gated (a mis-resolve cannot select a streamless playlist) and simply
absent when the VM abstains (BD-J discs, or non-convergence), in which
case the lower keys decide.
- `authoring-feature`: a title the disc's own menu authoring designates as
the feature (has video, is not itself a composite, and is within a sane
fraction of the longest title's duration) outranks everything else — a
size-independent signal an obfuscation decoy cannot forge.
- `standalone`: a title that is a play-all / wrapper COMPOSITE — its clip
set properly contains another substantial video title's whole clip set (a
real UHD title's decoy playlist `00245` = `[bumper][00001 feature
clip][outro]` case) — is demoted below the standalone title it wraps. This
is the STRUCTURAL anti-decoy signal; it does not depend on stream counts,
physical size, or disc capacity, so it survives where those are spoofed or
unknown.
- `has-video`: a title with no plausible video content (see
`DiscTitle::has_probable_video`) is demoted below every title that has
some — a floor that drops streamless menu/metadata playlists.
## rank_titles
This catches both the wrapper decoy (`00245 ⊋ 00001`) and the oversize
play-all — independent of capacity, which the `fits-disc` gate needs and
which is `0`/unknown on drives that fail READ CAPACITY.
### Wrapper-shape gate (issue #45)
A proper-subset relation with `Q.size ≥ 0.5·P.size` is NOT by itself proof
that `P` is a wrapper. A genuine **seamless-branch feature** `P` (body clip
plus intro/credits/alternate-branch segments) is a *superset* of its
single-clip body decoy `Q`: `Q` runs almost as long as `P` (e.g. 1h50 vs
2h03, ~0.89 of the duration) yet is markedly smaller, because `P`'s branch
segments add bytes without adding much played runtime. The old size-only
gate mis-flagged such a `P` as a "play-all wrapper" and demoted the real
2h03 feature below the shorter 1h50 body — a SAFETY bug (wrong title ripped).
So the composite flag additionally requires `P` to *have* a wrapper shape,
via one of two capacity-independent, per-title (`duration_secs`, `size_bytes`)
signals on the subset `Q`:
- **concat / play-all:** `Q.duration < 0.85·P.duration` — `Q` is a much
shorter cut and `P` pads its runtime with other segments (the true
play-all runs ~the SUM of its parts, so each part is well under half).
- **bumper / outro wrapper:** `Q.size ≥ 0.90·P.size` — `Q` already holds
almost all of `P`'s bytes and `P` merely prepends/appends a short
non-feature lead/tail (the `00245 ⊋ 00001` decoy: `Q` is ~0.96 of both the
duration and the size, so the size limb keeps it demoted).
Otherwise — `Q` runs almost as long as `P` (≥0.85) yet is well under 0.90 of
its size — `P` is the fuller branch set and is kept. `0.85` sits above a real
concat's per-part ratio (~0.5) and below a seamless body's (~0.89); `0.90`
sits above a bumper wrapper's near-1.0 size share and below a branch body's.
**Known residual:** a wrapper that pads runtime only modestly AND withholds
most of its bytes from the body clip is ambiguous by shape alone. The corpus
title-selection gate is the backstop across the disc hoard, and
higher-precedence nav/authoring signals override the composite key.
## aligned_unit_keys_validate
Conservative: a sample that is not AACS-scrambled proves nothing, and with
no scrambled sample at all there is nothing to disprove against, so it
returns `true` (accept). It returns `false` when ANY scrambled sample cannot
be restored to clear MPEG-TS by ANY unit key in the set. That covers two
distinct failure shapes: (1) a wholly wrong key (a keydb VK that does not
match this disc) — no sample decrypts; and (2) a *partially* applicable key
set on a multi-CPS-unit disc — the resolved keys cover CPS unit 0 but not
CPS unit 1. Accepting on the first sample that decrypts (the old behaviour)
would commit such a set, after which CPS-unit-1 sectors pass through as raw
encrypted bytes into the ISO/MKV with no error surfaced anywhere. Requiring
every scrambled sample to decrypt rejects the incomplete set so the caller
falls through to the next candidate. Reuses the ecosystem's single
`is_clean` content-clarity predicate and the full (bus + AACS) unit decrypt,
so it agrees with the actual mux decrypt.
## ensure_decryptable
The verdict, in order:
- `raw == true` → `Ok(())`. `--raw` intentionally skips decryption and needs
no key (the caller wants an encrypted image).
- `self.css_error.is_some()` → `Err(Error::CssNoDiscKey)`. The scan saw
scrambled CSS sectors but recovered no title key (`self.css` is `None` yet
the content IS encrypted). Treating `css.is_none()` as "unencrypted" would
mux scrambled MPEG as plaintext garbage. A DISC-LEVEL verdict
(`error::is_disc_level_no_key`) — the main feature's crack failed, so
every title fails the same way and the rip loop must stop rather than
skip each title in turn.
- AACS-encrypted (`self.aacs.is_some()`) with no usable key
(`decrypt_keys()` is `None`) → `Err(Error::NoDiscKey { .. })`, naming the
disc by hash.
- CSS-encrypted (`self.css.is_some()`) with no usable key →
`Err(Error::CssKeyMissing)`. (The disc-wide `decrypt_keys()` yields
`Css{..}` whenever `css.is_some()`, so this is defensive; the live
multi-VTS case is gated by `Self::ensure_decryptable_keys`.)
- otherwise → `Ok(())`. A genuinely unencrypted disc has `None` keys
legitimately, and a CSS disc whose keyless crack succeeded has a key.
## decrypt_keys_for_title
For a DVD the CSS title key MUST be recovered before descrambling: a
scrambled sector without a recoverable crib cannot self-crack, and CSS
leaves the pack/PES header clear, so a sector left un-descrambled would mux
as a structurally-valid but corrupt PES packet with no loss reported. Two
ways to get it:
- **Fast path** — the scan already cracked a key whose LBA span
(`crate::css::CssState::crack_span`) covers this title's VTS: reuse it. No
re-read, and on a live drive no second CSS bus-auth round-trip. CSS title
keys are per-VTS, so an overlapping span is the same key.
- **Crack** — when up-front detection missed (`self.css == None`) or the
title lives in a different VTS: crack the key from this title's OWN
extents in a SINGLE scan, in natural PLAYBACK ORDER (never
largest-cell-first, which starved the crack in a big cell's clear prefix —
the 1.5.1 bug). Playback order reaches the scrambled feature body after
only the small clear front matter (logo / rating card) that precedes it.
One scan = one CSS-locked early-bail, so a locked title is not
re-hammered per cell against a live drive (hard rule #2); its 50k-sector
budget is the same accepted bound the disc-wide scan uses.
Outcomes (`batch_sectors` sizes the crack's batched reads):
- `(Css{title_key}, false)` — descramble with the recovered/reused key.
- `(None, true)` — a genuinely-clear title needs no key; the gate passes it.
- `(None, false)` — scrambled but no key recoverable → hard failure via
`Self::ensure_title_decryptable`, never a silent garbage mux.
## inject_unit_keys
Injects pre-resolved AACS unit keys into a scanned disc — the deferred-mux /
resume path. The keys come from the mapfile's `# freemkv-uk:` header
(persisted at sweep time when the disc was keyed), so the mux decrypts
directly with NO key-service round-trip. Populates `self.aacs.unit_keys` so
`Self::decrypt_keys` returns them and marks the source `ExternalUk`.
If the scan built no AACS state (`self.aacs == None`) — which happens when
the keydb was absent at scan time (`scan_aacs_no_keydb` ->
`aacs_error = KeydbLoad`) — this synthesizes a minimal `ExternalUk` state
for an encrypted AACS disc. A Unit Key is the FINAL per-title decryption
key; the keydb is only needed to *derive* it, and that derivation already
happened at sweep (the UK is in the mapfile). So a UK alone is sufficient to
decrypt the on-disk ISO — AACS 2.0 bus decryption was applied by the drive
at read time, so `read_data_key` is unused for file-backed mux. Without
this, a keyed disc swept without a keydb would recover its UK yet still
report E8005 (no usable `decrypt_keys`) at remux. No-op for an unencrypted
or CSS (DVD) disc.
## scan_with_passes_the_halt_flag_to_the_bluray_enumerator
Every BD and DVD cancellation test in this crate calls
`scan_bluray_titles` / `scan_dvd_titles` DIRECTLY, so the three-line wiring
in `scan_with` that connects the operator's Stop to them was covered by
nothing at all: replacing `opts.halt.as_ref()` with `None` on either branch
left the whole suite green while a Stop during a BD or DVD scan silently did
nothing. Only the HD-DVD branch was pinned.
The fixture is deliberately the smallest disc that takes the BD branch — a
bare `/BDMV` with no PLAYLIST — because the point under test is the
ARGUMENT, not the enumerator's own (separately tested) halt polling. Nothing
between `scan_with`'s entry and the enumerator reads the flag (the disc is
unencrypted, so the AACS path is skipped), so a green here can only mean the
flag arrived. Mutation: `Self::scan_bluray_titles(reader, &udf_fs, None)?`
fails here.
## halted_reads_do_not_report_the_hddvd_scan_as_successful
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The HD-DVD enumerator must not swallow that into a
successful scan. Measured before this was fixed: the scan returned `Ok`
with both titles present and ZERO streams on each — a cancelled scan
wearing the shape of a disc whose clips carry no video or audio. Downstream
that is a title list to cache, display and rip from.
## scan_with_capacity_bytes tests
`scan_with`'s `capacity_bytes = capacity as u64 * 2048` feeds
`canonical_title_order`'s "bigger than the whole disc = play-all composite"
threshold.
`..._not_addition`: chosen so `capacity * 2048` clears both titles (neither
is oversize, so the bigger one — OTHER — ranks first), but
`capacity + 2048` lands BETWEEN the two sizes: MAIN stays non-oversize while
OTHER flips to oversize and gets demoted, changing `titles[0]`. A `*` -> `+`
mutation is caught by this ranking flip, not merely a wrong numeric
threshold value.
`..._not_division`: same mechanism, tuned to catch a `*` -> `/` mutation
instead: `capacity * 2048` clears both titles, but `capacity / 2048`
collapses to a threshold far below MAIN's declared size while still leaving
MAIN under it, so only OTHER flips to oversize.
## read_mkb_content_grows_prefix_past_16mib_when_records_run_longer
This fixture's record stream is deliberately built so the first 16 records
land EXACTLY on the 16 MiB boundary (so the 16 MiB prefix alone parses as a
clean, complete record stream — the case a naive "n < buf.len() means done"
check would wrongly accept) and 4 more MiB of records follow, terminated by
the explicit `00 000000` end marker. Only a caller that actually grows past
16 MiB recovers the true 20 MiB content length.
## main_feature_order_demotes_wrapper_composite_decoy
The decoy `00245.mpls` is a WRAPPER COMPOSITE — `[00339 2s bumper][00001 the
whole feature clip][00336 outro]` — so its first-PlayItem STN reports v=0,
and it is the LONGEST and LARGEST title (8350s / 78.4 GB vs the feature's
8038s / 76.5 GB). Muxing its zero frames produced E6008 forever. It must be
demoted STRUCTURALLY (its clip set properly contains the feature's), not by
the fragile first-PlayItem stream count — this is what makes the fix robust
to the video-bearing-composite variant (see
`main_feature_order_demotes_video_composite`).
## key_source_failure_is_not_reported_as_a_missing_disc_key
Drives the real chain end to end — `KeySource` -> `resolve_and_apply_traced`
-> `Disc::aacs_error` -> `ensure_decryptable` — twice over the SAME disc,
and asserts the two operator-visible verdicts differ:
- source returns `Err(KeyServiceUnavailable)` (the HTTP-502 outage) → E7028
"the key service could not answer; retry later",
- source returns `Ok(vec![])` (the service answered, no entry) → E7022 "no
key source has a decryption key for this disc".
The credential and rate-limit verdicts are asserted alongside, since each is
a different operator action (fix the token / back off).
## css_disc_wide_no_key_is_disc_level_while_per_title_stays_skippable
The two CSS no-key conditions are NOT the same verdict and must classify
oppositely through the public predicates:
- **disc-wide** — `css_error` is set: the MAIN feature's crack failed, so
every title of this disc fails identically. Must be
`crate::error::is_disc_level_no_key` (the rip loop fail-fasts) and must
NOT be `crate::error::is_skippable_title_stub`. While both conditions
shared `E_CSS_KEY_MISSING`, an uncrackable CSS disc iterated all N titles
logging "title skipped" and exited 0 — a total failure reported as
success.
- **per-title** — one title's own re-crack failed on a multi-VTS disc
(`title_is_clear == false`, no key): skipping it and finishing the rest is
correct policy, so it must STAY skippable and must NOT be disc-level.
Pinned in both directions so a future change cannot silently flip either.
## correct_truehd_channels_full_correction_and_atmos_promotion
Kills the `was_basic ==`, the channels/rate `!=`/`&&` guards' "already
correct" branch, the `!matches!` per-stream skip, the `is_atmos ==
Some(true)` branch, and the whole-function no-op mutant.
## byte_offset_in_title_accumulates_across_extents
Kills: `lba >= start && lba < end` flipped to `||` (the first extent's
disjunction would trivially match almost any lba and return the wrong,
too-early offset); `cumulative +=` flipped to `*=` (cumulative is seeded at
0, so `*=` freezes it at 0 forever); and `sector_count * SECTOR_BYTES_U64`
flipped to `+` or `/` (wrong per-extent byte length folded into cumulative).
## chapter_at_offset_concrete_arithmetic
Kills: the `/` (time-fraction) flipped to `%` or `*`; the `*` (duration
scale) flipped to `+`; the `<=` chapter-scan comparison flipped to `>`; the
final `chapter_idx + 1` flipped to `-` or `*`; and every whole-function
fixed-tuple replacement (none produce `(2, 60.0)`).
## locate_ranges_positive_bps_duration_and_at_risk
Kills `bps > 0.0` flipped to `==`/`<` (both would take the `else 0.0`
branch here, wrongly reporting 0); `(*size as f64) / bps` flipped to `%` or
`*`; and the trailing `* MILLIS_PER_SEC` flipped to `+` or `/`.
## decrypt_keys_for_title_css_span_reuse_is_half_open
A CSS title key is per-VTS: reusing the scan's cracked key for a title that
lives OUTSIDE the cracked span descrambles that title with the wrong key,
and the mux emits garbage at exit 0. `crack_span` is documented as the
half-open LBA span `[start, end)`, so overlap is `extent.start < span.end &&
span.start < extent.end` — both comparisons STRICT. A title that merely
ABUTS the span (ends exactly where it begins, or begins exactly where it
ends) shares no sector with it and must NOT reuse the key.
The reader serves only clear (all-zero) sectors, so a title that falls
through to its own crack is reported unencrypted — distinguishable from the
reused-key answer both in the key and in the is-clear flag.
## merge_content_key_ranges
Merges per-title AACS key ranges into the sorted, disjoint set the
whole-disc map needs (`crate::decrypt::AacsKeyMap::entry_for` requires
disjoint ranges). Titles that share a clip resolve the SAME physical span
(same LBAs -> same CPS unit -> same key). When a later range overlaps a kept
one that carries the SAME key index and phase, the two are UNIONED (end
extended to the max) — this covers both the exact-duplicate (shared clip)
case and any partial overlap without ever dropping coverage, so no encrypted
LBA is left in no range (which would pass through as ciphertext). A real disc
never produces two DIFFERENT keys for one LBA; if that malformed case ever
appeared, the later range is dropped to keep the set disjoint rather than
extend one key over another key's LBAs.
+9
View File
@@ -0,0 +1,9 @@
# `disc::profile`
Everything in `DiscProfile` is DERIVED from the `Disc` model at construction:
the `Stream` enum is split into three typed vectors (audio/video/subtitle),
the `qualifier` / `purpose` enums are decomposed into booleans, and the
"default track" selection is precomputed so downstream never recomputes it.
Every field is always populated — a sensible default (`"und"` language, empty
`name`, `false` flag) stands in where the model has nothing, so consumers
never handle a bare `Option`.
+17
View File
@@ -0,0 +1,17 @@
# disc_tests: capacity/layer derivation
`scan_image_reports_capacity_in_bytes_and_the_layer_count` covers the scan
reporting the medium's size on both axes it exposes, derived from the one
sector count the caller hands in:
* `capacity_bytes` is that sector count times the 2048-byte logical sector
(ECMA-167 / BD-ROM logical block size). It is what sizes a full-disc image
read and what the progress percentage divides by, so a wrong scale is a
wrong ISO length, not a cosmetic number.
* `layers` distinguishes single- from dual-layer media. The threshold sits
between the two real capacities: a single-layer BD-25 is 12,219,392
sectors (25,025,314,816 bytes / 2048) and a dual-layer BD-50 is 24,438,784
sectors, so BD-25 must report 1 layer and BD-50 must report 2.
`scan_image` takes the sector count as a parameter, so this exercises the
real derivation without a 50 GB fixture.
+23 -15
View File
@@ -29,7 +29,6 @@ seam).
| `init()` | Route to the matching registered unlocker (if any), then prepare for reads |
| `probe_disc()` | Probe disc surface for optimal speeds |
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
| `reset()` | Eject-cycle escape hatch. Caller-invoked only; not on the read path. |
| `lock_tray()` | Prevent tray ejection during rip |
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
| `eject()` | Eject disc tray |
@@ -104,14 +103,14 @@ descriptors or calls ioctls outside of a `ScsiTransport` implementation.
| Platform | Implementation | Device |
|----------|---------------|--------|
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
| Linux | `SgIoTransport` — synchronous `ioctl(fd, SG_IO, &hdr)` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
background thread) and a fresh fd opened — the kernel's USB error recovery
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
The Linux backend issues each command as one synchronous `ioctl(fd, SG_IO, &hdr)`
(`linux.rs`); the timeout is the kernel's, set via `hdr.timeout`, not a userspace
`poll()`. Opens with `O_RDWR | O_NONBLOCK`. (Pre-0.13.20 this backend ran an async
`write()` + `poll(1.5s)` + close-on-timeout + background-reopen pattern; it was
reverted in 0.13.20 — see the note at the top of `linux.rs`.)
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
@@ -127,14 +126,13 @@ The shim handles:
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
`SgIoTransport::reset` (Linux) does pure userspace state cleanup: an open +
close pair to make the kernel cancel any SG_IO commands queued against a
previous fd, a 2 s sleep to let the kernel finish that cancellation, then a
fresh fd to send ALLOW MEDIUM REMOVAL to clear any stale tray lock. It does
NOT issue `SG_SCSI_RESET` or escalate via STOP+START UNIT. Both were tried
in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA bridge); both failed
to recover wedged drives and made the wedge worse. The macOS reset (which
had been a no-op) was removed entirely in 0.13.6, and the top-level
The Linux backend has no `reset`. `SG_SCSI_RESET` and STOP+START UNIT
escalation were tried in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA
bridge); both failed to recover wedged drives and made the wedge worse, so no
Linux transport-reset path exists today — recovery is just a fresh fd opened on
the next command. The only `reset` on a transport in the crate is
`SptiTransport::reset` on Windows (`windows.rs`). The macOS reset (which had
been a no-op) was removed entirely in 0.13.6, and the top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
removed at the same time (no callers).
@@ -241,3 +239,13 @@ Available speeds:
| Blu-ray | 1x (4,500 KB/s) through 12x (54,000 KB/s) |
| DVD | 1x (1,385 KB/s) through 16x (22,160 KB/s) |
| Max | 0xFFFF (drive decides) |
---
## Windows Path Normalization
`normalize_path` in `src/drive/windows.rs` converts a device path ("D:",
"D:\\", "\\.\D:", "\\.\CdRom0") to Windows `\\.\X:` form. A near-identical
`normalize_device_path` exists in `scsi::windows`; both are kept because
they live in separate `cfg(windows)` modules that cannot easily share a
helper without introducing cross-module coupling.
+15
View File
@@ -0,0 +1,15 @@
# Linux `sg*` device enumeration
`enumerate_sg_names` (in `src/drive/linux.rs`) lists `/dev/sgN` SCSI-generic
device names to probe for optical drives.
Linux assigns `/dev/sgN` sequentially across *all* SCSI-generic devices
(disks, tape, HBAs, optical), not just optical drives. That means a fixed
`sg0..15` range can miss an optical drive on a host with many other SCSI
targets ahead of it in enumeration order.
To avoid that, the function prefers the exact present-device list from
`/sys/class/scsi_generic/`, which enumerates only the `sg*` nodes that
actually exist. It falls back to a bounded `sg0..15` probe only when sysfs
itself is unreadable, which happens in some minimal containers that lack a
mounted `/sys`.
+142
View File
@@ -0,0 +1,142 @@
# src/drive/mod.rs — internal notes
Long-form rationale relocated from `src/drive/mod.rs` comments that exceeded
the comment-guard's internal-comment cap. Each section is pointed to by a
short `// See docs/drive-mod.md — <topic>` comment at its original site.
## Error-recovery mode page (PER bit)
`MODE_PAGE_ERROR_RECOVERY` (0x01) is the SBC/MMC Read-Write Error Recovery
mode page. We flip the `PER` bit to make the drive REPORT a recovered read
(via CHECK CONDITION + sense key RECOVERED ERROR) instead of silently
returning best-effort data as GOOD status. On marginal/dirty media that
silent-GOOD data can be mis-corrected — a rip that "passed clean" but decoded
with errors. With PER on, freemkv sees the marginal read and re-reads it in
Pass N (a loud miss, never a silent commit). See
`build_error_recovery_select_payload`.
## `sleep_until_halted` — why it exists and why it was dormant
This was `#[cfg(test)]` for two releases, kept alive only by its unit tests,
while the two production paths that actually sleep — the `wait_ready` poll
backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
`std::thread::sleep` and so were deaf to the operator's Stop for ~30 s and
~15 s respectively. Every other drive path returns `Halted` at the next
`checked_exec` boundary; a Stop pressed during spin-up simply did nothing
visible until the poll ran out. The primitive existed; the sleeping code just
did not call it. It is production code again.
## `Drive::read` — single-shot contract and the removed inline retries
`recovery=true` uses `crate::scsi::READ_RECOVERY_TIMEOUT_MS` (60 s, matches
sg_dd) for the `freemkv_engine::recovery::patch` pass; `recovery=false` uses
`crate::scsi::READ_TIMEOUT_MS` (10 s) for `freemkv_engine::recovery::copy`'s
fast skip-forward sweep. Both budgets are generous enough that the drive can
finish ECC recovery on a marginal sector — pre-0.13.21 this was 1.5 s on the
fast path which forced the kernel mid-layer to time out and escalate while we
waited anyway. On any failure this returns `Err(DiscRead)` immediately;
orchestration (`freemkv_engine::recovery::patch` multi-pass, `DiscStream`
adaptive batch halving) handles retry policy.
Inline retry phases (5x gentle + reset+reopen + 5x more) were removed in
0.13.6: on some USB-SATA bridges the inline reset wedged drive firmware
without ever recovering a sector. The remaining recovery layers
(`freemkv_engine::recovery::patch` multi-pass, `DiscStream` batch halving) do
not touch the wedge-prone reset path.
## `find_drive` — media-preference selection policy
On a multi-drive system (common on Windows, where an empty/not-ready drive
can enumerate first) returning the first drive blindly can pick a drive with
no disc, dooming the operation. So `find_drive` opens each candidate in
enumeration order, queries `Drive::drive_status` (GET EVENT STATUS, which
works regardless of firmware state), and returns the first drive reporting
`DriveStatus::DiscPresent`. If no drive reports a disc — or `drive_status()`
is unavailable/returns `Unknown` everywhere (single-drive or quirky bridges)
— it falls back to the first drive that opened, preserving historical
behavior so those setups don't regress. For just listing drives without
opening (e.g. UI sidebar), use `scsi::list_drives()` instead — it returns
`DriveInfo` (path + identity) without the cost of running every drive's
profile + identity probe.
## `open_block_device_for_sg` — why no `O_DIRECT`
Resolves a `/dev/sg*` path to the corresponding `/dev/sr*` block device by
walking sysfs, then opens it for read. No `O_DIRECT`:
`posix_fadvise(POSIX_FADV_DONTNEED)` flushes the cache before each pread,
which avoids buffer-alignment requirements while still forcing fresh device
reads. Returns `None` on any error (sysfs not present, no matching block
device, open failed); callers treat that as "no fallback available" and
propagate the original SCSI READ error.
## `resolve_device` — staged, not yet wired
The cross-platform dispatch is kept ready for the caller that will consume
it, so the per-platform implementations below it (and their tests) stay
live. `allow(dead_code)` marks that deliberately — this is not an accidental
orphan.
## Test rationale: recovery/halt/chunking coverage
The `halt_tests` and `command_tests` modules exercise mutation-sensitive
edges (halt-aware sleep timing, chunked-read offset arithmetic, drive-status
byte decoding, spin-cycle/wait-ready halt responsiveness). The following
notes preserve the "why this exact scenario" reasoning that the inline test
doc comments were trimmed from:
- **`disc_is_dvd_matches_only_dvd_profile_family`**: must match the DVD
profile family (0x0010..=0x001F) and ONLY that family. A false positive on
a BD/UHD profile (0x0040+) would skip the drive unlock UHD reads require; a
false negative on a DVD would re-introduce the CSS read failure. Widening
the range to `..=0x0040` makes the BD-ROM assert fire; a failed/short GET
CONFIGURATION must default to NOT-DVD so the unlock still runs.
- **`media_event_reply` / media-status decoding**: MMC-6 §6.7 — byte 5 of the
GET EVENT STATUS reply is a Media Status only when the Event Header says a
media event descriptor follows (NEA clear AND Notification Class == 4).
Decoding byte 5 regardless of those preconditions reads a reserved/zero
byte as Media Status 0 and reports NoDisc on a drive that has a disc loaded
— the classic "works on my drive, not theirs" firmware split. The drive is
untrusted input, so an event-less reply must fall back to the TEST UNIT
READY status instead.
- **Chunked-read tests** (`read_chunks_write_into_correctly_offset_buffer_regions`,
`an_undersized_buffer_errors_on_the_chunked_path_just_like_the_single_one`,
`chunk_lba_near_u32_max_errors_not_overflows`): the multi-chunk path slices
the caller's buffer by `count * 2048` and advances the per-chunk LBA with
`lba + done`, both unchecked. An undersized buffer PANICKED
("range end index out of range") out of the public `read`/`read_fua`
instead of returning `Err(DiscRead)` like the single-chunk path; an LBA
range crossing `u32::MAX` overflowed (debug panic, or release wrap to a low
LBA silently read). These tests pin both to error-not-panic/overflow, and
one writes a distinct marker per chunk to confirm the byte-offset
arithmetic (`done * 2048`, not `done + 2048`) is correct — no other mock
touches the buffer contents, so a `*` -> `+`/`/` mutation would otherwise go
unasserted.
- **`wait_ready`/`spin_cycle` halt tests**: `wait_ready` was the one drive
path that called `self.scsi.as_mut().execute(..)` instead of
`checked_exec`, and its 60 x 500 ms loop never read `self.halt` — a cancel
during spin-up was ignored for ~30 s. `spin_cycle` had the same gap across
two bare `execute` calls and two plain `std::thread::sleep` pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`), and it runs from the
recovery path — precisely when an operator is most likely to press Stop.
The tests pin: a Stop set before the call returns `Halted` immediately with
no command issued; a Stop set mid-poll (on a deterministic Nth call, no
wall-clock race) exits at the next `checked_exec` boundary; a Stop that
lands during the spin-down pause wakes the halt-aware sleep within
~100 ms instead of blocking for the full 5 s.
- **`read_logs_a_good_status_short_transfer`**: a READ(10) that completes
with GOOD status but a residual underrun is correctly refused, but the
refusal used to log nothing — so a drive that residual-underruns on GOOD
status produced the same silent journal as a scratched disc, and the
operator was sent after the wrong remedy (replace the drive vs. clean the
disc). The test pins that the refusal now warns with lba/transferred/
expected/code.
## `open_block_device_for_sg` doc contract (kept short in code)
See the section above for the `O_DIRECT` rationale; the pointer comment near
the function keeps only the API contract.
+69
View File
@@ -0,0 +1,69 @@
# `dropgate` — audio drop bookkeeping rationale
## Keep/drop rule
A clean mux keeps every frame it can and drops the ones it can't — video
always survives (it's inter-frame predicted; a per-frame drop would cascade,
so video resyncs/conceals instead), audio keeps every decodable access unit,
and a damaged audio AU is dropped rather than shipped as a decoder-choking
glitch.
## Detection is per-codec
The DETECTION is inherently per-codec — each format carries its own
authoritative corruption check (DTS: the core sync/header parse per ETSI TS
102 114; AC-3: the header CRC per ETSI TS 102 366; FLAC: the frame CRC-16;
…). `DropTally` only carries the UNIFORM response so every audio parser
behaves identically:
1. **Count** kept vs dropped AUs and the dropped duration.
2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
once-per-track aggregate at `warn` so it surfaces without debug logging.
3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
a poison flag so the remainder is dropped too (a track that damaged isn't
worth muxing).
## Sync preservation
**Sync preservation is the caller's responsibility**, not this type's: the
parser must advance its PTS clock across a dropped AU exactly as it would for
an emitted one, so a drop becomes a silence gap and never a shift of the
following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
## `verified_dropped` vs `dropped`
`verified_dropped` counts AUs dropped because they were INDIVIDUALLY verified
undecodable (a failed CRC/header/parity check); only these feed the
whole-track poison verdict. `dropped` also counts *collateral* drops — AUs
discarded as a consequence of one corruption (TrueHD's resync-forward run, or
a poisoned track) — which must NOT amplify a few real errors into a false
whole-track loss.
## `record_collateral_drop`
A collateral drop is caused by another AU's corruption (TrueHD's
resync-forward run to the next major sync, or an already-poisoned track), not
by being individually verified undecodable. It is counted and logged for the
drop report, but deliberately does NOT feed the poison verdict, so one
corruption event can't amplify into a false whole-track loss.
## `maybe_poison`
Whole-track fallback: after enough AUs to judge, if more than half were
dropped the track is too damaged to be worth muxing — latch `poisoned` and
log it loudly once. The minimum-sample gate keeps a short damaged burst from
poisoning an otherwise-good track. Judge on VERIFIED drops vs all AUs seen: a
track is only poisoned when a majority of its access units are individually
undecodable — not when a couple of corruption events forced long collateral
resync runs.
## Test: `interleaved_keeps_are_in_the_poison_denominator`
The poison verdict is a RATIO — verified drops against every AU seen — so
the kept count is half of it. `does_not_poison_a_mostly_good_track` records
its keeps AFTER the single drop, and `maybe_poison` only runs inside
`record_drop`, so the keeps are never in the denominator when the verdict is
actually computed: that test passes even with the kept count never
incremented. Interleaving them puts the kept count on the critical path,
where losing it turns the ratio into "verified drops vs verified drops" —
always >50% — and silently discards a healthy track.
+309
View File
@@ -0,0 +1,309 @@
# DTS / DTS-HD elementary stream parser (`src/mux/codec/dts.rs`)
Internal design notes moved out of source comments (see `ci/comment-guard.py`
prose caps). Each section is pointed to by a short `//` comment at the
corresponding site in `dts.rs`.
## `DTS_HD_EXT_SYNC`
An access unit is delimited by the next CORE sync; the parser locates and
exactly sizes each extension substream (via `exss_frame_size`) so a false
core sync inside the EXSS payload can't split the AU and truncate the
lossless extension.
## `emit_or_drop`
Gates an assembled access unit through the decodability check and either
pushes it or drops it. `au_pts`/`dur_ns` are already stamped on the shared
PTS clock (which the caller advances whether or not the AU survives), so a
drop leaves the following audio on its true timeline — a gap, not a shift.
Every drop is logged (fail-loud, never silent).
## `stamp_pts`
`front` is the AU's own core-PES PTS (from `front_pts`); `dur_ns` its
decoded duration.
The model matches the (correct) AC-3 path: **re-base to each PES's own
container timestamp, and advance by one frame duration ONLY within a run of
AUs that share the same PES.** A new PES (its `front` differs from the
previous AU's) trusts its own timestamp — so the emitted timeline tracks the
container and never drifts. Advancing within a PES is what fixes the DVD
case (several DTS core frames packed in one PES, which otherwise all
collided on that single PES timestamp → "non monotonically increasing dts").
A global running clock was WRONG here: once accumulated frame durations
exceeded the PES spacing it never re-based, drifting the track minutes past
the real length over a feature-long title.
## `CORE_HEADER_MIN_BYTES`
Number of leading bytes that must be buffered before the core `fsize` field
(bytes 5-7) can be decoded. This is a HEADER-LAYOUT minimum — "enough bytes
to read the size field" — and is deliberately distinct from
`MIN_CORE_FRAME_BYTES` (the decoded-frame-size validity floor). They must
not be conflated: this one gates buffer reads of the header, the other
rejects implausible decoded sizes.
## `MIN_CORE_FRAME_BYTES`
Minimum plausible decoded DTS core frame size, per ETSI TS 102 114: the
on-wire FSIZE floor is 95, so a real core frame is at least 96 bytes. A
decoded `core_size` below this means we matched a false/corrupt core sync
(a lucky 0x7FFE8001 in extension-substream payload whose 14-bit `fsize`
decoded to a tiny value) rather than a real frame, so we resync instead of
closing an access unit at a junk boundary and dropping the DTS-HD extension
tail.
## `PTS_UNSET`
Sentinel for "no valid PTS base captured yet". Real PTS-in-ns values are
non-negative (derived from the unsigned 90 kHz PES timestamp), so a
negative value can never collide with a genuine timestamp. Used to mark the
PTS base invalid after a forced flush so the next PES sets it regardless of
buffer state.
## `flush_tail`
Emits the final buffered access unit (the last core + its extension
substreams, which had no following core sync to close it during
streaming), gated through the decodability check. Requires a complete core
frame; drops a bare partial sync tail.
## `SYNCWORD_BYTES`
Byte length shared by the DTS core sync and the DTS-HD EXSS sync (both
32-bit words). `next_core_boundary` finds the next *valid* core sync after
the current core frame, to delimit the access unit: extension-substream
payload can contain byte sequences that match the core syncword, so each
candidate is validated by decoding its core size — a match whose decoded
size is implausible (`< MIN_CORE_FRAME_BYTES` or `> MAX_AU_BYTES`) is a
false sync and is skipped, continuing the search.
## EXSS header field bit widths
`EXSS_USER_DEFINED_BITS`, `EXSS_INDEX_BITS`, `EXSS_HEADER_SIZE_TYPE_BITS`,
etc. are the DTS-HD extension-substream (EXSS) header field bit widths
(ETSI TS 102 114, ExtSS header). `bHeaderSizeType` selects the short form
(`nuExtSSHeaderSize` 8 bits, `nuExtSSFsize` 16 bits) or, for larger
substreams, the long form (12 / 20 bits).
## `exss_frame_size`
DTS-HD extension substream (EXSS) total byte size — INCLUDING the
`0x64582025` syncword — read precisely from its header. `buf` must begin
with `DTS_HD_EXT_SYNC`. Returns `None` when the size fields aren't fully
buffered.
`nuExtSSFsize` is the total frame size in bytes minus one. Parsing it lets
the AU framer skip the extension by its exact length instead of scanning
its (arbitrary) payload for a core sync.
## `next_core_boundary`
Offset where the current access unit ends (the start of the next core
frame). The AU is the core frame plus its trailing DTS-HD extension
substreams, which are skipped PRECISELY by their declared size — so a
chance core syncword inside the XLL lossless payload can never be mistaken
for the next AU boundary (the bug that truncated the extension and produced
the "Failed to decode block code(s)" class). Falls back to the heuristic
core-sync scan only when an extension can't be sized (malformed / truncated
input).
## `scan_for_next_core`
Heuristic fallback (the pre-fix behaviour): scan forward for the next core
syncword whose decoded size is plausible. Used only when precise extension
skipping can't proceed; a chance core syncword in extension payload usually
decodes to an implausible size and is skipped.
## `dts_core_frame_size`
`fsize` is the 14-bit field at bits 46-59 of the header (bytes 5-7). On the
wire `fsize` is the frame length minus one, so this returns `fsize + 1`,
i.e. the core frame length in bytes (range 1..=16384). Callers treat the
result as the actual byte length and the MIN..=MAX range checks assume so.
Returns `0` when `data` is shorter than `CORE_HEADER_MIN_BYTES` — every call
site rejects that via the minimum-frame lower bound, so a `0` is never
mistaken for a valid tiny frame.
## `dts_core_samples`
Samples in one DTS core frame: `(NBLKS + 1) * 32`. `NBLKS` (7 bits) is the
core-header PCM-sample-block count (ETSI TS 102 114) that fixes the frame's
decoded sample count. Bit layout after the 32-bit sync: FTYPE(1) SHORT(5)
CPF(1) **NBLKS(7)** FSIZE(14) …, so NBLKS = byte4 bit0 + byte5 bits7-2.
## `DTS_PCMBLOCK_SAMPLES` / `DTS_SUBBAND_SAMPLES` (core-header validity)
DTS core-header validity constants (ETSI TS 102 114). For a NORMAL frame
`deficit_samples` must equal `DTS_PCMBLOCK_SAMPLES` — a termination frame
may carry fewer; `npcmblocks` must be a multiple of `DTS_SUBBAND_SAMPLES`;
`audio_mode` must be below `DTS_AMODE_COUNT`; `lfe_present ==
DTS_LFE_FLAG_INVALID` is rejected.
## `DTS_AMODE_COUNT`
Number of LEGAL `AMODE` (channel-arrangement) codes. The 6-bit AMODE field
(ETSI TS 102 114 §5.3.1) has 16 defined channel arrangements, codes 0-15;
only 16-63 are reserved/user-defined and undecodable. The per-AMODE channel
counts in ETSI TS 102 114 §5.3.1 cover all 16, confirming they are
decodable — codes 10-15 are the 6/7/8-channel layouts. A frame is dropped
only when `audio_mode >= DTS_AMODE_COUNT` (i.e. a truly reserved 16-63
code); dropping a legal 10-15 multichannel core would silence recoverable
audio.
## `DTS_CORE_SR_VALID`
Sample rate (Hz) per core `SFREQ` code (ETSI TS 102 114 Table 6-4); a `0`
entry marks a reserved code that fails header validation as an invalid
sample rate. Valid entries are locked to the spec by
`dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
{0, 4, 5, 9, 10}.
## `core_header_drop_reason`
Decodability gate: the core-frame header validity checks from ETSI TS 102
114. Returns `Some(reason)` when the DTS core-frame header is invalid — in
which case the packet is undecodable ("Invalid data found") and dropping it
loses nothing a decoder could have used. Returns `None` (keep) for a
decodable header OR if the header can't be fully read (never false-drop on
our own buffer underrun; the framer only emits AUs whose core is fully
buffered and ≥ 96 bytes).
The 4-byte core sync is already validated by the framer, so this reads the
header fields that follow it. The 16-bit CPF header CRC is not verified (we
skip past it) and the audio-header/side-info CRCs are likewise not checked,
because decoders treat those bytes as optional/ignored — verifying them
would drop frames that decode fine (false positives).
## Test: `stamp_pts_reuses_front_when_no_running_cursor_yet`
`stamp_pts`'s third arm (`else if front != PTS_UNSET`) is only reached once
the first arm's `front != self.last_front_pts` has already failed (i.e.
`front == self.last_front_pts`) and no within-PES running cursor is
available yet (`next_pts_ns == PTS_UNSET`). In that state the AU must still
be stamped with its own (repeated) front timestamp, not silently collapsed
to 0 — a `!=` -> `==` typo on the guard would only take this arm when
`front` IS the unset sentinel, producing 0 instead here and the sentinel
value itself if front really were unset.
## Test: `drain_front_collapses_offset_zero_markers_instead_of_leaking`
Every drained access unit that shares its PES with the previous one pushes
a new `(0, pts)` marker once rebased; `drain_front` must collapse those down
to the single most-recent one each time, or `pts_marks` grows once per
access unit for the life of the track — an unbounded allocation on a
multi-hour disc. `front_pts()` alone can't observe this: it always finds the
last offset-0 entry regardless of how many duplicates precede it, so the
leak is invisible unless something checks that the collapse actually ran.
## Test: `make_exss`
Builds a real DTS-HD EXSS substream of `total` bytes (short header form),
with an optional false DTS core syncword embedded in its payload (decoding
to a plausible core size) — to prove precise sizing, not a payload scan,
bounds the extension.
## Test: `make_dts_ext`
Builds a minimal DTS-HD extension substream of `size` bytes (just the sync
+ zero-padding). The parser delimits extensions by the next CORE sync, not
by the extension's own size header, so a valid header isn't required — only
that the bytes carry no spurious core sync.
## Test: `bogus_tiny_core_sync`
Builds 4 bytes that look like a DTS core sync but whose `fsize` field
decodes to a tiny `core_size` (< MIN_CORE_FRAME_BYTES). With the dead-code
guards this passed validation and could close an access unit at a junk
boundary; with the fix it must be drained and resynced past.
## Test: `next_core_boundary_exact_syncword_length_is_not_need_more`
Only the 4-byte EXSS syncword is buffered after the core frame — no header
fields at all. The guard `buf.len() < pos + SYNCWORD_BYTES` must be strict
`<`: at exactly `pos + SYNCWORD_BYTES` the sync bytes ARE fully present, so
the function proceeds to identify them as an extension sync (then fails to
size the header and falls back, ending in `NextCore::None` here since
nothing after it looks like a core sync either). A `<=` typo would instead
return `NeedMore` at this exact length without ever inspecting what the 4
buffered bytes are.
## Test: `make_bad_dts_core`
Builds a structurally-framed but UNDECODABLE core: a valid `make_dts_core`
whose LFE flag is set to the reserved value 3 (`DTS_LFE_FLAG_INVALID`). It
still sizes and syncs correctly (so the framer delimits it normally), but
the core-frame header validity check rejects it as an invalid LFE flag
(ETSI TS 102 114 §5.3.1: an LFF value of 3 is invalid). LFE is byte10
bits2-1, and does NOT feed the frame duration (NBLKS + SFREQ only), so a
dropped bad core still carries the same `DTS_CORE_DUR_NS` as its good peers.
## Test: `reparse_real_dts_file`
Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
through `DtsParser` and writes the emitted access units back out, so the
garbage-extension → core-only drop can be validated against an actual
damaged stream (e.g. an extracted DTS-HD MA track) end-to-end with an
external DTS decoder. Env: `DTS_IN` (input), `DTS_OUT` (output).
cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
## Test: `find_sync_matches_a_buffer_that_is_exactly_the_syncword`
`find_sync` scans `0..=len-4`, so a buffer that is EXACTLY the syncword
still matches. The existing tests cover 0, 3 and "longer than 4", which
leaves the `len == 4` boundary open — and that is the case a syncword split
across PES packets lands on the moment its last byte arrives.
## Test: `a_core_header_of_exactly_the_minimum_length_is_decoded_not_deferred`
`CORE_HEADER_MIN_BYTES` is "enough bytes to DECODE the size field", so a
buffer holding exactly that many must be decoded, not deferred. The
distinction is visible precisely at the boundary: with 10 bytes of a core
sync whose decoded size is sub-spec, the parser must recognise the false
sync, drain past it and resync down to the 3-byte carry-over tail. Waiting
instead leaves the false sync sitting at the front of the buffer, where it
blocks every later real core behind it.
## Test: `flush_emits_a_core_that_exactly_fills_the_buffer`
The end-of-stream flush must emit a final access unit whose core is exactly
as long as the buffer — the ordinary case, since the last AU is closed by
end-of-stream rather than by a following core sync. Rejecting it at the
boundary silently drops the last frame of every DTS track.
## Test: `flush_discards_a_long_buffer_that_does_not_begin_with_a_core_sync`
The flush guard is a DISJUNCTION: a buffer that does not BEGIN with a core
sync is discarded whatever its length. Requiring both conditions instead
lets a long run of junk through to `dts_core_frame_size`, which happily
decodes a 14-bit size out of arbitrary bytes — and the flush then emits an
"access unit" that is not DTS at all.
The junk here is sized so that mis-decoding it yields a plausible core size
that the buffer fully covers, which is exactly when the wrong answer looks
like a right one.
## Test: `exss_frame_size_needs_the_worst_case_header_and_no_more`
`EXSS_HEADER_MIN_BYTES` is the WORST-CASE header length — the long form's
43 bits after the syncword, rounded up to 6 bytes, plus the 4-byte sync. It
gates whether an extension substream can be sized precisely, and that is
what keeps a chance core syncword inside XLL payload from being mistaken
for the next AU boundary; too large and every extension falls back to the
payload scan, too small and the reader runs off a truncated header.
Pinned at both sides of the boundary rather than by value, so the field
widths it is summed from stay honest.
## Test: `an_access_unit_carries_the_source_of_the_packet_its_core_arrived_in`
The whole point of the shared buffer: an access unit whose core arrived in
an EARLIER packet keeps that packet's source offset, not the offset of
whichever packet completed it. At a clip boundary the two belong to
different clips, and taking the later one puts the unit in the wrong clip —
which is what left nine audio and subtitle tracks unplaceable.
</content>
</invoke>
+149
View File
@@ -0,0 +1,149 @@
# DVD AC-3 physical sub-stream probing
## Why this exists (a real-disc wrong-substream bug)
A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
scan assigns each declared stream a `private_stream_1` sub-stream id purely
by per-codec ordinal — the first AC-3 stream becomes `0x80`, the second
`0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
physical sub-stream order on the wire matches the IFO declaration order.
On some discs it does NOT. A real R2 PAL DVD release's feature declares
ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but the
physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
`0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
ordinal `0x80` slot. Routing the declared 5.1 stream to `0x80` by ordinal
therefore muxes the 2.0 down-mix while labelling it 5.1 — the wrong
physical track.
The robust fix is data-driven and codec/disc agnostic: read each physical
AC-3 sub-stream's REAL channel count from the VOB (the `acmod`/`lfeon` of
its first frame after the `0x0B77` sync) and route each IFO-declared AC-3
stream to the physical sub-stream whose actual channel count matches the
IFO's declared count — instead of trusting the ordinal. This never
re-reads the disc beyond a bounded head-of-feature probe and degrades to
the original ordinal mapping when the probe yields nothing
(unreadable/short VOB).
## `PROBE_SECTORS` sizing
How many 2048-byte sectors of the first feature extent to probe. The head
of a DVD feature opens with logos/warnings whose audio is frequently a
thin 2.0 bed on the FIRST sub-stream only — the other physical `0x8x`
sub-streams and the main 5.1 mix do not appear until a sector or two
further in. 512 sectors (1 MiB) was too short: on one real disc it saw
ONLY `0x80`, and only its opening 2.0 frames. 1024 sectors (2 MiB)
reliably contains at least one frame of every physical AC-3 sub-stream
AND enough of `0x80` to reach its 5.1 frames. Still bounded so a live
drive is never hammered (see the project "don't hammer the live drive"
rule).
## Why the maximum channel count, not the first frame
The first frame of a sub-stream at the head of a feature is NOT
representative. A DVD opens with logos/warnings, and the main `0x80`
sub-stream there frequently carries a thin 2.0 bed before transitioning
to its real 5.1 main mix a fraction of a second later (observed on a real
disc: `0x80`'s first frames are acmod=2 → 2 channels, then it becomes
acmod=7+lfe → 6 channels within the same 2 MiB window). Recording only
the FIRST frame read `0x80=2` and missed the 5.1 entirely, defeating the
channel-match routing. The 5.1 capability of a sub-stream is the
*maximum* channel count any of its frames carries, so
`probe_ac3_substream_channels` scans them all and keeps the max.
## `remap_audio_pids` conservatism
Conservative — it only ever REASSIGNS among the physical sub-streams the
probe actually saw, and only when a better (exact-channel) match exists
than the stream's current assignment. A stream whose current sub-stream
already matches is left alone; a stream with no matching physical
sub-stream keeps its ordinal assignment. So a normal disc (physical order
== IFO order) is a no-op.
## Test helper notes (`dvd_audio_probe::tests`)
- `ac3_frame`: builds a single, correctly-SIZED AC-3 frame whose
`acmod`/`lfeon` encode a known channel count. `byte4` is
`fscod=0 | frmsizecod=0`, so `ac3_frame_size` reports 128 bytes and the
frame is zero-padded to exactly that — this lets `max_substream_channels`
advance frame-by-frame over a multi-frame payload exactly as it does on
real VOB data. The BSI bits are laid down with a writer so the test
never hand-miscomputes the lfeon offset, matching `acmod_channels`'
reader.
- `ps_ac3_frames`: builds a minimal `private_stream_1` PES carrying
`frames` for `sub_id`, each preceded only by the 4-byte AC-3 sub-header
at the PES head. Mirrors the on-disc layout the PS demux expects: PES
start `0x000001BD`, length, PES header (no PTS), sub-header
`[sub_id, frame_count, ptr_hi, ptr_lo]`, then the concatenated AC-3
frames.
### `probe_reads_max_channels_no_cross_contamination`
Real-disc regression — the probe must read each sub-stream's TRUE
(max-mix) channel count, not be poisoned by an unrepresentative head
frame, and must NOT cross-contaminate between sub-streams. Mirrors the
real on-disc layout that caused the mis-read: the feature head carries
`0x80` opening with a 2.0 frame and THEN a 5.1 frame (its real main mix),
interleaved with `0x81` carrying only 2.0. The old first-frame probe read
`0x80=2` (the logo bed) and missed the 5.1; the max-over-frames probe
must report `0x80=6` and `0x81=2`.
### `max_substream_channels_locates_sync_after_leading_non_sync_bytes`
Regression guard for a hand-checked mutation (`+` → `-` at the
`pos + rel` offset computation): with `pos` starting at 0 and the first
sync found 3 bytes in, `pos - rel` would underflow a `usize` and panic,
or (if it somehow didn't) index the wrong start entirely. `pos + rel` is
the only computation that is always in-bounds, since `rel` is itself
bounded by the length of the slice searched from `pos`.
### Unmappable-size fallback tests
When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
`fscod == 3`), `max_substream_channels` must fall back to stepping
`start + 2` bytes past the sync to re-lock onto the next genuine sync,
and must keep making forward progress doing so (never revisit the same
sync, which would loop forever, and never jump so far that it skips the
very next real frame).
- `max_substream_channels_unmappable_size_steps_forward_by_two` lays a
bogus-sized header at absolute offset 4 (so `start == 4`,
`start + 2 == 6`) immediately followed, at offset 6, by a real, fully
decodable 2.0 frame — the position the `+ 2` fallback must land on
exactly.
- `max_substream_channels_unmappable_size_at_start_steps_forward_not_back`
puts the unmappable-size sync at absolute offset 0 (`start == 0`) so
that stepping backward instead of forward (`start - 2`) would underflow
rather than merely land on the wrong byte. Also proves the real frame is
still found 6 bytes further in, confirming forward progress past the
bogus header.
### `remap_reads_current_substream_via_and_not_or_or_xor`
`remap_audio_pids` must read a stream's CURRENT physical sub-stream id
from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
`0x00FF`, both of which force the low byte to `0xFF` regardless of the
real PID and so always miss the "already matches" shortcut. That matters
observably when TWO physical sub-streams share the same probed channel
count: with a correct read, a stream already sitting on a matching
sub-stream is left alone (conservative, per the module's documented
behaviour); with the low byte forced to `0xFF`, `probed.get(&0xFF)` is
always `None`, so the code falls through to the "find any unclaimed
match" path and picks the FIRST (lowest-keyed, BTreeMap-ordered) matching
physical sub-stream instead — which here is a *different* sub-stream
(0x80) than the one the PID already correctly names (0x81), producing a
spurious PID change.
### `probe_and_remap_reroutes_swapped_substream_scenario_end_to_end`
End-to-end `probe_and_remap`: a real-disc-shaped MpegPs title (one
declared 5.1 AC-3 stream ordinally assigned 0x80) whose physical VOB
bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on 0x81. This must
reach the `remap_audio_pids` call and re-route the stream to 0xBD81. It
also, by construction, proves each of the guards along the way lets a
real, positive case through: the content-format check must NOT bail on
`MpegPs` (only on non-`MpegPs`), the AC-3 presence check must NOT bail
when AC-3 IS present, and the sector-count check must NOT bail when the
count is nonzero — any one of those inverted would skip the probe
entirely and leave the PID at its untouched ordinal value (0xBD80), which
the test's assertion would catch.
+177
View File
@@ -0,0 +1,177 @@
# DVD Title Scan (src/disc/dvd.rs) — Internal Notes
Rationale, regression history, and test-fixture layout notes that were
originally inline comments/doc-comments on private items in
`src/disc/dvd.rs`, relocated here to satisfy the comment-guard's internal-
comment cap. Each pointed-to site in the source carries a short `// See
docs/dvd.md — <topic>` line.
## `Disc::scan_dvd_titles` halt handling
Cancellation: `halt` is polled before the IFO tree is read, and an IFO read
that fails with `Error::Halted` — how a live drive reports a Stop, since
`Drive::checked_exec` fails every command once its flag is set — is
propagated rather than swallowed. Every other IFO failure keeps its
best-effort `Ok(vec![])`.
It has to be an error and not an empty title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles. This one was the worst of the three enumerators — a bare
`Err(_) => return Vec::new()` turned an operator Stop into ZERO titles at
rc=0, a disc reported as carrying no video at all.
`scan_dvd_titles` returns the scanned titles plus, when the disc's own
First-Play navigation deterministically dispatches to a title, that title's
`playlist_id` — the nav-resolved main feature. The caller feeds it into
`sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the BD
path uses. It is `None` (existing ranking heuristics stand) whenever the
navigation is menu-only, malformed, or does not converge.
## DefaultDuration / cadence TODO
`measured_cicp` TODO(spec): populate `top_field_first` + measured colour
from `Mpeg2Parser` once tracks are built after frame parsing — needs a
`CodecParser`->title channel like `codec_private`. Until wired, `None`
falls back to TFF / PAL-NTSC guess.
DefaultDuration TODO(spec): comes from IFO `frame_rate`, so a
soft-telecined 23.976-in-29.97 DVD reports 29.97 instead of true film
rate. Cadence is detectable via `nb_fields` (2:3 pulldown); needs the same
channel as `top_field_first` above.
## Test fixture builders: `build_vmg` / `build_vts`
`build_vmg` builds a VIDEO_TS.IFO: magic "DVDVIDEO-VMG"@0, TT_SRPT sector
ptr@0xC4. TT_SRPT lives at `tt_srpt_sector*2048`: `num_titles(u16)`@0, then
12-byte entries from +8. Each entry: `num_chapters(u16)`@+2, `vts_number`@+6,
`vts_title_num`@+7.
`build_vts` builds a VTS_XX_0.IFO. Layout per ifo.rs:
- magic "DVDVIDEO-VTS"@0
- `vtstt_vobs` (Title VOBS start sector, u32 BE)@0xC4 — the production
`vob_start_sector` the cell sectors are relative to. (0xC0 is
`vtsm_vobs`, the menu VOBS, which the scan must NOT use.)
- VTS_PGCIT sector ptr(u32 BE)@0xCC
- video attr byte@0x200
- `num_audio(u16 BE)`@0x202, audio blocks (8B) @0x204
- `num_subs(u16 BE)`@0x254, subtitle blocks (6B) @0x256
PGCIT (at `pgcit_sector*2048`): `num_pgcs(u16)`@0, PGC info entries (8B)
from +8 with PGC byte offset(u32 BE)@+4. PGC: `nr_programs`@0x02,
`nr_cells`@0x03, BCD time@0x04, `pgm_map` ptr@0xE6, `cell_playback` ptr@0xE8
(both u16 BE rel to PGC start).
## `HaltingReader` test double
A `SectorSource` that fails every read at or above `halt_at` with
`Error::Halted` — how a LIVE DRIVE behaves once the operator presses Stop:
`Drive::checked_exec` fails every SCSI command with `Halted` from then on,
and `Drive::read` deliberately preserves the variant. Reads below the
threshold still succeed, so the scan gets far enough to have something to
truncate.
## Test: `halted_ifo_read_is_not_reported_as_a_shorter_disc`
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The DVD enumerator must not swallow that into a
successful scan.
This was the worst of the three enumerators. RED BEFORE GREEN, two
distinct swallows, both measured with the fix reverted:
- `ifo::parse_vmg` treats a failed title set as a placeholder entry and
continues, so a cancel landing on VTS_02's IFO returned
`Ok([VTS_01_1.VOB])` — one title from a two-title disc.
- `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel landing
on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc reported as
holding no video at all.
Both are indistinguishable from a real disc, and both are now
`Err(Error::Halted)`.
Fixture: two title sets, VTS_01's IFO at `PART_START+6000`, VTS_02's at
`PART_START+7000`. Both ICBs sit far below, so filesystem metadata
resolves fine and only the second title set's CONTENT read is cancelled —
the truncation case.
## Test: `scan_dvd_titles_uses_title_vobs_not_menu_vobs`
Regression (first-play menu prepended to the feature): `vob_start` must
come from the **Title** VOBS pointer `vtstt_vobs` (VTS_IFO 0xC4), NOT the
**menu** VOBS pointer `vtsm_vobs` (0xC0). On discs with a per-title menu —
e.g. the Universal "the parental level has been set, press yes" first-play
still — `vtsm_vobs` points at that menu VOB, which sits just before the
title VOB. Cell `first_sector` values are relative to `vtstt_vobs`; reading
0xC0 prepended the menu and shifted every extent back by
`vtstt_vobs - vtsm_vobs`, so the rip opened on the parental prompt instead
of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); the test additionally
stamps a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve
from 3640.
## Test: `scan_dvd_titles_extent_is_absolute_three_term_sum`
ABSOLUTE-REBASE regression (real-disc menu-at-start fix): `ifo::parse_vts`
now sets `vob_start_sector = file_start_lba(IFO) + vtstt_vobs`, so an
extent's `start_lba` must equal the sum of THREE independent terms — the
IFO file's absolute on-disc LBA, the IFO-relative `vtstt_vobs` (0xC4), and
the cell's `first_sector` — none of which may be dropped. The earlier code
used the bare relative `vtstt_vobs`, placing every extent `ifo_lba`
sectors too early (the rip opened in the VMGI/menu region before drifting
into the movie). The other tests fold two of the three terms together
(zero cell offset, or a single combined expectation); this one keeps all
three distinct and non-overlapping so a regression to ANY two-term
combination is caught.
## Test: `scan_dvd_titles_lpcm_routes_to_a0_pid_range`
LPCM SCAN ROUTING (audit §2 / §5 #6): the 0xA0..=0xA7 PID range was never
exercised in the dvd.rs scan. An LPCM stream (coding_mode 4) at audio
position 1 must get `sub_stream_id 0xA1` → PID `0xBDA1` via
`dvd_audio_pid`, distinct from the AC-3 `0xBD80` space, with its real
channel count preserved.
## Test: `scan_dvd_titles_multiple_vobsub_tracks_distinct_pids`
MULTI-VOBSUB SCAN (audit §2 / §5 #6): the single-subtitle test covered one
track; a multi-subtitle VTS must emit one `Stream::Subtitle` per entry
with distinct PIDs (`0x20 + ordinal`) and per-language tags, all sharing
the PGC palette `codec_data`.
## `stamp_first_play_jumptt` test helper
Stamp a First-Play PGC into a VMG whose pre-command list is a single
unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
TT_SRPT pointers `build_vmg` already writes.
## Test: `scan_dvd_titles_nav_promotes_first_play_target`
The DVD First-Play nav promotion branch (issue #40): when
`resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
the matching `vts_number` — to the running `title_number`, and surface it
as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
join or a `vtsn`/`vts_ttn` field swap would promote the wrong title or
none.
## Test: `scan_dvd_titles_drops_leading_scene_index_cell`
End-to-end bug-4 fix: a feature PGC that opens with a leading interleaved-
angle sub-block cell (category 0x90 = in-block cell of an angle block)
must have that cell DROPPED from the muxed extents, so the rip starts at
the real feature. Chapters shift earlier by the dropped duration.
## Test: `scan_dvd_titles_mp2_audio_pid_fallback_is_additive`
Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio stream
has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio, per
`ifo::assign_audio_sub_stream_ids` — the PID falls back to `0xBD00 + i`
where `i` is the stream's positional index in the IFO audio-attribute
table. Two MPEG-audio (coding_mode 2) streams must land on two DISTINCT,
correctly-offset PIDs: 0xBD00 and 0xBD01. This pins the `+` (not `-`/`*`)
so the second stream doesn't collide with, or wrap under, the first.
+23
View File
@@ -0,0 +1,23 @@
# src/dvdnav/mod.rs
## Module rationale
Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
PGC/cell tables, PCI/HLI button packets); the VM command decoder is
verified against real discs.
Current contents: `vmcmd` — the VM command decoder; `nav` — the
First-Play navigation executor that resolves which title the disc's own
navigation selects as the feature.
## `resolve_feature_start` — parked status (issue #40)
PARKED (issue #40, `USE_NAV_RESOLVER=false`): this is intentionally a stub,
not an accidental dead one. It unconditionally returns `None` and ignores
`reader`/`udf` on purpose — the IFO/PCI parsing + nav executor (built on
`vmcmd`) land incrementally behind that flag. Until wired in, calling this
is behaviour-neutral (the caller always falls back); improvements to the
resolver take effect here without touching the call site. Do not delete.
`reader`/`udf` are the seam inputs the nav executor will consume to read
VIDEO_TS.IFO + the VTS IFOs/menu VOBs. Reserved until that lands.
+28
View File
@@ -0,0 +1,28 @@
# dvdsub — VobSub SPU boundary rule
The presence of a PTS — not merely an open `pending` — is the authoritative
SPU-boundary signal, so a lost continuation or a corrupt SPU_size can't merge
the next subtitle into the stuck unit.
## ycbcr_to_rgb range convention (deliberate)
This uses the **full-range (JFIF) BT.601** coefficients with no 16/235 luma
scaling. DVD IFO palette YCbCr is nominally studio-swing BT.601, so
studio-swing math would be more colorimetrically "correct" in isolation. But
the output here is a VobSub `.idx` `palette:` line, and the entire VobSub
ecosystem (the original tooling, mkvtoolnix, players that read the `.idx`
palette) is built around this full-range formula — it is the de-facto
on-disk convention. Emitting studio-swing-scaled RGB here would make
freemkv's palettes inconsistent with every other tool and wrong in players
that assume the VobSub convention. Do NOT "fix" this to studio-swing without
changing the consuming side in lockstep.
## ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb test fixture
A saturated RED entry appears on disc as Y=76, Cr=255, Cb=85 (full-range
BT.601 encoding of RGB #FF0000), i.e. bytes `[0x00, 76, 255, 85]`. Reading
byte 2 as Cb and byte 3 as Cr instead turns this entry BLUE, which is the
exact user-visible symptom. The fixture is deliberately NOT built from this
crate's own doc comments: those described the order wrongly for a long time,
and the previous version of this test inherited the error from them and
therefore could not detect it.
+62
View File
@@ -0,0 +1,62 @@
# src/mux/ebml.rs
## `fixed_width_vint8`
Encodes `data_size` as the FIXED-WIDTH 8-octet EBML VINT used to back-patch
a master element's size field: the `0x01` VINT_MARKER octet followed by the
56-bit VINT_DATA payload, big-endian (RFC 8794 section 4.4 — an 8-octet
VINT carries 7 octets of VINT_DATA).
Extracted so the full 56-bit payload can be exercised directly: reaching
the high payload bytes through `end_master` / `end_master_buf` would take
a multi-terabyte buffer, leaving them unconstrained by any test.
`data_size` must be below 2^56; both callers check that first.
## `start_master_buf` / `end_master_buf`
This is the seek-free twin of `start_master`/`end_master` for elements
small enough to assemble whole before hitting the file (a BlockGroup: one
Block plus a couple of tiny elements). Because `end_master` always
back-patches a FIXED-WIDTH 8-byte VINT (`0x01` + 7 payload bytes) rather
than a minimal-width one, the bytes produced by this pair are byte-for-byte
identical to the seek-and-back-patch pair — assembling in memory cannot
change the emitted Matroska.
## `read_exact_bounded`
Reads exactly `len` bytes WITHOUT trusting `len` to size the allocation.
`vec![0u8; len]` on an attacker-controlled EBML size would allocate
gigabytes before the read fails. Instead the reader is capped to `len`
and the buffer grows as bytes actually arrive: a malformed element that
claims a huge length but supplies few bytes allocates only what it
delivers, then errors on the short read.
## Test: `buffered_master_matches_seeking_master_byte_for_byte`
The buffer-based master helpers must produce byte-for-byte what the
seek-based ones produce. `write_block_group` was rewritten onto
`start_master_buf`/`end_master_buf` to eliminate ~4 seeks (and 4 MiB
BufWriter flushes) PER FRAME, which on an MPEG-2 title is ~350k frames — a
change only safe because the two encodings are identical.
Both write a FIXED-width 8-byte VINT placeholder (0x01 + 7 payload bytes)
and patch it in place; neither ever emits a minimal-width size. This test
pins that, so a future "optimisation" of either one to minimal-width sizes
cannot silently desync the two and change emitted Matroska.
## Test: `buffered_master_nests_correctly`
Nested masters must patch correctly too — the MVC BlockGroup nests
BlockAdditions > BlockMore inside the BlockGroup, and in-memory patching
works by index rather than by file offset, so nesting is where an
index-arithmetic slip would show up.
## Test: `fixed_width_vint8_is_big_endian_over_the_full_payload`
The fixed-width 8-octet VINT is `0x01` followed by the 56-bit VINT_DATA
payload in BIG-ENDIAN order (RFC 8794 section 4.4). Every payload octet is
distinct here, so a shifted, reversed or dropped octet is visible; the top
payload octets are unreachable through `end_master` without a
multi-terabyte buffer, which is why this is tested at the encoder.
+51
View File
@@ -0,0 +1,51 @@
# `dirimage::encode` — descriptor encoder notes
Overflow rationale for comments in `src/dirimage/encode.rs` that would
otherwise exceed the comment-guard caps. Each section below is pointed to
by a short `//` comment at the relevant spot in the code.
## Why a pure function
`encode` is a pure function from `Layout` to sectors — nothing here touches
the filesystem. That is what makes it testable against the production
parser in `udf.rs`: the same bytes this module writes are read back and
checked by the real UDF reader, so the encoder and the parser can't drift
apart silently.
## Why UDF 1.02 with a single Type-1 partition map
UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
the DVD-Video profile, it is the shape `read_filesystem` takes when
`num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
entirely. That also means a synthetic image never exercises the Metadata
Partition path in `udf.rs` (`:946-991`).
## `finish_tag`: why `tag_loc` correctness matters
Getting `tag_loc` wrong (absolute vs. partition-relative) is the classic
reason a hand-built volume mounts nowhere: a driver that validates the tag
location rejects the descriptor outright. `desc_len` is the descriptor's
total length including the tag; the CRC covers `buf[16..desc_len]`.
## `encode_cs0`: why ASCII, not Latin-1, for the 8-bit form
`parse_udf_name` (`udf.rs:1467`) decodes a compression-8 name with
`from_utf8_lossy`, so a 0x80-0xFF byte — legal CS0 under Latin-1 — would
come back as U+FFFD. Every character above 0x7F therefore takes the 16-bit
(UTF-16BE) form, which that parser decodes correctly.
## `file_entry`: why tag 261, not the Extended File Entry (266)
Real BD-ROMs use the Extended File Entry (tag 266), but an EFE requires
UDF 2.00+, and this image declares 1.02, so `file_entry` writes the plain
File Entry (tag 261) instead. `udf.rs` reads both — the 261 field offsets
it uses (`l_ea` 168, `l_ad` 172, ADs at `176 + l_ea`) are the ones written
here.
## `push_fid`: why FIDs may span logical blocks
FIDs are packed with no inter-descriptor padding beyond the 4-byte
alignment the spec mandates, and they are allowed to span logical blocks —
which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
directory extent as one flat byte run and stops at the first non-257 tag,
so any block-alignment gap would truncate the directory.
+151
View File
@@ -0,0 +1,151 @@
# encrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/encrypt.md — <topic>` pointer
left at the original site.
## HandshakeResult::drive_unlocked
True when the VID came from an unlocker (in `freemkv-unlock`) that
unlocked the drive. Such a drive serves CLEAR content, so AACS bus
encryption is already removed AT THE DRIVE — the same end state a
successful cert handshake's `read_data_key` provides, just via
firmware instead of the AKE. The bus-key gate MUST credit this as a
valid bus-removal: bus encryption is unremovable only when NEITHER the
firmware unlocked the drive NOR the cert handshake yielded a bus key.
Without this, a SUCCESSFUL unlock (VID present, `read_data_key: None`)
paradoxically trips the gate and blocks ALL key resolution (incl. the
online source).
## bus_encryption_removed
Single source of truth for "is AACS bus encryption gone for this
scan?". The gate asks ONLY this — `if !removed { error }` — never
enumerating cases. Bus encryption is gone when ANY of these holds:
the disc never had it; file/ISO reads (content already clear at read
time); an unlocker unlocked the drive (it serves clear content); or
the cert handshake produced the bus key. Add a NEW removal mechanism
HERE, never in the gate.
## AacsCertUnlocker
libfreemkv-side driver for the AACS cert route. It owns the host-cert
collection (a keysource concern that stays in libfreemkv) and then
dispatches the actual mutual-auth to the `freemkv-unlock` AACS
unlocker via the `crate::unlock_bridge`. The firmware (drive-prep) and
CSS routes dispatch the same way at their own call sites; this one
carries the host certs.
## AacsCertUnlocker::authenticate
Runs the host-certificate mutual-auth handshake: collect
non-compiled-in host certs from the key sources + credentials, then
hand them to the AACS unlocker (via the `freemkv-unlock` dispatch),
which tries each cert (wedge-guarded) and on success yields the
Volume ID + `read_data_key` (the AACS 2.0 bus key). Returns a
`CertUnlockFailure` on every no-VID outcome.
## unlock_error_to_error
Maps a `CertUnlockFailure` back to the `Error` variant
`do_handshake_cert` has always surfaced, so `scan_with`'s rendering
and the path-1 disc-hash → VUK fallback are byte-for-byte unchanged.
(`NoHostCert` keeps the `<no host cert>` sentinel.)
## handshake_has_volume_id
Did the cert handshake actually carry a Volume ID? Extracted so it can
be tested as a VALUE. It only ever reaches an operator as the
`has_volume_id` field of the `bus_key_unavailable` warn, and asserting
on a `tracing` field means installing a capturing subscriber — which
is thread-local, while `tracing`'s callsite-interest cache is global.
Those two facts race: the test failed roughly one run in ten under the
full parallel suite while passing every time in isolation, and
serialising the captures was not enough because the cache can be
re-evaluated against the process default rather than the thread-local
dispatch.
A predicate this small does not need a subscriber to verify. The
polarity is the whole point: an `==` here would tell an operator a VID
was absent on exactly the discs where one was present.
## Disc::do_handshake
SCSI handshake — drives the VID-acquisition flow and returns a
structured `HandshakeResult` for downstream key resolution.
VID acquisition runs through `do_handshake_cert`, which first uses the
OEM VID a firmware unlocker may have stashed at drive `init()` (a
drive-functionality capability decoupled from the host cert + HRL) and
falls back to the cert-based mutual-auth handshake (dispatched to the
`freemkv-unlock` AACS unlocker) when none is present. The cert path
also yields `read_data_key`, required for AACS 2.0 bus decryption.
Returns `(handshake, error)`:
- `(Some(_), None)` — VID acquired
- `(None, Some(_))` — specific failure mode (`AacsHostCertRejected` or
`AacsVidUnavailable`)
- `(None, None)` — handshake not attempted (no keydb; resolution will
proceed with VID=zero and rely on path 1 disc-hash → VUK lookup)
## Disc::do_handshake_cert
Cert-based AACS handshake — the cert route for VID acquisition.
Before running the cert mutual-auth, this checks for an OEM Volume ID
a firmware unlocker stashed at drive `init()`. Such an unlocker
unlocks *drive functionality*, not just the disc: VID retrieval via
the drive's OEM CDB is a capability separate from `unlock`. When one
served a VID, we use it and SKIP the cert handshake entirely — the
OEM path gets the VID *without* the host certificate + HRL, decoupling
VID from the cert chain. The OEM path yields no `read_data_key` (no
bus-key is derived); AACS 2.0 content needing read_data_key for bus
decryption must still use the cert path, so an unlocker with no OEM
VID capability returns `None` and we fall through to cert auth
unchanged.
## Disc::collect_host_certs
Collect every AACS host cert the caller carries, from BOTH the
explicit `DriveCredentials` and the key-source layer
(`crate::KeySource::host_certs` across each source), unioned. Host
certs are keysource-served, never compiled in; this is the one place
the OEM cert route gathers them. An empty result is the graceful
no-cert signal (the caller turns it into `Error::AacsNoHostCert`).
`mkb` is the disc's MKB generation when known, forwarded to each
source's `crate::KeySource::host_certs` so a source MAY return only
generation-appropriate certs (the default ignores it).
## Disc::resolve_vid_only
Build a keys-free AACS state that carries only the Volume ID (+
version metadata), for callers that resolve Unit Keys out-of-band and
have disabled the local keydb. The VID is on-disc content read during
the handshake; preserving it here lets the out-of-band path use it. No
keys are present (`unit_keys` empty, `vuk` None), so the disc reports
as "encrypted, no keys" until the caller re-scans with a resolved Unit
Key.
## build_mkb (test helper)
An MKB with one Type-and-Version record (type 0x10) carrying the
version as BE u32 at record offset 8, followed by a recorded EOF
record then trailing zero padding. mkb_content_len walks records and
stops at the first padding (type 0) byte (aacs/inf.rs).
## resolve_vid_only_no_cert_defaults_version_uhd (test)
No content cert at all → version defaults to UHD (major 2), matching
`read_aacs_version` so the scanned `AacsState.version` and the
out-of-band fetch agree on the Unit_Key_RO stride (audit #4: a wrong
BD-vs-UHD guess mis-parses unit keys). bus_encryption false
(unreadable → off).
## Skipped/deferred test coverage
Tests for `read_vid_oem` (response parsing) are skipped in this file
since `Drive` construction requires a live transport; parsing branches
are exercised through `read_vid_oem`'s callers in integration. The
`collect_host_certs` tests union `DriveCredentials` with the
key-source layer; empty means the route fails gracefully
(`AacsNoHostCert`).
+181
View File
@@ -0,0 +1,181 @@
# Error Codes
Overflow detail for `src/error.rs` doc comments that would otherwise exceed
the comment-guard caps. Each section below is pointed to by a short `//` or
`///` comment at the corresponding definition.
## Error Code Ranges
| Range | Category |
|-------|----------|
| E1xxx | Device errors |
| E2xxx | Profile errors |
| E3xxx | Unlock errors |
| E4xxx | SCSI errors |
| E5xxx | I/O errors |
| E6xxx | Disc format errors |
| E7xxx | AACS errors |
| E8xxx | Keydb errors |
| E9xxx | Stream/mux errors |
## E_UDF_NO_USABLE_EXTENT
A file that EXISTS and whose allocation descriptors resolved without error,
yet yields not one usable extent: an empty AD list, or a list every entry of
which is zero-length or points at LBA 0. Reported by the HD-DVD clip resolver
(`disc::hddvd`).
It has no `Error` variant on purpose. `UdfFs::file_extents` returns
`Ok(vec![])` here rather than failing — the emptiness is only a defect in the
eye of a caller that needs bytes — so the condition is DETECTED by the
caller, not returned to it. The code exists so that detection can be
accounted in the log with something other than a neighbouring error's code:
logging it as E6017 would file a zero-length AD list as an authoring hole
and send whoever triages it at the wrong population.
## E_CSS_NO_DISC_KEY / CssNoDiscKey history
While the disc-wide CSS failure and the per-title CSS failure both used
`E_CSS_KEY_MISSING`, an uncrackable CSS disc was iterated title by title,
each one logged as skipped, and the run exited reporting success — a total
failure reported as success. `E_CSS_NO_DISC_KEY` / `Error::CssNoDiscKey` was
split out so `is_disc_level_no_key` classifies it and a multi-title rip loop
fails fast on it instead.
## E_KEY_SERVICE_UNAVAILABLE
A seven-hour run of HTTP 502s reported as `E_NO_DISC_KEY` told operators
their disc was not in the key database and sent them hunting for a VUK that
was never missing; the correct action was to wait. `E_KEY_SERVICE_UNAVAILABLE`
exists so a key-source outage (transport error, DNS failure, timeout, TLS
failure, HTTP 5xx, unreadable reply) is distinguishable from a source that
answered and simply has no key.
## E_MKV_SOURCE_INVALID vs E_MKV_INVALID
`is_skippable_title_stub` classifies `E_MKV_INVALID` as a title that yielded
no muxable frames, which an all-titles rip may skip while finishing the
rest. A corrupt or truncated `mkv://` source file is a FAILURE, not a stub —
reporting it as skippable would let a broken source be silently passed over
by a run that then exits successfully. `E_MKV_SOURCE_INVALID` (and the
sibling codes `E_MUX_HEADER_BUFFER_EXCEEDED`, `E_MKV_LACING_INVALID`) exist
to keep that distinction, and are fatal.
## UdfEmbeddedData
A file that legitimately stores its data this way is tiny (an ICB caps it at
well under 2 KiB — the AACS `*.inf` key files are the usual case), and the
callers that expect one read it via `read_inline_data` long before extents
are ever requested. A stream file cannot be one. `read_directory` already
refuses the same shape for directories; `UdfEmbeddedData` is the file half
of that decision.
## MkvInvalid / MuxHeaderBufferExceeded history
`MuxHeaderBufferExceeded` exists because hundreds of megabytes of real
frames with unresolvable codec-init headers is NOT the same thing as
`MkvInvalid`'s canonical case (an empty nav/menu PGC stub). Reporting the
former as the latter would have `is_skippable_title_stub` silently drop a
main feature from a rip that then exits successfully.
## AacsBusKeyUnavailable
NOT raised for AACS 1.0 BD (no bus encryption, `read_data_key` legitimately
absent) nor for file-backed (ISO) scans, where bus encryption was already
removed at read time and no handshake runs. Only a live-drive AACS 2.0/UHD
scan that obtained the Volume ID but produced no bus key raises this.
## SourceTerminated
The alternative answer is a lie: a dead source that reports `Ok(0)` is
indistinguishable from end-of-stream, and `DiscStream::fill_extents`
legitimately reads a short count as a skippable hole — zero-filling and
advancing over every remaining sector of the title and still returning
success. Unlike a bad sector, this condition cannot be retried at a smaller
size or skipped past.
## error_code / From<Error> for io::Error round-trip
`mux_stream` hands back an `io::Error`, and a front-end reporting *why* a
title failed had no way to recover the code from it — the typed `Error` is
gone by then and only the `E<code>` string prefix survives. Parsing that
prefix is `error_code`'s job; every consumer re-implementing the parse is how
the string-matching this crate spent 1.5.x removing comes back.
`From<Error> for io::Error` boxes the typed value as the payload
(`io::Error::new(kind, e)`), whose `Display` is the same `E<code>[: …]`
string the stringifying version produced — so `error_code`'s parse is
unaffected, and `From<io::Error> for Error` can additionally `downcast` the
payload back to the exact typed error. Errors that did NOT come from this
crate carry no `E<code>` prefix and yield `None`.
## is_skippable_title_stub
This replaces the CLI's `E7023`/`E6008` string-match with a typed check on
the `io::Error` `mux_stream` returns.
`Error::MkvSourceInvalid`, `Error::MkvLacingInvalid`, `Error::MkvUnencodable`
all used to be raised as `Error::MkvInvalid` and therefore landed in the
skippable set, so a corrupt source was reported as a title worth silently
passing over by a run that then exited successfully. They now carry their
own codes and are fatal, as is `Error::MuxHeaderBufferExceeded`.
`Error::CssNoDiscKey` (the disc's CSS crack recovered no key at all) is the
same conflation on the decrypt axis: while it too was raised as
`Error::CssKeyMissing`, every title of an undecryptable disc classified as a
skippable stub, so the rip loop skipped all of them and exited successfully.
It belongs to `is_disc_level_no_key` instead, and is fatal here.
## is_disc_level_no_key
`E_CSS_NO_DISC_KEY` is the CSS side of the disc-level/per-title split, and it
exists because the disc-wide CSS failure used to be raised as the per-title
`E_CSS_KEY_MISSING`: an undecryptable CSS disc landed in
`is_skippable_title_stub`, so the rip loop skipped all N titles with an
"empty stub" notice and exited successfully.
The key-SOURCE failures (`E_KEY_SERVICE_UNAVAILABLE`,
`E_KEY_SERVICE_UNAUTHORIZED`, `E_KEY_SERVICE_RATE_LIMITED`) are classified
here for the same fail-fast reason and NOT because they mean "no key": a
service that is down, refusing the token, or throttling is down for every
title on the disc, so iterating N titles re-issues N doomed requests (and,
on 429, digs the rate-limit hole deeper). They are separate CODES precisely
so the front-end can say "retry later" / "fix the token" instead of
`E_NO_DISC_KEY`'s "no key source has a key for this disc".
## is_marginal_read
- MEDIUM ERROR (sense key 3) — canonical bad-sector signal.
- ABORTED COMMAND (sense key B) — transient; retry usually works.
- NOT READY (sense key 2) — the dominant bad-sector response on the BU40N
(ASC 0x04/ASCQ 0x3E); a pause + retry often recovers.
- RECOVERED ERROR (sense key 1) / NO SENSE (sense key 0) — not classified as
fatal; treat as recoverable.
Caller-agnostic predicate — describes a property of the *error*, not what
one specific call site should do with it. Used by
`freemkv_engine::recovery::copy`'s hysteresis dispatch.
## Test rationale: declared_error_codes / uniqueness
The uniqueness test used to carry a hand-maintained `vec![]` of the error
constants, with a doc comment claiming it "pins all code assignments". It did
not. At the time this was written the file declared 127 `pub const E_*` and
the vector named 109 of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
`E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`, `E_DIR_NAME_COLLISION`,
`E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`, `E_DIR_RAW_REJECTED`,
`E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`, `E_DRIVE_INQUIRY_SHORT`,
`E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`, `E_SEAM_PLAN_DROPPED_MOST`,
`E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`, `E_SOURCE_TERMINATED`,
`E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were outside the guarantee entirely,
so a new variant colliding with any of them passed green. Worse, an earlier
audit READ that doc comment and trusted it while assigning new codes.
The defect is not the eighteen omissions, it is that the list is
hand-maintained at all: adding a constant and forgetting the vector is a
silent no-op, which is the definition of a guarantee that decays. Deriving
the list from the declarations makes forgetting impossible — the only way to
escape the check is to stop declaring the constant. `include_str!` of the
same file is the cheapest seam that does that: no `build.rs`, no proc macro,
no dependency, and — being `#[cfg(test)]` — not one byte of the source
embedded in a release build.
+150
View File
@@ -0,0 +1,150 @@
# `src/disc/extract.rs` — decrypted file-tree extraction
## Relation to the ISO sweep/patch recovery passes
`extract_tree` is a sibling of the disc→ISO sector dump (the sweep/patch
recovery passes, which now live in the `freemkv-engine` crate), specialized
to write **per file** rather than a whole image, applying decryption on the
way out, and **without** any multipass / recovery orchestration — it is
1-shot, decrypt-only.
No new error codes are introduced: it reuses the existing `Error` set
(`error.rs` is numeric-only, no English).
## `extract_tree` contract details
`reader` is consumed for content reads (it is moved into a
`DecryptingSectorSource`); the structure + extent metadata are read first
with the raw reader. `dest` receives the tree STRAIGHT IN (no auto-named
subfolder). The caller must have run the pre-flight decrypt gate
(`Disc::ensure_decryptable`); this method resolves per-VTS CSS keys itself
for DVD.
Bad sectors become recorded zero-filled holes in the affected file (the run
does not abort); undecryptable units are counted as loss. Files are written
`<name>.partial` and renamed on success, so an interrupted run never leaves
a half-written file that looks complete.
## `resolve_vts_key` — why a failed crack must be a hard error
FALLIBLE, because the three outcomes of a crack are not two.
[`crate::css::CrackOutcome`] exists precisely to separate "no scrambled
sector was seen, so there is nothing to decrypt" from "scrambled sectors
were seen and no key came out", and its doc says callers MUST surface the
second as a hard error. An earlier version of this function used the
`Option`-returning `crack_key`, which collapses both into `None`, and then
fell back to the disc-wide key — descrambling this VTS with ANOTHER VTS's
title key. The hazard is described in the ordering note in the function
body, which was written about the same fallback; ordering makes the crack
far more likely to succeed, but it cannot make a failed crack safe.
## `Borrowed` — why a borrowing `SectorSource` wrapper
Lets the decrypting decorator "own" an inner source for its lifetime while
the caller keeps the underlying `&mut dyn SectorSource` (the decorator is a
`DecryptingSectorSource<S>` generic over `S`, so it does NOT require a
`'static` boxed inner — unlike the mux highway, which takes the reader by
value). The decorator is dropped before `extract_tree` returns, so the
borrow never escapes.
## `whole_unit_batch` — the trailing-partial rule
`remaining` is the sectors left in the current extent; the batch is capped
at `READ_BATCH_SECTORS` and rounded DOWN to a whole number of 3-sector units
UNLESS it is the extent's final (possibly short) tail — the tail always
begins on a unit boundary, so a 1-2 sector partial there is handled by
`decrypt_sectors`' trailing-partial contract. Used by `extract_one_file` so
this rounding rule lives in exactly one place.
## `available_space` (windows) — why the free-space gate matters
Windows has no `statvfs`. This once returned `None` unconditionally, which
does not merely skip a test — it skipped the free-space GATE, so a Windows
user extracting a disc to a full volume got a confusing failure part-way
through instead of a clear refusal up front. libfreemkv ships a Windows
GUI, so that is the platform where the friendly error matters most.
`FreeBytesAvailableToCaller` (not `TotalNumberOfFreeBytes`) is used because
it accounts for per-user quotas, matching what `statvfs`'s `f_bavail` gives
on the unix side.
## `sanitize_component` — why reserved names are substituted, not rejected
A name whose base matches a Windows reserved device name (`NUL`, `CON`,
`COM1`..) is **substituted** (prefixed with `_`) rather than rejected — a
Linux-authored disc may legally carry such a file and a single reserved
name must not abort the whole tree walk. An empty result after stripping
is an error.
## `names_differing_only_by_case_are_a_collision` — why this must be an error
Two disc names differing only by CASE are one host file on macOS APFS and
Windows NTFS, both case-insensitive by default. Keyed by the case-preserving
path, the collision map would see two entries, raise nothing, and the
second file extracted would overwrite the first — while both `PlannedFile`s
report `complete: true`. Silent data loss reported as a clean extract is
the one outcome this crate must never produce, so the host-equivalence key
has to model the host's namespace, not the disc's.
## `a_files_partial_path_colliding_with_another_files_final_name_is_an_error`
A disc carrying both `X` and `X.partial` plans two distinct final names, so
nothing collides in the final-name check — but extracting `X` writes
through `X.partial`, the same host path the other file owns. Whichever
lands second truncates the other, and both entries would still be reported
complete without this guard.
## `multi_extent_aacs_anchors_unit_base_per_extent` — the alignment bug
Regression (rc.6 audit, finding #449): a MULTI-EXTENT AACS file must
re-anchor the unit-alignment base PER extent, not once at the first extent.
The two extents in this test start at absolute LBAs whose difference is NOT
a multiple of 3 (`PART_START+5000` vs `PART_START+5004` — a 4-sector delta).
With a single first-extent unit base the second extent's first read is off
the unit grid (offset 4, `4 % 3 == 1`) → `is_unit_aligned` fails →
`DecryptFailed` → the whole extent becomes a zero-filled (false) hole. With
per-extent anchoring both extents read clean: `bytes_unreadable == 0` and
the file content survives. The fixture content is clear (no TS syncs) so
the decrypt step restores each unit verbatim — this isolates the GATE from
the cipher math.
## `extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it`
An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED: the
space belongs to the file and occupies its byte range, but nothing was ever
written there, and the standard defines its contents as zeros.
`read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
`read_file_limited` honours it by emitting zeros WITHOUT touching the
media. The tree extractor reads the same ICBs and must agree: reading those
sectors returns whatever the media happens to hold there — on an AACS disc,
ciphertext that decrypts to noise — and writes it into the extracted file
as if the disc had recorded it.
## `per_extent_base_is_aligned_first_extent_base_is_not`
Focused alignment-computation check underpinning the per-extent fix: when
each extent anchors its OWN start as the unit base, the extent's own batch
starts are always unit-aligned; anchoring a later extent against the FIRST
extent's base mis-aligns whenever the extents' starts differ by a
non-multiple of 3 sectors. This is the exact arithmetic the decrypt-on-read
gate (`aacs::content::is_unit_aligned`) performs.
## `css_two_vts_groups_do_not_cross_contaminate_keys`
Regression: the per-VTS key crack in `resolve_vts_key` must gather ONLY
this VTS's own title-VOB extents. Two VTS groups here carry DISTINCT
scrambling keys; if the group filter (`vts_group_of(..) == Some(vts) &&
is_title_vob(..)`) is loosened (`!=`, or `&&` -> `||`), one VTS's resolve
gathers the OTHER VTS's extents, cracks the wrong key, and that VTS's own
content silently fails to descramble to the expected plaintext.
## `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`
A VTS that IS scrambled but whose key could not be recovered must FAIL, not
borrow another VTS's key. `crack_key` returns `Option`, which collapses "no
scrambled sector was seen" with "scrambled sectors were seen and no key
came out"; the fallback then descrambled this VTS under the disc-wide key
and `extract_tree` reported `complete = true` at exit 0, because no read
had failed. `CrackOutcome` exists to keep those two apart, and every
sibling path in the crate already honours it. The fixture: VTS_01 is
crackable, VTS_02 is scrambled with NO periodic crib, so its crack
genuinely fails.
+36
View File
@@ -0,0 +1,36 @@
# `file_sector_source::linux` — read-side platform hints
## Why both `hint_sequential` and `drop_window`
`POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window so
each pread aggregates into fewer NFS round-trips. `DONTNEED` on the
consumed window (called periodically by the caller) drops the
already-read pages from the page cache so an 85 GB streaming ISO read
doesn't fill memory and starve concurrent writes (the MKV output
during mux). Together they mirror the write-side WritebackPipeline's
policy.
## History
Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
`FileSectorSource` silently dropped the read-side DONTNEED, and
0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
to it) removed the hint. Net effect: 85 GB of ISO reads pinned in the
page cache + no readahead widening → mux throughput collapse from
18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
## `prefetch`
Async-prefetch `len` bytes at `offset` into the page cache. The kernel
`readahead(2)` syscall queues the I/O and returns immediately — it
does NOT wait for completion. Called right after each consumed read
so the next batch's I/O overlaps with the caller's processing of the
current batch (decrypt + demux + mux).
Without this hint, with a synchronous demux consumer running at
~50 MB/s and a single-spindle disk capable of ~150 MB/s, the disk sits
idle ~70% of each iteration because kernel readahead alone (capped at
`/sys/block/<dev>/queue/read_ahead_kb`, default 128 KB) can only
pre-stage a tiny slice of the next batch. An explicit `readahead()` of
the same size as the current batch tells the kernel to queue the full
next-batch read now.
+29
View File
@@ -0,0 +1,29 @@
# file_sector_source::macos
macOS has no direct `POSIX_FADV_SEQUENTIAL` equivalent; the idiomatic hint is
`fcntl(F_RDADVISE, &radvisory)` describing the byte range you intend to read
soon. The open-time sequential hint points it at the whole file (clamped to
`RDADVISE_MAX_BYTES` so a multi-TB ISO doesn't ask the kernel to prefetch
everything at once).
## RDADVISE_MAX_BYTES
Cap on the byte length passed to `F_RDADVISE`. Asking for a multi-GB
readahead window is counterproductive — the OS doesn't have that much cache
to throw at one fd. 64 MiB is generous for our use case (sweep, mux) so the
kernel's prefetch keeps pace with our app-level pipeline depth.
## drop_window
macOS has no direct `POSIX_FADV_DONTNEED` equivalent for a byte range.
`fcntl(F_NOCACHE)` would disable caching globally on the fd (too coarse — we
want the unread region to still benefit). Best approximation: no-op.
macOS's unified buffer cache is generally less prone to the pin-everything
pathology that triggers the regression on Linux NFS clients.
## prefetch
Async-prefetch the byte range `[offset, offset+len)`. macOS uses the same
`fcntl(F_RDADVISE, &radvisory)` primitive as the open-time sequential hint,
just targeted at a moving window instead of the whole file. The kernel
queues I/O for the requested range and returns immediately.
+77
View File
@@ -0,0 +1,77 @@
# `FileSectorSource` design rationale
`FileSectorSource` reads 2048-byte sectors from an ISO file on disk via
direct `seek + read_exact` (`pread`-equivalent) calls, letting the kernel's
own readahead policy manage prefetch.
## Why no app-level buffer
Pre-0.21.3 this source held a 32 MiB (later 4 MiB) read-ahead buffer to
amortise per-sector NFS round-trips. Empirically that buffer hurt: 32 MiB
refills bursted the NFS TCP connection hard enough to starve the concurrent
writer, and even a 4 MiB window gave the kernel less freedom to pipeline
reads with writes. Direct pread per call lets Linux's readahead widen as it
detects the sequential pattern, and naturally interleaves with writeback.
## DONTNEED on the consumed window
Without page-cache eviction an 85 GB streaming ISO read pins the entire
file in memory, starves the concurrent writer, and collapses mux throughput
(observed: 2.7 MB/s mux on 0.21.5 vs. 70 MB/s isolated NFS reads). Every
`READ_DROP_CHUNK_BYTES_DEFAULT` of consumed bytes we call
`posix_fadvise(DONTNEED)` over that window, mirroring the write-side
`crate::io::writeback::WritebackPipeline` policy.
The drop window is accounted by a monotonic forward byte counter, which
matches the sequential streaming pattern the mux highway drives. Under
random or backward access the dropped range no longer lines up with the
bytes actually read — but `DONTNEED` is purely an advisory cache hint with
no correctness impact, so this degrades to a slightly imprecise hint rather
than a bug.
`READ_DROP_CHUNK_BYTES_DEFAULT` (32 MiB) is the empirically tuned value on
a 7200rpm HDD via SATA: smaller windows (8 / 16 MiB) shorten the
kernel-readahead overlap and slow the producer; larger windows (64 / 128
MiB) let the page cache pin enough of the ISO to pressure concurrent
writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
`READ_DROP_CHUNK_MIB_MAX` (64 GiB) bounds `FREEMKV_READ_DROP_CHUNK_MIB`
before the MiB→byte multiply, mirroring `WRITEBACK_CHUNK_MIB_MAX`, whose
identical multiply is bounded for exactly this reason: without the bound,
a value above 2^44 overflows — a panic on the first ISO open in an
overflow-checked build, and in release a wrap to a near-zero window that
fires `drop_window` on every read. Out-of-range values fall back to the
default.
## Platform open hint
On `open()` each platform issues its "sequential access expected" hint so
OS-level readahead widens. The hint and the DONTNEED call live in per-OS
sibling modules (`linux::hint_sequential` et al.) — no inline `#[cfg]` in
this file.
## Read-ahead prefetch
After every consumed read we issue an OS-level prefetch hint for the next
equivalent-sized window (`platform::prefetch`). The kernel queues that I/O
asynchronously and returns immediately, so the next batch's read overlaps
with the caller's processing of the current batch (decrypt + demux + mux).
Without this the disk sits idle ~70% of each iteration because kernel
SEQUENTIAL readahead alone (capped at `read_ahead_kb`, default 128 KB) is
far smaller than our 16 MiB app-level batch.
## Test notes
- `read_sectors_with_an_undersized_buffer_errors_rather_than_panicking`:
this is a public `SectorSource` impl, so buffer length is caller input,
and the guard used to be a `debug_assert!` — compiled out in release,
where the `out[..bytes]` slice then panicked with 'range end index out
of range'. `Drive::read_fua` already carries this exact guard with a
comment recording the same panic being fixed there, and
`PrefetchedSectorSource` has `direct_read_too_small_buffer_errors` for
the same case; this impl had neither.
- `dontneed_eviction_does_not_affect_data`: reads past the default 32 MiB
drop chunk (16384 sectors) so the eviction block fires at least once,
asserting every sector still reads correctly. Avoids mutating
`FREEMKV_READ_DROP_CHUNK_MIB` to sidestep a parallel-test env race with
`drop_chunk_size_env_override`.
+16
View File
@@ -0,0 +1,16 @@
# FLAC elementary-stream decodability gate (`src/mux/codec/flac.rs`)
FLAC frames carry no length field, so a raw stream is delimited only by
sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
it comes from mp4/mkv, where each packet is exactly one container-delimited
FLAC frame (a complete, pre-delimited frame per packet). So this parser
is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
(poly 0x8005, init 0, non-reflected) computed so the residue over the whole
frame — footer CRC included — is zero (per the FLAC format specification,
RFC 9639, frame footer). A nonzero residue is definitive corruption → drop
the frame (a silence gap, never a shift — each packet keeps its own PTS),
logged via the shared tally.
A packet that does not begin with the FLAC frame sync is not a delimited
frame we can validate, so it is passed through unchanged (never
false-dropped).
+36
View File
@@ -0,0 +1,36 @@
# fMP4 muxer (`src/mux/fmp4/mod.rs`) — stub status and design notes
## Status
**STUB**. The muxer can emit the init segment (`ftyp` + a minimal HEVC
`moov` skeleton with one video track) via
[`Fmp4Mux::write_init_segment`], so the shape and call site are
validated, but media fragments (`moof`/`mdat`) are NOT emitted.
[`Fmp4Mux::write_video`] therefore returns
[`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented)
rather than silently accepting and discarding frames. It buffers
nothing, so it cannot accumulate memory.
## Not yet implemented
- `moof` box: `mfhd` (sequence_number) + `traf` (`tfhd` + `tfdt`
+ `trun` with sample sizes, durations, flags, composition offsets).
- `mdat` box: concatenated sample data.
- Fragment cadence: one fragment per GOP or every N seconds,
whichever comes first.
- HEVC `hvcC` box inside `moov.trak.mdia.minf.stbl.stsd` so the
init segment is self-describing (`stsd` currently has zero entries).
- Sample-flags computation (sync vs. delta, depends_on, etc.).
- Edit lists / fragment_duration for accurate seeking.
Reference: ISO/IEC 14496-12 §8 (Movie Fragments).
## `wrap_box` size-cast rationale
All callers build tiny init-segment boxes (kilobytes at most), so the
`u32` size never overflows; the saturating cast plus the debug assert
documents and guards that invariant rather than silently emitting a
truncated, structurally corrupt size field. `body` is always
internally constructed here, never untrusted input — a future caller
feeding a multi-gigabyte body trips the debug assert instead of
writing a malformed box.
+70
View File
@@ -0,0 +1,70 @@
# Fox — `dcx.xml` label parser
Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
under `<disc>` it lists every playlist the disc plays. The main-feature
playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
language, editorial purpose and forced/SDH state outright — everything a
label parser wants, in attributes, with no bytecode to walk.
NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
is one authoring house's internal metadata that happens to press onto the
disc. Every field meaning below was read off a real disc; treat an
unfamiliar value as unknown rather than guessing.
## Confirmed schema
A real Fox release, `/BDMV/JAR/05001/dcx.xml`:
```xml
<dcx>
<disc>
<playlist id="00001" lang="eng" name="topmenu"/>
...
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
...
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
...
<subtitle id="11" lang="eng" type="text"/>
<properties> ...chapter marks... </properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
</disc>
</dcx>
```
## Field meanings (read off the disc, not documented)
* `<audio type>`: `feature` = a normal program track; `rnib` = a
descriptive/narration track ("Royal National Institute of Blind People"
described-video), mapped to `LabelPurpose::Descriptive`. A `type`
containing "comment" maps to `LabelPurpose::Commentary`.
* `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
hearing track (`LabelQualifier::Sdh`).
* `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
track (`LabelQualifier::Forced`); `feature`/`text` are full tracks.
* `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
(audio ids 01..N, subtitle ids 01..N independently), matching the vendor
`stream_number` convention the ordinal binder in `super::apply_labels`
reads.
The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
so extraction is scoped to a single `<playlist name="feature">` element —
never a document-wide `<audio>` scan, which would merge the regional
`00800` (eng) and `00801` (jpn) tables into one and collide their slots.
## `select_feature_playlist` scoping rationale
Among all `<playlist name="feature">` elements — Fox presses one per
regional variant (`00800` eng, `00801` jpn) — the one carrying the most
`<audio>`/`<subtitle>` streams wins, ties to the first.
Returning ONE element is the whole point: each feature playlist is its own
STN table with its own 1-based slot ids, so merging two would put two
different streams on slot `01`. The richest table is the fullest label set
and anchors to its matching title by language sequence in
`super::apply_labels`.
+13
View File
@@ -0,0 +1,13 @@
# Linux `f_type` classification
`classify_f_type` in `src/platform/fs_type/linux.rs` is the single source of
truth for the magic-number comparisons used by both the path-based
(`detect_impl`) and fd-based (`detect_fd_impl`) entry points.
## Why the `i64` cast
`statfs::f_type` is signed `__fsword_t` on glibc and unsigned `c_ulong` on
musl; a portable comparison needs a common type, so the value is cast to
`i64`. On glibc x86_64 both are already `i64` — clippy flags the cast as
unnecessary on that target only, but the cast is required for musl, hence
the `#[allow(clippy::unnecessary_cast)]`.
+12
View File
@@ -0,0 +1,12 @@
# fs_type module
The buffering architecture (Phase 2) selects different output sinks
for local vs network filesystems: NFS gets the adaptive
`WritebackFile` machinery on Linux; local disks get `LocalFileSink`
and rely on the kernel's default writeback policy. This module
provides the construction-site primitive that picks which one.
Per the per-OS file-split convention, the actual `statfs` call lives
in the matching platform file (`linux.rs`, `macos.rs`, `windows.rs`,
`other.rs`); this `mod.rs` exposes only the cross-platform enum and
the `detect` entry point.
+20
View File
@@ -0,0 +1,20 @@
# `io::fsync` — platform-aware crash-durability primitives
Two flush operations need OS-specific handling to make a write survive a
crash / power loss:
- **`dir`** — fsync a directory so a prior `rename(2)` into it is durable.
After a crash a renamed file's dirent can otherwise be lost even though
the rename returned, because it is still page-cache-only. This is a POSIX
concept: on Windows std cannot even open a directory as a `File` (it does
not set `FILE_FLAG_BACKUP_SEMANTICS`), and NTFS/ReFS commit the rename's
dirent without an explicit directory flush — so it is a no-op there
rather than a failed open that logs on every marker write.
- **`file_durable`** — fsync a file's contents + metadata. Opens the file
**read+write**: on Windows `File::sync_all` maps to `FlushFileBuffers`,
which requires a handle with write access and returns
`ERROR_ACCESS_DENIED` (os error 5) on a read-only handle. (A read-only
`File::open` + `sync_all` is legal on POSIX, which is why that bug only
bit Windows.) The open mode is platform-uniform, so this lives here with
no dispatch.
+37
View File
@@ -0,0 +1,37 @@
# `fvi://` sink notes
Rationale and design notes for `src/mux/fvi_sink.rs` that don't belong in the
public rustdoc contract.
## Relationship to `VideoMap`
The sink is a thin consumer of the reusable, pure-data
[`VideoMap`](crate::mux::videomap) model
([`MapHeader`](crate::mux::videomap::MapHeader) /
[`PictureRecord`](crate::mux::videomap::PictureRecord)): it builds the header
from the title, then writes one record per video
[`PesFrame`](crate::pes::PesFrame) straight to disk. Nothing here re-parses
the elementary stream, and the whole index is never buffered.
Serialization is inlined in the sink itself.
## Extensibility
A different output format would be a DIFFERENT sink reusing the same
`VideoMap` model (e.g. a future `fvi2://`), not a pluggable encoder —
extensibility is by adding a sink, like every other sink in this crate.
## `write_fvi_record`: coding-derived members (§7.1)
Coding-derived members (§7.1) are produced via the codec-agnostic
`PictureInfo` accessors, never the raw bitstream: `field_order` is emitted
only when measured (OMITTED on codec-type-only HEVC/H.264/VC-1),
`progressive` only when signalled.
## `write_fvi_record`: `dts` / future `recovered` member
`dts` is MAY per the format spec — the highway carries no DTS on a frame, so
it is always omitted.
TODO(provenance→recovery join): a `recovered` MAY member (sweep/patch mapfile
overlap with this AU's `src`) belongs here; unreachable at `PesFrame`.
+189
View File
@@ -0,0 +1,189 @@
# H.264 codec parser notes
## Slice-type mapping
`h264_slice_coding_type` maps an H.264 `slice_type` (Rec. ITU-T H.264 §7.4.3,
Table 7-6) to a coding type. Values 5..=9 repeat 0..=4 (the "all slices of
this type" forms), so `slice_type % 5`: 0 = P, 1 = B, 2 = I, 3 = SP
(predicted → P), 4 = SI (intra → I). Returns `None` for values outside 0..=9
(malformed header).
## `unescape_ebsp_prefix`
Copies the leading bytes of an EBSP with emulation-prevention bytes removed
(ISO/IEC 14496-10 §7.3.1: a 0x03 following 0x00 0x00 is not part of the
RBSP). Only a short prefix is produced, because the sole caller decodes two
ue(v) fields from the very start of a slice header. `first_mb_in_slice` and
`slice_type` are each at most 32 bits, so 16 octets is ample and keeps this
off the per-frame allocation path for anything larger.
## `unescape_ebsp`
Copies `ebsp` with emulation-prevention bytes removed, stopping after at most
`max_octets` output bytes (pass `ebsp.len()` for "no cap").
The zero run-length counter is cumulative across the whole scan and is reset
only when an escape byte is actually dropped — matching the reference
decoding process in ITU-T H.264 §7.3.1, which discards a 0x03 following ANY
run of two-or-more 0x00 bytes, not just an exact `00 00 03` sliding window. A
window-based scanner that advances by 3 on a match and by 1 otherwise
disagrees with this on a run of 3+ real zero bytes ahead of an 0x03:
`00 00 00 03` decodes here as `00 00 00` (the run is escaped, matching every
other H.264 decoder), where a window scan starting fresh after each
non-match would keep the 0x03 as real payload. Only one implementation of
this rule may exist in this module — see `parse_sps_high_profile_ext`, which
used to keep its own window-scan copy that disagreed with this one on
exactly that input.
## `handle_param_set`
Handles an SPS/PPS NAL (mirrors the HEVC fix). The strip/emit decision is
made against the currently-ACTIVE set `cur`, NOT the codecPrivate copy
`first`: a streaming MKV decoder applies avcC once at init and thereafter
updates a parameter set only from an in-band NAL, so a switch BACK to the
first-seen body (== codecPrivate) is still a change the decoder must be told
about. Stripping on `== first` silently dropped that revert.
- First of its type → seeds codecPrivate; stripped (decoder gets it from avcC).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Unconditional (not only when the active set differs from codecPrivate): a
streaming decoder applies the avcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IDRs
even though its body is unchanged; if the decoder then drops it (a reset
event), nothing re-sends it and every subsequent slice fails (param-set id
out of range) until the next genuine change. Re-asserting at EVERY keyframe
— what compliant muxers do at every IDR — makes streaming decode
self-healing. Re-sending an identical param set is benign; cost is a few
bytes per keyframe. This strictly supersets the change-only re-assert, so
the param-set-revert fix is unaffected.
## `parse_sps_high_profile_ext`
Parses `(chroma_format_idc, bit_depth_luma_minus8, bit_depth_chroma_minus8)`
from a High-Profile SPS NAL (profile_idc ∈ `HIGH_PROFILES` — the 14
chroma/bit-depth extended profiles `codec_private` invokes this for).
SPS RBSP layout (ITU-T H.264 §7.3.2.1.1) up to the fields we need:
```
byte 0 NAL header (already known to be type 7)
byte 1 profile_idc
byte 2 constraint_set_flags / reserved
byte 3 level_idc
ue(v) seq_parameter_set_id
— High-profile branch —
ue(v) chroma_format_idc
if chroma_format_idc == 3: u(1) separate_colour_plane_flag
ue(v) bit_depth_luma_minus8
ue(v) bit_depth_chroma_minus8
```
RBSP emulation-prevention bytes (0x00 0x00 0x03 → 0x00 0x00) are removed
before bit-parsing so the bit reader sees clean RBSP data.
Returns `None` if the SPS is too short or malformed (Exp-Golomb code
overflows 32 bits, leading-zero count > 31, etc.). The caller silently omits
the extension in that case.
## Test notes
### `open_gop_intra_access_unit_is_a_keyframe`
Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD
open-GOP tail instead of dropping every frame to EOF and truncating the
video. BD H.264 titles reach random access at a non-IDR I-frame (the
recovery point); HEVC gets this free from the CRA NAL type, H.264 does not,
so the measured intra coding type is the signal. Red before the intra
promotion in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
### `inter_coded_access_unit_is_not_a_keyframe`
Guards the promotion's precision: a non-intra (P) access unit must NOT be
promoted to a keyframe, or the resync gate would resume on a frame that
carries inter-references to dropped data and the MKV cue index would gain
bogus random-access points.
### `slice_header_parse_removes_emulation_prevention_bytes`
The slice-header parse must remove emulation-prevention bytes first
(ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is EBSP,
and a 0x03 following 0x00 0x00 is not part of the RBSP.
Reachable when `first_mb_in_slice` is 65535 or more — a ue(v) that needs
sixteen leading zero bits, so the payload opens 0x00 0x00 and an encoder
must escape it. A UHD frame is ~32,400 macroblocks so conforming Blu-ray
never gets there, but 8K does, and the disc is untrusted input. Reading the
raw NAL decoded `slice_type` against the inserted byte and produced the
wrong picture type.
### `unescape_ebsp_drops_escape_after_a_run_of_three_zeros`
Regression: `parse_sps_high_profile_ext` used to re-derive the
emulation-prevention rule with its own window scanner (match `00 00 03` at
position i, advance by 3; else advance by 1) instead of calling the shared
`unescape_ebsp`. On a run of 3+ real 0x00 bytes ahead of an 0x03 —
non-conformant, but this is untrusted disc input, not a spec-clean encoder —
the two disagreed: `unescape_ebsp`'s cumulative zero counter (the rule the
H.264 reference decoding process in §7.3.1 specifies) drops the 0x03 as an
escape, while a fresh 3-byte window starting right after the non-matching
first byte kept it as real payload. Pin the shared function's behaviour
here so a second hand-rolled copy doesn't quietly reappear.
### `keyframe_param_reassert_emits_exact_bytes`
The keyframe SPS/PPS re-assert must produce EXACTLY these bytes: the active
SPS, then the active PPS, then the access unit's own NALs, each as a 4-byte
big-endian length prefix followed by the NAL body (ISO/IEC 14496-15
length-prefixed form, lengthSizeMinusOne = 3).
This pins the full byte string, not just its length, so the in-place
`splice` that replaced the build-a-new-buffer-and-copy approach is proven
byte-for-byte equivalent — including the ORDER (parameter sets ahead of the
slices, which is what makes the keyframe self-contained). The literals were
captured from the pre-splice implementation's output.
### `keyframe_param_reassert_does_not_reallocate_the_frame`
MEASURED: the keyframe parameter-set re-assert must be spliced into the
front of the already-assembled access unit IN PLACE, not built as a fresh
full-size buffer.
It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-hundred-KB allocation — memcpy'd the
whole frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a UHD title is
~200,000 frames of 150-400 KB with a keyframe every 1-2 s, i.e. thousands of
avoidable multi-hundred-KB copies per title, each large enough to go through
mmap. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates; this
counts the reallocations that happen, which must be zero. Mirrors the HEVC
parser's test of the same name.
### `avcc_profile_244_appends_extension_bytes`
ISO 14496-15 §5.3.3.1.2 regression: profile_idc=244 (High 4:4:4 Predictive)
ALSO mandates the chroma/bit-depth extension. It was missing from
HIGH_PROFILES, so a 244 stream took the Baseline/Main path and emitted an
avcC with NO extension bytes — non-conforming, and strict parsers then
assume 8-bit 4:2:0. The extension must be appended.
### `sps_reader_read_ue_thirty_one_leading_zeros_is_still_valid`
`SpsReader::read_ue`'s truncation guard is `leading_zeros > 31`: 31 leading
zero bits is the longest legal code and must decode, not abort. Mirrors the
equivalent guard in the shared `BitReader` in `startcode.rs`, kept here
because `SpsReader` is a separate, unshared implementation used only for SPS
parsing.
+16
View File
@@ -0,0 +1,16 @@
# halt
## Why `Ordering::Relaxed`
`Ordering::Relaxed` is sufficient on both load and store because this flag
is purely advisory — no other memory operations piggyback on it for
happens-before ordering. Callers that need to publish data across threads
do so via channels or other synchronization, not via this bit.
## Why 250ms for `POLL_INTERVAL`
250 ms is the sweet spot between responsiveness (operator presses Stop,
sees it take effect within ~quarter-second) and waste (atomic load + clock
read is cheap but not free at thousands of hertz). It's centralised here
so the half-dozen halt-polling loops across `io` can't drift apart
silently.
+35
View File
@@ -0,0 +1,35 @@
# Seeded robustness harness (`src/harness.rs`)
Every parser reached from here takes bytes that came off a disc, and this
crate's primary boundary is that the disc is untrusted: a malformed, damaged
or hostile image must never crash the library. These tests assert exactly
that one property — the parser returns `Ok` or `Err`, and never panics.
## Why this exists rather than `cargo-fuzz`
`cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
for libFuzzer's coverage feedback) and this project pins stable. So the
generator lives here instead. It gives up coverage-guided mutation — the real
loss — and keeps everything else: millions of cases, structure-aware input,
and a crash corpus. It also gains determinism, which a fuzzer does not have:
the same seed replays the same cases on any machine.
## Why no `proptest` or `arbitrary`
This crate has exactly one dev-dependency. That is a deliberate posture, and
a randomness crate is not worth ten transitive dependencies when the parsers
take plain `&[u8]` and a good enough generator is forty lines.
## Budget
`FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
enough that the per-commit gate stays under a second). The overnight run sets
it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
fixed so a failure in CI reproduces locally verbatim.
## On failure
The panic message carries the seed, generator and case index. Re-run with
`FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
into `tests/corpus/` as a permanent regression fixture — discovery happens
here, defence happens there.
+465
View File
@@ -0,0 +1,465 @@
# HD-DVD title scanning (`src/disc/hddvd.rs`)
HD-DVD is a **tree-level peer** of DVD and Blu-ray (not a stream variant like
FMTS): its content lives in `HVDVD_TS/` as `.evo` clips — Enhanced VOB, an
MPEG **program** stream — each with a small `.map` timemap sidecar, navigated
by `.xpl`/`.ifo` playlists in `HVDVD_TS/` and `ADV_OBJ/`. Because it is a
different tree with a different playlist format, it gets its OWN scanner
(this file), a peer to `Disc::scan_bluray_titles` — the two-format design
rule: a genuinely different format is a new enumerator, not an extension
bolted into the BD path.
## Title composition (authoritative path)
Title composition is authoritative, from the Advanced-Content playlist. HD-DVD
ships a real player playlist at `ADV_OBJ/VPLST000.XPL` (DVD-Forum
`HDDVDVideo/Playlist` XML). The scanner parses it (with a real XML parser,
`roxmltree`) into one `DiscTitle` per `<Title>`: its `<PrimaryAudioVideoClip>`
clips in playback order (each an EVO, referenced via its `.MAP` sidecar), the
`titleDuration`, the `displayName`, and the `<ChapterList>`. A layer-break
split (`FEATURE_1` + `FEATURE_2`, or `feature`/`feature_Divide`) is composed
into ONE title with the two parts as clips, carrying each clip's title-time
in/out points (45 kHz ticks) so a seamless join can be spliced onto one
timeline. Container is `ContentFormat::MpegPs`, so the existing PS mux path
handles it. Per-clip streams are enumerated by demuxing the clip head and
building one `Stream` per distinct elementary stream (video + DD+ audio),
codec sniffed from the ES bytes.
## Fallback path
When no playlist is present (or it fails to parse), the scanner falls back to
the older clip-name heuristic: parse the `HVA*.VTI` clip table, join the
`feature*`-named clips into one title, and emit every other clip on its own.
## Known gaps
Not parsed yet: subtitles (8-bit RLC on `0xBD` sub `0x20..=0x3F`) and per-track
audio languages (the XPL carries `<Audio description=...>` but they are not yet
wired onto the streams). Extents and size are real (the ripper images the
clips).
## VTI parsing
`parse_vti_clip_order` parses the `ADVANCED-VTS` VTI clip-name table. The
table is a run of `VTI_CLIP_ENTRY_STRIDE`-spaced records, each carrying a
NUL-terminated `<name>.EVO`. Rather than trust the (imprecise) header pointer,
it collects every NUL-terminated `*.EVO` name and keeps the largest group
sharing one residue modulo the stride — the clip table — in offset order.
Returns empty for a non-VTI blob or one with no recognizable table.
## Codec sniffing
`sniff_video_codec` reads MPEG / Annex-B start codes: `00 00 01 B3` → MPEG-2
(sequence_header), `00 00 01 0F` → VC-1 (BD/HD-DVD sequence-header BDU),
`00 00 01 [x7]` H.264 SPS NAL (type 7, forbidden_zero_bit clear) → H.264.
Returns `None` when no recognizable start code is present. The scan prefers
the unambiguous MPEG-2 / VC-1 sequence headers; H.264 is inferred from an SPS
NAL so a stray slice/picture code can't be mistaken for a different codec.
`sniff_audio_codec` recognizes only Dolby Digital Plus (E-AC-3) today, via its
`0x0B77` syncword — what real retail HD-DVD titles carry on sub-ids
`0xC0..=0xC7`. Returns `None` for an unrecognized sample so the caller drops
the stream rather than mislabeling it.
## probe_evo_streams
Demuxes the head of an `.evo` clip (through the disc's `SectorSource`) and
builds one `Stream` per distinct elementary stream found: the video track
(mapped to the canonical `DVD_VIDEO_PID` for plain 0xE0-0xEF video, or the
extended-stream-id PID for HD-DVD VC-1) and every DD+ audio sub-stream
(mapped via `dvd_audio_pid`). Codec is sniffed from the demuxed ES bytes.
Mirrors the stream construction in `Disc::scan_dvd_titles`; resolution /
language / channels use sane HD-DVD defaults (the muxer reads the true pixel
dimensions from the H.264 SPS, and E-AC-3 channel counts are not decoded
here). Returns an empty vec when the clip cannot be read or carries no
recognizable stream (e.g. an AACS-encrypted clip probed as ciphertext).
Cancellation is checked before every chunk read and is the ONE condition that
returns `Err` rather than an empty vec: a probe cut short by a Stop has not
established that the clip carries no streams, and reporting it as if it had
would enumerate a stream-less title as a scanned fact. A read that fails for
any other reason keeps the existing best-effort behaviour.
## collect_es
VC-1 video rides extended-stream-id `0xFD` with `stream_id_extension` `0x55`.
HD audio (MLP/TrueHD) can also use `0xFD` with other extensions — routing
those to their own audio tracks is deferred (see the HD-DVD program-chain
follow-up); until then only the VC-1 extension is treated as video, so an
audio `0xFD` sub-stream can never mis-stamp the video PID.
## XPL amplification bounds
The XPL parser guards against a crafted `ADV_OBJ/VPLST000.XPL` amplifying a
small file into an enormous amount of work or memory. Four separate axes are
bounded, each independently, because they are separate attacks:
- **`MAX_XPL_DEPTH` (32)** bounds element NESTING. A real `VPLST000.XPL` is
about six levels deep at its deepest
(`Playlist`/`TitleSet`/`Title`/`PrimaryAudioVideoClip`/`Video`, or
`.../ChapterList/Chapter`); 32 leaves a 5x margin for authoring tools that
wrap extra `ApplicationSegment`/`ObjectMappingList` layers, while staying
trivially stack-safe.
- **`MAX_XPL_TITLES` (512)** bounds how many `<Title>` elements one playlist
may declare — BREADTH, separate from the depth guard: a flat playlist
passes the depth check trivially and can still declare a title per 51
bytes, so a 64 MiB `VPLST000.XPL` (exactly `udf::MAX_FILE_BYTES`) reaches
~1.3 million of them. This matters here and not on the Blu-ray side because
of where the count comes from: `bluray.rs` learns its playlists from the
MPLS files actually present in the UDF directory, so the medium bounds it —
an XPL DECLARES its titles, so nothing does. The cost is drive time, not
memory: `compose_xpl_titles` runs `probe_evo_streams` once per title, and
each probe reads real sectors, so an uncapped count turns a `Disc::scan()`
the operator expects to take seconds into tens of terabytes of optical
reads on a disc that is only pretending to be large. The bound that matters
is `cap x EVO_PROBE_SECTORS`, since `compose_xpl_titles` memoizes on the
title's RESOLVED extent list, and distinct titles built from different
combinations of the same clips have different lists, so each one misses the
memo — at 4096 that product was 64 GiB of optical reads, the same worst
case, and the same contradiction with "seconds", as the sibling
`MAX_HDDVD_CLIPS` path. 512 keeps it at 8 GiB and remains far above any
real disc (retail HD-DVDs carry tens of titles).
- **`MAX_HDDVD_CLIPS` (512)** caps `.evo` clips taken from the `HVDVD_TS/`
directory listing. `MAX_XPL_TITLES` and `xpl_depth_within_limit` bound the
`ADV_OBJ/*.XPL` path, but a crafted disc simply omits `/ADV_OBJ`; control
then reaches the clip-name fallback in `Disc::scan_hddvd_titles`, which
emits one title per directory entry and probes each — the same unbounded-
probe cost, reached without the playlist. The directory is the only bound
the medium supplies, and `udf::MAX_DIR_BYTES` (1 MiB) still leaves room for
~24,000 FIDs. Memoizing the probe (`EvoProbeCache`) collapses entries that
resolve to the SAME extents, closing the many-names-one-File-Entry case —
but not this one: a File Entry is a single sector, so an attacker can give
every FID its own, each declaring its own extent, for ~48 MiB of image;
every probe then misses the memo and costs a full `EVO_PROBE_SECTORS`
(16 MiB) read, hundreds of GiB on a scan the operator expects to take
seconds. The cap's job is to bound `cap x EVO_PROBE_SECTORS`; it was 4096
(matching `MAX_XPL_TITLES`, which bounds a different, no-per-item-read-cost
path) — 4096 x 16 MiB is 64 GiB, more than dual-layer HD-DVD media
physically holds (reachable anyway, since distinct extent lists may overlap
on the medium). 512 keeps the product at 8 GiB and still leaves a wide
margin over any real disc (retail HD-DVDs carry TENS of `.evo` clips, ~10x
the real-world maximum, the sizing convention `MAX_XPL_CHAPTERS_PER_TITLE`
also uses against the 99-chapter authoring ceiling). Pinned by
`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`, which
asserts the PRODUCT rather than either constant, so raising one without
re-examining the other fails.
- **`MAX_XPL_CLIPS_PER_TITLE` (256)** bounds how many
`<PrimaryAudioVideoClip>` elements a SINGLE `<Title>` may contribute — the
fourth amplification axis, the one the other three guards leave open. A
64 MiB XPL holds ~1.7 million `<PrimaryAudioVideoClip src="A.EVO"/>`
elements at ~38 bytes each, and the collector is `descendants()`, not
`children()`, so a clip nested inside N ancestor `<Title>` elements is
collected by EVERY one of them — the depth cap still permits 32 such
ancestors (measured: a bare `<Title>` root nests 32 deep and still parses,
because the guard counts the self-closing clip element as non-nesting).
The product is ~1.7M x 32 ≈ 56 million heap-allocated `XplClip`s, each with
its own `String`, and then as many `Clip`s again in `compose_xpl_titles`.
Unlike the title cap, the cost here is MEMORY rather than drive time:
`compose_xpl_titles` de-duplicates by `.evo` name before probing
(`seen_evos`), so repeats cost no extra sector reads, but the `Vec`s are
built and held regardless. 256 is far above any real disc — a feature is
one clip, or two across a layer break; even seamless-branching/multi-angle
is tens — and keeps the aggregate bounded at `MAX_XPL_TITLES` x 256 clips
rather than the tens of millions above.
- **`MAX_XPL_CHAPTERS_PER_TITLE` (1024)** bounds `<Chapter>` elements per
`<Title>` — the same `descendants()` amplification, on the second unbounded
`collect()` in the same loop, bounded separately so capping clips doesn't
simply move the attack next door. A chapter is a bare `f64` rather than a
`String`-carrying struct so it's cheaper, but a
`<Chapter titleTimeBegin="…"/>` element is also smaller, so the element
count an XPL can reach is comparable. 1024 is far above any real disc: the
DVD/HD-DVD authoring convention tops out at 99 chapters per title, a 10x
margin.
## scan_hddvd_titles
Scans HD-DVD titles from the `HVDVD_TS/` `.evo` clips. The main feature is
authored as one or more `.evo` clips (a layer-break split —
`FEATURE_1`/`FEATURE_2` or `feature`/`feature_Divide`). The `HVA*.VTI`
navigation file names every clip in authored order; this parses it to
concatenate the feature clips into ONE title (so the largest-title pick gets
the whole movie, not just part 1), emitting every other clip as its own
title. Falls back to one title per clip when the VTI is absent or
unparseable, so a disc with no readable navigation still enumerates.
`chapters`/duration are left empty pending deeper VTI parsing.
Cancellation: `halt` is polled once per clip and again before every probe
chunk, and a read that fails with `Error::Halted` — how a live drive reports
a Stop, since `Drive::checked_exec` fails every command once its flag is set
— is propagated rather than swallowed. `Halted` is the only error this
returns; every other read failure keeps its best-effort behaviour. It has to
be an error and not a short title list: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it.
## xpl_depth_within_limit
Rejects a disc-supplied XML blob whose element nesting exceeds
`MAX_XPL_DEPTH`, BEFORE it reaches the parser. `roxmltree` is
recursive-descent and its depth-10 limit applies only to ENTITY expansion, so
element nesting is unbounded: a few hundred KB of well-formed XML (far under
the read cap) overflows the thread stack. That is a process ABORT, not an
`Err` and not an unwind, so neither the `Document::parse` fallback nor
`catch_unwind` can contain it; the only fix is to not hand the document over
at all. A byte-size cap is deliberately NOT added: file size is already
bounded by the UDF read path, and a size cap does not close this class (a
180 KB document already aborts) — depth is the bound that matters.
This is a conservative scan, not a validator: it tracks `<name …>` /
`</name>` while skipping comments, CDATA, processing instructions and
declarations, and treats `<… />` as non-nesting. A miscount can only cost a
pathological document its fast path, and the failure mode is the existing
one — fall back to the clip-name heuristic.
## Test rationale
Detailed rationale for tests in `src/disc/hddvd.rs`'s `#[cfg(test)]` module,
kept here so the in-code doc can stay within the internal-comment cap.
**`scan_hddvd_feature_composition_saturates_absurd_disc_declared_sizes`** —
The SECOND size accumulator. `compose_xpl_titles` sums an XPL's clip sizes;
this one sums the composed FEATURE title's parts, and it reads the same
disc-declared `u64` Information Length from the same `clip_extents` map. A
split feature whose two parts each declare a size near `u64::MAX` overflows
it exactly as the XPL path did. Nothing reconciles the declared size against
the extents that back it — here each part declares `u64::MAX` while
occupying ten sectors — because the allocation descriptor's length field is
only 32 bits wide, so the disc can always claim more than it holds.
Mutation: restore `size_bytes += *size` in the feature-composition path and
this goes red (debug: attempt to add with overflow).
**`scan_hddvd_does_not_compose_a_feature_over_an_unrecorded_part`** — A split
feature (FEATURE_1 + FEATURE_2) where ONE part carries an unrecorded extent
must not be composed into a FEATURE title at all. Composing around the
unusable part is the dangerous outcome: the title would play through as if
it were the whole movie while silently missing that part's runtime, and its
size still counts it. The other clips remain available as their own titles
— refusing the composition is not the same as refusing the disc.
**`scan_hddvd_logs_an_unrecorded_feature_part_with_its_own_code`** — The
refusal above must also be ACCOUNTED, with the unrecorded extent's own code.
This warn site carried a hardcoded `code = 6017` literal — in the very file
whose sibling arm was changed to stop doing exactly that — and nothing
tested it, so neither the literal nor its absence broke anything. Refusing a
feature and saying nothing leaves the operator with a disc that scanned
"fine" and is quietly missing its main title. Mutations: deleting the
`tracing::warn!` (no event to find); restoring the literal `6017` still
passes here BY VALUE, which is the point — what is pinned is
`E_UDF_UNRECORDED_EXTENT`'s value reaching the log, so a renumbering that
the literal could not follow goes red; logging a neighbouring code (e.g.
`E_UDF_NO_USABLE_EXTENT`) goes red immediately.
**`scan_hddvd_does_not_compose_a_feature_over_a_part_with_no_usable_extent`**
— The `Ok`-but-empty twin of the test above, and the one that actually bites
on an ordinary disc: a feature part whose `file_extents` call SUCCEEDS and
yields nothing usable — here a zero-byte `FEATURE_2.EVO`, no crafted ICB
required — must refuse the composition too, and must say so in the log.
Before this, the empty-`Ok` clip entered neither `clip_extents` nor
`unusable`, so the `any(|n| unusable.contains(..))` guard missed it and the
`filter(|n| clip_extents.contains_key(..))` beside it quietly DELETED the
part: the scan composed a `FEATURE` title out of part one alone, still
advertising itself as the feature, at rc=0, with no diagnostic anywhere.
Half a movie presented as a whole one — strictly worse than the `Err` case
the guard was built for, and reachable without a hostile disc. Mutations
this catches, all of which the pre-fix code exhibited: dropping the
`extents.is_empty()` branch entirely (a `FEATURE` title appears with one
extent); inserting into `clip_extents` anyway (same); marking the clip
unusable but NOT logging (the log assertion fails — absence of a log is
itself the defect); logging the neighbouring `E_UDF_UNRECORDED_EXTENT`
instead of `E_UDF_NO_USABLE_EXTENT` (the code assertion fails, because a
zero-length file is not an authoring hole and triaging the two together
sends whoever reads it at the wrong population).
**`parse_xpl_titles_caps_a_playlist_declaring_absurdly_many_titles`** — A
disc-supplied playlist nested far deeper than any real one must be REFUSED
BEFORE the XML parser sees it. `roxmltree` is recursive-descent and its
depth-10 limit covers only ENTITY expansion, so unbounded element nesting
blows the thread stack — an ABORT, which no `catch_unwind` and no `Err`
fallback can contain. ~50k levels is a few hundred KB of well-formed XML,
far under the read cap. The correct outcome is the same as any other
unusable playlist: an empty `Vec`, so the caller falls back to the clip-name
heuristic. Separately, this test also exercises BREADTH, not depth: a flat
playlist passes the nesting guard trivially and can still declare a title
per ~51 bytes, so a 64 MiB XPL (exactly `udf::MAX_FILE_BYTES`) reaches ~1.3
million of them. The cost is drive time, not memory: `compose_xpl_titles`
runs `probe_evo_streams` once per title and each probe reads real sectors,
so an uncapped count turns a scan the operator expects to take seconds into
tens of terabytes of optical reads. `bluray.rs` is not exposed to this
because its playlist count comes from the MPLS files present in the UDF
directory; an XPL DECLARES its titles, so nothing bounds them. Mutation:
delete the `.take(MAX_XPL_TITLES)` and this goes red at the declared count.
**`nested_title_xpl`** — Builds a playlist that nests `nesting` `<Title>`
elements inside one another and puts `clips` self-closing
`<PrimaryAudioVideoClip>` elements at the innermost level, plus `chapters`
`<Chapter>` elements. `parse_xpl_titles` collects a title's clips with
`descendants()`, not `children()`, so EVERY one of the `nesting` ancestors
collects the SAME innermost clip list — the amplification this exercises.
**`parse_xpl_titles_caps_clips_per_title_against_descendant_amplification`**
— CLIPS PER TITLE is the axis the title cap, the depth cap and the
directory cap all leave open. `MAX_XPL_TITLES` bounds how many titles a
playlist may declare and `MAX_XPL_DEPTH` bounds how deeply it may nest, but
NEITHER bounds how many `<PrimaryAudioVideoClip>` elements a single title
collects. Worse, the collector is `descendants()`, so one clip nested inside
N ancestor `<Title>`s is collected N times over — the clip count MULTIPLIES
by the nesting the depth guard still permits. A 64 MiB XPL
(`udf::MAX_FILE_BYTES`) holds ~1.7M clip elements at ~38 bytes each; with
the 32 `<Title>` ancestors the depth cap still permits, that is ~56 million
heap-allocated `XplClip`s, and then as many `Clip`s again in
`compose_xpl_titles`. This test uses small numbers that exercise the same
multiplication without a 64 MiB fixture: pre-fix it collected 756 clips per
title, 18,900 across the 25 nested titles, from a document declaring only
756 clip elements — an exact 25x, the nesting count. Mutation: delete the
`clips.len() >= MAX_XPL_CLIPS_PER_TITLE` break and this goes red at the
unamplified per-title count.
**`parse_xpl_titles_caps_chapters_per_title`** — CHAPTERS PER TITLE is the
same `descendants()` amplification on the second unbounded `collect()` in
the same loop. Bounded separately so closing the clip axis does not simply
move the attack next door. Mutation: delete the
`.take(MAX_XPL_CHAPTERS_PER_TITLE)` and this goes red at the declared count.
**`parse_xpl_titles_keeps_every_clip_of_a_realistic_title`** — The control: a
realistic multi-clip title still resolves EVERY one of its clips and
chapters. `SYNTH_XPL`'s main movie is a layer-break split — two clips on one
timeline — plus two chapters; losing either to the cap would cost a genuine
disc half its feature. Mutation: set `MAX_XPL_CLIPS_PER_TITLE` to 1 and this
goes red.
**`parse_xpl_titles_accepts_real_world_nesting_depth`** — The depth guard
must not reject real discs. A genuine `VPLST000.XPL` is a handful of levels
deep; this is the same document the parsing test uses, plus self-closing
tags, a comment and a processing instruction that the pre-parse scanner has
to account for without inflating its depth count.
**`scan_hddvd_composes_titles_from_xpl_playlist`** — The authoritative path:
when a VPLST000.XPL is present, titles come from the playlist — the
layer-break split (FEATURE_1 + FEATURE_2) is composed into ONE title with
the real duration, name, chapters, and per-clip title-time offsets — not the
clip-name heuristic.
**`compose_xpl_titles_saturates_absurd_disc_declared_clip_sizes`** — A
title's `size_bytes` is a running sum of DISC-DECLARED clip sizes: the `u64`
UDF File Entry Information Length, which nothing cross-checks against the
extent list that file actually occupies. Two clips can therefore each
declare a size near `u64::MAX`. Summed with a plain `+=`, that overflows: a
panic in a debug build — a crash on untrusted disc content, which this
library must never do — and a wrap to a small garbage total in release,
which misreports the title's size while looking entirely normal. Mutation:
restore `size_bytes += *size` in `compose_xpl_titles` and this goes red
(debug: attempt to add with overflow).
**`compose_xpl_titles_dedups_repeated_clip_references_by_evo`** — A crafted
Advanced-Content playlist can name the SAME `.evo` in
`<PrimaryAudioVideoClip>` many times over (the XML has no fixed element
count, unlike MPLS's binary PlayItem count). Each repeat must NOT push
another copy of that clip's extent list onto the title — mirroring
bluray.rs's `first_ref = seen_clips.insert(...)` gate, which pushes a clip's
extents only the first time its id is seen. Here the analogous key is the
`.evo` filename (this file has no clip_id; `.evo` is what `clip_extents` is
keyed by). Without the gate, extents grows by the referenced clip's full
extent list on EVERY repetition — unbounded by the number of on-disc files,
bounded only by playlist size.
**`scan_hddvd_titles_drops_a_split_feature_whose_part_cannot_be_read`** — A
clip whose extents cannot be resolved must drop the SPLIT FEATURE. Catches
reverting the `Err(e) => { warn; unusable.insert(..) }` arm in
`Disc::scan_hddvd_titles` back to the bare `Err(_) => {}` it replaced. RED
BEFORE GREEN, and TWO earlier attempts at this test did NOT go red. The
first asserted "no title names the broken clip" — that passes either way,
because a clip that resolves to no extents is never inserted into
`clip_extents` and so yields no per-clip title regardless. The second fixed
that but shipped no `.vti`: `order` is built solely from
`parse_vti_clip_order` of the navigation file, so with no `.vti` it is
EMPTY, `feature` is empty, and no composed title is ever built — the
assertion held vacuously with the fix reverted. Hence the synthetic
`HVA00001.VTI` fixture: it is what makes the composer run at all. The
defect lives one level up from the per-clip titles, in the composed
feature: `unusable` is what tells the composer that a part is MISSING
rather than merely absent, and the old bare `Err(_) => {}` populated it for
nothing but an unrecorded extent. So a scratched sector under FEATURE_2's
ICB (`Error::DiscRead`) left FEATURE_1 composing a title named "FEATURE" by
itself — half a movie offered as the whole one.
**`scan_hddvd_titles_excludes_a_clip_with_zero_sectors`** — A clip whose
file has a zero-byte size (a degenerate/empty allocation: its ICB's
allocation descriptor has `data_len == 0`, the UDF AD-list terminator, so
`file_extents` yields no extent at all) must not produce a title. NOTE: this
exercises the *upstream* `data_len == 0` terminator path in
`crate::udf::UdfFs::file_extents`, not the `sectors > 0 && lba > 0` guard in
`scan_hddvd_titles` itself — with this fixture (`file_extents` never returns
a `(lba, 0)` tuple, and `PART_START` unconditionally makes every resolved
`lba` positive) that guard is unreachable in a divergent way; kept here as a
regression check on the zero-byte-file behavior in its own right.
**`scan_hddvd_titles_probes_once_for_entries_resolving_to_identical_extents`**
— THE THIRD AMPLIFICATION AXIS on HD-DVD title composition, past both
existing guards. `xpl_depth_within_limit` bounds XML NESTING;
`MAX_XPL_TITLES` + the `.evo` de-dup bound what an XPL can DECLARE. Both
live on the playlist path — and an attacker simply omits `/ADV_OBJ`, so
`read_adv_obj_xpl` returns `None` and neither guard is ever consulted.
Control then reaches the clip-name FALLBACK, which emits one title per
`.evo` DIRECTORY ENTRY and probes each. Nothing de-duplicates a FID's ICB
LBA, so every name in a 1 MiB directory (`udf::MAX_DIR_BYTES`, ~24,000 FIDs)
can point at ONE File Entry. Each resolves to the SAME extents and each
costs a full `EVO_PROBE_SECTORS` (16 MiB) probe: ~375 GiB of optical reads
from a directory that describes a single file. Memoizing the probe on the
resolved extent list collapses that to ONE probe, because all of those
names resolve to one extent list. Mutation: drop the memo lookup in
`EvoProbeCache::streams` (always probe) and this goes red at `ENTRIES`
probes.
**`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`** —
The clip cap and the per-probe read budget are ONE bound, not two, and only
their product is meaningful: `MAX_HDDVD_CLIPS` probes each cost up to
`EVO_PROBE_SECTORS`, so the pair fixes the worst-case drive time of a
`Disc::scan()`. `EvoProbeCache` does not help against a crafted disc here:
it keys on the RESOLVED extent list, and distinct-but-overlapping extent
lists all reading one physical region miss the memo every time while every
read succeeds. So the product is genuinely reachable, and it is the number
that has to stay within what a scan can absorb. 8 GiB is the ceiling
asserted here. It is far above any genuine disc — retail HD-DVDs carry tens
of `.evo` clips, i.e. well under 1 GiB of probing — and far below the 64 GiB
the pair used to permit, which exceeded the capacity of the dual-layer media
this format tops out at and took tens of minutes of optical reads. This is
a relationship, not a magic number: raising EITHER constant without
re-examining the other trips it.
**`scan_hddvd_titles_still_probes_each_distinct_clip`** — CONTROL:
memoization must not silently collapse DISTINCT clips. A legitimate disc
with several different `.evo` files still probes each one, and each title
keeps the streams of ITS OWN clip. Mutation: make `EvoProbeCache`'s key a
constant (ignore the extents) and this goes red — the junk clip inherits
the first clip's H.264 streams.
**`scan_hddvd_titles_caps_a_directory_declaring_absurdly_many_clips`** —
Memoization alone is NOT the whole fix. Collapsing identical extent lists
bounds the one-File-Entry attack, but distinct extent lists are only as
bounded as the File Entries an attacker cares to lay down — and a File
Entry is ONE sector. A 1 MiB directory's ~24,000 FIDs can each point at
their own 1-sector File Entry (a ~48 MiB image) declaring its own huge
extent, so every probe misses the memo and the amplification is back.
`MAX_HDDVD_CLIPS` is the bound that closes that. Mutation: drop the
`clips.len() < MAX_HDDVD_CLIPS` gate in the directory scan and this goes
red at `ENTRIES`.
## EvoProbeCache
`EvoProbeCache` memoizes `probe_evo_streams`, keyed on the RESOLVED extent
list. Probing is the expensive half of title composition: each pass reads up
to `EVO_PROBE_SECTORS` (16 MiB) off the medium. Both composition paths can
reach the same physical clip many times over — an XPL naming one `.evo` from
many `<Title>`s, or a directory whose FIDs all point at one File Entry — and
a probe is a pure function of the extents it reads, so the second and later
passes over an identical extent list are re-reads of bytes already seen. The
key is the extent list itself, NOT the clip name or title: two names for one
File Entry share a key (one probe), while two genuinely different clips have
different `start_lba`s and so keep their own probes and streams. Cache size
is bounded by the caller's title/clip cap (`MAX_XPL_TITLES` /
`MAX_HDDVD_CLIPS`).
+99
View File
@@ -0,0 +1,99 @@
# HEVC Annex B muxer (`src/mux/hevc/mod.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## Module overview
Writes a raw `.hevc` / `.h265` Annex B byte stream:
`00 00 00 01 | NAL_unit | 00 00 00 01 | NAL_unit | …` with no container
framing. On the first frame the muxer emits the codec_private's VPS, SPS, PPS
(parsed from a `HEVCDecoderConfigurationRecord` in `length-prefixed-in-hvcC`
form), then converts each PES frame's length-prefixed NAL units to Annex B
and writes them.
## `write_frame`
Input may be either:
- Length-prefixed: `[u32-BE len][NAL bytes]` repeated. This is the form
emitted by libfreemkv's HEVC parser (the MKV-native layout). Converted to
Annex B.
- Already Annex B: a buffer beginning with a `00 00 00 01` or `00 00 01`
start code. Passed through unchanged.
## `hvcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §8.3.3.1.2):
- 22-byte fixed header
- byte 22 = `numOfArrays`
- each array: `array_completeness:1 | reserved:1 | NAL_unit_type:6`,
`numNalus:u16-BE`, then `numNalus` × `(nalUnitLength:u16-BE + NAL bytes)`.
We don't filter on NAL type — VPS (32), SPS (33), PPS (34), and any SEI
arrays included in hvcC all get the same Annex B treatment.
## `length_prefixed_to_annex_b`
Already-Annex-B input (a buffer beginning with a `00 00 00 01` or `00 00 01`
start code) is detected up front and passed through unchanged — some
upstream paths (raw HEVC ES from disc) hand Annex B straight through the
PES layer, and a genuine start code would otherwise be misread as a u32-BE
length prefix.
Truncation policy (single source of truth across all muxers): if a length
prefix runs past the end of the buffer (e.g. a NAL truncated by a bad disc
sector), the truncated trailing NAL is dropped and only the valid Annex-B
prefix accumulated so far is emitted. We never emit a half-NAL nor leak raw
length-prefixed bytes into the Annex-B stream.
## `nal_length_size`
Number of octets each NAL length prefix occupies in the elementary data of a
track described by `record` — the `lengthSizeMinusOne + 1` field of the
decoder configuration record (ISO/IEC 14496-15).
- avcC (`AVCDecoderConfigurationRecord`, §5.3.3.1.2): byte 4 is
`bit(6) reserved | unsigned int(2) lengthSizeMinusOne`.
- hvcC (`HEVCDecoderConfigurationRecord`, §8.3.3.1.2): byte 21 is
`constantFrameRate(2) | numTemporalLayers(3) | temporalIdNested(1) |
lengthSizeMinusOne(2)`.
The spec permits only 1, 2 or 4 octets (`lengthSizeMinusOne` of 0, 1 or 3);
a declared 3 is non-conformant but is decoded rather than rejected, since
reading N octets is the same operation for every N. A record too short to
carry the field, or a codec with no such record, falls back to
`DEFAULT_NAL_LENGTH_SIZE` — the width every freemkv parser emits.
This exists because assuming 4 is silent corruption for a legal source:
reading a 2-octet-prefixed frame as u32-BE yields an absurd first length,
the conversion loop bails with nothing parsed, and the raw length-prefixed
bytes are passed through as though they were already Annex B — a stream
with no start codes at all, and no error anywhere.
## `append_length_prefixed_as_annex_b`
Same conversion as `length_prefixed_to_annex_b` but writes directly into a
caller-owned buffer, avoiding an intermediate allocation on hot paths (e.g.
per-frame video muxing). If `data` doesn't parse as length-prefixed (no
NALs extracted), it's appended unchanged on the assumption it's already
Annex B.
## `append_length_prefixed_as_annex_b_sized`
For a source whose NAL length prefixes are `length_size` octets wide rather
than the 4 this crate's own parsers emit. Derive `length_size` from the
track's configuration record with `nal_length_size` — ISO/IEC 14496-15 lets
a legal avcC/hvcC declare 1 or 2 octet prefixes, and reading those as
u32-BE mangles the frame.
`length_size` outside `1..=4` is clamped to `DEFAULT_NAL_LENGTH_SIZE`; the
field it comes from is 2 bits wide, so that is unreachable from real input.
## `avcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §5.3.3.1.2):
- 5-byte fixed header
- byte 5 = `[reserved:3 | numOfSequenceParameterSets:5]`
- `numOfSPS` × `(sequenceParameterSetLength:u16-BE + SPS bytes)`
- 1 byte = `numOfPictureParameterSets`
- `numOfPPS` × `(pictureParameterSetLength:u16-BE + PPS bytes)`
+265
View File
@@ -0,0 +1,265 @@
# HEVC parser (`src/mux/codec/hevc.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## `hevc_num_extra_slice_header_bits`
`num_extra_slice_header_bits` from a HEVC PPS NAL (H.265 §7.3.2.3): after the
2-byte NAL header, skip `pps_pic_parameter_set_id` + `pps_seq_parameter_set_id`
(both `ue(v)`) and `dependent_slice_segments_enabled_flag` +
`output_flag_present_flag` (`u(1)` each), then read `u(3)`. `None` if the PPS
is too short to parse — the caller then declines to guess a slice type.
## `hevc_first_slice_coding_type`
Measures the coding type from the FIRST coded slice of an access unit
(H.265 §7.3.6.1 `slice_segment_header`). Reads only the leading fields of the
first slice segment: `first_slice_segment_in_pic_flag` u(1), the IRAP
`no_output_of_prior_pics_flag` u(1), `slice_pic_parameter_set_id` ue(v), the
`num_extra_slice_header_bits` reserved bits, then `slice_type` ue(v). Returns
`None` for a non-first slice or on truncation — never a guess. `num_extra`
MUST come from the active PPS so the bit offset to `slice_type` is exact.
## `HevcParser::hdr10`
Combines the accumulated mastering-display and content-light SEI into a
complete `Hdr10Metadata`, or `None` until BOTH HDR10 SEI messages have been
seen. Requiring both means an SDR / partially-signalled stream never emits a
half-populated (confidently-wrong) HDR10 record.
## `HevcParser::scan_sei`
Scans an SEI NAL (`[2-byte NAL header][RBSP]`) for the two HDR10 payload
types and captures each the FIRST time it appears (per-stream constants).
RBSP structure (Rec. ITU-T H.265 D.2 `sei_rbsp` / `sei_message`): a sequence
of messages, each `payloadType` then `payloadSize` encoded as a run of 0xFF
bytes plus a final <0xFF byte (the "ff-extension" coding), followed by
`payloadSize` payload bytes. Emulation-prevention (00 00 03) is stripped
before reading — unlike a slice header, an SEI payload can be deep enough
that an emulation byte falls inside the fields we read. Unknown payload
types are skipped by their size so a later HDR10 message in the same NAL is
still reached.
## `HevcParser::mark_clip_boundary`
MPLS `connection_condition` 0x05 and 0x06 are the non-seamless values (per
the BD-ROM spec: 0x01 = first item / seamless, 0x05/0x06 = non-seamless).
The first CRA at/after this call is rewritten CRA_NUT (21) → BLA_W_LP (16)
so a linear decoder sets NoRaslOutput and discards the now-dangling RASL
leading pictures with no "could not find ref" error.
It is a no-op for the rewrite unless a CRA actually follows: an
IDR/IDR_W_RADL boundary needs no fix (it carries no cross-splice
references), and the flag is cleared by the first IRAP-class NAL it reaches.
SAFETY: never call this for connection_condition 0x01 (seamless/first item)
or within a single-clip title — doing so could convert a legitimate
mid-content CRA to BLA. The default (never called) path leaves output
byte-identical.
## `handle_param_set`
Handles a VPS/SPS/PPS NAL: decides whether to strip it (the decoder already
has the value) or emit it in-band, and tracks the currently-active body.
The decision MUST be made against the currently-active set (`cur`), NOT the
codecPrivate copy (`first`). The two player behaviours for hvcC-in-MKV
diverge exactly here:
- A *seek-capable / Annex-B* player (one that converts hvcC to Annex-B by
inserting the parameter sets) re-applies the hvcC sets at every keyframe.
`reassert_active` handles it.
- A *streaming* decode (a decoder consuming the MKV directly — what most
integrity checkers do) applies hvcC ONCE at init and thereafter updates a
parameter set ONLY from an in-band NAL.
So when a title redefines a set mid-stream (id 0 body A → B) and later
switches BACK to A (== codecPrivate), the change to A must STILL be emitted
in-band: the streaming decoder is sitting on B and will never revert
otherwise, decoding the whole A-segment against B → CABAC/cu_qp_delta
desync. Stripping on `== first` (the old behaviour) dropped exactly that
revert and corrupted every "switch back to the first body" segment.
Rules:
- First of its type → seeds codecPrivate; stripped (the decoder gets it from
hvcC at init).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change, in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active VPS/SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Why unconditional (not only when the active set differs from codecPrivate):
a streaming decoder applies the hvcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IRAPs
even though its body is unchanged; if the decoder then drops it (a CRA reset
or SPS event), nothing re-sends it and every subsequent slice fails with
"PPS id out of range" until the next genuine change (observed as a ~24 min
corrupt band on one dual-layer UHD title). Re-asserting the active set at
EVERY keyframe — what compliant Matroska muxers do at every IRAP — makes
streaming decode self-healing. Re-sending an identical param set is benign
(decoders expect it at IRAPs); cost is a few hundred bytes per keyframe.
This strictly supersets the earlier change-only re-assert, so the
param-set-revert fix is unaffected.
## `parse_mastering_display`
Parses a Mastering Display Colour Volume SEI payload (Rec. ITU-T H.265
D.2.28 / semantics D.3.28). Layout — 24 bytes total, all big-endian:
`display_primaries_x[c]` u(16), `display_primaries_y[c]` u(16) for c=0,1,2
(SEI primary order is c=0 Green, c=1 Blue, c=2 Red); white_point_x u(16),
white_point_y u(16); max_display_mastering_luminance u(32);
min_display_mastering_luminance u(32). Returns `None` if the payload is
shorter than 24 bytes (malformed → ignored, never partially populated).
## Test notes
**`hevc_hdr10_sei_keeps_the_first_value_and_ignores_later_repeats`** — the
`scan_sei` match arms are guarded with `self.sei_mastering.is_none()` /
`self.sei_content_light.is_none()`, so the FIRST mastering-display and
content-light SEI a title carries wins and later repeats (every AU of a real
HDR10 stream repeats both) are ignored. Each AU below carries only ONE of
the two messages, so the whole-scan early return
(`if self.sei_mastering.is_some() && self.sei_content_light.is_some()`)
never fires early and the per-arm guards are actually exercised.
**`sei_nal` (test helper)** — wraps one or more SEI messages in a prefix-SEI
NAL (type 39) preceded by an Annex-B start code. The assembled message bytes
are emulation-prevented (as a conforming encoder would) so they never form a
false start code; the 0x80 RBSP trailing-bits byte is appended.
**`scan_sei_stops_copying_once_both_hdr10_messages_are_captured`** — MEASURED,
not reasoned: an HDR10 stream carries a prefix SEI per access unit, and
`scan_sei` used to allocate + byte-copy the whole SEI RBSP through
`strip_emulation_prevention` on EVERY one, including after both HDR10
messages were already captured. On a ~200,000-frame UHD title that is
~200,000 wasted allocations/copies. Counted at the single
`strip_emulation_prevention` call site.
**`nonzero_num_extra_slice_header_bits_shifts_the_slice_type_offset`** —
`num_extra_slice_header_bits` (H.265 §7.3.2.3) is a PPS field, and the
`slice_reserved_flag[i]` bits it counts sit BETWEEN
`slice_pic_parameter_set_id` and `slice_type` in the slice segment header
(§7.3.6.1). Every earlier fixture used a PPS with the field == 0, so the
skip was never exercised: a parser that ignored the field entirely agreed
with all of them, and would then read `slice_type` from the wrong bit offset
on any real stream that sets it — mislabelling every picture's coding type.
**`reasserts_active_pps_at_bare_keyframe`** — regression (UHD banded
corruption): a stream redefines PPS id 0 mid-title, then a later keyframe
arrives WITHOUT repeating it (the source relies on the decoder retaining the
redefinition — valid for a raw bitstream). An hvcC player re-applies the
FIRST (codecPrivate) PPS at every keyframe, so the active redefinition must
be re-asserted in-band at that bare keyframe or the whole segment decodes
against the wrong parameter set.
**`keyframe_param_reassert_does_not_reallocate_the_frame`** — MEASURED: the
keyframe parameter-set re-assert must be spliced into the front of the
already-assembled access unit IN PLACE, not built as a fresh full-size
buffer. It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-MB allocation — memcpy'd the whole
frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a 2 h UHD title at
24 fps with a 1 s GOP is ~7,200 keyframes, ~14-28 GB of avoidable memcpy per
title. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates;
this test counts the reallocations that happen, which must be zero.
**`emits_switch_back_to_codecprivate_pps`** — regression (a UHD title, the
real bug): id 0 is body A (→ hvcC), then redefined to B, then the title
switches BACK to A. A streaming decoder (hvcC at init, in-band updates only)
is sitting on B; the switch back to A must be emitted IN-BAND even though A
== codecPrivate, or the whole A-segment decodes against B (cu_qp_delta
desync). Stripping on `== hvcC` dropped this revert.
**`cra_at_auto_detected_pts_backstep_rewritten_to_bla`** — regression for a
UHD Dolby Vision profile 7 dual-layer title: a multi-clip title is read as
one concatenated stream and the mpls connection_condition is never plumbed
to the parser, so the splice CRA opening the next clip kept its dangling
RASL leading pictures and a linear decoder flooded "Could not find ref with
POC N". The parser must AUTO-DETECT the boundary from the backward PES-PTS
reset between clips (each .m2ts has its own PTS base) and rewrite that
splice CRA → BLA_W_LP with no explicit `mark_clip_boundary` call.
**`cra_after_33bit_pts_wrap_not_rewritten`** — regression for the 33-bit PTS
wraparound false-trigger (rc.5.2 audit #1): a SINGLE clip whose raw 90 kHz
PES PTS crosses the 2^33 counter wrap (~26.5 h) must NOT be mistaken for a
non-seamless clip join. Before the fix the raw 2^33→0 backward step armed
`pending_clip_boundary` and the next in-clip CRA was wrongly rewritten
CRA→BLA_W_LP (dropping valid RASL pictures — visible corruption). After
unwrapping onto a monotonic timeline the wrap is absorbed and the CRA stays
CRA.
**`cra_splice_detected_at_large_pts_magnitude_not_masked_by_wrap_logic`** —
the wrap-vs-backstep test above keeps `high_pts` near the very top of the
33-bit range, close enough to the `PTS_WRAP_PERIOD / 2` threshold that
`high - unwrapped` and a hand-flipped `high + unwrapped` land on the SAME
side of the threshold at every step — it doesn't actually distinguish the
two. This test uses PTS magnitudes around 3e9 (order 2^32, well below the
wrap threshold but reached in well under an hour), where the subtraction and
the addition diverge: ordinary forward progression keeps `high - unwrapped`
small, but `high + unwrapped` would already exceed `PTS_WRAP_PERIOD / 2` on
the very next frame, which would pollute `pts_wrap_offset` by a full period
and mask the genuine two-clip splice that follows.
**`non_cra_nals_never_rewritten_at_boundary`** — non-CRA NALs are never
rewritten even when a boundary IS marked. IDR (19), RASL (8/9), VPS/SPS/PPS,
and a trailing slice all pass through unmodified; the IDR clears the pending
boundary so no later CRA is wrongly converted.
**`no_boundary_marker_is_byte_identical`** — a frame stream with NO boundary
marker is BYTE-IDENTICAL to a parser that has no splice-rewrite field at all
(the UHD-safety guarantee): asserts byte-equality of every emitted frame
across a multi-AU stream containing CRAs, IDRs, RASLs, VPS/SPS/PPS, and
trailing slices — none of which is ever marked.
**`seamless_boundary_no_rewrite`** — a SEAMLESS boundary (connection_condition
0x05/0x06) is expressed by NOT calling `mark_clip_boundary`, so a CRA across
a seamless join is left unchanged. This encodes the contract: only
non-seamless joins call `mark_clip_boundary`; seamless ones never do, so no
rewrite occurs.
**`redefined_pps_emitted_inline`** — a parameter set REDEFINED mid-stream
(same id, different body) must be emitted INLINE so the decoder
re-activates it. Some discs redefine PPS id 0 partway through the title;
the old parser kept only the first PPS, so the second segment decoded
against the wrong PPS (CABAC desync).
**`make_sps_full` (test helper)** — builds a stored SPS NAL with sub-layers
and a conformance window, so the parser must skip sub-layer PTL and the 4
conformance-window ue(v) fields before reaching the bit depths.
`max_sub_layers_minus1` controls the sub-layer loop.
**`hvcc_array_length_round_trips_above_256_bytes`** — the hvcC array length
is a 16-bit big-endian field written as two separate `push`es: `(len >> 8)
as u8` then `len as u8`. Every VPS/SPS/PPS the rest of the suite feeds is
well under 256 bytes, so the high byte is always 0 and a `>>` -> `<<`
mutation (which also always truncates to 0 for those inputs) is
unobservable there. A real HEVC SPS with an extended VUI/HRD block can
exceed 256 bytes, so this test uses 300+ byte VPS/SPS/PPS and decodes the
16-bit length fields back to confirm they round-trip to the exact NAL
length, not merely a byte that happens to be 0.
**`hevc_ps_reorder_is_installed_and_flush_drains_its_real_frames`** — HEVC
counterpart of `h264_ps_reorder_reconstructs_distinct_display_pts`. A
DVD/HD-DVD program stream stamps a PTS only on each GOP anchor, so the
parser must reconstruct display-order timestamps for the rest. Two things
are pinned here that nothing else constrains: `with_ps_reorder(true)` must
actually INSTALL the reorderer (a builder that returned a
default-constructed parser would silently leave the transport-stream path
in place on every DVD title), and `flush()` must drain the reorderer's real
buffered frames at EOF — not nothing, and not a manufactured empty frame at
PTS 0 (a zero-length Block whose timestamp jumps back behind every cluster
already written, RFC 9559 §5.1.3.2). Slice bodies follow H.265 §7.3.6.1:
`first_slice_segment_in_pic_flag`, the IRAP-only
`no_output_of_prior_pics_flag`, `ue(pps_id)`, `ue(slice_type)` (Table 7-7:
0 = B, 1 = P, 2 = I). PPS body 0xC0 sets `num_extra_slice_header_bits = 0`.
+26
View File
@@ -0,0 +1,26 @@
# hex
The single hex → bytes parser for the whole workspace.
## Why one parser
Key material arrives as hex from three third-party sources — the keydb, an
online key service, and the mapfile's `# freemkv-vid:` comment — and each
used to parse it slightly differently (one stripped `0x`/`0X`, one stripped
nothing, one stripped `0x` only). A key written with a prefix one parser
didn't expect was silently dropped → "can't decrypt" with no error. This is
the one parser they all call, so the prefix/case/validation rules live in
exactly one place.
## Byte-oriented, not char-oriented
Operates on BYTES, not `&str` char indices: the inputs are untrusted, so a
multi-byte UTF-8 scalar must reject as malformed, never panic on a
mid-codepoint slice.
## `byte()` internals
Combines two ASCII hex-digit bytes into one byte. `as char` is intentional:
for a non-ASCII byte it produces a Latin-1 scalar that `to_digit(16)` then
rejects — so non-hex (incl. `+`/`-` sign chars) and multi-byte input fail
cleanly rather than slipping through `from_str_radix`'s sign handling.
+75
View File
@@ -0,0 +1,75 @@
# identity.rs — drive identity test notes
## `OversizedCountTransport` (test double)
Returns the requested data length but reports a `bytes_transferred` larger
than the caller's buffer — models a drive that lies about its transfer
count. The old slicing code panicked on this; the clamps in `from_drive`
must keep it from indexing out of range.
## `inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive`
`ascii_field` with a buffer shorter than `start` returns empty string
rather than panicking.
- Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer (e.g.
a device that reports fewer than 8 bytes) must not panic.
- Mutation: removing the `data.len() > start` guard makes it panic on
short inputs.
A drive that answers INQUIRY with GOOD status but a short or empty data
phase must fail the probe, not present as a blank drive.
The buffer is pre-zeroed, so decoding it unconditionally yielded empty
vendor/product/revision strings and a byte 0 of 0x00. Every platform
enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
and 0x00 is DIRECT ACCESS — so the drive silently vanished from the device
list rather than reporting that its identity probe failed. A USB-SATA
bridge mid-wedge does exactly this.
The two GET CONFIGURATION calls in the same function already clamped on
`bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
lines above them, discarded it.
## `ascii_field_boundary_len_equals_start_is_empty`
`ascii_field`'s guard is `data.len() > start` (strictly greater), not
`>=`: a buffer whose length is exactly `start` has NO byte at that offset,
so it must still yield empty, not attempt to slice.
Mutation: `>` -> `>=` would try to slice `data[start..]` when
`data.len() == start`, which panics (empty range at the very end is fine,
but the guard's job is the `< start` case below it — pinning the exact
boundary catches an off-by-one either direction).
## `display_formats_trimmed_fields_space_separated`
`Display` renders the four trimmed identity fields space-separated — the
human-readable counterpart of `match_key`'s pipe-separated form. Not
exercised anywhere else in this test module.
Mutation: replacing the `fmt` body with `Ok(Default::default())` writes
nothing at all, so formatting any `DriveId` yields "".
## `FixedGcCountTransport` (test double) / `from_drive_gc_failure_yields_empty_firmware_date`
GET CONFIGURATION failure (transport error) must not abort the identity
probe — firmware_date is empty, raw_gc_010c is empty. Mutation:
propagating the GET_CONFIGURATION error with `?` aborts `from_drive`.
`FixedGcCountTransport` is a transport whose GET CONFIGURATION responses
report an exact, caller-chosen `bytes_transferred` for each of the two GC
features (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
boundary guards can be pinned precisely. INQUIRY always succeeds.
## `from_drive_firmware_date_boundary_exactly_12_is_empty`
`end > 12` in the firmware-date branch (`from_drive`) is a strict
inequality: `bytes_transferred == 12` reports the field absent (offset 12
is the first byte of the 12-char date; a count of exactly 12 covers bytes
0..12, none of which is the date), so `firmware_date` must be empty, not
the mutant's off-by-one read.
Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an empty
but non-panicking slice — silently reporting "present" data that is
actually all outside the transferred count.
+168
View File
@@ -0,0 +1,168 @@
# IFO parser notes
Overflow rationale relocated from `src/ifo.rs` comments (see `// See docs/ifo.md`
pointers in that file for the anchor point of each section below).
## `DvdTitle::feature_start_cell` (Bug-4: scene-selection/logo at head of feature)
The main feature's PGC can open with leading cells that are NOT part of the
movie — a scene-index segment or an interleaved-angle sub-block. This walks
the leading run and returns the index of the first cell that is a plain
feature cell (category `0x00`-class).
Conservative by construction — it can NEVER truncate a normal feature:
- It only ever skips a *prefix*; the scan stops at the first plain-feature
cell and keeps everything from there on.
- A normal single-angle feature has category `0x00` on cell 0, so the scan
stops immediately at index 0 and drops nothing.
- It never drops on duration or any heuristic — only on the spec category
bits — and it never drops the FIRST cell of an angle block (the angle we
keep).
- As a final guard it never returns past the last cell, and never drops when
that would leave zero cells.
For a real disc where every feature cell category is `0x00` and chapter 1
sits at 00:00:00, this returns 0 — a no-op — which is the correct result: the
disc's scene-index lives in a separate menu/title PGC, not in leading cells
of the feature PGC, so there is nothing to drop here.
## `bcd_to_frames` / `bcd_to_secs` (dvd_time_t is a timecode, not wall-clock time)
Format: `[hours_bcd, minutes_bcd, seconds_bcd, rate_and_frames]`
- Byte 0: hours in BCD (e.g. 0x01 = 1 hour, 0x12 = 12 hours)
- Byte 1: minutes in BCD
- Byte 2: seconds in BCD
- Byte 3: bits 7-6 = frame rate flag (01=25fps, 11=29.97fps),
bits 5-0 = frame count in BCD
The seconds field advances once every [`DvdRate::nominal_fps`] frames. On
NTSC discs that is every 30 frames, but 30 frames of 30000/1001 fps video
last 1001/1000 s — so reading H:M:S as literal seconds under-reports real
time by exactly 0.1% (3.6 s per hour). Callers that need real seconds must go
through the frame count, which is what `bcd_to_secs` does.
## `MAX_TT_SRPT_TITLES` / `parse_tt_srpt`
DVD-Video caps a disc at 99 titles (VMGI TT_SRPT `TT_Ns`, and the 99-title /
99-title-set structure the format is built around), but the on-disc count is
an untrusted `u16`: a ~800 KB crafted IFO can declare 65535 entries, each
re-parsing a PGC into a full `DvdTitle` (~540 MB of `DvdInfo`). Headroom: this
IS the format maximum, so it clips no conformant disc — a real DVD cannot
address a 100th title through TT_SRPT.
`parse_tt_srpt` has two bounds on untrusted input: the declared entry count
is clamped to `MAX_TT_SRPT_TITLES`, and entries naming a `(vts_number,
vts_title_num)` pair already seen are dropped. De-duplication is a
correctness fix as well as a bound: two TT_SRPT entries pointing at the same
VTS title are the same title, and `parse_pgcit` would otherwise re-parse
that one PGC into a separate `DvdTitle` per entry.
## `parse_audio_attr` visibility
`pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
`parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
calls it directly so its language-mapping tests run the real parser over real
on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
could agree with the muxer while both disagree with the disc. Narrowing this
would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
that can drift apart, or moving those tests away from the code they exist to
pin — both worse than the widened crate-internal visibility, which reaches no
public API.
## `assign_audio_sub_stream_ids`
On DVD-Video the sub-id's low nibble is the audio-stream *number* (0-7),
shared across all codecs — the single stream index the PGC `audio_control`
table / navigation registers select — and the high nibble is the codec base.
So the sub-id is `codec_base | position`, where `position` is the stream's
index in the IFO audio-attribute table (NOT a per-codec running count):
- AC-3 → `0x80 | i`
- DTS → `0x88 | i`
- LPCM → `0xA0 | i`
- MP1/MP2 and anything else → `None` (regular MPEG-audio PES, not a
private-stream-1 sub-id).
A per-codec ordinal was wrong: it only coincides with the wire id when a
codec's first stream is also the disc's audio stream #0. Any codec that is
not the first audio stream (e.g. a DTS track after an AC-3 track) then got
a sub-id one-too-low, so the demux routing key (`0xBD00 | sub_id`) never
matched and the track muxed silent. The positional index is the real wire
number, so distinct positions still give distinct sub-ids (no collision).
Position saturates at 7 so a malformed over-count never produces an
out-of-range sub-id.
## `parse_raw_dvd_lang_bytes`
The salvage is deliberately not narrowed to a-z: whatever survives is only
ever a lookup key for `dvd_lang_to_iso639_2`, which degrades anything it does
not recognize to `und`, so a stray `X` or `5` costs nothing and cannot reach
an output stream as a language code.
## `dvd_lang_to_iso639_2`
Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
frozen-1988 language list still puts on disc). The narrower `vocab::menu_lang`
table is deliberately NOT used here: it exists for Blu-ray menu-graphic
filename tokens and knows only 25 languages, so a Region-2 disc's Romanian,
Bulgarian, Croatian, Serbian, Slovak, Slovenian, Hebrew, Estonian, Latvian,
Lithuanian and Icelandic tracks would all fold onto `und` together. DVD
streams carry an empty `label`, so the language is the only thing
distinguishing one subtitle track from the next — a valid code that is
identical for six tracks is worse for the user than the invalid one it
replaced. Both tables normalize to ISO 639-2/T, so they agree wherever they
overlap.
An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's own
"undetermined" value) — a valid element value — rather than passing through
an invalid 2-letter code or an empty string. Never guesses.
## `video_attr_absolute_bytes_pin_real_layout` test
ABSOLUTE-BYTE pin (audit §3 #2): the existing video-attr tests build the byte
via `v_atr_byte(...)`, which uses the SAME shift constants the parser reads
with — a co-edit of constant + helper would silently re-introduce the
PAL-as-NTSC bug and every test would still pass. This test feeds
`parse_video_attr` HARDCODED bytes captured from real DVD-Video layouts
(DVD-Video video attributes: mpeg_version[7-6] video_format[5-4]
display_aspect[3-2] permitted_df[1-0]) — no `v_atr_byte`. If the parser's bit
positions drift, these fail.
## `dts_after_ac3_uses_positional_substream_id` test
Regression (The Punisher 2004): audio[0]=AC-3 5.1, audio[1]=DTS 5.0. The DTS
track sits at audio position 1, so its wire sub-id is 0x89 (0x88 | 1), NOT
the per-codec 0x88. With the old per-codec ordinal it got 0x88 → demux
routing key 0xBD88 had no match → every DTS packet (which carries 0x89) was
dropped → the track muxed present-but-silent while the AC-3 (at position 0,
where ordinal and position coincide) played fine. Positional numbering fixes
it end-to-end.
## `pgc_chapter_times_ntsc_no_pulldown_drift` test
Regression test for issue freemkv#25 (NTSC chapter drift). Builds a PGC from
the real cell table of the disc in the bug report and checks the chapter
marks against the reference tool's values. Before the fix, chapter 14 landed
4.019 s early; the drift was proportional to elapsed time (0.1%), so a short
synthetic fixture would not have caught it.
## Test additions note
The block of tests starting at `cell_category_low_flags_are_bit_isolated`
covers cell-category bit isolation, palette/program-map arithmetic, and the
malformed-language salvage paths.
## `build_pgc` test fixture
Builds a standalone PGC starting at offset 0. Layout: 0xEA-byte header, then
the program map (one byte per program, the 1-based first cell number), then
the 24-byte cell playback table. `cells` are `(category, BCD time,
first_sector, last_sector)`.
## `build_pgcit` test fixture
Builds a VTS_PGCIT at offset 0: VTS_PGC_Ns(2) + reserved(2) +
VTS_PGCIT_EA(4), then one 8-byte VTS_PGCI_SRP per PGC (VTS_PGC_CAT(4) +
VTS_PGCI_SA(4), the PGC's byte offset from the VTS_PGCIT start). `pgcs` are
appended after the SRP table.
+35
View File
@@ -0,0 +1,35 @@
# `src/io/image_writer.rs`
## Why this is not `freemkv_engine::copy`
The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
exists because an optical drive returns read errors on marginal media. A
file-backed or synthesized source has no marginal media: a read either
succeeds or the underlying file is broken, and retrying it is pointless.
Routing a non-drive source through the recovery path is not merely wasteful,
it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
both of which are empty for an already-decrypted source, so identity passes
for ANY such source: a second run with a different input to the same output
path would resume over the previous image and produce wrong content at exit
zero. Keeping the two paths separate makes that unrepresentable.
So: drive sources get `freemkv_engine::copy`. Everything else gets this.
## `write_image` rationale
The output is a faithful image of whatever the source presents — decrypted
if the caller wrapped the source in a
[`DecryptingSectorSource`](../src/sector/decrypting.rs), ciphertext if it did
not. `write_image` performs no decryption itself and makes no decryption
decision; that belongs to the caller, which knows whether the run is
`--raw`.
On cancellation the partial file is left in place — the caller decides
whether a partial image is worth keeping, and deleting a multi-gigabyte file
the user may want to inspect is not this function's call to make.
A short read from the source is treated as an error rather than zero-filled:
silently padding a truncated source would produce an image that looks
complete and is not.
+10
View File
@@ -0,0 +1,10 @@
# `/BDMV/index.bdmv` layout
Binary layout of the BD navigation index (First Play, Top Menu, and the
title table), as parsed by `src/bdnav/index.rs`.
`"INDX"` + version(4) + `indexes_start`(u32 @8) + … ; at `indexes_start`:
`index_len`(u32), First-Play object(12), Top-Menu object(12),
`num_titles`(u16), then `num_titles` title objects(12 each). Every object's
`object_type` is the top two bits of its first byte; for an HDMV object the
`id_ref` into `MovieObject.bdmv` is a big-endian u16 at object offset 6.
+7
View File
@@ -0,0 +1,7 @@
# FMTS index selection
Where the resolved index comes from is a separate concern from disposition
classification (`resolve_disc_index` in `src/aacs/index_select.rs`): today it
is read off the index keys the key source handed us; when Processing Keys are
available it will come from the VK derivation instead. Either way the
disposition logic in `unit_disposition` is identical.
+100
View File
@@ -0,0 +1,100 @@
# `aacs::inf` — notes
## `UnitKeyFile`'s hand-written `Debug`
Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
— plus the disc hash they are looked up by. A derived `Debug` printed every key
byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
message, a future `tracing::debug!` in this module) leaked them. Only
non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
The disc hash is the public keydb lookup key, printed as hex the same way
`DiscEntry` prints its own — never as raw bytes.
## `parse_vtkf`
Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
Dukes of Hazzard `VTKF000`):
```text
[0x00..0x0C] magic "DVD_HD_V_TKF"
[0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
[0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
[0x1C..0x80] reserved
[0x80..] 64 entries × 36 bytes:
BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
BIFO bit 7 (AV_FLG) set = this slot holds a title key
(pre-recorded discs fill the binding MAC with 0xFF)
[0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
```
The slot index (1-based) is the CPS unit number, so an absent slot is
SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
hand the wrong title key to CPS unit N+1. The title→CPS mapping is
playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
`title_cps_unit` is left empty here.
The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
(real multi-CPS-unit HD-DVD titles) yielded garbage keys for CPS unit ≥2.
## `synth_vtkf` test helper
Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
`VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
## `unit_key_file_debug_is_redacted` test
`UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
printed every byte; the hand-written impl must not. Sentinel key byte
0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
`#[derive(Debug)]` back fails this.
## `read_mkb_from_drive_returns_the_concatenated_pack_payload` test
`read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
Media Key ladder) consumes exactly what it returns. An empty return is
not a benign "no MKB" — it is a total read failure reported as success,
and every derivation then fails with a key-not-found code that points
the operator at their keydb rather than at the drive.
This pins the CONTENT: the concatenated payload of all packs, in pack
order, byte for byte.
## `read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack` test
The CDB is what the drive actually acts on, and every byte of it is
load-bearing: a wrong format code returns a different disc structure
entirely, and a wrong allocation length truncates the pack. The existing
test above pins the opcode, the format code and the pack number; this
pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
`[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
(BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
## `read_mkb_from_drive_accepts_a_full_size_pack` test
A pack payload filling the FULL 32768-byte window must come back whole.
The `len > 0 && len <= 32768` bound is what stands between a maximal
pack and a silently dropped one, and the small payloads used elsewhere
in this module never reach it.
## `read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds` test
A drive that DECLARES more payload than it returned must not be
believed. The BE16 length in the response header is drive-supplied data:
a firmware bug, a short transfer, or a hostile device can put a value in
it that runs past the 32772-byte buffer. Copying `len` bytes on that word
alone panics the rip thread mid-scan.
Both the first-pack read and the per-pack loop carry the same bound, so
both are exercised here: the over-declared pack contributes nothing and
the honest pack still comes through.
+34
View File
@@ -0,0 +1,34 @@
# `src/io/mod.rs` — background
## `WritebackFile`
Drop-in wrapper around `std::fs::File` for any call site that performs
large sequential writes (sweep, patch, mux, etc.). It implements `Write`
and `Seek` so existing code paths can swap `File` for `WritebackFile`
with no body changes. Internally it drives a `WritebackPipeline` that,
on Linux, drains dirty pages continuously at 32 MB granularity to avoid
the kernel's accumulate-then-burst flush behaviour. macOS and Windows
use a no-op pipeline — their default cache policies have not been shown
to exhibit the same pathology for this access pattern.
## `FileSectorSource`
The read-side dual of `WritebackFile` — it implements
[`crate::sector::SectorSource`] for an ISO file using direct
`pread`-equivalent calls so the kernel's own readahead policy runs
(which interleaves naturally with the concurrent writeback). It pairs
that with periodic `posix_fadvise(DONTNEED)` drops on the consumed
window so an 85 GB streaming ISO read doesn't fill the page cache and
starve the concurrent MKV write.
## `Pipeline` + `Sink`
The generic producer/consumer primitive used by sweep, patch, and mux
to overlap reads with writes via a bounded channel + dedicated consumer
thread.
## `byte_prefetcher`
The read-ahead producer feeding the mux pipeline for `io::Read`-backed
sources: a worker thread fills a recycled pool of buffers and ships
them through a channel, exposing `BytePrefetcher` / `PrefetchShell`.
+25
View File
@@ -0,0 +1,25 @@
# src/labels/jar.rs — relocated comment prose
## `MAX_CLASS_BYTES` rationale
A real BD-J `.class` is far under this ceiling (64 MiB); a lying header
simply gets truncated and the class fails to parse, which is skipped like
any other bad entry. The buffer is grown incrementally and the read is
capped here rather than pre-sized from the declared size.
## `for_each_jar` "top-level" vendor examples
"Top-level" means entries directly under `/BDMV/JAR/`, not nested under a
subdir. Pixelogic, Criterion, Paramount, etc. put their data files inside
`/BDMV/JAR/<x>/`; dbp and Deluxe put their jar directly at
`/BDMV/JAR/<name>.jar`.
## `read_is_bounded_by_cap` test rationale
The read is bounded by `MAX_CLASS_BYTES`: a stored entry whose real payload
exceeds the cap yields only the first `MAX_CLASS_BYTES` bytes to the
parser, never the full (unbounded) entry. Verified here on a small cap via
the entry-count path: the truncated bytes still parse a valid class
prefix, but no read beyond the cap occurs. The test asserts the entry is
still surfaced exactly once (the cap does not drop legitimate entries) and
that the call returns.
+68
View File
@@ -0,0 +1,68 @@
# KeyProvider (src/aacs/provider.rs)
Long-form notes for `src/aacs/provider.rs`, relocated from doc comments to
stay under the comment-guard caps. See the module's own (trimmed) `//!` doc
for the short version.
## Method flavors
`KeyProvider` methods come in two flavors:
- **Bulk material** (`device_keys`, `processing_keys`, `media_keys`) — the
resolver unions (and dedups) results across all providers and tries each
candidate.
- **Disc-keyed lookup** (`lookup_disc_by_hash`, `lookup_disc_by_vid`) — the
resolver short-circuits on the first hit, so providers are queried in
array order with fastest/closest first.
`host_certs` is a sixth method but is NOT consumed by the resolver chain:
the SCSI handshake reads host certs directly from the caller-supplied
credentials, not from the provider array. A provider that overrides
`host_certs` today has no effect on the handshake; the method is retained
as a forward-looking extension point only. Correspondingly,
`Providers::host_certs` (the union helper) is `#[allow(dead_code)]` and
unused by the resolver chain today.
Default impls return empty / `None` so backends only override the methods
they actually support — an external key service might implement only
`lookup_disc_by_hash`, while a local file might implement all six.
Calls may block (disk I/O, network round-trips). The resolver invokes each
method at most a handful of times per scan; for per-disc memoization,
implementations should cache internally.
## `Providers` (resolver-side aggregation helper)
The resolver wraps `ctx.providers` (`&[&dyn KeyProvider]`) in `Providers`;
its methods apply the union-vs-short-circuit policy described above per
method. The bulk unions dedup so overlapping providers don't make the
resolver re-walk/re-validate identical material.
## `SuppliedKey` (bridge for `Disc::decrypt_with`)
A `KeyProvider` backed by a single caller-supplied key's raw material — the
bridge for `crate::disc::Disc::decrypt_with`.
The application's key source did the lookup and handed in material at one
level (DK / PK / MK / VUK). This exposes exactly that material to the
version-dispatched resolver, which owns ALL derivation — so a source never
derives, and the lib remains the single home for the AACS chain across
1.0 / 2.0 / 2.1 / 2.x.
Each level fills only its own field; the rest stay empty, so the resolver
naturally runs the matching path (DK→…, PK→…, MK-pool brute, or a
disc-keyed VUK hit). `decrypt_with` already knows the disc, so the
`lookup_disc_by_*` hash/VID arguments are irrelevant — a present
`disc_entry` is returned for any query.
## Test: `providers_host_certs_unions_every_providers_certs_in_array_order`
`Providers::host_certs` is the union across the provider array. It is not
wired into the handshake today (see above), so nothing else in the crate
would notice a body that dropped every cert on the floor — and the day it
IS wired in, a silently-empty cert list means the drive AACS authentication
finds no host certificate to present and every disc fails to open, with no
indication that the caller's certs were discarded.
Unlike the bulk key unions this one does NOT dedup (`HostCert` is not
Ord/Hash), so the assertion is on the full concatenation in array order.
+254
View File
@@ -0,0 +1,254 @@
# keysource.rs — extended notes
Overflow rationale relocated from `src/keysource.rs` doc comments by the
comment-guard (prose caps: 8 lines for pub-item rustdoc, 3 lines for internal
comments). Each section below is pointed to by a short `//`/`///` comment at
the corresponding site in the source.
## Module overview — ownership split
libfreemkv still OWNS the crypto: the boil-down primitives and the AES live
here. A source owns only PATH ORCHESTRATION — deciding which primitive to
call with what input for the material it happens to hold. Source
implementations are published in the companion `freemkv-keysources` crate,
keeping key *policy* (which store, which order, online vs local) out of the
library. It knows what material it holds (a DK / MK / VUK / pre-decrypted UK)
and what it must fetch from the disc (VID, MKB, encrypted title keys, content
samples) to get there.
## `MIN_SAMPLE_UNITS` — why this many, why here
The key service identifies a key by which of the submitted units it decrypts,
so too few samples — especially on FMTS, where a segment interleaves several
variants at the unit level — can return a key that matches an incidental unit
rather than the one asked about (a false positive). This many distinct units
make the request unambiguous.
Canonical here (the base crate) so BOTH consumers agree on one value: the
online source in `freemkv-keysources` (which refuses to send an under-sampled
request) re-exports it, and libfreemkv's own FMTS forensic query
([`crate::mux`]) sizes its per-segment batch by it. Layering forbids the
reverse import (keysources depends on libfreemkv, not vice versa), so the
value lives at the lower layer both share.
## `DecodeSampleSet` — parse, don't validate
"Parse, don't validate": the only constructor, `DecodeSampleSet::new`, returns
`None` for an under-sized slice, so an online key request simply *cannot be
built* from too few samples. The runtime `len() < MIN_SAMPLE_UNITS` check that
used to live at the request site (and was silently forgotten by an
under-sampling caller, reading as "key service down") becomes a compile-time
obligation: a request builder that takes `&DecodeSampleSet` can never receive
an unchecked `Vec`.
The *count* enforced here is a runtime property of the disc (how many
encrypted units it yields); the *requested* count is a caller-side
compile-time constant that callers pin to `MIN_SAMPLE_UNITS` (see e.g.
autorip's `SAMPLE_UNITS`). Together the two make under-sampling
unrepresentable at the request boundary.
The wrapped samples are on-disc AACS ciphertext — the same bytes the sibling
`DiscInputs::samples` redacts as key MATERIAL — so `Debug` is hand-written and
redacting.
## `DecodeSampleSet`'s hand-written `Debug`
A derived `Debug` dumped every wrapped sample verbatim: a `DecodeSampleSet`
carries at least `MIN_SAMPLE_UNITS` 6144-byte aligned units (>= 49 KiB, in
practice multi-MB) of AACS ciphertext plus each unit's clear 16-byte
derivation seed, so one `tracing::debug!("{set:?}")` on a failed `/decode`
request — or an `assert_eq!` whose panic message formats it — wrote all of it
to the log that gets attached to a bug report. Same policy and same shape as
`DiscInputs`'s impl.
## `DiscInputs`'s hand-written `Debug`
Redacting `Debug`, per the policy `aacs::types` documents (and which
`aacs::types::Vid` already applies to this very Volume ID). `DiscInputs` is
public and returned by `crate::Disc::inputs`, so a consumer's
`tracing::debug!("{inputs:?}")` used to print the Volume ID, the whole
`Unit_Key_RO.inf` (the encrypted title keys), the entire MKB and every
ciphertext sample verbatim into a log that ends up attached to a bug report.
Only non-secret identity and shape (presence, lengths) is printed.
## `DiscInputsCtx::new` — parse-failure policy
A present-but-malformed `unit_key_ro` (truncated / wrong magic / wrong
stride) parses to an empty key set, so a later `enc_title_keys` returns
`Ok(&[])` indistinguishably from a disc that legitimately has no title
keys — the parse failure is swallowed here, not surfaced as an error.
## `KeySource` trait — the two resolve operations
Returning an empty `Vec` means "no key for this disc from this source"; an
`Err` means the source itself failed (I/O, parse, network). The caller
(`resolve_and_apply`) tries each source in order and validates the returned
keys against real ciphertext before committing them, so a wrong key from one
source transparently falls through to the next.
Two explicit resolve operations, one per key kind — never one overloaded call
whose meaning depends on how many keys came back:
* `get_unit_keys` — the disc's base per-CPS-unit Unit Keys (index space =
CPS-unit number). The common path for every disc.
* `get_fmts_indexes` — the AACS 2.1 forensic index keys (index space =
forensic index 1..N). Defaults to empty: a source with no forensic material
opts out, and only an FMTS disc ever asks.
What each source must do to answer is the source's own business: a keydb keys
on `disc_hash` and reads no samples; the online source submits the ctx's
content samples (a base batch for `get_unit_keys`, an index-1 anchor batch
for `get_fmts_indexes`) to the key service.
## `resolve_and_apply_traced` — one-shot semantics and CPS-unit numbering
One-shot per source: each source's `KeySource::get_unit_keys` is called
exactly once with a `DiscInputsCtx` over `inputs`. Non-empty Unit Keys are
mapped to terminal `Key::Unit`s and applied via `crate::Disc::decrypt_with`,
which validates them against `inputs.samples` and only mutates the disc on
success — so a wrong/partial key set is rejected and the loop continues.
CPS-unit numbering: a source returns Unit Keys carrying the POSITIONAL index
from `crate::aacs::derive::decrypt_unit_key`; the library's canonical
CPS-unit number is `position + 1` (matching
`crate::aacs::inf::parse_unit_key_ro`'s `(i + 1)`), so the committed
`AacsState.unit_keys` is byte-identical to the library-resolved path.
The NUMBER itself is not what descramble indexes by — but the ORDER is
load-bearing, so a source must return its keys in CPS-unit order. Trial
decrypt-and-check was deliberately deleted (see `crate::decrypt::AacsKeyMap`:
decryption is driven by the disc's CPS-unit / FMTS-segment structure, "never
by trial-decrypt-and-check per unit"), and `decrypt_sectors_mapped` indexes
the committed pool POSITIONALLY — `unit_keys[key_idx].1`, where `key_idx` is
a POSITION in the Vec a source returned, recorded by
`resolve_mux_key_map_cached` / `resolve_fmts_key_map`. Return the same keys
in a different order and every `AacsKeyMap` points at the wrong key: the
whole title decrypts under a neighbour's key, or a forensic range trips the
`is_clean` net into `DecryptFailed`. (The doc used to say the number "is
cosmetic for descramble (the decrypt path strips it and tries every key)",
which is what the DELETED trial-decrypt path did; the only place that still
tries every key is `Disc::decrypt_with`'s sample VALIDATION, which does not
descramble content.)
## `fetch_unit_keys` — the two callers
This is exactly what both paths do — only the samples differ:
* at disc open, `ctx` carries reachable-content samples → resolves the
up-front CPS units (the common one),
* in the read, on a decrypt miss, `ctx` carries the FAILING unit's ciphertext
→ resolves the CPS unit that wasn't sampled up front.
Same sources, same call; there is no separate "fetch". Unlike
`resolve_and_apply` this does not validate/commit to a disc — the read's
decorator re-decrypts with the returned keys, which is the validation.
## `FetchOutcome` — the error-signal struct
Whether a driver run resolved keys, and — when it did NOT — whether the miss
was a genuine "no source holds this key" (`errored == false`) or at least one
source FAILED (`errored == true`, e.g. a network source was unreachable). The
distinction gates negative-result memoization: an empty-because-absent
result is safe to cache, an empty-because-a-source-was-down result is
transient and must NOT be cached (the key may resolve once the source
recovers).
## `drive_unit_keys`
`fetch_unit_keys` plus the error signal: drive `sources` in order, return the
first source's non-empty Unit Keys, and flag whether any source that failed
to answer did so with an `Err` (a source failure) rather than an empty `Ok`
(genuine absence — see `KeySource::get_unit_keys`).
## `key_fetch` — one builder, memoization policy
One builder, used by every read path (sweep / patch / mux) and every
consumer (CLI, autorip) — neither application contains the fetch logic, only
its key-source config. Cheap to clone; build once, clone into each read
path. `make_sources` is invoked per fetch (the cold path) so the resolver
stays `Send + Sync` without requiring `KeySource: Send`.
Each operation is memoized per sample-batch fingerprint (keys are disc-level,
so repeats hit cache). A genuinely-empty reply is cached, but a source
FAILURE is not — a transient miss (see `FetchOutcome::errored`).
## `read_encrypted_units` — why here, and the CPI selection rule
Lives in the library, not a key-source crate: reading the disc and carving
AACS units is decryption *mechanism* (unit geometry anchored at each
extent's `start_lba`), which the library owns. A key source is *handed*
these bytes via `DiscInputs.samples`; it never reads the disc itself.
"Encrypted" is decided by `crate::aacs::content::aacs_unit_encrypted` — the
AACS Copy Permission Indicator (CPI) in the top 2 bits of byte 0, the
spec-correct signal (`buf[0] & 0xc0`). NOT the `is_clean` TS-sync heuristic:
a unit lacking clean TS syncs does not imply encryption (an FMTS variant
frame or an odd clear unit can lack syncs yet be unencrypted), and a clear
unit sent to a key server yields nothing to validate against — the "0
encrypted units" rejection. A clip opens with clear navigation units
(PAT/PMT, menus) whose CPI is clear; only CPI-flagged content units are
collected — probing several points spread across EACH extent so a title
whose encrypted body starts late (or whose midpoint lands in clear nav)
still yields samples. CPI is read at each extent's `start_lba`
(clip-file-anchored), so byte 0 is a real unit start and the flag is
meaningful.
## Test doc overflow
### `key_source_host_certs_defaults_to_empty`
Spec: a source holding no cert returns the empty vec; the `mkb` param is
forward-looking and the default ignores it. Mutation: a default returning a
non-empty vec would inject phantom certs into the OEM handshake.
### `errored_empty_is_not_cached_and_retries_when_source_recovers`
A transient source outage must NOT be memoized as a permanent "no key": a
fingerprint whose first fetch failed because the source errored must be
re-asked, and once the source recovers the key resolves. Regression guard
for the negative-result memoization fix — caching the errored empty would
permanently drop a recoverable unit for the rest of the op.
### `key_fetch_routes_unit_and_fmts_to_distinct_source_methods`
The two `KeyFetch` operations route to the two DISTINCT trait methods:
`unit_keys` drives `get_unit_keys`, `fmts_indexes` drives
`get_fmts_indexes`. A source that returns different keys per method proves
the seam no longer collapses "1 base key" and "the forensic set" into one
overloaded call — the operation, not the return length, decides which.
### `read_encrypted_units_finds_scrambled_content_off_the_midpoint`
#4 regression: encrypted content NOT at the extent midpoint (a
late-starting feature, or a midpoint landing in clear nav) must still be
sampled — empty samples make `decrypt_with` skip wrong-key validation. The
old midpoint-and-forward sampler returned empty; the probe-spread finds the
early scrambled band.
### `read_encrypted_units_selects_by_cpi_not_ts_sync`
DISCRIMINATING: selection is by the AACS CPI (byte 0), NOT the TS-sync
clarity heuristic. Half the units lack TS syncs but are CPI-CLEAR
(`byte0 & 0xC0 == 0`) — genuinely UNencrypted units that merely lack TS
syncs; the old sampler collected these and the key server rejected the POST
as "0 encrypted units". `read_encrypted_units` must skip them and return
ONLY CPI-flagged units. A regression to selecting by TS-sync clarity would
collect the CPI-clear units too and fail the `& 0xC0` assertion.
### `disc_inputs_ctx_parses_unit_keys_at_the_version_stride`
Audit #5 — a DISCRIMINATING test for the version→stride fix. A 2-key
`Unit_Key_RO.inf` whose SECOND key sits at the V20 (64-byte) offset; a V10
(48-byte) parse reads a DIFFERENT region. Confirms `DiscInputsCtx` parses at
the stride for `inputs.version` — a swapped `from_major` branch or a
hardcoded stride (the exact bug 1.2.0 fixes) would fail this, where the
prior single-key fixtures passed regardless of stride.
### `disc_inputs_debug_is_redacted`
`DiscInputs` is public and returned by `Disc::inputs`, so any consumer's
`tracing::debug!("{inputs:?}")` prints it. A derived `Debug` printed the
Volume ID (the value `aacs::types::Vid` deliberately renders as
`Vid(<redacted>)`), the whole `Unit_Key_RO.inf` (the encrypted title keys),
the entire MKB and every ciphertext sample verbatim. Sentinel byte 0xD5 =
decimal 213, matching `aacs::types::redaction_tests`. Mutation guard:
restoring `#[derive(Debug)]` fails this.
### `decode_sample_set_debug_is_redacted`
`DecodeSampleSet` is public and wraps the SAME on-disc ciphertext the
sibling `DiscInputs` redacts, so a derived `Debug` dumped >= MIN_SAMPLE_UNITS
x 6144 bytes of verbatim AACS ciphertext (plus every unit's clear 16-byte
derivation seed) into any log that formatted it. Sentinel byte 0xD5 =
decimal 213, matching `aacs::types::redaction_tests` and the `DiscInputs`
test above. Mutation guard: restoring `#[derive(Debug)]` fails this.
+556
View File
@@ -0,0 +1,556 @@
# src/labels/mod.rs — design notes
Long-form rationale relocated from doc comments per the comment-guard
audience cap (internal items cap at 3 lines; see `ci/comment-guard.py`).
## `label_at`
The VENDOR label occupying 1-based STN slot `n` of `stream_type`, if any.
Slot lookup is restricted to labels with no `StreamId`, and that
restriction is the whole point. `stream_number` is not one coordinate
system, it is two that share a name:
* on a vendor label it is a slot in the one stream table the config blob
describes, which is what this function's callers count against; while
* on a derived label it is that stream's slot in *its own* playlist's
table, which says nothing about any other playlist's numbering.
Reading a derived label out of a slot lookup therefore answers a question
about table A with a fact about table B. A label that carries a `StreamId`
names the stream it describes outright and is bound by that id instead —
see `apply_labels`.
## `find_anchor`
Find the title the label list is actually describing, for one stream type.
A vendor label list is a single stream table — one playlist's STN slots —
but it is handed to every title on the disc. When sibling playlists cover
the same clip with different stream subsets, per-title ordinal numbering
puts the same label on different physical PIDs in each, and the flags
contradict each other. The list itself carries no playlist id, but it does
carry a language per slot, and that sequence is a fingerprint: on the
corpus, exactly one title's per-type language sequence reproduces the label
list position for position, and content confirms that title's binding is
the correct one.
So: the anchor is the title that `anchor_score` admits and that confirms
the most of the list. `None` means no title matches — the list describes a
stream table this disc scan cannot see, and nothing may be bound
authoritatively.
Only VENDOR slots take part. Derived labels (MPLS / CLPI) name their own
stream and bind directly, so admitting them here would be asking a title to
agree with a numbering that describes a different playlist — the mistake
that made this gate a coin flip while the merged list was the only list
there was.
## `anchor_score`
How strongly this title's stream sequence matches the vendor label list, or
`None` if the title contradicts it and cannot be the table it describes.
A stream disqualifies the title when the vendor label on its slot states a
different language. A slot the vendor list does not name constrains nothing:
under-yield is the normal shape of these blobs — an authoring layer ships
editorial labels for the streams it considers interesting and leaves the
rest as bare slots — so a hole is the list's silence, not a disagreement.
The score is how much of the list the title positively CONFIRMS: slots where
both sides state a language and state the same one. A list that names no
languages at all scores zero everywhere and the ranking falls through to
table size, which is what it did before this became a fingerprint.
## `apply_labels`
Apply a pre-extracted set of labels to titles' streams.
Every label either NAMES the stream it describes or it does not, and that
split — not a slot number — decides how it binds. Four tiers, most certain
first:
1. **This title is the anchor.** `find_anchor` identifies the title
whose stream table the vendor list is describing, so its slots are the
list's own slots and bind directly.
2. **An anchor proved this PID.** Every slot of the anchor yields a
`(clip, PID) -> label` fact, and a PID is the same physical stream in
every playlist that plays that clip. Sibling playlists bind through
that map, so a vendor label lands on the same elementary stream no
matter which playlist enumerates it, and a slot the anchor never showed
us is not bound at all.
3. **The label names this stream.** A derived label carries a
`StreamId` read out of the very table the stream itself was built
from, so `(clip, PID)` equality is identity, not inference. This is the
floor that gives every stream its language and codec.
4. **Ordinal, language-checked.** A vendor label with no anchor behind it
falls back to the 1-based per-type STN slot — but only where tier 3 had
nothing to say, and only when the languages do not contradict.
Tier 3 outranking tier 4 is the substance of this ordering: a label that
states which stream it belongs to beats a guess about which stream it might
belong to, even when the guess is the richer label. The policy where a title
cannot be bound confidently is to leave the stream alone: an unlabelled track
is a much smaller harm than a full-dialogue track wearing `forced`, which
presents to the user as a duplicate of the track they wanted, and which the
muxer can only undo on discs that state `forced_on_flag`. A subtitle label
carries nothing but the qualifier, so an unverifiable one has no upside at
all to trade against that risk, and the ordinal path applies one only when
the label and the stream both state a language and it is the same one.
Audio streams update `purpose` + `label` (codec/variant info; never
English purpose text). Subtitle streams update `qualifier` and the
`forced` flag.
Extracted from `apply()` so the matching logic is unit-testable without
needing a SectorSource / UdfFs.
## `merge_mpls_floor`
Merge the MPLS-derived floor into a framework parser's label list.
The two lists do not share a coordinate system, and the merge must not
pretend they do. A framework `stream_number` is a slot in the one stream
table the vendor blob describes; an MPLS label's is that stream's slot in
its own playlist's table. Matching them by number — which is what this
function used to do — merges by an equality that means nothing, and the
merged entries then land on whatever stream happens to sit at that ordinal
in each title.
So nothing is merged BY slot. Every MPLS label whose stream is not already
named in `framework` is appended, keeping its own `StreamId`, and
`apply_labels` decides per stream which of the two reaches it: the
vendor's editorial label where an anchor puts it there, the floor
everywhere else. Framework labels stay richer and stay ahead — they are
simply no longer competing for a slot number.
Sorted at the end (vendor slots first, in slot order, then the named
streams) so callers see a deterministic list.
## `append_clpi_orphans`
Append CLPI ProgramInfo streams that no existing label already names.
These are "orphan" streams — physically present in the .m2ts per CLPI's
clip-authoritative view, but no MPLS playlist references them, so the
framework and the MPLS floor both missed them. Empirically they are
commentary or alternate-version streams the authoring tool left out of the
published playlist. Returns the number appended.
Each carries the `(clip, PID)` it was read under, so it binds to that
stream and to nothing else. It used to be given `stream_number =
max(existing per type) + 1, +2, …`, an ordinal invented here and shared
with the slot numbering the vendor list uses — which meant a title with
more streams than the list had slots could reach an orphan by counting, and
be labelled from a stream no playlist even plays. There is no ordinal now:
a stream that is in no playlist is in no title, so an orphan label binds to
nothing, which is exactly right and is now structural rather than lucky.
## `select_result`
Pick the winning parser result from `results` (built in PARSERS order):
highest `Confidence` among non-empty results, with the earliest array
position winning on a tie — matching `extract()`'s strict-`>` first-wins
scan.
`Iterator::max_by_key` returns the LAST maximal element, so the key is
`(confidence, Reverse(index))`: among equal-confidence entries the one
with the smallest index has the largest `Reverse(index)` and is selected,
i.e. first wins.
## `collect_chapter_summary`
Scan `/BDMV/PLAYLIST/*.mpls`, parse each, return a row per playlist with
chapter count (entry marks only) and total duration. Sorted by playlist
filename. Skipped entries (read error, parse error, no marks) silently
dropped — this is a diagnostic field, not a correctness-critical one.
## `jar_inventory`
List filenames found under any `/BDMV/JAR/<x>/` subdirectory of the disc.
Deduped, sorted. Returns an empty vec if no JAR dir is present.
`pub(crate)` so filename-based parsers (e.g. `png_filenames`) can scan
menu-asset names without a reader.
## `jar_inventory_from`
The body of `jar_inventory`, over the `/BDMV/JAR` children directly, so
it is unit-testable without a `UdfFs`.
A `BTreeSet`, not `Vec::contains`: the entry names come from the disc's own
UDF directory records, so both the file count and the name lengths are
attacker-controlled, and a linear `contains` doing a full `String` compare
per candidate is quadratic in the number of files. The set also subsumes
the trailing sort — it yields sorted, deduplicated output directly.
## test: `jar_inventory_dedup_does_not_hang_on_a_hostile_directory`
`jar_inventory` deduplicated with a linear `Vec::contains`, doing a full
`String` comparison per candidate — quadratic in a file count taken
straight from the disc's UDF directory records, with attacker-chosen
name lengths to inflate each comparison.
This is a HANG GUARD, and the name says so: a return to the linear scan
makes this fixture run for minutes (120 000² / 2 comparisons over a
180-byte shared prefix), which without the deadline would wedge CI
rather than fail it. It is not a complexity proof — no assertion here
can distinguish `BTreeSet` from any other sub-quadratic dedup, and the
clock-free half of the claim (dedup, sort, directory exclusion) belongs
to `jar_inventory_dedups_sorts_and_skips_dirs` below.
The deadline is a real margin, unlike the 6x one that made
`paramount.rs`'s wall-clock test flake under a loaded CI box: measured
at 0.14 s debug / 0.07 s release against 10 s, so ~70x. A shared CPU
does not close that; a quadratic dedup does not survive it.
## test: `parsers_registry_order_locked`
Lock the parser roster + order. If someone reorders the array or
adds/removes a parser, this test forces them to update the expectation
explicitly. The order is load-bearing: first matching `parse()` wins, so
reordering changes which parser claims a disc on overlapping detect
signals.
dbp + deluxe MUST stay at the end (their detect triggers on "any BD-J
disc"; placing them earlier would short-circuit the stricter parsers
above them).
## test: `select_result_first_wins_on_tie`
`select_result` must pick the highest-confidence non-empty result and,
on a confidence tie, the FIRST in array order (regression for the old
`analyze()` `max_by(...then(Equal))` no-op that picked the LAST).
`extract()` — the path that actually ships — used to re-derive this
same rule with its own inline `>` scan and had no test at all, so the
bug this test guards against could have recurred there unnoticed. It
now calls `select_result`, so this test covers both.
## test: `suppression_is_by_named_stream_not_by_slot_number`
Spec: the merge suppresses a floor entry only when the list already
NAMES that stream. A framework label occupying the same slot number is
not the same fact — a vendor slot and a playlist STN slot are different
coordinate systems — so it suppresses nothing.
This is the merge rule inverted from what it used to be: the old key was
`(stream_type, stream_number)`, which dropped a floor entry whenever
some unrelated vendor slot happened to share its number, and kept one
whenever it did not. Both outcomes were decided by an accident of
counting.
## test: log-injection guard on playlist-name logging
A playlist name is a raw UDF directory entry — disc-controlled bytes,
validated no further than a lossy UTF-8 decode. Logging it through
tracing's `%` (Display) sigil writes those bytes VERBATIM, so a crafted
`.mpls` filename carrying ANSI escapes or control characters forges
terminal output and log structure in any consumer rendering the event
(CWE-117). `?` (Debug) escapes them, and `str`'s Debug is exactly the
escaping this needs.
`info!` is not covered by the debug/trace-logging exemption: this fires
on an ordinary rip of an ordinary disc.
Mutation: put `%` back on `playlist` in `apply_labels` and this goes red.
## test: `forced_label_follows_the_pid_not_the_ordinal_across_sibling_playlists`
The cross-playlist mis-binding this module's two-tier binding exists to
stop, in its measured shape: two playlists cover the identical feature
clip, one of them enumerates one extra subtitle ahead of the forced
slots, and the vendor label list describes the shorter of the two.
Numbering the same list from 1 inside each title puts the `Forced`
label on a different PID in each — in the longer playlist, on the full
dialogue track. The user then sees two identical-looking English
subtitle tracks, one of them wrongly flagged forced.
Binding by the PID an anchored title proved, rather than by ordinal,
puts `forced` on the same two physical streams in both playlists and
leaves the extra one alone.
## test: `ordinal_binding_drops_a_subtitle_label_that_contradicts_the_stream_language`
No anchor to work from (nothing on the disc reproduces the label list's
language sequence), so binding falls back to the STN ordinal — and a
label that names a different language than the stream it would land on
is evidence the list is describing some other stream table. Drop it:
unlabelled beats mislabelled, because the payload here is `forced`.
## test: `generate_video_label_secondary_dolby_vision_el`
Spec: a secondary (dependent-view) video stream with Dolby Vision
enhancement layer gets the brand string "Dolby Vision EL"; every other
HDR format on a secondary stream gets no label at all (that wording is a
CLI concern). Mutation: delete the `HdrFormat::DolbyVision` arm so it
falls through to the `_ => String::new()` catch-all, losing the brand.
## test: `generate_video_label_480_boundary`
Spec: 480 lines is the SD floor — a stream with height exactly 480 must
get the "480p"/"480i" token (BD spec height boundary), not fall through
to the empty-resolution case. Mutation: `h >= 480` -> `h < 480` inverts
the boundary so a legitimate 480-line stream (h == 480) produces no
resolution token at all.
## test: `generate_video_label_sdr_produces_no_hdr_token`
Spec: SDR is the unmarked default — it must never appear as a token in
the generated label (only non-SDR formats get an explicit tag).
Mutation: delete the `HdrFormat::Sdr` arm so it falls through to
`_ => parts.push(hdr.name())`, appending a spurious "SDR" token.
## test: `generate_audio_label_atmos_folds_brand`
Spec: the Atmos-aware variant folds "Atmos" into the codec brand name
for TrueHD/DD+ carriers, distinct from the plain wrapper. Mutation: stub
the whole function to `String::new()` / a constant literal — either way
it stops reflecting the codec/channel inputs.
## test: `generate_audio_label_covers_pc_container_codecs`
Spec: every disc-audio codec in the enum has a full marketing name,
including the lossy PC-container codecs (AAC/MP2/MP3/FLAC/Opus) that
`generate_audio_label_all_codecs` above doesn't cover. Mutation: delete
any one of these match arms — the codec falls through to
`_ => return String::new()`, silently losing its label.
## `codec_hint_consistent`: chained-OR boundary hardening
The family-detection booleans chain `h.contains(..) || ...` synonym
checks. Each test below isolates ONE clause so a `||` -> `&&` flip there
changes the verdict.
### `codec_hint_consistent_truehd_space_synonym`
Isolates the `"true hd"` (space form) synonym in `says_truehd`, which
mutant testing hit at 396:44's `||`. If that `||` is weakened to `&&`,
"True HD" alone (no "truehd" substring) no longer sets `says_truehd`,
`names_family` goes false entirely (no other family clause matches), and
the function takes the "no family named" early-return path — turning a
should-be-`false` verdict for a mismatched codec into `true`.
### `codec_hint_consistent_ddp_ac3_plus_no_hyphen_synonym`
Isolates the `"ac3+"` (no-hyphen) synonym in `says_ddp` (398:9's `||`).
A hint matching only this clause must still classify as DD+, not fall
through to the plain-AC3 `says_ac3` check.
### `codec_hint_consistent_ddp_eac3_synonym`
Isolates the `"eac3"` synonym in `says_ddp` (401:9's `||`), the last
clause before the chain moves to "digital plus"/"dd+".
### `codec_hint_consistent_lpcm_bare_pcm_synonym`
Isolates the `"pcm"` (no "lpcm") synonym in `says_lpcm` (409:40's `||`).
A bare "PCM" hint on a non-LPCM stream must still be judged
inconsistent — if the `||` were `&&`, "PCM" alone would fail to set
`says_lpcm`, `names_family` would go false, and the function would take
the "no family named" path, wrongly returning `true` for ANY codec.
### `codec_hint_consistent_names_family_dts_ma_alone`
Isolates the `says_dts_ma || says_dts_hr` disjunction inside the
`names_family` chain (418:60). A hint that sets `says_dts_ma` alone
(e.g. "Master Audio", without "hd ma") must still make `names_family`
true; weakening that `||` to `&&` requires both clauses at once, so
`names_family` goes false and the function wrongly reports "consistent"
for a codec the hint never named.
### `codec_hint_consistent_truehd_arm_atmos_alone`
Isolates the `Codec::TrueHd => says_truehd || says_atmos` arm (433:38).
An Atmos-tagged hint that names a DIFFERENT lossless carrier by name
(DD+) must still be judged consistent with a TrueHd stream purely on the
Atmos marker — `||` -> `&&` would require the hint to ALSO say "truehd",
which an Atmos-only marker doesn't.
### `codec_hint_consistent_dts_arm_not_bypassed`
Spec: `Codec::Dts` is consistent ONLY when the hint's DTS-family
bookkeeping (`says_dts`) is true, not just because `names_family` is
true via some other carrier. Mutation: delete the `Codec::Dts =>
says_dts` arm (438:9) — it falls to `_ => true`, so ANY named family is
(wrongly) "consistent" with a Dts stream.
### `codec_hint_consistent_lpcm_arm_not_bypassed`
Spec: `Codec::Lpcm` is consistent ONLY when `says_lpcm` is true.
Mutation: delete the `Codec::Lpcm => says_lpcm` arm (439:9) — same
bypass-to-`_ => true` failure mode as the Dts arm above.
## test: `slot_lookup_never_returns_a_label_that_names_its_own_stream`
Spec: a label that names its own stream is never reachable through the
slot lookup, whatever number it carries.
The two numbers are different coordinate systems. A derived label's
`stream_number` is its slot in ITS OWN playlist's table; the vendor's is
a slot in the one table the config blob describes. `label_at` answers
questions about the second, so it must not return the first.
Mutation: drop the `l.stream_id.is_none()` term from `label_at` — the
derived label at slot 1 is found first and shadows the vendor's.
## test: `the_derived_floor_cannot_veto_the_anchor`
Spec: the derived floor does not vote on which title anchors the vendor
list — and in particular cannot VETO the title that does.
The vendor list names two subtitle slots, eng then fra, and the feature
reproduces them. The feature has a third subtitle the vendor said
nothing about; the merge dropped a derived label into "slot 3", numbered
in a different playlist's coordinate system, and it says Spanish. Read
as part of the vendor sequence that is a contradiction, so the feature
is rejected, no title anchors, and every flag the list carries is lost.
This is the measured cost of a whole-sequence gate without provenance:
the vendor's own slots stay correctly positioned while the merged ones
break the sequence between them.
Mutation: drop the `l.stream_id.is_none()` term from `label_at` — the
derived "spa" is read as vendor slot 3, `anchor_score` returns `None`,
and there is no anchor.
## test: `the_anchor_is_the_title_that_confirms_most_of_the_list`
Spec: among titles the vendor list admits, the one that CONFIRMS more of
it wins — a slot where both sides state a language and state the same
one is evidence; a slot where either is silent is not.
Here the shorter title matches both named slots outright while the
longer one states no language at all, so it merely fails to contradict.
Size alone would hand the anchor to the title that proved nothing.
Mutation: rank on `n` alone (drop `score` from the comparison) — the
three-stream title with no languages wins.
## test: `a_vendor_qualifier_does_not_leak_onto_a_featurettes_own_stream`
Spec: an editorial qualifier the vendor list states for ONE stream table
does not leak onto a different physical stream in a title that merely
counts to the same ordinal.
Measured shape: the feature carries eleven subtitles whose first is SDH;
a dozen featurettes each carry a single English subtitle, on a different
clip and a different PID. Both are "subtitle number 1", so the ordinal
fallback put SDH on all of them — and the languages agree, so the
language gate could not catch it. What separates them is that the
featurette's stream is named by the derived floor, and a label that
names a stream outranks a label guessed onto it.
Mutation: move the `by_id` lookup in `resolve` after the ordinal
fallback — the guess wins again and the featurette is SDH.
## test: `an_anchor_proves_pids_only_for_the_clip_its_table_came_from`
Spec: the anchor proves a `(clip, PID)` fact only for the clip its
stream table was READ FROM — the first play item — never for every clip
the anchor happens to play.
`disc::bluray` builds a title's stream list from `play_items[0]`'s STN
table, and tier 3 fifty lines below says exactly that by keying its
derived ids on `clip0`. Tier 2's harvest contradicted it: it recorded
the anchor's slot PIDs against EVERY clip the anchor plays, so a sibling
playlist that plays a LATER clip of the anchor bound the anchor's
editorial label onto whatever stream in that clip happens to reuse the
PID — a different physical stream, in a different clip, whose own
language says so.
Mutation: harvest over `&title.clips` instead of its first clip — the
featurette wears the feature's SDH again.
## test: `a_vendor_codec_claim_does_not_follow_the_ordinal_onto_a_bonus_clip`
Spec: a vendor codec/variant claim does not follow the ordinal onto a
bonus clip that carries a different codec.
Measured shape: the feature's first audio is object audio; a dozen menu
and bonus titles each carry one plain stereo track. All of them are
"audio number 1", and the hint names the same codec family as the stream
it lands on, so the consistency guard passes it through and every bonus
clip advertises the feature's format.
Mutation: as above — move `by_id` after the ordinal fallback.
## test: `a_title_shorter_than_the_vendor_list_cannot_anchor_it`
Spec: a title whose stream count is smaller than the vendor list's
highest slot cannot be the table that list describes.
Mutation: drop the `n < extent` term from `find_anchor` — the one-stream
title becomes eligible, and on a disc where it sorts first it takes the
anchor away from the title that actually has the slots.
## test: `an_unnamed_slot_does_not_disqualify_a_title`
Spec: a slot the vendor list never names constrains nothing.
These blobs under-yield by design — the authoring layer ships editorial
labels for the streams it finds interesting and leaves the rest as bare
slots. Treating a hole as a disagreement rejects the very title the list
describes. (Before provenance the holes were filled with labels numbered
in an unrelated coordinate system, so the gate neither rejected nor
admitted on evidence.)
Mutation: make `anchor_score` return `None` for an unnamed slot — no
title anchors and the forced flag is never delivered.
## test: `fill_gaps_leaves_order_untouched_when_nothing_added`
Spec: the sort-by-(type, number) pass only runs when the merge actually
added something (`added > 0`); when MPLS contributed nothing new,
`framework`'s existing order (however the caller built it) must be left
untouched. Mutation: `added > 0` -> `added >= 0` is always true, so the
sort runs unconditionally, silently reordering a framework list that
wasn't already in (type, number) order even on a no-op merge.
## `build_program_info` / `build_clpi` (test fixtures)
`build_program_info`: builds a CLPI ProgramInfo section for one program
with the given (pid, stream_coding_info) pairs. Layout mirrors
`crate::clpi::parse_program_info`'s expectations: length(4) + reserved(1)
+ num_programs(1), then per-program spn(4)+pmt_pid(2)+num_streams(1)
+num_groups(1), then per-stream pid(2)+sci_len(1)+sci.
`build_clpi`: builds a full CLPI byte buffer (HDMV header + ProgramInfo)
declaring the given (pid, coding_type, lang) streams. `sci` layout
follows `crate::clpi::parse_program_info`'s per-coding-type match arms:
PG/IG = coding_type + 3-byte lang; audio (primary or secondary) =
coding_type + format/rate byte + 3-byte lang.
## test: `orphans_state_no_stn_slot_and_name_their_stream`
(d) An orphan states NO STN slot, and is identified by the stream it was
read from instead.
This test used to assert the opposite — that orphans continue the slot
numbering from `max(existing) + 1`. That number was invented here and
shared with the coordinate system `label_at` counts vendor slots in, so
a title with more streams of a type than the vendor list had slots could
reach an orphan by counting and be labelled from a stream no playlist
plays. An orphan is by definition in no playlist, hence in no title, so
there is no ordinal to give it.
## `codec_hint_consistent`
Does the parser's `codec_hint` name a codec consistent with the stream's
actual `codec`? `apply_labels` uses this to keep richer-but-consistent
hints (e.g. "Dolby Atmos" on a TrueHD stream — Atmos is a TrueHD extension
the raw spec codec can't express) while rejecting mis-bound ones (e.g.
"AC-3 2.0" on a TrueHD stream, the shuffled-label bug). Matching is by codec
FAMILY parsed out of the hint string. "Atmos" with no carrier named is
treated as compatible with its lossless carriers (TrueHD / E-AC-3). A hint
naming no recognizable codec family (pure editorial, e.g. "Commentary") is
consistent — it isn't asserting a codec.
## `generate_audio_label_atmos`
Atmos-aware variant: same codec/channel string as `generate_audio_label`
with the object-audio marker folded into the codec brand (e.g. "Dolby
TrueHD Atmos 7.1"). The "Atmos" string lives here in the label layer, not
in the core parser. Used when a bitstream probe detected an Atmos
substream and the stream still carries the basic (non-editorial) label.
+26
View File
@@ -0,0 +1,26 @@
# src/labels/xml.rs
Tolerant XML scraping helpers — promoted from two near-duplicate hand-rolls
in `paramount.rs` (attribute extraction) and `criterion.rs` (tag-text
extraction).
These are NOT a full XML parser. They handle the subset of XML the BD-J
authoring tools we've seen actually emit: ASCII tag/attr names, no entity
references inside label strings, optional XML namespaces. Hardening goals
over the prior `find("<tag>")` / `find(r#"name=""#)` matchers:
1. **Case-insensitive** tag and attribute names — vendors casing is
inconsistent across authoring-tool revisions.
2. **Namespace-aware** — strip an optional `ns:` prefix so `<ns:playlist>`
and `<playlist>` both match.
3. **Whitespace-tolerant** — multiple/tab/newline characters around `=`
between attribute name and value; whitespace inside the opening tag.
4. **Quote-style tolerant** — both `"value"` and `'value'`.
5. **Self-closing tag handling** — `<tag />` and `<tag/>` both work;
`text()` returns `Some("")` for empty content.
Out of scope (intentionally simple): XML entity decoding (`&amp;`, `&lt;`,
etc.), CDATA sections, comments, processing instructions, DTD
declarations. None of the BD-J authored disc data we've observed
exercises any of those — labels are plain ASCII/Latin-1 in attribute
values.
+166
View File
@@ -0,0 +1,166 @@
# CSS content cipher (`src/css/lfsr.rs`)
## Module overview
The algorithm is the publicly documented CSS stream cipher, described in the
open CSS cryptanalysis literature. It is implemented here from that public
description; its constants (see [`super::tables`]) are the cipher's own
defined values. Nothing in this file is copied or translated from any
particular CSS software.
The cipher uses two table-driven linear-feedback circuits:
- **LFSR1** — a 17-bit register (a 9-bit and an 8-bit half seeded from
`key[0..2] XOR seed[0..2]`), stepped through `TAB2`/`TAB3`/`TAB5`.
- **LFSR0** — a 24-bit feedback register (seeded from `key[2..5] XOR
seed[2..5]`), stepped through a feedback polynomial and `TAB4`.
Each output byte is the sum-with-carry of the two register outputs. A body
byte is recovered as `plain = TAB1[cipher] ^ keystream` — a `TAB1`
substitution of the ciphertext byte followed by an XOR with the keystream
(so the cipher is deliberately not its own inverse).
## `descramble_sector`
The two feedback registers are seeded **directly** from
`title_key XOR sector_seed` (bytes `0x54..0x59`) — there is no title-key
mangling on the content path (that belongs to the disc/title-key hierarchy,
not the sector cipher). Only the body, bytes `0x80..0x800`, is transformed:
`body[i] = TAB1[body[i]] ^ (keystream & 0xff)`.
The scramble flag at byte `0x14` (bits 4-5) marks an encrypted sector. This
routine CLEARS that flag after unscrambling, so a descrambled sector reads as
`sector[0x14] & 0x30 == 0`; callers and tests use that to tell it from
ciphertext, and re-running descramble on an already-cleared sector is a no-op
(the flag guard below skips it). Clearing does not affect the recovered body.
`sector.len() < 2048` (the encrypted region `0x80..0x800` not fully present):
callers chunk by 2048, so a trailing partial chunk is left untouched. The
`debug_assert!` flags this misuse in debug/test builds; a DVD sector is
always exactly 2048 bytes.
## `scramble_sector` (test helper)
Exact inverse of `descramble_sector`: turn a plaintext sector body into
CSS ciphertext under `title_key`.
Descramble computes `plain = TAB1[cipher] ^ (keystream & 0xff)`, so the
inverse is `cipher = TAB1_INV[plain ^ (keystream & 0xff)]` with the identical
keystream. The keystream derivation is the same as `descramble_sector`;
only the final substitution differs. Bytes `0x80..0x800` are rewritten in
place; the scramble flag is set to `0x10` so a subsequent descramble runs.
Not on any production read path — it exists so the key-recovery tests (and
any caller that needs a known CSS-encrypted sector) can build genuine
ciphertext rather than approximating it.
## Test notes
### `descramble_produces_the_reference_css_vector`
Regression vector: the deterministic output of the CSS content cipher for
a fixed key/seed/body. The value is generated by this implementation and
is self-consistent with the scramble/descramble round-trip test — any
correct CSS descrambler yields the same bytes, since the cipher is
deterministic. Pins the implementation against accidental change.
key = 42 13 37 BE EF, seed (0x54..0x59) = DE AD BE EF 42, body = 0xAA.
### `css_descramble_inverts_scramble_over_body`
The content cipher is NOT a plain XOR involution (it applies TAB1 to the
ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
`scramble_sector`. Scrambling a plaintext body and then descrambling
with the same key must reproduce the original body exactly.
### `css_tab1_is_permutation`
Verify the structure of TAB1: it is a substitution table used in key
mangling. Check that no two inputs map to the same output (TAB1 is a
permutation of 0..255).
### `descramble_treats_high_bits_of_0x14_as_clear`
Only bits 4-5 of byte 0x14 are the CSS scramble flag: the code reads
`sector[0x14] & 0x30 == 0` (bits 6-7, i.e. 0x40/0x80, are masked out by
0x30). A sector with 0x14 == 0x40 or 0x80 must therefore be treated as
UNSCRAMBLED and left byte-for-byte unchanged. This guards against a
too-wide mask silently "descrambling" (and thus corrupting) clear data.
Grounding: CSS sector header byte 0x14 — copyright/scramble bits live in
bits 4-5; the masked value 0 means not scrambled.
Mutation: widen the mask `0x30` to `0x70`/`0xF0` -> 0x40/0x80 would be
seen as scrambled and the body would change.
### `descramble_triggers_on_either_flag_bit`
Each individual scramble bit (4 and 5) independently marks the sector as
encrypted: 0x10 and 0x20 must both trigger descrambling.
Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both nonzero.
Mutation: change `!= 0` early-return condition to `== 3` -> a sector
flagged only 0x10 or 0x20 would be skipped and left scrambled.
### `descramble_clear_preserves_high_bits_of_0x14`
After descrambling, ONLY the two scramble bits are cleared (`& 0xCF`);
bits 6 and 7 of byte 0x14 must be preserved. A sector with 0x14 == 0xF0
becomes 0xC0 (bits 6,7 kept, bits 4,5 cleared), NOT 0x00.
Grounding: code does `sector[0x14] &= 0xCF`; 0xF0 & 0xCF == 0xC0.
Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved high
bits assert fails.
### `descramble_leaves_header_and_seed_intact`
The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80 (the
header) must NOT be modified by the keystream — except byte 0x14 whose
flag is cleared. In particular the sector-seed bytes 0x54..0x59 (which
live inside the header) must survive untouched, since the descrambler
reads them but never writes them.
Grounding: loop is `sector.iter_mut().take(2048).skip(128)` -> indices
128..2048 only.
Mutation: change `.skip(128)` to `.skip(0)` -> header bytes (incl. the
seed) get XORed and this fails.
### `descramble_covers_final_body_byte`
The descrambler must touch the WHOLE body 0x80..0x800, not just a prefix.
With a constant body and constant key, the keystream is non-degenerate
enough that the very last sector byte (index 2047) is altered. This guards
the loop bound `.take(2048)` against an off-by-one that would leave the
final byte(s) scrambled.
Grounding: encrypted region end is 0x800 == 2048 (exclusive).
Mutation: change `.take(2048)` to `.take(2047)` -> last byte unchanged,
assert fires (this body is all-zero so any keystream XOR shows).
### `descramble_processes_the_first_sector_of_an_over_long_buffer`
The length guard is a FLOOR, not a ceiling: `descramble_sector` is a
no-op below one sector, and processes the FIRST sector of anything at
least that long (the loop is `.take(2048)`). `css::descramble_sector` is
a public entry taking `&mut [u8]` of any length, so a caller handing it a
multi-sector buffer must get its first sector descrambled — a guard that
rejected over-long buffers would hand that caller its ciphertext back
unchanged, with the scramble flag cleared as if it had worked.
### `descramble_output_depends_on_title_key`
Descramble is keyed by `title_key XOR seed`: two different title keys
produce two different bodies for the same scrambled input. A cipher that
ignored the title key (or mixed it in wrongly) would yield identical
output — silent wrong-key decryption.
Grounding: per-sector key = title_key[i] ^ sector[0x54+i].
Mutation: in the `key` array drop the `title_key[i] ^` term -> both keys
give the same body, assert fires.
### `descramble_output_depends_on_seed`
Descramble is keyed by the sector seed too: same title key, different
seed -> different body. Pins that bytes 0x54..0x59 actually feed the
keystream (not just the per-sector XOR key).
Mutation: replace `seed` array reads with a constant -> both seeds give
the same body, assert fires.
+53
View File
@@ -0,0 +1,53 @@
# libfreemkv Crate Overview
Overflow detail for the `src/lib.rs` crate-level doc comment, which is capped
by `ci/comment-guard.py`. Pointed to by the `//!` header there.
## Unlocking
Unlocking -- removing bus encryption (firmware unlock, AACS cert handshake,
CSS bus-auth) -- lives entirely in the `freemkv-unlock` crate; libfreemkv
consumes it privately and exposes none of it, so clients are oblivious to
unlockers (just as they are to the SCSI layer).
## Why a live `disc://` can't go through `input()`
A live `disc://` cannot be opened via [`input`] -- it returns
[`Error::DiscUrlNotDirect`] by design (use `Drive` + `Disc::scan` +
`DiscStream::new` directly for a live drive). Any file-backed source
(`iso://`, `m2ts://`) opens through [`input`].
## Architecture
```text
Drive -- open, identify, unlock, read sectors
├── ScsiTransport -- SG_IO (Linux), IOKit (macOS)
├── DriveId -- INQUIRY + GET_CONFIG identification
└── unlock_bridge -- private seam to the `freemkv-unlock` crate
(firmware / AACS cert / CSS bus-auth unlockers)
Disc -- scan titles, streams, AACS state
├── UDF reader -- Blu-ray UDF 2.50 with metadata partitions
├── MPLS parser -- playlists → titles + clips + STN streams
├── CLPI parser -- clip info → EP map → sector extents
├── JAR parser -- BD-J audio track labels
└── AACS -- encryption: key resolution + content decrypt
├── aacs -- KEYDB, VUK, MKB, unit decrypt
└── host_certs -- collect host certs (cert handshake lives in freemkv-unlock)
```
See also `docs/architecture.md` for the fuller design-level writeup.
## AACS Encryption
Disc scanning automatically detects and handles AACS encryption.
If a KEYDB.cfg is available (via `ScanOptions` or standard paths),
the library resolves keys and decrypts content transparently.
Supports AACS 1.0 (Blu-ray) and AACS 2.0 (UHD, with fallback).
## Error Codes
All errors are structured with numeric codes. No user-facing English
text -- applications format their own messages. See `docs/error-codes.md`
for the full range table.
+34
View File
@@ -0,0 +1,34 @@
# `LocalFileSink`
`BufWriter<File>`-backed sink for the common local-disk case
(`src/io/sink/local_file.rs`).
## Buffering
4 MiB internal `BufWriter`. Sized to coalesce the small per-PES writes
that come out of the muxer into kernel-page-aligned flushes without
making the buffer big enough to matter for memory pressure on a single
concurrent rip.
## Seek-flush ordering
`Seek` flushes the underlying `BufWriter` first; otherwise a seek could
leapfrog buffered data and silently corrupt the file. This is the same
shape `BufWriter` itself uses when it impls `Seek` in stdlib, and is
necessary for MKV's seek-back operations (cluster size patch, Cues
index, segment header backpatch) to land on the right offset.
## Trait wiring
[`SequentialSink`](../src/io/sink/mod.rs) is implemented explicitly
(not via a blanket impl) so its `finish()` flushes the `BufWriter` and
`fsync`s the file even when called through a `dyn` trait object;
`RandomAccessSink` is implemented over the `Seek` impl.
## Construction contract
Construction always opens the file `create + truncate + read + write`.
`read` is enabled so the same handle can be reused for a verification
re-read after the mux (the existing `FileSectorSink::create` pattern).
On Linux, `with_size_hint` additionally calls
`fallocate(FALLOC_FL_KEEP_SIZE)` to pre-reserve extents.
+29
View File
@@ -0,0 +1,29 @@
# BD/DVD LPCM wire format
BD LPCM PES packets (TS stream type 0x80) carry a 4-byte header on the
elementary-stream payload:
```
Bytes 0-1: audio frame number
Byte 2: reserved
Byte 3: quantization (bits 7-6), sample rate (bits 5-4), channel assignment (bits 3-0)
```
This header is part of the ES payload, so the BD parser must strip it.
DVD LPCM lives in private stream 1 (sub-stream 0xA0-0xA7). Its 7-byte
private sub-header (sub_id + frames + first-access-unit-ptr(2) + emphasis +
quant/freq + channels) is stripped by `PsDemuxer` while demuxing the
Program Stream. By the time a DVD LPCM `PesPacket` reaches this parser its
`data` is already raw PCM, so the parser must NOT strip any further bytes —
doing so drops one sample pair per PES and drifts the audio.
The two origins are distinguished by the `strip_header` flag: BD = strip,
DVD = leave intact. The raw PCM data is otherwise one complete audio frame
per PES; no framing is needed.
For MKV: both BD and DVD LPCM map to codec ID "A_PCM/INT/BIG" (big-endian).
DVD-Video LPCM is big-endian per the DVD-Video spec, and `mkv.rs` emits
"A_PCM/INT/BIG" unconditionally for `Codec::Lpcm` — there is no DVD/BD branch
and no "A_PCM/INT/LIT" path, so no byte-swap or alternate codec ID applies.
All frames are keyframes (uncompressed audio).

Some files were not shown because too many files have changed in this diff Show More