Compare commits

...
84 Commits
Author SHA1 Message Date
Matthew Jackson 477c0da6dd v1.6.12: bump version (freemkv-unlock git-pinned for the tag) 2026-08-27 22:57:29 +00:00
Matthew Jackson 41c5b8b785 changelog: 1.6.12 (unreleased) 2026-08-27 13:11:27 -07:00
Matthew Jackson 9f43ae94d3 Comment and Documentation Cleanup
- Cap every inline `//` comment block at 3 lines; distil the rationale
  rather than delete it. Enforced by a new comment-guard CI check.
- Fix stale and orphaned rustdoc: broken intra-doc links (renamed or
  removed items), doc blocks that had merged onto the wrong item (leaving
  the intended item undocumented), and doc-contradicts-code drift.
- Add a doc-links CI check (cargo doc, deny broken intra-doc links) so
  both classes of comment rot are caught going forward.
2026-08-27 12:07:53 -07:00
Matthew Jackson 9ed6f86255 Fix macOS optical E1000: hold a DiskArbitration claim for the session
The macOS SCSI transport took exclusive SCSI access plus a one-shot
`diskutil unmountDisk`, but never claimed the disc from DiskArbitration.
diskarbitrationd was therefore free to remount the volume mid-rip (a
Spotlight re-probe is the usual trigger), yanking the device out from
under an in-progress read; the failure surfaced to the user as E1000.

Hold a DADiskClaim plus a mount-approval dissenter scoped to our disc for
the transport lifetime, so the OS cannot remount it until shim_close().
Link the DiskArbitration framework in build.rs.
2026-08-27 11:17:01 -07:00
Matthew Jackson 70ae7b6c10 restore freemkv-unlock path dep for local dev (post-v1.6.11) 2026-08-25 23:17:11 -07:00
Matthew Jackson 3f03c6db06 v1.6.11: bump version (freemkv-unlock git-pinned for the tag) 2026-08-25 23:17:07 -07:00
Matthew Jackson 562bb8bee5 docs: neutralize tool-comparison and technique wording (opsec) 2026-08-25 22:57:01 -07:00
Matthew Jackson 7c13ed32b5 fix: align drive-prep unlocker token with unlock rename; neutralize product name 2026-08-25 22:54:38 -07:00
Matthew Jackson 574aa2c506 docs: neutralize retail title references (opsec) 2026-08-25 22:45:57 -07:00
Matthew Jackson 3912762b52 docs: neutralize comment provenance and external references (opsec) 2026-08-25 22:31:19 -07:00
Matthew Jackson 028768e71b docs: 1.6.11 changelog 2026-08-25 22:08:48 -07:00
Matthew Jackson bf4e5ac395 docs: standardize README badges (License, CI, codecov) 2026-08-25 20:26:28 -07:00
Matthew Jackson 11948d513b nav: taint control-transfer decisions only, and track SPRM-laundered GPRMs
Refine the First-Play SPRM-taint abstain to fix two defects:

- Over-abstention: a tainted compare that guards a SetGPRM store (not a
  control transfer) no longer forces abstention. The predicate only
  decides whether the store runs; a following unconditional dispatch is
  still resolvable. Taint now gates only JumpTT/Goto (the control
  transfers the resolver follows); a store under a tainted guard taints
  its destination GPRM (unknown post-state) instead of abstaining.

- Under-abstention: an SPRM laundered through a GPRM (SetGPRM g = SPRM,
  then a later GPRM-only compare on g) previously evaded taint and let
  the resolver commit to the cold-machine arm. Add per-GPRM taint bits:
  a store propagates its source's taint (SPRM or tainted GPRM) and a
  concrete/immediate store clears it; eval marks a compare tainted when
  it reads a tainted GPRM.

Adds red-before-green tests for both plus taint-clear guards.
2026-08-25 20:26:03 -07:00
Matthew Jackson 7d5c0a5c41 Fix round-3 audit findings: VM operand decode, MPLS field offset, log escaping, nav SPRM abstain, tests
1. dvdnav/vmcmd: the register operand of the if_v1/if_v3 compare layouts is the
   LOW byte of the reg-or-immediate field (b[5] / b[7]), not the high byte
   (b[4] / b[6]). Fix both layouts and their comment; add register-mode KATs
   with distinct high/low bytes.

2. mpls: connection_condition is the low nibble of PlayItem byte[10]
   (byte[9] is fully reserved), corroborated by the byte-11 stc_id and
   byte-12 in_time reads. Relocate every fixture's encoded value and update the
   regression test to assert the reserved byte does not leak.

3. labels/deluxe: escape the two disc-derived class-name tracing fields via the
   Debug path, matching the other log sites.

4. disc/dvd: add an integration test that drives scan_dvd_titles with a resolved
   First-Play nav target so the 1-based promotion join is verified end-to-end.

5. bdnav/vm: add unit tests for the SET/SWAP immediate gate — full register
   exchange, and that an immediate-flagged operand is never a write target.

6. dvdnav/nav: honor the documented contract — a First-Play compare that reads a
   system parameter is undecidable at scan time, so eval now taints on SPRM reads
   and the executor abstains rather than committing to the cold-power-on arm.
   GPRM reads never taint.

7. labels/deluxe: narrow four now-internal helpers from pub(crate) to private.
2026-08-25 20:26:03 -07:00
Matthew Jackson 1ede962aec ci(coverage): authenticate Codecov upload with org CODECOV_TOKEN 2026-08-25 19:32:30 -07:00
Matthew Jackson 014ea870a4 ci(coverage): add --ignore-run-fail so instrumentation flake never blocks upload 2026-08-25 19:04:41 -07:00
Matthew Jackson bdd2876d78 ci: add non-required Codecov coverage job, config, and badge
Add an additive coverage job to CI that generates lcov via cargo llvm-cov
and uploads to Codecov (tokenless). Mirrors the test job's pinned 1.94
toolchain and dual freemkv-unlock checkout for the path dep. Informational
only: no gate depends on it. Adds codecov.yml and a README badge.
2026-08-25 18:54:23 -07:00
Matthew Jackson 44de7a8d12 harness: fuzz the Deluxe bytecode decoder directly, not through a dead zip open
Audit round-2 (fix-adversary): the round-1 deluxe sweep fed the zip local-file-
header magic, but ZipArchive::new validates the end-of-central-directory at the
buffer's end — which the byte generators (front magic + noise) can never
synthesize — so the archive open always failed and decode_binding was never
reached: false coverage for the very bytecode-decoder panic path it guarded.
Drive decode_binding_class directly on a parsed .class with an empty master-enum
table instead, so the symbolic stack machine is actually exercised.
2026-08-25 18:45:28 -07:00
Matthew Jackson 552076a943 disc/labels/harness: audit round-1 batch — DoS bound, fuzz coverage, API surface
Continuing the /codeaudit HEAD..v1.6.10 round:

- disc/mod.rs rank_titles: bound the O(n^2 * clips) composite scan to a
  sub-second work budget (both title count and per-title clip count are
  untrusted disc metadata); above it, skip composite classification and degrade
  to the has-video + canonical order. Reorder the per-pair checks so the O(len)
  is_subset runs only after the cheap video/size gates. (regression test)
- bdnav/vm.rs SWAP: honour the imm_op1/imm_op2 store gates so a SWAP to an
  immediate operand does not corrupt a register.
- harness.rs: fuzz the previously-uncovered new parsers — the Deluxe bytecode
  decoder, the Fox dcx.xml scanner, and the combined bdnav index+mobj+VM resolve
  path (labels::{deluxe,fox} and bdnav::vm made pub(crate) for in-crate fuzzing).
- dvdnav: pub(crate) the module + submodules (was pub) to match bdnav and stop
  leaking the DVD IFO/VM internals as public API; drop an unused re-export.
- labels: escape disc-derived strings in diagnostics (Debug not Display) so a
  hostile disc cannot inject terminal control sequences into logs.
2026-08-25 18:33:37 -07:00
Matthew Jackson 3ecbeb7eba disc/bdnav: audit round-1 — VM title/PSR faithfulness + main_title never-panics
From the /codeaudit HEAD..v1.6.10 round (each verified against the libbluray
HDMV VM reference, and design-refuted SOUND on two models):

- bdnav/vm.rs title_obj: resolve First Play via the spec sentinel 0xFFFF (was
  num_titles+1). A JumpTitle 0xFFFF now reaches First Play, and an out-of-range
  JumpTitle(N+1) correctly abstains instead of mis-jumping there.
- bdnav/vm.rs wr: refuse stores to PSR-tagged registers (matches _store_reg),
  so an illegal SET/SWAP to a PSR is a no-op and later compares agree with a
  real player instead of diverging.
- disc/profile.rs main_title(): return Option<&TitleProfile> instead of indexing
  titles[0], which panicked on a disc that scanned to zero titles.

Regression tests added, each shown red-before-green.
2026-08-25 18:16:17 -07:00
Matthew Jackson b405154fd0 ci: run the consumers job at the highest dependent MSRV (1.98)
The MSRV split raised the dependents above libfreemkv's own floor — the
bins (freemkv, autorip, bdemu) are 1.98 and freemkv-keysources is 1.97 —
so the consumers job could no longer build them at libfreemkv's 1.94
toolchain (`rustc 1.94.0 is not supported by ... freemkv-keysources
requires rustc 1.97`), leaving dev red since the MSRV change. Pin that one
job to 1.98, which satisfies every dependent's floor. libfreemkv's own
lib/lint/test jobs stay at its 1.94 MSRV.
2026-08-25 13:49:35 -07:00
Matthew Jackson 8a630c8839 disc: payload-plausibility floor on nav/authoring feature promotion
A nav- or authoring-resolved playlist could promote a video-bearing but
negligible-payload branch above the substantial feature it sits beside:
Spotlight (2015) UHD navigates to a Dolby-Vision seamless-branch playlist
(00001, 0.4 GB across 102 tiny clips) instead of the 91.5 GB single-clip
feature (00002). A player plays the branch; a rip wants the feature.

Require a nav/authoring-promoted title to also carry at least 10% of the
largest video-bearing title's bytes; otherwise selection falls through to
the composite/size ranking, which recovers the real feature. Adds the
Spotlight-shaped regression test.

Verified against the 88-disc hoard: title selection is identical to the
prior release on every disc except the one it fixes (Spider-Man 3 UHD,
00245 wrapper decoy -> 00001 feature).

Also clears three 1.97 clippy lints in the HDMV VM and drops two
reference-implementation filename citations from the bdnav module docs.
2026-08-25 13:44:04 -07:00
Matthew Jackson 6c00090308 labels/fox: wire the extracted feature-playlist id into the selection hint 2026-08-25 13:01:25 -07:00
Matthew Jackson 7f9f5a9d3e Merge branch 'label-fox' into integrate-1.6.11 2026-08-25 13:00:06 -07:00
Matthew Jackson b3c23c19ee Merge branch 'label-deluxe' into integrate-1.6.11 2026-08-25 12:59:54 -07:00
Matthew Jackson f3923cf587 Merge branch 'dvd-nav' into integrate-1.6.11 2026-08-25 12:59:48 -07:00
Matthew Jackson 002912abb7 dvdnav: resolve DVD main feature by following First-Play navigation
Implement the DVD navigation feature resolver (#40): read the disc's own
First-Play program out of VMGI (VIDEO_TS.IFO) and execute it on a bounded
register machine to identify which title a player would land on, then prefer
that title in DVD main-feature selection.

- src/dvdnav/nav.rs: new module.
  * Parses the VMGI First-Play PGC command table (FP_PGC pointer at VMGI+0x84,
    PGC command-table pointer at PGC+0xE4, pre-command list) and TT_SRPT
    (title -> VTS/VTS_TTN, pointer at VMGI+0xC4).
  * Executes the pre-command list with the existing vmcmd decoder over 16 GPRMs
    + SPRMs from a cold machine: SetGPRM (mov/add/sub/mul/div/mod/and/or/xor),
    compare predicates, Goto, Break/Exit, and JumpTT (title dispatch). Menu /
    sub-domain jumps and intra-domain links are treated as undecidable.
  * resolve_from_vmg(&[u8]) is pure and total; resolve_main_title(reader, udf)
    reads the IFO and returns the resolved (title, vtsn, vts_ttn) or None.
  * Every offset/count is bounds-checked; a step budget bounds non-convergence.

- src/disc/dvd.rs: scan_dvd_titles now resolves the nav main title and returns
  its playlist_id alongside the titles; test call sites updated for the tuple.
- src/disc/mod.rs: the DVD nav pick feeds sort_titles_by_main_feature as
  nav_feature, the same seam the BD path uses, guarded to a video-bearing
  title. None -> today's leading-cell heuristic stands (strict improvement).
- src/harness.rs: add resolve_from_vmg to the never-panic sweep.

Tests: 11 nav unit tests (unconditional/conditional/set-then-dispatch convergence,
menu/empty/out-of-range/self-loop/truncation/overflow abstention) built to the
IFO spec using the real First-Play command bytes from the existing decoder KATs
(unconditional JumpTT 1; menu entry). Full suite green on 1.94.
2026-08-25 12:55:34 -07:00
Matthew Jackson b5fd193d54 disc: add normalized DiscProfile output surface
Add a thin, format-agnostic typed VIEW over the scanned Disc model so
downstream reads `profile.titles[i].subtitles[j].forced` (and analogous
audio/video flags) for any disc with no per-format conditionals.

New src/disc/profile.rs:
- DiscProfile / TitleProfile / VideoTrack / AudioTrack / SubtitleTrack
  (serde Serialize + Deserialize), re-exported from the disc module.
- DiscProfile::from_disc / From<&Disc>, Disc::profile(), and accessors
  (DiscProfile::main_title, TitleProfile::video/audio/subtitles).
- Splits each title's Stream enum into three typed vectors; flattens
  normalized fields (language default "und", forced/sdh from the subtitle
  qualifier, commentary/descriptive from the audio purpose, label -> name).
- Hoists derived flags so downstream never recomputes: main_title (the
  pre-sorted main feature at index 0), is_main, and the default
  audio/video track (first non-secondary, keep only the first), subtitle
  default always false.

Make the profile types the single serialization source of truth: the
json:// sink now serializes TitleProfile instead of hand-building JSON,
so json:// and any downstream share one schema. Stream arrays are split
by kind and qualifier/purpose decomposed into booleans; json://-only
extras (clips, full chapter_marks list) are spliced back on. meta_sink
tests reconciled to the new schema.

Tests: profile normalization across BdTs + MpegPs titles, typed-vector
split, populated fields, hoisted main/is_main/default flags,
qualifier/purpose mapping, and a serde round-trip.
2026-08-25 12:43:24 -07:00
Matthew Jackson 49011d577f labels/deluxe: decode real per-stream labels, emit High (Universal + others)
The Deluxe parser detected com/bydeluxe discs but emitted nothing, so
mpls_universal won at Low. Grounding the decoder on real Universal discs
(Fast Five / F9 / Bourne, studio="uni") surfaced three defects that
kept the full Phase A-D pipeline from ever resolving a stream on any
real disc, all fixed here:

1. Master enums were keyed by the ZIP ENTRY name ("pd.class") but the
   binding class references enum constants by JVM INTERNAL name
   (getstatic pd.<field>), so Phase C's getstatic count never matched
   the identified enums and found zero binding classes. Key master enums
   by this_class_name instead.
2. MasterEnumTable resolved getstatic operands by enum VALUE string
   ("English") but the operand is the obfuscated static-FIELD name
   ("a"), so every language/purpose ordinal came back unresolved.
   Capture the ordered putstatic field names (clinit_enum_field_names)
   and key the resolver on them.
3. The Language-enum fingerprint demanded 70+-4 values; Universal's
   enum has 65, so Phase A rejected it outright. Give each fingerprint
   its own count window (Language widened to span 65 and 70); the
   ordered 4+-string prefix stays the real discriminator.

Also: model anewarray/newarray so array construction keeps the symbolic
stack aligned, and skip constructors with an array parameter so the
Universal title-wrapper object (oq.<init>(...,[Lnp;,[Lwb;,...)) is not
mistaken for a subtitle binding. Recover SDH/RNIB from the language
display name (Universal encodes it as a distinct Language value, not a
Purpose). Add config.xml <TitleConfig studio="..."> detection. Raise
confidence to High when the binding table decodes >=1 stream; the
mpls_universal floor still back-fills un-named STN slots.

Fixtures: verbatim pd/lp/tl .class files from FastFive.iso decode to
English/French/Spanish + English Commentary (Dolby Digital) audio and
per-title eng/spa/fra subtitles. Validated end to end: labels-analyze on
FastFive/F9/Bourne now reports parser=deluxe High with real per-stream
languages, names, purposes and codecs instead of mpls_universal Low.
2026-08-25 12:25:38 -07:00
Matthew Jackson f8e39abe54 labels: harden class_reader/jar as first-class .class format readers
Treat Java bytecode as a documented format we READ (never execute), a
peer of mpls/clpi. The class reader's bounds checking was already sound
(typed Reader with checked u8/u16/u32/u64 + slice, saturating switch
arithmetic, iterator halt-on-truncation); this adds the untrusted-input
entry points the never-panic harness was missing and the malformed-input
regression tests the format warranted.

- harness: add class_reader (magic CAFEBABE) and jar (zip PK header)
  sweep targets. The class_reader target also drives the accessors a
  label parser exercises (this_class_name, member Code walk, cp_index,
  member_ref, load_constant_display) so their indexing is fuzzed, not
  just the structural parse. The jar target opens the sweep bytes as a
  zip and walks every .class through for_each_class exactly as deluxe/dbp
  do.
- class_reader tests: oversized constant_pool_count and interfaces_count
  reject with UnexpectedEof (u16 counts keep with_capacity bounded);
  out-of-range cp indices return None from every accessor; a Code
  attribute whose declared code_length runs past its body is rejected.
2026-08-25 12:25:19 -07:00
Matthew Jackson 65d719ee5d disc: wire bdnav as the top nav-feature tier in main-feature selection
scan_with runs the bdnav VM for BDMV discs and feeds the video-gated result
as the new top MAIN_FEATURE_ORDER key (nav-feature > authoring-feature >
standalone > has-video). A nav result is only honoured for a video-bearing
title, so a mis-resolve can never reopen the decoy hole; when the VM abstains
(BD-J, non-convergence) the structural/heuristic order decides exactly as
before. Adds nav selection tests and updates the diag decision reason.
2026-08-25 12:19:16 -07:00
Matthew Jackson c3bdbdcfc9 bdnav: faithful HDMV navigation VM (index.bdmv + MovieObject.bdmv)
Add src/bdnav: a libbluray-faithful, bounded, never-panic HDMV navigation VM
that runs First-Play with correct power-on PSR/GPR state to resolve the
playlist the disc itself plays as the feature. One generic VM handles the
obfuscated Sony dispatcher and every other HDMV disc with no vendor
special-casing; it abstains (None) on the BD-J boundary, malformed data, or
non-convergence. index/mobj parsers added to the never-panic harness.
2026-08-25 12:19:16 -07:00
Matthew Jackson 3f4e69f7b8 labels: add Fox dcx.xml vendor parser
Older Fox discs ship a loose plain-XML manifest at
/BDMV/JAR/<id>/dcx.xml (root <dcx>) that names every feature stream's
language, editorial purpose and forced/SDH state in attributes — fully
parseable and previously unowned (2 Fox discs matched no parser, 11 were
mis-owned by the deluxe stub).

New src/labels/fox.rs:
  - detect(): loose dcx.xml under /BDMV/JAR/*/ (cheap dir walk), falling
    back to a com/foxbd/ jar central-directory prefix so newer Fox discs
    are attributed to Fox even though their bytecode form isn't decoded
    yet (Phase 2, sketched in-file).
  - parse(): scopes extraction to one <playlist name="feature"> element
    (the richest, to avoid merging regional 00800/00801 STN tables),
    reusing labels/xml.rs. audio type=rnib -> Descriptive, comment* ->
    Commentary; subtitle form=sdh -> Sdh, type=embed -> Forced; nested
    id="NN" -> 1-based per-type STN slot. High confidence.
  - Feature playlist id recorded via tracing::debug for now; dev's
    ParseResult has no feature_playlist field (that FeaturePlaylistHint
    slot lives on selection-hardening). An in-code note documents wiring
    it at reconciliation, matching paramount::parse.

Registered ahead of mpls_universal, alongside the High-confidence vendor
parsers; registry-order lock test updated.

Tests embed the real Life of Pi dcx.xml (/BDMV/JAR/05001), asserting the
00800 feature selection + id, all 11 audio labels (incl. the eng rnib
descriptive track) and 11 subtitle labels (sdh/forced/none), the
no-cross-playlist-merge scope rule, plus negative-detect cases.
2026-08-25 12:09:37 -07:00
Matthew Jackson 89b03610e7 disc: demote play-all/wrapper composites in main-feature selection
The has-video gate alone fixed Spider-Man 3 UHD only by accident: the
00245 decoy = [bumper][00001 feature clip][outro], and streams come from
PlayItem 0 (the bumper) only, so it reported v=0. That is fragile — a
video-bearing composite [feature][outro] still beat the feature on size
(audit R2), and a real feature with a non-video first PlayItem would be
wrongly disqualified (R6).

Add structural clip-set composite detection: a title whose clip set
properly contains another substantial video title's whole clip set is a
play-all/wrapper composite and is demoted below the standalone title it
wraps — independent of stream counts, physical size, and disc capacity
(which is 0/unknown when READ CAPACITY fails, disabling the oversize
gate). Replace the has-video disqualification with the more-permissive
has_probable_video (real video, or size+duration+clips consistent with
video) so a bumper-lead feature is never disqualified while streamless
menus still are. Corroborate the authoring hint by duration (a hint at a
short branch/trailer can't override the feature) and require real video
streams for it. Precedence: authoring -> standalone-over-composite ->
has-video -> canonical (with a reserved top slot for the future BD-nav
key). Unify the session.rs key-sampling/verdict pickers and
image_crack_extents on has_probable_video.

Tests: video-bearing composite (T2), oversize composite with unknown
capacity (T3), short-branch hint not honoured (T4), bumper-lead feature
not disqualified (T6), all-streamless deterministic + single-source pick
(T7), equal-clip twins not composite, video-only title. T2/T3/T6 fail
under the prior has-video-only logic.
2026-08-25 11:54:04 -07:00
Matthew Jackson a7cbb3afcd disc: harden main-feature selection against streamless decoys
Main-feature selection ranked purely on physical size -> duration ->
audio (canonical_title_order), with no requirement that the chosen title
carry a decodable video stream. A playlist-obfuscation decoy can inflate
a streamless playlist past the real feature on size AND duration (Spider-
Man 3 UHD: 00245.mpls, 78.4GB/8350s/v=0 beat 00001.mpls, 76.5GB/8038s/
v=1), so the mux ran a title with zero streams -> zero frames -> E6008.

- Add Disc::main_feature_order layering two gates atop canonical_title_order:
  authoring-feature (a title the disc's menu authoring designates AND that
  has video) wins outright; then has-video demotes any streamless title
  below every title that has video. scan_with re-sorts with it after labels
  run so titles[0] is the selected feature. New MAIN_FEATURE_ORDER_KEYS +
  DiscTitle::has_video().
- Surface the authoring feature-playlist id: labels::FeaturePlaylistHint,
  captured from Paramount playlists.xml <playlist name="Feature" id=..>,
  threaded through ParseResult/extract; labels::apply now returns it.
- Unify pickers so key-sampling and the structural verdict (session.rs) and
  image_crack_extents prefer the has-video feature, not a streamless decoy.
- diag: decision row logs video= count + main_feature_order reason and a
  tag=decision.demoted line per no-video title (closes the field-diagnosis
  gap this incident exposed).

Tests: main_feature_order_demotes_streamless_decoy (pre-gate vs post-gate
contrast), _honours_authoring_hint, authoring_hint_never_promotes_a_
streamless_title, feature_playlist_hint_matches_on_id_or_filename; diag
reason test updated. QA green on 1.94 (fmt/clippy -D warnings/test).
2026-08-25 11:23:44 -07:00
Matthew Jackson f75766b4d5 chore: lower MSRV to 1.94, extend disc provenance tests, tidy comments
Lower the crate MSRV from 1.97 to 1.94 -- the lowest toolchain on which
`cargo build`, `cargo clippy --all-targets -D warnings`, and
`cargo test --tests` all pass clean. Dependencies floor the build at 1.90
(freemkv-unlock 1.90, zip 1.88), but clippy is only warning-clean from
1.94 (older clippy has a `duplicated_attributes` false positive on
`#![cfg(test)]` and flags a now-documented `let_and_return`). Update
`rust-version` and every `dtolnay/rust-toolchain` CI pin to 1.94.0, and
add a CHANGELOG Unreleased entry.

Extend the byte-provenance coverage added in f7d38ca (currently one
TS-path test) with two PS-path tests driving the real `Mpeg2Parser`:
- `ps_stream_stamps_source_provenance`: the PS path forwards `ps.source`
  onto `PesPacket`, so demuxed frames carry `Some(SourcePos)`.
- `ps_stream_carries_feed_base_across_reads`: the cumulative `fed_bytes`
  base carries ACROSS read buffers -- a frame decoded from the second
  sector is stamped `source.byte >= 2048`, not reset per read.

Also record the f7d38ca provenance fix in the CHANGELOG, inline a
`let_and_return` test binding, and trim a redundant sentence from
`PsDemuxer::feed_at`'s doc comment.
2026-08-25 10:35:47 -07:00
dependabot[bot] 1c313fe882 Bump actions/checkout from 5 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-24 08:52:57 -07:00
Pascal Harprecht f7d38caf0b mux/disc: pass feed base offset to demuxer to stamp byte provenance
On direct disc reads (`disc://...`), `DiscStream` called `demuxer.feed(&self.read_buf[..bytes])`
instead of `feed_at(buf_base, ...)`. Without a running feed base offset, demuxed `PesPacket`s
had `source = None`, stripping byte provenance from downstream `PesFrame`s.

Under a `SeamPlan`, when `src_byte` is `None`, `SeamPlan::place` falls back to a timestamp
heuristic across clips. On multi-clip playlists where a subsequent clip starts at the same
base timestamp as the initial clip (`in_time = 600s`), B-frame PTS reorder dips (where
`raw_ns < last` and `|raw_ns - next_in| <= 250ms`) falsely triggered a `stepped_back` clip
transition on the first reordered frame. Because the subsequent clip's span was short (e.g. 1s),
all subsequent video frames across the rest of the title were rejected and dropped as outside
clip marks. Audio tracks (which have monotonic PTS without reordering) were unaffected.

Track `fed_bytes` in `DiscStream` and route through `feed_at` for both TS and PS demuxers,
forwarding `ps.source` to `PesPacket` on the PS path.

Test: `mux::disc::tests::stream_surface_tests::disc_stream_stamps_source_provenance`
2026-08-24 08:52:19 -07:00
Matthew Jackson e98e98c8fd TrueHD: resync forward to next major sync on a source discontinuity
MLP/TrueHD carries predictor + restart state across access units, so after a source transport-stream discontinuity (continuity-counter gap) the parser was splicing post-gap audio mid-stream, decoding against stale state and producing a decoder-choking seam (a burst of "restart header sync incorrect" / "Invalid blocksize" errors that could flag the whole track corrupt).

Honor the source's own gap signal: drop forward to the next CRC-valid major-sync AU (a clean re-init point) before resuming, guarded so it never arms before a validated baseline exists (no stream-head whole-track drop). Mirrors the video keyframe resync and the ResyncGate design. Corrects the "audio has no inter-frame refs" comments in truehd/dts/resync. Adds a regression test.
2026-08-23 16:30:29 -07:00
Matthew Jackson 7c085db133 restore freemkv-unlock path dep for local dev (post-v1.6.9) 2026-08-22 08:34:31 -07:00
Matthew Jackson 587e26353c v1.6.9: bump version (freemkv-unlock git-pinned for the tag) 2026-08-22 08:34:26 -07:00
Matthew Jackson 3eb3e649b1 changelog: 1.6.9 unified-release entry 2026-08-22 08:24:55 -07:00
Matthew Jackson b01505983c restore freemkv-unlock path dep for local dev (post-v1.6.8) 2026-08-21 12:54:53 -07:00
Matthew Jackson 87bc07dad8 v1.6.8: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 12:54:48 -07:00
Matthew Jackson 743c5830b6 docs: 1.6.8 changelog (unified release) 2026-08-21 12:47:34 -07:00
Matthew Jackson 80c8e4ca24 ci: require a CHANGELOG entry for the crate version on main
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
2026-08-21 11:23:45 -07:00
Matthew Jackson f49e79c198 restore freemkv-unlock path dep for local dev (post-v1.6.7) 2026-08-21 09:31:40 -07:00
Matthew Jackson 231cbf3d76 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 21c7b12e2b docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 799039df21 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
Matthew Jackson 497b7726dc v1.6.6: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 20:03:44 -07:00
Matthew Jackson 94b549a9d8 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson fc2d35a19c v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew Jackson 3c51190c32 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson dfe0390445 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 8678a2bf94 drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson d953b46023 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 443ac6bcb1 Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 794393f24e CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson c6e2614c4c Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson b02c402dae CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 128e3bb120 Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson dc100f1c80 Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 286b5d4527 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson d9f3f82c65 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson e30209091e restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
Matthew Jackson ac46e2382f v1.6.4: bump version (freemkv-unlock git-pinned for the tag) 2026-08-15 09:13:26 -07:00
Matthew Jackson 862c1460c3 Trim single-clip titles to their playlist marks (1.6.4) 2026-08-15 08:14:31 -07:00
Matthew Jackson fea64d07e2 Stop an uncrackable VTS borrowing another VTS's title key
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.

`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.

Also, three things nothing could catch:

- The demux output filename took the stream's language raw while the
  base beside it was sanitised. A language code is three raw STN bytes
  through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
  encoding on real discs — a NUL in a path fails `File::create` with
  InvalidInput, taking the whole export down before one track file
  opened. `sanitize` now maps control characters too; it did not.

- `css::crack_key_scan`'s short-read handling was dead code under test:
  every source in the module returned the full request, so reverting
  `advance` to the requested count, or dropping the `.max(1)`, left the
  suite green. The `.max(1)` is load-bearing — without it a source
  returning `Ok(0)` never moves the cursor and never increments the
  budget, so the scan spins forever. That mutation now HANGS the test
  rather than failing it, which is the honest demonstration.

- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
  `#[cfg(test)] mod tests`, so it compiled in no configuration and could
  never fire — the dead gate the test above it was written to replace.
  Moved to module scope, and verified it now rejects a wrong constant at
  compile time.
2026-08-11 17:47:56 -07:00
Matthew Jackson 6e8da15c34 Bound the third hostile CSV, and stop two docs overclaiming
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.

The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.

`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
2026-08-11 16:59:33 -07:00
Matthew Jackson 8898351cb1 Round 2: stop two of round 1's tests claiming more than they prove
Both are mine, and both pass with the bound they "cover" deleted.

`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.

`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.

Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
2026-08-11 11:54:28 -07:00
Matthew Jackson 7a03be5496 Bound the commentary-index parse, and stop proving it with a clock
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.

It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.

`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.

The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.

Applies to both `sub_com1_idx` and `aud_com1_idx`.

NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
2026-08-11 11:30:53 -07:00
Matthew Jackson de94e12864 Cover the dir:// input door, and correct the CSS scramble-gate docs
Three round-1 audit findings on v1.6.0..HEAD.

Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.

Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.

Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
2026-08-11 10:23:37 -07:00
Matthew Jackson 21ec1d3f80 restore freemkv-unlock path dep for local dev (post-v1.6.3) 2026-08-10 09:04:40 -07:00
Matthew Jackson 6658a80819 v1.6.3: bump version (freemkv-unlock git-pinned for the tag) 2026-08-10 09:04:35 -07:00
Matthew Jackson 3db11a192c Carry the same licence and community files as the other repos
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.

SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
2026-08-09 22:08:22 -07:00
Matthew Jackson a425a71cd5 Keep the cross-platform jobs for the release candidate
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.

Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.

They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
2026-08-09 20:24:07 -07:00
Matthew Jackson f79745d815 Release 1.6.3
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
2026-08-09 20:01:33 -07:00
Matthew Jackson 5d8f77e049 Drop six unused crates, and align the rest with the workspace
Two problems, both invisible until the whole graph is looked at together.

DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are
declared here and referenced nowhere -- not in src, tests or benches.
They were being compiled, audited and offered version bumps forever for
no reason. Removing beats bumping.

cbc nearly survived the sweep: a substring search for "cbc" matches 44
occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked
used. Only a word-boundary search exposed it.

SKEW: this crate was the outlier on every shared dependency -- aes 0.8,
rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8
elsewhere. Cargo cannot unify across a major version, so it compiled
BOTH: 32 duplicated crates in the freemkv binary's graph, including two
complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two
digest stacks and two getrandom. Two crypto stacks in one product is
worth removing on its own.

The aes bump is an API rename -- BlockCipher-prefixed traits, Array for
GenericArray -- and the obvious translation uses Array::from_slice,
which the new version deprecates and clippy's -D warnings would reject.
These use the From<[T; N]> conversion the crate points at instead.

3441 tests pass in debug and release. The AACS crypto here is covered by
known-answer tests, so a byte-order or sizing mistake in that rename
could not have passed.
2026-08-09 18:37:24 -07:00
dependabot[bot] 79a16a9279 Bump softprops/action-gh-release from 2 to 3 (#4)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 17:14:30 -07:00
dependabot[bot] 219a77b581 Bump actions/checkout from 5 to 7 (#5)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:58:27 -07:00
Matthew Jackson be378cbc34 Have Dependabot watch the dependencies, and land its PRs on dev
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".

Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.

Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.

The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
2026-08-09 16:48:31 -07:00
Matthew Jackson 63200a1d1f Retire the README version bot
It fired on `release: published` and committed to main, the branch the
release points at its tag, so it could only ever leave main ahead of the
tag. release.sh now makes the same edit inside the release commit.

In this repo it was a no-op regardless: the pattern it rewrote does not
appear in this README, so it has been running and changing nothing.
2026-08-09 07:20:44 -07:00
Matthew Jackson 526dd3552b restore freemkv-unlock path dep for local dev (post-v1.6.2) 2026-08-08 22:18:33 -07:00
167 changed files with 17388 additions and 11100 deletions
+63
View File
@@ -0,0 +1,63 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+114 -21
View File
@@ -11,10 +11,36 @@ on:
pull_request:
jobs:
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
# forgot its changelog entry must fail here rather than ship a blank release.
# Hard-fails on main (and PRs targeting main); warns elsewhere so an
# in-progress dev/qa branch whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
set -euo pipefail
version=$(grep -m1 '^version = ' Cargo.toml | sed -E 's/.*"([^"]+)".*/\1/')
if grep -qE "^## \[${version}\]" CHANGELOG.md; then
echo "CHANGELOG.md has an entry for ${version}."
exit 0
fi
msg="CHANGELOG.md has no '## [${version}]' entry"
if [ "${GITHUB_REF_NAME:-}" = "main" ] || [ "${GITHUB_BASE_REF:-}" = "main" ]; then
echo "::error::${msg} — main requires a changelog entry for the released version."
exit 1
fi
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -26,12 +52,12 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
@@ -52,7 +78,7 @@ jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -64,12 +90,12 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -77,9 +103,22 @@ jobs:
working-directory: libfreemkv
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -91,12 +130,12 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -104,9 +143,22 @@ jobs:
working-directory: libfreemkv
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -118,12 +170,12 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -149,21 +201,21 @@ jobs:
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
@@ -178,7 +230,10 @@ jobs:
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
# The consumers span a higher MSRV than libfreemkv itself (the bins are
# 1.98, keysources 1.97), so build them with a toolchain that satisfies the
# HIGHEST dependent floor — libfreemkv's own 1.94 cannot compile them.
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
@@ -201,3 +256,41 @@ jobs:
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
# ── Coverage (non-required, additive) ───────────────────────────────────────
#
# Reports line coverage to Codecov. Deliberately NOT wired into any release
# gate: it is informational only (see codecov.yml) and no other job `needs:`
# it. Mirrors the `test` job's layout exactly — same pinned 1.94 toolchain,
# same dual checkout so the `../freemkv-unlock` path dep resolves, same cache
# workspace and working-directory. cargo-llvm-cov comes as a prebuilt binary
# from taiki-e/install-action, so it is toolchain-independent.
#
# No `--locked`: like the fmt/clippy/test jobs above, this crate's Cargo.lock
# is gitignored, so there is nothing to lock against on a fresh runner.
coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
slug: freemkv/libfreemkv
fail_ci_if_error: false
+15
View File
@@ -0,0 +1,15 @@
name: comment-guard
# Fails if any inline `//` comment block runs longer than 3 lines. Rustdoc
# (`///`, `//!`) is exempt. Keeps explanatory comments terse; long rationale
# belongs in a doc comment or a design note, not an inline wall.
on: [push, pull_request]
jobs:
comment-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Run comment-guard
run: python3 ci/comment-guard.py
+32
View File
@@ -0,0 +1,32 @@
name: doc-links
# Fails on broken intra-doc links in `///`/`//!` doc comments (e.g. a `[Type]`
# link to a renamed/removed item). Catches doc drift that `cargo build` misses.
#
# libfreemkv path-deps ../freemkv-unlock on the branch tip, so — like ci.yml —
# both repos are checked out into sibling subdirectories for the path dep to
# resolve.
on: [push, pull_request]
jobs:
doc-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- name: cargo doc (deny broken intra-doc links)
working-directory: libfreemkv
env:
RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links"
run: cargo doc --no-deps --document-private-items --lib --bins
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
+10 -10
View File
@@ -43,7 +43,7 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
@@ -71,15 +71,15 @@ jobs:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -92,15 +92,15 @@ jobs:
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
@@ -117,15 +117,15 @@ jobs:
windows-build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
+6 -6
View File
@@ -17,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -29,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# gated by the local precommit (same Rust 1.94, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -37,8 +37,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.97.0
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
@@ -56,8 +56,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
-36
View File
@@ -1,36 +0,0 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push
+273
View File
@@ -1,5 +1,278 @@
# Changelog
## [Unreleased]
## [1.6.12] — 2026-08-27
### Fixed
- macOS: hold a DiskArbitration claim (plus a mount-approval dissenter) for the SCSI transport's lifetime so `diskarbitrationd` can't remount the disc mid-rip — the cause of the E1000 "disc stolen mid-rip" failure on macOS.
### Changed
- Comment and documentation cleanup.
## [1.6.11] — 2026-08-26
### Added
- Navigation-driven main-feature selection: selection now follows the disc's
own on-disc menu/navigation logic — a Blu-ray HDMV navigation VM and a DVD
First-Play resolver — to reach the main feature, instead of guessing by
title size alone.
- Clip-set composite/decoy detection, so a title assembled from a composite
or decoy clip set is recognized as such during selection.
- A feature-payload floor, giving main-feature selection a minimum-size
sanity check.
- Normalized `DiscProfile` output.
- Vendor navigation-label parsing.
- Substantially expanded unit-test coverage across the navigation and
clip-selection paths.
### Fixed
- Main-feature selection no longer picks an obfuscated decoy playlist over
the real feature.
- Two rounds of code-audit hardening: navigation VM operand/field-offset
conformance, navigation now abstains rather than guessing on an
undecidable system-parameter branch, panic-safety fixes, log-escaping
fixes, and expanded fuzz/test coverage.
- Direct disc reads (`disc://…`): `DiscStream` now threads a cumulative feed
base offset into the demuxer (`feed_at`) for both the TS and PS paths and
forwards it onto each `PesPacket`, so demuxed frames carry byte-exact source
provenance (`SourcePos`). Without it, `SeamPlan::place` fell back to a PTS
heuristic that a B-frame PTS reorder dip could misread as a `stepped_back`
clip transition, dropping all subsequent video frames in multi-clip playlists.
### Changed
- Lowered the minimum supported Rust version (MSRV) from 1.97 to 1.94. This is
the lowest toolchain on which `cargo build`, `cargo clippy --all-targets
-D warnings`, and `cargo test --tests` all pass clean; CI toolchain pins were
moved to match. (The crate's dependencies floor `cargo build` at 1.90, but
clippy is only warning-clean from 1.94.)
## [1.6.10] — 2026-08-23
### Fixed
- TrueHD/MLP audio: on a source transport-stream discontinuity (lost or gapped
packets, signalled by a continuity-counter break), the parser now drops forward
to the next major-sync access unit before resuming, instead of splicing the
post-gap audio mid-stream. MLP carries predictor + restart state across access
units, so resuming mid-stream decoded against stale state and produced a
decoder-choking seam — streaming decoders emitted a burst of "restart header
sync incorrect" / "Invalid blocksize" errors and could flag the whole track
corrupt. This mirrors the video path (resync to the next keyframe after a gap)
and how the independent-frame audio codecs (AC-3, DTS, FLAC, AAC) already
recover; the fix is guarded so it never arms before a validated major-sync
baseline exists (no whole-track drop at stream head). The gapped instant itself
is in the source and cannot be recovered, but the muxed track is now
decoder-clean.
## [1.6.9] — 2026-08-22
### Changed
- Version aligned to 1.6.9 for the unified release. No functional changes to
this crate; the release was driven by autorip (automatic per-episode TV
ripping — each episode named `S{NN}E{MM}`, with TMDB runtime-aligned episode
numbering across multi-disc seasons — a Manual Rename option, and a unified
per-disc staging state file — see the autorip 1.6.9 notes).
## [1.6.8] — 2026-08-21
### Changed
- Version aligned to 1.6.8 for the unified release. No functional changes to
this crate; the release was driven by autorip (webhooks now fire per pipeline
stage — Rip / Mux / Move — with the Rip hook firing the moment the drive is
free again, plus a Ripper-tab activity-banner fix so it also shows during
moves — see the autorip 1.6.8 notes).
## [1.6.7] — 2026-08-21
### Changed
- Version aligned to 1.6.7 for the unified release. No functional changes to
this crate; the release was driven by autorip (per-webhook event selection,
a progress bar per moved artifact, and move-queue / webhook-error fixes —
see the autorip 1.6.7 notes).
## [1.6.6] — 2026-08-20
### Changed
- Version aligned to 1.6.6 for the unified release. No functional changes
to this crate; the release was driven by autorip (webhooks may now target
private/LAN addresses — see the autorip 1.6.6 notes).
## [1.6.5] — 2026-08-20
### Fixed
- **On some Blu-ray/UHD titles the picture stopped short of the declared
end.** When a title is stitched across a clip seam, freemkv drops video
after the join until it sees a fresh keyframe it can restart decoding
from. It only recognised one kind of keyframe (an IDR frame), but a
Blu-ray's final segment can open on a different kind of self-contained
frame instead — so on those discs the restart point was never
recognised and every remaining frame to the end of the film was
dropped, losing up to a group-of-pictures-plus tail of picture. That
kind of frame is now recognised as a valid restart point, matching how
the HEVC path already worked, so the tail is kept. HEVC and DVD titles
were never affected.
- **A split HD-DVD feature could be exported as half the film while still
claiming the full running time.** A feature stored across parts (for
example `FEATURE_1.EVO` + `FEATURE_2.EVO`) is composed back into one
title. If a part resolved with no usable data — a zero-length file, or
an extent map that yielded nothing — that part was neither used nor
flagged, so the feature was quietly composed from the surviving parts
alone, still advertising the whole runtime, and exited success with no
log. The missing part is now marked unusable and logged under its own
code (E6019), so half a movie can no longer be presented as a whole one.
- **A Blu-ray title with an unreadable clip could be exported short while
still reporting the full running time.** A scratched or malformed
clip-info (`.clpi`) sector, or a clip whose extents could not be
resolved (a bad sector, a broken allocation chain, an embedded-data
file), was silently skipped — but the title's duration had already been
counted from the playlist, so it shipped short of the runtime it claimed
at exit success, with nothing logged. Previously only one narrow failure
kind counted; now every unresolvable clip drops the title and warns with
the read's own error code. A truly absent optional file (such as the 3D
`.ssif` on a 2D disc) is still treated as benign.
- **Pressing Stop during a disc scan could leave the disc looking like it
holds fewer titles than it does — or none at all.** Once cancelled,
every remaining drive command fails, but the Blu-ray and DVD title
enumerators treated those failures as ordinary skips: the Blu-ray scan
returned a truncated title list and the DVD scan returned zero titles,
both at exit success, so a cancelled scan was indistinguishable from a
disc that simply held that many titles. A cancel is now propagated as an
error out of every enumerator and at every read site, so it can never be
cached, displayed, or ripped from as if it were the real disc. HD-DVD
already behaved this way and is the model the others now follow.
- **A decrypted HD-DVD could be refused with a DVD copy-protection error
(E7023).** HD-DVD and DVD share the same MPEG program-stream container,
and the scramble detector mistook an HD-DVD navigation packet for a
CSS-scrambled sector — so a good, already-decrypted HD-DVD was reported
as carrying an unrecoverable CSS key and hard-failed, sending anyone
triaging it hunting for a missing DVD key on a disc that never had one.
The detector now excludes the structural packet types that CSS never
scrambles, identified from a field that is readable even on ciphertext,
so a decrypted HD-DVD scans clean. The DVD CSS crack is unchanged and a
genuinely scrambled, uncrackable DVD still hard-fails as before, so
ciphertext can never be muxed as plaintext.
- **A file with an allocated-but-never-written region could splice
undefined sectors into the rip.** The UDF reader treated an
allocated-but-unrecorded extent as ordinary content and read whatever
happened to be on those sectors into the output, and it decoded
embedded-data files (which store their content inline, not as a sector
map) as if their bytes were an extent list, pointing the reader at
unrelated sectors. Such extents are now refused when they actually
occupy space, and embedded files are handled as their own case; a
legitimately zero-length file still reads as empty rather than dropping
its title.
- **Audio and subtitle tracks in many languages were all labelled
"undefined."** The language mapping recognised only a handful of codes
and collapsed fifteen others to `und`. Every ISO 639-1 language code is
now mapped, so those tracks carry their real language. The DVD subtitle
colour palette, which was written in the wrong order, is also corrected.
- **If the disc read-ahead thread died mid-rip, a whole title could be
fabricated and reported complete.** A prefetch producer that terminated
returned an end-of-data signal that the reader legitimately read as a
short read and zero-filled, so a failed read could be papered over with
zeros and the pass still reported as successful. It now reports a
distinct source-terminated error instead. Dead-bus drive faults are
likewise classified rather than flattened, so the wedged-drive recovery
path can see them, and the drive now responds to Stop during spin-up and
spin-down instead of staying deaf for up to ~30 seconds.
- **A holed extraction could climb to a clean 100% on the live progress
channel.** The progress feed hardcoded its unreadable-byte count to zero
and counted every zero-filled hole as good data, so a progress-only
consumer saw a damaged extraction finish spotless — even though the
authoritative result was already truthful. The real good/unreadable
split is now threaded through the live channel. Separately, an
unreadable HD-DVD authored clip order is now logged with its error code
instead of being silently discarded before falling back to the
per-clip heuristic.
### Security
- **Bounded several unbounded amplification axes in the HD-DVD and Blu-ray
scanners.** A crafted disc could drive the playlist nesting depth, the
title count, and the clips and chapters per title without limit, and
could force a repeated clip-name fallback probe — each of which alone
left the worst-case scan unbounded. All now carry positional caps (512
titles and clips, roughly ten times any retail disc) and the fallback
probe is memoized. No effect on a well-formed disc.
### Changed
- **How freemkv picks the per-platform drive code was consolidated, with
no change to reading a disc.** The drive layer chose its
operating-system-specific module separately inside each entry point; it
now selects that module once, so a new entry point cannot silently
forget a platform. Purely internal.
## [1.6.4] — 2026-08-15
### Fixed
- **On a few Blu-ray/UHD titles the sound ran on for half a minute after the
picture had ended.** A disc stores each part of a film as a clip, and the
playlist marks exactly where that clip's content begins and ends. Where a
title is a single clip, freemkv trimmed the picture to those marks but not the
sound — and some discs leave extra audio in the file past the end mark (a
quiet fade authored after the last frame of picture). That trailing audio was
copied through, so the file claimed one running time while carrying up to ~36
seconds more sound than picture. Measured on `The Bourne Supremacy`: the
picture ends at 1:48:26 as declared, but every sound track ran to 1:49:02.
Single-clip titles are now trimmed to their playlist marks the same way
multi-clip titles already were, so sound and picture end together at the
declared duration. A title that had no extra material past its marks is
byte-for-byte unchanged. Multi-clip titles were never affected.
- **A multi-title-set CSS DVD could descramble one title set under another
set's key.** When a DVD's second title set resisted the keyless title-key
recovery, the decrypt fell back to the disc-wide key instead of failing —
writing a corrupt title behind an intact header and reporting success at exit
0. A failed recovery is now a hard error, the same as every other path in the
crate already does; ordering makes the recovery more likely to succeed but
cannot make a failed one safe.
- **A disc whose stream language field was all-zero could abort the whole track
export.** An all-zero language code (the ordinary "undefined" value on real
discs) put a NUL byte into a demux output filename and failed file creation
before a single track opened. Control bytes in that field are now sanitised
the same way the rest of the name already was.
### Security
- **Bounded the last unbounded attacker-controlled list in the DVD label
parser.** A crafted IFO could grow the forced-subtitle index list without
limit; it now carries the same positional cap as the neighbouring command
lists. No effect on a well-formed disc.
## [1.6.3] — 2026-08-10
### Changed
- **Six crates that were never used have been removed, and nothing about reading
a disc has changed.** They were declared as dependencies but referenced
nowhere, so they were compiled into every build for nothing. The rest were
aligned with the versions the other freemkv crates use.
- **Two complete AES implementations were being built into the product.** Two
different releases of the cipher crates had been pulled in by different parts
of the tree and both were compiled. They are now one.
## [1.6.2] — 2026-08-08
### Fixed
+7 -13
View File
@@ -1,8 +1,8 @@
[package]
name = "libfreemkv"
version = "1.6.2"
version = "1.6.12"
edition = "2024"
rust-version = "1.97"
rust-version = "1.94"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@@ -22,20 +22,14 @@ codegen-units = 1
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.2" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.12" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
+2
View File
@@ -1,4 +1,6 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/libfreemkv/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/libfreemkv)
# libfreemkv
+22
View File
@@ -0,0 +1,22 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
+1
View File
@@ -5,6 +5,7 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
println!("cargo:rustc-link-lib=framework=DiskArbitration");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Fail if any inline `//` comment block runs longer than MAX_LINES.
Rustdoc (`///`, `//!`) is exempt — long API docs are fine. Only consecutive
non-doc `//` lines are counted; a blank line, code, or a doc line ends the run.
Block comments (`/* */`) are not inspected. Scans src/ and tests/ under the cwd.
"""
import sys
from pathlib import Path
MAX_LINES = 3
def is_inline_comment(line: str) -> bool:
s = line.lstrip()
return s.startswith("//") and not s.startswith("///") and not s.startswith("//!")
def violations(path: Path):
"""Yield (start_line, length) for each inline run longer than MAX_LINES."""
out = []
run_start = None
run_len = 0
for i, line in enumerate(path.read_text(encoding="utf-8", errors="replace").splitlines(), 1):
if is_inline_comment(line):
if run_start is None:
run_start = i
run_len += 1
else:
if run_len > MAX_LINES:
out.append((run_start, run_len))
run_start, run_len = None, 0
if run_len > MAX_LINES:
out.append((run_start, run_len))
return out
def main(argv):
root = Path(argv[1]) if len(argv) > 1 else Path.cwd()
files = sorted(p for d in ("src", "tests") for p in (root / d).rglob("*.rs"))
total = 0
for f in files:
for start, length in violations(f):
rel = f.relative_to(root)
print(f"{rel}:{start}: inline comment block is {length} lines (max {MAX_LINES})")
total += 1
if total:
print(f"\ncomment-guard: {total} inline comment block(s) exceed {MAX_LINES} lines.")
print("Trim to the essential why, or split the code so the comment isn't needed.")
return 1
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+18
View File
@@ -0,0 +1,18 @@
# Coverage is informational only: it never fails CI or blocks a merge/release.
# The `coverage` job in .github/workflows/ci.yml is non-required and no gate
# depends on it.
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
comment: false
ignore:
- "tests"
- "examples"
- "benches"
+77 -140
View File
@@ -4,7 +4,7 @@
#[cfg(test)]
use aes::Aes128;
#[cfg(test)]
use aes::cipher::{KeyInit, generic_array::GenericArray};
use aes::cipher::{Array, KeyInit};
use super::crypto::{aes_cbc_decrypt, aes_cbc_encrypt, aes_ecb_encrypt};
// Only this module's test fixtures build CBC ciphertext by hand now — the
@@ -14,7 +14,7 @@ use super::crypto::AACS_IV;
// ── AACS constants ──────────────────────────────────────────────────────────
/// Size of an AACS aligned unit (3 × 2048-byte sectors). [BD] §3.10.1.
/// Size of an AACS aligned unit (3 × 2048-byte sectors). `[BD]` §3.10.1.
pub const ALIGNED_UNIT_LEN: usize = 6144;
/// An AACS aligned unit spans this many 2048-byte sectors (3).
@@ -82,7 +82,7 @@ const PS_SCRAMBLE_MASK: u8 = 0x30;
///
/// - `BdTs` (BD / UHD / FMTS Transport Stream): the Copy Permission Indicator in
/// the top 2 bits of byte 0 — the first `TP_extra_header` byte, always left
/// clear (the first 16 bytes of every unit are the unencrypted SEED). [BD]
/// clear (the first 16 bytes of every unit are the unencrypted SEED). `[BD]`
/// §3.10.2. `(buf[0] & 0xC0) == 0` → clear; non-zero → bytes `16..6144` are
/// AES-CBC encrypted.
/// - `MpegPs` (HD-DVD `.evo`): the MPEG-2 `PES_scrambling_control` flag — see
@@ -99,10 +99,9 @@ pub fn aacs_unit_encrypted(unit: &[u8], format: crate::disc::ContentFormat) -> b
}
match format {
ContentFormat::BdTs => (unit[0] & 0xC0) != 0,
// UNVERIFIED-HDDVD-DECRYPT (1 of 2): the PES_scrambling_control location
// for HD-DVD `.evo` is derived from spec, never confirmed against a real
// ENCRYPTED HD DVD (we only have decrypted rips). If HD-DVD ripping ever
// misbehaves, verify this flag byte/mask against a genuine encrypted unit.
// UNVERIFIED-HDDVD-DECRYPT (1 of 2): PES_scrambling_control location for
// HD-DVD `.evo` is spec-derived, never confirmed against a real encrypted
// disc (only decrypted rips available). Verify against a real one if ripping misbehaves.
ContentFormat::MpegPs => (unit[PS_SCRAMBLE_OFF] & PS_SCRAMBLE_MASK) != 0,
}
}
@@ -137,12 +136,9 @@ pub fn aacs_unit_seed_encrypted(unit: &[u8], format: crate::disc::ContentFormat)
///
/// Like the flag itself this is only meaningful at the clip-FILE-anchored boundary.
pub fn aacs_unit_needs_decrypt(unit: &[u8], format: crate::disc::ContentFormat) -> bool {
// "Still needs the key applied" = flagged encrypted AND not yet structurally
// clean. There is ONE definition of clean — [`is_clean`] (the min(E,4) proof
// floor: E>4 needs any 4 synced, E<=4 needs all present). Never a second
// threshold: the old >50% majority false-flagged a bad-encoded-but-OPENED
// unit as still-scrambled, so the mux re-sampled it to the key service every
// batch (the storm) and could re-apply the key over already-clear bytes.
// "Still needs key" = flagged encrypted AND not structurally clean per the
// ONE definition, [`is_clean`]'s min(E,4) proof floor. Never a second threshold:
// the old >50% majority false-flagged bad-but-opened units, causing re-sample storms.
aacs_unit_encrypted(unit, format) && !is_clean(unit, format)
}
@@ -258,7 +254,7 @@ pub fn ts_packet_total(unit: &[u8]) -> usize {
/// the freebie `is_clean_ts` skips at packet 0); packs 1 and 2 (offsets 2048 and
/// 4096) are in the encrypted region, so a wrong key garbles them and this returns
/// false (64 bits of discrimination). Validated against real decrypted HD-DVD
/// `.evo` (ANCHORMAN / SHAUN_OF_THE_DEAD): pack starts are exactly 2048-aligned,
/// `.evo` (two real retail titles): pack starts are exactly 2048-aligned,
/// three per aligned unit, at offsets 0 / 2048 / 4096.
///
/// UNVERIFIED-HDDVD-DECRYPT (2 of 2): the pack STRUCTURE here is confirmed on
@@ -271,11 +267,9 @@ fn is_clean_ps(unit: &[u8]) -> bool {
return false;
}
const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
// Skip pack 0 (offset 0): its start bytes lie in the clear 16-byte seed —
// present for ANY key (and byte 0 may carry the Blu-ray-style CPI bits), so
// it proves nothing about decryption. This mirrors `is_clean_ts` skipping
// packet 0. Packs 1 and 2 (offsets 2048 / 4096) are in the encrypted region
// and are what discriminate the key (64 bits of proof).
// Skip pack 0: its start bytes lie in the clear 16-byte seed, present for ANY
// key, so it proves nothing (mirrors `is_clean_ts` skipping packet 0). Packs 1
// and 2 (offsets 2048/4096) are in the encrypted region and discriminate the key.
let mut o = SECTOR_BYTES;
while o + 4 <= ALIGNED_UNIT_LEN {
if unit[o..o + 4] != PACK_START {
@@ -302,7 +296,7 @@ fn is_clean_ps(unit: &[u8]) -> bool {
/// to "decrypt applies the key and nothing else". The only guard kept here is the
/// length check, since the crypto is defined only over a whole 6144-byte unit.
///
/// Block Key = AES-128E(Kcu, seed) ⊕ seed ([BD] §3.10.1 Fig 3-8: encrypt the clear
/// Block Key = AES-128E(Kcu, seed) ⊕ seed (`[BD]` §3.10.1 Fig 3-8: encrypt the clear
/// 16-byte seed under the CPS Unit Key, XOR the seed back in — the trailing ⊕seed
/// is load-bearing); then AES-128-CBC decrypt bytes 16..6144 under the AACS IV.
/// Source-zero padding packets (all 192 bytes zero on disc) are restored to zero:
@@ -395,12 +389,9 @@ pub fn encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) -> bool {
/// Remove bus encryption from an aligned unit (AACS 2.0 / UHD).
/// Bus encryption uses read_data_key, decrypting bytes 16..2048 of each 2048-byte sector.
pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
// Expand the key schedule ONCE for the whole unit. `read_data_key` is
// loop-invariant here (and constant for the entire disc), but calling
// `aes_cbc_decrypt` per sector rebuilt the AES-128 schedule per sector — three
// expansions per 6144-byte aligned unit, i.e. ~29 million redundant expansions
// over a 90 GB read on a stock drive, on the per-unit decrypt hot path.
// Measured by `decrypt_bus_expands_the_read_data_key_once_per_unit`.
// Expand the key schedule ONCE for the whole unit: `read_data_key` is
// loop-invariant, but per-sector `aes_cbc_decrypt` rebuilt it 3x per unit
// (~29M redundant expansions over a 90GB read) on the decrypt hot path.
let cipher = crate::aacs::crypto::new_cipher_for(read_data_key);
for sector_start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
if sector_start + SECTOR_BYTES > unit.len() {
@@ -418,7 +409,7 @@ pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
mod tests {
use super::super::crypto::aes_ecb_decrypt;
use super::*;
use aes::cipher::BlockEncrypt; // test fixtures build ciphertext directly
use aes::cipher::BlockCipherEncrypt; // test fixtures build ciphertext directly
/// [`encrypt_unit`] is the exact inverse of [`decrypt_unit`]: whatever an
/// authoring caller encrypts, the read path must recover byte-for-byte.
@@ -457,11 +448,6 @@ mod tests {
assert_eq!(unit, clear, "round trip must be byte-exact");
}
/// The documented padding asymmetry actually holds: `decrypt_unit` restores
/// all-zero-ON-DISC packets to zero, but an all-zero PLAINTEXT packet enciphers
/// to non-zero bytes, so it is not mistaken for padding and still round-trips.
/// This is the one place the two functions are deliberately not symmetric, so
/// the claim is worth pinning rather than asserting in prose alone.
/// A slice too short to encrypt must SAY so. The caller has already set the
/// container's encrypted flag by the time it calls this (the contract requires
/// flag-before-crypto, since the header is the key seed), so a silent no-op
@@ -487,6 +473,11 @@ mod tests {
assert!(encrypt_unit(&mut exact, &key), "a full unit must encrypt");
}
/// The documented padding asymmetry actually holds: `decrypt_unit` restores
/// all-zero-ON-DISC packets to zero, but an all-zero PLAINTEXT packet enciphers
/// to non-zero bytes, so it is not mistaken for padding and still round-trips.
/// This is the one place the two functions are deliberately not symmetric, so
/// the claim is worth pinning rather than asserting in prose alone.
#[test]
fn encrypt_unit_round_trips_all_zero_plaintext_packets() {
let key = [0xA5u8; 16];
@@ -546,11 +537,9 @@ mod tests {
#[test]
fn is_unit_aligned_lba_below_base_is_well_defined() {
// Latent-trap contract (rc.5.2 audit #5): a read never starts before its
// extent base, but if `lba < unit_base` the result must be well-defined,
// NOT the `wrapping_sub` underflow that — because 2^32 ≡ 1 (mod 3) —
// would falsely report alignment. `saturating_sub` clamps to offset 0,
// which is a unit boundary, so any `lba <= unit_base` reads as aligned.
// Latent-trap contract (rc.5.2 audit #5): if `lba < unit_base` the result must
// be well-defined, NOT `wrapping_sub` underflow (2^32 ≡ 1 mod 3 falsely
// reports alignment). `saturating_sub` clamps to 0, a unit boundary.
assert!(
is_unit_aligned(99, 100),
"lba just below base must not wrap"
@@ -569,11 +558,9 @@ mod tests {
#[test]
fn clear_unit_is_not_flagged_encrypted() {
// `decrypt_unit` is now PURE (applies the key unconditionally); the
// "leave a clear unit untouched" policy lives at the caller's gate,
// `aacs_unit_encrypted` / `aacs_unit_needs_decrypt`. A clear TS unit
// (byte-0 CPI bits clear, syncs intact) must report NOT-encrypted so the
// caller never hands it to decrypt_unit.
// `decrypt_unit` is PURE (applies the key unconditionally); "leave a clear
// unit untouched" lives at the caller's gate. A clear TS unit (CPI bits
// clear, syncs intact) must report NOT-encrypted so the caller skips decrypt_unit.
let ts = crate::disc::ContentFormat::BdTs;
let mut unit = vec![0u8; ALIGNED_UNIT_LEN];
let mut off = 4;
@@ -597,10 +584,8 @@ mod tests {
#[test]
fn ts_packet_total_no_off_by_one() {
// The maximum sync count is exactly the number of stride
// positions the counting loop visits (offset 4, 196, ...), i.e.
// len / 192, NOT (len - 4) / 192 + 1. For the 6144-byte aligned unit
// the loop checks offsets 4..=5956 → 32 positions.
// Max sync count = stride positions visited (offset 4, 196, ...) = len/192,
// NOT (len-4)/192+1. For the 6144-byte unit, offsets 4..=5956 → 32 positions.
let unit = vec![0u8; ALIGNED_UNIT_LEN];
assert_eq!(ts_packet_total(&unit), 32);
// Confirm the loop visits exactly that many stride positions.
@@ -612,10 +597,9 @@ mod tests {
#[test]
fn is_clean_min4_proof_floor() {
// ONE rule: a unit is clean iff `synced >= min(E, 4)` over the ENCRYPTED
// (non-padding) packets — E>4 needs any 4, E<=4 needs all present. Build
// NON-ZERO payloads (real content, not padding) so every packet counts
// toward E; place `n` TS syncs among packets 1..31 (packet 0 is skipped).
// ONE rule: clean iff `synced >= min(E, 4)` over ENCRYPTED (non-padding)
// packets — E>4 needs any 4, E<=4 needs all present. Build NON-ZERO payloads
// so every packet counts toward E; place syncs among packets 1..31 (0 skipped).
let unit_with = |synced: usize| {
let mut unit: Vec<u8> = (0..ALIGNED_UNIT_LEN)
.map(|i| ((i * 7 + 1) as u8) | 1)
@@ -686,11 +670,9 @@ mod tests {
let original = vec![0x42u8; 128]; // 8 blocks
let mut data = original.clone();
// Encrypt with the REAL production primitive. This test previously
// defined a local `fn aes_cbc_encrypt` that SHADOWED it, so it round-
// tripped a copy of the algorithm against itself and never exercised
// `crypto::aes_cbc_encrypt` at all — a mutation to the shipped function
// could not fail it.
// Encrypt with the REAL production primitive. This test previously defined a
// local `fn aes_cbc_encrypt` that SHADOWED it, round-tripping a copy of the
// algorithm against itself — a mutation to the shipped function couldn't fail it.
aes_cbc_encrypt(&key, &mut data);
assert_ne!(data, original); // should be different after encrypt
@@ -725,7 +707,7 @@ mod tests {
}
// CBC encrypt bytes 16..6143
let cipher = Aes128::new(GenericArray::from_slice(&encrypt_key));
let cipher = Aes128::new(&encrypt_key.into());
let mut prev = AACS_IV;
let num_blocks = (ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..num_blocks {
@@ -733,7 +715,9 @@ mod tests {
for j in 0..16 {
plain[off + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&plain[off..off + 16]);
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&plain[off..off + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.encrypt_block(&mut block);
plain[off..off + 16].copy_from_slice(&block);
prev.copy_from_slice(&plain[off..off + 16]);
@@ -795,14 +779,8 @@ mod tests {
}
// ── Padding-aware IsDecryptable (fragment-tail recovery) ───────────────
//
// A content fragment can end mid-unit, with the disc zero-padding the rest
// of the aligned unit to the next fragment (proven on Dunkirk: ~11 real
// video packets + source-zero pad). `decrypt_unit` must accept such a unit
// — its real packets decrypt; the source-zero tail is padding, not content
// — while still REJECTING a unit whose undecryptable tail is non-zero (a
// genuine misread / wrong key). The discriminator is the SOURCE bytes of the
// failing packets: zero ⇒ padding (still decryptable), non-zero ⇒ bad data.
// A fragment can end mid-unit, zero-padded to the next fragment. `decrypt_unit`
// must accept a zero tail (padding) but REJECT a non-zero tail (misread/wrong key).
/// Encrypt a full clear unit under `unit_key`, then overwrite the tail (from
/// packet `keep` onward) with `fill`. `0x00` models disc fragment padding; a
@@ -831,7 +809,7 @@ mod tests {
#[test]
fn fragment_tail_with_source_zero_pad_is_decryptable() {
// 11 real content packets, then source-zero padding (the Dunkirk shape).
// 11 real content packets, then source-zero padding (a real-disc fragment shape).
let key = [0x5Au8; 16];
let mut unit = tail_filled_unit(&key, 11, 0x00);
decrypt_unit(&mut unit, &key);
@@ -857,10 +835,8 @@ mod tests {
#[test]
fn fragment_tail_with_nonzero_garbage_decrypts_the_real_prefix() {
// Same shape, but the tail is NON-zero garbage. The crypto still recovers
// the 11 real packets; the garbage tail is whatever it is (the muxer drops
// it on sync-loss, and a genuine bad sector is caught by the physical read
// layer / mapfile — never by TS structure). Ground truth: the 11 real
// packets came back.
// the 11 real packets; the muxer drops the garbage tail on sync-loss (a
// genuine bad sector is caught by the read layer/mapfile, never TS structure).
let key = [0x5Au8; 16];
let mut unit = tail_filled_unit(&key, 11, 0xC3);
decrypt_unit(&mut unit, &key);
@@ -874,16 +850,8 @@ mod tests {
}
// ── Defect-tolerant "did a key OPEN this unit?" verdict ─────────────────
//
// The authored-bad-packet bug: a commercial disc carries the odd bad TS
// packet (a pressing/encoding defect, or an AACS 2.1 forensic-variant frame)
// — one non-conforming packet inside an otherwise perfectly-decrypted 6144
// unit. The OLD strict per-packet acceptance rejected the WHOLE unit over
// that single packet, so the mux concealed ~all of it as NULL and 21/22 good
// video packets were destroyed and tallied as loss ("false corruption").
// The read/decrypt path must instead ACCEPT the unit (the key opened it) and
// pass the defective packet through VERBATIM for the muxer to drop. TS-sync
// conformance is a muxer concern, never a decryption verdict.
// Authored-bad-packet bug: one non-conforming packet made OLD strict acceptance
// reject the whole unit, destroying good video. Must ACCEPT and pass defects verbatim.
/// Build a unit that decrypts to 32 content packets, with `defect_pkts`
/// marked authored-bad (a non-`0x47` at the sync position + non-zero payload
@@ -996,11 +964,9 @@ mod tests {
#[test]
fn is_clean_dispatches_by_container_format() {
use crate::disc::ContentFormat;
// Clean HD-DVD `.evo` Program-Stream unit: pack_start_code `00 00 01 BA`
// at each 2048-byte pack boundary (0/2048/4096) — the layout validated
// against real decrypted ANCHORMAN / SHAUN_OF_THE_DEAD `.evo`. Offset 0 is
// the clear-seed freebie; the packs at 2048/4096 are encrypted and are
// what actually discriminate a key.
// Clean HD-DVD `.evo` Program-Stream unit: pack_start_code `00 00 01 BA` at
// each 2048-byte pack boundary (0/2048/4096), validated against real retail rips.
// Offset 0 is the clear-seed freebie; packs at 2048/4096 are what discriminate a key.
let mut ps = vec![0u8; ALIGNED_UNIT_LEN];
for off in [0usize, 2048, 4096] {
ps[off..off + 4].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
@@ -1083,10 +1049,9 @@ mod tests {
#[test]
fn sparse_tail_needs_all_present_packets_min_e_4() {
// End-of-clip fragment tail: `min(E,4)` scales to the packets that exist,
// so a sparse unit needs ALL of its (few) encrypted packets — a wrong key
// gives 0, so this is a strong proof with no wrong-key hole, and a valid
// 1-packet tail is never false-rejected.
// End-of-clip fragment tail: `min(E,4)` scales to existing packets, so a
// sparse unit needs ALL its (few) encrypted packets. Wrong key gives 0 (no
// hole), and a valid 1-packet tail is never false-rejected.
assert!(
is_clean_ts(&decrypted_shape(1, 1, false)),
"E=1: the one packet syncs -> open"
@@ -1180,12 +1145,9 @@ mod tests {
#[test]
fn aes_ecb_matches_fips197_known_answer() {
// FIPS-197 Appendix C.1 AES-128 KAT:
// key = 000102030405060708090a0b0c0d0e0f
// plaintext = 00112233445566778899aabbccddeeff
// ciphertext= 69c4e0d86a7b0430d8cdb78070b4c55a
// This pins the AES primitive against a published vector — a wrong
// cipher (or a key/plaintext byte-order slip) fails it.
// FIPS-197 Appendix C.1 AES-128 KAT (key/plaintext/ciphertext = code arrays
// below: key 000102..0f, pt 001122..ff, ct 69c4e0..5a).
// Pins the AES primitive against a published vector.
let key = [
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D,
0x0E, 0x0F,
@@ -1209,7 +1171,7 @@ mod tests {
/// 2048-byte sector, and each call built its own AES-128 key schedule, so a
/// 6144-byte aligned unit performed THREE key expansions under the same
/// loop-invariant `read_data_key`. On a 90 GB UHD read on a stock (non-
/// LibreDrive) drive — ~14.6 million aligned units — that is ~29 million
/// firmware-unlocked) drive — ~14.6 million aligned units — that is ~29 million
/// redundant expansions on the per-unit decrypt hot path, for a key that is
/// constant for the whole disc. The counter is incremented inside
/// `crypto::new_cipher`, the single construction site.
@@ -1229,7 +1191,7 @@ mod tests {
}
/// The single-expansion refactor must be byte-identical: bus encryption
/// ([C] §4.2) covers bytes 16..2048 of every 2048-byte sector, so a
/// (`[C]` §4.2) covers bytes 16..2048 of every 2048-byte sector, so a
/// three-sector aligned unit round-trips through the forward direction
/// sector by sector and `decrypt_bus` must recover it exactly.
#[test]
@@ -1258,7 +1220,7 @@ mod tests {
/// The published `iv0` bytes, INDEPENDENT of the production constant.
///
/// [C] §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation.
/// `[C]` §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation.
/// Both IV tests below used to compute their expected value from
/// `crypto::AACS_IV` itself, so the constant was asserted against itself and
/// NOTHING in the suite pinned its bytes: swapping `AACS_IV` for `[0u8; 16]`
@@ -1272,7 +1234,7 @@ mod tests {
0x78,
];
/// Pins the fixed AACS CBC IV ([C] §2.1.2 `iv0`) against a literal, so a
/// Pins the fixed AACS CBC IV (`[C]` §2.1.2 `iv0`) against a literal, so a
/// change to `crypto::AACS_IV` fails HERE rather than silently shipping.
#[test]
fn aacs_iv_matches_published_iv0() {
@@ -1284,13 +1246,9 @@ mod tests {
#[test]
fn cbc_decrypt_first_block_xors_aacs_iv() {
// CBC: P[0] = AES-D(K, C[0]) XOR IV, and the IV is the fixed AACS
// constant (not zero). Encrypt a single block forward with the PUBLISHED
// iv0 literal, then confirm aes_cbc_decrypt recovers it — proving the IV
// the production code uses on block 0 is exactly that value. Building the
// fixture from `IV0_PUBLISHED` rather than from `AACS_IV` is what makes
// the claim real: a mutation that swaps AACS_IV for [0u8;16] now makes the
// recovered block wrong.
// CBC: P[0] = AES-D(K, C[0]) XOR IV, IV being the fixed AACS constant. The
// fixture is built from the PUBLISHED iv0 literal (not `AACS_IV`), so swapping
// AACS_IV for [0u8;16] makes the recovered block wrong — proving block 0's IV.
let key = [0x24u8; 16];
let plain = [0x5Au8; 16];
// Forward CBC for one block: C = AES-E(K, P XOR IV).
@@ -1308,30 +1266,9 @@ mod tests {
#[test]
fn aes_cbc_decrypt_matches_nist_sp800_38a_f2_2() {
// NIST SP 800-38A Appendix F.2.2 (CBC-AES128.Decrypt) published vector:
// Key = 2b7e151628aed2a6abf7158809cf4f3c
// IV = 000102030405060708090a0b0c0d0e0f
// CT = 7649abac8119b246cee98e9b12e9197d (block 0)
// 5086cb9b507219ee95db113a917678b2 (block 1)
// 73bed6b8e3c1743b7116e69e22229516 (block 2)
// 3ff1caa1681fac09120eca307586e1a7 (block 3)
// PT = 6bc1bee22e409f96e93d7e117393172a (block 0)
// ae2d8a571e03ac9c9eb76fac45af8e51 (block 1)
// 30c81c46a35ce411e5fbc1191a0a52ef (block 2)
// f69f2445df4f9b17ad2b417be66c3710 (block 3)
//
// `aes_cbc_decrypt` hardwires the fixed AACS IV for block 0 (it never
// takes a caller IV), so:
// * Blocks 1..=3 are independent of the IV — they MUST equal the NIST
// plaintext byte-for-byte (P[i] = AES-D(K, C[i]) XOR C[i-1]). This
// pins the real reverse-order CBC chaining against a published KAT.
// * Block 0 = AES-D(K, C[0]) XOR iv0 = NIST_PT[0] XOR NIST_IV XOR iv0 —
// the documented IV substitution. Asserting this exact relation pins
// both the AES decrypt of C[0] AND that block 0 uses iv0. The expected
// value is built from the `IV0_PUBLISHED` literal, NOT from
// `crypto::AACS_IV`: computing it from the production constant made
// the change cancel out of the triple XOR, so a swap to [0u8;16] still
// passed. It now fails.
// NIST SP 800-38A F.2.2 (CBC-AES128.Decrypt) KAT (vectors = code arrays below).
// `aes_cbc_decrypt` hardwires iv0 for block 0: blocks 1..=3 pin reverse-order
// chaining; block 0 = NIST_PT[0] XOR NIST_IV XOR the `IV0_PUBLISHED` literal.
let key = [
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
@@ -1455,9 +1392,8 @@ mod tests {
#[test]
fn cpi_gate_set_flag_decrypts_and_needs_decrypt_is_idempotent() {
// A CPI-set encrypted unit decrypts with the right key. CPI lives in the
// plaintext header, so it survives decryption — `aacs_unit_encrypted`
// still reports true afterward, but `aacs_unit_needs_decrypt` flips to
// false (syncs restored), keeping the re-decrypt paths idempotent.
// plaintext header, so `aacs_unit_encrypted` stays true after decryption, but
// `aacs_unit_needs_decrypt` flips false (syncs restored) — re-decrypt is idempotent.
let ts = crate::disc::ContentFormat::BdTs;
let key = [0x5au8; 16];
let mut unit = clear_unit();
@@ -1483,10 +1419,9 @@ mod tests {
#[test]
fn decrypt_bus_roundtrips_per_sector_skipping_first_16_bytes() {
// Bus encryption CBC-encrypts bytes 16..2048 of EACH 2048-byte sector
// (3 sectors per aligned unit), leaving the first 16 plaintext. Build
// the forward transform, then confirm decrypt_bus inverts it and
// leaves each sector's first 16 bytes untouched.
// Bus encryption CBC-encrypts bytes 16..2048 of EACH 2048-byte sector (3 per
// aligned unit), leaving the first 16 plaintext. Confirm decrypt_bus inverts the
// forward transform and leaves each sector's first 16 bytes untouched.
let rdk = [0x13u8; 16];
let mut unit = vec![0u8; ALIGNED_UNIT_LEN];
// Fill with a recognisable pattern.
@@ -1496,7 +1431,7 @@ mod tests {
let plain = unit.clone();
// Forward: CBC-encrypt unit[s+16 .. s+2048] per sector under AACS IV.
let cipher = Aes128::new(GenericArray::from_slice(&rdk));
let cipher = Aes128::new(&rdk.into());
for s in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
let mut prev = AACS_IV;
let body = s + 16;
@@ -1507,7 +1442,9 @@ mod tests {
for j in 0..16 {
unit[off + j] ^= prev[j];
}
let mut blk = GenericArray::clone_from_slice(&unit[off..off + 16]);
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&unit[off..off + 16]);
let mut blk: Array<u8, _> = chunk.into();
cipher.encrypt_block(&mut blk);
unit[off..off + 16].copy_from_slice(&blk);
prev.copy_from_slice(&unit[off..off + 16]);
@@ -1569,7 +1506,7 @@ mod tests {
/// `aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the
/// guard that stops a truncated encrypted fragment from being emitted as
/// clear content. It reads ONLY the two Copy Permission Indicator bits
/// ([BD] §3.10.2, byte 0 bits 6-7); the remaining six bits are
/// (`[BD]` §3.10.2, byte 0 bits 6-7); the remaining six bits are
/// `TP_extra_header` arrival-timestamp bits and carry no encryption
/// meaning.
///
+27 -30
View File
@@ -1,4 +1,4 @@
//! AACS common cryptographic primitives — [C] Chapter 2 / §3.2.2.
//! AACS common cryptographic primitives — `[C]` Chapter 2 / §3.2.2.
//!
//! Source: `[C]` = AACS Introduction and Common Cryptographic Elements Book,
//! Rev 0.953. The shared low-level building blocks — AES-128 ECB E/D, AES-G,
@@ -8,21 +8,16 @@
//! content / keys / variant modules.
use aes::Aes128;
use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit, generic_array::GenericArray};
use aes::cipher::{Array, BlockCipherDecrypt, BlockCipherEncrypt, KeyInit};
/// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`).
/// Fixed IV used by AACS for all AES-CBC operations. `[C]` §2.1.2 (default CBC IV, `iv0`).
pub(crate) const AACS_IV: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78,
];
// Per-thread count of AES-128 key schedules built through `new_cipher`.
// Test-only instrumentation: an AES-128 key expansion is 10 round-key
// derivations, and the CBC helpers here run on the per-aligned-unit decrypt hot
// path of a whole disc read, so "how many times was the schedule built for one
// loop-invariant key" is a property worth asserting rather than reasoning about.
// THREAD-LOCAL, not a global atomic: `cargo test` runs tests concurrently, so a
// shared counter would see every other test's expansions. See
// `content::tests::decrypt_bus_expands_the_read_data_key_once_per_unit`.
// Test-only: lets tests assert the CBC decrypt hot path builds one schedule
// per loop-invariant key. Thread-local, not atomic, since `cargo test` runs concurrently.
#[cfg(test)]
thread_local! {
pub(crate) static KEY_EXPANSIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
@@ -35,27 +30,27 @@ pub(crate) fn new_cipher_for(key: &[u8; 16]) -> Aes128 {
}
/// Build an AES-128 key schedule. The single construction site for the CBC
/// helpers, so [`KEY_EXPANSIONS`] can count them under test.
/// helpers, so `KEY_EXPANSIONS` can count them under test.
fn new_cipher(key: &[u8; 16]) -> Aes128 {
#[cfg(test)]
KEY_EXPANSIONS.with(|c| c.set(c.get() + 1));
Aes128::new(GenericArray::from_slice(key))
Aes128::new(&(*key).into())
}
/// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`).
/// AES-128-ECB encrypt a single 16-byte block. `[C]` §2.1.1 (`AES-128E`).
pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
cipher.encrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
out
}
/// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`).
/// AES-128-ECB decrypt a single 16-byte block. `[C]` §2.1.1 (`AES-128D`).
pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
cipher.decrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
@@ -63,7 +58,7 @@ pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
}
/// AES-128-CBC ENCRYPT in place under the fixed [`AACS_IV`] — the forward
/// direction of [`aes_cbc_decrypt`], and its exact inverse. [C] §2.1.2
/// direction of [`aes_cbc_decrypt`], and its exact inverse. `[C]` §2.1.2
/// (`AES-128CBCE`).
///
/// Precondition: `data.len()` is a multiple of 16; the assert
@@ -88,14 +83,14 @@ pub(crate) fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
for j in 0..16 {
block[j] = data[offset + j] ^ prev[j];
}
let mut ga = GenericArray::clone_from_slice(&block);
let mut ga: Array<u8, _> = block.into();
cipher.encrypt_block(&mut ga);
data[offset..offset + 16].copy_from_slice(&ga);
prev.copy_from_slice(&ga);
}
}
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. [C] §2.1.2
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. `[C]` §2.1.2
/// (`AES-128CBCD`).
///
/// Precondition: `data.len()` is a multiple of 16. Any trailing partial
@@ -124,7 +119,7 @@ pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
/// Split out of [`aes_cbc_decrypt`] so a caller that decrypts several regions
/// under one loop-invariant key expands the schedule once. `decrypt_bus`
/// ([`super::content::decrypt_bus`]) is that caller: bus encryption
/// ([C] §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// (`[C]` §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// 2048-byte sector, so a 6144-byte aligned unit is three regions under one
/// `read_data_key` — three key schedules where one suffices, on the per-unit
/// decrypt hot path of a whole 90 GB read.
@@ -140,7 +135,9 @@ pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
p.copy_from_slice(&data[(i - 1) * 16..i * 16]);
p
};
let mut block = GenericArray::clone_from_slice(&data[offset..offset + 16]);
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&data[offset..offset + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.decrypt_block(&mut block);
for j in 0..16 {
data[offset + j] = block[j] ^ prev[j];
@@ -148,7 +145,7 @@ pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
}
}
/// AES-G(x1, x2) = AES-128D(x1, x2) XOR x2. [C] §2.1.3 (note: uses AES-128**D**).
/// AES-G(x1, x2) = AES-128D(x1, x2) XOR x2. `[C]` §2.1.3 (note: uses AES-128**D**).
///
/// The Media Key Variant chain uses AES-G to derive both the variant
/// number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
@@ -163,14 +160,14 @@ pub(crate) fn aes_g(x1: &[u8; 16], x2: &[u8; 16]) -> [u8; 16] {
out
}
/// AACS-G3 seed constant (`s0`). [C] §3.2.2.
/// AACS-G3 seed constant (`s0`). `[C]` §3.2.2.
pub(crate) const AESG3_SEED: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B, 0xD9,
];
/// AACS-G3: derive a subkey from a parent key. [C] §3.2.2 (Triple AES Generator:
/// AACS-G3: derive a subkey from a parent key. `[C]` §3.2.2 (Triple AES Generator:
/// left=`D(k,s0)⊕s0` inc 0, pk=`D(k,s0+1)⊕(s0+1)` inc 1, right=`D(k,s0+2)⊕(s0+2)` inc 2).
/// seed[15] += inc, then AES-DEC(key, seed) XOR seed.
/// `seed[15] += inc`, then AES-DEC(key, seed) XOR seed.
///
/// Shared with [`super::variant`] (its variant chain runs the same SD
/// tree); a single definition keeps the two walks byte-identical.
@@ -188,7 +185,7 @@ pub(crate) fn aesg3(key: &[u8; 16], inc: u8) -> [u8; 16] {
mod tests {
use super::*;
/// The AACS-G3 seed `s0`, transcribed independently from [C] §3.2.2 rather
/// The AACS-G3 seed `s0`, transcribed independently from `[C]` §3.2.2 rather
/// than read from [`AESG3_SEED`] — a test that sourced the seed from the
/// production constant would assert that constant against itself and would
/// still pass if it were edited.
@@ -214,7 +211,7 @@ mod tests {
/// finding Media Keys, with no error to say why.
///
/// Pinned through the spec relation rather than a re-implementation:
/// [C] §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `[C]` §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `s = s0 + inc` (added into the last seed byte), so applying the
/// FORWARD primitive [`aes_ecb_encrypt`] — a different function from the
/// one under test — to `aesg3(k, inc) XOR s` must reproduce `s` exactly.
@@ -240,7 +237,7 @@ mod tests {
}
}
/// The Triple Generator's three outputs ([C] §3.2.2: left = inc 0, the
/// The Triple Generator's three outputs (`[C]` §3.2.2: left = inc 0, the
/// Processing Key = inc 1, right = inc 2) are the two child node keys and
/// the Processing Key of ONE tree node. They must be three different keys —
/// if `inc` were ignored, a descent would revisit its own parent and the
+54 -109
View File
@@ -1,5 +1,5 @@
//! Media-key derivation: DK/PK → Media Key via the subset-difference tree.
//! [C] §3.2.2–§3.2.5.
//! `[C]` §3.2.2–§3.2.5.
use super::crypto::*;
use super::inf::*;
@@ -71,11 +71,11 @@ pub(crate) fn try_pk_against_tables(
/// Validate a processing key against a cvalue/UV pair.
/// Returns the Media Key if valid.
///
/// Steps (media key: [C] §3.2.4; verify relation: [C] §3.2.5.1.4):
/// 1. `mk = AES-128D(pk, cvalue)` [C] §3.2.4
/// 2. `mk[12..16] ^= uv` (4 bytes XOR into the last 4 bytes only) [C] §3.2.4
/// 3. `dec_vd = AES-128D(mk, mk_dv)` [C] §3.2.5.1.4
/// 4. If `dec_vd[0..8] == 01 23 45 67 89 AB CD EF` → valid. [C] §3.2.5.1.4
/// Steps (media key: `[C]` §3.2.4; verify relation: `[C]` §3.2.5.1.4):
/// 1. `mk = AES-128D(pk, cvalue)` `[C]` §3.2.4
/// 2. `mk[12..16] ^= uv` (4 bytes XOR into the last 4 bytes only) `[C]` §3.2.4
/// 3. `dec_vd = AES-128D(mk, mk_dv)` `[C]` §3.2.5.1.4
/// 4. If `dec_vd[0..8] == 01 23 45 67 89 AB CD EF` → valid. `[C]` §3.2.5.1.4
pub(crate) fn validate_processing_key(
pk: &[u8; 16],
cvalue: &[u8],
@@ -105,7 +105,7 @@ pub(crate) fn validate_processing_key(
None
}
/// Compute v_mask from a UV value. [C] §3.2.3. Shared with [`super::variant`].
/// Compute v_mask from a UV value. `[C]` §3.2.3. Shared with [`super::variant`].
pub(super) fn calc_v_mask(uv: u32) -> u32 {
let mut v_mask: u32 = 0xFFFF_FFFF;
while (uv & !v_mask) == 0 && v_mask != 0 {
@@ -115,27 +115,22 @@ pub(super) fn calc_v_mask(uv: u32) -> u32 {
}
/// Derive processing key from device key using subset-difference tree traversal.
/// [C] §3.2.4 (device-tree descent, MSB-branch, terminal PK). Shared with [`super::variant`].
/// `[C]` §3.2.4 (device-tree descent, MSB-branch, terminal PK). Shared with [`super::variant`].
pub(super) fn calc_pk_from_dk(
dk: &[u8; 16],
uv: u32,
v_mask: u32,
dev_key_v_mask: u32,
) -> [u8; 16] {
// Descend from the device node to the record node, following the record's
// `uv` bits. At each level only the child we descend INTO is needed (the
// sibling is computed but never used), and the Processing Key is the
// `aesg3(.,1)` of the FINAL node — so we derive ONE child per level and the
// PK once at the end, instead of left/pk/right at every level. Identical
// result, ~3x fewer block ops. (left child = `aesg3(node,0)`, right = `,2`.)
// Descend device node -> record node by the record's `uv` bits. Only the child
// descended into matters, so derive ONE child per level (left=`aesg3(node,0)`,
// right=`,2`) and the PK=`aesg3(.,1)` once at the end — ~3x fewer block ops.
let mut node = *dk;
let mut current_v_mask = dev_key_v_mask;
// The subset-difference tree is at most 32 levels deep (u32 mask), so the
// walk must converge in <= 32 steps. The arithmetic `>> 1` sign-extends
// current_v_mask, so a v_mask coarser than dev_key_v_mask (reachable from
// a crafted/corrupt MKB) would otherwise saturate at 0xFFFF_FFFF and spin
// forever — bound the loop to keep a bad disc from hanging the rip thread.
// Tree is <=32 levels deep (u32 mask). The arithmetic `>> 1` sign-extends, so a
// v_mask coarser than dev_key_v_mask (crafted/corrupt MKB) would saturate to
// 0xFFFF_FFFF and spin forever; bound to 32 steps so a bad disc can't hang the rip.
let mut steps = 0u32;
while current_v_mask != v_mask {
if steps >= 32 {
@@ -203,14 +198,9 @@ pub fn derive_media_key_and_pk_from_dk(
let p_uv = &uvs[1 + 5 * uvs_idx..];
let u_mask_shift = uvs[5 * uvs_idx]; // byte before the UV value
// `num_uvs` was computed via `take_while(.. c[0] & 0xC0 == 0)`, so
// every iterated slot already has its revoked-marker bits clear — no
// inner `& 0xC0` re-check is needed (it would be unreachable).
//
// Shifts of 32..=63 (0x20..=0x3F) have those bits clear but would
// panic in debug / wrap to a wrong mask in release. The MKB byte is
// disc-controlled, so a crafted/corrupt MKB must not crash the ripper:
// skip an out-of-range slot rather than `<<` it.
// `num_uvs` used `take_while(.. c[0] & 0xC0 == 0)`, so revoked-marker bits
// are already clear (no re-check). But shifts 32..=63 panic/wrap on `<<` and
// the byte is disc-controlled, so skip an out-of-range slot rather than shift.
if u_mask_shift >= 32 {
continue;
}
@@ -262,7 +252,7 @@ pub fn derive_media_key_and_pk_from_dk(
/// 16 bytes) cannot be walked — the walk needs its tree node. This finds that
/// node empirically: for each MKB subset-difference record, it tries the device
/// at the record's node AND at every ancestor v-position (the device may sit one
/// or more levels ABOVE the record, descending via AES-G to reach it), deriving
/// or more levels ABOVE the record, descending via AES-G3 to reach it), deriving
/// the candidate Processing Key DIRECTLY (one [`calc_pk_from_dk`] per candidate,
/// no full re-walk) and checking it validates against that record's cvalue.
///
@@ -284,16 +274,14 @@ pub fn recover_dk_position(mkb: &[u8], key: &[u8; 16]) -> Option<DeviceKey> {
.count();
let n_cv = cvalues.len() / 16;
// Hoisted ONCE for the whole scan: the Processing Key the device produces if
// it sits EXACTLY at a record (zero descent) is `AES-G3(key, 1)` — it does
// not depend on the record, so the zero-descent probe of every slot reuses
// this single value instead of re-deriving it per slot.
// Hoisted once: the zero-descent Processing Key (device sits exactly at a record)
// is `AES-G3(key, 1)`, independent of the record, so every slot's zero-descent
// probe reuses this instead of re-deriving it per slot.
let pk_zero_descent = aesg3(key, 1);
// The slots are independent, so the scan parallelises — a UHD MKB has ~181k
// slots (~26s single-threaded). `find_map_any` returns the first matching
// node found by any thread and cancels the rest; a valid MKB has exactly one
// matching subset-difference, so which thread finds it is immaterial.
// Slots are independent so the scan parallelises (~181k slots, ~26s serial on a
// UHD MKB). `find_map_any` returns the first match and cancels the rest; a valid
// MKB has exactly one matching subset-difference, so which thread finds it is moot.
use rayon::prelude::*;
let found = (0..num_uvs.min(n_cv)).into_par_iter().find_map_any(|i| {
let u_mask_shift = uvs[5 * i];
@@ -415,7 +403,7 @@ pub mod probe {
// ── Volume key: Media Key + Volume ID → VUK → unit keys ──────────────────────
/// Derive VUK from Media Key and Volume ID. [PR] §3.3 / [BD] §3.3
/// Derive VUK from Media Key and Volume ID. `[PR]` §3.3 / `[BD]` §3.3
/// (`Kvu = AES-G(Km, IDv)`; AES-G uses AES-128D):
/// VUK = AES-128-ECB-DECRYPT(media_key, volume_id) XOR volume_id
pub fn derive_vuk(media_key: &[u8; 16], volume_id: &[u8; 16]) -> [u8; 16] {
@@ -426,8 +414,8 @@ pub fn derive_vuk(media_key: &[u8; 16], volume_id: &[u8; 16]) -> [u8; 16] {
vuk
}
/// Decrypt an encrypted unit key using the VUK (AES-128-ECB). [PR] §3.5
/// (Title Key unwrap `Kt = AES-128D(Ku, Kte)`); the BD "CPS Unit Key" synonym is [BD] §3.9.3.
/// Decrypt an encrypted unit key using the VUK (AES-128-ECB). `[PR]` §3.5
/// (Title Key unwrap `Kt = AES-128D(Ku, Kte)`); the BD "CPS Unit Key" synonym is `[BD]` §3.9.3.
pub fn decrypt_unit_key(vuk: &[u8; 16], encrypted_uk: &[u8; 16]) -> [u8; 16] {
aes_ecb_decrypt(vuk, encrypted_uk)
}
@@ -792,7 +780,7 @@ mod resolve_candidate_tests {
///
/// This file holds the whole subset-difference walk and had five tests for it.
/// There are no published AACS test vectors, but none are needed: the AACS
/// relations ([C] §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
/// relations (`[C]` §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
/// key can be constructed with `aes_ecb_encrypt` and the same `aesg3` the walk
/// uses as its node function. That is what `plant_mkb` below does — no real
/// key material, and the assertions check the DERIVED Media Key, not any
@@ -802,7 +790,7 @@ mod position_recovery_tests {
use super::*;
use crate::aacs::crypto::aes_ecb_encrypt;
/// [C] §3.2.5.1.4 Verify-Media-Key plaintext prefix.
/// `[C]` §3.2.5.1.4 Verify-Media-Key plaintext prefix.
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
/// An MKB record: 1-byte type + BE24 total length (header included) + body.
@@ -834,7 +822,7 @@ mod position_recovery_tests {
///
/// `uv = 0x0400` (lowest set bit 10) with `u_mask_shift = 12` is chosen so a
/// gating device node exists: `resolve_dk_node` flips one bit `b < 12`, and
/// the walk's gate ([C] §3.2.4) needs that bit inside `v_mask`
/// the walk's gate (`[C]` §3.2.4) needs that bit inside `v_mask`
/// (`0xFFFF_FFFF << 11`) and outside `u_mask` (`0xFFFF_FFFF << 12`) — i.e.
/// `b == 11`. `uv` is kept under 0x10000 because `DeviceKey::node` is a u16.
fn plant_mkb() -> Planted {
@@ -956,7 +944,7 @@ mod position_recovery_tests {
}
/// `resolve_dk_node` picks the `device_number` that passes the walk's
/// subset-difference gate ([C] §3.2.4). `None` here strands a key whose
/// subset-difference gate (`[C]` §3.2.4). `None` here strands a key whose
/// position was already successfully recovered, so it is the last step of
/// position recovery and fails the same way: usable key, discarded.
///
@@ -1015,22 +1003,12 @@ mod position_recovery_tests {
);
}
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT MKB where the device key sits ABOVE the matching slot.
//
// `plant_mkb` above is a ONE-slot, ZERO-descent fixture: the matching
// subset-difference is at index 0 and the device sits exactly on it. That
// leaves two whole behaviours of `recover_dk_position` unexercised —
// * slot INDEXING (`uvs[5*i]`, `uvs[1 + 5*i]`, `cvalues[i*16..]`), which
// is the identity permutation when i is always 0, and
// * the DESCENT branch, where the device is an ancestor of the slot and
// the candidate position is walked up bit by bit —
// so an MKB whose keyed slot is index 2 of 3, opened by a device one level
// above it, is what pins them.
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT MKB where the device key sits ABOVE the matching slot. Unlike
// `plant_mkb`'s one-slot zero-descent fixture, this pins slot INDEXING and the
// DESCENT branch: keyed slot is index 2 of 3, opened by a device one level above.
/// v-masks for the fixture's two positions, written as literals from
/// [C] §3.2.3 (`v_mask` is all-ones above the LOWEST set bit of `uv`, i.e.
/// `[C]` §3.2.3 (`v_mask` is all-ones above the LOWEST set bit of `uv`, i.e.
/// `0xFFFF_FFFF << (uv.trailing_zeros() + 1)`) rather than computed with
/// `calc_v_mask`, which is itself under test.
const UV_SLOT: u32 = 0x0000_9400; // lowest set bit 10
@@ -1039,7 +1017,7 @@ mod position_recovery_tests {
const V_MASK_ANCESTOR: u32 = 0xFFFF_F000; // 0xFFFF_FFFF << 12
const U_MASK_SHIFT: u8 = 16;
/// `calc_v_mask` implements [C] §3.2.3. Every subset-difference gate and
/// `calc_v_mask` implements `[C]` §3.2.3. Every subset-difference gate and
/// every descent in the walk is masked by its result, so a wrong mask makes
/// the walk match the wrong slots (or none) — pinned here against literal
/// expectations, not against a re-computation.
@@ -1088,10 +1066,8 @@ mod position_recovery_tests {
0xBE, 0xBF,
];
// The Processing Key the device produces after descending from
// UV_ANCESTOR to the slot. Built with the same descent the walk uses
// (as `plant_mkb` builds its cvalue with the same `aesg3`), but anchored
// to the FIXED ancestor position above — so a walk that computes a
// Processing Key from descending UV_ANCESTOR -> slot, via the same descent the
// walk uses but anchored to the FIXED ancestor above — so a walk computing a
// different candidate position derives a different Kp and fails.
let pk = calc_pk_from_dk(&dkey, UV_SLOT, V_MASK_SLOT, V_MASK_ANCESTOR);
@@ -1191,20 +1167,9 @@ mod position_recovery_tests {
assert!(recover_dk_position(&p.mkb, &stranger).is_none());
}
// ════════════════════════════════════════════════════════════════════
// A FOUR-LEVEL descent taking both branches.
//
// The fixtures above descend zero levels and one level (left). The tree
// walk's per-level branch decision — [C] §3.2.4: descend RIGHT
// (`aesg3(.,2)`) when the slot's `uv` has the level's bit set, LEFT
// (`aesg3(.,0)`) when it is clear, terminal Processing Key `aesg3(.,1)` —
// is only pinned by a descent that takes both branches more than once.
//
// The expected Processing Key here is written out as an EXPLICIT chain of
// `aesg3` calls, not computed with `calc_pk_from_dk`: a fixture built by
// the function under test moves with it, and every mutation of the descent
// would stay self-consistent.
// ════════════════════════════════════════════════════════════════════
// A FOUR-LEVEL descent taking both branches, pinning the per-level decision ([C]
// §3.2.4: RIGHT `aesg3(.,2)` if uv bit set, else LEFT `aesg3(.,0)`, PK `aesg3(.,1)`).
// Expected PK is an EXPLICIT `aesg3` chain, not `calc_pk_from_dk` (which would drift).
/// Slot `uv` for the four-level fixture: bits 8, 6 and 4 set. Lowest set
/// bit 4 → the descent reads bits 8, 7, 6, 5 (set, clear, set, clear).
@@ -1232,13 +1197,8 @@ mod position_recovery_tests {
0xDE, 0xDF,
];
// [C] §3.2.4, written out level by level. Descending from the ancestor
// node to the slot reads UV_SLOT4's bits 8, 7, 6, 5 in that order:
// bit 8 = 1 → right child, aesg3(., 2)
// bit 7 = 0 → left child, aesg3(., 0)
// bit 6 = 1 → right child, aesg3(., 2)
// bit 5 = 0 → left child, aesg3(., 0)
// and the Processing Key is aesg3(final_node, 1).
// [C] §3.2.4 level by level: ancestor -> slot reads UV_SLOT4 bits 8,7,6,5 =
// 1,0,1,0 -> right(2),left(0),right(2),left(0), then PK = aesg3(final_node, 1).
let n1 = aesg3(&dkey, 2);
let n2 = aesg3(&n1, 0);
let n3 = aesg3(&n2, 2);
@@ -1320,13 +1280,9 @@ mod position_recovery_tests {
);
}
// ════════════════════════════════════════════════════════════════════
// MALFORMED MKBs: a truncated cvalue table, and a revoked-marker slot.
//
// The MKB is disc-controlled data. Both of these shapes are reachable from
// a corrupt or crafted disc, and in both the walk must decline to derive a
// key rather than index past the end of a record.
// ════════════════════════════════════════════════════════════════════
// MALFORMED MKBs: a truncated cvalue table, and a revoked-marker slot. Both are
// reachable from a corrupt/crafted (disc-controlled) MKB, and in both the walk must
// decline to derive a key rather than index past the end of a record.
/// Assemble an MKB from an explicit slot list and cvalue table.
/// `slots` is `(u_mask_shift, uv)` per subset-difference entry.
@@ -1424,7 +1380,7 @@ mod position_recovery_tests {
}
/// The `0xC0` revoked marker in a slot's `u_mask_shift` byte TERMINATES the
/// subset-difference table ([C] §3.2.5.1.5). Slots after it are not part of
/// subset-difference table (`[C]` §3.2.5.1.5). Slots after it are not part of
/// the index and must not be walked — a walk that ran past the marker would
/// derive keys from entries the MKB has explicitly ended.
///
@@ -1490,7 +1446,7 @@ mod position_recovery_tests {
}
/// A device key applies to a subset-difference only when BOTH gates hold
/// ([C] §3.2.4): its u-mask must equal the slot's, AND its `uv` must agree
/// (`[C]` §3.2.4): its u-mask must equal the slot's, AND its `uv` must agree
/// with the slot's under the device's v-mask. A key filed with the wrong
/// `u_mask_shift` describes a different region of the tree and must not be
/// used, even though its tree position would otherwise line up — accepting
@@ -1524,7 +1480,7 @@ mod position_recovery_tests {
}
/// `validate_processing_key` XORs the slot's 4-byte `uv` into `mk[12..16]`
/// ([C] §3.2.4 step 2). XOR, not OR: the operation must be reversible, and
/// (`[C]` §3.2.4 step 2). XOR, not OR: the operation must be reversible, and
/// it must be able to CLEAR a bit the AES output set. A `uv` and a Media Key
/// that share set bits in those four bytes are what tell the two apart.
#[test]
@@ -1580,7 +1536,7 @@ mod position_recovery_tests {
/// `probe::aes_dec` is the single AACS verify primitive a reproduction
/// harness has: every claim such a harness makes about a Media Key is
/// `aes_dec(km, mk_dv)` starting with the [C] §3.2.5.1.4 magic. A body
/// `aes_dec(km, mk_dv)` starting with the `[C]` §3.2.5.1.4 magic. A body
/// returning a fixed block makes the harness answer the SAME way for every
/// key and every disc — either "nothing verifies" or, if the constant
/// happened to start with the magic, "everything verifies", which is the
@@ -1647,20 +1603,9 @@ mod position_recovery_tests {
);
}
// ════════════════════════════════════════════════════════════════════
// An ODD subset-difference `uv` — the depth-0 slot.
//
// `recover_dk_position`'s descent starts at `uv_r.trailing_zeros() + 1`.
// Every other fixture in this module uses an even `uv` (lowest set bit 4,
// 10 or 11), so `trailing_zeros()` was never 0 and the descent never began
// at level 1. That left the whole depth-0 case unexecuted: a slot whose `uv`
// is odd sits at the very bottom of the subset-difference tree, and it is a
// perfectly legal MKB shape.
//
// It is also the arithmetic boundary of the loop bound: at `p == 0` the
// `+ 1` is the only thing keeping `(p - 1)` — an unsigned underflow — off
// the range expression.
// ════════════════════════════════════════════════════════════════════
// An ODD subset-difference `uv` — the legal depth-0 slot. Descent starts at
// `uv_r.trailing_zeros() + 1`; other fixtures use even `uv`, so this alone exercises
// `trailing_zeros() == 0` and the `p == 0` boundary where `+ 1` blocks a `(p-1)` underflow.
/// Slot `uv` with bits 8, 6, 4 AND 0 set: lowest set bit 0, so
/// `trailing_zeros() == 0` and the descent must start at level 1.
@@ -1673,7 +1618,7 @@ mod position_recovery_tests {
/// An MKB with a single ODD-`uv` slot, keyed by a device one level above it.
///
/// The expected Processing Key is written as the EXPLICIT `aesg3` chain the
/// one-level descent produces ([C] §3.2.4): from the ancestor, bit 1 of
/// one-level descent produces (`[C]` §3.2.4): from the ancestor, bit 1 of
/// `UV_ODD` is CLEAR, so the walk takes the left child (`aesg3(.,0)`) and
/// then terminates with `aesg3(.,1)`. Not built with `calc_pk_from_dk` —
/// a fixture built by the walk moves with the walk's own mutations.
+6 -9
View File
@@ -63,11 +63,9 @@ pub fn unit_disposition(
None => UnitDisposition::Default,
// In a forensic segment → decide by whether it is our index.
Some(seg) => {
// `seg.index` is an untrusted u16 from IndividualSegment.tbl; a real
// forensic index is 1..=32. Compare in u16 space so a corrupt/crafted
// index above 255 can't truncate into a valid u8 and alias our index.
// The disposition carries a u8 for diagnostics (saturated — an
// out-of-range index is never ours anyway).
// `seg.index` is untrusted (real range 1..=32); compare in u16 space so a
// crafted index above 255 can't truncate into a valid u8 and alias ours.
// Diagnostics carry a saturated u8 since out-of-range is never ours.
let seg_index = seg.index;
let diag = seg_index.min(u8::MAX as u16) as u8;
match disc_index {
@@ -169,10 +167,9 @@ mod tests {
#[test]
fn out_of_range_index_does_not_truncate_into_ours() {
// A crafted/corrupt segment index of 288 (0x0120) truncates to 32 in a
// u8. With our disc index resolved as 32, the old `seg.index as u8`
// compare would alias it to OUR index and decrypt with the wrong key.
// The u16 compare must instead classify it as foreign.
// Index 288 (0x0120) truncates to 32 in a u8, which would alias our
// resolved index 32 under the old `as u8` compare; u16 compare must
// classify it as foreign instead.
let segs = tbl(&[(288, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
+14 -17
View File
@@ -67,6 +67,7 @@ pub fn disc_hash_hex(hash: &[u8; 20]) -> String {
/// Parse Unit_Key_RO.inf from raw bytes.
///
/// Format (from AACS spec):
/// ```text
/// [0..4] BE32: offset to key storage area (uk_pos)
/// [16] app_type (1 = BD-ROM)
/// [17] num_bdmv_dir
@@ -81,6 +82,7 @@ pub fn disc_hash_hex(hash: &[u8; 20]) -> String {
/// [uk_pos+48..] encrypted keys, 16 bytes each
/// AACS 1.0: 48-byte stride
/// AACS 2.0 / 2.1: 64-byte stride (48 + 16 extra)
/// ```
pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() < 20 {
return None;
@@ -135,18 +137,16 @@ pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFil
pos += stride;
}
// The loop above `break`s if the buffer runs out mid-key. A short list
// means the .inf is malformed/truncated — reject it rather than silently
// accepting fewer keys than the header declared, which would later map
// title CPS units to nonexistent keys.
// The loop `break`s if the buffer runs out mid-key; a short list means the .inf
// is malformed/truncated — reject rather than silently mapping title CPS units
// to fewer keys than the header declared.
if encrypted_keys.len() != num_uk {
return None;
}
// Title → CPS unit mapping (AACS Unit_Key_RO format): each on-disc CPS
// value is in `1..=num_uk` (else zeroes it) and converts the 1-based on-disc
// index to a 0-based key index. We mirror that so the stored value is a safe,
// ready-to-use key index rather than a raw 1-based number.
// Title → CPS unit mapping (AACS Unit_Key_RO format): validates the on-disc
// CPS value is in `1..=num_uk` (else zeroes it) and converts it from 1-based
// to a safe, ready-to-use 0-based key index.
let to_key_idx = |cps: u16| -> u16 {
if cps >= 1 && cps as usize <= num_uk {
cps - 1
@@ -228,7 +228,7 @@ const VTKF_AV_FLG: u8 = 0x80;
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
/// (real multi-CPS-unit HD-DVD titles) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
@@ -382,10 +382,8 @@ pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
return None;
}
// Content Certificate layout (per the AACS content-cert format):
// [0] certificate type (0x00 = AACS1, 0x10 = AACS2)
// [1] bit7 bus_encryption_enabled_flag (`p[1] >> 7`)
// [14..20] cc_id (6 bytes) (`p + 14`)
// Content Certificate layout: [0] cert type (0x00 AACS1/0x10 AACS2),
// [1] bit7 bus_encryption_enabled_flag, [14..20] cc_id (6 bytes).
let version = if data[0] == 0x00 {
AacsVersion::V10
} else {
@@ -444,10 +442,9 @@ mod vtkf_tests {
let data = synth_vtkf(&[k1, k2, k3]);
let ukf = parse_vtkf(&data).expect("valid VTKF must parse");
// Exactly the three present entries — the empty slots and the trailing
// 16-byte TKF MAC are NOT mistaken for keys. Critically, k2/k3 are read
// at the 36-byte stride (offsets 0xA4, 0xC8); the old 32-byte stride
// misread them from inside the previous entry's binding MAC.
// Exactly the three present entries — empty slots and the trailing 16-byte
// TKF MAC are not mistaken for keys. k2/k3 are read at the 36-byte stride
// (0xA4, 0xC8); the old 32-byte stride misread them from the prior MAC.
assert_eq!(ukf.encrypted_keys.len(), 3);
assert_eq!(
ukf.encrypted_keys[0],
+8 -10
View File
@@ -1,4 +1,4 @@
//! AACS Media Key Block — [C] Chapter 3.
//! AACS Media Key Block — `[C]` Chapter 3.
//!
//! The MKB record format (framing walker, the `MkbRecord` view, record-body
//! finders), the MKBType / AACS-generation classification, and MKB-file
@@ -7,10 +7,8 @@
//! still live side by side pending a follow-up that collapses them.
// ── MKB record types ([C] Chapter 3) ──────────────────────────────────────
// The ONE canonical set. Every record-type comparison in the `aacs` module
// references these, so a type byte is never a bare literal scattered across
// files (the `0x0c` variant-data record in particular used to appear in several
// hand-rolled forms).
// The ONE canonical set; every record-type comparison in `aacs` references these
// so a type byte is never a bare literal scattered across files.
/// Type-and-Version — carries the 32-bit MKBType / AACS generation.
pub(crate) const REC_TYPE_AND_VERSION: u8 = 0x10;
@@ -157,7 +155,7 @@ impl AacsVersion {
}
/// Find Verify Media Key Record (type 0x81 for AACS 1.0, 0x86 for AACS 2.0/2.1) in MKB.
/// 0x81: [C] §3.2.5.1.4. 0x86 (AACS 2.x): [RE] — not in the public spec (from real 2.x MKBs).
/// 0x81: `[C]` §3.2.5.1.4. 0x86 (AACS 2.x): `[RE]` — not in the public spec (from real 2.x MKBs).
pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
// Verify-Media-Key record (0x81 for AACS 1.0, 0x86 for AACS 2.x): mk_dv is
// the 16 bytes at record offset 4 (body offset 0). Needs rec_len >= 20.
@@ -189,12 +187,12 @@ pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
}
}
/// Find Subset-Difference records (type 0x04) in MKB. [C] §3.2.5.1.5.
/// Find Subset-Difference records (type 0x04) in MKB. `[C]` §3.2.5.1.5.
pub(crate) fn mkb_find_subdiff_records(mkb: &[u8]) -> Option<Vec<u8>> {
find_record_body(mkb, 0x04)
}
/// Find the Media Key Data Record (cvalues table) in an MKB. [C] §3.2.4 / §3.2.5.1.7.
/// Find the Media Key Data Record (cvalues table) in an MKB. `[C]` §3.2.4 / §3.2.5.1.7.
///
/// The cvalue table is record type `0x05` (Media Key Data) on BOTH AACS
/// 1.0 and AACS 2.x MKBs — its 16-byte cvalue entries are 1:1 with the
@@ -331,7 +329,7 @@ impl MkbType {
}
/// The raw 32-bit MKBType field from the Type-and-Version record (0x10), bytes
/// 4-7. `None` if no 0x10 record is present. [C] §3.2.5.1.1 Table 3-2.
/// 4-7. `None` if no 0x10 record is present. `[C]` §3.2.5.1.1 Table 3-2.
pub fn mkb_type_raw(mkb: &[u8]) -> Option<u32> {
// Type-and-Version record (0x10): the 32-bit MKBType is bytes 4-7 (body
// offset 0). Needs rec_len >= 8 (4 header + 4 type).
@@ -516,7 +514,7 @@ mod tests {
assert_eq!(recs[1].body, vec![0x55; 16]);
}
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim ([C] §3.2.5.1.1
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim (`[C]` §3.2.5.1.1
/// Table 3-2), including a value this build does not recognise — the caller
/// uses it to tell "unknown MKB generation" from "no Type record at all".
/// All four bytes must come from the record body; reading any of them from
+13 -34
View File
@@ -122,18 +122,9 @@ pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String>
AacsRole::Mkb => v.push(format!("/{d}/MKBROM.AACS")),
AacsRole::ContentCert => v.push(format!("/{d}/CONTENT_CERT.AACS")),
AacsRole::UnitKey => {
// Glob VTKF*.AACS — the title-key filename is not fixed at
// VTKF000 (Freedom ships VTKF090 + VTKF100). Sorted for a
// deterministic try order.
//
// Each VTKF%%%.AACS is bound to ONE playlist (VPLST%%%.XPL): the
// TKF's 12-byte PLAYLIST_NAME field (bytes 0x10..0x1C) names the
// playlist whose Title Keys it carries, and keys from a TKF whose
// name does not match the title's playlist must not be used. The
// caller resolves this by trying candidates in sorted order and
// decrypting with the one whose keys verify — correct for a
// single-playlist disc; a name-matched selection keyed on the
// active playlist is the precise form for multi-playlist discs.
// Glob VTKF*.AACS (not fixed at VTKF000; Freedom ships VTKF090 +
// VTKF100), sorted for a deterministic try order. Each is bound to
// ONE playlist; caller tries in order, uses whichever's keys verify.
let mut names: Vec<&str> = dir
.entries
.iter()
@@ -157,7 +148,7 @@ pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String>
///
/// `read` performs the actual per-path read (full file or bounded prefix), so
/// callers share the same first-present walk regardless of read style. Returns
/// [`Error::AacsNoKeys`] if no candidate is present. Generic over the path
/// [`Error::AacsNoKeys`](crate::Error::AacsNoKeys) if no candidate is present. Generic over the path
/// element (`&str` or owned `String`) so it accepts the `Vec<String>` that
/// [`role_paths`] builds from the discovered HD DVD directory.
pub(crate) fn read_first<S, F>(candidates: &[S], mut read: F) -> crate::error::Result<Vec<u8>>
@@ -173,15 +164,9 @@ where
Err(crate::error::Error::AacsNoKeys)
}
// The module structure IS the public API — consumers import from the owning
// module directly (e.g. `aacs::content::decrypt_unit`, `aacs::mkb::MkbType`,
// `aacs::derive::{derive_vuk, resolve_candidate}`, `aacs::resolve::resolve_keys_v2`).
// The `derive::probe` reproduction harness stays reachable via its module path.
//
// A small set of flat re-exports is kept for the typed key primitives and the
// content-decrypt entry points that downstream key-source crates import through
// the `aacs::` path. These are the stable, load-bearing names; keeping them here
// lets those crates track the module refactor without a lockstep re-pin.
// The module structure IS the public API — consumers import from the owning module
// (e.g. `aacs::content::decrypt_unit`). A small set of flat re-exports below is kept
// for typed key primitives and content-decrypt entry points that downstream crates rely on.
pub use content::ALIGNED_UNIT_LEN;
pub use derive::derive_vuk;
pub use types::{DeviceKey, HostCert, MediaKey, ProcessingKey, UnitKey, Vid, Vuk};
@@ -199,19 +184,16 @@ mod tests {
#[test]
fn aligned_unit_len_is_three_2048_byte_sectors() {
// ALIGNED_UNIT_LEN is the AACS aligned-unit size: 3 × 2048 = 6144.
// Re-exported from decrypt; pin the value here so the public constant
// and the spec stay in lockstep.
// ALIGNED_UNIT_LEN is the AACS aligned-unit size (3 × 2048 = 6144);
// pin it here so the public constant tracks the spec.
assert_eq!(ALIGNED_UNIT_LEN, 6144);
assert_eq!(ALIGNED_UNIT_LEN, 3 * 2048);
}
#[test]
fn version_strides_are_reexported_and_distinct() {
// The three AACS generations are part of the public surface, and the
// V10 (48) vs V20/V21 (64) stride distinction is the load-bearing
// difference. Confirm the enum re-export is usable and the variants
// are distinct values.
// V10 (48) vs V20/V21 (64) stride is the load-bearing distinction;
// confirm the enum re-export is usable and variants are distinct.
assert_ne!(AacsVersion::V10, AacsVersion::V20);
assert_ne!(AacsVersion::V20, AacsVersion::V21);
}
@@ -236,11 +218,8 @@ mod tests {
}
// ── HD DVD AACS directory / filename discovery ────────────────────────
//
// The HD DVD AACS dir name and title-key filename are authoring-specific
// and were previously hardcoded to `/ANY!/VTKF000.AACS`. These verify the
// discovery replacement against both real-disc shapes: Freedom (`AAC!` +
// `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
// Dir/filename are authoring-specific (previously hardcoded to `/ANY!/VTKF000.AACS`);
// verify against Freedom (`AAC!` + `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
#[test]
fn role_paths_discovers_hddvd_dir_and_globs_all_vtkf_variants() {
+70 -134
View File
@@ -5,10 +5,9 @@ use super::derive::*;
use super::inf::*;
use super::mkb::*;
//
// Canonical form is `<category>1003` (low 16 bits `0x1003` is a fixed marker).
// Types 3/4/10 are from the AACS Common Cryptographic Elements spec (0.953,
// §3.2.5.1.1); the Category-C 2.0/2.1 values are the standard MKB type constants.
// §3.2.5.1.1); Category-C 2.0/2.1 values are the standard MKB type constants.
// ── Full VUK resolution chain ───────────────────────────────────────────────
@@ -136,13 +135,13 @@ pub(crate) fn classify_resolve_failure(ctx: &ResolveContext<'_>) -> ResolveFailu
}
/// AACS 1.0 key resolution. Parses `Unit_Key_RO.inf` with 48-byte
/// stride. Tries paths 1 → 4 in order.
/// stride. Tries paths 1 → 5 in order (via `resolve_keys_classical`).
pub fn resolve_keys_v1(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
resolve_keys_classical(ctx, AacsVersion::V10)
}
/// AACS 2.0 key resolution. Parses `Unit_Key_RO.inf` with 64-byte
/// stride. Tries paths 1 → 4 in order. When paths 3/4 succeed against
/// stride. Tries paths 1 → 5 in order. When paths 3/4 succeed against
/// an MKB carrying Variant records (`0x2d` / `0x2f`), the result's
/// `version` is upgraded to [`AacsVersion::V21`] — derivation still
/// runs through the classical V2 path; the V21-specific Variant chain
@@ -210,9 +209,8 @@ pub fn resolve_keys_v21(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
if has_vid {
// Path 1: Variant chain (V21's analogue of classical Path 1's DK
// derivation). Derive the Processing Key from device keys first (DK → PK
// via the variant walk), then run the PK → Km variant primitive and
// derive the per-disc VUK from Km + VID.
// derivation): DK → PK via variant walk, then PK → Km, then VUK
// from Km + VID.
if let Some(mkb) = ctx.mkb {
let recs = super::mkb::walk_mkb(mkb);
let all_dks = providers.device_keys();
@@ -242,10 +240,9 @@ pub fn resolve_keys_v21(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
// Path 3: pre-computed MK + matching VID → derived VUK.
// Short-circuit: first provider with a matching VID wins.
if let Some(entry) = providers.lookup_disc_by_vid(ctx.volume_id) {
// The entry already matched by VID and derive_vuk needs only mk +
// ctx.volume_id, so a provider that matches by VID without
// populating disc_id (e.g. a webservice) must not have its MK
// dropped — gate on the MK alone.
// Already matched by VID and derive_vuk needs only mk + volume_id, so a
// provider matching by VID without disc_id (e.g. a webservice) must not
// lose its MK — gate on the MK alone.
if let Some(mk) = entry.media_key {
let vuk = derive_vuk(&mk, ctx.volume_id);
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_v21_path3_hit", "MK+VID entry matched volume_id");
@@ -371,13 +368,9 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
}
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_miss", pk_count = all_pks.len(), "PK derivation failed");
// Path 2.5: MK-pool brute. keydb stores Media Keys per-disc, but an
// MK is MKB-scoped (shared across a pressing/MKB-family). A disc
// whose own hash/VID isn't keyed can still resolve if ANY stored MK
// verifies against its MKB. Try every distinct MK via km_verifies;
// a UNIQUE pass is this disc's Km → derive VUK (needs VID) → UK.
// One AES-D + magic check per candidate (cheap). mk_dv is hoisted
// out of the loop so the MKB is not re-walked per candidate.
// Path 2.5: MK-pool brute. An MK is MKB-scoped, so a disc whose own
// hash/VID isn't keyed can still resolve if ANY stored MK verifies (a
// UNIQUE pass via km_verifies is this disc's Km); mk_dv is hoisted out.
let mks = providers.media_keys();
let chosen_mk = mkb_find_mk_dv(mkb).and_then(|mk_dv| {
unique_verifying_mk(&mks, |mk| {
@@ -398,10 +391,9 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
// Path 3: pre-computed MK + matching VID → derived VUK.
// Short-circuit: first provider with a matching VID wins.
if let Some(entry) = providers.lookup_disc_by_vid(ctx.volume_id) {
// The entry already matched by VID and derive_vuk needs only mk +
// ctx.volume_id, so a provider that matches by VID without
// populating disc_id (e.g. a webservice) must not have its MK
// dropped — gate on the MK alone.
// Already matched by VID and derive_vuk needs only mk + volume_id, so a
// provider matching by VID without disc_id (e.g. a webservice) must not
// lose its MK — gate on the MK alone.
if let Some(mk) = entry.media_key {
let vuk = derive_vuk(&mk, ctx.volume_id);
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path3_hit", "MK+VID entry matched volume_id");
@@ -586,11 +578,9 @@ mod tests {
#[test]
fn derive_media_key_from_dk_survives_out_of_range_u_mask_shift() {
// Regression: a crafted/corrupt MKB with a Subset-Difference
// u_mask_shift of 32..=63 (passes the 0xC0 revoked-marker check but
// overflows `0xFFFF_FFFF << shift`) used to panic in debug / compute a
// wrong mask in release. The walk must now skip the bad slot and
// return cleanly (no panic) on disc-controlled bytes.
// Regression: a u_mask_shift of 32..=63 (passes the 0xC0 revoked-marker
// check but overflows `0xFFFF_FFFF << shift`) used to panic in debug /
// compute a wrong mask in release. The walk must skip the bad slot.
let mut mkb: Vec<u8> = Vec::new();
// 0x81 record: 4-byte header + 16-byte mk_dv body (rec_len = 20).
mkb.extend_from_slice(&[0x81, 0x00, 0x00, 0x14]);
@@ -616,14 +606,9 @@ mod tests {
#[test]
fn test_decrypt_unit_key_from_vuk() {
// VUK → encrypted unit key → unit key roundtrip. The keydb-sourced
// variant of this test (which scanned a real KEYDB for VUK + unit
// keys) moved to freemkv-keysources; this rebuilt version exercises
// the same AES-G primitive (decrypt_unit_key ∘ aes_ecb_encrypt under a
// VUK) with directly-constructed material, so it needs no parser and
// keeps the crypto covered in libfreemkv. `aes_ecb_encrypt` is
// pub(crate), reachable here but not from keysources — the reason this
// half stays.
// VUK → encrypted unit key → unit key roundtrip. The keydb-sourced variant
// moved to freemkv-keysources; this exercises the same AES-G primitive
// with directly-constructed material — `aes_ecb_encrypt` is pub(crate).
use super::super::crypto::aes_ecb_encrypt;
let vuk = [0x5Au8; 16];
// A few representative "decrypted" unit keys.
@@ -743,11 +728,9 @@ mod tests {
#[test]
fn trim_mkb_never_zeroes_an_unrecognised_mkb() {
// Regression: the 0.31.0 read_aacs_inputs path truncated the MKB to
// mkb_content_len() unconditionally. For an MKB whose first record the
// parser can't read, mkb_content_len() returns 0 → an unconditional
// truncate zeroed the MKB, so autorip sent an EMPTY MKB to the key
// service (or skipped the request). trim_mkb must leave it intact.
// Regression: 0.31.0 truncated the MKB to mkb_content_len() unconditionally.
// For an unparseable first record, mkb_content_len() returns 0, so autorip
// sent an EMPTY MKB to the key service. trim_mkb must leave it intact.
let unrecognised = vec![0xFFu8; 4096]; // first "rec_type" 0xFF, rec_len huge → content_len 0
assert_eq!(
mkb_content_len(&unrecognised),
@@ -809,11 +792,9 @@ mod tests {
#[test]
fn probe_try_pk_against_tables_accepts_planted_pk_rejects_corrupt() {
// Lock in the terminal PK scan (`try_pk_against_tables`) used by the
// production PK path (`derive_media_key_from_pk`). Plant a terminal PK
// whose derived Media Key satisfies a synthetic verify record; confirm
// the scan ACCEPTS it against caller-supplied SD/cvalue tables and
// REJECTS a 1-byte corruption.
// Lock in the terminal PK scan (`try_pk_against_tables`) used by
// `derive_media_key_from_pk`: a planted PK whose derived MK satisfies a
// synthetic verify record must be ACCEPTED; a 1-byte corruption REJECTED.
use super::super::crypto::aes_ecb_encrypt as enc;
let pk: [u8; 16] = [
@@ -855,11 +836,9 @@ mod tests {
#[test]
fn validate_processing_key_round_trip_with_nonzero_uv() {
// Synthesise a (pk, uv, mk, cvalue, mk_dv) tuple that satisfies the
// AACS PK-validation relation, then confirm validate_processing_key
// recovers mk. Catches the bugs that landed pre-fix:
// * uv XOR step was missing → mk wrong whenever uv != 0
// * AES-128E + 12-zero check instead of AES-128D + magic
// Synthesise a (pk, uv, mk, cvalue, mk_dv) tuple satisfying the AACS
// PK-validation relation; confirms validate_processing_key recovers mk.
// Catches pre-fix bugs: missing uv XOR, and AES-128E+zero-check vs 128D+magic.
use super::super::crypto::{aes_ecb_decrypt as dec, aes_ecb_encrypt as enc};
let pk: [u8; 16] = [
@@ -872,10 +851,8 @@ mod tests {
];
let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00];
// cvalue is what AES-128E(pk, mk') gives, where mk' = mk with the
// last-4-bytes-uv XOR pre-undone:
// mk_raw[12..16] = mk[12..16] XOR uv (so the validate step XORs
// uv back in and recovers mk).
// cvalue is AES-128E(pk, mk') where mk' = mk with last-4-bytes-uv XOR
// pre-undone, so the validate step XORs uv back in and recovers mk.
let mut mk_raw = mk;
for a in 0..4 {
mk_raw[12 + a] ^= uv[a];
@@ -907,16 +884,9 @@ mod tests {
assert!(validate_processing_key(&pk, &cvalue, &wrong_uv, &mk_dv).is_none());
}
// ── MKB cvalue-record selection (issue #259 / #281) ─────────────────
//
// The cvalue (Media Key Data) table is record 0x05; the
// Subset-Difference index is record 0x04 (the standard AACS MKB layout:
// 0x05 = cvalues, 0x04 = subset-difference index). Record 0x07
// (Explicit Subset-Difference Record) is NOT the cvalue table. On real
// in-drive AACS 2.x UHD MKBs 0x07 is small (~96 entries) while the 0x05
// table is large (181270 entries, 1:1 with 0x04). An earlier
// `mkb_find_cvalues` preferred 0x07, which under-tested the SD walk and
// broke the DK→walk path. The selector must prefer 0x05.
// ── MKB cvalue-record selection (issue #259 / #281): cvalue (Media Key Data)
// is record 0x05, not 0x07 (Explicit SD, small ~96 entries vs 0x05's 181270).
// An earlier `mkb_find_cvalues` wrongly preferred 0x07, breaking DK→walk.
/// Build a 4-byte MKB record header (type + 3-byte big-endian total
/// length, header included) and append `body`.
@@ -1012,12 +982,9 @@ mod tests {
#[test]
fn real_aacs2_samples_select_large_0x05_not_small_0x07() {
// Real in-drive AACS 2.x UHD MKBs carry BOTH a small 0x07
// Explicit-Subset-Difference record (96 16-byte entries) AND the
// large 0x05 Media Key Data / cvalue table (181270 entries, 1:1
// with the 0x04 index). The production selector must return the
// LARGE 0x05 body, not the small 0x07 one. This is the exact
// regression #259 found. Skips when no sample dir is present.
// Real AACS 2.x UHD MKBs carry BOTH a small 0x07 record (96 entries) AND
// the large 0x05 cvalue table (181270 entries); selector must return the
// LARGE 0x05 body — the exact regression #259. Skips if no sample dir.
let samples = [
"sample-a/MKB_RO.inf",
"sample-b/MKB_RO.inf",
@@ -1116,19 +1083,15 @@ mod tests {
#[test]
fn resolve_keys_skips_paths_2_through_4_when_vid_is_zero() {
// No VID -> paths 2/3/4 cannot succeed. The function must
// return None WITHOUT touching the MKB / device keys, so we
// can pass an MKB that would otherwise cause expensive
// derivation work — it must not be consumed.
// No VID -> paths 2/3/4 cannot succeed. Must return None WITHOUT touching
// the MKB/device keys, so an MKB that would cause expensive derivation
// work can be passed here and must not be consumed.
let uk_ro = minimal_unit_key_ro();
let zero_vid = [0u8; 16];
// A provider carrying a dummy processing key but NO disc entry that
// matches this disc. `disc_entry: None` preserves the negative-miss
// the test asserts: with VID=0, paths 1/2/3 are skipped and the
// path-4/5 hash lookup must MISS (a SuppliedKey returns its
// disc_entry unconditionally, so the planted entry would WRONGLY hit
// path 4 — None keeps the miss).
// Dummy processing key, NO matching disc entry: `disc_entry: None` keeps
// the path-4/5 hash lookup a miss (SuppliedKey returns disc_entry
// unconditionally, so a planted entry would WRONGLY hit path 4).
let keydb = SuppliedKey {
device_keys: Vec::new(),
processing_keys: vec![[0u8; 16]],
@@ -1153,10 +1116,9 @@ mod tests {
#[test]
fn resolve_keys_path4_still_runs_when_vid_is_zero() {
// Path 4 (disc-hash → VUK) doesn't need VID. Confirm the
// short-circuit doesn't block it: install a keydb entry whose
// disc_hash matches the fixture's hash, with a known VUK, and
// verify resolve_keys returns it with key_source = 4.
// Path 4 (disc-hash → VUK) doesn't need VID; confirm the short-circuit
// doesn't block it: install a keydb entry whose disc_hash matches the
// fixture with a known VUK, verify resolve_keys returns key_source = 4.
let uk_ro = minimal_unit_key_ro();
let hash = disc_hash(&uk_ro);
// `find_disc` lowercases the incoming hash; the entry map is
@@ -1195,10 +1157,9 @@ mod tests {
#[test]
fn resolve_keys_path5_uses_keydb_unit_keys_when_vuk_absent() {
// Path 5: an entry with no VUK but with pre-decrypted unit
// keys matching the disc's CPS-unit numbering decrypts the
// disc directly. Covers the ~4,572 U-only KEYDB entries
// (mostly MKBv76+ UHDs) that the resolver previously ignored.
// Path 5: an entry with no VUK but pre-decrypted unit keys matching the
// disc's CPS-unit numbering decrypts directly. Covers the ~4,572 U-only
// KEYDB entries (mostly MKBv76+ UHDs) the resolver previously ignored.
let uk_ro = minimal_unit_key_ro();
let hash = disc_hash(&uk_ro);
let hash_hex = disc_hash_hex(&hash).to_lowercase();
@@ -1237,11 +1198,9 @@ mod tests {
}
#[test]
fn resolve_keys_path5_rejects_partial_unit_key_coverage() {
// If the disc declares a CPS unit that's not in the KEYDB
// entry's unit_keys, path 5 must NOT half-decrypt the disc.
// The match function returns None and the resolver falls
// through to None overall (no other paths available in this
// setup).
// If the disc declares a CPS unit not in the KEYDB entry's unit_keys, path
// 5 must NOT half-decrypt: match returns None and the resolver falls
// through to None overall (no other paths available here).
let uk_ro = minimal_unit_key_ro();
let hash = disc_hash(&uk_ro);
let hash_hex = disc_hash_hex(&hash).to_lowercase();
@@ -1295,13 +1254,9 @@ mod tests {
);
// This disc's inf (its hash will NOT be in keydb).
let uk_ro = minimal_unit_key_ro();
// The sibling's MK is lifted directly into the MK pool: a KeyDb
// aggregated per-disc media_keys into media_keys(), but SuppliedKey
// does NOT harvest its disc_entry's media_key — it has an explicit
// media_keys field. `disc_entry: None` preserves the miss on this
// disc's own hash/VID (the sibling matches neither), so ONLY the
// MK-pool brute (km_verifies) can resolve it — exactly the path under
// test.
// The sibling's MK is lifted into the MK pool via SuppliedKey's explicit
// media_keys field; `disc_entry: None` keeps this disc's own hash/VID a
// miss, so ONLY the MK-pool brute (km_verifies) can resolve it.
let keydb = SuppliedKey {
device_keys: Vec::new(),
processing_keys: Vec::new(),
@@ -1407,9 +1362,7 @@ mod tests {
data[1] = 0x01;
assert!(!parse_content_cert(&data).unwrap().bus_encryption);
}
// ════════════════════════════════════════════════════════════════════
// Hardening additions
// ════════════════════════════════════════════════════════════════════
// ════════════════ Hardening additions ════════════════
// ── VUK derivation: spec relation VUK = AES-D(MK, VID) XOR VID ─────────
#[test]
fn derive_vuk_matches_spec_relation_explicitly() {
@@ -1467,11 +1420,9 @@ mod tests {
}
#[test]
fn stride_v10_is_48_v20_is_64_and_picks_distinct_keys() {
// AACS 1.0 stride = 48, AACS 2.0/2.1 stride = 64 (aacs/inf.rs).
// Lay keys at 64-byte stride. Parsing at V20 stride must pick exactly
// those keys; parsing the SAME bytes at V10 (48) stride would read the
// wrong (intermediate) bytes for key 2 onward — proving the stride
// selector matters.
// AACS 1.0 stride = 48, 2.0/2.1 = 64. Keys laid at 64-byte stride: V20
// parsing must pick exactly those keys, while V10 (48) parsing the SAME
// bytes would read wrong intermediate bytes for key 2 onward.
let data = build_unit_key_ro(2, 64);
let v20 = parse_unit_key_ro(&data, AacsVersion::V20).unwrap();
assert_eq!(v20.encrypted_keys.len(), 2);
@@ -1533,11 +1484,9 @@ mod tests {
}
#[test]
fn parse_unit_key_ro_rejects_when_keys_run_off_end() {
// Finding #5: 3 keys declared but the buffer holds only 2 strides plus
// 8 trailing bytes (not a full 3rd 16-byte key). The extraction loop
// breaks at the buffer end (never reading OOB), and the post-loop
// length check rejects the short list with None — a truncated/malformed
// .inf must NOT be silently accepted with fewer keys than declared.
// Finding #5: 3 keys declared but buffer holds only 2 strides + 8 spare
// bytes. Extraction loop breaks at buffer end (never OOB), and the
// post-loop length check rejects the short list rather than accepting fewer.
let uk_pos = 0x60usize;
let stride = 48usize;
// Room for keys at uk_pos+48 and uk_pos+48+48, then only 8 spare bytes
@@ -1579,10 +1528,9 @@ mod tests {
}
#[test]
fn parse_unit_key_ro_title_cps_mapping_first_play_top_menu_then_titles() {
// [20..22] first_play, [22..24] top_menu, [24..26] num_titles, then
// per-title 2-byte pad + 2-byte CPS unit at 26 + i*4 + 2. Each on-disc
// 1-based CPS number in `1..=num_uk` is validated and converted to a
// 0-based key index (per the AACS Unit_Key_RO format); an out-of-range number → 0.
// [20..22] first_play, [22..24] top_menu, [24..26] num_titles, then per-title
// 2-byte pad + 2-byte CPS unit at 26 + i*4 + 2. Each on-disc 1-based CPS
// number in `1..=num_uk` converts to a 0-based key index; out-of-range → 0.
let mut data = build_unit_key_ro(4, 64); // num_uk = 4 → CPS 1..=4 valid
data[20..22].copy_from_slice(&1u16.to_be_bytes()); // first_play CPS 1
data[22..24].copy_from_slice(&2u16.to_be_bytes()); // top_menu CPS 2
@@ -2046,17 +1994,9 @@ mod tests {
#[test]
fn derive_media_key_and_pk_from_dk_returns_intermediate_pk() {
// Regression: a classical DK boil must yield the intermediate
// Processing Key, not just the Media Key. The key service banks the
// PK lineage (DK·PK·MK·VUK·UK); before the `_and_pk_` form existed it
// recovered the MK here but lost the PK silently.
//
// Build a minimal classical MKB (no 0x82/0x83) with:
// - 0x04 Subset-Difference: u_mask_shift=3, uv=0x00000002
// - 0x05 cvalues: one cvalue C planted so AES-D(Kp, C) XOR uv == mk
// - 0x86 Verify Media Key: mk_dv = AES-E(mk, magic || pad)
// and a DK with node=4, uv=2, u_mask_shift=3 so dev_key_v_mask ==
// v_mask: the calc_pk_from_dk loop is a no-op and Kp == aesg3(dk, 1).
// Regression: a classical DK boil must yield the intermediate PK too
// (key service banks DK·PK·MK·VUK·UK; before `_and_pk_` it lost the PK).
// Builds a minimal MKB with a DK whose calc_pk_from_dk walk is a no-op.
use super::super::crypto::aes_ecb_encrypt as enc;
let dk_bytes: [u8; 16] = [
@@ -2126,12 +2066,8 @@ mod tests {
}
// ── resolve_keys_with_reason / classify_resolve_failure ────────────────
//
// The rc.6 E7017/E7022 split is also exercised end-to-end through the
// `ensure_decryptable` gate in `disc/mod.rs`. These tests pin the
// *classifier* directly at the aacs::resolve seam and cover the branches the
// gate test does not: VID-present (must never be VidUnavailable), the
// processing-keys-only material path, and the version dispatch / Ok path.
// Also exercised end-to-end via `ensure_decryptable` in `disc/mod.rs`; these
// pin the *classifier* directly, covering branches that gate test doesn't.
/// A `SuppliedKey` provider with the given derivation material and no
/// disc-keyed entry. Mirrors the construction the gate test uses, lifted to
+5 -7
View File
@@ -181,7 +181,8 @@ pub fn clip_byte_to_lba(extents: &[crate::disc::Extent], clip_byte: u64) -> Opti
/// to clean TS and garbles the other ~40 (the second interleaved half), which the
/// demux then drops — yielding one coherent stream. Ranges outside every segment
/// are left for the map's default (the ordinary Unit Key). A segment that straddles
/// a UDF extent boundary is emitted as one range per whole-sector slice it covers.
/// a UDF extent boundary is skipped entirely, falling back to the Unit Key rather
/// than emitting a wrong span.
///
/// The result feeds [`AacsKeyMap::from_ranges`](crate::decrypt::AacsKeyMap::from_ranges)
/// with the Unit-Key index as the default — the same structure the CPS map uses,
@@ -200,12 +201,9 @@ pub fn fmts_key_ranges(
}
let start_byte = s.start_spn as u64 * SOURCE_PACKET_LEN;
let end_byte = (s.end_spn as u64 + 1) * SOURCE_PACKET_LEN; // exclusive
// A segment is unit-aligned and contiguous in clip bytes; map its first
// and last sector to LBAs. Segments are ~480 KB and extents are GB-sized,
// so a segment almost never crosses an extent boundary — but if the two
// ends land in different extents (non-contiguous LBAs), skip rather than
// emit a wrong span; the units there fall to the Unit Key (garble+drop),
// never a mis-decrypt.
// Map first/last sector to LBAs. Segments (~480 KB) rarely cross the
// GB-sized extent boundary, but if the two ends land in different
// extents, skip rather than emit a wrong span (falls back to Unit Key).
let (Some(a), Some(b)) = (
clip_byte_to_lba(extents, start_byte),
clip_byte_to_lba(extents, end_byte - 1),
+6 -16
View File
@@ -111,17 +111,9 @@ pub struct DiscEntry {
pub unit_keys: Vec<(u32, [u8; 16])>,
}
// ── Redacting `Debug` impls ──────────────────────────────────────────────────
//
// Every type above carries AACS secret material (device keys, host PRIVATE keys,
// media/volume/processing/unit keys). `#[derive(Debug)]` would print those bytes
// verbatim, so a stray `debug!("{:?}", …)` or a panic message would leak the
// keys. These hand-written impls print only NON-secret shape (presence, lengths,
// tree coordinates, indices) — never key bytes. `decrypt::DecryptKeys` follows
// the same policy by omitting `Debug` entirely; here we keep `Debug` because
// these are `PartialEq`/`Eq` value types used in `assert_eq!` and nested inside
// other `#[derive(Debug)]` structs, so the trait must exist — just not leak.
// Guarded by `redaction_tests` below.
// ── Redacting `Debug` impls: types above carry AACS secrets; derived `Debug`
// would leak key bytes via `debug!`/panics. These print only non-secret shape,
// kept (not omitted) since used in `assert_eq!`. See `redaction_tests`.
impl std::fmt::Debug for DeviceKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
@@ -270,11 +262,9 @@ mod unit_key_tests {
mod redaction_tests {
use super::*;
// Sentinel key byte 0xD5 = decimal 213. A derived `Debug` prints `[u8;N]`
// as decimal, so a leaked key surfaces the substring "213"; the redacting
// impls must not. No non-secret field below is 213, so "213" appearing means
// key bytes leaked. Each type must also carry a "redacted" marker (or omit
// the secret entirely) so re-adding `#[derive(Debug)]` fails this test.
// Sentinel byte 0xD5 = decimal 213: a derived `Debug` prints `[u8;N]` as
// decimal, so a leak surfaces "213" (no non-secret field is 213). Each type
// must also carry a "redacted" marker so re-adding `#[derive(Debug)]` fails.
const S: u8 = 0xD5;
fn assert_redacted(what: &str, dbg: &str) {
+68 -125
View File
@@ -54,11 +54,9 @@ use super::crypto::{aes_ecb_decrypt, aes_g};
use super::mkb::*;
use super::types::DeviceKey;
// The MKB record types this chain selects — `REC_MEDIA_KEY_VARIANT_DATA`
// (`0x0c`, the per-slot C table), `REC_VARIANT_DATA_AND_NONCE` (`0x2d`, VARIANTS
// + tail Nonce), `REC_VKD_TABLE` (`0x2f`), the subset-difference / cvalue records
// (`0x04` / `0x05` / `0x07`), and the verify records (`0x81` / `0x86`) — are the
// canonical set in [`super::mkb`], in scope here via the `use super::mkb::*` glob.
// MKB record types this chain selects: `REC_MEDIA_KEY_VARIANT_DATA` (`0x0c`),
// `REC_VARIANT_DATA_AND_NONCE` (`0x2d`), `REC_VKD_TABLE` (`0x2f`), the
// subset-difference/cvalue and verify records — canonical set in [`super::mkb`].
// ── Public constants ──────────────────────────────────────────────────────
@@ -137,10 +135,9 @@ pub(crate) fn variant_key_data(records: &[MkbRecord]) -> Option<&[u8]> {
// ── Subset-difference walk that exposes (Kp, uv) ──────────────────────────
// `calc_v_mask` and `calc_pk_from_dk` (and the AES-G3 seed step they ride
// on) are shared with the classical walk in [`super::derive`] — a single
// definition keeps the variant SD tree byte-identical to the classical one.
// (`aesg3` itself is imported separately in the test module.)
// `calc_v_mask`/`calc_pk_from_dk` (and the AES-G3 seed step) are shared with
// the classical walk in [`super::derive`], keeping the variant SD tree
// byte-identical. `aesg3` itself is imported separately in the test module.
use super::derive::{calc_pk_from_dk, calc_v_mask};
/// Outcome of a subset-difference walk against an MKB. Carries the
@@ -190,10 +187,11 @@ fn mkb_find_mk_dv(records: &[MkbRecord]) -> Option<[u8; 16]> {
/// separate on purpose and select MKB records in DELIBERATELY different
/// order:
///
/// - cvalues: this variant walk tries record `0x07`-then-`0x05`; the
/// classical walk tries `0x05`-then-`0x07`. On a variant MKB the
/// small `0x07` Explicit-Subset-Difference record carries the
/// cvalue the Precursor chain consumes, whereas a classical UHD MKB
/// - cvalues: this variant walk tries record `0x0c`-then-`0x07`-then-`0x05`
/// (`0x0c` is the real-disc per-uv cvalue source; `0x07`/`0x05` are
/// fixture fallbacks); the classical walk tries `0x05`-then-`0x07`. On a
/// variant MKB the small `0x07` Explicit-Subset-Difference record carries
/// the cvalue the Precursor chain consumes, whereas a classical UHD MKB
/// keeps its 1:1 cvalue table in the large `0x05` record (see the
/// note on [`super::derive::probe::mkb_cvalues`]). They must NOT be
/// unified to one order — each is correct for its own MKB shape.
@@ -211,10 +209,8 @@ pub fn walk_processing_key(
) -> Option<ProcessingKeyMatch> {
let mk_dv = mkb_find_mk_dv(records)?;
let uvs = mkb_find_body(records, REC_SUBSET_DIFFERENCE)?;
// Variant cvalue source: a real variant MKB carries its per-uv cvalue table
// in record `0x0c` (confirmed 46,101×16, one per `0x04` subset-difference
// slot). Fall back to `0x07`/`0x05` for the synthetic fixtures and any MKB
// shape that keeps its cvalues there.
// Real variant MKBs carry per-uv cvalues in record `0x0c` (46,101x16, one
// per `0x04` slot); fall back to `0x07`/`0x05` for synthetic fixtures.
let cvalues = mkb_find_body(records, REC_MEDIA_KEY_VARIANT_DATA)
.or_else(|| mkb_find_body(records, REC_EXPLICIT_SUBSET_DIFF))
.or_else(|| mkb_find_body(records, REC_MEDIA_KEY_DATA))?;
@@ -229,17 +225,13 @@ pub fn walk_processing_key(
for uvs_idx in 0..num_uvs {
let p_uv = &uvs[1 + 5 * uvs_idx..];
// `num_uvs` was computed by `take_while(.. (c[0] & 0xC0) == 0)`, so
// every chunk in `0..num_uvs` already has its revoked-marker bits
// clear — that `take_while` is the single authoritative place the
// parse stops, no inner re-check needed.
// `num_uvs` came from `take_while(.. (c[0] & 0xC0) == 0)`, so every
// chunk in `0..num_uvs` already has clear revoked-marker bits.
let u_mask_shift = uvs[5 * uvs_idx];
// 0x20..=0x3F (32..=63) have their revoked-marker bits clear (so they
// pass the take_while above) but are out of range for a u32 shift.
// `wrapping_shl` would silently compute shift % 32 (e.g. 32 → no shift
// → 0xFFFF_FFFF), matching a wrong uv slot and deriving a wrong key.
// Disc-controlled byte: skip the slot instead.
// 0x20..=0x3F pass the take_while but are out of range for a u32
// shift; `wrapping_shl` would silently wrap (shift % 32) and match
// a wrong uv slot. Disc-controlled byte: skip the slot instead.
if u_mask_shift >= 32 {
continue;
}
@@ -281,12 +273,9 @@ pub fn walk_processing_key(
}
let dec_vd = aes_ecb_decrypt(&km_candidate, &mk_dv);
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
// On a classical (non-variant) MKB this magic must
// match. On a variant MKB it won't — `km_candidate`
// is really Kmp and the magic check is moot. We
// still gate the walk on cvalue indexing being
// sane; the chain itself enforces the variant
// semantics downstream.
// On classical MKBs this magic must match. On variant MKBs
// it won't — `km_candidate` is really Kmp, so the magic
// check is moot; the chain enforces semantics downstream.
let classical_ok = dec_vd[..8] == VERIFY_MAGIC;
let variant_present = is_variant_mkb(records);
if !(classical_ok || variant_present) {
@@ -375,10 +364,9 @@ impl std::error::Error for MediaKeyVariantError {}
/// silent bad key).
fn variants_for_uv(records: &[MkbRecord], sd_slot_index: usize) -> Option<u16> {
let body = variant_data_record(records)?;
// The VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at
// the TAIL (see [`variant_nonce`]). Bound the read to the table region so a
// near-end slot can never read Nonce bytes as a VARIANTS entry. NO leading
// header (measured: a v70 `0x2d` body = 46_100*2 + 16 = 92_216).
// VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at the
// TAIL (see [`variant_nonce`]). Bound reads to the table region so a near-end
// slot never reads Nonce bytes (no header: v70 `0x2d` body = 46_100*2+16).
const NONCE: usize = 16;
let table_len = body.len().checked_sub(NONCE)?;
let off = sd_slot_index.checked_mul(2)?;
@@ -532,10 +520,9 @@ pub fn derive_media_key_variant(
}
let nonce = variant_nonce(mkb_records).ok_or(MediaKeyVariantError::MkbIncomplete)?;
let vkd_table = variant_key_data(mkb_records).ok_or(MediaKeyVariantError::MkbIncomplete)?;
// C for the Kmp step is the per-subset-difference `0x0c` table (one 16-byte
// C per slot) — the SAME source and index `walk_processing_key` uses. `0x2d`
// holds VARIANTS + Nonce, NOT C. Fall back to `0x07`/`0x05` for the synthetic
// fixtures that keep a single cvalue there.
// C for Kmp is the per-slot `0x0c` table, same source/index as
// `walk_processing_key` uses; `0x2d` holds VARIANTS + Nonce, not C. Fall
// back to `0x07`/`0x05` for synthetic fixtures with a single cvalue.
let cvalues = mkb_find_body(mkb_records, REC_MEDIA_KEY_VARIANT_DATA)
.or_else(|| mkb_find_body(mkb_records, REC_EXPLICIT_SUBSET_DIFF))
.or_else(|| mkb_find_body(mkb_records, REC_MEDIA_KEY_DATA))
@@ -672,11 +659,9 @@ mod tests {
#[test]
fn calc_pk_from_dk_terminates_on_nonconvergent_mask() {
// Regression for the unbounded-loop hang: pick a (dev_key_v_mask,
// v_mask) pair the arithmetic `>> 1` walk can never reconcile.
// dev_key_v_mask has the MSB set, so `>> 1` sign-extends and the
// mask saturates at 0xFFFF_FFFF, never reaching a coarser v_mask.
// The 32-step bound must let this return rather than spin forever.
// Regression for the unbounded-loop hang: dev_key_v_mask has the MSB
// set, so the `>> 1` walk sign-extends and saturates at 0xFFFF_FFFF,
// never reaching v_mask. The 32-step bound must still return.
let dk = [0x11u8; 16];
let pk = calc_pk_from_dk(&dk, 0x0000_0002, 0x0000_0000, 0xFFFF_FFFE);
// Bounded exit yields *some* key; we only assert it terminated.
@@ -844,9 +829,8 @@ mod tests {
mkb.extend_from_slice(&[0x04, 0x00, 0x00, 0x09]);
mkb.extend_from_slice(&[0x03, 0x00, 0x00, 0x00, 0x02]);
// Pick a known DK; with dk.uv == MKB.uv (==2) and
// dk.u_mask_shift == MKB.u_mask_shift (==3), dev_key_v_mask
// equals the MKB's v_mask and the calc_pk_from_dk loop is a
// Known DK with dk.uv == MKB.uv (==2) and dk.u_mask_shift == MKB's
// (==3): dev_key_v_mask == MKB's v_mask, so calc_pk_from_dk is a
// no-op — Kp = aesg3(dk, 1).
let dk_bytes: [u8; 16] = [
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE,
@@ -862,12 +846,9 @@ mod tests {
aes_d_result[15] ^= 0x02;
let c_block = aes_ecb_encrypt(&kp, &aes_d_result);
// cvalues record (0x07): the per-SD C the chain reads for `Kmp`. This
// fixture has no `0x0c`, so both the walk and the chain fall back to
// `0x07` — plant the computed `c_block` HERE so `AES-D(Kp, C) XOR uv ==
// Kmp` and the chosen `kmp15` bit lands. On a variant MKB the per-match
// magic check fails, but `variant_present` is true, so the walk still
// returns the match.
// cvalues record (0x07): no `0x0c` here, so walk and chain both fall
// back to `0x07` — plant `c_block` so AES-D(Kp,C)^uv == Kmp. Magic
// check fails on variant MKBs, but `variant_present` lets it match.
mkb.extend_from_slice(&[0x07, 0x00, 0x00, 0x14]);
mkb.extend_from_slice(&c_block);
@@ -939,14 +920,9 @@ mod tests {
#[test]
fn walk_mkb_be24_middle_byte_is_honored() {
// A record longer than 255 bytes needs the MIDDLE BE24 byte: total
// length 0x00_0110 (272) is `[0x00, 0x01, 0x10]`, so a parser reading
// only the low byte sees 0x10. The HIGH byte of this length is zero, so
// this test says nothing about the `<< 16` term — that is pinned
// separately by `mkb::tests::mkb_records_honors_the_high_byte_of_the_be24_length`,
// which uses a 0x01_0004 record. (Renamed from
// `walk_mkb_be24_high_byte_is_honored`, which claimed coverage this body
// does not deliver.)
// Needs the MIDDLE BE24 byte: length 0x00_0110 (272) is [0x00,0x01,0x10],
// so a low-byte-only reader sees 0x10. HIGH byte is zero here; that term
// is pinned by `mkb::tests::mkb_records_honors_the_high_byte_of_the_be24_length`.
let total = 0x0110usize; // 272
let mut mkb = vec![0x10, 0x00, 0x01, 0x10];
mkb.resize(total, 0xAB);
@@ -1049,10 +1025,9 @@ mod tests {
#[test]
fn chain_yields_no_key_for_non_covering_pk() {
// A complete variant MKB but a Processing Key that covers no slot → no
// Km verifies → an error (never a key). A non-covering key resolves to
// ProcessingKeyUnavailable, or to a correction-mode classification if its
// Kmp happens to set the soft/online bit — either way, no key is emitted.
// Complete variant MKB, but PK covers no slot → no Km verifies → error.
// Resolves to ProcessingKeyUnavailable, or a correction-mode
// classification if Kmp sets the soft/online bit; either way, no key.
let (recs, _dk, _, _) = synthetic_variant_setup(0x00);
let out = derive_media_key_variant(&recs, &[0x11; 16]);
assert!(out.is_err(), "non-covering PK must not yield a Media Key");
@@ -1062,9 +1037,8 @@ mod tests {
#[test]
fn chain_reports_mkb_incomplete_when_nonce_missing() {
// Build a variant MKB (still variant via 0x2f, and a DK can walk it)
// but WITHOUT the 0x2d record that carries C + the trailing Nonce →
// MkbIncomplete at the variant_nonce `?`.
// Variant MKB (still variant via 0x2f, DK can walk it) but WITHOUT the
// 0x2d record carrying C + Nonce → MkbIncomplete at variant_nonce `?`.
let (recs, _dk, kp, _) = synthetic_variant_setup(0x00);
// Reconstruct bytes without the 0x2d record.
let mut mkb = Vec::new();
@@ -1088,10 +1062,9 @@ mod tests {
#[test]
fn walk_processing_key_skips_shift_32_to_63_without_panic() {
// A subset-difference u_mask_shift in 0x20..=0x3F passes the 0xC0
// revoke check but is out of range for a u32 shift. The walk must skip
// the slot (continue) and not panic / not match a wrong uv. With only
// that one bad slot, no match → None.
// u_mask_shift in 0x20..=0x3F passes the 0xC0 revoke check but is out
// of u32-shift range. Walk must skip the slot without panicking; with
// only this bad slot present, result is None.
let mut mkb = vec![
0x10, 0x00, 0x00, 0x0C, 0x48, 0x14, 0x10, 0x03, 0x00, 0x00, 0x00, 0x4D,
];
@@ -1211,22 +1184,9 @@ mod tests {
assert_eq!(err, MediaKeyVariantError::SoftCorrectionRequired);
}
// ════════════════════════════════════════════════════════════════════
// A COMPLETE variant MKB — the AACS 2.1 happy path
//
// Every other test in this module asserts an ERROR classification, so
// until now no test ever drove `derive_media_key_variant` to a Media
// Key. That left the whole success path — the VARIANTS lookup, the VKD
// selection, the final `Km` unwrap and the verify gate — pinned by
// nothing: a body that answered a constant for any of those steps still
// produced the same errors these tests expect.
//
// No real key material is involved. Every AACS 2.1 relation in the chain
// is invertible, so the fixture below picks a Media Key and a Processing
// Key and computes the MKB records that connect them, exactly as
// `derive::position_recovery_tests::plant_mkb` does for the classical
// chain.
// ════════════════════════════════════════════════════════════════════
// A COMPLETE variant MKB — the AACS 2.1 happy path. Every other test here
// asserts an ERROR, leaving the success path (VARIANTS/VKD lookup, Km
// unwrap, verify gate) unpinned; fixture inverts the chain, no real keys.
/// A planted variant MKB and the values it was built from.
struct PlantedVariant {
@@ -1278,11 +1238,9 @@ mod tests {
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
];
// `uv = 2` puts its only non-zero byte at index 15, so byte 15 is the ONE
// position where the `Km`/`Kmp` uv-XOR is observable. Its 0x02 bit is
// deliberately CLEAR: with the bit set, `km[15] ^= 2` and `km[15] |= 2`
// agree (the XOR would only be clearing a bit the OR re-sets) and an
// OR-for-XOR substitution in the final step would be invisible.
// `uv = 2` has its only non-zero byte at index 15, the one position where
// the uv-XOR is observable. Its 0x02 bit is deliberately CLEAR: if set,
// `^= 2` and `|= 2` would agree, hiding an OR-for-XOR substitution bug.
let km: [u8; 16] = [
0xC0, 0xC1, 0xC2, 0xC3, 0xC4, 0xC5, 0xC6, 0xC7, 0xC8, 0xC9, 0xCA, 0xCB, 0xCC, 0xCD,
0xCE, 0xCD,
@@ -1613,9 +1571,8 @@ mod tests {
);
// Wrong C block: EVERY one-bit neighbour must fail to produce a key.
// (Which classification it lands in depends on the two condition bits
// the perturbed Kmp happens to carry — the property being pinned is
// that none of the 128 reaches `Ok`.)
// Classification varies with the perturbed Kmp's condition bits; the
// property pinned here is that none of the 128 reaches `Ok`.
for byte in 0..16usize {
for bit in 0..8u32 {
let mut c_bad = p.c_block;
@@ -1687,20 +1644,9 @@ mod tests {
);
}
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT variant MKB driven by a real DEVICE KEY
//
// `walk_processing_key` is the DK -> Kp step that feeds the whole 2.1
// chain. Every existing test of it either asserts `None` (out-of-range
// shift, uv == 0) or asserts only that SOME match came back — none pins
// WHICH Processing Key, cvalue or slot index it returns. And every one of
// them uses a SINGLE-slot MKB, where the slot index is 0: all the
// `uvs[1 + 5*idx]` / `cvalues[idx*16..]` stride arithmetic multiplies by
// zero and any stride at all gives the same answer.
//
// This fixture puts the covering slot at index 1, behind a decoy at
// index 0, so the strides are load-bearing.
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT variant MKB driven by a real DEVICE KEY. Prior tests used
// single-slot MKBs (index 0), where stride arithmetic multiplies by zero;
// this puts the covering slot at index 1 behind a decoy, so strides matter.
/// A two-slot variant MKB whose SECOND slot is opened by a device key.
struct PlantedWalk {
@@ -1718,7 +1664,7 @@ mod tests {
/// Positions follow the same reasoning as the classical
/// `derive::position_recovery_tests::plant_mkb`: `uv = 0x0400`
/// (`u_mask_shift = 12`) with a device node of `0x0C00` satisfies the
/// [C] §3.2.4 gate — equal under `u_mask = 0xFFFF_F000`, different under
/// `[C]` §3.2.4 gate — equal under `u_mask = 0xFFFF_F000`, different under
/// `v_mask = 0xFFFF_F800`. The device key's own `uv` equals the slot's, so
/// `dev_key_v_mask == v_mask` and [`calc_pk_from_dk`] descends zero levels:
/// `Kp = AES-G3(dk, 1)`, written out explicitly below rather than taken from
@@ -1763,10 +1709,9 @@ mod tests {
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&km, &vd);
// ── C blocks. Both are built so `Kmp[15]` has the 0x02 / 0x04 condition
// bits CLEAR, so both slots run the default-KCD path to completion and
// the decoy is rejected by the terminal verify gate rather than
// short-circuiting into a correction-mode classification.
// C blocks: both keep `Kmp[15]`'s 0x02/0x04 bits CLEAR, so both slots
// run the default-KCD path and the decoy is rejected by the terminal
// verify gate instead of short-circuiting into correction-mode.
let c_for = |kmp: &[u8; 16], uv: u32| -> [u8; 16] {
let mut c_plain = *kmp;
for (b, u) in c_plain[12..16].iter_mut().zip(uv.to_be_bytes()) {
@@ -1864,7 +1809,7 @@ mod tests {
/// them — and produced a Processing Key that opens nothing.
///
/// The expected `Kp` is written as the explicit `AES-G3(dk, 1)` zero-descent
/// relation from [C] §3.2.4, not taken from the walk's own output.
/// relation from `[C]` §3.2.4, not taken from the walk's own output.
#[test]
fn walk_processing_key_returns_the_covering_slots_key_cvalue_and_index() {
let p = plant_walk_variant_mkb();
@@ -1893,7 +1838,7 @@ mod tests {
);
}
/// The gate the walk applies is [C] §3.2.4's subset-difference test, and a
/// The gate the walk applies is `[C]` §3.2.4's subset-difference test, and a
/// device key that fails it must get NO match. Pinned across all four
/// coordinates the gate reads — node, uv, u_mask_shift and the key bytes —
/// because a body that dropped any half of the gate would hand back a
@@ -2000,7 +1945,7 @@ mod tests {
}
/// The classical-magic escape hatch. On a NON-variant MKB the walk must
/// return a match only when `AES-D(Kmp, mk_dv)` opens with the [C] §3.2.5.1.4
/// return a match only when `AES-D(Kmp, mk_dv)` opens with the `[C]` §3.2.5.1.4
/// verify magic; on a variant MKB that relation does not hold (the walk
/// yields a Precursor) and the presence of `0x2d`/`0x2f` is what lets the
/// match through to the chain's own terminal gate.
@@ -2036,7 +1981,7 @@ mod tests {
/// The OTHER half of `classical_ok || variant_present`: a non-variant MKB
/// whose cvalue really does open the Verify-Media-Key magic must yield a
/// match, and the [C] §3.2.4 relation that produces the candidate — AES-D(Kp,
/// match, and the `[C]` §3.2.4 relation that produces the candidate — AES-D(Kp,
/// cvalue) with `uv` XORed into the LOW FOUR BYTES — must be computed
/// exactly.
///
@@ -2059,11 +2004,9 @@ mod tests {
];
// Zero descent ([C] §3.2.4), written out as the primitive relation.
let kp = aesg3(&dkey, 1);
// `uv = 0x0400` puts its only non-zero byte at index 14, so byte 14 is
// the ONE position where the `uv` XOR is observable at all. Its 0x04 bit
// is deliberately CLEAR here: with the bit set, `km_candidate[14] |=
// 0x04` and `^= 0x04` agree (the XOR would only be clearing a bit the OR
// re-sets), and an OR-for-XOR substitution would be invisible.
// `uv = 0x0400` has its only non-zero byte at index 14, the one position
// where the uv XOR is observable. Its 0x04 bit is CLEAR: if set, `|= 0x04`
// and `^= 0x04` would agree, hiding an OR-for-XOR substitution bug.
let mk: [u8; 16] = [
0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7A, 0x7B, 0x7C, 0x7D,
0x7A, 0x7F,
+149
View File
@@ -0,0 +1,149 @@
//! Parse `/BDMV/index.bdmv` — the BD navigation index (First Play, Top Menu,
//! and the title table), per its documented binary layout. This is read as a
//! documented binary format, never executed: every field is bounds-checked and
//! any malformed input yields `None` (the nav resolver then abstains).
//!
//! Layout: `"INDX"` + version(4) + `indexes_start`(u32 @8) + … ; at
//! `indexes_start`: `index_len`(u32), First-Play object(12), Top-Menu object(12),
//! `num_titles`(u16), then `num_titles` title objects(12 each). Every object's
//! `object_type` is the top two bits of its first byte; for an HDMV object the
//! `id_ref` into `MovieObject.bdmv` is a big-endian u16 at object offset 6.
/// One playback/title object in `index.bdmv`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum PlaybackObj {
/// HDMV title. `id_ref` indexes `MovieObject.bdmv`; `0xffff` means "no
/// object".
Hdmv { id_ref: u16 },
/// BD-J title (a Java Xlet chooses what plays — the nav VM cannot resolve
/// it and must abstain).
BdJ,
/// Unrecognised object type.
Unknown,
}
/// The parsed index: the two entry objects plus the title table (title numbers
/// `1..=titles.len()` map to `titles[i-1]`; title 0 is the Top Menu).
#[derive(Debug, Clone)]
pub(crate) struct Index {
pub first_play: PlaybackObj,
pub top_menu: PlaybackObj,
pub titles: Vec<PlaybackObj>,
}
const OBJ_LEN: usize = 12;
/// Sanity cap on the title count (real discs have well under this).
const MAX_TITLES: usize = 4096;
fn be_u16(d: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*d.get(o)?, *d.get(o + 1)?]))
}
fn be_u32(d: &[u8], o: usize) -> Option<u32> {
Some(u32::from_be_bytes([
*d.get(o)?,
*d.get(o + 1)?,
*d.get(o + 2)?,
*d.get(o + 3)?,
]))
}
/// Parse one 12-byte object starting at `o`. `object_type` is the top two bits
/// of byte 0; the object body starts at byte 4, and an HDMV `id_ref` is the
/// big-endian u16 at body offset +2 (object offset +6).
fn parse_obj(d: &[u8], o: usize) -> Option<PlaybackObj> {
let b0 = *d.get(o)?;
d.get(o + OBJ_LEN - 1)?; // require the whole 12-byte record
Some(match (b0 >> 6) & 0x3 {
1 => PlaybackObj::Hdmv {
id_ref: be_u16(d, o + 6)?,
},
2 => PlaybackObj::BdJ,
_ => PlaybackObj::Unknown,
})
}
/// Parse `index.bdmv`. Returns `None` on any structural problem.
pub(crate) fn parse(d: &[u8]) -> Option<Index> {
if d.get(0..4)? != b"INDX" {
return None;
}
// version at 4..8 ("0100"/"0200"/"0300") is not load-bearing for resolution.
let indexes_start = be_u32(d, 8)? as usize;
// At indexes_start: u32 index_len, First-Play(12), Top-Menu(12), u16 titles.
let mut o = indexes_start.checked_add(4)?;
let first_play = parse_obj(d, o)?;
o = o.checked_add(OBJ_LEN)?;
let top_menu = parse_obj(d, o)?;
o = o.checked_add(OBJ_LEN)?;
let num_titles = be_u16(d, o)? as usize;
o = o.checked_add(2)?;
if num_titles == 0 || num_titles > MAX_TITLES {
return None;
}
let span = num_titles.checked_mul(OBJ_LEN)?;
if o.checked_add(span)? > d.len() {
return None;
}
let mut titles = Vec::with_capacity(num_titles);
for i in 0..num_titles {
titles.push(parse_obj(d, o + i * OBJ_LEN)?);
}
Some(Index {
first_play,
top_menu,
titles,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a 12-byte object record: `object_type` in the top two bits of
/// byte 0, HDMV `id_ref` big-endian at offset 6.
fn hdmv_obj(id_ref: u16) -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = 1 << 6;
b[6..8].copy_from_slice(&id_ref.to_be_bytes());
b
}
fn bdj_obj() -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = 2 << 6;
b
}
fn build(first: [u8; 12], top: [u8; 12], titles: &[[u8; 12]]) -> Vec<u8> {
let indexes_start = 48u32;
let mut d = vec![0u8; indexes_start as usize];
d[0..4].copy_from_slice(b"INDX");
d[4..8].copy_from_slice(b"0300");
d[8..12].copy_from_slice(&indexes_start.to_be_bytes());
d.extend_from_slice(&0u32.to_be_bytes()); // index_len (unused by parser)
d.extend_from_slice(&first);
d.extend_from_slice(&top);
d.extend_from_slice(&(titles.len() as u16).to_be_bytes());
for t in titles {
d.extend_from_slice(t);
}
d
}
#[test]
fn parses_hdmv_first_play_and_titles() {
let d = build(hdmv_obj(0), bdj_obj(), &[bdj_obj(), bdj_obj()]);
let idx = parse(&d).expect("parses");
assert_eq!(idx.first_play, PlaybackObj::Hdmv { id_ref: 0 });
assert_eq!(idx.top_menu, PlaybackObj::BdJ);
assert_eq!(idx.titles.len(), 2);
assert_eq!(idx.titles[0], PlaybackObj::BdJ);
}
#[test]
fn rejects_bad_magic_and_truncation() {
assert!(parse(b"NOPE").is_none());
let d = build(hdmv_obj(3), hdmv_obj(0), &[hdmv_obj(1)]);
// Truncating below the declared title span must yield None, never panic.
assert!(parse(&d[..d.len() - 4]).is_none());
}
}
+147
View File
@@ -0,0 +1,147 @@
//! Parse `/BDMV/MovieObject.bdmv` — the HDMV navigation programs, per their
//! documented binary layout. Read as a documented format, never executed here:
//! bounds-checked, never panics.
//!
//! Layout: `"MOBJ"` + version(4) + reserved… ; `MovieObjects()` at byte 40 is
//! `length`(u32) + reserved(u32) + `num_objects`(u16 @48), then objects from
//! byte 50. Each object is `flags`(1) + reserved(1) + `num_cmds`(u16) followed
//! by `num_cmds` 12-byte navigation commands.
/// One decoded 12-byte navigation command. Field layout (big-endian): byte 0 =
/// `op_cnt(3) grp(2) sub_grp(3)`; byte 1 = `imm_op1(1) imm_op2(1) branch_opt(4)`
/// (low nibble); byte 2 low nibble = `cmp_opt`; byte 3 low 5 bits = `set_opt`;
/// then `dst`(u32) and `src`(u32).
#[derive(Debug, Clone, Copy)]
pub(crate) struct Cmd {
pub op_cnt: u8,
pub grp: u8,
pub sub_grp: u8,
pub imm_op1: bool,
pub imm_op2: bool,
pub branch_opt: u8,
pub cmp_opt: u8,
pub set_opt: u8,
pub dst: u32,
pub src: u32,
}
/// One HDMV navigation program.
#[derive(Debug, Clone)]
pub(crate) struct MovieObject {
pub cmds: Vec<Cmd>,
}
const CMD_LEN: usize = 12;
/// Sanity caps (real discs are far under these; some densely-branched
/// dispatchers run several thousand commands in one object).
const MAX_OBJECTS: usize = 4096;
const MAX_CMDS: usize = 200_000;
fn be_u16(d: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*d.get(o)?, *d.get(o + 1)?]))
}
/// Decode one 12-byte command. Caller guarantees `b.len() == 12`.
fn decode_cmd(b: &[u8]) -> Cmd {
let (b0, b1, b2, b3) = (b[0], b[1], b[2], b[3]);
Cmd {
op_cnt: (b0 >> 5) & 0x7,
grp: (b0 >> 3) & 0x3,
sub_grp: b0 & 0x7,
imm_op1: (b1 >> 7) & 1 == 1,
imm_op2: (b1 >> 6) & 1 == 1,
branch_opt: b1 & 0x0f,
cmp_opt: b2 & 0x0f,
set_opt: b3 & 0x1f,
dst: u32::from_be_bytes([b[4], b[5], b[6], b[7]]),
src: u32::from_be_bytes([b[8], b[9], b[10], b[11]]),
}
}
/// Parse `MovieObject.bdmv` into its programs. Returns `None` on any structural
/// problem.
pub(crate) fn parse(d: &[u8]) -> Option<Vec<MovieObject>> {
if d.get(0..4)? != b"MOBJ" {
return None;
}
let num = be_u16(d, 48)? as usize;
if num > MAX_OBJECTS {
return None;
}
let mut off = 50usize;
let mut objs = Vec::with_capacity(num);
for _ in 0..num {
// Object header: flags(1) + reserved(1) + num_cmds(u16).
let num_cmds = be_u16(d, off + 2)? as usize;
off = off.checked_add(4)?;
if num_cmds > MAX_CMDS {
return None;
}
let span = num_cmds.checked_mul(CMD_LEN)?;
let end = off.checked_add(span)?;
if end > d.len() {
return None;
}
let mut cmds = Vec::with_capacity(num_cmds);
for i in 0..num_cmds {
let s = off + i * CMD_LEN;
cmds.push(decode_cmd(&d[s..s + CMD_LEN]));
}
off = end;
objs.push(MovieObject { cmds });
}
Some(objs)
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
/// Encode one command from its opcode bytes + operands.
pub(crate) fn cmd(b0: u8, b1: u8, b2: u8, b3: u8, dst: u32, src: u32) -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = b0;
b[1] = b1;
b[2] = b2;
b[3] = b3;
b[4..8].copy_from_slice(&dst.to_be_bytes());
b[8..12].copy_from_slice(&src.to_be_bytes());
b
}
pub(crate) fn build(objects: &[&[[u8; 12]]]) -> Vec<u8> {
let mut d = vec![0u8; 50];
d[0..4].copy_from_slice(b"MOBJ");
d[48..50].copy_from_slice(&(objects.len() as u16).to_be_bytes());
for obj in objects {
d.push(0); // flags
d.push(0); // reserved
d.extend_from_slice(&(obj.len() as u16).to_be_bytes());
for c in *obj {
d.extend_from_slice(c);
}
}
d
}
#[test]
fn parses_objects_and_commands() {
// op_cnt=1, grp=BRANCH(0), sub_grp=PLAY(2), branch_opt=PLAY_PL(0), imm dst.
let b0 = (1 << 5) | 2;
let play = cmd(b0, 0x80, 0, 0, 11, 0);
let d = build(&[&[play]]);
let objs = parse(&d).expect("parses");
assert_eq!(objs.len(), 1);
assert_eq!(objs[0].cmds.len(), 1);
assert_eq!(objs[0].cmds[0].dst, 11);
assert_eq!(objs[0].cmds[0].sub_grp, 2);
}
#[test]
fn rejects_truncation() {
let play = cmd((1 << 5) | 2, 0x80, 0, 0, 11, 0);
let d = build(&[&[play]]);
assert!(parse(&d[..d.len() - 3]).is_none());
assert!(parse(b"NOPE").is_none());
}
}
+50
View File
@@ -0,0 +1,50 @@
//! BD/UHD HDMV navigation resolver — "mimic a real player" for main-feature
//! title selection. It reads `/BDMV/index.bdmv` + `/BDMV/MovieObject.bdmv` and
//! runs a faithful, bounded HDMV navigation VM to find the playlist the disc's
//! own First-Play navigation plays as the feature.
//!
//! There is no per-disc special-casing: one correctly-booted VM resolves
//! densely-branched dispatchers as well as simple ones. The honest boundary is
//! BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
//! on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
//! and selection falls back to the structural/heuristic order.
//!
//! Contract: read-only, bounded, and never panics or hard-fails.
pub(crate) mod index;
pub(crate) mod mobj;
pub(crate) mod vm;
use crate::sector::SectorSource;
use crate::udf::UdfFs;
/// Resolve the playlist id the disc's First-Play navigation plays as the
/// feature, restricted to ids the caller marks as feature candidates
/// (`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
/// short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J discs,
/// missing/malformed nav data, or when navigation does not converge on a
/// candidate.
pub(crate) fn resolve_feature(
reader: &mut dyn SectorSource,
udf: &UdfFs,
is_feature_candidate: impl Fn(u16) -> bool,
) -> Option<u16> {
// Belt-and-suspenders: the parsers and VM are written panic-free and bounded,
// but a navigation resolver must NEVER take down a scan — swallow any
// unexpected panic and abstain.
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
resolve_inner(reader, udf, &is_feature_candidate)
}))
.ok()
.flatten()
}
fn resolve_inner(
reader: &mut dyn SectorSource,
udf: &UdfFs,
is_feature_candidate: &dyn Fn(u16) -> bool,
) -> Option<u16> {
let index = index::parse(&udf.read_file(reader, "/BDMV/index.bdmv").ok()?)?;
let mobjs = mobj::parse(&udf.read_file(reader, "/BDMV/MovieObject.bdmv").ok()?)?;
vm::resolve(&index, &mobjs, is_feature_candidate)
}
+468
View File
@@ -0,0 +1,468 @@
//! The HDMV navigation VM — a faithful, bounded re-implementation of the subset
//! of HDMV navigation semantics needed to follow First-Play to the feature
//! `PlayPlayList`. It "mimics a real player": correct power-on register state is
//! what lets a densely-branched dispatcher converge on the feature rather than
//! loop. Never panics; hard step/switch caps.
//!
//! It resolves generically — there is no per-disc special-casing. When
//! First-Play is a BD-J title (the feature is chosen by a Java Xlet), or the
//! program does not reach a `PlayPL` on a caller-approved feature candidate, it
//! abstains (`None`) and selection falls back to the structural/heuristic order.
use super::index::{Index, PlaybackObj};
use super::mobj::MovieObject;
/// Top bit of an operand selects PSR (else GPR).
const PSR_FLAG: u32 = 0x8000_0000;
/// Upper bound on VM steps before declaring non-convergence.
const MAX_STEPS: usize = 1_000_000;
/// Cap on object/title switches — a dispatcher runs in one object; this only
/// bounds pathological jump chains.
const MAX_SWITCHES: usize = 1024;
/// BD player power-on register defaults (indices 0..=61); unlisted = 0. The
/// load-bearing values are PSR4/5 = `0xffff` ("no title/chapter selected") and
/// PSR6/7/8 = 0, which steer dispatcher compares; the rest are set for fidelity
/// with real player behavior.
fn psr_init() -> [u32; 128] {
let mut p = [0u32; 128];
let vals: &[(usize, u32)] = &[
(0, 1),
(1, 0xff),
(2, 0x0fff_0fff),
(3, 1),
(4, 0xffff),
(5, 0xffff),
(10, 0xffff),
(12, 0xff),
(13, 0xff),
(14, 0xffff),
(15, 0xAAAA),
(16, 0x00ff_ffff),
(17, 0x00ff_ffff),
(18, 0x00ff_ffff),
(19, 0xffff),
(20, 2),
(29, 0x0000_0003),
(30, 0x0001_ffff),
(31, 0x0003_0200),
(36, 0xffff),
(37, 0xffff),
(42, 0xffff),
(44, 0xff),
];
for &(i, v) in vals {
p[i] = v;
}
for slot in p.iter_mut().take(62).skip(48) {
*slot = 0xffff_ffff;
}
p
}
struct Vm<'a> {
mobjs: &'a [MovieObject],
index: &'a Index,
gpr: [u32; 4096],
psr: [u32; 128],
}
impl Vm<'_> {
fn rd(&self, val: u32) -> u32 {
if val & PSR_FLAG != 0 {
self.psr[(val & 0x7f) as usize]
} else {
self.gpr[(val & 0xfff) as usize]
}
}
fn wr(&mut self, val: u32, x: u32) {
// Per the HDMV nav spec, a store to a PSR-tagged register is refused;
// only GPR writes take effect. PSR values come only from power-on init,
// never from a nav SET/SWAP operand.
if val & PSR_FLAG == 0 {
self.gpr[(val & 0xfff) as usize] = x;
}
}
fn fetch(&self, imm: bool, raw: u32) -> u32 {
if imm { raw } else { self.rd(raw) }
}
/// Resolve a JumpTitle target: title 0 = Top Menu, `1..=N` = `titles[i-1]`,
/// `0xFFFF` = First Play (BD-ROM title numbering). Any other value is an
/// invalid title reference and abstains (`None`).
fn title_obj(&self, title: u32) -> Option<PlaybackObj> {
let n = self.index.titles.len() as u32;
if title == 0 {
Some(self.index.top_menu)
} else if title <= n {
self.index.titles.get((title - 1) as usize).copied()
} else if title == 0xFFFF {
Some(self.index.first_play)
} else {
None
}
}
}
/// Run First-Play and return the first `PlayPL`/`PlayPL_PM`/`PlayPL_PI` whose
/// playlist id passes `is_feature`. `None` = BD-J boundary, non-convergence, or
/// no approved feature playlist reached.
pub(crate) fn resolve(
index: &Index,
mobjs: &[MovieObject],
is_feature: &dyn Fn(u16) -> bool,
) -> Option<u16> {
// Enter at First-Play when it is an HDMV object that exists; otherwise abstain
// (BD-J boundary, or `0xffff` "no object"). We don't chase Top-Menu's "Play
// Movie" target here — it lives in menu .m2ts IG button commands (future work).
let start = match index.first_play {
PlaybackObj::Hdmv { id_ref } if (id_ref as usize) < mobjs.len() => id_ref as usize,
_ => return None,
};
let mut vm = Vm {
mobjs,
index,
gpr: [0; 4096],
psr: psr_init(),
};
run(&mut vm, start, is_feature)
}
fn run(vm: &mut Vm, mut obj_id: usize, is_feature: &dyn Fn(u16) -> bool) -> Option<u16> {
let mut pc = 0usize;
let mut steps = 0usize;
let mut switches = 0usize;
loop {
steps += 1;
if steps > MAX_STEPS {
return None;
}
let obj = vm.mobjs.get(obj_id)?;
// Ran off the end without a feature PlayPL → abstain.
let c = *obj.cmds.get(pc)?;
let mut npc = pc + 1;
let dst = if c.op_cnt > 0 {
vm.fetch(c.imm_op1, c.dst)
} else {
0
};
let src = if c.op_cnt > 1 {
vm.fetch(c.imm_op2, c.src)
} else {
0
};
match c.grp {
// BRANCH
0 => match c.sub_grp {
// GOTO
0 => match c.branch_opt {
0x01 => npc = dst as usize, // GOTO
0x02 => return None, // BREAK — terminate
_ => {} // NOP / other
},
// JUMP
1 => match c.branch_opt {
0x00 | 0x02 => {
// JumpObject / CallObject (return address not modelled —
// for feature resolution we only follow the play path).
switches += 1;
if switches > MAX_SWITCHES || dst as usize >= vm.mobjs.len() {
return None;
}
obj_id = dst as usize;
pc = 0;
continue;
}
0x01 | 0x03 => {
// JumpTitle / CallTitle — resolve through the index table.
switches += 1;
if switches > MAX_SWITCHES {
return None;
}
match vm.title_obj(dst) {
Some(PlaybackObj::Hdmv { id_ref })
if (id_ref as usize) < vm.mobjs.len() =>
{
obj_id = id_ref as usize;
pc = 0;
continue;
}
// BD-J / unknown / invalid title → abstain.
_ => return None,
}
}
_ => {} // RESUME / other — fall through
},
// PLAY: PlayPL / PlayPL_PI / PlayPL_PM emit a playlist id (the
// rest — Terminate / Link — fall through). A logo/pre-roll whose
// id isn't a feature candidate lets autoplay resume at pc + 1.
2 if matches!(c.branch_opt, 0x00..=0x02) => {
let id = dst as u16;
if is_feature(id) {
return Some(id);
}
}
_ => {}
},
// CMP — skip the next command when the compare is false.
1 => {
let truth = match c.cmp_opt {
0x01 => (dst & !src) == 0,
0x02 => dst == src,
0x03 => dst != src,
0x04 => dst >= src,
0x05 => dst > src,
0x06 => dst <= src,
0x07 => dst < src,
_ => true,
};
if !truth {
npc = pc + 2;
}
}
// SET (sub_grp 0). SETSYSTEM (sub_grp 1) only mutates system PSRs
// whose values the feature path does not branch on — skip it.
2 if c.sub_grp == 0 => {
let r: Option<u32> = match c.set_opt {
0x01 => Some(src), // MOVE
0x02 => {
// SWAP exchanges the two operands; a store to an operand
// flagged immediate is refused, and `wr` already drops
// PSR stores.
if !c.imm_op1 {
vm.wr(c.dst, src);
}
if !c.imm_op2 {
vm.wr(c.src, dst);
}
None
}
0x03 => Some(dst.wrapping_add(src)),
0x04 => Some(dst.saturating_sub(src)),
0x05 => Some(dst.wrapping_mul(src)),
0x06 => Some(dst.checked_div(src).unwrap_or(0xffff_ffff)),
0x07 => Some(dst.checked_rem(src).unwrap_or(0xffff_ffff)),
0x08 => Some(dst), // RND — deterministic stand-in
0x09 => Some(dst & src),
0x0a => Some(dst | src),
0x0b => Some(dst ^ src),
0x0c => Some(dst | (1u32 << (src & 31))),
0x0d => Some(dst & !(1u32 << (src & 31))),
0x0e => Some(dst.wrapping_shl(src & 31)),
0x0f => Some(dst.wrapping_shr(src & 31)),
_ => None,
};
if let Some(r) = r
&& !c.imm_op1
{
vm.wr(c.dst, r);
}
}
_ => {}
}
pc = npc;
}
}
#[cfg(test)]
mod tests {
use super::super::index::{Index, PlaybackObj};
use super::super::mobj::{self, tests::build, tests::cmd};
use super::*;
fn play_pl(id: u16) -> [u8; 12] {
// op_cnt=1, grp=BRANCH(0), sub_grp=PLAY(2), branch_opt=PLAY_PL(0), imm dst.
cmd((1 << 5) | 2, 0x80, 0, 0, id as u32, 0)
}
fn jump_object(obj: u16) -> [u8; 12] {
// grp=BRANCH(0), sub_grp=JUMP(1), branch_opt=JUMP_OBJECT(0), imm dst.
cmd((1 << 5) | 1, 0x80, 0, 0, obj as u32, 0)
}
fn jump_title(title: u16) -> [u8; 12] {
// sub_grp=JUMP(1), branch_opt=JUMP_TITLE(1), imm dst.
cmd((1 << 5) | 1, 0x81, 0, 0, title as u32, 0)
}
fn set_move_gpr(reg: u16, imm: u16) -> [u8; 12] {
// grp=SET(2), sub_grp=SET(0), set_opt=MOVE(1); dst=reg (GPR), imm src.
// op_cnt=2, imm_op2=1 (src immediate), imm_op1=0 (dst is a register).
cmd((2 << 5) | (2 << 3), 0x40, 0, 0x01, reg as u32, imm as u32)
}
fn play_pl_reg(reg: u16) -> [u8; 12] {
// PLAY_PL with dst = GPR[reg] (op_cnt=1, imm_op1=0).
cmd((1 << 5) | 2, 0x00, 0, 0, reg as u32, 0)
}
fn idx(first: PlaybackObj, titles: Vec<PlaybackObj>) -> Index {
Index {
first_play: first,
top_menu: PlaybackObj::BdJ,
titles,
}
}
#[test]
fn resolves_immediate_playpl() {
// First-Play HDMV obj0 → JumpObject 1 → PlayPL 11.
let d = build(&[&[jump_object(1)], &[play_pl(11)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
assert_eq!(resolve(&index, &mobjs, &|id| id == 11), Some(11));
}
#[test]
fn skips_non_feature_playpl_then_returns_feature() {
// Autoplay chain: logo PlayPL 99 (not a candidate) → feature PlayPL 1.
let d = build(&[&[play_pl(99), play_pl(1)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
// Only 1 is a feature candidate; 99 is a logo.
assert_eq!(resolve(&index, &mobjs, &|id| id == 1), Some(1));
}
#[test]
fn abstains_when_first_play_jumps_to_bdj_title() {
// Computed-title-number shape: First-Play computes a title number in a
// GPR and JumpTitles to it; the target title is BD-J → abstain (None).
let d = build(&[&[set_move_gpr(0xFEB & 0xfff, 2), {
// JumpTitle with dst = GPR[0xFEB].
cmd((1 << 5) | 1, 0x01, 0, 0, 0x0000_0FEB, 0)
}]]);
let mobjs = mobj::parse(&d).unwrap();
// title 2 → titles[1] = BD-J.
let index = idx(
PlaybackObj::Hdmv { id_ref: 0 },
vec![PlaybackObj::Hdmv { id_ref: 5 }, PlaybackObj::BdJ],
);
assert_eq!(resolve(&index, &mobjs, &|_| true), None);
}
#[test]
fn abstains_when_first_play_is_bdj() {
let d = build(&[&[play_pl(1)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::BdJ, vec![]);
assert_eq!(resolve(&index, &mobjs, &|_| true), None);
}
#[test]
fn jumptitle_first_play_is_0xffff_not_n_plus_1() {
// BD-ROM title numbering: 0 = Top Menu, 0xFFFF = First Play,
// 1..=N = titles. `N+1` is an INVALID title ref.
let index = idx(
PlaybackObj::Hdmv { id_ref: 7 },
vec![PlaybackObj::Hdmv { id_ref: 1 }],
);
let vm = Vm {
mobjs: &[],
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
assert_eq!(vm.title_obj(0), Some(index.top_menu), "title 0 = Top Menu");
assert_eq!(
vm.title_obj(1),
Some(PlaybackObj::Hdmv { id_ref: 1 }),
"title 1 = titles[0]"
);
assert_eq!(
vm.title_obj(0xFFFF),
Some(PlaybackObj::Hdmv { id_ref: 7 }),
"0xFFFF = First Play (abstained wrongly before the fix)"
);
assert_eq!(
vm.title_obj(2),
None,
"N+1 is an invalid title ref, not First Play (resolved wrongly before the fix)"
);
}
#[test]
fn set_to_psr_is_refused() {
// SET MOVE to PSR6 must be a no-op (PSR stores are refused by spec).
// Program: MOVE PSR6<-5; CMP PSR6==5; PlayPL 100 else 200. Refused write
// keeps PSR6 at 0 -> PlayPL 200; an illegal write would wrongly pick 100.
let set_psr6 = cmd((2 << 5) | (2 << 3), 0x40, 0, 0x01, 0x8000_0006, 5);
let cmp_psr6_eq_5 = cmd((2 << 5) | (1 << 3), 0x40, 0x02, 0, 0x8000_0006, 5);
let d = build(&[&[set_psr6, cmp_psr6_eq_5, play_pl(100), play_pl(200)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
assert_eq!(
resolve(&index, &mobjs, &|id| id == 100 || id == 200),
Some(200),
"PSR store must be refused, so the compare is false and 200 is reached"
);
}
#[test]
fn register_fold_and_cmp_skip_reach_playpl() {
// SET GPR[3]=7; CMP GPR[3]==7 (true → do NOT skip); PlayPL GPR[3].
// grp=CMP(1), op_cnt=2, imm_op2=1 (src=7 immediate), cmp_opt=EQ(2); dst=GPR3.
let cmp_eq_reg = cmd((2 << 5) | (1 << 3), 0x40, 0x02, 0, 3, 0x0000_0007);
let d = build(&[&[set_move_gpr(3, 7), cmp_eq_reg, play_pl_reg(3), play_pl(999)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
// If the compare wrongly skipped, we'd hit PlayPL 999 instead of 7.
assert_eq!(resolve(&index, &mobjs, &|id| id == 7 || id == 999), Some(7));
}
#[test]
fn jumptitle_namespace_differs_from_jumpobject() {
// JumpTitle 1 → titles[0] = HDMV obj 1 (which PlayPLs 42). Proves title
// numbers resolve through the index, not as object indices.
let d = build(&[&[jump_title(1)], &[play_pl(42)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(
PlaybackObj::Hdmv { id_ref: 0 },
vec![PlaybackObj::Hdmv { id_ref: 1 }],
);
assert_eq!(resolve(&index, &mobjs, &|id| id == 42), Some(42));
}
#[test]
fn swap_exchanges_two_registers() {
// MOVE g0=11; MOVE g1=22; SWAP g0<->g1; PlayPL GPR[0]. SWAP: grp=SET(2),
// sub_grp=SET(0), set_opt=SWAP(2), both operands registers (imm_op*=0).
let swap01 = cmd((2 << 5) | (2 << 3), 0x00, 0, 0x02, 0, 1);
let d = build(&[&[
set_move_gpr(0, 11),
set_move_gpr(1, 22),
swap01,
play_pl_reg(0),
]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
let mut vm = Vm {
mobjs: &mobjs,
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
// After the swap GPR0 holds the old GPR1 (22) and PlayPL plays it.
assert_eq!(run(&mut vm, 0, &|id| id == 22), Some(22));
// Full exchange: GPR1 now holds the old GPR0 (11).
assert_eq!(vm.gpr[1], 11);
}
#[test]
fn swap_immediate_operand_is_not_written_back() {
// SWAP dst=GPR0 with src=IMMEDIATE 5 (imm_op1=0, imm_op2=1) must refuse
// writing to the aliased register: GPR[5] stays 77, only GPR0 gets 5.
// Dropping the `imm_op1`/`imm_op2` guard makes this assertion fail.
let swap_imm = cmd((2 << 5) | (2 << 3), 0x40, 0, 0x02, 0, 5);
let d = build(&[&[set_move_gpr(5, 77), swap_imm]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
let mut vm = Vm {
mobjs: &mobjs,
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
// No PlayPL → program runs off the end (None); inspect registers after.
assert_eq!(run(&mut vm, 0, &|_| false), None);
assert_eq!(
vm.gpr[5], 77,
"immediate operand must not be a write target"
);
assert_eq!(vm.gpr[0], 5, "register operand receives the immediate");
}
}
+8 -17
View File
@@ -4,7 +4,7 @@
//! The EP (Entry Point) map provides timestamp → SPN mapping.
//! SPN × 192 = byte offset in the m2ts file.
//!
//! Reference: https://github.com/lw/BluRay/wiki/CLPI
//! Format is documented in the BD-ROM Clip Information (CLPI) specification.
use crate::error::{Error, Result};
@@ -55,10 +55,9 @@ pub fn parse(data: &[u8]) -> Result<ClipInfo> {
0
};
// Parse ProgramInfo (per-stream language + codec). Best-effort:
// malformed program_info doesn't fail the parse, just gives an
// empty streams list. EP map is unaffected — sector-range lookups
// continue to work.
// Parse ProgramInfo (per-stream language + codec), best-effort: malformed
// program_info doesn't fail the parse, just yields an empty streams list.
// EP map is unaffected — sector-range lookups still work.
let streams = if prog_info_start > 0 && prog_info_start + 6 < data.len() {
parse_program_info(&data[prog_info_start..])
} else {
@@ -172,16 +171,9 @@ mod tests {
/// Build a minimal CLPI binary.
/// `cpi_data` is the raw CPI section bytes (starting with the 4-byte CPI length).
fn build_clpi(source_packet_count: u32, cpi_data: Option<&[u8]>) -> Vec<u8> {
// We need at least 60 bytes for the header area.
// Offsets:
// 0..4: "HDMV"
// 4..8: "0200"
// 8..12: seq_info_start (unused, set to 0)
// 12..16: prog_info_start (unused, set to 0)
// 16..20: cpi_start
// 20..40: reserved/padding
// 40..56: ClipInfo section area (length + stuff before source_packet_count)
// 56..60: source_packet_count
// Need at least 60 bytes for the header area: "HDMV"/"0200" magic,
// seq_info_start/prog_info_start (unused here), cpi_start, reserved
// padding, then the ClipInfo section ending in source_packet_count.
let cpi_start: u32 = if cpi_data.is_some() { 60 } else { 0 };
@@ -230,8 +222,7 @@ mod tests {
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the BD-ROM CLPI spec
// (https://github.com/lw/BluRay/wiki/CLPI).
// Added hardening tests, grounded in the BD-ROM CLPI spec byte layout.
// ─────────────────────────────────────────────────────────────────────
/// Build a ProgramInfo section. `streams` = Vec<(pid, sci_bytes)>.
+4
View File
@@ -114,6 +114,10 @@ pub mod coding_type {
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
+31 -56
View File
@@ -1,9 +1,9 @@
//! CSS title-key recovery — Frank A. Stevenson's divide-and-conquer attack
//! (1999), implemented from his published cryptanalysis ("Cryptanalysis of
//! Contents Scrambling System"). It recovers the 5-byte CSS title key from a
//! single scrambled DVD sector with no player keys and no disc-key crack, using
//! only known plaintext. Implemented from that public description; nothing here
//! is copied or translated from any particular CSS software.
//! CSS title-key recovery — a divide-and-conquer known-plaintext attack,
//! implemented from the openly published CSS cryptanalysis literature. It
//! recovers the 5-byte CSS title key from a single scrambled DVD sector with no
//! player keys and no disc-key crack, using only known plaintext. Implemented
//! from that public description; nothing here is copied or translated from any
//! particular CSS software.
//!
//! # The cipher this attacks
//!
@@ -41,8 +41,8 @@ const ENCRYPTED_START: usize = 0x80; // byte 128
const SEED_OFFSET: usize = 0x54; // sector seed at bytes 0x54-0x58
const FLAG_BYTE: usize = 0x14;
/// Recover the title key from cipher + known plaintext (the core of Stevenson's
/// attack). `crypted` is the ciphertext starting at sector byte 0x80;
/// Recover the title key from cipher + known plaintext (the core of the
/// recovery). `crypted` is the ciphertext starting at sector byte 0x80;
/// `decrypted` is the matching known plaintext; `seed` is `sector[0x54..0x59]`.
/// On success returns the recovered 5-byte title key; `None` if no LFSR seed
/// reproduces the keystream.
@@ -210,8 +210,8 @@ fn descramble_matches(sector: &[u8], title: &[u8; 5], plain: &[u8]) -> bool {
}
/// Find a repeating pattern just before the encrypted region and assume the
/// plaintext at 0x80 continues it — the known-plaintext step of Stevenson's
/// attack. Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// plaintext at 0x80 continues it — the known-plaintext step of the recovery.
/// Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// with a cycle length in 2..0x2F. If the run is long enough (`plen > 3` and at
/// least two full cycles), the known plaintext at 0x80 is taken to be the
/// periodic run continuing forward, and [`recover_title_key_from_plain`] is
@@ -224,10 +224,9 @@ pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
return None;
}
// Runaway guard: a single sector's crack is a bounded 2^16 LFSR search and
// should finish in well under a second on any modern CPU. If it ever
// exceeds ~2s wall-clock, something pathological is happening — log it so a
// hang is never silent.
// Runaway guard: this crack is a bounded 2^16 LFSR search, done well under 1s
// on any modern CPU. If it ever exceeds ~2s, something pathological is
// happening — log it so a hang is never silent.
let crack_t0 = std::time::Instant::now();
let result = crack_title_key_inner(sector);
@@ -280,21 +279,13 @@ pub(crate) fn attack_crib(sector: &[u8]) -> Option<[u8; 10]> {
// Need at least a few repeated bytes and at least one full cycle.
if best_plen > 3 && best_p > 0 && best_plen / best_p >= 2 {
// The known plaintext is the periodic run continuing past 0x80. The
// crib starts at `0x80 - (best_plen/best_p)*best_p` and continues
// through the encrypted region; the bytes at and after 0x80 are the
// predicted plaintext (the pattern repeats with period best_p).
// Crib starts at `0x80 - cycles*best_p`; bytes at/after 0x80 are
// predicted by the run repeating with period best_p.
let cycles = best_plen / best_p;
let plain_start = 0x80 - cycles * best_p;
// Each predicted byte is the run sample one or more periods back:
// `sec[plain_start + (i % best_p)]`. For in-run offsets
// (`plain_start + i < 0x80`) the run is exactly periodic, so this
// equals `sec[plain_start + i]`; for offsets at/after 0x80 the raw
// byte is ciphertext, so we MUST wrap within the period rather than
// read it. (Reading `&sec[plain_start..+10]` directly — as before —
// pulled ciphertext into the crib whenever the run covered fewer than
// 10 bytes before 0x80, producing false-negative key recovery.)
// Must wrap within the period (not read `&sec[plain_start..+10]`
// directly), since past 0x80 the raw byte is ciphertext, not plaintext.
let mut plain = [0u8; 10];
for (i, p) in plain.iter_mut().enumerate() {
*p = sector[plain_start + (i % best_p)];
@@ -359,14 +350,9 @@ mod tests {
let mut plaintext = vec![0u8; SECTOR_BYTES];
plaintext[FLAG_BYTE] = 0x10;
// A clean periodic run occupying the tail of the cleartext header
// (RUN_START..0x80) and continuing into the encrypted region. This
// mirrors a real VOB: a periodic data run just before the scrambled
// part. The run must NOT overlap the seed bytes (0x54..0x59), or the
// the crib detector would break mid-run. The phase is anchored to
// offset 0 so the run is consistent across the 0x80 boundary.
// Just above the seed (0x54..0x59); gives a 39-byte run (0x59..0x80)
// — enough for >=2 cycles of every tested period (<=19).
// Periodic run over the cleartext header tail (RUN_START..0x80) into
// the encrypted region, mirroring a real VOB. Must not overlap the seed
// bytes (0x54..0x59); RUN_START=0x59 gives >=2 cycles of every period.
const RUN_START: usize = 0x59;
let pat: Vec<u8> = (0..period)
.map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A)
@@ -754,11 +740,9 @@ mod tests {
None,
"scramble bits clear → no crack, even though one would succeed"
);
// `attack_crib` carries its own copy of the same gate, and it is the one
// that actually stops the crack (`crack_title_key`'s is defensive
// duplication). The crib doubles as the decrypt path's cached-key
// oracle, so a widened mask there would hand that path a "predicted
// plaintext" for sectors that were never scrambled.
// `attack_crib` carries its own copy of the same gate and is the one that
// actually stops the crack (`crack_title_key`'s is defensive duplication);
// it also doubles as the decrypt path's cached-key oracle.
assert_eq!(
attack_crib(&periodic),
None,
@@ -845,14 +829,9 @@ mod tests {
);
}
// ── attack_crib: known-answer vectors ──────────────────────────────────
//
// `attack_crib` is BOTH the cracker's known plaintext and the decrypt
// path's "did the cached key descramble correctly?" oracle. Until now it
// was only ever exercised end-to-end through `crack_title_key`, on a
// fixture whose periodic run covered 39 bytes (0x59..0x80) — long enough
// that the run start, the cycle count and the `i % best_p` wrap were all
// slack. A crib that silently drifts costs a rip its title key.
// ── attack_crib: known-answer vectors ──────────────────────
// Both the cracker's known plaintext and the decrypt path's cached-key
// oracle; was only exercised indirectly, leaving run/cycle/wrap slack.
/// Build a sector whose clear header ends in a `period`-length repeating
/// run of exactly `run_len` bytes immediately before 0x80.
@@ -1011,10 +990,8 @@ mod tests {
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
// The FLAG byte sits inside the header at 0x14, so it interrupts the
// pattern there; re-lay it and accept that 0x14 breaks the run — the
// scan still reaches offset 0x15 - 1 = 0x14 going backwards, i.e.
// j = 0x7f - 0x14 = 0x6b, well short of the bound. Instead put the
// The FLAG byte at 0x14 would interrupt the pattern, but j only needs to
// reach 0x7f - 0x14 = 0x6b, well short of the bound. So instead put the
// scramble flag bits into a byte value that IS the pattern's.
sector[FLAG_BYTE] = pat[FLAG_BYTE % period];
assert_ne!(
@@ -1081,11 +1058,9 @@ mod tests {
"{n} plaintext bytes is fewer than the ten the cipher iterates"
);
}
// Exactly ten of each is ACCEPTED as far as the search — the boundary is
// `< 10`, not `<= 10`. (Whether this particular keystream has a seed is
// immaterial; what must not happen is an early `None` from the guard.)
// Proven through the round-trip fixture, whose inputs are exactly ten
// bytes and which does recover its key.
// Exactly ten of each is ACCEPTED — the boundary is `< 10`, not `<= 10`.
// What must not happen is an early `None` from the guard; proven via the
// round-trip fixture, whose inputs are exactly ten bytes.
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
+2 -3
View File
@@ -1,9 +1,8 @@
//! CSS content cipher — an independent implementation of the publicly
//! documented Content Scramble System stream cipher.
//!
//! The algorithm is the one recovered and published in Frank A. Stevenson's
//! 1999 cryptanalysis ("Cryptanalysis of Contents Scrambling System") and
//! described in the open CSS literature. It is implemented here from that public
//! The algorithm is the publicly documented CSS stream cipher, described in the
//! open CSS cryptanalysis literature. It is implemented here from that public
//! description; its constants (see [`super::tables`]) are the cipher's own
//! defined values. Nothing in this file is copied or translated from any
//! particular CSS software.
+343 -166
View File
@@ -2,9 +2,9 @@
//!
//! CSS uses a weak 40-bit LFSR stream cipher (broken since 1999).
//!
//! The title key is recovered keylessly: [`crack_key`] runs the Stevenson
//! known-plaintext attack (see the [`stevenson`] module) on the scrambled
//! data, needing no player keys, disc-key crack, or external key file.
//! The title key is recovered keylessly: [`crack_key`] recovers it directly
//! from the scrambled data (see the [`keyless`] module),
//! needing no player keys, disc-key recovery, or external key file.
//! Sectors are then decrypted with [`descramble_sector`].
//!
//! Usage:
@@ -14,8 +14,8 @@
//! }
//! ```
pub mod keyless;
pub mod lfsr;
pub mod stevenson;
pub(crate) mod tables;
use crate::disc::Extent;
@@ -56,7 +56,7 @@ impl std::fmt::Debug for CssState {
}
/// Recover the CSS title key with no keys, by scanning scrambled sectors and
/// running the Stevenson known-plaintext attack (see the [`stevenson`] module).
/// running the known-plaintext attack (see the [`keyless`] module).
///
/// The crib comes from the periodic-run detector: a scrambled sector's cleartext region
/// (bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
@@ -75,7 +75,7 @@ pub fn crack_key(
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
crack_key_scan(reader, extents, batch_sectors, None).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -85,10 +85,10 @@ pub fn crack_key(
///
/// - [`CrackOutcome::Cracked`] — a scrambled sector yielded a title key.
/// - [`CrackOutcome::Unencrypted`] — NO scrambled sector was seen across the
/// scanned extents (`is_scrambled` never true): the content is genuinely
/// scanned extents (`is_scrambled_pack` never true): the content is genuinely
/// plaintext, so proceeding without a key is correct.
/// - [`CrackOutcome::ScrambledUncracked`] — scrambled sectors WERE seen but no
/// key could be recovered (the Stevenson attack found no crackable crib, or
/// key could be recovered (the recovery found no crackable crib, or
/// the scrambled region was unreadable). The content is encrypted; muxing it
/// as plaintext would emit garbage, so callers MUST surface a hard error
/// instead of falling through to "unencrypted" — the per-title
@@ -136,7 +136,7 @@ pub fn crack_key_outcome(
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
) -> CrackOutcome {
crack_key_scan(reader, extents, batch_sectors, halt, true)
crack_key_scan(reader, extents, batch_sectors, halt)
}
/// Resolve a DVD title's CSS descramble key from the reader when the caller
@@ -167,9 +167,8 @@ pub(crate) fn resolve_dvd_title_key(
halt: Option<&crate::halt::Halt>,
) -> std::io::Result<()> {
// `--raw` = deliberate ciphertext passthrough: never crack or descramble, and
// never hard-fail on scrambled-uncrackable — the user asked for the scrambled
// bytes. (In raw mode the caller hands us `None` on purpose; without this
// guard we'd install a real key and silently DECRYPT, or abort a raw mux.)
// never hard-fail on scrambled-uncrackable. Caller hands us `None` on
// purpose; without this guard we'd silently DECRYPT or abort a raw mux.
if raw {
return Ok(());
}
@@ -179,12 +178,9 @@ pub(crate) fn resolve_dvd_title_key(
// `halt` threads the caller's cancellation token so /api/stop can
// interrupt a long crack scan (the old scan-time crack honored it too).
let outcome = crack_key_outcome(reader, extents, batch_sectors, halt);
// A cancelled crack breaks out early, so its outcome is a TRUNCATED scan
// — not a real verdict. Interpreting it would either hard-fail a good disc
// as `ScrambledUncracked` (quarantining staging on a Stop) or, worse,
// read a half-scanned title as `Unencrypted` and mux scrambled bytes as
// plaintext. Surface the cancellation as `Halted` so the caller takes its
// graceful-stop path instead of trusting the partial outcome.
// A cancelled crack's outcome is a TRUNCATED scan, not a real verdict —
// surface it as `Halted` rather than trusting a partial `Unencrypted`
// or `ScrambledUncracked` and taking the wrong path.
if halt.map(|h| h.is_cancelled()).unwrap_or(false) {
return Err(crate::error::Error::Halted.into());
}
@@ -212,18 +208,10 @@ fn crack_key_scan(
extents: &[Extent],
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
// True only on the INITIAL scan: a fully CSS-locked (`05/6F/03`) result is a
// hard `ScrambledUncracked`. False on the per-VTS re-crack so a lapsed-AGID
// locked read returns None instead of killing a genuinely crackable title.
fail_on_locked: bool,
) -> CrackOutcome {
// Batch the reads: a live optical drive at 1 sector/read is glacial, and the
// crack only needs to FIND one scrambled sector whose 0x80 plaintext matches
// a known PES header. `batch_sectors` MUST be sized to the source — a drive
// rejects a READ(10) larger than its per-command max (DVD = 16) and
// `Drive::read` does not chunk, so an over-large batch fails every read and
// scans nothing. Callers pass `detect_max_batch_sectors(device_path)` for a
// live drive, a file-safe value for an image, or 1 to force per-sector.
// Batch the reads: a live drive at 1 sector/read is glacial. `batch_sectors`
// MUST be sized to the source — a drive rejects a READ(10) larger than its
// per-command max, and `Drive::read` does not chunk an over-large batch.
let batch = (batch_sectors.max(1)) as u32;
// Record the LBA span the key is being cracked from so the per-title mux
// path can tell whether a later title lives in the same VTS (overlaps the
@@ -237,18 +225,13 @@ fn crack_key_scan(
let max_tries = 50_000u32;
let mut buf = vec![0u8; batch as usize * 2048];
let mut hb = crate::progress::Heartbeat::new("css_crack");
// Track whether ANY scrambled sector was observed. If we exhaust the scan
// budget having seen scrambled data but never recovered a key, the content
// is encrypted-but-uncrackable — a HARD failure the caller must surface,
// NOT silently treat as unencrypted (which would mux scrambled MPEG as
// plaintext → garbage at exit 0). See `CrackOutcome::ScrambledUncracked`.
// Track whether ANY scrambled sector was observed: if the budget is
// exhausted with scrambled data seen but no key recovered, that's a HARD
// failure, not "unencrypted". See `CrackOutcome::ScrambledUncracked`.
let mut saw_scrambled = false;
// A read rejected with sense `05/6F/03` ("scrambled sector without
// authentication") is positive proof of CSS encryption — never collapse it
// to "unencrypted". A run of consecutive locked reads means the bus-auth
// gate is shut (it is global, so reads are all-or-nothing), so the scan
// early-bails. `consecutive_locked` resets on any readable batch, so a
// crackable title (gate open) never trips it.
// Sense `05/6F/03` is positive proof of CSS encryption. Consecutive locked
// reads mean the (global) bus-auth gate is shut, so the scan early-bails;
// this resets on any readable batch, so an open-gate title never trips it.
let mut saw_locked = false;
let mut consecutive_locked = 0u32;
@@ -277,40 +260,35 @@ fn crack_key_scan(
}
let n = (ext.sector_count - i).min(batch);
let want = n as usize * 2048;
// How far the cursor advances. Set from the bytes actually READ on
// the Ok path so a short read is RETRIED from where it stopped
// rather than skipped: bounding only the inspection (which is what
// stops stale buffer bytes being scanned) would otherwise leave
// those sectors unexamined, quietly shrinking the crack's coverage
// on exactly the damaged media where a key is hardest to find.
// Set from bytes actually READ on the Ok path so a short read is
// RETRIED from where it stopped, not skipped — else those sectors
// go unexamined on exactly the damaged media a key needs most.
let mut advance = n;
match reader.read_sectors(ext.start_lba + i, n as u16, &mut buf[..want], true) {
Ok(got) => {
// A readable batch: the gate is open — reset the locked run.
consecutive_locked = 0;
// Inspect only what was actually READ. The trait returns the
// byte count and a source may return Ok with fewer bytes
// than asked (a recovery read over a damaged region), while
// `buf` is reused across batches — so the tail still holds
// the PREVIOUS batch's sectors. Scanning those means
// cracking a key from data belonging to a different extent,
// and possibly a different VTS, while `crack_span` records
// the CURRENT one: a key that opens nothing here gets
// installed and its wrong descrambles are only partly caught
// by the per-sector crib.
// Inspect only what was READ: `buf` is reused across
// batches, so its tail may hold the PREVIOUS batch's
// sectors, risking a key crack for the wrong extent/VTS.
let usable = (got / 2048).min(n as usize);
// At least one, so a source returning Ok(0) cannot spin here.
advance = (usable as u32).max(1);
if usable == 0 {
// Nothing inspected, but the cursor still moves one
// sector — charge it here or `tried` stays frozen
// (mirrors the `Err` arm's `tried += n`).
tried += 1;
}
for s in 0..usable {
tried += 1;
let sect = &buf[s * 2048..(s + 1) * 2048];
// Use the HARDENED pack-gated check (Fix 3): a clear stub
// sector with stray bits at 0x14 must NOT count as
// scramble evidence, or a genuinely-unencrypted title
// would falsely report ScrambledUncracked (a false E7023).
// HARDENED pack-gated check: a clear stub sector with
// stray bits at 0x14 must NOT count as scramble evidence,
// or an unencrypted title falsely reports E7023.
if is_scrambled_pack(sect) {
saw_scrambled = true;
if let Some(key) = stevenson::crack_title_key(sect) {
if let Some(key) = keyless::crack_title_key(sect) {
return CrackOutcome::Cracked(CssState {
title_key: key,
crack_span,
@@ -323,9 +301,8 @@ fn crack_key_scan(
}
}
// A failed batch still counts toward the budget so a damaged
// region can't loop forever. A CSS-locked failure (`05/6F/03`)
// proves encryption and, in a long enough run, means the read
// gate is shut — track it and early-bail rather than grind.
// region can't loop forever. A CSS-locked failure proves
// encryption; a long enough run means the gate is shut.
Err(e) => {
tried += n;
if e.scsi_sense().is_some_and(|s| s.is_css_locked()) {
@@ -343,15 +320,10 @@ fn crack_key_scan(
}
}
// Budget exhausted / extents walked / early-bailed with no key recovered.
// The disc is ENCRYPTED-but-uncracked (a hard failure on the initial scan)
// when EITHER a scrambled sector was actually seen, OR — on the initial scan
// only (`fail_on_locked`) — every read was CSS-locked (`05/6F/03`), itself
// proof of scrambling. A re-crack (`fail_on_locked` false) stays soft: a
// lapsed-AGID locked read yields None, not a hard fail, so a crackable title
// in another VTS isn't killed. Only a scan that saw neither a scrambled
// sector nor a CSS-lock is genuinely unencrypted.
if saw_scrambled || (saw_locked && fail_on_locked) {
// ENCRYPTED-but-uncracked (hard failure) if a scrambled sector was seen or
// every read was CSS-locked; only a scan seeing neither is unencrypted. A
// prior "soft" `fail_on_locked` toggle was removed as dead/unsafe.
if saw_scrambled || saw_locked {
CrackOutcome::ScrambledUncracked
} else {
CrackOutcome::Unencrypted
@@ -402,39 +374,25 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
///
/// # Errors
///
/// [`Error::DecryptFailed`] when a sector's own crib proves the cached key stale
/// and the re-crack from that same sector also fails. CSS has no external key
/// source — the title key comes only from cracking the data — so on a readable
/// sector this is not a missing input, it is recovery failing on data we can
/// see. Emitting the sector anyway means one of two bad outcomes: descrambled
/// with the key its crib just rejected, which yields garbage behind an intact
/// clear header (valid pack start, passes every structural check the PS demuxer
/// applies, corruption confined to the PES payload where nothing looks); or
/// passed through still scrambled, which is ciphertext delivered where plaintext
/// is meant to be. Both are bad data reported as success.
/// Never returns `Err` — the signature is `Result` only to match the decrypt
/// seam it is dispatched from, and the `usize` is that seam's legacy
/// always-zero loss count (see [`crate::decrypt::decrypt_sectors`]).
///
/// This matches the AACS sibling, which returns [`Error::DecryptFailed`] rather
/// than apply a neighbouring CPS unit's key.
/// This section used to document an [`Error::DecryptFailed`](crate::Error::DecryptFailed) for the case where
/// a sector's crib rejects the cached key and the re-crack from that sector also
/// fails. That behaviour was tried and REVERTED, for the reason set out at the
/// `None =>` arm below: crib mismatch plus crack failure is the signature of a
/// crib FALSE POSITIVE, not of a stale key, and failing there made real discs
/// unrippable. The arm descrambles with the cached key and returns `Ok`.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
// `is_scrambled_pack`, NOT the looser `is_scrambled`. The raw flag test
// is only safe once a caller has committed to a title's VOB data, where
// every sector is a pack and byte 0x14 always means what it says. This
// function is handed arbitrary regions of a disc, so it also sees IFO,
// UDF and ISO 9660 sectors — raw structures where byte 0x14 is whatever
// that format stores there.
//
// Measured: `VIDEO_TS.IFO` on one disc holds 0x15 at offset 0x14 of its
// second sector. The raw test read bits 4-5 as "scrambled", descrambled
// it, and destroyed 1912 of 2048 bytes. That sector carries TT_SRPT, so
// the title table went with it: the disc enumerated 38 titles and an
// image decrypted from it enumerated 10, silently, at exit 0. Requiring
// the pack start code first costs nothing — a genuinely scrambled VOB
// sector always has it, and no IFO sector does.
// `is_scrambled_pack`, NOT the looser `is_scrambled`: this sees arbitrary
// regions (IFO/UDF/ISO 9660) where raw byte 0x14 isn't a reliable flag.
// Measured: an IFO misread this way was destroyed, dropping titles 38→10.
if chunk.len() < 2048 || !is_scrambled_pack(chunk) {
continue;
}
let crib = stevenson::attack_crib(chunk);
let crib = keyless::attack_crib(chunk);
// Snapshot the ciphertext (chunk is exactly 2048 here) only when there is
// a crib to validate against, so the common cache-hit path costs no
// per-sector heap allocation.
@@ -449,34 +407,15 @@ pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::erro
// Cached key is stale for this region — restore the ciphertext and
// crack this sector's own key.
chunk.copy_from_slice(&original);
match stevenson::crack_title_key(chunk) {
match keyless::crack_title_key(chunk) {
Some(fresh) => {
*title_key = fresh;
lfsr::descramble_sector(title_key, chunk);
}
None => {
// The re-crack found nothing. Descramble with the CACHED
// key anyway — it is the best available evidence, and this
// is very probably still the right key.
//
// `attack_crib` is a heuristic, not a proof. It finds a
// periodic run in the unscrambled header and predicts that
// the run continues past 0x80. When that prediction does
// not hold, the crib check reports a mismatch even though
// the cached key is correct, and the re-crack from this
// sector then fails BECAUSE the crib was never valid.
// Crib mismatch + crack failure is therefore the signature
// of a crib false positive, not of a stale key.
//
// Round 9 read this as "descrambling with a key we just
// proved stale" and made it Error::DecryptFailed to match
// the AACS path. That was wrong on both counts: the key is
// not proven stale, and CSS is not AACS — an AACS unit key
// either opens a unit or does not, whereas a CSS title key
// is recovered from data whose recoverability varies sector
// by sector. Real DVDs hit this constantly; the change made
// Greenland.iso unrippable and was caught by the real-media
// acceptance gate, not by any unit test.
// Re-crack found nothing; descramble with the CACHED key
// anyway — mismatch+failure signals a crib false positive,
// not a stale key (`DecryptFailed` here made discs unrippable).
lfsr::descramble_sector(title_key, chunk);
}
}
@@ -515,7 +454,7 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// Check if a sector is a CSS-scrambled DVD **video pack** — the HARDENED test
/// the crack scan uses to set its `saw_scrambled` evidence flag (Fix 3).
///
/// [`is_scrambled`] keys solely on bits 4-5 of byte 0x14. That single byte is
/// [`has_scramble_flag_bits`] keys solely on bits 4-5 of byte 0x14. That byte is
/// only meaningful inside a real DVD sector — an MPEG-2 Program Stream pack,
/// which ALWAYS begins with the 32-bit pack-start code `00 00 01 BA` at offset
/// 0x00. A tiny clear / nav-only stub (a 0.5 s menu loop, an FBI-warning title)
@@ -527,14 +466,53 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// structurally a DVD video pack can be counted as scramble evidence. This does
/// NOT weaken the genuine "encrypted but uncrackable" hard-fail: a real
/// scrambled feature is made of valid PS packs, so its scrambled sectors still
/// pass this check and still drive `ScrambledUncracked` when no key cracks. (The
/// descramble loop keeps the looser [`is_scrambled`]: by the time it runs we
/// already know the title is CSS, and it only needs to skip interleaved clear
/// NAV packs — a wrongly-skipped or wrongly-included sector there is recoverable
/// per-sector, whereas a false scramble verdict in the scan poisons the whole
/// title's outcome.)
/// pass this check and still drive `ScrambledUncracked` when no key cracks.
///
/// The pack-start check alone is not enough, and the sector's PES `stream_id`
/// (offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
/// prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
/// streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
/// (`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
/// stream_id even on a scrambled sector. The MPEG-PS structural packets that
/// are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
/// private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
/// because on those the byte at 0x14 is NOT a PES scrambling-control field but
/// raw payload/structure whose bits 4-5 land set by chance.
///
/// This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
/// MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
/// private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
/// set. With no `0x11` gate those nav packs flipped the crack scan's
/// `saw_scrambled` flag on a disc that carries no CSS at all; the crack then
/// found no key (there is none) and the scan returned `ScrambledUncracked`,
/// hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
/// `0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
/// can act on (the recovery only recovers a key from a scrambled ES
/// pack), so a decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
///
/// This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
/// real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
/// private_stream_1 (`0xBD`) packs, none of which are excluded, so its
/// scrambled sectors still set `saw_scrambled` and still drive
/// `ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
/// so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
///
/// The DESCRAMBLE path gates on this same function — [`descramble_sector`] and
/// [`descramble_region`] both call it, not the raw flag test — because the raw
/// test does not merely mis-skip a sector there: it descrambles one that was
/// never scrambled and destroys it. The measured case is written up on
/// [`descramble_region`]: a `VIDEO_TS.IFO` sector holding 0x15 at offset 0x14
/// lost 1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38
/// titles became 10 — silently, at exit 0. One gate, both paths.
pub fn is_scrambled_pack(sector: &[u8]) -> bool {
sector.len() >= 2048 && sector[0x00..0x04] == PACK_START && (sector[0x14] >> 4) & 0x03 != 0
use crate::consts::pes_stream_id::{PADDING_STREAM, PRIVATE_STREAM_2, SYSTEM_HEADER};
sector.len() >= 2048
&& sector[0x00..0x04] == PACK_START
&& !matches!(
sector[0x11],
SYSTEM_HEADER | PADDING_STREAM | PRIVATE_STREAM_2
)
&& (sector[0x14] >> 4) & 0x03 != 0
}
#[cfg(test)]
@@ -556,7 +534,7 @@ mod tests {
/// This test exists because round 9 read the same code as "descrambling
/// with a key we just proved stale" and made it `DecryptFailed` to match
/// the AACS path. Real DVDs hit this constantly — the change made
/// Greenland.iso unrippable, and no unit test caught it; the real-media
/// a real disc unrippable, and no unit test caught it; the real-media
/// acceptance gate did. CSS is not AACS: an AACS unit key either opens a
/// unit or does not, whereas a CSS title key is recovered from data whose
/// recoverability varies sector by sector.
@@ -577,11 +555,11 @@ mod tests {
"fixture must be a scrambled sector"
);
assert!(
stevenson::attack_crib(&sector).is_some(),
keyless::attack_crib(&sector).is_some(),
"fixture must yield a crib, or the mismatch branch is never entered"
);
assert!(
stevenson::crack_title_key(&sector).is_none(),
keyless::crack_title_key(&sector).is_none(),
"fixture must be uncrackable, or the failure branch is never entered"
);
@@ -590,7 +568,18 @@ mod tests {
let out = descramble_region(&mut sector, &mut key)
.expect("a crib false positive must NOT fail the rip");
assert_eq!(out, 0, "CSS reports no loss term of its own");
// "No loss term" belongs to the SEAM, not this function: assert it one
// level up (at `decrypt_sectors`) where the value is assembled and
// returned, so the arm dispatch and plumbing are exercised too.
assert_eq!(out, 0);
let mut seam_sector = sector;
let mut seam_keys = crate::decrypt::DecryptKeys::Css { title_key: key };
assert_eq!(
crate::decrypt::decrypt_sectors(&mut seam_sector, &mut seam_keys, 0)
.expect("a crib false positive must NOT fail the rip at the seam either"),
0,
"CSS reports no loss term of its own through decrypt_sectors"
);
assert_eq!(
key, key_before,
"a failed re-crack must leave the cached key in place — it is still \
@@ -618,9 +607,9 @@ mod tests {
);
}
// ── is_scrambled ───────────────────────────────────────────────────────
// ── has_scramble_flag_bits ─────────────────────────────────────────────
/// is_scrambled returns false for any buffer shorter than one sector,
/// has_scramble_flag_bits returns false for any buffer shorter than one sector,
/// WITHOUT indexing byte 0x14 (which would panic on a tiny buffer). The
/// length guard is short-circuited before the flag read.
///
@@ -630,13 +619,13 @@ mod tests {
/// (`(sector[0x14]...) && sector.len() >= 2048`) -> panics indexing a
/// 20-byte slice; this test catches it.
#[test]
fn is_scrambled_short_buffer_is_false_no_panic() {
fn has_scramble_flag_bits_short_buffer_is_false_no_panic() {
assert!(!has_scramble_flag_bits(&[]));
assert!(!has_scramble_flag_bits(&[0u8; 20])); // shorter than 0x14+1 even
assert!(!has_scramble_flag_bits(&[0xFFu8; 2047])); // one byte short of a sector
}
/// is_scrambled keys on bits 4-5 of byte 0x14 (the CSS scramble field).
/// has_scramble_flag_bits keys on bits 4-5 of byte 0x14 (the CSS scramble field).
/// A full sector flagged 0x10/0x20/0x30 is scrambled; 0x00 and the
/// high-bit-only values 0x40/0x80 are clear.
///
@@ -644,7 +633,7 @@ mod tests {
/// Mutation: widen mask to `& 0x0F` -> 0x40 reports scrambled, the 0x40
/// assert fails.
#[test]
fn is_scrambled_uses_bits_4_5_only() {
fn has_scramble_flag_bits_uses_bits_4_5_only() {
let mut s = vec![0u8; 2048];
for (flag, expected) in [
(0x00u8, false),
@@ -665,14 +654,14 @@ mod tests {
}
}
/// is_scrambled accepts exactly 2048 bytes as the minimum (boundary at the
/// has_scramble_flag_bits accepts exactly 2048 bytes as the minimum (boundary at the
/// inclusive value 2048).
///
/// Grounding: `sector.len() >= 2048`.
/// Mutation: change `>= 2048` to `> 2048` -> an exact 2048-byte scrambled
/// sector reports false; this fails.
#[test]
fn is_scrambled_exact_sector_length_accepted() {
fn has_scramble_flag_bits_exact_sector_length_accepted() {
let mut s = vec![0u8; 2048];
s[0x14] = 0x30;
assert!(
@@ -686,8 +675,8 @@ mod tests {
/// bits. A clear / nav-only stub whose bytes happen to set bits 4-5 of 0x14
/// but lacks the pack-start is NOT counted as scramble evidence — without
/// this the scan flips `saw_scrambled` and a genuinely unencrypted title
/// reports `ScrambledUncracked` (the false E7023). The looser `is_scrambled`
/// (descramble gate) still reads the same sector as flagged.
/// reports `ScrambledUncracked` (the false E7023). The raw flag test
/// `has_scramble_flag_bits` still reads the same sector as flagged.
///
/// Grounding: `sector[0x00..0x04] == 00 00 01 BA && (sector[0x14] >> 4)...`.
/// Mutation: drop the pack-start clause -> the 0x14-only sector counts as a
@@ -719,6 +708,52 @@ mod tests {
);
}
/// Detection fix: `is_scrambled_pack` must EXCLUDE the MPEG-PS structural
/// packets that CSS never scrambles — system_header (0xBB), padding (0xBE)
/// and private_stream_2 (0xBF, DVD PCI/DSI and HD-DVD `.evo` RDI nav) — by
/// their `stream_id` at offset 0x11. On those packets byte 0x14 is raw
/// payload/structure, not a PES scrambling-control field, so its bits 4-5
/// land set by chance.
///
/// This is the exact decrypted-HD-DVD defect: an `.evo` RDI pack is
/// private_stream_2 (0xBF) with the pack-start code and bits set at 0x14, so
/// the old gate flipped `saw_scrambled` on a CSS-free disc → the crack found
/// no key → `ScrambledUncracked` → E7023 on a good HD-DVD.
///
/// Grounding: `!matches!(sector[0x11], SYSTEM_HEADER | PADDING_STREAM |
/// PRIVATE_STREAM_2)`. Mutation: drop the 0x11 exclusion → the 0xBF nav pack
/// counts as scrambled evidence and the first assert fails. The video and
/// private_stream_1 packs prove the gate does NOT reject a real scrambled
/// DVD sector (the catastrophic direction).
#[test]
fn is_scrambled_pack_excludes_nav_and_structural_stream_ids() {
use crate::consts::pes_stream_id::{
PADDING_STREAM, PRIVATE_STREAM_1, PRIVATE_STREAM_2, SYSTEM_HEADER, VIDEO, VIDEO_MAX,
};
// A pack-start pack with 0x14 scramble bits set, varying only 0x11.
let mut s = vec![0u8; 2048];
s[0x00..0x04].copy_from_slice(&PACK_START);
s[0x14] = 0x30;
for excluded in [SYSTEM_HEADER, PADDING_STREAM, PRIVATE_STREAM_2] {
s[0x11] = excluded;
assert!(
!is_scrambled_pack(&s),
"stream_id {excluded:#04x} is a structural/nav pack CSS never scrambles — \
it must NOT count as scramble evidence (else a decrypted HD-DVD RDI pack → E7023)"
);
}
// A genuinely scramblable elementary-stream pack must STILL register —
// proving the exclusion did not weaken real-DVD CSS detection.
for scramblable in [VIDEO, VIDEO_MAX, PRIVATE_STREAM_1, 0xE2] {
s[0x11] = scramblable;
assert!(
is_scrambled_pack(&s),
"stream_id {scramblable:#04x} is a scramblable ES pack — a real CSS DVD's \
scrambled sector must still be counted, never passed through as plaintext"
);
}
}
// ── crack_key scanning over a mock SectorSource ────────────────────────
/// Records every (lba, count) read; returns a caller-supplied flag byte at
@@ -731,10 +766,20 @@ mod tests {
/// scrambled reads because the bus-auth gate isn't open).
lock_all: bool,
/// When set, the sector at `crackable.0` is served as a full
/// Stevenson-crackable scrambled sector (`crackable.1`, 2048 bytes)
/// crackable scrambled sector (`crackable.1`, 2048 bytes)
/// instead of the uniform `flag_byte` fill. Lets the scan actually
/// reach `CrackOutcome::Cracked` from a synthetic ISO.
crackable: Option<(u32, Vec<u8>)>,
/// Sectors actually filled per batch, however many were asked for —
/// the SHORT READ a `recovery: true` source is allowed to return over
/// a damaged region. `Some(0)` is the degenerate case that must not
/// spin the scan.
short_read: Option<usize>,
/// PES `stream_id` written at offset 0x11 of each uniform-fill sector.
/// `0x00` (the default) is a scramblable-looking pack; set to
/// private_stream_2 (`0xBF`) to model an HD-DVD `.evo` RDI nav pack,
/// which carries the pack-start code and 0x14 bits but no CSS.
stream_id: u8,
}
impl MockSource {
@@ -745,15 +790,17 @@ mod tests {
fail_all: false,
lock_all: false,
crackable: None,
short_read: None,
stream_id: 0x00,
}
}
}
/// Build a Stevenson-crackable scrambled sector for `(title_key, seed)`:
/// Build a crackable scrambled sector for `(title_key, seed)`:
/// the cleartext header (0x59..0x80) carries a periodic run that continues
/// across the 0x80 boundary into the encrypted region — the crib
/// `stevenson::crack_title_key` recovers a key from. Mirrors the
/// `synth_periodic_sector` fixture in the stevenson tests but built here
/// `keyless::crack_title_key` recovers a key from. Mirrors the
/// `synth_periodic_sector` fixture in the keyless tests but built here
/// from the crate-internal `scramble_sector`.
fn crackable_sector(title_key: &[u8; 5], seed: &[u8; 5], period: usize) -> Vec<u8> {
const RUN_START: usize = 0x59;
@@ -795,14 +842,19 @@ mod tests {
if self.fail_all {
return Err(Error::DecryptFailed);
}
let n = count as usize * 2048;
// A short read fills, and reports, fewer sectors than asked.
let filled = match self.short_read {
Some(k) => (k as u16).min(count),
None => count,
};
let n = filled as usize * 2048;
let end = n.min(buf.len());
for b in buf[..end].iter_mut() {
*b = 0;
}
// Fill each sector in the batch with the uniform flag byte, EXCEPT a
// designated crackable LBA which gets the full synthetic sector.
for s in 0..count as u32 {
for s in 0..filled as u32 {
let sect_lba = lba + s;
let base = s as usize * 2048;
if base + 2048 > end {
@@ -813,12 +865,11 @@ mod tests {
buf[base..base + 2048].copy_from_slice(sector);
}
_ => {
// Real DVD video sectors always open with the MPEG-PS
// pack-start code; `is_scrambled` (Fix 3) requires it
// before trusting the 0x14 scramble bits, so the fixture
// must include it for a `flag_byte` of 0x30 to register
// as scrambled.
// `is_scrambled_pack` requires the MPEG-PS pack-start code
// before trusting 0x14; `stream_id` defaults to 0x00
// (scramblable) vs. an RDI nav pack's excluded 0xBF.
buf[base..base + 4].copy_from_slice(&PACK_START);
buf[base + 0x11] = self.stream_id;
buf[base + 0x14] = self.flag_byte;
}
}
@@ -827,6 +878,89 @@ mod tests {
}
}
// ── Short reads: the branch nothing exercised ──────────
// `recovery = true` means a `SectorSource` may return Ok with fewer bytes
// than asked; no source here did, so `usable`/`advance`/`.max(1)` were dead.
/// A short batch is RE-READ from where it stopped, not skipped. Skipping
/// would quietly shrink the crack's coverage on exactly the damaged media
/// where a key is hardest to find.
#[test]
fn a_short_read_resumes_from_where_it_stopped() {
let mut src = MockSource::new(0x00);
src.short_read = Some(1);
let ext = [crate::disc::Extent {
start_lba: 100,
sector_count: 4,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().clone();
assert_eq!(
reads,
vec![100, 101, 102, 103],
"a source that filled one sector per batch must be asked for the \
next one, not advanced a whole batch past it"
);
}
/// A source that reads NOTHING must terminate. Without the `.max(1)` the
/// cursor never moves and `tried` never increments — the budget cannot end
/// the loop, so the scan spins forever inside a library whose whole job is
/// surviving hostile input.
#[test]
fn a_source_that_returns_zero_sectors_terminates() {
let mut src = MockSource::new(0x00);
src.short_read = Some(0);
let ext = [crate::disc::Extent {
start_lba: 0,
sector_count: 8,
}];
let outcome = crack_key_scan(&mut src, &ext, 4, None);
assert!(
matches!(outcome, CrackOutcome::Unencrypted),
"nothing was read, so nothing scrambled was seen"
);
assert!(
src.reads.borrow().len() <= 8,
"the cursor must advance even on an empty read; got {} reads over \
an 8-sector extent",
src.reads.borrow().len()
);
}
/// The 50_000-sector budget must hold whatever the source returns, not
/// only when the source delivers sectors.
///
/// `tried` is incremented ONLY per inspected sector, inside
/// `for s in 0..usable`. An `Ok(0)` inspects nothing, so that loop never
/// runs — yet `advance` is forced to 1 to stop the scan spinning, so the
/// cursor keeps walking. The budget is then never consulted and the scan
/// runs for the extent's full, disc-declared `sector_count`: a misbehaving
/// or adversarial source (an emulated drive, a bridge answering short)
/// converts the anti-grind bound into no bound at all.
///
/// Mutation: delete the `tried` charge in the `usable == 0` arm and this
/// goes red at 60_000 reads.
#[test]
fn a_source_that_returns_zero_sectors_still_obeys_the_scan_budget() {
const MAX_TRIES: usize = 50_000;
let mut src = MockSource::new(0x00);
src.short_read = Some(0);
// Deliberately LARGER than the budget: if the budget is what stops the
// scan, the extent's own length is never reached.
let ext = [crate::disc::Extent {
start_lba: 0,
sector_count: 60_000,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().len();
assert!(
reads <= MAX_TRIES,
"an Ok(0)-returning source must be stopped by the {MAX_TRIES}-sector \
budget, not by the disc-declared extent length; got {reads} reads"
);
}
/// crack_key caps total scanned sectors at 50_000 even when extents are
/// far larger, and counts EVERY scanned sector (clear ones included)
/// toward the budget. With one 200_000-sector extent of clear sectors, it
@@ -875,7 +1009,7 @@ mod tests {
}
/// THE Fix 6 regression: a scan that SEES scrambled sectors (flag set) but
/// recovers no key (the mock's zeroed data has no Stevenson crib) must
/// recovers no key (the mock's zeroed data has no recoverable crib) must
/// return `ScrambledUncracked` — a HARD failure — NOT `Unencrypted`. The
/// old code conflated this with "unencrypted" and muxed scrambled MPEG as
/// plaintext (garbage at exit 0).
@@ -1032,12 +1166,17 @@ mod tests {
);
}
/// MISSING #1 guard: the re-crack path (the `Option`-returning `crack_key`,
/// `fail_on_locked == false`) must NOT hard-fail on a CSS-locked read — it
/// returns `None`. A lapsed-AGID re-crack of another VTS stays soft so a
/// genuinely crackable title isn't killed by a transient locked read.
/// `crack_key` (the `Option`-returning convenience wrapper) collapses
/// `ScrambledUncracked` and `Unencrypted` alike to `None` via
/// [`CrackOutcome::into_state`], so an all-locked scan still reads `None`
/// here even though the scan itself now treats every CSS-lock as a hard
/// `ScrambledUncracked` (see `crack_key_scan`'s removal of the dead
/// `fail_on_locked` parameter). Callers that need to tell "locked/
/// uncrackable" apart from "genuinely clear" must use `crack_key_outcome`,
/// which the `all_locked_synthetic_iso_yields_css_key_missing_signal` test
/// pins directly.
#[test]
fn crack_key_recrack_locked_is_none_not_hard_fail() {
fn crack_key_all_locked_collapses_to_none() {
let mut src = MockSource::new(0x30);
src.lock_all = true;
let extents = [Extent {
@@ -1123,7 +1262,7 @@ mod tests {
/// budget). With a small failing extent, every sector is attempted and the
/// function returns None.
///
/// Grounding: `if reader.read_sectors(...).is_ok() && is_scrambled(...)` —
/// Grounding: `if reader.read_sectors(...).is_ok() && is_scrambled_pack(...)` —
/// an Err simply falls through to `i += 1`.
/// Mutation: change the read-error handling to `reader.read_sectors(...)?`
/// (propagate) -> crack_key would stop after the first error and read only
@@ -1184,7 +1323,7 @@ mod tests {
/// SCAN-LEVEL CRACKED (audit gap "MockSource never yields a crackable
/// sector"): drive the full `crack_key_scan` over a synthetic ISO whose
/// scan hits a Stevenson-crackable scrambled sector. The outcome must be
/// scan hits a crackable scrambled sector. The outcome must be
/// `CrackOutcome::Cracked` with a key that round-trips the sector, AND the
/// `crack_span` must be recorded as the half-open extent span (the per-VTS
/// routing key the mux path needs). Previously only the leaf crack and the
@@ -1248,7 +1387,7 @@ mod tests {
sector_count: 50,
}];
// Cracked: a real Stevenson-crackable sector in an otherwise clear scan.
// Cracked: a real crackable sector in an otherwise clear scan.
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let mut cracked_src = MockSource::new(0x00);
@@ -1362,6 +1501,44 @@ mod tests {
);
}
/// The decrypted-HD-DVD regression, end to end through `resolve_dvd_title_key`.
/// An `.evo` is MPEG-PS (`ContentFormat::MpegPs`) exactly like a DVD `.vob`
/// and carries `None` keys once decrypted, so it reaches the CSS crack. Its
/// RDI navigation packs are private_stream_2 (0xBF) with the pack-start code
/// and bits set at offset 0x14 — but HD-DVD carries no CSS at all. The 0x11
/// exclusion in `is_scrambled_pack` keeps those nav packs from flipping the
/// scan's `saw_scrambled` gate, so the scan returns `Unencrypted` and the
/// title muxes cleanly instead of hard-failing.
///
/// Catches the mutation of dropping the 0x11 exclusion: without it every 0xBF
/// RDI pack counts as scramble evidence, the crack finds no key (there is
/// none), and the scan returns `ScrambledUncracked` → `CssKeyMissing` (E7023)
/// on a perfectly good HD-DVD — the exact defect a real CI run produced.
#[test]
fn resolve_dvd_title_key_decrypted_hddvd_rdi_packs_scan_clean_no_e7023() {
let mut src = MockSource::new(0x30); // 0x14 bits set…
src.stream_id = crate::consts::pes_stream_id::PRIVATE_STREAM_2; // …but a 0xBF nav pack
let extents = [Extent {
start_lba: 0,
sector_count: 64,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
8,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("a decrypted HD-DVD's RDI nav packs are not CSS — the scan must not hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"no CSS key exists on an HD-DVD; keys must stay None and the title mux clean"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
@@ -1527,8 +1704,8 @@ mod tests {
/// PER-VTS RE-CRACK SUCCESS (audit gap "success path missing"): the prior
/// re-crack test only covered the locked→None path. Here a re-crack
/// (`crack_key`, `fail_on_locked == false`) over a DIFFERENT VTS's extents
/// finds that VTS's own crackable sector and returns a `CssState` whose
/// (`crack_key`) over a DIFFERENT VTS's extents finds that VTS's own
/// crackable sector and returns a `CssState` whose
/// `crack_span` matches the new extents — proving a key cracked for one VTS
/// is genuinely re-derived (not reused) for another.
#[test]
+4 -3
View File
@@ -4,7 +4,8 @@
//! Content Scramble System. They are mathematical constants derived from
//! the CSS specification, published in academic literature since 1999.
/// Table 1: byte substitution used in key mangling and sector seed processing.
/// Table 1: content-cipher substitution table, `plain = TAB1[cipher] ^ keystream`
/// (used by descramble/recover in lfsr.rs and keyless.rs).
pub const TAB1: [u8; 256] = [
0x33, 0x73, 0x3b, 0x26, 0x63, 0x23, 0x6b, 0x76, 0x3e, 0x7e, 0x36, 0x2b, 0x6e, 0x2e, 0x66, 0x7b,
0xd3, 0x93, 0xdb, 0x06, 0x43, 0x03, 0x4b, 0x96, 0xde, 0x9e, 0xd6, 0x0b, 0x4e, 0x0e, 0x46, 0x9b,
@@ -88,7 +89,7 @@ pub const TAB3: [u8; 512] = [
0x00, 0x24, 0x49, 0x6d, 0x92, 0xb6, 0xdb, 0xff, 0x00, 0x24, 0x49, 0x6d, 0x92, 0xb6, 0xdb, 0xff,
];
/// Table 4: LFSR0 byte permutation (used in initialization and output).
/// Table 4: permutes the LFSR0 output byte (`o0 = TAB4[o0]`).
pub const TAB4: [u8; 256] = [
0x00, 0x80, 0x40, 0xc0, 0x20, 0xa0, 0x60, 0xe0, 0x10, 0x90, 0x50, 0xd0, 0x30, 0xb0, 0x70, 0xf0,
0x08, 0x88, 0x48, 0xc8, 0x28, 0xa8, 0x68, 0xe8, 0x18, 0x98, 0x58, 0xd8, 0x38, 0xb8, 0x78, 0xf8,
@@ -111,7 +112,7 @@ pub const TAB4: [u8; 256] = [
/// Table 5: LFSR1 output permutation used in the keystream combiner.
/// `TAB5[i] == TAB4[i] ^ 0xFF` (bitwise complement of the TAB4 bit-reversal
/// table). Applied on the normal descramble/recrypt path (lfsr.rs) as well as
/// in the key-recovery fallback (crack.rs).
/// in the key-recovery fallback ([`super::keyless`]).
pub const TAB5: [u8; 256] = [
0xff, 0x7f, 0xbf, 0x3f, 0xdf, 0x5f, 0x9f, 0x1f, 0xef, 0x6f, 0xaf, 0x2f, 0xcf, 0x4f, 0x8f, 0x0f,
0xf7, 0x77, 0xb7, 0x37, 0xd7, 0x57, 0x97, 0x17, 0xe7, 0x67, 0xa7, 0x27, 0xc7, 0x47, 0x87, 0x07,
+70 -105
View File
@@ -72,14 +72,9 @@ static DECRYPT_POOL: RwLock<Option<Arc<rayon::ThreadPool>>> = RwLock::new(None);
pub fn set_decrypt_threads(n: usize) {
let clamped = n.min(MAX_THREADS);
DECRYPT_THREADS.store(clamped, Ordering::Relaxed);
// Drop the existing pool. Next decrypt_pool() call rebuilds with
// the new resolved thread count.
//
// Recover the guard on poisoning, exactly as `decrypt_pool` does. Skipping
// the swap on a poisoned lock silently kept the STALE pool alive while the
// atomic above already reported the new thread count, so the setting appeared
// to take effect and never did. The pool Arc is immutable once stored, so a
// prior panic cannot have left it half-written.
// Drop the existing pool so the next decrypt_pool() call rebuilds with the
// new thread count. Recover the guard on poisoning like `decrypt_pool` does —
// skipping the swap would keep the STALE pool alive, making the setting no-op.
let mut guard = DECRYPT_POOL.write().unwrap_or_else(|e| e.into_inner());
*guard = None;
}
@@ -100,10 +95,9 @@ fn decrypt_pool() -> Option<Arc<rayon::ThreadPool>> {
return Some(Arc::clone(pool));
}
}
// Slow path: build a new one under the write lock. Recover the guard
// on poisoning (a prior panic) rather than propagating a secondary
// panic — we simply rebuild. Double-check after acquiring in case
// another caller built it first.
// Slow path: build a new one under the write lock, recovering the guard on
// poisoning (a prior panic) rather than propagating a secondary panic — we
// simply rebuild. Double-check after acquiring in case another caller won.
let mut guard = DECRYPT_POOL.write().unwrap_or_else(|e| e.into_inner());
if let Some(pool) = guard.as_ref() {
return Some(Arc::clone(pool));
@@ -128,10 +122,9 @@ pub fn decrypt_threads() -> usize {
if explicit > 0 {
return explicit;
}
// Resolve the `FREEMKV_THREADS` env var + `available_parallelism()` ONCE and
// cache it — this runs on the per-buffer decrypt hot path, and a getenv +
// String alloc + parallelism syscall per call is pure overhead. The explicit
// `set_decrypt_threads` override above still takes effect dynamically.
// Resolve `FREEMKV_THREADS` + `available_parallelism()` ONCE and cache it:
// this runs on the per-buffer decrypt hot path, so a getenv/alloc/syscall
// per call is pure overhead (the `set_decrypt_threads` override still works).
static DEFAULT_THREADS: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
*DEFAULT_THREADS.get_or_init(|| {
let env = std::env::var("FREEMKV_THREADS")
@@ -208,11 +201,9 @@ fn unit_is_our_phase(unit_lba: u32, range_start: u32, unit_sectors: u32, phase:
Phase::Even => false,
Phase::Odd => true,
};
// `saturating_sub` and `max(1)`: both inputs come from the key map, which is
// built from disc structure. A unit below its own range start, or a zero
// unit size, means the map is malformed — that must not panic (debug
// overflow / divide-by-zero) inside a library used by a long-running
// service. Unit 0 of the range is even, which is the safe default.
// `saturating_sub`/`max(1)` guard against a malformed key map (unit below its
// range start, or zero unit size) — must not panic (overflow/div-by-zero) in
// a long-running service. Unit 0 of the range is even, the safe default.
let unit_ix = unit_lba.saturating_sub(range_start) / unit_sectors.max(1);
(unit_ix % 2 == 1) == want_odd
}
@@ -402,18 +393,6 @@ impl AacsKeyMap {
}
}
/// Decrypt a buffer of sectors in-place using a resolved [`AacsKeyMap`] — the
/// mux's TRUSTED decrypt. `base_lba` is the absolute LBA of `buf`'s first sector;
/// each aligned unit (3 sectors) is decrypted with the key the map assigns to its
/// LBA. There is NO key trial and NO `is_clean` verdict: the map already decided
/// the key from disc structure, so we apply it and move on — a unit that decrypts
/// to authored-bad TS passes through for the muxer to drop, never re-fetched.
///
/// Only [`DecryptKeys::Aacs`] uses a map (CSS self-cracks per region inside
/// [`decrypt_sectors`]; `None` is clear) — other variants are a no-op here so the
/// decorator can dispatch uniformly. A map index outside the held pool is a
/// fail-loud [`Error::DecryptFailed`]: the resolver's job is to guarantee every
/// selectable index is present, so a gap here is a resolver bug, not silent loss.
/// Decrypt `buf` with a resolved AACS key map. Thin wrapper over
/// [`decrypt_span`] — the map is the AACS scheme's input, not a second
/// orchestrator.
@@ -427,12 +406,20 @@ pub(crate) fn decrypt_sectors_mapped(
decrypt_span(buf, &mut keys, base_lba, Some(map), None).map(|_| ())
}
/// AACS scheme step: apply `map`'s per-unit keys to `buf`.
/// AACS scheme step: apply `map`'s per-unit keys to `buf`, in-place. `base_lba`
/// is the absolute LBA of `buf`'s first sector; each aligned unit (3 sectors) is
/// decrypted with the key the map assigns to its LBA. There is NO key trial and
/// NO `is_clean` verdict: the map already decided the key from disc structure,
/// so this applies it and moves on — a unit that decrypts to authored-bad TS
/// passes through for the muxer to drop, never re-fetched.
///
/// A SCHEME, not a policy. It reports what it could not open by returning
/// `Err(DecryptFailed)`; the decision that an unopenable unit must never be
/// emitted belongs to [`decrypt_span`], which is the one place that decides it
/// for every scheme.
/// for every scheme. A map index outside the held pool is likewise a fail-loud
/// [`Error::DecryptFailed`](crate::Error::DecryptFailed): the resolver's job is
/// to guarantee every selectable index is present, so a gap here is a resolver
/// bug, not silent loss.
fn apply_aacs_map(
buf: &mut [u8],
keys: &DecryptKeys,
@@ -462,23 +449,16 @@ fn apply_aacs_map(
}
}
// Cheap safety net for the "map must be right" model: with a correct
// phase-aware map, every CORRECT-PHASE forensic unit decrypts to clean TS, so
// this never fires in the happy path — but a map bug (wrong phase/key for a
// segment) surfaces as a loud DecryptFailed instead of silent corruption. Only
// forensic (Even/Odd) ranges are verified; base / multi-CPS (All) stays
// trust-only, so the common disc is byte-for-byte unchanged.
// Cheap safety net: a correct map always decrypts CORRECT-PHASE forensic
// units to clean TS, so this never fires happy-path — a map bug surfaces as
// loud DecryptFailed, not silent corruption. Only forensic ranges are verified.
let verify_failed = std::sync::atomic::AtomicBool::new(false);
let decrypt_one = |idx_in_buf: usize, chunk: &mut [u8]| {
if chunk.len() != unit_len {
// Trailing partial unit (buffer/region tail shorter than a whole unit).
// Normally a genuinely-clear content tail (source-zero padding or a
// short final fragment) — leave as-is. But a partial that is BOTH inside
// a mapped (encrypted) range AND flagged encrypted in its clear seed is
// an encrypted unit split across a boundary: a CBC fragment we cannot
// decrypt, so emitting it verbatim would ship ciphertext as clear. Fail
// loud instead (restores the guard the removed `decrypt_sectors` had).
// Trailing partial unit: normally a genuinely-clear tail, left as-is. But
// one that is BOTH inside a mapped range AND flagged encrypted is a CBC
// fragment we cannot decrypt — fail loud instead of shipping it as clear.
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
if map.entry_for(unit_lba).is_some()
&& aacs::content::aacs_unit_seed_encrypted(chunk, format)
@@ -488,20 +468,9 @@ fn apply_aacs_map(
return;
}
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
// No range covers this LBA. That is expected for clear filesystem / nav
// on a whole-disc read — but "the map has no key here" and "there is
// nothing to decrypt here" are different statements, and only the
// second makes passing the unit through correct.
//
// An ENCRYPTED unit outside every range is content we cannot key: on a
// multi-CPS disc that is an orphan clip referenced by no playlist, so
// it sits in no title extent and therefore in no range. Emitting it
// verbatim ships ciphertext where plaintext is meant to be, and extract
// then counts those bytes as good and reports the file complete.
//
// The split-unit branch immediately above already draws exactly this
// distinction. This one did not, so it was reached before the
// `aacs_unit_encrypted` gate below ever ran.
// No range covers this LBA — expected for clear filesystem/nav, but an
// ENCRYPTED unit outside every range is an orphan clip we cannot key;
// emitting it verbatim would ship ciphertext as clear content.
let Some((key_idx, phase, range_start)) = map.entry_for(unit_lba) else {
if aacs::content::aacs_unit_seed_encrypted(chunk, format) {
verify_failed.store(true, std::sync::atomic::Ordering::Relaxed);
@@ -568,8 +537,8 @@ fn apply_aacs_map(
/// reader was built without installing its map (a bug), so it fails loud rather
/// than apply a guessed key.
///
/// `unit_key_idx` and `content` are legacy parameters kept so the CSS / `None`
/// wrapper signatures stay stable; they are ignored (the CSS arm self-gates on its
/// `unit_key_idx` is a legacy parameter kept so the CSS / `None` wrapper
/// signature stays stable; it is ignored (the CSS arm self-gates on its
/// per-sector scramble flag). Returns `Err` if decryption was expected but
/// impossible; never produces silently corrupted output. The `usize` return is a
/// legacy unverified-byte count that is always `0` for the CSS / `None` arms.
@@ -629,12 +598,9 @@ fn decrypt_span(
let dropped: usize = match keys {
DecryptKeys::None => 0,
DecryptKeys::Aacs { .. } => {
// AACS decrypts EXCLUSIVELY through a resolved key map: the map keys
// every content unit up front and a missing key fails at RESOLVE
// time. No map here means an AACS reader was built without
// installing one — the old trial-decrypt path (try each held key,
// keep the first-tried plaintext on a miss) is gone precisely
// because it silently applied wrong keys.
// AACS decrypts EXCLUSIVELY via a resolved key map (missing key fails at
// RESOLVE time). No map here means the reader was built without one — the
// old trial-decrypt path is gone: it silently applied wrong keys on a miss.
let Some(map) = map else {
return Err(crate::error::Error::DecryptFailed);
};
@@ -642,12 +608,9 @@ fn decrypt_span(
0
}
DecryptKeys::Css { title_key } => {
// CSS SELF-recovers: the title key changes per VOB region and is
// re-cracked constantly, but always FROM THE DATA ITSELF — no external
// input. So the whole descramble-and-rekey is self-contained here (see
// `css::descramble_region`), and CSS does not need the post-decrypt
// recovery seam that AACS key-fetch / FMTS segment-skip use (those DO
// consume external inputs a `decrypt_sectors` caller cannot supply).
// CSS SELF-recovers: the title key changes per VOB region and is re-cracked
// constantly, but always FROM THE DATA ITSELF (see `css::descramble_region`),
// so it needs none of the external-input recovery seam AACS key-fetch uses.
css::descramble_region(buf, title_key)?
}
};
@@ -849,10 +812,10 @@ mod tests {
);
}
/// Build a Stevenson-crackable scrambled CSS sector for `title_key` (mirrors
/// Build a crackable scrambled CSS sector for `title_key` (mirrors
/// `crackable_sector` in the css::mod tests): a periodic run in the clear
/// header continues past 0x80 into the encrypted region, so
/// `stevenson::crack_title_key` recovers the key. Distinct `seed` values give
/// `keyless::crack_title_key` recovers the key. Distinct `seed` values give
/// two sectors different cribs, standing in for two VOB regions.
fn crackable_css_sector(title_key: &[u8; 5], seed: &[u8; 5]) -> Vec<u8> {
const RUN_START: usize = 0x59;
@@ -1030,11 +993,9 @@ mod tests {
/// scramble flag.
fn make_css_sector(title_key: &[u8; 5], seed: &[u8; 5], body_fill: u8) -> (Vec<u8>, Vec<u8>) {
let mut sector = vec![body_fill; 2048];
// A real scrambled DVD sector is an MPEG-2 PS pack, so it begins with
// the pack start code. The descrambler requires it before trusting
// byte 0x14 — without it this fixture is a sector shape that cannot
// occur on a disc, and the test would pass while the production gate
// rejected every sector like it.
// A real scrambled DVD sector is an MPEG-2 PS pack starting with the pack
// start code; the descrambler requires it before trusting byte 0x14. Without
// it the fixture is a shape that can't occur on disc, masking a rejecting gate.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] = 0x30; // scramble flag (bits 4-5)
sector[0x54..0x59].copy_from_slice(seed);
@@ -1157,12 +1118,12 @@ mod tests {
// Precondition: each sector must be crackable on its own (the rekey
// depends on it). If this fails the fixture, not the path, is at fault.
assert_eq!(
crate::css::stevenson::crack_title_key(&s0),
crate::css::keyless::crack_title_key(&s0),
Some(key_a),
"fixture s0 must crack to key_a standalone"
);
assert_eq!(
crate::css::stevenson::crack_title_key(&s1),
crate::css::keyless::crack_title_key(&s1),
Some(key_b),
"fixture s1 must crack to key_b standalone"
);
@@ -1437,10 +1398,9 @@ mod tests {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// Case A — range_start is itself unaligned (1001 % 3 == 2) and the extent
// begins on it, so unit offsets are 0, 3, 6, ... Under this shape the
// formula `(lba + range_start) / us` shifts every index by an ODD amount
// and inverts the kept half.
// Case A — range_start is itself unaligned (1001 % 3 == 2), so unit offsets
// are 0, 3, 6, ... Under this shape `(lba + range_start) / us` shifts every
// index by an ODD amount and inverts the kept half.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
@@ -1462,11 +1422,9 @@ mod tests {
units are the even-indexed ones counting from the range start"
);
// Case B — the extent begins one unit-remainder away from the range start
// (1001 - 1000 = 1), so offsets are 1, 4, 7, 10. Here `(lba - range_start)
// * us` inverts the halves instead: the division is what maps a byte
// offset onto a unit index, and multiplying happens to preserve parity
// only when the offset is already a multiple of the unit size.
// Case B — the extent begins one unit-remainder from the range start (1001 -
// 1000 = 1), so offsets are 1, 4, 7, 10. Here `(lba - range_start) * us`
// inverts the halves: multiplying only preserves parity for aligned offsets.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
@@ -1876,22 +1834,16 @@ mod tests {
}
// The index must be RANGE-RELATIVE: `(lba - start)`, not `(lba + start)`.
// Most value pairs give the same parity either way, so pin one where
// they genuinely disagree: (5-1)/2 = 2 (even, ours) but
// Pin a case where they genuinely disagree: (5-1)/2 = 2 (even, ours) but
// (5+1)/2 = 3 (odd, not ours).
assert!(
unit_is_our_phase(5, 1, 2, Phase::Even),
"the index must be measured from the range start"
);
// The `/ unit_sectors` -> `* unit_sectors` mutant is EQUIVALENT here,
// and deliberately not chased. Proof: aligned offsets are exact
// multiples of the unit size, so offset = k*u for unit index k.
// Dividing gives k; multiplying gives k*u^2, whose parity is
// parity(k)*parity(u^2) = parity(k) whenever u is ODD. `unit_sectors`
// is `ALIGNED_UNIT_LEN / 2048` = 3, a compile-time constant, so u is
// always odd and the two agree on every reachable input. Only an even
// unit size would separate them, and none exists.
// The `/ unit_sectors` -> `* unit_sectors` mutant is EQUIVALENT here and
// deliberately not chased: aligned offsets are k*u, and dividing vs.
// multiplying give the same parity since `unit_sectors` (=3) is always odd.
assert!(!unit_is_our_phase(33, 30, 3, Phase::Even));
}
@@ -1899,11 +1851,24 @@ mod tests {
/// below its own range start, or a zero unit size, are both map bugs — they
/// must return a defined answer rather than panicking on debug overflow or
/// dividing by zero.
///
/// Every case here asserts the DEFINED answer, not merely the absence of a
/// panic. The zero-unit-size case used to be written
/// `assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
/// accepts both answers and so pinned nothing at all: the guards could
/// invert and it would still pass. The answer is knowable —
/// `saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
/// is even — so pin it.
#[test]
fn phase_gate_does_not_panic_on_a_malformed_map() {
use super::{Phase, unit_is_our_phase};
// Unit below its own range start: saturating_sub clamps to 0, and unit
// 0 is even.
assert!(unit_is_our_phase(10, 100, 3, Phase::Even));
assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true);
// Zero unit size: max(1) makes the divisor 1, so the index is the raw
// offset 70 — even.
assert!(unit_is_our_phase(100, 30, 0, Phase::Even));
// Both malformations at once: offset 0 over divisor 1 is unit 0, even.
assert!(unit_is_our_phase(5, 5, 0, Phase::Even));
}
}
+35 -20
View File
@@ -92,12 +92,9 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
}
}
// `channel_count` and `sample_rate_hz` lived here as a third copy of the
// AudioChannels/SampleRate mappings. They were the only HONEST copy — returning
// 0 for Unknown where the canonical accessors fabricated 6 channels at 48 kHz —
// and their only caller was the trace line below, in this same file. The
// canonical accessors are honest now, so the duplicates are gone rather than
// left to drift a fourth time.
// `channel_count`/`sample_rate_hz` used to live here as a third, HONEST copy of
// the AudioChannels/SampleRate mappings (returning 0 for Unknown where the
// canonical accessors fabricated 6ch/48kHz). Canonical accessors are honest now.
// ── DVD cell-category dump (from the IFO scan, pre-lowering) ─────────────────
@@ -110,9 +107,8 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
pub fn dvd_cell_row(idx: usize, cell: &crate::ifo::DvdCell, dropped: bool) -> String {
let c = CellCategory::decode(cell.category);
// Per-cell keep/skip REASON (self-sufficient bug log): a dropped cell is a
// leading secondary angle/interleave block piece; a kept cell is either the
// first feature cell or genuine feature content. This makes the
// leading-cell-filter decision auditable from the log without the disc.
// leading secondary angle/interleave piece; a kept cell is the first feature
// cell or genuine content. Makes the leading-cell-filter decision auditable.
let verdict = if dropped {
"DROP(leading-secondary-block-piece)"
} else if c.is_secondary_block_piece() {
@@ -472,14 +468,31 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!(
target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason={}",
size={}B clips={} video={} reason={}",
main.playlist,
main.duration_secs,
main.size_bytes,
main.clips.len(),
main.video_streams().count(),
main_feature_reason(),
);
}
// Titles disqualified from main-feature candidacy for carrying no video — the
// streamless obfuscation decoys the `has-video` gate demotes. Logged so a field
// bug report shows WHY a playlist wasn't picked (the gap behind E6008 on `00245`).
for (ti, t) in disc.titles.iter().enumerate() {
if !t.has_video() {
tracing::debug!(
target: DIAG,
"tag=decision.demoted title_idx={} playlist={:?} dur={:.1}s size={}B reason=no-video",
ti,
t.playlist,
t.duration_secs,
t.size_bytes,
);
}
}
}
/// The `reason=` token on the main-feature decision row.
@@ -491,10 +504,12 @@ pub fn dump_disc(disc: &Disc) {
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
let keys: Vec<&str> = Disc::MAIN_FEATURE_ORDER_KEYS
.iter()
.chain(Disc::CANONICAL_TITLE_ORDER_KEYS.iter())
.copied()
.collect();
format!("main_feature_order({})", keys.join(", "))
}
fn dump_aacs(disc: &Disc) {
@@ -652,10 +667,9 @@ mod tests {
.collect(),
..DiscTitle::empty()
};
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's
// primary key among disc-fitting titles is LARGEST SIZE. "fewest clips"
// would predict the opposite, so the drifted string described a rule
// the comparator does not implement.
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's primary
// key among disc-fitting titles is LARGEST SIZE, not "fewest clips" (the
// drifted string described a rule the comparator does not implement).
let many_clips_big = sized(8_000_000_000, 40);
let one_clip_small = sized(1_000_000_000, 1);
assert_eq!(
@@ -670,8 +684,9 @@ mod tests {
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
reason,
"main_feature_order(nav-feature, authoring-feature, standalone, has-video, fits-disc, largest-size, longest, richest-audio)",
"the reason must name the selection keys (authoring, standalone-over-composite, and has-video gates, then the physical keys) in priority order"
);
}
+15 -15
View File
@@ -29,8 +29,12 @@ use super::layout::{DirNode, Layout};
use crate::error::{Error, Result};
use std::collections::BTreeMap;
/// Logical block / sector size. Fixed for every optical profile this crate reads.
pub(super) const SECTOR: usize = 2048;
/// Logical block / sector size. Fixed for every optical profile this crate
/// reads, and the same quantity as [`crate::consts::SECTOR_BYTES`] — aliased
/// rather than re-declared so the two cannot drift apart. The short name is
/// kept because it appears in ~25 extent and offset expressions across
/// `dirimage`, where the longer one would bury the arithmetic.
pub(super) use crate::consts::SECTOR_BYTES as SECTOR;
/// Descriptor version recorded in every tag. 2 = ECMA-167 2nd edition, which
/// is what UDF revisions up to and including 2.00 require.
@@ -455,10 +459,9 @@ fn file_entry(
s[160..168].copy_from_slice(&unique_id.to_le_bytes());
s[168..172].copy_from_slice(&0u32.to_le_bytes()); // length of EAs
let l_ad = extents.len() * 8;
// A short AD is 8 bytes and the entry has 2048 - 176 = 1872 bytes for
// them, i.e. 234 extents — over 200 GiB at the per-AD ceiling. Beyond
// that an Allocation Extent Descriptor chain would be required; refuse
// rather than write a truncated list.
// A short AD is 8 bytes, leaving room for 234 extents (~200 GiB) before an
// Allocation Extent Descriptor chain would be needed; refuse rather than
// write a truncated list.
if 176 + l_ad > SECTOR {
return Err(Error::DirImageTooLarge);
}
@@ -496,10 +499,9 @@ fn push_fid(buf: &mut Vec<u8>, name: &str, icb_lba: u32, is_dir: bool, is_parent
chars |= 0x08;
}
fid[18] = chars;
// The planner refuses any name whose encoding exceeds what this byte can
// hold (`layout::MAX_CS0_NAME_BYTES`), so this cannot wrap in practice. The
// assert states the invariant where it is relied on rather than trusting a
// check three files away; a wrap here would desynchronise the directory.
// Planner enforces `layout::MAX_CS0_NAME_BYTES` so this can't wrap in
// practice; the assert states that invariant where it's relied on
// instead of trusting a check three files away.
debug_assert!(
l_fi <= u8::MAX as usize,
"FID name length must fit one byte"
@@ -599,11 +601,9 @@ fn write_dir(out: &mut MetaSectors, layout: &Layout, dir: &DirNode) -> Result<()
fix_fid_tag_locations(&mut fids, dir.data_lba);
debug_assert_eq!(fids.len(), dir.data_bytes as usize);
// A directory's link count is 1 (its own FID in the parent) plus one for
// each child directory's parent FID pointing back at it.
// The planner caps subdirectory fan-out (`layout::MAX_SUBDIRS`) so this
// cannot overflow; saturating rather than wrapping keeps a future change to
// that cap from silently producing a wrong count.
// Link count is 1 (own FID in parent) plus one per child dir's parent FID.
// `layout::MAX_SUBDIRS` bounds fan-out so this can't overflow; saturating
// guards against a future change to that cap silently miscounting.
let link_count = (dir.dirs.len() as u16).saturating_add(1);
let fe = file_entry(
true,
+51 -98
View File
@@ -67,6 +67,15 @@ const MAX_SUBDIRS: usize = (u16::MAX - 1) as usize;
#[cfg(test)]
const MAX_SUBDIRS: usize = 4;
/// One entry per child plus the parent's own must still fit the 16-bit field.
///
/// At MODULE scope, and it has to be. This assertion previously sat inside
/// `mod tests`, which is `#[cfg(test)]`, while carrying `#[cfg(not(test))]`
/// itself — so it was compiled in NO configuration and could never fire, which
/// made it exactly the dead gate the test above it was written to replace.
#[cfg(not(test))]
const _: () = assert!(MAX_SUBDIRS + 1 == u16::MAX as usize);
/// Largest image this planner will synthesize, in sectors (128 GiB).
///
/// A DVD title set records where its VOBS begins as an offset in its own IFO,
@@ -209,18 +218,14 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
if is_excluded(&name) {
continue;
}
// `file_type()` does NOT follow symlinks; `metadata()` does. A symlink
// to a file is materialized as that file (a legitimate way to assemble
// a folder), a symlink to a directory is skipped — following one can
// loop, and UDF has no link this maps onto.
// `file_type()` does NOT follow symlinks; `metadata()` does. A symlink to
// a file is materialized as that file; a symlink to a directory is
// skipped — following one can loop, and UDF has no link this maps onto.
let ft = entry.file_type().map_err(Error::from)?;
let child_path = format!("{}/{}", disc_path.trim_end_matches('/'), name);
// A File Identifier Descriptor records the encoded name length in ONE
// byte. A longer name would wrap that field and desynchronise every
// later entry in the directory, so refuse while planning rather than
// encode something unreadable. 255 ASCII bytes is a legal name on
// ext4/APFS/NTFS and already exceeds this once the CS0 compression
// byte is added, so it is reachable without anything exotic.
// A FID records the encoded name length in ONE byte; a longer name would
// wrap it and desynchronise the directory, so refuse while planning. 255
// ASCII bytes (legal on ext4/APFS/NTFS) already exceeds it with CS0 added.
if crate::dirimage::encode::encode_cs0(&name).len() > MAX_CS0_NAME_BYTES {
return Err(Error::DirNameTooLong { path: child_path });
}
@@ -228,29 +233,13 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
if *entries > MAX_ENTRIES {
return Err(Error::DirImageTooLarge);
}
// Key the uniqueness check on the name AS THE READER WILL SEE IT, by
// round-tripping through the very encoder and parser that will be used.
//
// The host name is not that name. `parse_udf_name` trims leading and
// trailing whitespace and drops any code unit `char::from_u32` rejects
// (a lone surrogate half, i.e. every non-BMP character the encoder
// emits as a surrogate pair). So " A.M2TS" and "A.M2TS", or "A<astral>.M2TS"
// and "A.M2TS", are distinct hosts that collapse to ONE name on read —
// and `find`/`read_file` take the first match, so a title silently
// resolves to the wrong file's extents and muxes the wrong bytes at
// exit 0. Comparing the raw host names cannot see that; deriving the
// key from the same two functions cannot drift from it.
// Key uniqueness on the name AS THE READER WILL SEE IT (round-trip through
// the same encoder/parser it uses): `parse_udf_name` trims/drops chars, so
// distinct hosts (" A.M2TS" vs "A.M2TS") can collapse and `find` mismatches.
let as_read = crate::udf::parse_udf_name(&crate::dirimage::encode::encode_cs0(&name));
// Nothing left after the reader is done with it — a name made entirely
// of characters `parse_udf_name` drops, e.g. a sidecar folder named
// with a single emoji. The entry would exist in the image and be
// addressable by nothing.
//
// SKIP it, do not fail the plan. Failing turned an irrelevant extra
// file into an un-rippable folder that 1.6.0 handled fine (the
// unreachable entry merely sat there, harming nothing), and reported it
// as a collision with a file that does not exist. It is also not
// pushed to `names`, so it cannot collide with anything either.
// Nothing left after the reader drops it (e.g. a name of a single emoji):
// it would exist in the image but be addressable by nothing. SKIP rather
// than fail the plan (1.6.0 handled such extras fine); not pushed to `names`.
if as_read.is_empty() {
tracing::warn!(
target: "freemkv::dirimage",
@@ -261,10 +250,9 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
}
names.push(as_read.to_ascii_uppercase());
if ft.is_dir() {
// A directory's File Entry records its link count in 16 bits: one
// per child directory plus one for its own entry in the parent.
// The global entry cap alone permits a single directory holding
// more than that, which would wrap the count.
// Link count (child dirs + 1) is stored in 16 bits; the global entry
// cap alone permits a single directory holding more than that, which
// would wrap the count.
if dirs.len() >= MAX_SUBDIRS {
return Err(Error::DirImageFanout {
path: disc_path.to_string(),
@@ -274,30 +262,16 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
} else {
let meta = match std::fs::metadata(entry.path()) {
Ok(m) => m,
// A broken symlink or a file that vanished between readdir and
// stat: skip it rather than plan an extent that cannot be read.
// Anything else — permission denied, an I/O error on the host
// volume — is NOT a missing file, and skipping it would drop a
// real file out of the image with nothing reported.
// A broken symlink or a file gone between readdir and stat: skip it
// rather than plan an unreadable extent. Anything else (e.g. perm
// denied) is NOT a missing file, so skipping it would drop it silently.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => continue,
Err(e) => return Err(Error::from(e)),
};
if !meta.is_file() {
// Not a plain file after following the link. The common case is
// a SYMLINK TO A DIRECTORY — a normal way to keep 40 GB of
// streams off the system disk — and `entry.file_type()` above
// reports the link, not the target, so such a subtree never
// enters `dirs` and is silently absent from the image.
//
// The result was the worst class this release exists to close:
// BDMV/PLAYLIST and CLIPINF still synthesize, so the folder
// scans and enumerates titles, every clip resolves to no
// extents, and the mux takes its clean-EOF path and writes a
// near-empty MKV at exit 0.
//
// It is still skipped rather than followed — following link
// targets invites cycles and escapes from the folder — but it
// is now SAID, at the same volume as the name-skip above.
// Not a plain file after following the link: commonly a SYMLINK TO A
// DIRECTORY, silently vanishing before (clips → no extents, near-empty
// MKV at exit 0). Still skipped, not followed, but now SAID.
tracing::warn!(
target: "freemkv::dirimage",
path = %child_path,
@@ -319,10 +293,9 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
}
}
// `UdfFs::find_dir` / `read_file` match path components case-insensitively,
// so two entries in one directory differing only in case are
// indistinguishable to every consumer — the second would silently shadow
// the first. Only reachable on a case-sensitive host volume.
// `find_dir`/`read_file` match path components case-insensitively, so two
// entries differing only in case are indistinguishable to every consumer —
// the second would silently shadow the first. Only on a case-sensitive host.
names.sort();
for pair in names.windows(2) {
if pair[0] == pair[1] {
@@ -419,12 +392,9 @@ fn be_u32(buf: &[u8], off: usize) -> Option<u32> {
fn read_head(path: &Path, n: usize) -> Result<Vec<u8>> {
use std::io::Read;
let mut buf = vec![0u8; n];
// `read_exact`, not `read`: a single `read` may legally return fewer bytes
// than asked on a network or FUSE mount — and a NAS-hosted backup is the
// normal case for this feature. A short buffer sends every placement offset
// through `unwrap_or(0)`, recording NO constraint, so the planner and the
// reader disagree about where a VOB begins and the rip reads the wrong
// sectors for a whole title at exit 0.
// `read_exact`, not `read`: a single `read` may legally return fewer bytes on
// a network/FUSE mount (a NAS-hosted backup is normal here). A short buffer
// records NO constraint, so the rip reads the wrong sectors at exit 0.
let mut f = std::fs::File::open(path).map_err(Error::from)?;
f.read_exact(&mut buf).map_err(Error::from)?;
Ok(buf)
@@ -473,10 +443,9 @@ fn place_video_ts(vts: &mut DirNode, start: u32) -> Result<u32> {
// Required start blocks, resolved as each IFO is placed.
let mut menu_req: std::collections::HashMap<u32, u32> = std::collections::HashMap::new();
let mut title_req: std::collections::HashMap<u32, u32> = std::collections::HashMap::new();
// Groups whose IFO has already been placed. Kept separately from the
// constraint maps because an IFO that declares no offsets inserts into
// neither, so keying the duplicate check on those maps would miss a second
// IFO for the same set — which is the collision the check exists for.
// Groups whose IFO has already been placed, kept separately from the
// constraint maps: an IFO declaring no offsets inserts into neither, so
// keying the duplicate check on those maps would miss the collision.
let mut seen_ifo: std::collections::HashSet<u32> = std::collections::HashSet::new();
let mut cursor = start;
@@ -508,11 +477,9 @@ fn place_video_ts(vts: &mut DirNode, start: u32) -> Result<u32> {
{
let head = read_head(&vts.files[i].host, 0xC8)?;
let menu = be_u32(&head, 0xC0).unwrap_or(0);
// One group per title set. Two files whose names differ only in
// how the number is written — `VTS_01_0.IFO` and `VTS_1_0.IFO` —
// parse to the same group, and the second insert would overwrite
// the first's constraint, placing a VOB at an address the IFO the
// reader uses does not point to. Refuse instead of picking one.
// One group per title set: `VTS_01_0.IFO` and `VTS_1_0.IFO` parse to
// the same group, and a second insert would overwrite the first's
// constraint, placing a VOB where the reader's IFO doesn't point.
if !seen_ifo.insert(c.group) {
return Err(Error::DirNameCollision {
host: vts.files[i].disc_path.clone(),
@@ -521,12 +488,9 @@ fn place_video_ts(vts: &mut DirNode, start: u32) -> Result<u32> {
if menu != 0 {
menu_req.insert(c.group, lba.saturating_add(menu));
}
// Only a VTS IFO carries a title VOBS pointer at 0xC4. In the VMG
// (`VIDEO_TS.IFO`) that offset is TT_SRPT, the title search pointer
// table — a sector offset INSIDE the IFO, not a pointer to another
// file, so treating it as one would place `VIDEO_TS.VOB` at a
// meaningless address. (`ifo.rs::parse_vmg` reads the same field as
// TT_SRPT; the two must not drift.)
// Only a VTS IFO carries a title VOBS pointer at 0xC4; in the VMG that
// offset is TT_SRPT (a sector offset INSIDE the IFO, not a file
// pointer). `ifo.rs::parse_vmg` reads the same field; must not drift.
if c.group > 0 {
let title = be_u32(&head, 0xC4).unwrap_or(0);
if title != 0 {
@@ -714,13 +678,9 @@ pub(super) fn plan(root: &Path) -> Result<Layout> {
.position(|d| d.name.eq_ignore_ascii_case("VIDEO_TS"))
{
cursor = place_video_ts(&mut tree.dirs[idx], cursor)?;
// `place_video_ts` places only the FILES directly inside VIDEO_TS,
// because only those carry the IFO-relative constraints. Anything in a
// subdirectory of VIDEO_TS still needs data placed: without this its
// File Entry is written with the file's real size but no extents, so it
// appears in the tree at full length and reads back as nothing, at
// exit 0. The same folder under BDMV/ was always placed correctly,
// which is what made the gap easy to miss.
// `place_video_ts` places only the FILES directly inside VIDEO_TS (the ones
// with IFO-relative constraints); subdirectories still need placing, or
// their File Entry has real size but no extents and reads as nothing.
for sub in tree.dirs[idx].dirs.iter_mut() {
place_generic(sub, &mut cursor)?;
}
@@ -980,11 +940,9 @@ mod tests {
/// plainly that it does NOT exercise `walk`.
#[test]
fn the_subdir_cap_refuses_a_folder_with_too_many_subdirectories() {
// Executes the guard, rather than restating the constant. A directory's
// File Entry records its link count in 16 bits — one per child
// directory plus one for its own entry in the parent — so exceeding it
// would wrap the count and produce an image whose directory structure
// lies about itself.
// Executes the guard rather than restating the constant: link count (child
// dirs + 1) is 16 bits, so exceeding it wraps and the image lies about
// its own directory structure.
let dir = std::env::temp_dir().join(format!("fmkv-fanout-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
@@ -998,11 +956,6 @@ mod tests {
"expected DirImageFanout, got {err:?}"
);
// And the real production value is what ships: one per child plus the
// parent's own entry must still fit in the 16-bit field.
#[cfg(not(test))]
const _: () = assert!(MAX_SUBDIRS + 1 == u16::MAX as usize);
let _ = std::fs::remove_dir_all(&dir);
}
}
+15 -45
View File
@@ -127,18 +127,9 @@ impl DirImage {
let mut files = Vec::with_capacity(nodes.len());
let mut ranges = Vec::new();
for (idx, node) in nodes.iter().enumerate() {
// Carry the plan-time mtime ONLY for files whose CONTENT the plan
// read — the DVD IFOs, whose bytes 0xC0/0xC4 decide where every VOB
// is placed (`layout::place_video_ts` -> `read_head`).
//
// For every other file the plan depends on the SIZE alone, and size
// is already checked. Comparing mtime on those buys nothing and
// costs real false positives: disc backups commonly live on
// exFAT/FAT32, which stores local time, so a long rip spanning a
// DST transition sees a whole-hour shift on a file nobody touched
// and would abort hours in, blaming a change that did not happen.
// The multi-gigabyte VOBs are exactly the files a long rip re-opens
// after the handle cache evicts them.
// Carry plan-time mtime ONLY for files whose CONTENT the plan read (DVD
// IFOs, whose 0xC0/0xC4 bytes place every VOB); others depend only on
// SIZE, and comparing mtime there false-positives on exFAT/FAT32 + DST.
let content_sensitive = node
.disc_path
.rsplit('.')
@@ -228,16 +219,9 @@ impl DirImage {
}
let f = File::open(&self.files[file].host).map_err(Error::from)?;
let md = f.metadata().map_err(Error::from)?;
// Size AND mtime. Size alone is content-blind, and this plan depends on
// content: a DVD's VOB placement comes from bytes 0xC0/0xC4 of its IFO,
// and an IFO rewritten in place keeps its length because IFOs occupy a
// whole number of sectors. The size check would pass while every title
// extent pointed at the wrong sectors — corrupt video behind an intact
// structure, reported complete at exit 0.
//
// Only compared when both sides have a timestamp; a platform or
// filesystem that reports none simply falls back to the size check
// rather than failing every read.
// Size AND mtime: size alone is content-blind, and VOB placement depends on
// IFO bytes 0xC0/0xC4 — an IFO rewritten in place keeps its sector-aligned
// length, so size alone would miss it. mtime only compared if both present.
let changed_size = md.len() != self.files[file].size;
let changed_mtime = match (self.files[file].mtime, md.modified().ok()) {
(Some(planned), Some(live)) => planned != live,
@@ -271,19 +255,9 @@ impl DirImage {
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Release the window just read, every time.
//
// The ISO source accumulates and drops in chunks because it reads
// one file linearly, so a running start offset always names the
// bytes it has consumed. Reads here jump between files, so there is
// no single cursor to accumulate against — an accumulated byte
// count paired with one read's offset names 1/Nth of what was
// actually consumed and leaves the rest pinned, which is how the
// first version of this got it wrong.
//
// Dropping per read costs one advisory syscall per batch (4-16 MiB),
// which is nothing against the read itself, and it is correct
// regardless of how reads interleave across files.
// Release the window just read, every time (unlike the ISO source's
// accumulate-and-drop, which relies on one linear cursor — reads here
// jump between files, so an accumulated count would leave bytes pinned).
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
@@ -319,18 +293,14 @@ impl SectorSource for DirImage {
return Err(Error::UdfBufferTooSmall);
}
buf[..need].fill(0);
// Walk the request in RUNS, not sector by sector. A mux batch is 8192
// sectors and almost always lands entirely inside one stream file's
// extent; per-sector seek+read would issue 8192 syscalls for what is
// one 16 MiB sequential read.
// Walk the request in RUNS, not sector by sector: a mux batch (8192
// sectors) almost always lands in one extent, so per-sector seek+read
// would cost 8192 syscalls for what is one 16 MiB sequential read.
let mut i = 0u32;
while i < count as u32 {
// Checked: callers saturate their LBAs (`disc/dvd.rs` builds a cell
// start as `vob_start_sector.saturating_add(cell.first_sector)`, and
// the prefetcher adds an offset the same way), so a crafted IFO can
// present a request at the very top of the address space. Wrapping
// here would fold `at` back to a LOW sector and hand the muxer a
// different file's bytes with nothing reported.
// Checked: callers saturate their LBAs, so a crafted IFO can present a
// request at the top of the address space. Wrapping here would fold
// `at` to a LOW sector, handing the muxer a different file's bytes.
let Some(at) = lba.checked_add(i) else {
break;
};
+119 -17
View File
@@ -17,18 +17,31 @@ struct Scratch(PathBuf);
impl Scratch {
fn new(tag: &str) -> Self {
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{:?}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
let p = Self::unique_path(tag);
std::fs::create_dir_all(&p).unwrap();
Self(p)
}
/// The path a scratch dir takes — kept separate from directory creation so
/// its uniqueness is testable without a syscall between draws. A monotonic
/// counter, NOT a timestamp, is what guarantees it: `SystemTime::now()`
/// resolves to only a MICROSECOND on macOS, so two of the seven parallel
/// tests that share the "bdmv" tag routinely read the same value within one
/// microsecond, collide on the same directory, and the first to finish
/// `remove_dir_all`s it out from under the others' reads — an intermittent
/// `read_sectors` ENOENT. The counter is unique regardless of clock
/// granularity; pairing it with the pid keeps it unique across test-binary
/// processes.
fn unique_path(tag: &str) -> PathBuf {
static SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let uniq = SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{uniq}",
std::process::id()
));
p
}
fn path(&self) -> &Path {
&self.0
}
@@ -69,6 +82,28 @@ fn bdmv_scratch() -> (Scratch, Vec<u8>, Vec<u8>) {
(s, index, clip)
}
/// Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
/// tag and run in parallel; if two land the same path, the first to drop
/// `remove_dir_all`s it out from under the other's reads, which surfaced as an
/// intermittent `read_sectors` failure. Regression: with the old
/// `SystemTime::now()` name this is RED, because macOS's clock advances only
/// per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
/// between them to nudge the clock, mirroring the real cross-thread collision —
/// hands back the same value many times over. The counter-based name is unique
/// regardless of clock granularity.
#[test]
fn scratch_paths_are_unique_even_at_clock_resolution() {
let paths: Vec<PathBuf> = (0..1000).map(|_| Scratch::unique_path("uniq")).collect();
let distinct: std::collections::HashSet<&PathBuf> = paths.iter().collect();
assert_eq!(
distinct.len(),
paths.len(),
"every Scratch must own a distinct path; a timestamp-based name collides \
under macOS's microsecond clock and lets one parallel test delete \
another's files mid-read"
);
}
// ── The de-risking spike ────────────────────────────────────────────────────
/// THE load-bearing assertion of the whole design: metadata synthesized here
@@ -510,10 +545,8 @@ fn scan_image_enumerates_a_bdmv_folder() {
let (s, _, _) = bdmv_scratch();
let mut img = DirImage::open(s.path()).unwrap();
let cap = img.capacity_sectors();
// The playlist here is filler, so no titles are expected — what is being
// asserted is that the scan reaches the BD enumerator at all (it would
// return `UdfNotFilesystem` if the synthesized volume did not parse) and
// reports the structure it found.
// Playlist is filler, no titles expected; this asserts the scan reaches the
// BD enumerator at all (would return `UdfNotFilesystem` if unparseable).
let disc = crate::disc::Disc::scan_image(&mut img, cap, &crate::disc::ScanOptions::default())
.expect("scan_image must accept a synthesized BDMV image");
assert!(!disc.encrypted, "a decrypted folder has no AACS directory");
@@ -650,6 +683,76 @@ fn an_aacs_folder_with_scrambled_content_is_rejected() {
assert_eq!(err.code(), crate::error::E_DIR_IMAGE_ENCRYPTED);
}
// ── The OTHER door: `dir://` through the PES input path ─────────────────────
// `scan_dir` and `mux::resolve::input("dir://…")` once disagreed: only `scan_dir`
// re-judged the verdict from CONTENT. Fixed via `apply_folder_encryption_verdict`.
/// The two doors must AGREE. Same folder, same verdict, same extents.
///
/// Stated as a differential rather than a bare `is_ok()` so it cannot go
/// vacuous: if the fixture ever stops producing an AACS state, a one-sided
/// `Ok` assertion would still pass while guarding nothing, whereas "both doors
/// see the same title" is the invariant the shared function actually exists to
/// hold.
#[test]
fn both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory() {
// `s` MUST outlive `stream`: the pipeline's producer thread is still
// reading the folder until the stream is dropped, and `Scratch::drop`
// removes the directory out from under it.
let s = playable_bdmv("dirdoor", false);
s.file("AACS/Unit_Key_RO.inf", &[0u8; 64]);
s.file("AACS/MKB_RO.inf", &[0u8; 64]);
// Door 1 — the scan path.
let (disc, _reader) =
crate::session::scan_dir(s.path(), crate::disc::ScanOptions::default()).unwrap();
assert!(
!disc.encrypted,
"door 1 must judge the clear folder decrypted"
);
let scanned_extents = disc.titles[0].extents.clone();
assert!(
!scanned_extents.is_empty(),
"fixture must have real extents"
);
// Door 2 — the PES input path the CLI actually rips through.
let url = format!("dir://{}", s.path().display());
let stream = crate::input(&url, &crate::InputOptions::default())
.expect("dir:// input must open a clear folder, exactly as scan_dir does");
assert_eq!(
stream.info().extents,
scanned_extents,
"the two doors selected different titles from the same folder"
);
drop(stream);
}
/// The other verdict, through the same door: a folder whose content units are
/// really scrambled is refused with the TYPED code, not muxed into garbage.
///
/// This is the load-bearing half. `E_DIR_IMAGE_ENCRYPTED` is produced at
/// exactly one site in the crate (`session::apply_folder_encryption_verdict`),
/// reachable from here only through the `is_folder` argument — so this test
/// cannot pass if that argument is dropped, whatever else changes.
#[test]
fn a_scrambled_folder_is_refused_through_the_dir_url_door_too() {
let s = playable_bdmv("dirdoorenc", true);
s.file("AACS/Unit_Key_RO.inf", &[0u8; 64]);
s.file("AACS/MKB_RO.inf", &[0u8; 64]);
let url = format!("dir://{}", s.path().display());
let err = match crate::input(&url, &crate::InputOptions::default()) {
Ok(_) => panic!("a scrambled folder must not open as a PES source"),
Err(e) => e,
};
assert_eq!(
crate::error::error_code(&err),
Some(crate::error::E_DIR_IMAGE_ENCRYPTED),
"expected the typed dir-source-encrypted code, got: {err}"
);
}
// ── External oracle ─────────────────────────────────────────────────────────
/// Write a synthesized image to a real file and ask the OS to mount it.
@@ -864,10 +967,9 @@ fn dump_vts_ifo_reads_for_an_image() {
let p = format!("/VIDEO_TS/VTS_{n:02}_0.IFO");
match fs.read_file(&mut img, &p) {
Ok(b) => {
// Hash the CONTENT, not just the length: two IFOs of equal
// size and magic can still differ, and comparing only length
// and magic is what made an earlier pass of this diagnostic
// wrongly report the images as identical.
// Hash the CONTENT, not just length: two IFOs of equal size and
// magic can still differ, which is why an earlier pass of this
// diagnostic wrongly reported the images as identical.
let mut h: u64 = 0xcbf2_9ce4_8422_2325;
for byte in &b {
h ^= *byte as u64;
+815 -189
View File
File diff suppressed because it is too large Load Diff
+325 -105
View File
@@ -7,15 +7,47 @@ use crate::udf;
impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
///
/// Returns the scanned titles plus, when the disc's own First-Play
/// navigation deterministically dispatches to a title, that title's
/// `playlist_id` — the nav-resolved main feature. The caller feeds it into
/// `sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the
/// BD path uses. It is `None` (→ existing ranking heuristics stand) whenever
/// the navigation is menu-only, malformed, or does not converge.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Vec<DiscTitle> {
halt: Option<&crate::halt::Halt>,
) -> Result<(Vec<DiscTitle>, Option<u16>)> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(_) => return Vec::new(),
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok((Vec::new(), None)),
};
// Follow the disc's First-Play navigation like a player would, to find the
// nav-chosen feature title (issue #40). Read-only; any failure/non-convergence
// yields `None`, so this only re-ranks, never removes, a title.
let nav_target = crate::dvdnav::resolve_main_title(reader, udf_fs);
let mut nav_feature: Option<u16> = None;
let mut titles = Vec::new();
let mut title_number: u16 = 0;
@@ -39,50 +71,27 @@ impl Disc {
crate::ifo::TvSystem::Pal => ColorSpace::Bt470bg,
crate::ifo::TvSystem::Ntsc => ColorSpace::Smpte170m,
},
// DVD pixels are anamorphic 720x480/576; the real display shape
// is the IFO aspect flag, not the pixel grid. Carry it so the
// MKV muxer writes a correct 16:9 / 4:3 DisplayWidth/Height
// instead of the square-pixel 3:2 / 5:4 it would otherwise emit.
// DVD pixels are anamorphic 720x480/576; real display shape comes from the
// IFO aspect flag, not the pixel grid. Carry it so the MKV muxer emits correct
// 16:9/4:3 DisplayWidth/Height instead of the wrong square-pixel 3:2/5:4.
display_aspect: Some(match ts.video.aspect {
crate::ifo::DvdAspect::R16x9 => (16, 9),
crate::ifo::DvdAspect::R4x3 => (4, 3),
}),
secondary: false,
label: String::new(),
// TODO(spec): populate from the MPEG-2 picture coding extension
// once the codec parser surfaces it. `Mpeg2Parser` already reads
// `top_field_first` (mux/codec/mpeg2.rs `picture_nb_fields`) but
// tracks are built from the IFO scan BEFORE any frame is parsed;
// wiring it requires a CodecParser accessor surfaced through
// PipelinedPesStream/DiscStream into the output title (mirroring
// the existing `codec_private` handshake). Until then `None`
// means the muxer falls back to TFF (correct for ~all DVDs).
// TODO(spec): DVD MPEG-2 carries no VUI; the colour signalling is
// the sequence_display_extension colour_description when present.
// Surface it from `Mpeg2Parser` (same handshake as above) and set
// this so a disc that states e.g. BT.601-625 colour overrides the
// PAL/NTSC guess in `color_space`. `None` uses the enum fallback.
// TODO(spec): populate top_field_first + measured colour from Mpeg2Parser once
// tracks are built after frame parsing -- needs a CodecParser->title channel
// like `codec_private`. Until wired, `None` falls back to TFF / PAL-NTSC guess.
measured_cicp: None,
});
// TODO(spec): DefaultDuration is derived from the declared IFO
// frame_rate (25 / 29.97). A soft-telecined 23.976-in-29.97 DVD then
// reports 29.97 fps instead of the true 23.976 film rate. The pulldown
// cadence is detectable from the parser's per-picture `nb_fields`
// (repeat_first_field) — when most frames are 3-field-then-2-field
// 2:3 pulldown the film rate is frame_rate × 4/5. Emitting the film
// DefaultDuration needs the parser to report the measured cadence
// through the same parser→title channel as `top_field_first` above;
// left as a follow-up to avoid a speculative rate change here.
// TODO(spec): DefaultDuration comes from IFO frame_rate, so a soft-telecined
// 23.976-in-29.97 DVD reports 29.97 instead of true film rate. Cadence is
// detectable via nb_fields (2:3 pulldown); needs same channel as top_field_first above.
// Map DvdAudioAttr to Stream::Audio. The PID is derived from the
// stream's REAL on-wire private_stream_1 sub-stream id (assigned
// by per-codec ordinal in the IFO scan) via the same
// `dvd_audio_pid` table the demuxer's `PsPacket::dvd_pid` uses,
// so a mixed-codec title (AC-3 + DTS + LPCM) routes correctly
// instead of colliding on 0xBD00. Streams carried as a regular
// MPEG-audio PES (MP1/MP2, no sub-id) fall back to a distinct
// 0xBD00+ordinal PID — disjoint from the 0xBD80+ canonical audio
// space — though they are not routed via `dvd_pid` today.
// Map DvdAudioAttr to Stream::Audio. PID derives from the on-wire private_stream_1
// sub-id via the same `dvd_audio_pid` table the demuxer uses, so mixed-codec
// titles route correctly instead of colliding; MP1/MP2 (no sub-id) use 0xBD00+ordinal.
let audio_streams: Vec<Stream> = ts
.audio_streams
.iter()
@@ -109,24 +118,24 @@ impl Disc {
for (vts_title_idx, dvd_title) in ts.titles.iter().enumerate() {
title_number += 1;
// If the nav executor picked this title (by VTS number + 1-based position,
// the same `vts_ttn` convention used below), remember its global
// `playlist_id` so the caller can promote it in ranking.
if let Some(rt) = nav_target
&& rt.vtsn == ts.vts_number
&& rt.vts_ttn as usize == vts_title_idx + 1
{
nav_feature = Some(title_number);
}
// Diagnostic dump (--log-level 3): per-cell category table +
// chapter map for this title, BEFORE lowering drops the
// per-cell IFO detail. No-op unless freemkv::diag is enabled.
crate::diag::dump_dvd_cells(ts.vts_number, title_number, dvd_title);
// Feature start cell. Prefer the DVD nav-VM resolver, which
// PARKED (#40, menu-at-start playback). The "menu at the start"
// symptom (e.g. SOTL) was a sector-mapping fault — the absolute
// VOB rebase in `ifo::parse_vts` (`vob_start_sector =
// file_start_lba + vtstt_vobs`) — NOT a navigation problem, so
// feature-start resolution is unnecessary for correct rips. The
// nav resolver + verified VM decoder (`dvdnav`) are kept compiled
// but deliberately bypassed; flip `USE_NAV_RESOLVER` to re-enable
// once the nav executor is finished. The fallback is the
// structural leading-cell filter (`feature_start_cell`), which
// drops leading scene-index / interleaved-angle sub-block cells
// and is a no-op for a normal feature (category 0x00 on cell 0).
// See `dvdnav::resolve_feature_start`.
// Feature-start nav resolution is PARKED (#40): the "menu at start" symptom
// was actually a sector-mapping fault in `ifo::parse_vts`'s VOB rebase, not
// navigation. Bypassed via `USE_NAV_RESOLVER = false`; fallback is the filter.
const USE_NAV_RESOLVER: bool = false;
let feature_start = if USE_NAV_RESOLVER {
crate::dvdnav::resolve_feature_start(
@@ -174,14 +183,9 @@ impl Disc {
let size_bytes: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
// Build pre-formatted VobSub `.idx` codec_data (size: + palette:
// lines) for VobSub subtitle streams. The `size:` line carries
// the coded video frame the subpicture was authored against
// (720x480 NTSC / 720x576 PAL) so players place and scale the
// bitmap correctly.
// format_palette guards on (0, 0) and omits its `size:` line,
// so an unresolved resolution degrades to a palette-only .idx
// rather than one claiming a 0x0 frame.
// Build pre-formatted VobSub `.idx` codec_data. The `size:` line carries the
// coded video frame (720x480 NTSC / 720x576 PAL) so players place/scale the
// subpicture correctly; format_palette omits it on unresolved (0,0).
let (vid_w, vid_h) = ts.video.resolution.pixels().unwrap_or((0, 0));
let codec_data = dvd_title
.palette
@@ -214,10 +218,9 @@ impl Disc {
streams.extend(audio_streams.iter().cloned());
streams.extend(subtitle_streams);
// Chapter times are absolute from the PGC start. When leading
// cells are dropped the muxed video shifts earlier by exactly
// their total duration, so shift the chapter marks too (clamping
// any that fell inside the dropped head to 0).
// Chapter times are absolute from the PGC start. When leading cells are
// dropped, the muxed video shifts earlier by their total duration, so shift
// chapter marks too (clamping any that fell inside the dropped head to 0).
let chapters: Vec<Chapter> = dvd_title
.chapter_times
.iter()
@@ -243,7 +246,13 @@ impl Disc {
}
}
titles
// Polled again AFTER the loop: a cancel raised during per-title-set IFO reads in
// `parse_vmg` has nothing left to poll otherwise, so without this a partially
// enumerated disc could still be handed back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok((titles, nav_feature))
}
}
@@ -253,11 +262,9 @@ mod tests {
use crate::sector::SectorSource;
use std::collections::HashMap;
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image (single physical partition,
// metadata_start == partition_start). Offsets cited against
// udf.rs::read_filesystem / ECMA-167.
// ---------------------------------------------------------------
const PART_START: u32 = 3000;
@@ -528,6 +535,118 @@ mod tests {
// Tests
// ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO at PART_START+6000, VTS_02's at PART_START+7000.
// Both ICBs sit far below, so filesystem metadata resolves fine and only the
// second title set's CONTENT read is cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|(ts, _)| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|(ts, _)| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test]
@@ -535,7 +654,12 @@ mod tests {
let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]);
assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0
.is_empty()
);
}
/// Single VTS, single title, one cell. Extent absolute LBA =
@@ -571,7 +695,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -597,7 +723,7 @@ mod tests {
/// before the title VOB. Cell `first_sector` values are relative to
/// `vtstt_vobs`; reading 0xC0 prepended the menu and shifted every extent
/// back by `vtstt_vobs - vtsm_vobs`, so the rip opened on the parental
/// prompt instead of the movie (Greenland NTSC R1: vtsm=44, vtstt=3640).
/// prompt instead of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
///
/// Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); we additionally stamp
/// a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve from 3640.
@@ -627,7 +753,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -647,7 +775,7 @@ mod tests {
);
}
/// ABSOLUTE-REBASE regression (THESILENCEOFTHELAMBS / Greenland fix):
/// ABSOLUTE-REBASE regression (real-disc menu-at-start fix):
/// `ifo::parse_vts` now sets `vob_start_sector = file_start_lba(IFO) +
/// vtstt_vobs`, so an extent's `start_lba` must equal the sum of THREE
/// independent terms — the IFO file's absolute on-disc LBA, the
@@ -685,7 +813,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -743,7 +873,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -784,7 +916,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let v = t
.streams
.iter()
@@ -839,7 +973,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let v = t
.streams
.iter()
@@ -869,12 +1005,9 @@ mod tests {
fn scan_dvd_titles_mixed_audio_codecs_distinct_pids() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// audio b0: coding_mode is (b0 >> 5) & 7. AC-3 = 0 → b0=0x00.
// DTS = 6 → b0 = 6<<5 = 0xC0. b1 channels nibble = (channels-1) in bits
// 2-0: a REAL 5.1 layout is 5 (0x05), a REAL 2.0 is 1 (0x01). The old
// fixture used 0x10/0x50 (both decode to 1 channel) — a placeholder that
// would pass even against code mishandling channel counts. Pin real
// layouts: AC-3 5.1 eng, DTS 2.0 fra.
// audio b0: coding_mode = (b0>>5)&7 (AC-3=0->0x00, DTS=6->0xC0). b1 channels nibble
// = (channels-1) in bits 2-0 (5.1=0x05, 2.0=0x01). Old fixture used 0x10/0x50
// (both -> 1ch), a placeholder that passed even with broken channel handling.
let vts = build_vts(
0,
0x00,
@@ -900,7 +1033,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -911,7 +1046,7 @@ mod tests {
.collect();
assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "en");
assert_eq!(audios[0].language, "eng");
assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -925,10 +1060,9 @@ mod tests {
2,
"DTS 2.0 nibble must decode to 2 channels"
);
// PIDs route via the positional sub-id table: AC-3 @ pos 0 → 0x80 →
// 0xBD80, DTS @ pos 1 → 0x89 → 0xBD89 (the shared audio-stream number
// in the low nibble, NOT a per-codec ordinal). Distinct AND the exact
// canonical wire PIDs the demux routes on.
// PIDs route via the positional sub-id table: AC-3 @ pos 0 -> 0x80 -> 0xBD80, DTS @
// pos 1 -> 0x89 -> 0xBD89 (shared audio-stream number in the low nibble, NOT a
// per-codec ordinal). Distinct AND the exact canonical wire PIDs the demux routes on.
assert_eq!(audios[0].pid, 0xBD80, "AC-3 @ pos 0 → 0xBD80");
assert_eq!(audios[1].pid, 0xBD89, "DTS @ pos 1 → 0xBD89");
assert_ne!(audios[0].pid, audios[1].pid);
@@ -970,7 +1104,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1027,7 +1163,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1040,7 +1178,7 @@ mod tests {
// Languages preserved in order.
assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["en", "fr", "de"]
vec!["eng", "fra", "deu"]
);
// PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1087,7 +1225,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let sub = t
.streams
.iter()
@@ -1097,7 +1237,7 @@ mod tests {
})
.expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "en");
assert_eq!(sub.language, "eng");
assert!(
sub.codec_data.is_some(),
"non-zero palette must yield codec_data"
@@ -1110,11 +1250,9 @@ mod tests {
#[test]
fn scan_dvd_titles_numbering_increments_per_title() {
let mut disc = MemDisc::new();
// Two titles in VTS 1 (title nums 1 and 2). num_pgcs must cover
// pgc_index = vts_title - 1, so we need >=2 PGC entries; our
// build_vts only emits 1 PGC. So the second title's PGC index (1)
// exceeds num_pgcs (1) and is skipped. To exercise numbering we use
// two separate VTS sets instead.
// Two titles in VTS 1 (nums 1,2): pgc_index = vts_title-1 needs >=2 PGC entries, but
// build_vts only emits 1, so the second title's PGC index (1) exceeds num_pgcs (1)
// and is skipped. Use two separate VTS sets instead to exercise numbering.
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
@@ -1141,7 +1279,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1154,6 +1294,79 @@ mod tests {
assert_eq!(titles[1].extents[0].start_lba, 10200); // 10000 + 200
}
/// Stamp a First-Play PGC into a VMG whose pre-command list is a single
/// unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
/// title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
/// TT_SRPT pointers `build_vmg` already writes.
fn stamp_first_play_jumptt(vmg: &mut [u8], ttn: u8) {
const FP_PGC_PTR: usize = 0x84; // u32 byte offset of the First-Play PGC
const CMD_TBL_PTR: usize = 0xE4; // u16 cmd-table offset (rel to PGC)
let fp_pgc: u32 = 0x200;
let cmd_tbl_rel: u16 = 0x100;
vmg[FP_PGC_PTR..FP_PGC_PTR + 4].copy_from_slice(&fp_pgc.to_be_bytes());
let pgc = fp_pgc as usize;
vmg[pgc + CMD_TBL_PTR..pgc + CMD_TBL_PTR + 2].copy_from_slice(&cmd_tbl_rel.to_be_bytes());
let tbl = pgc + cmd_tbl_rel as usize;
vmg[tbl..tbl + 2].copy_from_slice(&1u16.to_be_bytes()); // nr_of_pre = 1
// JumpTT command (type 1, direct=1, cmd=2): ttn in byte 5.
vmg[tbl + 8..tbl + 16].copy_from_slice(&[0x30, 0x02, 0, 0, 0, ttn, 0, 0]);
}
/// The DVD First-Play nav promotion branch (issue #40): when
/// `resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
/// must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
/// the matching `vts_number` — to the running `title_number`, and surface it
/// as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
/// the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
/// the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
/// join or a vtsn/vts_ttn field swap would promote the wrong title or none.
#[test]
fn scan_dvd_titles_nav_promotes_first_play_target() {
let mut disc = MemDisc::new();
// TT_SRPT: title 1 → VTS 1 (in-set title 1); title 2 → VTS 2 (in-set 1).
let mut vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
stamp_first_play_jumptt(&mut vmg, 2); // First-Play → JumpTT title 2
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
let (titles, nav_feature) = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// Sanity: the resolver reached the branch with the VTS-2 target.
assert_eq!(
crate::dvdnav::resolve_main_title(&mut disc, &udf),
Some(crate::dvdnav::nav::ResolvedTitle {
title: 2,
vtsn: 2,
vts_ttn: 1,
})
);
// The promotion mapped that target to the second scanned title.
assert_eq!(nav_feature, Some(2));
assert_eq!(titles[1].playlist_id, 2);
}
/// chapter_times from the IFO become Chapter entries with ordinal
/// names (dvd.rs maps chapter_times → Chapter{time_secs, chapter_name}).
#[test]
@@ -1178,7 +1391,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1233,9 +1448,8 @@ mod tests {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(2, 1, 1)]);
// Cell 0: leading scene-index/angle sub-block (cat 0x90), 5s, sectors 0..9.
// Cell 1: feature start (cat 0x00), 59s, sectors 100..199.
// Cell 2: feature (cat 0x00), 59s, sectors 300..399.
// Programs: prog0 → cell 1 (feature start), prog1 → cell 3.
// Cell 1: feature start (cat 0x00), 59s, 100..199. Cell 2: feature, 59s, 300..399.
// Programs: prog0 -> cell 1 (feature start), prog1 -> cell 3.
let vts = build_vts_cells(
1000,
0x00,
@@ -1263,7 +1477,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1287,7 +1503,7 @@ mod tests {
/// Conservative guard end-to-end: a normal feature (every cell category
/// 0x00) is muxed in full — the filter drops nothing and chapters are
/// unshifted. This is the "Silence of the Lambs" case.
/// unshifted. This is the plain-single-angle-feature case.
#[test]
fn scan_dvd_titles_plain_feature_untouched() {
let mut disc = MemDisc::new();
@@ -1315,7 +1531,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1358,7 +1576,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
+14 -19
View File
@@ -1,6 +1,6 @@
//! Physical AC-3 sub-stream probing for DVD audio routing.
//!
//! ## Why this exists (Silence-of-the-Lambs wrong-substream bug)
//! ## Why this exists (a real-disc wrong-substream bug)
//!
//! A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
//! scan assigns each declared stream a `private_stream_1` sub-stream id purely
@@ -8,7 +8,7 @@
//! `0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
//! physical sub-stream order on the wire matches the IFO declaration order.
//!
//! On some discs it does NOT. The R2 PAL "The Silence of the Lambs" feature
//! On some discs it does NOT. A real R2 PAL DVD release's feature
//! declares ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but
//! the physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
//! `0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
@@ -34,7 +34,7 @@ use std::collections::BTreeMap;
/// a DVD feature opens with logos/warnings whose audio is frequently a thin 2.0
/// bed on the FIRST sub-stream only — the other physical `0x8x` sub-streams and
/// the main 5.1 mix do not appear until a sector or two further in. 512 sectors
/// (1 MiB) was too short: on Greenland it saw ONLY `0x80`, and only its opening
/// (1 MiB) was too short: on one real disc it saw ONLY `0x80`, and only its opening
/// 2.0 frames. 1024 sectors (2 MiB) reliably contains at least one frame of
/// every physical AC-3 sub-stream AND enough of `0x80` to reach its 5.1 frames.
/// Still bounded so a live drive is never hammered (see the project "don't
@@ -54,7 +54,7 @@ const PROBE_SECTORS: u16 = 1024;
/// The first frame of a sub-stream at the head of a feature is NOT
/// representative. A DVD opens with logos/warnings, and the main `0x80`
/// sub-stream there frequently carries a thin 2.0 bed before transitioning to
/// its real 5.1 main mix a fraction of a second later (observed on Greenland:
/// its real 5.1 main mix a fraction of a second later (observed on a real disc:
/// `0x80`'s first frames are acmod=2 → 2 channels, then it becomes acmod=7+lfe →
/// 6 channels within the same 2 MiB window). Recording only the FIRST frame read
/// `0x80=2` and missed the 5.1 entirely, defeating the channel-match routing.
@@ -75,11 +75,9 @@ pub fn probe_ac3_substream_channels(ps_bytes: &[u8]) -> BTreeMap<u8, u8> {
if !(0x80..=0x87).contains(&sub) {
continue;
}
// The PS demux strips the 4-byte AC-3 sub-header but does not align to a
// frame. Walk EVERY 0x0B77 sync in this sub-stream's payload, decode
// each frame's channel count, and keep the largest — the sub-stream's
// real (main-mix) channel capability. See the doc comment above for why
// the first frame alone is unreliable.
// The PS demux strips the AC-3 sub-header but doesn't align to a frame, so
// walk every 0x0B77 sync in the payload and keep the largest decoded
// channel count — the sub-stream's real main-mix capability (see doc above).
if let Some(ch) = max_substream_channels(&p.data) {
let slot = found.entry(sub).or_insert(0);
*slot = (*slot).max(ch);
@@ -202,7 +200,7 @@ pub fn remap_audio_pids(streams: &mut [Stream], probed: &BTreeMap<u8, u8>) -> us
///
/// `reader` MUST yield PLAINTEXT VOB bytes (i.e. a `DecryptingSectorSource` on a
/// CSS disc) — probing scrambled sectors yields no AC-3 syncs and is a safe
/// no-op. Returns the number of audio streams whose PID changed.
/// no-op. Remaps `title`'s AC-3 streams in place; returns nothing.
pub fn probe_and_remap<S: SectorSource + ?Sized>(
reader: &mut S,
title: &mut crate::disc::DiscTitle,
@@ -347,7 +345,7 @@ mod tests {
assert_eq!(probed.get(&0x81), Some(&6), "0x81 is the 5.1 main mix");
}
/// GREENLAND regression — the probe must read each sub-stream's TRUE
/// Real-disc regression — the probe must read each sub-stream's TRUE
/// (max-mix) channel count, not be poisoned by an unrepresentative head
/// frame, and must NOT cross-contaminate between sub-streams.
///
@@ -500,12 +498,9 @@ mod tests {
#[test]
fn max_substream_channels_unmappable_size_steps_forward_by_two() {
let mut real = ac3_frame(2, false);
// Overwrite the (unchecked) CRC bytes of the real frame — these double
// as byte4/byte5 of the bogus header 2 bytes earlier, at absolute
// offset 4: byte4 = 0xC0 (fscod=3 reserved -> ac3_frame_size == 0,
// unmappable), byte5 = 0xF8 (bsid=31 >= 11 -> acmod_channels == None,
// so the bogus header itself never contributes a spurious channel
// count).
// Overwrite the real frame's (unchecked) CRC bytes, which double as byte4/5
// of the bogus header at offset 4: 0xC0 (fscod=3 -> ac3_frame_size == 0,
// unmappable) and 0xF8 (bsid=31 -> acmod_channels == None, no spurious count).
real[2] = 0xC0;
real[3] = 0xF8;
let mut data = vec![0xAA, 0xAA, 0xAA, 0xAA]; // offsets 0..4, no sync
@@ -590,7 +585,7 @@ mod tests {
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// End-to-end `probe_and_remap`: a real-disc-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
@@ -603,7 +598,7 @@ mod tests {
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
fn probe_and_remap_reroutes_swapped_substream_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
+52 -109
View File
@@ -102,11 +102,9 @@ impl AacsCertUnlocker<'_> {
.ok()
.and_then(|m| aacs::mkb::mkb_version(&m));
// Host certs are keysource-served, never compiled in — unioned from the
// explicit `DriveCredentials` and the key-source layer. With ZERO certs
// the cert route cannot run: NoHostCert (folded to AacsNoHostCert by the
// caller, preserving the graceful path-1 disc-hash → VUK fallback). This
// is detected here, where the MKB generation is still in hand.
// Host certs are keysource-served, never compiled in — unioned from
// `DriveCredentials` and the key-source layer. With ZERO certs, NoHostCert
// (folded to AacsNoHostCert) preserves the graceful disc-hash → VUK fallback.
let host_certs = Disc::collect_host_certs(self.opts, mkb_gen);
if host_certs.is_empty() {
tracing::warn!(
@@ -117,10 +115,9 @@ impl AacsCertUnlocker<'_> {
return Err(CertUnlockFailure::NoHostCert { mkb: mkb_gen });
}
// Hand the collected certs to the AACS unlocker. The cert-route bus
// removal depends on the read_data_key, NOT a drive unlock. The
// borrow checker can't split `session` across `scsi_mut()` + `&drive_id`
// through method calls, so clone the (cheap) identity first.
// Hand the collected certs to the AACS unlocker; the cert-route bus removal
// depends on read_data_key, NOT a drive unlock. Borrow checker can't split
// `session` across `scsi_mut()` + `&drive_id`, so clone the cheap identity.
let drive_id = session.drive_id.clone();
let fu_certs = crate::unlock_bridge::map_host_certs(&host_certs);
let (_, unlock_res) = crate::unlock_bridge::run_bus(
@@ -251,35 +248,13 @@ impl Disc {
/// read_data_key for bus decryption must still use the cert path, so an
/// unlocker with no OEM VID capability returns `None` and we fall through
/// to cert auth unchanged.
/// Collect every AACS host cert the caller carries, from BOTH the explicit
/// [`DriveCredentials`] and the key-source layer
/// ([`crate::KeySource::host_certs`] across each source), unioned. Host certs
/// are keysource-served, never compiled in; this is the one place the OEM
/// cert route gathers them. An empty result is the graceful no-cert signal
/// (the caller turns it into [`Error::AacsNoHostCert`]).
/// `mkb` is the disc's MKB generation when known, forwarded to each source's
/// [`crate::KeySource::host_certs`] so a source MAY return only
/// generation-appropriate certs (the default ignores it).
fn collect_host_certs(
opts: &ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::types::HostCert> {
// Delegates to the shared cert primitive (the external freemkv-unlock-aacs
// plugin uses the same one). Kept as a thin Disc method so the existing
// collect_host_certs_* unit tests and call sites are unchanged.
crate::aacs::host_certs::collect_host_certs(opts, mkb)
}
fn do_handshake_cert(
session: &mut crate::drive::Drive,
opts: &ScanOptions,
) -> (Option<HandshakeResult>, Option<Error>) {
// OEM VID shortcut: a matching unlocker stashed the disc's Volume ID at
// drive `init()` (the new `unlock()` folds in the old `read_volume_id`).
// Use it and SKIP the cert handshake — the OEM path
// decouples the VID from the host cert + HRL. It yields no
// `read_data_key`; a bus-encrypted disc that needs the bus key is caught
// by the bus-key gate in `resolve_vid_only`.
// drive `init()`; use it and SKIP the cert handshake. It yields no
// `read_data_key`, caught by the bus-key gate in `resolve_vid_only`.
if let Some(volume_id) = session.oem_vid() {
tracing::debug!(
target: "freemkv::disc",
@@ -307,12 +282,9 @@ impl Disc {
"No drive-unlocker Volume ID; running the in-tree AACS host-certificate handshake (AacsCertUnlocker)."
);
// Cert path: the in-tree `AacsCertUnlocker` peer absorbs the host-cert
// mutual-auth. It collects host certs from the key sources + credentials,
// runs `aacs_authenticate` per cert (wedge-guarded), and on success reads
// the VID + read_data_key. Its `UnlockError` is folded back to the same
// `Error` variants this function has always surfaced, so `scan_with`'s
// error rendering and the path-1 disc-hash → VUK fallback are unchanged.
// Cert path: `AacsCertUnlocker` collects host certs, runs
// `aacs_authenticate` per cert, and on success reads VID + read_data_key.
// Its `UnlockError` folds back to this function's usual `Error` variants.
let unlocker = AacsCertUnlocker { opts };
match unlocker.authenticate(session) {
Ok(hs) => (Some(hs), None),
@@ -328,6 +300,25 @@ impl Disc {
}
}
/// Collect every AACS host cert the caller carries, from BOTH the explicit
/// [`DriveCredentials`] and the key-source layer
/// ([`crate::KeySource::host_certs`] across each source), unioned. Host certs
/// are keysource-served, never compiled in; this is the one place the OEM
/// cert route gathers them. An empty result is the graceful no-cert signal
/// (the caller turns it into [`Error::AacsNoHostCert`]).
/// `mkb` is the disc's MKB generation when known, forwarded to each source's
/// [`crate::KeySource::host_certs`] so a source MAY return only
/// generation-appropriate certs (the default ignores it).
fn collect_host_certs(
opts: &ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::types::HostCert> {
// Delegates to the shared cert primitive (the external freemkv-unlock-aacs
// plugin uses the same one). Kept as a thin Disc method so the existing
// collect_host_certs_* unit tests and call sites are unchanged.
crate::aacs::host_certs::collect_host_certs(opts, mkb)
}
/// Build a keys-free AACS state that carries only the Volume ID (+ version
/// metadata), for callers that resolve Unit Keys out-of-band and have
/// disabled the local keydb. The VID is on-disc content read during the
@@ -355,36 +346,17 @@ impl Disc {
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let bus_encryption = cc.as_ref().map(|c| c.bus_encryption).unwrap_or(false);
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so the
// scanned `AacsState.version` and the out-of-band fetch agree. A wrong
// stride on the main resolve path fails loudly (sample validation) rather
// than silently, so the conservative V20 default is safe here too.
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so
// scanned/out-of-band versions agree; the main path fails loudly on a
// wrong stride, so this conservative default is safe here too.
let version = cc
.as_ref()
.map(|c| c.version.major())
.unwrap_or(aacs::mkb::AACS_MAJOR_UHD);
// Bus-encryption gate (wrong-keys guard). A bus-encrypted disc (Content
// Certificate bus-encryption bit set) carries bus encryption on its
// sectors, which MUST be removed before any AACS key can decrypt them.
// There are TWO ways it gets removed, and bus encryption is unremovable
// only when NEITHER succeeded:
// 1. An unlocker unlocked the drive → it serves CLEAR content
// (`drive_unlocked`). This is the common live-drive case and yields
// no `read_data_key` — it doesn't need one.
// 2. The AACS host-certificate cert-auth handshake produced the bus key
// (`read_data_key`).
// The old gate credited ONLY (2), so a SUCCESSFUL drive unlock (VID
// present, `read_data_key: None`, `drive_unlocked: true`) tripped it and
// blocked ALL key resolution — including the online source — even though
// the drive was serving clear content. That was the bug.
//
// Also skipped when `handshake = None` (file-backed/ISO scans — bus
// encryption already removed at read time) and when `bus_encryption` is
// false (AACS 1.0 BD is not bus-encrypted).
// ONE question — "is AACS bus encryption gone?" — asked of the single
// `bus_encryption_removed` predicate, which OWNS every case (never had it,
// file/ISO, drive unlock, cert bus key). The gate enumerates nothing.
// Bus-encryption gate: removed by a drive unlock (`drive_unlocked`) or a
// cert handshake `read_data_key`. The old gate credited only the latter,
// wrongly blocking key resolution after a successful drive unlock.
if !bus_encryption_removed(bus_encryption, handshake) {
let (rdk_err, has_vid) = handshake
.map(|h| (h.read_data_key_err, handshake_has_volume_id(h)))
@@ -400,20 +372,9 @@ impl Disc {
);
return Err(Error::AacsBusKeyUnavailable);
}
// Read the MKB record stream via the SAME bounded reader the
// out-of-band `read_aacs_inputs` uses (`read_mkb_content`: a prefix-grow
// read + trim), NOT a full `read_file`. MKB_RO/RW is allocated to a
// fixed ~128 MiB of zero padding, and a full `read_file` of it FAILS on
// file-backed / large readers — which left `a.mkb` empty here, silently
// breaking online key resolution: `Disc::inputs()` shipped `mkb=0` to
// the decode service and it 404'd, while autorip's separate
// `read_aacs_inputs` path (this same helper) worked. One reader now, so
// `Disc::inputs()` is the single complete source of AACS inputs.
// A read ERROR is surfaced (logged), not silently emptied: an empty MKB
// here is invisible until an online key service rejects the request, so
// a transient I/O hiccup must not masquerade as "no MKB". We still
// continue with an empty MKB (disc-hash-keyed keydb lookups don't need
// it), but the cause is now on the log.
// Use the SAME bounded reader as `read_aacs_inputs` (`read_mkb_content`),
// NOT `read_file` (fails on the ~128 MiB zero-padded MKB_RO/RW, silently
// emptying `a.mkb`). A read ERROR is logged; we still continue with an empty MKB.
let mkb_bytes = match Self::read_mkb_content(reader, udf_fs) {
Ok(m) => m,
Err(e) => {
@@ -484,11 +445,8 @@ mod tests {
);
}
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical
// partition (metadata_start == partition_start). Offsets cited
// against udf.rs::read_filesystem / ECMA-167.
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical partition
// (metadata_start == partition_start), per udf.rs::read_filesystem / ECMA-167.
const PART_START: u32 = 4000;
@@ -856,12 +814,9 @@ mod tests {
assert_eq!(st.read_data_key, Some(rdk));
}
// ---------------------------------------------------------------
// OEM bus-key gate: a bus-encrypted disc scanned on a LIVE drive
// (handshake present) with no read_data_key must HARD-ERROR
// (AacsBusKeyUnavailable) rather than silently yield garbage. The
// three non-regressing cases must still succeed.
// ---------------------------------------------------------------
// OEM bus-key gate: a bus-encrypted disc on a LIVE drive with no
// read_data_key must HARD-ERROR (AacsBusKeyUnavailable); three
// non-regressing cases must still succeed.
fn disc_with_cert(cert_type: u8, bus_encryption: bool) -> (MemDisc, udf::UdfFs) {
let mut disc = MemDisc::new();
@@ -965,8 +920,8 @@ mod tests {
assert_eq!(st.read_data_key, None);
}
/// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy
/// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`).
/// `handshake_has_volume_id` treats an all-zero Volume ID as absent and
/// any non-zero Volume ID as present.
#[test]
fn handshake_has_volume_id_reports_presence_not_absence() {
let with_vid = HandshakeResult {
@@ -1017,21 +972,12 @@ mod tests {
assert!(matches!(err, Error::AacsBusKeyUnavailable));
}
// ---------------------------------------------------------------
// Tests: read_vid_oem (response parsing). The OEM path issues a
// READ_BUFFER CDB and parses a 36-byte response; we can't easily
// fixture a real Drive, but the response-shape contract (3-byte
// signature 00 22 00, VID at [4..20]) is documented and worth a
// direct guard via a fake transport. Skipped here because Drive
// construction requires a live transport; the parsing branches are
// Tests: read_vid_oem (response parsing) are skipped here since Drive
// construction requires a live transport; parsing branches are
// exercised through `read_vid_oem`'s callers in integration.
// ---------------------------------------------------------------
// ---------------------------------------------------------------
// Tests: collect_host_certs — the OEM cert route's cert-gathering.
// Unions DriveCredentials with the key-source layer; empty means
// the route fails gracefully (AacsNoHostCert), never panics.
// ---------------------------------------------------------------
// Tests: collect_host_certs — unions DriveCredentials with the key-source
// layer; empty means the route fails gracefully (AacsNoHostCert).
fn fake_cert(tag: u8) -> aacs::types::HostCert {
aacs::types::HostCert {
@@ -1109,11 +1055,8 @@ mod tests {
assert_eq!(tags, vec![1, 2, 3]);
}
// ---------------------------------------------------------------
// AacsCertUnlocker outcome mapping: UnlockError → Error (preserving
// the legacy do_handshake_cert surface) and → UnlockOutcome (the
// structured trace step). No English in either.
// ---------------------------------------------------------------
// AacsCertUnlocker outcome mapping: UnlockError → Error (legacy surface)
// and → UnlockOutcome (structured trace step). No English in either.
#[test]
fn unlock_error_maps_to_legacy_error_variants() {
+360 -161
View File
@@ -107,8 +107,10 @@ struct PlannedFile {
size: u64,
/// Inline (ICB-embedded) data, if any. When `Some`, `extents` is empty.
inline: Option<Vec<u8>>,
/// Absolute disc extents `(abs_lba, byte_len)`.
extents: Vec<(u32, u32)>,
/// Absolute disc extents, each carrying whether it was ever RECORDED (an
/// ECMA-167 4/14.14.1.1 type-1 extent is allocated but not recorded: it
/// occupies the file's byte space and its contents are zeros).
extents: Vec<crate::udf::AbsExtent>,
}
impl Disc {
@@ -144,7 +146,7 @@ impl Disc {
let fs = udf::read_filesystem(reader)?;
let mut planned: Vec<PlannedFile> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
let mut seen_hosts: std::collections::HashMap<PathBuf, String> =
let mut seen_hosts: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
plan_tree(
reader,
@@ -188,28 +190,9 @@ impl Disc {
// disc-wide keys for every file.
let mut base_keys = self.decrypt_keys();
// AACS key map for the extract, chosen by CPS-unit count:
//
// * SINGLE CPS (the overwhelming majority, incl. every single-key UHD): one
// Unit Key opens EVERY encrypted unit on the disc — content in a parsed
// title AND an orphan clip that no playlist references. A blanket key-0
// map over the whole LBA space is exact and covers orphans; clear
// filesystem/nav (encrypted-flag off) passes through untouched.
//
// * MULTI-CPS: each clip is protected by a different Unit Key, so a blanket
// key-0 map would mis-decrypt every secondary-CPS file into garbage
// (silently, since Phase::All is trust-only). Build the EXACT per-CPS
// content map instead (each title's extents → the CPS key that opens a
// real sample from it), up front before the decorator takes the reader. A
// content unit whose key the pool lacks fails loud at resolve (extract has
// no CPS/forensic fetch source), never emits a wrong-key garble.
//
// KNOWN LIMITATION (by design): an orphan encrypted clip on a multi-CPS
// disc — referenced by no playlist, so in no title extent — is in no range
// and passes through as ciphertext. There is no correct key to apply (its
// CPS unit is unknown without a playlist reference), and blind trial-decrypt
// is exactly what this keymap-only model removes. Single-CPS is unaffected
// (the blanket key-0 map above covers orphans).
// AACS key map, by CPS-unit count. Single-CPS: one Unit Key opens
// everything, so a blanket key-0 map covers even orphan clips. Multi-CPS
// builds an exact per-title map instead (a blanket map would mis-decrypt).
let key_map =
match &base_keys {
DecryptKeys::Aacs { unit_keys, .. } if unit_keys.len() <= 1 => {
@@ -223,10 +206,9 @@ impl Disc {
_ => None,
};
// ── Phase 2: stream each file through the decrypting decorator ────
// The decorator owns its inner source for its lifetime. We hand it a
// borrowing wrapper (so the caller keeps `reader`), swap keys per CSS
// VTS group via `set_keys`; AACS/None keep `base_keys` throughout.
// Phase 2: stream each file through the decrypting decorator, which owns a
// borrowing wrapper so the caller keeps `reader`. Keys swap per CSS VTS
// group via `set_keys`; AACS/None keep `base_keys` throughout.
let mut dec = DecryptingSectorSource::new(Borrowed(reader), base_keys.clone());
if let Some(map) = key_map {
dec = dec.with_key_map(map);
@@ -235,6 +217,9 @@ impl Disc {
let mut result = ExtractResult::default();
let total_bytes = required;
let mut done_bytes: u64 = 0;
// Cumulative zero-filled-unreadable bytes, so the live progress channel
// can report the good/unreadable split instead of pinning unreadable at 0.
let mut done_unreadable: u64 = 0;
// CSS per-VTS key cache; only consulted for CSS discs.
let is_css = matches!(base_keys, DecryptKeys::Css { .. });
@@ -254,7 +239,7 @@ impl Disc {
let key = match vts_keys.get(&vts) {
Some(k) => k.clone(),
None => {
let k = self.resolve_vts_key(&vts, &planned, &mut dec, &base_keys);
let k = self.resolve_vts_key(&vts, &planned, &mut dec, &base_keys)?;
vts_keys.insert(vts.clone(), k.clone());
k
}
@@ -265,12 +250,17 @@ impl Disc {
}
}
// A unit that fails to decrypt fails the read loud (extract runs
// non-tolerate), so extract_one_file already zero-filled it and
// counted it in bytes_unreadable — one 'lost' bucket covers both
// media damage and decrypt failure.
let (fr, halted) =
extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?;
// A unit that fails to decrypt fails the read loud; extract_one_file
// zero-fills it into bytes_unreadable, the same bucket as media damage.
let (fr, halted) = extract_one_file(
&mut dec,
dest,
pf,
total_bytes,
&mut done_bytes,
&mut done_unreadable,
opts,
)?;
result.bytes_good = result.bytes_good.saturating_add(fr.bytes_good);
result.bytes_unreadable = result.bytes_unreadable.saturating_add(fr.bytes_unreadable);
@@ -287,82 +277,71 @@ impl Disc {
Ok(result)
}
/// Resolve the CSS title key for a VTS group by cracking from its title
/// VOB extents (keyless Stevenson attack). The disc-wide key is reused when
/// Resolve the CSS title key for a VTS group by recovering it from the
/// group's scrambled title-VOB sectors. The disc-wide key is reused when
/// it already covers this VTS (single-VTS discs, or this VTS's span). The
/// reader is borrowed from the decrypting decorator (its inner source).
///
/// FALLIBLE, because the three outcomes of a crack are not two.
/// [`crate::css::CrackOutcome`] exists precisely to separate "no scrambled
/// sector was seen, so there is nothing to decrypt" from "scrambled sectors
/// were seen and no key came out", and its doc says callers MUST surface
/// the second as a hard error. This function used the `Option`-returning
/// `crack_key`, which collapses both into `None`, and then fell back to the
/// disc-wide key — descrambling this VTS with ANOTHER VTS's title key. The
/// hazard is described in the ordering note below, which was written about
/// the same fallback; ordering makes the crack far more likely to succeed,
/// but it cannot make a failed crack safe.
fn resolve_vts_key<S: SectorSource>(
&self,
vts: &str,
planned: &[PlannedFile],
dec: &mut DecryptingSectorSource<S>,
base_keys: &DecryptKeys,
) -> DecryptKeys {
// Gather the title VOB extents for this VTS (VTS_xx_1.VOB .. _9.VOB;
// VTS_xx_0.VOB is the menu and is clear, so excluded from the crack).
// Gather this VTS's title VOBs BY NAME, ascending.
//
// `planned` comes from walking the UDF directory, which yields File
// Identifier Descriptors in on-disc order — authoring order, with no
// guarantee it matches playback order. Relying on it would leave the
// crack starting wherever the disc's directory happened to list first.
//
// DVD-Video numbers a title set's VOBs in playback order by spec
// (VTS_xx_1.VOB .. VTS_xx_9.VOB, single digit), so ascending name IS
// playback order and it is deterministic regardless of directory
// layout. That matters because ORDER decides correctness here: see the
// note below on the shared crack budget.
) -> Result<DecryptKeys> {
// Gather this VTS's title VOBs (VTS_xx_1..9.VOB, _0.VOB menu excluded) BY
// NAME, ascending. Directory order is authoring order not playback order,
// but DVD-Video numbers VOBs in playback order, so name-sort is correct.
let mut files: Vec<&PlannedFile> = planned
.iter()
.filter(|pf| {
vts_group_of(&pf.disc_name).as_deref() == Some(vts) && is_title_vob(&pf.disc_name)
})
.collect();
// Case-INSENSITIVE, matching `vts_group_of`/`is_title_vob`, which
// selected these files case-insensitively. A byte-wise sort put
// 'V' (0x56) before 'v' (0x76), so a set holding `vts_01_1.vob`
// alongside `VTS_01_2.VOB` — reachable on a case-sensitive volume —
// started the crack at part 2 and could exhaust the budget in a clear
// run, which is the whole failure this ordering exists to prevent.
// Case-INSENSITIVE, matching the selection above. A byte-wise sort could
// start the crack at the wrong part on a case-sensitive volume, exhausting
// the budget in a clear run — the exact failure this ordering prevents.
files.sort_by_key(|f| f.disc_name.to_ascii_uppercase());
let mut extents: Vec<crate::disc::Extent> = Vec::new();
for pf in files {
for &(abs_lba, byte_len) in &pf.extents {
// Unrecorded extents hold no bytes the VOB ever wrote and carry no
// scrambled sector, so feeding them in wastes the shared crack budget.
for ext in pf.extents.iter().filter(|e| e.recorded) {
extents.push(crate::disc::Extent {
start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
start_lba: ext.lba,
sector_count: (ext.len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
});
}
}
if extents.is_empty() {
return base_keys.clone();
return Ok(base_keys.clone());
}
// PLAYBACK ORDER — do NOT sort. This is the 1.5.1 garbage bug, and it
// grew back here in a new code path: the comment this replaced said it
// "matched the scan heuristic", but that heuristic WAS the bug and was
// already fixed in `Disc::decrypt_keys_for_title`, which documents the
// rule ("PLAYBACK ORDER, never largest-cell-first") and pins it with
// `decrypt_keys_for_title_scans_playback_order_not_largest_first`.
//
// Why order decides correctness: a CSS DVD's biggest cell opens with a
// long CLEAR run, and `crack_key`'s sector budget is shared across the
// whole extent list. Starting at the largest cell can exhaust the budget
// without ever MEETING a scrambled sector — and CSS recovers the title
// key from scrambled data itself, so the scan has to reach some. The
// crack then returns None, this function falls back to `base_keys`, and
// every VOB in the VTS is descrambled with the wrong key: corrupt PES
// behind an intact header, written out as a complete extract at exit 0.
//
// The files above are ordered by name and each file's own extents stay
// in file order, which together is playback order for a DVD title set.
// Crack against the raw (still-scrambled) inner reader, NOT the
// decrypting view — `crack_key` runs the descrambler itself.
match crate::css::crack_key(dec.inner_mut(), &extents, 64) {
Some(state) => DecryptKeys::Css {
// PLAYBACK ORDER — do NOT sort (the 1.5.1 garbage bug: see
// `Disc::decrypt_keys_for_title`'s "never largest-cell-first" rule).
// Crack against the raw inner reader, NOT the decrypting view.
match crate::css::crack_key_outcome(dec.inner_mut(), &extents, 64, None) {
crate::css::CrackOutcome::Cracked(state) => Ok(DecryptKeys::Css {
title_key: state.title_key,
},
None => base_keys.clone(),
}),
// No scrambled sector anywhere in this VTS: the content is clear,
// and any key descrambles it as a no-op. The disc-wide key is the
// right answer, and this is the ONLY case that ever was.
crate::css::CrackOutcome::Unencrypted => Ok(base_keys.clone()),
// Scrambled sectors WERE seen and no key came out. `CssKeyMissing`
// (unlike MUX, which skips on it) is `?`-propagated here, aborting
// the whole extract rather than reusing the wrong key silently.
crate::css::CrackOutcome::ScrambledUncracked => Err(Error::CssKeyMissing),
}
}
}
@@ -409,7 +388,7 @@ fn plan_tree(
is_root: bool,
files: &mut Vec<PlannedFile>,
dirs: &mut Vec<PathBuf>,
seen_hosts: &mut std::collections::HashMap<PathBuf, String>,
seen_hosts: &mut std::collections::HashMap<String, String>,
) -> Result<()> {
for entry in &dir.entries {
if entry.name.is_empty() {
@@ -427,13 +406,23 @@ fn plan_tree(
let safe = sanitize_component(&entry.name)?;
let child_rel = host_rel.join(&safe);
let child_disc = format!("{disc_path}/{}", entry.name);
// Collision: two distinct disc paths → same host path.
if let Some(prev) = seen_hosts.insert(child_rel.clone(), child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: child_rel.to_string_lossy().into_owned(),
});
// Collision: two distinct disc paths → same host FILE (case-insensitive
// fold of `Movie`/`movie`, or `.partial` sharing the final-name namespace
// so `X`/`X.partial` collide too). Fold is ASCII/simple-Unicode only.
let mut register = |key: PathBuf| -> Result<()> {
let folded = key.to_string_lossy().to_lowercase();
if let Some(prev) = seen_hosts.insert(folded, child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: key.to_string_lossy().into_owned(),
});
}
Ok(())
};
register(child_rel.clone())?;
if !entry.is_dir {
register(with_partial_suffix(&child_rel))?;
}
if entry.is_dir {
dirs.push(child_rel.clone());
@@ -479,6 +468,7 @@ fn extract_one_file<S: SectorSource>(
pf: &PlannedFile,
total_bytes: u64,
done_bytes: &mut u64,
done_unreadable: &mut u64,
opts: &ExtractOptions,
) -> Result<(FileResult, bool)> {
let final_path = dest.join(&pf.host_rel);
@@ -508,36 +498,58 @@ fn extract_one_file<S: SectorSource>(
finalize_file(writer, &partial_path, pf.size, &final_path)?;
fr.complete = true;
*done_bytes = done_bytes.saturating_add(pf.size);
report(opts, *done_bytes, total_bytes);
report(opts, *done_bytes, *done_unreadable, total_bytes);
return Ok((fr, false));
}
let mut written: u64 = 0;
let mut buf = vec![0u8; READ_BATCH_SECTORS as usize * SECTOR_BYTES];
'extents: for &(abs_lba, byte_len) in &pf.extents {
'extents: for &crate::udf::AbsExtent {
lba: abs_lba,
len: byte_len,
recorded,
} in &pf.extents
{
if written >= pf.size {
break;
}
// Anchor AACS unit alignment at THIS extent's start (clip-anchored
// gate, not absolute LBA 0 and NOT the file's first extent). A file
// may span multiple extents (fragmented / Long-AD / continuation ICB
// allocation); each extent's start LBA is arbitrary and generally not
// a multiple of 3 sectors from the previous extent. The decrypt-on-
// read gate (`is_unit_aligned(lba, unit_base)`) measures every read's
// LBA relative to `unit_base`, so the base MUST be re-anchored per
// extent — otherwise the first read of every later extent fails the
// gate and the whole extent is zero-filled as a (false) hole. Matches
// the per-extent re-anchoring in the mux read paths
// (`mux/disc.rs`, `sector/prefetched.rs`). No-op for CSS / None.
// ECMA-167 4/14.14.1.1 type 1: allocated but NOT recorded. Write zeros
// WITHOUT reading the media (mirrors `UdfFs::read_file_limited`) — skipping
// the extent entirely would slide every later extent's bytes down.
if !recorded {
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64);
let hole_bytes = (sectors * SECTOR_BYTES_U64).min(pf.size.saturating_sub(written));
let mut left = hole_bytes;
for b in buf.iter_mut() {
*b = 0;
}
while left > 0 {
let n = left.min(buf.len() as u64) as usize;
write_all(&mut writer, &buf[..n], &partial_path)?;
written = written.saturating_add(n as u64);
*done_bytes = done_bytes.saturating_add(n as u64);
left -= n as u64;
}
fr.bytes_good = fr.bytes_good.saturating_add(hole_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
if opts.cancelled(cont) {
return Ok((fr, true));
}
if written >= pf.size {
break 'extents;
}
continue;
}
// Anchor AACS unit alignment at THIS extent's start, not LBA 0 or the
// file's first extent: a fragmented extent's start LBA is arbitrary, so
// the gate must re-anchor per extent or a later read false-holes. No-op for CSS/None.
dec.set_unit_base(abs_lba);
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32;
let mut sector_off: u32 = 0;
while sector_off < sectors {
// AACS: read whole units (see `whole_unit_batch`). The tail batch may
// be a 1–2 sector partial unit, which `decrypt_sectors` handles via
// its trailing-partial contract: a clear partial stays clear (AACS
// leaves the final short unit unencrypted on disc), a scrambled
// partial fails loud as DecryptFailed.
// AACS: read whole units (see `whole_unit_batch`). A 1-2 sector tail
// partial is handled by `decrypt_sectors`'s trailing-partial contract:
// clear stays clear, scrambled fails loud as DecryptFailed.
let batch = whole_unit_batch(sectors - sector_off);
let lba = abs_lba + sector_off;
let want = batch as usize * SECTOR_BYTES;
@@ -557,14 +569,14 @@ fn extract_one_file<S: SectorSource>(
}
write_all(&mut writer, &buf[..usable], &partial_path)?;
fr.bytes_unreadable = fr.bytes_unreadable.saturating_add(usable as u64);
*done_unreadable = done_unreadable.saturating_add(usable as u64);
}
written = written.saturating_add(usable as u64);
*done_bytes = done_bytes.saturating_add(usable as u64);
let cont = report(opts, *done_bytes, total_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
sector_off += batch;
if opts.cancelled(cont) {
// Leave the `.partial`; do NOT rename. The aggregate run
// records the halt and this file stays incomplete.
// Leave the `.partial`; do NOT rename — this file stays incomplete.
return Ok((fr, true));
}
if written >= pf.size {
@@ -586,9 +598,8 @@ fn extract_one_file<S: SectorSource>(
/// [`READ_BATCH_SECTORS`] and rounded DOWN to a whole number of 3-sector units
/// UNLESS it is the extent's final (possibly short) tail — the tail always
/// begins on a unit boundary, so a 1–2 sector partial there is handled by
/// `decrypt_sectors`' trailing-partial contract. Shared by `extract_one_file`
/// (write) and `verify_one_clip` (dead-range) so this rounding rule lives in
/// exactly one place.
/// `decrypt_sectors`' trailing-partial contract. Used by `extract_one_file`
/// so this rounding rule lives in exactly one place.
fn whole_unit_batch(remaining: u32) -> u32 {
let mut batch = remaining.min(READ_BATCH_SECTORS);
if batch >= AACS_UNIT_SECTORS && batch < remaining {
@@ -650,11 +661,9 @@ fn finalize_file(
f.set_len(size).map_err(|e| Error::DirWriteFailed {
errno: e.raw_os_error(),
})?;
// `set_len` is a separate kernel op on this second handle (opened solely to
// truncate); the content fsync above was on the now-dropped writer handle.
// Without an fsync here, a crash between `set_len` and the rename can leave
// the file at its pre-truncation length — e.g. a sparse/over-read tail keeps
// its oversized form. Sync the new metadata (length) before publishing.
// `set_len` is a separate kernel op on this second handle; without an fsync
// here, a crash between `set_len` and the rename can leave the file at its
// pre-truncation length. Sync the new metadata (length) before publishing.
f.sync_all().map_err(|e| Error::DirWriteFailed {
errno: e.raw_os_error(),
})?;
@@ -673,15 +682,18 @@ fn finalize_file(
/// Emit a progress report. Returns `true` to continue, `false` if the sink
/// requested an early stop (or there is no sink — always continue).
fn report(opts: &ExtractOptions, done: u64, total: u64) -> bool {
fn report(opts: &ExtractOptions, done: u64, unreadable: u64, total: u64) -> bool {
match opts.progress {
Some(p) => {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Mux,
work_done: done,
work_total: total,
bytes_good_total: done,
bytes_unreadable_total: 0,
// `done` counts good AND unreadable bytes together; split them so a
// live-progress-only consumer sees a holed extraction as holed,
// not a clean climb to 100% (this used to pin unreadable at 0).
bytes_good_total: done.saturating_sub(unreadable),
bytes_unreadable_total: unreadable,
bytes_pending_total: 0,
bytes_retryable_total: 0,
bytes_total_disc: total,
@@ -715,10 +727,8 @@ fn available_space(dir: &Path) -> Option<u64> {
if rc != 0 {
return None;
}
// `statvfs` field integer widths differ by platform (e.g. macOS `u64`,
// Linux glibc `u64`/`c_ulong`); cast both to `u64` for the product. The
// cast is a no-op where the field is already `u64` — allow the lint so the
// same source stays portable.
// `statvfs` field integer widths differ by platform; cast both to `u64`
// for the product (no-op where already `u64` — allow lint for portability).
#[allow(clippy::unnecessary_cast, clippy::useless_conversion)]
let avail = (st.f_bavail as u64).saturating_mul(st.f_frsize as u64);
Some(avail)
@@ -818,13 +828,9 @@ fn sanitize_component(name: &str) -> Result<String> {
host: name.to_string(),
});
}
// Windows reserved device names (case-insensitive, base name before any
// extension). `NUL` silently discards all writes on Windows, so a disc file
// literally named `NUL.cfg` (legal on UDF/Linux) must NOT pass through
// verbatim — but it also must not abort the whole extraction. Substitute by
// prefixing `_`, which can never itself collide with another reserved name
// and is host-legal everywhere. The substituted name still flows through the
// caller's host-path collision check.
// Windows reserved device names (case-insensitive, base before extension).
// `NUL` silently discards writes on Windows, so `NUL.cfg` (legal on
// UDF/Linux) must not pass through verbatim; prefix `_` instead of aborting.
let base = trimmed.split('.').next().unwrap_or(trimmed);
if is_windows_reserved(base) {
return Ok(format!("_{trimmed}"));
@@ -888,14 +894,9 @@ mod tests {
use crate::css::lfsr;
use std::collections::HashMap;
// ── Self-contained in-memory UDF fixture toolkit ──────────────────────
// Modeled on the bluray.rs / dvd.rs test fixtures: a `MemDisc` SectorSource
// backed by an explicit absolute-LBA map, plus recursive UDF builders.
// PART_START == metadata_start so file/ICB LBAs share one address space
// (single partition map). Encrypted content is faked at the buffer level
// (CSS via `lfsr::scramble_sector`); AACS-content tests use clear bytes
// through `DecryptKeys::None`, since per-unit AACS decrypt is already
// exhaustively tested in `decrypt.rs` / `decrypting.rs`.
// ── Self-contained in-memory UDF fixture toolkit ──────────────────
// Modeled on the bluray.rs / dvd.rs fixtures: a `MemDisc` SectorSource
// backed by an absolute-LBA map; AACS tests use clear bytes.
const PART_START: u32 = 2000;
@@ -1165,6 +1166,17 @@ mod tests {
s
}
/// Build a file ICB whose FIRST short AD is an ECMA-167 4/14.14.1.1 type-1
/// (allocated, NOT recorded) extent and whose second is ordinary recorded
/// data. Both are `sectors_each` sectors long.
fn build_hole_then_data_icb(sectors_each: u32, hole_lba: u32, data_lba: u32) -> [u8; 2048] {
let mut s = build_two_extent_icb(sectors_each, hole_lba, data_lba);
let len = sectors_each * SECTOR_BYTES as u32;
// Re-stamp AD #0 with extent type 1 in bits 30..31 of the length field.
s[216..220].copy_from_slice(&(0x4000_0000u32 | (len & 0x3FFF_FFFF)).to_le_bytes());
s
}
/// Encrypt the clear unit from `clear_aacs_unit(tag)` under `unit_key` so
/// `aacs::content::decrypt_unit` recovers it cleanly (zero decrypt loss).
/// `tag` distinguishes two units' payloads.
@@ -1366,7 +1378,8 @@ mod tests {
}
/// A CSS-scrambled title VOB is descrambled on extraction: the producer
/// cracks the per-VTS key (keyless Stevenson) and the output VOB is plain.
/// recovers the per-VTS key from the scrambled sectors and the output VOB
/// is plain.
#[test]
fn css_title_vob_is_descrambled() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
@@ -1578,6 +1591,62 @@ mod tests {
assert!(matches!(err, Error::DirNameCollision { .. }));
}
/// Two disc names differing only by CASE are one host file on macOS APFS
/// and Windows NTFS, both case-insensitive by default. Keyed by the
/// case-preserving path, the collision map sees two entries, raises
/// nothing, and the second file extracted overwrites the first — while both
/// `PlannedFile`s report `complete: true`. Silent data loss reported as a
/// clean extract is the one outcome this crate must never produce, so the
/// host-equivalence key has to model the host's namespace, not the disc's.
#[test]
fn names_differing_only_by_case_are_a_collision() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("Movie", 30, 31, b"a".to_vec(), false),
file("movie", 32, 33, b"b".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("case_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err("two names that fold to one host file must collide");
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A file's in-flight `.partial` path shares the host namespace with every
/// other planned file's FINAL path. A disc carrying both `X` and
/// `X.partial` plans two distinct final names, so nothing collides — but
/// extracting `X` writes through `X.partial`, the same host path the other
/// file owns. Whichever lands second truncates the other, and both entries
/// are still reported complete.
#[test]
fn a_files_partial_path_colliding_with_another_files_final_name_is_an_error() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("X", 30, 31, b"aaaa".to_vec(), false),
file("X.partial", 32, 33, b"bbbb".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("partial_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err(
"X's temp path IS X.partial's final path — one host file for two \
disc files, so it must be refused up front",
);
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A non-empty target dir is refused without `--force`, and accepted with.
#[test]
fn non_empty_target_requires_force() {
@@ -1707,6 +1776,63 @@ mod tests {
);
}
/// An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED:
/// the space belongs to the file and occupies its byte range, but nothing
/// was ever written there, and the standard defines its contents as zeros.
/// `read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
/// `read_file_limited` honours it by emitting zeros WITHOUT touching the
/// media. The tree extractor reads the same ICBs and must agree: reading
/// those sectors returns whatever the media happens to hold there — on an
/// AACS disc, ciphertext that decrypts to noise — and writes it into the
/// extracted file as if the disc had recorded it.
///
/// The hole here is filled with a recognisable non-zero pattern, so
/// "emitted zeros" and "read the media" are told apart by CONTENT.
#[test]
fn extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it() {
const SECTORS_EACH: u32 = 1;
const HOLE: u32 = 5000;
const DATA: u32 = 5004;
let hole_bytes = vec![0xEEu8; SECTOR_BYTES];
let data_bytes = vec![0x5Au8; SECTOR_BYTES];
// The file's byte space: the hole's zeros FIRST, then the real data.
let mut expect = vec![0u8; SECTOR_BYTES];
expect.extend_from_slice(&data_bytes);
let mut disc = MemDisc::new();
build_udf_skeleton(&mut disc, 10);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "INDEX.BDMV", 42, false, false);
disc.put(
PART_START + 42,
build_hole_then_data_icb(SECTORS_EACH, HOLE, DATA),
);
disc.put_bytes(PART_START + HOLE, &hole_bytes);
disc.put_bytes(PART_START + DATA, &data_bytes);
disc.put(PART_START + 10, build_dir_icb(11, root_fids.len() as u32));
disc.put_bytes(PART_START + 11, &root_fids);
let out = TmpDir::new("unrecorded_extent");
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract");
let got = read_out(out.path(), "INDEX.BDMV").expect("file written");
assert_eq!(
got, expect,
"an unrecorded extent contributes zeros to the file, never the \
bytes that happen to sit on those sectors"
);
assert_eq!(
res.bytes_unreadable, 0,
"a hole is not a read failure — nothing was attempted"
);
assert!(res.complete);
}
/// Focused alignment-computation check underpinning the per-extent fix:
/// when each extent anchors its OWN start as the unit base, the extent's
/// own batch starts are always unit-aligned; anchoring a later extent
@@ -1841,12 +1967,9 @@ mod tests {
fn scrambled_vob(title_key: [u8; 5], marker: u8) -> (Vec<u8>, Vec<u8>) {
let seed = [0x11u8, 0x22, 0x33, 0x44, marker];
let mut plain = vec![0u8; 2048];
// The crack scan's hardened `is_scrambled_pack` gate requires the
// MPEG-PS pack-start signature before it will even ATTEMPT a
// Stevenson crack (see `css::is_scrambled_pack`) -- without it,
// `resolve_vts_key` silently falls back to `base_keys` for BOTH
// VTS groups regardless of which extents were gathered, masking
// this exact regression.
// The crack scan's `is_scrambled_pack` gate requires the MPEG-PS pack-
// start signature before attempting a crack; without it, `resolve_vts_key`
// falls back to `base_keys` for both groups, masking this regression.
plain[0x00..0x04].copy_from_slice(&crate::css::PACK_START);
// Pack start code: a real scrambled sector is an MPEG-2 PS pack, and
// the descrambler requires it before trusting byte 0x14.
@@ -1889,10 +2012,8 @@ mod tests {
let out = TmpDir::new("css_two_vts");
let mut d = clear_disc();
d.content_format = crate::disc::ContentFormat::MpegPs;
// Disc-wide key deliberately matches NEITHER VTS's real key: if the
// per-VTS group filter is broken and a crack attempt fails (or is
// skipped), falling back to this key must still mismatch, so the
// fallback path can never accidentally mask a broken filter.
// Disc-wide key deliberately matches NEITHER VTS's real key, so a
// broken filter falling back to it can never accidentally mask itself.
d.css = Some(crate::css::CssState {
title_key: [0xFFu8; 5],
crack_span: None,
@@ -1918,6 +2039,86 @@ mod tests {
assert!(res.complete);
}
/// A VTS that IS scrambled but whose key could not be recovered must
/// FAIL, not borrow another VTS's key.
///
/// `crack_key` returns `Option`, which collapses "no scrambled sector was
/// seen" with "scrambled sectors were seen and no key came out"; the
/// fallback then descrambled this VTS under the disc-wide key and
/// `extract_tree` reported `complete = true` at exit 0, because no read had
/// failed. `CrackOutcome` exists to keep those two apart, and every sibling
/// path in the crate already honours it.
///
/// The fixture: VTS_01 is crackable, VTS_02 is scrambled with NO periodic
/// crib, so its crack genuinely fails.
#[test]
fn a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key() {
let key_1 = [0x10u8, 0x20, 0x30, 0x40, 0x50];
let (_plain_1, scrambled_1) = {
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x01];
let mut plain = vec![0u8; 2048];
plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plain[0x14] = 0x10;
let pat: Vec<u8> = (0..8)
.map(|k| (0xA0u8.wrapping_add(k as u8) ^ 0x01) ^ 0x5A)
.collect();
for (i, b) in plain.iter_mut().enumerate().skip(0x59) {
*b = pat[i % 8];
}
plain[0x54..0x59].copy_from_slice(&seed);
let mut scrambled = plain.clone();
lfsr::scramble_sector(&key_1, &mut scrambled);
(plain, scrambled)
};
// Scrambled — pack header and 0x14 flag set, so the scan SEES ciphertext —
// but the cleartext region has no periodic run, so no key is recoverable.
let uncrackable = {
let mut sect = vec![0u8; 2048];
sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sect[0x14] = 0x10;
for (i, b) in sect.iter_mut().enumerate().skip(0x59) {
// Non-repeating, so no run of any period survives to 0x80.
*b = (i as u8).wrapping_mul(37).wrapping_add(11);
}
sect
};
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "VIDEO_TS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTS_01_1.VOB", 30, 5000, scrambled_1, false),
file("VTS_02_1.VOB", 32, 6000, uncrackable, false),
],
subdirs: vec![],
}],
};
let mut disc = build_disc(root);
let out = TmpDir::new("css_uncrackable_vts");
let mut d = clear_disc();
d.content_format = crate::disc::ContentFormat::MpegPs;
d.css = Some(crate::css::CssState {
title_key: [0xFFu8; 5],
crack_span: None,
});
let err = d
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err(
"a VTS whose key could not be recovered must be a hard error, \
not a silent extract under another VTS's key",
);
assert!(
matches!(err, Error::CssKeyMissing),
"expected CssKeyMissing, got {err:?}"
);
}
/// `Borrowed` is a thin forwarding wrapper the decrypting decorator uses
/// to avoid taking ownership of the caller's reader -- every
/// `SectorSource` method must forward to the wrapped `&mut dyn
@@ -1987,10 +2188,8 @@ mod tests {
// with sector_off > 0.
const SECTORS_A: u32 = 1600;
const LBA_A: u32 = 5000;
// The disc region immediately following extent A's true end. Must
// NEVER be read as part of extent A: sized to cover a full erroneous
// READ_BATCH_SECTORS second batch starting right after extent A's
// real tail.
// The disc region immediately following extent A's true end. Must NEVER
// be read as part of extent A: sized to cover a full erroneous second batch.
const LBA_FILLER: u32 = LBA_A + SECTORS_A;
const SECTORS_FILLER: u32 = 1472;
// Extent B: the file's real second extent, at a completely different
+1719 -112
View File
File diff suppressed because it is too large Load Diff
+1140 -401
View File
File diff suppressed because it is too large Load Diff
+90 -208
View File
@@ -192,11 +192,9 @@ fn plan_windows(sector_count: u32, share: u32) -> Vec<SampleWindow> {
(0..windows)
.map(|i| SampleWindow {
offset: if windows == 1 {
// A title cut into many clips gives each extent a share worth one
// window. Putting that window at the extent's head samples every
// clip at the same relative position — and for the FIRST clip
// that position is the start of the feature, the one stretch a
// film reliably has no subtitles in. Take the middle instead.
// A single window per extent placed at the head would sample the
// first clip's start — the one span a film reliably lacks subtitles.
// Use the middle instead.
align_down(span / 2)
} else {
// u64: `span * i` overflows u32 for a large extent.
@@ -267,8 +265,8 @@ impl TrackEvidence {
/// The coverage is the whole point. Evidence from a sampled (or budget-cut) read
/// describes the sectors that were actually fed to the demuxer, not the extent —
/// memoising it under the extent's full key and replaying it to another playlist
/// asserts a completeness the read never had. Recording `covered` and `complete`
/// lets a later run decide whether the entry answers ITS question or whether the
/// asserts a completeness the read never had. Recording `covered` alongside the
/// evidence's own `sampled` flag lets a later run decide whether the entry answers ITS question or whether the
/// extent has to be read again.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub(crate) struct CachedEvidence {
@@ -296,7 +294,7 @@ impl CachedEvidence {
///
/// Many playlists on one disc reference the same clips (main feature, play-all,
/// seamless-branch variants) but rarely with byte-identical extent LISTS: 00800
/// = [A, B], 00801 = [A], 00802 = [B] are three different lists over two clips.
/// = `[A, B]`, 00801 = `[A]`, 00802 = `[B]` are three different lists over two clips.
/// Keying on the whole list de-duplicated only exactly-identical playlists and
/// re-read every shared clip once per list — up to `PROBE_BUDGET_SECTORS`
/// (256 MiB) of optical-drive time each. Per-extent keying reads each physical
@@ -398,11 +396,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
})
.collect();
// Budget allocation: each extent gets a share of the sector budget in
// proportion to its size, computed over the title's WHOLE extent list rather
// than over the extents that happen to be uncached. That keeps an extent's
// sampling plan — and therefore the coverage its cache entry claims — a
// function of the title alone, not of cache state or probe order.
// Each extent's share of the sector budget is proportional to its size,
// computed over the title's whole extent list (not just uncached ones), so the
// sampling plan is a function of the title alone, not cache state or order.
let total_sectors: u64 = title
.extents
.iter()
@@ -439,10 +435,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
slot.merge(hit.evidence);
}
}
// No entry, or one whose coverage does not support the claim this
// run needs from it. (A playlist that declares a PGS PID a
// previous playlist did not also lands here, so the extra track is
// genuinely probed.)
// No entry, or one whose coverage doesn't support this run's claim —
// includes a PGS PID a previous playlist didn't declare, so it's
// genuinely probed.
_ => fresh.push(pid),
}
}
@@ -471,10 +466,8 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
.collect()
}
// Whether ANY declared track could still have its outcome changed by reading
// more. This is the per-track early exit: the moment a track is disproven
// (and there is nothing left for its evidence to correct) it stops asking for
// budget, and when no track is asking, the run stops.
// Per-track early exit: once a track is disproven with nothing left to
// correct, it stops asking for budget; when no track is asking, the run stops.
let anything_left_to_learn = |live: &[(u16, TrackEvidence)]| -> bool {
let disc_uses_forced_flag = live.iter().any(|(_, e)| e.forced_seen);
let busiest = live.iter().map(|(_, e)| e.displays).max().unwrap_or(0);
@@ -488,19 +481,15 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
if !vendor_forced.get(pid).copied().unwrap_or(false) {
return false;
}
// Disproven but labelled forced: keep reading only while the evidence
// a demotion needs is still incomplete (see `demotable`) — otherwise
// the run would stop one window short of being ALLOWED to act on what
// it has already seen.
// Disproven but labelled forced: keep reading only while the evidence a
// demotion needs (see `demotable`) is still incomplete.
!crate::mux::codec::pgs::demotable(e.facts(), disc_uses_forced_flag, busiest)
})
};
if todo.is_empty() {
// Every extent's evidence came from a run that reached a designed stop and
// covered at least as much as this run planned to, so an absence claim
// over the composed evidence is as sound as the run that produced each
// part.
// Every extent's evidence reached a designed stop and covered at least as
// much as planned, so an absence claim over it is as sound as its source.
apply_verdicts(title, &verdicts(&evidence, true));
return;
}
@@ -512,10 +501,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
// asserted from what was observed.
let mut stop = StopReason::Exhausted;
'outer: for (ext, fresh_pids) in &todo {
// Trackers are PER EXTENT and only for the tracks this extent still owes
// evidence for — a track the cache already answered is not demuxed again,
// so no observation is ever counted twice, and budget is not spent on
// tracks that have nothing left to say.
// Trackers are per-extent, for tracks this extent still owes evidence for;
// an already-answered track isn't demuxed again, so no observation is
// double-counted and no budget is spent on tracks with nothing left to say.
let mut trackers: HashMap<u16, ForcedTracker> = fresh_pids
.iter()
.map(|&p| (p, ForcedTracker::new()))
@@ -531,21 +519,16 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
// Sectors of this extent actually fed to the demuxer — the coverage the
// memo will claim, never more.
let mut covered: u32 = 0;
// AACS aligned units are anchored at THIS extent's start LBA, so tell a
// decrypt-on-read source to gate relative to it rather than absolute disc
// LBA 0 — without this the very first read of a clip whose start_lba is
// not itself 3-aligned is rejected. Mirrors the mux read paths. Every
// window offset is a whole number of units from that base, so sampling
// does not disturb the gate.
// AACS units are anchored at this extent's start LBA, so gate a decrypt-
// on-read source relative to it (not disc LBA 0), or the first read of a
// non-3-aligned clip is rejected. Mirrors the mux read paths.
reader.set_unit_base(ext.start_lba);
// `None` = the extent's whole plan ran. `Some(reason)` = it stopped early.
let mut cut_short: Option<StopReason> = None;
for window in &plan {
// Demux/parse state is PER WINDOW, because a window is a
// DISCONTIGUOUS run of the clip: carrying a demuxer across the gap
// would splice two unrelated byte runs into one PES. The cost is at
// most a display set straddling a window edge — the same trade the
// per-extent reset already made at extent edges.
// Demux/parse state is per-window: a window is a discontiguous run of
// the clip, so carrying a demuxer across the gap would splice unrelated
// byte runs into one PES (same trade as the per-extent reset).
let mut demux = TsDemuxer::new(fresh_pids);
let mut parsers: HashMap<u16, PgsParser> =
fresh_pids.iter().map(|&p| (p, PgsParser::new())).collect();
@@ -585,58 +568,22 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
break;
}
};
// Advance by what was actually READ, not by what was requested. A
// short-but-nonzero read (a source whose batch is smaller than the
// request — `PrefetchedSectorSource` returns its producer's batch)
// used to advance `lba`/`remaining`/`sectors_read` by the full
// `count`, silently SKIPPING the unread tail of the chunk while
// `stop` stayed `Exhausted` — so an absence-based forced verdict was
// asserted (and memoised) over data that was never seen.
// Advance by what was actually READ, not requested: a short-but-
// nonzero read (e.g. `PrefetchedSectorSource`) used to advance by
// the full `count`, silently skipping the unread tail as `Exhausted`.
let served = (n.min(want) / SECTOR_BYTES) as u32;
// ...but advancing by the raw sector count breaks the OTHER invariant
// this loop rests on: with a decrypting source every read must BEGIN a
// whole number of AACS aligned units past `ext.start_lba`, which is why
// `CHUNK_SECTORS` is a multiple of `ALIGNED_UNIT_SECTORS` (see the
// const-assert above). A short read of, say, 64 sectors is not a whole
// number of units, so the next `lba` would be off the unit grid,
// `DecryptingSectorSource` would reject it with `DecryptFailed` before
// reading, and the run would end `ReadFailed` — inconclusive, nothing
// memoised, forced detection silently back to the vendor label on
// exactly the encrypted discs the aligned chunk size was chosen for.
//
// So a read that did NOT satisfy the whole request advances only by
// whole aligned units, and the residue sectors are simply RE-READ from
// the next unit boundary on the following pass: at most two sectors of
// duplicated drive work, no gap, no double-feed of any byte to the
// demuxer (only the aligned prefix is fed), and no trailing partial
// unit — whose plaintext a unit-anchored decrypt cannot produce anyway
// — is ever handed to the parsers.
//
// A read that satisfied the whole request advances by all of it even
// when `count` itself was not unit-aligned: `count` is only ever below
// `CHUNK_SECTORS` on the extent's final chunk or at the sector budget,
// and both end the loop before another read of this extent.
// Advancing by raw sector count would break unit-alignment (every
// read must begin on an AACS aligned unit), so short reads advance only
// by whole units; the residue sectors are simply re-read next pass.
let got = if served >= u32::from(count) {
u32::from(count)
} else {
served - served % crate::aacs::content::ALIGNED_UNIT_SECTORS
};
if got == 0 {
// Less than one whole aligned unit came back, so the read position
// cannot move: the next aligned boundary IS the one just read. The
// bytes are real, so feed them (never lose an observation), then
// RETRY the same aligned lba — a source that short-changed one call
// commonly satisfies the next, and only when it repeatedly cannot
// yield a whole unit is the run the truncated prefix an error is.
// Bounded retries are what keep a source that never yields a unit
// (including one that claims sectors and returns none) from
// spinning here for ever. A retry that re-serves the same bytes
// feeds them twice: the BOOLEAN evidence is monotone (observed /
// saw-a-non-forced-set / saw-a-forced-set), so a repeat cannot
// change it, and the display COUNT can over-count by at most
// [`STALL_RETRY_LIMIT`] repeats of a sub-unit (< 6 KB) read —
// too little to move the shape comparison the demotion gate
// makes, which is against the busiest track on the disc.
// Less than one aligned unit came back: feed the real bytes (never
// lose an observation), then retry the same lba, bounded by
// [`STALL_RETRY_LIMIT`] since boolean evidence is monotone to repeats.
for pes in demux.feed(&buf[..n.min(want)]) {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
@@ -670,10 +617,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
}
}
}
// Saturating: a malformed extent can put the window's last chunk at
// the top of the 32-bit LBA space, and `remaining` has already
// reached 0 by then, so pinning the position is harmless — wrapping
// (or panicking in a debug build) is not.
// Saturating: a malformed extent can put the last chunk at the top of
// the 32-bit LBA space; `remaining` is already 0 by then, so pinning is
// harmless — wrapping (or a debug-build panic) is not.
lba = lba.saturating_add(got);
remaining -= got;
sectors_read += got;
@@ -687,11 +633,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
}
}
// Drain the window's tail: the demuxer holds the last PES open until
// the next PUSI arrives, and with a sampled read that PUSI is in a
// different window (or nowhere), so without this the last display set
// of EVERY window is thrown away — 16 lost observations per extent,
// exactly where the sample is thinnest.
// Drain the window's tail: the demuxer holds the last PES open until the
// next PUSI, which a sampled read may put in another window (or nowhere),
// so without this every window loses its last display set.
for pes in demux.flush() {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
@@ -714,22 +658,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
}
}
// Fold this extent's evidence in, and memoise it if the read reached a
// DESIGNED stop — the whole plan ran, the sector budget was hit, or every
// track had already settled. The budget is a designed stop for exactly the
// reason [`StopReason`] documents, and it is the stop that fires on every
// disc that HAS a forced track — excluding it from the cache would mean
// nothing is ever memoised on precisely those discs.
//
// What makes THAT sound is the coverage stored alongside: the entry says
// how much of the extent was actually read, so a later playlist that
// intended to read more of it will re-read instead of inheriting an
// absence claim from a prefix (see [`CachedEvidence::answers`]).
//
// A halt or a read fault is different: the cut-off point is arbitrary, so
// its evidence is real for THIS title (nothing observed is retracted) but
// must not be frozen in as the extent's answer, or one transient fault
// would be replayed onto every other playlist sharing the clip.
// Fold in this extent's evidence; memoise only on a DESIGNED stop (plan
// done, budget hit, or all tracks settled), with coverage stored so a later
// playlist re-reads rather than inherits a halt/fault's arbitrary cutoff.
let cacheable = cut_short.is_none_or(StopReason::absence_is_conclusive);
let sampled = !(complete_plan && cut_short.is_none());
for (&pid, t) in trackers.iter() {
@@ -742,11 +673,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
evidence: ev,
covered,
};
// Never replace a richer memo with a thinner one: an extent
// re-read under a smaller share would otherwise DOWNGRADE what is
// known about it. Facts merge (they are monotone), and the
// coverage claimed is the larger of the two — conservative, since
// the two samples together cover at least that much.
// Never replace a richer memo with a thinner one (a re-read under a
// smaller share would downgrade it): facts merge monotonically, and
// the coverage claimed is the larger of the two, which is conservative.
cache
.entry((ext.start_lba, ext.sector_count, pid))
.and_modify(|prev| {
@@ -800,7 +729,7 @@ fn tracker_evidence(t: &ForcedTracker, sampled: bool) -> TrackEvidence {
/// Compose the per-track verdicts a run is ENTITLED to assert from the evidence
/// it gathered. A track absent from the result keeps its vendor-derived flag.
///
/// Three gates, all PER TRACK, because the evidence is per track:
/// Four gates, all PER TRACK, because the evidence is per track:
/// * `observed` — saw no display set at all, so nothing is known. (Never assert
/// "not forced" from having seen nothing.)
/// * on a truncated run, `non_forced` — the track saw an actual non-forced
@@ -814,6 +743,9 @@ fn tracker_evidence(t: &ForcedTracker, sampled: bool) -> TrackEvidence {
/// that flag. Discs like that exist (measured: not one track on the disc
/// carries it), and without this gate the probe would strip the correct
/// forced label off every track on every one of them.
/// * for the FORCED verdict, a minimum display-set count (`PROMOTE_MIN_DISPLAY_SETS`)
/// unless the read was complete — over a SAMPLE of a mixed track, one flagged
/// set alone is not enough to promote the whole track to forced.
fn verdicts(evidence: &HashMap<u16, TrackEvidence>, conclusive: bool) -> HashMap<u16, bool> {
// Disc-level facts the demotion gate rests on, over the tracks judged
// together: does the authoring house set the flag at all, and how busy is the
@@ -832,13 +764,9 @@ fn verdicts(evidence: &HashMap<u16, TrackEvidence>, conclusive: bool) -> HashMap
busiest,
);
}
// Calling a track FORCED is also an absence claim — "no display set
// here was un-flagged" — and over a SAMPLE of a track that mixes
// flagged and un-flagged sets (measured: such tracks exist, flagging
// a quarter of their sets), catching one flagged set and nothing else
// is exactly what a wrong promotion looks like. A read that covered
// every extent end to end has no such gap and may promote off a
// single set (measured: single-sign forced tracks exist too).
// Calling a track FORCED is also an absence claim ("no set here was
// un-flagged"); over a SAMPLE of a mixed track, one flagged set alone
// is a wrong promotion, so require a minimum unless the read was complete.
!e.sampled || e.displays >= PROMOTE_MIN_DISPLAY_SETS
})
.map(|(&pid, e)| (pid, !e.non_forced))
@@ -854,18 +782,9 @@ fn apply_verdicts(title: &mut DiscTitle, verdicts: &HashMap<u16, bool>) {
&& let Some(&forced) = verdicts.get(&sub.pid)
{
sub.forced = forced;
// A demoted track must not go on describing itself as forced. The
// flag and the qualifier are two renderings of one fact for
// different consumers — the muxer writes `FlagForced` from
// `forced`, the JSON sidecar writes its qualifier string from
// `qualifier` — so leaving `Forced` behind here published a track
// whose sidecar said "forced" and whose Matroska header said it was
// not. The probe read the content; it outranks the vendor's claim.
//
// Only this direction is a contradiction. `qualifier == None` on a
// track the probe promoted is not one: `None` is the absence of an
// editorial qualifier, not an assertion that the track is ordinary,
// and the vendor never claimed otherwise.
// A demoted track must not go on describing itself as forced: flag and
// qualifier render one fact for different consumers, so leaving `Forced`
// behind contradicts the header. The probe outranks the vendor's claim.
if !forced && sub.qualifier == crate::disc::LabelQualifier::Forced {
sub.qualifier = crate::disc::LabelQualifier::None;
}
@@ -969,11 +888,9 @@ mod tests {
"the probe must never read past the budget, got {}",
reader.served
);
// ...and must actually SPEND it: a plan that quietly reads a fraction of
// the budget would look "safe" while observing even less than the
// head-first read it replaced. One window's slack is allowed (windows are
// whole numbers of aligned units, so 16 of them need not divide the
// budget exactly).
// ...and must actually SPEND it: quietly reading a fraction would look
// "safe" while observing less than the head-first read it replaced.
// One window's slack is allowed (windows need not divide the budget exactly).
assert!(
reader.served + WINDOW_SECTORS > PROBE_BUDGET_SECTORS,
"the probe must spend the budget it is given, got {}",
@@ -1096,10 +1013,9 @@ mod tests {
}
}
// PGS PCS layout (matches the private constants in mux::codec::pgs): a
// display-set frame begins with a PCS (segment type 0x16); byte 13 is
// number_of_composition_objects; byte 17 is the first object's flags, whose
// 0x40 bit is forced_on_flag.
// PGS PCS layout (matches mux::codec::pgs): a display-set frame begins with a
// PCS (segment type 0x16); byte 13 is number_of_composition_objects; byte 17
// is the first object's flags, whose 0x40 bit is forced_on_flag.
const PCS_SEG: u8 = 0x16;
const PCS_NUM_OBJECTS_OFF: usize = 13;
const PCS_FLAGS_OFF: usize = 17;
@@ -1172,10 +1088,8 @@ mod tests {
pos: 0,
};
let mut title = pgs_title(pid, false); // vendor label says NOT forced
// One sector, which is exactly what the reader serves: an extent that
// claims more sectors than the source yields is a SHORT read, and a short
// read is (correctly) inconclusive — see
// `read_error_after_partial_content_preserves_vendor_forced`.
// One sector, exactly what the reader serves: an extent claiming more
// sectors than the source yields is a SHORT read, correctly inconclusive.
title.extents = vec![Extent {
start_lba: 0,
sector_count: 1,
@@ -1193,9 +1107,8 @@ mod tests {
#[test]
fn nonforced_display_sets_clear_a_not_yet_forced_track() {
// A non-forced display set observed on the wire settles the track as
// not-forced. The vendor label here already agrees, so this pins the
// verdict itself (the track must not come back FORCED off one non-forced
// set) without engaging the demotion guard — that gets its own tests.
// not-forced; this pins the verdict itself (must not come back FORCED off
// one non-forced set) without engaging the demotion guard.
let pid = 0x1200u16;
let mut reader = TsReader {
data: ts_stream(pid, &pcs_display(false)),
@@ -1288,10 +1201,9 @@ mod tests {
#[test]
fn read_error_after_partial_content_preserves_vendor_forced() {
// The defect: a forced verdict rests on the ABSENCE of a non-forced
// display set. When the read dies mid-title that absence means nothing —
// the rest of the track was never seen — yet the probe used to apply it as
// authoritative, overwriting the vendor flag from a fraction of the data.
// The defect: a forced verdict rests on the ABSENCE of a non-forced set.
// When the read dies mid-title that absence means nothing, yet the probe
// used to apply it as authoritative, overwriting the vendor flag.
let pid = 0x1200u16;
let mut reader = PartialTsReader::new(ts_stream(pid, &pcs_display(true)), ThenWhat::Error);
let mut title = multi_read_pgs_title(pid, false); // vendor label: NOT forced
@@ -1304,9 +1216,8 @@ mod tests {
!s.forced,
"a forced verdict from a read-truncated prefix must not overwrite the vendor flag"
);
// And it must not be memoised: the cache is keyed on the extent list, so a
// cached inconclusive result would spread this one read fault across every
// playlist sharing these clips and block any later re-read.
// And it must not be memoised: caching an inconclusive result would spread
// this one read fault across every playlist sharing these clips.
assert!(
cache.is_empty(),
"an inconclusive probe must not be cached against these extents"
@@ -1315,12 +1226,9 @@ mod tests {
#[test]
fn read_error_keeps_a_track_that_already_settled_not_forced() {
// Per-track, not per-title: "not forced" is POSITIVE evidence — a
// non-forced display set was actually seen — and no amount of unread data
// could retract it. That verdict survives the truncation even though a
// forced verdict would not. The vendor label already says not-forced, so
// what is pinned here is that the truncation does not let the track come
// back FORCED.
// Per-track, not per-title: "not forced" is POSITIVE evidence (a non-forced
// set was actually seen), so no amount of unread data retracts it — that
// verdict survives truncation even though a forced verdict would not.
let pid = 0x1200u16;
let mut reader = PartialTsReader::new(ts_stream(pid, &pcs_display(false)), ThenWhat::Error);
let mut title = multi_read_pgs_title(pid, false);
@@ -1339,9 +1247,8 @@ mod tests {
#[test]
fn budget_stop_still_applies_the_forced_verdict() {
// The budget is a DESIGNED stop, not a failure: the probe's natural exit
// never fires for a genuinely forced track, so the budget is exactly the
// mechanism by which a forced verdict gets accepted from a bounded prefix.
// Classifying it as inconclusive would disable forced detection.
// never fires for a genuinely forced track, so the budget is how a forced
// verdict gets accepted from a bounded prefix; inconclusive would disable it.
let pid = 0x1200u16;
let mut reader = PartialTsReader::new(ts_stream(pid, &pcs_display(true)), ThenWhat::Zeros);
let mut title = multi_read_pgs_title(pid, false); // vendor label: NOT forced
@@ -1355,9 +1262,8 @@ mod tests {
"a forced verdict from a budget-bounded prefix must still be applied"
);
// One entry per (extent, PGS track): both extents reached a designed stop
// (the first was read to its end, the second stopped at the budget), so
// both are memoised — the budget is the stop that fires on every disc that
// HAS a forced track, so excluding it would memoise nothing there.
// (first read to end, second stopped at budget), so both are memoised —
// excluding the budget stop would memoise nothing on forced-track discs.
assert_eq!(cache.len(), 2, "a conclusive probe is memoised per extent");
assert!(cache.contains_key(&(0, 4, pid)));
assert!(cache.contains_key(&(100, u32::MAX, pid)));
@@ -1367,9 +1273,7 @@ mod tests {
fn cancelled_probe_is_neither_asserted_nor_cached() {
// A halt lands at an arbitrary chunk boundary, so an absence claim from it
// is worth no more than one from a read fault. Here the cancel arrives
// AFTER a forced display set has been observed — the same setup that,
// uncancelled, reaches the budget and legitimately asserts forced (see
// budget_stop_still_applies_the_forced_verdict).
// AFTER a forced set was observed, unlike the uncancelled budget case.
let pid = 0x1200u16;
let halt = crate::halt::Halt::new();
let mut reader = PartialTsReader {
@@ -1386,11 +1290,9 @@ mod tests {
!s.forced,
"a verdict from a cancelled probe must not overwrite the vendor flag"
);
// The cancel landed inside the SECOND extent, whose read is therefore an
// arbitrary prefix: that extent must not be memoised, or the one
// cancellation would be replayed onto every other playlist sharing the
// clip. (The first extent WAS read to its end before the cancel, so its
// own evidence is sound and keeping it is the point of per-extent keying.)
// The cancel landed inside the SECOND extent, an arbitrary prefix that must
// not be memoised, or the cancellation replays onto every playlist sharing
// the clip. (The first extent read to its end, so it stays cached.)
assert!(
!cache.contains_key(&(100, u32::MAX, pid)),
"a cancelled probe must not poison the cancelled extent's cache entry"
@@ -1898,23 +1800,9 @@ mod tests {
/// anything — and therefore whether the probe reports the run as truncated.
#[test]
fn a_stop_reason_decides_whether_absence_is_conclusive() {
// The debug line at the end of `probe_and_set_forced` is gated on
// `!stop.absence_is_conclusive()`. Assert that PREDICATE rather than
// counting emitted events.
//
// Reading a log line back needs a capturing subscriber, which is
// thread-local, while tracing's callsite-interest cache is global.
// Those race: the sibling test in disc/encrypt.rs that did this failed
// roughly one full-suite run in ten while passing every time in
// isolation, and serialising the captures crate-wide was not enough —
// the cache can still be re-evaluated against the process default
// dispatch. A boolean does not need a subscriber to check.
//
// ECMA of the decision: a stop that saw everything it was ever going to
// see (Exhausted) or stopped by DESIGN at the budget is conclusive, so
// the absence of a display set means the track is not forced and there
// is nothing to report. A stop that was cut short (Halted, ReadFailed)
// is not, and that is exactly what the operator needs told.
// Assert the PREDICATE directly rather than counting log events (capturing
// a tracing log line is racy — see the sibling test in disc/encrypt.rs).
// Exhausted/Budget stops are conclusive; Halted/ReadFailed are not.
assert!(
StopReason::Exhausted.absence_is_conclusive(),
"reading every extent to its end is a complete observation"
@@ -1936,15 +1824,9 @@ mod tests {
);
}
// ── sampling: reading where the subtitles actually are ──────────────────
//
// SYNTHETIC FIXTURES. The clips below are generated, not captured: they
// reproduce the measured SHAPE of real discs — a feature's subtitles begin
// minutes into the title, a full dialogue track carries one to two thousand
// display sets spread over the whole runtime, a forced-narrative track
// carries tens — and nothing else. They prove the probe's LOGIC over that
// shape. They do not prove the shape itself; that comes from measurement on
// real discs, and if the shape is wrong these fixtures are wrong with it.
// ── sampling: reading where the subtitles actually are ──────────
// SYNTHETIC FIXTURES reproducing the measured SHAPE of real discs; they
// prove the probe's LOGIC over that shape, not the shape itself.
/// One synthetic PGS track: `count` display sets, the first at
/// `first_sector`, then every `period_sectors`.
+505
View File
@@ -0,0 +1,505 @@
//! Normalized, format-agnostic disc profile — a thin typed VIEW over the scanned
//! [`Disc`] model.
//!
//! The scan already normalizes every source (DVD/BD/UHD/HD-DVD) into the same
//! [`Disc`] / [`DiscTitle`] / [`Stream`] shapes, carrying language / forced /
//! qualifier / purpose / secondary / label per stream. This module hoists those
//! into a flat, serde-friendly surface so a consumer reads
//! `profile.titles[i].subtitles[j].forced` (and the analogous audio/video flags)
//! for ANY disc with no per-format conditionals.
//!
//! Everything here is DERIVED from the model at construction: the enum `Stream`
//! is split into three typed vectors, the `qualifier` / `purpose` enums are
//! decomposed into booleans, and the "default track" selection is precomputed so
//! downstream never recomputes it. Every field is always populated — a sensible
//! default (`"und"` language, empty `name`, `false` flag) stands in where the
//! model has nothing, so consumers never handle a bare `Option`.
use serde::{Deserialize, Serialize};
use super::{
AudioStream, Disc, DiscFormat, DiscTitle, LabelPurpose, LabelQualifier, Stream, SubtitleStream,
VideoStream,
};
/// A disc's complete normalized profile: identity plus every title's typed
/// track breakdown, with the main-feature selection hoisted to `main_title`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct DiscProfile {
/// Disc container family (`"bluray"`, `"uhd"`, `"dvd"`, `"hddvd"`, …).
pub format: String,
/// Best available display name (disc metadata title, else volume id).
pub disc_name: String,
/// Stable disc identifier (the UDF volume identifier).
pub disc_id: String,
/// Index into `titles` of the selected main feature. The scan pre-sorts
/// titles so `titles[0]` is the main feature, so this is `0` whenever any
/// title exists.
pub main_title: usize,
/// Every title, in the scan's main-feature order.
pub titles: Vec<TitleProfile>,
}
/// One title's normalized profile: identity, duration/size, chapter count, the
/// main-feature flag, and its streams split into typed per-kind vectors.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct TitleProfile {
/// Position of this title within [`DiscProfile::titles`].
pub index: usize,
/// Playlist / program identifier (e.g. `"00800.mpls"`).
pub playlist: String,
/// Duration in seconds.
pub duration_secs: f64,
/// Total size in bytes.
pub size_bytes: u64,
/// Number of chapter points.
pub chapters: usize,
/// Whether this is the disc's main feature (`index == main_title`).
pub is_main: bool,
/// Video tracks, in declared order.
pub video: Vec<VideoTrack>,
/// Audio tracks, in declared order.
pub audio: Vec<AudioTrack>,
/// Subtitle tracks, in declared order.
pub subtitles: Vec<SubtitleTrack>,
}
/// A normalized video track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct VideoTrack {
/// Compact codec id (e.g. `"hevc"`, `"h264"`, `"vc1"`, `"mpeg2"`).
pub codec: String,
/// Resolution label (e.g. `"2160p"`, `"1080p"`, `"576i"`).
pub resolution: String,
/// HDR format id (e.g. `"sdr"`, `"hdr10"`, `"hdr10+"`, `"dv"`, `"hlg"`).
pub hdr: String,
/// Frame-rate label (e.g. `"23.976"`, `"25"`).
pub frame_rate: String,
/// Whether this is the title's default video track (first non-secondary).
pub default: bool,
/// Whether this is a secondary stream (PiP / dependent view).
pub secondary: bool,
}
/// A normalized audio track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct AudioTrack {
/// ISO 639-2 language code, `"und"` when the source stated none.
pub language: String,
/// Compact codec id (e.g. `"truehd"`, `"dtshd_ma"`, `"ac3"`).
pub codec: String,
/// Channel layout label (e.g. `"stereo"`, `"5.1"`, `"unknown"`).
pub channels: String,
/// Whether this is the title's default audio track (first non-secondary).
pub default: bool,
/// Editorial purpose flag: commentary track.
pub commentary: bool,
/// Editorial purpose flag: descriptive / audio-description track.
pub descriptive: bool,
/// Codec / variant text label; empty when the source stated none.
pub name: String,
}
/// A normalized subtitle track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SubtitleTrack {
/// ISO 639-2 language code, `"und"` when the source stated none.
pub language: String,
/// Compact codec id (e.g. `"pgs"`, `"dvdsub"`).
pub codec: String,
/// Forced-narrative flag.
pub forced: bool,
/// SDH (subtitles for the deaf / hard-of-hearing) flag.
pub sdh: bool,
/// Whether this is the title's default subtitle track. Always `false`: the
/// mux never marks a subtitle default, and the normalized view mirrors that.
pub default: bool,
/// Track name; empty (the subtitle model carries no free-text label).
pub name: String,
}
/// Container family as a compact, stable id. Kept here (not on [`DiscFormat`])
/// because it is a serialization concern of this view.
fn format_id(format: DiscFormat) -> &'static str {
match format {
DiscFormat::Uhd => "uhd",
DiscFormat::Fmts => "uhd_fmts",
DiscFormat::BluRay => "bluray",
DiscFormat::HdDvd => "hddvd",
DiscFormat::Dvd => "dvd",
DiscFormat::Unknown => "unknown",
}
}
/// `"und"` for an empty language code, else the code unchanged. Mirrors the
/// muxer's `language_or_und`: the one place a source with no language table has
/// to fall back to the ISO 639-2 "undetermined" code.
fn language_or_und(lang: &str) -> String {
if lang.is_empty() {
"und".to_string()
} else {
lang.to_string()
}
}
impl VideoTrack {
fn from_stream(v: &VideoStream, default: bool) -> Self {
Self {
codec: v.codec.id().to_string(),
resolution: v.resolution.to_string(),
hdr: v.hdr.id().to_string(),
frame_rate: v.frame_rate.to_string(),
default,
secondary: v.secondary,
}
}
}
impl AudioTrack {
fn from_stream(a: &AudioStream, default: bool) -> Self {
Self {
language: language_or_und(&a.language),
codec: a.codec.id().to_string(),
channels: a.channels.to_string(),
default,
commentary: a.purpose == LabelPurpose::Commentary,
descriptive: a.purpose == LabelPurpose::Descriptive,
name: a.label.clone(),
}
}
}
impl SubtitleTrack {
fn from_stream(s: &SubtitleStream) -> Self {
Self {
language: language_or_und(&s.language),
codec: s.codec.id().to_string(),
// The authoritative forced flag (`forced`, set by the content probe /
// STN) OR the label-derived qualifier — either is sufficient.
forced: s.forced || s.qualifier == LabelQualifier::Forced,
sdh: s.qualifier == LabelQualifier::Sdh,
// The mux never defaults a subtitle track; mirror that here.
default: false,
name: String::new(),
}
}
}
impl TitleProfile {
/// Build a title's profile. `index` / `is_main` are disc-level facts the
/// caller supplies (see [`DiscProfile::from_disc`]); the streams are split
/// and the per-kind default track is hoisted here.
pub fn from_title(title: &DiscTitle, index: usize, is_main: bool) -> Self {
let mut video = Vec::new();
let mut audio = Vec::new();
let mut subtitles = Vec::new();
// "First non-secondary is the default; keep only the first" — the same
// rule the Matroska path applies (`is_default = !secondary`, then only
// the first video and first audio survive as default).
let mut video_default_taken = false;
let mut audio_default_taken = false;
for s in &title.streams {
match s {
Stream::Video(v) => {
let default = !v.secondary && !video_default_taken;
video_default_taken |= default;
video.push(VideoTrack::from_stream(v, default));
}
Stream::Audio(a) => {
let default = !a.secondary && !audio_default_taken;
audio_default_taken |= default;
audio.push(AudioTrack::from_stream(a, default));
}
Stream::Subtitle(t) => subtitles.push(SubtitleTrack::from_stream(t)),
}
}
Self {
index,
playlist: title.playlist.clone(),
duration_secs: title.duration_secs,
size_bytes: title.size_bytes,
chapters: title.chapters.len(),
is_main,
video,
audio,
subtitles,
}
}
/// The title's video tracks.
pub fn video(&self) -> &[VideoTrack] {
&self.video
}
/// The title's audio tracks.
pub fn audio(&self) -> &[AudioTrack] {
&self.audio
}
/// The title's subtitle tracks.
pub fn subtitles(&self) -> &[SubtitleTrack] {
&self.subtitles
}
}
impl DiscProfile {
/// Build the normalized profile from a scanned [`Disc`].
pub fn from_disc(disc: &Disc) -> Self {
// The scan pre-sorts titles so `titles[0]` is the selected main feature
// (`sort_titles_by_main_feature`), so the main title is index 0 whenever
// any title exists.
let main_title = 0;
let has_titles = !disc.titles.is_empty();
let titles = disc
.titles
.iter()
.enumerate()
.map(|(i, t)| TitleProfile::from_title(t, i, has_titles && i == main_title))
.collect();
Self {
format: format_id(disc.format).to_string(),
disc_name: disc
.meta_title
.clone()
.unwrap_or_else(|| disc.volume_id.clone()),
disc_id: disc.volume_id.clone(),
main_title,
titles,
}
}
/// The selected main-feature title, or `None` when the disc scanned to zero
/// titles (a data-only image with no /BDMV, /HVDVD_TS or /VIDEO_TS). Never
/// panics — mirrors the `Option`-returning main-feature accessors elsewhere
/// (`titles.first()`, `dvdnav::resolve_main_title`).
pub fn main_title(&self) -> Option<&TitleProfile> {
self.titles.get(self.main_title)
}
}
impl From<&Disc> for DiscProfile {
fn from(disc: &Disc) -> Self {
DiscProfile::from_disc(disc)
}
}
impl Disc {
/// This disc's normalized, format-agnostic [`DiscProfile`].
pub fn profile(&self) -> DiscProfile {
DiscProfile::from_disc(self)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{
AudioChannels, AudioStream, Codec, ColorSpace, ContentFormat, DiscRegion, DiscTitle,
FrameRate, HdrFormat, Resolution, SampleRate, Stream, SubtitleStream, VideoStream,
};
/// A minimal titleless [`Disc`] for the disc-level tests.
fn test_disc() -> Disc {
Disc {
volume_id: String::new(),
meta_title: None,
format: DiscFormat::BluRay,
capacity_sectors: 0,
capacity_bytes: 0,
layers: 1,
titles: Vec::new(),
region: DiscRegion::Free,
aacs: None,
css: None,
encrypted: false,
aacs_error: None,
css_error: None,
content_format: ContentFormat::BdTs,
}
}
#[test]
fn main_title_is_none_on_empty_disc() {
// A data-only image (no /BDMV, /HVDVD_TS, /VIDEO_TS) scans to zero
// titles; main_title() must return None, not panic on titles[0].
let profile = DiscProfile::from_disc(&test_disc());
assert!(profile.titles.is_empty());
assert!(profile.main_title().is_none());
}
fn video(codec: Codec, res: Resolution, secondary: bool) -> Stream {
Stream::Video(VideoStream {
pid: 0x1011,
codec,
resolution: res,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
display_aspect: None,
secondary,
label: String::new(),
measured_cicp: None,
})
}
fn audio(lang: &str, secondary: bool, purpose: LabelPurpose, label: &str) -> Stream {
Stream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Stereo,
language: lang.into(),
sample_rate: SampleRate::S48,
secondary,
purpose,
label: label.into(),
})
}
fn subtitle(lang: &str, forced: bool, qualifier: LabelQualifier) -> Stream {
Stream::Subtitle(SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: lang.into(),
forced,
qualifier,
codec_data: None,
})
}
/// A BD transport-stream title with a mix of streams: two audio tracks
/// (main + commentary), two subtitles (forced + SDH), a video track.
fn bdts_title() -> DiscTitle {
let mut t = DiscTitle::empty();
t.content_format = ContentFormat::BdTs;
t.playlist = "00800.mpls".into();
t.duration_secs = 7200.0;
t.size_bytes = 30_000_000_000;
t.streams = vec![
video(Codec::Hevc, Resolution::R2160p, false),
audio("eng", false, LabelPurpose::Normal, "Dolby TrueHD 5.1"),
audio("eng", true, LabelPurpose::Commentary, "Director"),
subtitle("eng", true, LabelQualifier::Forced),
subtitle("eng", false, LabelQualifier::Sdh),
];
t
}
/// A DVD (MPEG program stream) title: two audio tracks (both non-secondary,
/// second is descriptive), one language-less subtitle, one video track.
fn dvd_title() -> DiscTitle {
let mut t = DiscTitle::empty();
t.content_format = ContentFormat::MpegPs;
t.playlist = "VTS_01".into();
t.duration_secs = 5400.0;
t.size_bytes = 4_000_000_000;
t.streams = vec![
video(Codec::Mpeg2, Resolution::R576i, false),
audio("eng", false, LabelPurpose::Normal, ""),
audio("eng", false, LabelPurpose::Descriptive, ""),
subtitle("", false, LabelQualifier::None),
];
t
}
#[test]
fn splits_streams_into_typed_vectors_across_formats() {
for title in [bdts_title(), dvd_title()] {
let p = TitleProfile::from_title(&title, 0, true);
assert_eq!(p.video.len(), 1, "one video track");
assert_eq!(p.audio.len(), 2, "two audio tracks");
assert_eq!(p.subtitles.len(), 1.max(p.subtitles.len()));
// Accessors return the same vectors.
assert_eq!(p.video(), p.video.as_slice());
assert_eq!(p.audio(), p.audio.as_slice());
assert_eq!(p.subtitles(), p.subtitles.as_slice());
}
}
#[test]
fn hoists_default_flags_first_non_secondary() {
let p = TitleProfile::from_title(&bdts_title(), 0, true);
// First (non-secondary) video/audio is the default.
assert!(p.video[0].default, "first video defaults");
assert!(p.audio[0].default, "first non-secondary audio defaults");
// The commentary track is secondary → never default.
assert!(
!p.audio[1].default,
"secondary/commentary audio not default"
);
// Subtitles are never default.
assert!(p.subtitles.iter().all(|s| !s.default));
}
#[test]
fn second_default_cleared_when_two_non_secondary_audio() {
// DVD title: BOTH audio tracks are non-secondary. Only the first keeps
// the default flag (mirrors the muxer's "keep only first default").
let p = TitleProfile::from_title(&dvd_title(), 0, true);
assert!(p.audio[0].default, "first audio default");
assert!(
!p.audio[1].default,
"second non-secondary audio default cleared"
);
assert!(p.audio[1].descriptive, "second audio is descriptive");
assert!(!p.audio[1].commentary);
}
#[test]
fn maps_qualifier_and_purpose_and_language_defaults() {
let p = TitleProfile::from_title(&bdts_title(), 0, true);
// Audio purpose → commentary/descriptive booleans.
assert!(!p.audio[0].commentary && !p.audio[0].descriptive);
assert!(p.audio[1].commentary);
assert_eq!(p.audio[1].name, "Director", "label maps to name");
// Subtitle qualifier → forced/sdh booleans.
assert!(p.subtitles[0].forced && !p.subtitles[0].sdh);
assert!(p.subtitles[1].sdh && !p.subtitles[1].forced);
// Language default: DVD title's blank subtitle language → "und".
let dvd = TitleProfile::from_title(&dvd_title(), 0, true);
assert_eq!(dvd.subtitles[0].language, "und");
assert_eq!(dvd.audio[0].language, "eng");
// Codec/resolution/hdr flattened as compact ids/labels.
assert_eq!(p.video[0].codec, "hevc");
assert_eq!(p.video[0].resolution, "2160p");
assert_eq!(p.video[0].hdr, "hdr10");
assert_eq!(dvd.video[0].codec, "mpeg2");
}
#[test]
fn from_disc_hoists_main_and_is_main() {
let mut disc = test_disc();
disc.format = DiscFormat::BluRay;
disc.meta_title = Some("SOME MOVIE".into());
disc.volume_id = "VOL_ID".into();
disc.titles = vec![bdts_title(), dvd_title()];
let profile = disc.profile();
assert_eq!(profile.format, "bluray");
assert_eq!(profile.disc_name, "SOME MOVIE");
assert_eq!(profile.disc_id, "VOL_ID");
assert_eq!(profile.main_title, 0);
assert!(profile.titles[0].is_main, "titles[0] is the main feature");
assert!(!profile.titles[1].is_main);
assert_eq!(profile.titles[1].index, 1);
assert_eq!(profile.main_title().unwrap().playlist, "00800.mpls");
}
#[test]
fn disc_name_falls_back_to_volume_id() {
let mut disc = test_disc();
disc.meta_title = None;
disc.volume_id = "PLAIN_VOLUME".into();
assert_eq!(disc.profile().disc_name, "PLAIN_VOLUME");
}
#[test]
fn serde_round_trip() {
let mut disc = test_disc();
disc.format = DiscFormat::Dvd;
disc.volume_id = "RT".into();
disc.titles = vec![bdts_title(), dvd_title()];
let profile = disc.profile();
let json = serde_json::to_string(&profile).expect("serialize");
let back: DiscProfile = serde_json::from_str(&json).expect("deserialize");
assert_eq!(profile, back, "profile round-trips through serde");
}
}
+3 -4
View File
@@ -150,10 +150,9 @@ mod tests {
#[test]
fn mask_string_preserves_non_ascii_chars() {
// is_ascii_alphabetic/is_ascii_digit are false for non-ASCII, so
// multibyte chars pass through unchanged (no mojibake, no panic).
// 'c','a','f' are ASCII letters → 'A'; 'é' is non-ASCII →
// preserved; '9' → '0'.
// is_ascii_alphabetic/is_ascii_digit are false for non-ASCII, so multibyte
// chars pass through unchanged (no mojibake, no panic): ASCII letters → 'A',
// digits → '0', 'é' preserved.
assert_eq!(mask_string("café9"), "AAAé0");
}
+3 -8
View File
@@ -84,14 +84,9 @@ pub fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
let sr_id = DriveId::from_drive(sr_transport.as_mut())?;
drop(sr_transport);
for (sg_path, sg_id) in find_drives() {
// Require a non-empty serial before treating vendor/product/
// serial as a unique match. serial_number falls back to an
// empty string when GET CONFIGURATION 0108h is unavailable
// (common on OEM drives); two same-model drives would then
// both compare equal and the first in enumeration order would
// win silently, resolving sr1 to sr0's sg node. An empty
// serial can't disambiguate, so fall through to the no-match
// path instead.
// Require a non-empty serial before treating vendor/product/serial as a unique
// match: serial_number falls back to "" when GET CONFIGURATION 0108h is
// unavailable (OEM drives), which would let same-model drives collide silently.
if !sr_id.serial_number.is_empty()
&& sg_id.vendor_id == sr_id.vendor_id
&& sg_id.product_id == sr_id.product_id
+447 -177
View File
@@ -8,6 +8,12 @@ pub fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
// A failed `ioctl(SG_IO)` or vanished device is a dead-bus fault, not a
// recoverable bad sector — map to SCSI_STATUS_TRANSPORT_FAILURE so sweep /
// patch / fill_extents abort the pass instead of zero-filling a wedged device.
Error::IoError { .. } | Error::DeviceNotFound { .. } => {
(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None)
}
_ => (0, None),
}
}
@@ -24,6 +30,15 @@ pub(crate) mod macos;
#[cfg(windows)]
pub(crate) mod windows;
// Pick the platform module ONCE, here, so the cross-platform entry points below
// dispatch through `platform::…` with no per-function `#[cfg]` in their bodies.
#[cfg(target_os = "linux")]
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
pub(crate) use macos as platform;
#[cfg(windows)]
pub(crate) use windows as platform;
use crate::error::{Error, Result};
use crate::event::Event;
use crate::identity::DriveId;
@@ -279,22 +294,26 @@ impl Drive {
for attempt in 0..60u64 {
hb.tick(attempt, 60);
let mut buf = [0u8; 0];
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
// `checked_exec`, not bare `execute`: this poll (60 x 500 ms = ~30 s) must
// see `self.halt`, or a Stop during cold spin-up is ignored for half a
// minute. A TUR that FAILS is just not-ready-yet and keeps the loop going.
match self.checked_exec(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
}
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => {}
}
std::thread::sleep(std::time::Duration::from_millis(500));
// Halt-aware backoff: the flag can also flip DURING the 500 ms
// gap, which is where most of the 30 s is actually spent.
sleep_until_halted(&self.halt, std::time::Duration::from_millis(500))?;
}
tracing::warn!(
target: "freemkv::drive",
@@ -331,21 +350,9 @@ impl Drive {
5_000,
);
// MMC-6 §6.7: byte 5 is a Media Status only when the Event Header
// actually announces a Media Event Descriptor behind it. The header is
// Event Descriptor Length (big-endian, bytes 0-1), then byte 2 = NEA
// (bit 7) + Notification Class (bits 2-0), then byte 3 = Supported
// Event Classes. With NEA set the drive is telling us there is NO event
// to report and returns the header alone; with a different Notification
// Class the descriptor that follows is not a media one and its second
// byte means something else entirely.
//
// Decoding byte 5 unconditionally turns both of those into Media Status
// 0 == "tray closed, no disc" — a drive with a disc loaded reported as
// empty, from a reply that said nothing about the media at all. The
// drive is untrusted input here, so an event-less or foreign-class reply
// yields no media state and the TEST UNIT READY fallback answers
// instead.
// MMC-6 §6.7: byte 5 is Media Status only when the Event Header (NEA flag +
// Notification Class) announces a Media Event Descriptor. Otherwise decoding
// it reads as "no disc" for untrusted input, so fall back to TUR instead.
const NEA: u8 = 0x80;
const NOTIFICATION_CLASS_MASK: u8 = 0x07;
const NOTIFICATION_CLASS_MEDIA: u8 = 0x04;
@@ -385,10 +392,9 @@ impl Drive {
0x00 => DriveStatus::NoDisc, // tray closed, no disc
0x01 => DriveStatus::TrayOpen, // tray open, no media
0x02 => DriveStatus::DiscPresent, // tray closed, disc present
// 0x03 = tray-open bit AND media-present bit both set:
// a contradictory/transient state. Don't report it as
// ready — autorip must not start a rip on a drive that
// is still settling. Treat as tray-open.
// 0x03 = tray-open AND media-present both set: a contradictory,
// transient state. Don't report ready — autorip must not start on
// a drive that's still settling — so treat as tray-open.
0x03 => DriveStatus::TrayOpen,
_ => DriveStatus::Unknown,
}
@@ -487,26 +493,17 @@ impl Drive {
pub fn init(&mut self) -> Result<()> {
let t0 = std::time::Instant::now();
tracing::info!(target: "freemkv::drive", phase = "init", "begin");
// Drive-prep runs for EVERY disc, DVD INCLUDED. The drive-level firmware
// unlock lifts riplock and readies max read speed regardless of disc type
// — drive features are disc-independent. At init the disc kind is not yet
// probed (Unknown), so only the identity-keyed DRIVE unlocker matches
// here; the AACS host-cert handshake and the CSS bus-auth handshake run
// LATER, each gated on the actual disc kind, on TOP of the already-
// unlocked drive. A genuine transport fault means the bus is dead — abort
// init (the v1.1.0 invariant; `if let Ok` was silently swallowing it).
// Every other error (NotApplicable / no match) is "nothing applied" —
// fall through to stock mode. NOTE: the `if disc_is_dvd() { return }` skip
// that used to sit here was the v1.0.0-rc.1 regression — it skipped the
// drive-prep for DVD, leaving DVDs riplocked at stock speed.
// Drive-prep runs for EVERY disc, DVD INCLUDED — drive features are
// disc-independent; AACS/CSS handshakes run LATER, gated on disc kind. A
// transport fault aborts init (v1.1.0 invariant); other errors fall through.
self.init_ran = true;
let (matched, unlock_res) =
crate::unlock_bridge::run_features(self.scsi.as_mut(), &self.drive_id);
let r: Result<()> = match unlock_res {
Ok(unlocked) => {
// Record WHICH drive-prep unlocker actually ran — "LibreDrive"
// (MediaTek) or "Renesas" — not the ld-only identity lookup, so a
// Renesas drive reports itself honestly rather than as nothing.
// Record WHICH drive-prep unlocker ran ("MT1959" bus-unlock or
// "Renesas" cert route), not the id-only lookup, so a Renesas
// drive reports itself honestly rather than as nothing.
self.unlocker_name = Some(matched.to_string());
// Stash the OEM Volume ID the unlocker returned for the AACS
// handshake phase (do_handshake reads it via `oem_vid()`). A
@@ -523,19 +520,14 @@ impl Drive {
}),
Err(_) => Ok(()),
};
// Raise the drive to its maximum read speed — UNCONDITIONALLY whenever
// the bus is alive (init didn't transport-fault), whether or not an
// unlocker matched, and for ANY disc type (DVD now flows through here too,
// so it gets max speed on the freshly firmware-unlocked drive instead of
// the stock riplock). A stock-mode drive with no firmware unlocker still
// wants max speed. Best-effort: a failure here must NOT fail the rip.
// Raise to max read speed UNCONDITIONALLY whenever the bus is alive, for
// ANY disc type — DVD included, and even a stock-mode drive with no
// firmware unlocker still wants it. Best-effort: failure must NOT fail the rip.
if r.is_ok() {
self.set_speed(Self::SPEED_MAX_KBPS);
// Ask the drive to REPORT recovered/marginal reads rather than
// silently commit best-effort data as GOOD (the dirty-disc
// "passed-clean-but-decodes-with-errors" trap). Best-effort: a drive
// that doesn't honor it just keeps its defaults — no regression, and
// on a clean disc it changes nothing.
// Ask the drive to REPORT recovered/marginal reads instead of silently
// committing best-effort data as GOOD (the dirty-disc "passed-clean-but-
// decodes-with-errors" trap). Best-effort: unsupported drives keep defaults.
self.enable_recovered_error_reporting();
}
tracing::info!(
@@ -812,29 +804,20 @@ impl Drive {
"Drive::read enter"
);
// Cap each CDB to the transport's max data-in transfer. A single
// READ larger than the adapter limit fails outright on some
// backends (notably Windows SPTI, where a 16 MiB read exceeds the
// adapter MaximumTransferLength → DeviceIoControl fails → we'd
// mis-read it as a transport failure and spam tiny-read fallbacks).
// For the common small read (count <= max_sectors) this is a single
// read_one call with no behavior change.
// Cap each CDB to the transport's max data-in transfer: a single READ past the
// adapter limit fails outright on some backends (e.g. Windows SPTI, where a
// 16 MiB read exceeds MaximumTransferLength and we'd mis-read it as transport failure).
let max_sectors = (self.scsi.max_transfer_bytes() / 2048).max(1) as u32;
if count as u32 <= max_sectors {
return self.read_one(lba, count, buf, timeout_ms, recovery, fua);
}
// Large read: split into chunks of at most `max_sectors` sectors,
// each a self-contained READ(10) with the same validation. Any
// chunk error reports that chunk's LBA (more precise than the whole
// request's base LBA).
// Large read: split into `max_sectors`-sized chunks, each a self-contained
// READ(10). Any chunk error reports that chunk's LBA, not the base LBA.
let count = count as u32;
// Check the caller's buffer ONCE, up front. The chunk loop slices `buf`
// by `count * 2048`; without this an undersized buffer PANICKED ('range
// end index out of range') out of the public `read`/`read_fua`, while the
// single-chunk path above tolerates the same undersized buffer and returns
// `Err(DiscRead)` from `checked_exec`. Behaviour on an undersized buffer
// must not depend on the transport's transfer limit.
// Check the caller's buffer ONCE, up front: the chunk loop slices `buf` by
// `count * 2048`, and without this an undersized buffer PANICKED out of the
// public `read`/`read_fua` instead of returning `Err(DiscRead)` like the single-chunk path.
if buf.len() < count as usize * 2048 {
return Err(Error::DiscRead {
sector: lba as u64,
@@ -842,11 +825,9 @@ impl Drive {
sense: None,
});
}
// The whole range must be addressable: SBC-3 READ(10) carries a 32-bit
// LOGICAL BLOCK ADDRESS, so a request whose last chunk crosses `u32::MAX`
// has no valid CDB. `lba + done` below was unchecked — a debug panic out
// of the public API, and in release a wrap to a low LBA that was read and
// returned as if it were the requested one.
// The whole range must be addressable: READ(10) carries a 32-bit LBA, so a
// request crossing `u32::MAX` has no valid CDB. `lba + done` was unchecked —
// a debug panic, or in release a wrap to a low LBA read and returned as requested.
if lba.checked_add(count.saturating_sub(1)).is_none() {
return Err(Error::DiscRead {
sector: lba as u64,
@@ -889,13 +870,9 @@ impl Drive {
// re-fetch the medium past its cache.
fua: bool,
) -> Result<usize> {
// FUA is OFF on the bulk path — unconditionally forcing every READ(10)
// past the cache disabled the drive's readahead/streaming cache on the
// sequential sweep and collapsed throughput ~10x (UHD 15-25 → ~2 MB/s,
// DVD → ~0.5 MB/s), disc-type-agnostic — the cache IS the streaming
// throughput. It is set ONLY when a Pass-N recovery handler (FuaRetry)
// asks for it per marginal-sector re-read, where cache-masking of a
// stochastic sector actually matters (#55).
// FUA is OFF on the bulk path: forcing every READ(10) past the cache disabled
// readahead on the sequential sweep and collapsed throughput ~10x (UHD → ~2
// MB/s, DVD → ~0.5 MB/s). Set ONLY per marginal-sector re-read by FuaRetry (#55).
let cdb = [
crate::scsi::SCSI_READ_10,
if fua { 0x08 } else { 0x00 },
@@ -918,19 +895,25 @@ impl Drive {
Ok(result) if result.bytes_transferred == count as usize * 2048 => {
Ok(result.bytes_transferred)
}
// A READ(10) that completes with GOOD status but a residual
// underrun (bytes_transferred < requested) is a SHORT transfer:
// the tail of `buf` still holds stale bytes from a prior read.
// Committing those as recovered/Good is silent data corruption, so
// treat a short transfer as a failed read — the caller marks the
// range NonTrimmed and retries (a loud miss, never a silent commit).
// The sector/file path enforces the same invariant in
// sector/prefetched.rs; this is the live-drive counterpart.
Ok(_) => Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
}),
// GOOD status with a residual underrun is a SHORT transfer: `buf`'s tail
// holds stale bytes, so fail (NonTrimmed, retried) rather than commit stale
// data — and log it, so it isn't indistinguishable from a scratched disc.
Ok(result) => {
tracing::warn!(
target: "freemkv::drive",
lba,
count,
transferred = result.bytes_transferred,
expected = count as usize * 2048,
code = crate::error::E_DISC_READ,
"READ(10) returned GOOD status with a residual underrun; refusing the short transfer"
);
Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
})
}
Err(Error::Halted) => Err(Error::Halted),
Err(e) => {
let (status, sense) = extract_scsi_context(&e);
@@ -943,11 +926,9 @@ impl Drive {
"Drive::read checked_exec failed"
);
// /dev/sr0 pread fallback (Linux only). The kernel
// sr_mod driver auto-retries failed reads (~5× per
// command). Empirically (BU40N + a UHD disc,
// 2026-05-08) dd via /dev/sr0 recovers ~50% of bad
// sectors that a single-shot SG_IO READ misses.
// /dev/sr0 pread fallback (Linux only): sr_mod auto-retries reads
// (~5x). Empirically (BU40N + UHD disc, 2026-05-08) this recovers
// ~50% of bad sectors that a single-shot SG_IO READ misses.
#[cfg(target_os = "linux")]
if recovery
&& let Some(fd) = self.block_dev_fd
@@ -1086,14 +1067,19 @@ impl Drive {
let stop = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x00, 0]; // START=0, LOEJ=0 → spin down
let start = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x01, 0]; // START=1, LOEJ=0 → spin up
let mut buf = [0u8; 0];
self.scsi
.as_mut()
.execute(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS));
self.scsi
.as_mut()
.execute(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS));
// `checked_exec` + `sleep_until_halted`, not blind `thread::sleep`: this ~15 s
// wait runs from the recovery path, exactly when Stop is likely pressed. A
// half-finished spin cycle is fine — the next command spins the drive back up.
self.checked_exec(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS),
)?;
self.checked_exec(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS),
)?;
Ok(())
}
@@ -1299,11 +1285,18 @@ pub(crate) fn decode_read_capacity(buf: &[u8; 8], bytes_transferred: usize) -> R
last_lba.checked_add(1).ok_or(Error::DiscCapacityOverflow)
}
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
/// to true. Kept for the unit tests that cover the slicing behaviour;
/// production code paths no longer sleep on the recovery hot path
/// (recovery loop removed in 0.13.6).
#[cfg(test)]
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping to
/// true, returning [`Error::Halted`].
///
/// This was `#[cfg(test)]` for two releases, kept alive only by the four unit
/// tests below, while the two production paths that actually sleep — the
/// `wait_ready` poll backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle
/// pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
/// `std::thread::sleep` and so were DEAF to the operator's Stop for ~30 s and
/// ~15 s respectively. Every other drive path returns `Halted` at the next
/// `checked_exec` boundary; a Stop pressed during spin-up simply did nothing
/// visible until the poll ran out. The primitive existed; the sleeping code
/// just did not call it. It is production code again.
fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<()> {
const SLICE: std::time::Duration = std::time::Duration::from_millis(100);
let deadline = std::time::Instant::now() + total;
@@ -1322,18 +1315,7 @@ fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<(
/// Internal: discover drive paths + IDs without opening full Drive objects.
fn discover_drives() -> Vec<(String, DriveId)> {
#[cfg(target_os = "linux")]
{
linux::find_drives()
}
#[cfg(target_os = "macos")]
{
macos::find_drives()
}
#[cfg(windows)]
{
windows::find_drives()
}
platform::find_drives()
}
/// Structured outcome of [`resolve_device`] — a machine-readable signal
@@ -1353,20 +1335,14 @@ pub enum DeviceResolution {
/// Resolve a device path to its raw SCSI device. Returns the resolved
/// path plus a structured [`DeviceResolution`] signal describing whether
/// any substitution happened; the application layer maps that to UX text.
///
/// Staged, not yet wired: the cross-platform dispatch is kept ready for the
/// caller that will consume it, so the per-platform implementations below it
/// (and their tests) stay live. `allow(dead_code)` marks that deliberately —
/// this is not an accidental orphan.
#[allow(dead_code)]
pub(crate) fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
#[cfg(target_os = "linux")]
{
linux::resolve_device(path)
}
#[cfg(target_os = "macos")]
{
macos::resolve_device(path)
}
#[cfg(windows)]
{
windows::resolve_device(path)
}
platform::resolve_device(path)
}
#[cfg(test)]
@@ -1406,17 +1382,9 @@ mod halt_tests {
let r = sleep_until_halted(&flag, Duration::from_secs(10));
assert!(matches!(r, Err(Error::Halted)));
let waited = t0.elapsed();
// What this test is for: the sleep must end because the flag flipped,
// NOT because the 10 s timeout expired. Anything comfortably under
// that proves it, and the lower bound proves it did not return early
// for some other reason.
//
// The upper bound used to be 350 ms — flag at ~150 ms plus one 100 ms
// poll slice, with a little slack. That measures the SCHEDULER, not
// this function: a loaded CI runner took 377 ms and failed, which says
// nothing about whether the wake worked. Bound it well below the
// timeout instead, so the assertion still distinguishes the two
// outcomes it exists to distinguish.
// The sleep must end because the flag flipped, not the 10 s timeout; the lower
// bound rules out returning early otherwise. Upper bound was 350 ms but that
// measures the scheduler: a loaded CI runner hit 377 ms and failed for no reason.
assert!(waited < Duration::from_secs(2), "waited {waited:?}");
assert!(waited >= Duration::from_millis(140), "waited {waited:?}");
}
@@ -1792,11 +1760,9 @@ mod command_tests {
#[test]
fn read_builds_read10_cdb_with_be_lba_and_count() {
// Drive::read issues READ(10) (0x28). LBA bytes 2..5 big-endian,
// transfer length bytes 7..8 big-endian (MMC-6). FUA is DISABLED (byte 1
// == 0x00) so the drive cache/readahead is allowed — forcing FUA on the
// bulk sweep collapsed throughput ~10x. Distinct nibbles catch a swapped
// shift.
// READ(10) (0x28): LBA bytes 2..5 BE, length bytes 7..8 BE (MMC-6). FUA is
// DISABLED (byte 1 == 0x00) — forcing FUA on the bulk sweep collapsed
// throughput ~10x. Distinct nibbles catch a swapped shift.
let RecordingHarness {
drive: mut d,
cdb,
@@ -1936,6 +1902,90 @@ mod command_tests {
assert!(err.scsi_sense().is_none());
}
/// `extract_scsi_context` must map the two non-SCSI dead-bus faults
/// (a failed `ioctl(SG_IO)` → `Error::IoError`, a vanished fd →
/// `Error::DeviceNotFound`) to the 0xFF TRANSPORT_FAILURE sentinel, not
/// to 0x00. Everything else keeps the (0, None) catch-all.
/// Spec: `Error::is_scsi_transport_failure` (error.rs) declares both
/// variants transport failures so sweep/patch/fill_extents abort
/// the pass instead of zero-filling against a wedged device.
/// Mutation: returning (0, None) here flattens IoError into
/// `DiscRead { status: Some(0) }`, which is_scsi_transport_failure
/// rejects — a wedged USB bridge zero-fills the whole title.
#[test]
fn extract_scsi_context_maps_dead_bus_faults_to_transport_failure() {
let (status, sense) = extract_scsi_context(&Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a failed ioctl(SG_IO) is a transport-layer fault"
);
assert!(sense.is_none(), "no SCSI reply means no sense data");
let (status, sense) = extract_scsi_context(&Error::DeviceNotFound {
path: "/dev/sg9".into(),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a vanished device is a transport-layer fault"
);
assert!(sense.is_none());
// Control: the catch-all still yields (0, None) for unrelated errors,
// so the two asserts above are about these variants specifically.
assert_eq!(extract_scsi_context(&Error::Halted), (0, None));
// Control: real SCSI replies still pass their own status through.
let s = crate::scsi::ScsiSense {
sense_key: 3,
asc: 0x11,
ascq: 0x00,
};
assert_eq!(
extract_scsi_context(&Error::ScsiError {
opcode: 0x28,
status: 0x02,
sense: Some(s),
}),
(0x02, Some(s))
);
}
/// End-to-end: an `Error::IoError` raised by the transport must still be
/// classified as a transport failure after `Drive::read` flattens it into
/// `Error::DiscRead`. Without the `extract_scsi_context` mapping the
/// variant is destroyed (status becomes `Some(0)`) and the caller treats a
/// dead bus as a recoverable bad sector.
#[test]
fn read_io_error_surfaces_as_transport_failure_not_a_bad_sector() {
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
},
}));
let mut buf = vec![0u8; 2048];
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a wedged bus must abort the pass, not zero-fill: got {err:?}"
);
// The same for a device that vanished mid-read.
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::DeviceNotFound {
path: "/dev/sg9".into(),
},
}));
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a vanished device must abort the pass: got {err:?}"
);
}
#[test]
fn read_returns_halted_before_dispatch_without_touching_transport() {
// When the halt flag is set, checked_exec returns Halted BEFORE
@@ -2296,10 +2346,9 @@ mod command_tests {
#[test]
fn select_drive_falls_back_to_first_when_none_have_media() {
// No drive reports a disc → fall back to the FIRST opened drive so
// single-drive / quirky setups still get a drive (historical
// behavior preserved). Tag drive #1 distinctly (TrayOpen 0x01) and
// confirm it, not #2 (NoDisc 0x00), is returned.
// No drive reports a disc → fall back to the FIRST opened drive so single-drive
// / quirky setups still get one (historical behavior). Tag drive #1 distinctly
// (TrayOpen 0x01) and confirm it, not #2 (NoDisc 0x00), is returned.
let drives = vec![drive_with_media_byte(0x01), drive_with_media_byte(0x00)];
let mut picked = select_drive_with_media(drives.into_iter()).expect("a fallback drive");
assert_eq!(
@@ -2367,10 +2416,9 @@ mod command_tests {
#[test]
fn drive_status_short_transfer_falls_back_to_tur() {
// bytes_transferred < 6 means the GET EVENT reply is unusable;
// the code falls back to a TUR. FixedTransport always returns
// Ok, so the TUR "succeeds" → DiscPresent. (Buffer length 8 but
// payload only 4 bytes → bytes_transferred = 4.)
// bytes_transferred < 6 (buffer len 8, payload 4) means the GET EVENT reply
// is unusable, so code falls back to a TUR; FixedTransport always returns
// Ok, so the TUR "succeeds" → DiscPresent.
let mut d = drive_with(vec![0u8; 4]);
assert_eq!(d.drive_status(), DriveStatus::DiscPresent);
}
@@ -2470,11 +2518,9 @@ mod command_tests {
#[test]
fn get_config_feature_encodes_feature_code_be_in_cdb() {
// GET CONFIGURATION (0x46), RT field byte 1 = 0x02 (report the
// named feature), then the 16-bit feature code big-endian in
// bytes 2..4. 0x010D has a nonzero high byte, so a swapped shift
// (>> vs <<) or byte order bug asks the drive for the wrong
// feature entirely.
// GET CONFIGURATION (0x46), RT byte 1 = 0x02, feature code big-endian in
// bytes 2..4. 0x010D has a nonzero high byte, so a swapped shift (>> vs <<)
// or byte order bug would ask the drive for the wrong feature.
let RecordingHarness {
drive: mut d,
cdb,
@@ -2589,6 +2635,231 @@ mod command_tests {
);
}
/// A Stop pressed before the poll starts must be answered at once.
///
/// `wait_ready` was the ONE drive path that called
/// `self.scsi.as_mut().execute(..)` instead of `checked_exec`, and its
/// 60 x 500 ms loop never read `self.halt`. A cancel during spin-up was
/// therefore ignored for ~30 s — the test above measures exactly how long
/// — while every other drive path returns `Halted` at its next command
/// boundary. Thirty seconds of a dead Stop button is indistinguishable
/// from a hung application.
///
/// Mutation: restoring the bare `execute` makes this run the full poll and
/// return `DeviceNotReady`, failing both assertions.
#[test]
fn wait_ready_returns_halted_when_stopped_before_the_poll() {
struct NeverReady;
impl ScsiTransport for NeverReady {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(NeverReady));
d.halt();
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop is the operator, not a drive that failed to spin up: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"the poll must abandon immediately, not run its ~30 s course"
);
}
/// ...and a Stop pressed PART WAY THROUGH the poll must be answered at the
/// next command boundary, not at the end of the 30 s.
///
/// The flag is flipped by the transport itself on its third TEST UNIT
/// READY, which is deterministic (no wall-clock race): `checked_exec`
/// re-reads the flag after the command completes, so the loop must exit on
/// attempt three of sixty.
#[test]
fn wait_ready_returns_halted_when_stopped_during_the_poll() {
struct StopsOnThirdPoll {
halt: Arc<AtomicBool>,
seen: usize,
}
impl ScsiTransport for StopsOnThirdPoll {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
self.seen += 1;
if self.seen == 3 {
self.halt.store(true, Ordering::Relaxed);
}
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(StopsOnThirdPoll {
halt: Arc::new(AtomicBool::new(false)),
seen: 0,
}));
// Hand the transport the drive's OWN flag, so setting it is exactly
// what an operator's Stop does.
let flag = d.halt_flag();
d.scsi = Box::new(StopsOnThirdPoll {
halt: flag,
seen: 0,
});
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a mid-poll Stop must surface as Halted, not as DeviceNotReady \
after the full 30 s: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"must exit on the third poll, not the sixtieth"
);
}
/// `spin_cycle` must not be deaf to Stop for its ~15 s of deliberate
/// waiting.
///
/// Both START STOP UNIT commands went out through a bare `execute` and
/// both pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) were plain
/// `std::thread::sleep`, so the whole routine ignored the halt flag. It
/// runs from the RECOVERY path — precisely when a run is going badly and
/// the operator is most likely to press Stop.
///
/// The flag is set BEFORE the call, so this is deterministic: the very
/// first `checked_exec` must refuse.
///
/// Mutation: restoring either bare `execute` lets the first command
/// through and the test then waits out a real 5 s sleep, failing the
/// elapsed bound.
#[test]
fn spin_cycle_returns_halted_when_stopped_before_it_starts() {
let RecordingHarness {
drive: mut d,
cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
d.halt();
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop must abort the spin cycle: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"no sleep may run after the flag is set"
);
assert!(
cdb.lock().unwrap().is_empty(),
"not one START STOP UNIT may be issued after Stop"
);
}
/// ...and a Stop that lands DURING the spin-down pause must wake it.
///
/// This is the half a `checked_exec` alone cannot fix: the flag flips
/// while the thread is parked inside the 5 s `SPIN_DOWN_IDLE_SECS` sleep,
/// so only a halt-aware sleep can observe it. `sleep_until_halted` — which
/// already lived in this file with four tests, marked `#[cfg(test)]` and
/// called from nowhere — wakes within ~100 ms.
///
/// The margin is deliberately enormous (≈0.3 s against 5 s) so the bound
/// is not a wall-clock race: a plain `thread::sleep` cannot come in under
/// two seconds, and the halt-aware one cannot take that long.
#[test]
fn spin_cycle_wakes_from_its_spin_down_pause_when_stopped() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
let flag = d.halt_flag();
let stopper = std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(200));
flag.store(true, Ordering::Relaxed);
});
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
stopper.join().expect("stopper thread");
assert!(
matches!(r, Err(Error::Halted)),
"a Stop during the spin-down pause must surface as Halted: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"the pause must be halt-aware, not a blind {SPIN_DOWN_IDLE_SECS}s \
thread::sleep; took {:?}",
t0.elapsed()
);
}
/// A READ(10) that completes with GOOD status but a residual underrun is
/// correctly refused — and must SAY SO.
///
/// The refusal itself already existed and is right: the tail of the buffer
/// still holds stale bytes, so committing them would be silent
/// corruption. What was missing is any trace of it. The sibling `Err(e)`
/// arm warns with lba/count/status; this arm logged NOTHING, so a drive
/// that residual-underruns on GOOD status produced exactly the same
/// journal as a scratched disc — two populations with opposite remedies
/// (replace the drive vs. clean the disc) collapsed into one, with the
/// operator sent after the wrong one.
///
/// Mutation: deleting the `tracing::warn!` leaves no event; logging a
/// different code, or dropping the transferred/expected pair that is the
/// entire signal, fails the field assertions.
#[test]
fn read_logs_a_good_status_short_transfer() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(1024)); // half of one sector
let mut buf = vec![0u8; 2048];
let (r, events) = crate::testlog::capture(|| d.read(7, 1, &mut buf, false));
assert!(
matches!(r, Err(Error::DiscRead { sector: 7, .. })),
"a short transfer stays a failed read: {r:?}"
);
let line = events
.iter()
.find(|e| e.target == "freemkv::drive" && e.field("transferred").is_some())
.unwrap_or_else(|| {
panic!("a silently refused short transfer is the defect; got {events:?}")
});
assert_eq!(line.level, tracing::Level::WARN);
assert_eq!(line.field("lba"), Some("7"));
assert_eq!(line.field("transferred"), Some("1024"));
assert_eq!(
line.field("expected"),
Some("2048"),
"the underrun is only legible as transferred-vs-expected"
);
assert_eq!(
line.field("code"),
Some(crate::error::E_DISC_READ.to_string().as_str())
);
}
// ── report_key / mode_sense / read_buffer empty-vs-some ─────────
#[test]
@@ -2639,10 +2910,9 @@ mod command_tests {
#[test]
fn init_without_unlocker_is_ok_oem_fallback() {
// The test transport's identity matches no registered unlocker, so
// route_unlock returns None. init() must succeed (leaving the drive
// in stock mode for the host-cert handshake), not error — the OEM
// route is the no-match fallback, not a failure.
// No registered unlocker matches, so route_unlock returns None. init() must
// still succeed (stock mode for the host-cert handshake) — no-match is the
// OEM fallback, not a failure.
let mut d = drive_with(vec![]);
assert!(
d.init().is_ok(),
+7 -4
View File
@@ -8,11 +8,14 @@
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against real discs.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
//! that resolves the start cell build on top of this.
//! Current contents: [`vmcmd`] — the VM command decoder; [`nav`] — the
//! First-Play navigation executor that resolves which title the disc's own
//! navigation selects as the feature.
pub mod vmcmd;
pub(crate) mod nav;
pub(crate) mod vmcmd;
pub(crate) use nav::resolve_main_title;
use crate::sector::SectorSource;
+737
View File
@@ -0,0 +1,737 @@
//! DVD-Video navigation executor — the DVD arm of "mimic a real player" title
//! selection (issue #40).
//!
//! A player enters a disc at the **First-Play PGC** (FP_PGC): a short program of
//! VM commands (parsed by [`super::vmcmd`]) that runs before any menu draws and
//! typically dispatches straight to the feature title. This module reads that
//! program out of `VIDEO_TS.IFO` (VMGI), executes its pre-command list on a
//! minimal register machine, and — when the program deterministically reaches a
//! title dispatch (`JumpTT`) — maps the selected VMG title through the title
//! search pointer table (`TT_SRPT`) to its `(VTS number, title-in-set)`
//! coordinates. That is the title a player would land on, so DVD title
//! selection can prefer it over the structural leading-cell heuristic.
//!
//! Contract: this is a strict improvement over the fallback. Any parse error,
//! interactive/menu-only entry, or non-convergence yields `None`, and the caller
//! keeps today's behaviour. Every table count and offset is attacker-controlled,
//! so all arithmetic is checked and all reads are bounds-guarded — the entry
//! point [`resolve_from_vmg`] never panics on any input (exercised by the
//! never-panic harness).
use super::vmcmd::{self, Compare, Instr};
use crate::consts::SECTOR_BYTES;
use crate::sector::SectorSource;
use crate::udf::UdfFs;
// ── VMGI management-table byte offsets into VIDEO_TS.IFO (DVD-Video spec) ─────
// The VMGI_MAT header carries these fixed offsets: per-disc values live at
// disc-constant offsets.
const VMGI_MAGIC: &[u8; 12] = b"DVDVIDEO-VMG";
/// u32 **byte** offset (from the start of VIDEO_TS.IFO) of the First-Play PGC.
const VMGI_FP_PGC_PTR: usize = 0x84;
/// u32 **sector** offset (from the start of VIDEO_TS.IFO) of TT_SRPT.
const VMGI_TT_SRPT_PTR: usize = 0xC4;
/// Within a PGC, the command-table pointer is a u16 at `PGC + 0xE4` giving the
/// command table's byte offset relative to the PGC start (DVD-Video PGC layout;
/// the same layout `ifo::parse_pgc` reads for cell/program tables).
const PGC_CMD_TBL_PTR: usize = 0xE4;
/// Upper bound on commands executed before declaring non-convergence. A
/// conformant First-Play routine reaches a title in a handful of steps; the cap
/// stops a crafted command list (self-`Goto`, mutual jumps) from spinning.
const STEP_BUDGET: usize = 1024;
/// Maximum commands honoured in one PGC command list. The DVD-Video VM caps a
/// list at 128 pre / 128 post / 128 cell commands; the on-disc count is an
/// untrusted u16, so it is clamped to this format maximum.
const MAX_CMDS: usize = 128;
/// Maximum TT_SRPT entries honoured — the DVD-Video 99-title format maximum
/// (mirrors `ifo::MAX_TT_SRPT_TITLES`; the on-disc count is an untrusted u16).
const MAX_TT_SRPT_TITLES: usize = 99;
/// The title the First-Play navigation selects, in VMG coordinates.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResolvedTitle {
/// 1-based VMG title number (the TT_SRPT index the `JumpTT` named).
pub title: u8,
/// 1-based VTS (title set) number the title lives in.
pub vtsn: u8,
/// 1-based title-within-set number (TT_SRPT `VTS_TTN`).
pub vts_ttn: u8,
}
// ── Bounds-guarded big-endian reads (return None past the end) ────────────────
#[inline]
fn u16_at(b: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*b.get(o)?, *b.get(o.checked_add(1)?)?]))
}
#[inline]
fn u32_at(b: &[u8], o: usize) -> Option<u32> {
Some(u32::from_be_bytes([
*b.get(o)?,
*b.get(o.checked_add(1)?)?,
*b.get(o.checked_add(2)?)?,
*b.get(o.checked_add(3)?)?,
]))
}
/// A register operand index names a system parameter (SPRM) when it carries the
/// `0x80` bias — matching [`Vm::reg`]'s split of GPRM (`< 128`) vs SPRM
/// (`>= 128`). Only SPRM reads are session-specific and therefore taint a
/// compare; GPRM reads do not.
#[inline]
fn is_sprm(idx: u8) -> bool {
idx >= 128
}
// ── Minimal navigation register machine ──────────────────────────────────────
/// The subset of the DVD-Video VM register file the First-Play resolver needs:
/// 16 general parameters (GPRMs) and the system parameters (SPRMs). Execution
/// starts from a cold machine (all zero) — the same starting point a player has
/// before any user interaction. The SPRMs a First-Play routine might branch on
/// (region, parental level, language) are player-/session-specific and unknown
/// at scan time; leaving them zero means the resolver only commits to a title
/// when the program reaches one *without* depending on them (an unconditional
/// dispatch, or a conditional whose zero-register path a player also takes).
/// When a branch genuinely depends on an unknown SPRM the routine is treated as
/// undecidable and the caller falls back — see the module contract.
struct Vm {
gprm: [u16; 16],
sprm: [u16; 24],
/// Per-GPRM taint: set when the register holds a value derived from a system
/// parameter (SPRM). An SPRM read taints a compare directly ([`is_sprm`]);
/// this bit tracks an SPRM *laundered through* a GPRM store so a later
/// GPRM-only compare that reads it is still recognised as undecidable.
gprm_tainted: [bool; 16],
}
impl Vm {
fn new() -> Self {
Self {
gprm: [0; 16],
sprm: [0; 24],
gprm_tainted: [false; 16],
}
}
/// Whether a register *operand read* is session-specific: an SPRM read is
/// always undecidable at scan time; a GPRM read is undecidable only when the
/// register currently holds an SPRM-derived (tainted) value.
fn reg_tainted(&self, idx: u8) -> bool {
if is_sprm(idx) {
true
} else {
self.gprm_tainted[(idx & 0x0F) as usize]
}
}
/// Mark a GPRM's post-state as unknown. Used when a store to it was guarded
/// by an undecidable (tainted) predicate: whether the write happened is
/// itself unknown, so the destination is conservatively tainted.
fn taint_gprm(&mut self, reg: u8) {
self.gprm_tainted[(reg & 0x0F) as usize] = true;
}
/// Read a register operand. Per the VM command model, indices `< 16` are
/// GPRMs and `>= 128` are SPRMs (0-23 after the `0x80` bias); anything out
/// of range reads as 0 rather than panicking.
fn reg(&self, idx: u8) -> u16 {
if idx >= 128 {
self.sprm.get((idx - 128) as usize).copied().unwrap_or(0)
} else {
self.gprm.get((idx & 0x0F) as usize).copied().unwrap_or(0)
}
}
/// Evaluate a compare predicate (op codes per [`Compare`]). Returns the
/// predicate result together with a *taint* flag that is set when either
/// operand read a system parameter (SPRM). An SPRM's value is
/// player-/session-specific (region, parental level, language) and unknown
/// at scan time, so a branch decision that depends on one is undecidable —
/// the caller abstains rather than commit to the cold-power-on arm. A GPRM
/// read taints only when that register was itself loaded from an SPRM
/// (tracked by [`Vm::gprm_tainted`]); a GPRM holding a program-established
/// value does not taint.
fn eval(&self, c: &Compare) -> (bool, bool) {
let mut tainted = self.reg_tainted(c.lhs_reg);
let l = self.reg(c.lhs_reg);
let r = if c.immediate {
c.imm
} else {
tainted |= self.reg_tainted(c.rhs_reg);
self.reg(c.rhs_reg)
};
let result = match c.op {
1 => (l & r) != 0,
2 => l == r,
3 => l != r,
4 => l >= r,
5 => l > r,
6 => l <= r,
7 => l < r,
_ => false,
};
(result, tainted)
}
/// Apply a `SetGPRM`. Only the arithmetic/logic set-ops that a First-Play
/// routine uses to prepare a dispatch are modelled; unmodelled ops
/// (swap/rnd) leave the register unchanged (best-effort, never panic).
fn set(&mut self, reg: u8, op: u8, immediate: bool, imm: u16, src: u8) {
let idx = (reg & 0x0F) as usize;
let v = if immediate { imm } else { self.reg(src) };
// The value's taint follows its inputs: an immediate is concrete
// (untainted); a register source carries its own taint ([`reg_tainted`]
// covers both a direct SPRM read and a laundered GPRM).
let src_tainted = !immediate && self.reg_tainted(src);
let cur = self.gprm[idx];
let cur_tainted = self.gprm_tainted[idx];
// Each arm sets (new value, new taint). `mov` overwrites with the source's
// taint alone (clearing prior taint); accumulating ops union current and
// source taint. Unmodelled ops leave value and taint intact.
let (nv, nt) = match op {
1 => (v, src_tainted), // mov
3 => (cur.wrapping_add(v), cur_tainted || src_tainted), // add
4 => (cur.wrapping_sub(v), cur_tainted || src_tainted), // sub
5 => (cur.wrapping_mul(v), cur_tainted || src_tainted), // mul
6 => (
cur.checked_div(v).unwrap_or(cur),
cur_tainted || src_tainted,
), // div (÷0 → unchanged)
7 => (
cur.checked_rem(v).unwrap_or(cur),
cur_tainted || src_tainted,
), // mod (÷0 → unchanged)
9 => (cur & v, cur_tainted || src_tainted), // and
10 => (cur | v, cur_tainted || src_tainted), // or
11 => (cur ^ v, cur_tainted || src_tainted), // xor
_ => (cur, cur_tainted), // swap/rnd/unmodelled
};
self.gprm[idx] = nv;
self.gprm_tainted[idx] = nt;
}
}
// ── VMGI structure readers ───────────────────────────────────────────────────
/// Extract the First-Play PGC pre-command list from VMGI bytes. Returns `None`
/// when there is no First-Play PGC, no command table, or the table is out of
/// bounds — all of which mean "nothing to execute" → fall back.
fn fp_pre_commands(vmg: &[u8]) -> Option<Vec<[u8; 8]>> {
let pgc = u32_at(vmg, VMGI_FP_PGC_PTR)? as usize;
if pgc == 0 {
return None; // no First-Play PGC on this disc
}
let cmd_ptr = u16_at(vmg, pgc.checked_add(PGC_CMD_TBL_PTR)?)? as usize;
if cmd_ptr == 0 {
return None; // First-Play PGC carries no command table
}
let tbl = pgc.checked_add(cmd_ptr)?;
// Command table header: nr_of_pre (u16) at +0, then pre-commands at +8.
let nr_pre = (u16_at(vmg, tbl)? as usize).min(MAX_CMDS);
let base = tbl.checked_add(8)?;
let mut cmds = Vec::with_capacity(nr_pre);
for i in 0..nr_pre {
let o = base.checked_add(i.checked_mul(8)?)?;
if o.checked_add(8)? > vmg.len() {
break; // truncated table — execute what parsed
}
let mut c = [0u8; 8];
c.copy_from_slice(&vmg[o..o + 8]);
cmds.push(c);
}
Some(cmds)
}
/// Parse TT_SRPT into a per-title `(vtsn, vts_ttn)` list indexed by 1-based VMG
/// title number. This is the mapping `JumpTT` needs: a title number → the VTS
/// title-set coordinates that identify the corresponding scanned title.
fn tt_srpt_titles(vmg: &[u8]) -> Option<Vec<(u8, u8)>> {
let sector = u32_at(vmg, VMGI_TT_SRPT_PTR)? as usize;
let base = sector.checked_mul(SECTOR_BYTES)?;
let n = (u16_at(vmg, base)? as usize).min(MAX_TT_SRPT_TITLES);
let entries = base.checked_add(8)?;
let mut out = Vec::with_capacity(n);
for i in 0..n {
// Title entries are 12 bytes: VTSN at +6, VTS_TTN at +7.
let e = entries.checked_add(i.checked_mul(12)?)?;
if e.checked_add(12)? > vmg.len() {
break; // truncated — map what parsed
}
out.push((vmg[e + 6], vmg[e + 7]));
}
Some(out)
}
// ── First-Play executor ──────────────────────────────────────────────────────
/// Execute the First-Play pre-command list and return the VMG title number it
/// dispatches to, if it reaches one deterministically. Returns `None` for a
/// program that ends in a menu/sub-domain, links within the menu domain, exits,
/// or does not converge inside the step budget — every case the caller falls
/// back on.
fn run_first_play(cmds: &[[u8; 8]]) -> Option<u8> {
let mut vm = Vm::new();
let mut pc = 0usize;
let mut budget = STEP_BUDGET;
while pc < cmds.len() {
if budget == 0 {
return None; // non-convergence guard
}
budget -= 1;
let cmd = vmcmd::decode(&cmds[pc]);
// A guarded command's predicate decides whether its instruction runs. `taken`
// is the predicate result; `tainted` marks it undecidable (read a system
// parameter, directly or via GPRM). No predicate = always runs, never tainted.
let (taken, tainted) = match cmd.compare {
Some(cmp) => vm.eval(&cmp),
None => (true, false),
};
// A decidable false predicate skips this line — no effect, taint
// unchanged. When the predicate is tainted we cannot decide `taken`, so
// we do not skip; the per-instruction handling below is conservative.
if !tainted && !taken {
pc += 1;
continue;
}
match cmd.instr {
// Title dispatch — a control transfer, the answer we are looking for.
// Abstain only when an undecidable predicate gates it: committing
// would pick the cold-power-on arm of an SPRM-dependent decision.
Instr::JumpTt { ttn } => {
if tainted {
return None;
}
return Some(ttn);
}
// Control flow within the pre list (Goto lines are 1-based) — also a
// control transfer, so an undecidable guard abstains.
Instr::Goto { line } => {
if tainted {
return None;
}
let l = line as usize;
if l == 0 || l > cmds.len() {
return None;
}
pc = l - 1;
continue;
}
// Register prep before dispatch is a store, not a control transfer, so a
// tainted guard does NOT abstain. Undecidable guard -> unknown post-state
// -> taint the destination; otherwise apply the store normally.
Instr::SetGprm {
reg,
op,
immediate,
imm,
src,
} => {
if tainted {
vm.taint_gprm(reg);
} else {
vm.set(reg, op, immediate, imm, src);
}
}
// No-effect (for resolution) instructions: keep executing. SetSystem
// writes an SPRM (never a GPRM in this decoder), and SPRM reads are
// caught at read time, so no GPRM taint is needed here.
Instr::Nop | Instr::SetSystem => {}
// Everything else leaves the deterministically-followable path: Break/Exit
// end the pre list with no title; JumpSS/Link land in a menu or depend on
// an interactive button selection that can't be resolved statically.
_ => return None,
}
pc += 1;
}
None
}
// ── Public entry points ──────────────────────────────────────────────────────
/// Resolve the main-feature title from raw VMGI (`VIDEO_TS.IFO`) bytes.
///
/// Pure and total: any malformed/hostile input returns `None`, never panics.
/// This is the harness entry point.
pub fn resolve_from_vmg(vmg: &[u8]) -> Option<ResolvedTitle> {
if vmg.len() < 0xC8 || &vmg[0..12] != VMGI_MAGIC {
return None;
}
let cmds = fp_pre_commands(vmg)?;
let ttn = run_first_play(&cmds)?;
if ttn == 0 {
return None; // title 0 is not addressable
}
let titles = tt_srpt_titles(vmg)?;
let (vtsn, vts_ttn) = titles.get((ttn as usize) - 1).copied()?;
if vtsn == 0 {
return None; // invalid TT_SRPT entry
}
Some(ResolvedTitle {
title: ttn,
vtsn,
vts_ttn,
})
}
/// Resolve the main-feature title by reading the disc's VMGI and following its
/// First-Play navigation. Returns `None` (→ caller keeps the heuristic) when the
/// IFO cannot be read or the navigation does not deterministically reach a
/// title.
pub fn resolve_main_title(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ResolvedTitle> {
let vmg = udf.read_file(reader, "/VIDEO_TS/VIDEO_TS.IFO").ok()?;
let resolved = resolve_from_vmg(&vmg)?;
tracing::debug!(
target: "freemkv::dvdnav",
title = resolved.title,
vtsn = resolved.vtsn,
vts_ttn = resolved.vts_ttn,
"nav resolved main-feature title from First-Play program"
);
Some(resolved)
}
#[cfg(test)]
mod tests {
use super::*;
// ── Fixture builders ────────────────────────────────────────────────────
/// Assemble a VMGI (`VIDEO_TS.IFO`) image with a First-Play PGC whose
/// pre-command list is `pre`, and a TT_SRPT at sector `tt_srpt_sector`
/// mapping each title to `(vtsn, vts_ttn)`.
///
/// Layout: the fixed VMGI header (magic + the FP_PGC and TT_SRPT pointers),
/// then the FP_PGC (with a command table holding `pre`), then the TT_SRPT at
/// its sector. Kept minimal but spec-shaped so the same code path a real
/// disc takes is exercised.
fn build_vmgi(pre: &[[u8; 8]], tt_srpt_sector: u32, titles: &[(u8, u8)]) -> Vec<u8> {
// Place the FP_PGC at a fixed byte offset past the header.
let fp_pgc_off: u32 = 0x400;
// Command table sits 0x100 bytes into the PGC.
let cmd_tbl_rel: u16 = 0x100;
let mut v = vec![0u8; 0x800];
v[0..12].copy_from_slice(VMGI_MAGIC);
v[VMGI_FP_PGC_PTR..VMGI_FP_PGC_PTR + 4].copy_from_slice(&fp_pgc_off.to_be_bytes());
v[VMGI_TT_SRPT_PTR..VMGI_TT_SRPT_PTR + 4].copy_from_slice(&tt_srpt_sector.to_be_bytes());
// FP_PGC: command-table pointer at +0xE4.
let pgc = fp_pgc_off as usize;
v[pgc + PGC_CMD_TBL_PTR..pgc + PGC_CMD_TBL_PTR + 2]
.copy_from_slice(&cmd_tbl_rel.to_be_bytes());
// Command table: nr_of_pre at +0, pre-commands from +8.
let tbl = pgc + cmd_tbl_rel as usize;
v[tbl..tbl + 2].copy_from_slice(&(pre.len() as u16).to_be_bytes());
for (i, c) in pre.iter().enumerate() {
let o = tbl + 8 + i * 8;
v[o..o + 8].copy_from_slice(c);
}
// TT_SRPT at its sector: count at +0, 12-byte entries from +8.
let base = tt_srpt_sector as usize * SECTOR_BYTES;
let need = base + 8 + titles.len() * 12;
if v.len() < need {
v.resize(need, 0);
}
v[base..base + 2].copy_from_slice(&(titles.len() as u16).to_be_bytes());
for (i, &(vtsn, vts_ttn)) in titles.iter().enumerate() {
let e = base + 8 + i * 12;
v[e + 6] = vtsn;
v[e + 7] = vts_ttn;
}
v
}
fn h(s: &str) -> [u8; 8] {
let v: Vec<u8> = (0..8)
.map(|i| u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).unwrap())
.collect();
v.try_into().unwrap()
}
// ── Executor: convergence to a title ────────────────────────────────────
/// Unconditional-dispatch shape: First-Play is an unconditional `JumpTT 1`,
/// and TT_SRPT maps title 1 to the feature title set (here VTS_02, title 1).
/// The resolver must return that title.
#[test]
fn first_play_jumptt_1_resolves_to_its_title_set() {
// 3002...0001 = JumpTT ttn=1.
let vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1), (3, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// A First-Play that sets up a register and then unconditionally dispatches
/// still resolves (the SetGPRM is a no-op for an unconditional JumpTT).
#[test]
fn setgprm_then_jumptt_resolves() {
let vmgi = build_vmgi(
&[h("7100000603e80000"), h("3002000000020000")],
1,
&[(1, 1), (4, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 2,
vtsn: 4,
vts_ttn: 1
})
);
}
/// A conditional dispatch whose predicate is true on the cold (zero)
/// machine is taken. `if g0 == 0 -> JumpTT 1` with g0 defaulting to 0.
#[test]
fn conditional_jumptt_true_on_cold_machine_is_taken() {
// 30 22: jump with EQ compare (op=2), operands are registers b6/b7 both
// 0 (g0 == g0) → true; JumpTT ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000010000")], 1, &[(5, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 5,
vts_ttn: 1
})
);
}
/// A conditional dispatch that is false falls through to the next line's
/// unconditional dispatch.
#[test]
fn false_conditional_falls_through_to_next_dispatch() {
// line 1: 30 32 with GT compare (op=3 = !=) g0(b6=1?) ... build a clearly
// false compare: if g0 != g0 -> JumpTT 9 (false, skipped);
// line 2: unconditional JumpTT 1.
let vmgi = build_vmgi(
&[h("3032000000090000"), h("3002000000010000")],
1,
&[(7, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 7,
vts_ttn: 1
})
);
}
// ── Executor: abstention (→ caller falls back) ──────────────────────────
/// Menu-entry shape: First-Play jumps into a VTS menu (`JumpSS VTSM root`).
/// That is an interactive menu, not a static dispatch — the resolver must
/// abstain so the caller keeps the leading-cell heuristic.
#[test]
fn first_play_into_menu_abstains() {
// 3006...0183 = JumpSS VTSM root.
let vmgi = build_vmgi(&[h("3006000101830000")], 1, &[(3, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A dispatch gated on a system parameter (SPRM) is undecidable at scan
/// time — the resolver must abstain rather than commit to the cold-power-on
/// arm. Here `if SPRM0 == g0 -> JumpTT 1` is *true* on the cold machine
/// (both read 0), so without SPRM taint-tracking the resolver would wrongly
/// return title 1; with it, the SPRM read taints the branch and it abstains.
#[test]
fn sprm_gated_dispatch_abstains() {
// 30 22: jump with EQ compare (op=2); if_v2 operands are registers
// b6/b7. b6=0x80 = SPRM0 (session-specific), b7=0x00 = g0. ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000018000")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// The mirror of the SPRM case: a compare that reads only GPRMs is decidable
/// and must NOT over-abstain. `if g0 == g1 -> JumpTT 1` is true on the cold
/// machine and resolves to title 1 (no SPRM operand → no taint).
#[test]
fn gprm_only_compare_does_not_taint() {
// 30 22: jump EQ; b6=0x00 = g0, b7=0x01 = g1 (both 0 on the cold
// machine → equal). ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000010001")], 1, &[(6, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 6,
vts_ttn: 1
})
);
}
/// Defect-1 regression (over-abstention): a store guarded by an SPRM-tainted
/// compare does NOT gate a control transfer, so it must not abstain. Here
/// `if SPRM0 == g0 -> g0 = 5` (a guarded store) is followed by an
/// UNCONDITIONAL `JumpTT 1`. The tainted predicate only decides whether the
/// store runs; the dispatch that follows is SPRM-independent, so the resolver
/// must resolve to title 1.
#[test]
fn sprm_guarded_store_then_unconditional_dispatch_resolves() {
// line 0: 71 20 | lhs=b2=0x80 (SPRM0), cmp EQ(op=2, register), rhs=b7=0
// (g0) → tainted guard; SetGPRM g0 = imm(bytes4-5)=5 (mov).
// line 1: unconditional JumpTT 1.
let vmgi = build_vmgi(
&[h("7120800000050000"), h("3002000000010000")],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// Defect-2 regression (under-abstention): an SPRM laundered through a GPRM
/// must still taint a later branch. `SetGPRM g0 = SPRM20` (register source,
/// index >= 128) followed by `if g0 == g1 -> JumpTT 1` — the compare reads
/// only GPRMs, but g0 now holds an SPRM-derived value, so the decision is
/// undecidable and the resolver must abstain.
#[test]
fn sprm_laundered_through_gprm_abstains() {
// line 0: 61 00 | SetGPRM g0 = SPRM20 (mov, register src=b5=0x94).
// line 1: 30 22 | if g0 == g1 -> JumpTT 1 (both 0 on cold machine).
let vmgi = build_vmgi(
&[h("6100000000940000"), h("3022000000010001")],
1,
&[(2, 1)],
);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// Guard: an unconditional immediate store CLEARS a prior taint (no sticky
/// latch). Taint g0 from an SPRM, overwrite it with an immediate, then branch
/// on g0 — the branch is now decidable and must resolve, not abstain.
#[test]
fn unconditional_immediate_store_clears_taint() {
// line 0: SetGPRM g0 = SPRM20 (taints g0).
// line 1: SetGPRM g0 = imm 0 (mov, immediate) → clears taint, g0 = 0.
// line 2: if g0 == g1 -> JumpTT 1 (both 0) → resolves.
let vmgi = build_vmgi(
&[
h("6100000000940000"),
h("7100000000000000"),
h("3022000000010001"),
],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// Guard: copying an UNtainted GPRM does not taint the destination. `g0 = g1`
/// (register source, g1 untainted) then `if g0 == g2 -> JumpTT 1` must
/// resolve (no SPRM ever entered g0).
#[test]
fn gprm_copy_of_untainted_gprm_does_not_taint() {
// line 0: 61 00 | SetGPRM g0 = g1 (mov, register src=b5=1).
// line 1: 30 22 | if g0 == g2 -> JumpTT 1 (both 0) → resolves.
let vmgi = build_vmgi(
&[h("6100000000010000"), h("3022000000010002")],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// An empty First-Play pre list selects no title.
#[test]
fn empty_first_play_abstains() {
let vmgi = build_vmgi(&[], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A JumpTT whose title number is past the TT_SRPT table resolves to no
/// coordinates (→ abstain) rather than indexing out of bounds.
#[test]
fn jumptt_past_tt_srpt_abstains() {
// JumpTT ttn=9 but TT_SRPT declares only 1 title.
let vmgi = build_vmgi(&[h("3002000000090000")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A self-referential `Goto` cannot spin forever — the step budget stops it
/// and the resolver abstains.
#[test]
fn self_goto_hits_budget_and_abstains() {
// Goto line 1 (0-special sub 1), byte7 = 01.
let vmgi = build_vmgi(&[h("0001000000000001")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
// ── Robustness ──────────────────────────────────────────────────────────
#[test]
fn bad_magic_abstains() {
let mut vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
vmgi[0] = b'X';
assert_eq!(resolve_from_vmg(&vmgi), None);
}
#[test]
fn truncated_and_short_inputs_never_panic() {
// Below the header minimum, and a valid header truncated at every length.
assert_eq!(resolve_from_vmg(&[]), None);
assert_eq!(resolve_from_vmg(VMGI_MAGIC), None);
let full = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
for len in 0..full.len() {
let _ = resolve_from_vmg(&full[..len]);
}
}
/// A TT_SRPT pointer that would overflow when scaled to a byte offset must
/// be rejected, not wrap. Build a normal fixture, then overwrite just the
/// TT_SRPT sector pointer with the extreme value (building a real table at
/// that sector would allocate terabytes in the fixture, not the resolver).
#[test]
fn overflowing_tt_srpt_sector_abstains() {
let mut vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
vmgi[VMGI_TT_SRPT_PTR..VMGI_TT_SRPT_PTR + 4].copy_from_slice(&u32::MAX.to_be_bytes());
assert_eq!(resolve_from_vmg(&vmgi), None);
}
}
+35 -22
View File
@@ -159,13 +159,9 @@ fn be16(b: &[u8; 8], o: usize) -> u16 {
((b[o] as u16) << 8) | b[o + 1] as u16
}
// Compare-operand layouts ("if_version"s) per the DVD-Video VM. The op
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
// v1 (special + link): lhs reg = b[3]; rhs imm = bytes4-5 / rhs reg = b[4].
// v2 (jump + system-set): lhs reg = b[6]; rhs reg = b[7] (registers only).
// v3 (set-GPRM): lhs reg = b[2]; rhs imm = bytes6-7 / rhs reg = b[6].
// Compare-operand layouts ("if_version"s): op nibble = byte1 bits 6-4, imm
// flag = byte1 bit 7; offsets differ by family: v1(special/link) lhs=b[3],
// rhs=b[4:6]/b[5]; v2(jump/sys-set) lhs=b[6],rhs=b[7]; v3(set-GPRM) lhs=b[2],rhs=b[6:8]/b[7].
fn if_v1(b: &[u8; 8]) -> Option<Compare> {
let op = (b[1] >> 4) & 7;
(op != 0).then(|| Compare {
@@ -173,7 +169,7 @@ fn if_v1(b: &[u8; 8]) -> Option<Compare> {
lhs_reg: b[3],
immediate: b[1] >> 7 != 0,
imm: be16(b, 4),
rhs_reg: b[4],
rhs_reg: b[5],
})
}
fn if_v2(b: &[u8; 8]) -> Option<Compare> {
@@ -193,7 +189,7 @@ fn if_v3(b: &[u8; 8]) -> Option<Compare> {
lhs_reg: b[2],
immediate: b[1] >> 7 != 0,
imm: be16(b, 6),
rhs_reg: b[6],
rhs_reg: b[7],
})
}
@@ -295,16 +291,16 @@ mod tests {
v.try_into().unwrap()
}
// KATs taken from the real SOTL / Greenland discs (decoded in the PoC).
// KATs taken from real-world First-Play command streams.
#[test]
fn greenland_first_play_is_jumptt_1() {
fn first_play_jumptt_1_decodes() {
let c = decode(&h("3002000000010000"));
assert_eq!(c.instr, Instr::JumpTt { ttn: 1 });
assert!(c.compare.is_none());
}
#[test]
fn sotl_first_play_is_jumpss_vtsm_root() {
fn first_play_jumpss_vtsm_root_decodes() {
// 30 06 ... byte5=0x83 -> sub 2 (VTSM), vts=byte4=1, menu=byte5&0xF=3 (root)
let c = decode(&h("3006000101830000"));
assert_eq!(
@@ -318,8 +314,9 @@ mod tests {
}
#[test]
fn sotl_title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (cell 2 = the 5:02 start)
fn title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (the cell where the
// feature begins).
let c = decode(&h("20a6000000020002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
@@ -330,7 +327,7 @@ mod tests {
}
#[test]
fn sotl_root_button_is_linkpgcn_37() {
fn root_button_is_linkpgcn_37() {
assert_eq!(
decode(&h("2004000000000025")).instr,
Instr::LinkPgcn { pgcn: 37 }
@@ -338,7 +335,7 @@ mod tests {
}
#[test]
fn greenland_scene_button_is_linkpgn() {
fn scene_button_is_linkpgn() {
assert_eq!(
decode(&h("2006000000001401")).instr,
Instr::LinkPgn { pgn: 1 }
@@ -354,7 +351,7 @@ mod tests {
#[test]
fn setgprm_immediate_mov() {
// SOTL First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
// First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
match decode(&h("7100000603e80000")).instr {
Instr::SetGprm {
reg,
@@ -382,16 +379,32 @@ mod tests {
);
}
// if_version_1 register compare: rhs register is byte4 (not byte5).
// if_version_1 register compare: the register operand is the LOW byte of the
// reg-or-immediate field (byte5), NOT the high byte (byte4). Distinct values
// in byte4 (0xAA) vs byte5 (0x05) pin the field down.
#[test]
fn link_register_compare_rhs_is_byte4() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN
let c = decode(&h("2026000304000002"));
fn link_register_compare_rhs_is_low_byte() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN (pgn=byte7=2)
let c = decode(&h("20260003aa050002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 3);
assert_eq!(cmp.rhs_reg, 4);
assert_eq!(cmp.rhs_reg, 5); // low byte of bytes4-5, not the 0xAA high byte
}
// if_version_3 (set-GPRM) register compare: the register operand is the LOW
// byte of the reg-or-immediate field (byte7), NOT the high byte (byte6).
// Distinct values in byte6 (0xBB) vs byte7 (0x07) pin the field down.
#[test]
fn setgprm_register_compare_rhs_is_low_byte() {
// 71 20: set-GPRM (type 3), cmp=EQ(2), dircmp=0(register); lhs=byte2=2.
let c = decode(&h("712002000000bb07"));
let cmp = c.compare.expect("conditional");
assert_eq!(cmp.op, 2);
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 2);
assert_eq!(cmp.rhs_reg, 7); // low byte of bytes6-7, not the 0xBB high byte
}
// if_version_2 jump compare: both operands are registers in byte6 / byte7.
+250 -166
View File
@@ -45,6 +45,7 @@ pub const E_INVALID_CDB_LENGTH: u16 = 4001;
// I/O (5xxx)
pub const E_IO_ERROR: u16 = 5000;
pub const E_SOURCE_TERMINATED: u16 = 5001;
// Disc format (6xxx)
pub const E_DISC_READ: u16 = 6000;
@@ -63,6 +64,22 @@ pub const E_SELECTION_PID_UNKNOWN: u16 = 6014;
pub const E_UDF_BUFFER_TOO_SMALL: u16 = 6012;
pub const E_UDF_NOT_FILESYSTEM: u16 = 6013;
pub const E_IMAGE_TRUNCATED: u16 = 6015;
pub const E_UDF_AD_CHAIN_TOO_LONG: u16 = 6016;
pub const E_UDF_UNRECORDED_EXTENT: u16 = 6017;
pub const E_UDF_EMBEDDED_DATA: u16 = 6018;
/// A file that EXISTS and whose allocation descriptors resolved without error,
/// yet yields not one usable extent: an empty AD list, or a list every entry of
/// which is zero-length or points at LBA 0. Reported by the HD-DVD clip
/// resolver (`disc::hddvd`).
///
/// It has no [`Error`] variant on purpose. `UdfFs::file_extents` returns
/// `Ok(vec![])` here rather than failing — the emptiness is only a defect in
/// the eye of a caller that needs bytes — so the condition is DETECTED by the
/// caller, not returned to it. The code exists so that detection can be
/// accounted in the log with something other than a neighbouring error's code:
/// logging it as E6017 would file a zero-length AD list as an authoring hole
/// and send whoever triages it at the wrong population.
pub const E_UDF_NO_USABLE_EXTENT: u16 = 6019;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -362,7 +379,7 @@ pub enum Error {
/// fields):
/// - [`Error::is_scsi_transport_failure`] — bail; bridge/transport wedge
/// - [`Error::is_marginal_read`] — drive said this read was marginal; smaller block may recover
/// - [`Error::scsi_sense`] — borrow the sense triple for finer routing ([`ScsiSense::is_medium_error`] etc.)
/// - [`Error::scsi_sense`] — borrow the sense triple for finer routing ([`ScsiSense::is_medium_error`](crate::scsi::ScsiSense::is_medium_error) etc.)
ScsiError {
opcode: u8,
status: u8,
@@ -393,6 +410,18 @@ pub enum Error {
UdfNotFound {
path: String,
},
/// The file's ICB allocation list contains an unrecorded (ECMA-167
/// 4/14.14.1.1 type-1/type-2) extent: space allocated to the file at that
/// location but never written, so its true content there is zeros while
/// the media holds something else.
///
/// Raised by [`crate::udf::UdfFs::file_extents`] because a
/// `(lba, sector_count)` read plan cannot express a hole — reading it
/// splices undefined sectors into the rip as content, and dropping it
/// slides every later extent's byte space.
UdfUnrecordedExtent {
path: String,
},
/// The reader was addressable but the bytes are structurally NOT a UDF
/// filesystem — a deterministic tag/format mismatch (e.g. no Anchor Volume
/// Descriptor Pointer at sector 256, no partition descriptor, no File Set
@@ -405,6 +434,32 @@ pub enum Error {
/// 2048-byte sector. A contract violation on the public reader API —
/// returned instead of panicking on the slice.
UdfBufferTooSmall,
/// A file's allocation-descriptor continuation chain did not end within the
/// hop budget the UDF reader allows.
///
/// The budget exists so a crafted or corrupt disc cannot loop the reader
/// forever. Hitting it is NOT the end of the chain: the extents beyond that
/// point are unknown, so the extent list in hand describes only part of the
/// file. Returning that list would let a caller zero-pad the remainder to
/// the declared size and report a mostly-empty file as a complete
/// extraction, so the read fails instead.
UdfAdChainTooLong,
/// A file's ICB declares its data EMBEDDED inline (ECMA-167 4/14.6.8
/// allocation-descriptor type 3), so it has no out-of-line extents at all.
///
/// Returned only when a caller asked for a read plan over such a file.
/// The bytes in the allocation-descriptor field are then the file's own
/// CONTENT, not descriptors, so decoding them as (length, LBA) pairs
/// manufactures extents out of arbitrary data and points the reader at
/// unrelated sectors — a rip that completes at rc=0 carrying whatever
/// happened to be there. `read_directory` already refuses the same shape
/// for directories; this is the file half of that decision.
///
/// A file that legitimately stores its data this way is tiny (an ICB caps
/// it at well under 2 KiB — the AACS `*.inf` key files are the usual
/// case), and the callers that expect one read it via `read_inline_data`
/// long before extents are ever requested. A stream file cannot be one.
UdfEmbeddedData,
DiscTitleRange {
index: usize,
count: usize,
@@ -733,6 +788,19 @@ pub enum Error {
/// it silently leaves encrypted. The producer surfaces this rather
/// than emit still-encrypted bytes.
ExtentNotUnitAligned,
/// A [`crate::sector::SectorSource`] that feeds its reads from a
/// producer thread has terminated for good — the thread exited after
/// an error or before delivering the extents it was given — so it can
/// never return another byte.
///
/// It exists because the alternative answer is a lie: a dead source
/// that reports `Ok(0)` is indistinguishable from end-of-stream, and
/// `DiscStream::fill_extents` legitimately reads a short count as a
/// skippable hole — zero-filling and advancing over every remaining
/// sector of the title and still returning success. Unlike a bad
/// sector, this condition cannot be retried at a smaller size or
/// skipped past, so every consumer must abort the pass on it.
SourceTerminated,
/// An MPEG-TS packet under construction violated the 188-byte fixed
/// size (over-long adaptation field, overflowing payload, or a
/// short/mis-assembled packet). Indicates a muxer invariant break,
@@ -850,13 +918,17 @@ impl Error {
Error::ScsiError { .. } => E_SCSI_ERROR,
Error::InvalidCdbLength { .. } => E_INVALID_CDB_LENGTH,
Error::IoError { .. } => E_IO_ERROR,
Error::SourceTerminated => E_SOURCE_TERMINATED,
Error::DiscRead { .. } => E_DISC_READ,
Error::Halted => E_HALTED,
Error::MplsParse => E_MPLS_PARSE,
Error::ClpiParse => E_CLPI_PARSE,
Error::UdfNotFound { .. } => E_UDF_NOT_FOUND,
Error::UdfUnrecordedExtent { .. } => E_UDF_UNRECORDED_EXTENT,
Error::UdfNotFilesystem => E_UDF_NOT_FILESYSTEM,
Error::UdfBufferTooSmall => E_UDF_BUFFER_TOO_SMALL,
Error::UdfAdChainTooLong => E_UDF_AD_CHAIN_TOO_LONG,
Error::UdfEmbeddedData => E_UDF_EMBEDDED_DATA,
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::ShortImageRead { .. } => E_SHORT_IMAGE_READ,
Error::EmptyImage => E_EMPTY_IMAGE,
@@ -1063,6 +1135,7 @@ impl std::fmt::Display for Error {
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::UdfUnrecordedExtent { path } => write!(f, "E{}: {}", self.code(), path),
Error::SeamPlanDroppedMost { dropped, written } => {
write!(f, "E{} {dropped}/{written}", self.code())
}
@@ -1128,22 +1201,26 @@ impl std::error::Error for Error {
impl From<std::io::Error> for Error {
fn from(e: std::io::Error) -> Self {
Error::IoError { source: e }
// If this `io::Error` is ours, unwrap to the original typed `Error` rather
// than re-wrapping as `IoError` (which `is_scsi_transport_failure` reads as
// a bridge wedge) — preserves classification, SCSI status, and sense data.
match e.downcast::<Error>() {
Ok(typed) => typed,
// A genuine OS/`std` error — the `IoError` wrapper is correct here.
Err(io) => Error::IoError { source: io },
}
}
}
impl From<Error> for std::io::Error {
fn from(e: Error) -> Self {
// An `Error::IoError` is just a wrapper around an underlying
// `io::Error` that entered via `From<io::Error> for Error`.
// Round-trip it back unchanged so the original `ErrorKind` and
// raw OS error code survive instead of being flattened to
// `Other` with a stringified message.
// `Error::IoError` just wraps an `io::Error` that entered via
// `From<io::Error> for Error` — round-trip it back unchanged so the
// original `ErrorKind`/OS code survive instead of flattening to `Other`.
if let Error::IoError { source } = e {
return source;
}
let code = e.code();
let msg = e.to_string();
// Map our error categories to io::ErrorKind
let kind = match code {
// Device access-denied semantics map to PermissionDenied;
@@ -1226,12 +1303,9 @@ impl From<Error> for std::io::Error {
// 9027 insufficient space / 9029 write failed: a filesystem-level
// failure, not bad input.
E_DIR_INSUFFICIENT_SPACE | E_DIR_WRITE_FAILED => std::io::ErrorKind::Other,
// dir:// SOURCE gates (9061–9064, 9066–9068): the folder handed in
// cannot be turned into a disc image — a 3D SSIF tree, an
// unsatisfiable VIDEO_TS placement, still-encrypted content, an
// unrecognized tree, one too large to address, a name too long to
// record, or a directory whose fan-out overflows its link count.
// All are properties of the input, decided before a byte is read.
// dir:// SOURCE gates (9061-9064, 9066-9068): the folder can't become a
// disc image (3D SSIF, bad VIDEO_TS placement, encrypted, unrecognized
// tree, too large, name too long, fan-out overflow) — input properties.
E_DIR_IMAGE_SSIF_UNSUPPORTED
| E_DIR_IMAGE_PLACEMENT
| E_DIR_IMAGE_ENCRYPTED
@@ -1244,7 +1318,10 @@ impl From<Error> for std::io::Error {
E_DIR_IMAGE_FILE_CHANGED => std::io::ErrorKind::InvalidData,
_ => std::io::ErrorKind::Other,
};
std::io::Error::new(kind, msg)
// Carry the typed value itself as the payload, not its stringification.
// `Display` is unchanged (same `E<code>[: …]` string), so `error_code` is
// unaffected — but the typed error now SURVIVES and can round-trip back.
std::io::Error::new(kind, e)
}
}
@@ -1263,13 +1340,14 @@ pub type Result<T> = std::result::Result<T, Error>;
/// removing comes back.
///
/// [`From<Error> for io::Error`] is the ONLY path from a typed [`Error`] to an
/// `io::Error` in this crate, and it stringifies (`io::Error::new(kind, msg)`
/// where `msg` is the `Error`'s `E<code>[: …]` [`Display`](std::fmt::Display)
/// string) rather than boxing the typed value — no code path constructs an
/// `io::Error` that still holds a `crate::error::Error` via `get_ref`. So the
/// only recognised shape is the round-tripped `E<code>` message prefix.
/// `io::Error` in this crate. It boxes the typed value as the payload
/// (`io::Error::new(kind, e)`), whose [`Display`](std::fmt::Display) is the
/// same `E<code>[: …]` string the stringifying version produced — so this
/// parse is unaffected, and `From<io::Error> for Error` can additionally
/// `downcast` the payload back to the exact typed error. Errors that did NOT
/// come from this crate carry no `E<code>` prefix and yield `None`.
pub fn error_code(e: &std::io::Error) -> Option<u16> {
// Round-tripped: `From<Error> for io::Error` stringifies as "E<code>[: …]".
// Round-tripped: `From<Error> for io::Error` renders as "E<code>[: …]".
let s = e.to_string();
let digits = s.strip_prefix('E')?;
let end = digits
@@ -1387,17 +1465,30 @@ impl Error {
..
}
) || matches!(
// A failed `ioctl(SG_IO)` (Error::IoError, e.g. ENODEV/EIO on an
// unplugged USB bridge) and a vanished device (Error::DeviceNotFound,
// fd gone) are dead-bus / transport-layer faults too — NOT recoverable
// bad sectors. Treat them as transport failures so sweep / patch /
// fill_extents abort the pass and re-enumerate the bridge instead of
// zero-filling every read against a wedged device.
// A failed ioctl (IoError, e.g. ENODEV/EIO) or vanished device
// (DeviceNotFound) is a dead-bus fault too, not a bad sector — treat
// as transport failure so the pass aborts instead of zero-filling.
self,
Error::IoError { .. } | Error::DeviceNotFound { .. }
)
}
/// True if the read SOURCE itself is gone, as opposed to one range of
/// media being unreadable. Kept separate from
/// [`is_scsi_transport_failure`](Self::is_scsi_transport_failure) —
/// which is about the bus/bridge and drives "power-cycle the drive"
/// advice — because a terminated producer thread is neither a wedged
/// bridge nor a bad sector, and reporting it as SCSI status 0xFF would
/// be a fabricated status byte.
///
/// What it shares with a transport failure is the only thing the read
/// loops need to know: retrying smaller or skipping ahead cannot
/// recover anything, so the pass must abort rather than fabricate
/// zeros for the rest of the title.
pub fn is_source_terminated(&self) -> bool {
matches!(self, Error::SourceTerminated)
}
/// True if this error indicates bridge degradation — the SCSI status
/// is neither GOOD (0x00), CHECK CONDITION (0x02), nor transport failure
/// (0xFF). Observed on the Initio INIC-1618L USB bridge preceding a full
@@ -1450,24 +1541,17 @@ mod tests {
#[test]
fn is_skippable_title_stub_excludes_malformed_mkv_input() {
// The two skippable per-title codes, round-tripped through io::Error
// exactly as `mux_stream` returns them. `MkvInvalid` now means ONLY the
// no-muxable-frames stub (the driver's headers-never-resolved gate and the
// MKV muxer's zero-frame `finish()` guard) — the meaning the doc on this
// predicate has always described.
// The two skippable per-title codes, round-tripped through io::Error as
// `mux_stream` returns them. `MkvInvalid` now means ONLY the no-muxable-
// frames stub, matching this predicate's documented meaning.
let mkv: std::io::Error = Error::MkvInvalid.into();
let css: std::io::Error = Error::CssKeyMissing.into();
assert!(is_skippable_title_stub(&mkv));
assert!(is_skippable_title_stub(&css));
// A malformed / truncated `mkv://` SOURCE is a FAILURE, not a stub. Every
// read-path rejection in `mux::mkvstream` and `mux::ebml`'s read
// primitives used to be raised as `MkvInvalid`, so a bad VINT, a cluster
// timestamp past i64::MAX or a BlockGroup child overrunning its group all
// landed in the skippable set above: an all-titles rip would pass silently
// over a corrupt input and exit reporting success. Reverting
// `Error::MkvSourceInvalid` back to `Error::MkvInvalid` at those raise
// sites turns this assertion red.
// A malformed/truncated mkv:// SOURCE is a FAILURE, not a stub. It used to
// be raised as `MkvInvalid`, so a corrupt source landed in the skippable
// set and an all-titles rip silently passed over it, exiting successfully.
let corrupt: std::io::Error = Error::MkvSourceInvalid.into();
assert!(
!is_skippable_title_stub(&corrupt),
@@ -1490,11 +1574,9 @@ mod tests {
let plain = std::io::Error::from(std::io::ErrorKind::BrokenPipe);
assert!(!is_skippable_title_stub(&plain));
// A header-buffer cap overflow is a REAL title whose codec init data
// never resolved — hundreds of MiB of frames, not an empty nav/menu
// stub. It used to be reported as `MkvInvalid`, which lands in the
// skippable set above, so an all-titles rip dropped a main feature and
// still exited successfully. It must have its own, non-skippable code.
// A header-buffer cap overflow is a REAL title whose codec init data never
// resolved, not an empty stub. Reported as `MkvInvalid` it would land in
// the skippable set, so an all-titles rip would drop a main feature silently.
let cap: std::io::Error = Error::MuxHeaderBufferExceeded {
bytes: 512 * 1024 * 1024 + 1,
}
@@ -1671,10 +1753,9 @@ mod tests {
e,
s
);
// Crude English filter — `Display` should never emit ASCII words
// longer than 4 chars (codes/paths/identifiers like `/dev/sg4`,
// `renesas`, `freebsd` all pass; "exclusive access denied" would
// not).
// Crude English filter — `Display` should never emit ASCII words longer
// than 4 chars (identifiers like `/dev/sg4`, `renesas` pass;
// "exclusive access denied" would not).
for word in s.split(|c: char| !c.is_ascii_alphabetic()) {
assert!(
word.len() <= 8,
@@ -1804,126 +1885,129 @@ mod tests {
// ── New comprehensive tests ────────────────────────────────────────────────
/// Every `pub const E_*` code declared in this file, as
/// `(name, value)`, PARSED OUT OF THE SOURCE at compile time.
///
/// WHY THIS IS PARSED AND NOT LISTED. The uniqueness test below used to
/// carry a hand-maintained `vec![]` of the constants, with a doc comment
/// claiming it "pins all code assignments". It did not. At the time this
/// was written the file declared **127** `pub const E_*` and the vector
/// named **109** of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
/// `E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`,
/// `E_DIR_NAME_COLLISION`, `E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`,
/// `E_DIR_RAW_REJECTED`, `E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`,
/// `E_DRIVE_INQUIRY_SHORT`, `E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`,
/// `E_SEAM_PLAN_DROPPED_MOST`, `E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`,
/// `E_SOURCE_TERMINATED`, `E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were
/// outside the guarantee entirely, so a new variant colliding with any of
/// them passed green. Worse, an earlier audit READ that doc comment and
/// trusted it while assigning new codes.
///
/// The defect is not the eighteen omissions, it is that the list is
/// hand-maintained at all: adding a constant and forgetting the vector is
/// a silent no-op, which is the definition of a guarantee that decays.
/// Deriving the list from the declarations makes forgetting impossible —
/// the only way to escape the check is to stop declaring the constant.
///
/// `include_str!` of this very file is the cheapest seam that does that:
/// no `build.rs`, no proc macro, no dependency, and — being `#[cfg(test)]`
/// — not one byte of the source embedded in a release build.
fn declared_error_codes() -> Vec<(&'static str, u16)> {
const SRC: &str = include_str!("error.rs");
SRC.lines()
.filter_map(|line| {
// Only real declarations at column 0. A retired code is
// recorded as `// 2001: burned/retired`, which carries no `pub
// const` and is therefore correctly invisible here.
let decl = line.strip_prefix("pub const ")?;
let (name, value) = decl.split_once(": u16 = ")?;
if !name.starts_with("E_") {
return None;
}
let value = value.strip_suffix(';')?;
Some((
name,
value
.parse::<u16>()
.unwrap_or_else(|_| panic!("non-literal error code for `{name}`")),
))
})
.collect()
}
/// The parser above must actually FIND the declarations, and read their
/// values correctly. Without this, a `declared_error_codes` that returned
/// an empty vector would make the uniqueness test below pass vacuously —
/// exactly the failure mode (a guarantee that is really a no-op) this
/// whole change exists to remove.
///
/// Mutation: a `strip_prefix` typo, an off-by-one in the name slice, or a
/// parser that drops the last line fails here.
#[test]
fn declared_error_codes_parses_the_declarations_it_claims_to() {
let declared = declared_error_codes();
// Independent count of the declarations, computed a different way
// from the parser under test.
let expected = include_str!("error.rs")
.lines()
.filter(|l| l.starts_with("pub const E_"))
.count();
assert_eq!(
declared.len(),
expected,
"the parser must see every `pub const E_*` line"
);
assert!(
expected >= 120,
"sanity floor: this file declares well over a hundred codes, got {expected}"
);
// Names and values must match the compiled constants, so a parser that
// mis-slices the name or mis-reads the digits cannot pass.
for (name, value) in [
("E_DEVICE_NOT_FOUND", E_DEVICE_NOT_FOUND),
("E_UDF_UNRECORDED_EXTENT", E_UDF_UNRECORDED_EXTENT),
("E_KEYDB_PARSE", E_KEYDB_PARSE),
] {
assert!(
declared.contains(&(name, value)),
"`{name}` = {value} not parsed out of the source; got {:?}",
declared
.iter()
.filter(|(n, _)| *n == name)
.collect::<Vec<_>>()
);
}
// A comment-only retired code must NOT be picked up as a constant.
assert!(
!declared.iter().any(|(n, _)| n.is_empty()),
"no empty names"
);
}
/// Every published error code constant must be unique.
/// This pins all code assignments: a new variant that accidentally reuses
/// an existing code will make this test fail.
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating E_KEYDB_CONNECT) fails here.
///
/// The set is derived from the declarations by
/// [`declared_error_codes`], so — unlike the hand-kept vector this
/// replaced — a newly added constant is covered the moment it is
/// written, with no second edit to remember.
///
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating
/// E_KEYDB_CONNECT) fails here, and so now does the same collision on any
/// of the eighteen codes the old hand-kept list had never heard of.
#[test]
fn all_error_code_constants_are_unique() {
let mut codes = vec![
E_DEVICE_NOT_FOUND,
E_DEVICE_PERMISSION,
E_DEVICE_NOT_READY,
E_DEVICE_RESET_FAILED,
E_SCSI_INTERFACE_UNAVAILABLE,
E_DEVICE_LOCKED,
E_IOKIT_PLUGIN_FAILED,
E_UNSUPPORTED_DRIVE,
E_PROFILE_PARSE,
E_UNSUPPORTED_PLATFORM,
E_PLATFORM_NOT_IMPLEMENTED,
E_UNLOCK_FAILED,
E_SIGNATURE_MISMATCH,
E_SCSI_ERROR,
E_INVALID_CDB_LENGTH,
E_IO_ERROR,
E_DISC_READ,
E_MPLS_PARSE,
E_CLPI_PARSE,
E_UDF_NOT_FOUND,
E_DISC_TITLE_RANGE,
E_IFO_PARSE,
E_MKV_INVALID,
E_NO_STREAMS,
E_HALTED,
E_MAPFILE_INVALID,
E_IMAGE_TRUNCATED,
E_UDF_BUFFER_TOO_SMALL,
E_UDF_NOT_FILESYSTEM,
E_AACS_NO_KEYS,
E_AACS_CERT_SHORT,
E_AACS_AGID_ALLOC,
E_AACS_CERT_REJECTED,
E_AACS_CERT_READ,
E_AACS_CERT_VERIFY,
E_AACS_KEY_READ,
E_AACS_KEY_REJECTED,
E_AACS_KEY_VERIFY,
E_AACS_VID_READ,
E_AACS_VID_MAC,
E_AACS_DATA_KEY,
E_DECRYPT_FAILED,
E_CSS_AUTH_FAILED,
E_AACS_HOST_CERT_REJECTED,
E_AACS_RAW_READ_UNSUPPORTED,
E_AACS_VID_UNAVAILABLE,
E_AACS_MK_UNAVAILABLE,
E_AACS_VUK_NOT_IN_KEYDB,
E_DRIVE_PROFILE_MISSING,
E_VID_CDB_UNAVAILABLE,
E_NO_DISC_KEY,
E_CSS_KEY_MISSING,
E_CSS_NO_DISC_KEY,
E_KEY_SERVICE_UNAVAILABLE,
E_KEY_SERVICE_UNAUTHORIZED,
E_KEY_SERVICE_RATE_LIMITED,
E_AACS_NO_HOST_CERT,
E_AACS_BUS_KEY_UNAVAILABLE,
E_FMTS_KEY_MISSING,
E_KEYDB_CONNECT,
E_KEYDB_HTTP,
E_KEYDB_INVALID,
E_KEYDB_WRITE,
E_KEYDB_PARSE,
E_KEYDB_LOAD,
E_KEYDB_UNSUPPORTED_SCHEME,
E_KEYDB_TOO_MANY_REDIRECTS,
E_STREAM_READ_ONLY,
E_STREAM_WRITE_ONLY,
E_STREAM_URL_INVALID,
E_STREAM_URL_MISSING_PATH,
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MUX_HEADER_BUFFER_EXCEEDED,
E_MKV_LACING_INVALID,
E_MKV_SOURCE_INVALID,
E_MKV_UNENCODABLE,
E_MP4_NO_VIDEO_TRACK,
E_MP4_INVALID,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
E_ISO_TOO_LARGE,
E_NO_METADATA,
E_DISC_URL_NOT_DIRECT,
E_HEVC_PARAM_PARSE,
E_MUX_TRACK_RANGE,
E_FMP4_UNIMPLEMENTED,
E_DEMUX_THREAD_PANICKED,
E_PIPELINE_JOIN_TIMEOUT,
E_PIPELINE_CONSUMER_PANICKED,
E_SWEEP_CONSUMER_GONE,
E_PIPELINE_CONSUMER_GONE,
E_DISC_CAPACITY_OVERFLOW,
E_M2TS_PACKET_MALFORMED,
E_EXTENT_NOT_UNIT_ALIGNED,
E_DISC_CAPACITY_MALFORMED,
E_DIR_IMAGE_SSIF_UNSUPPORTED,
E_DIR_IMAGE_PLACEMENT,
E_DIR_IMAGE_ENCRYPTED,
E_DIR_IMAGE_UNSUPPORTED_TREE,
E_DIR_IMAGE_FILE_CHANGED,
E_DIR_IMAGE_TOO_LARGE,
];
let original_len = codes.len();
codes.sort();
codes.dedup();
assert_eq!(
codes.len(),
original_len,
"duplicate error code constants detected — check error.rs"
let mut by_code: std::collections::BTreeMap<u16, Vec<&str>> =
std::collections::BTreeMap::new();
for (name, value) in declared_error_codes() {
by_code.entry(value).or_default().push(name);
}
let dupes: Vec<_> = by_code
.iter()
.filter(|(_, names)| names.len() > 1)
.collect();
assert!(
dupes.is_empty(),
"duplicate error code constants detected — check error.rs: {dupes:?}"
);
}
+115 -9
View File
@@ -136,11 +136,9 @@ fn sweep_n<F: FnMut(&[u8])>(target: &str, magic: &[u8], n: usize, mut f: F) {
run(target, "magic+noise", s, i, &buf, &mut f);
}
// 3. Structured mutation of a plausible record: a valid magic, then mostly
// zeroes, with a handful of bytes corrupted and a truncation. Length and
// offset fields live in those early bytes, so this is what reaches the
// arithmetic — the offsets, counts and sizes a hostile image would lie
// about.
// 3. Structured mutation: valid magic, mostly zeroes, a handful of bytes
// corrupted plus truncation. Length/offset fields live in those early
// bytes, so this reaches the arithmetic a hostile image would lie about.
let mut rng = Rng::new(s ^ 0x1234_5678_9ABC_DEF0);
for i in 0..n {
let mut buf = vec![0u8; 512];
@@ -180,6 +178,30 @@ fn clpi_parse_never_panics() {
});
}
#[test]
fn bdnav_index_parse_never_panics() {
sweep("bdnav_index", b"INDX", |b| {
let _ = crate::bdnav::index::parse(b);
});
}
#[test]
fn bdnav_mobj_parse_never_panics() {
sweep("bdnav_mobj", b"MOBJ", |b| {
let _ = crate::bdnav::mobj::parse(b);
});
}
#[test]
fn dvd_nav_resolve_never_panics() {
// The DVD First-Play resolver parses VMGI (FP_PGC + TT_SRPT, all
// attacker-controlled) and executes an untrusted command list on a bounded
// VM. Magic is the VMGI signature, so the sweep reaches the executor.
sweep("dvd_nav", b"DVDVIDEO-VMG", |b| {
let _ = crate::dvdnav::nav::resolve_from_vmg(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
@@ -189,12 +211,96 @@ fn udf_name_parse_never_panics() {
});
}
#[test]
fn class_reader_parse_never_panics() {
// Java `.class` files are READ (never executed) to recover BD-J stream
// labels. Every count/length/constant-pool index is attacker-controlled
// disc metadata, so the parser must reject malformed input, never panic.
sweep("class_reader", &[0xCA, 0xFE, 0xBA, 0xBE], |b| {
if let Ok(class) = crate::labels::class_reader::ClassFile::parse(b) {
// Exercise the accessors a label parser drives, so their indexing
// is fuzzed too, not just the structural parse.
let _ = class.this_class_name();
let _ = class.super_class_name();
for m in &class.methods {
let _ = class.member_name(m);
if let Some(code) = m.code(&class.constant_pool) {
for insn in code.instructions() {
let _ = insn.cp_index();
if let Some(idx) = insn.cp_index() {
let _ = class.constant_pool.get(idx);
let _ = class.constant_pool.member_ref(idx);
let _ = class.constant_pool.load_constant_display(idx);
}
}
}
}
}
});
}
#[test]
fn jar_class_walk_never_panics() {
// The jar layer opens a `/BDMV/JAR/<x>.jar` zip and walks every `.class`
// inside, handing each to the class reader (as `deluxe`/`dbp` do). Neither
// the zip open nor the class walk may panic. Magic: `PK\x03\x04`.
sweep("jar", &[0x50, 0x4B, 0x03, 0x04], |b| {
if let Ok(mut archive) = zip::ZipArchive::new(std::io::Cursor::new(b.to_vec())) {
crate::labels::jar::for_each_class(&mut archive, |_name, _class| {});
}
});
}
#[test]
fn deluxe_bytecode_decode_never_panics() {
// Deluxe's BD-J parser runs a symbolic stack machine over `<clinit>`
// bytecode. Fuzzing THROUGH a zip is vacuous (generators can't fake a valid
// end-of-central-directory), so drive the decoder DIRECTLY on a `.class`.
let master = crate::labels::deluxe::MasterEnumTable::from(&[] as &[(
&'static str,
crate::labels::deluxe::MasterEnum,
)]);
sweep("deluxe_decode", &[0xCA, 0xFE, 0xBA, 0xBE], |b| {
if let Ok(class) = crate::labels::class_reader::ClassFile::parse(b) {
let _ = crate::labels::deluxe::decode_binding_class(&class, &master);
}
});
}
#[test]
fn fox_dcx_labels_never_panics() {
// Fox's `dcx.xml` is untrusted disc-authored XML; `labels_from_dcx` scans it
// with byte-offset XML helpers documented as UTF-8-boundary-prone. No magic —
// feed lossy UTF-8 so multi-byte boundaries inside tags/attributes are hit.
sweep("fox_dcx", b"<", |b| {
let text = String::from_utf8_lossy(b);
let _ = crate::labels::fox::labels_from_dcx(&text);
});
}
#[test]
fn bdnav_vm_resolve_never_panics() {
// The HDMV nav VM executes an untrusted MovieObject.bdmv command list.
// Parse sweep bytes as a MovieObject set and run the VM from First-Play —
// the combined parse+execute path the per-parser index/mobj sweeps miss.
use crate::bdnav::index::{Index, PlaybackObj};
sweep("bdnav_vm", b"MOBJ", |b| {
if let Some(mobjs) = crate::bdnav::mobj::parse(b) {
let index = Index {
first_play: PlaybackObj::Hdmv { id_ref: 0 },
top_menu: PlaybackObj::Hdmv { id_ref: 0 },
titles: Vec::new(),
};
let _ = crate::bdnav::vm::resolve(&index, &mobjs, &|_| false);
}
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
// each case is fed to a FRESH demuxer and then a shared one. The shared pass
// is what exercises cross-feed state — a start code split over a boundary,
// a held PES completed by later bytes, the carry-over cap.
// Stateful, unlike the others: the demuxer carries a buffer across feeds.
// Each case is fed to a FRESH demuxer, then a shared one — the shared pass
// exercises cross-feed state: split start codes, held PES, the carry cap.
let mut shared = crate::mux::ps::PsDemuxer::new();
sweep("ps_demux", &[0x00, 0x00, 0x01, 0xBA], |b| {
let mut fresh = crate::mux::ps::PsDemuxer::new();
+13 -23
View File
@@ -17,19 +17,19 @@ use crate::scsi::{DataDirection, ScsiTransport};
/// - MMC-6 §5.3.10 for Firmware Information
#[derive(Debug, Clone)]
pub struct DriveId {
/// T10 VENDOR IDENTIFICATION — INQUIRY bytes [8:16]
/// T10 VENDOR IDENTIFICATION — INQUIRY bytes `[8:16]`
/// SPC-4 §6.4.2
pub vendor_id: String,
/// PRODUCT IDENTIFICATION — INQUIRY bytes [16:32]
/// PRODUCT IDENTIFICATION — INQUIRY bytes `[16:32]`
/// SPC-4 §6.4.2
pub product_id: String,
/// PRODUCT REVISION LEVEL — INQUIRY bytes [32:36]
/// PRODUCT REVISION LEVEL — INQUIRY bytes `[32:36]`
/// SPC-4 §6.4.2
pub product_revision: String,
/// VENDOR SPECIFIC — INQUIRY bytes [36:43]
/// VENDOR SPECIFIC — INQUIRY bytes `[36:43]`
/// SPC-4 §6.4.2
/// Content varies by vendor: firmware type code (MTK), date (Pioneer), etc.
pub vendor_specific: String,
@@ -60,26 +60,18 @@ impl DriveId {
let mut inquiry = vec![0u8; 96];
let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00];
let inq = transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
// `bytes_transferred` is device-reported and untrusted — the same rule
// the two GET CONFIGURATION calls below already apply. It was ignored
// here, and the buffer is pre-zeroed, so a drive answering GOOD with a
// short or empty data phase (a USB-SATA bridge mid-wedge does exactly
// this) decoded to blank identity strings and a byte 0 of 0x00. Every
// platform enumerator gates on `raw_inquiry[0] & 0x1F == OPTICAL`, so
// 0x00 reads as DIRECT ACCESS and the drive silently disappears from
// the device list instead of reporting a failed probe.
// `bytes_transferred` is device-reported and untrusted. Unchecked, a
// GOOD status with a short/empty data phase decoded to blank identity +
// byte0 0x00 — which reads as DIRECT ACCESS, so the drive vanishes silently.
if inq.bytes_transferred < INQUIRY_STANDARD_LEN {
return Err(crate::error::Error::DriveInquiryShort);
}
// Never decode past what the drive actually sent.
inquiry.truncate(inq.bytes_transferred.min(inquiry.len()));
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6.
// Best-effort: 010Ch (Firmware Information) is an optional feature.
// A drive that lacks it may CHECK CONDITION rather than return an
// empty descriptor, so a failure here is treated as feature-absent
// (empty firmware date + empty raw bytes) instead of aborting the
// whole identity probe.
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6. Best-effort: an optional
// feature a drive may lack (CHECK CONDITION), so failure here is
// feature-absent (empty firmware date + raw bytes), not a probe abort.
let mut gc = vec![0u8; 256];
let cdb_gc = [0x46, 0x02, 0x01, 0x0C, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00];
// `bytes_transferred` is device-reported and untrusted; clamp every
@@ -100,11 +92,9 @@ impl DriveId {
Err(_) => (String::new(), Vec::new()),
};
// GET CONFIGURATION Feature 0108h — Serial Number.
// Best-effort, like 010Ch above: the serial-number feature is
// optional, so a drive that lacks it (CHECK CONDITION) or reports
// too few bytes deliberately yields an empty serial rather than
// failing the identity probe.
// GET CONFIGURATION Feature 0108h — Serial Number. Best-effort like
// 010Ch: optional feature, so lacking it (CHECK CONDITION) or too few
// bytes deliberately yields an empty serial rather than failing.
let mut gc_serial = vec![0u8; 256];
let cdb_serial = [0x46, 0x02, 0x01, 0x08, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00];
let serial_number = if let Ok(r) =
+177 -186
View File
@@ -49,7 +49,7 @@ pub struct DvdTitle {
pub cells: Vec<DvdCell>,
/// Chapter start times in seconds (derived from program map + cell times)
pub chapter_times: Vec<f64>,
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cb, Cr].
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cr, Cb].
pub palette: Option<Vec<[u8; 4]>>,
}
@@ -149,8 +149,8 @@ impl DvdTitle {
/// - As a final guard it never returns past the last cell, and never drops
/// when that would leave zero cells.
///
/// For "The Silence of the Lambs" (every feature cell category `0x00`,
/// chapter 1 at 00:00:00) this returns 0 — a no-op — which is the correct
/// For a real disc where every feature cell category is `0x00` and
/// chapter 1 sits at 00:00:00, this returns 0 — a no-op — which is the correct
/// result: the disc's scene-index lives in a separate menu/title PGC, not
/// in leading cells of the feature PGC, so there is nothing to drop here.
pub fn feature_start_cell(&self) -> usize {
@@ -173,7 +173,7 @@ impl DvdTitle {
if idx >= n { 0 } else { idx }
}
/// The feature cells after the leading-cell filter ([`feature_start_cell`]).
/// The feature cells after the leading-cell filter ([`Self::feature_start_cell`]).
pub fn feature_cells(&self) -> &[DvdCell] {
&self.cells[self.feature_start_cell()..]
}
@@ -425,14 +425,14 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
for (&vts_number, titles_info) in &title_set_map {
match parse_vts(reader, udf, vts_number, titles_info) {
Ok(ts) => title_sets.push(ts),
// Halted means the drive itself stopped, not that this title set is a
// placeholder — once set, every later parse_vts call also fails, so
// swallowing it here would silently truncate the scan. Propagate it.
Err(Error::Halted) => return Err(Error::Halted),
Err(e) => {
// Some discs carry placeholder TT_SRPT entries for title sets
// that are not really there, so one failure is not fatal. But
// the failure must not be INVISIBLE: every skipped set is a
// title the user will never see, and swallowing the reason made
// a disc that enumerated 38 titles from one image and 10 from
// another look like a scan difference rather than 28 dropped
// reads.
// Some discs carry placeholder TT_SRPT entries for title sets that
// aren't really there, so one failure isn't fatal — but it must not
// be silent, since swallowing it can hide many dropped titles.
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
@@ -556,25 +556,9 @@ fn parse_vts(
// VTS_PGCIT sector pointer
let pgcit_sector = be_u32(&vts_data, VTS_PGCIT_OFFSET)?;
// First sector of the VTS **Title** VOBS (`vtstt_vobs`, VTSTT_VOBS_OFFSET).
// The cell `first_sector` / `last_sector` values in the title PGCs are
// relative to this. Offset 0xC0 is `vtsm_vobs` — the VTS *menu* VOBS
// (VTS_xx_0.VOB), which on discs with a per-title menu (e.g. a Universal
// "the parental level has been set, press yes" first-play still) holds that
// interactive prompt. Reading the menu base instead prepended the menu VOB
// to the feature and shifted every cell extent back by
// `vtstt_vobs - vtsm_vobs` sectors, so the rip opened on the parental
// prompt instead of the movie. The title content lives at `vtstt_vobs`.
//
// `vtstt_vobs` is a sector address **relative to the start of this VTS_xx_0.IFO
// file**, not an absolute disc LBA. The cell `first_sector`/`last_sector`
// values are in turn relative to `vtstt_vobs`. To turn them into the absolute
// disc LBAs the reader needs, add the IFO file's own on-disc location (from
// the UDF FS). Without this rebase every extent started `ifo_lba` sectors too
// early — for THESILENCEOFTHELAMBS the feature began at LBA 126 (the VMGI /
// VIDEO_TS.VOB main-menu region) instead of 132886 (VTS_03_1.VOB), so the
// first ~4.5 min of muxed video was the disc's main menu before the stream
// drifted into the movie.
// First sector of the VTS **Title** VOBS (`vtstt_vobs`), which PGC cell extents
// are relative to (offset 0xC0/`vtsm_vobs` is the *menu* VOBS, not the movie).
// It's relative to this IFO file, so rebase by the IFO's on-disc LBA (UDF FS).
let vtstt_vobs = be_u32(&vts_data, VTSTT_VOBS_OFFSET)?;
let ifo_lba = udf.file_start_lba(reader, &path)?;
let vob_start_sector = ifo_lba.saturating_add(vtstt_vobs);
@@ -593,11 +577,9 @@ fn parse_vts(
}
audio_streams.push(parse_audio_attr(&vts_data, aoff)?);
}
// Assign each audio stream its on-wire private_stream_1 sub-stream id
// by per-codec ordinal — the same convention DVD authoring uses (AC-3
// 0x80+, DTS 0x88+, LPCM 0xA0+). This is the routing key shared with
// the muxer; per-codec ordinals (not the positional index) are what
// keep mixed-codec titles from colliding.
// Assign each stream its on-wire private_stream_1 id by per-codec ordinal
// (AC-3 0x80+, DTS 0x88+, LPCM 0xA0+), the DVD authoring convention the
// muxer routes on — the positional index would collide in mixed-codec titles.
assign_audio_sub_stream_ids(&mut audio_streams);
// Subtitle streams: count at 0x254 (u16 BE), then 6 bytes each starting at 0x256
@@ -634,12 +616,9 @@ fn parse_vts(
// ── Attribute parsers ───────────────────────────────────────────────────────
// ── VTS_V_ATR byte 0 bitfield layout (DVD-Video spec, MSB first) ──────────
// bits 7-6 mpeg_version | bits 5-4 video_format | bits 3-2 display_aspect
// | bits 1-0 permitted_df
// Naming the positions is the guard against the original bug: video_format is
// bits 5-4, NOT bits 1-0 (those are the pan&scan/letterbox permission). Reading
// the low two bits mis-detected every PAL disc as NTSC → 720x480 not 720x576.
// VTS_V_ATR byte 0 (MSB first): bits 7-6 mpeg_version | bits 5-4 video_format |
// bits 3-2 display_aspect | bits 1-0 permitted_df. video_format is bits 5-4, NOT
// bits 1-0 (pan&scan/letterbox permission) — the earlier bug misread PAL as NTSC.
const V_ATR_VIDEO_FORMAT_SHIFT: u8 = 4;
const V_ATR_ASPECT_SHIFT: u8 = 2;
const V_ATR_FIELD_MASK: u8 = 0x03;
@@ -691,16 +670,23 @@ fn parse_video_attr(data: &[u8]) -> Result<DvdVideoAttr> {
}
/// Parse one audio stream attribute block (8 bytes at `offset`).
fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
/// `pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
/// `parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
/// calls it directly so its language-mapping tests run the real parser over real
/// on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
/// could agree with the muxer while both disagree with the disc. Narrowing this
/// would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
/// that can drift apart, or moving those tests away from the code they exist to
/// pin — both worse than the widened crate-internal visibility, which reaches no
/// public API.
pub(crate) fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let b0 = byte_at(data, offset)?;
let b1 = byte_at(data, offset + 1)?;
let coding_mode = (b0 >> 5) & 0x07;
// DVD-Video audio_coding_mode. Modes 2 and 3 are both MPEG audio Layer II
// (3 adds the MPEG-2 multichannel extension), so both are Codec::Mp2.
// Mode 2 previously mapped to Codec::Mpeg1 — a VIDEO variant, so
// Codec::kind() reported Video and the audio stream was classified and
// handled as video everywhere downstream.
// Modes 2 and 3 are both MPEG audio Layer II (3 adds the multichannel
// extension), so both map to Codec::Mp2. Mode 2 previously mapped to the
// video codec Codec::Mpeg1, so the audio stream was classified as video.
let codec = match coding_mode {
0 => Codec::Ac3,
2 | 3 => Codec::Mp2,
@@ -718,25 +704,9 @@ fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let channels = (b1 & 0x07) + 1; // (channels - 1) in low 3 bits of byte 1
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
{
String::from_utf8_lossy(lang_bytes).to_string()
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
// Try to interpret as printable ASCII
let s: String = lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdAudioAttr {
codec,
@@ -785,9 +755,27 @@ fn assign_audio_sub_stream_ids(streams: &mut [DvdAudioAttr]) {
/// Parse one subtitle stream attribute block (6 bytes at `offset`).
fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdSubtitleAttr { language })
}
/// Decode the raw 2-byte on-disc language code shared by the audio and
/// subtitle attribute blocks. A pair of lowercase a-z bytes is taken
/// verbatim (the ISO 639-1 code the DVD-Video spec puts there); an all-zero
/// pair means unspecified (empty string); anything else falls through to an
/// ASCII-alphanumeric salvage — letters (either case) and digits are kept,
/// everything else (control bytes, punctuation, high bytes from a corrupt or
/// hostile disc) is dropped.
///
/// The salvage is deliberately not narrowed to a-z: whatever survives is only
/// ever a lookup key for [`dvd_lang_to_iso639_2`], which degrades anything it
/// does not recognize to `und`, so a stray `X` or `5` costs nothing and cannot
/// reach an output stream as a language code.
fn parse_raw_dvd_lang_bytes(lang_bytes: &[u8]) -> String {
if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
@@ -796,15 +784,40 @@ fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
let s: String = lang_bytes
lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
.collect()
}
}
Ok(DvdSubtitleAttr { language })
/// Convert a DVD IFO audio/subtitle attribute's language code — ISO 639-1
/// (2 lowercase letters) per the DVD-Video spec, or empty when unspecified —
/// to the ISO 639-2 form every downstream consumer (`disc::AudioStream` /
/// `disc::SubtitleStream::language`, and in turn Matroska's `Language`
/// element per RFC 9559 §12 and the MP4 sink's `mdhd` language) requires.
///
/// Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
/// 639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
/// frozen-1988 language list still puts on disc). The narrower
/// `vocab::menu_lang` table is deliberately NOT used here: it exists for
/// Blu-ray menu-graphic filename tokens and knows only 25 languages, so a
/// Region-2 disc's Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian,
/// Hebrew, Estonian, Latvian, Lithuanian and Icelandic tracks would all fold
/// onto `und` together. DVD streams carry an empty `label`, so the language
/// is the only thing distinguishing one subtitle track from the next — a
/// valid code that is identical for six tracks is worse for the user than the
/// invalid one it replaced. Both tables normalize to ISO 639-2/T, so they
/// agree wherever they overlap.
///
/// An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's
/// own "undetermined" value) — a valid element value — rather than passing
/// through an invalid 2-letter code or an empty string. Never guesses.
fn dvd_lang_to_iso639_2(raw: &str) -> String {
crate::labels::vocab::iso639_1_to_iso639_2(raw)
.unwrap_or("und")
.to_string()
}
// ── PGC parser ──────────────────────────────────────────────────────────────
@@ -825,10 +838,9 @@ fn parse_pgcit(
let entries_start = pgcit_offset + 8;
let mut titles = Vec::new();
// Every `continue` below drops a title the user will never see. None of
// them may be silent: `parse_vmg` already counts and warns per skipped
// title SET, and this function was the remaining place where a disc could
// quietly report fewer titles than it has.
// Every `continue` below drops a title the user will never see and must not
// be silent: `parse_vmg` warns per skipped title SET, but this was the
// remaining place a disc could quietly report fewer titles than it has.
let mut skipped = 0usize;
for &(chapter_count, vts_title_num) in titles_info {
@@ -867,14 +879,9 @@ fn parse_pgcit(
match parse_pgc(data, pgc_abs, chapter_count) {
Ok(title) => titles.push(title),
Err(e) => {
// By design a single unparseable PGC (truncated/corrupt entry,
// authoring-tool quirk) must not lose the whole title list. But
// "not fatal" is not the same as "not worth saying": every PGC
// skipped here is a title the user will never see, and this was
// the only remaining silent one — `parse_vmg` above already
// counts and warns per skipped title SET for exactly this
// reason. A disc quietly reporting fewer titles than it has is
// the failure this release exists to stop.
// A single unparseable PGC (truncated/corrupt entry, authoring quirk)
// must not lose the whole title list, but must not be silent either:
// a disc quietly reporting fewer titles than it has is what this warns.
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
@@ -908,11 +915,8 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
return Err(Error::IfoParse);
}
// PGC layout:
// 0x00-0x01: misc flags
// 0x02: nr_of_programs
// 0x03: nr_of_cells
// 0x04-0x07: playback_time (4 BCD bytes)
// PGC layout: 0x00-0x01 misc flags, 0x02 nr_of_programs,
// 0x03 nr_of_cells, 0x04-0x07 playback_time (4 BCD bytes).
let num_cells = byte_at(data, pgc_offset + 0x03)? as usize;
let time_bytes = sub_slice(data, pgc_offset + 0x04, 4)?;
let duration_secs = bcd_to_secs(time_bytes);
@@ -968,14 +972,9 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
let mut times = Vec::new();
if pgm_map_offset > 0 && nr_of_programs > 0 && cell_playback_offset > 0 {
let pgm_base = pgc_offset + pgm_map_offset;
// Collect per-cell durations as exact timecode FRAME counts. Summing
// frames (integers) and converting once per chapter keeps every mark
// on an exact frame boundary — summing f64 seconds would let rounding
// drift accumulate across a two-hour title.
//
// A cell whose rate flag is unspecified has no frame count, so it
// contributes plain seconds instead; that is tracked separately and
// added on top. In practice a PGC never mixes rates.
// Collect durations as exact FRAME counts, converting once per chapter so
// marks stay on frame boundaries (summing f64 seconds would drift). Cells
// with no rate flag fall back to plain seconds, tracked separately.
let mut cell_frames: Vec<(u64, f64)> = Vec::with_capacity(num_cells);
let mut rate: Option<DvdRate> = None;
let cell_base = pgc_offset + cell_playback_offset;
@@ -1016,7 +1015,7 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
times
};
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cb, Cr]
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cr, Cb]
let palette = if pgc_offset + 0xA4 + 64 <= data.len() {
let mut colors = Vec::with_capacity(16);
for i in 0..16 {
@@ -1065,29 +1064,18 @@ mod tests {
#[test]
fn tt_srpt_title_count_is_capped_and_deduplicated() {
// The TT_SRPT count is an untrusted u16. A ~800 KB crafted IFO declares
// 65535 entries, and each one re-parses a PGC into another full DvdTitle.
//
// The entries are DISTINCT, which matters: with 65535 identical entries
// the de-duplication collapses them to one title on its own and the count
// cap is never what bounds the result — the test would then pass with the
// cap removed entirely, proving only that dedup works. Distinct entries
// defeat dedup, so the cap is the only thing left holding the line.
// De-duplication gets its own assertion below, on its own fixture.
// Both title fields are u8, so vary each within its own range to get
// ~65025 distinct pairs out of the 65535 entries.
// The TT_SRPT count is an untrusted u16. Entries must be DISTINCT: identical
// entries would let dedup alone collapse them to one title, so the cap would
// never be exercised — vary both u8 fields for ~65025 distinct pairs.
let entries: Vec<(u16, u8, u8)> = (0..u16::MAX)
.map(|i| (5u16, (i % 255) as u8 + 1, ((i / 255) % 255) as u8 + 1))
.collect();
let data = tt_srpt_bytes(u16::MAX, &entries);
let map = parse_tt_srpt(&data, 0).expect("well-formed TT_SRPT");
let total: usize = map.values().map(|v| v.len()).sum();
// Asserted against the DVD-Video format maximum as a LITERAL, not against
// MAX_TT_SRPT_TITLES. Comparing a result to the very constant under test
// passes vacuously the moment someone raises that constant, which is the
// most likely regression here — and is the tautology class this audit has
// now found six times. 99 is the format's own ceiling, so this pins the
// cap to the spec rather than to whatever the code currently says.
// Asserted against the DVD-Video max as a LITERAL, not MAX_TT_SRPT_TITLES:
// comparing to the constant under test would pass vacuously if that constant
// were raised. 99 is the format's own ceiling, pinning the cap to the spec.
assert!(
total <= 99,
"TT_SRPT produced {total} titles from a crafted 65535-entry table; \
@@ -1135,10 +1123,8 @@ mod tests {
// 0 hours, 1 minute, 30 seconds, 15 frames of NTSC timecode.
let bcd = [0x00, 0x01, 0x30, 0b11_010101];
let secs = bcd_to_secs(&bcd);
// 0b010101 = 0x15, BCD = 15 frames.
// Timecode 00:01:30:15 is 90*30 + 15 = 2715 frames, and each frame
// lasts 1001/30000 s, so real time is 2715*1001/30000 = 90.5905 s
// (NOT 90.5 s — NTSC timecode runs 0.1% slow against the clock).
// 0b010101 = 0x15 BCD = 15 frames. Timecode 00:01:30:15 = 2715 frames at
// 1001/30000 s each = 90.5905 s (NOT 90.5 s — NTSC runs 0.1% slow).
let expected = 2715.0 * 1001.0 / 30000.0;
assert!((secs - expected).abs() < 1e-9, "got {}", secs);
assert!((secs - 90.5905).abs() < 1e-9, "got {}", secs);
@@ -1395,15 +1381,9 @@ mod tests {
/// bit positions drift, these fail.
#[test]
fn video_attr_absolute_bytes_pin_real_layout() {
// (byte @0x200, expected standard, expected aspect, expected resolution).
// PAL 16:9 anamorphic = mpeg(00) format(01=PAL) aspect(11=16:9) df(00)
// = 0b0001_1100 = 0x1C (e.g. a PAL 16:9 R2 feature disc).
// PAL 4:3 = 0b0001_0000 = 0x10.
// NTSC 16:9 = 0b0000_1100 = 0x0C.
// NTSC 4:3 = 0b0000_0000 = 0x00.
// A real disc also sets mpeg_version=01 (MPEG-2) in bits 7-6, which the
// parser must IGNORE; OR it in (|0x40) to prove it doesn't leak into the
// video_format read.
// (byte @0x200, expected standard, expected aspect, expected resolution):
// PAL 16:9=0x1C, PAL 4:3=0x10, NTSC 16:9=0x0C, NTSC 4:3=0x00. A real disc
// also sets mpeg_version=01 in bits 7-6 (|0x40), which the parser must ignore.
let cases: &[(u8, TvSystem, DvdAspect, Resolution)] = &[
(0x1C, TvSystem::Pal, DvdAspect::R16x9, Resolution::R576i),
(0x10, TvSystem::Pal, DvdAspect::R4x3, Resolution::R576i),
@@ -1420,10 +1400,9 @@ mod tests {
assert_eq!(attr.aspect, aspect, "byte {b0:#04x} → aspect");
assert_eq!(attr.resolution, res, "byte {b0:#04x} → resolution");
}
// Anti-bug anchor: the original bug read the TV system from bits 1-0
// (permitted_df). A PAL byte whose low 2 bits are 0 (0x1C) must NOT be
// misread as NTSC — and a byte with low bits set but format=NTSC
// (0x03 = NTSC, df=11) must stay NTSC, proving the low bits are ignored.
// Anti-bug anchor: the original bug read TV system from bits 1-0
// (permitted_df). A byte with low bits set but format=NTSC (0x03, df=11)
// must stay NTSC here, proving the low bits are ignored.
let mut df = vec![0u8; 0x204];
df[0x200] = 0x03; // format=NTSC(00), df=11
assert_eq!(
@@ -1441,7 +1420,7 @@ mod tests {
data[0] = 0x00;
// b1: bits 2-0=101 (channels-1=5) => 0x05
data[1] = 0x05;
// language "en"
// on-disc language "en" (ISO 639-1) -> parsed as ISO 639-2 "eng"
data[2] = b'e';
data[3] = b'n';
@@ -1449,15 +1428,14 @@ mod tests {
assert_eq!(attr.codec, Codec::Ac3);
assert_eq!(attr.sample_rate, 48000);
assert_eq!(attr.channels, 6);
assert_eq!(attr.language, "en");
assert_eq!(attr.language, "eng");
}
#[test]
fn mixed_codec_sub_stream_ids_are_distinct() {
// A title mixing AC-3, DTS and LPCM: the sub-id low nibble is the
// POSITIONAL audio-stream number (shared across codecs), OR'd with the
// codec base. So idx 1 (DTS) → 0x89, idx 3 (AC-3) → 0x83 — the real
// wire ids the demux routes on. All distinct (positions are unique).
// A title mixing AC-3, DTS and LPCM: sub-id low nibble is the positional
// audio-stream number OR'd with the codec base (idx 1 DTS → 0x89, idx 3
// AC-3 → 0x83) — the real wire ids the demux routes on, all distinct.
let mut streams = vec![
DvdAudioAttr {
codec: Codec::Ac3,
@@ -1579,13 +1557,11 @@ mod tests {
assert_eq!(attr.codec, Codec::Dts);
assert_eq!(attr.sample_rate, 96000);
assert_eq!(attr.channels, 2);
assert_eq!(attr.language, "fr");
assert_eq!(attr.language, "fra");
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the DVD-Video IFO spec
// (DVD-Video IFO format; http://dvd.sourceforge.net).
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests, grounded in the DVD-Video IFO spec
// (http://dvd.sourceforge.net).
/// BCD frame-rate flag: bits 7-6 of byte[3]. 0b01 = 25fps (PAL),
/// 0b11 = 29.97fps (NTSC). 0b00/0b10 are "unknown" → frames ignored.
@@ -1693,15 +1669,17 @@ mod tests {
}
/// Audio language bytes [offset+2..+4]: when both bytes are 0x00 the
/// language is the empty string (unspecified), per source.
/// on-disc code is unspecified, and `dvd_lang_to_iso639_2` maps that to
/// the valid ISO 639-2 "und" (undetermined) rather than an empty string,
/// which is not a legal Matroska `Language` element value.
#[test]
fn audio_attr_zero_language_is_empty() {
fn audio_attr_zero_language_becomes_und() {
let mut data = vec![0u8; 8];
data[0] = 0x00;
data[2] = 0x00;
data[3] = 0x00;
let attr = parse_audio_attr(&data, 0).unwrap();
assert_eq!(attr.language, "");
assert_eq!(attr.language, "und");
}
/// Audio sample_rate flag (b0>>3 & 0x03): 0=48kHz, 1=96kHz, else 48kHz.
@@ -1722,11 +1700,11 @@ mod tests {
data[2] = b'd';
data[3] = b'e';
let attr = parse_subtitle_attr(&data, 0).unwrap();
assert_eq!(attr.language, "de");
assert_eq!(attr.language, "deu");
let zero = vec![0u8; 6];
let attr2 = parse_subtitle_attr(&zero, 0).unwrap();
assert_eq!(attr2.language, "");
assert_eq!(attr2.language, "und");
}
/// assign_audio_sub_stream_ids: MP1/MP2 and other non-private-stream-1
@@ -1812,8 +1790,8 @@ mod tests {
assert_eq!(title.cells[1].first_sector, 20);
}
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cb, Cr].
/// A palette with at least one non-zero Y/Cb/Cr is returned as Some;
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cr, Cb].
/// A palette with at least one non-zero Y/Cr/Cb is returned as Some;
/// an all-zero palette returns None (source filters empty palettes).
#[test]
fn pgc_palette_present_and_empty() {
@@ -1833,14 +1811,14 @@ mod tests {
assert!(title2.palette.is_none());
}
/// parse_pgc palette layout: each color is [padding, Y, Cb, Cr] and the
/// parse_pgc palette layout: each color is [padding, Y, Cr, Cb] and the
/// "non-empty" test ignores the padding byte (index 0). A palette whose
/// ONLY non-zero bytes are padding must still be treated as empty (None).
#[test]
fn pgc_palette_padding_only_is_empty() {
let mut pgc = vec![0u8; 0xEA];
pgc[0x03] = 0;
// Set padding byte (index 0) of color 0 non-zero, but Y/Cb/Cr zero.
// Set padding byte (index 0) of color 0 non-zero, but Y/Cr/Cb zero.
pgc[0xA4] = 0xFF;
let title = parse_pgc(&pgc, 0, 1).unwrap();
assert!(
@@ -1893,8 +1871,8 @@ mod tests {
/// Regression test for issue freemkv#25 (NTSC chapter drift).
///
/// Builds a PGC from the real cell table of the DOLL_MASTERS_7 disc in the
/// bug report and checks the chapter marks against MakeMKV's values. Before
/// Builds a PGC from the real cell table of the disc in the
/// bug report and checks the chapter marks against the reference tool's values. Before
/// the fix, chapter 14 landed 4.019 s early; the drift was proportional to
/// elapsed time (0.1%), so a short synthetic fixture would not have caught it.
#[test]
@@ -1942,7 +1920,7 @@ mod tests {
}
let title = parse_pgc(&pgc, 0, pgm.len() as u16).unwrap();
// MakeMKV / source truth, in seconds.
// Reference / source truth, in seconds.
let expected = [
0.0,
62.062,
@@ -2070,7 +2048,7 @@ mod tests {
/// A normal single-angle feature (every cell category 0x00) is never
/// filtered: feature_start_cell == 0, feature_cells == all cells. This is
/// the "Silence of the Lambs" case — the filter must be a no-op.
/// the plain-single-angle-feature case — the filter must be a no-op.
#[test]
fn feature_filter_noop_on_plain_feature() {
let t = DvdTitle {
@@ -2214,10 +2192,8 @@ mod tests {
);
}
// ─────────────────────────────────────────────────────────────────────
// Added: cell-category bit isolation, palette/program-map arithmetic,
// and the malformed-language salvage paths.
// ─────────────────────────────────────────────────────────────────────
/// Each low flag of the cell-category byte comes from its OWN bit and no
/// other: bit3 seamless_play, bit2 interleaved, bit1 stc_discontinuity,
@@ -2278,10 +2254,13 @@ mod tests {
/// two bytes at +2. Only a pair of lowercase a-z bytes is taken verbatim;
/// anything else falls through to the ASCII-alphanumeric salvage, which
/// keeps only the usable characters. A byte outside a-z must never end up
/// in the language string.
/// in the raw salvaged string. This exercises `parse_raw_dvd_lang_bytes`
/// directly — the byte-level salvage step — separately from the ISO
/// 639-1 -> 639-2 mapping `parse_audio_attr`/`parse_subtitle_attr` apply
/// on top (see `dvd_two_letter_and_malformed_language_becomes_iso639_2`).
#[test]
fn language_code_rejects_non_lowercase_bytes() {
// (byte0, byte1, expected language)
// (byte0, byte1, expected raw salvage)
let cases: [(u8, u8, &str); 8] = [
(b'e', b'n', "en"), // both in range → verbatim
(0x21, b'n', "n"), // '!' is below 'a'
@@ -2292,6 +2271,31 @@ mod tests {
(0x00, b'E', "E"), // only the first byte is zero
(0x00, 0x00, ""), // both zero → unset
];
for (b0, b1, want) in cases {
assert_eq!(
parse_raw_dvd_lang_bytes(&[b0, b1]),
want,
"raw language salvage for ({b0:#04x}, {b1:#04x})"
);
}
}
/// The full pipeline `parse_audio_attr`/`parse_subtitle_attr` apply on
/// top of the raw salvage: a valid ISO 639-1 code maps to its ISO 639-2
/// equivalent, and anything the raw salvage does NOT produce a mapped
/// code for (empty, or a single leftover letter from a malformed byte
/// pair) degrades to "und" — never an invalid 2-letter/1-letter code and
/// never an empty string, both of which are illegal Matroska `Language`
/// element values (RFC 9559 §12).
#[test]
fn dvd_two_letter_and_malformed_language_becomes_iso639_2() {
// (byte0, byte1, expected final language)
let cases: [(u8, u8, &str); 4] = [
(b'e', b'n', "eng"), // valid ISO 639-1 → mapped
(0x21, b'n', "und"), // malformed → salvage "n", unmapped → und
(b'E', 0x00, "und"), // malformed → salvage "E", unmapped → und
(0x00, 0x00, "und"), // unspecified → und
];
for (b0, b1, want) in cases {
let mut audio = vec![0u8; 8];
audio[2] = b0;
@@ -2496,13 +2500,12 @@ mod tests {
}
/// The subtitle palette is 16 entries of 4 bytes at PGC+0xA4, each
/// `[padding, Y, Cb, Cr]`. Every byte of every entry is distinct here, so
/// `[padding, Y, Cr, Cb]`. Every byte of every entry is distinct here, so
/// a wrong stride, a wrong base or a shifted component shows up.
#[test]
// The loop variable is the DOMAIN VALUE being checked (a palette entry number), not a
// cursor into a collection: it is what the assertion message names and
// what the table is keyed by. `.iter().enumerate()` would rename the
// thing under test to `i` and read worse.
// The loop variable is the DOMAIN VALUE under test (a palette entry number),
// not a collection cursor — it's what the assertion message names, so
// `.iter().enumerate()` would rename the thing under test and read worse.
#[allow(clippy::needless_range_loop)]
fn pgc_palette_entries_read_at_correct_stride() {
let mut pal = [[0u8; 4]; 16];
@@ -2524,7 +2527,7 @@ mod tests {
}
}
/// A palette is "present" when ANY of Y, Cb or Cr is non-zero in ANY
/// A palette is "present" when ANY of Y, Cr or Cb is non-zero in ANY
/// entry — a single non-zero chroma component is enough. Only the
/// padding byte [0] is ignored.
#[test]
@@ -2641,16 +2644,8 @@ mod tests {
}
// ── TT_SRPT: the title table (`parse_tt_srpt` / `parse_vmg`) ────────────
//
// This is the table that says which titles a DVD has, and it had no tests
// at all until 1.6.2 — while being at the centre of both DVD stories in
// 1.6.1. The CSS descramble bug destroyed the sector carrying it, so a disc
// enumerating 38 titles produced an image enumerating 10, silently, at exit
// 0. And `dir://` was pulled from the release on 2026-08-05 because a
// folder produced the wrong title list — the same table, reached through a
// synthesized image.
//
// Both were found on real discs. Nothing hermetic covered the table itself.
// This table says which titles a DVD has and had no tests until 1.6.2, despite
// two real-disc 1.6.1 bugs that silently mis-enumerated titles through it.
/// Build a VMG with a TT_SRPT at `tt_srpt_sector` declaring `entries`
/// of `(num_chapters, vts_number, vts_title_num)`.
@@ -2694,11 +2689,9 @@ mod tests {
let at_sector = parse_tt_srpt(&vmg, sector as usize * crate::consts::SECTOR_BYTES)
.expect("reading at the sector offset works");
assert_eq!(at_sector[&1], vec![(3, 1)], "sector {sector}");
// Reading the same pointer as a BYTE offset lands 1/2048th of the
// way in — inside the header here — and yields a different table.
// It does not error: it silently reports the wrong titles, which is
// how 28 dropped titles looked like a scan difference rather than a
// failure.
// Reading the same pointer as a BYTE offset lands 1/2048th of the way
// in and yields a different table without erroring — silently wrong
// titles, which is how dropped titles looked like a scan difference.
let as_bytes = parse_tt_srpt(&vmg, sector as usize).expect("does not error");
assert_ne!(
as_bytes, at_sector,
@@ -2746,11 +2739,9 @@ mod tests {
/// unclamped 65535 is a ~540 MB allocation from a ~800 KB crafted file.
#[test]
fn tt_srpt_clamps_an_absurd_declared_title_count() {
// The fixture must actually CONTAIN more entries than the cap, or the
// walk stops when the buffer runs out and the clamp is never what
// bounded it. An earlier version of this test declared u16::MAX over a
// two-kilobyte fixture: the loop broke on `base + 12 > len` after three
// iterations and the assertion held with the clamp deleted entirely.
// The fixture must actually CONTAIN more entries than the cap, or the walk
// stops when the buffer runs out and the clamp is never exercised — an
// earlier version declared u16::MAX over a 2KB fixture and passed vacuously.
const PRESENT: usize = MAX_TT_SRPT_TITLES + 40;
let entries: Vec<(u16, u8, u8)> = (0..PRESENT)
.map(|i| (1u16, 1u8, (i % 250 + 1) as u8))
+9 -18
View File
@@ -109,19 +109,14 @@ where
return Err(BoundedError::Halted);
}
// Rendezvous channel: the worker sends exactly one value (the
// op's return) and then exits. Capacity-0 means the send blocks
// until we receive — fine on the happy path; on the timeout /
// halt path the receiver is dropped and the worker's send
// returns Err, which the worker ignores.
// Rendezvous channel: worker sends one value then exits. On timeout/halt
// the receiver is dropped, so the worker's send returns Err (ignored).
let (tx, rx) = sync_channel::<R>(0);
let _ = thread::Builder::new()
.name("freemkv-bounded-syscall".into())
.spawn(move || {
// Ignore the send error: if we time out (or get halted)
// before the worker finishes, the receiver is dropped
// and `tx.send` returns Err. Either way, the worker has
// nothing more to do.
// Ignore send error: if we time out/halt before the worker finishes,
// the receiver is dropped and `tx.send` returns Err — nothing more to do.
let _ = tx.send(op());
});
@@ -179,10 +174,8 @@ mod tests {
fn halt_fires_during_wait() {
let halt = Halt::new();
let halt2 = halt.clone();
// Flip the halt from a side thread after ~300 ms — long
// enough that the receive loop has rolled at least one
// 250 ms slice and is sitting in `recv_timeout` again when
// the bit flips.
// Flip halt after ~300ms, long enough that the receive loop has rolled
// at least one 250ms slice and is back in `recv_timeout`.
thread::spawn(move || {
thread::sleep(Duration::from_millis(300));
halt2.cancel();
@@ -196,11 +189,9 @@ mod tests {
#[test]
fn worker_panics() {
// Worker panics → sender drops without sending → recv sees
// Disconnected → WorkerLost. We use an explicit panic in the
// op closure rather than `panic!()` from inside the channel
// machinery; the spawned thread's panic is contained (no
// process abort) because we don't `.join()` it.
// Worker panics → sender drops without sending → recv sees Disconnected →
// WorkerLost. Panic is in the op closure, not the channel machinery; it's
// contained (no process abort) because we don't `.join()` the thread.
let r = bounded_syscall(None, Duration::from_secs(2), || -> u32 {
panic!("intentional test panic");
});
+24 -57
View File
@@ -75,18 +75,15 @@ impl BytePrefetcher {
chunk_bytes: usize,
halt: Option<Halt>,
) -> std::io::Result<Self> {
// A zero-length chunk makes every recycled buffer an empty
// slice; `reader.read(&mut [])` returns Ok(0), which the loop
// below treats as EOF — the consumer would see a clean,
// silent zero-byte stream. Callers pass the downstream
// demuxer's batch size, which is always > 0.
// A zero-length chunk makes every recycled buffer empty; `reader.read(&mut [])`
// returns Ok(0), which the loop below treats as EOF, producing a silent
// zero-byte stream. Callers must pass the downstream demuxer's batch size (> 0).
debug_assert!(chunk_bytes > 0, "BytePrefetcher chunk_bytes must be > 0");
let (tx, rx) = bounded::<Batch>(FORWARD_DEPTH);
let (recycle_tx, recycle_rx) = bounded::<Vec<u8>>(RECYCLE_DEPTH);
// Seed the recycle pool. Without these the first
// `recycle_rx.recv()` would block forever (no consumer has
// returned a buffer yet).
// Seed the recycle pool; otherwise the first `recycle_rx.recv()` blocks
// forever since no consumer has returned a buffer yet.
for _ in 0..RECYCLE_DEPTH {
let _ = recycle_tx.send(vec![0u8; chunk_bytes]);
}
@@ -94,21 +91,13 @@ impl BytePrefetcher {
let producer = std::thread::Builder::new()
.name("freemkv-byte-prefetch".into())
.spawn(move || {
// Wrap the feed loop in catch_unwind so a panic in the inner
// `reader.read` (e.g. a decrypt-on-read slice/arith bug) is NOT
// indistinguishable from a clean finish at the demux boundary. A
// clean exit (EOF, halt, consumer disconnect) returns and drops
// `tx` → the demux loop reads RecvError as EOF (correct). A PANIC
// sends an explicit error sentinel first so the demux loop's
// `Ok(Err(_))` arm fires and propagates a typed error instead of
// converting the dropped channel into a clean `DemuxBatch::Eof`
// that would finalize a TRUNCATED mux while reporting success.
// catch_unwind: a clean exit drops `tx` (demux reads RecvError as EOF),
// but a panic sends an error sentinel first so demux gets a typed error
// instead of finalizing a truncated mux as success.
let body = std::panic::AssertUnwindSafe(|| {
let cancelled = || halt.as_ref().map(|h| h.is_cancelled()).unwrap_or(false);
// Liveness heartbeat: the producer blocks on the recycle and
// forward channels; a stalled consumer or a wedged reader shows
// up as the beat going silent. Total is unknown, so `pos` is
// cumulative bytes read.
// Liveness heartbeat: a stalled consumer or wedged reader shows up
// as the beat going silent. Total is unknown, so `pos` is cumulative.
let mut hb = crate::progress::Heartbeat::new("byte_prefetch");
let mut produced_bytes: u64 = 0;
loop {
@@ -116,10 +105,9 @@ impl BytePrefetcher {
if cancelled() {
return;
}
// Park on the recycle channel, but re-poll halt
// every POLL_INTERVAL: a pure-AtomicBool Halt does
// not disconnect the channel, so a blocking recv()
// would never re-reach the cancel check.
// Re-poll halt every POLL_INTERVAL: a pure-AtomicBool Halt does
// not disconnect the channel, so a blocking recv() would never
// re-reach the cancel check.
let mut buf = loop {
match recycle_rx.recv_timeout(POLL_INTERVAL) {
Ok(b) => break b,
@@ -132,11 +120,9 @@ impl BytePrefetcher {
Err(RecvTimeoutError::Disconnected) => return,
}
};
// Re-expose the full extent. After a short read the
// prior iteration truncated to n < chunk_bytes, so
// this regrows the length back to chunk_bytes
// without reallocating (capacity was fixed at
// construction and never shrinks).
// Regrow to chunk_bytes: a prior short read truncated len to
// n < chunk_bytes. No realloc — capacity was fixed at
// construction and never shrinks.
if buf.len() < chunk_bytes {
buf.resize(chunk_bytes, 0);
} else {
@@ -196,27 +182,12 @@ impl BytePrefetcher {
/// drains `rx`, runs the demuxer in place on each filled buffer,
/// and recycles back through `recycle_tx`.
pub fn into_channels(self) -> (Receiver<Batch>, Sender<Vec<u8>>, PrefetchShell) {
// MOVE the three fields out cleanly — never clone. Each of
// `rx` and `recycle_tx` ends up with exactly ONE live copy:
// the one in the returned tuple. The pre-1.0.0 implementation
// cloned both and then `mem::forget`-ed `self`, leaking the
// originals so an extra live receiver + sender survived
// forever. That defeated the channel-disconnection shutdown:
// when the demux consumer exited early (halt, or a `tx.send`
// error in `demux_thread`), the producer's `recycle_rx.recv()`
// and `tx.send()` never saw all-peers-dropped, so the producer
// never returned and `PrefetchShell::drop`'s `join()` hung.
//
// `ManuallyDrop` + `ptr::read` reads each field out by value
// and suppresses `self`'s own `Drop` (which would otherwise
// double-`join`), leaving NO extra live endpoint behind. This
// is the panic-free equivalent of the `Option::take` approach
// and mirrors `sector::prefetched::into_channels`.
// MOVE the fields out, never clone: the pre-1.0.0 impl cloned + `mem::forget`-ed
// `self`, leaking endpoints that defeated disconnection shutdown (producer join
// hung forever). `ManuallyDrop` + `ptr::read` moves fields out cleanly instead.
let me = std::mem::ManuallyDrop::new(self);
// SAFETY: `me` is `ManuallyDrop`, so none of these fields will
// be dropped by `me`. Each `ptr::read` performs exactly one
// bitwise move out; every field is read exactly once and never
// touched again, so there are no double-frees and no aliasing.
// SAFETY: `me` is `ManuallyDrop`, so none of these fields are dropped by `me`.
// Each `ptr::read` is one bitwise move, read exactly once — no double-frees.
let producer = unsafe { std::ptr::read(&me.producer) };
// SAFETY: `rx` and `recycle_tx` are always `Some` here —
// `into_channels` is the only way to consume a live
@@ -230,13 +201,9 @@ impl BytePrefetcher {
impl Drop for BytePrefetcher {
fn drop(&mut self) {
// Drop channel endpoints BEFORE joining the producer so the
// producer observes SendTimeoutError::Disconnected (forward tx)
// or RecvTimeoutError::Disconnected (recycle rx) and exits
// promptly. Without this, a non-EOF source fills the depth-2
// forward channel and then spins in send_timeout(POLL_INTERVAL)
// forever because rx is never drained, causing join() to
// deadlock.
// Drop channel endpoints BEFORE joining so the producer observes Disconnected
// (send/recv) and exits promptly. Otherwise a non-EOF source fills the forward
// channel and spins in send_timeout forever since rx is never drained.
drop(self.rx.take());
drop(self.recycle_tx.take());
if let Some(h) = self.producer.take() {
+13 -24
View File
@@ -33,7 +33,7 @@
//!
//! On `open()` each platform issues its "sequential access expected"
//! hint so OS-level readahead widens. The hint and the DONTNEED call
//! live in per-OS sibling modules ([`linux::hint_sequential`] et al.)
//! live in per-OS sibling modules (`linux::hint_sequential` et al.)
//! — no inline `#[cfg]` in this file.
//!
//! ## Read-ahead prefetch
@@ -190,13 +190,9 @@ impl SectorSource for FileSectorSource {
) -> Result<usize> {
let count = count as u32;
let bytes = count as usize * SECTOR_BYTES;
// A real check, not a debug_assert: this is a public `SectorSource` impl,
// so an undersized `out` is caller input, and `out[..bytes]` below would
// panic with 'range end index out of range' in release where the assert is
// compiled away. `Drive::read_fua` already carries exactly this guard, with
// a comment recording the same panic being fixed there — this impl was
// simply never given it, and `PrefetchedSectorSource` has a regression test
// for the case that this one lacked.
// A real check, not debug_assert: `out` is caller input to a public
// `SectorSource` impl, and `out[..bytes]` below would panic in release
// where the assert is compiled away. `Drive::read_fua` carries this guard.
if out.len() < bytes {
return Err(Error::DiscRead {
sector: lba as u64,
@@ -215,18 +211,14 @@ impl SectorSource for FileSectorSource {
.read_exact(&mut out[..bytes])
.map_err(|e| Error::IoError { source: e })?;
// Queue the next batch's read with the kernel before the
// caller starts processing what we just returned. readahead()
// is non-blocking — it queues I/O and returns, so the kernel
// pulls those pages into cache while the consumer (decrypt +
// demux + mux) runs. Next read_sectors call hits a warm cache.
// Queue the next batch's read before the caller processes what we returned.
// readahead() is non-blocking; the kernel pulls pages into cache while the
// consumer runs, so the next read_sectors call hits a warm cache.
platform::prefetch(&self.file, offset + bytes as u64, bytes as u64);
// Periodic page-cache eviction on the read side. Without
// this, an 85 GB streaming ISO read pins the entire file in
// the kernel page cache, which starves concurrent writes and
// collapses mux throughput. Mirrors the write-side
// WritebackPipeline's DONTNEED policy.
// Periodic page-cache eviction on the read side: an 85 GB streaming ISO
// read would otherwise pin the whole file in cache, starving concurrent
// writes. Mirrors WritebackPipeline's DONTNEED policy on the write side.
self.bytes_read_since_drop += bytes as u64;
if self.bytes_read_since_drop >= self.drop_chunk_bytes {
let drop_start = self.drop_window_start;
@@ -407,12 +399,9 @@ mod tests {
#[test]
fn drop_chunk_size_env_override() {
// Explicit 8 MiB via env var.
// SAFETY: tests in this crate are single-threaded per the
// default cargo test harness, but std::env::set_var is
// declared `unsafe` since Rust 2024 (it can race with other
// threads / TLS). For a test that runs in-process before any
// FileSectorSource construction this is safe in practice.
// Explicit 8 MiB via env var. SAFETY: set_var is `unsafe` since Rust 2024
// (can race with other threads/TLS), but this test runs single-threaded
// and in-process before any FileSectorSource construction.
unsafe {
std::env::set_var("FREEMKV_READ_DROP_CHUNK_MIB", "8");
}
+4 -14
View File
@@ -21,6 +21,7 @@
//!
//! So: drive sources get `freemkv_engine::copy`. Everything else gets this.
use crate::consts::SECTOR_BYTES;
use crate::error::{Error, Result};
use crate::halt::Halt;
use crate::sector::SectorSource;
@@ -33,9 +34,6 @@ use std::path::Path;
/// a notable allocation and cancellation stays responsive.
const BATCH_SECTORS: u32 = 2048;
/// Bytes per sector. Fixed for every medium this crate reads.
const SECTOR_BYTES: usize = 2048;
/// Write `total_sectors` sectors from `reader` to `dest`.
///
/// Reads sequentially from LBA 0 and writes in order, so the output is a faithful
@@ -103,17 +101,9 @@ pub fn write_image(
on_progress(written);
}
// flush() only pushes the BufWriter's bytes into the kernel via write(2).
// It makes no durability promise at all, so returning Ok here would report
// a finished image while up to several gigabytes of it still sit in the
// page cache. A crash, a power loss, or yanking the removable/network
// volume the image was written to then leaves a truncated or empty file
// that the caller was told was complete.
//
// For a 6-90 GB image that is exactly the failure this crate treats as
// worst: success reported over wrong output. `into_inner` is used rather
// than `flush` so a buffered-write error is surfaced instead of being
// dropped on the floor by BufWriter's Drop.
// flush() only pushes bytes into the kernel via write(2), no durability promise —
// a crash or yanked volume could leave a truncated file reported as complete.
// `into_inner` (not `flush`) also surfaces buffered-write errors, not silently drop them.
let file = out.into_inner().map_err(|e| Error::IoError {
source: e.into_error(),
})?;
+47 -111
View File
@@ -1,13 +1,12 @@
//! Generic bounded producer/consumer pipeline.
//!
//! `Pipeline<I, R>` spawns a single consumer thread, hands it items
//! through a bounded `mpsc::sync_channel`, and joins it on `finish()`.
//! through a bounded `crossbeam_channel`, and joins it on `finish()`.
//! The consumer's behaviour is supplied by a [`Sink`] implementation:
//! `apply` is called once per item, `close` is called once at the end.
//!
//! Call sites in libfreemkv that want a producer/consumer split —
//! sweep (`disc/sweep.rs::SweepSink`) and the file-backed mux highway
//! — are built on this primitive.
//! The file-backed mux highway is built on this primitive. (An earlier
//! sweep sink also used it; sweep moved to freemkv-engine in 1.6.0.)
//!
//! ## Cancellation and error semantics
//!
@@ -170,19 +169,9 @@ fn finish_with_grace<R: Send + 'static>(
}
thread::sleep(POLL_INTERVAL);
}
// Grace expired. CLAIM abandonment, then log and leak. Claiming BEFORE
// dropping the handle guarantees the leaked consumer observes it the moment
// its wedged syscall returns: it then skips any further `apply` and skips
// `close()`, rather than running on to finalise the abandoned output file.
//
// A compare-exchange, not a store, because the consumer may have committed to
// `close()` in the instant between our last `is_finished()` poll and now. It
// then cannot be stopped — the finalise IS happening — so abandoning it would
// report the rip as interrupted while a valid, fully finalised container
// lands on disk. Losing the race means waiting for the result the consumer is
// already producing instead. `AcqRel` pairs with the consumer's own
// compare-exchange and with the `Acquire` loads in its drain loop, so the flag
// is reliably observed even on weak memory models (ARM64/POWER).
// Grace expired. CLAIM abandonment before dropping the handle so the leaked consumer
// skips further `apply`/`close()` once its wedged syscall returns. Compare-exchange,
// not a store: if it already committed to `close()`, wait for its result instead.
if state
.compare_exchange(
state::RUNNING,
@@ -221,11 +210,9 @@ fn finish_with_grace<R: Send + 'static>(
(output will not be finalised)",
FINISH_GRACE_SECS
);
// Dropping `handle` without joining detaches from the thread — the
// consumer keeps running until its kernel call returns or the process
// exits. This is the intentional "leak" documented in
// `finish_with_halt`'s contract; the `abandoned` flag bounds what the
// leaked thread is allowed to do to the output before it exits.
// Dropping `handle` without joining detaches the thread; the consumer keeps
// running until its kernel call returns or the process exits. This is the
// intentional "leak" from `finish_with_halt`'s contract, bounded by `abandoned`.
drop(handle);
Err(leak_err)
}
@@ -365,11 +352,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let mut first_err: Option<Error> = None;
let mut stopped = false;
// Rolling apply-throughput summary. The per-item "apply: OK"
// line was 99% of the mux log; collapse it into a periodic
// summary (count, avg ms, items/s) emitted ~every 5 s while
// debug tracing is on. The individual slow-apply ("took … s")
// STALL events below stay visible — those are signal, not noise.
// Rolling apply-throughput summary: the per-item "apply: OK" line was
// 99% of the mux log, so collapse it into a periodic summary (count,
// avg ms, items/s) every ~5s. Slow-apply STALL events stay visible below.
let mut summary_count: u64 = 0;
let mut summary_nanos: u128 = 0;
let mut summary_since = Instant::now();
@@ -381,13 +366,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline receive: item={}", std::any::type_name::<I>());
}
// If the caller has abandoned us (grace period expired
// after a halt/timeout and the JoinHandle was dropped),
// stop applying items. We keep draining so the producer
// — if it is somehow still alive — never blocks on a
// dead receiver, but we touch the output no further. The
// final post-loop abandonment check returns the error
// and skips `close()`.
// Abandoned (grace expired, JoinHandle dropped): keep draining so a
// still-alive producer never blocks on a dead receiver, but touch
// the output no further. Post-loop check returns error, skips close().
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
continue;
}
@@ -417,11 +398,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
}
first_err = Some(e);
// Publish the failure so the producer can stop
// FEEDING a dead write side instead of only learning
// about it at `finish()` — by which time it has read
// the rest of the disc. `Release` pairs with the
// `Acquire` load in `send_with_halt`.
// Publish the failure so the producer stops feeding a dead
// write side instead of learning at `finish()`, after reading
// the rest of the disc. `Release` pairs with `send_with_halt`.
failed_consumer.store(true, Ordering::Release);
}
}
@@ -473,16 +452,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
);
}
// Final abandonment check: the common leak case is a
// consumer wedged inside `apply` (a blocking write
// syscall). When that syscall finally returns, the
// producer has long since dropped `tx`, so `recv`
// yields `Err` and we fall through to here. If the
// caller abandoned us in the meantime, skip `close()`
// entirely — finalising the output (e.g. writing the
// MKV Cues + patching the segment header) on a file the
// caller already reported as failed is exactly the
// write race we must not run.
// Final abandonment check: a consumer wedged in a blocking `apply` write
// can outlive the producer dropping `tx`, landing here via `recv -> Err`.
// Skip `close()` if abandoned meanwhile — it would race the write.
match first_err {
// No `close()` on this path, so there is nothing to claim —
// just report, unless the caller has already given up on us.
@@ -493,14 +465,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Err(e)
}
}
// CLAIM the finalise. A plain load here left a window in which
// the caller stored `abandoned` AFTER we read it as clear, so
// `close()` ran anyway and finalised (Cues + Segment-size
// patch) an output the caller had already reported as
// interrupted — a truncated rip indistinguishable from a
// complete one. The compare-exchange closes that window: if the
// caller got there first we skip `close()`, and if we get there
// first the caller waits for us instead of abandoning.
// CLAIM the finalise: a plain load could race the caller marking
// `abandoned`, letting `close()` finalise output already reported
// interrupted. Compare-exchange: loser skips `close()`, waits for winner.
None => {
if state_consumer
.compare_exchange(
@@ -645,12 +612,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let end = Instant::now() + deadline;
let mut pending = item;
loop {
// The consumer's `apply` has failed fatally: everything sent from here
// is drained and discarded, so hand the item back at once. Without this
// the producer saw every send succeed (the channel is always being
// drained) and went on reading the whole remaining title — an hour of
// drive time on a UHD — for a write that died on the first frame, only
// learning about it at `finish()`.
// The consumer's `apply` failed fatally: hand the item back now, since
// otherwise sends keep succeeding and the producer reads a whole UHD
// title before learning at `finish()` the write died on frame one.
if self.consumer_failed() {
if debug_enabled() {
tracing::debug!(
@@ -680,9 +644,8 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
}
return Err(pending);
}
// Wait for space-available or halt-check tick, whichever
// is sooner. Crossbeam's send_timeout is kernel-wakeup
// based: the consumer's recv on a saturated channel
// Wait for space-available or halt-check tick, whichever is sooner.
// send_timeout is kernel-wakeup based: recv on a saturated channel
// signals this thread the moment a slot opens up.
let slice = SEND_HALT_CHECK_INTERVAL.min(end.saturating_duration_since(now));
match self.tx.send_timeout(pending, slice) {
@@ -857,14 +820,9 @@ mod tests {
#[test]
fn back_pressure_blocks_sender() {
// depth=2 + 5 sends + 50ms/apply: with the consumer pinned at
// 50 ms per item, the producer can buffer 2 (channel cap) +
// 1 (consumer in flight) = 3 items before sends 4 and 5 must
// block on consumer progress. Wall-clock floor across all 5
// sends is therefore ~2 * 50ms = 100ms (sends 4 and 5 each
// wait roughly one apply-cycle). Use 80 ms as the assertion
// floor to stay above the 50ms-per-item progress floor while
// tolerating CI jitter — it still proves blocking is real.
// depth=2 + 5 sends + 50ms/apply: producer buffers 3 items (2 channel cap +
// 1 in flight) before sends 4 and 5 must block, giving a ~100ms wall-clock
// floor. Assert 80ms to tolerate CI jitter while still proving blocking.
let count = Arc::new(AtomicUsize::new(0));
let sink = SlowSink {
delay: Duration::from_millis(50),
@@ -985,10 +943,9 @@ mod tests {
)
.expect("spawn should succeed");
// Send 10 items. After Stop, subsequent sends may either
// succeed (already buffered) or fail with Err(I) (channel
// closed). Both are valid — we don't assert on the send
// results.
// Send 10 items. After Stop, subsequent sends may either succeed (already
// buffered) or fail with Err(I) (channel closed); both are valid, so we
// don't assert on the send results.
for i in 0..10u64 {
let _ = pipe.send(i);
}
@@ -1087,14 +1044,9 @@ mod tests {
#[test]
fn send_with_halt_returns_item_on_deadline() {
// depth=1 + consumer wedged in apply on the first item, AND
// the channel buffer already loaded with a second item, means
// any further `try_send` sees Full; with a 200 ms deadline and
// no halt fired, send_with_halt must return `Err(item)` within
// roughly the deadline. Synchronising on `started` ensures the
// consumer has actually started its wedged apply BEFORE we
// load the channel-buffer slot — without that, the consumer
// could still drain in a race window.
// depth=1 + wedged consumer + a loaded buffer slot means further `try_send`
// sees Full, so send_with_halt must return `Err(item)` around the 200ms
// deadline. Sync on `started` first so the consumer is wedged before we load the slot.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
@@ -1208,13 +1160,9 @@ mod tests {
matches!(res, Err(Error::Halted)),
"expected Err(Halted), got {res:?}"
);
// Bailed out within the grace period plus a healthy margin.
// The grace spin-poll adds up to FINISH_GRACE_SECS (5s) of
// extra wait for a truly wedged consumer; the test's consumer
// is deliberately never released before this assert so we
// exercise the "grace expires → leak" path. 15s is well under
// the 10-minute JOIN_TIMEOUT backstop and proves the new code
// doesn't block forever.
// Bailed within the grace period plus margin: grace spin-poll adds up to
// FINISH_GRACE_SECS (5s) for the deliberately-unreleased, wedged consumer.
// 15s stays well under the 10-minute JOIN_TIMEOUT, proving it doesn't block forever.
assert!(
elapsed < Duration::from_secs(15),
"halt observation took too long: {elapsed:?}"
@@ -1571,10 +1519,9 @@ mod tests {
halt.cancel();
let start = Instant::now();
// finish_with_halt drops tx (signalling EOF), then observes the
// pre-cancelled halt and enters the grace spin. The consumer
// finishes close() within 500ms, so finish_with_halt must join
// cleanly and return Ok with the correct total.
// finish_with_halt drops tx (EOF), observes the pre-cancelled halt, and
// enters the grace spin; the consumer finishes close() within 500ms, so
// it must join cleanly and return Ok with the correct total.
let res = pipe.finish_with_halt(Some(&halt));
let elapsed = start.elapsed();
@@ -1855,20 +1802,9 @@ mod tests {
// caller genuinely reaches the abandon decision with a close in flight.
let rel = release.clone();
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
//
// These intervals used to be 600 ms against a 300 ms grace, which
// left NO margin: two 300 ms windows end at 600 ms, and the 250 ms
// poll cadence can push the observation later still, so on a loaded
// runner the second window expired first and the caller abandoned —
// failing with Err(Halted) against a race, not a defect.
//
// Scaled up so the jitter is small relative to the intervals: the
// first window ends at ~1.0-1.25 s and the second at ~2.0-2.25 s,
// so releasing at 1.6 s sits well inside the second with roughly
// 350 ms of slack on either side. The ordering under test is
// unchanged; only the margin is.
// Release inside the second grace window. Old 600ms/300ms-grace intervals
// left no margin and flaked on loaded runners; scaled up so windows end
// at ~1.0-1.25s and ~2.0-2.25s, releasing at 1.6s for ~350ms slack.
thread::sleep(Duration::from_millis(1600));
rel.store(true, Ordering::SeqCst);
});
+4 -3
View File
@@ -3,7 +3,7 @@
//! The `libc` crate doesn't expose these symbols across all macOS SDK
//! versions, so we define them locally with values from
//! `/usr/include/sys/fcntl.h`. Two call sites (
//! [`crate::io::writeback_file`] and [`crate::io::sink::preallocate`])
//! [`crate::io::writeback_file`] and `crate::io::sink::preallocate`)
//! need the same constants and `fstore_t` layout — keeping a single
//! source of truth here prevents the two copies from drifting.
//!
@@ -16,8 +16,9 @@ pub(crate) const F_PREALLOCATE: libc::c_int = 42;
/// Anchor preallocation at the current physical EOF.
pub(crate) const F_PEOFPOSMODE: libc::c_int = 3;
/// Prefer a contiguous allocation. Try this first; on `EINVAL` (no
/// contiguous run of that size), fall back to `F_ALLOCATEALL`.
/// Prefer a contiguous allocation. Call sites OR this together with
/// `F_ALLOCATEALL` on the first attempt; on failure they fall back to
/// `F_ALLOCATEALL` alone.
pub(crate) const F_ALLOCATECONTIG: libc::c_uint = 0x0000_0002;
/// Allow non-contiguous allocation. Stronger guarantee than just
+3 -5
View File
@@ -94,11 +94,9 @@ pub(crate) fn open_for_mkv(
return Ok(Box::new(wf));
}
}
// Only Linux differentiates the sink by filesystem type (NFS gets
// the WritebackFile machinery); every other OS always uses
// `LocalFileSink`. Reference `detect` as a value (no call, no
// `statfs` syscall) so it isn't flagged dead on non-Linux while
// still avoiding the wasted probe whose result we'd discard.
// Only Linux differentiates the sink by filesystem type; other OSes always use
// `LocalFileSink`. Reference `detect` as a value (no call/syscall) so it isn't
// flagged dead on non-Linux while avoiding a wasted probe.
#[cfg(not(target_os = "linux"))]
let _ = crate::platform::fs_type::detect;
+3 -5
View File
@@ -18,11 +18,9 @@ pub(super) fn preallocate_impl(file: &File, size_bytes: u64) {
// would wrap a >= 2^63 size to a negative length.
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX) as libc::off_t;
let mut store = Fstore {
// Prefer a contiguous run but accept scattered extents to
// satisfy the full length. Without F_ALLOCATEALL the first
// attempt is best-effort and can return rc=0 with a partial
// allocation, so the fallback below would never fire. Matches
// writeback_file/macos.rs.
// Prefer contiguous but accept scattered extents. Without F_ALLOCATEALL
// the first attempt is best-effort and can return rc=0 with a partial
// allocation, so the fallback below would never fire. Matches writeback_file/macos.rs.
fst_flags: F_ALLOCATECONTIG | F_ALLOCATEALL,
fst_posmode: F_PEOFPOSMODE,
fst_offset: 0,
+15 -25
View File
@@ -56,12 +56,9 @@ impl SocketSink {
/// `("host", 1234)`, a `SocketAddr`, etc.
pub fn connect<A: ToSocketAddrs>(addr: A, sndbuf_bytes: Option<usize>) -> io::Result<Self> {
let stream = TcpStream::connect(addr)?;
// `set_nodelay(true)` keeps small writes (TS packet trains, fMP4
// moof headers) from sitting in Nagle's algorithm until the buffer
// fills. The BufWriter already absorbs syscall overhead; Nagle
// would just add latency without coalescing more. It is a latency
// hint, not a correctness requirement, so a platform that rejects
// TCP_NODELAY must not fail the connect — demote the error.
// `set_nodelay(true)` keeps small writes (TS packet trains, fMP4 moof headers)
// from sitting in Nagle's algorithm; BufWriter already absorbs syscall overhead.
// A latency hint, not a correctness requirement — a rejecting platform must not fail connect.
let _ = stream.set_nodelay(true);
if let Some(n) = sndbuf_bytes {
set_send_buffer(&stream, n)?;
@@ -119,10 +116,9 @@ impl UdpSocketSink {
///
/// `sndbuf_bytes`, when present, is a hint to `SO_SNDBUF`.
pub fn connect<A: ToSocketAddrs>(peer: A, sndbuf_bytes: Option<usize>) -> io::Result<Self> {
// Resolve the peer first so the local bind matches its address
// family. Binding `0.0.0.0:0` (IPv4) and then connecting to an
// IPv6 peer fails with EAFNOSUPPORT, so pick the wildcard that
// matches the resolved family.
// Resolve the peer first so the local bind matches its address family:
// binding `0.0.0.0:0` (IPv4) then connecting to an IPv6 peer fails
// with EAFNOSUPPORT.
let peer_addr = peer
.to_socket_addrs()?
.next()
@@ -164,12 +160,9 @@ impl SequentialSink for UdpSocketSink {
}
}
// ── Platform `SO_SNDBUF` tuning ────────────────────────────────────────────
//
// std's `TcpStream` / `UdpSocket` don't expose `SO_SNDBUF`. We drop to
// libc on Linux + macOS (the libc-dep targets in Cargo.toml). On other
// targets the hint is silently ignored — the socket still works, the
// kernel just picks its own send-buffer size.
// ── Platform `SO_SNDBUF` tuning ─────────────────────────────────────────
// std doesn't expose `SO_SNDBUF`; drop to libc on Linux + macOS. Other
// targets silently ignore the hint — socket still works, kernel picks its own size.
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn set_send_buffer(stream: &TcpStream, bytes: usize) -> io::Result<()> {
@@ -185,10 +178,9 @@ fn set_udp_send_buffer(socket: &UdpSocket, bytes: usize) -> io::Result<()> {
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
fn set_send_buffer(_stream: &TcpStream, _bytes: usize) -> io::Result<()> {
// Non-Linux-non-macOS targets aren't in Cargo.toml's libc dep list;
// silently ignore the hint rather than failing the connect. Callers
// can detect via the lack of an explicit "sndbuf applied" signal
// (not provided, intentionally — this is a hint, not a guarantee).
// Non-Linux-non-macOS targets aren't in Cargo.toml's libc dep list; silently
// ignore the hint rather than failing the connect (intentionally no
// "sndbuf applied" signal — this is a hint, not a guarantee).
Ok(())
}
@@ -265,11 +257,9 @@ mod tests {
});
let mut sink = SocketSink::connect(addr, None).unwrap();
_assert_seq(&mut sink);
// The negative is harder to assert directly (no `is_not<T>`),
// but `SocketSink` does not impl `Seek`, so it can't unify with
// `RandomAccessSink`'s super-bound. The Phase 2 blanket impl
// `impl<T: SequentialSink + Seek> RandomAccessSink for T {}` thus
// excludes it by construction.
// No `is_not<T>` to assert the negative directly, but `SocketSink` doesn't
// impl `Seek`, so it can't unify with `RandomAccessSink`'s super-bound —
// the blanket `impl<T: SequentialSink + Seek> RandomAccessSink for T {}` excludes it.
}
#[test]
+1 -1
View File
@@ -3,7 +3,7 @@
//! queue bounded. On macOS and Windows, a no-op stub.
//!
//! The platform decision lives entirely in this file (the cfg-gated
//! `pub use` below). Callers — and `DiskWriter` itself — are
//! `pub use` below). Callers — and `WritebackFile` itself — are
//! platform-independent.
#[cfg(target_os = "linux")]
+22 -34
View File
@@ -179,18 +179,16 @@ impl WritebackPipeline {
if pos < self.last_flush_pos.saturating_add(self.chunk_bytes) {
return;
}
// Byte offsets are unsigned throughout; the signed cast happens
// only at the libc call boundary where the kernel ABI requires
// `i64`. `saturating_sub` documents and hardens the line-above
// guard that `pos >= last_flush_pos`.
// Byte offsets are unsigned throughout; the signed cast happens only at the
// libc call boundary where the kernel ABI requires `i64`. `saturating_sub`
// hardens the line-above guard that `pos >= last_flush_pos`.
let chunk_off: u64 = self.last_flush_pos;
let chunk_len: u64 = pos.saturating_sub(self.last_flush_pos);
let mut wait_ms: u64 = 0;
let mut fadvise_ms: u64 = 0;
// Async kickoff for the just-completed chunk runs on every
// path (NFS, degraded, normal) — it's nominally non-blocking
// by spec and gives the kernel an early hint that this range
// is ready to flush.
// Async kickoff for the just-completed chunk runs on every path (NFS,
// degraded, normal) — non-blocking by spec, an early hint that this
// range is ready to flush.
let kickoff_rc = unsafe {
libc::sync_file_range(
self.fd,
@@ -211,16 +209,13 @@ impl WritebackPipeline {
}
if let Some((prev_off, prev_len)) = self.pending.take() {
if self.skip_wait() {
// NFS branch (or degraded fallback after a prior
// timeout): the WAIT_AFTER + DONTNEED dance is what
// hangs on NFS — skip it. We still advance `pending`
// so the next call has a stable cycle.
// NFS branch (or degraded fallback after a prior timeout): the
// WAIT_AFTER + DONTNEED dance hangs on NFS — skip it, but still
// advance `pending` so the next call has a stable cycle.
} else {
// Normal local-storage branch with belt-and-braces
// timeout. If WAIT_AFTER hangs > WAIT_AFTER_TIMEOUT
// we mark the pipeline degraded, log a loud error,
// and fall through to the skip path on subsequent
// calls.
// Normal local-storage branch with belt-and-braces timeout: if
// WAIT_AFTER hangs > WAIT_AFTER_TIMEOUT, mark degraded, log loudly,
// and fall through to the skip path on subsequent calls.
match wait_after_with_timeout(self.clone_for_worker(), self.fd, prev_off, prev_len)
{
Some(ms) => {
@@ -238,17 +233,13 @@ impl WritebackPipeline {
self.record_wait(wait_ms);
}
None => {
// Timeout branch: switch to NFS-style skip
// for the rest of the pipeline's life. Do
// NOT call DONTNEED — if WAIT_AFTER hasn't
// Timeout branch: switch to NFS-style skip for the rest of the
// pipeline's life. Do NOT call DONTNEED — if WAIT_AFTER hasn't
// returned, the pages aren't safely flushed.
self.degraded.store(true, Ordering::Relaxed);
// Once degraded we skip DONTNEED, so every subsequent
// chunk's pages stay resident until close — the same
// page-cache exposure profile as NFS. Shrink to the
// floor so that exposure window is as small as the NFS
// path keeps it, instead of whatever the adaptive sizing
// had grown chunk_bytes to (up to 256 MiB).
// Skipping DONTNEED leaves pages resident until close (same
// exposure as NFS), so shrink chunk_bytes to the floor rather
// than whatever adaptive sizing had grown it to (up to 256 MiB).
self.chunk_bytes = CHUNK_BYTES_MIN;
tracing::error!(
target: "mux",
@@ -294,10 +285,9 @@ impl WritebackPipeline {
if self.wait_after_window.len() < ADAPTIVE_WINDOW {
return;
}
// p95 index, derived from the window size so it stays valid if
// ADAPTIVE_WINDOW changes (a hard-coded `[14]` would panic OOB
// for a window <= 14). For the default 16 this is index 15
// (ceil(16 * 95 / 100) - 1 = 15), i.e. the top sample.
// p95 index, derived from window size so it stays valid if ADAPTIVE_WINDOW
// changes (a hard-coded `[14]` would panic OOB for a window <= 14). For the
// default 16 this is index 15, i.e. the top sample.
let mut sorted: Vec<u64> = self.wait_after_window.iter().copied().collect();
sorted.sort_unstable();
let p95_idx = (ADAPTIVE_WINDOW * 95).div_ceil(100).min(ADAPTIVE_WINDOW) - 1;
@@ -442,10 +432,8 @@ fn wait_after_with_timeout(
Err(crate::io::bounded::BoundedError::Timeout)
| Err(crate::io::bounded::BoundedError::Halted) => None,
Err(crate::io::bounded::BoundedError::WorkerLost) => {
// Worker thread spawn failed or panicked before sending.
// Treat as a benign success (no syscall ran) rather than
// a degrade trigger — falling through with elapsed_ms=0
// matches the no-op behaviour.
// Worker thread spawn failed or panicked before sending. Treat as benign
// success (no syscall ran), not a degrade trigger — elapsed_ms=0 matches no-op.
Some(0)
}
}
+3 -4
View File
@@ -17,10 +17,9 @@ use std::time::Duration;
/// Best-effort: a non-zero rc is logged but not propagated, since the
/// caller would just continue with the unreserved file anyway.
pub(super) fn preallocate(file: &File, size_bytes: u64) {
// FALLOC_FL_KEEP_SIZE = 0x01 — keep the reported file size at 0
// (writes grow it normally) while still pre-reserving the extents.
// Clamp to the signed `off_t` range; an unchecked `as i64` cast
// would wrap a >= 2^63 size to a negative length (EINVAL no-op).
// FALLOC_FL_KEEP_SIZE keeps the reported file size at 0 (writes grow it normally)
// while pre-reserving extents. Clamp to `off_t` range; an unchecked `as i64`
// cast would wrap a >= 2^63 size to a negative length (EINVAL no-op).
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX);
let rc = unsafe { libc::fallocate(file.as_raw_fd(), libc::FALLOC_FL_KEEP_SIZE, 0, len) };
tracing::debug!(
+3 -5
View File
@@ -215,11 +215,9 @@ mod tests {
"a halted F_FULLFSYNC must not be reported as a completed sync"
);
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each
// carries its own numeric code through the "E<code>" prefix that
// `From<Error> for io::Error` mints — the only shape `error_code`
// recognises. A bare `ErrorKind` cannot be classified, which is how a
// user cancel here used to read as a hard I/O failure.
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each carries its
// own numeric code via the "E<code>" prefix `From<Error> for io::Error` mints —
// a bare `ErrorKind` can't be classified, so a user cancel used to read as I/O failure.
let lost = bounded_failure_to_result(BoundedError::WorkerLost)
.expect_err("a lost F_FULLFSYNC worker must be an error");
assert!(
+12 -24
View File
@@ -143,8 +143,9 @@ impl WritebackFile {
/// Like [`Self::create`] but pre-reserves `size_bytes` of disk
/// space via the platform's extent-preallocation primitive (Linux
/// `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`, Windows
/// `SetFileValidData` stub). The reported file size is unchanged
/// `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`; Windows has no
/// equivalent, so it is a debug-logged no-op there). The reported
/// file size is unchanged
/// (writes still grow the file naturally) — only the on-disk extent
/// allocation is preallocated, which reduces extent fragmentation
/// on large sequential writes (mux output, especially on slow
@@ -228,18 +229,13 @@ impl Write for WritebackFile {
impl Seek for WritebackFile {
fn seek(&mut self, from: SeekFrom) -> io::Result<u64> {
let p = self.file.seek(from)?;
// Only treat seeks that actually move the position as
// boundaries — sweep does a redundant `seek(Current(pos))`
// before every write, and we don't want that to drain the
// pipeline on every iteration.
// Only treat seeks that actually move the position as boundaries — sweep does
// a redundant `seek(Current(pos))` before every write, which shouldn't drain
// the pipeline on every iteration.
if p != self.pos {
// Diagnostic for the NFS mux hang: the MKV format requires
// the muxer to seek back occasionally (cluster size
// patching, Cues index write, Segment header backpatch).
// Each such seek invalidates the writeback chunk tracking
// and forces a finalize → WAIT_AFTER on the in-flight
// chunk. Logging the seek delta lets us correlate hang
// offsets with specific muxer operations.
// Diagnostic for the NFS mux hang: MKV requires occasional backward seeks
// (cluster patching, Cues, Segment backpatch), each invalidating writeback
// chunk tracking. Logging the delta correlates hang offsets with muxer ops.
let from_pos = self.pos;
let to_pos = p;
let delta: i64 = (to_pos as i64).wrapping_sub(from_pos as i64);
@@ -274,17 +270,9 @@ impl super::sink::RandomAccessSink for WritebackFile {}
impl Drop for WritebackFile {
fn drop(&mut self) {
// Run the pipeline's tail finalize so the last in-flight chunk
// gets its `WAIT_AFTER` + `posix_fadvise(DONTNEED)`. Without
// this, callers that drop a `WritebackFile` without calling
// `sync_all` (panic, early-return, idiomatic `let _ = w;`)
// leave the trailing chunk in cache; the kernel still flushes
// on close, but the bounded-cache invariant fails at the tail.
// We deliberately do *not* call `self.file.sync_all()` here —
// close already triggers a flush, and an `fsync` from `Drop`
// would silently swallow its `io::Error` anyway. `finalize` is
// idempotent so an explicit `sync_all` followed by drop is
// still safe.
// Run the pipeline's tail finalize (WAIT_AFTER + DONTNEED); otherwise a drop
// without `sync_all` leaves the trailing chunk in cache. No `self.file.sync_all()`
// here — `Drop`-triggered fsync would swallow errors; `finalize` is idempotent.
self.pipeline.finalize();
}
}
+26 -67
View File
@@ -381,12 +381,9 @@ pub fn resolve_and_apply_traced(
let mut trace = crate::aacs::trace::ResolutionTrace::new();
// The FIRST source failure seen, if any. A source that returns `Err` did not
// answer "no key for this disc" — it could not answer at all — and that
// reason is stamped onto `disc.aacs_error` below so the decrypt gate reports
// THAT instead of the generic `NoDiscKey`. First-wins (not last) so the
// ordered sources' most-preferred failure is the one the operator is told
// about, matching the first-valid-wins rule for successes.
// The FIRST source failure seen, if any. An `Err` means the source could not
// answer at all (not "no key"), so its reason is stamped onto `disc.aacs_error`
// below, reporting the ordered sources' most-preferred failure to the operator.
let mut source_failure: Option<crate::error::Error> = None;
// The ctx parses Unit_Key_RO.inf at the stride for `inputs.version` (the
@@ -430,26 +427,9 @@ pub fn resolve_and_apply_traced(
outcome: KeyOutcome::NoKey,
});
}
// The source could NOT answer — it was unreachable, it errored, or it
// refused. Nothing is known about whether a key exists, so the path
// is EMPTY: recording `NoEntry` here is exactly the conflation that
// made a seven-hour run of HTTP 502s render as
// `key: online > no entry > NO KEY` + `E7022 No key source has a
// decryption key for this disc`, and sent operators hunting for a VUK
// that was never missing.
//
// The reason itself rides out on `disc.aacs_error` (below), the
// channel `Disc::ensure_decryptable_keys` already reads for the
// E7017-vs-E7022 split — so the decrypt gate raises the SOURCE's code
// (`KeyServiceUnavailable` / `KeyServiceUnauthorized` /
// `KeyServiceRateLimited`) instead of the generic `NoDiscKey`.
//
// `KeyOutcome` deliberately gains no variant: it is matched
// exhaustively by every front-end's trace renderer (freemkv's
// `pipe::render_resolution_trace`, autorip's
// `keysource::render_resolution_trace`), and this fix must not turn
// into a breaking change across four repos to say something the error
// code already says precisely.
// The source could NOT answer (unreachable/errored/refused), so path is
// EMPTY, not `NoEntry` — conflating the two once rendered 502s as "no
// key" and misled operators. The real reason rides on `disc.aacs_error`.
Err(e) => {
if source_failure.is_none() {
source_failure = Some(e);
@@ -462,10 +442,9 @@ pub fn resolve_and_apply_traced(
}
}
}
// Nothing resolved. If a source FAILED rather than answered, stamp that
// reason onto the disc so the decrypt gate can report it — but never clobber
// a reason the scan already captured (e.g. `AacsVidUnavailable`), which is
// closer to the disc itself than a source outage is.
// Nothing resolved. If a source FAILED rather than answered, stamp that reason
// onto the disc for the decrypt gate — but never clobber a reason the scan
// already captured (e.g. `AacsVidUnavailable`), which is closer to the disc.
if let Some(e) = source_failure
&& disc.aacs_error.is_none()
{
@@ -573,22 +552,9 @@ pub fn key_fetch(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
) -> crate::sector::KeyFetch {
// One driver behind both operations: rebuild the sources, inject `samples`
// as the ctx's content samples, run `drive` (the per-kind fetch), map the
// resolved UnitKeys to raw keys. Memoized by the fingerprint of the sample
// batch: the resolved keys are disc-level (a clip's index / CPS keys are
// identical for every title that references it), so the first batch resolves
// over the network and every repeat is answered from the cache with no
// request. A GENUINELY-empty reply (every source ran and none held the key)
// is cached too — the key the service lacks for a batch won't appear on a
// re-ask, so re-hitting the network buys nothing. But an empty reply caused
// by a source FAILURE (network down, source unreachable) is NOT cached: that
// is a transient miss, and caching it would permanently drop a unit that
// could be recovered once the source recovers — the `errored` flag on
// `FetchOutcome` draws exactly that line. Each operation gets its OWN cache:
// a base batch and a forensic anchor never collide, and the same bytes could
// legitimately resolve differently per op.
// The per-kind driver: `drive_unit_keys` or `drive_fmts_indexes`.
// One driver behind both operations, memoized per sample-batch fingerprint
// (keys are disc-level, so repeats hit cache). A genuinely-empty reply is
// cached, but a source FAILURE is not — a transient miss (see `errored`).
type FetchDriver = fn(&[Box<dyn KeySource>], &dyn ResolveCtx) -> FetchOutcome;
fn make_op(
inputs: DiscInputs,
@@ -614,9 +580,8 @@ pub fn key_fetch(
let mut di = inputs.clone();
di.samples = samples.to_vec();
// Parse Unit_Key_RO.inf at the disc's OWN stride (carried on `inputs`):
// an online /decode reply that returns a VUK (not a terminal UK) then
// derives unit keys from `enc_title_keys`, which a V10 disc parses at
// the 48-byte stride — hardcoding the V20 stride here corrupted them.
// a V10 disc parses `enc_title_keys` at the 48-byte stride, so
// hardcoding the V20 stride here would corrupt the derived unit keys.
let ctx = DiscInputsCtx::new(&di);
let outcome = drive(&sources, &ctx);
let keys: Vec<[u8; 16]> = outcome.keys.into_iter().map(|u| u.key).collect();
@@ -666,11 +631,9 @@ pub fn read_encrypted_units(
) -> Vec<Vec<u8>> {
use crate::aacs::content::{ALIGNED_UNIT_LEN, ALIGNED_UNIT_SECTORS, aacs_unit_encrypted};
const CHUNK_UNITS: u32 = 15; // 45 sectors/read — under the drive transfer cap
// Probe several evenly-spaced points across EACH extent rather than only the
// midpoint-and-forward: a title whose encrypted feature starts late, or whose
// midpoint lands in a clear nav stretch, must STILL yield scrambled samples.
// Empty samples make `Disc::decrypt_with` skip wrong-key validation, so a
// real encrypted title returning nothing here is a silent wrong-key hazard.
// Probe several evenly-spaced points across EACH extent, not just midpoint
// forward: a title starting late or landing in a clear nav stretch must still
// yield samples — empty samples make `decrypt_with` skip wrong-key validation.
const PROBES_PER_EXTENT: u32 = 8;
let mut out: Vec<Vec<u8>> = Vec::new();
@@ -687,20 +650,17 @@ pub fn read_encrypted_units(
continue;
}
let units_this = CHUNK_UNITS.min(total_units - unit);
// Saturate: start_lba comes from attacker-controlled UDF/MPLS
// extents; a malformed extent near u32::MAX would otherwise panic
// (debug) or wrap to a wrong LBA (release). An over-capacity LBA then
// fails cleanly via the read_sectors().is_err() skip below.
// Saturate: start_lba comes from attacker-controlled UDF/MPLS extents;
// near u32::MAX it would otherwise panic (debug) or wrap (release).
// An over-capacity LBA fails cleanly via the is_err() skip below.
let lba = ext
.start_lba
.saturating_add(unit.saturating_mul(ALIGNED_UNIT_SECTORS));
let count = (units_this * ALIGNED_UNIT_SECTORS) as u16;
let mut buf = vec![0u8; count as usize * 2048];
// `false` = no recovery retries; the reader is the raw drive/file
// (no decrypt decorator), so these are the on-disc encrypted bytes. A
// read error at one probe skips THAT probe only — it must not abandon
// the rest of the extent (the old `break` blinded the sampler on a
// single transient miss).
// `false` = no recovery retries; reader is the raw drive/file (no
// decrypt decorator). A read error skips only THAT probe — it must not
// abandon the rest of the extent (the old `break` blinded the sampler).
if reader.read_sectors(lba, count, &mut buf, false).is_err() {
continue;
}
@@ -1298,10 +1258,9 @@ mod tests {
use crate::error::Result;
use crate::sector::SectorSource;
// Even units: CPI-clear (byte0 & 0xC0 == 0) AND sync-destroyed (no 0x47).
// Odd units: CPI-set (byte0 = 0xC0) with a scrambled body.
// Neither has clean TS syncs, so `is_clean` is FALSE for BOTH;
// `aacs_unit_encrypted` flags only the odd units.
// Even units: CPI-clear, sync-destroyed. Odd units: CPI-set, scrambled body.
// Neither has clean TS syncs (`is_clean` is FALSE for both), but
// `aacs_unit_encrypted` must flag only the odd (CPI-set) units.
struct MixSource {
ext_start: u32,
total_units: u32,
+18 -25
View File
@@ -47,7 +47,9 @@ pub struct DiscMetadata {
pub titles: BTreeMap<String, String>,
/// First-line / short description, per lang
pub descriptions: BTreeMap<String, String>,
/// Disc N of M for box sets (None if not a box set)
/// Disc N of M, returned whenever both `<di:discNumber>` and
/// `<di:numSets>` are present and sane (including `(1, 1)` for a
/// single-disc release); `None` if either is missing or invalid.
pub disc_number: Option<(u32, u32)>,
}
@@ -76,10 +78,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<DiscMetadata>
let Some(lang) = lang_code_from_filename(&entry.name) else {
continue;
};
// entry.size is attacker-controlled UDF metadata and flows into
// a Vec::with_capacity in read_file. A real BDMV bdmt XML is a
// few KB; cap well above that so a crafted multi-GB size can't
// trigger a huge allocation before any parsing.
// entry.size is attacker-controlled and flows into a Vec::with_capacity
// in read_file. Cap well above a real few-KB bdmt XML so a crafted
// multi-GB size can't trigger a huge allocation before parsing.
if !bdmt_size_acceptable(entry.size) {
continue;
}
@@ -178,11 +179,9 @@ fn looks_like_xml(s: &str) -> bool {
/// helpers are case- and namespace-insensitive, so callers pass the
/// bare local name (no `di:` prefix).
fn extract_title(xml_text: &str) -> Option<String> {
// Order matches the module-level convention: <di:name> first
// (Paramount-style), then <di:title>, then the nested
// tableOfContents/titleName form.
// xml::text already trims its result, so an empty string after
// extraction means a genuinely empty element.
// Priority: <di:name> (Paramount-style), then <di:title>, then nested
// tableOfContents/titleName. xml::text trims, so an empty string here
// means a genuinely empty element.
for tag in ["name", "title"] {
if let Some(s) = xml::text(xml_text, tag)
&& !s.is_empty()
@@ -351,10 +350,9 @@ mod tests {
#[test]
fn multiple_languages_keyed_correctly() {
// Simulate driving parse_bdmt_xml from two synthetic XML
// blobs and aggregating into DiscMetadata the same way parse()
// would. This exercises the BTreeMap key handling without
// needing a UdfFs.
// Drive parse_bdmt_xml from two synthetic XML blobs and aggregate
// into DiscMetadata like parse() would, exercising BTreeMap key
// handling without needing a UdfFs.
let eng_xml = r#"<discInfo xmlns:di="urn:BDA:bdmv;disclibmeta">
<di:name>Aurora Drift</di:name>
</discInfo>"#;
@@ -395,12 +393,9 @@ mod tests {
#[test]
fn malformed_xml_returns_none() {
// Random gibberish has no recognizable title element. We
// document the contract: parse_bdmt_xml returns None, and
// parse() (the caller) skips the file. Aggregating across
// zero files leaves DiscMetadata::default() — which parse()
// surfaces as None to its caller. Either is documented as
// acceptable per the module spec.
// Gibberish has no recognizable title element: parse_bdmt_xml returns
// None and parse() skips the file. Zero files aggregated leaves
// DiscMetadata::default(), which parse() surfaces as None.
let bad = "this is not xml &&& <<< nope";
assert!(parse_bdmt_xml(bad).is_none());
@@ -411,11 +406,9 @@ mod tests {
#[test]
fn description_with_only_child_xml_is_dropped() {
// Real-world bug: <di:description> contained only
// <di:thumbnail/> child elements with no actual prose. The
// previous parser surfaced the raw XML fragment as the
// description string. Now we reject candidates that begin
// with `<`.
// Real-world bug: <di:description> contained only <di:thumbnail/>
// children with no prose, and the old parser surfaced the raw XML
// fragment as the description. Now candidates starting with `<` are rejected.
let xml = r#"<discInfo>
<di:name>Skyline Run</di:name>
<di:description>
+109 -67
View File
@@ -10,10 +10,9 @@
//! for any label parser that needs structured access to .class files
//! inside a `/BDMV/JAR/<x>.jar`.
// Foundation module — public API is staged for `labels::deluxe` (which
// will exercise the bytecode walker) and `labels::dbp`'s refactor onto
// the constant-pool iterator. The dead-code allow comes off as those
// callers land. Tests below cover the API in isolation.
// Foundation module — public API is staged for `labels::deluxe` (bytecode
// walker) and `labels::dbp`'s refactor onto the constant-pool iterator.
// The dead-code allow comes off once those callers land.
#![allow(dead_code)]
const CLASS_MAGIC: u32 = 0xCAFEBABE;
@@ -32,10 +31,9 @@ pub enum Error {
BadInstruction { pc: usize, opcode: u8 },
}
// No Display/std::error::Error impl: this is a crate-internal, typed error
// used only for `match`/`?` within the label parsers (callers discard it via
// `let Ok(_) = ... else continue`). Per the library's zero-English rule there
// is no user-facing text; the variant fields carry the structured detail.
// No Display/std::error::Error impl: crate-internal typed error for `match`/`?`
// within label parsers. Per the zero-English rule there is no user-facing
// text; variant fields carry the structured detail.
pub type Result<T> = std::result::Result<T, Error>;
@@ -155,7 +153,8 @@ impl ConstantPool {
}
/// For `ldc` / `ldc_w` operands: resolve a constant-pool index to a
/// best-effort string. Supports Utf8, String, Integer, Class.
/// best-effort string. Supports Utf8, String, Integer, Float, Long,
/// Double, and Class.
pub fn load_constant_display(&self, index: u16) -> Option<String> {
Some(match self.get(index)? {
CpInfo::Utf8(s) => format!("utf8:{:?}", s),
@@ -588,7 +587,7 @@ impl Instruction<'_> {
/// ldc_w, ldc2_w, new, getstatic, putstatic, getfield, putfield,
/// invokevirtual, invokespecial, invokestatic, invokeinterface,
/// invokedynamic, checkcast, instanceof, anewarray, multianewarray,
/// ldc with cp index in operands[0] — return the index. Returns
/// ldc with cp index in `operands[0]` — return the index. Returns
/// None for opcodes whose operand is not a CP index.
pub fn cp_index(&self) -> Option<u16> {
match self.opcode {
@@ -657,10 +656,9 @@ fn instruction_size(code: &[u8], pc: usize) -> Option<usize> {
if high < low {
return None;
}
// `high - low + 1` can overflow i32 for adversarial bytecode
// (e.g. low=i32::MIN/high=0, or low=0/high=i32::MAX), so widen
// to i64 before adding. The product and final sum are saturating
// so they cannot overflow usize on a 32-bit target either.
// `high - low + 1` can overflow i32 for adversarial bytecode, so widen
// to i64 before adding; the product/sum below saturate so they can't
// overflow usize on a 32-bit target either.
let entries = (high as i64 - low as i64 + 1) as u64;
let table_bytes = entries.saturating_mul(4);
let base = (padded_start - pc + 12) as u64;
@@ -989,14 +987,9 @@ impl<'a> Reader<'a> {
}
fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> {
// `n` is attacker-supplied: it comes from a JVMS `u4` attribute_length
// / code_length (§4.7, §4.7.3) or a `u2` Utf8 length (§4.4.7). Unlike
// the fixed-width readers above, whose `self.pos + k` cannot leave the
// buffer's own address range, `self.pos + n` can wrap — on a 32-bit
// target a `u4` length near 0xFFFF_FFFF plus a non-zero `pos` panics
// in debug and in release wraps to a SMALL end offset that passes the
// bounds check, after which the slice index itself panics. Checked, so
// an out-of-range length is the EOF error it always should have been.
// `n` is attacker-supplied (a JVMS u4/u2 length), so unlike the fixed-width
// readers above, `self.pos + n` can wrap on a 32-bit target and pass a naive
// bounds check. checked_add turns an out-of-range length into an EOF error.
let Some(end) = self.pos.checked_add(n) else {
return Err(Error::UnexpectedEof { needed });
};
@@ -1130,10 +1123,9 @@ mod tests {
#[test]
fn instruction_size_tableswitch_overflow_does_not_panic() {
// Adversarial low/high spanning the full i32 range. `high - low + 1`
// overflows i32; the widened i64 count then saturates the byte
// products. Must return a value (possibly None on a 32-bit usize)
// without panicking.
// Adversarial low/high spanning the full i32 range overflows `high - low + 1`
// in i32; the widened i64 count then saturates the byte products. Must
// return a value (possibly None on a 32-bit usize) without panicking.
for (low, high) in [
(i32::MIN, 0i32),
(0i32, i32::MAX),
@@ -1225,14 +1217,9 @@ mod tests {
assert_eq!(i.cp_index(), None);
}
// ── Robustness smoke tests ──────────────────────────────────────────────
//
// ClassFile::parse must NEVER panic on adversarial input, only
// return Err. These tests feed a battery of malformed byte
// sequences and assert Err results — they're the lightweight
// alternative to a full cargo-fuzz target (which would need
// nightly + separate crate). If we adopt cargo-fuzz later, these
// tests stay as deterministic regression cases.
// ── Robustness smoke tests ── ClassFile::parse must NEVER panic on
// adversarial input, only return Err. Lightweight alternative to a full
// cargo-fuzz target; stays useful as deterministic regression cases.
/// Tiny pseudo-random byte generator — deterministic + reproducible
/// without needing a `rand` dep. xorshift64*; good enough for
@@ -1306,10 +1293,9 @@ mod tests {
#[test]
fn parse_does_not_panic_on_random_bytes() {
// 200 deterministic-pseudo-random byte buffers of varying
// lengths. The contract: never panic, only return Err (or in
// the vanishingly unlikely case of a coincidentally-valid
// buffer, Ok — we don't assert one or the other).
// 200 deterministic pseudo-random buffers of varying lengths.
// Contract: never panic, only return Err (or Ok in the vanishingly
// unlikely coincidentally-valid case — we don't assert which).
let mut state: u64 = 0xDEADBEEF_DEADBEEF;
for _ in 0..200 {
let len = (xorshift(&mut state) % 256) as usize;
@@ -1325,10 +1311,9 @@ mod tests {
#[test]
fn parse_does_not_panic_on_valid_magic_random_tail() {
// 100 buffers that start with the magic + plausible
// minor/major but have garbage afterwards. These are the
// most adversarial — they pass the magic check and then
// exercise every other parser path.
// 100 buffers with valid magic + plausible minor/major but garbage
// afterwards — the most adversarial, passing the magic check then
// exercising every other parser path.
let mut state: u64 = 0xCAFEBABE_DEADBEEF;
for _ in 0..100 {
let mut buf = vec![0xCA, 0xFE, 0xBA, 0xBE, 0x00, 0x00, 0x00, 0x34];
@@ -1389,14 +1374,9 @@ mod tests {
}
}
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness
//
// These exercise plain data accessors on an already-parsed pool
// (built via the test-only `from_entries` constructor) — not the
// untrusted-bytes parsing path, just "does the right variant map to
// the right Option value."
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness: exercise data accessors
// on an already-parsed pool (test-only `from_entries` ctor), checking each
// variant maps to the right Option value.
fn sample_pool() -> ConstantPool {
// index: 0=Empty (reserved), 1=Utf8("Hello"), 2=Integer(42),
@@ -1547,14 +1527,9 @@ mod tests {
assert_ne!(cf.member_descriptor(&m), Some("doStuff"));
}
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness
//
// Mirrors `slice_boundary_is_inclusive_of_the_final_byte`: an
// exact-fit read must succeed, one byte short must fail. Plus
// positive-value tests so a scrambled byte assembly (not just an
// out-of-bounds read) would be caught.
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness: an exact-fit read
// succeeds, one byte short fails, plus positive-value tests so a
// scrambled byte assembly (not just OOB) is caught.
#[test]
fn u16_boundary_is_inclusive_of_the_final_byte() {
@@ -1656,10 +1631,9 @@ mod tests {
#[test]
fn constant_pool_long_entry_occupies_two_slots_via_real_parse() {
// Real class-file bytes (not the `from_entries` synthetic ctor):
// magic + minor/major + cp_count=4 + tag=5 (Long, 8-byte payload
// at index 1, reserved slot at index 2) + tag=1 (Utf8 at index 3)
// + empty access_flags/this/super/interfaces/fields/methods/attrs.
// Real class-file bytes (not the from_entries ctor): magic, minor/major,
// cp_count=4, tag=5 (Long, payload at index 1, reserved slot 2),
// tag=1 (Utf8 at index 3), then empty header tail sections.
let mut buf = vec![
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, // minor
@@ -1691,12 +1665,9 @@ mod tests {
}
}
// -----------------------------------------------------------------
// instruction_size: tableswitch/lookupswitch with non-degenerate
// low/high/npairs (the existing tests only cover low==high==0 and
// npairs==0, which can't distinguish `-` from `+` in the entry-count
// arithmetic).
// -----------------------------------------------------------------
// low/high/npairs — existing tests only cover the all-zero case, which
// can't distinguish `-` from `+` in the entry-count arithmetic.
#[test]
fn instruction_size_tableswitch_non_degenerate_range() {
@@ -1723,4 +1694,75 @@ mod tests {
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
// ── Malformed-input hardening: every count/length/index is attacker- ──────
// controlled, so an oversized or lying field must yield Err/None, never a
// panic or an unbounded allocation.
/// A `constant_pool_count` far larger than the data behind it must fail
/// cleanly (running out of bytes), not pre-allocate gigabytes or panic. The
/// count is a u16, so `Vec::with_capacity` is bounded at 65535 regardless.
#[test]
fn parse_rejects_oversized_constant_pool_count() {
let bytes = [
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, 0x00, 0x00, // minor / major
0xFF, 0xFF, // constant_pool_count = 65535, but no entries follow
];
assert!(matches!(
ClassFile::parse(&bytes),
Err(Error::UnexpectedEof { .. })
));
}
/// An oversized `interfaces_count` / `fields_count` with no data behind it
/// must fail on the first missing element, not panic.
#[test]
fn parse_rejects_oversized_interface_count() {
let bytes = [
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, 0x00, 0x00, // minor / major
0x00, 0x01, // constant_pool_count = 1 (no entries)
0x00, 0x00, // access_flags
0x00, 0x00, // this_class
0x00, 0x00, // super_class
0xFF, 0xFF, // interfaces_count = 65535, none follow
];
assert!(matches!(
ClassFile::parse(&bytes),
Err(Error::UnexpectedEof { .. })
));
}
/// Out-of-range constant-pool indices resolve to `None` from every
/// accessor a label parser drives — no slice panic.
#[test]
fn constant_pool_out_of_range_index_is_none_everywhere() {
let pool = sample_pool();
let oob = 9999u16;
assert!(pool.get(oob).is_none());
assert!(pool.utf8(oob).is_none());
assert!(pool.class_name(oob).is_none());
assert!(pool.string(oob).is_none());
assert!(pool.integer(oob).is_none());
assert!(pool.member_ref(oob).is_none());
assert!(pool.load_constant_display(oob).is_none());
}
/// A `Code` attribute whose declared `code_length` runs past the attribute
/// body must be rejected, not read out of bounds.
#[test]
fn code_attribute_rejects_length_past_its_body() {
// max_stack(2) + max_locals(2) + code_length(4) then a code_length that
// exceeds the remaining bytes.
let mut info = Vec::new();
info.extend_from_slice(&0u16.to_be_bytes()); // max_stack
info.extend_from_slice(&0u16.to_be_bytes()); // max_locals
info.extend_from_slice(&0xFFFF_FFFFu32.to_be_bytes()); // code_length: absurd
info.extend_from_slice(&[0x00, 0x01]); // only 2 code bytes present
assert!(matches!(
parse_code_attribute(&info),
Err(Error::UnexpectedEof { .. })
));
}
}
+3 -4
View File
@@ -116,10 +116,9 @@ impl ClpiVsMplsAudit {
/// merged view. Missing files (read errors, parse failures) are
/// silently skipped — this is diagnostic, not correctness-critical.
pub fn audit(reader: &mut dyn SectorSource, udf: &UdfFs) -> ClpiVsMplsAudit {
// Aggregate by PID across all CLPI files. If a PID appears in
// multiple clips (typical — main movie clip + trailers reference
// the same audio stream PIDs), first encountered wins (they should
// all agree per BD spec).
// Aggregate by PID across all CLPI files. A PID commonly repeats across
// clips (main movie + trailers sharing audio PIDs); first encountered
// wins, since they should all agree per BD spec.
let mut clpi_by_pid: BTreeMap<u16, (u8, String)> = BTreeMap::new();
if let Some(dir) = udf.find_dir("/BDMV/CLIPINF") {
let names: Vec<String> = dir
+4 -7
View File
@@ -101,10 +101,8 @@ fn assign_stream_numbers(
const NUMBER_SPACE_END: u32 = u16::MAX as u32 + 1;
// Numbers already claimed by the map, per type. A map value of 0 is NOT a
// claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable.
// Treat 0 as "unmapped" here (defense in depth — parse_playback_config also
// filters it) so such a stream gets a real synthesized number instead of an
// orphan 0 that collides with / shadows a genuine stream 1.
// claim (apply_labels binds 1-based numbers, so 0 is unmatchable); treat it
// as unmapped so the stream gets a real number instead of colliding with 1.
let mut taken_audio: Vec<u16> = Vec::new();
let mut taken_sub: Vec<u16> = Vec::new();
for info in infos {
@@ -646,9 +644,8 @@ mod tests {
assert_eq!(map.get("sub1").copied(), Some(3));
}
/// Spec: high confidence is returned when streamproperties.xml is fully
/// structured (no fallback). This is the Criterion parser's claim.
/// Mutation: change to ParseResult::medium → confidence assertion fails.
/// Spec: `LangInfoID` values are lowercased so they match
/// `apply_labels`' lookup (an uppercase "ENG" must parse as "eng").
#[test]
fn parse_stream_infos_language_lowercased() {
// LangInfoID values must be lowercased so they match apply_labels' lookup.
+15 -23
View File
@@ -56,11 +56,9 @@ fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
label.name = mb_match.name.clone();
}
}
// Append any menu_base-only stream (present in mb but not in ls by
// (stream_type, stream_number)). Without this the both-files path
// silently drops streams the menu_base-only path would have emitted:
// language_streams is authoritative for type/purpose but is not
// necessarily a superset of menu_base.
// Append menu_base-only streams (present in mb but not in ls by
// (stream_type, stream_number)); language_streams is authoritative for
// type/purpose but not necessarily a superset of menu_base.
for mb_label in mb {
let already = result.iter().any(|l| {
l.stream_type == mb_label.stream_type && l.stream_number == mb_label.stream_number
@@ -197,11 +195,9 @@ fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
"csp" | "cs" | "lsp" | "ls" | "cf" | "pf" | "bp" | "pp" => {
variant_code = variant.clone();
}
// Everything else: defer to vocab::codec as the single
// source of codec-name truth. If it recognizes the token
// (returns something other than the input) it's a known
// codec — store the canonical name. Otherwise it's an
// unknown token, stored as-is.
// Everything else: defer to vocab::codec as the single source of
// codec-name truth — a known token gets its canonical name,
// otherwise it's stored as-is.
_ => codec_hint = vocab::codec(&variant).to_string(),
}
}
@@ -222,10 +218,9 @@ fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
labels
}
// NOTE: `parse_menu_base` / `parse_menu_base_text` are defined just below this
// module and structurally belong above it. They are left in place (with the
// lint allowed) rather than relocated here — a ~120-line block move that is
// safer to do as its own focused change than inline.
// NOTE: `parse_menu_base` / `parse_menu_base_text` structurally belong above this
// module but are left below (lint allowed) — a ~120-line move is safer as its
// own focused change.
#[allow(clippy::items_after_test_module)]
#[cfg(test)]
mod tests {
@@ -311,10 +306,9 @@ mod tests {
#[test]
fn dual_flag_entry_resolves_to_audio_with_no_subtitle_qualifier() {
// An entry tripping BOTH flags (audio_ prefix sets is_audio,
// class "SubtitleButton" sets is_subtitle). Audio wins the type,
// and the subtitle qualifier (SDH) must NOT be carried onto the
// resulting Audio label. Regression for the type/qualifier split.
// An entry tripping BOTH flags (audio_ prefix sets is_audio, class
// "SubtitleButton" sets is_subtitle): audio wins the type, and the
// subtitle qualifier (SDH) must NOT carry onto the Audio label.
let labels = parse_props(
"audio_1.class=SubtitleButton\n\
audio_1.streamNumber=6\n\
@@ -885,11 +879,9 @@ fn parse_menu_base_text(text: &str) -> Vec<StreamLabel> {
let name = props.get("name").cloned().unwrap_or_default();
// Purpose: ask vocab first (word-boundary matched — avoids the
// "Commenter" false positive the prior `name.contains("comment")`
// had). Then fall back to the structural prefix check
// (`audio_commentary.foo`-style keys group commentary streams
// regardless of display name).
// Ask vocab first (word-boundary matched, avoiding the "Commenter"
// false positive of the old `name.contains("comment")`), then fall
// back to the structural prefix check (`audio_commentary.foo`-style).
let purpose = match vocab::purpose(&name) {
LabelPurpose::Normal if prefix_is_commentary(prefix) => LabelPurpose::Commentary,
p => p,
+3 -5
View File
@@ -67,11 +67,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
// BTreeMap so we keep the highest-numbered (last-written) label
// for each stream slot deterministic across runs. The same
// TextField,Audio1,... string can appear in multiple classes
// (button-state variants, localization fallbacks). Last write
// wins — they should all agree, but the structure is defensive.
// BTreeMap keeps the last-written label per stream slot deterministic.
// The same TextField,Audio1,... string can appear in multiple classes
// (button-state variants, fallbacks); last write should agree, but wins defensively.
let mut audios: BTreeMap<u16, String> = BTreeMap::new();
let mut subs: BTreeMap<u16, String> = BTreeMap::new();
+455 -215
View File
File diff suppressed because it is too large Load Diff
+522
View File
@@ -0,0 +1,522 @@
//! Fox — loose `/BDMV/JAR/<id>/dcx.xml` plain-XML manifest.
//!
//! Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
//! human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
//! under `<disc>` it lists every playlist the disc plays. The main-feature
//! playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
//! language, editorial purpose and forced/SDH state outright — everything a
//! label parser wants, in attributes, with no bytecode to walk.
//!
//! NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
//! is one authoring house's internal metadata that happens to press onto the
//! disc. Every field meaning below was read off a real disc; treat an
//! unfamiliar value as unknown rather than guessing.
//!
//! Confirmed schema (a real Fox release, `/BDMV/JAR/05001/dcx.xml`):
//!
//! ```xml
//! <dcx>
//! <disc>
//! <playlist id="00001" lang="eng" name="topmenu"/>
//! ...
//! <playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
//! <audio id="01" lang="eng" type="feature"/>
//! <audio id="02" lang="eng" type="rnib"/>
//! <audio id="03" lang="spa" dial="lat" type="feature"/>
//! ...
//! <subtitle id="01" lang="eng" type="feature" form="sdh"/>
//! <subtitle id="02" lang="spa" dial="lat" type="embed"/>
//! ...
//! <subtitle id="11" lang="eng" type="text"/>
//! <properties> ...chapter marks... </properties>
//! </playlist>
//! <playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
//! </disc>
//! </dcx>
//! ```
//!
//! Field meanings (read off the disc, not documented):
//! * `<audio type>`: `feature` = a normal program track; `rnib` = a
//! descriptive/narration track ("Royal National Institute of Blind People"
//! described-video), mapped to [`LabelPurpose::Descriptive`]. A `type`
//! containing "comment" maps to [`LabelPurpose::Commentary`].
//! * `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
//! hearing track ([`LabelQualifier::Sdh`]).
//! * `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
//! track ([`LabelQualifier::Forced`]); `feature`/`text` are full tracks.
//! * `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
//! (audio ids 01..N, subtitle ids 01..N independently), matching the vendor
//! `stream_number` convention the ordinal binder in [`super::apply_labels`]
//! reads.
//!
//! The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
//! so extraction is scoped to a single `<playlist name="feature">` element —
//! never a document-wide `<audio>` scan, which would merge the regional
//! `00800` (eng) and `00801` (jpn) tables into one and collide their slots.
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorSource;
use crate::udf::UdfFs;
/// Detect a Fox disc.
///
/// Primary, cheap signal: a loose `dcx.xml` under some `/BDMV/JAR/<id>/`
/// subdirectory — the manifest [`parse`] reads. Checked first because it is a
/// directory walk with no sector reads.
///
/// Secondary signal: a `com/foxbd/` prefix in a top-level BD-J jar, which newer
/// Fox discs (later franchise sequels) carry instead of a loose `dcx.xml`. Recognising
/// it attributes the disc to Fox rather than the generic deluxe stub even
/// though [`parse`] cannot decode that bytecode form yet (see the Phase 2 note
/// at the bottom of this file); such a disc detects here but yields no labels,
/// so a lower-confidence parser still wins.
pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
if super::jar_file_exists(udf, "dcx.xml") {
return true;
}
super::jar::for_each_jar(reader, udf, |_, jar| {
super::jar::has_path_prefix(jar, "com/foxbd/").then_some(())
})
.is_some()
}
pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult> {
let data = super::read_jar_file(reader, udf, "dcx.xml")?;
let text = std::str::from_utf8(&data).ok()?;
let labels = labels_from_dcx(text);
if labels.is_empty() {
return None;
}
// High confidence: the manifest is fully structured and every field
// extracted here has its meaning fixed by real discs.
let mut result = ParseResult::high(labels);
// Surface the feature playlist's authoring id (e.g. "00800") so title
// selection can prefer the disc's own feature over a size-inflated decoy —
// the same signal `paramount::parse` provides.
result.feature_playlist = feature_playlist_id(text).map(|id| super::FeaturePlaylistHint {
playlist_id: id.parse::<u16>().ok(),
filename: Some(format!("{id}.mpls")),
});
Some(result)
}
/// Build the stream labels from a `dcx.xml` document. Split out from [`parse`]
/// so the schema mapping is unit-testable without a `SectorSource`/`UdfFs`.
///
/// Scoped to ONE feature playlist element (see the module note): the richest
/// `<playlist name="feature">`. Its nested `<audio>`/`<subtitle>` children are
/// the vendor's STN table for that title.
pub(crate) fn labels_from_dcx(text: &str) -> Vec<StreamLabel> {
let Some(feature) = select_feature_playlist(text) else {
return Vec::new();
};
let mut labels = Vec::new();
// Audio streams.
let mut from = 0;
while let Some((s, e)) = xml::find_element(feature, "audio", from) {
let el = &feature[s..e];
from = e;
let Some(stream_number) = stream_number_from_id(&xml::attr(el, "id")) else {
continue;
};
let Some(language) = xml::attr(el, "lang") else {
continue;
};
let ty = xml::attr(el, "type")
.unwrap_or_default()
.to_ascii_lowercase();
labels.push(StreamLabel {
stream_id: None,
stream_number,
stream_type: StreamLabelType::Audio,
language: normalize_language(&language),
name: String::new(),
purpose: audio_purpose(&ty),
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
});
}
// Subtitle streams.
let mut from = 0;
while let Some((s, e)) = xml::find_element(feature, "subtitle", from) {
let el = &feature[s..e];
from = e;
let Some(stream_number) = stream_number_from_id(&xml::attr(el, "id")) else {
continue;
};
let Some(language) = xml::attr(el, "lang") else {
continue;
};
let ty = xml::attr(el, "type")
.unwrap_or_default()
.to_ascii_lowercase();
let form = xml::attr(el, "form")
.unwrap_or_default()
.to_ascii_lowercase();
labels.push(StreamLabel {
stream_id: None,
stream_number,
stream_type: StreamLabelType::Subtitle,
language: normalize_language(&language),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: subtitle_qualifier(&ty, &form),
codec_hint: String::new(),
variant: String::new(),
});
}
labels
}
/// Pick the disc's primary feature playlist element (the full `<playlist ...>
/// ... </playlist>` text). Among all `<playlist name="feature">` elements —
/// Fox presses one per regional variant (`00800` eng, `00801` jpn) — the one
/// carrying the most `<audio>`/`<subtitle>` streams wins, ties to the first.
///
/// Returning ONE element is the whole point: each feature playlist is its own
/// STN table with its own 1-based slot ids, so merging two would put two
/// different streams on slot `01`. The richest table is the fullest label set
/// and anchors to its matching title by language sequence in
/// [`super::apply_labels`].
fn select_feature_playlist(text: &str) -> Option<&str> {
let mut best: Option<&str> = None;
let mut best_streams = 0usize;
let mut from = 0;
while let Some((s, e)) = xml::find_element(text, "playlist", from) {
let element = &text[s..e];
from = e;
// `name` is read from the element; the opening tag's `name="feature"`
// is the first `name=` in document order, ahead of any nested child
// (nested `<audio>`/`<subtitle>` carry no `name`).
let is_feature =
xml::attr(element, "name").is_some_and(|n| n.eq_ignore_ascii_case("feature"));
if !is_feature {
continue;
}
let streams = count_elements(element, "audio") + count_elements(element, "subtitle");
// First feature wins the tie (matches the registry's first-wins rule);
// a strictly richer table displaces it. `best.is_none()` so a feature
// with zero nested streams is still selected — its id feeds the hint.
if best.is_none() || streams > best_streams {
best_streams = streams;
best = Some(element);
}
}
best
}
/// The authoring id (digits only, e.g. "00800") of the selected feature
/// playlist, or `None` when no feature playlist / no id is present.
fn feature_playlist_id(text: &str) -> Option<String> {
let feature = select_feature_playlist(text)?;
let id = xml::attr(feature, "id")?;
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
Some(digits)
}
/// Count `<tag>` elements inside `element`.
fn count_elements(element: &str, tag: &str) -> usize {
let mut n = 0;
let mut from = 0;
while let Some((_, e)) = xml::find_element(element, tag, from) {
n += 1;
from = e;
}
n
}
/// Parse a nested stream `id` (e.g. "01", "11") into its 1-based STN slot.
/// Digits only, so a stray quote/space cannot poison it; `None` (skip the
/// stream) when it names no slot — 0 is the module's `NO_STN_SLOT` sentinel and
/// cannot be bound by the ordinal path.
fn stream_number_from_id(id: &Option<String>) -> Option<u16> {
let id = id.as_deref()?;
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
digits.parse::<u16>().ok().filter(|&n| n != 0)
}
/// Map an `<audio type>` value to a [`LabelPurpose`]. `rnib` is described-video
/// (descriptive/narration); a `comment*` value is commentary; everything else
/// (`feature`, unknown) is a normal program track.
fn audio_purpose(ty: &str) -> LabelPurpose {
if ty == "rnib" {
LabelPurpose::Descriptive
} else if ty.contains("comment") {
LabelPurpose::Commentary
} else {
LabelPurpose::Normal
}
}
/// Map a `<subtitle>`'s `form`/`type` to a [`LabelQualifier`]. `form="sdh"`
/// takes precedence (an SDH track is the editorially meaningful flag);
/// otherwise `type="embed"` is a dedicated forced/embedded track. `feature`
/// and `text` full tracks carry no qualifier.
fn subtitle_qualifier(ty: &str, form: &str) -> LabelQualifier {
if form == "sdh" {
LabelQualifier::Sdh
} else if ty == "embed" {
LabelQualifier::Forced
} else {
LabelQualifier::None
}
}
/// Trim + lowercase the raw ISO 639-2 code (`"eng"`, `"fra"`, ...).
fn normalize_language(raw: &str) -> String {
raw.trim().to_ascii_lowercase()
}
// Phase 2 (design only, not implemented): newer Fox discs ship no loose `dcx.xml`,
// wrapping the same per-stream data in `com/foxbd` BD-J `.class` bytecode instead. A
// follow-on parser would reuse `super::class_reader`/`jar` (as `dbp`/`deluxe` do).
#[cfg(test)]
mod tests {
use super::*;
/// A real Fox-release manifest, reduced only by truncating the giant
/// chapter-mark `<properties>` blocks (which carry no stream labels). Every
/// `<audio>`/`<subtitle>` element and both regional feature playlists are
/// verbatim from a real disc's `/BDMV/JAR/05001/dcx.xml`, so this fixture exercises the
/// exact bytes production sees: feature selection across two `name="feature"`
/// playlists, the nested-scope rule, and the rnib/sdh/embed flags.
const FOX_DCX_SAMPLE: &str = r#"<dcx>
<disc>
<properties region="ABC" regioncheckon="false" parentallevel="PG" hdronlydisc="true" bootstrap.bdjo="88888"
vstaskenabled="false" defaultversion="1" topmenushowmarkid="0" topmenuloopmarkid="0"/>
<playlist id="00001" lang="eng" name="topmenu"/>
<playlist id="00100" lang="eng" name="foxlogo"/>
<playlist id="00300" lang="eng" name="copyright"/> <!-- English -->
<playlist id="00600" name="black" lang="eng"/>
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
<audio id="04" lang="fra" dial="par" type="feature"/>
<audio id="05" lang="dan" type="feature"/>
<audio id="06" lang="nld" type="feature"/>
<audio id="07" lang="fin" type="feature"/>
<audio id="08" lang="deu" type="feature"/>
<audio id="09" lang="ita" type="feature"/>
<audio id="10" lang="nor" type="feature"/>
<audio id="11" lang="swe" type="feature"/>
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
<subtitle id="03" lang="fra" dial="par" type="embed"/>
<subtitle id="04" lang="dan" type="embed"/>
<subtitle id="05" lang="nld" type="embed"/>
<subtitle id="06" lang="fin" type="feature"/>
<subtitle id="07" lang="deu" type="embed"/>
<subtitle id="08" lang="ita" type="embed"/>
<subtitle id="09" lang="nor" type="embed"/>
<subtitle id="10" lang="swe" type="embed"/>
<subtitle id="11" lang="eng" type="text"/>
<properties>
<entry.marks ids="0,2,4,6,8,10"/>
<playlist.marks timecodes="00:00:00:00,00:03:58:18"/>
</properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="jpn" type="feature"/>
<subtitle id="01" lang="jpn" type="feature"/>
<subtitle id="02" lang="eng" type="feature" form="sdh"/>
<subtitle id="03" lang="jpn" type="text"/>
<subtitle id="04" lang="eng" type="text"/>
<properties>
<entry.marks ids="0,2,4"/>
</properties>
</playlist>
</disc>
</dcx>"#;
fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.collect()
}
fn subs(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.collect()
}
/// The primary feature is the richer `00800` (eng) table, not `00801`
/// (jpn). Both are `name="feature"`; selection must pick by stream count,
/// and the id we surface for the (future) FeaturePlaylistHint is `00800`.
#[test]
fn selects_richest_feature_playlist_and_its_id() {
assert_eq!(
feature_playlist_id(FOX_DCX_SAMPLE),
Some("00800".to_string())
);
}
/// Full real-disc parse: the 00800 audio table. Eleven tracks, id order =
/// STN slot, and slot 2 (`eng rnib`) is the descriptive/narration track.
#[test]
fn fox_dcx_audio_labels() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let a = audio(&labels);
assert_eq!(
a.len(),
11,
"00800 has eleven audio tracks, 00801 not merged"
);
// Slots and languages, in id order.
let got: Vec<(u16, &str, LabelPurpose)> = a
.iter()
.map(|l| (l.stream_number, l.language.as_str(), l.purpose))
.collect();
assert_eq!(
got,
vec![
(1, "eng", LabelPurpose::Normal),
(2, "eng", LabelPurpose::Descriptive), // rnib = described video
(3, "spa", LabelPurpose::Normal),
(4, "fra", LabelPurpose::Normal),
(5, "dan", LabelPurpose::Normal),
(6, "nld", LabelPurpose::Normal),
(7, "fin", LabelPurpose::Normal),
(8, "deu", LabelPurpose::Normal),
(9, "ita", LabelPurpose::Normal),
(10, "nor", LabelPurpose::Normal),
(11, "swe", LabelPurpose::Normal),
]
);
// Vendor labels: no StreamId, they bind by ordinal STN slot.
assert!(a.iter().all(|l| l.stream_id.is_none()));
}
/// Full real-disc parse: the 00800 subtitle table. `form="sdh"` → Sdh,
/// `type="embed"` → Forced, `feature`/`text` → no qualifier.
#[test]
fn fox_dcx_subtitle_labels() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let s = subs(&labels);
assert_eq!(s.len(), 11, "00800 has eleven subtitle tracks");
let got: Vec<(u16, &str, LabelQualifier)> = s
.iter()
.map(|l| (l.stream_number, l.language.as_str(), l.qualifier))
.collect();
assert_eq!(
got,
vec![
(1, "eng", LabelQualifier::Sdh), // feature + form=sdh
(2, "spa", LabelQualifier::Forced), // embed
(3, "fra", LabelQualifier::Forced),
(4, "dan", LabelQualifier::Forced),
(5, "nld", LabelQualifier::Forced),
(6, "fin", LabelQualifier::None), // feature
(7, "deu", LabelQualifier::Forced),
(8, "ita", LabelQualifier::Forced),
(9, "nor", LabelQualifier::Forced),
(10, "swe", LabelQualifier::Forced),
(11, "eng", LabelQualifier::None), // text
]
);
}
/// The nested-scope rule: audio/subtitle come from ONE feature playlist,
/// never a document-wide scan that would merge 00800 and 00801 and collide
/// their `id="01"` slots. If scoping regressed, slot 1 audio would appear
/// twice and the count would jump past 11.
#[test]
fn does_not_merge_regional_feature_playlists() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let a = audio(&labels);
// Exactly one audio label per STN slot 1..=11.
let slot1: Vec<_> = a.iter().filter(|l| l.stream_number == 1).collect();
assert_eq!(slot1.len(), 1, "one stream on slot 1, not one per playlist");
assert_eq!(audio(&labels).len() + subs(&labels).len(), 22);
}
/// The `rnib` described-video mapping in isolation, plus the commentary
/// path (no track in the sample manifest uses it, so it is pinned synthetically).
#[test]
fn audio_purpose_mapping() {
assert_eq!(audio_purpose("feature"), LabelPurpose::Normal);
assert_eq!(audio_purpose("rnib"), LabelPurpose::Descriptive);
assert_eq!(audio_purpose("commentary"), LabelPurpose::Commentary);
assert_eq!(
audio_purpose("director-commentary"),
LabelPurpose::Commentary
);
assert_eq!(audio_purpose("unknown"), LabelPurpose::Normal);
}
/// `form="sdh"` outranks `type`; `embed` is forced; full tracks are None.
#[test]
fn subtitle_qualifier_mapping() {
assert_eq!(subtitle_qualifier("feature", "sdh"), LabelQualifier::Sdh);
assert_eq!(subtitle_qualifier("embed", ""), LabelQualifier::Forced);
assert_eq!(subtitle_qualifier("feature", ""), LabelQualifier::None);
assert_eq!(subtitle_qualifier("text", ""), LabelQualifier::None);
// An SDH embedded track (hypothetical) still flags SDH, the richer fact.
assert_eq!(subtitle_qualifier("embed", "sdh"), LabelQualifier::Sdh);
}
/// `id="NN"` becomes the 1-based STN slot; a missing/zero id names no slot.
#[test]
fn stream_number_from_id_digits_only() {
assert_eq!(stream_number_from_id(&Some("01".into())), Some(1));
assert_eq!(stream_number_from_id(&Some("11".into())), Some(11));
assert_eq!(stream_number_from_id(&Some(" 07 ".into())), Some(7));
assert_eq!(stream_number_from_id(&Some("00".into())), None);
assert_eq!(stream_number_from_id(&None), None);
assert_eq!(stream_number_from_id(&Some("".into())), None);
}
// ── Negative detection ─────────────────────────────────────────────────
/// A document that is not a Fox feature manifest yields no labels. Three
/// shapes: empty, a `<dcx>` whose playlists are all non-feature menu/logo
/// clips (no `name="feature"`), and unrelated XML. None must be mistaken
/// for a Fox feature table.
#[test]
fn non_feature_manifest_yields_no_labels() {
assert!(labels_from_dcx("").is_empty());
assert!(labels_from_dcx("<not-dcx><playlist name=\"x\"/></not-dcx>").is_empty());
let menus_only = r#"<dcx><disc>
<playlist id="00001" lang="eng" name="topmenu"/>
<playlist id="00100" lang="eng" name="foxlogo"/>
<playlist id="00600" name="black" lang="eng"/>
</disc></dcx>"#;
assert!(
labels_from_dcx(menus_only).is_empty(),
"no <playlist name=\"feature\"> → nothing to label"
);
assert_eq!(feature_playlist_id(menus_only), None);
}
/// A feature playlist with no nested streams (an authoring edge) produces
/// no labels rather than a spurious empty-slot entry.
#[test]
fn feature_without_streams_yields_no_labels() {
let doc = r#"<dcx><disc>
<playlist id="00800" lang="eng" name="feature" durs="7628"/>
</disc></dcx>"#;
assert!(labels_from_dcx(doc).is_empty());
// ...but the id is still recoverable for the feature hint.
assert_eq!(feature_playlist_id(doc), Some("00800".to_string()));
}
}
+2 -2
View File
@@ -74,12 +74,12 @@ pub fn has_path_prefix(archive: &Jar, prefix: &str) -> bool {
}
/// Iterate every `.class` entry in the jar, parse it with
/// [`class_reader`], and call `f` with `(entry_name, &ClassFile)`.
/// [`class_reader`](super::class_reader), and call `f` with `(entry_name, &ClassFile)`.
///
/// Entries that fail to read or parse are silently skipped — this is
/// label-extraction code, robustness matters more than completeness.
/// Callers that need to know which classes failed should use the
/// lower-level [`class_reader`] API directly.
/// lower-level [`class_reader`](super::class_reader) API directly.
pub fn for_each_class<F>(archive: &mut Jar, mut f: F)
where
F: FnMut(&str, &ClassFile),
+335 -160
View File
@@ -15,7 +15,8 @@ pub mod clpi_audit;
mod criterion;
mod ctrm;
mod dbp;
mod deluxe;
pub(crate) mod deluxe;
pub(crate) mod fox;
pub(crate) mod jar;
mod mpls_universal;
mod paramount;
@@ -118,15 +119,12 @@ pub enum LabelQualifier {
}
// ── Parser registry ────────────────────────────────────────────────────────
//
// Each entry: (name, detect_fn, parse_fn). Order = tiebreaker only —
// the registry picks the highest-confidence parse result, falling back
// to array order on confidence ties.
// Each entry: (name, detect_fn, parse_fn). Order = tiebreaker only — the
// registry picks the highest-confidence parse result, falling back to order.
// `detect` takes the reader too, so a parser can look INSIDE a jar's central
// directory (real vendor-prefix / project-file check) rather than firing on
// "any jar present". Precise detection is what lets the registry scale to many
// parsers without cross-parser collisions.
// "any jar present", letting the registry scale without cross-parser collisions.
type DetectFn = fn(&mut dyn SectorSource, &UdfFs) -> bool;
type ParseFn = fn(&mut dyn SectorSource, &UdfFs) -> Option<ParseResult>;
@@ -149,6 +147,38 @@ pub enum Confidence {
High,
}
/// The disc's own authoring-designated main-feature playlist, surfaced by a
/// parser that reads the menu/authoring metadata (e.g. Paramount `playlists.xml`
/// carries `<playlist name="Feature" id="00222"/>`). Consumed by title
/// selection (`Disc::main_feature_order`) as a size-independent anti-decoy
/// signal — a playlist obfuscation trick can inflate a streamless decoy past
/// the real feature on size/duration, but it can't forge the authoring id.
/// Best-effort: an unknown field is `None`.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct FeaturePlaylistHint {
/// Playlist number (e.g. 222 for `00222.mpls`) when the authoring id parses.
pub playlist_id: Option<u16>,
/// Playlist filename (e.g. "00222.mpls") when derivable.
pub filename: Option<String>,
}
impl FeaturePlaylistHint {
/// Whether this hint designates the playlist a title carries. Matches on
/// the numeric id OR the filename (case-insensitive), so a hint that pins
/// only one still binds.
pub fn matches(&self, title_playlist_id: u16, title_playlist: &str) -> bool {
if self.playlist_id == Some(title_playlist_id) {
return true;
}
matches!(&self.filename, Some(f) if f.eq_ignore_ascii_case(title_playlist))
}
/// Whether the hint carries no usable identity.
pub fn is_empty(&self) -> bool {
self.playlist_id.is_none() && self.filename.is_none()
}
}
/// Successful parser result. `None` from `parse()` still means "this
/// isn't my disc" (no labels at all); `Some(ParseResult { labels, .. })`
/// with `labels.is_empty()` is also a "no labels" case but reachable
@@ -158,6 +188,9 @@ pub enum Confidence {
pub struct ParseResult {
pub labels: Vec<StreamLabel>,
pub confidence: Confidence,
/// The disc's authoring-named feature playlist, when this parser read it
/// from the menu metadata. `None` for parsers that don't expose one.
pub feature_playlist: Option<FeaturePlaylistHint>,
}
impl ParseResult {
@@ -167,6 +200,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::High,
feature_playlist: None,
}
}
@@ -175,6 +209,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::Medium,
feature_playlist: None,
}
}
@@ -185,6 +220,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::Low,
feature_playlist: None,
}
}
}
@@ -194,19 +230,18 @@ const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
("criterion", criterion::detect, criterion::parse),
("pixelogic", pixelogic::detect, pixelogic::parse),
("ctrm", ctrm::detect, ctrm::parse),
// dbp and deluxe now detect via the real `com/<vendor>/` central-directory
// prefix (reader-backed), so they claim only their own discs. Order between
// them is the tiebreaker on equal confidence; dbp goes first because its
// parse path is cheaper (constant-pool iteration vs. deluxe's bytecode
// walking).
// dbp and deluxe detect via the real `com/<vendor>/` central-directory
// prefix (reader-backed), so they claim only their own discs. dbp goes
// first on ties: its parse path is cheaper (constant-pool vs. bytecode).
("dbp", dbp::detect, dbp::parse),
("deluxe", deluxe::detect, deluxe::parse),
// Universal MPLS fallback. Returns Confidence::Low so framework
// parsers always win when they match. Closes the "no framework
// matched" gap (e.g. HDMV-only discs) with spec-derived language
// + base codec for every stream the playlist references. Runs
// last in registry order so it's only the chosen parser when
// nothing else fired.
// Fox — older discs ship a loose plain-XML `/BDMV/JAR/<id>/dcx.xml` manifest
// with per-stream language/purpose/qualifier (High confidence). Newer Fox
// wraps this in `com/foxbd` bytecode (fox.rs Phase 2); detected, not yet decoded.
("fox", fox::detect, fox::parse),
// Universal MPLS fallback. Returns Confidence::Low so framework parsers
// always win when they match. Closes the "no framework matched" gap (e.g.
// HDMV-only discs) with spec-derived language + codec for every stream.
(
"mpls_universal",
mpls_universal::detect,
@@ -214,20 +249,27 @@ const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
),
// Menu-graphic filename language hints (Low). AFTER mpls_universal so the
// richer spec-derived floor wins the Low tie whenever it produces anything;
// this only becomes the chosen parser when even MPLS yields nothing but the
// menu artwork still names its languages. A last-resort language source.
// only chosen when even MPLS yields nothing but the menu art still names languages.
("png_filenames", png_filenames::detect, png_filenames::parse),
];
/// Search disc for config files, extract labels, apply to streams.
/// This is 100% optional — if anything fails, streams are untouched.
pub fn apply(reader: &mut dyn SectorSource, udf: &UdfFs, titles: &mut [DiscTitle]) {
let labels = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| extract(reader, udf)))
.unwrap_or_default();
if labels.is_empty() {
return;
/// Returns the disc's authoring-designated feature playlist when a parser
/// surfaced one (`None` otherwise), so the caller can bias title selection
/// toward it. Label binding onto streams is a side effect on `titles`.
pub fn apply(
reader: &mut dyn SectorSource,
udf: &UdfFs,
titles: &mut [DiscTitle],
) -> Option<FeaturePlaylistHint> {
let (labels, feature_playlist) =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| extract(reader, udf)))
.unwrap_or_default();
if !labels.is_empty() {
apply_labels(&labels, titles);
}
apply_labels(&labels, titles);
feature_playlist.filter(|h| !h.is_empty())
}
/// Minimum number of streams of one type a title must carry before its
@@ -430,10 +472,9 @@ fn anchor_score(
pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
use std::collections::HashMap;
// `(clip id, PID) -> index into `labels``, harvested from the anchor
// title of each stream type. Keyed by clip because a PID is only unique
// within one clip: two unrelated .m2ts files both open their audio at
// 0x1100.
// `(clip id, PID) -> index into `labels``, harvested from the anchor title
// of each stream type. Keyed by clip because a PID is only unique within
// one clip: two unrelated .m2ts files both open their audio at 0x1100.
let mut pid_map: HashMap<(&str, u16), usize> = HashMap::new();
let mut anchors: [Option<usize>; 2] = [None; 2];
for stream_type in [StreamLabelType::Audio, StreamLabelType::Subtitle] {
@@ -450,13 +491,16 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
}) else {
continue;
};
for clip in &title.clips {
// ONLY the anchor's first clip: `disc::bluray` builds streams from
// `play_items[0]`'s STN table, the only clip these PIDs were seen in.
// A later clip could reuse the PID for a different stream otherwise.
if let Some(clip) = title.clips.first() {
pid_map.insert((clip.clip_id.as_str(), *pid), pos);
}
}
tracing::info!(
stream_type = ?stream_type,
playlist = %titles[anchor].playlist,
playlist = ?titles[anchor].playlist,
slots = slots_of(&titles[anchor], stream_type).len(),
"label list anchored to a title by its stream-language sequence",
);
@@ -468,12 +512,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
.map(|((c, p), v)| ((c.to_string(), p), v))
.collect();
// Labels that name their own stream, indexed by that name. No anchor, no
// sequence and no ordinal is involved in reaching one of these: the id was
// read from the same STN / ProgramInfo table `disc::bluray` built the
// stream from, so equality here is the same elementary stream by
// construction. Later duplicates lose, matching the first-wins rule the
// rest of this module uses.
// Labels that name their own stream, indexed by that name. The id was read
// from the same STN / ProgramInfo table `disc::bluray` built the stream
// from, so equality here is the same elementary stream by construction.
let by_id: HashMap<&StreamId, usize> = labels
.iter()
.enumerate()
@@ -509,9 +550,8 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
.unwrap_or_default();
// Resolve one stream to (label, authoritative). Authoritative means the
// label is known to belong to THIS stream rather than guessed onto it:
// tiers 1-3 of the doc comment above. Only tier 4, the bare ordinal, is
// not.
// label is known to belong to THIS stream rather than guessed onto it
// (tiers 1-3 of the doc comment above); only tier 4, the bare ordinal, isn't.
let resolve = |stream_type: StreamLabelType, idx: u16, pid: u16, lang: &str| {
if anchors[type_tag(stream_type) as usize] == Some(title_idx)
&& let Some(l) = label_at(labels, stream_type, idx)
@@ -547,15 +587,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
// Structured fields — callers translate purpose to UI text.
a.purpose = label.purpose;
// Codec descriptor: trust the parser's `codec_hint` ONLY
// when it's consistent with the stream's actual codec — it
// may legitimately be richer (e.g. "Dolby Atmos" on a TrueHD
// stream, which the raw spec codec can't express). If the
// hint CONTRADICTS the stream (a mis-bound / shuffled label,
// e.g. "AC-3 2.0" on a TrueHD track, or "TrueHD" on a DD+
// track), discard it and derive the descriptor from the
// stream itself — that's correct per-stream and can never be
// shuffled. An empty hint is left for `fill_defaults`.
// Trust the parser's `codec_hint` ONLY when consistent with the
// stream's actual codec (it may be richer, e.g. "Dolby Atmos" on
// TrueHD); if it CONTRADICTS, derive from the stream itself.
let codec_desc = if label.codec_hint.is_empty() {
// No codec hint — leave for fill_defaults.
String::new()
@@ -597,10 +631,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
sub_idx += 1;
if let Some((label, authoritative)) =
resolve(StreamLabelType::Subtitle, sub_idx, s.pid, &s.language)
// A subtitle label carries nothing but the qualifier,
// so an unverifiable one is all risk and no gain: off
// the authoritative path, require the label and the
// stream to state the same language.
// A subtitle label carries nothing but the qualifier, so an
// unverifiable one is all risk, no gain: off the authoritative
// path, require the label and stream to state the same language.
&& (authoritative
|| languages_agree(&label.language, &s.language))
{
@@ -735,19 +768,17 @@ fn codec_hint_consistent(hint: &str, codec: &crate::disc::Codec) -> bool {
let says_dts = !says_dts_ma && !says_dts_hr && h.contains("dts");
let says_lpcm = h.contains("lpcm") || h.contains("pcm");
let says_atmos = h.contains("atmos");
// DTS:X is an object-audio extension carried on a DTS-HD MA (or HR)
// core, exactly as Atmos rides TrueHD / DD+. The spec Codec enum has
// no DtsX variant, so a correctly-authored DTS:X hint must be judged
// consistent with its DtsHdMa/DtsHdHr carrier rather than discarded.
// DTS:X is an object-audio extension carried on a DTS-HD MA (or HR) core,
// as Atmos rides TrueHD/DD+. The spec Codec enum has no DtsX variant, so a
// correctly-authored hint must be judged consistent with its carrier.
let says_dtsx = h.contains("dts:x") || h.contains("dts-x") || h.contains("dtsx");
let names_family =
says_truehd || says_ddp || says_ac3 || says_dts_ma || says_dts_hr || says_dts || says_lpcm;
// Pure-editorial hint (no codec family named) isn't asserting a codec →
// consistent. "Atmos" alone implies a lossless carrier (TrueHD or DD+).
// ("DTS:X" always also matches the "dts" family above, so it never
// reaches this branch — it is handled in the DtsHdMa/DtsHdHr arms.)
// Pure-editorial hint (no codec family named) isn't asserting a codec, so
// it's consistent. "Atmos" alone implies a lossless carrier (TrueHD/DD+).
// ("DTS:X" always matches "dts" above, so it never reaches this branch.)
if !names_family {
return if says_atmos {
matches!(codec, Codec::TrueHd | Codec::Ac3Plus)
@@ -855,7 +886,10 @@ fn generate_audio_label_inner(
}
}
fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
fn extract(
reader: &mut dyn SectorSource,
udf: &UdfFs,
) -> (Vec<StreamLabel>, Option<FeaturePlaylistHint>) {
let mut candidates: Vec<(&'static str, ParseResult)> = Vec::new();
for (name, detect, parse) in PARSERS {
if !detect(reader, udf) {
@@ -870,16 +904,11 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
}
candidates.push((name, result));
}
// One tie-break rule, one implementation. `select_result` owns it and
// carries a regression test for a past bug where the LAST equal-confidence
// parser won instead of the first. `extract` — the path that actually
// ships — used to re-derive the same rule inline with a hand-rolled `>`
// scan and had no test of its own, so that fixed bug could have silently
// recurred here. Array order encodes a trust ordering (the hand-vetted
// parsers are registered ahead of the ones that detect on any BD-J disc),
// so "first wins on a tie" is load-bearing, not incidental.
// One tie-break rule, one implementation. `select_result` regression-tests a
// past bug where the LAST equal-confidence parser won instead of the first.
// Array order encodes trust (hand-vetted parsers before generic BD-J ones).
let best = select_result(&candidates).map(|(n, r)| (*n, r.clone()));
let (name, mut labels) = match best {
let (name, mut labels, feature_playlist) = match best {
Some((n, r)) => {
tracing::info!(
parser = n,
@@ -887,35 +916,29 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
label_count = r.labels.len(),
"label parser selected",
);
(n, r.labels)
(n, r.labels, r.feature_playlist)
}
None => {
tracing::info!("no label parser matched");
return Vec::new();
return (Vec::new(), None);
}
};
// The MPLS floor: framework parsers under-yield on multi-track discs
// because their authoring layer only ships editorial labels for
// "interesting" streams (Director's Cut, Atmos, SDH) and leaves the rest as
// plain numbered slots. MPLS sees every stream a playlist references, and
// names each one, so merging it in gives the user every track even when
// only the "interesting" ones have editorial names. Skipped when
// mpls_universal was itself the chosen parser (its labels ARE the labels).
// The MPLS floor: framework parsers under-yield on multi-track discs, only
// shipping editorial labels for "interesting" streams (Atmos, SDH). MPLS
// names every stream a playlist references, filling the gaps.
if name != "mpls_universal"
&& let Some(mpls_result) = mpls_universal::parse(reader, udf)
{
merge_mpls_floor(&mut labels, &mpls_result.labels);
}
// CLPI orphan streams: PIDs in /BDMV/CLIPINF/*.clpi ProgramInfo that no
// MPLS playlist references. Empirically a small fraction of streams are
// CLPI-only — physically on disc, not menu-reachable. They too are
// appended under the id they name, so a title reaches one only if it
// actually carries that stream.
// CLPI orphan streams: PIDs in /BDMV/CLIPINF/*.clpi ProgramInfo that no MPLS
// playlist references (physically on disc, not menu-reachable). Appended
// under the id they name, so a title reaches one only if it carries it.
let _orphans_added = append_clpi_orphans(&mut labels, reader, udf);
labels
(labels, feature_playlist)
}
/// Merge the MPLS-derived floor into a framework parser's label list.
@@ -1009,12 +1032,9 @@ fn append_clpi_orphans(
udf: &UdfFs,
) -> usize {
use crate::consts::coding_type as c;
// Two exclusions, one exact and one fuzzy. Exact: a stream some label
// already NAMES is not an orphan, whatever it looks like. Fuzzy: the
// pre-existing (type, language, codec_hint) test, kept because it is what
// bounds this list to a handful of entries per disc rather than one per
// stream per clip; it can only ever drop a candidate, and an orphan that
// never binds costs nothing when it is dropped.
// Two exclusions: exact (a stream some label already NAMES is not an orphan)
// and fuzzy (the pre-existing (type, language, codec_hint) test, kept to
// bound this list to a handful of entries per disc, not one per stream).
use std::collections::HashSet;
let named: HashSet<&StreamId> = labels.iter().filter_map(|l| l.stream_id.as_ref()).collect();
let existing: HashSet<(StreamLabelType, String, String)> = labels
@@ -1054,10 +1074,9 @@ fn append_clpi_orphans(
if !seen_pids.insert(s.pid) {
continue;
}
// Translate CLPI coding_type → label stream_type.
// 0x90 = Presentation Graphics (PG subtitle). 0x91 =
// Interactive Graphics (BD-J menu overlay), NOT a user-facing
// subtitle — skip it, matching the MPLS path which drops IG.
// Translate CLPI coding_type → label stream_type. 0x90 = Presentation
// Graphics (PG subtitle); 0x91 = Interactive Graphics (BD-J menu
// overlay), NOT a user-facing subtitle — skip it, matching MPLS.
let stype = match s.coding_type {
c::LPCM..=c::DTS_HD_MA | c::AC3_PLUS_SECONDARY | c::DTS_HD_SECONDARY => {
StreamLabelType::Audio
@@ -1169,16 +1188,13 @@ pub fn analyze(reader: &mut dyn SectorSource, udf: &UdfFs) -> LabelAnalysis {
};
// The MPLS floor: same merge as `extract()`. Skipped when MPLS was itself
// the chosen parser (its labels ARE the labels).
//
// Note this diagnostic path stops here — `extract()` also appends the CLPI
// orphans, which `analyze` has never reported.
// the chosen parser (its labels ARE the labels). This diagnostic path stops
// here — `extract()` also appends the CLPI orphans, which `analyze` never does.
let gap_fill_added = if parser.is_some() && parser != Some("mpls_universal") {
let before = labels.len();
// Re-run MPLS unconditionally — we only ran framework parsers
// above (we want to know which one to pick), and in the
// common case where MPLS would have detected but wasn't
// chosen we still need its labels for the merge.
// Re-run MPLS unconditionally: we only ran framework parsers above (to
// know which to pick), and in the common case where MPLS would have
// detected but wasn't chosen we still need its labels for the merge.
if let Some(mpls_result) = mpls_universal::parse(reader, udf) {
merge_mpls_floor(&mut labels, &mpls_result.labels);
}
@@ -1196,10 +1212,9 @@ pub fn analyze(reader: &mut dyn SectorSource, udf: &UdfFs) -> LabelAnalysis {
);
}
// bdmt runs independently of the parser registry: it's disc-level
// metadata (localized titles, box-set position), not per-stream
// labels, so the "highest confidence wins" logic doesn't apply.
// Always run if detected; surface result as a separate field.
// bdmt runs independently of the parser registry: it's disc-level metadata
// (localized titles, box-set position), not per-stream labels, so "highest
// confidence wins" doesn't apply. Always run if detected, as a separate field.
let disc_metadata = if bdmt::detect(udf) {
bdmt::parse(reader, udf)
} else {
@@ -1254,10 +1269,9 @@ fn collect_chapter_summary(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<Ch
if chapter_count == 0 {
continue;
}
// Duration: sum of (out_time - in_time) across play items,
// each in 45kHz PTS ticks → seconds. Approximates the disc
// module's per-title duration; we don't claim sample accuracy
// here, just enough to identify "the long one" (main movie).
// Duration: sum of (out_time - in_time) across play items, in 45kHz PTS
// ticks. Approximates the disc module's per-title duration; not
// sample-accurate, just enough to identify "the long one" (main movie).
let duration_ticks: u64 = playlist
.play_items
.iter()
@@ -1311,7 +1325,7 @@ pub struct LabelAnalysis {
/// tool.
pub gap_fill_added: usize,
/// Per-playlist chapter summary: `(playlist_filename, chapter_count, duration_secs)`.
/// Sourced from MPLS PlaylistMark entries with `mark_type ≤ 1`
/// Sourced from MPLS PlaylistMark entries with `mark_type == 1`
/// (chapter entries). Ordered by playlist filename. Empty if no
/// MPLS files have parseable marks, or the disc isn't Blu-ray.
pub chapter_summary: Vec<ChapterSummary>,
@@ -1418,11 +1432,20 @@ mod registry_tests {
/// straight from the disc's UDF directory records, with attacker-chosen
/// name lengths to inflate each comparison.
///
/// Proof is by deadline. With the linear scan this fixture measures well
/// past the deadline; with a set it is milliseconds. Bounded so a
/// regression fails fast instead of hanging CI.
/// This is a HANG GUARD, and the name says so: a return to the linear scan
/// makes this fixture run for minutes (120 000² / 2 comparisons over a
/// 180-byte shared prefix), which without the deadline would wedge CI
/// rather than fail it. It is not a complexity proof — no assertion here
/// can distinguish `BTreeSet` from any other sub-quadratic dedup, and the
/// clock-free half of the claim (dedup, sort, directory exclusion) belongs
/// to `jar_inventory_dedups_sorts_and_skips_dirs` below.
///
/// The deadline is a real margin, unlike the 6x one that made
/// `paramount.rs`'s wall-clock test flake under a loaded CI box: measured
/// at 0.14 s debug / 0.07 s release against 10 s, so ~70x. A shared CPU
/// does not close that; a quadratic dedup does not survive it.
#[test]
fn jar_inventory_dedup_is_not_quadratic() {
fn jar_inventory_dedup_does_not_hang_on_a_hostile_directory() {
const FILES: usize = 120_000;
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
@@ -1508,15 +1531,16 @@ mod registry_tests {
"ctrm",
"dbp",
"deluxe",
"fox",
"mpls_universal",
"png_filenames",
],
"PARSERS array order changed — file-presence/reader-gated High \
parsers (paramount/criterion/pixelogic/ctrm) stay first; dbp + \
deluxe (now real com/<vendor>/ prefix detect) stay before \
mpls_universal; mpls_universal stays the universal Low fallback; \
png_filenames (Low, language-only hint) stays LAST so MPLS wins \
the Low tie whenever it produces anything."
deluxe (now real com/<vendor>/ prefix detect) and fox (dcx.xml / \
com/foxbd) stay before mpls_universal; mpls_universal stays the \
universal Low fallback; png_filenames (Low, language-only hint) \
stays LAST so MPLS wins the Low tie whenever it produces anything."
);
}
@@ -1538,9 +1562,37 @@ mod registry_tests {
ParseResult {
labels: vec![one_label()],
confidence: conf,
feature_playlist: None,
}
}
#[test]
fn feature_playlist_hint_matches_on_id_or_filename() {
let h = FeaturePlaylistHint {
playlist_id: Some(222),
filename: Some("00222.mpls".into()),
};
assert!(h.matches(222, "99999.mpls"), "numeric id match");
assert!(
h.matches(1, "00222.MPLS"),
"filename match is case-insensitive"
);
assert!(
!h.matches(1, "00001.mpls"),
"neither id nor filename → no match"
);
let id_only = FeaturePlaylistHint {
playlist_id: Some(5),
filename: None,
};
assert!(id_only.matches(5, "anything.mpls"));
assert!(!id_only.matches(6, "00005.mpls"));
assert!(FeaturePlaylistHint::default().is_empty());
assert!(!h.is_empty());
}
/// `select_result` must pick the highest-confidence non-empty result
/// and, on a confidence tie, the FIRST in array order (regression for the
/// old `analyze()` `max_by(...then(Equal))` no-op that picked the LAST).
@@ -1574,6 +1626,7 @@ mod registry_tests {
let empty = ParseResult {
labels: Vec::new(),
confidence: Confidence::High,
feature_playlist: None,
};
// High-confidence but empty must be skipped in favour of a
// non-empty lower-confidence result.
@@ -1590,18 +1643,14 @@ mod registry_tests {
#[test]
fn parsers_registry_all_entries_populated() {
for (name, detect, parse) in PARSERS {
// Function pointers can't be Null in safe Rust, so the
// assertion is just that the array entry was constructed
// — which the iter above already implies. The test
// exists to fail compile if someone changes the tuple
// shape (e.g. adds a 4th field) without updating callers,
// and as a marker for "these parsers exist."
// Function pointers can't be null in safe Rust, so this just fails to
// compile if the tuple shape changes (e.g. a 4th field is added)
// without updating callers; it also marks "these parsers exist."
let _ = (name, detect, parse);
}
// The loop above touches every registry entry. The non-empty
// invariant is covered separately by `parsers_registry_order_locked`,
// whose assert_eq! on the expected order fails if PARSERS is empty.
// This test fails to compile if the tuple shape changes.
// Non-empty invariant is covered separately by
// `parsers_registry_order_locked`, whose assert_eq! on the expected
// order fails if PARSERS is empty.
}
}
@@ -1761,11 +1810,9 @@ mod gap_fill_tests {
#[test]
fn orphan_append_skips_matching_type_lang_codec_tuples() {
// If a "would-be orphan" actually shares (type, lang, codec) with a
// label we already have, drop it — the user-facing rendering would be a
// confusing duplicate. This fuzzy test is what bounds the orphan list
// to a handful of entries per disc; it is a population rule only, and
// no longer decides where anything binds.
// If a "would-be orphan" shares (type, lang, codec) with a label we
// already have, drop it to avoid a confusing duplicate. This fuzzy
// test bounds the orphan list; it's a population rule, not a binding rule.
let labels = [
label(StreamLabelType::Audio, 1, "eng", "TrueHD"),
label(StreamLabelType::Audio, 2, "fra", "AC-3"),
@@ -1814,10 +1861,8 @@ mod gap_fill_tests {
}
// ── apply() integration tests ──────────────────────────────────────────────
//
// End-to-end coverage for the apply_labels + fill_defaults pipeline
// without needing a SectorSource / UdfFs. Synthetic DiscTitle +
// StreamLabel inputs, assert on the resulting Stream field values.
// End-to-end coverage for apply_labels + fill_defaults without needing a
// SectorSource / UdfFs. Synthetic DiscTitle + StreamLabel inputs.
#[cfg(test)]
mod apply_tests {
@@ -1898,6 +1943,72 @@ mod apply_tests {
}
}
/// Sentinel embedded in the crafted playlist name below. The capture keeps
/// ONLY fields whose rendered form contains it, so the assertion cannot be
/// fooled by another test's log output.
const LOG_INJECTION_SENTINEL: &str = "FMKV-LOG-INJECTION-PROBE";
/// A playlist name is a raw UDF directory entry — disc-controlled bytes,
/// validated no further than a lossy UTF-8 decode. Logging it through
/// tracing's `%` (Display) sigil writes those bytes VERBATIM, so a crafted
/// `.mpls` filename carrying ANSI escapes or control characters forges
/// terminal output and log structure in any consumer rendering the event
/// (CWE-117). `?` (Debug) escapes them, and `str`'s Debug is exactly the
/// escaping this needs.
///
/// `info!` is not covered by the debug/trace-logging exemption: this fires
/// on an ordinary rip of an ordinary disc.
///
/// Mutation: put `%` back on `playlist` in `apply_labels` and this goes red.
#[test]
fn a_disc_derived_playlist_name_is_escaped_in_the_log_not_written_verbatim() {
// A name whose bytes would clear the line and repaint it.
let evil = format!("\u{1b}[2K\u{1b}[31m{LOG_INJECTION_SENTINEL}\u{7}\u{1b}[0m.mpls");
let labels = vec![
sub_label(1, "eng", LabelQualifier::None),
sub_label(2, "spa", LabelQualifier::None),
sub_label(3, "fra", LabelQualifier::None),
];
let mut titles = vec![title_on_clip(
&evil,
"00294",
vec![
subtitle(0x12A0, "eng"),
subtitle(0x12A1, "spa"),
subtitle(0x12A2, "fra"),
],
)];
// `testlog::capture` installs the crate's ONE global `tracing` subscriber,
// routing events to a thread-local sink; `tracing` caches interest
// globally, so a per-test subscriber would poison other tests' callsites.
let ((), events) = crate::testlog::capture(|| {
apply_labels(&labels, &mut titles);
});
let playlist: Vec<&str> = events
.iter()
.filter(|e| e.target.starts_with("libfreemkv::labels"))
.filter_map(|e| e.field("playlist"))
.filter(|v| v.contains(LOG_INJECTION_SENTINEL))
.collect();
assert!(
!playlist.is_empty(),
"the anchoring event must actually have fired, or this test proves \
nothing; captured: {events:?}"
);
for rendered in playlist {
assert!(
!rendered.contains('\u{1b}') && !rendered.contains('\u{7}'),
"a disc-controlled playlist name reached the log with its raw \
control bytes intact: {rendered:?}"
);
assert!(
rendered.contains(LOG_INJECTION_SENTINEL),
"the name must still be legible once escaped: {rendered:?}"
);
}
}
fn sub_label(num: u16, lang: &str, qualifier: LabelQualifier) -> StreamLabel {
StreamLabel {
stream_id: None,
@@ -2041,10 +2152,9 @@ mod apply_tests {
#[test]
fn apply_keeps_consistent_dtsx_hint_on_dts_hd_ma() {
// DTS:X rides a DTS-HD MA core just as Atmos rides TrueHD. A
// correctly-authored "DTS:X" hint on a DtsHdMa stream is richer
// than the spec codec yet consistent, so it's kept verbatim —
// not discarded and regenerated to "DTS-HD Master Audio".
// DTS:X rides a DTS-HD MA core just as Atmos rides TrueHD. A correctly
// authored "DTS:X" hint on a DtsHdMa stream is richer than the spec
// codec yet consistent, so it's kept verbatim, not regenerated.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::DtsHdMa,
@@ -2079,10 +2189,9 @@ mod apply_tests {
#[test]
fn apply_normalizes_plain_consistent_hint_to_marketing() {
// A French DD+ track: a DD+ stream whose hint "AC-3+ 5.1" is correct
// but short-form. A sibling DD+ track that fell back uses the marketing
// form — keeping the short form here would read inconsistently, so a
// plain (non-richer) consistent hint is normalized to the stream's own.
// A DD+ stream whose hint "AC-3+ 5.1" is correct but short-form; a sibling
// fallback track uses the marketing form, so a plain (non-richer)
// consistent hint is normalized to the stream's own to stay consistent.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::Ac3Plus,
@@ -2828,13 +2937,8 @@ mod apply_tests {
}
// ── codec_hint_consistent: chained-OR boundary hardening ────────────────
//
// The family-detection booleans are built from chains of `h.contains(..)
// || h.contains(..) || ...` synonym checks. Each test below isolates ONE
// synonym clause (a hint string that matches that clause and NO other
// clause in the same chain) so a `||` -> `&&` flip at that specific
// position changes the family verdict — and, downstream, whether the
// codec match arm returns the spec-correct answer.
// The family-detection booleans chain `h.contains(..) || ...` synonym checks.
// Each test isolates ONE clause so a `||` -> `&&` flip there changes the verdict.
/// Isolates the `"true hd"` (space form) synonym in `says_truehd`,
/// which mutant testing hit at 396:44's `||`. If that `||` is
@@ -3094,6 +3198,78 @@ mod apply_tests {
);
}
/// A title that plays SEVERAL clips in order — the shape the anchor's
/// PID facts are harvested from.
fn title_on_clips(playlist: &str, clip_ids: &[&str], streams: Vec<Stream>) -> DiscTitle {
DiscTitle {
playlist: playlist.into(),
clips: clip_ids
.iter()
.map(|id| crate::disc::Clip {
feed_span: None,
clip_id: (*id).into(),
in_time: 0,
out_time: 0,
duration_secs: 3600.0,
source_packets: 0,
})
.collect(),
..title_with(streams)
}
}
/// Spec: the anchor proves a `(clip, PID)` fact only for the clip its
/// stream table was READ FROM — the first play item — never for every clip
/// the anchor happens to play.
///
/// `disc::bluray` builds a title's stream list from `play_items[0]`'s STN
/// table, and tier 3 fifty lines below says exactly that by keying its
/// derived ids on `clip0`. Tier 2's harvest contradicted it: it recorded
/// the anchor's slot PIDs against EVERY clip the anchor plays, so a sibling
/// playlist that plays a LATER clip of the anchor bound the anchor's
/// editorial label onto whatever stream in that clip happens to reuse the
/// PID — a different physical stream, in a different clip, whose own
/// language says so.
///
/// Mutation: harvest over `&title.clips` instead of its first clip — the
/// featurette wears the feature's SDH again.
#[test]
fn an_anchor_proves_pids_only_for_the_clip_its_table_came_from() {
let labels = vec![
sub_label(1, "eng", LabelQualifier::Sdh),
sub_label(2, "fra", LabelQualifier::None),
];
let mut titles = vec![
// The anchor: its stream table is clip 00082's (the first play
// item); it merely CONTINUES into 00090.
title_on_clips(
"00800.mpls",
&["00082", "00090"],
vec![subtitle(0x1200, "eng"), subtitle(0x1201, "fra")],
),
// Sibling playlist over the anchor's SECOND clip; reuses PID 0x1200
// (PIDs are only unique within a clip) but is Spanish, so the
// anchor's English SDH slot doesn't describe it.
title_on_clips("00451.mpls", &["00090"], vec![subtitle(0x1200, "spa")]),
];
apply_labels(&labels, &mut titles);
assert_eq!(
sub_state(&titles[0]),
vec![
(0x1200, false, LabelQualifier::Sdh),
(0x1201, false, LabelQualifier::None)
],
"the anchor itself keeps the qualifiers the list states for it"
);
assert_eq!(
sub_state(&titles[1]),
vec![(0x1200, false, LabelQualifier::None)],
"a PID in a clip the anchor's table never described is not that \
table's stream 1"
);
}
/// Spec: a vendor codec/variant claim does not follow the ordinal onto a
/// bonus clip that carries a different codec.
///
@@ -3106,10 +3282,9 @@ mod apply_tests {
/// Mutation: as above — move `by_id` after the ordinal fallback.
#[test]
fn a_vendor_codec_claim_does_not_follow_the_ordinal_onto_a_bonus_clip() {
// The measured shape: this framework states no codec_hint at all and
// puts its descriptor in `name`, which `apply_labels` falls back to
// verbatim. Nothing about the stream is consulted on that path, so the
// codec-consistency guard never runs and cannot catch the mis-binding.
// This framework states no codec_hint and puts its descriptor in `name`,
// which `apply_labels` falls back to verbatim, so the codec-consistency
// guard never runs and cannot catch the mis-binding.
let feature_audio = StreamLabel {
name: "English Dolby Atmos".into(),
..audio_label(1, "eng", "", "")
+22 -42
View File
@@ -78,11 +78,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
// MPLS gives language + codec but never editorial info. Low confidence
// means framework parsers always win when they match; MPLS is only
// chosen when nothing else fired — the universal-fallback role we want.
Some(ParseResult::low(labels))
}
@@ -93,7 +91,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// needing a synthetic on-disc UDF image.
///
/// Identity is `(clip, PID)` — what the STN entry states — and it is both the
/// dedup key and the label's [`StreamId`]. A stream twenty playlists list is
/// dedup key and the label's [`StreamId`](super::StreamId). A stream twenty playlists list is
/// one label; two clips that both open their first audio at 0x1100 are two.
/// This replaced a disc-global dense counter that numbered surviving entries
/// 1, 2, 3, … in playlist-directory order: that number was not an STN slot in
@@ -104,25 +102,16 @@ fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
let mut seen: HashSet<super::StreamId> = HashSet::new();
for playlist in playlists {
// `Playlist::streams` is the FIRST play item's STN table, so every
// entry here is a stream of that play item's clip — the same clip
// `disc::bluray` records as the title's `clips[0]`. That pairing is
// what makes the PID an identity rather than a 16-bit number.
//
// Streams cannot be non-empty without a play item to have read them
// from, so the empty case is unreachable on a real disc; entries we
// cannot identify are skipped rather than emitted as unbindable
// labels.
// `Playlist::streams` is the FIRST play item's STN table, so every entry
// here is a stream of that play item's clip, matching `disc::bluray`'s
// `clips[0]` — that pairing is what makes the PID an identity.
let Some(clip_id) = playlist.play_items.first().map(|pi| pi.clip_id.clone()) else {
continue;
};
// 1-based STN slot within THIS playlist's table, per type — the
// `stream_number` field's documented meaning, counted the same way
// `disc::bluray` counts the stream list it builds from these entries.
// Nothing binds through it (these labels bind by id); it is stated
// truthfully rather than invented so that a reader of the label list
// sees where on its own playlist each stream sits.
// 1-based STN slot within THIS playlist's table, per type — matches how
// `disc::bluray` counts the same entries. Nothing binds through it (labels
// bind by id); it's stated truthfully so the label list shows real slots.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
@@ -201,13 +190,9 @@ fn label_type_for(entry: &crate::mpls::StreamEntry) -> Option<StreamLabelType> {
}
fn has_mpls_extension(name: &str) -> bool {
// Case-insensitive ".mpls" suffix. Some discs use uppercase,
// some lowercase; UDF filenames preserve case but we don't.
//
// UDF names are decoded via from_utf8_lossy, so a multi-byte
// replacement char (EF BF BD) can straddle byte index n-5; a raw
// byte slice there panics on a non-char-boundary. `ends_with` on a
// lowercased copy is char-boundary-safe and still case-insensitive.
// Case-insensitive ".mpls" suffix (discs use both cases). UDF names are
// decoded via from_utf8_lossy, so a multi-byte replacement char can straddle
// byte index n-5; `ends_with` on a lowercased copy avoids that boundary panic.
name.len() >= 5 && name.to_ascii_lowercase().ends_with(".mpls")
}
@@ -573,11 +558,9 @@ mod tests {
assert_eq!(id("fra"), Some(("00001".into(), 0x1101)));
assert_eq!(id("deu"), Some(("00001".into(), 0x1102)));
// `stream_number` is the entry's slot in ITS OWN playlist's STN table
// — deu is pl2's second audio, so 2, not "the third distinct stream
// seen while scanning the disc". It used to be the latter: a dense
// disc-global counter that named no table anyone could count against,
// handed to a binder that reads the field as an STN slot.
// `stream_number` is the entry's slot in ITS OWN playlist's STN table —
// deu is pl2's second audio, so 2, not "third distinct stream on disc".
// (Used to be a dense disc-global counter fed to a binder expecting a slot.)
let num = |lang: &str| {
labels
.iter()
@@ -735,10 +718,8 @@ mod tests {
#[test]
fn secondary_audio_becomes_audio_label() {
// stream_type 5 = secondary audio. The conversion should
// still produce an Audio label (the registry's apply path
// can ignore secondary if it wants — this module just
// surfaces what's there).
// stream_type 5 = secondary audio; conversion still produces an Audio
// label (the registry's apply path can ignore secondary if it wants).
let mut sec = audio_entry(0x1A00, 0x83, 3, 1, "eng");
sec.stream_type = 5;
sec.secondary = true;
@@ -875,11 +856,10 @@ mod tests {
#[test]
fn normalize_language_lowercases_and_trims() {
assert_eq!(
super::super::mpls_universal::language_display_name(&{
let trimmed = " ENG ".trim().to_ascii_lowercase();
// feed through production normalize_language logic
trimmed
}),
// trim + lowercase, mirroring production normalize_language
super::super::mpls_universal::language_display_name(
&" ENG ".trim().to_ascii_lowercase(),
),
"English"
);
}
+227 -98
View File
@@ -42,10 +42,27 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
if labels.is_empty() {
return None;
}
// Surface the feature playlist's identity so title selection can prefer
// it over a size-inflated decoy. The id is 5-digit zero-padded on BD
// (matching `NNNNN.mpls`); keep only digits to resist a stray quote/space.
let feature_playlist = super::xml::attr(&feature, "id").and_then(|id| {
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
Some(super::FeaturePlaylistHint {
playlist_id: digits.parse::<u16>().ok(),
filename: Some(format!("{digits}.mpls")),
})
});
// High confidence: this format is fully structured and we extract
// every field whose meaning the corpus establishes. "Documented" would
// be the wrong word — see the module note; nothing about it is.
Some(ParseResult::high(labels))
let mut result = ParseResult::high(labels);
result.feature_playlist = feature_playlist;
Some(result)
}
/// One cell of the `forced_sub` CSV.
@@ -98,6 +115,74 @@ enum ForcedSub {
ForcedNarrative,
}
/// The one number behind every cap in this parser: the highest CSV cell
/// position that can ever be addressed.
///
/// The labelling loops number cells 1-based into a `u16` and `break` at
/// `u16::try_from(i + 1)`, so cell `MAX_COM_INDICES` and everything past it is
/// never visited. Two different things are measured against that, and they are
/// not the same bound:
///
/// - **A VALUE at or beyond it cannot match any cell.** This is what caps the
/// set: values are filtered before insertion, so at most `MAX_COM_INDICES`
/// distinct entries can ever be stored, however long the attribute is. The
/// `HashSet` that replaced a linear scan fixed the LOOKUP cost; this is what
/// fixes the ALLOCATION, and a disc declaring half a billion indices no
/// longer costs half a billion entries.
/// - **A POSITION at or beyond it describes nothing new.** This caps the WORK,
/// not the memory — the value filter already made the set small, but without
/// it every one of those half a billion cells is still split and parsed. It
/// is an early exit at the first position whose contents provably cannot
/// matter, and it is why `forced_sub` — which holds no set at all, and so
/// gets no protection from the value rule — is bounded too.
///
/// Real authoring is nowhere near either limit: the BD STN table admits at
/// most 32 streams per playlist, so nothing legitimate is lost.
const MAX_COM_INDICES: usize = u16::MAX as usize;
/// Parse a `*_com1_idx` attribute into the set the labelling loops query.
///
/// Extracted so the BOUND is observable. Asserting it through
/// `labels_from_feature` is not possible: a `HashSet` collapses repeated
/// values, and an out-of-range index changes no label either way, so such a
/// test passes whether or not the cap exists — an assertion that cannot fail.
/// Returning the set lets a test hand in tens of thousands of DISTINCT
/// unaddressable indices and see them refused.
fn com_indices(attr: Option<String>) -> HashSet<usize> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.filter_map(|i| i.trim().parse().ok())
.filter(|&i| i < MAX_COM_INDICES)
.collect()
})
.unwrap_or_default()
}
/// Parse the `forced_sub` attribute into the cell list the subtitle loop
/// queries — the third attacker-controlled CSV in this file, and the last one
/// that was still unbounded.
///
/// Bounded by POSITION, and it has no other choice: a `*_com1_idx` list holds
/// values that can be filtered, and that filter is what caps its set, but a
/// `forced_sub` cell is a classification of the position it sits at, so there
/// is nothing to filter and nothing else would ever cap this. The Vec is read
/// only as `forced.get(i)` from a loop that stops at `MAX_COM_INDICES`, so
/// every cell past that is unreachable by construction.
///
/// Extracted, like [`com_indices`], so the bound is OBSERVABLE. Through
/// `labels_from_feature` it is not: the subtitle loop cannot reach those cells
/// either, so a label-level assertion passes whether or not the cap exists.
fn forced_subs(attr: Option<String>) -> Vec<ForcedSub> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.map(forced_sub_cell)
.collect()
})
.unwrap_or_default()
}
fn forced_sub_cell(cell: &str) -> ForcedSub {
match cell.trim() {
"1" => ForcedSub::ContainsForcedSegments,
@@ -115,34 +200,14 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse audio streams
if let Some(aud) = xml::attr(feature, "aud") {
// aud_com1_idx is a trimmed, comma-separated list of CSV positions
// (some authoring tools emit whitespace, and multiple commentary
// tracks are possible) — symmetric with sub_com1_idx below.
// A HashSet, not a Vec: `com_indices` is parsed straight out of an
// attacker-controlled attribute with no length bound and was scanned
// linearly once per stream, so `aud="..."` and `aud_com1_idx="..."`
// both grown large make this quadratic in the size of one XML file.
// Membership is the only operation performed on it.
let com_indices: HashSet<usize> = xml::attr(feature, "aud_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
// aud_com1_idx: trimmed CSV positions, symmetric with sub_com1_idx.
// HashSet not Vec: parsed from an attacker-controlled attribute and
// only membership-tested; a Vec would make large inputs quadratic.
let com_indices = com_indices(xml::attr(feature, "aud_com1_idx"));
// The CSV *is* the STN list: one cell per stream, in stream order,
// and `aud_com1_idx` is a 0-based index into those same cells. So
// `stream_number` is the cell's own 1-based position — NOT a counter
// that only advances on cells carrying a language.
//
// A cell with an empty language still occupies its STN slot; it just
// has nothing to label. Renumbering the surviving cells 1..N shifts
// every label behind an empty cell one slot forward, which is how a
// marker authored for one stream ends up written onto the stream in
// front of it (see the subtitle side, where the marker is `forced`).
//
// `u16::try_from` rather than `saturating_add`: past the 1-based u16
// numbering space every cell would collapse onto `u16::MAX`, binding
// several streams to one label. Stop emitting instead. Unreachable on
// real media — the BD STN_table admits at most 32 primary audio
// streams per playlist.
// The CSV *is* the STN list, so `stream_number` is each cell's 1-based
// position, not a counter over labeled cells (empty cells still occupy
// their slot). `u16::try_from`: stop rather than wrap onto `u16::MAX`.
for (i, lang) in aud.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
@@ -172,21 +237,15 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse subtitle streams
if let Some(sub) = xml::attr(feature, "sub") {
let forced: Vec<ForcedSub> = xml::attr(feature, "forced_sub")
.map(|s| s.split(',').map(forced_sub_cell).collect())
.unwrap_or_default();
let forced = forced_subs(xml::attr(feature, "forced_sub"));
// HashSet for the same reason as the audio side above: unbounded
// parsed input, membership-only use, linear scan once per stream.
let com_indices: HashSet<usize> = xml::attr(feature, "sub_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
let com_indices = com_indices(xml::attr(feature, "sub_com1_idx"));
// As with audio: the cell position IS the STN slot. `forced_sub` and
// `sub_com1_idx` are indexed against those same cells, so an empty
// cell must not renumber the cells behind it — a forced marker
// authored for one PG slot would otherwise be written onto an
// earlier, full-dialogue subtitle track.
// As with audio: the cell position IS the STN slot, indexed by
// `forced_sub`/`sub_com1_idx`, so an empty cell must not renumber
// the rest — else a forced marker lands on the wrong subtitle track.
for (i, lang) in sub.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
@@ -315,61 +374,133 @@ mod tests {
);
}
/// `sub_com1_idx` is an unbounded index list parsed straight out of the
/// disc's `playlists.xml` and was membership-tested with a linear
/// `Vec::contains` once per subtitle stream — quadratic in the size of a
/// single attacker-supplied file.
/// `sub_com1_idx` is parsed straight out of the disc's `playlists.xml`,
/// which is attacker-controlled and has no length bound of its own.
///
/// Proof is by deadline rather than micro-benchmark. With the linear scan
/// the original fixture (200 000 streams x 1 000 001 indices) measured
/// 31 s in a release build and far longer in debug; with a set it measured
/// 0.03 s release / 0.56 s debug. A 10 s deadline sits ~18x above the
/// slowest passing measurement and ~3x below the fastest failing one, and
/// makes a regression fail fast instead of hanging CI.
/// This replaces a WALL-CLOCK test. That one built a 200 000 x 1 000 001
/// fixture and failed if it took over 10 s, to prove the membership test
/// was a set rather than a linear scan. Measured on the machine that
/// wrote this: 1.62 s alone, and OVER 10 s — a real failure — when the
/// suite's other 3 347 tests were running concurrently. A 6x margin
/// against a shared CPU is not a margin; it is a CI failure that looks
/// like a flake and gets re-run until it passes.
///
/// The CSV now stops at the end of the 1-based `u16` stream-numbering
/// space, so only the first 65 535 cells are scanned. `INDICES` is raised
/// to keep the linear-scan work product (`cells x indices`) at or above
/// the original fixture's, preserving that deadline margin.
/// It also measured the wrong thing. Making the lookup O(1) bounded the
/// QUERY, not the PARSE: the set was still built from every entry the
/// disc declared, so a hostile playlist could still force an unbounded
/// allocation before any lookup happened. `MAX_COM_INDICES` bounds that.
///
/// Correctness is pinned on fixture-derived literals: indices 0, 2 and 4
/// are the commentary tracks, 1 and 3 are not.
/// What THIS test guards is that bounding did not change what a
/// legitimate playlist MEANS: it goes red if the bound is set too LOW
/// (verified at 2 — the real indices `0,2,4` stop resolving and the
/// purposes change). It does NOT go red if the bound is deleted
/// entirely, because the out-of-range filler is unobservable at the
/// label level and a `HashSet` collapses the repeats. Enforcement is
/// proven separately, by
/// `distinct_unaddressable_indices_are_refused_not_stored`, which reads
/// the set itself. Two tests, two properties; neither pretends to the
/// other's job.
#[test]
fn commentary_index_lookup_is_not_quadratic() {
/// Cells offered. Everything past `u16::MAX` is unnumberable and the
/// parser stops there, so the scanned prefix is 65 535 cells.
const STREAMS: usize = 200_000;
const SCANNED: usize = u16::MAX as usize;
const INDICES: usize = 3_100_000;
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(STREAMS));
feature.pop();
// Three real commentary indices, then a long run of one
// out-of-range value: nothing here is bounded by the stream count.
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(INDICES));
feature.pop();
feature.push_str(r#"" />"#);
let _ = tx.send(labels_from_feature(&feature));
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(labels) => {
worker.join().expect("worker panicked");
assert_eq!(labels.len(), SCANNED);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
Err(_) => panic!(
"labels_from_feature did not finish {STREAMS} streams x \
{INDICES} commentary indices within 10s — the membership \
test is still a linear scan"
),
}
fn bounding_the_parse_does_not_change_a_legitimate_playlist() {
// Three real indices, then far more entries than can address a cell.
const OVERSIZED: usize = MAX_COM_INDICES + 10_000;
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(8));
feature.pop();
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(OVERSIZED));
feature.pop();
feature.push_str(r#"" />"#);
let labels = labels_from_feature(&feature);
// The fixture's real indices still decide the purposes: bounding the
// parse must not change what a legitimate playlist means.
assert_eq!(labels.len(), 8);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
/// The set REFUSES unaddressable indices, so a hostile playlist cannot
/// inflate it. DISTINCT values on purpose: a `HashSet` collapses repeats,
/// so a million copies of one index costs one entry and would prove
/// nothing. Fifty thousand distinct out-of-range indices cost fifty
/// thousand entries without the filter, and none with it — so this test
/// goes red if the bound is removed, which the label-level assertions
/// below cannot do.
#[test]
fn distinct_unaddressable_indices_are_refused_not_stored() {
let hostile: String = (MAX_COM_INDICES..MAX_COM_INDICES + 50_000)
.map(|i| i.to_string())
.collect::<Vec<_>>()
.join(",");
let set = com_indices(Some(hostile));
assert!(
set.is_empty(),
"kept {} unaddressable indices — the parse is still unbounded",
set.len()
);
// The addressable ones are still kept.
assert_eq!(com_indices(Some("0,2,4".to_string())).len(), 3);
}
/// `forced_sub` is bounded too — the third CSV in the same function, and
/// the one that had no value filter to hide behind.
///
/// Read through `forced_subs` rather than through the labels for the same
/// reason the two tests above read the set: the subtitle loop stops at
/// `MAX_COM_INDICES`, so a label-level assertion cannot tell a bounded
/// parse from an unbounded one.
#[test]
fn forced_sub_cells_past_the_last_addressable_one_are_not_parsed() {
let hostile = "0,".repeat(MAX_COM_INDICES + 50_000);
let cells = forced_subs(Some(hostile));
assert_eq!(
cells.len(),
MAX_COM_INDICES,
"parsed {} cells — the forced_sub parse is still unbounded",
cells.len()
);
}
/// Bounding it must not change what a legitimate playlist means: the
/// cells that CAN address a stream still classify exactly as before.
#[test]
fn bounding_forced_sub_leaves_the_addressable_cells_alone() {
let cells = forced_subs(Some("0,1,3,2".to_string()));
assert_eq!(
cells,
vec![
ForcedSub::None,
ForcedSub::ContainsForcedSegments,
ForcedSub::ForcedNarrative,
ForcedSub::ForcedNarrative,
]
);
}
/// An index that cannot address any cell is dropped rather than STORED.
///
/// Asserted through `com_indices`, not through the labels: the labelling
/// loop never queries a cell position that high, so at the label level
/// retaining the value is unobservable and the assertion could not fail.
/// Reading the set is what makes the claim checkable.
#[test]
fn an_index_that_cannot_address_any_cell_is_not_retained() {
let set = com_indices(Some(format!(
"1,{},{}",
MAX_COM_INDICES,
MAX_COM_INDICES + 1
)));
assert_eq!(
set.len(),
1,
"only the addressable index belongs in the set, got {set:?}"
);
assert!(set.contains(&1));
}
/// Headroom: the BD STN_table admits at most 32 PG streams per playlist,
@@ -468,10 +599,9 @@ mod tests {
#[test]
fn aud_com1_idx_trimmed_and_multivalue() {
// Whitespace around the index, and a multi-value list, must both
// resolve. com index is positional against the raw CSV, so with
// an empty slot at position 1, " 2 " marks the 'fra' track
// (CSV index 2, STN slot 3) as commentary.
// Whitespace and multi-value lists must both resolve; com index is
// positional against the raw CSV, so with an empty slot at position 1,
// " 2 " marks 'fra' (CSV index 2, STN slot 3) as commentary.
let feature = r#"<playlist aud="eng,,fra" aud_com1_idx=" 2 " />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -630,10 +760,9 @@ mod tests {
/// Mutation: cast `i + 1` to u16 → stream numbers wrap to 0, skipping apply.
#[test]
fn audio_stream_number_never_wraps() {
// 65535 audio tracks is impossible on a real disc but the parser must
// not panic or produce 0. Build a comma-separated list of 65535 "eng"s.
// We only run the number-assignment logic via labels_from_feature.
// Limit: CSV with 300 slots is sufficient to test the counter.
// 65535 tracks is impossible on a real disc but must not panic/produce 0.
// 300 slots is sufficient to exercise the number-assignment logic
// via labels_from_feature without building the full 65535-entry CSV.
let aud: String = (0..300).map(|_| "eng").collect::<Vec<_>>().join(",");
let feature = format!(r#"<playlist name="Feature" aud="{}" />"#, aud);
let labels = labels_from_feature(&feature);
+47 -126
View File
@@ -111,11 +111,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// shortest meaningful run is 4 chars (lang + underscore).
let strings = text::extract_ascii_strings(&data, 4);
// Tracked across all parse_token calls in this run: which uncatalogued
// token components did the blob contain? If any, we downgrade confidence
// to Medium — the labels are still valid but the disc surfaced something
// we don't catalogue — and report them once (see `UnknownParts`). Parsing
// is single-threaded and sequential, so a plain owned collector suffices.
// Collects uncatalogued token components; if any, confidence downgrades to Medium
// and they're reported once (see `UnknownParts`). Sequential parse, so a plain
// owned collector suffices.
let mut unknown = UnknownParts::default();
let labels = assign_labels(&strings, &mut unknown);
@@ -133,7 +131,11 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
} else {
Confidence::Medium
};
Some(ParseResult { labels, confidence })
Some(ParseResult {
labels,
confidence,
feature_playlist: None,
})
}
/// Walk the extracted token strings of the feature section and emit a
@@ -141,14 +143,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// from `parse` so the section/numbering logic is unit-testable without
/// a `SectorSource`/`UdfFs`.
fn assign_labels(strings: &[String], unknown: &mut UnknownParts) -> Vec<StreamLabel> {
// The authoritative per-feature stream list lives in the `FPL_`
// (FeaturePLaylist) section, in STN order. `SEG_*` entries are menu
// segments (intros, logos, disclaimers, previews) that can also carry
// stray stream tokens — e.g. a `SEG_MainFeature` preview segment that
// lists only a commentary track. Anchoring on such a segment grabs the
// wrong streams and misnumbers them. So when the project ships any
// `FPL_` playlist, anchor exclusively on it; only fall back to
// `SEG_MainFeature` on discs that have no `FPL_` section at all.
// The authoritative per-feature stream list lives in `FPL_` (FeaturePlaylist), in
// STN order. `SEG_*` menu segments can carry stray tokens, so anchor on `FPL_` when
// present; fall back to `SEG_MainFeature` only if no `FPL_` section exists.
let has_fpl = strings.iter().any(|s| s.starts_with("FPL_"));
let mut labels = Vec::new();
@@ -191,27 +188,9 @@ fn assign_labels(strings: &[String], unknown: &mut UnknownParts) -> Vec<StreamLa
continue;
}
// Second way a section ends, and the only one some discs give us.
//
// The `SEG_`/`SF_`/`FPL_` markers above are section NAMES, and not every
// section is named: a project's trailing sections (the per-language
// notice, disclaimer and dub-credit cards) are emitted with a plain
// clip name, or none at all. When the feature playlist is the last
// NAMED section in the blob — which it is on most of the corpus — the
// name-only rule never fires again and the walk swallows the whole
// tail of the file as if it were more of the feature's stream list.
// Those trailing clip names are per-language (`{lang3}_{card}`), so
// they pass `is_stream_token` and each one silently advances an STN
// counter; the ones whose card name collides with a catalogued
// component (an `AC` card reads as the AC-3 codec) even emit a label,
// for an STN slot the feature playlist does not have.
//
// What every section does have, named or not, is a stream list that
// OPENS with its video slots. So a `Video Stream N` entry that repeats
// one this section already listed cannot be another slot in this
// section — it is the first entry of the next one, and this section is
// over. Distinct video entries are kept (a section may legitimately
// list a secondary video stream alongside the primary).
// Second section-end signal (the only one some discs give): every section's stream
// list OPENS with its video slots, so a `Video Stream N` repeating one this section
// listed marks the next section's start. Guards against swallowing trailing cards.
if s.starts_with("Video Stream") {
if video_slots.contains(&s.as_str()) || video_slots.len() >= MAX_VIDEO_SLOTS {
break;
@@ -226,26 +205,9 @@ fn assign_labels(strings: &[String], unknown: &mut UnknownParts) -> Vec<StreamLa
break;
}
// Every entry in the section's stream list occupies one STN slot,
// whether or not it carries anything worth labelling. Pixelogic lists
// a slot as an editorial `{lang}_{codec|purpose|region}_…` token, or as
// a bare `Audio Stream N` / `PG Stream N` placeholder when no editorial
// label was authored. Both must advance the per-type counter, and so
// must an editorial token the grammar cannot classify — a
// region-only token (`fra_CF_`, `spa_LS_`: REGIONS sets `variant` but
// neither `is_audio` nor `is_subtitle`, so `parse_token_inner` returns
// `None`) or one whose only distinguishing component is uncatalogued
// (`jpn_DUB_`). Numbering only the slots that PARSE renumbers the rest
// 1..N and lands every surviving label on the wrong stream: on
// one observed UHD feature the seven `*_TXT_FOR_` forced tokens at PG
// STN 11-16/18 collapsed onto STN 2-8, flagging that disc's FULL
// subtitle tracks `forced` and leaving the real forced-narrative
// tracks unflagged.
//
// A slot the grammar cannot classify carries no stream type either, so
// it advances the list currently being enumerated: pixelogic sections
// run video → audio → PG, and `domain` follows the last slot whose type
// WAS known (a typed token or a typed placeholder), starting at Audio.
// Every stream-list entry occupies one STN slot (editorial token, bare placeholder,
// or unclassifiable token) and ALL must advance the per-type counter, else labels
// renumber onto wrong streams. Unclassifiable slots follow `domain` (video→audio→PG).
if let Some(kind) = placeholder_kind(s) {
domain = kind;
match kind {
@@ -365,12 +327,9 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
if raw_part.is_empty() {
continue;
}
// Token components are spec-uppercase (codec IDs, ADES/ACOM/SDH,
// region codes). vocab elsewhere is deliberately case-insensitive,
// so normalize each component to uppercase before the gate to
// avoid silently dropping a lowercase-authored token (which would
// fall through to the unknown branch and, with no is_audio/
// is_subtitle set, get the whole stream discarded below).
// Token components are spec-uppercase (codec IDs, ADES/ACOM/SDH, region codes).
// Normalize each to uppercase before the gate so a lowercase-authored token isn't
// silently dropped through the unknown branch (no is_audio/is_subtitle set).
let part_up = raw_part.to_ascii_uppercase();
let part = part_up.as_str();
if AUDIO_CODECS.contains(&part) {
@@ -395,37 +354,15 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
} else if part == "STRI" || part == "TXT" {
is_subtitle = true;
} else if part == "FOR" {
// `FOR` (forced) is a subtitle-domain qualifier. A token whose
// only non-language component is FOR (e.g. `eng_FOR_`) would
// otherwise classify as neither audio nor subtitle and be
// dropped at the `!is_audio && !is_subtitle` guard below. Treat
// a forced marker as a subtitle signal so the stream survives.
// `FOR` (forced) is a subtitle-domain qualifier. Treat it as a subtitle signal
// so a token whose only non-language component is FOR (e.g. `eng_FOR_`) isn't
// dropped at the `!is_audio && !is_subtitle` guard below.
qualifier = LabelQualifier::Forced;
is_subtitle = true;
} else if part == "DUB" {
// `DUB` names the forced-narrative subtitle track authored to
// accompany that language's DUBBED audio presentation — the
// signs/on-screen-text pass a viewer needs when the dialogue is
// already dubbed. Same editorial class as `*_TXT_FOR_`, spelled
// differently by some authoring runs.
//
// Evidence (two independent discs in the corpus): the token
// appears only inside the PG list, embedded in an otherwise
// contiguous run of `{lang}[_{region}]_TXT_FOR_` siblings — one
// forced-narrative slot per localized language — and takes the
// slot where that language's forced entry belongs. The stream it
// lands on is a sparse PG track (sub-megabyte, a few dozen
// display sets), the signature of a forced-narrative pass rather
// than a full subtitle track.
//
// Deliberately NOT added to `vocab::qualifier`: that maps
// free-form ENGLISH label text, where a bare word "dub" means
// dubbed AUDIO. The forced-subtitle reading is specific to this
// vendor's token grammar, so it stays token-local here.
//
// Like `FOR`, this is a subtitle-domain signal: a token whose
// only non-language component is `DUB` must still classify as a
// subtitle or the `!is_audio && !is_subtitle` guard drops it.
// `DUB` = forced-narrative subtitle for a language's dubbed audio (same class as
// `*_TXT_FOR_`). Token-local, NOT in `vocab::qualifier` (English "dub" = dubbed
// AUDIO). Like FOR, a subtitle-domain signal so the guard keeps the stream.
qualifier = LabelQualifier::Forced;
is_subtitle = true;
} else if REGIONS.contains(&part) {
@@ -433,15 +370,10 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
} else if part.starts_with("PGSTREAM") {
is_subtitle = true;
} else {
// Unknown token component — skip this single part rather
// than discarding the entire stream record. Pre-refactor
// behavior was `return None` here, which silently dropped
// any stream containing a single uncatalogued token (e.g.
// a new codec ID or framework variant). Better to surface
// what we know than discard a whole stream over one part,
// but flag the parse as Medium-confidence so callers know
// some data was elided.
tracing::debug!(part = %part, "pixelogic: unrecognized token component, skipping");
// Unknown token component — skip this single part rather than discarding the
// whole stream record (pre-refactor `return None` dropped streams over one
// uncatalogued token). Recorded so the parse downgrades to Medium confidence.
tracing::debug!(part = ?part, "pixelogic: unrecognized token component, skipping");
if let Some(acc) = unknown.as_deref_mut() {
acc.record(part);
}
@@ -452,12 +384,9 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
return None;
}
// Tie-break for tokens that signal both domains (e.g. `eng_MLP_SDH_`
// sets is_audio via the codec and is_subtitle via SDH). An audio
// codec hint is the stronger, audio-domain signal, so prefer Audio
// when one is present (keeps the parsed codec_hint instead of
// discarding it); otherwise file as Subtitle. Pure-subtitle and
// pure-audio tokens are unaffected.
// Tie-break for tokens signalling both domains (e.g. `eng_MLP_SDH_`: is_audio via codec,
// is_subtitle via SDH). An audio codec is the stronger signal, so prefer Audio when
// present (keeps codec_hint); otherwise Subtitle. Pure tokens are unaffected.
let has_audio_codec = is_audio && !codec.is_empty();
let stream_type = if is_subtitle && !has_audio_codec {
StreamLabelType::Subtitle
@@ -624,11 +553,9 @@ mod tests {
#[test]
fn parse_token_components_are_case_insensitive() {
// Regression for the case-sensitive gate: a lowercase codec/
// qualifier component must classify identically to uppercase
// rather than falling through to the unknown branch and getting
// the whole stream dropped. The ISO 639-2 lang prefix is still
// required lowercase.
// Regression for the case-sensitive gate: a lowercase codec/qualifier component must
// classify identically to uppercase, not fall through to the unknown branch and drop
// the stream. The ISO 639-2 lang prefix is still required lowercase.
let l = parse_token_inner("eng_mlp_", None).unwrap();
assert_eq!(l.stream_type, StreamLabelType::Audio);
assert_eq!(l.codec_hint, "TrueHD");
@@ -653,11 +580,9 @@ mod tests {
#[test]
fn assign_labels_numbers_commentary_behind_placeholders() {
// Observed case: the FPL_MainFeature playlist lists three unlabelled main
// audio tracks as `Audio Stream N` placeholders, then a lone
// `eng_ACOM_` commentary at STN slot 4. The commentary must land on
// audio #4, not collapse onto #1 (which would tag the main feature
// track as commentary).
// Observed case: FPL_MainFeature lists three unlabelled `Audio Stream N`
// placeholders, then a lone `eng_ACOM_` commentary at STN slot 4. It must land on
// audio #4, not collapse onto #1 (which would tag the main track as commentary).
let mut flag = UnknownParts::default();
let tokens = strs(&[
"FPL_MainFeature",
@@ -759,10 +684,9 @@ mod tests {
#[test]
fn assign_labels_prefers_fpl_over_seg_mainfeature() {
// A `SEG_MainFeature` menu/preview segment carries a stray commentary
// token, but the real playlist is `FPL_MainFeature`. When an FPL_
// section exists, the SEG_ one must be ignored as an anchor — so we
// number from the FPL playlist, putting the commentary at slot 2.
// A `SEG_MainFeature` segment carries a stray commentary token, but the real
// playlist is `FPL_MainFeature`. When an FPL_ section exists the SEG_ one is ignored
// as anchor, so numbering comes from the FPL playlist (commentary at slot 2).
let mut flag = UnknownParts::default();
let tokens = strs(&[
"SEG_MainFeature",
@@ -903,8 +827,8 @@ mod tests {
}
/// Spec: is_audio wins over is_subtitle when codec explicitly identified.
/// Tests the commentary audio case — `ACOM` sets both is_audio purpose and SDH-only-is-subtitle:
/// MLP codec wins → Audio.
/// `SDH` alone would suggest a subtitle, but an explicit `AC3` codec token
/// wins the tiebreak → Audio.
/// Mutation: flip the tie-break → Subtitle returned when codec present.
#[test]
fn parse_token_codec_always_wins_type_tiebreak() {
@@ -1070,12 +994,9 @@ mod tests {
],
"only the feature playlist's own slots are labelled"
);
// A card's name is emitted BEFORE its own section's video slot, so a
// forward-only walk meets the first one while still nominally inside
// the feature section and counts it. That residue is one entry, at the
// tail of a list nothing follows in — it cannot renumber any label,
// only cost the parse its High confidence. Every card behind it is
// past the boundary and never seen. Pinned rather than papered over.
// A card's name is emitted BEFORE its section's video slot, so a forward-only walk
// counts the first one while still nominally inside the feature section. That lone
// tail entry can't renumber any label, only cost High confidence. Pinned, not hidden.
assert_eq!(
flag.seen.iter().map(String::as_str).collect::<Vec<_>>(),
vec!["WARNING"],
+1 -1
View File
@@ -7,7 +7,7 @@
//! of menu languages the disc ships.
//!
//! This is a language-only hint (no per-stream purpose/codec), so it runs at
//! [`Confidence::Low`] — it never displaces a real framework parser, and it
//! [`Confidence::Low`](super::Confidence::Low) — it never displaces a real framework parser, and it
//! sits at the same tier as the MPLS floor. It is here so the pattern is a
//! first-class, testable parser that keeps picking up discs as the corpus
//! grows, rather than lost logic. Detection is precise: it fires only on the
Binary file not shown.
Binary file not shown.
Binary file not shown.
+333 -8
View File
@@ -219,11 +219,243 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
Some(code)
}
// ── ISO 639-1 → ISO 639-2 ────────────────────────────────────────────────────
/// The complete ISO 639-1 set, paired with its ISO 639-2/**T** (terminological)
/// code. Every two-letter code ISO 639-1 defines appears exactly once.
///
/// /T is the variant the rest of this crate uses — [`lang`] and [`menu_lang`]
/// both normalize to it (`deu` not `ger`, `fra` not `fre`, `zho` not `chi`,
/// `ces`, `nld`, `ell`, `ron`, `slk`, `isl`, `eus`, `hrv`) — so the three
/// tables cannot disagree. `iso639_1_agrees_with_menu_lang` pins that.
///
/// For the 165 codes where 639-2/B and /T are identical this distinction does
/// not arise; it only matters for the 20-odd languages with a distinct
/// bibliographic code.
const ISO_639_1_TO_2: &[(&str, &str)] = &[
("aa", "aar"),
("ab", "abk"),
("ae", "ave"),
("af", "afr"),
("ak", "aka"),
("am", "amh"),
("an", "arg"),
("ar", "ara"),
("as", "asm"),
("av", "ava"),
("ay", "aym"),
("az", "aze"),
("ba", "bak"),
("be", "bel"),
("bg", "bul"),
("bh", "bih"),
("bi", "bis"),
("bm", "bam"),
("bn", "ben"),
("bo", "bod"),
("br", "bre"),
("bs", "bos"),
("ca", "cat"),
("ce", "che"),
("ch", "cha"),
("co", "cos"),
("cr", "cre"),
("cs", "ces"),
("cu", "chu"),
("cv", "chv"),
("cy", "cym"),
("da", "dan"),
("de", "deu"),
("dv", "div"),
("dz", "dzo"),
("ee", "ewe"),
("el", "ell"),
("en", "eng"),
("eo", "epo"),
("es", "spa"),
("et", "est"),
("eu", "eus"),
("fa", "fas"),
("ff", "ful"),
("fi", "fin"),
("fj", "fij"),
("fo", "fao"),
("fr", "fra"),
("fy", "fry"),
("ga", "gle"),
("gd", "gla"),
("gl", "glg"),
("gn", "grn"),
("gu", "guj"),
("gv", "glv"),
("ha", "hau"),
("he", "heb"),
("hi", "hin"),
("ho", "hmo"),
("hr", "hrv"),
("ht", "hat"),
("hu", "hun"),
("hy", "hye"),
("hz", "her"),
("ia", "ina"),
("id", "ind"),
("ie", "ile"),
("ig", "ibo"),
("ii", "iii"),
("ik", "ipk"),
("io", "ido"),
("is", "isl"),
("it", "ita"),
("iu", "iku"),
("ja", "jpn"),
("jv", "jav"),
("ka", "kat"),
("kg", "kon"),
("ki", "kik"),
("kj", "kua"),
("kk", "kaz"),
("kl", "kal"),
("km", "khm"),
("kn", "kan"),
("ko", "kor"),
("kr", "kau"),
("ks", "kas"),
("ku", "kur"),
("kv", "kom"),
("kw", "cor"),
("ky", "kir"),
("la", "lat"),
("lb", "ltz"),
("lg", "lug"),
("li", "lim"),
("ln", "lin"),
("lo", "lao"),
("lt", "lit"),
("lu", "lub"),
("lv", "lav"),
("mg", "mlg"),
("mh", "mah"),
("mi", "mri"),
("mk", "mkd"),
("ml", "mal"),
("mn", "mon"),
("mr", "mar"),
("ms", "msa"),
("mt", "mlt"),
("my", "mya"),
("na", "nau"),
("nb", "nob"),
("nd", "nde"),
("ne", "nep"),
("ng", "ndo"),
("nl", "nld"),
("nn", "nno"),
("no", "nor"),
("nr", "nbl"),
("nv", "nav"),
("ny", "nya"),
("oc", "oci"),
("oj", "oji"),
("om", "orm"),
("or", "ori"),
("os", "oss"),
("pa", "pan"),
("pi", "pli"),
("pl", "pol"),
("ps", "pus"),
("pt", "por"),
("qu", "que"),
("rm", "roh"),
("rn", "run"),
("ro", "ron"),
("ru", "rus"),
("rw", "kin"),
("sa", "san"),
("sc", "srd"),
("sd", "snd"),
("se", "sme"),
("sg", "sag"),
("si", "sin"),
("sk", "slk"),
("sl", "slv"),
("sm", "smo"),
("sn", "sna"),
("so", "som"),
("sq", "sqi"),
("sr", "srp"),
("ss", "ssw"),
("st", "sot"),
("su", "sun"),
("sv", "swe"),
("sw", "swa"),
("ta", "tam"),
("te", "tel"),
("tg", "tgk"),
("th", "tha"),
("ti", "tir"),
("tk", "tuk"),
("tl", "tgl"),
("tn", "tsn"),
("to", "ton"),
("tr", "tur"),
("ts", "tso"),
("tt", "tat"),
("tw", "twi"),
("ty", "tah"),
("ug", "uig"),
("uk", "ukr"),
("ur", "urd"),
("uz", "uzb"),
("ve", "ven"),
("vi", "vie"),
("vo", "vol"),
("wa", "wln"),
("wo", "wol"),
("xh", "xho"),
("yi", "yid"),
("yo", "yor"),
("za", "zha"),
("zh", "zho"),
("zu", "zul"),
];
/// The three two-letter codes ISO 639-1 has since withdrawn, mapped to their
/// replacements. DVD-Video froze its language list on the 1988 edition, so
/// discs authored to the spec carry these spellings and no other table sees
/// them: `iw` Hebrew (now `he`), `in` Indonesian (now `id`), `ji` Yiddish
/// (now `yi`).
const ISO_639_1_DEPRECATED: &[(&str, &str)] = &[("iw", "he"), ("in", "id"), ("ji", "yi")];
/// Map an ISO 639-1 two-letter language code to its ISO 639-2/T three-letter
/// code, accepting the withdrawn DVD-era spellings (`iw`, `in`, `ji`) as
/// aliases for their replacements.
///
/// Covers the WHOLE of ISO 639-1, unlike [`menu_lang`], whose table only spans
/// the languages that show up in Blu-ray menu-graphic filenames. Callers that
/// convert a spec field — a DVD IFO attribute block, say — need the whole set:
/// narrowing it to the menu vocabulary would fold every other language onto
/// one value and make a disc's tracks indistinguishable from each other.
///
/// Case-insensitive and trimmed. Returns `None` for anything that is not an
/// ISO 639-1 code, so callers decide the fallback rather than getting a guess.
pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
let c = code.trim().to_ascii_lowercase();
let c = ISO_639_1_DEPRECATED
.iter()
.find(|(old, _)| *old == c)
.map_or(c.as_str(), |(_, new)| new);
ISO_639_1_TO_2
.iter()
.find(|(two, _)| *two == c)
.map(|(_, three)| *three)
}
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
///
/// Recognized keywords (case-insensitive, word-boundary matched):
/// Recognized keywords (case-insensitive; single-word keywords are
/// word-boundary matched, multi-word phrases are substring matched):
/// - "commentary", "director's commentary" → `Commentary`
/// - "descriptive", "description", "audio description", "described" → `Descriptive`
/// - "score", "music only" → `Score`
@@ -231,7 +463,8 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
/// - anything else → `Normal`
///
/// Word-boundary matching means "Commentary track" matches but
/// "Commenter Pro audio" does not.
/// "Commenter Pro audio" does not. Multi-word phrases like "audio
/// description" and "music only" are matched as plain substrings.
pub fn purpose(text: &str) -> LabelPurpose {
let lower = text.to_lowercase();
// Multi-word compounds first — they're more specific.
@@ -300,12 +533,9 @@ fn has_word(haystack: &str, needle: &str) -> bool {
if needle.is_empty() {
return false;
}
// Boundary check is char-aware (not byte-level): a non-ASCII letter
// adjacent to the match (e.g. an accented or CJK char, which is
// multiple UTF-8 bytes) is alphanumeric and so is NOT a boundary,
// preventing false positives like "sdh" inside "cafésch". Needles
// are ASCII tokens, so a byte-offset match aligns with char
// boundaries in `haystack`.
// Char-aware, not byte-level: an accented/CJK char adjacent to the match is
// alphanumeric and so NOT a boundary, preventing false positives like "sdh"
// inside "cafésch". Needles are ASCII, so byte offsets align with char bounds.
for (idx, _) in haystack.match_indices(needle) {
// Char immediately before the match.
let before_is_alnum = haystack[..idx]
@@ -789,4 +1019,99 @@ mod tests {
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
/// Structural invariants of `ISO_639_1_TO_2`: it must hold the complete
/// ISO 639-1 set (184 codes), every key a distinct pair of lowercase
/// letters and every value three lowercase letters. A typo'd or duplicated
/// row fails here rather than silently mislabelling a track.
#[test]
fn iso639_1_table_is_complete_and_well_formed() {
assert_eq!(
ISO_639_1_TO_2.len(),
184,
"ISO 639-1 defines 184 two-letter codes; the table must hold all \
of them"
);
let mut keys: Vec<&str> = ISO_639_1_TO_2.iter().map(|(two, _)| *two).collect();
keys.sort_unstable();
let unique = keys.len();
keys.dedup();
assert_eq!(unique, keys.len(), "no ISO 639-1 code may appear twice");
for (two, three) in ISO_639_1_TO_2 {
assert!(
two.len() == 2 && two.bytes().all(|b| b.is_ascii_lowercase()),
"{two:?} is not a two-letter lowercase ISO 639-1 code"
);
assert!(
three.len() == 3 && three.bytes().all(|b| b.is_ascii_lowercase()),
"{three:?} is not a three-letter lowercase ISO 639-2 code"
);
}
// The withdrawn DVD-era spellings resolve, and are not themselves
// rows in the main table (they are aliases, not codes).
for (old, new) in ISO_639_1_DEPRECATED {
assert!(
!ISO_639_1_TO_2.iter().any(|(two, _)| two == old),
"withdrawn code {old:?} must not be a table row"
);
assert_eq!(
iso639_1_to_iso639_2(old),
iso639_1_to_iso639_2(new),
"withdrawn code {old:?} must resolve exactly as {new:?}"
);
}
}
/// The two tables must not disagree. Every two-letter token `menu_lang`
/// accepts has to yield the same ISO 639-2/T code through
/// `iso639_1_to_iso639_2`, so a DVD-sourced language and a Blu-ray
/// menu-label language for the same tongue never produce different
/// `Language` elements.
#[test]
fn iso639_1_agrees_with_menu_lang() {
for (two, three) in ISO_639_1_TO_2 {
if let Some(via_menu) = menu_lang(two) {
assert_eq!(
via_menu, *three,
"menu_lang({two:?}) = {via_menu:?} disagrees with the ISO \
639-1 table's {three:?}"
);
}
}
// Spot-check the /T choice itself, on the languages where /B differs.
for (two, t_code) in [
("de", "deu"),
("fr", "fra"),
("zh", "zho"),
("cs", "ces"),
("nl", "nld"),
("el", "ell"),
("ro", "ron"),
("sk", "slk"),
("is", "isl"),
("hy", "hye"),
("ka", "kat"),
("fa", "fas"),
] {
assert_eq!(
iso639_1_to_iso639_2(two),
Some(t_code),
"the crate standardises on ISO 639-2/T, so {two:?} is \
{t_code:?} and never the bibliographic form"
);
}
}
/// Trimming, case-insensitivity, and the no-guess contract.
#[test]
fn iso639_1_normalizes_input_and_never_guesses() {
assert_eq!(iso639_1_to_iso639_2("RO"), Some("ron"));
assert_eq!(iso639_1_to_iso639_2(" Ro "), Some("ron"));
assert_eq!(iso639_1_to_iso639_2("IW"), Some("heb"));
assert_eq!(iso639_1_to_iso639_2("zz"), None);
assert_eq!(iso639_1_to_iso639_2(""), None);
assert_eq!(iso639_1_to_iso639_2("e"), None);
// A three-letter code is not ISO 639-1 input — that is menu_lang's job.
assert_eq!(iso639_1_to_iso639_2("eng"), None);
}
}
+8 -13
View File
@@ -110,10 +110,9 @@ pub fn attr(element: &str, name: &str) -> Option<String> {
/// prior behavior in criterion.rs).
pub fn text(xml: &str, tag: &str) -> Option<String> {
let (_open_end, body_start) = find_open_tag(xml, tag, 0)?;
// For self-closing tags, body_start is past `/>` and there is no
// content. Detect with a *byte* comparison: slicing `&xml[..]` two
// bytes back can land inside a multi-byte UTF-8 char and panic
// (untrusted on-disc XML), but indexing the byte slice never does.
// For self-closing tags, body_start is past `/>` with no content. Detect via
// a *byte* comparison: slicing `&xml[..]` two bytes back can land inside a
// multi-byte UTF-8 char and panic on untrusted XML; byte indexing never does.
let b = xml.as_bytes();
if body_start >= 2 && b[body_start - 2] == b'/' && b[body_start - 1] == b'>' {
return Some(String::new());
@@ -462,10 +461,9 @@ mod tests {
#[test]
fn text_multibyte_before_self_close_does_not_panic() {
// A multi-byte UTF-8 char ending right before the `/>` used to
// panic on a non-char-boundary str slice in `text()`. The
// byte-level self-closing check must handle it cleanly.
// 'é' (0xC3 0xA9) directly precedes the `/>`.
// A multi-byte UTF-8 char ('é' = 0xC3 0xA9) right before `/>` used to
// panic on a non-char-boundary str slice in `text()`; the byte-level
// self-closing check must handle it cleanly.
assert_eq!(text("<x>é</x>", "x"), Some("é".into()));
// Self-closing form with a multi-byte char in an attr value.
assert_eq!(text(r#"<x a="é"/>"#, "x"), Some("".into()));
@@ -636,11 +634,8 @@ mod tests {
}
// ── Malformed / truncated input (untrusted on-disc XML) ────────────────
//
// These scrapers run on XML lifted out of BD-J jar entries, which is
// attacker-controllable. Every scan in this module must terminate and
// stay in bounds on truncated or unbalanced input rather than panic.
// XML 1.0 §2.3 defines the Name production these boundary rules model.
// These scrapers run on attacker-controllable BD-J jar XML, so every scan
// must terminate and stay in bounds on truncated input (XML 1.0 §2.3).
/// A quoted attribute value that is never closed must terminate the
/// scan at EOF rather than reading past the end of the buffer.
+27 -81
View File
@@ -101,6 +101,7 @@ pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFI
pub(crate) const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs;
pub(crate) mod bdnav;
pub(crate) mod clpi;
pub mod consts;
pub mod css;
@@ -109,7 +110,7 @@ pub mod diag;
pub mod dirimage;
pub mod disc;
pub mod drive;
pub mod dvdnav;
pub(crate) mod dvdnav;
pub mod error;
pub mod event;
pub mod halt;
@@ -129,69 +130,49 @@ pub mod progress;
pub mod scsi;
pub mod sector;
pub mod session;
#[cfg(test)]
pub(crate) mod testlog;
pub(crate) mod udf;
pub(crate) mod unlock_bridge;
// ─── Drive lifecycle ────────────────────────────────────────────────────────
//
// `Drive::open(path)` → `wait_ready()` → `init()` → `Disc::scan()`. `Drive`
// owns the SCSI session; `DriveCapture` etc. let advanced callers introspect
// drive identity / profile data for sharing.
// `Drive::open(path)` → `wait_ready()` → `init()` → `Disc::scan()`. `Drive` owns
// the SCSI session; `DriveCapture` etc. expose drive identity/profile data.
pub use drive::capture::{
CapturedFeature, DriveCapture, capture_drive_data, mask_bytes, mask_string,
};
pub use drive::{Drive, DriveStatus, extract_scsi_context, find_drive};
// ─── Disc session (drive open + SCSI bring-up hoist) ─────────────────────────
//
// One entry point that opens a drive and brings the transport up, so consumers
// stop hand-rolling `open → wait_ready → init → probe_disc → identify → scan`.
// Owns the `Drive` by value; forwards consumer-built key material into
// `ScanOptions` (the library derives no certs — see `KeySpec`).
// One entry point opens a drive, brings transport up, and forwards caller-built
// key material into `ScanOptions` (the library derives no certs; see `KeySpec`).
pub use session::{
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_dir,
scan_iso,
};
// ─── Errors ─────────────────────────────────────────────────────────────────
//
// All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a
// numeric `code()`; **no English text in the library** — applications map
// codes to localized messages. See `error.rs` for the full taxonomy.
// All fallible APIs return `Result<T, Error>`; `Error` is a typed enum with a
// numeric `code()` — no English text in the library; see `error.rs` for taxonomy.
pub use error::{
Error, Result, error_code, is_disc_level_no_key, is_halt, is_skippable_title_stub,
};
// ─── Cooperative cancellation ───────────────────────────────────────────────
//
// One-bit cooperative cancellation token, shared by every long-running loop —
// libfreemkv's mux, and the recovery passes (sweep/patch) that now live in the
// freemkv-engine crate. Clone it cheaply; pass it by value into each component;
// poll `is_cancelled()` inside the loop body.
// One-bit cancellation token shared by every long-running loop (mux, and the
// engine's sweep/patch passes); clone cheaply, poll `is_cancelled()` in loops.
pub use halt::Halt;
// Generic bounded producer/consumer primitive used by the mux pipeline (and,
// via this re-export, by the engine's sweep/patch recovery passes) to overlap
// reads with writes via a dedicated consumer thread.
// `Pipeline::spawn(name, depth, sink)` spawns a named consumer; `pipe.send(item)`
// pushes one item with back-pressure; `pipe.finish()` joins the
// consumer and surfaces its `close()` output. Callers implement `Sink`
// to define per-item behaviour and end-of-stream finalisation.
//
// `DEFAULT_PIPELINE_DEPTH` (=4) is for callers without specific needs;
// most should use WRITE_PIPELINE_DEPTH instead.
// Patch uses `WRITE_THROUGH_DEPTH` (=1). Returning `Flow::Stop` from
// `apply` ends the consumer cleanly (still calls `close()`).
// Bounded producer/consumer primitive used by mux (and engine's sweep/patch) to
// overlap reads with writes via a consumer thread. `Sink::apply` defines
// per-item behavior; `Flow::Stop` ends the consumer cleanly, still calling `close()`.
pub use io::pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
// ─── Bounded-cache buffered file writer ─────────────────────────────────────
//
// Drop-in `std::fs::File` replacement used everywhere the lib writes large
// sequential output (mux, extract, sweep, patch) — drains dirty pages
// continuously instead of bursting. General I/O infra, not recovery policy;
// promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too.
// Drop-in `std::fs::File` replacement for large sequential output (mux, extract,
// sweep, patch); drains dirty pages continuously. `pub` for freemkv-engine reuse.
pub use io::WritebackFile;
/// Write an image-level source out as a sector image — what an `iso://`
/// DESTINATION means for any source that is not a physical drive. Drive sources
@@ -204,32 +185,17 @@ pub use event::{BatchSizeReason, Event, EventKind};
pub use identity::DriveId;
// ─── Unlock seam ────────────────────────────────────────────────────────────
//
// Drive/disc unlocking (removing bus encryption — firmware, AACS cert, CSS
// bus-auth) lives entirely in the `freemkv-unlock` crate. libfreemkv consumes
// it through the private `unlock_bridge` and exposes nothing of it: clients are
// oblivious to unlockers, exactly as they are to the SCSI layer. There is no
// public unlock surface to import.
// Drive/disc unlocking (firmware, AACS cert, CSS bus-auth) lives entirely in
// `freemkv-unlock`; libfreemkv consumes it via `unlock_bridge` and exposes nothing.
// ─── Decryption (AACS / CSS) ────────────────────────────────────────────────
//
// `Disc::scan()` resolves keys and stores them on `Disc`; in most flows you
// don't touch `DecryptKeys` directly — `DiscStream::new(reader, title, keys, …)`
// accepts whatever `Disc::decrypt_keys()` returned. `decrypt_sectors()` is
// for callers that operate on raw sector buffers (e.g. ISO patching).
// `Disc::scan()` resolves keys onto `Disc`; `DiscStream::new(...)` consumes
// them directly. `decrypt_sectors()` is for raw sector buffers (ISO patching).
pub use decrypt::{AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_threads};
// ─── Disc structure ─────────────────────────────────────────────────────────
//
// `Disc::scan()` produces a fully-populated `Disc` (titles, streams, AACS
// state). `Disc::identify()` is the fast path — UDF only, no playlist parse,
// for displaying disc name + format quickly while a full scan runs in the
// background. The codec / channel / resolution enums are the canonical
// structured representation; never compare against display strings.
// Note: `disc::Stream` here is the codec enum (audio / video / sub kind)
// — not the `pes::Stream` trait re-exported below as `PesStream`. Two
// different concepts, the same short name; the trait gets the `Pes`
// prefix at the crate root to keep both addressable.
// `Disc::scan()` fully populates `Disc`; `Disc::identify()` is a UDF-only fast path
// for name/format display. Codec enums are canonical; never compare display strings.
pub use dirimage::DirImage;
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
@@ -240,23 +206,8 @@ pub use disc::{
pub use keysource::{DiscInputs, KeySource, read_encrypted_units, resolve_and_apply};
// ─── Streams ────────────────────────────────────────────────────────────────
//
// All stream types implement `pes::Stream` — read PES frames from a source,
// write PES frames to a sink. Pick the right type at construction:
//
// - `DiscStream` — physical drive or ISO (any `SectorSource`). Read-only.
// - `MkvStream` — Matroska container. Read on `open()`, write on `create()`.
// - `M2tsStream` — Blu-ray Transport Stream. Write-only sink (`create()`).
// - `NetworkStream` — TCP. Read on `listen()`, write on `connect()`.
// - `NullStream` — write-only black-hole sink. Useful for benchmarks.
// - `StdioStream` — pipe to/from stdin/stdout. Read or write.
//
// Most consumers use the URL resolvers (`input()` / `output()`) which pick
// the right type from a scheme:// URL. Direct construction is for callers
// that need to wire custom readers (e.g. autorip's drive-session reuse).
// The trait is re-exported as `PesStream` here to disambiguate from
// `disc::Stream` (the codec-kind enum re-exported above), which would
// otherwise collide at the crate root.
// All types implement `pes::Stream` (re-exported `PesStream` to avoid colliding
// with `disc::Stream`). Prefer `input()`/`output()` URL resolvers over direct construction.
pub use pes::PesFrame;
pub use pes::Stream as PesStream;
@@ -272,13 +223,8 @@ pub use mux::{Medium, SourceInfo};
pub use mux::{Mp4FitReport, Mp4Sink, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
// `ScsiTransport` is the platform-abstraction trait Drive uses; expose for
// out-of-tree platform backends. `SectorSource` / `SectorSink` are the
// direction-typed read/write traits; `FileSectorSource` and `FileSectorSink`
// are the ISO-on-disk implementations. [`DecryptingSectorSource`] is the
// single decrypt-on-read decorator (AACS / CSS / none) — wrap any
// `SectorSource` to get plaintext sectors out.
// `ScsiTransport` is the platform trait Drive uses, exposed for out-of-tree
// backends. `DecryptingSectorSource` wraps any `SectorSource` to decrypt (AACS/CSS).
pub use mux::build_iso_pipeline;
pub use mux::resolve_mux_key_map;
pub use mux::select::{PidFilter, StreamSelection};
+47 -57
View File
@@ -4,7 +4,7 @@
//! Contains play items (clips) with in/out timestamps,
//! stream info (video, audio, subtitle tracks).
//!
//! Reference: https://github.com/lw/BluRay/wiki/MPLS
//! Format is documented in the BD-ROM PlayList (MPLS) specification.
use crate::error::{Error, Result};
@@ -146,17 +146,16 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
}
let clip_id = String::from_utf8_lossy(&item[0..5]).to_string();
let connection_condition = item[9] & 0x0F;
// After clip_id[0..5] and codec_id[5..9], item[9] is fully reserved;
// item[10] holds is_multi_angle (bit 4) plus connection_condition in its
// low nibble. stc_id follows at item[11] and in_time at item[12..16].
let connection_condition = item[10] & 0x0F;
let in_time = u32::from_be_bytes([item[12], item[13], item[14], item[15]]);
let out_time = u32::from_be_bytes([item[16], item[17], item[18], item[19]]);
// Parse STN table from the first play item
// PlayItem layout after out_time:
// [20:28] UO_mask_table (8 bytes)
// [28] misc flags (1 byte)
// [29] still_mode (1 byte)
// [30:32] still_time (2 bytes)
// [32:] STN_table
// Parse STN table from the first play item. PlayItem layout after out_time:
// UO_mask_table(8) + misc flags(1) + still_mode(1) + still_time(2), then
// STN_table starting at STN_OFFSET.
const STN_OFFSET: usize = 32;
if item_idx == 0 && item.len() > STN_OFFSET + 16 {
// STN header: length(2) + reserved(2) + counts(8) + reserved(4) = 16 bytes
@@ -235,11 +234,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
entry.stream_type = 6;
entry.secondary = true;
streams.push(entry);
// Skip extra ref bytes (audio refs + PG refs).
// Use `next < item.len()` to match the sibling secondary
// blocks; the inner `after_arefs < item.len()` re-guards
// the second read, so the stricter `+2` only mis-aligned
// spos when the aref count sits in the last 1-2 bytes.
// Skip extra ref bytes (audio refs + PG refs). `next < item.len()`
// matches the sibling secondary blocks; the inner
// `after_arefs < item.len()` re-guards the second read near the end.
if next < item.len() {
let n_arefs = item[next] as usize;
let after_arefs = next + 2 + n_arefs + (n_arefs % 2);
@@ -367,13 +364,9 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
return None;
}
// PID location depends on the stream-entry type (BD spec stream_entry()):
// type 1 (stream in the PlayItem's Clip): PID at +2
// type 2 (stream in a SubPath SubClip): +subpath_id(1)+subclip_id(1) → PID at +4
// type 3 / 4 (SubPath clip; type 4 = Dolby Vision +subpath_id(1) → PID at +3
// enhancement layer, e.g. PID 0x1015):
// Previously only type 1 was handled, so the DV EL (type 4) and any
// sub-path stream fell through to PID 0 and were dropped by the mux.
// PID location depends on stream-entry type (BD spec stream_entry()): type 1
// (PlayItem's Clip) → +2; type 2 (SubPath SubClip) → +4; type 3/4 (SubPath
// clip / DV enhancement layer) → +3. Previously only type 1 was handled.
let pid_off = match item[pos + 1] {
STREAM_ENTRY_PLAYITEM_CLIP => 2,
STREAM_ENTRY_SUBPATH_SUBCLIP => 4,
@@ -410,11 +403,9 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
let mut color_space_val = 0u8;
let mut language = String::new();
// `stream_type` here is the STN category passed by the caller, which is
// only ever a primary category (VIDEO/AUDIO/PG_SUBTITLE/IG). Secondary
// audio/video and the DV enhancement layer are parsed through their
// matching primary category (identical attribute layout) and re-tagged by
// the caller after this returns, so there are no secondary arms here.
// `stream_type` is the STN category from the caller — always a primary category
// (VIDEO/AUDIO/PG_SUBTITLE/IG). Secondary audio/video and the DV enhancement layer
// share the primary's attribute layout and are re-tagged by the caller afterward.
match stream_type {
STREAM_CATEGORY_VIDEO => {
// Video: coding_type(1) + format_rate(1) + [hdr_info(1) if HEVC]
@@ -533,10 +524,12 @@ mod tests {
item.extend_from_slice(*clip_id);
// [5..9] codec_id ("M2TS")
item.extend_from_slice(b"M2TS");
// [9] connection_condition in low nibble
// [9] fully reserved
item.push(0);
// [10] is_multi_angle (bit 4) + connection_condition (low nibble)
item.push(*conn & 0x0F);
// [10..12] reserved
item.extend_from_slice(&[0u8; 2]);
// [11] stc_id / reserved
item.push(0);
// [12..16] in_time
item.extend_from_slice(&in_time.to_be_bytes());
// [16..20] out_time
@@ -964,12 +957,9 @@ mod tests {
#[test]
fn mark_type_read_from_correct_offset() {
// Regression for the mark_type off-by-one: each PlayListMark entry is
// reserved(1) + mark_type(1) + .... The parser must read byte[1], not
// byte[0]. build_mpls_with_marks writes reserved=0 at byte[0] and the
// mark_type at byte[1], so a parser that read byte[0] would see 0 for
// every mark. Use distinct non-zero, non-1 types to make the offset
// error unmistakable.
// Regression for the mark_type off-by-one: PlayListMark is reserved(1)+mark_type(1)+...,
// so the parser must read byte[1], not byte[0]. build_mpls_with_marks sets reserved=0
// and mark_type=byte[1]; distinct non-zero/non-1 types make an offset error unmistakable.
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
let marks = vec![
TestMark {
@@ -1022,10 +1012,7 @@ mod tests {
assert_eq!(playlist.marks.len(), 0);
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests below. Grounded in the BD-ROM MPLS spec
// (https://github.com/lw/BluRay/wiki/MPLS) byte layout.
// ─────────────────────────────────────────────────────────────────────
// ─── Added hardening tests below, grounded in the BD-ROM MPLS spec byte layout ───
/// Header guard: parse() requires `playlist_start + 10 <= data.len()`
/// before reading the PlayList header (num_play_items at pl[6..8]).
@@ -1039,22 +1026,24 @@ mod tests {
assert!(parse(&data).is_err());
}
/// Spec: connection_condition is the LOW nibble of PlayItem byte[9]
/// (high nibble is reserved/flags). A byte 0xF5 must yield 5, not 0xF5.
/// connection_condition is the LOW nibble of PlayItem byte[10] (the high
/// nibble carries reserved bits and the is_multi_angle flag at bit 4).
/// byte[9] is entirely reserved and must NOT be read. A byte[10] of 0xF5
/// must yield 5, and a non-zero byte[9] must not leak into the value.
#[test]
fn connection_condition_is_low_nibble_only() {
// Build a custom item where byte[9] = 0xF5 (high nibble set).
// build_mpls masks with &0x0F when writing, so write raw to verify
// the PARSER masks. We patch the item byte directly after building.
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
let mut data = build_mpls(
&[(b"00001", 0, 0, 9000000)],
(1, 0, 0, 0, 0, 0, 0, 0),
&[video],
);
// Locate PlayItem byte[9]: header(40) + pl_header(10) + item_len(2) + 9.
let conn_idx = 40 + 10 + 2 + 9;
data[conn_idx] = 0xF5;
// PlayItem base: header(40) + pl_header(10) + item_len(2).
let item_base = 40 + 10 + 2;
// byte[9] fully reserved — set it to prove it does not leak in.
data[item_base + 9] = 0xFF;
// byte[10] high nibble set — parser must mask to the low nibble.
data[item_base + 10] = 0xF5;
let pl = parse(&data).expect("should parse");
assert_eq!(pl.play_items[0].connection_condition, 0x05);
}
@@ -1203,11 +1192,9 @@ mod tests {
/// PG, and the PG must keep its correct PID (proving IG advanced spos).
#[test]
fn ig_consumed_but_not_retained_and_dv_after_aligned() {
// STN parse order is video, audio, PG, IG, sec_audio, sec_video,
// pip_pg, DV. The IG entry must be consumed (advancing spos) but
// never retained. To PROVE IG advanced the cursor, place a Dolby
// Vision EL after the IG: if IG didn't advance spos, the DV parse
// would land on the IG bytes and read the wrong PID.
// STN parse order is video, audio, PG, IG, sec_audio, sec_video, pip_pg, DV.
// The IG entry must be consumed (advancing spos) but never retained; placing
// a DV EL right after IG proves this — a wrong spos would misread the DV PID.
let video = build_stream_entry_video(0x1011, 0x24, 8, 1, Some(0x12));
let ig = build_stream_entry_pg(0x1400, 0x91, b"eng"); // IG entry bytes
let dv = build_stream_entry_video(0x1015, 0x24, 8, 1, Some(0x12)); // DV EL
@@ -1399,8 +1386,9 @@ mod tests {
let mut item = Vec::new();
item.extend_from_slice(b"00009");
item.extend_from_slice(b"M2TS");
item.push(0x01); // connection_condition
item.extend_from_slice(&[0u8; 2]);
item.push(0); // [9] reserved
item.push(0x01); // [10] connection_condition (low nibble)
item.push(0); // [11] stc_id / reserved
item.extend_from_slice(&90000u32.to_be_bytes()); // in_time
item.extend_from_slice(&180000u32.to_be_bytes()); // out_time
item.resize(32, 0); // pad through STN_OFFSET; item.len()==32 so no STN
@@ -1456,13 +1444,15 @@ mod tests {
}
/// The 20 bytes a PlayItem needs for clip_id(5) + codec_id(4) +
/// connection_condition(1) + reserved(2) + IN_time(4) + OUT_time(4).
/// reserved(1) + connection_condition byte(1) + stc_id/reserved(1) +
/// IN_time(4) + OUT_time(4).
fn play_item_20(clip: &[u8; 5], cc: u8, in_t: u32, out_t: u32) -> Vec<u8> {
let mut it = Vec::new();
it.extend_from_slice(clip);
it.extend_from_slice(b"M2TS");
it.push(cc);
it.extend_from_slice(&[0u8; 2]);
it.push(0); // [9] reserved
it.push(cc); // [10] connection_condition (low nibble)
it.push(0); // [11] stc_id / reserved
it.extend_from_slice(&in_t.to_be_bytes());
it.extend_from_slice(&out_t.to_be_bytes());
assert_eq!(it.len(), 20);
+26 -51
View File
@@ -245,11 +245,8 @@ impl AuAssembler {
dts,
source,
});
// Backstop: the `buf`-size cap prunes marks only when bytes accumulate.
// A run of zero-length (or start-code-free) timed fragments grows no
// bytes, so bound the deque directly — drop the oldest (stalest) mark,
// which belongs to an already-emitted or lost AU. A real AU spans far
// fewer fragments than this cap.
// Backstop: zero-length/start-code-free fragments grow no bytes, so the
// `buf`-size cap never prunes marks. Bound the deque directly instead.
if self.marks.len() > MAX_MARKS {
self.marks.pop_front();
}
@@ -318,12 +315,9 @@ impl AuAssembler {
}
let end_abs = self.base + end as u64;
// The AU's own timing/source: take the FIRST Some of each field
// across every mark in this AU's range [base, end_abs), independently
// — one PES fragment may carry the source while a later fragment of
// the same AU carries the PTS (and vice versa), so reading only the
// front mark would drop the other field. This restores the semantics
// of the pre-consolidation separate pts/source mark deques.
// Take the first Some of each field independently across marks in
// [base, end_abs): one fragment may carry source while another carries
// PTS, so reading only the front mark would drop a field.
let (mut pts, mut dts, mut source) = (None, None, None);
while self.marks.front().is_some_and(|m| m.off < end_abs) {
let m = self.marks.pop_front().unwrap();
@@ -399,10 +393,8 @@ impl AuAssembler {
}
let data = self.buf[..end].to_vec();
self.buf.drain(..end);
// Shrink toward what this AU actually needed (the tail plus room for
// another AU of about this size). Only the short tail is copied, and it
// brings `cap` back under the `2*end` threshold so the next AU of this
// size hands over instead of copying.
// Shrink toward this AU's actual need so `cap` drops under the `2*end`
// threshold and the next similarly-sized AU hands over instead of copying.
self.buf.shrink_to(end.max(tail_len));
return data;
}
@@ -447,10 +439,9 @@ impl AuAssembler {
fn au_boundary_resumable(&mut self) -> Option<usize> {
match self.mode {
Mode::StartCode(marker) => {
// Stateless: the AU ends at the next delimiter after the opener at
// buf[0]. Resume from the furthest searched offset (never before 4,
// to skip the opening delimiter). find_start_code needs 4 bytes, so
// back up 3 to catch a code straddling the previous buffer end.
// Resume from the furthest searched offset (never before 4, to skip the
// opening delimiter at buf[0]); find_start_code needs 4 bytes, so back
// up 3 to catch a code straddling the previous buffer end.
let from = self.scan_pos.max(4);
match find_start_code(&self.buf, from, marker) {
Some(e) => Some(e),
@@ -683,10 +674,8 @@ mod tests {
#[test]
fn au_merges_pts_and_source_from_different_fragments() {
// One fragment of an AU may carry the source stamp while a later fragment
// of the SAME AU carries the PTS (each PES gets a source; only the anchor
// gets a PTS). The AU must keep BOTH — reading only the front mark would
// drop whichever field the first fragment lacked.
// One fragment may carry the source stamp while a later fragment of the same
// AU carries the PTS; the AU must keep both, not just the front mark's field.
let src = crate::pes::SourcePos::at_byte(4242);
let mut a = AuAssembler::for_codec(Codec::H264);
let full = au(0xAB, 80);
@@ -745,10 +734,8 @@ mod tests {
#[test]
fn vc1_i_frame_keeps_its_preceding_seq_and_entry_headers() {
// An I-frame AU is [seq 0x0F][entry 0x0E][frame 0x0D][slices]; a following
// P-frame is just [frame 0x0D][slices]. A plain 0x0D split would strand the
// seq/entry headers on the P-frame's AU — the decode bug. The VC-1 mode must
// group them with the I-frame that follows them.
// A plain 0x0D split would strand the seq/entry headers on the following
// P-frame's AU (a decode bug); VC-1 mode must group them with the I-frame.
let mut a = AuAssembler::for_codec(Codec::Vc1);
let mut iframe = bdu(VC1_SEQ, 0xAA, 8);
iframe.extend(bdu(VC1_ENTRY, 0xBB, 6));
@@ -875,11 +862,8 @@ mod tests {
#[test]
fn resumable_boundary_matches_from_scratch_across_all_fragmentations() {
// The incremental scan_pos cursor must produce byte-identical AUs to a
// whole-buffer rescan, at EVERY fragment granularity (this is what makes
// the O(n) resume equivalent to the old O(n^2) from-scratch scan). Build a
// multi-AU stream per codec, reassemble it fed 1 byte at a time up to
// whole, and require one canonical result.
// The incremental scan_pos cursor must match a whole-buffer rescan at every
// fragment granularity, from 1 byte at a time up to whole (O(n) vs O(n^2)).
let h264 = {
let mut s = au(0x11, 40); // AU1 (AUD + payload)
s.extend(au(0x22, 70)); // AU2
@@ -971,11 +955,9 @@ mod tests {
"an ordinary AU at the head of a clean run is continuous"
);
// Now start-code-free junk past the cap. The FIRST over-cap run still
// has the next AU's delimiter at buf[0], so it is force-flushed as an
// (over-long) access unit — nothing is discarded and nothing is lost.
// Only once the buffer holds no opener at all does the backstop throw
// bytes away, which is the case this test is about.
// The first over-cap run still has a delimiter at buf[0], so it force-flushes
// as an over-long AU with nothing lost. Only once no opener remains does the
// backstop discard bytes, which is what this test covers.
let junk = vec![0xAB; MAX_AU_BUFFER + 4096];
a.push(&junk, Some(2000), None, None, false);
a.push(&junk, Some(2100), None, None, false);
@@ -1118,10 +1100,8 @@ mod tests {
before,
"the emitted AU must own the buffer's allocation (no whole-frame copy)"
);
// The replacement buffer keeps room for another AU of about this size, so
// the next AU does not re-grow — but it is NOT pinned to the OLD capacity,
// which would make `buf` a permanent high-water mark and send every later
// smaller AU down the copy path (see
// Must not pin to the old capacity, or `buf` becomes a permanent high-water
// mark sending every later smaller AU down the copy path (see
// `handover_survives_a_large_au_instead_of_copying_every_later_one`).
assert!(
a.buf.capacity() >= au1.len(),
@@ -1149,10 +1129,9 @@ mod tests {
#[test]
fn handover_survives_a_large_au_instead_of_copying_every_later_one() {
let mut a = AuAssembler::for_codec(Codec::H264);
// Production shape: BD-TS aligns one access unit per PES, so each `push`
// carries about one AU and the buffer holds ~one AU at a time. One large AU
// (the IDR) followed by a run of much smaller ones (P/B frames). Each AU is
// pushed with the NEXT AU's opener so the previous one closes.
// BD-TS aligns one AU per PES, so each `push` carries about one AU. One large
// AU (IDR) is followed by smaller ones (P/B); each push includes the next
// AU's opener so the previous one closes.
const SMALL: usize = 64 * 1024;
let mut pending = au(0x11, 2 * 1024 * 1024);
for i in 0..20u8 {
@@ -1172,12 +1151,8 @@ mod tests {
}
// ── AU-opener detection: the per-mode start-code rule ─────────────────
//
// `au_opener_from` is the SECOND implementation of a rule each codec parser
// also encodes (h264 `NAL_AUD`, hevc `NAL_AUD`, vc1 `SC_*`, mpeg2
// `PICTURE_CODE`/`SEQ_HEADER_CODE`/`GOP_CODE`). Two independent copies of one
// rule drift; these cases pin this copy to the normative byte values and to
// the codes that are explicitly NOT openers, so a drift shows up here.
// `au_opener_from` duplicates a rule each codec parser also encodes; these
// cases pin it to the normative byte values so a drift between copies shows up.
/// The opener offset must be the position of the real start code, never a
/// fixed 0. A constant `Some(0)` makes every pre-sync run of junk bytes look
+414 -187
View File
@@ -33,12 +33,12 @@ use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// Sample rates indexed by fscod (0=48kHz, 1=44.1kHz, 2=32kHz). fscod=3 is
/// reserved in AC-3; in E-AC-3 it signals "fscod2" (reduced rates: 24/22.05/16
/// kHz, selected by byte-4 bits [5:4]). `frame_sample_rate` decodes fscod2 in
/// kHz, selected by byte-4 bits `[5:4]`). `frame_sample_rate` decodes fscod2 in
/// the E-AC-3 case; this table's index-3 entry (48 kHz) is only the fallback
/// when the header is too short to read fscod2.
const SAMPLE_RATES: [u32; 4] = [48_000, 44_100, 32_000, 48_000];
/// E-AC-3 reduced sample rates indexed by fscod2 (byte-4 bits [5:4]), used when
/// E-AC-3 reduced sample rates indexed by fscod2 (byte-4 bits `[5:4]`), used when
/// fscod==3. Index 3 is reserved; we fall back to 48 kHz for it.
const EAC3_REDUCED_RATES: [u32; 4] = [24_000, 22_050, 16_000, 48_000];
@@ -91,6 +91,18 @@ pub struct Ac3Parser {
/// across the PES boundary because it may be the core of an AC-3-core +
/// E-AC-3-dependent frame set whose remaining substreams are in the next PES.
saw_extension: bool,
/// The access unit held open across the last PES boundary, ALREADY
/// scanned. The carry-over begins at its first byte, so without this the
/// next call re-scans and re-CRCs every syncframe of it from byte 0 — and
/// an access unit that keeps gaining substreams grows to [`MAX_AC3_BUF`]
/// (1 MiB) before the resync guard drops it, which on a ~2 KiB DVD PES is
/// three orders of magnitude of repeated work per packet.
held: Option<HeldAu>,
/// Test-only: syncframes examined (sized + CRC-gated) by
/// `scan_access_units`. Pins the resume above — the property it exists for
/// is a WORK bound, which no frame-level assertion can observe.
#[cfg(test)]
frames_scanned: u64,
}
impl Default for Ac3Parser {
@@ -107,6 +119,9 @@ impl Ac3Parser {
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
saw_extension: false,
held: None,
#[cfg(test)]
frames_scanned: 0,
}
}
@@ -149,20 +164,56 @@ impl Ac3Parser {
anchor: Option<PtsAnchor>,
at_eos: bool,
marks: &[(usize, super::pesbuf::PesFacts)],
) -> (Vec<Frame>, usize, i64) {
held: Option<HeldAu>,
) -> ScanOut {
let mut frames = Vec::new();
let mut pos = 0usize;
// Running PTS for the next access unit to emit in this call.
let mut frame_pts_ns = base_pts_ns;
let mut anchor = anchor;
let mut pending: Option<PendingAu> = None;
// How far this call has proved there is no further syncframe to
// process; carried over so the held access unit's own bytes (and the
// junk after them) are not searched again next call.
let mut scanned_to = 0usize;
// Resume a held access unit instead of re-deriving it. `keep_from` was
// its first byte, so it starts at 0 of this buffer, and every frame in
// it was sized and CRC-gated on the call that built it.
if let Some(h) = held {
let mut drop_reason = h.drop_reason;
// The one verdict that can have changed since: the track may have
// become poisoned while this access unit was held, and a re-scan
// would have picked that up.
if drop_reason.is_none() && self.tally.is_poisoned() {
drop_reason = Some("track-poisoned");
}
pending = Some(PendingAu {
start: 0,
end: h.end,
pts_ns: base_pts_ns,
duration_ns: h.duration_ns,
drop_reason,
bsid: h.bsid,
});
frame_pts_ns = base_pts_ns + h.duration_ns as i64;
pos = h.scanned_to;
scanned_to = h.scanned_to;
}
while pos < data.len() {
let sync = find_ac3_sync(&data[pos..]);
let start = match sync {
Some(offset) => pos + offset,
None => break,
None => {
// No syncword in `data[pos..]` at all: every byte but the
// last is proved sync-free (a syncword is two bytes and the
// second may still arrive).
scanned_to = data.len().saturating_sub(1).max(pos);
break;
}
};
scanned_to = start;
let remaining = &data[start..];
@@ -182,6 +233,7 @@ impl Ac3Parser {
// Invalid/sub-header frame size (e.g. an E-AC-3 frmsiz of 0/1
// sizing to a 2/4-byte fragment) — skip this sync word.
pos = start + 2;
scanned_to = pos;
continue;
}
@@ -191,18 +243,20 @@ impl Ac3Parser {
}
let frame = &data[start..start + frame_size];
// Decodability gate: a syncframe with an out-of-range bsid (> 16) or
// a failed native CRC (payload corruption) poisons the access unit it
// belongs to — a dependent substream is useless without its parent and
// vice versa, so the whole access unit is dropped as one silence gap.
#[cfg(test)]
{
self.frames_scanned += 1;
}
// Decodability gate: an out-of-range bsid (> 16) or a failed native CRC
// poisons the whole access unit — a dependent substream is useless
// without its parent and vice versa — so it's dropped as one silence gap.
let reason = ac3_drop_reason(&self.tally, frame, bsid);
if substream_role(remaining, bsid) == SubstreamRole::Extends {
match pending.as_mut() {
// A dependent substream — or an additional independent
// substream (`substreamid` 1..7) — extends the frame set it
// directly follows. Requiring byte contiguity keeps skipped
// junk out of the emitted access unit.
// A dependent (or additional independent, substreamid 1..7)
// substream extending the frame set it directly follows; byte
// contiguity is required to keep skipped junk out of the AU.
Some(au) if au.end == start => {
au.end = start + frame_size;
if au.drop_reason.is_none() {
@@ -210,18 +264,9 @@ impl Ac3Parser {
}
self.saw_extension = true;
}
// A substream with no open access unit — a mid-frame-set
// resync, or a stream whose first syncframe is not
// `substreamid` 0 (Annex E orders a frame set independent
// substream 0 first, so joining mid-set is exactly the case
// here). It belongs to a frame set whose mandatory
// `substreamid`-0 substream was never seen, so it is neither
// decodable on its own (a dependent substream) nor timeable
// (an additional independent substream carries the frame set's
// time period, not its own PTS). Skip it rather than ship a
// fragment a decoder cannot use and re-time the whole
// timeline around; resync at the next `substreamid`-0
// substream. No PTS advance: it carries no duration.
// No open access unit: mid-frame-set resync, or a stream whose
// mandatory substreamid-0 substream was never seen. Neither
// decodable alone nor timeable, so skip it with no PTS advance.
_ => {
tracing::debug!(
target: "mux",
@@ -233,9 +278,8 @@ impl Ac3Parser {
if let Some(au) = pending.take() {
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
// First access unit that starts in this PES's own bytes: adopt
// this PES's timestamp so a genuine PTS jump is followed instead
// of the running cadence drifting past it.
// First AU starting in this PES's own bytes: adopt its timestamp so
// a genuine PTS jump is followed, not drifted past.
if let Some(a) = &anchor
&& start >= a.at
{
@@ -258,45 +302,42 @@ impl Ac3Parser {
}
pos = start + frame_size;
scanned_to = pos;
}
// Close or HOLD the trailing access unit. The rest of its frame set — its
// dependent substreams and any additional independent substreams — may
// still be in the next PES, so an access unit that can still grow is
// held: the carry-over rewinds to its first byte and the whole access
// unit is re-scanned (and only then counted/emitted) next call. An
// E-AC-3 `substreamid`-0 substream can always gain more of its frame set;
// a legacy AC-3 syncframe only in the AC-3-core + E-AC-3-dependent
// arrangement, so it is held only once this track has actually shown a
// substream that extends an access unit — a plain AC-3 track keeps
// emitting every frame in-call.
// Close or HOLD the trailing access unit: its frame set may still continue
// in the next PES, so a growable AU is held and re-scanned next call, but
// only once this track has shown a substream extending an AU (E-AC-3).
let mut hold_from = None;
let mut held_out = None;
if let Some(au) = pending {
if !at_eos && (au.bsid >= 11 || self.saw_extension) {
frame_pts_ns = au.pts_ns;
hold_from = Some(au.start);
// Everything below `scanned_to` is already searched and CRC-gated;
// record it rebased onto the carry-over (starting at `au.start`)
// so the next call resumes instead of redoing the work.
held_out = Some(HeldAu {
end: au.end - au.start,
scanned_to: scanned_to.max(au.end) - au.start,
duration_ns: au.duration_ns,
drop_reason: au.drop_reason,
bsid: au.bsid,
});
} else {
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
}
// Keep unconsumed data for the next call. `pos` is the start of the
// last unprocessed search region. On the `start + frame_size > len`
// break it sits exactly at the straddling frame's syncword; on the
// `remaining.len() < 6` break it is the value from the top of that
// iteration, with the syncword possibly sitting after some pre-sync
// junk — so the re-scan below (from `pos`, NOT a recomputed sync) is
// required to locate the carry-over syncword. Carry from `pos`, NOT
// from the next syncword: discarding bytes between `pos` and the next
// sync would drop the partial frame we are deliberately keeping across
// the boundary. A held access unit wins: it starts before `pos`.
// Keep unconsumed data for the next call: re-scan from `pos`, not a
// recomputed sync, to avoid dropping the partial frame kept across the
// PES boundary. Held AU wins (its start is before `pos`).
let keep_from = match hold_from {
Some(h) => h,
None if pos < data.len() => {
// A syncword at/after `pos` marks the carry-over start (anything
// before it is junk with no sync). With no full sync, retain the
// whole tail — including a lone trailing 0x0B that may be the first
// half of a syncword split across the PES boundary.
// A syncword at/after `pos` marks the carry-over start. With no full
// sync, retain the whole tail, including a lone trailing 0x0B that
// may be the first half of a syncword split across the boundary.
match find_ac3_sync(&data[pos..]) {
Some(o) => pos + o,
None if data.last() == Some(&0x0B) => data.len() - 1,
@@ -306,7 +347,12 @@ impl Ac3Parser {
None => data.len(),
};
(frames, keep_from, frame_pts_ns)
ScanOut {
frames,
keep_from,
frame_pts_ns,
held: held_out,
}
}
}
@@ -319,6 +365,36 @@ struct PtsAnchor {
pts_ns: i64,
}
/// What one `scan_access_units` pass produced: the access units it emitted,
/// the offset in the scanned buffer from which bytes must be carried over to
/// the next call, the PTS to stamp on the access unit that begins that
/// carry-over, and — when the trailing access unit was HELD — the state that
/// lets the next call resume rather than re-derive it.
struct ScanOut {
frames: Vec<Frame>,
keep_from: usize,
frame_pts_ns: i64,
held: Option<HeldAu>,
}
/// A trailing access unit held across the PES boundary, already scanned.
/// Offsets are relative to the carry-over, which begins at the access unit's
/// first byte — so the access unit occupies `0..end`.
#[derive(Clone, Copy)]
struct HeldAu {
/// End of the access unit's bytes.
end: usize,
/// How far the scan that built it had searched (`>= end`). Bytes below it
/// hold no further syncframe to process.
scanned_to: usize,
/// Duration contributed by the access unit's `substreamid`-0 substream.
duration_ns: u64,
/// Decodability verdict reached for it so far.
drop_reason: Option<&'static str>,
/// bsid of the substream that opened it.
bsid: u8,
}
/// An access unit (frame set) under construction: `data[start..end]` is the
/// `substreamid`-0 independent substream frame plus every substream appended to it
/// so far — its dependents, and any additional independent substreams 1..7 with
@@ -464,60 +540,43 @@ fn ac3_drop_reason(
impl CodecParser for Ac3Parser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
// B1: a concealed/lost gap means the bytes held in `buf` are a TRUNCATED
// frame. Appending the post-gap bytes would splice them into one corrupt
// frame (wrong frame_size, bad CRC → "exponent out of range" / garbage).
// Drop the partial and resync on the next syncword — a clean single-frame
// gap instead of a frankenstein frame. (The video parsers carry this via
// the ResyncGate; audio has no inter-frame refs, so dropping the spliced
// partial is the whole fix.)
//
// Handle the discontinuity BEFORE the empty-data guard so the signal can
// never be stranded by an empty post-gap PES (the demuxer only emits
// non-empty PES today; this is defensive for any future caller).
// B1: a concealed/lost gap means `buf` holds a TRUNCATED frame; appending
// post-gap bytes would splice a corrupt frame, so drop the partial and
// resync on the next syncword. Handled before the empty-data guard.
if pes.discontinuity {
self.acc.clear();
// The held access unit's bytes went with it.
self.held = None;
}
if pes.data.is_empty() {
return Vec::new();
}
// This PES's timestamp applies to the first access unit that STARTS in
// its own bytes; anything already buffered began in an earlier PES and
// keeps the running cadence (`flush_pts_ns`). Each subsequent access unit
// in the same call advances by the previous one's duration, so a PES that
// carries several access units stamps a monotonically increasing PTS
// instead of the same PES timestamp on all of them (which collapses their
// timecodes and drifts A/V).
//
// A PES with no PTS (rare for audio, but legal — and the case demuxers
// guard at a post-gap continuation) must NOT reset the timeline to 0: with
// no anchor the running cadence simply continues. The discontinuity-
// carrying PES is a PUSI with a PTS in practice, so this is
// defense-in-depth.
// This PES's timestamp applies to the first AU that STARTS in its own bytes;
// later AUs advance by the previous one's duration (avoiding A/V drift). A
// PES with no PTS must NOT reset the timeline: `flush_pts_ns` just continues.
let carry_len = self.acc.len();
let anchor = pes.pts.map(|p| PtsAnchor {
at: carry_len,
pts_ns: pts_to_ns(p),
});
// Prepend leftover from previous PES, then take the whole buffer into a
// local so the scanner can call `self.tally` (the bytes are no longer
// borrowed from `self`). The unconsumed tail is written back at the end.
// Prepend leftover from the previous PES, then take the scratch buffer out
// of `self` so the scanner can borrow `self.tally`; it's put back at the
// end, keeping its capacity. There is no early return after this point.
self.acc.push(pes);
// Copy the working bytes out so the scanner can borrow `self.tally`;
// the buffer keeps its marks, so the unconsumed tail stays attributed
// to the packet that carried it.
// Take the scratch OUT of `self` so the scanner can borrow `self.tally`
// while reading it; it is put back at the end of the call, keeping its
// capacity for the next PES. There is no early return after this point.
let mut buf = std::mem::take(&mut self.scratch);
buf.clear();
buf.extend_from_slice(self.acc.as_slice());
let marks = self.acc.marks_snapshot();
let data = &buf;
let (frames, keep_from, frame_pts_ns) =
self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks);
let held = self.held.take();
let ScanOut {
frames,
keep_from,
frame_pts_ns,
held: still_held,
} = self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks, held);
if keep_from < data.len() {
let tail = &data[keep_from..];
@@ -531,23 +590,22 @@ impl CodecParser for Ac3Parser {
MAX_AC3_BUF
);
self.acc.clear();
// Advance the cadence, as both sibling branches below do, so the
// three paths out of this block cannot disagree. Defensive: no
// input reaching this parser was found that both parses frames and
// leaves a residue this large, so the stale-cadence bug this
// prevents is not currently reachable and has no regression test.
self.held = None;
// Advance the cadence, as the other two paths out of this block do,
// so all three cannot disagree. Defensive: no input parsing frames
// and leaving a residue this large is currently reachable.
self.flush_pts_ns = frame_pts_ns;
} else {
self.acc.drain(keep_from);
// The carried bytes, when later completed and emitted (next call
// or by flush() at EOS), are timed at the PTS the scanner reached
// here: the PTS of the next access unit in presentation order, or
// — for a HELD access unit — that access unit's own PTS, so the
// hold never shifts it.
self.held = still_held;
// Carried bytes, when later completed and emitted, are timed at the
// PTS the scanner reached here — the next AU's PTS, or for a HELD
// AU, its own PTS, so the hold never shifts it.
self.flush_pts_ns = frame_pts_ns;
}
} else {
self.acc.clear();
self.held = None;
// Nothing carried, but keep the cadence so a following PES with no
// PTS (no anchor) continues the timeline instead of reusing a stale
// value.
@@ -561,17 +619,16 @@ impl CodecParser for Ac3Parser {
}
fn flush(&mut self) -> Vec<Frame> {
// Drain the carry-over buffer at EOS: a complete access unit (including
// one held back waiting for a possible dependent substream) may sit there
// with no following PES to close it, and without this drain the last
// ~32 ms of audio is lost. `at_eos` closes the trailing access unit
// instead of holding it; a partial/garbage tail yields nothing.
// Drain the carry-over at EOS: an AU (possibly held for a dependent
// substream) may sit there with no following PES to close it, else the
// last ~32ms of audio is lost. `at_eos` closes it instead of holding.
let buf = self.acc.as_slice().to_vec();
let marks = self.acc.marks_snapshot();
self.acc.clear();
let held = self.held.take();
let out = self
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks)
.0;
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks, held)
.frames;
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
@@ -605,7 +662,7 @@ fn eac3_samples_per_frame(data: &[u8]) -> u32 {
/// Sample rate (Hz) of an AC-3/E-AC-3 frame from its fscod field (byte 4 bits
/// 7-6). For E-AC-3 (`bsid >= 11`) an fscod of 3 selects a reduced rate via
/// fscod2 (byte 4 bits [5:4]); decoding it keeps the frame duration correct
/// fscod2 (byte 4 bits `[5:4]`); decoding it keeps the frame duration correct
/// instead of mistiming reduced-rate frames at 48 kHz (A/V drift).
fn frame_sample_rate(data: &[u8], bsid: u8) -> u32 {
if data.len() < 5 {
@@ -752,7 +809,7 @@ pub(crate) fn ac3_frame_size(data: &[u8]) -> usize {
}
}
/// AC-3 frame size table: [frmsizecod] -> [48kHz words, 44.1kHz words, 32kHz words]
/// AC-3 frame size table: `[frmsizecod]` -> `[48kHz words, 44.1kHz words, 32kHz words]`
const AC3_FRAME_SIZES: [[usize; 3]; 38] = [
[64, 69, 96],
[64, 70, 96],
@@ -853,11 +910,9 @@ mod tests {
"the buffer must be handed back to the parser, not dropped"
);
// The discriminator: a LARGE packet, then a small one. A reused buffer
// keeps the large capacity; a fresh `Vec` sized to each packet drops
// back to the small one. Feeding equal-sized packets cannot tell those
// apart — the first version of this test did exactly that and a
// to_vec()-per-call implementation passed it.
// Discriminator: LARGE packet then small. A reused buffer keeps the large
// capacity; a fresh `Vec` drops to the small size. Equal-sized packets
// can't tell those apart — an earlier to_vec()-per-call bug passed that test.
let big = [frame.clone(), frame.clone(), frame.clone(), frame.clone()].concat();
parser.parse(&PesPacket {
source: None,
@@ -955,11 +1010,9 @@ mod tests {
#[test]
fn discontinuity_drops_truncated_partial() {
// B1: a partial AC-3 frame is buffered, then a concealed gap arrives
// (PES marked discontinuity) carrying a fresh complete frame. The
// truncated partial must be DROPPED, not spliced — otherwise the parser
// emits one corrupt frame built from [stale partial | head of fresh] and
// strands the tail (decoders report "incomplete frame" / wrong sync).
// B1: a partial frame is buffered, then a discontinuity-marked PES brings a
// fresh complete frame. The partial must be DROPPED, not spliced, or the
// parser emits one corrupt frame from [stale partial | head of fresh].
let mut parser = Ac3Parser::new();
let frame_data = make_ac3_frame(0, 2); // 160 bytes, starts with 0x0B77
@@ -1103,11 +1156,9 @@ mod tests {
#[test]
fn buffer_stays_bounded_across_many_garbage_pes() {
// The carry-over buffer must never grow without bound. Feed
// many large PES packets that contain no usable frame and assert the
// retained buffer stays tiny — carry-from-`pos` drops all pre-sync junk,
// and a never-completing frame is bounded by the 8192-byte frame cap and
// the MAX_AC3_BUF resync guard.
// The carry-over buffer must never grow without bound: carry-from-`pos`
// drops pre-sync junk, and a never-completing frame is bounded by the
// 8192-byte frame cap and the MAX_AC3_BUF resync guard.
let mut parser = Ac3Parser::new();
for i in 0..256 {
// Vary the trailing byte so we also exercise the lone-0x0B retain.
@@ -1278,10 +1329,9 @@ mod tests {
#[test]
fn eac3_fscod2_reduced_rate_duration() {
// E-AC-3 with fscod==3 (reduced rate) and fscod2==0 → 24 kHz, not 48.
// bsid>=11 selects the E-AC-3 path. When fscod==3 the block count is
// fixed at 6 → 1536 samples. Byte 4 layout: fscod(2)|fscod2(2)|...
// fscod=3 (0b11), fscod2=0 (0b00) → byte4 = 0b1100_0000 = 0xC0.
// E-AC-3 with fscod==3 (reduced rate) and fscod2==0 → 24 kHz, not 48; block
// count is then fixed at 6 → 1536 samples. Byte 4: fscod(2)|fscod2(2)|...,
// so fscod=3 (0b11), fscod2=0 (0b00) → byte4 = 0b1100_0000 = 0xC0.
let data = [0x0B, 0x77, 0x00, 0x00, 0xC0, 16 << 3];
let bsid = get_bsid(&data);
assert!(bsid >= 11, "test frame is E-AC-3");
@@ -1522,11 +1572,9 @@ mod tests {
#[test]
fn eac3_frame_at_min_frame_bytes_passes_sizing_then_crc_gate() {
// The smallest frame the SIZING layer accepts is MIN_FRAME_BYTES = 6
// (frmsiz=2). A synthetic all-zero 6-byte frame passes sizing (so it
// reaches the decodability gate — proven by it being COUNTED as a drop,
// not silently size-skipped) but fails the CRC gate and is dropped; the
// following real AC-3 frame (valid CRC) is emitted.
// Smallest frame SIZING accepts is MIN_FRAME_BYTES = 6 (frmsiz=2). An
// all-zero 6-byte frame passes sizing (proven COUNTED as a drop, not
// size-skipped) but fails CRC; the following real AC-3 frame is emitted.
let mut parser = Ac3Parser::new();
// 0x0B 0x77 | byte2=0 byte3=2 (frmsiz=2 → 6 bytes) | byte4=0 | byte5 bsid
let mut data = vec![0x0B, 0x77, 0x00, 0x02, 0x00, 16 << 3];
@@ -1544,10 +1592,9 @@ mod tests {
#[test]
fn eac3_max_frmsiz_frame_within_window_accepted() {
// E-AC-3 frmsiz is an 11-bit field (3 bits of byte2 + 8 bits of byte3),
// so its maximum value is 0x7FF = 2047 → (2048)*2 = 4096 bytes, which is
// inside the MIN_FRAME_BYTES..=8192 accept window and, with a valid CRC,
// must be emitted.
// E-AC-3 frmsiz is an 11-bit field, so its max value 0x7FF = 2047 →
// (2048)*2 = 4096 bytes, inside the MIN_FRAME_BYTES..=8192 accept window
// and, with a valid CRC, must be emitted.
let mut parser = Ac3Parser::new();
let mut frame = vec![0u8; 4096];
frame[0] = 0x0B;
@@ -1556,9 +1603,8 @@ mod tests {
frame[3] = 0xFF; // frmsiz low → 0x7FF = 2047 → 4096 bytes
frame[5] = 16 << 3; // bsid 16 (E-AC-3)
finalize_ac3_crc(&mut frame); // pass the decodability gate
// The trailing E-AC-3 access unit is HELD at the end of the call (a
// dependent substream may follow in the next PES), so it is closed by
// flush() at EOS rather than in-call. Content and size are unchanged.
// Trailing E-AC-3 AU is HELD at end of call (a dependent substream may
// follow), so it's closed by flush() at EOS, not in-call.
let mut f = parser.parse(&make_eac3_pes(frame));
f.extend(parser.flush());
assert_eq!(f.len(), 1, "4096-byte E-AC-3 frame within window accepted");
@@ -1605,10 +1651,9 @@ mod tests {
#[test]
fn flush_rejects_frame_extending_past_buffer() {
// A buffered sync whose decoded frame size exceeds the buffered bytes
// must be dropped by flush (never emit fewer bytes than the size field
// declares). Build a real AC-3 header (160-byte frame) but only buffer
// 100 bytes.
// A buffered sync whose decoded frame size exceeds the buffered bytes must
// be dropped by flush (never emit fewer bytes than declared). Build a real
// 160-byte AC-3 frame but only buffer 100 bytes.
let mut parser = Ac3Parser::new();
let frame = make_ac3_frame(0, 2); // sizes to 160
parser.acc.seed(&frame[..100]);
@@ -1669,11 +1714,9 @@ mod tests {
#[test]
fn acmod_channels_stereo_2_0_no_lfe() {
// acmod=2 (2/0 L,R), no LFE → 2 channels. Verifies the channel count is
// read from the AC-3 bitstream's acmod, independent of any IFO claim.
// (A disc whose IFO lists 5.1 but where the wrong physical substream is
// selected is a separate stream-SELECTION bug, not this label path —
// tracked for rc.5.2.)
// acmod=2 (2/0 L,R), no LFE → 2 channels; verifies the channel count comes
// from the AC-3 bitstream's acmod, independent of any IFO claim (a disc
// IFO/substream mismatch is a separate SELECTION bug, tracked for rc.5.2).
assert_eq!(acmod_channels(&make_bsi(2, false)), Some(2));
}
@@ -1781,9 +1824,8 @@ mod tests {
#[test]
fn crc_drop_preserves_pts_sync_no_shift() {
// THE INVARIANT: dropping a corrupt frame must not shift the audio after
// it. good / corrupt / good in one PES — the corrupt frame is dropped but
// the trailing clean frame keeps the EXACT PTS it would have had with no
// drop (base + 2 frame durations): a silence gap, not a shift.
// it. good/corrupt/good in one PES — the trailing clean frame keeps the
// EXACT PTS it would have had with no drop: a silence gap, not a shift.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2); // f0
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2)); // dropped
@@ -1853,13 +1895,9 @@ mod tests {
#[test]
fn eac3_independent_plus_dependent_is_one_access_unit() {
// THE FIX: per ETSI TS 102 366 Annex E an access unit is the independent
// substream (strmtyp 0) plus every dependent substream (strmtyp 1) that
// follows it — the 7.1 Dolby Digital Plus arrangement. Both syncframes
// must emerge as ONE frame carrying the INDEPENDENT substream's PTS and
// exactly ONE frame duration (previously each syncframe was emitted as
// its own access unit and each advanced the clock, doubling the timeline
// and handing decoders a parentless dependent substream).
// THE FIX: per ETSI TS 102 366 Annex E an AU is the independent substream
// plus every dependent substream that follows it — both must emerge as ONE
// frame with the independent PTS and ONE duration (not double it).
let mut parser = Ac3Parser::new();
let indep = make_eac3_frame(0, 0, 160);
let dep = make_eac3_frame(1, 0, 96);
@@ -1899,10 +1937,9 @@ mod tests {
#[test]
fn eac3_grouped_timeline_is_not_doubled() {
// Two complete access units (independent + dependent each) in one PES,
// followed by a third independent substream that closes the second.
// The emitted PTS cadence must be one frame duration per access unit —
// the 2x-runtime / A-V-drift symptom of ungrouped substreams.
// Two complete access units (independent + dependent each) in one PES, plus
// a third independent substream closing the second. PTS cadence must be
// one frame duration per AU — else the 2x-runtime / A-V-drift symptom.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..2 {
@@ -1923,10 +1960,9 @@ mod tests {
#[test]
fn plain_ac3_frames_are_not_grouped_or_delayed() {
// NO REGRESSION for legacy AC-3 (bsid < 11): it has no substream
// structure (byte 2 is crc1, not strmtyp), so every syncframe is a
// complete access unit, emitted in the SAME call — never merged with its
// neighbour and never held back for a dependent that cannot exist.
// NO REGRESSION for legacy AC-3 (bsid < 11): it has no substream structure
// (byte 2 is crc1, not strmtyp), so every syncframe is a complete AU,
// emitted in the SAME call — never merged, never held for a dependent.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..3 {
@@ -1945,10 +1981,9 @@ mod tests {
#[test]
fn eac3_access_unit_split_across_pes_is_grouped() {
// The access unit boundary is only known at the NEXT independent
// substream, so a trailing independent substream is held across the PES
// boundary: the dependent half arriving in the next PES still joins it,
// and the AU keeps the FIRST PES's PTS (its independent substream's).
// The AU boundary is only known at the NEXT independent substream, so a
// trailing one is held across the PES boundary: the dependent half in the
// next PES still joins it, keeping the FIRST PES's PTS.
let mut parser = Ac3Parser::new();
let indep = make_eac3_frame(0, 0, 160);
let dep = make_eac3_frame(1, 0, 96);
@@ -2059,14 +2094,9 @@ mod tests {
#[test]
fn eac3_additional_independent_substream_stays_in_the_frame_set() {
// THE FIX: an Annex E frame set is independent substream 0 (mandatory,
// first) with its dependents, then the OPTIONAL additional independent
// substreams 1..7 with theirs — ALL covering the SAME time period. An
// additional independent substream (an associated / commentary service)
// therefore belongs to the frame set already open; keying the access-unit
// boundary on `strmtyp` alone made it close the AU and advance the running
// PTS a SECOND time over the same 32 ms, doubling the timeline (~1 s of
// A/V drift per second of audio).
// THE FIX: a frame set is substream 0 plus dependents, then OPTIONAL
// additional independent substreams 1..7 — ALL the SAME time period.
// Keying the AU boundary on `strmtyp` alone doubled the PTS advance.
let mut parser = Ac3Parser::new();
let ind0 = make_eac3_frame(0, 0, 160); // main programme
let dep0 = make_eac3_frame(1, 0, 96); // its dependent (7.1 extension)
@@ -2113,13 +2143,9 @@ mod tests {
#[test]
fn eac3_stream_joined_mid_frame_set_resyncs_at_substreamid_0() {
// A stream whose first syncframe is an ADDITIONAL independent substream
// (here substreamid 3) joined a frame set whose mandatory substreamid-0
// substream was never seen. Mirroring the orphan-dependent rule, it is
// skipped: it carries the frame set's time period rather than its own, so
// emitting it as an access unit would invent a period for audio whose
// main programme is missing. Grouping resyncs at the next substreamid-0
// substream, and the orphan consumes none of the timeline.
// A stream whose first syncframe is ADDITIONAL substream (substreamid 3)
// joined a set whose mandatory substreamid-0 was never seen. It's skipped
// (carries the set's time period, not its own); resyncs at substreamid-0.
let mut parser = Ac3Parser::new();
let orphan = make_eac3_frame(0, 3, 128);
let orphan_dep = make_eac3_frame(1, 3, 96);
@@ -2176,10 +2202,9 @@ mod tests {
#[test]
fn corrupt_dependent_substream_drops_the_whole_access_unit() {
// A dependent substream that fails its native CRC poisons the access unit
// it belongs to: emitting the independent half alone would ship an AU a
// decoder must reassemble from a corrupt pair. The drop accounts for one
// frame duration, so the following AU keeps its true PTS (gap, not shift).
// A dependent substream failing its native CRC poisons the whole AU:
// emitting the independent half alone ships a pair no decoder can
// reassemble. The drop still accounts for one duration (gap, not shift).
let mut parser = Ac3Parser::new();
let mut dep = make_eac3_frame(1, 0, 96);
dep[20] ^= 0xFF; // break the dependent substream's CRC
@@ -2199,6 +2224,125 @@ mod tests {
);
}
/// A 256-byte E-AC-3 syncframe with a valid CRC. `strmtyp`/`substreamid`
/// go into byte 2 (A/52 Annex E), which is what `substream_role` reads:
/// (0, 0) OPENS an access unit, (1, 0) is a dependent substream that
/// EXTENDS the open one.
fn eac3_substream_frame(strmtyp: u8, substreamid: u8) -> Vec<u8> {
const SIZE: usize = 256;
let frmsiz = SIZE / 2 - 1; // (frmsiz + 1) * 2 == SIZE
let mut f = vec![0u8; SIZE];
f[0] = 0x0B;
f[1] = 0x77;
f[2] = (strmtyp << 6) | (substreamid << 3) | ((frmsiz >> 8) as u8 & 0x07);
f[3] = (frmsiz & 0xFF) as u8;
f[5] = 16 << 3; // bsid 16 → E-AC-3
finalize_ac3_crc(&mut f);
f
}
/// An access unit closes only at the next `substreamid`-0 independent
/// substream, so one that keeps gaining dependent substreams stays OPEN
/// across PES boundaries and its bytes stay in the carry-over. The
/// carry-over must not be re-scanned — and re-CRCed — from the access
/// unit's first byte on every packet: the buffer only stops growing at
/// MAX_AC3_BUF (1 MiB), and a PES on a DVD is about 2 KiB, so re-deriving
/// the held access unit costs work quadratic in the packets fed.
///
/// Measured directly, because a work bound has no frame-level shadow:
/// `frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
/// Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
///
/// Mutation: pass `None` for `held` in `parse` (or drop the `if let
/// Some(h) = held` resume) — the count returns to the quadratic figure.
#[test]
fn a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet() {
const DEPENDENTS: usize = 200;
let mut parser = Ac3Parser::new();
// Opens the access unit.
let emitted = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
emitted.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..DEPENDENTS {
let f = parser.parse(&make_eac3_pes(eac3_substream_frame(1, 0)));
assert!(f.is_empty(), "a dependent substream extends the open unit");
}
let fed = (DEPENDENTS + 1) as u64;
assert!(
parser.frames_scanned <= 2 * fed,
"the scanner examined {} syncframes for {fed} fed — a held access \
unit must be resumed, not re-derived",
parser.frames_scanned
);
// ...and the resume must not have cost correctness: the whole frame
// set is still one access unit, emitted intact at EOS.
let out = parser.flush();
assert_eq!(out.len(), 1, "the frame set is a single access unit");
assert_eq!(
out[0].data.len(),
256 * (DEPENDENTS + 1),
"every substream of the frame set belongs to it"
);
}
/// A concealed gap must drop the HELD access unit, not just the byte
/// buffer.
///
/// `parse` clears `self.acc` on a discontinuity because the buffered bytes
/// are a truncated frame. The held access unit is described by OFFSETS into
/// exactly those bytes, so it has to go with them. Without
/// `self.held = None`, the next packet resumes a HeldAu whose `start`/`end`
/// were computed against the pre-gap buffer but are applied to the
/// unrelated post-gap bytes — splicing audio across the gap at best, and
/// indexing past the end of the new, shorter buffer at worst.
///
/// The two existing discontinuity tests use plain AC-3 (bsid < 11), which
/// never holds an access unit open, so neither of them reaches this reset.
#[test]
fn a_discontinuity_drops_the_held_access_unit_with_its_bytes() {
let mut parser = Ac3Parser::new();
// Open an access unit and extend it, so a HeldAu exists describing
// offsets into a large buffer.
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(0, 0)))
.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..8 {
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(1, 0)))
.is_empty(),
"a dependent substream extends the open unit"
);
}
// The gap. Its post-gap payload is deliberately far SHORTER than the
// held unit's bytes, so a stale HeldAu indexes past its end.
let mut gap = make_eac3_pes(eac3_substream_frame(0, 0));
gap.discontinuity = true;
let _ = parser.parse(&gap);
// Whatever comes out, nothing may carry pre-gap bytes: the truncated
// unit was dropped, so the only access unit that can be emitted is the
// one opened after the gap.
let out = parser.flush();
let total: usize = out.iter().map(|f| f.data.len()).sum();
assert!(
total <= 256,
"a post-gap access unit must not be spliced onto the 9 frames held \
before the gap; got {total} bytes across {} frame(s)",
out.len()
);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
@@ -2248,4 +2392,87 @@ mod tests {
"the unit belongs to the packet its FIRST byte came from"
);
}
/// A track that becomes POISONED while an access unit is held open across a
/// PES boundary must not emit that access unit when it resumes.
///
/// Mutation this catches: deleting (or inverting) the resume-path re-check
/// `if drop_reason.is_none() && self.tally.is_poisoned()` at the top of
/// `scan_access_units`. The verdict on a held access unit is frozen at the
/// moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
/// previous access unit is closed — so the very drop that crosses the
/// poison threshold lands after the next unit's verdict was already taken
/// as `None`. Without the re-check that unit is emitted as an ordinary
/// frame after the track has been judged too damaged to mux: corrupt audio
/// passed through as success, and worse, it is the ONE frame that escapes
/// a whole-track fallback whose entire point is that nothing after the
/// verdict ships.
///
/// Neither existing held-AU test reaches this: both keep a pristine tally.
///
/// The fixture drives the exact interleaving above. `DropTally` poisons
/// once `verified_dropped * 2 > kept + dropped` past the 200-AU gate, so
/// one PES carries 200 CRC-failing `substreamid`-0 syncframes followed by a
/// clean one. Closing corrupt unit #200 (which happens only when the clean
/// frame is reached) latches the poison — after that clean frame's own
/// verdict was computed. Being an E-AC-3 unit that can still gain
/// substreams, it is then HELD across the boundary with `drop_reason:
/// None`, which is precisely the state the re-check exists for.
#[test]
fn a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes() {
// One more than the verdict gate: the Nth close is what poisons.
const CORRUPT_AUS: usize = 200;
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..CORRUPT_AUS {
let mut f = eac3_substream_frame(0, 0);
// Corrupt a payload byte AFTER CRC finalization: header stays intact so
// the frame is still parsed whole and fails only CRC — a VERIFIED
// drop, the only kind that feeds the poison verdict.
f[100] ^= 0xFF;
assert!(!frame_crc_ok(&f), "the fixture frame must fail its CRC");
data.extend_from_slice(&f);
}
// The clean access unit. Last in the PES, so it is held open.
let clean = eac3_substream_frame(0, 0);
assert!(
frame_crc_ok(&clean),
"the held unit is individually decodable"
);
data.extend_from_slice(&clean);
let emitted = parser.parse(&make_eac3_pes(data));
assert!(
emitted.is_empty(),
"every corrupt unit is dropped and the clean one is held; got {} frame(s)",
emitted.len()
);
// The state the re-check depends on: the track IS poisoned, and the
// held unit was opened before that verdict existed.
assert!(
parser.tally.is_poisoned(),
"fixture must actually cross the whole-track poison threshold"
);
// Resume. The next PES opens a new unit, which closes the held one.
let out = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
out.is_empty(),
"an access unit held across the boundary must not be emitted once \
the track is poisoned; got {} frame(s) totalling {} bytes",
out.len(),
out.iter().map(|f| f.data.len()).sum::<usize>()
);
// ...and nothing may leak at end of stream either.
let tail = parser.flush();
assert!(
tail.is_empty(),
"a poisoned track emits nothing at EOS; got {} frame(s)",
tail.len()
);
assert!(
parser.dropped_frames() > CORRUPT_AUS as u64,
"the held unit must be ACCOUNTED as a drop, not silently discarded"
);
}
}
+3 -7
View File
@@ -50,13 +50,9 @@ fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
// The floor is the header the frame SAYS it carries, not a constant.
// protection_absent (byte1 bit0) clear means a 16-bit crc_check follows the
// 7-byte fixed+variable header, so the frame cannot be shorter than 9 —
// aac_frame_length counts the header and the CRC, not just the payload.
// Comparing against a flat 7 let a CRC-present frame declaring 7 or 8
// through as structurally Valid, and the muxer then carried a frame whose
// own header says it is impossible.
// Floor depends on protection_absent: CRC-present (byte1 bit0 clear) adds a
// 16-bit crc_check after the 7-byte header, so the min is 9, not a flat 7 —
// else a CRC frame declaring length 7-8 wrongly passed as Valid.
let header_bytes = if data[1] & 0x01 == 0 { 9 } else { 7 };
if frame_length < header_bytes {
return AdtsVerdict::Invalid;

Some files were not shown because too many files have changed in this diff Show More