Compare commits

...
53 Commits
Author SHA1 Message Date
Matthew Jackson d40540ce9e v1.6.13: bump version (freemkv-unlock git-pinned for the tag) 2026-08-28 15:48:58 +00:00
Matthew Jackson ba14d66c18 Release 1.6.13: capture additional drive buffers during info --share 2026-08-27 21:22:11 -07:00
Matthew Jackson f16abe1cf4 restore freemkv-unlock path dep for local dev (post-v1.6.12) 2026-08-27 22:57:33 +00:00
Matthew Jackson 477c0da6dd v1.6.12: bump version (freemkv-unlock git-pinned for the tag) 2026-08-27 22:57:29 +00:00
Matthew Jackson 41c5b8b785 changelog: 1.6.12 (unreleased) 2026-08-27 13:11:27 -07:00
Matthew Jackson 9f43ae94d3 Comment and Documentation Cleanup
- Cap every inline `//` comment block at 3 lines; distil the rationale
  rather than delete it. Enforced by a new comment-guard CI check.
- Fix stale and orphaned rustdoc: broken intra-doc links (renamed or
  removed items), doc blocks that had merged onto the wrong item (leaving
  the intended item undocumented), and doc-contradicts-code drift.
- Add a doc-links CI check (cargo doc, deny broken intra-doc links) so
  both classes of comment rot are caught going forward.
2026-08-27 12:07:53 -07:00
Matthew Jackson 9ed6f86255 Fix macOS optical E1000: hold a DiskArbitration claim for the session
The macOS SCSI transport took exclusive SCSI access plus a one-shot
`diskutil unmountDisk`, but never claimed the disc from DiskArbitration.
diskarbitrationd was therefore free to remount the volume mid-rip (a
Spotlight re-probe is the usual trigger), yanking the device out from
under an in-progress read; the failure surfaced to the user as E1000.

Hold a DADiskClaim plus a mount-approval dissenter scoped to our disc for
the transport lifetime, so the OS cannot remount it until shim_close().
Link the DiskArbitration framework in build.rs.
2026-08-27 11:17:01 -07:00
Matthew Jackson 70ae7b6c10 restore freemkv-unlock path dep for local dev (post-v1.6.11) 2026-08-25 23:17:11 -07:00
Matthew Jackson 3f03c6db06 v1.6.11: bump version (freemkv-unlock git-pinned for the tag) 2026-08-25 23:17:07 -07:00
Matthew Jackson 562bb8bee5 docs: neutralize tool-comparison and technique wording (opsec) 2026-08-25 22:57:01 -07:00
Matthew Jackson 7c13ed32b5 fix: align drive-prep unlocker token with unlock rename; neutralize product name 2026-08-25 22:54:38 -07:00
Matthew Jackson 574aa2c506 docs: neutralize retail title references (opsec) 2026-08-25 22:45:57 -07:00
Matthew Jackson 3912762b52 docs: neutralize comment provenance and external references (opsec) 2026-08-25 22:31:19 -07:00
Matthew Jackson 028768e71b docs: 1.6.11 changelog 2026-08-25 22:08:48 -07:00
Matthew Jackson bf4e5ac395 docs: standardize README badges (License, CI, codecov) 2026-08-25 20:26:28 -07:00
Matthew Jackson 11948d513b nav: taint control-transfer decisions only, and track SPRM-laundered GPRMs
Refine the First-Play SPRM-taint abstain to fix two defects:

- Over-abstention: a tainted compare that guards a SetGPRM store (not a
  control transfer) no longer forces abstention. The predicate only
  decides whether the store runs; a following unconditional dispatch is
  still resolvable. Taint now gates only JumpTT/Goto (the control
  transfers the resolver follows); a store under a tainted guard taints
  its destination GPRM (unknown post-state) instead of abstaining.

- Under-abstention: an SPRM laundered through a GPRM (SetGPRM g = SPRM,
  then a later GPRM-only compare on g) previously evaded taint and let
  the resolver commit to the cold-machine arm. Add per-GPRM taint bits:
  a store propagates its source's taint (SPRM or tainted GPRM) and a
  concrete/immediate store clears it; eval marks a compare tainted when
  it reads a tainted GPRM.

Adds red-before-green tests for both plus taint-clear guards.
2026-08-25 20:26:03 -07:00
Matthew Jackson 7d5c0a5c41 Fix round-3 audit findings: VM operand decode, MPLS field offset, log escaping, nav SPRM abstain, tests
1. dvdnav/vmcmd: the register operand of the if_v1/if_v3 compare layouts is the
   LOW byte of the reg-or-immediate field (b[5] / b[7]), not the high byte
   (b[4] / b[6]). Fix both layouts and their comment; add register-mode KATs
   with distinct high/low bytes.

2. mpls: connection_condition is the low nibble of PlayItem byte[10]
   (byte[9] is fully reserved), corroborated by the byte-11 stc_id and
   byte-12 in_time reads. Relocate every fixture's encoded value and update the
   regression test to assert the reserved byte does not leak.

3. labels/deluxe: escape the two disc-derived class-name tracing fields via the
   Debug path, matching the other log sites.

4. disc/dvd: add an integration test that drives scan_dvd_titles with a resolved
   First-Play nav target so the 1-based promotion join is verified end-to-end.

5. bdnav/vm: add unit tests for the SET/SWAP immediate gate — full register
   exchange, and that an immediate-flagged operand is never a write target.

6. dvdnav/nav: honor the documented contract — a First-Play compare that reads a
   system parameter is undecidable at scan time, so eval now taints on SPRM reads
   and the executor abstains rather than committing to the cold-power-on arm.
   GPRM reads never taint.

7. labels/deluxe: narrow four now-internal helpers from pub(crate) to private.
2026-08-25 20:26:03 -07:00
Matthew Jackson 1ede962aec ci(coverage): authenticate Codecov upload with org CODECOV_TOKEN 2026-08-25 19:32:30 -07:00
Matthew Jackson 014ea870a4 ci(coverage): add --ignore-run-fail so instrumentation flake never blocks upload 2026-08-25 19:04:41 -07:00
Matthew Jackson bdd2876d78 ci: add non-required Codecov coverage job, config, and badge
Add an additive coverage job to CI that generates lcov via cargo llvm-cov
and uploads to Codecov (tokenless). Mirrors the test job's pinned 1.94
toolchain and dual freemkv-unlock checkout for the path dep. Informational
only: no gate depends on it. Adds codecov.yml and a README badge.
2026-08-25 18:54:23 -07:00
Matthew Jackson 44de7a8d12 harness: fuzz the Deluxe bytecode decoder directly, not through a dead zip open
Audit round-2 (fix-adversary): the round-1 deluxe sweep fed the zip local-file-
header magic, but ZipArchive::new validates the end-of-central-directory at the
buffer's end — which the byte generators (front magic + noise) can never
synthesize — so the archive open always failed and decode_binding was never
reached: false coverage for the very bytecode-decoder panic path it guarded.
Drive decode_binding_class directly on a parsed .class with an empty master-enum
table instead, so the symbolic stack machine is actually exercised.
2026-08-25 18:45:28 -07:00
Matthew Jackson 552076a943 disc/labels/harness: audit round-1 batch — DoS bound, fuzz coverage, API surface
Continuing the /codeaudit HEAD..v1.6.10 round:

- disc/mod.rs rank_titles: bound the O(n^2 * clips) composite scan to a
  sub-second work budget (both title count and per-title clip count are
  untrusted disc metadata); above it, skip composite classification and degrade
  to the has-video + canonical order. Reorder the per-pair checks so the O(len)
  is_subset runs only after the cheap video/size gates. (regression test)
- bdnav/vm.rs SWAP: honour the imm_op1/imm_op2 store gates so a SWAP to an
  immediate operand does not corrupt a register.
- harness.rs: fuzz the previously-uncovered new parsers — the Deluxe bytecode
  decoder, the Fox dcx.xml scanner, and the combined bdnav index+mobj+VM resolve
  path (labels::{deluxe,fox} and bdnav::vm made pub(crate) for in-crate fuzzing).
- dvdnav: pub(crate) the module + submodules (was pub) to match bdnav and stop
  leaking the DVD IFO/VM internals as public API; drop an unused re-export.
- labels: escape disc-derived strings in diagnostics (Debug not Display) so a
  hostile disc cannot inject terminal control sequences into logs.
2026-08-25 18:33:37 -07:00
Matthew Jackson 3ecbeb7eba disc/bdnav: audit round-1 — VM title/PSR faithfulness + main_title never-panics
From the /codeaudit HEAD..v1.6.10 round (each verified against the libbluray
HDMV VM reference, and design-refuted SOUND on two models):

- bdnav/vm.rs title_obj: resolve First Play via the spec sentinel 0xFFFF (was
  num_titles+1). A JumpTitle 0xFFFF now reaches First Play, and an out-of-range
  JumpTitle(N+1) correctly abstains instead of mis-jumping there.
- bdnav/vm.rs wr: refuse stores to PSR-tagged registers (matches _store_reg),
  so an illegal SET/SWAP to a PSR is a no-op and later compares agree with a
  real player instead of diverging.
- disc/profile.rs main_title(): return Option<&TitleProfile> instead of indexing
  titles[0], which panicked on a disc that scanned to zero titles.

Regression tests added, each shown red-before-green.
2026-08-25 18:16:17 -07:00
Matthew Jackson b405154fd0 ci: run the consumers job at the highest dependent MSRV (1.98)
The MSRV split raised the dependents above libfreemkv's own floor — the
bins (freemkv, autorip, bdemu) are 1.98 and freemkv-keysources is 1.97 —
so the consumers job could no longer build them at libfreemkv's 1.94
toolchain (`rustc 1.94.0 is not supported by ... freemkv-keysources
requires rustc 1.97`), leaving dev red since the MSRV change. Pin that one
job to 1.98, which satisfies every dependent's floor. libfreemkv's own
lib/lint/test jobs stay at its 1.94 MSRV.
2026-08-25 13:49:35 -07:00
Matthew Jackson 8a630c8839 disc: payload-plausibility floor on nav/authoring feature promotion
A nav- or authoring-resolved playlist could promote a video-bearing but
negligible-payload branch above the substantial feature it sits beside:
Spotlight (2015) UHD navigates to a Dolby-Vision seamless-branch playlist
(00001, 0.4 GB across 102 tiny clips) instead of the 91.5 GB single-clip
feature (00002). A player plays the branch; a rip wants the feature.

Require a nav/authoring-promoted title to also carry at least 10% of the
largest video-bearing title's bytes; otherwise selection falls through to
the composite/size ranking, which recovers the real feature. Adds the
Spotlight-shaped regression test.

Verified against the 88-disc hoard: title selection is identical to the
prior release on every disc except the one it fixes (Spider-Man 3 UHD,
00245 wrapper decoy -> 00001 feature).

Also clears three 1.97 clippy lints in the HDMV VM and drops two
reference-implementation filename citations from the bdnav module docs.
2026-08-25 13:44:04 -07:00
Matthew Jackson 6c00090308 labels/fox: wire the extracted feature-playlist id into the selection hint 2026-08-25 13:01:25 -07:00
Matthew Jackson 7f9f5a9d3e Merge branch 'label-fox' into integrate-1.6.11 2026-08-25 13:00:06 -07:00
Matthew Jackson b3c23c19ee Merge branch 'label-deluxe' into integrate-1.6.11 2026-08-25 12:59:54 -07:00
Matthew Jackson f3923cf587 Merge branch 'dvd-nav' into integrate-1.6.11 2026-08-25 12:59:48 -07:00
Matthew Jackson 002912abb7 dvdnav: resolve DVD main feature by following First-Play navigation
Implement the DVD navigation feature resolver (#40): read the disc's own
First-Play program out of VMGI (VIDEO_TS.IFO) and execute it on a bounded
register machine to identify which title a player would land on, then prefer
that title in DVD main-feature selection.

- src/dvdnav/nav.rs: new module.
  * Parses the VMGI First-Play PGC command table (FP_PGC pointer at VMGI+0x84,
    PGC command-table pointer at PGC+0xE4, pre-command list) and TT_SRPT
    (title -> VTS/VTS_TTN, pointer at VMGI+0xC4).
  * Executes the pre-command list with the existing vmcmd decoder over 16 GPRMs
    + SPRMs from a cold machine: SetGPRM (mov/add/sub/mul/div/mod/and/or/xor),
    compare predicates, Goto, Break/Exit, and JumpTT (title dispatch). Menu /
    sub-domain jumps and intra-domain links are treated as undecidable.
  * resolve_from_vmg(&[u8]) is pure and total; resolve_main_title(reader, udf)
    reads the IFO and returns the resolved (title, vtsn, vts_ttn) or None.
  * Every offset/count is bounds-checked; a step budget bounds non-convergence.

- src/disc/dvd.rs: scan_dvd_titles now resolves the nav main title and returns
  its playlist_id alongside the titles; test call sites updated for the tuple.
- src/disc/mod.rs: the DVD nav pick feeds sort_titles_by_main_feature as
  nav_feature, the same seam the BD path uses, guarded to a video-bearing
  title. None -> today's leading-cell heuristic stands (strict improvement).
- src/harness.rs: add resolve_from_vmg to the never-panic sweep.

Tests: 11 nav unit tests (unconditional/conditional/set-then-dispatch convergence,
menu/empty/out-of-range/self-loop/truncation/overflow abstention) built to the
IFO spec using the real First-Play command bytes from the existing decoder KATs
(unconditional JumpTT 1; menu entry). Full suite green on 1.94.
2026-08-25 12:55:34 -07:00
Matthew Jackson b5fd193d54 disc: add normalized DiscProfile output surface
Add a thin, format-agnostic typed VIEW over the scanned Disc model so
downstream reads `profile.titles[i].subtitles[j].forced` (and analogous
audio/video flags) for any disc with no per-format conditionals.

New src/disc/profile.rs:
- DiscProfile / TitleProfile / VideoTrack / AudioTrack / SubtitleTrack
  (serde Serialize + Deserialize), re-exported from the disc module.
- DiscProfile::from_disc / From<&Disc>, Disc::profile(), and accessors
  (DiscProfile::main_title, TitleProfile::video/audio/subtitles).
- Splits each title's Stream enum into three typed vectors; flattens
  normalized fields (language default "und", forced/sdh from the subtitle
  qualifier, commentary/descriptive from the audio purpose, label -> name).
- Hoists derived flags so downstream never recomputes: main_title (the
  pre-sorted main feature at index 0), is_main, and the default
  audio/video track (first non-secondary, keep only the first), subtitle
  default always false.

Make the profile types the single serialization source of truth: the
json:// sink now serializes TitleProfile instead of hand-building JSON,
so json:// and any downstream share one schema. Stream arrays are split
by kind and qualifier/purpose decomposed into booleans; json://-only
extras (clips, full chapter_marks list) are spliced back on. meta_sink
tests reconciled to the new schema.

Tests: profile normalization across BdTs + MpegPs titles, typed-vector
split, populated fields, hoisted main/is_main/default flags,
qualifier/purpose mapping, and a serde round-trip.
2026-08-25 12:43:24 -07:00
Matthew Jackson 49011d577f labels/deluxe: decode real per-stream labels, emit High (Universal + others)
The Deluxe parser detected com/bydeluxe discs but emitted nothing, so
mpls_universal won at Low. Grounding the decoder on real Universal discs
(Fast Five / F9 / Bourne, studio="uni") surfaced three defects that
kept the full Phase A-D pipeline from ever resolving a stream on any
real disc, all fixed here:

1. Master enums were keyed by the ZIP ENTRY name ("pd.class") but the
   binding class references enum constants by JVM INTERNAL name
   (getstatic pd.<field>), so Phase C's getstatic count never matched
   the identified enums and found zero binding classes. Key master enums
   by this_class_name instead.
2. MasterEnumTable resolved getstatic operands by enum VALUE string
   ("English") but the operand is the obfuscated static-FIELD name
   ("a"), so every language/purpose ordinal came back unresolved.
   Capture the ordered putstatic field names (clinit_enum_field_names)
   and key the resolver on them.
3. The Language-enum fingerprint demanded 70+-4 values; Universal's
   enum has 65, so Phase A rejected it outright. Give each fingerprint
   its own count window (Language widened to span 65 and 70); the
   ordered 4+-string prefix stays the real discriminator.

Also: model anewarray/newarray so array construction keeps the symbolic
stack aligned, and skip constructors with an array parameter so the
Universal title-wrapper object (oq.<init>(...,[Lnp;,[Lwb;,...)) is not
mistaken for a subtitle binding. Recover SDH/RNIB from the language
display name (Universal encodes it as a distinct Language value, not a
Purpose). Add config.xml <TitleConfig studio="..."> detection. Raise
confidence to High when the binding table decodes >=1 stream; the
mpls_universal floor still back-fills un-named STN slots.

Fixtures: verbatim pd/lp/tl .class files from FastFive.iso decode to
English/French/Spanish + English Commentary (Dolby Digital) audio and
per-title eng/spa/fra subtitles. Validated end to end: labels-analyze on
FastFive/F9/Bourne now reports parser=deluxe High with real per-stream
languages, names, purposes and codecs instead of mpls_universal Low.
2026-08-25 12:25:38 -07:00
Matthew Jackson f8e39abe54 labels: harden class_reader/jar as first-class .class format readers
Treat Java bytecode as a documented format we READ (never execute), a
peer of mpls/clpi. The class reader's bounds checking was already sound
(typed Reader with checked u8/u16/u32/u64 + slice, saturating switch
arithmetic, iterator halt-on-truncation); this adds the untrusted-input
entry points the never-panic harness was missing and the malformed-input
regression tests the format warranted.

- harness: add class_reader (magic CAFEBABE) and jar (zip PK header)
  sweep targets. The class_reader target also drives the accessors a
  label parser exercises (this_class_name, member Code walk, cp_index,
  member_ref, load_constant_display) so their indexing is fuzzed, not
  just the structural parse. The jar target opens the sweep bytes as a
  zip and walks every .class through for_each_class exactly as deluxe/dbp
  do.
- class_reader tests: oversized constant_pool_count and interfaces_count
  reject with UnexpectedEof (u16 counts keep with_capacity bounded);
  out-of-range cp indices return None from every accessor; a Code
  attribute whose declared code_length runs past its body is rejected.
2026-08-25 12:25:19 -07:00
Matthew Jackson 65d719ee5d disc: wire bdnav as the top nav-feature tier in main-feature selection
scan_with runs the bdnav VM for BDMV discs and feeds the video-gated result
as the new top MAIN_FEATURE_ORDER key (nav-feature > authoring-feature >
standalone > has-video). A nav result is only honoured for a video-bearing
title, so a mis-resolve can never reopen the decoy hole; when the VM abstains
(BD-J, non-convergence) the structural/heuristic order decides exactly as
before. Adds nav selection tests and updates the diag decision reason.
2026-08-25 12:19:16 -07:00
Matthew Jackson c3bdbdcfc9 bdnav: faithful HDMV navigation VM (index.bdmv + MovieObject.bdmv)
Add src/bdnav: a libbluray-faithful, bounded, never-panic HDMV navigation VM
that runs First-Play with correct power-on PSR/GPR state to resolve the
playlist the disc itself plays as the feature. One generic VM handles the
obfuscated Sony dispatcher and every other HDMV disc with no vendor
special-casing; it abstains (None) on the BD-J boundary, malformed data, or
non-convergence. index/mobj parsers added to the never-panic harness.
2026-08-25 12:19:16 -07:00
Matthew Jackson 3f4e69f7b8 labels: add Fox dcx.xml vendor parser
Older Fox discs ship a loose plain-XML manifest at
/BDMV/JAR/<id>/dcx.xml (root <dcx>) that names every feature stream's
language, editorial purpose and forced/SDH state in attributes — fully
parseable and previously unowned (2 Fox discs matched no parser, 11 were
mis-owned by the deluxe stub).

New src/labels/fox.rs:
  - detect(): loose dcx.xml under /BDMV/JAR/*/ (cheap dir walk), falling
    back to a com/foxbd/ jar central-directory prefix so newer Fox discs
    are attributed to Fox even though their bytecode form isn't decoded
    yet (Phase 2, sketched in-file).
  - parse(): scopes extraction to one <playlist name="feature"> element
    (the richest, to avoid merging regional 00800/00801 STN tables),
    reusing labels/xml.rs. audio type=rnib -> Descriptive, comment* ->
    Commentary; subtitle form=sdh -> Sdh, type=embed -> Forced; nested
    id="NN" -> 1-based per-type STN slot. High confidence.
  - Feature playlist id recorded via tracing::debug for now; dev's
    ParseResult has no feature_playlist field (that FeaturePlaylistHint
    slot lives on selection-hardening). An in-code note documents wiring
    it at reconciliation, matching paramount::parse.

Registered ahead of mpls_universal, alongside the High-confidence vendor
parsers; registry-order lock test updated.

Tests embed the real Life of Pi dcx.xml (/BDMV/JAR/05001), asserting the
00800 feature selection + id, all 11 audio labels (incl. the eng rnib
descriptive track) and 11 subtitle labels (sdh/forced/none), the
no-cross-playlist-merge scope rule, plus negative-detect cases.
2026-08-25 12:09:37 -07:00
Matthew Jackson 89b03610e7 disc: demote play-all/wrapper composites in main-feature selection
The has-video gate alone fixed Spider-Man 3 UHD only by accident: the
00245 decoy = [bumper][00001 feature clip][outro], and streams come from
PlayItem 0 (the bumper) only, so it reported v=0. That is fragile — a
video-bearing composite [feature][outro] still beat the feature on size
(audit R2), and a real feature with a non-video first PlayItem would be
wrongly disqualified (R6).

Add structural clip-set composite detection: a title whose clip set
properly contains another substantial video title's whole clip set is a
play-all/wrapper composite and is demoted below the standalone title it
wraps — independent of stream counts, physical size, and disc capacity
(which is 0/unknown when READ CAPACITY fails, disabling the oversize
gate). Replace the has-video disqualification with the more-permissive
has_probable_video (real video, or size+duration+clips consistent with
video) so a bumper-lead feature is never disqualified while streamless
menus still are. Corroborate the authoring hint by duration (a hint at a
short branch/trailer can't override the feature) and require real video
streams for it. Precedence: authoring -> standalone-over-composite ->
has-video -> canonical (with a reserved top slot for the future BD-nav
key). Unify the session.rs key-sampling/verdict pickers and
image_crack_extents on has_probable_video.

Tests: video-bearing composite (T2), oversize composite with unknown
capacity (T3), short-branch hint not honoured (T4), bumper-lead feature
not disqualified (T6), all-streamless deterministic + single-source pick
(T7), equal-clip twins not composite, video-only title. T2/T3/T6 fail
under the prior has-video-only logic.
2026-08-25 11:54:04 -07:00
Matthew Jackson a7cbb3afcd disc: harden main-feature selection against streamless decoys
Main-feature selection ranked purely on physical size -> duration ->
audio (canonical_title_order), with no requirement that the chosen title
carry a decodable video stream. A playlist-obfuscation decoy can inflate
a streamless playlist past the real feature on size AND duration (Spider-
Man 3 UHD: 00245.mpls, 78.4GB/8350s/v=0 beat 00001.mpls, 76.5GB/8038s/
v=1), so the mux ran a title with zero streams -> zero frames -> E6008.

- Add Disc::main_feature_order layering two gates atop canonical_title_order:
  authoring-feature (a title the disc's menu authoring designates AND that
  has video) wins outright; then has-video demotes any streamless title
  below every title that has video. scan_with re-sorts with it after labels
  run so titles[0] is the selected feature. New MAIN_FEATURE_ORDER_KEYS +
  DiscTitle::has_video().
- Surface the authoring feature-playlist id: labels::FeaturePlaylistHint,
  captured from Paramount playlists.xml <playlist name="Feature" id=..>,
  threaded through ParseResult/extract; labels::apply now returns it.
- Unify pickers so key-sampling and the structural verdict (session.rs) and
  image_crack_extents prefer the has-video feature, not a streamless decoy.
- diag: decision row logs video= count + main_feature_order reason and a
  tag=decision.demoted line per no-video title (closes the field-diagnosis
  gap this incident exposed).

Tests: main_feature_order_demotes_streamless_decoy (pre-gate vs post-gate
contrast), _honours_authoring_hint, authoring_hint_never_promotes_a_
streamless_title, feature_playlist_hint_matches_on_id_or_filename; diag
reason test updated. QA green on 1.94 (fmt/clippy -D warnings/test).
2026-08-25 11:23:44 -07:00
Matthew Jackson f75766b4d5 chore: lower MSRV to 1.94, extend disc provenance tests, tidy comments
Lower the crate MSRV from 1.97 to 1.94 -- the lowest toolchain on which
`cargo build`, `cargo clippy --all-targets -D warnings`, and
`cargo test --tests` all pass clean. Dependencies floor the build at 1.90
(freemkv-unlock 1.90, zip 1.88), but clippy is only warning-clean from
1.94 (older clippy has a `duplicated_attributes` false positive on
`#![cfg(test)]` and flags a now-documented `let_and_return`). Update
`rust-version` and every `dtolnay/rust-toolchain` CI pin to 1.94.0, and
add a CHANGELOG Unreleased entry.

Extend the byte-provenance coverage added in f7d38ca (currently one
TS-path test) with two PS-path tests driving the real `Mpeg2Parser`:
- `ps_stream_stamps_source_provenance`: the PS path forwards `ps.source`
  onto `PesPacket`, so demuxed frames carry `Some(SourcePos)`.
- `ps_stream_carries_feed_base_across_reads`: the cumulative `fed_bytes`
  base carries ACROSS read buffers -- a frame decoded from the second
  sector is stamped `source.byte >= 2048`, not reset per read.

Also record the f7d38ca provenance fix in the CHANGELOG, inline a
`let_and_return` test binding, and trim a redundant sentence from
`PsDemuxer::feed_at`'s doc comment.
2026-08-25 10:35:47 -07:00
dependabot[bot] 1c313fe882 Bump actions/checkout from 5 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-24 08:52:57 -07:00
Pascal Harprecht f7d38caf0b mux/disc: pass feed base offset to demuxer to stamp byte provenance
On direct disc reads (`disc://...`), `DiscStream` called `demuxer.feed(&self.read_buf[..bytes])`
instead of `feed_at(buf_base, ...)`. Without a running feed base offset, demuxed `PesPacket`s
had `source = None`, stripping byte provenance from downstream `PesFrame`s.

Under a `SeamPlan`, when `src_byte` is `None`, `SeamPlan::place` falls back to a timestamp
heuristic across clips. On multi-clip playlists where a subsequent clip starts at the same
base timestamp as the initial clip (`in_time = 600s`), B-frame PTS reorder dips (where
`raw_ns < last` and `|raw_ns - next_in| <= 250ms`) falsely triggered a `stepped_back` clip
transition on the first reordered frame. Because the subsequent clip's span was short (e.g. 1s),
all subsequent video frames across the rest of the title were rejected and dropped as outside
clip marks. Audio tracks (which have monotonic PTS without reordering) were unaffected.

Track `fed_bytes` in `DiscStream` and route through `feed_at` for both TS and PS demuxers,
forwarding `ps.source` to `PesPacket` on the PS path.

Test: `mux::disc::tests::stream_surface_tests::disc_stream_stamps_source_provenance`
2026-08-24 08:52:19 -07:00
Matthew Jackson e98e98c8fd TrueHD: resync forward to next major sync on a source discontinuity
MLP/TrueHD carries predictor + restart state across access units, so after a source transport-stream discontinuity (continuity-counter gap) the parser was splicing post-gap audio mid-stream, decoding against stale state and producing a decoder-choking seam (a burst of "restart header sync incorrect" / "Invalid blocksize" errors that could flag the whole track corrupt).

Honor the source's own gap signal: drop forward to the next CRC-valid major-sync AU (a clean re-init point) before resuming, guarded so it never arms before a validated baseline exists (no stream-head whole-track drop). Mirrors the video keyframe resync and the ResyncGate design. Corrects the "audio has no inter-frame refs" comments in truehd/dts/resync. Adds a regression test.
2026-08-23 16:30:29 -07:00
Matthew Jackson 7c085db133 restore freemkv-unlock path dep for local dev (post-v1.6.9) 2026-08-22 08:34:31 -07:00
Matthew Jackson 587e26353c v1.6.9: bump version (freemkv-unlock git-pinned for the tag) 2026-08-22 08:34:26 -07:00
Matthew Jackson 3eb3e649b1 changelog: 1.6.9 unified-release entry 2026-08-22 08:24:55 -07:00
Matthew Jackson b01505983c restore freemkv-unlock path dep for local dev (post-v1.6.8) 2026-08-21 12:54:53 -07:00
Matthew Jackson 87bc07dad8 v1.6.8: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 12:54:48 -07:00
Matthew Jackson 743c5830b6 docs: 1.6.8 changelog (unified release) 2026-08-21 12:47:34 -07:00
Matthew Jackson 80c8e4ca24 ci: require a CHANGELOG entry for the crate version on main
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
2026-08-21 11:23:45 -07:00
Matthew Jackson f49e79c198 restore freemkv-unlock path dep for local dev (post-v1.6.7) 2026-08-21 09:31:40 -07:00
Matthew Jackson 231cbf3d76 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 21c7b12e2b docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 799039df21 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
162 changed files with 9808 additions and 10987 deletions
+72 -5
View File
@@ -11,6 +11,32 @@ on:
pull_request:
jobs:
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
# forgot its changelog entry must fail here rather than ship a blank release.
# Hard-fails on main (and PRs targeting main); warns elsewhere so an
# in-progress dev/qa branch whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
set -euo pipefail
version=$(grep -m1 '^version = ' Cargo.toml | sed -E 's/.*"([^"]+)".*/\1/')
if grep -qE "^## \[${version}\]" CHANGELOG.md; then
echo "CHANGELOG.md has an entry for ${version}."
exit 0
fi
msg="CHANGELOG.md has no '## [${version}]' entry"
if [ "${GITHUB_REF_NAME:-}" = "main" ] || [ "${GITHUB_BASE_REF:-}" = "main" ]; then
echo "::error::${msg} — main requires a changelog entry for the released version."
exit 1
fi
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
runs-on: ubuntu-latest
steps:
@@ -31,7 +57,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
@@ -69,7 +95,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -109,7 +135,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -149,7 +175,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -204,7 +230,10 @@ jobs:
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
# The consumers span a higher MSRV than libfreemkv itself (the bins are
# 1.98, keysources 1.97), so build them with a toolchain that satisfies the
# HIGHEST dependent floor — libfreemkv's own 1.94 cannot compile them.
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
@@ -227,3 +256,41 @@ jobs:
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
# ── Coverage (non-required, additive) ───────────────────────────────────────
#
# Reports line coverage to Codecov. Deliberately NOT wired into any release
# gate: it is informational only (see codecov.yml) and no other job `needs:`
# it. Mirrors the `test` job's layout exactly — same pinned 1.94 toolchain,
# same dual checkout so the `../freemkv-unlock` path dep resolves, same cache
# workspace and working-directory. cargo-llvm-cov comes as a prebuilt binary
# from taiki-e/install-action, so it is toolchain-independent.
#
# No `--locked`: like the fmt/clippy/test jobs above, this crate's Cargo.lock
# is gitignored, so there is nothing to lock against on a fresh runner.
coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
slug: freemkv/libfreemkv
fail_ci_if_error: false
+15
View File
@@ -0,0 +1,15 @@
name: comment-guard
# Fails if any inline `//` comment block runs longer than 3 lines. Rustdoc
# (`///`, `//!`) is exempt. Keeps explanatory comments terse; long rationale
# belongs in a doc comment or a design note, not an inline wall.
on: [push, pull_request]
jobs:
comment-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Run comment-guard
run: python3 ci/comment-guard.py
+32
View File
@@ -0,0 +1,32 @@
name: doc-links
# Fails on broken intra-doc links in `///`/`//!` doc comments (e.g. a `[Type]`
# link to a renamed/removed item). Catches doc drift that `cargo build` misses.
#
# libfreemkv path-deps ../freemkv-unlock on the branch tip, so — like ci.yml —
# both repos are checked out into sibling subdirectories for the path dep to
# resolve.
on: [push, pull_request]
jobs:
doc-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- name: cargo doc (deny broken intra-doc links)
working-directory: libfreemkv
env:
RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links"
run: cargo doc --no-deps --document-private-items --lib --bins
+3 -3
View File
@@ -79,7 +79,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -100,7 +100,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
@@ -125,7 +125,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
+2 -2
View File
@@ -29,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# gated by the local precommit (same Rust 1.94, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -38,7 +38,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
+105
View File
@@ -1,5 +1,110 @@
# Changelog
## [Unreleased]
## [1.6.13] — 2026-08-28
### Added
- Capture a few additional drive buffers during `info --share` collection on supported drives.
## [1.6.12] — 2026-08-27
### Fixed
- macOS: hold a DiskArbitration claim (plus a mount-approval dissenter) for the SCSI transport's lifetime so `diskarbitrationd` can't remount the disc mid-rip — the cause of the E1000 "disc stolen mid-rip" failure on macOS.
### Changed
- Comment and documentation cleanup.
## [1.6.11] — 2026-08-26
### Added
- Navigation-driven main-feature selection: selection now follows the disc's
own on-disc menu/navigation logic — a Blu-ray HDMV navigation VM and a DVD
First-Play resolver — to reach the main feature, instead of guessing by
title size alone.
- Clip-set composite/decoy detection, so a title assembled from a composite
or decoy clip set is recognized as such during selection.
- A feature-payload floor, giving main-feature selection a minimum-size
sanity check.
- Normalized `DiscProfile` output.
- Vendor navigation-label parsing.
- Substantially expanded unit-test coverage across the navigation and
clip-selection paths.
### Fixed
- Main-feature selection no longer picks an obfuscated decoy playlist over
the real feature.
- Two rounds of code-audit hardening: navigation VM operand/field-offset
conformance, navigation now abstains rather than guessing on an
undecidable system-parameter branch, panic-safety fixes, log-escaping
fixes, and expanded fuzz/test coverage.
- Direct disc reads (`disc://…`): `DiscStream` now threads a cumulative feed
base offset into the demuxer (`feed_at`) for both the TS and PS paths and
forwards it onto each `PesPacket`, so demuxed frames carry byte-exact source
provenance (`SourcePos`). Without it, `SeamPlan::place` fell back to a PTS
heuristic that a B-frame PTS reorder dip could misread as a `stepped_back`
clip transition, dropping all subsequent video frames in multi-clip playlists.
### Changed
- Lowered the minimum supported Rust version (MSRV) from 1.97 to 1.94. This is
the lowest toolchain on which `cargo build`, `cargo clippy --all-targets
-D warnings`, and `cargo test --tests` all pass clean; CI toolchain pins were
moved to match. (The crate's dependencies floor `cargo build` at 1.90, but
clippy is only warning-clean from 1.94.)
## [1.6.10] — 2026-08-23
### Fixed
- TrueHD/MLP audio: on a source transport-stream discontinuity (lost or gapped
packets, signalled by a continuity-counter break), the parser now drops forward
to the next major-sync access unit before resuming, instead of splicing the
post-gap audio mid-stream. MLP carries predictor + restart state across access
units, so resuming mid-stream decoded against stale state and produced a
decoder-choking seam — streaming decoders emitted a burst of "restart header
sync incorrect" / "Invalid blocksize" errors and could flag the whole track
corrupt. This mirrors the video path (resync to the next keyframe after a gap)
and how the independent-frame audio codecs (AC-3, DTS, FLAC, AAC) already
recover; the fix is guarded so it never arms before a validated major-sync
baseline exists (no whole-track drop at stream head). The gapped instant itself
is in the source and cannot be recovered, but the muxed track is now
decoder-clean.
## [1.6.9] — 2026-08-22
### Changed
- Version aligned to 1.6.9 for the unified release. No functional changes to
this crate; the release was driven by autorip (automatic per-episode TV
ripping — each episode named `S{NN}E{MM}`, with TMDB runtime-aligned episode
numbering across multi-disc seasons — a Manual Rename option, and a unified
per-disc staging state file — see the autorip 1.6.9 notes).
## [1.6.8] — 2026-08-21
### Changed
- Version aligned to 1.6.8 for the unified release. No functional changes to
this crate; the release was driven by autorip (webhooks now fire per pipeline
stage — Rip / Mux / Move — with the Rip hook firing the moment the drive is
free again, plus a Ripper-tab activity-banner fix so it also shows during
moves — see the autorip 1.6.8 notes).
## [1.6.7] — 2026-08-21
### Changed
- Version aligned to 1.6.7 for the unified release. No functional changes to
this crate; the release was driven by autorip (per-webhook event selection,
a progress bar per moved artifact, and move-queue / webhook-error fixes —
see the autorip 1.6.7 notes).
## [1.6.6] — 2026-08-20
### Changed
+3 -3
View File
@@ -1,8 +1,8 @@
[package]
name = "libfreemkv"
version = "1.6.6"
version = "1.6.13"
edition = "2024"
rust-version = "1.97"
rust-version = "1.94"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@@ -26,7 +26,7 @@ aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.5" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.13" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
+2
View File
@@ -1,4 +1,6 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/libfreemkv/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/libfreemkv)
# libfreemkv
+1
View File
@@ -5,6 +5,7 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
println!("cargo:rustc-link-lib=framework=DiskArbitration");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Fail if any inline `//` comment block runs longer than MAX_LINES.
Rustdoc (`///`, `//!`) is exempt — long API docs are fine. Only consecutive
non-doc `//` lines are counted; a blank line, code, or a doc line ends the run.
Block comments (`/* */`) are not inspected. Scans src/ and tests/ under the cwd.
"""
import sys
from pathlib import Path
MAX_LINES = 3
def is_inline_comment(line: str) -> bool:
s = line.lstrip()
return s.startswith("//") and not s.startswith("///") and not s.startswith("//!")
def violations(path: Path):
"""Yield (start_line, length) for each inline run longer than MAX_LINES."""
out = []
run_start = None
run_len = 0
for i, line in enumerate(path.read_text(encoding="utf-8", errors="replace").splitlines(), 1):
if is_inline_comment(line):
if run_start is None:
run_start = i
run_len += 1
else:
if run_len > MAX_LINES:
out.append((run_start, run_len))
run_start, run_len = None, 0
if run_len > MAX_LINES:
out.append((run_start, run_len))
return out
def main(argv):
root = Path(argv[1]) if len(argv) > 1 else Path.cwd()
files = sorted(p for d in ("src", "tests") for p in (root / d).rglob("*.rs"))
total = 0
for f in files:
for start, length in violations(f):
rel = f.relative_to(root)
print(f"{rel}:{start}: inline comment block is {length} lines (max {MAX_LINES})")
total += 1
if total:
print(f"\ncomment-guard: {total} inline comment block(s) exceed {MAX_LINES} lines.")
print("Trim to the essential why, or split the code so the comment isn't needed.")
return 1
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+18
View File
@@ -0,0 +1,18 @@
# Coverage is informational only: it never fails CI or blocks a merge/release.
# The `coverage` job in .github/workflows/ci.yml is non-required and no gate
# depends on it.
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
comment: false
ignore:
- "tests"
- "examples"
- "benches"
+67 -134
View File
@@ -14,7 +14,7 @@ use super::crypto::AACS_IV;
// ── AACS constants ──────────────────────────────────────────────────────────
/// Size of an AACS aligned unit (3 × 2048-byte sectors). [BD] §3.10.1.
/// Size of an AACS aligned unit (3 × 2048-byte sectors). `[BD]` §3.10.1.
pub const ALIGNED_UNIT_LEN: usize = 6144;
/// An AACS aligned unit spans this many 2048-byte sectors (3).
@@ -82,7 +82,7 @@ const PS_SCRAMBLE_MASK: u8 = 0x30;
///
/// - `BdTs` (BD / UHD / FMTS Transport Stream): the Copy Permission Indicator in
/// the top 2 bits of byte 0 — the first `TP_extra_header` byte, always left
/// clear (the first 16 bytes of every unit are the unencrypted SEED). [BD]
/// clear (the first 16 bytes of every unit are the unencrypted SEED). `[BD]`
/// §3.10.2. `(buf[0] & 0xC0) == 0` → clear; non-zero → bytes `16..6144` are
/// AES-CBC encrypted.
/// - `MpegPs` (HD-DVD `.evo`): the MPEG-2 `PES_scrambling_control` flag — see
@@ -99,10 +99,9 @@ pub fn aacs_unit_encrypted(unit: &[u8], format: crate::disc::ContentFormat) -> b
}
match format {
ContentFormat::BdTs => (unit[0] & 0xC0) != 0,
// UNVERIFIED-HDDVD-DECRYPT (1 of 2): the PES_scrambling_control location
// for HD-DVD `.evo` is derived from spec, never confirmed against a real
// ENCRYPTED HD DVD (we only have decrypted rips). If HD-DVD ripping ever
// misbehaves, verify this flag byte/mask against a genuine encrypted unit.
// UNVERIFIED-HDDVD-DECRYPT (1 of 2): PES_scrambling_control location for
// HD-DVD `.evo` is spec-derived, never confirmed against a real encrypted
// disc (only decrypted rips available). Verify against a real one if ripping misbehaves.
ContentFormat::MpegPs => (unit[PS_SCRAMBLE_OFF] & PS_SCRAMBLE_MASK) != 0,
}
}
@@ -137,12 +136,9 @@ pub fn aacs_unit_seed_encrypted(unit: &[u8], format: crate::disc::ContentFormat)
///
/// Like the flag itself this is only meaningful at the clip-FILE-anchored boundary.
pub fn aacs_unit_needs_decrypt(unit: &[u8], format: crate::disc::ContentFormat) -> bool {
// "Still needs the key applied" = flagged encrypted AND not yet structurally
// clean. There is ONE definition of clean — [`is_clean`] (the min(E,4) proof
// floor: E>4 needs any 4 synced, E<=4 needs all present). Never a second
// threshold: the old >50% majority false-flagged a bad-encoded-but-OPENED
// unit as still-scrambled, so the mux re-sampled it to the key service every
// batch (the storm) and could re-apply the key over already-clear bytes.
// "Still needs key" = flagged encrypted AND not structurally clean per the
// ONE definition, [`is_clean`]'s min(E,4) proof floor. Never a second threshold:
// the old >50% majority false-flagged bad-but-opened units, causing re-sample storms.
aacs_unit_encrypted(unit, format) && !is_clean(unit, format)
}
@@ -258,7 +254,7 @@ pub fn ts_packet_total(unit: &[u8]) -> usize {
/// the freebie `is_clean_ts` skips at packet 0); packs 1 and 2 (offsets 2048 and
/// 4096) are in the encrypted region, so a wrong key garbles them and this returns
/// false (64 bits of discrimination). Validated against real decrypted HD-DVD
/// `.evo` (ANCHORMAN / SHAUN_OF_THE_DEAD): pack starts are exactly 2048-aligned,
/// `.evo` (two real retail titles): pack starts are exactly 2048-aligned,
/// three per aligned unit, at offsets 0 / 2048 / 4096.
///
/// UNVERIFIED-HDDVD-DECRYPT (2 of 2): the pack STRUCTURE here is confirmed on
@@ -271,11 +267,9 @@ fn is_clean_ps(unit: &[u8]) -> bool {
return false;
}
const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
// Skip pack 0 (offset 0): its start bytes lie in the clear 16-byte seed —
// present for ANY key (and byte 0 may carry the Blu-ray-style CPI bits), so
// it proves nothing about decryption. This mirrors `is_clean_ts` skipping
// packet 0. Packs 1 and 2 (offsets 2048 / 4096) are in the encrypted region
// and are what discriminate the key (64 bits of proof).
// Skip pack 0: its start bytes lie in the clear 16-byte seed, present for ANY
// key, so it proves nothing (mirrors `is_clean_ts` skipping packet 0). Packs 1
// and 2 (offsets 2048/4096) are in the encrypted region and discriminate the key.
let mut o = SECTOR_BYTES;
while o + 4 <= ALIGNED_UNIT_LEN {
if unit[o..o + 4] != PACK_START {
@@ -302,7 +296,7 @@ fn is_clean_ps(unit: &[u8]) -> bool {
/// to "decrypt applies the key and nothing else". The only guard kept here is the
/// length check, since the crypto is defined only over a whole 6144-byte unit.
///
/// Block Key = AES-128E(Kcu, seed) ⊕ seed ([BD] §3.10.1 Fig 3-8: encrypt the clear
/// Block Key = AES-128E(Kcu, seed) ⊕ seed (`[BD]` §3.10.1 Fig 3-8: encrypt the clear
/// 16-byte seed under the CPS Unit Key, XOR the seed back in — the trailing ⊕seed
/// is load-bearing); then AES-128-CBC decrypt bytes 16..6144 under the AACS IV.
/// Source-zero padding packets (all 192 bytes zero on disc) are restored to zero:
@@ -395,12 +389,9 @@ pub fn encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) -> bool {
/// Remove bus encryption from an aligned unit (AACS 2.0 / UHD).
/// Bus encryption uses read_data_key, decrypting bytes 16..2048 of each 2048-byte sector.
pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
// Expand the key schedule ONCE for the whole unit. `read_data_key` is
// loop-invariant here (and constant for the entire disc), but calling
// `aes_cbc_decrypt` per sector rebuilt the AES-128 schedule per sector — three
// expansions per 6144-byte aligned unit, i.e. ~29 million redundant expansions
// over a 90 GB read on a stock drive, on the per-unit decrypt hot path.
// Measured by `decrypt_bus_expands_the_read_data_key_once_per_unit`.
// Expand the key schedule ONCE for the whole unit: `read_data_key` is
// loop-invariant, but per-sector `aes_cbc_decrypt` rebuilt it 3x per unit
// (~29M redundant expansions over a 90GB read) on the decrypt hot path.
let cipher = crate::aacs::crypto::new_cipher_for(read_data_key);
for sector_start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
if sector_start + SECTOR_BYTES > unit.len() {
@@ -457,11 +448,6 @@ mod tests {
assert_eq!(unit, clear, "round trip must be byte-exact");
}
/// The documented padding asymmetry actually holds: `decrypt_unit` restores
/// all-zero-ON-DISC packets to zero, but an all-zero PLAINTEXT packet enciphers
/// to non-zero bytes, so it is not mistaken for padding and still round-trips.
/// This is the one place the two functions are deliberately not symmetric, so
/// the claim is worth pinning rather than asserting in prose alone.
/// A slice too short to encrypt must SAY so. The caller has already set the
/// container's encrypted flag by the time it calls this (the contract requires
/// flag-before-crypto, since the header is the key seed), so a silent no-op
@@ -487,6 +473,11 @@ mod tests {
assert!(encrypt_unit(&mut exact, &key), "a full unit must encrypt");
}
/// The documented padding asymmetry actually holds: `decrypt_unit` restores
/// all-zero-ON-DISC packets to zero, but an all-zero PLAINTEXT packet enciphers
/// to non-zero bytes, so it is not mistaken for padding and still round-trips.
/// This is the one place the two functions are deliberately not symmetric, so
/// the claim is worth pinning rather than asserting in prose alone.
#[test]
fn encrypt_unit_round_trips_all_zero_plaintext_packets() {
let key = [0xA5u8; 16];
@@ -546,11 +537,9 @@ mod tests {
#[test]
fn is_unit_aligned_lba_below_base_is_well_defined() {
// Latent-trap contract (rc.5.2 audit #5): a read never starts before its
// extent base, but if `lba < unit_base` the result must be well-defined,
// NOT the `wrapping_sub` underflow that — because 2^32 ≡ 1 (mod 3) —
// would falsely report alignment. `saturating_sub` clamps to offset 0,
// which is a unit boundary, so any `lba <= unit_base` reads as aligned.
// Latent-trap contract (rc.5.2 audit #5): if `lba < unit_base` the result must
// be well-defined, NOT `wrapping_sub` underflow (2^32 ≡ 1 mod 3 falsely
// reports alignment). `saturating_sub` clamps to 0, a unit boundary.
assert!(
is_unit_aligned(99, 100),
"lba just below base must not wrap"
@@ -569,11 +558,9 @@ mod tests {
#[test]
fn clear_unit_is_not_flagged_encrypted() {
// `decrypt_unit` is now PURE (applies the key unconditionally); the
// "leave a clear unit untouched" policy lives at the caller's gate,
// `aacs_unit_encrypted` / `aacs_unit_needs_decrypt`. A clear TS unit
// (byte-0 CPI bits clear, syncs intact) must report NOT-encrypted so the
// caller never hands it to decrypt_unit.
// `decrypt_unit` is PURE (applies the key unconditionally); "leave a clear
// unit untouched" lives at the caller's gate. A clear TS unit (CPI bits
// clear, syncs intact) must report NOT-encrypted so the caller skips decrypt_unit.
let ts = crate::disc::ContentFormat::BdTs;
let mut unit = vec![0u8; ALIGNED_UNIT_LEN];
let mut off = 4;
@@ -597,10 +584,8 @@ mod tests {
#[test]
fn ts_packet_total_no_off_by_one() {
// The maximum sync count is exactly the number of stride
// positions the counting loop visits (offset 4, 196, ...), i.e.
// len / 192, NOT (len - 4) / 192 + 1. For the 6144-byte aligned unit
// the loop checks offsets 4..=5956 → 32 positions.
// Max sync count = stride positions visited (offset 4, 196, ...) = len/192,
// NOT (len-4)/192+1. For the 6144-byte unit, offsets 4..=5956 → 32 positions.
let unit = vec![0u8; ALIGNED_UNIT_LEN];
assert_eq!(ts_packet_total(&unit), 32);
// Confirm the loop visits exactly that many stride positions.
@@ -612,10 +597,9 @@ mod tests {
#[test]
fn is_clean_min4_proof_floor() {
// ONE rule: a unit is clean iff `synced >= min(E, 4)` over the ENCRYPTED
// (non-padding) packets — E>4 needs any 4, E<=4 needs all present. Build
// NON-ZERO payloads (real content, not padding) so every packet counts
// toward E; place `n` TS syncs among packets 1..31 (packet 0 is skipped).
// ONE rule: clean iff `synced >= min(E, 4)` over ENCRYPTED (non-padding)
// packets — E>4 needs any 4, E<=4 needs all present. Build NON-ZERO payloads
// so every packet counts toward E; place syncs among packets 1..31 (0 skipped).
let unit_with = |synced: usize| {
let mut unit: Vec<u8> = (0..ALIGNED_UNIT_LEN)
.map(|i| ((i * 7 + 1) as u8) | 1)
@@ -686,11 +670,9 @@ mod tests {
let original = vec![0x42u8; 128]; // 8 blocks
let mut data = original.clone();
// Encrypt with the REAL production primitive. This test previously
// defined a local `fn aes_cbc_encrypt` that SHADOWED it, so it round-
// tripped a copy of the algorithm against itself and never exercised
// `crypto::aes_cbc_encrypt` at all — a mutation to the shipped function
// could not fail it.
// Encrypt with the REAL production primitive. This test previously defined a
// local `fn aes_cbc_encrypt` that SHADOWED it, round-tripping a copy of the
// algorithm against itself — a mutation to the shipped function couldn't fail it.
aes_cbc_encrypt(&key, &mut data);
assert_ne!(data, original); // should be different after encrypt
@@ -797,14 +779,8 @@ mod tests {
}
// ── Padding-aware IsDecryptable (fragment-tail recovery) ───────────────
//
// A content fragment can end mid-unit, with the disc zero-padding the rest
// of the aligned unit to the next fragment (proven on Dunkirk: ~11 real
// video packets + source-zero pad). `decrypt_unit` must accept such a unit
// — its real packets decrypt; the source-zero tail is padding, not content
// — while still REJECTING a unit whose undecryptable tail is non-zero (a
// genuine misread / wrong key). The discriminator is the SOURCE bytes of the
// failing packets: zero ⇒ padding (still decryptable), non-zero ⇒ bad data.
// A fragment can end mid-unit, zero-padded to the next fragment. `decrypt_unit`
// must accept a zero tail (padding) but REJECT a non-zero tail (misread/wrong key).
/// Encrypt a full clear unit under `unit_key`, then overwrite the tail (from
/// packet `keep` onward) with `fill`. `0x00` models disc fragment padding; a
@@ -833,7 +809,7 @@ mod tests {
#[test]
fn fragment_tail_with_source_zero_pad_is_decryptable() {
// 11 real content packets, then source-zero padding (the Dunkirk shape).
// 11 real content packets, then source-zero padding (a real-disc fragment shape).
let key = [0x5Au8; 16];
let mut unit = tail_filled_unit(&key, 11, 0x00);
decrypt_unit(&mut unit, &key);
@@ -859,10 +835,8 @@ mod tests {
#[test]
fn fragment_tail_with_nonzero_garbage_decrypts_the_real_prefix() {
// Same shape, but the tail is NON-zero garbage. The crypto still recovers
// the 11 real packets; the garbage tail is whatever it is (the muxer drops
// it on sync-loss, and a genuine bad sector is caught by the physical read
// layer / mapfile — never by TS structure). Ground truth: the 11 real
// packets came back.
// the 11 real packets; the muxer drops the garbage tail on sync-loss (a
// genuine bad sector is caught by the read layer/mapfile, never TS structure).
let key = [0x5Au8; 16];
let mut unit = tail_filled_unit(&key, 11, 0xC3);
decrypt_unit(&mut unit, &key);
@@ -876,16 +850,8 @@ mod tests {
}
// ── Defect-tolerant "did a key OPEN this unit?" verdict ─────────────────
//
// The authored-bad-packet bug: a commercial disc carries the odd bad TS
// packet (a pressing/encoding defect, or an AACS 2.1 forensic-variant frame)
// — one non-conforming packet inside an otherwise perfectly-decrypted 6144
// unit. The OLD strict per-packet acceptance rejected the WHOLE unit over
// that single packet, so the mux concealed ~all of it as NULL and 21/22 good
// video packets were destroyed and tallied as loss ("false corruption").
// The read/decrypt path must instead ACCEPT the unit (the key opened it) and
// pass the defective packet through VERBATIM for the muxer to drop. TS-sync
// conformance is a muxer concern, never a decryption verdict.
// Authored-bad-packet bug: one non-conforming packet made OLD strict acceptance
// reject the whole unit, destroying good video. Must ACCEPT and pass defects verbatim.
/// Build a unit that decrypts to 32 content packets, with `defect_pkts`
/// marked authored-bad (a non-`0x47` at the sync position + non-zero payload
@@ -998,11 +964,9 @@ mod tests {
#[test]
fn is_clean_dispatches_by_container_format() {
use crate::disc::ContentFormat;
// Clean HD-DVD `.evo` Program-Stream unit: pack_start_code `00 00 01 BA`
// at each 2048-byte pack boundary (0/2048/4096) — the layout validated
// against real decrypted ANCHORMAN / SHAUN_OF_THE_DEAD `.evo`. Offset 0 is
// the clear-seed freebie; the packs at 2048/4096 are encrypted and are
// what actually discriminate a key.
// Clean HD-DVD `.evo` Program-Stream unit: pack_start_code `00 00 01 BA` at
// each 2048-byte pack boundary (0/2048/4096), validated against real retail rips.
// Offset 0 is the clear-seed freebie; packs at 2048/4096 are what discriminate a key.
let mut ps = vec![0u8; ALIGNED_UNIT_LEN];
for off in [0usize, 2048, 4096] {
ps[off..off + 4].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
@@ -1085,10 +1049,9 @@ mod tests {
#[test]
fn sparse_tail_needs_all_present_packets_min_e_4() {
// End-of-clip fragment tail: `min(E,4)` scales to the packets that exist,
// so a sparse unit needs ALL of its (few) encrypted packets — a wrong key
// gives 0, so this is a strong proof with no wrong-key hole, and a valid
// 1-packet tail is never false-rejected.
// End-of-clip fragment tail: `min(E,4)` scales to existing packets, so a
// sparse unit needs ALL its (few) encrypted packets. Wrong key gives 0 (no
// hole), and a valid 1-packet tail is never false-rejected.
assert!(
is_clean_ts(&decrypted_shape(1, 1, false)),
"E=1: the one packet syncs -> open"
@@ -1182,12 +1145,9 @@ mod tests {
#[test]
fn aes_ecb_matches_fips197_known_answer() {
// FIPS-197 Appendix C.1 AES-128 KAT:
// key = 000102030405060708090a0b0c0d0e0f
// plaintext = 00112233445566778899aabbccddeeff
// ciphertext= 69c4e0d86a7b0430d8cdb78070b4c55a
// This pins the AES primitive against a published vector — a wrong
// cipher (or a key/plaintext byte-order slip) fails it.
// FIPS-197 Appendix C.1 AES-128 KAT (key/plaintext/ciphertext = code arrays
// below: key 000102..0f, pt 001122..ff, ct 69c4e0..5a).
// Pins the AES primitive against a published vector.
let key = [
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D,
0x0E, 0x0F,
@@ -1211,7 +1171,7 @@ mod tests {
/// 2048-byte sector, and each call built its own AES-128 key schedule, so a
/// 6144-byte aligned unit performed THREE key expansions under the same
/// loop-invariant `read_data_key`. On a 90 GB UHD read on a stock (non-
/// LibreDrive) drive — ~14.6 million aligned units — that is ~29 million
/// firmware-unlocked) drive — ~14.6 million aligned units — that is ~29 million
/// redundant expansions on the per-unit decrypt hot path, for a key that is
/// constant for the whole disc. The counter is incremented inside
/// `crypto::new_cipher`, the single construction site.
@@ -1231,7 +1191,7 @@ mod tests {
}
/// The single-expansion refactor must be byte-identical: bus encryption
/// ([C] §4.2) covers bytes 16..2048 of every 2048-byte sector, so a
/// (`[C]` §4.2) covers bytes 16..2048 of every 2048-byte sector, so a
/// three-sector aligned unit round-trips through the forward direction
/// sector by sector and `decrypt_bus` must recover it exactly.
#[test]
@@ -1260,7 +1220,7 @@ mod tests {
/// The published `iv0` bytes, INDEPENDENT of the production constant.
///
/// [C] §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation.
/// `[C]` §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation.
/// Both IV tests below used to compute their expected value from
/// `crypto::AACS_IV` itself, so the constant was asserted against itself and
/// NOTHING in the suite pinned its bytes: swapping `AACS_IV` for `[0u8; 16]`
@@ -1274,7 +1234,7 @@ mod tests {
0x78,
];
/// Pins the fixed AACS CBC IV ([C] §2.1.2 `iv0`) against a literal, so a
/// Pins the fixed AACS CBC IV (`[C]` §2.1.2 `iv0`) against a literal, so a
/// change to `crypto::AACS_IV` fails HERE rather than silently shipping.
#[test]
fn aacs_iv_matches_published_iv0() {
@@ -1286,13 +1246,9 @@ mod tests {
#[test]
fn cbc_decrypt_first_block_xors_aacs_iv() {
// CBC: P[0] = AES-D(K, C[0]) XOR IV, and the IV is the fixed AACS
// constant (not zero). Encrypt a single block forward with the PUBLISHED
// iv0 literal, then confirm aes_cbc_decrypt recovers it — proving the IV
// the production code uses on block 0 is exactly that value. Building the
// fixture from `IV0_PUBLISHED` rather than from `AACS_IV` is what makes
// the claim real: a mutation that swaps AACS_IV for [0u8;16] now makes the
// recovered block wrong.
// CBC: P[0] = AES-D(K, C[0]) XOR IV, IV being the fixed AACS constant. The
// fixture is built from the PUBLISHED iv0 literal (not `AACS_IV`), so swapping
// AACS_IV for [0u8;16] makes the recovered block wrong — proving block 0's IV.
let key = [0x24u8; 16];
let plain = [0x5Au8; 16];
// Forward CBC for one block: C = AES-E(K, P XOR IV).
@@ -1310,30 +1266,9 @@ mod tests {
#[test]
fn aes_cbc_decrypt_matches_nist_sp800_38a_f2_2() {
// NIST SP 800-38A Appendix F.2.2 (CBC-AES128.Decrypt) published vector:
// Key = 2b7e151628aed2a6abf7158809cf4f3c
// IV = 000102030405060708090a0b0c0d0e0f
// CT = 7649abac8119b246cee98e9b12e9197d (block 0)
// 5086cb9b507219ee95db113a917678b2 (block 1)
// 73bed6b8e3c1743b7116e69e22229516 (block 2)
// 3ff1caa1681fac09120eca307586e1a7 (block 3)
// PT = 6bc1bee22e409f96e93d7e117393172a (block 0)
// ae2d8a571e03ac9c9eb76fac45af8e51 (block 1)
// 30c81c46a35ce411e5fbc1191a0a52ef (block 2)
// f69f2445df4f9b17ad2b417be66c3710 (block 3)
//
// `aes_cbc_decrypt` hardwires the fixed AACS IV for block 0 (it never
// takes a caller IV), so:
// * Blocks 1..=3 are independent of the IV — they MUST equal the NIST
// plaintext byte-for-byte (P[i] = AES-D(K, C[i]) XOR C[i-1]). This
// pins the real reverse-order CBC chaining against a published KAT.
// * Block 0 = AES-D(K, C[0]) XOR iv0 = NIST_PT[0] XOR NIST_IV XOR iv0 —
// the documented IV substitution. Asserting this exact relation pins
// both the AES decrypt of C[0] AND that block 0 uses iv0. The expected
// value is built from the `IV0_PUBLISHED` literal, NOT from
// `crypto::AACS_IV`: computing it from the production constant made
// the change cancel out of the triple XOR, so a swap to [0u8;16] still
// passed. It now fails.
// NIST SP 800-38A F.2.2 (CBC-AES128.Decrypt) KAT (vectors = code arrays below).
// `aes_cbc_decrypt` hardwires iv0 for block 0: blocks 1..=3 pin reverse-order
// chaining; block 0 = NIST_PT[0] XOR NIST_IV XOR the `IV0_PUBLISHED` literal.
let key = [
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
@@ -1457,9 +1392,8 @@ mod tests {
#[test]
fn cpi_gate_set_flag_decrypts_and_needs_decrypt_is_idempotent() {
// A CPI-set encrypted unit decrypts with the right key. CPI lives in the
// plaintext header, so it survives decryption — `aacs_unit_encrypted`
// still reports true afterward, but `aacs_unit_needs_decrypt` flips to
// false (syncs restored), keeping the re-decrypt paths idempotent.
// plaintext header, so `aacs_unit_encrypted` stays true after decryption, but
// `aacs_unit_needs_decrypt` flips false (syncs restored) — re-decrypt is idempotent.
let ts = crate::disc::ContentFormat::BdTs;
let key = [0x5au8; 16];
let mut unit = clear_unit();
@@ -1485,10 +1419,9 @@ mod tests {
#[test]
fn decrypt_bus_roundtrips_per_sector_skipping_first_16_bytes() {
// Bus encryption CBC-encrypts bytes 16..2048 of EACH 2048-byte sector
// (3 sectors per aligned unit), leaving the first 16 plaintext. Build
// the forward transform, then confirm decrypt_bus inverts it and
// leaves each sector's first 16 bytes untouched.
// Bus encryption CBC-encrypts bytes 16..2048 of EACH 2048-byte sector (3 per
// aligned unit), leaving the first 16 plaintext. Confirm decrypt_bus inverts the
// forward transform and leaves each sector's first 16 bytes untouched.
let rdk = [0x13u8; 16];
let mut unit = vec![0u8; ALIGNED_UNIT_LEN];
// Fill with a recognisable pattern.
@@ -1573,7 +1506,7 @@ mod tests {
/// `aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the
/// guard that stops a truncated encrypted fragment from being emitted as
/// clear content. It reads ONLY the two Copy Permission Indicator bits
/// ([BD] §3.10.2, byte 0 bits 6-7); the remaining six bits are
/// (`[BD]` §3.10.2, byte 0 bits 6-7); the remaining six bits are
/// `TP_extra_header` arrival-timestamp bits and carry no encryption
/// meaning.
///
+17 -22
View File
@@ -1,4 +1,4 @@
//! AACS common cryptographic primitives — [C] Chapter 2 / §3.2.2.
//! AACS common cryptographic primitives — `[C]` Chapter 2 / §3.2.2.
//!
//! Source: `[C]` = AACS Introduction and Common Cryptographic Elements Book,
//! Rev 0.953. The shared low-level building blocks — AES-128 ECB E/D, AES-G,
@@ -10,19 +10,14 @@
use aes::Aes128;
use aes::cipher::{Array, BlockCipherDecrypt, BlockCipherEncrypt, KeyInit};
/// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`).
/// Fixed IV used by AACS for all AES-CBC operations. `[C]` §2.1.2 (default CBC IV, `iv0`).
pub(crate) const AACS_IV: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78,
];
// Per-thread count of AES-128 key schedules built through `new_cipher`.
// Test-only instrumentation: an AES-128 key expansion is 10 round-key
// derivations, and the CBC helpers here run on the per-aligned-unit decrypt hot
// path of a whole disc read, so "how many times was the schedule built for one
// loop-invariant key" is a property worth asserting rather than reasoning about.
// THREAD-LOCAL, not a global atomic: `cargo test` runs tests concurrently, so a
// shared counter would see every other test's expansions. See
// `content::tests::decrypt_bus_expands_the_read_data_key_once_per_unit`.
// Test-only: lets tests assert the CBC decrypt hot path builds one schedule
// per loop-invariant key. Thread-local, not atomic, since `cargo test` runs concurrently.
#[cfg(test)]
thread_local! {
pub(crate) static KEY_EXPANSIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
@@ -35,14 +30,14 @@ pub(crate) fn new_cipher_for(key: &[u8; 16]) -> Aes128 {
}
/// Build an AES-128 key schedule. The single construction site for the CBC
/// helpers, so [`KEY_EXPANSIONS`] can count them under test.
/// helpers, so `KEY_EXPANSIONS` can count them under test.
fn new_cipher(key: &[u8; 16]) -> Aes128 {
#[cfg(test)]
KEY_EXPANSIONS.with(|c| c.set(c.get() + 1));
Aes128::new(&(*key).into())
}
/// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`).
/// AES-128-ECB encrypt a single 16-byte block. `[C]` §2.1.1 (`AES-128E`).
pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
@@ -52,7 +47,7 @@ pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
out
}
/// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`).
/// AES-128-ECB decrypt a single 16-byte block. `[C]` §2.1.1 (`AES-128D`).
pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
@@ -63,7 +58,7 @@ pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
}
/// AES-128-CBC ENCRYPT in place under the fixed [`AACS_IV`] — the forward
/// direction of [`aes_cbc_decrypt`], and its exact inverse. [C] §2.1.2
/// direction of [`aes_cbc_decrypt`], and its exact inverse. `[C]` §2.1.2
/// (`AES-128CBCE`).
///
/// Precondition: `data.len()` is a multiple of 16; the assert
@@ -95,7 +90,7 @@ pub(crate) fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
}
}
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. [C] §2.1.2
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. `[C]` §2.1.2
/// (`AES-128CBCD`).
///
/// Precondition: `data.len()` is a multiple of 16. Any trailing partial
@@ -124,7 +119,7 @@ pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
/// Split out of [`aes_cbc_decrypt`] so a caller that decrypts several regions
/// under one loop-invariant key expands the schedule once. `decrypt_bus`
/// ([`super::content::decrypt_bus`]) is that caller: bus encryption
/// ([C] §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// (`[C]` §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// 2048-byte sector, so a 6144-byte aligned unit is three regions under one
/// `read_data_key` — three key schedules where one suffices, on the per-unit
/// decrypt hot path of a whole 90 GB read.
@@ -150,7 +145,7 @@ pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
}
}
/// AES-G(x1, x2) = AES-128D(x1, x2) XOR x2. [C] §2.1.3 (note: uses AES-128**D**).
/// AES-G(x1, x2) = AES-128D(x1, x2) XOR x2. `[C]` §2.1.3 (note: uses AES-128**D**).
///
/// The Media Key Variant chain uses AES-G to derive both the variant
/// number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
@@ -165,14 +160,14 @@ pub(crate) fn aes_g(x1: &[u8; 16], x2: &[u8; 16]) -> [u8; 16] {
out
}
/// AACS-G3 seed constant (`s0`). [C] §3.2.2.
/// AACS-G3 seed constant (`s0`). `[C]` §3.2.2.
pub(crate) const AESG3_SEED: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B, 0xD9,
];
/// AACS-G3: derive a subkey from a parent key. [C] §3.2.2 (Triple AES Generator:
/// AACS-G3: derive a subkey from a parent key. `[C]` §3.2.2 (Triple AES Generator:
/// left=`D(k,s0)⊕s0` inc 0, pk=`D(k,s0+1)⊕(s0+1)` inc 1, right=`D(k,s0+2)⊕(s0+2)` inc 2).
/// seed[15] += inc, then AES-DEC(key, seed) XOR seed.
/// `seed[15] += inc`, then AES-DEC(key, seed) XOR seed.
///
/// Shared with [`super::variant`] (its variant chain runs the same SD
/// tree); a single definition keeps the two walks byte-identical.
@@ -190,7 +185,7 @@ pub(crate) fn aesg3(key: &[u8; 16], inc: u8) -> [u8; 16] {
mod tests {
use super::*;
/// The AACS-G3 seed `s0`, transcribed independently from [C] §3.2.2 rather
/// The AACS-G3 seed `s0`, transcribed independently from `[C]` §3.2.2 rather
/// than read from [`AESG3_SEED`] — a test that sourced the seed from the
/// production constant would assert that constant against itself and would
/// still pass if it were edited.
@@ -216,7 +211,7 @@ mod tests {
/// finding Media Keys, with no error to say why.
///
/// Pinned through the spec relation rather than a re-implementation:
/// [C] §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `[C]` §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `s = s0 + inc` (added into the last seed byte), so applying the
/// FORWARD primitive [`aes_ecb_encrypt`] — a different function from the
/// one under test — to `aesg3(k, inc) XOR s` must reproduce `s` exactly.
@@ -242,7 +237,7 @@ mod tests {
}
}
/// The Triple Generator's three outputs ([C] §3.2.2: left = inc 0, the
/// The Triple Generator's three outputs (`[C]` §3.2.2: left = inc 0, the
/// Processing Key = inc 1, right = inc 2) are the two child node keys and
/// the Processing Key of ONE tree node. They must be three different keys —
/// if `inc` were ignored, a descent would revisit its own parent and the
+54 -109
View File
@@ -1,5 +1,5 @@
//! Media-key derivation: DK/PK → Media Key via the subset-difference tree.
//! [C] §3.2.2–§3.2.5.
//! `[C]` §3.2.2–§3.2.5.
use super::crypto::*;
use super::inf::*;
@@ -71,11 +71,11 @@ pub(crate) fn try_pk_against_tables(
/// Validate a processing key against a cvalue/UV pair.
/// Returns the Media Key if valid.
///
/// Steps (media key: [C] §3.2.4; verify relation: [C] §3.2.5.1.4):
/// 1. `mk = AES-128D(pk, cvalue)` [C] §3.2.4
/// 2. `mk[12..16] ^= uv` (4 bytes XOR into the last 4 bytes only) [C] §3.2.4
/// 3. `dec_vd = AES-128D(mk, mk_dv)` [C] §3.2.5.1.4
/// 4. If `dec_vd[0..8] == 01 23 45 67 89 AB CD EF` → valid. [C] §3.2.5.1.4
/// Steps (media key: `[C]` §3.2.4; verify relation: `[C]` §3.2.5.1.4):
/// 1. `mk = AES-128D(pk, cvalue)` `[C]` §3.2.4
/// 2. `mk[12..16] ^= uv` (4 bytes XOR into the last 4 bytes only) `[C]` §3.2.4
/// 3. `dec_vd = AES-128D(mk, mk_dv)` `[C]` §3.2.5.1.4
/// 4. If `dec_vd[0..8] == 01 23 45 67 89 AB CD EF` → valid. `[C]` §3.2.5.1.4
pub(crate) fn validate_processing_key(
pk: &[u8; 16],
cvalue: &[u8],
@@ -105,7 +105,7 @@ pub(crate) fn validate_processing_key(
None
}
/// Compute v_mask from a UV value. [C] §3.2.3. Shared with [`super::variant`].
/// Compute v_mask from a UV value. `[C]` §3.2.3. Shared with [`super::variant`].
pub(super) fn calc_v_mask(uv: u32) -> u32 {
let mut v_mask: u32 = 0xFFFF_FFFF;
while (uv & !v_mask) == 0 && v_mask != 0 {
@@ -115,27 +115,22 @@ pub(super) fn calc_v_mask(uv: u32) -> u32 {
}
/// Derive processing key from device key using subset-difference tree traversal.
/// [C] §3.2.4 (device-tree descent, MSB-branch, terminal PK). Shared with [`super::variant`].
/// `[C]` §3.2.4 (device-tree descent, MSB-branch, terminal PK). Shared with [`super::variant`].
pub(super) fn calc_pk_from_dk(
dk: &[u8; 16],
uv: u32,
v_mask: u32,
dev_key_v_mask: u32,
) -> [u8; 16] {
// Descend from the device node to the record node, following the record's
// `uv` bits. At each level only the child we descend INTO is needed (the
// sibling is computed but never used), and the Processing Key is the
// `aesg3(.,1)` of the FINAL node — so we derive ONE child per level and the
// PK once at the end, instead of left/pk/right at every level. Identical
// result, ~3x fewer block ops. (left child = `aesg3(node,0)`, right = `,2`.)
// Descend device node -> record node by the record's `uv` bits. Only the child
// descended into matters, so derive ONE child per level (left=`aesg3(node,0)`,
// right=`,2`) and the PK=`aesg3(.,1)` once at the end — ~3x fewer block ops.
let mut node = *dk;
let mut current_v_mask = dev_key_v_mask;
// The subset-difference tree is at most 32 levels deep (u32 mask), so the
// walk must converge in <= 32 steps. The arithmetic `>> 1` sign-extends
// current_v_mask, so a v_mask coarser than dev_key_v_mask (reachable from
// a crafted/corrupt MKB) would otherwise saturate at 0xFFFF_FFFF and spin
// forever — bound the loop to keep a bad disc from hanging the rip thread.
// Tree is <=32 levels deep (u32 mask). The arithmetic `>> 1` sign-extends, so a
// v_mask coarser than dev_key_v_mask (crafted/corrupt MKB) would saturate to
// 0xFFFF_FFFF and spin forever; bound to 32 steps so a bad disc can't hang the rip.
let mut steps = 0u32;
while current_v_mask != v_mask {
if steps >= 32 {
@@ -203,14 +198,9 @@ pub fn derive_media_key_and_pk_from_dk(
let p_uv = &uvs[1 + 5 * uvs_idx..];
let u_mask_shift = uvs[5 * uvs_idx]; // byte before the UV value
// `num_uvs` was computed via `take_while(.. c[0] & 0xC0 == 0)`, so
// every iterated slot already has its revoked-marker bits clear — no
// inner `& 0xC0` re-check is needed (it would be unreachable).
//
// Shifts of 32..=63 (0x20..=0x3F) have those bits clear but would
// panic in debug / wrap to a wrong mask in release. The MKB byte is
// disc-controlled, so a crafted/corrupt MKB must not crash the ripper:
// skip an out-of-range slot rather than `<<` it.
// `num_uvs` used `take_while(.. c[0] & 0xC0 == 0)`, so revoked-marker bits
// are already clear (no re-check). But shifts 32..=63 panic/wrap on `<<` and
// the byte is disc-controlled, so skip an out-of-range slot rather than shift.
if u_mask_shift >= 32 {
continue;
}
@@ -262,7 +252,7 @@ pub fn derive_media_key_and_pk_from_dk(
/// 16 bytes) cannot be walked — the walk needs its tree node. This finds that
/// node empirically: for each MKB subset-difference record, it tries the device
/// at the record's node AND at every ancestor v-position (the device may sit one
/// or more levels ABOVE the record, descending via AES-G to reach it), deriving
/// or more levels ABOVE the record, descending via AES-G3 to reach it), deriving
/// the candidate Processing Key DIRECTLY (one [`calc_pk_from_dk`] per candidate,
/// no full re-walk) and checking it validates against that record's cvalue.
///
@@ -284,16 +274,14 @@ pub fn recover_dk_position(mkb: &[u8], key: &[u8; 16]) -> Option<DeviceKey> {
.count();
let n_cv = cvalues.len() / 16;
// Hoisted ONCE for the whole scan: the Processing Key the device produces if
// it sits EXACTLY at a record (zero descent) is `AES-G3(key, 1)` — it does
// not depend on the record, so the zero-descent probe of every slot reuses
// this single value instead of re-deriving it per slot.
// Hoisted once: the zero-descent Processing Key (device sits exactly at a record)
// is `AES-G3(key, 1)`, independent of the record, so every slot's zero-descent
// probe reuses this instead of re-deriving it per slot.
let pk_zero_descent = aesg3(key, 1);
// The slots are independent, so the scan parallelises — a UHD MKB has ~181k
// slots (~26s single-threaded). `find_map_any` returns the first matching
// node found by any thread and cancels the rest; a valid MKB has exactly one
// matching subset-difference, so which thread finds it is immaterial.
// Slots are independent so the scan parallelises (~181k slots, ~26s serial on a
// UHD MKB). `find_map_any` returns the first match and cancels the rest; a valid
// MKB has exactly one matching subset-difference, so which thread finds it is moot.
use rayon::prelude::*;
let found = (0..num_uvs.min(n_cv)).into_par_iter().find_map_any(|i| {
let u_mask_shift = uvs[5 * i];
@@ -415,7 +403,7 @@ pub mod probe {
// ── Volume key: Media Key + Volume ID → VUK → unit keys ──────────────────────
/// Derive VUK from Media Key and Volume ID. [PR] §3.3 / [BD] §3.3
/// Derive VUK from Media Key and Volume ID. `[PR]` §3.3 / `[BD]` §3.3
/// (`Kvu = AES-G(Km, IDv)`; AES-G uses AES-128D):
/// VUK = AES-128-ECB-DECRYPT(media_key, volume_id) XOR volume_id
pub fn derive_vuk(media_key: &[u8; 16], volume_id: &[u8; 16]) -> [u8; 16] {
@@ -426,8 +414,8 @@ pub fn derive_vuk(media_key: &[u8; 16], volume_id: &[u8; 16]) -> [u8; 16] {
vuk
}
/// Decrypt an encrypted unit key using the VUK (AES-128-ECB). [PR] §3.5
/// (Title Key unwrap `Kt = AES-128D(Ku, Kte)`); the BD "CPS Unit Key" synonym is [BD] §3.9.3.
/// Decrypt an encrypted unit key using the VUK (AES-128-ECB). `[PR]` §3.5
/// (Title Key unwrap `Kt = AES-128D(Ku, Kte)`); the BD "CPS Unit Key" synonym is `[BD]` §3.9.3.
pub fn decrypt_unit_key(vuk: &[u8; 16], encrypted_uk: &[u8; 16]) -> [u8; 16] {
aes_ecb_decrypt(vuk, encrypted_uk)
}
@@ -792,7 +780,7 @@ mod resolve_candidate_tests {
///
/// This file holds the whole subset-difference walk and had five tests for it.
/// There are no published AACS test vectors, but none are needed: the AACS
/// relations ([C] §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
/// relations (`[C]` §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
/// key can be constructed with `aes_ecb_encrypt` and the same `aesg3` the walk
/// uses as its node function. That is what `plant_mkb` below does — no real
/// key material, and the assertions check the DERIVED Media Key, not any
@@ -802,7 +790,7 @@ mod position_recovery_tests {
use super::*;
use crate::aacs::crypto::aes_ecb_encrypt;
/// [C] §3.2.5.1.4 Verify-Media-Key plaintext prefix.
/// `[C]` §3.2.5.1.4 Verify-Media-Key plaintext prefix.
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
/// An MKB record: 1-byte type + BE24 total length (header included) + body.
@@ -834,7 +822,7 @@ mod position_recovery_tests {
///
/// `uv = 0x0400` (lowest set bit 10) with `u_mask_shift = 12` is chosen so a
/// gating device node exists: `resolve_dk_node` flips one bit `b < 12`, and
/// the walk's gate ([C] §3.2.4) needs that bit inside `v_mask`
/// the walk's gate (`[C]` §3.2.4) needs that bit inside `v_mask`
/// (`0xFFFF_FFFF << 11`) and outside `u_mask` (`0xFFFF_FFFF << 12`) — i.e.
/// `b == 11`. `uv` is kept under 0x10000 because `DeviceKey::node` is a u16.
fn plant_mkb() -> Planted {
@@ -956,7 +944,7 @@ mod position_recovery_tests {
}
/// `resolve_dk_node` picks the `device_number` that passes the walk's
/// subset-difference gate ([C] §3.2.4). `None` here strands a key whose
/// subset-difference gate (`[C]` §3.2.4). `None` here strands a key whose
/// position was already successfully recovered, so it is the last step of
/// position recovery and fails the same way: usable key, discarded.
///
@@ -1015,22 +1003,12 @@ mod position_recovery_tests {
);
}
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT MKB where the device key sits ABOVE the matching slot.
//
// `plant_mkb` above is a ONE-slot, ZERO-descent fixture: the matching
// subset-difference is at index 0 and the device sits exactly on it. That
// leaves two whole behaviours of `recover_dk_position` unexercised —
// * slot INDEXING (`uvs[5*i]`, `uvs[1 + 5*i]`, `cvalues[i*16..]`), which
// is the identity permutation when i is always 0, and
// * the DESCENT branch, where the device is an ancestor of the slot and
// the candidate position is walked up bit by bit —
// so an MKB whose keyed slot is index 2 of 3, opened by a device one level
// above it, is what pins them.
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT MKB where the device key sits ABOVE the matching slot. Unlike
// `plant_mkb`'s one-slot zero-descent fixture, this pins slot INDEXING and the
// DESCENT branch: keyed slot is index 2 of 3, opened by a device one level above.
/// v-masks for the fixture's two positions, written as literals from
/// [C] §3.2.3 (`v_mask` is all-ones above the LOWEST set bit of `uv`, i.e.
/// `[C]` §3.2.3 (`v_mask` is all-ones above the LOWEST set bit of `uv`, i.e.
/// `0xFFFF_FFFF << (uv.trailing_zeros() + 1)`) rather than computed with
/// `calc_v_mask`, which is itself under test.
const UV_SLOT: u32 = 0x0000_9400; // lowest set bit 10
@@ -1039,7 +1017,7 @@ mod position_recovery_tests {
const V_MASK_ANCESTOR: u32 = 0xFFFF_F000; // 0xFFFF_FFFF << 12
const U_MASK_SHIFT: u8 = 16;
/// `calc_v_mask` implements [C] §3.2.3. Every subset-difference gate and
/// `calc_v_mask` implements `[C]` §3.2.3. Every subset-difference gate and
/// every descent in the walk is masked by its result, so a wrong mask makes
/// the walk match the wrong slots (or none) — pinned here against literal
/// expectations, not against a re-computation.
@@ -1088,10 +1066,8 @@ mod position_recovery_tests {
0xBE, 0xBF,
];
// The Processing Key the device produces after descending from
// UV_ANCESTOR to the slot. Built with the same descent the walk uses
// (as `plant_mkb` builds its cvalue with the same `aesg3`), but anchored
// to the FIXED ancestor position above — so a walk that computes a
// Processing Key from descending UV_ANCESTOR -> slot, via the same descent the
// walk uses but anchored to the FIXED ancestor above — so a walk computing a
// different candidate position derives a different Kp and fails.
let pk = calc_pk_from_dk(&dkey, UV_SLOT, V_MASK_SLOT, V_MASK_ANCESTOR);
@@ -1191,20 +1167,9 @@ mod position_recovery_tests {
assert!(recover_dk_position(&p.mkb, &stranger).is_none());
}
// ════════════════════════════════════════════════════════════════════
// A FOUR-LEVEL descent taking both branches.
//
// The fixtures above descend zero levels and one level (left). The tree
// walk's per-level branch decision — [C] §3.2.4: descend RIGHT
// (`aesg3(.,2)`) when the slot's `uv` has the level's bit set, LEFT
// (`aesg3(.,0)`) when it is clear, terminal Processing Key `aesg3(.,1)` —
// is only pinned by a descent that takes both branches more than once.
//
// The expected Processing Key here is written out as an EXPLICIT chain of
// `aesg3` calls, not computed with `calc_pk_from_dk`: a fixture built by
// the function under test moves with it, and every mutation of the descent
// would stay self-consistent.
// ════════════════════════════════════════════════════════════════════
// A FOUR-LEVEL descent taking both branches, pinning the per-level decision ([C]
// §3.2.4: RIGHT `aesg3(.,2)` if uv bit set, else LEFT `aesg3(.,0)`, PK `aesg3(.,1)`).
// Expected PK is an EXPLICIT `aesg3` chain, not `calc_pk_from_dk` (which would drift).
/// Slot `uv` for the four-level fixture: bits 8, 6 and 4 set. Lowest set
/// bit 4 → the descent reads bits 8, 7, 6, 5 (set, clear, set, clear).
@@ -1232,13 +1197,8 @@ mod position_recovery_tests {
0xDE, 0xDF,
];
// [C] §3.2.4, written out level by level. Descending from the ancestor
// node to the slot reads UV_SLOT4's bits 8, 7, 6, 5 in that order:
// bit 8 = 1 → right child, aesg3(., 2)
// bit 7 = 0 → left child, aesg3(., 0)
// bit 6 = 1 → right child, aesg3(., 2)
// bit 5 = 0 → left child, aesg3(., 0)
// and the Processing Key is aesg3(final_node, 1).
// [C] §3.2.4 level by level: ancestor -> slot reads UV_SLOT4 bits 8,7,6,5 =
// 1,0,1,0 -> right(2),left(0),right(2),left(0), then PK = aesg3(final_node, 1).
let n1 = aesg3(&dkey, 2);
let n2 = aesg3(&n1, 0);
let n3 = aesg3(&n2, 2);
@@ -1320,13 +1280,9 @@ mod position_recovery_tests {
);
}
// ════════════════════════════════════════════════════════════════════
// MALFORMED MKBs: a truncated cvalue table, and a revoked-marker slot.
//
// The MKB is disc-controlled data. Both of these shapes are reachable from
// a corrupt or crafted disc, and in both the walk must decline to derive a
// key rather than index past the end of a record.
// ════════════════════════════════════════════════════════════════════
// MALFORMED MKBs: a truncated cvalue table, and a revoked-marker slot. Both are
// reachable from a corrupt/crafted (disc-controlled) MKB, and in both the walk must
// decline to derive a key rather than index past the end of a record.
/// Assemble an MKB from an explicit slot list and cvalue table.
/// `slots` is `(u_mask_shift, uv)` per subset-difference entry.
@@ -1424,7 +1380,7 @@ mod position_recovery_tests {
}
/// The `0xC0` revoked marker in a slot's `u_mask_shift` byte TERMINATES the
/// subset-difference table ([C] §3.2.5.1.5). Slots after it are not part of
/// subset-difference table (`[C]` §3.2.5.1.5). Slots after it are not part of
/// the index and must not be walked — a walk that ran past the marker would
/// derive keys from entries the MKB has explicitly ended.
///
@@ -1490,7 +1446,7 @@ mod position_recovery_tests {
}
/// A device key applies to a subset-difference only when BOTH gates hold
/// ([C] §3.2.4): its u-mask must equal the slot's, AND its `uv` must agree
/// (`[C]` §3.2.4): its u-mask must equal the slot's, AND its `uv` must agree
/// with the slot's under the device's v-mask. A key filed with the wrong
/// `u_mask_shift` describes a different region of the tree and must not be
/// used, even though its tree position would otherwise line up — accepting
@@ -1524,7 +1480,7 @@ mod position_recovery_tests {
}
/// `validate_processing_key` XORs the slot's 4-byte `uv` into `mk[12..16]`
/// ([C] §3.2.4 step 2). XOR, not OR: the operation must be reversible, and
/// (`[C]` §3.2.4 step 2). XOR, not OR: the operation must be reversible, and
/// it must be able to CLEAR a bit the AES output set. A `uv` and a Media Key
/// that share set bits in those four bytes are what tell the two apart.
#[test]
@@ -1580,7 +1536,7 @@ mod position_recovery_tests {
/// `probe::aes_dec` is the single AACS verify primitive a reproduction
/// harness has: every claim such a harness makes about a Media Key is
/// `aes_dec(km, mk_dv)` starting with the [C] §3.2.5.1.4 magic. A body
/// `aes_dec(km, mk_dv)` starting with the `[C]` §3.2.5.1.4 magic. A body
/// returning a fixed block makes the harness answer the SAME way for every
/// key and every disc — either "nothing verifies" or, if the constant
/// happened to start with the magic, "everything verifies", which is the
@@ -1647,20 +1603,9 @@ mod position_recovery_tests {
);
}
// ════════════════════════════════════════════════════════════════════
// An ODD subset-difference `uv` — the depth-0 slot.
//
// `recover_dk_position`'s descent starts at `uv_r.trailing_zeros() + 1`.
// Every other fixture in this module uses an even `uv` (lowest set bit 4,
// 10 or 11), so `trailing_zeros()` was never 0 and the descent never began
// at level 1. That left the whole depth-0 case unexecuted: a slot whose `uv`
// is odd sits at the very bottom of the subset-difference tree, and it is a
// perfectly legal MKB shape.
//
// It is also the arithmetic boundary of the loop bound: at `p == 0` the
// `+ 1` is the only thing keeping `(p - 1)` — an unsigned underflow — off
// the range expression.
// ════════════════════════════════════════════════════════════════════
// An ODD subset-difference `uv` — the legal depth-0 slot. Descent starts at
// `uv_r.trailing_zeros() + 1`; other fixtures use even `uv`, so this alone exercises
// `trailing_zeros() == 0` and the `p == 0` boundary where `+ 1` blocks a `(p-1)` underflow.
/// Slot `uv` with bits 8, 6, 4 AND 0 set: lowest set bit 0, so
/// `trailing_zeros() == 0` and the descent must start at level 1.
@@ -1673,7 +1618,7 @@ mod position_recovery_tests {
/// An MKB with a single ODD-`uv` slot, keyed by a device one level above it.
///
/// The expected Processing Key is written as the EXPLICIT `aesg3` chain the
/// one-level descent produces ([C] §3.2.4): from the ancestor, bit 1 of
/// one-level descent produces (`[C]` §3.2.4): from the ancestor, bit 1 of
/// `UV_ODD` is CLEAR, so the walk takes the left child (`aesg3(.,0)`) and
/// then terminates with `aesg3(.,1)`. Not built with `calc_pk_from_dk` —
/// a fixture built by the walk moves with the walk's own mutations.
+6 -9
View File
@@ -63,11 +63,9 @@ pub fn unit_disposition(
None => UnitDisposition::Default,
// In a forensic segment → decide by whether it is our index.
Some(seg) => {
// `seg.index` is an untrusted u16 from IndividualSegment.tbl; a real
// forensic index is 1..=32. Compare in u16 space so a corrupt/crafted
// index above 255 can't truncate into a valid u8 and alias our index.
// The disposition carries a u8 for diagnostics (saturated — an
// out-of-range index is never ours anyway).
// `seg.index` is untrusted (real range 1..=32); compare in u16 space so a
// crafted index above 255 can't truncate into a valid u8 and alias ours.
// Diagnostics carry a saturated u8 since out-of-range is never ours.
let seg_index = seg.index;
let diag = seg_index.min(u8::MAX as u16) as u8;
match disc_index {
@@ -169,10 +167,9 @@ mod tests {
#[test]
fn out_of_range_index_does_not_truncate_into_ours() {
// A crafted/corrupt segment index of 288 (0x0120) truncates to 32 in a
// u8. With our disc index resolved as 32, the old `seg.index as u8`
// compare would alias it to OUR index and decrypt with the wrong key.
// The u16 compare must instead classify it as foreign.
// Index 288 (0x0120) truncates to 32 in a u8, which would alias our
// resolved index 32 under the old `as u8` compare; u16 compare must
// classify it as foreign instead.
let segs = tbl(&[(288, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
+14 -17
View File
@@ -67,6 +67,7 @@ pub fn disc_hash_hex(hash: &[u8; 20]) -> String {
/// Parse Unit_Key_RO.inf from raw bytes.
///
/// Format (from AACS spec):
/// ```text
/// [0..4] BE32: offset to key storage area (uk_pos)
/// [16] app_type (1 = BD-ROM)
/// [17] num_bdmv_dir
@@ -81,6 +82,7 @@ pub fn disc_hash_hex(hash: &[u8; 20]) -> String {
/// [uk_pos+48..] encrypted keys, 16 bytes each
/// AACS 1.0: 48-byte stride
/// AACS 2.0 / 2.1: 64-byte stride (48 + 16 extra)
/// ```
pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() < 20 {
return None;
@@ -135,18 +137,16 @@ pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFil
pos += stride;
}
// The loop above `break`s if the buffer runs out mid-key. A short list
// means the .inf is malformed/truncated — reject it rather than silently
// accepting fewer keys than the header declared, which would later map
// title CPS units to nonexistent keys.
// The loop `break`s if the buffer runs out mid-key; a short list means the .inf
// is malformed/truncated — reject rather than silently mapping title CPS units
// to fewer keys than the header declared.
if encrypted_keys.len() != num_uk {
return None;
}
// Title → CPS unit mapping (AACS Unit_Key_RO format): each on-disc CPS
// value is in `1..=num_uk` (else zeroes it) and converts the 1-based on-disc
// index to a 0-based key index. We mirror that so the stored value is a safe,
// ready-to-use key index rather than a raw 1-based number.
// Title → CPS unit mapping (AACS Unit_Key_RO format): validates the on-disc
// CPS value is in `1..=num_uk` (else zeroes it) and converts it from 1-based
// to a safe, ready-to-use 0-based key index.
let to_key_idx = |cps: u16| -> u16 {
if cps >= 1 && cps as usize <= num_uk {
cps - 1
@@ -228,7 +228,7 @@ const VTKF_AV_FLG: u8 = 0x80;
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
/// (real multi-CPS-unit HD-DVD titles) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
@@ -382,10 +382,8 @@ pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
return None;
}
// Content Certificate layout (per the AACS content-cert format):
// [0] certificate type (0x00 = AACS1, 0x10 = AACS2)
// [1] bit7 bus_encryption_enabled_flag (`p[1] >> 7`)
// [14..20] cc_id (6 bytes) (`p + 14`)
// Content Certificate layout: [0] cert type (0x00 AACS1/0x10 AACS2),
// [1] bit7 bus_encryption_enabled_flag, [14..20] cc_id (6 bytes).
let version = if data[0] == 0x00 {
AacsVersion::V10
} else {
@@ -444,10 +442,9 @@ mod vtkf_tests {
let data = synth_vtkf(&[k1, k2, k3]);
let ukf = parse_vtkf(&data).expect("valid VTKF must parse");
// Exactly the three present entries — the empty slots and the trailing
// 16-byte TKF MAC are NOT mistaken for keys. Critically, k2/k3 are read
// at the 36-byte stride (offsets 0xA4, 0xC8); the old 32-byte stride
// misread them from inside the previous entry's binding MAC.
// Exactly the three present entries — empty slots and the trailing 16-byte
// TKF MAC are not mistaken for keys. k2/k3 are read at the 36-byte stride
// (0xA4, 0xC8); the old 32-byte stride misread them from the prior MAC.
assert_eq!(ukf.encrypted_keys.len(), 3);
assert_eq!(
ukf.encrypted_keys[0],
+8 -10
View File
@@ -1,4 +1,4 @@
//! AACS Media Key Block — [C] Chapter 3.
//! AACS Media Key Block — `[C]` Chapter 3.
//!
//! The MKB record format (framing walker, the `MkbRecord` view, record-body
//! finders), the MKBType / AACS-generation classification, and MKB-file
@@ -7,10 +7,8 @@
//! still live side by side pending a follow-up that collapses them.
// ── MKB record types ([C] Chapter 3) ──────────────────────────────────────
// The ONE canonical set. Every record-type comparison in the `aacs` module
// references these, so a type byte is never a bare literal scattered across
// files (the `0x0c` variant-data record in particular used to appear in several
// hand-rolled forms).
// The ONE canonical set; every record-type comparison in `aacs` references these
// so a type byte is never a bare literal scattered across files.
/// Type-and-Version — carries the 32-bit MKBType / AACS generation.
pub(crate) const REC_TYPE_AND_VERSION: u8 = 0x10;
@@ -157,7 +155,7 @@ impl AacsVersion {
}
/// Find Verify Media Key Record (type 0x81 for AACS 1.0, 0x86 for AACS 2.0/2.1) in MKB.
/// 0x81: [C] §3.2.5.1.4. 0x86 (AACS 2.x): [RE] — not in the public spec (from real 2.x MKBs).
/// 0x81: `[C]` §3.2.5.1.4. 0x86 (AACS 2.x): `[RE]` — not in the public spec (from real 2.x MKBs).
pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
// Verify-Media-Key record (0x81 for AACS 1.0, 0x86 for AACS 2.x): mk_dv is
// the 16 bytes at record offset 4 (body offset 0). Needs rec_len >= 20.
@@ -189,12 +187,12 @@ pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
}
}
/// Find Subset-Difference records (type 0x04) in MKB. [C] §3.2.5.1.5.
/// Find Subset-Difference records (type 0x04) in MKB. `[C]` §3.2.5.1.5.
pub(crate) fn mkb_find_subdiff_records(mkb: &[u8]) -> Option<Vec<u8>> {
find_record_body(mkb, 0x04)
}
/// Find the Media Key Data Record (cvalues table) in an MKB. [C] §3.2.4 / §3.2.5.1.7.
/// Find the Media Key Data Record (cvalues table) in an MKB. `[C]` §3.2.4 / §3.2.5.1.7.
///
/// The cvalue table is record type `0x05` (Media Key Data) on BOTH AACS
/// 1.0 and AACS 2.x MKBs — its 16-byte cvalue entries are 1:1 with the
@@ -331,7 +329,7 @@ impl MkbType {
}
/// The raw 32-bit MKBType field from the Type-and-Version record (0x10), bytes
/// 4-7. `None` if no 0x10 record is present. [C] §3.2.5.1.1 Table 3-2.
/// 4-7. `None` if no 0x10 record is present. `[C]` §3.2.5.1.1 Table 3-2.
pub fn mkb_type_raw(mkb: &[u8]) -> Option<u32> {
// Type-and-Version record (0x10): the 32-bit MKBType is bytes 4-7 (body
// offset 0). Needs rec_len >= 8 (4 header + 4 type).
@@ -516,7 +514,7 @@ mod tests {
assert_eq!(recs[1].body, vec![0x55; 16]);
}
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim ([C] §3.2.5.1.1
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim (`[C]` §3.2.5.1.1
/// Table 3-2), including a value this build does not recognise — the caller
/// uses it to tell "unknown MKB generation" from "no Type record at all".
/// All four bytes must come from the record body; reading any of them from
+13 -34
View File
@@ -122,18 +122,9 @@ pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String>
AacsRole::Mkb => v.push(format!("/{d}/MKBROM.AACS")),
AacsRole::ContentCert => v.push(format!("/{d}/CONTENT_CERT.AACS")),
AacsRole::UnitKey => {
// Glob VTKF*.AACS — the title-key filename is not fixed at
// VTKF000 (Freedom ships VTKF090 + VTKF100). Sorted for a
// deterministic try order.
//
// Each VTKF%%%.AACS is bound to ONE playlist (VPLST%%%.XPL): the
// TKF's 12-byte PLAYLIST_NAME field (bytes 0x10..0x1C) names the
// playlist whose Title Keys it carries, and keys from a TKF whose
// name does not match the title's playlist must not be used. The
// caller resolves this by trying candidates in sorted order and
// decrypting with the one whose keys verify — correct for a
// single-playlist disc; a name-matched selection keyed on the
// active playlist is the precise form for multi-playlist discs.
// Glob VTKF*.AACS (not fixed at VTKF000; Freedom ships VTKF090 +
// VTKF100), sorted for a deterministic try order. Each is bound to
// ONE playlist; caller tries in order, uses whichever's keys verify.
let mut names: Vec<&str> = dir
.entries
.iter()
@@ -157,7 +148,7 @@ pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String>
///
/// `read` performs the actual per-path read (full file or bounded prefix), so
/// callers share the same first-present walk regardless of read style. Returns
/// [`Error::AacsNoKeys`] if no candidate is present. Generic over the path
/// [`Error::AacsNoKeys`](crate::Error::AacsNoKeys) if no candidate is present. Generic over the path
/// element (`&str` or owned `String`) so it accepts the `Vec<String>` that
/// [`role_paths`] builds from the discovered HD DVD directory.
pub(crate) fn read_first<S, F>(candidates: &[S], mut read: F) -> crate::error::Result<Vec<u8>>
@@ -173,15 +164,9 @@ where
Err(crate::error::Error::AacsNoKeys)
}
// The module structure IS the public API — consumers import from the owning
// module directly (e.g. `aacs::content::decrypt_unit`, `aacs::mkb::MkbType`,
// `aacs::derive::{derive_vuk, resolve_candidate}`, `aacs::resolve::resolve_keys_v2`).
// The `derive::probe` reproduction harness stays reachable via its module path.
//
// A small set of flat re-exports is kept for the typed key primitives and the
// content-decrypt entry points that downstream key-source crates import through
// the `aacs::` path. These are the stable, load-bearing names; keeping them here
// lets those crates track the module refactor without a lockstep re-pin.
// The module structure IS the public API — consumers import from the owning module
// (e.g. `aacs::content::decrypt_unit`). A small set of flat re-exports below is kept
// for typed key primitives and content-decrypt entry points that downstream crates rely on.
pub use content::ALIGNED_UNIT_LEN;
pub use derive::derive_vuk;
pub use types::{DeviceKey, HostCert, MediaKey, ProcessingKey, UnitKey, Vid, Vuk};
@@ -199,19 +184,16 @@ mod tests {
#[test]
fn aligned_unit_len_is_three_2048_byte_sectors() {
// ALIGNED_UNIT_LEN is the AACS aligned-unit size: 3 × 2048 = 6144.
// Re-exported from decrypt; pin the value here so the public constant
// and the spec stay in lockstep.
// ALIGNED_UNIT_LEN is the AACS aligned-unit size (3 × 2048 = 6144);
// pin it here so the public constant tracks the spec.
assert_eq!(ALIGNED_UNIT_LEN, 6144);
assert_eq!(ALIGNED_UNIT_LEN, 3 * 2048);
}
#[test]
fn version_strides_are_reexported_and_distinct() {
// The three AACS generations are part of the public surface, and the
// V10 (48) vs V20/V21 (64) stride distinction is the load-bearing
// difference. Confirm the enum re-export is usable and the variants
// are distinct values.
// V10 (48) vs V20/V21 (64) stride is the load-bearing distinction;
// confirm the enum re-export is usable and variants are distinct.
assert_ne!(AacsVersion::V10, AacsVersion::V20);
assert_ne!(AacsVersion::V20, AacsVersion::V21);
}
@@ -236,11 +218,8 @@ mod tests {
}
// ── HD DVD AACS directory / filename discovery ────────────────────────
//
// The HD DVD AACS dir name and title-key filename are authoring-specific
// and were previously hardcoded to `/ANY!/VTKF000.AACS`. These verify the
// discovery replacement against both real-disc shapes: Freedom (`AAC!` +
// `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
// Dir/filename are authoring-specific (previously hardcoded to `/ANY!/VTKF000.AACS`);
// verify against Freedom (`AAC!` + `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
#[test]
fn role_paths_discovers_hddvd_dir_and_globs_all_vtkf_variants() {
+70 -134
View File
@@ -5,10 +5,9 @@ use super::derive::*;
use super::inf::*;
use super::mkb::*;
//
// Canonical form is `<category>1003` (low 16 bits `0x1003` is a fixed marker).
// Types 3/4/10 are from the AACS Common Cryptographic Elements spec (0.953,
// §3.2.5.1.1); the Category-C 2.0/2.1 values are the standard MKB type constants.
// §3.2.5.1.1); Category-C 2.0/2.1 values are the standard MKB type constants.
// ── Full VUK resolution chain ───────────────────────────────────────────────
@@ -136,13 +135,13 @@ pub(crate) fn classify_resolve_failure(ctx: &ResolveContext<'_>) -> ResolveFailu
}
/// AACS 1.0 key resolution. Parses `Unit_Key_RO.inf` with 48-byte
/// stride. Tries paths 1 → 4 in order.
/// stride. Tries paths 1 → 5 in order (via `resolve_keys_classical`).
pub fn resolve_keys_v1(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
resolve_keys_classical(ctx, AacsVersion::V10)
}
/// AACS 2.0 key resolution. Parses `Unit_Key_RO.inf` with 64-byte
/// stride. Tries paths 1 → 4 in order. When paths 3/4 succeed against
/// stride. Tries paths 1 → 5 in order. When paths 3/4 succeed against
/// an MKB carrying Variant records (`0x2d` / `0x2f`), the result's
/// `version` is upgraded to [`AacsVersion::V21`] — derivation still
/// runs through the classical V2 path; the V21-specific Variant chain
@@ -210,9 +209,8 @@ pub fn resolve_keys_v21(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
if has_vid {
// Path 1: Variant chain (V21's analogue of classical Path 1's DK
// derivation). Derive the Processing Key from device keys first (DK → PK
// via the variant walk), then run the PK → Km variant primitive and
// derive the per-disc VUK from Km + VID.
// derivation): DK → PK via variant walk, then PK → Km, then VUK
// from Km + VID.
if let Some(mkb) = ctx.mkb {
let recs = super::mkb::walk_mkb(mkb);
let all_dks = providers.device_keys();
@@ -242,10 +240,9 @@ pub fn resolve_keys_v21(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
// Path 3: pre-computed MK + matching VID → derived VUK.
// Short-circuit: first provider with a matching VID wins.
if let Some(entry) = providers.lookup_disc_by_vid(ctx.volume_id) {
// The entry already matched by VID and derive_vuk needs only mk +
// ctx.volume_id, so a provider that matches by VID without
// populating disc_id (e.g. a webservice) must not have its MK
// dropped — gate on the MK alone.
// Already matched by VID and derive_vuk needs only mk + volume_id, so a
// provider matching by VID without disc_id (e.g. a webservice) must not
// lose its MK — gate on the MK alone.
if let Some(mk) = entry.media_key {
let vuk = derive_vuk(&mk, ctx.volume_id);
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_v21_path3_hit", "MK+VID entry matched volume_id");
@@ -371,13 +368,9 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
}
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_miss", pk_count = all_pks.len(), "PK derivation failed");
// Path 2.5: MK-pool brute. keydb stores Media Keys per-disc, but an
// MK is MKB-scoped (shared across a pressing/MKB-family). A disc
// whose own hash/VID isn't keyed can still resolve if ANY stored MK
// verifies against its MKB. Try every distinct MK via km_verifies;
// a UNIQUE pass is this disc's Km → derive VUK (needs VID) → UK.
// One AES-D + magic check per candidate (cheap). mk_dv is hoisted
// out of the loop so the MKB is not re-walked per candidate.
// Path 2.5: MK-pool brute. An MK is MKB-scoped, so a disc whose own
// hash/VID isn't keyed can still resolve if ANY stored MK verifies (a
// UNIQUE pass via km_verifies is this disc's Km); mk_dv is hoisted out.
let mks = providers.media_keys();
let chosen_mk = mkb_find_mk_dv(mkb).and_then(|mk_dv| {
unique_verifying_mk(&mks, |mk| {
@@ -398,10 +391,9 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
// Path 3: pre-computed MK + matching VID → derived VUK.
// Short-circuit: first provider with a matching VID wins.
if let Some(entry) = providers.lookup_disc_by_vid(ctx.volume_id) {
// The entry already matched by VID and derive_vuk needs only mk +
// ctx.volume_id, so a provider that matches by VID without
// populating disc_id (e.g. a webservice) must not have its MK
// dropped — gate on the MK alone.
// Already matched by VID and derive_vuk needs only mk + volume_id, so a
// provider matching by VID without disc_id (e.g. a webservice) must not
// lose its MK — gate on the MK alone.
if let Some(mk) = entry.media_key {
let vuk = derive_vuk(&mk, ctx.volume_id);
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path3_hit", "MK+VID entry matched volume_id");
@@ -586,11 +578,9 @@ mod tests {
#[test]
fn derive_media_key_from_dk_survives_out_of_range_u_mask_shift() {
// Regression: a crafted/corrupt MKB with a Subset-Difference
// u_mask_shift of 32..=63 (passes the 0xC0 revoked-marker check but
// overflows `0xFFFF_FFFF << shift`) used to panic in debug / compute a
// wrong mask in release. The walk must now skip the bad slot and
// return cleanly (no panic) on disc-controlled bytes.
// Regression: a u_mask_shift of 32..=63 (passes the 0xC0 revoked-marker
// check but overflows `0xFFFF_FFFF << shift`) used to panic in debug /
// compute a wrong mask in release. The walk must skip the bad slot.
let mut mkb: Vec<u8> = Vec::new();
// 0x81 record: 4-byte header + 16-byte mk_dv body (rec_len = 20).
mkb.extend_from_slice(&[0x81, 0x00, 0x00, 0x14]);
@@ -616,14 +606,9 @@ mod tests {
#[test]
fn test_decrypt_unit_key_from_vuk() {
// VUK → encrypted unit key → unit key roundtrip. The keydb-sourced
// variant of this test (which scanned a real KEYDB for VUK + unit
// keys) moved to freemkv-keysources; this rebuilt version exercises
// the same AES-G primitive (decrypt_unit_key ∘ aes_ecb_encrypt under a
// VUK) with directly-constructed material, so it needs no parser and
// keeps the crypto covered in libfreemkv. `aes_ecb_encrypt` is
// pub(crate), reachable here but not from keysources — the reason this
// half stays.
// VUK → encrypted unit key → unit key roundtrip. The keydb-sourced variant
// moved to freemkv-keysources; this exercises the same AES-G primitive
// with directly-constructed material — `aes_ecb_encrypt` is pub(crate).
use super::super::crypto::aes_ecb_encrypt;
let vuk = [0x5Au8; 16];
// A few representative "decrypted" unit keys.
@@ -743,11 +728,9 @@ mod tests {
#[test]
fn trim_mkb_never_zeroes_an_unrecognised_mkb() {
// Regression: the 0.31.0 read_aacs_inputs path truncated the MKB to
// mkb_content_len() unconditionally. For an MKB whose first record the
// parser can't read, mkb_content_len() returns 0 → an unconditional
// truncate zeroed the MKB, so autorip sent an EMPTY MKB to the key
// service (or skipped the request). trim_mkb must leave it intact.
// Regression: 0.31.0 truncated the MKB to mkb_content_len() unconditionally.
// For an unparseable first record, mkb_content_len() returns 0, so autorip
// sent an EMPTY MKB to the key service. trim_mkb must leave it intact.
let unrecognised = vec![0xFFu8; 4096]; // first "rec_type" 0xFF, rec_len huge → content_len 0
assert_eq!(
mkb_content_len(&unrecognised),
@@ -809,11 +792,9 @@ mod tests {
#[test]
fn probe_try_pk_against_tables_accepts_planted_pk_rejects_corrupt() {
// Lock in the terminal PK scan (`try_pk_against_tables`) used by the
// production PK path (`derive_media_key_from_pk`). Plant a terminal PK
// whose derived Media Key satisfies a synthetic verify record; confirm
// the scan ACCEPTS it against caller-supplied SD/cvalue tables and
// REJECTS a 1-byte corruption.
// Lock in the terminal PK scan (`try_pk_against_tables`) used by
// `derive_media_key_from_pk`: a planted PK whose derived MK satisfies a
// synthetic verify record must be ACCEPTED; a 1-byte corruption REJECTED.
use super::super::crypto::aes_ecb_encrypt as enc;
let pk: [u8; 16] = [
@@ -855,11 +836,9 @@ mod tests {
#[test]
fn validate_processing_key_round_trip_with_nonzero_uv() {
// Synthesise a (pk, uv, mk, cvalue, mk_dv) tuple that satisfies the
// AACS PK-validation relation, then confirm validate_processing_key
// recovers mk. Catches the bugs that landed pre-fix:
// * uv XOR step was missing → mk wrong whenever uv != 0
// * AES-128E + 12-zero check instead of AES-128D + magic
// Synthesise a (pk, uv, mk, cvalue, mk_dv) tuple satisfying the AACS
// PK-validation relation; confirms validate_processing_key recovers mk.
// Catches pre-fix bugs: missing uv XOR, and AES-128E+zero-check vs 128D+magic.
use super::super::crypto::{aes_ecb_decrypt as dec, aes_ecb_encrypt as enc};
let pk: [u8; 16] = [
@@ -872,10 +851,8 @@ mod tests {
];
let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00];
// cvalue is what AES-128E(pk, mk') gives, where mk' = mk with the
// last-4-bytes-uv XOR pre-undone:
// mk_raw[12..16] = mk[12..16] XOR uv (so the validate step XORs
// uv back in and recovers mk).
// cvalue is AES-128E(pk, mk') where mk' = mk with last-4-bytes-uv XOR
// pre-undone, so the validate step XORs uv back in and recovers mk.
let mut mk_raw = mk;
for a in 0..4 {
mk_raw[12 + a] ^= uv[a];
@@ -907,16 +884,9 @@ mod tests {
assert!(validate_processing_key(&pk, &cvalue, &wrong_uv, &mk_dv).is_none());
}
// ── MKB cvalue-record selection (issue #259 / #281) ─────────────────
//
// The cvalue (Media Key Data) table is record 0x05; the
// Subset-Difference index is record 0x04 (the standard AACS MKB layout:
// 0x05 = cvalues, 0x04 = subset-difference index). Record 0x07
// (Explicit Subset-Difference Record) is NOT the cvalue table. On real
// in-drive AACS 2.x UHD MKBs 0x07 is small (~96 entries) while the 0x05
// table is large (181270 entries, 1:1 with 0x04). An earlier
// `mkb_find_cvalues` preferred 0x07, which under-tested the SD walk and
// broke the DK→walk path. The selector must prefer 0x05.
// ── MKB cvalue-record selection (issue #259 / #281): cvalue (Media Key Data)
// is record 0x05, not 0x07 (Explicit SD, small ~96 entries vs 0x05's 181270).
// An earlier `mkb_find_cvalues` wrongly preferred 0x07, breaking DK→walk.
/// Build a 4-byte MKB record header (type + 3-byte big-endian total
/// length, header included) and append `body`.
@@ -1012,12 +982,9 @@ mod tests {
#[test]
fn real_aacs2_samples_select_large_0x05_not_small_0x07() {
// Real in-drive AACS 2.x UHD MKBs carry BOTH a small 0x07
// Explicit-Subset-Difference record (96 16-byte entries) AND the
// large 0x05 Media Key Data / cvalue table (181270 entries, 1:1
// with the 0x04 index). The production selector must return the
// LARGE 0x05 body, not the small 0x07 one. This is the exact
// regression #259 found. Skips when no sample dir is present.
// Real AACS 2.x UHD MKBs carry BOTH a small 0x07 record (96 entries) AND
// the large 0x05 cvalue table (181270 entries); selector must return the
// LARGE 0x05 body — the exact regression #259. Skips if no sample dir.
let samples = [
"sample-a/MKB_RO.inf",
"sample-b/MKB_RO.inf",
@@ -1116,19 +1083,15 @@ mod tests {
#[test]
fn resolve_keys_skips_paths_2_through_4_when_vid_is_zero() {
// No VID -> paths 2/3/4 cannot succeed. The function must
// return None WITHOUT touching the MKB / device keys, so we
// can pass an MKB that would otherwise cause expensive
// derivation work — it must not be consumed.
// No VID -> paths 2/3/4 cannot succeed. Must return None WITHOUT touching
// the MKB/device keys, so an MKB that would cause expensive derivation
// work can be passed here and must not be consumed.
let uk_ro = minimal_unit_key_ro();
let zero_vid = [0u8; 16];
// A provider carrying a dummy processing key but NO disc entry that
// matches this disc. `disc_entry: None` preserves the negative-miss
// the test asserts: with VID=0, paths 1/2/3 are skipped and the
// path-4/5 hash lookup must MISS (a SuppliedKey returns its
// disc_entry unconditionally, so the planted entry would WRONGLY hit
// path 4 — None keeps the miss).
// Dummy processing key, NO matching disc entry: `disc_entry: None` keeps
// the path-4/5 hash lookup a miss (SuppliedKey returns disc_entry
// unconditionally, so a planted entry would WRONGLY hit path 4).
let keydb = SuppliedKey {
device_keys: Vec::new(),
processing_keys: vec![[0u8; 16]],
@@ -1153,10 +1116,9 @@ mod tests {
#[test]
fn resolve_keys_path4_still_runs_when_vid_is_zero() {
// Path 4 (disc-hash → VUK) doesn't need VID. Confirm the
// short-circuit doesn't block it: install a keydb entry whose
// disc_hash matches the fixture's hash, with a known VUK, and
// verify resolve_keys returns it with key_source = 4.
// Path 4 (disc-hash → VUK) doesn't need VID; confirm the short-circuit
// doesn't block it: install a keydb entry whose disc_hash matches the
// fixture with a known VUK, verify resolve_keys returns key_source = 4.
let uk_ro = minimal_unit_key_ro();
let hash = disc_hash(&uk_ro);
// `find_disc` lowercases the incoming hash; the entry map is
@@ -1195,10 +1157,9 @@ mod tests {
#[test]
fn resolve_keys_path5_uses_keydb_unit_keys_when_vuk_absent() {
// Path 5: an entry with no VUK but with pre-decrypted unit
// keys matching the disc's CPS-unit numbering decrypts the
// disc directly. Covers the ~4,572 U-only KEYDB entries
// (mostly MKBv76+ UHDs) that the resolver previously ignored.
// Path 5: an entry with no VUK but pre-decrypted unit keys matching the
// disc's CPS-unit numbering decrypts directly. Covers the ~4,572 U-only
// KEYDB entries (mostly MKBv76+ UHDs) the resolver previously ignored.
let uk_ro = minimal_unit_key_ro();
let hash = disc_hash(&uk_ro);
let hash_hex = disc_hash_hex(&hash).to_lowercase();
@@ -1237,11 +1198,9 @@ mod tests {
}
#[test]
fn resolve_keys_path5_rejects_partial_unit_key_coverage() {
// If the disc declares a CPS unit that's not in the KEYDB
// entry's unit_keys, path 5 must NOT half-decrypt the disc.
// The match function returns None and the resolver falls
// through to None overall (no other paths available in this
// setup).
// If the disc declares a CPS unit not in the KEYDB entry's unit_keys, path
// 5 must NOT half-decrypt: match returns None and the resolver falls
// through to None overall (no other paths available here).
let uk_ro = minimal_unit_key_ro();
let hash = disc_hash(&uk_ro);
let hash_hex = disc_hash_hex(&hash).to_lowercase();
@@ -1295,13 +1254,9 @@ mod tests {
);
// This disc's inf (its hash will NOT be in keydb).
let uk_ro = minimal_unit_key_ro();
// The sibling's MK is lifted directly into the MK pool: a KeyDb
// aggregated per-disc media_keys into media_keys(), but SuppliedKey
// does NOT harvest its disc_entry's media_key — it has an explicit
// media_keys field. `disc_entry: None` preserves the miss on this
// disc's own hash/VID (the sibling matches neither), so ONLY the
// MK-pool brute (km_verifies) can resolve it — exactly the path under
// test.
// The sibling's MK is lifted into the MK pool via SuppliedKey's explicit
// media_keys field; `disc_entry: None` keeps this disc's own hash/VID a
// miss, so ONLY the MK-pool brute (km_verifies) can resolve it.
let keydb = SuppliedKey {
device_keys: Vec::new(),
processing_keys: Vec::new(),
@@ -1407,9 +1362,7 @@ mod tests {
data[1] = 0x01;
assert!(!parse_content_cert(&data).unwrap().bus_encryption);
}
// ════════════════════════════════════════════════════════════════════
// Hardening additions
// ════════════════════════════════════════════════════════════════════
// ════════════════ Hardening additions ════════════════
// ── VUK derivation: spec relation VUK = AES-D(MK, VID) XOR VID ─────────
#[test]
fn derive_vuk_matches_spec_relation_explicitly() {
@@ -1467,11 +1420,9 @@ mod tests {
}
#[test]
fn stride_v10_is_48_v20_is_64_and_picks_distinct_keys() {
// AACS 1.0 stride = 48, AACS 2.0/2.1 stride = 64 (aacs/inf.rs).
// Lay keys at 64-byte stride. Parsing at V20 stride must pick exactly
// those keys; parsing the SAME bytes at V10 (48) stride would read the
// wrong (intermediate) bytes for key 2 onward — proving the stride
// selector matters.
// AACS 1.0 stride = 48, 2.0/2.1 = 64. Keys laid at 64-byte stride: V20
// parsing must pick exactly those keys, while V10 (48) parsing the SAME
// bytes would read wrong intermediate bytes for key 2 onward.
let data = build_unit_key_ro(2, 64);
let v20 = parse_unit_key_ro(&data, AacsVersion::V20).unwrap();
assert_eq!(v20.encrypted_keys.len(), 2);
@@ -1533,11 +1484,9 @@ mod tests {
}
#[test]
fn parse_unit_key_ro_rejects_when_keys_run_off_end() {
// Finding #5: 3 keys declared but the buffer holds only 2 strides plus
// 8 trailing bytes (not a full 3rd 16-byte key). The extraction loop
// breaks at the buffer end (never reading OOB), and the post-loop
// length check rejects the short list with None — a truncated/malformed
// .inf must NOT be silently accepted with fewer keys than declared.
// Finding #5: 3 keys declared but buffer holds only 2 strides + 8 spare
// bytes. Extraction loop breaks at buffer end (never OOB), and the
// post-loop length check rejects the short list rather than accepting fewer.
let uk_pos = 0x60usize;
let stride = 48usize;
// Room for keys at uk_pos+48 and uk_pos+48+48, then only 8 spare bytes
@@ -1579,10 +1528,9 @@ mod tests {
}
#[test]
fn parse_unit_key_ro_title_cps_mapping_first_play_top_menu_then_titles() {
// [20..22] first_play, [22..24] top_menu, [24..26] num_titles, then
// per-title 2-byte pad + 2-byte CPS unit at 26 + i*4 + 2. Each on-disc
// 1-based CPS number in `1..=num_uk` is validated and converted to a
// 0-based key index (per the AACS Unit_Key_RO format); an out-of-range number → 0.
// [20..22] first_play, [22..24] top_menu, [24..26] num_titles, then per-title
// 2-byte pad + 2-byte CPS unit at 26 + i*4 + 2. Each on-disc 1-based CPS
// number in `1..=num_uk` converts to a 0-based key index; out-of-range → 0.
let mut data = build_unit_key_ro(4, 64); // num_uk = 4 → CPS 1..=4 valid
data[20..22].copy_from_slice(&1u16.to_be_bytes()); // first_play CPS 1
data[22..24].copy_from_slice(&2u16.to_be_bytes()); // top_menu CPS 2
@@ -2046,17 +1994,9 @@ mod tests {
#[test]
fn derive_media_key_and_pk_from_dk_returns_intermediate_pk() {
// Regression: a classical DK boil must yield the intermediate
// Processing Key, not just the Media Key. The key service banks the
// PK lineage (DK·PK·MK·VUK·UK); before the `_and_pk_` form existed it
// recovered the MK here but lost the PK silently.
//
// Build a minimal classical MKB (no 0x82/0x83) with:
// - 0x04 Subset-Difference: u_mask_shift=3, uv=0x00000002
// - 0x05 cvalues: one cvalue C planted so AES-D(Kp, C) XOR uv == mk
// - 0x86 Verify Media Key: mk_dv = AES-E(mk, magic || pad)
// and a DK with node=4, uv=2, u_mask_shift=3 so dev_key_v_mask ==
// v_mask: the calc_pk_from_dk loop is a no-op and Kp == aesg3(dk, 1).
// Regression: a classical DK boil must yield the intermediate PK too
// (key service banks DK·PK·MK·VUK·UK; before `_and_pk_` it lost the PK).
// Builds a minimal MKB with a DK whose calc_pk_from_dk walk is a no-op.
use super::super::crypto::aes_ecb_encrypt as enc;
let dk_bytes: [u8; 16] = [
@@ -2126,12 +2066,8 @@ mod tests {
}
// ── resolve_keys_with_reason / classify_resolve_failure ────────────────
//
// The rc.6 E7017/E7022 split is also exercised end-to-end through the
// `ensure_decryptable` gate in `disc/mod.rs`. These tests pin the
// *classifier* directly at the aacs::resolve seam and cover the branches the
// gate test does not: VID-present (must never be VidUnavailable), the
// processing-keys-only material path, and the version dispatch / Ok path.
// Also exercised end-to-end via `ensure_decryptable` in `disc/mod.rs`; these
// pin the *classifier* directly, covering branches that gate test doesn't.
/// A `SuppliedKey` provider with the given derivation material and no
/// disc-keyed entry. Mirrors the construction the gate test uses, lifted to
+5 -7
View File
@@ -181,7 +181,8 @@ pub fn clip_byte_to_lba(extents: &[crate::disc::Extent], clip_byte: u64) -> Opti
/// to clean TS and garbles the other ~40 (the second interleaved half), which the
/// demux then drops — yielding one coherent stream. Ranges outside every segment
/// are left for the map's default (the ordinary Unit Key). A segment that straddles
/// a UDF extent boundary is emitted as one range per whole-sector slice it covers.
/// a UDF extent boundary is skipped entirely, falling back to the Unit Key rather
/// than emitting a wrong span.
///
/// The result feeds [`AacsKeyMap::from_ranges`](crate::decrypt::AacsKeyMap::from_ranges)
/// with the Unit-Key index as the default — the same structure the CPS map uses,
@@ -200,12 +201,9 @@ pub fn fmts_key_ranges(
}
let start_byte = s.start_spn as u64 * SOURCE_PACKET_LEN;
let end_byte = (s.end_spn as u64 + 1) * SOURCE_PACKET_LEN; // exclusive
// A segment is unit-aligned and contiguous in clip bytes; map its first
// and last sector to LBAs. Segments are ~480 KB and extents are GB-sized,
// so a segment almost never crosses an extent boundary — but if the two
// ends land in different extents (non-contiguous LBAs), skip rather than
// emit a wrong span; the units there fall to the Unit Key (garble+drop),
// never a mis-decrypt.
// Map first/last sector to LBAs. Segments (~480 KB) rarely cross the
// GB-sized extent boundary, but if the two ends land in different
// extents, skip rather than emit a wrong span (falls back to Unit Key).
let (Some(a), Some(b)) = (
clip_byte_to_lba(extents, start_byte),
clip_byte_to_lba(extents, end_byte - 1),
+6 -16
View File
@@ -111,17 +111,9 @@ pub struct DiscEntry {
pub unit_keys: Vec<(u32, [u8; 16])>,
}
// ── Redacting `Debug` impls ──────────────────────────────────────────────────
//
// Every type above carries AACS secret material (device keys, host PRIVATE keys,
// media/volume/processing/unit keys). `#[derive(Debug)]` would print those bytes
// verbatim, so a stray `debug!("{:?}", …)` or a panic message would leak the
// keys. These hand-written impls print only NON-secret shape (presence, lengths,
// tree coordinates, indices) — never key bytes. `decrypt::DecryptKeys` follows
// the same policy by omitting `Debug` entirely; here we keep `Debug` because
// these are `PartialEq`/`Eq` value types used in `assert_eq!` and nested inside
// other `#[derive(Debug)]` structs, so the trait must exist — just not leak.
// Guarded by `redaction_tests` below.
// ── Redacting `Debug` impls: types above carry AACS secrets; derived `Debug`
// would leak key bytes via `debug!`/panics. These print only non-secret shape,
// kept (not omitted) since used in `assert_eq!`. See `redaction_tests`.
impl std::fmt::Debug for DeviceKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
@@ -270,11 +262,9 @@ mod unit_key_tests {
mod redaction_tests {
use super::*;
// Sentinel key byte 0xD5 = decimal 213. A derived `Debug` prints `[u8;N]`
// as decimal, so a leaked key surfaces the substring "213"; the redacting
// impls must not. No non-secret field below is 213, so "213" appearing means
// key bytes leaked. Each type must also carry a "redacted" marker (or omit
// the secret entirely) so re-adding `#[derive(Debug)]` fails this test.
// Sentinel byte 0xD5 = decimal 213: a derived `Debug` prints `[u8;N]` as
// decimal, so a leak surfaces "213" (no non-secret field is 213). Each type
// must also carry a "redacted" marker so re-adding `#[derive(Debug)]` fails.
const S: u8 = 0xD5;
fn assert_redacted(what: &str, dbg: &str) {
+68 -125
View File
@@ -54,11 +54,9 @@ use super::crypto::{aes_ecb_decrypt, aes_g};
use super::mkb::*;
use super::types::DeviceKey;
// The MKB record types this chain selects — `REC_MEDIA_KEY_VARIANT_DATA`
// (`0x0c`, the per-slot C table), `REC_VARIANT_DATA_AND_NONCE` (`0x2d`, VARIANTS
// + tail Nonce), `REC_VKD_TABLE` (`0x2f`), the subset-difference / cvalue records
// (`0x04` / `0x05` / `0x07`), and the verify records (`0x81` / `0x86`) — are the
// canonical set in [`super::mkb`], in scope here via the `use super::mkb::*` glob.
// MKB record types this chain selects: `REC_MEDIA_KEY_VARIANT_DATA` (`0x0c`),
// `REC_VARIANT_DATA_AND_NONCE` (`0x2d`), `REC_VKD_TABLE` (`0x2f`), the
// subset-difference/cvalue and verify records — canonical set in [`super::mkb`].
// ── Public constants ──────────────────────────────────────────────────────
@@ -137,10 +135,9 @@ pub(crate) fn variant_key_data(records: &[MkbRecord]) -> Option<&[u8]> {
// ── Subset-difference walk that exposes (Kp, uv) ──────────────────────────
// `calc_v_mask` and `calc_pk_from_dk` (and the AES-G3 seed step they ride
// on) are shared with the classical walk in [`super::derive`] — a single
// definition keeps the variant SD tree byte-identical to the classical one.
// (`aesg3` itself is imported separately in the test module.)
// `calc_v_mask`/`calc_pk_from_dk` (and the AES-G3 seed step) are shared with
// the classical walk in [`super::derive`], keeping the variant SD tree
// byte-identical. `aesg3` itself is imported separately in the test module.
use super::derive::{calc_pk_from_dk, calc_v_mask};
/// Outcome of a subset-difference walk against an MKB. Carries the
@@ -190,10 +187,11 @@ fn mkb_find_mk_dv(records: &[MkbRecord]) -> Option<[u8; 16]> {
/// separate on purpose and select MKB records in DELIBERATELY different
/// order:
///
/// - cvalues: this variant walk tries record `0x07`-then-`0x05`; the
/// classical walk tries `0x05`-then-`0x07`. On a variant MKB the
/// small `0x07` Explicit-Subset-Difference record carries the
/// cvalue the Precursor chain consumes, whereas a classical UHD MKB
/// - cvalues: this variant walk tries record `0x0c`-then-`0x07`-then-`0x05`
/// (`0x0c` is the real-disc per-uv cvalue source; `0x07`/`0x05` are
/// fixture fallbacks); the classical walk tries `0x05`-then-`0x07`. On a
/// variant MKB the small `0x07` Explicit-Subset-Difference record carries
/// the cvalue the Precursor chain consumes, whereas a classical UHD MKB
/// keeps its 1:1 cvalue table in the large `0x05` record (see the
/// note on [`super::derive::probe::mkb_cvalues`]). They must NOT be
/// unified to one order — each is correct for its own MKB shape.
@@ -211,10 +209,8 @@ pub fn walk_processing_key(
) -> Option<ProcessingKeyMatch> {
let mk_dv = mkb_find_mk_dv(records)?;
let uvs = mkb_find_body(records, REC_SUBSET_DIFFERENCE)?;
// Variant cvalue source: a real variant MKB carries its per-uv cvalue table
// in record `0x0c` (confirmed 46,101×16, one per `0x04` subset-difference
// slot). Fall back to `0x07`/`0x05` for the synthetic fixtures and any MKB
// shape that keeps its cvalues there.
// Real variant MKBs carry per-uv cvalues in record `0x0c` (46,101x16, one
// per `0x04` slot); fall back to `0x07`/`0x05` for synthetic fixtures.
let cvalues = mkb_find_body(records, REC_MEDIA_KEY_VARIANT_DATA)
.or_else(|| mkb_find_body(records, REC_EXPLICIT_SUBSET_DIFF))
.or_else(|| mkb_find_body(records, REC_MEDIA_KEY_DATA))?;
@@ -229,17 +225,13 @@ pub fn walk_processing_key(
for uvs_idx in 0..num_uvs {
let p_uv = &uvs[1 + 5 * uvs_idx..];
// `num_uvs` was computed by `take_while(.. (c[0] & 0xC0) == 0)`, so
// every chunk in `0..num_uvs` already has its revoked-marker bits
// clear — that `take_while` is the single authoritative place the
// parse stops, no inner re-check needed.
// `num_uvs` came from `take_while(.. (c[0] & 0xC0) == 0)`, so every
// chunk in `0..num_uvs` already has clear revoked-marker bits.
let u_mask_shift = uvs[5 * uvs_idx];
// 0x20..=0x3F (32..=63) have their revoked-marker bits clear (so they
// pass the take_while above) but are out of range for a u32 shift.
// `wrapping_shl` would silently compute shift % 32 (e.g. 32 → no shift
// → 0xFFFF_FFFF), matching a wrong uv slot and deriving a wrong key.
// Disc-controlled byte: skip the slot instead.
// 0x20..=0x3F pass the take_while but are out of range for a u32
// shift; `wrapping_shl` would silently wrap (shift % 32) and match
// a wrong uv slot. Disc-controlled byte: skip the slot instead.
if u_mask_shift >= 32 {
continue;
}
@@ -281,12 +273,9 @@ pub fn walk_processing_key(
}
let dec_vd = aes_ecb_decrypt(&km_candidate, &mk_dv);
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
// On a classical (non-variant) MKB this magic must
// match. On a variant MKB it won't — `km_candidate`
// is really Kmp and the magic check is moot. We
// still gate the walk on cvalue indexing being
// sane; the chain itself enforces the variant
// semantics downstream.
// On classical MKBs this magic must match. On variant MKBs
// it won't — `km_candidate` is really Kmp, so the magic
// check is moot; the chain enforces semantics downstream.
let classical_ok = dec_vd[..8] == VERIFY_MAGIC;
let variant_present = is_variant_mkb(records);
if !(classical_ok || variant_present) {
@@ -375,10 +364,9 @@ impl std::error::Error for MediaKeyVariantError {}
/// silent bad key).
fn variants_for_uv(records: &[MkbRecord], sd_slot_index: usize) -> Option<u16> {
let body = variant_data_record(records)?;
// The VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at
// the TAIL (see [`variant_nonce`]). Bound the read to the table region so a
// near-end slot can never read Nonce bytes as a VARIANTS entry. NO leading
// header (measured: a v70 `0x2d` body = 46_100*2 + 16 = 92_216).
// VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at the
// TAIL (see [`variant_nonce`]). Bound reads to the table region so a near-end
// slot never reads Nonce bytes (no header: v70 `0x2d` body = 46_100*2+16).
const NONCE: usize = 16;
let table_len = body.len().checked_sub(NONCE)?;
let off = sd_slot_index.checked_mul(2)?;
@@ -532,10 +520,9 @@ pub fn derive_media_key_variant(
}
let nonce = variant_nonce(mkb_records).ok_or(MediaKeyVariantError::MkbIncomplete)?;
let vkd_table = variant_key_data(mkb_records).ok_or(MediaKeyVariantError::MkbIncomplete)?;
// C for the Kmp step is the per-subset-difference `0x0c` table (one 16-byte
// C per slot) — the SAME source and index `walk_processing_key` uses. `0x2d`
// holds VARIANTS + Nonce, NOT C. Fall back to `0x07`/`0x05` for the synthetic
// fixtures that keep a single cvalue there.
// C for Kmp is the per-slot `0x0c` table, same source/index as
// `walk_processing_key` uses; `0x2d` holds VARIANTS + Nonce, not C. Fall
// back to `0x07`/`0x05` for synthetic fixtures with a single cvalue.
let cvalues = mkb_find_body(mkb_records, REC_MEDIA_KEY_VARIANT_DATA)
.or_else(|| mkb_find_body(mkb_records, REC_EXPLICIT_SUBSET_DIFF))
.or_else(|| mkb_find_body(mkb_records, REC_MEDIA_KEY_DATA))
@@ -672,11 +659,9 @@ mod tests {
#[test]
fn calc_pk_from_dk_terminates_on_nonconvergent_mask() {
// Regression for the unbounded-loop hang: pick a (dev_key_v_mask,
// v_mask) pair the arithmetic `>> 1` walk can never reconcile.
// dev_key_v_mask has the MSB set, so `>> 1` sign-extends and the
// mask saturates at 0xFFFF_FFFF, never reaching a coarser v_mask.
// The 32-step bound must let this return rather than spin forever.
// Regression for the unbounded-loop hang: dev_key_v_mask has the MSB
// set, so the `>> 1` walk sign-extends and saturates at 0xFFFF_FFFF,
// never reaching v_mask. The 32-step bound must still return.
let dk = [0x11u8; 16];
let pk = calc_pk_from_dk(&dk, 0x0000_0002, 0x0000_0000, 0xFFFF_FFFE);
// Bounded exit yields *some* key; we only assert it terminated.
@@ -844,9 +829,8 @@ mod tests {
mkb.extend_from_slice(&[0x04, 0x00, 0x00, 0x09]);
mkb.extend_from_slice(&[0x03, 0x00, 0x00, 0x00, 0x02]);
// Pick a known DK; with dk.uv == MKB.uv (==2) and
// dk.u_mask_shift == MKB.u_mask_shift (==3), dev_key_v_mask
// equals the MKB's v_mask and the calc_pk_from_dk loop is a
// Known DK with dk.uv == MKB.uv (==2) and dk.u_mask_shift == MKB's
// (==3): dev_key_v_mask == MKB's v_mask, so calc_pk_from_dk is a
// no-op — Kp = aesg3(dk, 1).
let dk_bytes: [u8; 16] = [
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE,
@@ -862,12 +846,9 @@ mod tests {
aes_d_result[15] ^= 0x02;
let c_block = aes_ecb_encrypt(&kp, &aes_d_result);
// cvalues record (0x07): the per-SD C the chain reads for `Kmp`. This
// fixture has no `0x0c`, so both the walk and the chain fall back to
// `0x07` — plant the computed `c_block` HERE so `AES-D(Kp, C) XOR uv ==
// Kmp` and the chosen `kmp15` bit lands. On a variant MKB the per-match
// magic check fails, but `variant_present` is true, so the walk still
// returns the match.
// cvalues record (0x07): no `0x0c` here, so walk and chain both fall
// back to `0x07` — plant `c_block` so AES-D(Kp,C)^uv == Kmp. Magic
// check fails on variant MKBs, but `variant_present` lets it match.
mkb.extend_from_slice(&[0x07, 0x00, 0x00, 0x14]);
mkb.extend_from_slice(&c_block);
@@ -939,14 +920,9 @@ mod tests {
#[test]
fn walk_mkb_be24_middle_byte_is_honored() {
// A record longer than 255 bytes needs the MIDDLE BE24 byte: total
// length 0x00_0110 (272) is `[0x00, 0x01, 0x10]`, so a parser reading
// only the low byte sees 0x10. The HIGH byte of this length is zero, so
// this test says nothing about the `<< 16` term — that is pinned
// separately by `mkb::tests::mkb_records_honors_the_high_byte_of_the_be24_length`,
// which uses a 0x01_0004 record. (Renamed from
// `walk_mkb_be24_high_byte_is_honored`, which claimed coverage this body
// does not deliver.)
// Needs the MIDDLE BE24 byte: length 0x00_0110 (272) is [0x00,0x01,0x10],
// so a low-byte-only reader sees 0x10. HIGH byte is zero here; that term
// is pinned by `mkb::tests::mkb_records_honors_the_high_byte_of_the_be24_length`.
let total = 0x0110usize; // 272
let mut mkb = vec![0x10, 0x00, 0x01, 0x10];
mkb.resize(total, 0xAB);
@@ -1049,10 +1025,9 @@ mod tests {
#[test]
fn chain_yields_no_key_for_non_covering_pk() {
// A complete variant MKB but a Processing Key that covers no slot → no
// Km verifies → an error (never a key). A non-covering key resolves to
// ProcessingKeyUnavailable, or to a correction-mode classification if its
// Kmp happens to set the soft/online bit — either way, no key is emitted.
// Complete variant MKB, but PK covers no slot → no Km verifies → error.
// Resolves to ProcessingKeyUnavailable, or a correction-mode
// classification if Kmp sets the soft/online bit; either way, no key.
let (recs, _dk, _, _) = synthetic_variant_setup(0x00);
let out = derive_media_key_variant(&recs, &[0x11; 16]);
assert!(out.is_err(), "non-covering PK must not yield a Media Key");
@@ -1062,9 +1037,8 @@ mod tests {
#[test]
fn chain_reports_mkb_incomplete_when_nonce_missing() {
// Build a variant MKB (still variant via 0x2f, and a DK can walk it)
// but WITHOUT the 0x2d record that carries C + the trailing Nonce →
// MkbIncomplete at the variant_nonce `?`.
// Variant MKB (still variant via 0x2f, DK can walk it) but WITHOUT the
// 0x2d record carrying C + Nonce → MkbIncomplete at variant_nonce `?`.
let (recs, _dk, kp, _) = synthetic_variant_setup(0x00);
// Reconstruct bytes without the 0x2d record.
let mut mkb = Vec::new();
@@ -1088,10 +1062,9 @@ mod tests {
#[test]
fn walk_processing_key_skips_shift_32_to_63_without_panic() {
// A subset-difference u_mask_shift in 0x20..=0x3F passes the 0xC0
// revoke check but is out of range for a u32 shift. The walk must skip
// the slot (continue) and not panic / not match a wrong uv. With only
// that one bad slot, no match → None.
// u_mask_shift in 0x20..=0x3F passes the 0xC0 revoke check but is out
// of u32-shift range. Walk must skip the slot without panicking; with
// only this bad slot present, result is None.
let mut mkb = vec![
0x10, 0x00, 0x00, 0x0C, 0x48, 0x14, 0x10, 0x03, 0x00, 0x00, 0x00, 0x4D,
];
@@ -1211,22 +1184,9 @@ mod tests {
assert_eq!(err, MediaKeyVariantError::SoftCorrectionRequired);
}
// ════════════════════════════════════════════════════════════════════
// A COMPLETE variant MKB — the AACS 2.1 happy path
//
// Every other test in this module asserts an ERROR classification, so
// until now no test ever drove `derive_media_key_variant` to a Media
// Key. That left the whole success path — the VARIANTS lookup, the VKD
// selection, the final `Km` unwrap and the verify gate — pinned by
// nothing: a body that answered a constant for any of those steps still
// produced the same errors these tests expect.
//
// No real key material is involved. Every AACS 2.1 relation in the chain
// is invertible, so the fixture below picks a Media Key and a Processing
// Key and computes the MKB records that connect them, exactly as
// `derive::position_recovery_tests::plant_mkb` does for the classical
// chain.
// ════════════════════════════════════════════════════════════════════
// A COMPLETE variant MKB — the AACS 2.1 happy path. Every other test here
// asserts an ERROR, leaving the success path (VARIANTS/VKD lookup, Km
// unwrap, verify gate) unpinned; fixture inverts the chain, no real keys.
/// A planted variant MKB and the values it was built from.
struct PlantedVariant {
@@ -1278,11 +1238,9 @@ mod tests {
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
];
// `uv = 2` puts its only non-zero byte at index 15, so byte 15 is the ONE
// position where the `Km`/`Kmp` uv-XOR is observable. Its 0x02 bit is
// deliberately CLEAR: with the bit set, `km[15] ^= 2` and `km[15] |= 2`
// agree (the XOR would only be clearing a bit the OR re-sets) and an
// OR-for-XOR substitution in the final step would be invisible.
// `uv = 2` has its only non-zero byte at index 15, the one position where
// the uv-XOR is observable. Its 0x02 bit is deliberately CLEAR: if set,
// `^= 2` and `|= 2` would agree, hiding an OR-for-XOR substitution bug.
let km: [u8; 16] = [
0xC0, 0xC1, 0xC2, 0xC3, 0xC4, 0xC5, 0xC6, 0xC7, 0xC8, 0xC9, 0xCA, 0xCB, 0xCC, 0xCD,
0xCE, 0xCD,
@@ -1613,9 +1571,8 @@ mod tests {
);
// Wrong C block: EVERY one-bit neighbour must fail to produce a key.
// (Which classification it lands in depends on the two condition bits
// the perturbed Kmp happens to carry — the property being pinned is
// that none of the 128 reaches `Ok`.)
// Classification varies with the perturbed Kmp's condition bits; the
// property pinned here is that none of the 128 reaches `Ok`.
for byte in 0..16usize {
for bit in 0..8u32 {
let mut c_bad = p.c_block;
@@ -1687,20 +1644,9 @@ mod tests {
);
}
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT variant MKB driven by a real DEVICE KEY
//
// `walk_processing_key` is the DK -> Kp step that feeds the whole 2.1
// chain. Every existing test of it either asserts `None` (out-of-range
// shift, uv == 0) or asserts only that SOME match came back — none pins
// WHICH Processing Key, cvalue or slot index it returns. And every one of
// them uses a SINGLE-slot MKB, where the slot index is 0: all the
// `uvs[1 + 5*idx]` / `cvalues[idx*16..]` stride arithmetic multiplies by
// zero and any stride at all gives the same answer.
//
// This fixture puts the covering slot at index 1, behind a decoy at
// index 0, so the strides are load-bearing.
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT variant MKB driven by a real DEVICE KEY. Prior tests used
// single-slot MKBs (index 0), where stride arithmetic multiplies by zero;
// this puts the covering slot at index 1 behind a decoy, so strides matter.
/// A two-slot variant MKB whose SECOND slot is opened by a device key.
struct PlantedWalk {
@@ -1718,7 +1664,7 @@ mod tests {
/// Positions follow the same reasoning as the classical
/// `derive::position_recovery_tests::plant_mkb`: `uv = 0x0400`
/// (`u_mask_shift = 12`) with a device node of `0x0C00` satisfies the
/// [C] §3.2.4 gate — equal under `u_mask = 0xFFFF_F000`, different under
/// `[C]` §3.2.4 gate — equal under `u_mask = 0xFFFF_F000`, different under
/// `v_mask = 0xFFFF_F800`. The device key's own `uv` equals the slot's, so
/// `dev_key_v_mask == v_mask` and [`calc_pk_from_dk`] descends zero levels:
/// `Kp = AES-G3(dk, 1)`, written out explicitly below rather than taken from
@@ -1763,10 +1709,9 @@ mod tests {
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&km, &vd);
// ── C blocks. Both are built so `Kmp[15]` has the 0x02 / 0x04 condition
// bits CLEAR, so both slots run the default-KCD path to completion and
// the decoy is rejected by the terminal verify gate rather than
// short-circuiting into a correction-mode classification.
// C blocks: both keep `Kmp[15]`'s 0x02/0x04 bits CLEAR, so both slots
// run the default-KCD path and the decoy is rejected by the terminal
// verify gate instead of short-circuiting into correction-mode.
let c_for = |kmp: &[u8; 16], uv: u32| -> [u8; 16] {
let mut c_plain = *kmp;
for (b, u) in c_plain[12..16].iter_mut().zip(uv.to_be_bytes()) {
@@ -1864,7 +1809,7 @@ mod tests {
/// them — and produced a Processing Key that opens nothing.
///
/// The expected `Kp` is written as the explicit `AES-G3(dk, 1)` zero-descent
/// relation from [C] §3.2.4, not taken from the walk's own output.
/// relation from `[C]` §3.2.4, not taken from the walk's own output.
#[test]
fn walk_processing_key_returns_the_covering_slots_key_cvalue_and_index() {
let p = plant_walk_variant_mkb();
@@ -1893,7 +1838,7 @@ mod tests {
);
}
/// The gate the walk applies is [C] §3.2.4's subset-difference test, and a
/// The gate the walk applies is `[C]` §3.2.4's subset-difference test, and a
/// device key that fails it must get NO match. Pinned across all four
/// coordinates the gate reads — node, uv, u_mask_shift and the key bytes —
/// because a body that dropped any half of the gate would hand back a
@@ -2000,7 +1945,7 @@ mod tests {
}
/// The classical-magic escape hatch. On a NON-variant MKB the walk must
/// return a match only when `AES-D(Kmp, mk_dv)` opens with the [C] §3.2.5.1.4
/// return a match only when `AES-D(Kmp, mk_dv)` opens with the `[C]` §3.2.5.1.4
/// verify magic; on a variant MKB that relation does not hold (the walk
/// yields a Precursor) and the presence of `0x2d`/`0x2f` is what lets the
/// match through to the chain's own terminal gate.
@@ -2036,7 +1981,7 @@ mod tests {
/// The OTHER half of `classical_ok || variant_present`: a non-variant MKB
/// whose cvalue really does open the Verify-Media-Key magic must yield a
/// match, and the [C] §3.2.4 relation that produces the candidate — AES-D(Kp,
/// match, and the `[C]` §3.2.4 relation that produces the candidate — AES-D(Kp,
/// cvalue) with `uv` XORed into the LOW FOUR BYTES — must be computed
/// exactly.
///
@@ -2059,11 +2004,9 @@ mod tests {
];
// Zero descent ([C] §3.2.4), written out as the primitive relation.
let kp = aesg3(&dkey, 1);
// `uv = 0x0400` puts its only non-zero byte at index 14, so byte 14 is
// the ONE position where the `uv` XOR is observable at all. Its 0x04 bit
// is deliberately CLEAR here: with the bit set, `km_candidate[14] |=
// 0x04` and `^= 0x04` agree (the XOR would only be clearing a bit the OR
// re-sets), and an OR-for-XOR substitution would be invisible.
// `uv = 0x0400` has its only non-zero byte at index 14, the one position
// where the uv XOR is observable. Its 0x04 bit is CLEAR: if set, `|= 0x04`
// and `^= 0x04` would agree, hiding an OR-for-XOR substitution bug.
let mk: [u8; 16] = [
0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7A, 0x7B, 0x7C, 0x7D,
0x7A, 0x7F,
+149
View File
@@ -0,0 +1,149 @@
//! Parse `/BDMV/index.bdmv` — the BD navigation index (First Play, Top Menu,
//! and the title table), per its documented binary layout. This is read as a
//! documented binary format, never executed: every field is bounds-checked and
//! any malformed input yields `None` (the nav resolver then abstains).
//!
//! Layout: `"INDX"` + version(4) + `indexes_start`(u32 @8) + … ; at
//! `indexes_start`: `index_len`(u32), First-Play object(12), Top-Menu object(12),
//! `num_titles`(u16), then `num_titles` title objects(12 each). Every object's
//! `object_type` is the top two bits of its first byte; for an HDMV object the
//! `id_ref` into `MovieObject.bdmv` is a big-endian u16 at object offset 6.
/// One playback/title object in `index.bdmv`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum PlaybackObj {
/// HDMV title. `id_ref` indexes `MovieObject.bdmv`; `0xffff` means "no
/// object".
Hdmv { id_ref: u16 },
/// BD-J title (a Java Xlet chooses what plays — the nav VM cannot resolve
/// it and must abstain).
BdJ,
/// Unrecognised object type.
Unknown,
}
/// The parsed index: the two entry objects plus the title table (title numbers
/// `1..=titles.len()` map to `titles[i-1]`; title 0 is the Top Menu).
#[derive(Debug, Clone)]
pub(crate) struct Index {
pub first_play: PlaybackObj,
pub top_menu: PlaybackObj,
pub titles: Vec<PlaybackObj>,
}
const OBJ_LEN: usize = 12;
/// Sanity cap on the title count (real discs have well under this).
const MAX_TITLES: usize = 4096;
fn be_u16(d: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*d.get(o)?, *d.get(o + 1)?]))
}
fn be_u32(d: &[u8], o: usize) -> Option<u32> {
Some(u32::from_be_bytes([
*d.get(o)?,
*d.get(o + 1)?,
*d.get(o + 2)?,
*d.get(o + 3)?,
]))
}
/// Parse one 12-byte object starting at `o`. `object_type` is the top two bits
/// of byte 0; the object body starts at byte 4, and an HDMV `id_ref` is the
/// big-endian u16 at body offset +2 (object offset +6).
fn parse_obj(d: &[u8], o: usize) -> Option<PlaybackObj> {
let b0 = *d.get(o)?;
d.get(o + OBJ_LEN - 1)?; // require the whole 12-byte record
Some(match (b0 >> 6) & 0x3 {
1 => PlaybackObj::Hdmv {
id_ref: be_u16(d, o + 6)?,
},
2 => PlaybackObj::BdJ,
_ => PlaybackObj::Unknown,
})
}
/// Parse `index.bdmv`. Returns `None` on any structural problem.
pub(crate) fn parse(d: &[u8]) -> Option<Index> {
if d.get(0..4)? != b"INDX" {
return None;
}
// version at 4..8 ("0100"/"0200"/"0300") is not load-bearing for resolution.
let indexes_start = be_u32(d, 8)? as usize;
// At indexes_start: u32 index_len, First-Play(12), Top-Menu(12), u16 titles.
let mut o = indexes_start.checked_add(4)?;
let first_play = parse_obj(d, o)?;
o = o.checked_add(OBJ_LEN)?;
let top_menu = parse_obj(d, o)?;
o = o.checked_add(OBJ_LEN)?;
let num_titles = be_u16(d, o)? as usize;
o = o.checked_add(2)?;
if num_titles == 0 || num_titles > MAX_TITLES {
return None;
}
let span = num_titles.checked_mul(OBJ_LEN)?;
if o.checked_add(span)? > d.len() {
return None;
}
let mut titles = Vec::with_capacity(num_titles);
for i in 0..num_titles {
titles.push(parse_obj(d, o + i * OBJ_LEN)?);
}
Some(Index {
first_play,
top_menu,
titles,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a 12-byte object record: `object_type` in the top two bits of
/// byte 0, HDMV `id_ref` big-endian at offset 6.
fn hdmv_obj(id_ref: u16) -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = 1 << 6;
b[6..8].copy_from_slice(&id_ref.to_be_bytes());
b
}
fn bdj_obj() -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = 2 << 6;
b
}
fn build(first: [u8; 12], top: [u8; 12], titles: &[[u8; 12]]) -> Vec<u8> {
let indexes_start = 48u32;
let mut d = vec![0u8; indexes_start as usize];
d[0..4].copy_from_slice(b"INDX");
d[4..8].copy_from_slice(b"0300");
d[8..12].copy_from_slice(&indexes_start.to_be_bytes());
d.extend_from_slice(&0u32.to_be_bytes()); // index_len (unused by parser)
d.extend_from_slice(&first);
d.extend_from_slice(&top);
d.extend_from_slice(&(titles.len() as u16).to_be_bytes());
for t in titles {
d.extend_from_slice(t);
}
d
}
#[test]
fn parses_hdmv_first_play_and_titles() {
let d = build(hdmv_obj(0), bdj_obj(), &[bdj_obj(), bdj_obj()]);
let idx = parse(&d).expect("parses");
assert_eq!(idx.first_play, PlaybackObj::Hdmv { id_ref: 0 });
assert_eq!(idx.top_menu, PlaybackObj::BdJ);
assert_eq!(idx.titles.len(), 2);
assert_eq!(idx.titles[0], PlaybackObj::BdJ);
}
#[test]
fn rejects_bad_magic_and_truncation() {
assert!(parse(b"NOPE").is_none());
let d = build(hdmv_obj(3), hdmv_obj(0), &[hdmv_obj(1)]);
// Truncating below the declared title span must yield None, never panic.
assert!(parse(&d[..d.len() - 4]).is_none());
}
}
+147
View File
@@ -0,0 +1,147 @@
//! Parse `/BDMV/MovieObject.bdmv` — the HDMV navigation programs, per their
//! documented binary layout. Read as a documented format, never executed here:
//! bounds-checked, never panics.
//!
//! Layout: `"MOBJ"` + version(4) + reserved… ; `MovieObjects()` at byte 40 is
//! `length`(u32) + reserved(u32) + `num_objects`(u16 @48), then objects from
//! byte 50. Each object is `flags`(1) + reserved(1) + `num_cmds`(u16) followed
//! by `num_cmds` 12-byte navigation commands.
/// One decoded 12-byte navigation command. Field layout (big-endian): byte 0 =
/// `op_cnt(3) grp(2) sub_grp(3)`; byte 1 = `imm_op1(1) imm_op2(1) branch_opt(4)`
/// (low nibble); byte 2 low nibble = `cmp_opt`; byte 3 low 5 bits = `set_opt`;
/// then `dst`(u32) and `src`(u32).
#[derive(Debug, Clone, Copy)]
pub(crate) struct Cmd {
pub op_cnt: u8,
pub grp: u8,
pub sub_grp: u8,
pub imm_op1: bool,
pub imm_op2: bool,
pub branch_opt: u8,
pub cmp_opt: u8,
pub set_opt: u8,
pub dst: u32,
pub src: u32,
}
/// One HDMV navigation program.
#[derive(Debug, Clone)]
pub(crate) struct MovieObject {
pub cmds: Vec<Cmd>,
}
const CMD_LEN: usize = 12;
/// Sanity caps (real discs are far under these; some densely-branched
/// dispatchers run several thousand commands in one object).
const MAX_OBJECTS: usize = 4096;
const MAX_CMDS: usize = 200_000;
fn be_u16(d: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*d.get(o)?, *d.get(o + 1)?]))
}
/// Decode one 12-byte command. Caller guarantees `b.len() == 12`.
fn decode_cmd(b: &[u8]) -> Cmd {
let (b0, b1, b2, b3) = (b[0], b[1], b[2], b[3]);
Cmd {
op_cnt: (b0 >> 5) & 0x7,
grp: (b0 >> 3) & 0x3,
sub_grp: b0 & 0x7,
imm_op1: (b1 >> 7) & 1 == 1,
imm_op2: (b1 >> 6) & 1 == 1,
branch_opt: b1 & 0x0f,
cmp_opt: b2 & 0x0f,
set_opt: b3 & 0x1f,
dst: u32::from_be_bytes([b[4], b[5], b[6], b[7]]),
src: u32::from_be_bytes([b[8], b[9], b[10], b[11]]),
}
}
/// Parse `MovieObject.bdmv` into its programs. Returns `None` on any structural
/// problem.
pub(crate) fn parse(d: &[u8]) -> Option<Vec<MovieObject>> {
if d.get(0..4)? != b"MOBJ" {
return None;
}
let num = be_u16(d, 48)? as usize;
if num > MAX_OBJECTS {
return None;
}
let mut off = 50usize;
let mut objs = Vec::with_capacity(num);
for _ in 0..num {
// Object header: flags(1) + reserved(1) + num_cmds(u16).
let num_cmds = be_u16(d, off + 2)? as usize;
off = off.checked_add(4)?;
if num_cmds > MAX_CMDS {
return None;
}
let span = num_cmds.checked_mul(CMD_LEN)?;
let end = off.checked_add(span)?;
if end > d.len() {
return None;
}
let mut cmds = Vec::with_capacity(num_cmds);
for i in 0..num_cmds {
let s = off + i * CMD_LEN;
cmds.push(decode_cmd(&d[s..s + CMD_LEN]));
}
off = end;
objs.push(MovieObject { cmds });
}
Some(objs)
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
/// Encode one command from its opcode bytes + operands.
pub(crate) fn cmd(b0: u8, b1: u8, b2: u8, b3: u8, dst: u32, src: u32) -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = b0;
b[1] = b1;
b[2] = b2;
b[3] = b3;
b[4..8].copy_from_slice(&dst.to_be_bytes());
b[8..12].copy_from_slice(&src.to_be_bytes());
b
}
pub(crate) fn build(objects: &[&[[u8; 12]]]) -> Vec<u8> {
let mut d = vec![0u8; 50];
d[0..4].copy_from_slice(b"MOBJ");
d[48..50].copy_from_slice(&(objects.len() as u16).to_be_bytes());
for obj in objects {
d.push(0); // flags
d.push(0); // reserved
d.extend_from_slice(&(obj.len() as u16).to_be_bytes());
for c in *obj {
d.extend_from_slice(c);
}
}
d
}
#[test]
fn parses_objects_and_commands() {
// op_cnt=1, grp=BRANCH(0), sub_grp=PLAY(2), branch_opt=PLAY_PL(0), imm dst.
let b0 = (1 << 5) | 2;
let play = cmd(b0, 0x80, 0, 0, 11, 0);
let d = build(&[&[play]]);
let objs = parse(&d).expect("parses");
assert_eq!(objs.len(), 1);
assert_eq!(objs[0].cmds.len(), 1);
assert_eq!(objs[0].cmds[0].dst, 11);
assert_eq!(objs[0].cmds[0].sub_grp, 2);
}
#[test]
fn rejects_truncation() {
let play = cmd((1 << 5) | 2, 0x80, 0, 0, 11, 0);
let d = build(&[&[play]]);
assert!(parse(&d[..d.len() - 3]).is_none());
assert!(parse(b"NOPE").is_none());
}
}
+50
View File
@@ -0,0 +1,50 @@
//! BD/UHD HDMV navigation resolver — "mimic a real player" for main-feature
//! title selection. It reads `/BDMV/index.bdmv` + `/BDMV/MovieObject.bdmv` and
//! runs a faithful, bounded HDMV navigation VM to find the playlist the disc's
//! own First-Play navigation plays as the feature.
//!
//! There is no per-disc special-casing: one correctly-booted VM resolves
//! densely-branched dispatchers as well as simple ones. The honest boundary is
//! BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
//! on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
//! and selection falls back to the structural/heuristic order.
//!
//! Contract: read-only, bounded, and never panics or hard-fails.
pub(crate) mod index;
pub(crate) mod mobj;
pub(crate) mod vm;
use crate::sector::SectorSource;
use crate::udf::UdfFs;
/// Resolve the playlist id the disc's First-Play navigation plays as the
/// feature, restricted to ids the caller marks as feature candidates
/// (`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
/// short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J discs,
/// missing/malformed nav data, or when navigation does not converge on a
/// candidate.
pub(crate) fn resolve_feature(
reader: &mut dyn SectorSource,
udf: &UdfFs,
is_feature_candidate: impl Fn(u16) -> bool,
) -> Option<u16> {
// Belt-and-suspenders: the parsers and VM are written panic-free and bounded,
// but a navigation resolver must NEVER take down a scan — swallow any
// unexpected panic and abstain.
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
resolve_inner(reader, udf, &is_feature_candidate)
}))
.ok()
.flatten()
}
fn resolve_inner(
reader: &mut dyn SectorSource,
udf: &UdfFs,
is_feature_candidate: &dyn Fn(u16) -> bool,
) -> Option<u16> {
let index = index::parse(&udf.read_file(reader, "/BDMV/index.bdmv").ok()?)?;
let mobjs = mobj::parse(&udf.read_file(reader, "/BDMV/MovieObject.bdmv").ok()?)?;
vm::resolve(&index, &mobjs, is_feature_candidate)
}
+468
View File
@@ -0,0 +1,468 @@
//! The HDMV navigation VM — a faithful, bounded re-implementation of the subset
//! of HDMV navigation semantics needed to follow First-Play to the feature
//! `PlayPlayList`. It "mimics a real player": correct power-on register state is
//! what lets a densely-branched dispatcher converge on the feature rather than
//! loop. Never panics; hard step/switch caps.
//!
//! It resolves generically — there is no per-disc special-casing. When
//! First-Play is a BD-J title (the feature is chosen by a Java Xlet), or the
//! program does not reach a `PlayPL` on a caller-approved feature candidate, it
//! abstains (`None`) and selection falls back to the structural/heuristic order.
use super::index::{Index, PlaybackObj};
use super::mobj::MovieObject;
/// Top bit of an operand selects PSR (else GPR).
const PSR_FLAG: u32 = 0x8000_0000;
/// Upper bound on VM steps before declaring non-convergence.
const MAX_STEPS: usize = 1_000_000;
/// Cap on object/title switches — a dispatcher runs in one object; this only
/// bounds pathological jump chains.
const MAX_SWITCHES: usize = 1024;
/// BD player power-on register defaults (indices 0..=61); unlisted = 0. The
/// load-bearing values are PSR4/5 = `0xffff` ("no title/chapter selected") and
/// PSR6/7/8 = 0, which steer dispatcher compares; the rest are set for fidelity
/// with real player behavior.
fn psr_init() -> [u32; 128] {
let mut p = [0u32; 128];
let vals: &[(usize, u32)] = &[
(0, 1),
(1, 0xff),
(2, 0x0fff_0fff),
(3, 1),
(4, 0xffff),
(5, 0xffff),
(10, 0xffff),
(12, 0xff),
(13, 0xff),
(14, 0xffff),
(15, 0xAAAA),
(16, 0x00ff_ffff),
(17, 0x00ff_ffff),
(18, 0x00ff_ffff),
(19, 0xffff),
(20, 2),
(29, 0x0000_0003),
(30, 0x0001_ffff),
(31, 0x0003_0200),
(36, 0xffff),
(37, 0xffff),
(42, 0xffff),
(44, 0xff),
];
for &(i, v) in vals {
p[i] = v;
}
for slot in p.iter_mut().take(62).skip(48) {
*slot = 0xffff_ffff;
}
p
}
struct Vm<'a> {
mobjs: &'a [MovieObject],
index: &'a Index,
gpr: [u32; 4096],
psr: [u32; 128],
}
impl Vm<'_> {
fn rd(&self, val: u32) -> u32 {
if val & PSR_FLAG != 0 {
self.psr[(val & 0x7f) as usize]
} else {
self.gpr[(val & 0xfff) as usize]
}
}
fn wr(&mut self, val: u32, x: u32) {
// Per the HDMV nav spec, a store to a PSR-tagged register is refused;
// only GPR writes take effect. PSR values come only from power-on init,
// never from a nav SET/SWAP operand.
if val & PSR_FLAG == 0 {
self.gpr[(val & 0xfff) as usize] = x;
}
}
fn fetch(&self, imm: bool, raw: u32) -> u32 {
if imm { raw } else { self.rd(raw) }
}
/// Resolve a JumpTitle target: title 0 = Top Menu, `1..=N` = `titles[i-1]`,
/// `0xFFFF` = First Play (BD-ROM title numbering). Any other value is an
/// invalid title reference and abstains (`None`).
fn title_obj(&self, title: u32) -> Option<PlaybackObj> {
let n = self.index.titles.len() as u32;
if title == 0 {
Some(self.index.top_menu)
} else if title <= n {
self.index.titles.get((title - 1) as usize).copied()
} else if title == 0xFFFF {
Some(self.index.first_play)
} else {
None
}
}
}
/// Run First-Play and return the first `PlayPL`/`PlayPL_PM`/`PlayPL_PI` whose
/// playlist id passes `is_feature`. `None` = BD-J boundary, non-convergence, or
/// no approved feature playlist reached.
pub(crate) fn resolve(
index: &Index,
mobjs: &[MovieObject],
is_feature: &dyn Fn(u16) -> bool,
) -> Option<u16> {
// Enter at First-Play when it is an HDMV object that exists; otherwise abstain
// (BD-J boundary, or `0xffff` "no object"). We don't chase Top-Menu's "Play
// Movie" target here — it lives in menu .m2ts IG button commands (future work).
let start = match index.first_play {
PlaybackObj::Hdmv { id_ref } if (id_ref as usize) < mobjs.len() => id_ref as usize,
_ => return None,
};
let mut vm = Vm {
mobjs,
index,
gpr: [0; 4096],
psr: psr_init(),
};
run(&mut vm, start, is_feature)
}
fn run(vm: &mut Vm, mut obj_id: usize, is_feature: &dyn Fn(u16) -> bool) -> Option<u16> {
let mut pc = 0usize;
let mut steps = 0usize;
let mut switches = 0usize;
loop {
steps += 1;
if steps > MAX_STEPS {
return None;
}
let obj = vm.mobjs.get(obj_id)?;
// Ran off the end without a feature PlayPL → abstain.
let c = *obj.cmds.get(pc)?;
let mut npc = pc + 1;
let dst = if c.op_cnt > 0 {
vm.fetch(c.imm_op1, c.dst)
} else {
0
};
let src = if c.op_cnt > 1 {
vm.fetch(c.imm_op2, c.src)
} else {
0
};
match c.grp {
// BRANCH
0 => match c.sub_grp {
// GOTO
0 => match c.branch_opt {
0x01 => npc = dst as usize, // GOTO
0x02 => return None, // BREAK — terminate
_ => {} // NOP / other
},
// JUMP
1 => match c.branch_opt {
0x00 | 0x02 => {
// JumpObject / CallObject (return address not modelled —
// for feature resolution we only follow the play path).
switches += 1;
if switches > MAX_SWITCHES || dst as usize >= vm.mobjs.len() {
return None;
}
obj_id = dst as usize;
pc = 0;
continue;
}
0x01 | 0x03 => {
// JumpTitle / CallTitle — resolve through the index table.
switches += 1;
if switches > MAX_SWITCHES {
return None;
}
match vm.title_obj(dst) {
Some(PlaybackObj::Hdmv { id_ref })
if (id_ref as usize) < vm.mobjs.len() =>
{
obj_id = id_ref as usize;
pc = 0;
continue;
}
// BD-J / unknown / invalid title → abstain.
_ => return None,
}
}
_ => {} // RESUME / other — fall through
},
// PLAY: PlayPL / PlayPL_PI / PlayPL_PM emit a playlist id (the
// rest — Terminate / Link — fall through). A logo/pre-roll whose
// id isn't a feature candidate lets autoplay resume at pc + 1.
2 if matches!(c.branch_opt, 0x00..=0x02) => {
let id = dst as u16;
if is_feature(id) {
return Some(id);
}
}
_ => {}
},
// CMP — skip the next command when the compare is false.
1 => {
let truth = match c.cmp_opt {
0x01 => (dst & !src) == 0,
0x02 => dst == src,
0x03 => dst != src,
0x04 => dst >= src,
0x05 => dst > src,
0x06 => dst <= src,
0x07 => dst < src,
_ => true,
};
if !truth {
npc = pc + 2;
}
}
// SET (sub_grp 0). SETSYSTEM (sub_grp 1) only mutates system PSRs
// whose values the feature path does not branch on — skip it.
2 if c.sub_grp == 0 => {
let r: Option<u32> = match c.set_opt {
0x01 => Some(src), // MOVE
0x02 => {
// SWAP exchanges the two operands; a store to an operand
// flagged immediate is refused, and `wr` already drops
// PSR stores.
if !c.imm_op1 {
vm.wr(c.dst, src);
}
if !c.imm_op2 {
vm.wr(c.src, dst);
}
None
}
0x03 => Some(dst.wrapping_add(src)),
0x04 => Some(dst.saturating_sub(src)),
0x05 => Some(dst.wrapping_mul(src)),
0x06 => Some(dst.checked_div(src).unwrap_or(0xffff_ffff)),
0x07 => Some(dst.checked_rem(src).unwrap_or(0xffff_ffff)),
0x08 => Some(dst), // RND — deterministic stand-in
0x09 => Some(dst & src),
0x0a => Some(dst | src),
0x0b => Some(dst ^ src),
0x0c => Some(dst | (1u32 << (src & 31))),
0x0d => Some(dst & !(1u32 << (src & 31))),
0x0e => Some(dst.wrapping_shl(src & 31)),
0x0f => Some(dst.wrapping_shr(src & 31)),
_ => None,
};
if let Some(r) = r
&& !c.imm_op1
{
vm.wr(c.dst, r);
}
}
_ => {}
}
pc = npc;
}
}
#[cfg(test)]
mod tests {
use super::super::index::{Index, PlaybackObj};
use super::super::mobj::{self, tests::build, tests::cmd};
use super::*;
fn play_pl(id: u16) -> [u8; 12] {
// op_cnt=1, grp=BRANCH(0), sub_grp=PLAY(2), branch_opt=PLAY_PL(0), imm dst.
cmd((1 << 5) | 2, 0x80, 0, 0, id as u32, 0)
}
fn jump_object(obj: u16) -> [u8; 12] {
// grp=BRANCH(0), sub_grp=JUMP(1), branch_opt=JUMP_OBJECT(0), imm dst.
cmd((1 << 5) | 1, 0x80, 0, 0, obj as u32, 0)
}
fn jump_title(title: u16) -> [u8; 12] {
// sub_grp=JUMP(1), branch_opt=JUMP_TITLE(1), imm dst.
cmd((1 << 5) | 1, 0x81, 0, 0, title as u32, 0)
}
fn set_move_gpr(reg: u16, imm: u16) -> [u8; 12] {
// grp=SET(2), sub_grp=SET(0), set_opt=MOVE(1); dst=reg (GPR), imm src.
// op_cnt=2, imm_op2=1 (src immediate), imm_op1=0 (dst is a register).
cmd((2 << 5) | (2 << 3), 0x40, 0, 0x01, reg as u32, imm as u32)
}
fn play_pl_reg(reg: u16) -> [u8; 12] {
// PLAY_PL with dst = GPR[reg] (op_cnt=1, imm_op1=0).
cmd((1 << 5) | 2, 0x00, 0, 0, reg as u32, 0)
}
fn idx(first: PlaybackObj, titles: Vec<PlaybackObj>) -> Index {
Index {
first_play: first,
top_menu: PlaybackObj::BdJ,
titles,
}
}
#[test]
fn resolves_immediate_playpl() {
// First-Play HDMV obj0 → JumpObject 1 → PlayPL 11.
let d = build(&[&[jump_object(1)], &[play_pl(11)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
assert_eq!(resolve(&index, &mobjs, &|id| id == 11), Some(11));
}
#[test]
fn skips_non_feature_playpl_then_returns_feature() {
// Autoplay chain: logo PlayPL 99 (not a candidate) → feature PlayPL 1.
let d = build(&[&[play_pl(99), play_pl(1)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
// Only 1 is a feature candidate; 99 is a logo.
assert_eq!(resolve(&index, &mobjs, &|id| id == 1), Some(1));
}
#[test]
fn abstains_when_first_play_jumps_to_bdj_title() {
// Computed-title-number shape: First-Play computes a title number in a
// GPR and JumpTitles to it; the target title is BD-J → abstain (None).
let d = build(&[&[set_move_gpr(0xFEB & 0xfff, 2), {
// JumpTitle with dst = GPR[0xFEB].
cmd((1 << 5) | 1, 0x01, 0, 0, 0x0000_0FEB, 0)
}]]);
let mobjs = mobj::parse(&d).unwrap();
// title 2 → titles[1] = BD-J.
let index = idx(
PlaybackObj::Hdmv { id_ref: 0 },
vec![PlaybackObj::Hdmv { id_ref: 5 }, PlaybackObj::BdJ],
);
assert_eq!(resolve(&index, &mobjs, &|_| true), None);
}
#[test]
fn abstains_when_first_play_is_bdj() {
let d = build(&[&[play_pl(1)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::BdJ, vec![]);
assert_eq!(resolve(&index, &mobjs, &|_| true), None);
}
#[test]
fn jumptitle_first_play_is_0xffff_not_n_plus_1() {
// BD-ROM title numbering: 0 = Top Menu, 0xFFFF = First Play,
// 1..=N = titles. `N+1` is an INVALID title ref.
let index = idx(
PlaybackObj::Hdmv { id_ref: 7 },
vec![PlaybackObj::Hdmv { id_ref: 1 }],
);
let vm = Vm {
mobjs: &[],
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
assert_eq!(vm.title_obj(0), Some(index.top_menu), "title 0 = Top Menu");
assert_eq!(
vm.title_obj(1),
Some(PlaybackObj::Hdmv { id_ref: 1 }),
"title 1 = titles[0]"
);
assert_eq!(
vm.title_obj(0xFFFF),
Some(PlaybackObj::Hdmv { id_ref: 7 }),
"0xFFFF = First Play (abstained wrongly before the fix)"
);
assert_eq!(
vm.title_obj(2),
None,
"N+1 is an invalid title ref, not First Play (resolved wrongly before the fix)"
);
}
#[test]
fn set_to_psr_is_refused() {
// SET MOVE to PSR6 must be a no-op (PSR stores are refused by spec).
// Program: MOVE PSR6<-5; CMP PSR6==5; PlayPL 100 else 200. Refused write
// keeps PSR6 at 0 -> PlayPL 200; an illegal write would wrongly pick 100.
let set_psr6 = cmd((2 << 5) | (2 << 3), 0x40, 0, 0x01, 0x8000_0006, 5);
let cmp_psr6_eq_5 = cmd((2 << 5) | (1 << 3), 0x40, 0x02, 0, 0x8000_0006, 5);
let d = build(&[&[set_psr6, cmp_psr6_eq_5, play_pl(100), play_pl(200)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
assert_eq!(
resolve(&index, &mobjs, &|id| id == 100 || id == 200),
Some(200),
"PSR store must be refused, so the compare is false and 200 is reached"
);
}
#[test]
fn register_fold_and_cmp_skip_reach_playpl() {
// SET GPR[3]=7; CMP GPR[3]==7 (true → do NOT skip); PlayPL GPR[3].
// grp=CMP(1), op_cnt=2, imm_op2=1 (src=7 immediate), cmp_opt=EQ(2); dst=GPR3.
let cmp_eq_reg = cmd((2 << 5) | (1 << 3), 0x40, 0x02, 0, 3, 0x0000_0007);
let d = build(&[&[set_move_gpr(3, 7), cmp_eq_reg, play_pl_reg(3), play_pl(999)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
// If the compare wrongly skipped, we'd hit PlayPL 999 instead of 7.
assert_eq!(resolve(&index, &mobjs, &|id| id == 7 || id == 999), Some(7));
}
#[test]
fn jumptitle_namespace_differs_from_jumpobject() {
// JumpTitle 1 → titles[0] = HDMV obj 1 (which PlayPLs 42). Proves title
// numbers resolve through the index, not as object indices.
let d = build(&[&[jump_title(1)], &[play_pl(42)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(
PlaybackObj::Hdmv { id_ref: 0 },
vec![PlaybackObj::Hdmv { id_ref: 1 }],
);
assert_eq!(resolve(&index, &mobjs, &|id| id == 42), Some(42));
}
#[test]
fn swap_exchanges_two_registers() {
// MOVE g0=11; MOVE g1=22; SWAP g0<->g1; PlayPL GPR[0]. SWAP: grp=SET(2),
// sub_grp=SET(0), set_opt=SWAP(2), both operands registers (imm_op*=0).
let swap01 = cmd((2 << 5) | (2 << 3), 0x00, 0, 0x02, 0, 1);
let d = build(&[&[
set_move_gpr(0, 11),
set_move_gpr(1, 22),
swap01,
play_pl_reg(0),
]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
let mut vm = Vm {
mobjs: &mobjs,
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
// After the swap GPR0 holds the old GPR1 (22) and PlayPL plays it.
assert_eq!(run(&mut vm, 0, &|id| id == 22), Some(22));
// Full exchange: GPR1 now holds the old GPR0 (11).
assert_eq!(vm.gpr[1], 11);
}
#[test]
fn swap_immediate_operand_is_not_written_back() {
// SWAP dst=GPR0 with src=IMMEDIATE 5 (imm_op1=0, imm_op2=1) must refuse
// writing to the aliased register: GPR[5] stays 77, only GPR0 gets 5.
// Dropping the `imm_op1`/`imm_op2` guard makes this assertion fail.
let swap_imm = cmd((2 << 5) | (2 << 3), 0x40, 0, 0x02, 0, 5);
let d = build(&[&[set_move_gpr(5, 77), swap_imm]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
let mut vm = Vm {
mobjs: &mobjs,
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
// No PlayPL → program runs off the end (None); inspect registers after.
assert_eq!(run(&mut vm, 0, &|_| false), None);
assert_eq!(
vm.gpr[5], 77,
"immediate operand must not be a write target"
);
assert_eq!(vm.gpr[0], 5, "register operand receives the immediate");
}
}
+8 -17
View File
@@ -4,7 +4,7 @@
//! The EP (Entry Point) map provides timestamp → SPN mapping.
//! SPN × 192 = byte offset in the m2ts file.
//!
//! Reference: https://github.com/lw/BluRay/wiki/CLPI
//! Format is documented in the BD-ROM Clip Information (CLPI) specification.
use crate::error::{Error, Result};
@@ -55,10 +55,9 @@ pub fn parse(data: &[u8]) -> Result<ClipInfo> {
0
};
// Parse ProgramInfo (per-stream language + codec). Best-effort:
// malformed program_info doesn't fail the parse, just gives an
// empty streams list. EP map is unaffected — sector-range lookups
// continue to work.
// Parse ProgramInfo (per-stream language + codec), best-effort: malformed
// program_info doesn't fail the parse, just yields an empty streams list.
// EP map is unaffected — sector-range lookups still work.
let streams = if prog_info_start > 0 && prog_info_start + 6 < data.len() {
parse_program_info(&data[prog_info_start..])
} else {
@@ -172,16 +171,9 @@ mod tests {
/// Build a minimal CLPI binary.
/// `cpi_data` is the raw CPI section bytes (starting with the 4-byte CPI length).
fn build_clpi(source_packet_count: u32, cpi_data: Option<&[u8]>) -> Vec<u8> {
// We need at least 60 bytes for the header area.
// Offsets:
// 0..4: "HDMV"
// 4..8: "0200"
// 8..12: seq_info_start (unused, set to 0)
// 12..16: prog_info_start (unused, set to 0)
// 16..20: cpi_start
// 20..40: reserved/padding
// 40..56: ClipInfo section area (length + stuff before source_packet_count)
// 56..60: source_packet_count
// Need at least 60 bytes for the header area: "HDMV"/"0200" magic,
// seq_info_start/prog_info_start (unused here), cpi_start, reserved
// padding, then the ClipInfo section ending in source_packet_count.
let cpi_start: u32 = if cpi_data.is_some() { 60 } else { 0 };
@@ -230,8 +222,7 @@ mod tests {
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the BD-ROM CLPI spec
// (https://github.com/lw/BluRay/wiki/CLPI).
// Added hardening tests, grounded in the BD-ROM CLPI spec byte layout.
// ─────────────────────────────────────────────────────────────────────
/// Build a ProgramInfo section. `streams` = Vec<(pid, sci_bytes)>.
+31 -56
View File
@@ -1,9 +1,9 @@
//! CSS title-key recovery — Frank A. Stevenson's divide-and-conquer attack
//! (1999), implemented from his published cryptanalysis ("Cryptanalysis of
//! Contents Scrambling System"). It recovers the 5-byte CSS title key from a
//! single scrambled DVD sector with no player keys and no disc-key crack, using
//! only known plaintext. Implemented from that public description; nothing here
//! is copied or translated from any particular CSS software.
//! CSS title-key recovery — a divide-and-conquer known-plaintext attack,
//! implemented from the openly published CSS cryptanalysis literature. It
//! recovers the 5-byte CSS title key from a single scrambled DVD sector with no
//! player keys and no disc-key crack, using only known plaintext. Implemented
//! from that public description; nothing here is copied or translated from any
//! particular CSS software.
//!
//! # The cipher this attacks
//!
@@ -41,8 +41,8 @@ const ENCRYPTED_START: usize = 0x80; // byte 128
const SEED_OFFSET: usize = 0x54; // sector seed at bytes 0x54-0x58
const FLAG_BYTE: usize = 0x14;
/// Recover the title key from cipher + known plaintext (the core of Stevenson's
/// attack). `crypted` is the ciphertext starting at sector byte 0x80;
/// Recover the title key from cipher + known plaintext (the core of the
/// recovery). `crypted` is the ciphertext starting at sector byte 0x80;
/// `decrypted` is the matching known plaintext; `seed` is `sector[0x54..0x59]`.
/// On success returns the recovered 5-byte title key; `None` if no LFSR seed
/// reproduces the keystream.
@@ -210,8 +210,8 @@ fn descramble_matches(sector: &[u8], title: &[u8; 5], plain: &[u8]) -> bool {
}
/// Find a repeating pattern just before the encrypted region and assume the
/// plaintext at 0x80 continues it — the known-plaintext step of Stevenson's
/// attack. Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// plaintext at 0x80 continues it — the known-plaintext step of the recovery.
/// Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// with a cycle length in 2..0x2F. If the run is long enough (`plen > 3` and at
/// least two full cycles), the known plaintext at 0x80 is taken to be the
/// periodic run continuing forward, and [`recover_title_key_from_plain`] is
@@ -224,10 +224,9 @@ pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
return None;
}
// Runaway guard: a single sector's crack is a bounded 2^16 LFSR search and
// should finish in well under a second on any modern CPU. If it ever
// exceeds ~2s wall-clock, something pathological is happening — log it so a
// hang is never silent.
// Runaway guard: this crack is a bounded 2^16 LFSR search, done well under 1s
// on any modern CPU. If it ever exceeds ~2s, something pathological is
// happening — log it so a hang is never silent.
let crack_t0 = std::time::Instant::now();
let result = crack_title_key_inner(sector);
@@ -280,21 +279,13 @@ pub(crate) fn attack_crib(sector: &[u8]) -> Option<[u8; 10]> {
// Need at least a few repeated bytes and at least one full cycle.
if best_plen > 3 && best_p > 0 && best_plen / best_p >= 2 {
// The known plaintext is the periodic run continuing past 0x80. The
// crib starts at `0x80 - (best_plen/best_p)*best_p` and continues
// through the encrypted region; the bytes at and after 0x80 are the
// predicted plaintext (the pattern repeats with period best_p).
// Crib starts at `0x80 - cycles*best_p`; bytes at/after 0x80 are
// predicted by the run repeating with period best_p.
let cycles = best_plen / best_p;
let plain_start = 0x80 - cycles * best_p;
// Each predicted byte is the run sample one or more periods back:
// `sec[plain_start + (i % best_p)]`. For in-run offsets
// (`plain_start + i < 0x80`) the run is exactly periodic, so this
// equals `sec[plain_start + i]`; for offsets at/after 0x80 the raw
// byte is ciphertext, so we MUST wrap within the period rather than
// read it. (Reading `&sec[plain_start..+10]` directly — as before —
// pulled ciphertext into the crib whenever the run covered fewer than
// 10 bytes before 0x80, producing false-negative key recovery.)
// Must wrap within the period (not read `&sec[plain_start..+10]`
// directly), since past 0x80 the raw byte is ciphertext, not plaintext.
let mut plain = [0u8; 10];
for (i, p) in plain.iter_mut().enumerate() {
*p = sector[plain_start + (i % best_p)];
@@ -359,14 +350,9 @@ mod tests {
let mut plaintext = vec![0u8; SECTOR_BYTES];
plaintext[FLAG_BYTE] = 0x10;
// A clean periodic run occupying the tail of the cleartext header
// (RUN_START..0x80) and continuing into the encrypted region. This
// mirrors a real VOB: a periodic data run just before the scrambled
// part. The run must NOT overlap the seed bytes (0x54..0x59), or the
// the crib detector would break mid-run. The phase is anchored to
// offset 0 so the run is consistent across the 0x80 boundary.
// Just above the seed (0x54..0x59); gives a 39-byte run (0x59..0x80)
// — enough for >=2 cycles of every tested period (<=19).
// Periodic run over the cleartext header tail (RUN_START..0x80) into
// the encrypted region, mirroring a real VOB. Must not overlap the seed
// bytes (0x54..0x59); RUN_START=0x59 gives >=2 cycles of every period.
const RUN_START: usize = 0x59;
let pat: Vec<u8> = (0..period)
.map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A)
@@ -754,11 +740,9 @@ mod tests {
None,
"scramble bits clear → no crack, even though one would succeed"
);
// `attack_crib` carries its own copy of the same gate, and it is the one
// that actually stops the crack (`crack_title_key`'s is defensive
// duplication). The crib doubles as the decrypt path's cached-key
// oracle, so a widened mask there would hand that path a "predicted
// plaintext" for sectors that were never scrambled.
// `attack_crib` carries its own copy of the same gate and is the one that
// actually stops the crack (`crack_title_key`'s is defensive duplication);
// it also doubles as the decrypt path's cached-key oracle.
assert_eq!(
attack_crib(&periodic),
None,
@@ -845,14 +829,9 @@ mod tests {
);
}
// ── attack_crib: known-answer vectors ──────────────────────────────────
//
// `attack_crib` is BOTH the cracker's known plaintext and the decrypt
// path's "did the cached key descramble correctly?" oracle. Until now it
// was only ever exercised end-to-end through `crack_title_key`, on a
// fixture whose periodic run covered 39 bytes (0x59..0x80) — long enough
// that the run start, the cycle count and the `i % best_p` wrap were all
// slack. A crib that silently drifts costs a rip its title key.
// ── attack_crib: known-answer vectors ──────────────────────
// Both the cracker's known plaintext and the decrypt path's cached-key
// oracle; was only exercised indirectly, leaving run/cycle/wrap slack.
/// Build a sector whose clear header ends in a `period`-length repeating
/// run of exactly `run_len` bytes immediately before 0x80.
@@ -1011,10 +990,8 @@ mod tests {
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
// The FLAG byte sits inside the header at 0x14, so it interrupts the
// pattern there; re-lay it and accept that 0x14 breaks the run — the
// scan still reaches offset 0x15 - 1 = 0x14 going backwards, i.e.
// j = 0x7f - 0x14 = 0x6b, well short of the bound. Instead put the
// The FLAG byte at 0x14 would interrupt the pattern, but j only needs to
// reach 0x7f - 0x14 = 0x6b, well short of the bound. So instead put the
// scramble flag bits into a byte value that IS the pattern's.
sector[FLAG_BYTE] = pat[FLAG_BYTE % period];
assert_ne!(
@@ -1081,11 +1058,9 @@ mod tests {
"{n} plaintext bytes is fewer than the ten the cipher iterates"
);
}
// Exactly ten of each is ACCEPTED as far as the search — the boundary is
// `< 10`, not `<= 10`. (Whether this particular keystream has a seed is
// immaterial; what must not happen is an early `None` from the guard.)
// Proven through the round-trip fixture, whose inputs are exactly ten
// bytes and which does recover its key.
// Exactly ten of each is ACCEPTED — the boundary is `< 10`, not `<= 10`.
// What must not happen is an early `None` from the guard; proven via the
// round-trip fixture, whose inputs are exactly ten bytes.
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
+2 -3
View File
@@ -1,9 +1,8 @@
//! CSS content cipher — an independent implementation of the publicly
//! documented Content Scramble System stream cipher.
//!
//! The algorithm is the one recovered and published in Frank A. Stevenson's
//! 1999 cryptanalysis ("Cryptanalysis of Contents Scrambling System") and
//! described in the open CSS literature. It is implemented here from that public
//! The algorithm is the publicly documented CSS stream cipher, described in the
//! open CSS cryptanalysis literature. It is implemented here from that public
//! description; its constants (see [`super::tables`]) are the cipher's own
//! defined values. Nothing in this file is copied or translated from any
//! particular CSS software.
+67 -160
View File
@@ -2,9 +2,9 @@
//!
//! CSS uses a weak 40-bit LFSR stream cipher (broken since 1999).
//!
//! The title key is recovered keylessly: [`crack_key`] runs the Stevenson
//! known-plaintext attack (see the [`stevenson`] module) on the scrambled
//! data, needing no player keys, disc-key crack, or external key file.
//! The title key is recovered keylessly: [`crack_key`] recovers it directly
//! from the scrambled data (see the [`keyless`] module),
//! needing no player keys, disc-key recovery, or external key file.
//! Sectors are then decrypted with [`descramble_sector`].
//!
//! Usage:
@@ -14,8 +14,8 @@
//! }
//! ```
pub mod keyless;
pub mod lfsr;
pub mod stevenson;
pub(crate) mod tables;
use crate::disc::Extent;
@@ -56,7 +56,7 @@ impl std::fmt::Debug for CssState {
}
/// Recover the CSS title key with no keys, by scanning scrambled sectors and
/// running the Stevenson known-plaintext attack (see the [`stevenson`] module).
/// running the known-plaintext attack (see the [`keyless`] module).
///
/// The crib comes from the periodic-run detector: a scrambled sector's cleartext region
/// (bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
@@ -88,7 +88,7 @@ pub fn crack_key(
/// scanned extents (`is_scrambled_pack` never true): the content is genuinely
/// plaintext, so proceeding without a key is correct.
/// - [`CrackOutcome::ScrambledUncracked`] — scrambled sectors WERE seen but no
/// key could be recovered (the Stevenson attack found no crackable crib, or
/// key could be recovered (the recovery found no crackable crib, or
/// the scrambled region was unreadable). The content is encrypted; muxing it
/// as plaintext would emit garbage, so callers MUST surface a hard error
/// instead of falling through to "unencrypted" — the per-title
@@ -167,9 +167,8 @@ pub(crate) fn resolve_dvd_title_key(
halt: Option<&crate::halt::Halt>,
) -> std::io::Result<()> {
// `--raw` = deliberate ciphertext passthrough: never crack or descramble, and
// never hard-fail on scrambled-uncrackable — the user asked for the scrambled
// bytes. (In raw mode the caller hands us `None` on purpose; without this
// guard we'd install a real key and silently DECRYPT, or abort a raw mux.)
// never hard-fail on scrambled-uncrackable. Caller hands us `None` on
// purpose; without this guard we'd silently DECRYPT or abort a raw mux.
if raw {
return Ok(());
}
@@ -179,12 +178,9 @@ pub(crate) fn resolve_dvd_title_key(
// `halt` threads the caller's cancellation token so /api/stop can
// interrupt a long crack scan (the old scan-time crack honored it too).
let outcome = crack_key_outcome(reader, extents, batch_sectors, halt);
// A cancelled crack breaks out early, so its outcome is a TRUNCATED scan
// — not a real verdict. Interpreting it would either hard-fail a good disc
// as `ScrambledUncracked` (quarantining staging on a Stop) or, worse,
// read a half-scanned title as `Unencrypted` and mux scrambled bytes as
// plaintext. Surface the cancellation as `Halted` so the caller takes its
// graceful-stop path instead of trusting the partial outcome.
// A cancelled crack's outcome is a TRUNCATED scan, not a real verdict —
// surface it as `Halted` rather than trusting a partial `Unencrypted`
// or `ScrambledUncracked` and taking the wrong path.
if halt.map(|h| h.is_cancelled()).unwrap_or(false) {
return Err(crate::error::Error::Halted.into());
}
@@ -213,13 +209,9 @@ fn crack_key_scan(
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
) -> CrackOutcome {
// Batch the reads: a live optical drive at 1 sector/read is glacial, and the
// crack only needs to FIND one scrambled sector whose 0x80 plaintext matches
// a known PES header. `batch_sectors` MUST be sized to the source — a drive
// rejects a READ(10) larger than its per-command max (DVD = 16) and
// `Drive::read` does not chunk, so an over-large batch fails every read and
// scans nothing. Callers pass `detect_max_batch_sectors(device_path)` for a
// live drive, a file-safe value for an image, or 1 to force per-sector.
// Batch the reads: a live drive at 1 sector/read is glacial. `batch_sectors`
// MUST be sized to the source — a drive rejects a READ(10) larger than its
// per-command max, and `Drive::read` does not chunk an over-large batch.
let batch = (batch_sectors.max(1)) as u32;
// Record the LBA span the key is being cracked from so the per-title mux
// path can tell whether a later title lives in the same VTS (overlaps the
@@ -233,18 +225,13 @@ fn crack_key_scan(
let max_tries = 50_000u32;
let mut buf = vec![0u8; batch as usize * 2048];
let mut hb = crate::progress::Heartbeat::new("css_crack");
// Track whether ANY scrambled sector was observed. If we exhaust the scan
// budget having seen scrambled data but never recovered a key, the content
// is encrypted-but-uncrackable — a HARD failure the caller must surface,
// NOT silently treat as unencrypted (which would mux scrambled MPEG as
// plaintext → garbage at exit 0). See `CrackOutcome::ScrambledUncracked`.
// Track whether ANY scrambled sector was observed: if the budget is
// exhausted with scrambled data seen but no key recovered, that's a HARD
// failure, not "unencrypted". See `CrackOutcome::ScrambledUncracked`.
let mut saw_scrambled = false;
// A read rejected with sense `05/6F/03` ("scrambled sector without
// authentication") is positive proof of CSS encryption — never collapse it
// to "unencrypted". A run of consecutive locked reads means the bus-auth
// gate is shut (it is global, so reads are all-or-nothing), so the scan
// early-bails. `consecutive_locked` resets on any readable batch, so a
// crackable title (gate open) never trips it.
// Sense `05/6F/03` is positive proof of CSS encryption. Consecutive locked
// reads mean the (global) bus-auth gate is shut, so the scan early-bails;
// this resets on any readable batch, so an open-gate title never trips it.
let mut saw_locked = false;
let mut consecutive_locked = 0u32;
@@ -273,52 +260,35 @@ fn crack_key_scan(
}
let n = (ext.sector_count - i).min(batch);
let want = n as usize * 2048;
// How far the cursor advances. Set from the bytes actually READ on
// the Ok path so a short read is RETRIED from where it stopped
// rather than skipped: bounding only the inspection (which is what
// stops stale buffer bytes being scanned) would otherwise leave
// those sectors unexamined, quietly shrinking the crack's coverage
// on exactly the damaged media where a key is hardest to find.
// Set from bytes actually READ on the Ok path so a short read is
// RETRIED from where it stopped, not skipped — else those sectors
// go unexamined on exactly the damaged media a key needs most.
let mut advance = n;
match reader.read_sectors(ext.start_lba + i, n as u16, &mut buf[..want], true) {
Ok(got) => {
// A readable batch: the gate is open — reset the locked run.
consecutive_locked = 0;
// Inspect only what was actually READ. The trait returns the
// byte count and a source may return Ok with fewer bytes
// than asked (a recovery read over a damaged region), while
// `buf` is reused across batches — so the tail still holds
// the PREVIOUS batch's sectors. Scanning those means
// cracking a key from data belonging to a different extent,
// and possibly a different VTS, while `crack_span` records
// the CURRENT one: a key that opens nothing here gets
// installed and its wrong descrambles are only partly caught
// by the per-sector crib.
// Inspect only what was READ: `buf` is reused across
// batches, so its tail may hold the PREVIOUS batch's
// sectors, risking a key crack for the wrong extent/VTS.
let usable = (got / 2048).min(n as usize);
// At least one, so a source returning Ok(0) cannot spin here.
advance = (usable as u32).max(1);
if usable == 0 {
// Nothing was inspected, so the per-sector `tried`
// charge below never runs — but the cursor still moves
// one sector (the `.max(1)` above). Charge that sector
// to the budget, or `tried` stays frozen and the loop
// is bounded only by the disc-declared
// `ext.sector_count`: the anti-grind budget stops
// applying to exactly the misbehaving source it exists
// for. Mirrors the `Err` arm's `tried += n`, which
// likewise charges an uninspected advance.
// Nothing inspected, but the cursor still moves one
// sector — charge it here or `tried` stays frozen
// (mirrors the `Err` arm's `tried += n`).
tried += 1;
}
for s in 0..usable {
tried += 1;
let sect = &buf[s * 2048..(s + 1) * 2048];
// Use the HARDENED pack-gated check (Fix 3): a clear stub
// sector with stray bits at 0x14 must NOT count as
// scramble evidence, or a genuinely-unencrypted title
// would falsely report ScrambledUncracked (a false E7023).
// HARDENED pack-gated check: a clear stub sector with
// stray bits at 0x14 must NOT count as scramble evidence,
// or an unencrypted title falsely reports E7023.
if is_scrambled_pack(sect) {
saw_scrambled = true;
if let Some(key) = stevenson::crack_title_key(sect) {
if let Some(key) = keyless::crack_title_key(sect) {
return CrackOutcome::Cracked(CssState {
title_key: key,
crack_span,
@@ -331,9 +301,8 @@ fn crack_key_scan(
}
}
// A failed batch still counts toward the budget so a damaged
// region can't loop forever. A CSS-locked failure (`05/6F/03`)
// proves encryption and, in a long enough run, means the read
// gate is shut — track it and early-bail rather than grind.
// region can't loop forever. A CSS-locked failure proves
// encryption; a long enough run means the gate is shut.
Err(e) => {
tried += n;
if e.scsi_sense().is_some_and(|s| s.is_css_locked()) {
@@ -351,29 +320,9 @@ fn crack_key_scan(
}
}
// Budget exhausted / extents walked / early-bailed with no key recovered.
// The disc is ENCRYPTED-but-uncracked (a hard failure) when EITHER a
// scrambled sector was actually seen, OR every read was CSS-locked
// (`05/6F/03`), itself proof of scrambling. Only a scan that saw neither a
// scrambled sector nor a CSS-lock is genuinely unencrypted.
//
// A prior revision made this conditional on a caller-supplied
// `fail_on_locked: bool`, documented as "false on the per-VTS re-crack, so
// a lapsed-AGID locked read returns None instead of killing a genuinely
// crackable title." That parameter never had an observable effect: its
// ONLY non-test caller with `false` was [`crack_key`], whose `Option`
// return collapses `ScrambledUncracked` and `Unencrypted` alike to `None`
// via [`CrackOutcome::into_state`] — so the branch this comment describes
// was unreachable from the moment it was introduced (see the crate's audit
// notes for the git-archaeology). The one production caller that DOES
// observe the `Cracked` / `Unencrypted` / `ScrambledUncracked` split for a
// per-VTS re-crack (`Disc::decrypt_keys_for_title`) has always gone
// through [`crack_key_outcome`], which hardcoded this to always-hard-fail.
// Reconnecting the soft variant there would mean a locked read on a live
// drive gets reported as `Unencrypted` (`title_is_clear = true`) and the
// title is muxed with NO key — precisely the silent-garbage failure mode
// `CrackOutcome` exists to prevent. So the parameter is removed rather
// than revived: the scan is unconditionally hard-fail-on-locked.
// ENCRYPTED-but-uncracked (hard failure) if a scrambled sector was seen or
// every read was CSS-locked; only a scan seeing neither is unencrypted. A
// prior "soft" `fail_on_locked` toggle was removed as dead/unsafe.
if saw_scrambled || saw_locked {
CrackOutcome::ScrambledUncracked
} else {
@@ -429,7 +378,7 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
/// seam it is dispatched from, and the `usize` is that seam's legacy
/// always-zero loss count (see [`crate::decrypt::decrypt_sectors`]).
///
/// This section used to document an [`Error::DecryptFailed`] for the case where
/// This section used to document an [`Error::DecryptFailed`](crate::Error::DecryptFailed) for the case where
/// a sector's crib rejects the cached key and the re-crack from that sector also
/// fails. That behaviour was tried and REVERTED, for the reason set out at the
/// `None =>` arm below: crib mismatch plus crack failure is the signature of a
@@ -437,24 +386,13 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
/// unrippable. The arm descrambles with the cached key and returns `Ok`.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
// `is_scrambled_pack`, NOT the looser `is_scrambled`. The raw flag test
// is only safe once a caller has committed to a title's VOB data, where
// every sector is a pack and byte 0x14 always means what it says. This
// function is handed arbitrary regions of a disc, so it also sees IFO,
// UDF and ISO 9660 sectors — raw structures where byte 0x14 is whatever
// that format stores there.
//
// Measured: `VIDEO_TS.IFO` on one disc holds 0x15 at offset 0x14 of its
// second sector. The raw test read bits 4-5 as "scrambled", descrambled
// it, and destroyed 1912 of 2048 bytes. That sector carries TT_SRPT, so
// the title table went with it: the disc enumerated 38 titles and an
// image decrypted from it enumerated 10, silently, at exit 0. Requiring
// the pack start code first costs nothing — a genuinely scrambled VOB
// sector always has it, and no IFO sector does.
// `is_scrambled_pack`, NOT the looser `is_scrambled`: this sees arbitrary
// regions (IFO/UDF/ISO 9660) where raw byte 0x14 isn't a reliable flag.
// Measured: an IFO misread this way was destroyed, dropping titles 38→10.
if chunk.len() < 2048 || !is_scrambled_pack(chunk) {
continue;
}
let crib = stevenson::attack_crib(chunk);
let crib = keyless::attack_crib(chunk);
// Snapshot the ciphertext (chunk is exactly 2048 here) only when there is
// a crib to validate against, so the common cache-hit path costs no
// per-sector heap allocation.
@@ -469,34 +407,15 @@ pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::erro
// Cached key is stale for this region — restore the ciphertext and
// crack this sector's own key.
chunk.copy_from_slice(&original);
match stevenson::crack_title_key(chunk) {
match keyless::crack_title_key(chunk) {
Some(fresh) => {
*title_key = fresh;
lfsr::descramble_sector(title_key, chunk);
}
None => {
// The re-crack found nothing. Descramble with the CACHED
// key anyway — it is the best available evidence, and this
// is very probably still the right key.
//
// `attack_crib` is a heuristic, not a proof. It finds a
// periodic run in the unscrambled header and predicts that
// the run continues past 0x80. When that prediction does
// not hold, the crib check reports a mismatch even though
// the cached key is correct, and the re-crack from this
// sector then fails BECAUSE the crib was never valid.
// Crib mismatch + crack failure is therefore the signature
// of a crib false positive, not of a stale key.
//
// Round 9 read this as "descrambling with a key we just
// proved stale" and made it Error::DecryptFailed to match
// the AACS path. That was wrong on both counts: the key is
// not proven stale, and CSS is not AACS — an AACS unit key
// either opens a unit or does not, whereas a CSS title key
// is recovered from data whose recoverability varies sector
// by sector. Real DVDs hit this constantly; the change made
// Greenland.iso unrippable and was caught by the real-media
// acceptance gate, not by any unit test.
// Re-crack found nothing; descramble with the CACHED key
// anyway — mismatch+failure signals a crib false positive,
// not a stale key (`DecryptFailed` here made discs unrippable).
lfsr::descramble_sector(title_key, chunk);
}
}
@@ -568,7 +487,7 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// found no key (there is none) and the scan returned `ScrambledUncracked`,
/// hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
/// `0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
/// can act on (the Stevenson attack only recovers a key from a scrambled ES
/// can act on (the recovery only recovers a key from a scrambled ES
/// pack), so a decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
///
/// This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
@@ -615,7 +534,7 @@ mod tests {
/// This test exists because round 9 read the same code as "descrambling
/// with a key we just proved stale" and made it `DecryptFailed` to match
/// the AACS path. Real DVDs hit this constantly — the change made
/// Greenland.iso unrippable, and no unit test caught it; the real-media
/// a real disc unrippable, and no unit test caught it; the real-media
/// acceptance gate did. CSS is not AACS: an AACS unit key either opens a
/// unit or does not, whereas a CSS title key is recovered from data whose
/// recoverability varies sector by sector.
@@ -636,11 +555,11 @@ mod tests {
"fixture must be a scrambled sector"
);
assert!(
stevenson::attack_crib(&sector).is_some(),
keyless::attack_crib(&sector).is_some(),
"fixture must yield a crib, or the mismatch branch is never entered"
);
assert!(
stevenson::crack_title_key(&sector).is_none(),
keyless::crack_title_key(&sector).is_none(),
"fixture must be uncrackable, or the failure branch is never entered"
);
@@ -649,13 +568,9 @@ mod tests {
let out = descramble_region(&mut sector, &mut key)
.expect("a crib false positive must NOT fail the rip");
// The "no loss term" contract belongs to the SEAM, not to this
// function: `decrypt_sectors`' `usize` is a legacy always-zero count
// that the CSS arm feeds from here. Asserting `out == 0` on
// `descramble_region` alone only restates its single `Ok(0)` return —
// a body replaced by `Ok(0)` satisfies it just as well. Assert it one
// level up, where the value is actually assembled and returned, so the
// arm dispatch and the plumbing are exercised too.
// "No loss term" belongs to the SEAM, not this function: assert it one
// level up (at `decrypt_sectors`) where the value is assembled and
// returned, so the arm dispatch and plumbing are exercised too.
assert_eq!(out, 0);
let mut seam_sector = sector;
let mut seam_keys = crate::decrypt::DecryptKeys::Css { title_key: key };
@@ -851,7 +766,7 @@ mod tests {
/// scrambled reads because the bus-auth gate isn't open).
lock_all: bool,
/// When set, the sector at `crackable.0` is served as a full
/// Stevenson-crackable scrambled sector (`crackable.1`, 2048 bytes)
/// crackable scrambled sector (`crackable.1`, 2048 bytes)
/// instead of the uniform `flag_byte` fill. Lets the scan actually
/// reach `CrackOutcome::Cracked` from a synthetic ISO.
crackable: Option<(u32, Vec<u8>)>,
@@ -881,11 +796,11 @@ mod tests {
}
}
/// Build a Stevenson-crackable scrambled sector for `(title_key, seed)`:
/// Build a crackable scrambled sector for `(title_key, seed)`:
/// the cleartext header (0x59..0x80) carries a periodic run that continues
/// across the 0x80 boundary into the encrypted region — the crib
/// `stevenson::crack_title_key` recovers a key from. Mirrors the
/// `synth_periodic_sector` fixture in the stevenson tests but built here
/// `keyless::crack_title_key` recovers a key from. Mirrors the
/// `synth_periodic_sector` fixture in the keyless tests but built here
/// from the crate-internal `scramble_sector`.
fn crackable_sector(title_key: &[u8; 5], seed: &[u8; 5], period: usize) -> Vec<u8> {
const RUN_START: usize = 0x59;
@@ -950,13 +865,9 @@ mod tests {
buf[base..base + 2048].copy_from_slice(sector);
}
_ => {
// Real DVD video sectors always open with the MPEG-PS
// pack-start code; `is_scrambled_pack` requires it before
// trusting the 0x14 scramble bits, so the fixture must
// include it for a `flag_byte` of 0x30 to register as
// scrambled. `stream_id` (byte 0x11) defaults to 0x00 (a
// scramblable pack); an HD-DVD RDI nav pack sets it to
// private_stream_2 (0xBF), which the 0x11 exclusion drops.
// `is_scrambled_pack` requires the MPEG-PS pack-start code
// before trusting 0x14; `stream_id` defaults to 0x00
// (scramblable) vs. an RDI nav pack's excluded 0xBF.
buf[base..base + 4].copy_from_slice(&PACK_START);
buf[base + 0x11] = self.stream_id;
buf[base + 0x14] = self.flag_byte;
@@ -967,13 +878,9 @@ mod tests {
}
}
// ── Short reads: the branch nothing exercised ─────────────────────────
//
// `crack_key_scan` passes `recovery = true`, which is precisely the mode
// where a `SectorSource` may return Ok with fewer bytes than asked. Every
// source in this module returned the full request, so `usable`, `advance`
// and the `.max(1)` anti-spin guard were dead code under test: reverting
// `advance` to `n`, or dropping the `.max(1)`, left the whole suite green.
// ── Short reads: the branch nothing exercised ──────────
// `recovery = true` means a `SectorSource` may return Ok with fewer bytes
// than asked; no source here did, so `usable`/`advance`/`.max(1)` were dead.
/// A short batch is RE-READ from where it stopped, not skipped. Skipping
/// would quietly shrink the crack's coverage on exactly the damaged media
@@ -1102,7 +1009,7 @@ mod tests {
}
/// THE Fix 6 regression: a scan that SEES scrambled sectors (flag set) but
/// recovers no key (the mock's zeroed data has no Stevenson crib) must
/// recovers no key (the mock's zeroed data has no recoverable crib) must
/// return `ScrambledUncracked` — a HARD failure — NOT `Unencrypted`. The
/// old code conflated this with "unencrypted" and muxed scrambled MPEG as
/// plaintext (garbage at exit 0).
@@ -1416,7 +1323,7 @@ mod tests {
/// SCAN-LEVEL CRACKED (audit gap "MockSource never yields a crackable
/// sector"): drive the full `crack_key_scan` over a synthetic ISO whose
/// scan hits a Stevenson-crackable scrambled sector. The outcome must be
/// scan hits a crackable scrambled sector. The outcome must be
/// `CrackOutcome::Cracked` with a key that round-trips the sector, AND the
/// `crack_span` must be recorded as the half-open extent span (the per-VTS
/// routing key the mux path needs). Previously only the leaf crack and the
@@ -1480,7 +1387,7 @@ mod tests {
sector_count: 50,
}];
// Cracked: a real Stevenson-crackable sector in an otherwise clear scan.
// Cracked: a real crackable sector in an otherwise clear scan.
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let mut cracked_src = MockSource::new(0x00);
+4 -3
View File
@@ -4,7 +4,8 @@
//! Content Scramble System. They are mathematical constants derived from
//! the CSS specification, published in academic literature since 1999.
/// Table 1: byte substitution used in key mangling and sector seed processing.
/// Table 1: content-cipher substitution table, `plain = TAB1[cipher] ^ keystream`
/// (used by descramble/recover in lfsr.rs and keyless.rs).
pub const TAB1: [u8; 256] = [
0x33, 0x73, 0x3b, 0x26, 0x63, 0x23, 0x6b, 0x76, 0x3e, 0x7e, 0x36, 0x2b, 0x6e, 0x2e, 0x66, 0x7b,
0xd3, 0x93, 0xdb, 0x06, 0x43, 0x03, 0x4b, 0x96, 0xde, 0x9e, 0xd6, 0x0b, 0x4e, 0x0e, 0x46, 0x9b,
@@ -88,7 +89,7 @@ pub const TAB3: [u8; 512] = [
0x00, 0x24, 0x49, 0x6d, 0x92, 0xb6, 0xdb, 0xff, 0x00, 0x24, 0x49, 0x6d, 0x92, 0xb6, 0xdb, 0xff,
];
/// Table 4: LFSR0 byte permutation (used in initialization and output).
/// Table 4: permutes the LFSR0 output byte (`o0 = TAB4[o0]`).
pub const TAB4: [u8; 256] = [
0x00, 0x80, 0x40, 0xc0, 0x20, 0xa0, 0x60, 0xe0, 0x10, 0x90, 0x50, 0xd0, 0x30, 0xb0, 0x70, 0xf0,
0x08, 0x88, 0x48, 0xc8, 0x28, 0xa8, 0x68, 0xe8, 0x18, 0x98, 0x58, 0xd8, 0x38, 0xb8, 0x78, 0xf8,
@@ -111,7 +112,7 @@ pub const TAB4: [u8; 256] = [
/// Table 5: LFSR1 output permutation used in the keystream combiner.
/// `TAB5[i] == TAB4[i] ^ 0xFF` (bitwise complement of the TAB4 bit-reversal
/// table). Applied on the normal descramble/recrypt path (lfsr.rs) as well as
/// in the key-recovery fallback (crack.rs).
/// in the key-recovery fallback ([`super::keyless`]).
pub const TAB5: [u8; 256] = [
0xff, 0x7f, 0xbf, 0x3f, 0xdf, 0x5f, 0x9f, 0x1f, 0xef, 0x6f, 0xaf, 0x2f, 0xcf, 0x4f, 0x8f, 0x0f,
0xf7, 0x77, 0xb7, 0x37, 0xd7, 0x57, 0x97, 0x17, 0xe7, 0x67, 0xa7, 0x27, 0xc7, 0x47, 0x87, 0x07,
+56 -104
View File
@@ -72,14 +72,9 @@ static DECRYPT_POOL: RwLock<Option<Arc<rayon::ThreadPool>>> = RwLock::new(None);
pub fn set_decrypt_threads(n: usize) {
let clamped = n.min(MAX_THREADS);
DECRYPT_THREADS.store(clamped, Ordering::Relaxed);
// Drop the existing pool. Next decrypt_pool() call rebuilds with
// the new resolved thread count.
//
// Recover the guard on poisoning, exactly as `decrypt_pool` does. Skipping
// the swap on a poisoned lock silently kept the STALE pool alive while the
// atomic above already reported the new thread count, so the setting appeared
// to take effect and never did. The pool Arc is immutable once stored, so a
// prior panic cannot have left it half-written.
// Drop the existing pool so the next decrypt_pool() call rebuilds with the
// new thread count. Recover the guard on poisoning like `decrypt_pool` does —
// skipping the swap would keep the STALE pool alive, making the setting no-op.
let mut guard = DECRYPT_POOL.write().unwrap_or_else(|e| e.into_inner());
*guard = None;
}
@@ -100,10 +95,9 @@ fn decrypt_pool() -> Option<Arc<rayon::ThreadPool>> {
return Some(Arc::clone(pool));
}
}
// Slow path: build a new one under the write lock. Recover the guard
// on poisoning (a prior panic) rather than propagating a secondary
// panic — we simply rebuild. Double-check after acquiring in case
// another caller built it first.
// Slow path: build a new one under the write lock, recovering the guard on
// poisoning (a prior panic) rather than propagating a secondary panic — we
// simply rebuild. Double-check after acquiring in case another caller won.
let mut guard = DECRYPT_POOL.write().unwrap_or_else(|e| e.into_inner());
if let Some(pool) = guard.as_ref() {
return Some(Arc::clone(pool));
@@ -128,10 +122,9 @@ pub fn decrypt_threads() -> usize {
if explicit > 0 {
return explicit;
}
// Resolve the `FREEMKV_THREADS` env var + `available_parallelism()` ONCE and
// cache it — this runs on the per-buffer decrypt hot path, and a getenv +
// String alloc + parallelism syscall per call is pure overhead. The explicit
// `set_decrypt_threads` override above still takes effect dynamically.
// Resolve `FREEMKV_THREADS` + `available_parallelism()` ONCE and cache it:
// this runs on the per-buffer decrypt hot path, so a getenv/alloc/syscall
// per call is pure overhead (the `set_decrypt_threads` override still works).
static DEFAULT_THREADS: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
*DEFAULT_THREADS.get_or_init(|| {
let env = std::env::var("FREEMKV_THREADS")
@@ -208,11 +201,9 @@ fn unit_is_our_phase(unit_lba: u32, range_start: u32, unit_sectors: u32, phase:
Phase::Even => false,
Phase::Odd => true,
};
// `saturating_sub` and `max(1)`: both inputs come from the key map, which is
// built from disc structure. A unit below its own range start, or a zero
// unit size, means the map is malformed — that must not panic (debug
// overflow / divide-by-zero) inside a library used by a long-running
// service. Unit 0 of the range is even, which is the safe default.
// `saturating_sub`/`max(1)` guard against a malformed key map (unit below its
// range start, or zero unit size) — must not panic (overflow/div-by-zero) in
// a long-running service. Unit 0 of the range is even, the safe default.
let unit_ix = unit_lba.saturating_sub(range_start) / unit_sectors.max(1);
(unit_ix % 2 == 1) == want_odd
}
@@ -402,18 +393,6 @@ impl AacsKeyMap {
}
}
/// Decrypt a buffer of sectors in-place using a resolved [`AacsKeyMap`] — the
/// mux's TRUSTED decrypt. `base_lba` is the absolute LBA of `buf`'s first sector;
/// each aligned unit (3 sectors) is decrypted with the key the map assigns to its
/// LBA. There is NO key trial and NO `is_clean` verdict: the map already decided
/// the key from disc structure, so we apply it and move on — a unit that decrypts
/// to authored-bad TS passes through for the muxer to drop, never re-fetched.
///
/// Only [`DecryptKeys::Aacs`] uses a map (CSS self-cracks per region inside
/// [`decrypt_sectors`]; `None` is clear) — other variants are a no-op here so the
/// decorator can dispatch uniformly. A map index outside the held pool is a
/// fail-loud [`Error::DecryptFailed`]: the resolver's job is to guarantee every
/// selectable index is present, so a gap here is a resolver bug, not silent loss.
/// Decrypt `buf` with a resolved AACS key map. Thin wrapper over
/// [`decrypt_span`] — the map is the AACS scheme's input, not a second
/// orchestrator.
@@ -427,12 +406,20 @@ pub(crate) fn decrypt_sectors_mapped(
decrypt_span(buf, &mut keys, base_lba, Some(map), None).map(|_| ())
}
/// AACS scheme step: apply `map`'s per-unit keys to `buf`.
/// AACS scheme step: apply `map`'s per-unit keys to `buf`, in-place. `base_lba`
/// is the absolute LBA of `buf`'s first sector; each aligned unit (3 sectors) is
/// decrypted with the key the map assigns to its LBA. There is NO key trial and
/// NO `is_clean` verdict: the map already decided the key from disc structure,
/// so this applies it and moves on — a unit that decrypts to authored-bad TS
/// passes through for the muxer to drop, never re-fetched.
///
/// A SCHEME, not a policy. It reports what it could not open by returning
/// `Err(DecryptFailed)`; the decision that an unopenable unit must never be
/// emitted belongs to [`decrypt_span`], which is the one place that decides it
/// for every scheme.
/// for every scheme. A map index outside the held pool is likewise a fail-loud
/// [`Error::DecryptFailed`](crate::Error::DecryptFailed): the resolver's job is
/// to guarantee every selectable index is present, so a gap here is a resolver
/// bug, not silent loss.
fn apply_aacs_map(
buf: &mut [u8],
keys: &DecryptKeys,
@@ -462,23 +449,16 @@ fn apply_aacs_map(
}
}
// Cheap safety net for the "map must be right" model: with a correct
// phase-aware map, every CORRECT-PHASE forensic unit decrypts to clean TS, so
// this never fires in the happy path — but a map bug (wrong phase/key for a
// segment) surfaces as a loud DecryptFailed instead of silent corruption. Only
// forensic (Even/Odd) ranges are verified; base / multi-CPS (All) stays
// trust-only, so the common disc is byte-for-byte unchanged.
// Cheap safety net: a correct map always decrypts CORRECT-PHASE forensic
// units to clean TS, so this never fires happy-path — a map bug surfaces as
// loud DecryptFailed, not silent corruption. Only forensic ranges are verified.
let verify_failed = std::sync::atomic::AtomicBool::new(false);
let decrypt_one = |idx_in_buf: usize, chunk: &mut [u8]| {
if chunk.len() != unit_len {
// Trailing partial unit (buffer/region tail shorter than a whole unit).
// Normally a genuinely-clear content tail (source-zero padding or a
// short final fragment) — leave as-is. But a partial that is BOTH inside
// a mapped (encrypted) range AND flagged encrypted in its clear seed is
// an encrypted unit split across a boundary: a CBC fragment we cannot
// decrypt, so emitting it verbatim would ship ciphertext as clear. Fail
// loud instead (restores the guard the removed `decrypt_sectors` had).
// Trailing partial unit: normally a genuinely-clear tail, left as-is. But
// one that is BOTH inside a mapped range AND flagged encrypted is a CBC
// fragment we cannot decrypt — fail loud instead of shipping it as clear.
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
if map.entry_for(unit_lba).is_some()
&& aacs::content::aacs_unit_seed_encrypted(chunk, format)
@@ -488,20 +468,9 @@ fn apply_aacs_map(
return;
}
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
// No range covers this LBA. That is expected for clear filesystem / nav
// on a whole-disc read — but "the map has no key here" and "there is
// nothing to decrypt here" are different statements, and only the
// second makes passing the unit through correct.
//
// An ENCRYPTED unit outside every range is content we cannot key: on a
// multi-CPS disc that is an orphan clip referenced by no playlist, so
// it sits in no title extent and therefore in no range. Emitting it
// verbatim ships ciphertext where plaintext is meant to be, and extract
// then counts those bytes as good and reports the file complete.
//
// The split-unit branch immediately above already draws exactly this
// distinction. This one did not, so it was reached before the
// `aacs_unit_encrypted` gate below ever ran.
// No range covers this LBA — expected for clear filesystem/nav, but an
// ENCRYPTED unit outside every range is an orphan clip we cannot key;
// emitting it verbatim would ship ciphertext as clear content.
let Some((key_idx, phase, range_start)) = map.entry_for(unit_lba) else {
if aacs::content::aacs_unit_seed_encrypted(chunk, format) {
verify_failed.store(true, std::sync::atomic::Ordering::Relaxed);
@@ -568,8 +537,8 @@ fn apply_aacs_map(
/// reader was built without installing its map (a bug), so it fails loud rather
/// than apply a guessed key.
///
/// `unit_key_idx` and `content` are legacy parameters kept so the CSS / `None`
/// wrapper signatures stay stable; they are ignored (the CSS arm self-gates on its
/// `unit_key_idx` is a legacy parameter kept so the CSS / `None` wrapper
/// signature stays stable; it is ignored (the CSS arm self-gates on its
/// per-sector scramble flag). Returns `Err` if decryption was expected but
/// impossible; never produces silently corrupted output. The `usize` return is a
/// legacy unverified-byte count that is always `0` for the CSS / `None` arms.
@@ -629,12 +598,9 @@ fn decrypt_span(
let dropped: usize = match keys {
DecryptKeys::None => 0,
DecryptKeys::Aacs { .. } => {
// AACS decrypts EXCLUSIVELY through a resolved key map: the map keys
// every content unit up front and a missing key fails at RESOLVE
// time. No map here means an AACS reader was built without
// installing one — the old trial-decrypt path (try each held key,
// keep the first-tried plaintext on a miss) is gone precisely
// because it silently applied wrong keys.
// AACS decrypts EXCLUSIVELY via a resolved key map (missing key fails at
// RESOLVE time). No map here means the reader was built without one — the
// old trial-decrypt path is gone: it silently applied wrong keys on a miss.
let Some(map) = map else {
return Err(crate::error::Error::DecryptFailed);
};
@@ -642,12 +608,9 @@ fn decrypt_span(
0
}
DecryptKeys::Css { title_key } => {
// CSS SELF-recovers: the title key changes per VOB region and is
// re-cracked constantly, but always FROM THE DATA ITSELF — no external
// input. So the whole descramble-and-rekey is self-contained here (see
// `css::descramble_region`), and CSS does not need the post-decrypt
// recovery seam that AACS key-fetch / FMTS segment-skip use (those DO
// consume external inputs a `decrypt_sectors` caller cannot supply).
// CSS SELF-recovers: the title key changes per VOB region and is re-cracked
// constantly, but always FROM THE DATA ITSELF (see `css::descramble_region`),
// so it needs none of the external-input recovery seam AACS key-fetch uses.
css::descramble_region(buf, title_key)?
}
};
@@ -849,10 +812,10 @@ mod tests {
);
}
/// Build a Stevenson-crackable scrambled CSS sector for `title_key` (mirrors
/// Build a crackable scrambled CSS sector for `title_key` (mirrors
/// `crackable_sector` in the css::mod tests): a periodic run in the clear
/// header continues past 0x80 into the encrypted region, so
/// `stevenson::crack_title_key` recovers the key. Distinct `seed` values give
/// `keyless::crack_title_key` recovers the key. Distinct `seed` values give
/// two sectors different cribs, standing in for two VOB regions.
fn crackable_css_sector(title_key: &[u8; 5], seed: &[u8; 5]) -> Vec<u8> {
const RUN_START: usize = 0x59;
@@ -1030,11 +993,9 @@ mod tests {
/// scramble flag.
fn make_css_sector(title_key: &[u8; 5], seed: &[u8; 5], body_fill: u8) -> (Vec<u8>, Vec<u8>) {
let mut sector = vec![body_fill; 2048];
// A real scrambled DVD sector is an MPEG-2 PS pack, so it begins with
// the pack start code. The descrambler requires it before trusting
// byte 0x14 — without it this fixture is a sector shape that cannot
// occur on a disc, and the test would pass while the production gate
// rejected every sector like it.
// A real scrambled DVD sector is an MPEG-2 PS pack starting with the pack
// start code; the descrambler requires it before trusting byte 0x14. Without
// it the fixture is a shape that can't occur on disc, masking a rejecting gate.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] = 0x30; // scramble flag (bits 4-5)
sector[0x54..0x59].copy_from_slice(seed);
@@ -1157,12 +1118,12 @@ mod tests {
// Precondition: each sector must be crackable on its own (the rekey
// depends on it). If this fails the fixture, not the path, is at fault.
assert_eq!(
crate::css::stevenson::crack_title_key(&s0),
crate::css::keyless::crack_title_key(&s0),
Some(key_a),
"fixture s0 must crack to key_a standalone"
);
assert_eq!(
crate::css::stevenson::crack_title_key(&s1),
crate::css::keyless::crack_title_key(&s1),
Some(key_b),
"fixture s1 must crack to key_b standalone"
);
@@ -1437,10 +1398,9 @@ mod tests {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// Case A — range_start is itself unaligned (1001 % 3 == 2) and the extent
// begins on it, so unit offsets are 0, 3, 6, ... Under this shape the
// formula `(lba + range_start) / us` shifts every index by an ODD amount
// and inverts the kept half.
// Case A — range_start is itself unaligned (1001 % 3 == 2), so unit offsets
// are 0, 3, 6, ... Under this shape `(lba + range_start) / us` shifts every
// index by an ODD amount and inverts the kept half.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
@@ -1462,11 +1422,9 @@ mod tests {
units are the even-indexed ones counting from the range start"
);
// Case B — the extent begins one unit-remainder away from the range start
// (1001 - 1000 = 1), so offsets are 1, 4, 7, 10. Here `(lba - range_start)
// * us` inverts the halves instead: the division is what maps a byte
// offset onto a unit index, and multiplying happens to preserve parity
// only when the offset is already a multiple of the unit size.
// Case B — the extent begins one unit-remainder from the range start (1001 -
// 1000 = 1), so offsets are 1, 4, 7, 10. Here `(lba - range_start) * us`
// inverts the halves: multiplying only preserves parity for aligned offsets.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
@@ -1876,22 +1834,16 @@ mod tests {
}
// The index must be RANGE-RELATIVE: `(lba - start)`, not `(lba + start)`.
// Most value pairs give the same parity either way, so pin one where
// they genuinely disagree: (5-1)/2 = 2 (even, ours) but
// Pin a case where they genuinely disagree: (5-1)/2 = 2 (even, ours) but
// (5+1)/2 = 3 (odd, not ours).
assert!(
unit_is_our_phase(5, 1, 2, Phase::Even),
"the index must be measured from the range start"
);
// The `/ unit_sectors` -> `* unit_sectors` mutant is EQUIVALENT here,
// and deliberately not chased. Proof: aligned offsets are exact
// multiples of the unit size, so offset = k*u for unit index k.
// Dividing gives k; multiplying gives k*u^2, whose parity is
// parity(k)*parity(u^2) = parity(k) whenever u is ODD. `unit_sectors`
// is `ALIGNED_UNIT_LEN / 2048` = 3, a compile-time constant, so u is
// always odd and the two agree on every reachable input. Only an even
// unit size would separate them, and none exists.
// The `/ unit_sectors` -> `* unit_sectors` mutant is EQUIVALENT here and
// deliberately not chased: aligned offsets are k*u, and dividing vs.
// multiplying give the same parity since `unit_sectors` (=3) is always odd.
assert!(!unit_is_our_phase(33, 30, 3, Phase::Even));
}
+35 -20
View File
@@ -92,12 +92,9 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
}
}
// `channel_count` and `sample_rate_hz` lived here as a third copy of the
// AudioChannels/SampleRate mappings. They were the only HONEST copy — returning
// 0 for Unknown where the canonical accessors fabricated 6 channels at 48 kHz —
// and their only caller was the trace line below, in this same file. The
// canonical accessors are honest now, so the duplicates are gone rather than
// left to drift a fourth time.
// `channel_count`/`sample_rate_hz` used to live here as a third, HONEST copy of
// the AudioChannels/SampleRate mappings (returning 0 for Unknown where the
// canonical accessors fabricated 6ch/48kHz). Canonical accessors are honest now.
// ── DVD cell-category dump (from the IFO scan, pre-lowering) ─────────────────
@@ -110,9 +107,8 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
pub fn dvd_cell_row(idx: usize, cell: &crate::ifo::DvdCell, dropped: bool) -> String {
let c = CellCategory::decode(cell.category);
// Per-cell keep/skip REASON (self-sufficient bug log): a dropped cell is a
// leading secondary angle/interleave block piece; a kept cell is either the
// first feature cell or genuine feature content. This makes the
// leading-cell-filter decision auditable from the log without the disc.
// leading secondary angle/interleave piece; a kept cell is the first feature
// cell or genuine content. Makes the leading-cell-filter decision auditable.
let verdict = if dropped {
"DROP(leading-secondary-block-piece)"
} else if c.is_secondary_block_piece() {
@@ -472,14 +468,31 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!(
target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason={}",
size={}B clips={} video={} reason={}",
main.playlist,
main.duration_secs,
main.size_bytes,
main.clips.len(),
main.video_streams().count(),
main_feature_reason(),
);
}
// Titles disqualified from main-feature candidacy for carrying no video — the
// streamless obfuscation decoys the `has-video` gate demotes. Logged so a field
// bug report shows WHY a playlist wasn't picked (the gap behind E6008 on `00245`).
for (ti, t) in disc.titles.iter().enumerate() {
if !t.has_video() {
tracing::debug!(
target: DIAG,
"tag=decision.demoted title_idx={} playlist={:?} dur={:.1}s size={}B reason=no-video",
ti,
t.playlist,
t.duration_secs,
t.size_bytes,
);
}
}
}
/// The `reason=` token on the main-feature decision row.
@@ -491,10 +504,12 @@ pub fn dump_disc(disc: &Disc) {
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
let keys: Vec<&str> = Disc::MAIN_FEATURE_ORDER_KEYS
.iter()
.chain(Disc::CANONICAL_TITLE_ORDER_KEYS.iter())
.copied()
.collect();
format!("main_feature_order({})", keys.join(", "))
}
fn dump_aacs(disc: &Disc) {
@@ -652,10 +667,9 @@ mod tests {
.collect(),
..DiscTitle::empty()
};
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's
// primary key among disc-fitting titles is LARGEST SIZE. "fewest clips"
// would predict the opposite, so the drifted string described a rule
// the comparator does not implement.
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's primary
// key among disc-fitting titles is LARGEST SIZE, not "fewest clips" (the
// drifted string described a rule the comparator does not implement).
let many_clips_big = sized(8_000_000_000, 40);
let one_clip_small = sized(1_000_000_000, 1);
assert_eq!(
@@ -670,8 +684,9 @@ mod tests {
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
reason,
"main_feature_order(nav-feature, authoring-feature, standalone, has-video, fits-disc, largest-size, longest, richest-audio)",
"the reason must name the selection keys (authoring, standalone-over-composite, and has-video gates, then the physical keys) in priority order"
);
}
+9 -13
View File
@@ -459,10 +459,9 @@ fn file_entry(
s[160..168].copy_from_slice(&unique_id.to_le_bytes());
s[168..172].copy_from_slice(&0u32.to_le_bytes()); // length of EAs
let l_ad = extents.len() * 8;
// A short AD is 8 bytes and the entry has 2048 - 176 = 1872 bytes for
// them, i.e. 234 extents — over 200 GiB at the per-AD ceiling. Beyond
// that an Allocation Extent Descriptor chain would be required; refuse
// rather than write a truncated list.
// A short AD is 8 bytes, leaving room for 234 extents (~200 GiB) before an
// Allocation Extent Descriptor chain would be needed; refuse rather than
// write a truncated list.
if 176 + l_ad > SECTOR {
return Err(Error::DirImageTooLarge);
}
@@ -500,10 +499,9 @@ fn push_fid(buf: &mut Vec<u8>, name: &str, icb_lba: u32, is_dir: bool, is_parent
chars |= 0x08;
}
fid[18] = chars;
// The planner refuses any name whose encoding exceeds what this byte can
// hold (`layout::MAX_CS0_NAME_BYTES`), so this cannot wrap in practice. The
// assert states the invariant where it is relied on rather than trusting a
// check three files away; a wrap here would desynchronise the directory.
// Planner enforces `layout::MAX_CS0_NAME_BYTES` so this can't wrap in
// practice; the assert states that invariant where it's relied on
// instead of trusting a check three files away.
debug_assert!(
l_fi <= u8::MAX as usize,
"FID name length must fit one byte"
@@ -603,11 +601,9 @@ fn write_dir(out: &mut MetaSectors, layout: &Layout, dir: &DirNode) -> Result<()
fix_fid_tag_locations(&mut fids, dir.data_lba);
debug_assert_eq!(fids.len(), dir.data_bytes as usize);
// A directory's link count is 1 (its own FID in the parent) plus one for
// each child directory's parent FID pointing back at it.
// The planner caps subdirectory fan-out (`layout::MAX_SUBDIRS`) so this
// cannot overflow; saturating rather than wrapping keeps a future change to
// that cap from silently producing a wrong count.
// Link count is 1 (own FID in parent) plus one per child dir's parent FID.
// `layout::MAX_SUBDIRS` bounds fan-out so this can't overflow; saturating
// guards against a future change to that cap silently miscounting.
let link_count = (dir.dirs.len() as u16).saturating_add(1);
let fe = file_entry(
true,
+42 -93
View File
@@ -218,18 +218,14 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
if is_excluded(&name) {
continue;
}
// `file_type()` does NOT follow symlinks; `metadata()` does. A symlink
// to a file is materialized as that file (a legitimate way to assemble
// a folder), a symlink to a directory is skipped — following one can
// loop, and UDF has no link this maps onto.
// `file_type()` does NOT follow symlinks; `metadata()` does. A symlink to
// a file is materialized as that file; a symlink to a directory is
// skipped — following one can loop, and UDF has no link this maps onto.
let ft = entry.file_type().map_err(Error::from)?;
let child_path = format!("{}/{}", disc_path.trim_end_matches('/'), name);
// A File Identifier Descriptor records the encoded name length in ONE
// byte. A longer name would wrap that field and desynchronise every
// later entry in the directory, so refuse while planning rather than
// encode something unreadable. 255 ASCII bytes is a legal name on
// ext4/APFS/NTFS and already exceeds this once the CS0 compression
// byte is added, so it is reachable without anything exotic.
// A FID records the encoded name length in ONE byte; a longer name would
// wrap it and desynchronise the directory, so refuse while planning. 255
// ASCII bytes (legal on ext4/APFS/NTFS) already exceeds it with CS0 added.
if crate::dirimage::encode::encode_cs0(&name).len() > MAX_CS0_NAME_BYTES {
return Err(Error::DirNameTooLong { path: child_path });
}
@@ -237,29 +233,13 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
if *entries > MAX_ENTRIES {
return Err(Error::DirImageTooLarge);
}
// Key the uniqueness check on the name AS THE READER WILL SEE IT, by
// round-tripping through the very encoder and parser that will be used.
//
// The host name is not that name. `parse_udf_name` trims leading and
// trailing whitespace and drops any code unit `char::from_u32` rejects
// (a lone surrogate half, i.e. every non-BMP character the encoder
// emits as a surrogate pair). So " A.M2TS" and "A.M2TS", or "A<astral>.M2TS"
// and "A.M2TS", are distinct hosts that collapse to ONE name on read —
// and `find`/`read_file` take the first match, so a title silently
// resolves to the wrong file's extents and muxes the wrong bytes at
// exit 0. Comparing the raw host names cannot see that; deriving the
// key from the same two functions cannot drift from it.
// Key uniqueness on the name AS THE READER WILL SEE IT (round-trip through
// the same encoder/parser it uses): `parse_udf_name` trims/drops chars, so
// distinct hosts (" A.M2TS" vs "A.M2TS") can collapse and `find` mismatches.
let as_read = crate::udf::parse_udf_name(&crate::dirimage::encode::encode_cs0(&name));
// Nothing left after the reader is done with it — a name made entirely
// of characters `parse_udf_name` drops, e.g. a sidecar folder named
// with a single emoji. The entry would exist in the image and be
// addressable by nothing.
//
// SKIP it, do not fail the plan. Failing turned an irrelevant extra
// file into an un-rippable folder that 1.6.0 handled fine (the
// unreachable entry merely sat there, harming nothing), and reported it
// as a collision with a file that does not exist. It is also not
// pushed to `names`, so it cannot collide with anything either.
// Nothing left after the reader drops it (e.g. a name of a single emoji):
// it would exist in the image but be addressable by nothing. SKIP rather
// than fail the plan (1.6.0 handled such extras fine); not pushed to `names`.
if as_read.is_empty() {
tracing::warn!(
target: "freemkv::dirimage",
@@ -270,10 +250,9 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
}
names.push(as_read.to_ascii_uppercase());
if ft.is_dir() {
// A directory's File Entry records its link count in 16 bits: one
// per child directory plus one for its own entry in the parent.
// The global entry cap alone permits a single directory holding
// more than that, which would wrap the count.
// Link count (child dirs + 1) is stored in 16 bits; the global entry
// cap alone permits a single directory holding more than that, which
// would wrap the count.
if dirs.len() >= MAX_SUBDIRS {
return Err(Error::DirImageFanout {
path: disc_path.to_string(),
@@ -283,30 +262,16 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
} else {
let meta = match std::fs::metadata(entry.path()) {
Ok(m) => m,
// A broken symlink or a file that vanished between readdir and
// stat: skip it rather than plan an extent that cannot be read.
// Anything else — permission denied, an I/O error on the host
// volume — is NOT a missing file, and skipping it would drop a
// real file out of the image with nothing reported.
// A broken symlink or a file gone between readdir and stat: skip it
// rather than plan an unreadable extent. Anything else (e.g. perm
// denied) is NOT a missing file, so skipping it would drop it silently.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => continue,
Err(e) => return Err(Error::from(e)),
};
if !meta.is_file() {
// Not a plain file after following the link. The common case is
// a SYMLINK TO A DIRECTORY — a normal way to keep 40 GB of
// streams off the system disk — and `entry.file_type()` above
// reports the link, not the target, so such a subtree never
// enters `dirs` and is silently absent from the image.
//
// The result was the worst class this release exists to close:
// BDMV/PLAYLIST and CLIPINF still synthesize, so the folder
// scans and enumerates titles, every clip resolves to no
// extents, and the mux takes its clean-EOF path and writes a
// near-empty MKV at exit 0.
//
// It is still skipped rather than followed — following link
// targets invites cycles and escapes from the folder — but it
// is now SAID, at the same volume as the name-skip above.
// Not a plain file after following the link: commonly a SYMLINK TO A
// DIRECTORY, silently vanishing before (clips → no extents, near-empty
// MKV at exit 0). Still skipped, not followed, but now SAID.
tracing::warn!(
target: "freemkv::dirimage",
path = %child_path,
@@ -328,10 +293,9 @@ fn walk(dir: &Path, disc_path: &str, depth: u32, entries: &mut usize) -> Result<
}
}
// `UdfFs::find_dir` / `read_file` match path components case-insensitively,
// so two entries in one directory differing only in case are
// indistinguishable to every consumer — the second would silently shadow
// the first. Only reachable on a case-sensitive host volume.
// `find_dir`/`read_file` match path components case-insensitively, so two
// entries differing only in case are indistinguishable to every consumer —
// the second would silently shadow the first. Only on a case-sensitive host.
names.sort();
for pair in names.windows(2) {
if pair[0] == pair[1] {
@@ -428,12 +392,9 @@ fn be_u32(buf: &[u8], off: usize) -> Option<u32> {
fn read_head(path: &Path, n: usize) -> Result<Vec<u8>> {
use std::io::Read;
let mut buf = vec![0u8; n];
// `read_exact`, not `read`: a single `read` may legally return fewer bytes
// than asked on a network or FUSE mount — and a NAS-hosted backup is the
// normal case for this feature. A short buffer sends every placement offset
// through `unwrap_or(0)`, recording NO constraint, so the planner and the
// reader disagree about where a VOB begins and the rip reads the wrong
// sectors for a whole title at exit 0.
// `read_exact`, not `read`: a single `read` may legally return fewer bytes on
// a network/FUSE mount (a NAS-hosted backup is normal here). A short buffer
// records NO constraint, so the rip reads the wrong sectors at exit 0.
let mut f = std::fs::File::open(path).map_err(Error::from)?;
f.read_exact(&mut buf).map_err(Error::from)?;
Ok(buf)
@@ -482,10 +443,9 @@ fn place_video_ts(vts: &mut DirNode, start: u32) -> Result<u32> {
// Required start blocks, resolved as each IFO is placed.
let mut menu_req: std::collections::HashMap<u32, u32> = std::collections::HashMap::new();
let mut title_req: std::collections::HashMap<u32, u32> = std::collections::HashMap::new();
// Groups whose IFO has already been placed. Kept separately from the
// constraint maps because an IFO that declares no offsets inserts into
// neither, so keying the duplicate check on those maps would miss a second
// IFO for the same set — which is the collision the check exists for.
// Groups whose IFO has already been placed, kept separately from the
// constraint maps: an IFO declaring no offsets inserts into neither, so
// keying the duplicate check on those maps would miss the collision.
let mut seen_ifo: std::collections::HashSet<u32> = std::collections::HashSet::new();
let mut cursor = start;
@@ -517,11 +477,9 @@ fn place_video_ts(vts: &mut DirNode, start: u32) -> Result<u32> {
{
let head = read_head(&vts.files[i].host, 0xC8)?;
let menu = be_u32(&head, 0xC0).unwrap_or(0);
// One group per title set. Two files whose names differ only in
// how the number is written — `VTS_01_0.IFO` and `VTS_1_0.IFO` —
// parse to the same group, and the second insert would overwrite
// the first's constraint, placing a VOB at an address the IFO the
// reader uses does not point to. Refuse instead of picking one.
// One group per title set: `VTS_01_0.IFO` and `VTS_1_0.IFO` parse to
// the same group, and a second insert would overwrite the first's
// constraint, placing a VOB where the reader's IFO doesn't point.
if !seen_ifo.insert(c.group) {
return Err(Error::DirNameCollision {
host: vts.files[i].disc_path.clone(),
@@ -530,12 +488,9 @@ fn place_video_ts(vts: &mut DirNode, start: u32) -> Result<u32> {
if menu != 0 {
menu_req.insert(c.group, lba.saturating_add(menu));
}
// Only a VTS IFO carries a title VOBS pointer at 0xC4. In the VMG
// (`VIDEO_TS.IFO`) that offset is TT_SRPT, the title search pointer
// table — a sector offset INSIDE the IFO, not a pointer to another
// file, so treating it as one would place `VIDEO_TS.VOB` at a
// meaningless address. (`ifo.rs::parse_vmg` reads the same field as
// TT_SRPT; the two must not drift.)
// Only a VTS IFO carries a title VOBS pointer at 0xC4; in the VMG that
// offset is TT_SRPT (a sector offset INSIDE the IFO, not a file
// pointer). `ifo.rs::parse_vmg` reads the same field; must not drift.
if c.group > 0 {
let title = be_u32(&head, 0xC4).unwrap_or(0);
if title != 0 {
@@ -723,13 +678,9 @@ pub(super) fn plan(root: &Path) -> Result<Layout> {
.position(|d| d.name.eq_ignore_ascii_case("VIDEO_TS"))
{
cursor = place_video_ts(&mut tree.dirs[idx], cursor)?;
// `place_video_ts` places only the FILES directly inside VIDEO_TS,
// because only those carry the IFO-relative constraints. Anything in a
// subdirectory of VIDEO_TS still needs data placed: without this its
// File Entry is written with the file's real size but no extents, so it
// appears in the tree at full length and reads back as nothing, at
// exit 0. The same folder under BDMV/ was always placed correctly,
// which is what made the gap easy to miss.
// `place_video_ts` places only the FILES directly inside VIDEO_TS (the ones
// with IFO-relative constraints); subdirectories still need placing, or
// their File Entry has real size but no extents and reads as nothing.
for sub in tree.dirs[idx].dirs.iter_mut() {
place_generic(sub, &mut cursor)?;
}
@@ -989,11 +940,9 @@ mod tests {
/// plainly that it does NOT exercise `walk`.
#[test]
fn the_subdir_cap_refuses_a_folder_with_too_many_subdirectories() {
// Executes the guard, rather than restating the constant. A directory's
// File Entry records its link count in 16 bits — one per child
// directory plus one for its own entry in the parent — so exceeding it
// would wrap the count and produce an image whose directory structure
// lies about itself.
// Executes the guard rather than restating the constant: link count (child
// dirs + 1) is 16 bits, so exceeding it wraps and the image lies about
// its own directory structure.
let dir = std::env::temp_dir().join(format!("fmkv-fanout-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
+15 -45
View File
@@ -127,18 +127,9 @@ impl DirImage {
let mut files = Vec::with_capacity(nodes.len());
let mut ranges = Vec::new();
for (idx, node) in nodes.iter().enumerate() {
// Carry the plan-time mtime ONLY for files whose CONTENT the plan
// read — the DVD IFOs, whose bytes 0xC0/0xC4 decide where every VOB
// is placed (`layout::place_video_ts` -> `read_head`).
//
// For every other file the plan depends on the SIZE alone, and size
// is already checked. Comparing mtime on those buys nothing and
// costs real false positives: disc backups commonly live on
// exFAT/FAT32, which stores local time, so a long rip spanning a
// DST transition sees a whole-hour shift on a file nobody touched
// and would abort hours in, blaming a change that did not happen.
// The multi-gigabyte VOBs are exactly the files a long rip re-opens
// after the handle cache evicts them.
// Carry plan-time mtime ONLY for files whose CONTENT the plan read (DVD
// IFOs, whose 0xC0/0xC4 bytes place every VOB); others depend only on
// SIZE, and comparing mtime there false-positives on exFAT/FAT32 + DST.
let content_sensitive = node
.disc_path
.rsplit('.')
@@ -228,16 +219,9 @@ impl DirImage {
}
let f = File::open(&self.files[file].host).map_err(Error::from)?;
let md = f.metadata().map_err(Error::from)?;
// Size AND mtime. Size alone is content-blind, and this plan depends on
// content: a DVD's VOB placement comes from bytes 0xC0/0xC4 of its IFO,
// and an IFO rewritten in place keeps its length because IFOs occupy a
// whole number of sectors. The size check would pass while every title
// extent pointed at the wrong sectors — corrupt video behind an intact
// structure, reported complete at exit 0.
//
// Only compared when both sides have a timestamp; a platform or
// filesystem that reports none simply falls back to the size check
// rather than failing every read.
// Size AND mtime: size alone is content-blind, and VOB placement depends on
// IFO bytes 0xC0/0xC4 — an IFO rewritten in place keeps its sector-aligned
// length, so size alone would miss it. mtime only compared if both present.
let changed_size = md.len() != self.files[file].size;
let changed_mtime = match (self.files[file].mtime, md.modified().ok()) {
(Some(planned), Some(live)) => planned != live,
@@ -271,19 +255,9 @@ impl DirImage {
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Release the window just read, every time.
//
// The ISO source accumulates and drops in chunks because it reads
// one file linearly, so a running start offset always names the
// bytes it has consumed. Reads here jump between files, so there is
// no single cursor to accumulate against — an accumulated byte
// count paired with one read's offset names 1/Nth of what was
// actually consumed and leaves the rest pinned, which is how the
// first version of this got it wrong.
//
// Dropping per read costs one advisory syscall per batch (4-16 MiB),
// which is nothing against the read itself, and it is correct
// regardless of how reads interleave across files.
// Release the window just read, every time (unlike the ISO source's
// accumulate-and-drop, which relies on one linear cursor — reads here
// jump between files, so an accumulated count would leave bytes pinned).
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
@@ -319,18 +293,14 @@ impl SectorSource for DirImage {
return Err(Error::UdfBufferTooSmall);
}
buf[..need].fill(0);
// Walk the request in RUNS, not sector by sector. A mux batch is 8192
// sectors and almost always lands entirely inside one stream file's
// extent; per-sector seek+read would issue 8192 syscalls for what is
// one 16 MiB sequential read.
// Walk the request in RUNS, not sector by sector: a mux batch (8192
// sectors) almost always lands in one extent, so per-sector seek+read
// would cost 8192 syscalls for what is one 16 MiB sequential read.
let mut i = 0u32;
while i < count as u32 {
// Checked: callers saturate their LBAs (`disc/dvd.rs` builds a cell
// start as `vob_start_sector.saturating_add(cell.first_sector)`, and
// the prefetcher adds an offset the same way), so a crafted IFO can
// present a request at the very top of the address space. Wrapping
// here would fold `at` back to a LOW sector and hand the muxer a
// different file's bytes with nothing reported.
// Checked: callers saturate their LBAs, so a crafted IFO can present a
// request at the top of the address space. Wrapping here would fold
// `at` to a LOW sector, handing the muxer a different file's bytes.
let Some(at) = lba.checked_add(i) else {
break;
};
+7 -17
View File
@@ -545,10 +545,8 @@ fn scan_image_enumerates_a_bdmv_folder() {
let (s, _, _) = bdmv_scratch();
let mut img = DirImage::open(s.path()).unwrap();
let cap = img.capacity_sectors();
// The playlist here is filler, so no titles are expected — what is being
// asserted is that the scan reaches the BD enumerator at all (it would
// return `UdfNotFilesystem` if the synthesized volume did not parse) and
// reports the structure it found.
// Playlist is filler, no titles expected; this asserts the scan reaches the
// BD enumerator at all (would return `UdfNotFilesystem` if unparseable).
let disc = crate::disc::Disc::scan_image(&mut img, cap, &crate::disc::ScanOptions::default())
.expect("scan_image must accept a synthesized BDMV image");
assert!(!disc.encrypted, "a decrypted folder has no AACS directory");
@@ -686,15 +684,8 @@ fn an_aacs_folder_with_scrambled_content_is_rejected() {
}
// ── The OTHER door: `dir://` through the PES input path ─────────────────────
//
// `session::scan_dir` (covered above) and `mux::resolve::input("dir://…")` are
// two doors into the same folder, and they once disagreed: a folder that ripped
// fine through one failed through the other, because only `scan_dir` re-judged
// the tree-shape encryption verdict from CONTENT. The fix was to share
// `session::apply_folder_encryption_verdict` between them — see its doc
// comment, which names this exact failure. Nothing tested the second door, so
// dropping the `is_folder` argument at `mux::resolve`'s call site restored the
// bug silently.
// `scan_dir` and `mux::resolve::input("dir://…")` once disagreed: only `scan_dir`
// re-judged the verdict from CONTENT. Fixed via `apply_folder_encryption_verdict`.
/// The two doors must AGREE. Same folder, same verdict, same extents.
///
@@ -976,10 +967,9 @@ fn dump_vts_ifo_reads_for_an_image() {
let p = format!("/VIDEO_TS/VTS_{n:02}_0.IFO");
match fs.read_file(&mut img, &p) {
Ok(b) => {
// Hash the CONTENT, not just the length: two IFOs of equal
// size and magic can still differ, and comparing only length
// and magic is what made an earlier pass of this diagnostic
// wrongly report the images as identical.
// Hash the CONTENT, not just length: two IFOs of equal size and
// magic can still differ, which is why an earlier pass of this
// diagnostic wrongly reported the images as identical.
let mut h: u64 = 0xcbf2_9ce4_8422_2325;
for byte in &b {
h ^= *byte as u64;
+71 -232
View File
@@ -117,26 +117,17 @@ impl Disc {
let mut extents = Vec::new();
let mut total_size: u64 = 0;
// Set when any clip resolves to a STREAM/SSIF/<clip>.ssif — a Blu-ray 3D
// interleaved stream carrying both the base (left) and MVC dependent
// (right) views. Drives reading the SSIF for both eyes and adding the
// dependent-view stream below.
// interleaved stream carrying both base and MVC dependent views. Drives
// reading the SSIF for both eyes and adding the dependent-view below.
let mut is_3d = false;
let mut clips = Vec::with_capacity(parsed.play_items.len());
// BD playlists legally reference the same .m2ts clip_id from
// multiple PlayItems (multi-angle, seamless splits, looped
// segments). The physical extents and packet count must be
// counted ONCE per unique clip — mux reads extents in order, so
// a duplicate would mux the A/V twice and inflate size_bytes.
// Per-PlayItem Clip entries (differing in/out times) still get
// recorded.
// BD playlists legally reference the same .m2ts clip_id from multiple
// PlayItems; extents/packet count must be counted ONCE per unique clip
// (mux reads in order) or a duplicate mux's the A/V twice, inflating size.
let mut seen_clips: std::collections::HashSet<String> = std::collections::HashSet::new();
// Byte offset of the next extent within the TITLE'S FEED — the
// concatenation of `extents` in the order the mux reads them. Each
// clip's span is recorded so a frame's source offset identifies its
// clip by lookup rather than by guessing from timestamps, which is
// ambiguous inside an overlap. A clip referenced a SECOND time pushes
// no extents (see `first_ref`), so it reuses the span of its first
// reference — the same bytes, read once.
// Byte offset of the next extent within the TITLE'S FEED (the concatenation
// of `extents` in read order). Each clip's span is recorded so a frame's
// source offset identifies its clip by lookup, not by ambiguous timestamps.
let mut feed_pos: u64 = 0;
let mut spans: std::collections::HashMap<String, (u64, u64)> =
std::collections::HashMap::new();
@@ -145,32 +136,22 @@ impl Disc {
let clip_dur = play_item.out_time.saturating_sub(play_item.in_time) as f64 / 45000.0;
let clpi_path = format!("/BDMV/CLIPINF/{}.clpi", play_item.clip_id);
// A `.clpi` that cannot be read or parsed is NOT a benign miss.
// `duration_ticks` was already summed from the PlayItems above, so
// the title keeps claiming its full runtime; skipping this block
// (what the old `if let Ok(..) && let Ok(..)` did) pushed no
// extents and added nothing to `total_size`, so the title shipped
// with the clip's bytes silently absent — and, because the `if let`
// DISCARDED the error, without a single log line to say so. Same
// classification as the extent resolver below: drop the title, and
// log the error's OWN code.
// A `.clpi` that cannot be read or parsed is NOT a benign miss:
// `duration_ticks` already claims the full runtime, so drop the
// title instead of silently shipping it missing bytes.
let clip_info = match udf_fs
.read_file(reader, &clpi_path)
.and_then(|clpi_data| clpi::parse(&clpi_data))
{
Ok(info) => info,
// The operator's Stop, not a disc defect. Every remaining
// command fails the same way once the drive's flag is set, so
// classifying it as an unresolvable clip would drop each
// remaining playlist in turn and hand back a truncated title
// list at success.
// The operator's Stop, not a disc defect: every remaining command
// fails the same way once the drive's flag is set, so classifying
// it as unresolvable would truncate the title list at success.
Err(Error::Halted) => return Err(Error::Halted),
Err(e) => {
// The REAL code, not a fixed one: a scratched CLIPINF
// sector (DiscRead), a missing `.clpi` (UdfNotFound) and a
// malformed one (ClpiParse) are different populations, and
// flattening them would send anyone triaging the first
// after the third.
// The REAL code, not a fixed one: DiscRead, UdfNotFound and
// ClpiParse are different populations, and flattening them
// would send anyone triaging the first after the third.
tracing::warn!(
target: "freemkv::disc",
playlist = ?filename,
@@ -182,12 +163,9 @@ impl Disc {
};
let pkt_count: u32 = clip_info.source_packet_count;
// The clip is marked seen only after its .clpi parses. That
// ordering used to matter because a transient failure on the first
// PlayItem referencing a clip must not permanently suppress its
// extents/size for a later PlayItem referencing the same clip;
// a failure now drops the whole title, so the ordering is kept
// simply because it is still the correct place for it.
// The clip is marked seen only after its .clpi parses. That ordering
// used to matter because a transient failure must not permanently
// suppress extents for a later PlayItem; kept as still-correct.
let first_ref = seen_clips.insert(play_item.clip_id.clone());
// Only fetch/push the physical extents and add to the
@@ -195,54 +173,13 @@ impl Disc {
if first_ref {
total_size += pkt_count as u64 * 192;
// Get stream file extents from UDF allocation descriptors.
// Dual-layer discs split files across layers — UDF knows the real layout.
//
// The clip's stream file is normally `.m2ts`, but AACS 2.1
// (FMTS) discs name the main feature `.fmts` and 3D discs
// use `.ssif` (see [`CLIP_STREAM_EXTS`]). A normal `.m2ts`
// clip is unchanged — the fallback only runs when `.m2ts`
// is absent, which is exactly when `file_extents` errors.
// 3D discs interleave the left (base) and right (MVC
// dependent) views in STREAM/SSIF/<clip>.ssif — note the
// SSIF/ subdir. Prefer it when present: the SSIF is one
// transport stream carrying BOTH eyes on distinct PIDs,
// so muxing it captures the full 3D. 2D clips fall back to
// the base .m2ts / .fmts as before.
// Get stream file extents from UDF allocation descriptors (dual-
// layer discs split files across layers). Normally `.m2ts`; AACS 2.1
// uses `.fmts`, 3D interleaves both eyes in STREAM/SSIF/<clip>.ssif.
let ssif = format!("/BDMV/STREAM/SSIF/{}.ssif", play_item.clip_id);
// A clip stream that carries an unrecorded (never-written)
// extent cannot be turned into a truthful read plan — see
// `UdfFs::file_extents`. Track it separately from an
// ordinary "file absent" error: absence is what the
// extension fallback exists for, whereas a hole means the
// bytes this title needs do not exist on the disc.
//
// ABSENCE is the only benign failure. A missing `.ssif` is
// the ordinary case (every 2D disc), and the extension
// fallback below exists precisely for it. Every OTHER
// error means the bytes this title needs could not be
// resolved: a scratched sector under the clip's ICB
// (DiscRead), an allocation-descriptor chain that never
// terminated (UdfAdChainTooLong), a file whose data is
// embedded rather than extent-mapped (UdfEmbeddedData).
// Those used to fall through to the not-found path, so the
// clip contributed zero extents while `total_size` and the
// play-item timing still counted it — a title advertising
// its full runtime with a piece silently missing, and no
// log line anywhere.
//
// `Halted` is NOT a disc defect and must not be treated as
// one either. It is the operator cancelling: once the flag is
// set, EVERY drive command returns it, so classifying it here
// would drop each remaining playlist in turn and hand back a
// truncated title list at success — the same shape this
// refusal exists to prevent, wearing a cancel. It used to be
// merely EXEMPTED (neither classified nor propagated, because
// this function returned `Option`); exempting it is no longer
// enough now that there is a channel. A cancel landing on
// `file_extents` would otherwise return a title with this
// clip's bytes silently missing — the flagship defect shape,
// wearing a cancel. It is propagated below instead.
// ABSENCE (`UdfNotFound`) is the only benign failure — the fallback
// exists for it. DiscRead/UdfAdChainTooLong/UdfEmbeddedData mean
// bytes are unresolved; `Halted` is the operator cancelling, propagated.
let mut unresolved: Option<u16> = None;
let mut halted = false;
let mut note = |e: &Error| {
@@ -271,26 +208,19 @@ impl Disc {
})
}
};
// Propagate the cancel BEFORE the classification below: a
// halted `file_extents` resolves nothing, so the title would
// otherwise be dropped (or, with a partial resolve, emitted
// short) and the scan would carry on as if the disc were at
// fault.
// Propagate the cancel BEFORE the classification below, or the
// title is dropped (or emitted short) and the scan carries on
// as if the disc were at fault.
if halted {
return Err(Error::Halted);
}
// Nothing resolved AND a hole was the reason: drop the
// whole title. Letting the clip contribute no extents
// (the ordinary not-found path) would emit a title whose
// feed is silently missing this clip's runtime while its
// durations, spans and size still count it — data loss
// wearing the shape of a normal rip.
// Nothing resolved AND a hole was the reason: drop the whole
// title, or the clip contributes no extents while durations/size
// still count it — data loss wearing the shape of a normal rip.
match (&file_exts, unresolved) {
(None, Some(code)) => {
// The REAL code, not a fixed one. A scratched sector
// (E6000) and an over-long AD chain (E6016) logged as
// E6017 would send anyone triaging them after authoring
// holes and hide the population that actually exists.
// The REAL code, not a fixed one: e.g. E6000 vs E6016
// logged as E6017 would hide the population that exists.
tracing::warn!(
target: "freemkv::disc",
playlist = ?filename,
@@ -299,32 +229,9 @@ impl Disc {
);
return Ok(None);
}
// A non-absence failure that the FALLBACK then papered
// over. `unresolved` was recorded and then thrown away:
// the `if let (None, Some(code))` above is the only reader
// of it, so when `/BDMV/STREAM/SSIF/<clip>.ssif` failed
// with `DiscRead` / `UdfAdChainTooLong` / `UdfEmbeddedData`
// but the `.m2ts` beside it resolved, the code was dropped
// on the floor. The title then shipped BASE-VIEW 2D off a
// disc that carries 3D, at rc=0, with not one log line —
// the operator gets a rip that looks complete and is
// missing the dependent view, and there is nothing in the
// journal to explain it or to distinguish it from a disc
// that was only ever 2D.
//
// This site's own doctrine, three paragraphs up, is
// "ABSENCE is the only benign failure" — `note()` already
// filters `UdfNotFound` out, so anything left in
// `unresolved` is by construction NOT benign. Reaching
// here means it was silently tolerated anyway.
//
// It is LOGGED, not refused. The fallback genuinely
// produced a truthful read plan for the base view, so the
// title is complete as 2D; dropping it would trade a
// degraded rip for no rip, which is the worse of the two
// and is not what the refusal above is for. The line
// carries the error's OWN code for the same reason as
// every other site in this file.
// A non-absence failure the FALLBACK papered over: `.ssif` failed
// non-benignly but `.m2ts` resolved. LOGGED, not refused — the
// fallback IS a truthful 2D read plan; dropping trades it for none.
(Some(_), Some(code)) => {
tracing::warn!(
target: "freemkv::disc",
@@ -337,32 +244,9 @@ impl Disc {
}
_ => {}
}
// KNOWN GAP, deliberately left open: `file_extents` can also
// return `Ok(vec![])`, or a vector every entry of which the
// `sectors > 0 && lba > 0` filter below discards (zero-length
// placeholder ADs — see `UdfFs::file_extents`). That clip then
// contributes no extents and no span while `total_size` and
// the play-item timing still count it: the same shape the
// refusal above exists to prevent, reached by a different
// route. It is NOT closed with a post-loop "every unique
// clip_id must appear in `spans`" invariant because it is not
// settled that an empty-but-Ok resolve is always a defect
// rather than a legitimate healthy-disc state, and dropping
// healthy titles is a worse failure than the residual gap.
// Recorded here so the next audit finds the decision instead
// of re-deriving it.
//
// ASYMMETRY, DELIBERATE: `disc::hddvd` CLOSES this same hole.
// Two things differ there. It already keeps an `unusable` set,
// so refusing costs nothing new (here it would mean inventing
// a post-loop "every clip_id must appear in `spans`"
// invariant). And the consequence is worse: an HD-DVD feature
// is COMPOSED from parts, so a missing part is spliced out of
// a title that keeps claiming the full runtime, whereas here
// the clip is one PlayItem of an otherwise whole title. Same
// hole, different price — see the long note at the
// `extents.is_empty()` branch in `hddvd.rs`. If BD ever grows
// an equivalent set, revisit this together with that one.
// KNOWN GAP, deliberately left open: an empty-but-Ok `file_extents`
// leaves the clip with no extents while size/timing still count it —
// not closed since it's unclear that's always a defect (see hddvd.rs).
if let Some(file_exts) = file_exts {
let span_start = feed_pos;
for (lba, sectors) in file_exts {
@@ -423,21 +307,13 @@ impl Disc {
// not special-cased here.
display_aspect: None,
secondary: s.secondary,
// No user-facing English in the library (numeric-code
// rule): the Dolby Vision enhancement layer is signalled
// structurally (secondary video + DolbyVision hdr) and
// the CLI/UI render the localized descriptor. `label`
// stays empty for disc video streams.
// No user-facing English (numeric-code rule): Dolby Vision
// is signalled structurally (secondary + DolbyVision hdr);
// `label` stays empty for disc video streams.
label: String::new(),
// TODO(spec): for 1080i HEVC/H.264/VC-1 titles, surface
// the measured field order (H.264/HEVC pic_struct, VC-1
// pulldown) from the codec parser instead of the TFF
// fallback; needs the parser→title channel (see dvd.rs).
// TODO(spec): prefer the HEVC/H.264 VUI colour_description
// (measured CICP) over this MPLS playlist-nibble guess
// once the parser surfaces it through the output title.
// `None` keeps the enum fallback. (HDR MaxCLL/Mastering
// metadata is a separate task and intentionally not here.)
// TODO(spec): surface measured field order instead of the TFF
// fallback (needs a parser→title channel, see dvd.rs); prefer
// measured VUI CICP over this MPLS nibble guess once surfaced.
measured_cicp: None,
})),
2 | 5 => {
@@ -487,22 +363,9 @@ impl Disc {
})
.collect();
// 3D: add the MVC dependent (right-eye) video stream. The base STN table
// lists only the left-eye video; the dependent view is a second video
// PID (stream_type 0x20) carried in the SSIF. The on-disc PAT/PMT are
// AACS-encrypted (unreadable pre-key) and the base STN omits the
// dependent view (it lives in the MPLS STN_table_SS), so we use the
// BD-3D PID convention: dependent = base-view video PID + 1
// (e.g. 0x1011 -> 0x1012). Reading the SSIF (above) provides its packets.
//
// Limitation: `is_3d` latches per PLAYLIST, not per clip. A playlist that
// mixed a 3D clip (has an SSIF) with a 2D clip (no SSIF) would tag the
// whole title 3D; the 2D clip's frames then mux as plain Blocks (no
// dependent PID → no BlockAdditional) under a track that still advertises
// the mvcC mapping. That output is valid (per-frame BlockAdditional is
// optional) but over-claims 3D for those frames. Real 3D main-feature
// playlists are single-clip or uniformly 3D, so this is not exercised;
// per-clip 3D would need per-clip stream sets (a larger change).
// 3D: add the MVC dependent (right-eye) stream, PID = base PID + 1 (base
// STN omits it; lives in MPLS STN_table_SS). `is_3d` latches per
// PLAYLIST, so a mixed 3D/2D playlist over-claims 3D for 2D frames.
if is_3d
&& let Some(base) = streams.iter().find_map(|s| match s {
Stream::Video(v) => Some(v.clone()),
@@ -523,16 +386,9 @@ impl Disc {
}
}
// Convert marks to chapters. mark_type == 1 is an entry-mark
// (chapter); type 2 is a link point and type 0 is reserved, so
// neither is a chapter.
//
// Each mark's timestamp is in the timebase of the PlayItem it
// references (play_item_ref). The chapter's position on the
// muxed timeline is the summed duration of every preceding
// PlayItem plus the mark's offset within its own PlayItem. Using
// play_items[0].in_time for every mark would misplace chapters in
// multi-PlayItem playlists.
// Convert marks to chapters: mark_type 1 is an entry-mark; 2/0 aren't.
// Each mark's timestamp is in its own PlayItem's timebase, so position
// sums preceding durations plus offset (not play_items[0].in_time).
let chapters: Vec<Chapter> = parsed
.marks
.iter()
@@ -632,11 +488,8 @@ impl Disc {
mod tests {
use super::*;
use crate::udf::fixture::*;
// ---------------------------------------------------------------
// MPLS builder (BD-ROM PlayList spec). Mirrors the layout the
// `mpls::parse` consumer reads (header@0, PlayList@playlist_start,
// PlayListMark@mark_start). Offsets cited against mpls.rs.
// ---------------------------------------------------------------
// MPLS builder (BD-ROM PlayList spec). Mirrors the layout `mpls::parse`
// reads (header@0, PlayList@playlist_start, PlayListMark@mark_start).
struct PiSpec {
clip_id: [u8; 5],
@@ -719,8 +572,9 @@ mod tests {
let mut item = Vec::new();
item.extend_from_slice(&pi.clip_id); // [0..5]
item.extend_from_slice(b"M2TS"); // [5..9] codec_id
item.push(0); // [9] connection_condition
item.extend_from_slice(&[0u8; 2]); // [10..12] reserved
item.push(0); // [9] reserved
item.push(0); // [10] is_multi_angle (bit 4) + connection_condition (low nibble)
item.push(0); // [11] stc_id / reserved
item.extend_from_slice(&pi.in_time.to_be_bytes()); // [12..16]
item.extend_from_slice(&pi.out_time.to_be_bytes()); // [16..20]
item.extend_from_slice(&[0u8; 8]); // [20..28] UO_mask
@@ -770,11 +624,8 @@ mod tests {
buf
}
// ---------------------------------------------------------------
// CLPI builder. `clpi::parse` reads "HDMV" magic, prog_info_start@12,
// cpi_start@16, source_packet_count@56. Zeroing prog_info/cpi starts
// disables those sections cleanly.
// ---------------------------------------------------------------
// cpi_start@16, source_packet_count@56. Zeroing prog_info/cpi disables them.
fn build_clpi(source_packet_count: u32) -> Vec<u8> {
let mut d = vec![0u8; 60];
@@ -856,9 +707,8 @@ mod tests {
);
}
/// Build a full BDMV tree with one STREAM/.m2ts (Long-AD ICB) and one
/// CLPINF/.clpi, returning the navigable UdfFs plus a populated disc.
/// This is the canonical 0.31.0 extent-assembly fixture.
/// Build the minimal fixture: an empty `BDMV/` directory (no STREAM,
/// no CLPINF), returning the navigable UdfFs plus a populated disc.
fn make_min_fs(disc: &mut MemDisc) -> udf::UdfFs {
// Empty BDMV/PLAYLIST so directory navigation in parse_playlist's
// clip lookups still works even when no clip files exist.
@@ -1362,17 +1212,9 @@ mod tests {
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
// Rewrite the .m2ts ICB (laid at PART_START + 100 by `lay_dir`)
// with a two-descriptor short-AD list:
// AD0: ECMA-167 4/14.14.1.1 type 1 (allocated, NOT recorded),
// length 0, at LBA 4999 — a zero-length descriptor that
// SURVIVES `read_icb_extents` (only a zero-length TYPE 0
// descriptor is the AD-list terminator), so it reaches
// `file_extents` as an extent of div_ceil(0, 2048) = 0
// sectors. This is the shape a crafted disc uses to put a
// readable-looking but empty range into a title's extent
// list.
// AD1: type 0, 4096 bytes at LBA 5000 — the real content.
// Rewrite the .m2ts ICB with a two-descriptor short-AD list: AD0 is
// a type-1 (allocated, not recorded) zero-length descriptor at LBA
// 4999 that survives `read_icb_extents`; AD1 is the real content.
let mut icb = build_file_icb(4096, 5000, false);
icb[212..216].copy_from_slice(&16u32.to_le_bytes()); // l_ad: two short ADs
icb[216..220].copy_from_slice(&0x4000_0800u32.to_le_bytes()); // type 1, 2048 bytes
@@ -1383,9 +1225,8 @@ mod tests {
udf::read_filesystem(&mut disc).expect("fs")
};
// The fixture must really carry the unrecorded descriptor, or the
// behaviour under test is never reached. `file_extents_addressing`
// shows what is there: the hole in its byte-space position, followed
// by the real content.
// behaviour under test is never reached: the hole's byte-space position,
// then the real content.
assert_eq!(
udf.file_extents_addressing(&mut disc, "/BDMV/STREAM/00001.m2ts")
.expect("extents"),
@@ -1466,10 +1307,9 @@ mod tests {
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
// Corrupt ONLY the descriptor tag, leaving a structurally valid
// ICB behind it. That is what an unreadable/garbled sector looks
// like to the parser, and it is deliberately NOT an unrecorded
// extent — the point is the error class the old code ignored.
// Corrupt ONLY the descriptor tag, leaving a structurally valid ICB
// behind it: an unreadable/garbled sector, deliberately NOT an
// unrecorded extent — the point is the error class the old code ignored.
let mut icb = build_file_icb(4096, 5000, false);
icb[0..2].copy_from_slice(&999u16.to_le_bytes());
disc.put_bytes(PART_START + 100, &icb);
@@ -2560,10 +2400,9 @@ mod tests {
&[],
&[],
);
// ONE sector is cancelled: the .m2ts ICB (metadata LBA 100), which is
// precisely and only where `file_extents` looks. The .clpi (ICB 102,
// data 8000) still reads, so the clip is sized and the earlier
// CLIPINF arm is not the thing under test here.
// ONE sector is cancelled: the .m2ts ICB (LBA 100), exactly where
// `file_extents` looks. The .clpi still reads, so the earlier CLIPINF
// arm is not the thing under test here.
let mut reader = HaltingReader {
inner: &mut disc,
halt_range: PART_START + 100..PART_START + 101,
+189 -111
View File
@@ -21,20 +21,33 @@ impl Disc {
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
///
/// Returns the scanned titles plus, when the disc's own First-Play
/// navigation deterministically dispatches to a title, that title's
/// `playlist_id` — the nav-resolved main feature. The caller feeds it into
/// `sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the
/// BD path uses. It is `None` (→ existing ranking heuristics stand) whenever
/// the navigation is menu-only, malformed, or does not converge.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
halt: Option<&crate::halt::Halt>,
) -> Result<Vec<DiscTitle>> {
) -> Result<(Vec<DiscTitle>, Option<u16>)> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok(Vec::new()),
Err(_) => return Ok((Vec::new(), None)),
};
// Follow the disc's First-Play navigation like a player would, to find the
// nav-chosen feature title (issue #40). Read-only; any failure/non-convergence
// yields `None`, so this only re-ranks, never removes, a title.
let nav_target = crate::dvdnav::resolve_main_title(reader, udf_fs);
let mut nav_feature: Option<u16> = None;
let mut titles = Vec::new();
let mut title_number: u16 = 0;
@@ -58,50 +71,27 @@ impl Disc {
crate::ifo::TvSystem::Pal => ColorSpace::Bt470bg,
crate::ifo::TvSystem::Ntsc => ColorSpace::Smpte170m,
},
// DVD pixels are anamorphic 720x480/576; the real display shape
// is the IFO aspect flag, not the pixel grid. Carry it so the
// MKV muxer writes a correct 16:9 / 4:3 DisplayWidth/Height
// instead of the square-pixel 3:2 / 5:4 it would otherwise emit.
// DVD pixels are anamorphic 720x480/576; real display shape comes from the
// IFO aspect flag, not the pixel grid. Carry it so the MKV muxer emits correct
// 16:9/4:3 DisplayWidth/Height instead of the wrong square-pixel 3:2/5:4.
display_aspect: Some(match ts.video.aspect {
crate::ifo::DvdAspect::R16x9 => (16, 9),
crate::ifo::DvdAspect::R4x3 => (4, 3),
}),
secondary: false,
label: String::new(),
// TODO(spec): populate from the MPEG-2 picture coding extension
// once the codec parser surfaces it. `Mpeg2Parser` already reads
// `top_field_first` (mux/codec/mpeg2.rs `picture_nb_fields`) but
// tracks are built from the IFO scan BEFORE any frame is parsed;
// wiring it requires a CodecParser accessor surfaced through
// PipelinedPesStream/DiscStream into the output title (mirroring
// the existing `codec_private` handshake). Until then `None`
// means the muxer falls back to TFF (correct for ~all DVDs).
// TODO(spec): DVD MPEG-2 carries no VUI; the colour signalling is
// the sequence_display_extension colour_description when present.
// Surface it from `Mpeg2Parser` (same handshake as above) and set
// this so a disc that states e.g. BT.601-625 colour overrides the
// PAL/NTSC guess in `color_space`. `None` uses the enum fallback.
// TODO(spec): populate top_field_first + measured colour from Mpeg2Parser once
// tracks are built after frame parsing -- needs a CodecParser->title channel
// like `codec_private`. Until wired, `None` falls back to TFF / PAL-NTSC guess.
measured_cicp: None,
});
// TODO(spec): DefaultDuration is derived from the declared IFO
// frame_rate (25 / 29.97). A soft-telecined 23.976-in-29.97 DVD then
// reports 29.97 fps instead of the true 23.976 film rate. The pulldown
// cadence is detectable from the parser's per-picture `nb_fields`
// (repeat_first_field) — when most frames are 3-field-then-2-field
// 2:3 pulldown the film rate is frame_rate × 4/5. Emitting the film
// DefaultDuration needs the parser to report the measured cadence
// through the same parser→title channel as `top_field_first` above;
// left as a follow-up to avoid a speculative rate change here.
// TODO(spec): DefaultDuration comes from IFO frame_rate, so a soft-telecined
// 23.976-in-29.97 DVD reports 29.97 instead of true film rate. Cadence is
// detectable via nb_fields (2:3 pulldown); needs same channel as top_field_first above.
// Map DvdAudioAttr to Stream::Audio. The PID is derived from the
// stream's REAL on-wire private_stream_1 sub-stream id (assigned
// by per-codec ordinal in the IFO scan) via the same
// `dvd_audio_pid` table the demuxer's `PsPacket::dvd_pid` uses,
// so a mixed-codec title (AC-3 + DTS + LPCM) routes correctly
// instead of colliding on 0xBD00. Streams carried as a regular
// MPEG-audio PES (MP1/MP2, no sub-id) fall back to a distinct
// 0xBD00+ordinal PID — disjoint from the 0xBD80+ canonical audio
// space — though they are not routed via `dvd_pid` today.
// Map DvdAudioAttr to Stream::Audio. PID derives from the on-wire private_stream_1
// sub-id via the same `dvd_audio_pid` table the demuxer uses, so mixed-codec
// titles route correctly instead of colliding; MP1/MP2 (no sub-id) use 0xBD00+ordinal.
let audio_streams: Vec<Stream> = ts
.audio_streams
.iter()
@@ -128,24 +118,24 @@ impl Disc {
for (vts_title_idx, dvd_title) in ts.titles.iter().enumerate() {
title_number += 1;
// If the nav executor picked this title (by VTS number + 1-based position,
// the same `vts_ttn` convention used below), remember its global
// `playlist_id` so the caller can promote it in ranking.
if let Some(rt) = nav_target
&& rt.vtsn == ts.vts_number
&& rt.vts_ttn as usize == vts_title_idx + 1
{
nav_feature = Some(title_number);
}
// Diagnostic dump (--log-level 3): per-cell category table +
// chapter map for this title, BEFORE lowering drops the
// per-cell IFO detail. No-op unless freemkv::diag is enabled.
crate::diag::dump_dvd_cells(ts.vts_number, title_number, dvd_title);
// Feature start cell. Prefer the DVD nav-VM resolver, which
// PARKED (#40, menu-at-start playback). The "menu at the start"
// symptom (e.g. SOTL) was a sector-mapping fault — the absolute
// VOB rebase in `ifo::parse_vts` (`vob_start_sector =
// file_start_lba + vtstt_vobs`) — NOT a navigation problem, so
// feature-start resolution is unnecessary for correct rips. The
// nav resolver + verified VM decoder (`dvdnav`) are kept compiled
// but deliberately bypassed; flip `USE_NAV_RESOLVER` to re-enable
// once the nav executor is finished. The fallback is the
// structural leading-cell filter (`feature_start_cell`), which
// drops leading scene-index / interleaved-angle sub-block cells
// and is a no-op for a normal feature (category 0x00 on cell 0).
// See `dvdnav::resolve_feature_start`.
// Feature-start nav resolution is PARKED (#40): the "menu at start" symptom
// was actually a sector-mapping fault in `ifo::parse_vts`'s VOB rebase, not
// navigation. Bypassed via `USE_NAV_RESOLVER = false`; fallback is the filter.
const USE_NAV_RESOLVER: bool = false;
let feature_start = if USE_NAV_RESOLVER {
crate::dvdnav::resolve_feature_start(
@@ -193,14 +183,9 @@ impl Disc {
let size_bytes: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
// Build pre-formatted VobSub `.idx` codec_data (size: + palette:
// lines) for VobSub subtitle streams. The `size:` line carries
// the coded video frame the subpicture was authored against
// (720x480 NTSC / 720x576 PAL) so players place and scale the
// bitmap correctly.
// format_palette guards on (0, 0) and omits its `size:` line,
// so an unresolved resolution degrades to a palette-only .idx
// rather than one claiming a 0x0 frame.
// Build pre-formatted VobSub `.idx` codec_data. The `size:` line carries the
// coded video frame (720x480 NTSC / 720x576 PAL) so players place/scale the
// subpicture correctly; format_palette omits it on unresolved (0,0).
let (vid_w, vid_h) = ts.video.resolution.pixels().unwrap_or((0, 0));
let codec_data = dvd_title
.palette
@@ -233,10 +218,9 @@ impl Disc {
streams.extend(audio_streams.iter().cloned());
streams.extend(subtitle_streams);
// Chapter times are absolute from the PGC start. When leading
// cells are dropped the muxed video shifts earlier by exactly
// their total duration, so shift the chapter marks too (clamping
// any that fell inside the dropped head to 0).
// Chapter times are absolute from the PGC start. When leading cells are
// dropped, the muxed video shifts earlier by their total duration, so shift
// chapter marks too (clamping any that fell inside the dropped head to 0).
let chapters: Vec<Chapter> = dvd_title
.chapter_times
.iter()
@@ -262,14 +246,13 @@ impl Disc {
}
}
// Polled again AFTER the loop: a cancel raised during the IFO reads
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
// Polled again AFTER the loop: a cancel raised during per-title-set IFO reads in
// `parse_vmg` has nothing left to poll otherwise, so without this a partially
// enumerated disc could still be handed back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok(titles)
Ok((titles, nav_feature))
}
}
@@ -279,11 +262,9 @@ mod tests {
use crate::sector::SectorSource;
use std::collections::HashMap;
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image (single physical partition,
// metadata_start == partition_start). Offsets cited against
// udf.rs::read_filesystem / ECMA-167.
// ---------------------------------------------------------------
const PART_START: u32 = 3000;
@@ -597,10 +578,9 @@ mod tests {
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
// Two title sets: VTS_01's IFO at PART_START+6000, VTS_02's at PART_START+7000.
// Both ICBs sit far below, so filesystem metadata resolves fine and only the
// second title set's CONTENT read is cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
@@ -633,6 +613,7 @@ mod tests {
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0
.len(),
2,
"fixture must offer two title sets"
@@ -647,7 +628,7 @@ mod tests {
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|ts| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
res.map(|(ts, _)| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
@@ -662,7 +643,7 @@ mod tests {
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|ts| ts.len())
res.map(|(ts, _)| ts.len())
);
}
@@ -676,6 +657,7 @@ mod tests {
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0
.is_empty()
);
}
@@ -713,7 +695,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -739,7 +723,7 @@ mod tests {
/// before the title VOB. Cell `first_sector` values are relative to
/// `vtstt_vobs`; reading 0xC0 prepended the menu and shifted every extent
/// back by `vtstt_vobs - vtsm_vobs`, so the rip opened on the parental
/// prompt instead of the movie (Greenland NTSC R1: vtsm=44, vtstt=3640).
/// prompt instead of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
///
/// Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); we additionally stamp
/// a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve from 3640.
@@ -769,7 +753,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -789,7 +775,7 @@ mod tests {
);
}
/// ABSOLUTE-REBASE regression (THESILENCEOFTHELAMBS / Greenland fix):
/// ABSOLUTE-REBASE regression (real-disc menu-at-start fix):
/// `ifo::parse_vts` now sets `vob_start_sector = file_start_lba(IFO) +
/// vtstt_vobs`, so an extent's `start_lba` must equal the sum of THREE
/// independent terms — the IFO file's absolute on-disc LBA, the
@@ -827,7 +813,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -885,7 +873,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -926,7 +916,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let v = t
.streams
.iter()
@@ -981,7 +973,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let v = t
.streams
.iter()
@@ -1011,12 +1005,9 @@ mod tests {
fn scan_dvd_titles_mixed_audio_codecs_distinct_pids() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// audio b0: coding_mode is (b0 >> 5) & 7. AC-3 = 0 → b0=0x00.
// DTS = 6 → b0 = 6<<5 = 0xC0. b1 channels nibble = (channels-1) in bits
// 2-0: a REAL 5.1 layout is 5 (0x05), a REAL 2.0 is 1 (0x01). The old
// fixture used 0x10/0x50 (both decode to 1 channel) — a placeholder that
// would pass even against code mishandling channel counts. Pin real
// layouts: AC-3 5.1 eng, DTS 2.0 fra.
// audio b0: coding_mode = (b0>>5)&7 (AC-3=0->0x00, DTS=6->0xC0). b1 channels nibble
// = (channels-1) in bits 2-0 (5.1=0x05, 2.0=0x01). Old fixture used 0x10/0x50
// (both -> 1ch), a placeholder that passed even with broken channel handling.
let vts = build_vts(
0,
0x00,
@@ -1042,7 +1033,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1067,10 +1060,9 @@ mod tests {
2,
"DTS 2.0 nibble must decode to 2 channels"
);
// PIDs route via the positional sub-id table: AC-3 @ pos 0 → 0x80 →
// 0xBD80, DTS @ pos 1 → 0x89 → 0xBD89 (the shared audio-stream number
// in the low nibble, NOT a per-codec ordinal). Distinct AND the exact
// canonical wire PIDs the demux routes on.
// PIDs route via the positional sub-id table: AC-3 @ pos 0 -> 0x80 -> 0xBD80, DTS @
// pos 1 -> 0x89 -> 0xBD89 (shared audio-stream number in the low nibble, NOT a
// per-codec ordinal). Distinct AND the exact canonical wire PIDs the demux routes on.
assert_eq!(audios[0].pid, 0xBD80, "AC-3 @ pos 0 → 0xBD80");
assert_eq!(audios[1].pid, 0xBD89, "DTS @ pos 1 → 0xBD89");
assert_ne!(audios[0].pid, audios[1].pid);
@@ -1112,7 +1104,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1169,7 +1163,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1229,7 +1225,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let sub = t
.streams
.iter()
@@ -1252,11 +1250,9 @@ mod tests {
#[test]
fn scan_dvd_titles_numbering_increments_per_title() {
let mut disc = MemDisc::new();
// Two titles in VTS 1 (title nums 1 and 2). num_pgcs must cover
// pgc_index = vts_title - 1, so we need >=2 PGC entries; our
// build_vts only emits 1 PGC. So the second title's PGC index (1)
// exceeds num_pgcs (1) and is skipped. To exercise numbering we use
// two separate VTS sets instead.
// Two titles in VTS 1 (nums 1,2): pgc_index = vts_title-1 needs >=2 PGC entries, but
// build_vts only emits 1, so the second title's PGC index (1) exceeds num_pgcs (1)
// and is skipped. Use two separate VTS sets instead to exercise numbering.
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
@@ -1283,7 +1279,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1296,6 +1294,79 @@ mod tests {
assert_eq!(titles[1].extents[0].start_lba, 10200); // 10000 + 200
}
/// Stamp a First-Play PGC into a VMG whose pre-command list is a single
/// unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
/// title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
/// TT_SRPT pointers `build_vmg` already writes.
fn stamp_first_play_jumptt(vmg: &mut [u8], ttn: u8) {
const FP_PGC_PTR: usize = 0x84; // u32 byte offset of the First-Play PGC
const CMD_TBL_PTR: usize = 0xE4; // u16 cmd-table offset (rel to PGC)
let fp_pgc: u32 = 0x200;
let cmd_tbl_rel: u16 = 0x100;
vmg[FP_PGC_PTR..FP_PGC_PTR + 4].copy_from_slice(&fp_pgc.to_be_bytes());
let pgc = fp_pgc as usize;
vmg[pgc + CMD_TBL_PTR..pgc + CMD_TBL_PTR + 2].copy_from_slice(&cmd_tbl_rel.to_be_bytes());
let tbl = pgc + cmd_tbl_rel as usize;
vmg[tbl..tbl + 2].copy_from_slice(&1u16.to_be_bytes()); // nr_of_pre = 1
// JumpTT command (type 1, direct=1, cmd=2): ttn in byte 5.
vmg[tbl + 8..tbl + 16].copy_from_slice(&[0x30, 0x02, 0, 0, 0, ttn, 0, 0]);
}
/// The DVD First-Play nav promotion branch (issue #40): when
/// `resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
/// must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
/// the matching `vts_number` — to the running `title_number`, and surface it
/// as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
/// the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
/// the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
/// join or a vtsn/vts_ttn field swap would promote the wrong title or none.
#[test]
fn scan_dvd_titles_nav_promotes_first_play_target() {
let mut disc = MemDisc::new();
// TT_SRPT: title 1 → VTS 1 (in-set title 1); title 2 → VTS 2 (in-set 1).
let mut vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
stamp_first_play_jumptt(&mut vmg, 2); // First-Play → JumpTT title 2
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
let (titles, nav_feature) = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// Sanity: the resolver reached the branch with the VTS-2 target.
assert_eq!(
crate::dvdnav::resolve_main_title(&mut disc, &udf),
Some(crate::dvdnav::nav::ResolvedTitle {
title: 2,
vtsn: 2,
vts_ttn: 1,
})
);
// The promotion mapped that target to the second scanned title.
assert_eq!(nav_feature, Some(2));
assert_eq!(titles[1].playlist_id, 2);
}
/// chapter_times from the IFO become Chapter entries with ordinal
/// names (dvd.rs maps chapter_times → Chapter{time_secs, chapter_name}).
#[test]
@@ -1320,7 +1391,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1375,9 +1448,8 @@ mod tests {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(2, 1, 1)]);
// Cell 0: leading scene-index/angle sub-block (cat 0x90), 5s, sectors 0..9.
// Cell 1: feature start (cat 0x00), 59s, sectors 100..199.
// Cell 2: feature (cat 0x00), 59s, sectors 300..399.
// Programs: prog0 → cell 1 (feature start), prog1 → cell 3.
// Cell 1: feature start (cat 0x00), 59s, 100..199. Cell 2: feature, 59s, 300..399.
// Programs: prog0 -> cell 1 (feature start), prog1 -> cell 3.
let vts = build_vts_cells(
1000,
0x00,
@@ -1405,7 +1477,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1429,7 +1503,7 @@ mod tests {
/// Conservative guard end-to-end: a normal feature (every cell category
/// 0x00) is muxed in full — the filter drops nothing and chapters are
/// unshifted. This is the "Silence of the Lambs" case.
/// unshifted. This is the plain-single-angle-feature case.
#[test]
fn scan_dvd_titles_plain_feature_untouched() {
let mut disc = MemDisc::new();
@@ -1457,7 +1531,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1500,7 +1576,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
+14 -19
View File
@@ -1,6 +1,6 @@
//! Physical AC-3 sub-stream probing for DVD audio routing.
//!
//! ## Why this exists (Silence-of-the-Lambs wrong-substream bug)
//! ## Why this exists (a real-disc wrong-substream bug)
//!
//! A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
//! scan assigns each declared stream a `private_stream_1` sub-stream id purely
@@ -8,7 +8,7 @@
//! `0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
//! physical sub-stream order on the wire matches the IFO declaration order.
//!
//! On some discs it does NOT. The R2 PAL "The Silence of the Lambs" feature
//! On some discs it does NOT. A real R2 PAL DVD release's feature
//! declares ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but
//! the physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
//! `0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
@@ -34,7 +34,7 @@ use std::collections::BTreeMap;
/// a DVD feature opens with logos/warnings whose audio is frequently a thin 2.0
/// bed on the FIRST sub-stream only — the other physical `0x8x` sub-streams and
/// the main 5.1 mix do not appear until a sector or two further in. 512 sectors
/// (1 MiB) was too short: on Greenland it saw ONLY `0x80`, and only its opening
/// (1 MiB) was too short: on one real disc it saw ONLY `0x80`, and only its opening
/// 2.0 frames. 1024 sectors (2 MiB) reliably contains at least one frame of
/// every physical AC-3 sub-stream AND enough of `0x80` to reach its 5.1 frames.
/// Still bounded so a live drive is never hammered (see the project "don't
@@ -54,7 +54,7 @@ const PROBE_SECTORS: u16 = 1024;
/// The first frame of a sub-stream at the head of a feature is NOT
/// representative. A DVD opens with logos/warnings, and the main `0x80`
/// sub-stream there frequently carries a thin 2.0 bed before transitioning to
/// its real 5.1 main mix a fraction of a second later (observed on Greenland:
/// its real 5.1 main mix a fraction of a second later (observed on a real disc:
/// `0x80`'s first frames are acmod=2 → 2 channels, then it becomes acmod=7+lfe →
/// 6 channels within the same 2 MiB window). Recording only the FIRST frame read
/// `0x80=2` and missed the 5.1 entirely, defeating the channel-match routing.
@@ -75,11 +75,9 @@ pub fn probe_ac3_substream_channels(ps_bytes: &[u8]) -> BTreeMap<u8, u8> {
if !(0x80..=0x87).contains(&sub) {
continue;
}
// The PS demux strips the 4-byte AC-3 sub-header but does not align to a
// frame. Walk EVERY 0x0B77 sync in this sub-stream's payload, decode
// each frame's channel count, and keep the largest — the sub-stream's
// real (main-mix) channel capability. See the doc comment above for why
// the first frame alone is unreliable.
// The PS demux strips the AC-3 sub-header but doesn't align to a frame, so
// walk every 0x0B77 sync in the payload and keep the largest decoded
// channel count — the sub-stream's real main-mix capability (see doc above).
if let Some(ch) = max_substream_channels(&p.data) {
let slot = found.entry(sub).or_insert(0);
*slot = (*slot).max(ch);
@@ -202,7 +200,7 @@ pub fn remap_audio_pids(streams: &mut [Stream], probed: &BTreeMap<u8, u8>) -> us
///
/// `reader` MUST yield PLAINTEXT VOB bytes (i.e. a `DecryptingSectorSource` on a
/// CSS disc) — probing scrambled sectors yields no AC-3 syncs and is a safe
/// no-op. Returns the number of audio streams whose PID changed.
/// no-op. Remaps `title`'s AC-3 streams in place; returns nothing.
pub fn probe_and_remap<S: SectorSource + ?Sized>(
reader: &mut S,
title: &mut crate::disc::DiscTitle,
@@ -347,7 +345,7 @@ mod tests {
assert_eq!(probed.get(&0x81), Some(&6), "0x81 is the 5.1 main mix");
}
/// GREENLAND regression — the probe must read each sub-stream's TRUE
/// Real-disc regression — the probe must read each sub-stream's TRUE
/// (max-mix) channel count, not be poisoned by an unrepresentative head
/// frame, and must NOT cross-contaminate between sub-streams.
///
@@ -500,12 +498,9 @@ mod tests {
#[test]
fn max_substream_channels_unmappable_size_steps_forward_by_two() {
let mut real = ac3_frame(2, false);
// Overwrite the (unchecked) CRC bytes of the real frame — these double
// as byte4/byte5 of the bogus header 2 bytes earlier, at absolute
// offset 4: byte4 = 0xC0 (fscod=3 reserved -> ac3_frame_size == 0,
// unmappable), byte5 = 0xF8 (bsid=31 >= 11 -> acmod_channels == None,
// so the bogus header itself never contributes a spurious channel
// count).
// Overwrite the real frame's (unchecked) CRC bytes, which double as byte4/5
// of the bogus header at offset 4: 0xC0 (fscod=3 -> ac3_frame_size == 0,
// unmappable) and 0xF8 (bsid=31 -> acmod_channels == None, no spurious count).
real[2] = 0xC0;
real[3] = 0xF8;
let mut data = vec![0xAA, 0xAA, 0xAA, 0xAA]; // offsets 0..4, no sync
@@ -590,7 +585,7 @@ mod tests {
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// End-to-end `probe_and_remap`: a real-disc-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
@@ -603,7 +598,7 @@ mod tests {
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
fn probe_and_remap_reroutes_swapped_substream_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
+52 -109
View File
@@ -102,11 +102,9 @@ impl AacsCertUnlocker<'_> {
.ok()
.and_then(|m| aacs::mkb::mkb_version(&m));
// Host certs are keysource-served, never compiled in — unioned from the
// explicit `DriveCredentials` and the key-source layer. With ZERO certs
// the cert route cannot run: NoHostCert (folded to AacsNoHostCert by the
// caller, preserving the graceful path-1 disc-hash → VUK fallback). This
// is detected here, where the MKB generation is still in hand.
// Host certs are keysource-served, never compiled in — unioned from
// `DriveCredentials` and the key-source layer. With ZERO certs, NoHostCert
// (folded to AacsNoHostCert) preserves the graceful disc-hash → VUK fallback.
let host_certs = Disc::collect_host_certs(self.opts, mkb_gen);
if host_certs.is_empty() {
tracing::warn!(
@@ -117,10 +115,9 @@ impl AacsCertUnlocker<'_> {
return Err(CertUnlockFailure::NoHostCert { mkb: mkb_gen });
}
// Hand the collected certs to the AACS unlocker. The cert-route bus
// removal depends on the read_data_key, NOT a drive unlock. The
// borrow checker can't split `session` across `scsi_mut()` + `&drive_id`
// through method calls, so clone the (cheap) identity first.
// Hand the collected certs to the AACS unlocker; the cert-route bus removal
// depends on read_data_key, NOT a drive unlock. Borrow checker can't split
// `session` across `scsi_mut()` + `&drive_id`, so clone the cheap identity.
let drive_id = session.drive_id.clone();
let fu_certs = crate::unlock_bridge::map_host_certs(&host_certs);
let (_, unlock_res) = crate::unlock_bridge::run_bus(
@@ -251,35 +248,13 @@ impl Disc {
/// read_data_key for bus decryption must still use the cert path, so an
/// unlocker with no OEM VID capability returns `None` and we fall through
/// to cert auth unchanged.
/// Collect every AACS host cert the caller carries, from BOTH the explicit
/// [`DriveCredentials`] and the key-source layer
/// ([`crate::KeySource::host_certs`] across each source), unioned. Host certs
/// are keysource-served, never compiled in; this is the one place the OEM
/// cert route gathers them. An empty result is the graceful no-cert signal
/// (the caller turns it into [`Error::AacsNoHostCert`]).
/// `mkb` is the disc's MKB generation when known, forwarded to each source's
/// [`crate::KeySource::host_certs`] so a source MAY return only
/// generation-appropriate certs (the default ignores it).
fn collect_host_certs(
opts: &ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::types::HostCert> {
// Delegates to the shared cert primitive (the external freemkv-unlock-aacs
// plugin uses the same one). Kept as a thin Disc method so the existing
// collect_host_certs_* unit tests and call sites are unchanged.
crate::aacs::host_certs::collect_host_certs(opts, mkb)
}
fn do_handshake_cert(
session: &mut crate::drive::Drive,
opts: &ScanOptions,
) -> (Option<HandshakeResult>, Option<Error>) {
// OEM VID shortcut: a matching unlocker stashed the disc's Volume ID at
// drive `init()` (the new `unlock()` folds in the old `read_volume_id`).
// Use it and SKIP the cert handshake — the OEM path
// decouples the VID from the host cert + HRL. It yields no
// `read_data_key`; a bus-encrypted disc that needs the bus key is caught
// by the bus-key gate in `resolve_vid_only`.
// drive `init()`; use it and SKIP the cert handshake. It yields no
// `read_data_key`, caught by the bus-key gate in `resolve_vid_only`.
if let Some(volume_id) = session.oem_vid() {
tracing::debug!(
target: "freemkv::disc",
@@ -307,12 +282,9 @@ impl Disc {
"No drive-unlocker Volume ID; running the in-tree AACS host-certificate handshake (AacsCertUnlocker)."
);
// Cert path: the in-tree `AacsCertUnlocker` peer absorbs the host-cert
// mutual-auth. It collects host certs from the key sources + credentials,
// runs `aacs_authenticate` per cert (wedge-guarded), and on success reads
// the VID + read_data_key. Its `UnlockError` is folded back to the same
// `Error` variants this function has always surfaced, so `scan_with`'s
// error rendering and the path-1 disc-hash → VUK fallback are unchanged.
// Cert path: `AacsCertUnlocker` collects host certs, runs
// `aacs_authenticate` per cert, and on success reads VID + read_data_key.
// Its `UnlockError` folds back to this function's usual `Error` variants.
let unlocker = AacsCertUnlocker { opts };
match unlocker.authenticate(session) {
Ok(hs) => (Some(hs), None),
@@ -328,6 +300,25 @@ impl Disc {
}
}
/// Collect every AACS host cert the caller carries, from BOTH the explicit
/// [`DriveCredentials`] and the key-source layer
/// ([`crate::KeySource::host_certs`] across each source), unioned. Host certs
/// are keysource-served, never compiled in; this is the one place the OEM
/// cert route gathers them. An empty result is the graceful no-cert signal
/// (the caller turns it into [`Error::AacsNoHostCert`]).
/// `mkb` is the disc's MKB generation when known, forwarded to each source's
/// [`crate::KeySource::host_certs`] so a source MAY return only
/// generation-appropriate certs (the default ignores it).
fn collect_host_certs(
opts: &ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::types::HostCert> {
// Delegates to the shared cert primitive (the external freemkv-unlock-aacs
// plugin uses the same one). Kept as a thin Disc method so the existing
// collect_host_certs_* unit tests and call sites are unchanged.
crate::aacs::host_certs::collect_host_certs(opts, mkb)
}
/// Build a keys-free AACS state that carries only the Volume ID (+ version
/// metadata), for callers that resolve Unit Keys out-of-band and have
/// disabled the local keydb. The VID is on-disc content read during the
@@ -355,36 +346,17 @@ impl Disc {
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let bus_encryption = cc.as_ref().map(|c| c.bus_encryption).unwrap_or(false);
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so the
// scanned `AacsState.version` and the out-of-band fetch agree. A wrong
// stride on the main resolve path fails loudly (sample validation) rather
// than silently, so the conservative V20 default is safe here too.
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so
// scanned/out-of-band versions agree; the main path fails loudly on a
// wrong stride, so this conservative default is safe here too.
let version = cc
.as_ref()
.map(|c| c.version.major())
.unwrap_or(aacs::mkb::AACS_MAJOR_UHD);
// Bus-encryption gate (wrong-keys guard). A bus-encrypted disc (Content
// Certificate bus-encryption bit set) carries bus encryption on its
// sectors, which MUST be removed before any AACS key can decrypt them.
// There are TWO ways it gets removed, and bus encryption is unremovable
// only when NEITHER succeeded:
// 1. An unlocker unlocked the drive → it serves CLEAR content
// (`drive_unlocked`). This is the common live-drive case and yields
// no `read_data_key` — it doesn't need one.
// 2. The AACS host-certificate cert-auth handshake produced the bus key
// (`read_data_key`).
// The old gate credited ONLY (2), so a SUCCESSFUL drive unlock (VID
// present, `read_data_key: None`, `drive_unlocked: true`) tripped it and
// blocked ALL key resolution — including the online source — even though
// the drive was serving clear content. That was the bug.
//
// Also skipped when `handshake = None` (file-backed/ISO scans — bus
// encryption already removed at read time) and when `bus_encryption` is
// false (AACS 1.0 BD is not bus-encrypted).
// ONE question — "is AACS bus encryption gone?" — asked of the single
// `bus_encryption_removed` predicate, which OWNS every case (never had it,
// file/ISO, drive unlock, cert bus key). The gate enumerates nothing.
// Bus-encryption gate: removed by a drive unlock (`drive_unlocked`) or a
// cert handshake `read_data_key`. The old gate credited only the latter,
// wrongly blocking key resolution after a successful drive unlock.
if !bus_encryption_removed(bus_encryption, handshake) {
let (rdk_err, has_vid) = handshake
.map(|h| (h.read_data_key_err, handshake_has_volume_id(h)))
@@ -400,20 +372,9 @@ impl Disc {
);
return Err(Error::AacsBusKeyUnavailable);
}
// Read the MKB record stream via the SAME bounded reader the
// out-of-band `read_aacs_inputs` uses (`read_mkb_content`: a prefix-grow
// read + trim), NOT a full `read_file`. MKB_RO/RW is allocated to a
// fixed ~128 MiB of zero padding, and a full `read_file` of it FAILS on
// file-backed / large readers — which left `a.mkb` empty here, silently
// breaking online key resolution: `Disc::inputs()` shipped `mkb=0` to
// the decode service and it 404'd, while autorip's separate
// `read_aacs_inputs` path (this same helper) worked. One reader now, so
// `Disc::inputs()` is the single complete source of AACS inputs.
// A read ERROR is surfaced (logged), not silently emptied: an empty MKB
// here is invisible until an online key service rejects the request, so
// a transient I/O hiccup must not masquerade as "no MKB". We still
// continue with an empty MKB (disc-hash-keyed keydb lookups don't need
// it), but the cause is now on the log.
// Use the SAME bounded reader as `read_aacs_inputs` (`read_mkb_content`),
// NOT `read_file` (fails on the ~128 MiB zero-padded MKB_RO/RW, silently
// emptying `a.mkb`). A read ERROR is logged; we still continue with an empty MKB.
let mkb_bytes = match Self::read_mkb_content(reader, udf_fs) {
Ok(m) => m,
Err(e) => {
@@ -484,11 +445,8 @@ mod tests {
);
}
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical
// partition (metadata_start == partition_start). Offsets cited
// against udf.rs::read_filesystem / ECMA-167.
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical partition
// (metadata_start == partition_start), per udf.rs::read_filesystem / ECMA-167.
const PART_START: u32 = 4000;
@@ -856,12 +814,9 @@ mod tests {
assert_eq!(st.read_data_key, Some(rdk));
}
// ---------------------------------------------------------------
// OEM bus-key gate: a bus-encrypted disc scanned on a LIVE drive
// (handshake present) with no read_data_key must HARD-ERROR
// (AacsBusKeyUnavailable) rather than silently yield garbage. The
// three non-regressing cases must still succeed.
// ---------------------------------------------------------------
// OEM bus-key gate: a bus-encrypted disc on a LIVE drive with no
// read_data_key must HARD-ERROR (AacsBusKeyUnavailable); three
// non-regressing cases must still succeed.
fn disc_with_cert(cert_type: u8, bus_encryption: bool) -> (MemDisc, udf::UdfFs) {
let mut disc = MemDisc::new();
@@ -965,8 +920,8 @@ mod tests {
assert_eq!(st.read_data_key, None);
}
/// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy
/// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`).
/// `handshake_has_volume_id` treats an all-zero Volume ID as absent and
/// any non-zero Volume ID as present.
#[test]
fn handshake_has_volume_id_reports_presence_not_absence() {
let with_vid = HandshakeResult {
@@ -1017,21 +972,12 @@ mod tests {
assert!(matches!(err, Error::AacsBusKeyUnavailable));
}
// ---------------------------------------------------------------
// Tests: read_vid_oem (response parsing). The OEM path issues a
// READ_BUFFER CDB and parses a 36-byte response; we can't easily
// fixture a real Drive, but the response-shape contract (3-byte
// signature 00 22 00, VID at [4..20]) is documented and worth a
// direct guard via a fake transport. Skipped here because Drive
// construction requires a live transport; the parsing branches are
// Tests: read_vid_oem (response parsing) are skipped here since Drive
// construction requires a live transport; parsing branches are
// exercised through `read_vid_oem`'s callers in integration.
// ---------------------------------------------------------------
// ---------------------------------------------------------------
// Tests: collect_host_certs — the OEM cert route's cert-gathering.
// Unions DriveCredentials with the key-source layer; empty means
// the route fails gracefully (AacsNoHostCert), never panics.
// ---------------------------------------------------------------
// Tests: collect_host_certs — unions DriveCredentials with the key-source
// layer; empty means the route fails gracefully (AacsNoHostCert).
fn fake_cert(tag: u8) -> aacs::types::HostCert {
aacs::types::HostCert {
@@ -1109,11 +1055,8 @@ mod tests {
assert_eq!(tags, vec![1, 2, 3]);
}
// ---------------------------------------------------------------
// AacsCertUnlocker outcome mapping: UnlockError → Error (preserving
// the legacy do_handshake_cert surface) and → UnlockOutcome (the
// structured trace step). No English in either.
// ---------------------------------------------------------------
// AacsCertUnlocker outcome mapping: UnlockError → Error (legacy surface)
// and → UnlockOutcome (structured trace step). No English in either.
#[test]
fn unlock_error_maps_to_legacy_error_variants() {
+64 -198
View File
@@ -190,28 +190,9 @@ impl Disc {
// disc-wide keys for every file.
let mut base_keys = self.decrypt_keys();
// AACS key map for the extract, chosen by CPS-unit count:
//
// * SINGLE CPS (the overwhelming majority, incl. every single-key UHD): one
// Unit Key opens EVERY encrypted unit on the disc — content in a parsed
// title AND an orphan clip that no playlist references. A blanket key-0
// map over the whole LBA space is exact and covers orphans; clear
// filesystem/nav (encrypted-flag off) passes through untouched.
//
// * MULTI-CPS: each clip is protected by a different Unit Key, so a blanket
// key-0 map would mis-decrypt every secondary-CPS file into garbage
// (silently, since Phase::All is trust-only). Build the EXACT per-CPS
// content map instead (each title's extents → the CPS key that opens a
// real sample from it), up front before the decorator takes the reader. A
// content unit whose key the pool lacks fails loud at resolve (extract has
// no CPS/forensic fetch source), never emits a wrong-key garble.
//
// KNOWN LIMITATION (by design): an orphan encrypted clip on a multi-CPS
// disc — referenced by no playlist, so in no title extent — is in no range
// and passes through as ciphertext. There is no correct key to apply (its
// CPS unit is unknown without a playlist reference), and blind trial-decrypt
// is exactly what this keymap-only model removes. Single-CPS is unaffected
// (the blanket key-0 map above covers orphans).
// AACS key map, by CPS-unit count. Single-CPS: one Unit Key opens
// everything, so a blanket key-0 map covers even orphan clips. Multi-CPS
// builds an exact per-title map instead (a blanket map would mis-decrypt).
let key_map =
match &base_keys {
DecryptKeys::Aacs { unit_keys, .. } if unit_keys.len() <= 1 => {
@@ -225,10 +206,9 @@ impl Disc {
_ => None,
};
// ── Phase 2: stream each file through the decrypting decorator ────
// The decorator owns its inner source for its lifetime. We hand it a
// borrowing wrapper (so the caller keeps `reader`), swap keys per CSS
// VTS group via `set_keys`; AACS/None keep `base_keys` throughout.
// Phase 2: stream each file through the decrypting decorator, which owns a
// borrowing wrapper so the caller keeps `reader`. Keys swap per CSS VTS
// group via `set_keys`; AACS/None keep `base_keys` throughout.
let mut dec = DecryptingSectorSource::new(Borrowed(reader), base_keys.clone());
if let Some(map) = key_map {
dec = dec.with_key_map(map);
@@ -270,10 +250,8 @@ impl Disc {
}
}
// A unit that fails to decrypt fails the read loud (extract runs
// non-tolerate), so extract_one_file already zero-filled it and
// counted it in bytes_unreadable — one 'lost' bucket covers both
// media damage and decrypt failure.
// A unit that fails to decrypt fails the read loud; extract_one_file
// zero-fills it into bytes_unreadable, the same bucket as media damage.
let (fr, halted) = extract_one_file(
&mut dec,
dest,
@@ -299,8 +277,8 @@ impl Disc {
Ok(result)
}
/// Resolve the CSS title key for a VTS group by cracking from its title
/// VOB extents (keyless Stevenson attack). The disc-wide key is reused when
/// Resolve the CSS title key for a VTS group by recovering it from the
/// group's scrambled title-VOB sectors. The disc-wide key is reused when
/// it already covers this VTS (single-VTS discs, or this VTS's span). The
/// reader is borrowed from the decrypting decorator (its inner source).
///
@@ -321,40 +299,24 @@ impl Disc {
dec: &mut DecryptingSectorSource<S>,
base_keys: &DecryptKeys,
) -> Result<DecryptKeys> {
// Gather the title VOB extents for this VTS (VTS_xx_1.VOB .. _9.VOB;
// VTS_xx_0.VOB is the menu and is clear, so excluded from the crack).
// Gather this VTS's title VOBs BY NAME, ascending.
//
// `planned` comes from walking the UDF directory, which yields File
// Identifier Descriptors in on-disc order — authoring order, with no
// guarantee it matches playback order. Relying on it would leave the
// crack starting wherever the disc's directory happened to list first.
//
// DVD-Video numbers a title set's VOBs in playback order by spec
// (VTS_xx_1.VOB .. VTS_xx_9.VOB, single digit), so ascending name IS
// playback order and it is deterministic regardless of directory
// layout. That matters because ORDER decides correctness here: see the
// note below on the shared crack budget.
// Gather this VTS's title VOBs (VTS_xx_1..9.VOB, _0.VOB menu excluded) BY
// NAME, ascending. Directory order is authoring order not playback order,
// but DVD-Video numbers VOBs in playback order, so name-sort is correct.
let mut files: Vec<&PlannedFile> = planned
.iter()
.filter(|pf| {
vts_group_of(&pf.disc_name).as_deref() == Some(vts) && is_title_vob(&pf.disc_name)
})
.collect();
// Case-INSENSITIVE, matching `vts_group_of`/`is_title_vob`, which
// selected these files case-insensitively. A byte-wise sort put
// 'V' (0x56) before 'v' (0x76), so a set holding `vts_01_1.vob`
// alongside `VTS_01_2.VOB` — reachable on a case-sensitive volume —
// started the crack at part 2 and could exhaust the budget in a clear
// run, which is the whole failure this ordering exists to prevent.
// Case-INSENSITIVE, matching the selection above. A byte-wise sort could
// start the crack at the wrong part on a case-sensitive volume, exhausting
// the budget in a clear run — the exact failure this ordering prevents.
files.sort_by_key(|f| f.disc_name.to_ascii_uppercase());
let mut extents: Vec<crate::disc::Extent> = Vec::new();
for pf in files {
// Unrecorded extents hold no bytes the VOB ever wrote, so they can
// carry no scrambled sector for the crack to work from — feeding
// them in spends the shared sector budget on media that is not part
// of the title.
// Unrecorded extents hold no bytes the VOB ever wrote and carry no
// scrambled sector, so feeding them in wastes the shared crack budget.
for ext in pf.extents.iter().filter(|e| e.recorded) {
extents.push(crate::disc::Extent {
start_lba: ext.lba,
@@ -365,26 +327,9 @@ impl Disc {
if extents.is_empty() {
return Ok(base_keys.clone());
}
// PLAYBACK ORDER — do NOT sort. This is the 1.5.1 garbage bug, and it
// grew back here in a new code path: the comment this replaced said it
// "matched the scan heuristic", but that heuristic WAS the bug and was
// already fixed in `Disc::decrypt_keys_for_title`, which documents the
// rule ("PLAYBACK ORDER, never largest-cell-first") and pins it with
// `decrypt_keys_for_title_scans_playback_order_not_largest_first`.
//
// Why order decides correctness: a CSS DVD's biggest cell opens with a
// long CLEAR run, and `crack_key`'s sector budget is shared across the
// whole extent list. Starting at the largest cell can exhaust the budget
// without ever MEETING a scrambled sector — and CSS recovers the title
// key from scrambled data itself, so the scan has to reach some. The
// crack then returns None, this function falls back to `base_keys`, and
// every VOB in the VTS is descrambled with the wrong key: corrupt PES
// behind an intact header, written out as a complete extract at exit 0.
//
// The files above are ordered by name and each file's own extents stay
// in file order, which together is playback order for a DVD title set.
// Crack against the raw (still-scrambled) inner reader, NOT the
// decrypting view — `crack_key` runs the descrambler itself.
// PLAYBACK ORDER — do NOT sort (the 1.5.1 garbage bug: see
// `Disc::decrypt_keys_for_title`'s "never largest-cell-first" rule).
// Crack against the raw inner reader, NOT the decrypting view.
match crate::css::crack_key_outcome(dec.inner_mut(), &extents, 64, None) {
crate::css::CrackOutcome::Cracked(state) => Ok(DecryptKeys::Css {
title_key: state.title_key,
@@ -393,24 +338,9 @@ impl Disc {
// and any key descrambles it as a no-op. The disc-wide key is the
// right answer, and this is the ONLY case that ever was.
crate::css::CrackOutcome::Unencrypted => Ok(base_keys.clone()),
// Scrambled sectors WERE seen and no key came out. Reusing the
// disc-wide key here writes corrupt PES behind an intact header and
// reports a complete extract at exit 0.
//
// `CssKeyMissing` is the per-TITLE code rather than the disc-level
// `CssNoDiscKey` because the two are treated differently elsewhere:
// it is what `error::is_skippable_title_stub` matches, and the MUX
// path really does skip a title on it and carry on with the rest of
// the disc.
//
// EXTRACT does not skip. This error is `?`-propagated by the only
// caller, out of the loop over every planned file, so one
// uncrackable VTS aborts the whole extract — including files from
// sibling VTS groups that would have cracked. That is deliberate:
// aborting loudly is the safe failure here, and pinned by
// `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`.
// Do not read the code choice as a promise that extraction
// continues past this point.
// Scrambled sectors WERE seen and no key came out. `CssKeyMissing`
// (unlike MUX, which skips on it) is `?`-propagated here, aborting
// the whole extract rather than reusing the wrong key silently.
crate::css::CrackOutcome::ScrambledUncracked => Err(Error::CssKeyMissing),
}
}
@@ -476,34 +406,9 @@ fn plan_tree(
let safe = sanitize_component(&entry.name)?;
let child_rel = host_rel.join(&safe);
let child_disc = format!("{disc_path}/{}", entry.name);
// Collision: two distinct disc paths → same host FILE. The key must
// model the HOST's namespace, not the disc's, and the two differ twice
// over:
//
// * CASE. macOS APFS and Windows NTFS are case-insensitive by default,
// so `Movie` and `movie` are one file there. Keyed by the
// case-preserving path they were two entries, nothing collided, and
// the second extraction overwrote the first with both files still
// reported `complete: true`.
// * The `.partial` SUFFIX. `extract_one_file` streams through
// `<final>.partial` before renaming, so a file's temp path lives in
// the same namespace as every other file's final path: a disc
// holding both `X` and `X.partial` planned two distinct final names,
// and extracting `X` then truncated the real `X.partial`.
//
// Only files get the `.partial` alias — directories are created
// directly and never stream through a temp name — but it is checked
// against every entry's primary key, so a directory `X.partial` beside
// a file `X` is caught too. The alias stores the OWNING file's disc
// path, so the `prev != child_disc` test keeps its meaning: a repeated
// identical disc path is not a collision, two different ones are.
//
// The fold is `to_lowercase`, which closes the ASCII and simple-Unicode
// case classes. It is NOT full case folding, and it does NOT normalize:
// APFS also unifies NFC/NFD, so `é` recorded as U+00E9 and as
// `e`+U+0301 remain two keys here and one file there. That residual is
// a false NEGATIVE (a missed collision, never a spurious one), so this
// is a strict improvement rather than a complete model of the host.
// Collision: two distinct disc paths → same host FILE (case-insensitive
// fold of `Movie`/`movie`, or `.partial` sharing the final-name namespace
// so `X`/`X.partial` collide too). Fold is ASCII/simple-Unicode only.
let mut register = |key: PathBuf| -> Result<()> {
let folded = key.to_string_lossy().to_lowercase();
if let Some(prev) = seen_hosts.insert(folded, child_disc.clone())
@@ -608,14 +513,9 @@ fn extract_one_file<S: SectorSource>(
if written >= pf.size {
break;
}
// ECMA-167 4/14.14.1.1 type 1: allocated but NOT recorded. The extent
// is part of the file's byte space and its contents are defined to be
// zeros, so write the zeros WITHOUT reading the media — those sectors
// hold nothing this file ever wrote (on an AACS disc, ciphertext that
// decrypts to noise). This mirrors `UdfFs::read_file_limited`, which
// takes the same decision from the same flag. Skipping the extent
// entirely instead would slide every later extent's bytes down by the
// hole's length.
// ECMA-167 4/14.14.1.1 type 1: allocated but NOT recorded. Write zeros
// WITHOUT reading the media (mirrors `UdfFs::read_file_limited`) — skipping
// the extent entirely would slide every later extent's bytes down.
if !recorded {
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64);
let hole_bytes = (sectors * SECTOR_BYTES_U64).min(pf.size.saturating_sub(written));
@@ -640,26 +540,16 @@ fn extract_one_file<S: SectorSource>(
}
continue;
}
// Anchor AACS unit alignment at THIS extent's start (clip-anchored
// gate, not absolute LBA 0 and NOT the file's first extent). A file
// may span multiple extents (fragmented / Long-AD / continuation ICB
// allocation); each extent's start LBA is arbitrary and generally not
// a multiple of 3 sectors from the previous extent. The decrypt-on-
// read gate (`is_unit_aligned(lba, unit_base)`) measures every read's
// LBA relative to `unit_base`, so the base MUST be re-anchored per
// extent — otherwise the first read of every later extent fails the
// gate and the whole extent is zero-filled as a (false) hole. Matches
// the per-extent re-anchoring in the mux read paths
// (`mux/disc.rs`, `sector/prefetched.rs`). No-op for CSS / None.
// Anchor AACS unit alignment at THIS extent's start, not LBA 0 or the
// file's first extent: a fragmented extent's start LBA is arbitrary, so
// the gate must re-anchor per extent or a later read false-holes. No-op for CSS/None.
dec.set_unit_base(abs_lba);
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32;
let mut sector_off: u32 = 0;
while sector_off < sectors {
// AACS: read whole units (see `whole_unit_batch`). The tail batch may
// be a 1–2 sector partial unit, which `decrypt_sectors` handles via
// its trailing-partial contract: a clear partial stays clear (AACS
// leaves the final short unit unencrypted on disc), a scrambled
// partial fails loud as DecryptFailed.
// AACS: read whole units (see `whole_unit_batch`). A 1-2 sector tail
// partial is handled by `decrypt_sectors`'s trailing-partial contract:
// clear stays clear, scrambled fails loud as DecryptFailed.
let batch = whole_unit_batch(sectors - sector_off);
let lba = abs_lba + sector_off;
let want = batch as usize * SECTOR_BYTES;
@@ -686,8 +576,7 @@ fn extract_one_file<S: SectorSource>(
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
sector_off += batch;
if opts.cancelled(cont) {
// Leave the `.partial`; do NOT rename. The aggregate run
// records the halt and this file stays incomplete.
// Leave the `.partial`; do NOT rename — this file stays incomplete.
return Ok((fr, true));
}
if written >= pf.size {
@@ -709,9 +598,8 @@ fn extract_one_file<S: SectorSource>(
/// [`READ_BATCH_SECTORS`] and rounded DOWN to a whole number of 3-sector units
/// UNLESS it is the extent's final (possibly short) tail — the tail always
/// begins on a unit boundary, so a 1–2 sector partial there is handled by
/// `decrypt_sectors`' trailing-partial contract. Shared by `extract_one_file`
/// (write) and `verify_one_clip` (dead-range) so this rounding rule lives in
/// exactly one place.
/// `decrypt_sectors`' trailing-partial contract. Used by `extract_one_file`
/// so this rounding rule lives in exactly one place.
fn whole_unit_batch(remaining: u32) -> u32 {
let mut batch = remaining.min(READ_BATCH_SECTORS);
if batch >= AACS_UNIT_SECTORS && batch < remaining {
@@ -773,11 +661,9 @@ fn finalize_file(
f.set_len(size).map_err(|e| Error::DirWriteFailed {
errno: e.raw_os_error(),
})?;
// `set_len` is a separate kernel op on this second handle (opened solely to
// truncate); the content fsync above was on the now-dropped writer handle.
// Without an fsync here, a crash between `set_len` and the rename can leave
// the file at its pre-truncation length — e.g. a sparse/over-read tail keeps
// its oversized form. Sync the new metadata (length) before publishing.
// `set_len` is a separate kernel op on this second handle; without an fsync
// here, a crash between `set_len` and the rename can leave the file at its
// pre-truncation length. Sync the new metadata (length) before publishing.
f.sync_all().map_err(|e| Error::DirWriteFailed {
errno: e.raw_os_error(),
})?;
@@ -803,11 +689,9 @@ fn report(opts: &ExtractOptions, done: u64, unreadable: u64, total: u64) -> bool
kind: crate::progress::PassKind::Mux,
work_done: done,
work_total: total,
// `done` counts good AND zero-filled-unreadable bytes together;
// split them so a consumer driven only by the live progress
// channel sees a holed extraction as holed rather than as a clean
// climb to 100%. The final ExtractResult already carries the true
// split — this used to pin unreadable at 0 and call every byte good.
// `done` counts good AND unreadable bytes together; split them so a
// live-progress-only consumer sees a holed extraction as holed,
// not a clean climb to 100% (this used to pin unreadable at 0).
bytes_good_total: done.saturating_sub(unreadable),
bytes_unreadable_total: unreadable,
bytes_pending_total: 0,
@@ -843,10 +727,8 @@ fn available_space(dir: &Path) -> Option<u64> {
if rc != 0 {
return None;
}
// `statvfs` field integer widths differ by platform (e.g. macOS `u64`,
// Linux glibc `u64`/`c_ulong`); cast both to `u64` for the product. The
// cast is a no-op where the field is already `u64` — allow the lint so the
// same source stays portable.
// `statvfs` field integer widths differ by platform; cast both to `u64`
// for the product (no-op where already `u64` — allow lint for portability).
#[allow(clippy::unnecessary_cast, clippy::useless_conversion)]
let avail = (st.f_bavail as u64).saturating_mul(st.f_frsize as u64);
Some(avail)
@@ -946,13 +828,9 @@ fn sanitize_component(name: &str) -> Result<String> {
host: name.to_string(),
});
}
// Windows reserved device names (case-insensitive, base name before any
// extension). `NUL` silently discards all writes on Windows, so a disc file
// literally named `NUL.cfg` (legal on UDF/Linux) must NOT pass through
// verbatim — but it also must not abort the whole extraction. Substitute by
// prefixing `_`, which can never itself collide with another reserved name
// and is host-legal everywhere. The substituted name still flows through the
// caller's host-path collision check.
// Windows reserved device names (case-insensitive, base before extension).
// `NUL` silently discards writes on Windows, so `NUL.cfg` (legal on
// UDF/Linux) must not pass through verbatim; prefix `_` instead of aborting.
let base = trimmed.split('.').next().unwrap_or(trimmed);
if is_windows_reserved(base) {
return Ok(format!("_{trimmed}"));
@@ -1016,14 +894,9 @@ mod tests {
use crate::css::lfsr;
use std::collections::HashMap;
// ── Self-contained in-memory UDF fixture toolkit ──────────────────────
// Modeled on the bluray.rs / dvd.rs test fixtures: a `MemDisc` SectorSource
// backed by an explicit absolute-LBA map, plus recursive UDF builders.
// PART_START == metadata_start so file/ICB LBAs share one address space
// (single partition map). Encrypted content is faked at the buffer level
// (CSS via `lfsr::scramble_sector`); AACS-content tests use clear bytes
// through `DecryptKeys::None`, since per-unit AACS decrypt is already
// exhaustively tested in `decrypt.rs` / `decrypting.rs`.
// ── Self-contained in-memory UDF fixture toolkit ──────────────────
// Modeled on the bluray.rs / dvd.rs fixtures: a `MemDisc` SectorSource
// backed by an absolute-LBA map; AACS tests use clear bytes.
const PART_START: u32 = 2000;
@@ -1505,7 +1378,8 @@ mod tests {
}
/// A CSS-scrambled title VOB is descrambled on extraction: the producer
/// cracks the per-VTS key (keyless Stevenson) and the output VOB is plain.
/// recovers the per-VTS key from the scrambled sectors and the output VOB
/// is plain.
#[test]
fn css_title_vob_is_descrambled() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
@@ -2093,12 +1967,9 @@ mod tests {
fn scrambled_vob(title_key: [u8; 5], marker: u8) -> (Vec<u8>, Vec<u8>) {
let seed = [0x11u8, 0x22, 0x33, 0x44, marker];
let mut plain = vec![0u8; 2048];
// The crack scan's hardened `is_scrambled_pack` gate requires the
// MPEG-PS pack-start signature before it will even ATTEMPT a
// Stevenson crack (see `css::is_scrambled_pack`) -- without it,
// `resolve_vts_key` silently falls back to `base_keys` for BOTH
// VTS groups regardless of which extents were gathered, masking
// this exact regression.
// The crack scan's `is_scrambled_pack` gate requires the MPEG-PS pack-
// start signature before attempting a crack; without it, `resolve_vts_key`
// falls back to `base_keys` for both groups, masking this regression.
plain[0x00..0x04].copy_from_slice(&crate::css::PACK_START);
// Pack start code: a real scrambled sector is an MPEG-2 PS pack, and
// the descrambler requires it before trusting byte 0x14.
@@ -2141,10 +2012,8 @@ mod tests {
let out = TmpDir::new("css_two_vts");
let mut d = clear_disc();
d.content_format = crate::disc::ContentFormat::MpegPs;
// Disc-wide key deliberately matches NEITHER VTS's real key: if the
// per-VTS group filter is broken and a crack attempt fails (or is
// skipped), falling back to this key must still mismatch, so the
// fallback path can never accidentally mask a broken filter.
// Disc-wide key deliberately matches NEITHER VTS's real key, so a
// broken filter falling back to it can never accidentally mask itself.
d.css = Some(crate::css::CssState {
title_key: [0xFFu8; 5],
crack_span: None,
@@ -2201,9 +2070,8 @@ mod tests {
lfsr::scramble_sector(&key_1, &mut scrambled);
(plain, scrambled)
};
// Scrambled — the pack header and the 0x14 flag are set, so the scan
// SEES ciphertext — but the cleartext region carries no periodic run,
// so the Stevenson crib never forms and no key is recoverable.
// Scrambled — pack header and 0x14 flag set, so the scan SEES ciphertext —
// but the cleartext region has no periodic run, so no key is recoverable.
let uncrackable = {
let mut sect = vec![0u8; 2048];
sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
@@ -2320,10 +2188,8 @@ mod tests {
// with sector_off > 0.
const SECTORS_A: u32 = 1600;
const LBA_A: u32 = 5000;
// The disc region immediately following extent A's true end. Must
// NEVER be read as part of extent A: sized to cover a full erroneous
// READ_BATCH_SECTORS second batch starting right after extent A's
// real tail.
// The disc region immediately following extent A's true end. Must NEVER
// be read as part of extent A: sized to cover a full erroneous second batch.
const LBA_FILLER: u32 = LBA_A + SECTORS_A;
const SECTORS_FILLER: u32 = 1472;
// Extent B: the file's real second extent, at a completely different
+78 -193
View File
@@ -107,10 +107,9 @@ fn parse_vti_clip_order(vti: &[u8]) -> Vec<String> {
count += 1;
}
}
// Pick the largest residue bucket (the clip table). On a size tie, break
// deterministically by the bucket's smallest offset — `HashMap` iteration
// order is randomized, so `max_by_key` alone could pick a different bucket
// run-to-run on identical bytes.
// Pick the largest residue bucket (the clip table). On a size tie, break by the
// bucket's smallest offset, since `HashMap` iteration order is randomized and
// `max_by_key` alone could pick a different bucket run-to-run on identical bytes.
let Some(mut best) = buckets
.into_values()
.max_by_key(|g| (g.len(), std::cmp::Reverse(g.iter().map(|(o, _)| *o).min())))
@@ -166,7 +165,7 @@ fn sniff_video_codec(es: &[u8]) -> Option<Codec> {
/// Sniff an audio codec from a `private_stream_1` sub-stream sample. Today only
/// Dolby Digital Plus (E-AC-3) is recognized — its `0x0B77` syncword — which is
/// what ANCHORMAN / SHAUN carry on sub-ids `0xC0..=0xC7`. Returns `None` for an
/// what real retail HD-DVD titles carry on sub-ids `0xC0..=0xC7`. Returns `None` for an
/// unrecognized sample so the caller drops the stream rather than mislabeling it.
fn sniff_audio_codec(es: &[u8]) -> Option<Codec> {
let has_sync = es.windows(2).any(|w| w[0] == 0x0B && w[1] == 0x77);
@@ -175,8 +174,10 @@ fn sniff_audio_codec(es: &[u8]) -> Option<Codec> {
/// Demux the head of an `.evo` clip (through the disc's [`SectorSource`]) and
/// build one [`Stream`] per distinct elementary stream found: the video track
/// (mapped to the canonical [`DVD_VIDEO_PID`]) and every DD+ audio sub-stream
/// (mapped via [`dvd_audio_pid`]). Codec is sniffed from the demuxed ES bytes.
/// (mapped to the canonical [`DVD_VIDEO_PID`](crate::mux::ps::DVD_VIDEO_PID) for
/// plain 0xE0-0xEF video, or the extended-stream-id PID for HD-DVD VC-1) and
/// every DD+ audio sub-stream (mapped via [`dvd_audio_pid`]). Codec is sniffed
/// from the demuxed ES bytes.
///
/// Mirrors the stream construction in `Disc::scan_dvd_titles`; resolution /
/// language / channels use sane HD-DVD defaults (the muxer reads the true pixel
@@ -196,11 +197,9 @@ fn probe_evo_streams(
) -> Result<Vec<Stream>> {
let mut demux = PsDemuxer::new();
let mut video: Vec<u8> = Vec::new();
// Routing PID of the video track, captured from the first video PES seen:
// `DVD_VIDEO_PID` for a plain 0xE0-0xEF stream (Anchorman's H.264 on 0xE2),
// or `0xFD00 | stream_id_extension` for an HD-DVD extended-stream-id video
// (Shaun's VC-1 on 0xFD ext 0x55). Kept in lockstep with `PsPacket::dvd_pid`
// so the emitted `Stream` PID matches what the demuxer routes at mux time.
// Routing PID of the video track, from the first video PES seen: `DVD_VIDEO_PID`
// for a plain 0xE0-0xEF stream, or `0xFD00 | stream_id_extension` for HD-DVD
// extended-stream-id video. Kept in lockstep with `PsPacket::dvd_pid`.
let mut video_pid: Option<u16> = None;
// sub_id -> ES sample, ordered so audio tracks surface in sub-id order.
let mut audio: BTreeMap<u8, Vec<u8>> = BTreeMap::new();
@@ -210,11 +209,9 @@ fn probe_evo_streams(
let mut lba = ext.start_lba;
let mut left = ext.sector_count;
while left > 0 && remaining > 0 {
// 1 MiB read chunks (512 sectors) keep buffers small.
// Poll before every chunk, as the PGS probe does: a clip's probe
// budget is 16 MiB of blocking reads, and on marginal media each
// one can sit in the drive's retry path, so per-clip granularity
// alone would leave a Stop waiting on a dead disc.
// 1 MiB chunks (512 sectors). Poll every chunk, not just per-clip:
// a 16 MiB probe can sit in the drive's retry path on marginal
// media, and per-clip granularity alone would leave a Stop waiting.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(crate::error::Error::Halted);
}
@@ -222,10 +219,9 @@ fn probe_evo_streams(
let mut buf = vec![0u8; n as usize * crate::consts::SECTOR_BYTES];
match reader.read_sectors(lba, n, &mut buf, false) {
Ok(_) => {}
// A live-drive Stop surfaces HERE, not through the token:
// `Drive::checked_exec` fails every command with `Halted` once
// the flag is set. Swallowing it (as any other read error is
// swallowed) reports an un-probed clip as a probed one.
// A live-drive Stop surfaces HERE, via `Drive::checked_exec`
// failing with `Halted`. Swallowing it like other read errors
// would report an un-probed clip as probed.
Err(crate::error::Error::Halted) => {
return Err(crate::error::Error::Halted);
}
@@ -244,12 +240,9 @@ fn probe_evo_streams(
}
let mut streams = Vec::new();
// Emit the video stream only when the codec was actually identified from the
// sampled head. Guessing (e.g. defaulting to H.264) would tag a VC-1 — or a
// still-encrypted — clip with the wrong codec, so the mux applies the wrong
// parser and produces a corrupt track; dropping it is the honest outcome
// (matches the audio path below), and a real clear clip always carries its
// sequence header / SPS at the head, so this never fires on a normal disc.
// Only emit video once the codec is actually identified from the sampled
// head; guessing would tag a VC-1 (or still-encrypted) clip wrong and feed
// it the wrong parser. A real clear clip always carries its header early.
if let (Some(pid), Some(codec)) = (video_pid, sniff_video_codec(&video)) {
streams.push(Stream::Video(VideoStream {
pid,
@@ -343,18 +336,9 @@ fn collect_es(
}
}
// ─────────────────────── Advanced-Content playlist (XPL) ──────────────────
//
// HD-DVD Advanced Content ships an authoritative playlist at
// `ADV_OBJ/VPLST000.XPL` (DVD-Forum `HDDVDVideo/Playlist` XML). It is the real
// player playlist: a `<TitleSet>` of `<Title>`s, each naming its
// `<PrimaryAudioVideoClip>` clips (an EVO, via its `.MAP` sidecar) in playback
// order with title-time in/out points, a `titleDuration`, a `displayName`, and a
// `<ChapterList>`. Parsing it gives authoritative title composition — clips,
// duration, name, chapters — instead of the `feature*` clip-name heuristic, plus
// the per-clip title-time offsets needed to splice a layer-break split
// (`FEATURE_1` + `FEATURE_2`) onto one continuous timeline. Parsed with a real
// XML parser (`roxmltree`), not a hand-rolled scanner — the XPL is genuine XML.
// ─── Advanced-Content playlist (XPL) ───
// `ADV_OBJ/VPLST000.XPL` is the real player playlist: `<Title>`s naming clips
// in order with in/out points, duration, name, chapters — parsed as real XML.
/// One clip reference inside an XPL `<Title>`: the resolved `.evo` name (lower
/// case) and the clip's placement on the title timeline, in seconds.
@@ -694,16 +678,9 @@ fn parse_xpl_titles(xpl: &[u8]) -> Vec<XplTitle> {
.descendants()
.filter(|n| local(n, "PrimaryAudioVideoClip"))
{
// Bounded by MAX_XPL_CLIPS_PER_TITLE. `descendants()` is deliberate
// — a real playlist wraps its clips in `<PrimaryAudioVideoClipList>`
// or an `ApplicationSegment`, so `children()` would miss them — but
// it also means a clip nested inside N ancestor `<Title>`s is
// collected N times over, and the depth guard still allows 32 such
// ancestors. `break` rather than dropping surplus: unlike the
// directory scan (which must keep reading to find the `.vti`),
// nothing later in THIS loop is needed, and stopping bounds the walk
// as well as the allocation. Chapters are collected by a separate
// iterator below, so they are unaffected.
// Bounded by MAX_XPL_CLIPS_PER_TITLE. `descendants()`, not
// `children()`, is needed to reach wrapped clips, but that also
// means nested `<Title>`s multiply hits, so `break` bounds the walk.
if clips.len() >= MAX_XPL_CLIPS_PER_TITLE {
break;
}
@@ -779,25 +756,18 @@ fn compose_xpl_titles(
// a crafted one can name a single `.evo` from all MAX_XPL_TITLES titles.
let mut probes = EvoProbeCache::default();
for t in xpl_titles {
// A clip that exists but has no truthful read plan poisons every
// title that names it: composing around it would emit a title short
// by that clip's bytes while its durations and chapter offsets still
// assume them.
// A clip with no truthful read plan poisons every title naming it:
// composing around it emits a title short by that clip's bytes while
// its durations and chapter offsets still assume them.
if t.clips.iter().any(|c| unusable.contains(&c.evo)) {
continue;
}
let mut extents = Vec::new();
let mut size_bytes = 0u64;
let mut parts = Vec::new();
// A crafted `<PrimaryAudioVideoClip>` list can name the same `.evo`
// any number of times (XML has no fixed element count, unlike MPLS's
// binary PlayItem count). Mirrors bluray.rs's
// `first_ref = seen_clips.insert(...)` gate (`:113`/`:117`): push a
// clip's physical extents and size only the first time its key is
// seen, so a clip named N times contributes its extent list ONCE
// instead of N times over. The analogous key here is the `.evo`
// filename — this format has no clip_id, and `.evo` is what
// `clip_extents` is already keyed by.
// A crafted clip list can name the same `.evo` any number of times.
// Mirrors bluray.rs's `seen_clips.insert(...)` gate: push a clip's
// extents/size only the first time its `.evo` key is seen.
let mut seen_evos: std::collections::HashSet<&str> = std::collections::HashSet::new();
for c in &t.clips {
let Some((orig, size, exts)) = clip_extents.get(&c.evo) else {
@@ -893,11 +863,9 @@ impl Disc {
}
let lower = e.name.to_ascii_lowercase();
if lower.ends_with(HDDVD_CLIP_EXT) {
// Bounded by MAX_HDDVD_CLIPS: each clip taken here costs an ICB
// read to resolve its extents and (in the fallback path below) a
// full probe_evo_streams pass. Surplus entries are DROPPED rather
// than `break`ing the loop, so a crafted disc cannot hide the
// `.vti` navigation file behind a wall of `.evo` names.
// Bounded by MAX_HDDVD_CLIPS. Surplus entries are DROPPED, not
// `break`ed on, so a crafted disc can't hide the `.vti` behind
// a wall of `.evo` names (each clip costs an ICB read + probe).
if clips.len() < MAX_HDDVD_CLIPS {
clips.push((e.name.clone(), e.size));
}
@@ -906,18 +874,9 @@ impl Disc {
}
}
// Authored clip order from the VTI clip table (empty if no VTI).
//
// The VTI name came from `ts_dir.entries`, so a failed read here is a
// real I/O error (a scratched sector under the `.vti`), never an absent
// file. `.ok()` used to flatten the two — dropping the authored order
// with no diagnostic, so a split feature then composed from the per-clip
// heuristic and the operator was never told the authored order existed
// but could not be read. The clip-extent arms below log every read
// failure with its own code; this one now does too. The fallback itself
// is unchanged (no order => per-clip, exactly as an unauthored disc),
// because there is nothing to compose from without the table. Logging is
// exempt from the no-English rule (errors stay numeric).
// Authored clip order from the VTI clip table (empty if no VTI). The
// name came from `ts_dir.entries`, so a failed read is a real I/O
// error, not an absent file — log it rather than silently falling back.
let order: Vec<String> = match vti_name {
None => Vec::new(),
Some(n) => match udf_fs.read_file(reader, &format!("/HVDVD_TS/{n}")) {
@@ -936,12 +895,9 @@ impl Disc {
// Resolve each clip's physical extents once, keyed by lower-case name.
let mut clip_extents: BTreeMap<String, (String, u64, Vec<Extent>)> = BTreeMap::new();
// Clips that EXIST but cannot be given a truthful read plan, because
// the file carries an unrecorded (never-written) extent — see
// `UdfFs::file_extents`. Kept apart from "did not resolve": a title
// built without one of these is short by that clip's runtime while
// still claiming its duration and size, which is data loss shaped
// like an ordinary rip. Any title naming one is dropped instead.
// Clips that EXIST but have no truthful read plan (unrecorded extent,
// see `UdfFs::file_extents`). Composing around one silently shorts a
// title's runtime, so any title naming one is dropped instead.
let mut unusable: std::collections::HashSet<String> = std::collections::HashSet::new();
for (name, size) in &clips {
if halt.is_some_and(|h| h.is_cancelled()) {
@@ -960,21 +916,9 @@ impl Disc {
}
}
Err(crate::error::Error::UdfUnrecordedExtent { .. }) => {
// Say so. Marking the clip unusable drops every title that
// names it, and in the fallback case the OTHER part of a
// split feature is still offered standalone — so half a
// movie is presented as a whole one. Silently returning Ok
// with the feature missing is the shape of data loss this
// refusal exists to prevent, pointed the other way.
//
// The Blu-ray half of this same fix already warns; this
// file had no diagnostics at all. Logging is exempt from
// the crate's no-English rule (errors stay numeric).
//
// Named constant, not the literal 6017 this used to carry:
// a hardcoded code in the very file whose sibling arm was
// changed to stop doing that is the next drift waiting to
// happen, and the literal cannot follow a renumbering.
// Marking unusable drops every title naming this clip;
// silently composing without it would present half a movie
// as a whole one. Named constant, not the old literal 6017.
tracing::warn!(
target: "freemkv::disc",
clip = ?name,
@@ -984,22 +928,9 @@ impl Disc {
unusable.insert(name.to_ascii_lowercase());
}
Err(crate::error::Error::Halted) => return Err(crate::error::Error::Halted),
// EVERY other failure means the same thing: no truthful read
// plan for this clip. A scratched sector under its ICB
// (DiscRead), an allocation-descriptor chain that never
// terminated (UdfAdChainTooLong), a file whose data is
// embedded rather than extent-mapped (UdfEmbeddedData) — all
// of them used to land in a bare `Err(_) => {}`: no log, and
// the clip NOT marked unusable, so a split feature still
// composed from FEATURE_1 alone and presented half a movie as
// a whole one. That is the very outcome the arm above was
// written to prevent, reachable through every error but one.
//
// The code logged is the error's own. Reusing 6017 here would
// account a scratched disc as an authoring hole.
//
// (`UdfNotFound` needs no special case: these names came from
// `ts_dir.entries`, so the lookup cannot miss.)
// EVERY other failure (scratched ICB, dead AD chain, embedded
// data, ...) means no truthful read plan; used to fall through
// a bare `Err(_) => {}`, unmarked and unlogged.
Err(e) => {
tracing::warn!(
target: "freemkv::disc",
@@ -1010,42 +941,9 @@ impl Disc {
unusable.insert(name.to_ascii_lowercase());
}
}
// A clip that resolved with NO usable extent is unusable too, and
// for exactly the same reason as the `Err` arms above.
//
// Round 1 accounted for every `Err` from `file_extents` and left
// this route open: `Ok` can still yield an empty AD list, or a list
// every entry of which the `sectors > 0 && lba > 0` filter above
// discards (a zero-length placeholder AD, or one pointing at LBA
// 0 — see `UdfFs::file_extents`). The clip then entered NEITHER
// `clip_extents` NOR `unusable`, and nothing was logged. The
// composer's `any(|n| unusable.contains(..))` guard below therefore
// missed it and the `filter(|n| clip_extents.contains_key(..))`
// beside it quietly deleted the part: a `FEATURE_2.EVO` of size 0
// next to a healthy `FEATURE_1.EVO` composed a FEATURE title out of
// part one alone, still advertising the whole runtime, at rc=0,
// silently. That is strictly WORSE than the `Err` case this guard
// was built for — half a movie presented as a whole one — and it is
// reachable from an ordinary zero-byte file, no crafting needed.
//
// WHY THE BLU-RAY HALF DELIBERATELY DIFFERS. `bluray.rs` documents
// this same `Ok`-but-empty hole as a KNOWN GAP and leaves it open,
// and that decision still stands there — not here. Two things are
// different. (1) Cost: BD has no `unusable` set, so closing it
// there means a post-loop "every clip_id must appear in `spans`"
// invariant that DROPS the title, and it is not settled that an
// empty-but-Ok resolve is always a defect rather than a legitimate
// healthy-disc state; dropping healthy titles is worse than the
// gap. Here the set already exists, so refusing costs nothing new.
// (2) Consequence: on BD the clip is one PlayItem of a title that
// is otherwise whole; on HD-DVD the authored feature is COMPOSED
// from parts, so the missing one is silently spliced out of a title
// that keeps claiming the full runtime. Same hole, different price.
// The asymmetry is deliberate; it is written down here so the next
// audit reads a decision instead of finding an oversight.
//
// `insert` returning false means an `Err` arm above already logged
// this clip — no second line for one clip.
// `Ok` with NO usable extent (empty/filtered AD list) is unusable
// too: else a zero-byte `FEATURE_2.EVO` beside a healthy part one
// silently composed a whole-runtime title missing half the movie.
if extents.is_empty() {
if unusable.insert(name.to_ascii_lowercase()) {
// Not the neighbouring 6017: an empty AD list is not an
@@ -1063,11 +961,9 @@ impl Disc {
}
}
// Authoritative composition from the Advanced-Content playlist
// (`ADV_OBJ/VPLST*.XPL`): real per-title clip lists, durations, names,
// chapters, and title-time offsets. This is the primary path; the
// clip-name heuristic below is the fallback when the playlist is absent
// or unparseable (or resolves to no on-disc clips).
// Authoritative composition from `ADV_OBJ/VPLST*.XPL`, if present and
// parseable. The clip-name heuristic below is the fallback when it's
// absent, unparseable, or resolves to no on-disc clips.
if let Some(xpl) = read_adv_obj_xpl(reader, udf_fs) {
let composed = compose_xpl_titles(
reader,
@@ -1081,12 +977,9 @@ impl Disc {
}
}
// Feature clips, in authored order, that actually resolved to extents.
// If ANY authored feature part has no truthful read plan, the composed
// feature title would silently omit that part's bytes — emit no
// composed feature at all rather than a short one. (The per-clip
// titles below are unaffected: each stands alone and an unusable clip
// simply yields no title.)
// Feature clips, in authored order, that resolved to extents. If ANY
// authored feature part has no truthful read plan, emit no composed
// feature at all rather than one silently short that part's bytes.
let feature: Vec<String> = if order
.iter()
.filter(|n| is_feature_clip(n))
@@ -1325,11 +1218,9 @@ mod tests {
#[test]
fn parse_vti_clip_order_is_deterministic_on_a_bucket_size_tie() {
// Two residue buckets of EQUAL size must resolve to the SAME winner every
// call — `HashMap` iteration is randomized, so a `max_by_key` without a
// deterministic tie-break could pick a different bucket run-to-run on
// identical bytes. Build a VTI whose stray `.EVO` names tie the real
// table's bucket count, then assert the result is stable across calls.
// Two equal-size buckets must resolve to the SAME winner every call:
// `HashMap` iteration is randomized, so `max_by_key` without a
// deterministic tie-break could pick a different bucket run-to-run.
let mut vti = vec![0u8; 0x600];
vti[..HDDVD_VTI_MAGIC.len()].copy_from_slice(HDDVD_VTI_MAGIC);
let put = |v: &mut Vec<u8>, off: usize, name: &str| {
@@ -1355,7 +1246,7 @@ mod tests {
#[test]
fn is_feature_clip_matches_the_feature_naming_variants() {
// Layer-break split (Shaun / Anchorman) and the divide form (Harry Potter).
// Layer-break split (seen on real discs) and the divide form (also seen on real discs).
assert!(is_feature_clip("FEATURE_1.EVO"));
assert!(is_feature_clip("FEATURE_2.EVO"));
assert!(is_feature_clip("feature.EVO"));
@@ -1420,10 +1311,9 @@ mod tests {
#[test]
fn scan_hddvd_composes_split_feature_into_one_title() {
// A disc whose feature is FEATURE_1 + FEATURE_2 (a layer-break split), plus
// a TRAILER extra. The VTI names them in authored order; the scan must
// JOIN the two feature parts into one title (so the largest-title pick is
// the whole movie) and keep the trailer as its own title.
// FEATURE_1 + FEATURE_2 (a layer-break split) plus a TRAILER extra;
// the scan must JOIN the feature parts into one title and keep the
// trailer separate.
let mut disc = MemDisc::new();
let vti = synthetic_vti(&["FEATURE_1.EVO", "FEATURE_2.EVO", "TRAILER.EVO"]);
let files = vec![
@@ -1720,11 +1610,9 @@ mod tests {
assert_eq!(sniff_video_codec(&[0x00, 0x00, 0x01, 0x61, 0x9A]), None);
assert_eq!(sniff_video_codec(&[0xDE, 0xAD, 0xBE, 0xEF]), None);
// Overlap regression: a picture_start_code (0x00) whose payload begins
// with 00 00 must advance a full 4 bytes so the code byte isn't re-read
// as the start of a new marker. Here the picture is followed by a real
// MPEG-2 sequence header — the scan must reach it cleanly and return
// Mpeg2 (and, critically, not be confused by the 1-byte overlap).
// Overlap regression: a picture_start_code (0x00) with payload 00 00
// must advance a full 4 bytes so the code byte isn't re-read as a new
// marker; the real MPEG-2 sequence header after it must still be found.
assert_eq!(
sniff_video_codec(&[0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0xB3, 0x2D]),
Some(Codec::Mpeg2)
@@ -1784,7 +1672,7 @@ mod tests {
}
/// Synthetic EVO program-stream: pack header, a video PES (H.264 SPS+IDR on
/// stream_id 0xE2, exactly as ANCHORMAN carries it), two DD+ audio PES
/// stream_id 0xE2, exactly as a real retail disc carries it), two DD+ audio PES
/// (sub-ids 0xC0/0xC1, each with the 4-byte sub-header + E-AC-3 syncword),
/// then program-end.
fn synthetic_evo() -> Vec<u8> {
@@ -1793,7 +1681,7 @@ mod tests {
d.extend_from_slice(&[
0x00, 0x00, 0x01, 0xBA, 0x44, 0x00, 0x04, 0x00, 0x04, 0x01, 0x01, 0x89, 0xC3, 0xF8,
]);
// Video PES on stream_id 0xE2 (Anchorman's H.264 sub-id in the 0xE0-0xEF
// Video PES on stream_id 0xE2 (a real disc's H.264 sub-id in the 0xE0-0xEF
// range): SPS (type 7) + IDR (type 5) Annex-B.
let video_es = [
0x00, 0x00, 0x01, 0x67, 0x42, 0x00, 0x1E, 0xAB, 0xCD, // SPS
@@ -1954,7 +1842,7 @@ mod tests {
}
/// Synthetic EVO carrying VC-1 video on the extended-stream-id 0xFD (ext
/// 0x55), as SHAUN OF THE DEAD does, plus one DD+ audio PES.
/// 0x55), as a real retail HD-DVD title does, plus one DD+ audio PES.
fn synthetic_evo_vc1() -> Vec<u8> {
let mut d = Vec::new();
d.extend_from_slice(&[
@@ -1987,10 +1875,9 @@ mod tests {
#[test]
fn collect_es_routes_only_vc1_0xfd_to_video() {
use crate::mux::ps::hddvd_extended_pid;
// The 0xFD guard: only the VC-1 extension (0x55) is video. An HD-audio
// 0xFD sub-stream (e.g. 0x72) that arrives FIRST must NOT stamp video_pid
// with its PID or pollute the video sample — else the real video track is
// lost. (Routing 0xFD audio to its own track is deferred.)
// The 0xFD guard: only VC-1 (ext 0x55) is video. An HD-audio 0xFD
// sub-stream (e.g. 0x72) arriving FIRST must not stamp video_pid or
// pollute the video sample, else the real video track is lost.
let mut video = Vec::new();
let mut video_pid: Option<u16> = None;
let mut audio = BTreeMap::new();
@@ -3150,10 +3037,9 @@ mod tests {
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
// Blank FEATURE_2's ICB (laid at PART_START + 101). Its descriptor tag
// is then 0, neither 261 nor 266 — what the parser sees when the sector
// holding an ICB cannot be read back intact. Deliberately NOT an
// unrecorded extent: that class was already handled.
// Blank FEATURE_2's ICB: descriptor tag becomes 0 (neither 261 nor
// 266), what the parser sees when an ICB sector can't be read back
// intact. Deliberately not an unrecorded extent — that's covered elsewhere.
disc.put_bytes(PART_START + 101, &[0u8; 2048]);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
@@ -3490,10 +3376,9 @@ mod tests {
let budget = |cap: usize| {
cap as u64 * u64::from(EVO_PROBE_SECTORS) * crate::consts::SECTOR_BYTES as u64
};
// BOTH title-composition paths pay one probe per item, and a crafted
// disc reaches either one: the directory fallback when `/ADV_OBJ` is
// omitted, the playlist path when it is present. Capping only one moves
// the amplification next door.
// Both paths pay one probe per item, and a crafted disc reaches
// either: the directory fallback when `/ADV_OBJ` is omitted, the
// playlist path when present. Capping only one moves the amplification.
for (name, cap) in [
("MAX_HDDVD_CLIPS", MAX_HDDVD_CLIPS),
("MAX_XPL_TITLES", MAX_XPL_TITLES),
+845 -398
View File
File diff suppressed because it is too large Load Diff
+90 -208
View File
@@ -192,11 +192,9 @@ fn plan_windows(sector_count: u32, share: u32) -> Vec<SampleWindow> {
(0..windows)
.map(|i| SampleWindow {
offset: if windows == 1 {
// A title cut into many clips gives each extent a share worth one
// window. Putting that window at the extent's head samples every
// clip at the same relative position — and for the FIRST clip
// that position is the start of the feature, the one stretch a
// film reliably has no subtitles in. Take the middle instead.
// A single window per extent placed at the head would sample the
// first clip's start — the one span a film reliably lacks subtitles.
// Use the middle instead.
align_down(span / 2)
} else {
// u64: `span * i` overflows u32 for a large extent.
@@ -267,8 +265,8 @@ impl TrackEvidence {
/// The coverage is the whole point. Evidence from a sampled (or budget-cut) read
/// describes the sectors that were actually fed to the demuxer, not the extent —
/// memoising it under the extent's full key and replaying it to another playlist
/// asserts a completeness the read never had. Recording `covered` and `complete`
/// lets a later run decide whether the entry answers ITS question or whether the
/// asserts a completeness the read never had. Recording `covered` alongside the
/// evidence's own `sampled` flag lets a later run decide whether the entry answers ITS question or whether the
/// extent has to be read again.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub(crate) struct CachedEvidence {
@@ -296,7 +294,7 @@ impl CachedEvidence {
///
/// Many playlists on one disc reference the same clips (main feature, play-all,
/// seamless-branch variants) but rarely with byte-identical extent LISTS: 00800
/// = [A, B], 00801 = [A], 00802 = [B] are three different lists over two clips.
/// = `[A, B]`, 00801 = `[A]`, 00802 = `[B]` are three different lists over two clips.
/// Keying on the whole list de-duplicated only exactly-identical playlists and
/// re-read every shared clip once per list — up to `PROBE_BUDGET_SECTORS`
/// (256 MiB) of optical-drive time each. Per-extent keying reads each physical
@@ -398,11 +396,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
})
.collect();
// Budget allocation: each extent gets a share of the sector budget in
// proportion to its size, computed over the title's WHOLE extent list rather
// than over the extents that happen to be uncached. That keeps an extent's
// sampling plan — and therefore the coverage its cache entry claims — a
// function of the title alone, not of cache state or probe order.
// Each extent's share of the sector budget is proportional to its size,
// computed over the title's whole extent list (not just uncached ones), so the
// sampling plan is a function of the title alone, not cache state or order.
let total_sectors: u64 = title
.extents
.iter()
@@ -439,10 +435,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
slot.merge(hit.evidence);
}
}
// No entry, or one whose coverage does not support the claim this
// run needs from it. (A playlist that declares a PGS PID a
// previous playlist did not also lands here, so the extra track is
// genuinely probed.)
// No entry, or one whose coverage doesn't support this run's claim —
// includes a PGS PID a previous playlist didn't declare, so it's
// genuinely probed.
_ => fresh.push(pid),
}
}
@@ -471,10 +466,8 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
.collect()
}
// Whether ANY declared track could still have its outcome changed by reading
// more. This is the per-track early exit: the moment a track is disproven
// (and there is nothing left for its evidence to correct) it stops asking for
// budget, and when no track is asking, the run stops.
// Per-track early exit: once a track is disproven with nothing left to
// correct, it stops asking for budget; when no track is asking, the run stops.
let anything_left_to_learn = |live: &[(u16, TrackEvidence)]| -> bool {
let disc_uses_forced_flag = live.iter().any(|(_, e)| e.forced_seen);
let busiest = live.iter().map(|(_, e)| e.displays).max().unwrap_or(0);
@@ -488,19 +481,15 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
if !vendor_forced.get(pid).copied().unwrap_or(false) {
return false;
}
// Disproven but labelled forced: keep reading only while the evidence
// a demotion needs is still incomplete (see `demotable`) — otherwise
// the run would stop one window short of being ALLOWED to act on what
// it has already seen.
// Disproven but labelled forced: keep reading only while the evidence a
// demotion needs (see `demotable`) is still incomplete.
!crate::mux::codec::pgs::demotable(e.facts(), disc_uses_forced_flag, busiest)
})
};
if todo.is_empty() {
// Every extent's evidence came from a run that reached a designed stop and
// covered at least as much as this run planned to, so an absence claim
// over the composed evidence is as sound as the run that produced each
// part.
// Every extent's evidence reached a designed stop and covered at least as
// much as planned, so an absence claim over it is as sound as its source.
apply_verdicts(title, &verdicts(&evidence, true));
return;
}
@@ -512,10 +501,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
// asserted from what was observed.
let mut stop = StopReason::Exhausted;
'outer: for (ext, fresh_pids) in &todo {
// Trackers are PER EXTENT and only for the tracks this extent still owes
// evidence for — a track the cache already answered is not demuxed again,
// so no observation is ever counted twice, and budget is not spent on
// tracks that have nothing left to say.
// Trackers are per-extent, for tracks this extent still owes evidence for;
// an already-answered track isn't demuxed again, so no observation is
// double-counted and no budget is spent on tracks with nothing left to say.
let mut trackers: HashMap<u16, ForcedTracker> = fresh_pids
.iter()
.map(|&p| (p, ForcedTracker::new()))
@@ -531,21 +519,16 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
// Sectors of this extent actually fed to the demuxer — the coverage the
// memo will claim, never more.
let mut covered: u32 = 0;
// AACS aligned units are anchored at THIS extent's start LBA, so tell a
// decrypt-on-read source to gate relative to it rather than absolute disc
// LBA 0 — without this the very first read of a clip whose start_lba is
// not itself 3-aligned is rejected. Mirrors the mux read paths. Every
// window offset is a whole number of units from that base, so sampling
// does not disturb the gate.
// AACS units are anchored at this extent's start LBA, so gate a decrypt-
// on-read source relative to it (not disc LBA 0), or the first read of a
// non-3-aligned clip is rejected. Mirrors the mux read paths.
reader.set_unit_base(ext.start_lba);
// `None` = the extent's whole plan ran. `Some(reason)` = it stopped early.
let mut cut_short: Option<StopReason> = None;
for window in &plan {
// Demux/parse state is PER WINDOW, because a window is a
// DISCONTIGUOUS run of the clip: carrying a demuxer across the gap
// would splice two unrelated byte runs into one PES. The cost is at
// most a display set straddling a window edge — the same trade the
// per-extent reset already made at extent edges.
// Demux/parse state is per-window: a window is a discontiguous run of
// the clip, so carrying a demuxer across the gap would splice unrelated
// byte runs into one PES (same trade as the per-extent reset).
let mut demux = TsDemuxer::new(fresh_pids);
let mut parsers: HashMap<u16, PgsParser> =
fresh_pids.iter().map(|&p| (p, PgsParser::new())).collect();
@@ -585,58 +568,22 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
break;
}
};
// Advance by what was actually READ, not by what was requested. A
// short-but-nonzero read (a source whose batch is smaller than the
// request — `PrefetchedSectorSource` returns its producer's batch)
// used to advance `lba`/`remaining`/`sectors_read` by the full
// `count`, silently SKIPPING the unread tail of the chunk while
// `stop` stayed `Exhausted` — so an absence-based forced verdict was
// asserted (and memoised) over data that was never seen.
// Advance by what was actually READ, not requested: a short-but-
// nonzero read (e.g. `PrefetchedSectorSource`) used to advance by
// the full `count`, silently skipping the unread tail as `Exhausted`.
let served = (n.min(want) / SECTOR_BYTES) as u32;
// ...but advancing by the raw sector count breaks the OTHER invariant
// this loop rests on: with a decrypting source every read must BEGIN a
// whole number of AACS aligned units past `ext.start_lba`, which is why
// `CHUNK_SECTORS` is a multiple of `ALIGNED_UNIT_SECTORS` (see the
// const-assert above). A short read of, say, 64 sectors is not a whole
// number of units, so the next `lba` would be off the unit grid,
// `DecryptingSectorSource` would reject it with `DecryptFailed` before
// reading, and the run would end `ReadFailed` — inconclusive, nothing
// memoised, forced detection silently back to the vendor label on
// exactly the encrypted discs the aligned chunk size was chosen for.
//
// So a read that did NOT satisfy the whole request advances only by
// whole aligned units, and the residue sectors are simply RE-READ from
// the next unit boundary on the following pass: at most two sectors of
// duplicated drive work, no gap, no double-feed of any byte to the
// demuxer (only the aligned prefix is fed), and no trailing partial
// unit — whose plaintext a unit-anchored decrypt cannot produce anyway
// — is ever handed to the parsers.
//
// A read that satisfied the whole request advances by all of it even
// when `count` itself was not unit-aligned: `count` is only ever below
// `CHUNK_SECTORS` on the extent's final chunk or at the sector budget,
// and both end the loop before another read of this extent.
// Advancing by raw sector count would break unit-alignment (every
// read must begin on an AACS aligned unit), so short reads advance only
// by whole units; the residue sectors are simply re-read next pass.
let got = if served >= u32::from(count) {
u32::from(count)
} else {
served - served % crate::aacs::content::ALIGNED_UNIT_SECTORS
};
if got == 0 {
// Less than one whole aligned unit came back, so the read position
// cannot move: the next aligned boundary IS the one just read. The
// bytes are real, so feed them (never lose an observation), then
// RETRY the same aligned lba — a source that short-changed one call
// commonly satisfies the next, and only when it repeatedly cannot
// yield a whole unit is the run the truncated prefix an error is.
// Bounded retries are what keep a source that never yields a unit
// (including one that claims sectors and returns none) from
// spinning here for ever. A retry that re-serves the same bytes
// feeds them twice: the BOOLEAN evidence is monotone (observed /
// saw-a-non-forced-set / saw-a-forced-set), so a repeat cannot
// change it, and the display COUNT can over-count by at most
// [`STALL_RETRY_LIMIT`] repeats of a sub-unit (< 6 KB) read —
// too little to move the shape comparison the demotion gate
// makes, which is against the busiest track on the disc.
// Less than one aligned unit came back: feed the real bytes (never
// lose an observation), then retry the same lba, bounded by
// [`STALL_RETRY_LIMIT`] since boolean evidence is monotone to repeats.
for pes in demux.feed(&buf[..n.min(want)]) {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
@@ -670,10 +617,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
}
}
}
// Saturating: a malformed extent can put the window's last chunk at
// the top of the 32-bit LBA space, and `remaining` has already
// reached 0 by then, so pinning the position is harmless — wrapping
// (or panicking in a debug build) is not.
// Saturating: a malformed extent can put the last chunk at the top of
// the 32-bit LBA space; `remaining` is already 0 by then, so pinning is
// harmless — wrapping (or a debug-build panic) is not.
lba = lba.saturating_add(got);
remaining -= got;
sectors_read += got;
@@ -687,11 +633,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
}
}
// Drain the window's tail: the demuxer holds the last PES open until
// the next PUSI arrives, and with a sampled read that PUSI is in a
// different window (or nowhere), so without this the last display set
// of EVERY window is thrown away — 16 lost observations per extent,
// exactly where the sample is thinnest.
// Drain the window's tail: the demuxer holds the last PES open until the
// next PUSI, which a sampled read may put in another window (or nowhere),
// so without this every window loses its last display set.
for pes in demux.flush() {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
@@ -714,22 +658,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
}
}
// Fold this extent's evidence in, and memoise it if the read reached a
// DESIGNED stop — the whole plan ran, the sector budget was hit, or every
// track had already settled. The budget is a designed stop for exactly the
// reason [`StopReason`] documents, and it is the stop that fires on every
// disc that HAS a forced track — excluding it from the cache would mean
// nothing is ever memoised on precisely those discs.
//
// What makes THAT sound is the coverage stored alongside: the entry says
// how much of the extent was actually read, so a later playlist that
// intended to read more of it will re-read instead of inheriting an
// absence claim from a prefix (see [`CachedEvidence::answers`]).
//
// A halt or a read fault is different: the cut-off point is arbitrary, so
// its evidence is real for THIS title (nothing observed is retracted) but
// must not be frozen in as the extent's answer, or one transient fault
// would be replayed onto every other playlist sharing the clip.
// Fold in this extent's evidence; memoise only on a DESIGNED stop (plan
// done, budget hit, or all tracks settled), with coverage stored so a later
// playlist re-reads rather than inherits a halt/fault's arbitrary cutoff.
let cacheable = cut_short.is_none_or(StopReason::absence_is_conclusive);
let sampled = !(complete_plan && cut_short.is_none());
for (&pid, t) in trackers.iter() {
@@ -742,11 +673,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
evidence: ev,
covered,
};
// Never replace a richer memo with a thinner one: an extent
// re-read under a smaller share would otherwise DOWNGRADE what is
// known about it. Facts merge (they are monotone), and the
// coverage claimed is the larger of the two — conservative, since
// the two samples together cover at least that much.
// Never replace a richer memo with a thinner one (a re-read under a
// smaller share would downgrade it): facts merge monotonically, and
// the coverage claimed is the larger of the two, which is conservative.
cache
.entry((ext.start_lba, ext.sector_count, pid))
.and_modify(|prev| {
@@ -800,7 +729,7 @@ fn tracker_evidence(t: &ForcedTracker, sampled: bool) -> TrackEvidence {
/// Compose the per-track verdicts a run is ENTITLED to assert from the evidence
/// it gathered. A track absent from the result keeps its vendor-derived flag.
///
/// Three gates, all PER TRACK, because the evidence is per track:
/// Four gates, all PER TRACK, because the evidence is per track:
/// * `observed` — saw no display set at all, so nothing is known. (Never assert
/// "not forced" from having seen nothing.)
/// * on a truncated run, `non_forced` — the track saw an actual non-forced
@@ -814,6 +743,9 @@ fn tracker_evidence(t: &ForcedTracker, sampled: bool) -> TrackEvidence {
/// that flag. Discs like that exist (measured: not one track on the disc
/// carries it), and without this gate the probe would strip the correct
/// forced label off every track on every one of them.
/// * for the FORCED verdict, a minimum display-set count (`PROMOTE_MIN_DISPLAY_SETS`)
/// unless the read was complete — over a SAMPLE of a mixed track, one flagged
/// set alone is not enough to promote the whole track to forced.
fn verdicts(evidence: &HashMap<u16, TrackEvidence>, conclusive: bool) -> HashMap<u16, bool> {
// Disc-level facts the demotion gate rests on, over the tracks judged
// together: does the authoring house set the flag at all, and how busy is the
@@ -832,13 +764,9 @@ fn verdicts(evidence: &HashMap<u16, TrackEvidence>, conclusive: bool) -> HashMap
busiest,
);
}
// Calling a track FORCED is also an absence claim — "no display set
// here was un-flagged" — and over a SAMPLE of a track that mixes
// flagged and un-flagged sets (measured: such tracks exist, flagging
// a quarter of their sets), catching one flagged set and nothing else
// is exactly what a wrong promotion looks like. A read that covered
// every extent end to end has no such gap and may promote off a
// single set (measured: single-sign forced tracks exist too).
// Calling a track FORCED is also an absence claim ("no set here was
// un-flagged"); over a SAMPLE of a mixed track, one flagged set alone
// is a wrong promotion, so require a minimum unless the read was complete.
!e.sampled || e.displays >= PROMOTE_MIN_DISPLAY_SETS
})
.map(|(&pid, e)| (pid, !e.non_forced))
@@ -854,18 +782,9 @@ fn apply_verdicts(title: &mut DiscTitle, verdicts: &HashMap<u16, bool>) {
&& let Some(&forced) = verdicts.get(&sub.pid)
{
sub.forced = forced;
// A demoted track must not go on describing itself as forced. The
// flag and the qualifier are two renderings of one fact for
// different consumers — the muxer writes `FlagForced` from
// `forced`, the JSON sidecar writes its qualifier string from
// `qualifier` — so leaving `Forced` behind here published a track
// whose sidecar said "forced" and whose Matroska header said it was
// not. The probe read the content; it outranks the vendor's claim.
//
// Only this direction is a contradiction. `qualifier == None` on a
// track the probe promoted is not one: `None` is the absence of an
// editorial qualifier, not an assertion that the track is ordinary,
// and the vendor never claimed otherwise.
// A demoted track must not go on describing itself as forced: flag and
// qualifier render one fact for different consumers, so leaving `Forced`
// behind contradicts the header. The probe outranks the vendor's claim.
if !forced && sub.qualifier == crate::disc::LabelQualifier::Forced {
sub.qualifier = crate::disc::LabelQualifier::None;
}
@@ -969,11 +888,9 @@ mod tests {
"the probe must never read past the budget, got {}",
reader.served
);
// ...and must actually SPEND it: a plan that quietly reads a fraction of
// the budget would look "safe" while observing even less than the
// head-first read it replaced. One window's slack is allowed (windows are
// whole numbers of aligned units, so 16 of them need not divide the
// budget exactly).
// ...and must actually SPEND it: quietly reading a fraction would look
// "safe" while observing less than the head-first read it replaced.
// One window's slack is allowed (windows need not divide the budget exactly).
assert!(
reader.served + WINDOW_SECTORS > PROBE_BUDGET_SECTORS,
"the probe must spend the budget it is given, got {}",
@@ -1096,10 +1013,9 @@ mod tests {
}
}
// PGS PCS layout (matches the private constants in mux::codec::pgs): a
// display-set frame begins with a PCS (segment type 0x16); byte 13 is
// number_of_composition_objects; byte 17 is the first object's flags, whose
// 0x40 bit is forced_on_flag.
// PGS PCS layout (matches mux::codec::pgs): a display-set frame begins with a
// PCS (segment type 0x16); byte 13 is number_of_composition_objects; byte 17
// is the first object's flags, whose 0x40 bit is forced_on_flag.
const PCS_SEG: u8 = 0x16;
const PCS_NUM_OBJECTS_OFF: usize = 13;
const PCS_FLAGS_OFF: usize = 17;
@@ -1172,10 +1088,8 @@ mod tests {
pos: 0,
};
let mut title = pgs_title(pid, false); // vendor label says NOT forced
// One sector, which is exactly what the reader serves: an extent that
// claims more sectors than the source yields is a SHORT read, and a short
// read is (correctly) inconclusive — see
// `read_error_after_partial_content_preserves_vendor_forced`.
// One sector, exactly what the reader serves: an extent claiming more
// sectors than the source yields is a SHORT read, correctly inconclusive.
title.extents = vec![Extent {
start_lba: 0,
sector_count: 1,
@@ -1193,9 +1107,8 @@ mod tests {
#[test]
fn nonforced_display_sets_clear_a_not_yet_forced_track() {
// A non-forced display set observed on the wire settles the track as
// not-forced. The vendor label here already agrees, so this pins the
// verdict itself (the track must not come back FORCED off one non-forced
// set) without engaging the demotion guard — that gets its own tests.
// not-forced; this pins the verdict itself (must not come back FORCED off
// one non-forced set) without engaging the demotion guard.
let pid = 0x1200u16;
let mut reader = TsReader {
data: ts_stream(pid, &pcs_display(false)),
@@ -1288,10 +1201,9 @@ mod tests {
#[test]
fn read_error_after_partial_content_preserves_vendor_forced() {
// The defect: a forced verdict rests on the ABSENCE of a non-forced
// display set. When the read dies mid-title that absence means nothing —
// the rest of the track was never seen — yet the probe used to apply it as
// authoritative, overwriting the vendor flag from a fraction of the data.
// The defect: a forced verdict rests on the ABSENCE of a non-forced set.
// When the read dies mid-title that absence means nothing, yet the probe
// used to apply it as authoritative, overwriting the vendor flag.
let pid = 0x1200u16;
let mut reader = PartialTsReader::new(ts_stream(pid, &pcs_display(true)), ThenWhat::Error);
let mut title = multi_read_pgs_title(pid, false); // vendor label: NOT forced
@@ -1304,9 +1216,8 @@ mod tests {
!s.forced,
"a forced verdict from a read-truncated prefix must not overwrite the vendor flag"
);
// And it must not be memoised: the cache is keyed on the extent list, so a
// cached inconclusive result would spread this one read fault across every
// playlist sharing these clips and block any later re-read.
// And it must not be memoised: caching an inconclusive result would spread
// this one read fault across every playlist sharing these clips.
assert!(
cache.is_empty(),
"an inconclusive probe must not be cached against these extents"
@@ -1315,12 +1226,9 @@ mod tests {
#[test]
fn read_error_keeps_a_track_that_already_settled_not_forced() {
// Per-track, not per-title: "not forced" is POSITIVE evidence — a
// non-forced display set was actually seen — and no amount of unread data
// could retract it. That verdict survives the truncation even though a
// forced verdict would not. The vendor label already says not-forced, so
// what is pinned here is that the truncation does not let the track come
// back FORCED.
// Per-track, not per-title: "not forced" is POSITIVE evidence (a non-forced
// set was actually seen), so no amount of unread data retracts it — that
// verdict survives truncation even though a forced verdict would not.
let pid = 0x1200u16;
let mut reader = PartialTsReader::new(ts_stream(pid, &pcs_display(false)), ThenWhat::Error);
let mut title = multi_read_pgs_title(pid, false);
@@ -1339,9 +1247,8 @@ mod tests {
#[test]
fn budget_stop_still_applies_the_forced_verdict() {
// The budget is a DESIGNED stop, not a failure: the probe's natural exit
// never fires for a genuinely forced track, so the budget is exactly the
// mechanism by which a forced verdict gets accepted from a bounded prefix.
// Classifying it as inconclusive would disable forced detection.
// never fires for a genuinely forced track, so the budget is how a forced
// verdict gets accepted from a bounded prefix; inconclusive would disable it.
let pid = 0x1200u16;
let mut reader = PartialTsReader::new(ts_stream(pid, &pcs_display(true)), ThenWhat::Zeros);
let mut title = multi_read_pgs_title(pid, false); // vendor label: NOT forced
@@ -1355,9 +1262,8 @@ mod tests {
"a forced verdict from a budget-bounded prefix must still be applied"
);
// One entry per (extent, PGS track): both extents reached a designed stop
// (the first was read to its end, the second stopped at the budget), so
// both are memoised — the budget is the stop that fires on every disc that
// HAS a forced track, so excluding it would memoise nothing there.
// (first read to end, second stopped at budget), so both are memoised —
// excluding the budget stop would memoise nothing on forced-track discs.
assert_eq!(cache.len(), 2, "a conclusive probe is memoised per extent");
assert!(cache.contains_key(&(0, 4, pid)));
assert!(cache.contains_key(&(100, u32::MAX, pid)));
@@ -1367,9 +1273,7 @@ mod tests {
fn cancelled_probe_is_neither_asserted_nor_cached() {
// A halt lands at an arbitrary chunk boundary, so an absence claim from it
// is worth no more than one from a read fault. Here the cancel arrives
// AFTER a forced display set has been observed — the same setup that,
// uncancelled, reaches the budget and legitimately asserts forced (see
// budget_stop_still_applies_the_forced_verdict).
// AFTER a forced set was observed, unlike the uncancelled budget case.
let pid = 0x1200u16;
let halt = crate::halt::Halt::new();
let mut reader = PartialTsReader {
@@ -1386,11 +1290,9 @@ mod tests {
!s.forced,
"a verdict from a cancelled probe must not overwrite the vendor flag"
);
// The cancel landed inside the SECOND extent, whose read is therefore an
// arbitrary prefix: that extent must not be memoised, or the one
// cancellation would be replayed onto every other playlist sharing the
// clip. (The first extent WAS read to its end before the cancel, so its
// own evidence is sound and keeping it is the point of per-extent keying.)
// The cancel landed inside the SECOND extent, an arbitrary prefix that must
// not be memoised, or the cancellation replays onto every playlist sharing
// the clip. (The first extent read to its end, so it stays cached.)
assert!(
!cache.contains_key(&(100, u32::MAX, pid)),
"a cancelled probe must not poison the cancelled extent's cache entry"
@@ -1898,23 +1800,9 @@ mod tests {
/// anything — and therefore whether the probe reports the run as truncated.
#[test]
fn a_stop_reason_decides_whether_absence_is_conclusive() {
// The debug line at the end of `probe_and_set_forced` is gated on
// `!stop.absence_is_conclusive()`. Assert that PREDICATE rather than
// counting emitted events.
//
// Reading a log line back needs a capturing subscriber, which is
// thread-local, while tracing's callsite-interest cache is global.
// Those race: the sibling test in disc/encrypt.rs that did this failed
// roughly one full-suite run in ten while passing every time in
// isolation, and serialising the captures crate-wide was not enough —
// the cache can still be re-evaluated against the process default
// dispatch. A boolean does not need a subscriber to check.
//
// ECMA of the decision: a stop that saw everything it was ever going to
// see (Exhausted) or stopped by DESIGN at the budget is conclusive, so
// the absence of a display set means the track is not forced and there
// is nothing to report. A stop that was cut short (Halted, ReadFailed)
// is not, and that is exactly what the operator needs told.
// Assert the PREDICATE directly rather than counting log events (capturing
// a tracing log line is racy — see the sibling test in disc/encrypt.rs).
// Exhausted/Budget stops are conclusive; Halted/ReadFailed are not.
assert!(
StopReason::Exhausted.absence_is_conclusive(),
"reading every extent to its end is a complete observation"
@@ -1936,15 +1824,9 @@ mod tests {
);
}
// ── sampling: reading where the subtitles actually are ──────────────────
//
// SYNTHETIC FIXTURES. The clips below are generated, not captured: they
// reproduce the measured SHAPE of real discs — a feature's subtitles begin
// minutes into the title, a full dialogue track carries one to two thousand
// display sets spread over the whole runtime, a forced-narrative track
// carries tens — and nothing else. They prove the probe's LOGIC over that
// shape. They do not prove the shape itself; that comes from measurement on
// real discs, and if the shape is wrong these fixtures are wrong with it.
// ── sampling: reading where the subtitles actually are ──────────
// SYNTHETIC FIXTURES reproducing the measured SHAPE of real discs; they
// prove the probe's LOGIC over that shape, not the shape itself.
/// One synthetic PGS track: `count` display sets, the first at
/// `first_sector`, then every `period_sectors`.
+505
View File
@@ -0,0 +1,505 @@
//! Normalized, format-agnostic disc profile — a thin typed VIEW over the scanned
//! [`Disc`] model.
//!
//! The scan already normalizes every source (DVD/BD/UHD/HD-DVD) into the same
//! [`Disc`] / [`DiscTitle`] / [`Stream`] shapes, carrying language / forced /
//! qualifier / purpose / secondary / label per stream. This module hoists those
//! into a flat, serde-friendly surface so a consumer reads
//! `profile.titles[i].subtitles[j].forced` (and the analogous audio/video flags)
//! for ANY disc with no per-format conditionals.
//!
//! Everything here is DERIVED from the model at construction: the enum `Stream`
//! is split into three typed vectors, the `qualifier` / `purpose` enums are
//! decomposed into booleans, and the "default track" selection is precomputed so
//! downstream never recomputes it. Every field is always populated — a sensible
//! default (`"und"` language, empty `name`, `false` flag) stands in where the
//! model has nothing, so consumers never handle a bare `Option`.
use serde::{Deserialize, Serialize};
use super::{
AudioStream, Disc, DiscFormat, DiscTitle, LabelPurpose, LabelQualifier, Stream, SubtitleStream,
VideoStream,
};
/// A disc's complete normalized profile: identity plus every title's typed
/// track breakdown, with the main-feature selection hoisted to `main_title`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct DiscProfile {
/// Disc container family (`"bluray"`, `"uhd"`, `"dvd"`, `"hddvd"`, …).
pub format: String,
/// Best available display name (disc metadata title, else volume id).
pub disc_name: String,
/// Stable disc identifier (the UDF volume identifier).
pub disc_id: String,
/// Index into `titles` of the selected main feature. The scan pre-sorts
/// titles so `titles[0]` is the main feature, so this is `0` whenever any
/// title exists.
pub main_title: usize,
/// Every title, in the scan's main-feature order.
pub titles: Vec<TitleProfile>,
}
/// One title's normalized profile: identity, duration/size, chapter count, the
/// main-feature flag, and its streams split into typed per-kind vectors.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct TitleProfile {
/// Position of this title within [`DiscProfile::titles`].
pub index: usize,
/// Playlist / program identifier (e.g. `"00800.mpls"`).
pub playlist: String,
/// Duration in seconds.
pub duration_secs: f64,
/// Total size in bytes.
pub size_bytes: u64,
/// Number of chapter points.
pub chapters: usize,
/// Whether this is the disc's main feature (`index == main_title`).
pub is_main: bool,
/// Video tracks, in declared order.
pub video: Vec<VideoTrack>,
/// Audio tracks, in declared order.
pub audio: Vec<AudioTrack>,
/// Subtitle tracks, in declared order.
pub subtitles: Vec<SubtitleTrack>,
}
/// A normalized video track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct VideoTrack {
/// Compact codec id (e.g. `"hevc"`, `"h264"`, `"vc1"`, `"mpeg2"`).
pub codec: String,
/// Resolution label (e.g. `"2160p"`, `"1080p"`, `"576i"`).
pub resolution: String,
/// HDR format id (e.g. `"sdr"`, `"hdr10"`, `"hdr10+"`, `"dv"`, `"hlg"`).
pub hdr: String,
/// Frame-rate label (e.g. `"23.976"`, `"25"`).
pub frame_rate: String,
/// Whether this is the title's default video track (first non-secondary).
pub default: bool,
/// Whether this is a secondary stream (PiP / dependent view).
pub secondary: bool,
}
/// A normalized audio track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct AudioTrack {
/// ISO 639-2 language code, `"und"` when the source stated none.
pub language: String,
/// Compact codec id (e.g. `"truehd"`, `"dtshd_ma"`, `"ac3"`).
pub codec: String,
/// Channel layout label (e.g. `"stereo"`, `"5.1"`, `"unknown"`).
pub channels: String,
/// Whether this is the title's default audio track (first non-secondary).
pub default: bool,
/// Editorial purpose flag: commentary track.
pub commentary: bool,
/// Editorial purpose flag: descriptive / audio-description track.
pub descriptive: bool,
/// Codec / variant text label; empty when the source stated none.
pub name: String,
}
/// A normalized subtitle track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SubtitleTrack {
/// ISO 639-2 language code, `"und"` when the source stated none.
pub language: String,
/// Compact codec id (e.g. `"pgs"`, `"dvdsub"`).
pub codec: String,
/// Forced-narrative flag.
pub forced: bool,
/// SDH (subtitles for the deaf / hard-of-hearing) flag.
pub sdh: bool,
/// Whether this is the title's default subtitle track. Always `false`: the
/// mux never marks a subtitle default, and the normalized view mirrors that.
pub default: bool,
/// Track name; empty (the subtitle model carries no free-text label).
pub name: String,
}
/// Container family as a compact, stable id. Kept here (not on [`DiscFormat`])
/// because it is a serialization concern of this view.
fn format_id(format: DiscFormat) -> &'static str {
match format {
DiscFormat::Uhd => "uhd",
DiscFormat::Fmts => "uhd_fmts",
DiscFormat::BluRay => "bluray",
DiscFormat::HdDvd => "hddvd",
DiscFormat::Dvd => "dvd",
DiscFormat::Unknown => "unknown",
}
}
/// `"und"` for an empty language code, else the code unchanged. Mirrors the
/// muxer's `language_or_und`: the one place a source with no language table has
/// to fall back to the ISO 639-2 "undetermined" code.
fn language_or_und(lang: &str) -> String {
if lang.is_empty() {
"und".to_string()
} else {
lang.to_string()
}
}
impl VideoTrack {
fn from_stream(v: &VideoStream, default: bool) -> Self {
Self {
codec: v.codec.id().to_string(),
resolution: v.resolution.to_string(),
hdr: v.hdr.id().to_string(),
frame_rate: v.frame_rate.to_string(),
default,
secondary: v.secondary,
}
}
}
impl AudioTrack {
fn from_stream(a: &AudioStream, default: bool) -> Self {
Self {
language: language_or_und(&a.language),
codec: a.codec.id().to_string(),
channels: a.channels.to_string(),
default,
commentary: a.purpose == LabelPurpose::Commentary,
descriptive: a.purpose == LabelPurpose::Descriptive,
name: a.label.clone(),
}
}
}
impl SubtitleTrack {
fn from_stream(s: &SubtitleStream) -> Self {
Self {
language: language_or_und(&s.language),
codec: s.codec.id().to_string(),
// The authoritative forced flag (`forced`, set by the content probe /
// STN) OR the label-derived qualifier — either is sufficient.
forced: s.forced || s.qualifier == LabelQualifier::Forced,
sdh: s.qualifier == LabelQualifier::Sdh,
// The mux never defaults a subtitle track; mirror that here.
default: false,
name: String::new(),
}
}
}
impl TitleProfile {
/// Build a title's profile. `index` / `is_main` are disc-level facts the
/// caller supplies (see [`DiscProfile::from_disc`]); the streams are split
/// and the per-kind default track is hoisted here.
pub fn from_title(title: &DiscTitle, index: usize, is_main: bool) -> Self {
let mut video = Vec::new();
let mut audio = Vec::new();
let mut subtitles = Vec::new();
// "First non-secondary is the default; keep only the first" — the same
// rule the Matroska path applies (`is_default = !secondary`, then only
// the first video and first audio survive as default).
let mut video_default_taken = false;
let mut audio_default_taken = false;
for s in &title.streams {
match s {
Stream::Video(v) => {
let default = !v.secondary && !video_default_taken;
video_default_taken |= default;
video.push(VideoTrack::from_stream(v, default));
}
Stream::Audio(a) => {
let default = !a.secondary && !audio_default_taken;
audio_default_taken |= default;
audio.push(AudioTrack::from_stream(a, default));
}
Stream::Subtitle(t) => subtitles.push(SubtitleTrack::from_stream(t)),
}
}
Self {
index,
playlist: title.playlist.clone(),
duration_secs: title.duration_secs,
size_bytes: title.size_bytes,
chapters: title.chapters.len(),
is_main,
video,
audio,
subtitles,
}
}
/// The title's video tracks.
pub fn video(&self) -> &[VideoTrack] {
&self.video
}
/// The title's audio tracks.
pub fn audio(&self) -> &[AudioTrack] {
&self.audio
}
/// The title's subtitle tracks.
pub fn subtitles(&self) -> &[SubtitleTrack] {
&self.subtitles
}
}
impl DiscProfile {
/// Build the normalized profile from a scanned [`Disc`].
pub fn from_disc(disc: &Disc) -> Self {
// The scan pre-sorts titles so `titles[0]` is the selected main feature
// (`sort_titles_by_main_feature`), so the main title is index 0 whenever
// any title exists.
let main_title = 0;
let has_titles = !disc.titles.is_empty();
let titles = disc
.titles
.iter()
.enumerate()
.map(|(i, t)| TitleProfile::from_title(t, i, has_titles && i == main_title))
.collect();
Self {
format: format_id(disc.format).to_string(),
disc_name: disc
.meta_title
.clone()
.unwrap_or_else(|| disc.volume_id.clone()),
disc_id: disc.volume_id.clone(),
main_title,
titles,
}
}
/// The selected main-feature title, or `None` when the disc scanned to zero
/// titles (a data-only image with no /BDMV, /HVDVD_TS or /VIDEO_TS). Never
/// panics — mirrors the `Option`-returning main-feature accessors elsewhere
/// (`titles.first()`, `dvdnav::resolve_main_title`).
pub fn main_title(&self) -> Option<&TitleProfile> {
self.titles.get(self.main_title)
}
}
impl From<&Disc> for DiscProfile {
fn from(disc: &Disc) -> Self {
DiscProfile::from_disc(disc)
}
}
impl Disc {
/// This disc's normalized, format-agnostic [`DiscProfile`].
pub fn profile(&self) -> DiscProfile {
DiscProfile::from_disc(self)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{
AudioChannels, AudioStream, Codec, ColorSpace, ContentFormat, DiscRegion, DiscTitle,
FrameRate, HdrFormat, Resolution, SampleRate, Stream, SubtitleStream, VideoStream,
};
/// A minimal titleless [`Disc`] for the disc-level tests.
fn test_disc() -> Disc {
Disc {
volume_id: String::new(),
meta_title: None,
format: DiscFormat::BluRay,
capacity_sectors: 0,
capacity_bytes: 0,
layers: 1,
titles: Vec::new(),
region: DiscRegion::Free,
aacs: None,
css: None,
encrypted: false,
aacs_error: None,
css_error: None,
content_format: ContentFormat::BdTs,
}
}
#[test]
fn main_title_is_none_on_empty_disc() {
// A data-only image (no /BDMV, /HVDVD_TS, /VIDEO_TS) scans to zero
// titles; main_title() must return None, not panic on titles[0].
let profile = DiscProfile::from_disc(&test_disc());
assert!(profile.titles.is_empty());
assert!(profile.main_title().is_none());
}
fn video(codec: Codec, res: Resolution, secondary: bool) -> Stream {
Stream::Video(VideoStream {
pid: 0x1011,
codec,
resolution: res,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
display_aspect: None,
secondary,
label: String::new(),
measured_cicp: None,
})
}
fn audio(lang: &str, secondary: bool, purpose: LabelPurpose, label: &str) -> Stream {
Stream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Stereo,
language: lang.into(),
sample_rate: SampleRate::S48,
secondary,
purpose,
label: label.into(),
})
}
fn subtitle(lang: &str, forced: bool, qualifier: LabelQualifier) -> Stream {
Stream::Subtitle(SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: lang.into(),
forced,
qualifier,
codec_data: None,
})
}
/// A BD transport-stream title with a mix of streams: two audio tracks
/// (main + commentary), two subtitles (forced + SDH), a video track.
fn bdts_title() -> DiscTitle {
let mut t = DiscTitle::empty();
t.content_format = ContentFormat::BdTs;
t.playlist = "00800.mpls".into();
t.duration_secs = 7200.0;
t.size_bytes = 30_000_000_000;
t.streams = vec![
video(Codec::Hevc, Resolution::R2160p, false),
audio("eng", false, LabelPurpose::Normal, "Dolby TrueHD 5.1"),
audio("eng", true, LabelPurpose::Commentary, "Director"),
subtitle("eng", true, LabelQualifier::Forced),
subtitle("eng", false, LabelQualifier::Sdh),
];
t
}
/// A DVD (MPEG program stream) title: two audio tracks (both non-secondary,
/// second is descriptive), one language-less subtitle, one video track.
fn dvd_title() -> DiscTitle {
let mut t = DiscTitle::empty();
t.content_format = ContentFormat::MpegPs;
t.playlist = "VTS_01".into();
t.duration_secs = 5400.0;
t.size_bytes = 4_000_000_000;
t.streams = vec![
video(Codec::Mpeg2, Resolution::R576i, false),
audio("eng", false, LabelPurpose::Normal, ""),
audio("eng", false, LabelPurpose::Descriptive, ""),
subtitle("", false, LabelQualifier::None),
];
t
}
#[test]
fn splits_streams_into_typed_vectors_across_formats() {
for title in [bdts_title(), dvd_title()] {
let p = TitleProfile::from_title(&title, 0, true);
assert_eq!(p.video.len(), 1, "one video track");
assert_eq!(p.audio.len(), 2, "two audio tracks");
assert_eq!(p.subtitles.len(), 1.max(p.subtitles.len()));
// Accessors return the same vectors.
assert_eq!(p.video(), p.video.as_slice());
assert_eq!(p.audio(), p.audio.as_slice());
assert_eq!(p.subtitles(), p.subtitles.as_slice());
}
}
#[test]
fn hoists_default_flags_first_non_secondary() {
let p = TitleProfile::from_title(&bdts_title(), 0, true);
// First (non-secondary) video/audio is the default.
assert!(p.video[0].default, "first video defaults");
assert!(p.audio[0].default, "first non-secondary audio defaults");
// The commentary track is secondary → never default.
assert!(
!p.audio[1].default,
"secondary/commentary audio not default"
);
// Subtitles are never default.
assert!(p.subtitles.iter().all(|s| !s.default));
}
#[test]
fn second_default_cleared_when_two_non_secondary_audio() {
// DVD title: BOTH audio tracks are non-secondary. Only the first keeps
// the default flag (mirrors the muxer's "keep only first default").
let p = TitleProfile::from_title(&dvd_title(), 0, true);
assert!(p.audio[0].default, "first audio default");
assert!(
!p.audio[1].default,
"second non-secondary audio default cleared"
);
assert!(p.audio[1].descriptive, "second audio is descriptive");
assert!(!p.audio[1].commentary);
}
#[test]
fn maps_qualifier_and_purpose_and_language_defaults() {
let p = TitleProfile::from_title(&bdts_title(), 0, true);
// Audio purpose → commentary/descriptive booleans.
assert!(!p.audio[0].commentary && !p.audio[0].descriptive);
assert!(p.audio[1].commentary);
assert_eq!(p.audio[1].name, "Director", "label maps to name");
// Subtitle qualifier → forced/sdh booleans.
assert!(p.subtitles[0].forced && !p.subtitles[0].sdh);
assert!(p.subtitles[1].sdh && !p.subtitles[1].forced);
// Language default: DVD title's blank subtitle language → "und".
let dvd = TitleProfile::from_title(&dvd_title(), 0, true);
assert_eq!(dvd.subtitles[0].language, "und");
assert_eq!(dvd.audio[0].language, "eng");
// Codec/resolution/hdr flattened as compact ids/labels.
assert_eq!(p.video[0].codec, "hevc");
assert_eq!(p.video[0].resolution, "2160p");
assert_eq!(p.video[0].hdr, "hdr10");
assert_eq!(dvd.video[0].codec, "mpeg2");
}
#[test]
fn from_disc_hoists_main_and_is_main() {
let mut disc = test_disc();
disc.format = DiscFormat::BluRay;
disc.meta_title = Some("SOME MOVIE".into());
disc.volume_id = "VOL_ID".into();
disc.titles = vec![bdts_title(), dvd_title()];
let profile = disc.profile();
assert_eq!(profile.format, "bluray");
assert_eq!(profile.disc_name, "SOME MOVIE");
assert_eq!(profile.disc_id, "VOL_ID");
assert_eq!(profile.main_title, 0);
assert!(profile.titles[0].is_main, "titles[0] is the main feature");
assert!(!profile.titles[1].is_main);
assert_eq!(profile.titles[1].index, 1);
assert_eq!(profile.main_title().unwrap().playlist, "00800.mpls");
}
#[test]
fn disc_name_falls_back_to_volume_id() {
let mut disc = test_disc();
disc.meta_title = None;
disc.volume_id = "PLAIN_VOLUME".into();
assert_eq!(disc.profile().disc_name, "PLAIN_VOLUME");
}
#[test]
fn serde_round_trip() {
let mut disc = test_disc();
disc.format = DiscFormat::Dvd;
disc.volume_id = "RT".into();
disc.titles = vec![bdts_title(), dvd_title()];
let profile = disc.profile();
let json = serde_json::to_string(&profile).expect("serialize");
let back: DiscProfile = serde_json::from_str(&json).expect("deserialize");
assert_eq!(profile, back, "profile round-trips through serde");
}
}
+51 -4
View File
@@ -20,6 +20,12 @@ pub struct DriveCapture {
pub rb_f1: Option<Vec<u8>>,
/// READ_BUFFER mode 6 (MTK vendor data)
pub rb_mode6: Option<Vec<u8>>,
/// READ_BUFFER 0xB0 @0x04 (Renesas signature only)
pub rb_b0_04: Option<Vec<u8>>,
/// WRITE_BUFFER 0x41 @0xA5AAAA (Renesas signature only)
pub wb_41: Option<Vec<u8>>,
/// READ_BUFFER 0xB0 @0x500000 (Renesas signature only)
pub rb_b0_500000: Option<Vec<u8>>,
}
/// A single GET CONFIGURATION feature response from the drive.
@@ -76,6 +82,33 @@ pub fn capture_drive_data(session: &mut Drive) -> Result<DriveCapture> {
let rb_f1 = session.read_buffer(0x02, 0xF1, 48); // Pioneer
let rb_mode6 = session.read_buffer(0x06, 0x00, 32); // MTK
// Renesas signature: RB 0xF1 bytes [16..19] == "SAT".
let (mut rb_b0_04, mut wb_41, mut rb_b0_500000) = (None, None, None);
if rb_f1
.as_ref()
.is_some_and(|f| f.len() >= 19 && &f[16..19] == b"SAT")
{
use crate::scsi::{DataDirection as D, build_read_buffer};
rb_b0_04 = raw(
session,
&build_read_buffer(0x02, 0xB0, 0x04, 164),
D::FromDevice,
164,
);
wb_41 = raw(
session,
&[0x3B, 0x02, 0x41, 0xA5, 0xAA, 0xAA, 0, 0, 0, 0],
D::None,
0,
);
rb_b0_500000 = raw(
session,
&build_read_buffer(0x02, 0xB0, 0x500000, 164),
D::FromDevice,
164,
);
}
// Standard queries
let rpc_state = session.report_key_rpc_state();
let mode_2a = session.mode_sense_page(0x2A);
@@ -88,9 +121,24 @@ pub fn capture_drive_data(session: &mut Drive) -> Result<DriveCapture> {
mode_2a,
rb_f1,
rb_mode6,
rb_b0_04,
wb_41,
rb_b0_500000,
})
}
/// Run a raw CDB; `Some(data)` on GOOD status (empty for a write), else `None`.
fn raw(
session: &mut Drive,
cdb: &[u8],
dir: crate::scsi::DataDirection,
len: usize,
) -> Option<Vec<u8>> {
let mut buf = vec![0u8; len];
let r = session.scsi_execute(cdb, dir, &mut buf, 5_000).ok()?;
(r.status == 0).then(|| buf[..r.bytes_transferred.min(buf.len())].to_vec())
}
/// Mask a string for privacy (letters->A, digits->0).
pub fn mask_string(s: &str) -> String {
s.chars()
@@ -150,10 +198,9 @@ mod tests {
#[test]
fn mask_string_preserves_non_ascii_chars() {
// is_ascii_alphabetic/is_ascii_digit are false for non-ASCII, so
// multibyte chars pass through unchanged (no mojibake, no panic).
// 'c','a','f' are ASCII letters → 'A'; 'é' is non-ASCII →
// preserved; '9' → '0'.
// is_ascii_alphabetic/is_ascii_digit are false for non-ASCII, so multibyte
// chars pass through unchanged (no mojibake, no panic): ASCII letters → 'A',
// digits → '0', 'é' preserved.
assert_eq!(mask_string("café9"), "AAAé0");
}
+3 -8
View File
@@ -84,14 +84,9 @@ pub fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
let sr_id = DriveId::from_drive(sr_transport.as_mut())?;
drop(sr_transport);
for (sg_path, sg_id) in find_drives() {
// Require a non-empty serial before treating vendor/product/
// serial as a unique match. serial_number falls back to an
// empty string when GET CONFIGURATION 0108h is unavailable
// (common on OEM drives); two same-model drives would then
// both compare equal and the first in enumeration order would
// win silently, resolving sr1 to sr0's sg node. An empty
// serial can't disambiguate, so fall through to the no-match
// path instead.
// Require a non-empty serial before treating vendor/product/serial as a unique
// match: serial_number falls back to "" when GET CONFIGURATION 0108h is
// unavailable (OEM drives), which would let same-model drives collide silently.
if !sr_id.serial_number.is_empty()
&& sg_id.vendor_id == sr_id.vendor_id
&& sg_id.product_id == sr_id.product_id
+65 -159
View File
@@ -8,14 +8,9 @@ pub fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
// A failed `ioctl(SG_IO)` and a vanished device never produced a SCSI
// reply at all — they are dead-bus faults, not recoverable bad sectors.
// `Error::is_scsi_transport_failure` (error.rs) already declares both
// variants transport failures so sweep / patch / fill_extents abort the
// pass instead of zero-filling against a wedged device; callers that
// flatten an error through here (Drive::read_one, DiscStream::fill_extents,
// freemkv-engine's recovery sweep/patch) would otherwise collapse them to
// status 0 and silently destroy that classification.
// A failed `ioctl(SG_IO)` or vanished device is a dead-bus fault, not a
// recoverable bad sector — map to SCSI_STATUS_TRANSPORT_FAILURE so sweep /
// patch / fill_extents abort the pass instead of zero-filling a wedged device.
Error::IoError { .. } | Error::DeviceNotFound { .. } => {
(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None)
}
@@ -299,16 +294,9 @@ impl Drive {
for attempt in 0..60u64 {
hb.tick(attempt, 60);
let mut buf = [0u8; 0];
// `checked_exec`, not a bare `execute`. This poll was the ONE
// drive path that talked to the transport directly, and it is the
// longest-running one: 60 x 500 ms = ~30 s. It never read
// `self.halt`, so an operator Stop pressed while a cold drive spun
// up was ignored for up to half a minute while every other path in
// this file returned `Halted` at its next command boundary. A Stop
// that does nothing for 30 s is indistinguishable from a hung app.
//
// A TUR that FAILS is the ordinary not-ready-yet answer and must
// keep the loop going — only `Halted` aborts it.
// `checked_exec`, not bare `execute`: this poll (60 x 500 ms = ~30 s) must
// see `self.halt`, or a Stop during cold spin-up is ignored for half a
// minute. A TUR that FAILS is just not-ready-yet and keeps the loop going.
match self.checked_exec(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(
@@ -362,21 +350,9 @@ impl Drive {
5_000,
);
// MMC-6 §6.7: byte 5 is a Media Status only when the Event Header
// actually announces a Media Event Descriptor behind it. The header is
// Event Descriptor Length (big-endian, bytes 0-1), then byte 2 = NEA
// (bit 7) + Notification Class (bits 2-0), then byte 3 = Supported
// Event Classes. With NEA set the drive is telling us there is NO event
// to report and returns the header alone; with a different Notification
// Class the descriptor that follows is not a media one and its second
// byte means something else entirely.
//
// Decoding byte 5 unconditionally turns both of those into Media Status
// 0 == "tray closed, no disc" — a drive with a disc loaded reported as
// empty, from a reply that said nothing about the media at all. The
// drive is untrusted input here, so an event-less or foreign-class reply
// yields no media state and the TEST UNIT READY fallback answers
// instead.
// MMC-6 §6.7: byte 5 is Media Status only when the Event Header (NEA flag +
// Notification Class) announces a Media Event Descriptor. Otherwise decoding
// it reads as "no disc" for untrusted input, so fall back to TUR instead.
const NEA: u8 = 0x80;
const NOTIFICATION_CLASS_MASK: u8 = 0x07;
const NOTIFICATION_CLASS_MEDIA: u8 = 0x04;
@@ -416,10 +392,9 @@ impl Drive {
0x00 => DriveStatus::NoDisc, // tray closed, no disc
0x01 => DriveStatus::TrayOpen, // tray open, no media
0x02 => DriveStatus::DiscPresent, // tray closed, disc present
// 0x03 = tray-open bit AND media-present bit both set:
// a contradictory/transient state. Don't report it as
// ready — autorip must not start a rip on a drive that
// is still settling. Treat as tray-open.
// 0x03 = tray-open AND media-present both set: a contradictory,
// transient state. Don't report ready — autorip must not start on
// a drive that's still settling — so treat as tray-open.
0x03 => DriveStatus::TrayOpen,
_ => DriveStatus::Unknown,
}
@@ -518,26 +493,17 @@ impl Drive {
pub fn init(&mut self) -> Result<()> {
let t0 = std::time::Instant::now();
tracing::info!(target: "freemkv::drive", phase = "init", "begin");
// Drive-prep runs for EVERY disc, DVD INCLUDED. The drive-level firmware
// unlock lifts riplock and readies max read speed regardless of disc type
// — drive features are disc-independent. At init the disc kind is not yet
// probed (Unknown), so only the identity-keyed DRIVE unlocker matches
// here; the AACS host-cert handshake and the CSS bus-auth handshake run
// LATER, each gated on the actual disc kind, on TOP of the already-
// unlocked drive. A genuine transport fault means the bus is dead — abort
// init (the v1.1.0 invariant; `if let Ok` was silently swallowing it).
// Every other error (NotApplicable / no match) is "nothing applied" —
// fall through to stock mode. NOTE: the `if disc_is_dvd() { return }` skip
// that used to sit here was the v1.0.0-rc.1 regression — it skipped the
// drive-prep for DVD, leaving DVDs riplocked at stock speed.
// Drive-prep runs for EVERY disc, DVD INCLUDED — drive features are
// disc-independent; AACS/CSS handshakes run LATER, gated on disc kind. A
// transport fault aborts init (v1.1.0 invariant); other errors fall through.
self.init_ran = true;
let (matched, unlock_res) =
crate::unlock_bridge::run_features(self.scsi.as_mut(), &self.drive_id);
let r: Result<()> = match unlock_res {
Ok(unlocked) => {
// Record WHICH drive-prep unlocker actually ran — "LibreDrive"
// (MediaTek) or "Renesas" — not the ld-only identity lookup, so a
// Renesas drive reports itself honestly rather than as nothing.
// Record WHICH drive-prep unlocker ran ("MT1959" bus-unlock or
// "Renesas" cert route), not the id-only lookup, so a Renesas
// drive reports itself honestly rather than as nothing.
self.unlocker_name = Some(matched.to_string());
// Stash the OEM Volume ID the unlocker returned for the AACS
// handshake phase (do_handshake reads it via `oem_vid()`). A
@@ -554,19 +520,14 @@ impl Drive {
}),
Err(_) => Ok(()),
};
// Raise the drive to its maximum read speed — UNCONDITIONALLY whenever
// the bus is alive (init didn't transport-fault), whether or not an
// unlocker matched, and for ANY disc type (DVD now flows through here too,
// so it gets max speed on the freshly firmware-unlocked drive instead of
// the stock riplock). A stock-mode drive with no firmware unlocker still
// wants max speed. Best-effort: a failure here must NOT fail the rip.
// Raise to max read speed UNCONDITIONALLY whenever the bus is alive, for
// ANY disc type — DVD included, and even a stock-mode drive with no
// firmware unlocker still wants it. Best-effort: failure must NOT fail the rip.
if r.is_ok() {
self.set_speed(Self::SPEED_MAX_KBPS);
// Ask the drive to REPORT recovered/marginal reads rather than
// silently commit best-effort data as GOOD (the dirty-disc
// "passed-clean-but-decodes-with-errors" trap). Best-effort: a drive
// that doesn't honor it just keeps its defaults — no regression, and
// on a clean disc it changes nothing.
// Ask the drive to REPORT recovered/marginal reads instead of silently
// committing best-effort data as GOOD (the dirty-disc "passed-clean-but-
// decodes-with-errors" trap). Best-effort: unsupported drives keep defaults.
self.enable_recovered_error_reporting();
}
tracing::info!(
@@ -843,29 +804,20 @@ impl Drive {
"Drive::read enter"
);
// Cap each CDB to the transport's max data-in transfer. A single
// READ larger than the adapter limit fails outright on some
// backends (notably Windows SPTI, where a 16 MiB read exceeds the
// adapter MaximumTransferLength → DeviceIoControl fails → we'd
// mis-read it as a transport failure and spam tiny-read fallbacks).
// For the common small read (count <= max_sectors) this is a single
// read_one call with no behavior change.
// Cap each CDB to the transport's max data-in transfer: a single READ past the
// adapter limit fails outright on some backends (e.g. Windows SPTI, where a
// 16 MiB read exceeds MaximumTransferLength and we'd mis-read it as transport failure).
let max_sectors = (self.scsi.max_transfer_bytes() / 2048).max(1) as u32;
if count as u32 <= max_sectors {
return self.read_one(lba, count, buf, timeout_ms, recovery, fua);
}
// Large read: split into chunks of at most `max_sectors` sectors,
// each a self-contained READ(10) with the same validation. Any
// chunk error reports that chunk's LBA (more precise than the whole
// request's base LBA).
// Large read: split into `max_sectors`-sized chunks, each a self-contained
// READ(10). Any chunk error reports that chunk's LBA, not the base LBA.
let count = count as u32;
// Check the caller's buffer ONCE, up front. The chunk loop slices `buf`
// by `count * 2048`; without this an undersized buffer PANICKED ('range
// end index out of range') out of the public `read`/`read_fua`, while the
// single-chunk path above tolerates the same undersized buffer and returns
// `Err(DiscRead)` from `checked_exec`. Behaviour on an undersized buffer
// must not depend on the transport's transfer limit.
// Check the caller's buffer ONCE, up front: the chunk loop slices `buf` by
// `count * 2048`, and without this an undersized buffer PANICKED out of the
// public `read`/`read_fua` instead of returning `Err(DiscRead)` like the single-chunk path.
if buf.len() < count as usize * 2048 {
return Err(Error::DiscRead {
sector: lba as u64,
@@ -873,11 +825,9 @@ impl Drive {
sense: None,
});
}
// The whole range must be addressable: SBC-3 READ(10) carries a 32-bit
// LOGICAL BLOCK ADDRESS, so a request whose last chunk crosses `u32::MAX`
// has no valid CDB. `lba + done` below was unchecked — a debug panic out
// of the public API, and in release a wrap to a low LBA that was read and
// returned as if it were the requested one.
// The whole range must be addressable: READ(10) carries a 32-bit LBA, so a
// request crossing `u32::MAX` has no valid CDB. `lba + done` was unchecked —
// a debug panic, or in release a wrap to a low LBA read and returned as requested.
if lba.checked_add(count.saturating_sub(1)).is_none() {
return Err(Error::DiscRead {
sector: lba as u64,
@@ -920,13 +870,9 @@ impl Drive {
// re-fetch the medium past its cache.
fua: bool,
) -> Result<usize> {
// FUA is OFF on the bulk path — unconditionally forcing every READ(10)
// past the cache disabled the drive's readahead/streaming cache on the
// sequential sweep and collapsed throughput ~10x (UHD 15-25 → ~2 MB/s,
// DVD → ~0.5 MB/s), disc-type-agnostic — the cache IS the streaming
// throughput. It is set ONLY when a Pass-N recovery handler (FuaRetry)
// asks for it per marginal-sector re-read, where cache-masking of a
// stochastic sector actually matters (#55).
// FUA is OFF on the bulk path: forcing every READ(10) past the cache disabled
// readahead on the sequential sweep and collapsed throughput ~10x (UHD → ~2
// MB/s, DVD → ~0.5 MB/s). Set ONLY per marginal-sector re-read by FuaRetry (#55).
let cdb = [
crate::scsi::SCSI_READ_10,
if fua { 0x08 } else { 0x00 },
@@ -949,26 +895,9 @@ impl Drive {
Ok(result) if result.bytes_transferred == count as usize * 2048 => {
Ok(result.bytes_transferred)
}
// A READ(10) that completes with GOOD status but a residual
// underrun (bytes_transferred < requested) is a SHORT transfer:
// the tail of `buf` still holds stale bytes from a prior read.
// Committing those as recovered/Good is silent data corruption, so
// treat a short transfer as a failed read — the caller marks the
// range NonTrimmed and retries (a loud miss, never a silent commit).
// The sector/file path enforces the same invariant in
// sector/prefetched.rs; this is the live-drive counterpart.
//
// SAY SO. The refusal was correct and completely silent: the
// sibling `Err(e)` arm below warns with lba/count/status, this one
// logged nothing at all, so a drive that residual-underruns on
// GOOD status was indistinguishable in the logs from a scratched
// disc — two different populations (replace the drive vs. clean
// the disc) collapsed into one. Absence of a log is itself the
// defect here; the read still fails either way.
//
// `scsi_status`/`sense` are deliberately absent from the line
// rather than faked: the command SUCCEEDED, so there is no sense
// data to report. `transferred` vs `expected` is the whole signal.
// GOOD status with a residual underrun is a SHORT transfer: `buf`'s tail
// holds stale bytes, so fail (NonTrimmed, retried) rather than commit stale
// data — and log it, so it isn't indistinguishable from a scratched disc.
Ok(result) => {
tracing::warn!(
target: "freemkv::drive",
@@ -997,11 +926,9 @@ impl Drive {
"Drive::read checked_exec failed"
);
// /dev/sr0 pread fallback (Linux only). The kernel
// sr_mod driver auto-retries failed reads (~5× per
// command). Empirically (BU40N + a UHD disc,
// 2026-05-08) dd via /dev/sr0 recovers ~50% of bad
// sectors that a single-shot SG_IO READ misses.
// /dev/sr0 pread fallback (Linux only): sr_mod auto-retries reads
// (~5x). Empirically (BU40N + UHD disc, 2026-05-08) this recovers
// ~50% of bad sectors that a single-shot SG_IO READ misses.
#[cfg(target_os = "linux")]
if recovery
&& let Some(fd) = self.block_dev_fd
@@ -1140,15 +1067,9 @@ impl Drive {
let stop = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x00, 0]; // START=0, LOEJ=0 → spin down
let start = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x01, 0]; // START=1, LOEJ=0 → spin up
let mut buf = [0u8; 0];
// `checked_exec` + `sleep_until_halted`, not `execute` + a blind
// `thread::sleep`. This routine is ~15 s of deliberate waiting
// (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) issued from the
// recovery path, i.e. exactly when a run is going badly and the
// operator is most likely to press Stop. Both commands bypassed the
// halt flag and both sleeps were unconditional, so the whole cycle was
// deaf: Stop appeared to hang. Nothing here is un-interruptible — a
// half-finished spin cycle leaves the drive spun down, which the next
// command spins back up.
// `checked_exec` + `sleep_until_halted`, not blind `thread::sleep`: this ~15 s
// wait runs from the recovery path, exactly when Stop is likely pressed. A
// half-finished spin cycle is fine — the next command spins the drive back up.
self.checked_exec(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
@@ -1461,17 +1382,9 @@ mod halt_tests {
let r = sleep_until_halted(&flag, Duration::from_secs(10));
assert!(matches!(r, Err(Error::Halted)));
let waited = t0.elapsed();
// What this test is for: the sleep must end because the flag flipped,
// NOT because the 10 s timeout expired. Anything comfortably under
// that proves it, and the lower bound proves it did not return early
// for some other reason.
//
// The upper bound used to be 350 ms — flag at ~150 ms plus one 100 ms
// poll slice, with a little slack. That measures the SCHEDULER, not
// this function: a loaded CI runner took 377 ms and failed, which says
// nothing about whether the wake worked. Bound it well below the
// timeout instead, so the assertion still distinguishes the two
// outcomes it exists to distinguish.
// The sleep must end because the flag flipped, not the 10 s timeout; the lower
// bound rules out returning early otherwise. Upper bound was 350 ms but that
// measures the scheduler: a loaded CI runner hit 377 ms and failed for no reason.
assert!(waited < Duration::from_secs(2), "waited {waited:?}");
assert!(waited >= Duration::from_millis(140), "waited {waited:?}");
}
@@ -1847,11 +1760,9 @@ mod command_tests {
#[test]
fn read_builds_read10_cdb_with_be_lba_and_count() {
// Drive::read issues READ(10) (0x28). LBA bytes 2..5 big-endian,
// transfer length bytes 7..8 big-endian (MMC-6). FUA is DISABLED (byte 1
// == 0x00) so the drive cache/readahead is allowed — forcing FUA on the
// bulk sweep collapsed throughput ~10x. Distinct nibbles catch a swapped
// shift.
// READ(10) (0x28): LBA bytes 2..5 BE, length bytes 7..8 BE (MMC-6). FUA is
// DISABLED (byte 1 == 0x00) — forcing FUA on the bulk sweep collapsed
// throughput ~10x. Distinct nibbles catch a swapped shift.
let RecordingHarness {
drive: mut d,
cdb,
@@ -2435,10 +2346,9 @@ mod command_tests {
#[test]
fn select_drive_falls_back_to_first_when_none_have_media() {
// No drive reports a disc → fall back to the FIRST opened drive so
// single-drive / quirky setups still get a drive (historical
// behavior preserved). Tag drive #1 distinctly (TrayOpen 0x01) and
// confirm it, not #2 (NoDisc 0x00), is returned.
// No drive reports a disc → fall back to the FIRST opened drive so single-drive
// / quirky setups still get one (historical behavior). Tag drive #1 distinctly
// (TrayOpen 0x01) and confirm it, not #2 (NoDisc 0x00), is returned.
let drives = vec![drive_with_media_byte(0x01), drive_with_media_byte(0x00)];
let mut picked = select_drive_with_media(drives.into_iter()).expect("a fallback drive");
assert_eq!(
@@ -2506,10 +2416,9 @@ mod command_tests {
#[test]
fn drive_status_short_transfer_falls_back_to_tur() {
// bytes_transferred < 6 means the GET EVENT reply is unusable;
// the code falls back to a TUR. FixedTransport always returns
// Ok, so the TUR "succeeds" → DiscPresent. (Buffer length 8 but
// payload only 4 bytes → bytes_transferred = 4.)
// bytes_transferred < 6 (buffer len 8, payload 4) means the GET EVENT reply
// is unusable, so code falls back to a TUR; FixedTransport always returns
// Ok, so the TUR "succeeds" → DiscPresent.
let mut d = drive_with(vec![0u8; 4]);
assert_eq!(d.drive_status(), DriveStatus::DiscPresent);
}
@@ -2609,11 +2518,9 @@ mod command_tests {
#[test]
fn get_config_feature_encodes_feature_code_be_in_cdb() {
// GET CONFIGURATION (0x46), RT field byte 1 = 0x02 (report the
// named feature), then the 16-bit feature code big-endian in
// bytes 2..4. 0x010D has a nonzero high byte, so a swapped shift
// (>> vs <<) or byte order bug asks the drive for the wrong
// feature entirely.
// GET CONFIGURATION (0x46), RT byte 1 = 0x02, feature code big-endian in
// bytes 2..4. 0x010D has a nonzero high byte, so a swapped shift (>> vs <<)
// or byte order bug would ask the drive for the wrong feature.
let RecordingHarness {
drive: mut d,
cdb,
@@ -3003,10 +2910,9 @@ mod command_tests {
#[test]
fn init_without_unlocker_is_ok_oem_fallback() {
// The test transport's identity matches no registered unlocker, so
// route_unlock returns None. init() must succeed (leaving the drive
// in stock mode for the host-cert handshake), not error — the OEM
// route is the no-match fallback, not a failure.
// No registered unlocker matches, so route_unlock returns None. init() must
// still succeed (stock mode for the host-cert handshake) — no-match is the
// OEM fallback, not a failure.
let mut d = drive_with(vec![]);
assert!(
d.init().is_ok(),
+7 -4
View File
@@ -8,11 +8,14 @@
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against real discs.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
//! that resolves the start cell build on top of this.
//! Current contents: [`vmcmd`] — the VM command decoder; [`nav`] — the
//! First-Play navigation executor that resolves which title the disc's own
//! navigation selects as the feature.
pub mod vmcmd;
pub(crate) mod nav;
pub(crate) mod vmcmd;
pub(crate) use nav::resolve_main_title;
use crate::sector::SectorSource;
+737
View File
@@ -0,0 +1,737 @@
//! DVD-Video navigation executor — the DVD arm of "mimic a real player" title
//! selection (issue #40).
//!
//! A player enters a disc at the **First-Play PGC** (FP_PGC): a short program of
//! VM commands (parsed by [`super::vmcmd`]) that runs before any menu draws and
//! typically dispatches straight to the feature title. This module reads that
//! program out of `VIDEO_TS.IFO` (VMGI), executes its pre-command list on a
//! minimal register machine, and — when the program deterministically reaches a
//! title dispatch (`JumpTT`) — maps the selected VMG title through the title
//! search pointer table (`TT_SRPT`) to its `(VTS number, title-in-set)`
//! coordinates. That is the title a player would land on, so DVD title
//! selection can prefer it over the structural leading-cell heuristic.
//!
//! Contract: this is a strict improvement over the fallback. Any parse error,
//! interactive/menu-only entry, or non-convergence yields `None`, and the caller
//! keeps today's behaviour. Every table count and offset is attacker-controlled,
//! so all arithmetic is checked and all reads are bounds-guarded — the entry
//! point [`resolve_from_vmg`] never panics on any input (exercised by the
//! never-panic harness).
use super::vmcmd::{self, Compare, Instr};
use crate::consts::SECTOR_BYTES;
use crate::sector::SectorSource;
use crate::udf::UdfFs;
// ── VMGI management-table byte offsets into VIDEO_TS.IFO (DVD-Video spec) ─────
// The VMGI_MAT header carries these fixed offsets: per-disc values live at
// disc-constant offsets.
const VMGI_MAGIC: &[u8; 12] = b"DVDVIDEO-VMG";
/// u32 **byte** offset (from the start of VIDEO_TS.IFO) of the First-Play PGC.
const VMGI_FP_PGC_PTR: usize = 0x84;
/// u32 **sector** offset (from the start of VIDEO_TS.IFO) of TT_SRPT.
const VMGI_TT_SRPT_PTR: usize = 0xC4;
/// Within a PGC, the command-table pointer is a u16 at `PGC + 0xE4` giving the
/// command table's byte offset relative to the PGC start (DVD-Video PGC layout;
/// the same layout `ifo::parse_pgc` reads for cell/program tables).
const PGC_CMD_TBL_PTR: usize = 0xE4;
/// Upper bound on commands executed before declaring non-convergence. A
/// conformant First-Play routine reaches a title in a handful of steps; the cap
/// stops a crafted command list (self-`Goto`, mutual jumps) from spinning.
const STEP_BUDGET: usize = 1024;
/// Maximum commands honoured in one PGC command list. The DVD-Video VM caps a
/// list at 128 pre / 128 post / 128 cell commands; the on-disc count is an
/// untrusted u16, so it is clamped to this format maximum.
const MAX_CMDS: usize = 128;
/// Maximum TT_SRPT entries honoured — the DVD-Video 99-title format maximum
/// (mirrors `ifo::MAX_TT_SRPT_TITLES`; the on-disc count is an untrusted u16).
const MAX_TT_SRPT_TITLES: usize = 99;
/// The title the First-Play navigation selects, in VMG coordinates.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResolvedTitle {
/// 1-based VMG title number (the TT_SRPT index the `JumpTT` named).
pub title: u8,
/// 1-based VTS (title set) number the title lives in.
pub vtsn: u8,
/// 1-based title-within-set number (TT_SRPT `VTS_TTN`).
pub vts_ttn: u8,
}
// ── Bounds-guarded big-endian reads (return None past the end) ────────────────
#[inline]
fn u16_at(b: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*b.get(o)?, *b.get(o.checked_add(1)?)?]))
}
#[inline]
fn u32_at(b: &[u8], o: usize) -> Option<u32> {
Some(u32::from_be_bytes([
*b.get(o)?,
*b.get(o.checked_add(1)?)?,
*b.get(o.checked_add(2)?)?,
*b.get(o.checked_add(3)?)?,
]))
}
/// A register operand index names a system parameter (SPRM) when it carries the
/// `0x80` bias — matching [`Vm::reg`]'s split of GPRM (`< 128`) vs SPRM
/// (`>= 128`). Only SPRM reads are session-specific and therefore taint a
/// compare; GPRM reads do not.
#[inline]
fn is_sprm(idx: u8) -> bool {
idx >= 128
}
// ── Minimal navigation register machine ──────────────────────────────────────
/// The subset of the DVD-Video VM register file the First-Play resolver needs:
/// 16 general parameters (GPRMs) and the system parameters (SPRMs). Execution
/// starts from a cold machine (all zero) — the same starting point a player has
/// before any user interaction. The SPRMs a First-Play routine might branch on
/// (region, parental level, language) are player-/session-specific and unknown
/// at scan time; leaving them zero means the resolver only commits to a title
/// when the program reaches one *without* depending on them (an unconditional
/// dispatch, or a conditional whose zero-register path a player also takes).
/// When a branch genuinely depends on an unknown SPRM the routine is treated as
/// undecidable and the caller falls back — see the module contract.
struct Vm {
gprm: [u16; 16],
sprm: [u16; 24],
/// Per-GPRM taint: set when the register holds a value derived from a system
/// parameter (SPRM). An SPRM read taints a compare directly ([`is_sprm`]);
/// this bit tracks an SPRM *laundered through* a GPRM store so a later
/// GPRM-only compare that reads it is still recognised as undecidable.
gprm_tainted: [bool; 16],
}
impl Vm {
fn new() -> Self {
Self {
gprm: [0; 16],
sprm: [0; 24],
gprm_tainted: [false; 16],
}
}
/// Whether a register *operand read* is session-specific: an SPRM read is
/// always undecidable at scan time; a GPRM read is undecidable only when the
/// register currently holds an SPRM-derived (tainted) value.
fn reg_tainted(&self, idx: u8) -> bool {
if is_sprm(idx) {
true
} else {
self.gprm_tainted[(idx & 0x0F) as usize]
}
}
/// Mark a GPRM's post-state as unknown. Used when a store to it was guarded
/// by an undecidable (tainted) predicate: whether the write happened is
/// itself unknown, so the destination is conservatively tainted.
fn taint_gprm(&mut self, reg: u8) {
self.gprm_tainted[(reg & 0x0F) as usize] = true;
}
/// Read a register operand. Per the VM command model, indices `< 16` are
/// GPRMs and `>= 128` are SPRMs (0-23 after the `0x80` bias); anything out
/// of range reads as 0 rather than panicking.
fn reg(&self, idx: u8) -> u16 {
if idx >= 128 {
self.sprm.get((idx - 128) as usize).copied().unwrap_or(0)
} else {
self.gprm.get((idx & 0x0F) as usize).copied().unwrap_or(0)
}
}
/// Evaluate a compare predicate (op codes per [`Compare`]). Returns the
/// predicate result together with a *taint* flag that is set when either
/// operand read a system parameter (SPRM). An SPRM's value is
/// player-/session-specific (region, parental level, language) and unknown
/// at scan time, so a branch decision that depends on one is undecidable —
/// the caller abstains rather than commit to the cold-power-on arm. A GPRM
/// read taints only when that register was itself loaded from an SPRM
/// (tracked by [`Vm::gprm_tainted`]); a GPRM holding a program-established
/// value does not taint.
fn eval(&self, c: &Compare) -> (bool, bool) {
let mut tainted = self.reg_tainted(c.lhs_reg);
let l = self.reg(c.lhs_reg);
let r = if c.immediate {
c.imm
} else {
tainted |= self.reg_tainted(c.rhs_reg);
self.reg(c.rhs_reg)
};
let result = match c.op {
1 => (l & r) != 0,
2 => l == r,
3 => l != r,
4 => l >= r,
5 => l > r,
6 => l <= r,
7 => l < r,
_ => false,
};
(result, tainted)
}
/// Apply a `SetGPRM`. Only the arithmetic/logic set-ops that a First-Play
/// routine uses to prepare a dispatch are modelled; unmodelled ops
/// (swap/rnd) leave the register unchanged (best-effort, never panic).
fn set(&mut self, reg: u8, op: u8, immediate: bool, imm: u16, src: u8) {
let idx = (reg & 0x0F) as usize;
let v = if immediate { imm } else { self.reg(src) };
// The value's taint follows its inputs: an immediate is concrete
// (untainted); a register source carries its own taint ([`reg_tainted`]
// covers both a direct SPRM read and a laundered GPRM).
let src_tainted = !immediate && self.reg_tainted(src);
let cur = self.gprm[idx];
let cur_tainted = self.gprm_tainted[idx];
// Each arm sets (new value, new taint). `mov` overwrites with the source's
// taint alone (clearing prior taint); accumulating ops union current and
// source taint. Unmodelled ops leave value and taint intact.
let (nv, nt) = match op {
1 => (v, src_tainted), // mov
3 => (cur.wrapping_add(v), cur_tainted || src_tainted), // add
4 => (cur.wrapping_sub(v), cur_tainted || src_tainted), // sub
5 => (cur.wrapping_mul(v), cur_tainted || src_tainted), // mul
6 => (
cur.checked_div(v).unwrap_or(cur),
cur_tainted || src_tainted,
), // div (÷0 → unchanged)
7 => (
cur.checked_rem(v).unwrap_or(cur),
cur_tainted || src_tainted,
), // mod (÷0 → unchanged)
9 => (cur & v, cur_tainted || src_tainted), // and
10 => (cur | v, cur_tainted || src_tainted), // or
11 => (cur ^ v, cur_tainted || src_tainted), // xor
_ => (cur, cur_tainted), // swap/rnd/unmodelled
};
self.gprm[idx] = nv;
self.gprm_tainted[idx] = nt;
}
}
// ── VMGI structure readers ───────────────────────────────────────────────────
/// Extract the First-Play PGC pre-command list from VMGI bytes. Returns `None`
/// when there is no First-Play PGC, no command table, or the table is out of
/// bounds — all of which mean "nothing to execute" → fall back.
fn fp_pre_commands(vmg: &[u8]) -> Option<Vec<[u8; 8]>> {
let pgc = u32_at(vmg, VMGI_FP_PGC_PTR)? as usize;
if pgc == 0 {
return None; // no First-Play PGC on this disc
}
let cmd_ptr = u16_at(vmg, pgc.checked_add(PGC_CMD_TBL_PTR)?)? as usize;
if cmd_ptr == 0 {
return None; // First-Play PGC carries no command table
}
let tbl = pgc.checked_add(cmd_ptr)?;
// Command table header: nr_of_pre (u16) at +0, then pre-commands at +8.
let nr_pre = (u16_at(vmg, tbl)? as usize).min(MAX_CMDS);
let base = tbl.checked_add(8)?;
let mut cmds = Vec::with_capacity(nr_pre);
for i in 0..nr_pre {
let o = base.checked_add(i.checked_mul(8)?)?;
if o.checked_add(8)? > vmg.len() {
break; // truncated table — execute what parsed
}
let mut c = [0u8; 8];
c.copy_from_slice(&vmg[o..o + 8]);
cmds.push(c);
}
Some(cmds)
}
/// Parse TT_SRPT into a per-title `(vtsn, vts_ttn)` list indexed by 1-based VMG
/// title number. This is the mapping `JumpTT` needs: a title number → the VTS
/// title-set coordinates that identify the corresponding scanned title.
fn tt_srpt_titles(vmg: &[u8]) -> Option<Vec<(u8, u8)>> {
let sector = u32_at(vmg, VMGI_TT_SRPT_PTR)? as usize;
let base = sector.checked_mul(SECTOR_BYTES)?;
let n = (u16_at(vmg, base)? as usize).min(MAX_TT_SRPT_TITLES);
let entries = base.checked_add(8)?;
let mut out = Vec::with_capacity(n);
for i in 0..n {
// Title entries are 12 bytes: VTSN at +6, VTS_TTN at +7.
let e = entries.checked_add(i.checked_mul(12)?)?;
if e.checked_add(12)? > vmg.len() {
break; // truncated — map what parsed
}
out.push((vmg[e + 6], vmg[e + 7]));
}
Some(out)
}
// ── First-Play executor ──────────────────────────────────────────────────────
/// Execute the First-Play pre-command list and return the VMG title number it
/// dispatches to, if it reaches one deterministically. Returns `None` for a
/// program that ends in a menu/sub-domain, links within the menu domain, exits,
/// or does not converge inside the step budget — every case the caller falls
/// back on.
fn run_first_play(cmds: &[[u8; 8]]) -> Option<u8> {
let mut vm = Vm::new();
let mut pc = 0usize;
let mut budget = STEP_BUDGET;
while pc < cmds.len() {
if budget == 0 {
return None; // non-convergence guard
}
budget -= 1;
let cmd = vmcmd::decode(&cmds[pc]);
// A guarded command's predicate decides whether its instruction runs. `taken`
// is the predicate result; `tainted` marks it undecidable (read a system
// parameter, directly or via GPRM). No predicate = always runs, never tainted.
let (taken, tainted) = match cmd.compare {
Some(cmp) => vm.eval(&cmp),
None => (true, false),
};
// A decidable false predicate skips this line — no effect, taint
// unchanged. When the predicate is tainted we cannot decide `taken`, so
// we do not skip; the per-instruction handling below is conservative.
if !tainted && !taken {
pc += 1;
continue;
}
match cmd.instr {
// Title dispatch — a control transfer, the answer we are looking for.
// Abstain only when an undecidable predicate gates it: committing
// would pick the cold-power-on arm of an SPRM-dependent decision.
Instr::JumpTt { ttn } => {
if tainted {
return None;
}
return Some(ttn);
}
// Control flow within the pre list (Goto lines are 1-based) — also a
// control transfer, so an undecidable guard abstains.
Instr::Goto { line } => {
if tainted {
return None;
}
let l = line as usize;
if l == 0 || l > cmds.len() {
return None;
}
pc = l - 1;
continue;
}
// Register prep before dispatch is a store, not a control transfer, so a
// tainted guard does NOT abstain. Undecidable guard -> unknown post-state
// -> taint the destination; otherwise apply the store normally.
Instr::SetGprm {
reg,
op,
immediate,
imm,
src,
} => {
if tainted {
vm.taint_gprm(reg);
} else {
vm.set(reg, op, immediate, imm, src);
}
}
// No-effect (for resolution) instructions: keep executing. SetSystem
// writes an SPRM (never a GPRM in this decoder), and SPRM reads are
// caught at read time, so no GPRM taint is needed here.
Instr::Nop | Instr::SetSystem => {}
// Everything else leaves the deterministically-followable path: Break/Exit
// end the pre list with no title; JumpSS/Link land in a menu or depend on
// an interactive button selection that can't be resolved statically.
_ => return None,
}
pc += 1;
}
None
}
// ── Public entry points ──────────────────────────────────────────────────────
/// Resolve the main-feature title from raw VMGI (`VIDEO_TS.IFO`) bytes.
///
/// Pure and total: any malformed/hostile input returns `None`, never panics.
/// This is the harness entry point.
pub fn resolve_from_vmg(vmg: &[u8]) -> Option<ResolvedTitle> {
if vmg.len() < 0xC8 || &vmg[0..12] != VMGI_MAGIC {
return None;
}
let cmds = fp_pre_commands(vmg)?;
let ttn = run_first_play(&cmds)?;
if ttn == 0 {
return None; // title 0 is not addressable
}
let titles = tt_srpt_titles(vmg)?;
let (vtsn, vts_ttn) = titles.get((ttn as usize) - 1).copied()?;
if vtsn == 0 {
return None; // invalid TT_SRPT entry
}
Some(ResolvedTitle {
title: ttn,
vtsn,
vts_ttn,
})
}
/// Resolve the main-feature title by reading the disc's VMGI and following its
/// First-Play navigation. Returns `None` (→ caller keeps the heuristic) when the
/// IFO cannot be read or the navigation does not deterministically reach a
/// title.
pub fn resolve_main_title(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ResolvedTitle> {
let vmg = udf.read_file(reader, "/VIDEO_TS/VIDEO_TS.IFO").ok()?;
let resolved = resolve_from_vmg(&vmg)?;
tracing::debug!(
target: "freemkv::dvdnav",
title = resolved.title,
vtsn = resolved.vtsn,
vts_ttn = resolved.vts_ttn,
"nav resolved main-feature title from First-Play program"
);
Some(resolved)
}
#[cfg(test)]
mod tests {
use super::*;
// ── Fixture builders ────────────────────────────────────────────────────
/// Assemble a VMGI (`VIDEO_TS.IFO`) image with a First-Play PGC whose
/// pre-command list is `pre`, and a TT_SRPT at sector `tt_srpt_sector`
/// mapping each title to `(vtsn, vts_ttn)`.
///
/// Layout: the fixed VMGI header (magic + the FP_PGC and TT_SRPT pointers),
/// then the FP_PGC (with a command table holding `pre`), then the TT_SRPT at
/// its sector. Kept minimal but spec-shaped so the same code path a real
/// disc takes is exercised.
fn build_vmgi(pre: &[[u8; 8]], tt_srpt_sector: u32, titles: &[(u8, u8)]) -> Vec<u8> {
// Place the FP_PGC at a fixed byte offset past the header.
let fp_pgc_off: u32 = 0x400;
// Command table sits 0x100 bytes into the PGC.
let cmd_tbl_rel: u16 = 0x100;
let mut v = vec![0u8; 0x800];
v[0..12].copy_from_slice(VMGI_MAGIC);
v[VMGI_FP_PGC_PTR..VMGI_FP_PGC_PTR + 4].copy_from_slice(&fp_pgc_off.to_be_bytes());
v[VMGI_TT_SRPT_PTR..VMGI_TT_SRPT_PTR + 4].copy_from_slice(&tt_srpt_sector.to_be_bytes());
// FP_PGC: command-table pointer at +0xE4.
let pgc = fp_pgc_off as usize;
v[pgc + PGC_CMD_TBL_PTR..pgc + PGC_CMD_TBL_PTR + 2]
.copy_from_slice(&cmd_tbl_rel.to_be_bytes());
// Command table: nr_of_pre at +0, pre-commands from +8.
let tbl = pgc + cmd_tbl_rel as usize;
v[tbl..tbl + 2].copy_from_slice(&(pre.len() as u16).to_be_bytes());
for (i, c) in pre.iter().enumerate() {
let o = tbl + 8 + i * 8;
v[o..o + 8].copy_from_slice(c);
}
// TT_SRPT at its sector: count at +0, 12-byte entries from +8.
let base = tt_srpt_sector as usize * SECTOR_BYTES;
let need = base + 8 + titles.len() * 12;
if v.len() < need {
v.resize(need, 0);
}
v[base..base + 2].copy_from_slice(&(titles.len() as u16).to_be_bytes());
for (i, &(vtsn, vts_ttn)) in titles.iter().enumerate() {
let e = base + 8 + i * 12;
v[e + 6] = vtsn;
v[e + 7] = vts_ttn;
}
v
}
fn h(s: &str) -> [u8; 8] {
let v: Vec<u8> = (0..8)
.map(|i| u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).unwrap())
.collect();
v.try_into().unwrap()
}
// ── Executor: convergence to a title ────────────────────────────────────
/// Unconditional-dispatch shape: First-Play is an unconditional `JumpTT 1`,
/// and TT_SRPT maps title 1 to the feature title set (here VTS_02, title 1).
/// The resolver must return that title.
#[test]
fn first_play_jumptt_1_resolves_to_its_title_set() {
// 3002...0001 = JumpTT ttn=1.
let vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1), (3, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// A First-Play that sets up a register and then unconditionally dispatches
/// still resolves (the SetGPRM is a no-op for an unconditional JumpTT).
#[test]
fn setgprm_then_jumptt_resolves() {
let vmgi = build_vmgi(
&[h("7100000603e80000"), h("3002000000020000")],
1,
&[(1, 1), (4, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 2,
vtsn: 4,
vts_ttn: 1
})
);
}
/// A conditional dispatch whose predicate is true on the cold (zero)
/// machine is taken. `if g0 == 0 -> JumpTT 1` with g0 defaulting to 0.
#[test]
fn conditional_jumptt_true_on_cold_machine_is_taken() {
// 30 22: jump with EQ compare (op=2), operands are registers b6/b7 both
// 0 (g0 == g0) → true; JumpTT ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000010000")], 1, &[(5, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 5,
vts_ttn: 1
})
);
}
/// A conditional dispatch that is false falls through to the next line's
/// unconditional dispatch.
#[test]
fn false_conditional_falls_through_to_next_dispatch() {
// line 1: 30 32 with GT compare (op=3 = !=) g0(b6=1?) ... build a clearly
// false compare: if g0 != g0 -> JumpTT 9 (false, skipped);
// line 2: unconditional JumpTT 1.
let vmgi = build_vmgi(
&[h("3032000000090000"), h("3002000000010000")],
1,
&[(7, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 7,
vts_ttn: 1
})
);
}
// ── Executor: abstention (→ caller falls back) ──────────────────────────
/// Menu-entry shape: First-Play jumps into a VTS menu (`JumpSS VTSM root`).
/// That is an interactive menu, not a static dispatch — the resolver must
/// abstain so the caller keeps the leading-cell heuristic.
#[test]
fn first_play_into_menu_abstains() {
// 3006...0183 = JumpSS VTSM root.
let vmgi = build_vmgi(&[h("3006000101830000")], 1, &[(3, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A dispatch gated on a system parameter (SPRM) is undecidable at scan
/// time — the resolver must abstain rather than commit to the cold-power-on
/// arm. Here `if SPRM0 == g0 -> JumpTT 1` is *true* on the cold machine
/// (both read 0), so without SPRM taint-tracking the resolver would wrongly
/// return title 1; with it, the SPRM read taints the branch and it abstains.
#[test]
fn sprm_gated_dispatch_abstains() {
// 30 22: jump with EQ compare (op=2); if_v2 operands are registers
// b6/b7. b6=0x80 = SPRM0 (session-specific), b7=0x00 = g0. ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000018000")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// The mirror of the SPRM case: a compare that reads only GPRMs is decidable
/// and must NOT over-abstain. `if g0 == g1 -> JumpTT 1` is true on the cold
/// machine and resolves to title 1 (no SPRM operand → no taint).
#[test]
fn gprm_only_compare_does_not_taint() {
// 30 22: jump EQ; b6=0x00 = g0, b7=0x01 = g1 (both 0 on the cold
// machine → equal). ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000010001")], 1, &[(6, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 6,
vts_ttn: 1
})
);
}
/// Defect-1 regression (over-abstention): a store guarded by an SPRM-tainted
/// compare does NOT gate a control transfer, so it must not abstain. Here
/// `if SPRM0 == g0 -> g0 = 5` (a guarded store) is followed by an
/// UNCONDITIONAL `JumpTT 1`. The tainted predicate only decides whether the
/// store runs; the dispatch that follows is SPRM-independent, so the resolver
/// must resolve to title 1.
#[test]
fn sprm_guarded_store_then_unconditional_dispatch_resolves() {
// line 0: 71 20 | lhs=b2=0x80 (SPRM0), cmp EQ(op=2, register), rhs=b7=0
// (g0) → tainted guard; SetGPRM g0 = imm(bytes4-5)=5 (mov).
// line 1: unconditional JumpTT 1.
let vmgi = build_vmgi(
&[h("7120800000050000"), h("3002000000010000")],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// Defect-2 regression (under-abstention): an SPRM laundered through a GPRM
/// must still taint a later branch. `SetGPRM g0 = SPRM20` (register source,
/// index >= 128) followed by `if g0 == g1 -> JumpTT 1` — the compare reads
/// only GPRMs, but g0 now holds an SPRM-derived value, so the decision is
/// undecidable and the resolver must abstain.
#[test]
fn sprm_laundered_through_gprm_abstains() {
// line 0: 61 00 | SetGPRM g0 = SPRM20 (mov, register src=b5=0x94).
// line 1: 30 22 | if g0 == g1 -> JumpTT 1 (both 0 on cold machine).
let vmgi = build_vmgi(
&[h("6100000000940000"), h("3022000000010001")],
1,
&[(2, 1)],
);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// Guard: an unconditional immediate store CLEARS a prior taint (no sticky
/// latch). Taint g0 from an SPRM, overwrite it with an immediate, then branch
/// on g0 — the branch is now decidable and must resolve, not abstain.
#[test]
fn unconditional_immediate_store_clears_taint() {
// line 0: SetGPRM g0 = SPRM20 (taints g0).
// line 1: SetGPRM g0 = imm 0 (mov, immediate) → clears taint, g0 = 0.
// line 2: if g0 == g1 -> JumpTT 1 (both 0) → resolves.
let vmgi = build_vmgi(
&[
h("6100000000940000"),
h("7100000000000000"),
h("3022000000010001"),
],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// Guard: copying an UNtainted GPRM does not taint the destination. `g0 = g1`
/// (register source, g1 untainted) then `if g0 == g2 -> JumpTT 1` must
/// resolve (no SPRM ever entered g0).
#[test]
fn gprm_copy_of_untainted_gprm_does_not_taint() {
// line 0: 61 00 | SetGPRM g0 = g1 (mov, register src=b5=1).
// line 1: 30 22 | if g0 == g2 -> JumpTT 1 (both 0) → resolves.
let vmgi = build_vmgi(
&[h("6100000000010000"), h("3022000000010002")],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// An empty First-Play pre list selects no title.
#[test]
fn empty_first_play_abstains() {
let vmgi = build_vmgi(&[], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A JumpTT whose title number is past the TT_SRPT table resolves to no
/// coordinates (→ abstain) rather than indexing out of bounds.
#[test]
fn jumptt_past_tt_srpt_abstains() {
// JumpTT ttn=9 but TT_SRPT declares only 1 title.
let vmgi = build_vmgi(&[h("3002000000090000")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A self-referential `Goto` cannot spin forever — the step budget stops it
/// and the resolver abstains.
#[test]
fn self_goto_hits_budget_and_abstains() {
// Goto line 1 (0-special sub 1), byte7 = 01.
let vmgi = build_vmgi(&[h("0001000000000001")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
// ── Robustness ──────────────────────────────────────────────────────────
#[test]
fn bad_magic_abstains() {
let mut vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
vmgi[0] = b'X';
assert_eq!(resolve_from_vmg(&vmgi), None);
}
#[test]
fn truncated_and_short_inputs_never_panic() {
// Below the header minimum, and a valid header truncated at every length.
assert_eq!(resolve_from_vmg(&[]), None);
assert_eq!(resolve_from_vmg(VMGI_MAGIC), None);
let full = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
for len in 0..full.len() {
let _ = resolve_from_vmg(&full[..len]);
}
}
/// A TT_SRPT pointer that would overflow when scaled to a byte offset must
/// be rejected, not wrap. Build a normal fixture, then overwrite just the
/// TT_SRPT sector pointer with the extreme value (building a real table at
/// that sector would allocate terabytes in the fixture, not the resolver).
#[test]
fn overflowing_tt_srpt_sector_abstains() {
let mut vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
vmgi[VMGI_TT_SRPT_PTR..VMGI_TT_SRPT_PTR + 4].copy_from_slice(&u32::MAX.to_be_bytes());
assert_eq!(resolve_from_vmg(&vmgi), None);
}
}
+35 -22
View File
@@ -159,13 +159,9 @@ fn be16(b: &[u8; 8], o: usize) -> u16 {
((b[o] as u16) << 8) | b[o + 1] as u16
}
// Compare-operand layouts ("if_version"s) per the DVD-Video VM. The op
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
// v1 (special + link): lhs reg = b[3]; rhs imm = bytes4-5 / rhs reg = b[4].
// v2 (jump + system-set): lhs reg = b[6]; rhs reg = b[7] (registers only).
// v3 (set-GPRM): lhs reg = b[2]; rhs imm = bytes6-7 / rhs reg = b[6].
// Compare-operand layouts ("if_version"s): op nibble = byte1 bits 6-4, imm
// flag = byte1 bit 7; offsets differ by family: v1(special/link) lhs=b[3],
// rhs=b[4:6]/b[5]; v2(jump/sys-set) lhs=b[6],rhs=b[7]; v3(set-GPRM) lhs=b[2],rhs=b[6:8]/b[7].
fn if_v1(b: &[u8; 8]) -> Option<Compare> {
let op = (b[1] >> 4) & 7;
(op != 0).then(|| Compare {
@@ -173,7 +169,7 @@ fn if_v1(b: &[u8; 8]) -> Option<Compare> {
lhs_reg: b[3],
immediate: b[1] >> 7 != 0,
imm: be16(b, 4),
rhs_reg: b[4],
rhs_reg: b[5],
})
}
fn if_v2(b: &[u8; 8]) -> Option<Compare> {
@@ -193,7 +189,7 @@ fn if_v3(b: &[u8; 8]) -> Option<Compare> {
lhs_reg: b[2],
immediate: b[1] >> 7 != 0,
imm: be16(b, 6),
rhs_reg: b[6],
rhs_reg: b[7],
})
}
@@ -295,16 +291,16 @@ mod tests {
v.try_into().unwrap()
}
// KATs taken from the real SOTL / Greenland discs (decoded in the PoC).
// KATs taken from real-world First-Play command streams.
#[test]
fn greenland_first_play_is_jumptt_1() {
fn first_play_jumptt_1_decodes() {
let c = decode(&h("3002000000010000"));
assert_eq!(c.instr, Instr::JumpTt { ttn: 1 });
assert!(c.compare.is_none());
}
#[test]
fn sotl_first_play_is_jumpss_vtsm_root() {
fn first_play_jumpss_vtsm_root_decodes() {
// 30 06 ... byte5=0x83 -> sub 2 (VTSM), vts=byte4=1, menu=byte5&0xF=3 (root)
let c = decode(&h("3006000101830000"));
assert_eq!(
@@ -318,8 +314,9 @@ mod tests {
}
#[test]
fn sotl_title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (cell 2 = the 5:02 start)
fn title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (the cell where the
// feature begins).
let c = decode(&h("20a6000000020002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
@@ -330,7 +327,7 @@ mod tests {
}
#[test]
fn sotl_root_button_is_linkpgcn_37() {
fn root_button_is_linkpgcn_37() {
assert_eq!(
decode(&h("2004000000000025")).instr,
Instr::LinkPgcn { pgcn: 37 }
@@ -338,7 +335,7 @@ mod tests {
}
#[test]
fn greenland_scene_button_is_linkpgn() {
fn scene_button_is_linkpgn() {
assert_eq!(
decode(&h("2006000000001401")).instr,
Instr::LinkPgn { pgn: 1 }
@@ -354,7 +351,7 @@ mod tests {
#[test]
fn setgprm_immediate_mov() {
// SOTL First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
// First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
match decode(&h("7100000603e80000")).instr {
Instr::SetGprm {
reg,
@@ -382,16 +379,32 @@ mod tests {
);
}
// if_version_1 register compare: rhs register is byte4 (not byte5).
// if_version_1 register compare: the register operand is the LOW byte of the
// reg-or-immediate field (byte5), NOT the high byte (byte4). Distinct values
// in byte4 (0xAA) vs byte5 (0x05) pin the field down.
#[test]
fn link_register_compare_rhs_is_byte4() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN
let c = decode(&h("2026000304000002"));
fn link_register_compare_rhs_is_low_byte() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN (pgn=byte7=2)
let c = decode(&h("20260003aa050002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 3);
assert_eq!(cmp.rhs_reg, 4);
assert_eq!(cmp.rhs_reg, 5); // low byte of bytes4-5, not the 0xAA high byte
}
// if_version_3 (set-GPRM) register compare: the register operand is the LOW
// byte of the reg-or-immediate field (byte7), NOT the high byte (byte6).
// Distinct values in byte6 (0xBB) vs byte7 (0x07) pin the field down.
#[test]
fn setgprm_register_compare_rhs_is_low_byte() {
// 71 20: set-GPRM (type 3), cmp=EQ(2), dircmp=0(register); lhs=byte2=2.
let c = decode(&h("712002000000bb07"));
let cmp = c.compare.expect("conditional");
assert_eq!(cmp.op, 2);
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 2);
assert_eq!(cmp.rhs_reg, 7); // low byte of bytes6-7, not the 0xBB high byte
}
// if_version_2 jump compare: both operands are registers in byte6 / byte7.
+28 -56
View File
@@ -379,7 +379,7 @@ pub enum Error {
/// fields):
/// - [`Error::is_scsi_transport_failure`] — bail; bridge/transport wedge
/// - [`Error::is_marginal_read`] — drive said this read was marginal; smaller block may recover
/// - [`Error::scsi_sense`] — borrow the sense triple for finer routing ([`ScsiSense::is_medium_error`] etc.)
/// - [`Error::scsi_sense`] — borrow the sense triple for finer routing ([`ScsiSense::is_medium_error`](crate::scsi::ScsiSense::is_medium_error) etc.)
ScsiError {
opcode: u8,
status: u8,
@@ -1201,16 +1201,9 @@ impl std::error::Error for Error {
impl From<std::io::Error> for Error {
fn from(e: std::io::Error) -> Self {
// If this `io::Error` is one WE produced (`From<Error> for io::Error`
// carries the typed value in its boxed payload), give the original
// back instead of burying it in `Error::IoError`. That wrapper is not
// neutral: `is_scsi_transport_failure` treats `IoError` as a
// transport-layer fault (dead bus / wedged bridge), so re-wrapping a
// round-tripped `DiscRead` MEDIUM ERROR turned a skippable bad sector
// into a pass-aborting bridge wedge — the exact inverse of what that
// arm exists for. Any error that crosses a thread boundary as an
// `io::Error` (the prefetch channel's `Batch`) keeps its
// classification, its SCSI status, and its sense data.
// If this `io::Error` is ours, unwrap to the original typed `Error` rather
// than re-wrapping as `IoError` (which `is_scsi_transport_failure` reads as
// a bridge wedge) — preserves classification, SCSI status, and sense data.
match e.downcast::<Error>() {
Ok(typed) => typed,
// A genuine OS/`std` error — the `IoError` wrapper is correct here.
@@ -1221,11 +1214,9 @@ impl From<std::io::Error> for Error {
impl From<Error> for std::io::Error {
fn from(e: Error) -> Self {
// An `Error::IoError` is just a wrapper around an underlying
// `io::Error` that entered via `From<io::Error> for Error`.
// Round-trip it back unchanged so the original `ErrorKind` and
// raw OS error code survive instead of being flattened to
// `Other` with a stringified message.
// `Error::IoError` just wraps an `io::Error` that entered via
// `From<io::Error> for Error` — round-trip it back unchanged so the
// original `ErrorKind`/OS code survive instead of flattening to `Other`.
if let Error::IoError { source } = e {
return source;
}
@@ -1312,12 +1303,9 @@ impl From<Error> for std::io::Error {
// 9027 insufficient space / 9029 write failed: a filesystem-level
// failure, not bad input.
E_DIR_INSUFFICIENT_SPACE | E_DIR_WRITE_FAILED => std::io::ErrorKind::Other,
// dir:// SOURCE gates (9061–9064, 9066–9068): the folder handed in
// cannot be turned into a disc image — a 3D SSIF tree, an
// unsatisfiable VIDEO_TS placement, still-encrypted content, an
// unrecognized tree, one too large to address, a name too long to
// record, or a directory whose fan-out overflows its link count.
// All are properties of the input, decided before a byte is read.
// dir:// SOURCE gates (9061-9064, 9066-9068): the folder can't become a
// disc image (3D SSIF, bad VIDEO_TS placement, encrypted, unrecognized
// tree, too large, name too long, fan-out overflow) — input properties.
E_DIR_IMAGE_SSIF_UNSUPPORTED
| E_DIR_IMAGE_PLACEMENT
| E_DIR_IMAGE_ENCRYPTED
@@ -1330,12 +1318,9 @@ impl From<Error> for std::io::Error {
E_DIR_IMAGE_FILE_CHANGED => std::io::ErrorKind::InvalidData,
_ => std::io::ErrorKind::Other,
};
// Carry the typed value itself as the payload rather than its
// stringification. `Display` is unchanged (`io::Error` delegates to the
// boxed error, whose `Display` is the same `E<code>[: …]` string), so
// `error_code` and every consumer built on it are unaffected — but the
// typed error now SURVIVES the conversion and `From<io::Error> for
// Error` can hand it back intact.
// Carry the typed value itself as the payload, not its stringification.
// `Display` is unchanged (same `E<code>[: …]` string), so `error_code` is
// unaffected — but the typed error now SURVIVES and can round-trip back.
std::io::Error::new(kind, e)
}
}
@@ -1480,12 +1465,9 @@ impl Error {
..
}
) || matches!(
// A failed `ioctl(SG_IO)` (Error::IoError, e.g. ENODEV/EIO on an
// unplugged USB bridge) and a vanished device (Error::DeviceNotFound,
// fd gone) are dead-bus / transport-layer faults too — NOT recoverable
// bad sectors. Treat them as transport failures so sweep / patch /
// fill_extents abort the pass and re-enumerate the bridge instead of
// zero-filling every read against a wedged device.
// A failed ioctl (IoError, e.g. ENODEV/EIO) or vanished device
// (DeviceNotFound) is a dead-bus fault too, not a bad sector — treat
// as transport failure so the pass aborts instead of zero-filling.
self,
Error::IoError { .. } | Error::DeviceNotFound { .. }
)
@@ -1559,24 +1541,17 @@ mod tests {
#[test]
fn is_skippable_title_stub_excludes_malformed_mkv_input() {
// The two skippable per-title codes, round-tripped through io::Error
// exactly as `mux_stream` returns them. `MkvInvalid` now means ONLY the
// no-muxable-frames stub (the driver's headers-never-resolved gate and the
// MKV muxer's zero-frame `finish()` guard) — the meaning the doc on this
// predicate has always described.
// The two skippable per-title codes, round-tripped through io::Error as
// `mux_stream` returns them. `MkvInvalid` now means ONLY the no-muxable-
// frames stub, matching this predicate's documented meaning.
let mkv: std::io::Error = Error::MkvInvalid.into();
let css: std::io::Error = Error::CssKeyMissing.into();
assert!(is_skippable_title_stub(&mkv));
assert!(is_skippable_title_stub(&css));
// A malformed / truncated `mkv://` SOURCE is a FAILURE, not a stub. Every
// read-path rejection in `mux::mkvstream` and `mux::ebml`'s read
// primitives used to be raised as `MkvInvalid`, so a bad VINT, a cluster
// timestamp past i64::MAX or a BlockGroup child overrunning its group all
// landed in the skippable set above: an all-titles rip would pass silently
// over a corrupt input and exit reporting success. Reverting
// `Error::MkvSourceInvalid` back to `Error::MkvInvalid` at those raise
// sites turns this assertion red.
// A malformed/truncated mkv:// SOURCE is a FAILURE, not a stub. It used to
// be raised as `MkvInvalid`, so a corrupt source landed in the skippable
// set and an all-titles rip silently passed over it, exiting successfully.
let corrupt: std::io::Error = Error::MkvSourceInvalid.into();
assert!(
!is_skippable_title_stub(&corrupt),
@@ -1599,11 +1574,9 @@ mod tests {
let plain = std::io::Error::from(std::io::ErrorKind::BrokenPipe);
assert!(!is_skippable_title_stub(&plain));
// A header-buffer cap overflow is a REAL title whose codec init data
// never resolved — hundreds of MiB of frames, not an empty nav/menu
// stub. It used to be reported as `MkvInvalid`, which lands in the
// skippable set above, so an all-titles rip dropped a main feature and
// still exited successfully. It must have its own, non-skippable code.
// A header-buffer cap overflow is a REAL title whose codec init data never
// resolved, not an empty stub. Reported as `MkvInvalid` it would land in
// the skippable set, so an all-titles rip would drop a main feature silently.
let cap: std::io::Error = Error::MuxHeaderBufferExceeded {
bytes: 512 * 1024 * 1024 + 1,
}
@@ -1780,10 +1753,9 @@ mod tests {
e,
s
);
// Crude English filter — `Display` should never emit ASCII words
// longer than 4 chars (codes/paths/identifiers like `/dev/sg4`,
// `renesas`, `freebsd` all pass; "exclusive access denied" would
// not).
// Crude English filter — `Display` should never emit ASCII words longer
// than 4 chars (identifiers like `/dev/sg4`, `renesas` pass;
// "exclusive access denied" would not).
for word in s.split(|c: char| !c.is_ascii_alphabetic()) {
assert!(
word.len() <= 8,
+115 -9
View File
@@ -136,11 +136,9 @@ fn sweep_n<F: FnMut(&[u8])>(target: &str, magic: &[u8], n: usize, mut f: F) {
run(target, "magic+noise", s, i, &buf, &mut f);
}
// 3. Structured mutation of a plausible record: a valid magic, then mostly
// zeroes, with a handful of bytes corrupted and a truncation. Length and
// offset fields live in those early bytes, so this is what reaches the
// arithmetic — the offsets, counts and sizes a hostile image would lie
// about.
// 3. Structured mutation: valid magic, mostly zeroes, a handful of bytes
// corrupted plus truncation. Length/offset fields live in those early
// bytes, so this reaches the arithmetic a hostile image would lie about.
let mut rng = Rng::new(s ^ 0x1234_5678_9ABC_DEF0);
for i in 0..n {
let mut buf = vec![0u8; 512];
@@ -180,6 +178,30 @@ fn clpi_parse_never_panics() {
});
}
#[test]
fn bdnav_index_parse_never_panics() {
sweep("bdnav_index", b"INDX", |b| {
let _ = crate::bdnav::index::parse(b);
});
}
#[test]
fn bdnav_mobj_parse_never_panics() {
sweep("bdnav_mobj", b"MOBJ", |b| {
let _ = crate::bdnav::mobj::parse(b);
});
}
#[test]
fn dvd_nav_resolve_never_panics() {
// The DVD First-Play resolver parses VMGI (FP_PGC + TT_SRPT, all
// attacker-controlled) and executes an untrusted command list on a bounded
// VM. Magic is the VMGI signature, so the sweep reaches the executor.
sweep("dvd_nav", b"DVDVIDEO-VMG", |b| {
let _ = crate::dvdnav::nav::resolve_from_vmg(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
@@ -189,12 +211,96 @@ fn udf_name_parse_never_panics() {
});
}
#[test]
fn class_reader_parse_never_panics() {
// Java `.class` files are READ (never executed) to recover BD-J stream
// labels. Every count/length/constant-pool index is attacker-controlled
// disc metadata, so the parser must reject malformed input, never panic.
sweep("class_reader", &[0xCA, 0xFE, 0xBA, 0xBE], |b| {
if let Ok(class) = crate::labels::class_reader::ClassFile::parse(b) {
// Exercise the accessors a label parser drives, so their indexing
// is fuzzed too, not just the structural parse.
let _ = class.this_class_name();
let _ = class.super_class_name();
for m in &class.methods {
let _ = class.member_name(m);
if let Some(code) = m.code(&class.constant_pool) {
for insn in code.instructions() {
let _ = insn.cp_index();
if let Some(idx) = insn.cp_index() {
let _ = class.constant_pool.get(idx);
let _ = class.constant_pool.member_ref(idx);
let _ = class.constant_pool.load_constant_display(idx);
}
}
}
}
}
});
}
#[test]
fn jar_class_walk_never_panics() {
// The jar layer opens a `/BDMV/JAR/<x>.jar` zip and walks every `.class`
// inside, handing each to the class reader (as `deluxe`/`dbp` do). Neither
// the zip open nor the class walk may panic. Magic: `PK\x03\x04`.
sweep("jar", &[0x50, 0x4B, 0x03, 0x04], |b| {
if let Ok(mut archive) = zip::ZipArchive::new(std::io::Cursor::new(b.to_vec())) {
crate::labels::jar::for_each_class(&mut archive, |_name, _class| {});
}
});
}
#[test]
fn deluxe_bytecode_decode_never_panics() {
// Deluxe's BD-J parser runs a symbolic stack machine over `<clinit>`
// bytecode. Fuzzing THROUGH a zip is vacuous (generators can't fake a valid
// end-of-central-directory), so drive the decoder DIRECTLY on a `.class`.
let master = crate::labels::deluxe::MasterEnumTable::from(&[] as &[(
&'static str,
crate::labels::deluxe::MasterEnum,
)]);
sweep("deluxe_decode", &[0xCA, 0xFE, 0xBA, 0xBE], |b| {
if let Ok(class) = crate::labels::class_reader::ClassFile::parse(b) {
let _ = crate::labels::deluxe::decode_binding_class(&class, &master);
}
});
}
#[test]
fn fox_dcx_labels_never_panics() {
// Fox's `dcx.xml` is untrusted disc-authored XML; `labels_from_dcx` scans it
// with byte-offset XML helpers documented as UTF-8-boundary-prone. No magic —
// feed lossy UTF-8 so multi-byte boundaries inside tags/attributes are hit.
sweep("fox_dcx", b"<", |b| {
let text = String::from_utf8_lossy(b);
let _ = crate::labels::fox::labels_from_dcx(&text);
});
}
#[test]
fn bdnav_vm_resolve_never_panics() {
// The HDMV nav VM executes an untrusted MovieObject.bdmv command list.
// Parse sweep bytes as a MovieObject set and run the VM from First-Play —
// the combined parse+execute path the per-parser index/mobj sweeps miss.
use crate::bdnav::index::{Index, PlaybackObj};
sweep("bdnav_vm", b"MOBJ", |b| {
if let Some(mobjs) = crate::bdnav::mobj::parse(b) {
let index = Index {
first_play: PlaybackObj::Hdmv { id_ref: 0 },
top_menu: PlaybackObj::Hdmv { id_ref: 0 },
titles: Vec::new(),
};
let _ = crate::bdnav::vm::resolve(&index, &mobjs, &|_| false);
}
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
// each case is fed to a FRESH demuxer and then a shared one. The shared pass
// is what exercises cross-feed state — a start code split over a boundary,
// a held PES completed by later bytes, the carry-over cap.
// Stateful, unlike the others: the demuxer carries a buffer across feeds.
// Each case is fed to a FRESH demuxer, then a shared one — the shared pass
// exercises cross-feed state: split start codes, held PES, the carry cap.
let mut shared = crate::mux::ps::PsDemuxer::new();
sweep("ps_demux", &[0x00, 0x00, 0x01, 0xBA], |b| {
let mut fresh = crate::mux::ps::PsDemuxer::new();
+13 -23
View File
@@ -17,19 +17,19 @@ use crate::scsi::{DataDirection, ScsiTransport};
/// - MMC-6 §5.3.10 for Firmware Information
#[derive(Debug, Clone)]
pub struct DriveId {
/// T10 VENDOR IDENTIFICATION — INQUIRY bytes [8:16]
/// T10 VENDOR IDENTIFICATION — INQUIRY bytes `[8:16]`
/// SPC-4 §6.4.2
pub vendor_id: String,
/// PRODUCT IDENTIFICATION — INQUIRY bytes [16:32]
/// PRODUCT IDENTIFICATION — INQUIRY bytes `[16:32]`
/// SPC-4 §6.4.2
pub product_id: String,
/// PRODUCT REVISION LEVEL — INQUIRY bytes [32:36]
/// PRODUCT REVISION LEVEL — INQUIRY bytes `[32:36]`
/// SPC-4 §6.4.2
pub product_revision: String,
/// VENDOR SPECIFIC — INQUIRY bytes [36:43]
/// VENDOR SPECIFIC — INQUIRY bytes `[36:43]`
/// SPC-4 §6.4.2
/// Content varies by vendor: firmware type code (MTK), date (Pioneer), etc.
pub vendor_specific: String,
@@ -60,26 +60,18 @@ impl DriveId {
let mut inquiry = vec![0u8; 96];
let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00];
let inq = transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
// `bytes_transferred` is device-reported and untrusted — the same rule
// the two GET CONFIGURATION calls below already apply. It was ignored
// here, and the buffer is pre-zeroed, so a drive answering GOOD with a
// short or empty data phase (a USB-SATA bridge mid-wedge does exactly
// this) decoded to blank identity strings and a byte 0 of 0x00. Every
// platform enumerator gates on `raw_inquiry[0] & 0x1F == OPTICAL`, so
// 0x00 reads as DIRECT ACCESS and the drive silently disappears from
// the device list instead of reporting a failed probe.
// `bytes_transferred` is device-reported and untrusted. Unchecked, a
// GOOD status with a short/empty data phase decoded to blank identity +
// byte0 0x00 — which reads as DIRECT ACCESS, so the drive vanishes silently.
if inq.bytes_transferred < INQUIRY_STANDARD_LEN {
return Err(crate::error::Error::DriveInquiryShort);
}
// Never decode past what the drive actually sent.
inquiry.truncate(inq.bytes_transferred.min(inquiry.len()));
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6.
// Best-effort: 010Ch (Firmware Information) is an optional feature.
// A drive that lacks it may CHECK CONDITION rather than return an
// empty descriptor, so a failure here is treated as feature-absent
// (empty firmware date + empty raw bytes) instead of aborting the
// whole identity probe.
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6. Best-effort: an optional
// feature a drive may lack (CHECK CONDITION), so failure here is
// feature-absent (empty firmware date + raw bytes), not a probe abort.
let mut gc = vec![0u8; 256];
let cdb_gc = [0x46, 0x02, 0x01, 0x0C, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00];
// `bytes_transferred` is device-reported and untrusted; clamp every
@@ -100,11 +92,9 @@ impl DriveId {
Err(_) => (String::new(), Vec::new()),
};
// GET CONFIGURATION Feature 0108h — Serial Number.
// Best-effort, like 010Ch above: the serial-number feature is
// optional, so a drive that lacks it (CHECK CONDITION) or reports
// too few bytes deliberately yields an empty serial rather than
// failing the identity probe.
// GET CONFIGURATION Feature 0108h — Serial Number. Best-effort like
// 010Ch: optional feature, so lacking it (CHECK CONDITION) or too few
// bytes deliberately yields an empty serial rather than failing.
let mut gc_serial = vec![0u8; 256];
let cdb_serial = [0x46, 0x02, 0x01, 0x08, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00];
let serial_number = if let Ok(r) =
+66 -148
View File
@@ -149,8 +149,8 @@ impl DvdTitle {
/// - As a final guard it never returns past the last cell, and never drops
/// when that would leave zero cells.
///
/// For "The Silence of the Lambs" (every feature cell category `0x00`,
/// chapter 1 at 00:00:00) this returns 0 — a no-op — which is the correct
/// For a real disc where every feature cell category is `0x00` and
/// chapter 1 sits at 00:00:00, this returns 0 — a no-op — which is the correct
/// result: the disc's scene-index lives in a separate menu/title PGC, not
/// in leading cells of the feature PGC, so there is nothing to drop here.
pub fn feature_start_cell(&self) -> usize {
@@ -173,7 +173,7 @@ impl DvdTitle {
if idx >= n { 0 } else { idx }
}
/// The feature cells after the leading-cell filter ([`feature_start_cell`]).
/// The feature cells after the leading-cell filter ([`Self::feature_start_cell`]).
pub fn feature_cells(&self) -> &[DvdCell] {
&self.cells[self.feature_start_cell()..]
}
@@ -425,21 +425,14 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
for (&vts_number, titles_info) in &title_set_map {
match parse_vts(reader, udf, vts_number, titles_info) {
Ok(ts) => title_sets.push(ts),
// The operator's Stop is not a placeholder title set. Once the
// drive's halt flag is set EVERY command fails with `Halted`, so
// swallowing it here would skip every REMAINING title set in turn
// and return `Ok` with a truncated list — a cancelled scan that
// is indistinguishable from a disc genuinely holding fewer
// titles. `scan_dvd_titles` propagates it to the caller.
// Halted means the drive itself stopped, not that this title set is a
// placeholder — once set, every later parse_vts call also fails, so
// swallowing it here would silently truncate the scan. Propagate it.
Err(Error::Halted) => return Err(Error::Halted),
Err(e) => {
// Some discs carry placeholder TT_SRPT entries for title sets
// that are not really there, so one failure is not fatal. But
// the failure must not be INVISIBLE: every skipped set is a
// title the user will never see, and swallowing the reason made
// a disc that enumerated 38 titles from one image and 10 from
// another look like a scan difference rather than 28 dropped
// reads.
// Some discs carry placeholder TT_SRPT entries for title sets that
// aren't really there, so one failure isn't fatal — but it must not
// be silent, since swallowing it can hide many dropped titles.
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
@@ -563,25 +556,9 @@ fn parse_vts(
// VTS_PGCIT sector pointer
let pgcit_sector = be_u32(&vts_data, VTS_PGCIT_OFFSET)?;
// First sector of the VTS **Title** VOBS (`vtstt_vobs`, VTSTT_VOBS_OFFSET).
// The cell `first_sector` / `last_sector` values in the title PGCs are
// relative to this. Offset 0xC0 is `vtsm_vobs` — the VTS *menu* VOBS
// (VTS_xx_0.VOB), which on discs with a per-title menu (e.g. a Universal
// "the parental level has been set, press yes" first-play still) holds that
// interactive prompt. Reading the menu base instead prepended the menu VOB
// to the feature and shifted every cell extent back by
// `vtstt_vobs - vtsm_vobs` sectors, so the rip opened on the parental
// prompt instead of the movie. The title content lives at `vtstt_vobs`.
//
// `vtstt_vobs` is a sector address **relative to the start of this VTS_xx_0.IFO
// file**, not an absolute disc LBA. The cell `first_sector`/`last_sector`
// values are in turn relative to `vtstt_vobs`. To turn them into the absolute
// disc LBAs the reader needs, add the IFO file's own on-disc location (from
// the UDF FS). Without this rebase every extent started `ifo_lba` sectors too
// early — for THESILENCEOFTHELAMBS the feature began at LBA 126 (the VMGI /
// VIDEO_TS.VOB main-menu region) instead of 132886 (VTS_03_1.VOB), so the
// first ~4.5 min of muxed video was the disc's main menu before the stream
// drifted into the movie.
// First sector of the VTS **Title** VOBS (`vtstt_vobs`), which PGC cell extents
// are relative to (offset 0xC0/`vtsm_vobs` is the *menu* VOBS, not the movie).
// It's relative to this IFO file, so rebase by the IFO's on-disc LBA (UDF FS).
let vtstt_vobs = be_u32(&vts_data, VTSTT_VOBS_OFFSET)?;
let ifo_lba = udf.file_start_lba(reader, &path)?;
let vob_start_sector = ifo_lba.saturating_add(vtstt_vobs);
@@ -600,11 +577,9 @@ fn parse_vts(
}
audio_streams.push(parse_audio_attr(&vts_data, aoff)?);
}
// Assign each audio stream its on-wire private_stream_1 sub-stream id
// by per-codec ordinal — the same convention DVD authoring uses (AC-3
// 0x80+, DTS 0x88+, LPCM 0xA0+). This is the routing key shared with
// the muxer; per-codec ordinals (not the positional index) are what
// keep mixed-codec titles from colliding.
// Assign each stream its on-wire private_stream_1 id by per-codec ordinal
// (AC-3 0x80+, DTS 0x88+, LPCM 0xA0+), the DVD authoring convention the
// muxer routes on — the positional index would collide in mixed-codec titles.
assign_audio_sub_stream_ids(&mut audio_streams);
// Subtitle streams: count at 0x254 (u16 BE), then 6 bytes each starting at 0x256
@@ -641,12 +616,9 @@ fn parse_vts(
// ── Attribute parsers ───────────────────────────────────────────────────────
// ── VTS_V_ATR byte 0 bitfield layout (DVD-Video spec, MSB first) ──────────
// bits 7-6 mpeg_version | bits 5-4 video_format | bits 3-2 display_aspect
// | bits 1-0 permitted_df
// Naming the positions is the guard against the original bug: video_format is
// bits 5-4, NOT bits 1-0 (those are the pan&scan/letterbox permission). Reading
// the low two bits mis-detected every PAL disc as NTSC → 720x480 not 720x576.
// VTS_V_ATR byte 0 (MSB first): bits 7-6 mpeg_version | bits 5-4 video_format |
// bits 3-2 display_aspect | bits 1-0 permitted_df. video_format is bits 5-4, NOT
// bits 1-0 (pan&scan/letterbox permission) — the earlier bug misread PAL as NTSC.
const V_ATR_VIDEO_FORMAT_SHIFT: u8 = 4;
const V_ATR_ASPECT_SHIFT: u8 = 2;
const V_ATR_FIELD_MASK: u8 = 0x03;
@@ -712,11 +684,9 @@ pub(crate) fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAtt
let b1 = byte_at(data, offset + 1)?;
let coding_mode = (b0 >> 5) & 0x07;
// DVD-Video audio_coding_mode. Modes 2 and 3 are both MPEG audio Layer II
// (3 adds the MPEG-2 multichannel extension), so both are Codec::Mp2.
// Mode 2 previously mapped to Codec::Mpeg1 — a VIDEO variant, so
// Codec::kind() reported Video and the audio stream was classified and
// handled as video everywhere downstream.
// Modes 2 and 3 are both MPEG audio Layer II (3 adds the multichannel
// extension), so both map to Codec::Mp2. Mode 2 previously mapped to the
// video codec Codec::Mpeg1, so the audio stream was classified as video.
let codec = match coding_mode {
0 => Codec::Ac3,
2 | 3 => Codec::Mp2,
@@ -868,10 +838,9 @@ fn parse_pgcit(
let entries_start = pgcit_offset + 8;
let mut titles = Vec::new();
// Every `continue` below drops a title the user will never see. None of
// them may be silent: `parse_vmg` already counts and warns per skipped
// title SET, and this function was the remaining place where a disc could
// quietly report fewer titles than it has.
// Every `continue` below drops a title the user will never see and must not
// be silent: `parse_vmg` warns per skipped title SET, but this was the
// remaining place a disc could quietly report fewer titles than it has.
let mut skipped = 0usize;
for &(chapter_count, vts_title_num) in titles_info {
@@ -910,14 +879,9 @@ fn parse_pgcit(
match parse_pgc(data, pgc_abs, chapter_count) {
Ok(title) => titles.push(title),
Err(e) => {
// By design a single unparseable PGC (truncated/corrupt entry,
// authoring-tool quirk) must not lose the whole title list. But
// "not fatal" is not the same as "not worth saying": every PGC
// skipped here is a title the user will never see, and this was
// the only remaining silent one — `parse_vmg` above already
// counts and warns per skipped title SET for exactly this
// reason. A disc quietly reporting fewer titles than it has is
// the failure this release exists to stop.
// A single unparseable PGC (truncated/corrupt entry, authoring quirk)
// must not lose the whole title list, but must not be silent either:
// a disc quietly reporting fewer titles than it has is what this warns.
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
@@ -951,11 +915,8 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
return Err(Error::IfoParse);
}
// PGC layout:
// 0x00-0x01: misc flags
// 0x02: nr_of_programs
// 0x03: nr_of_cells
// 0x04-0x07: playback_time (4 BCD bytes)
// PGC layout: 0x00-0x01 misc flags, 0x02 nr_of_programs,
// 0x03 nr_of_cells, 0x04-0x07 playback_time (4 BCD bytes).
let num_cells = byte_at(data, pgc_offset + 0x03)? as usize;
let time_bytes = sub_slice(data, pgc_offset + 0x04, 4)?;
let duration_secs = bcd_to_secs(time_bytes);
@@ -1011,14 +972,9 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
let mut times = Vec::new();
if pgm_map_offset > 0 && nr_of_programs > 0 && cell_playback_offset > 0 {
let pgm_base = pgc_offset + pgm_map_offset;
// Collect per-cell durations as exact timecode FRAME counts. Summing
// frames (integers) and converting once per chapter keeps every mark
// on an exact frame boundary — summing f64 seconds would let rounding
// drift accumulate across a two-hour title.
//
// A cell whose rate flag is unspecified has no frame count, so it
// contributes plain seconds instead; that is tracked separately and
// added on top. In practice a PGC never mixes rates.
// Collect durations as exact FRAME counts, converting once per chapter so
// marks stay on frame boundaries (summing f64 seconds would drift). Cells
// with no rate flag fall back to plain seconds, tracked separately.
let mut cell_frames: Vec<(u64, f64)> = Vec::with_capacity(num_cells);
let mut rate: Option<DvdRate> = None;
let cell_base = pgc_offset + cell_playback_offset;
@@ -1108,29 +1064,18 @@ mod tests {
#[test]
fn tt_srpt_title_count_is_capped_and_deduplicated() {
// The TT_SRPT count is an untrusted u16. A ~800 KB crafted IFO declares
// 65535 entries, and each one re-parses a PGC into another full DvdTitle.
//
// The entries are DISTINCT, which matters: with 65535 identical entries
// the de-duplication collapses them to one title on its own and the count
// cap is never what bounds the result — the test would then pass with the
// cap removed entirely, proving only that dedup works. Distinct entries
// defeat dedup, so the cap is the only thing left holding the line.
// De-duplication gets its own assertion below, on its own fixture.
// Both title fields are u8, so vary each within its own range to get
// ~65025 distinct pairs out of the 65535 entries.
// The TT_SRPT count is an untrusted u16. Entries must be DISTINCT: identical
// entries would let dedup alone collapse them to one title, so the cap would
// never be exercised — vary both u8 fields for ~65025 distinct pairs.
let entries: Vec<(u16, u8, u8)> = (0..u16::MAX)
.map(|i| (5u16, (i % 255) as u8 + 1, ((i / 255) % 255) as u8 + 1))
.collect();
let data = tt_srpt_bytes(u16::MAX, &entries);
let map = parse_tt_srpt(&data, 0).expect("well-formed TT_SRPT");
let total: usize = map.values().map(|v| v.len()).sum();
// Asserted against the DVD-Video format maximum as a LITERAL, not against
// MAX_TT_SRPT_TITLES. Comparing a result to the very constant under test
// passes vacuously the moment someone raises that constant, which is the
// most likely regression here — and is the tautology class this audit has
// now found six times. 99 is the format's own ceiling, so this pins the
// cap to the spec rather than to whatever the code currently says.
// Asserted against the DVD-Video max as a LITERAL, not MAX_TT_SRPT_TITLES:
// comparing to the constant under test would pass vacuously if that constant
// were raised. 99 is the format's own ceiling, pinning the cap to the spec.
assert!(
total <= 99,
"TT_SRPT produced {total} titles from a crafted 65535-entry table; \
@@ -1178,10 +1123,8 @@ mod tests {
// 0 hours, 1 minute, 30 seconds, 15 frames of NTSC timecode.
let bcd = [0x00, 0x01, 0x30, 0b11_010101];
let secs = bcd_to_secs(&bcd);
// 0b010101 = 0x15, BCD = 15 frames.
// Timecode 00:01:30:15 is 90*30 + 15 = 2715 frames, and each frame
// lasts 1001/30000 s, so real time is 2715*1001/30000 = 90.5905 s
// (NOT 90.5 s — NTSC timecode runs 0.1% slow against the clock).
// 0b010101 = 0x15 BCD = 15 frames. Timecode 00:01:30:15 = 2715 frames at
// 1001/30000 s each = 90.5905 s (NOT 90.5 s — NTSC runs 0.1% slow).
let expected = 2715.0 * 1001.0 / 30000.0;
assert!((secs - expected).abs() < 1e-9, "got {}", secs);
assert!((secs - 90.5905).abs() < 1e-9, "got {}", secs);
@@ -1438,15 +1381,9 @@ mod tests {
/// bit positions drift, these fail.
#[test]
fn video_attr_absolute_bytes_pin_real_layout() {
// (byte @0x200, expected standard, expected aspect, expected resolution).
// PAL 16:9 anamorphic = mpeg(00) format(01=PAL) aspect(11=16:9) df(00)
// = 0b0001_1100 = 0x1C (e.g. a PAL 16:9 R2 feature disc).
// PAL 4:3 = 0b0001_0000 = 0x10.
// NTSC 16:9 = 0b0000_1100 = 0x0C.
// NTSC 4:3 = 0b0000_0000 = 0x00.
// A real disc also sets mpeg_version=01 (MPEG-2) in bits 7-6, which the
// parser must IGNORE; OR it in (|0x40) to prove it doesn't leak into the
// video_format read.
// (byte @0x200, expected standard, expected aspect, expected resolution):
// PAL 16:9=0x1C, PAL 4:3=0x10, NTSC 16:9=0x0C, NTSC 4:3=0x00. A real disc
// also sets mpeg_version=01 in bits 7-6 (|0x40), which the parser must ignore.
let cases: &[(u8, TvSystem, DvdAspect, Resolution)] = &[
(0x1C, TvSystem::Pal, DvdAspect::R16x9, Resolution::R576i),
(0x10, TvSystem::Pal, DvdAspect::R4x3, Resolution::R576i),
@@ -1463,10 +1400,9 @@ mod tests {
assert_eq!(attr.aspect, aspect, "byte {b0:#04x} → aspect");
assert_eq!(attr.resolution, res, "byte {b0:#04x} → resolution");
}
// Anti-bug anchor: the original bug read the TV system from bits 1-0
// (permitted_df). A PAL byte whose low 2 bits are 0 (0x1C) must NOT be
// misread as NTSC — and a byte with low bits set but format=NTSC
// (0x03 = NTSC, df=11) must stay NTSC, proving the low bits are ignored.
// Anti-bug anchor: the original bug read TV system from bits 1-0
// (permitted_df). A byte with low bits set but format=NTSC (0x03, df=11)
// must stay NTSC here, proving the low bits are ignored.
let mut df = vec![0u8; 0x204];
df[0x200] = 0x03; // format=NTSC(00), df=11
assert_eq!(
@@ -1497,10 +1433,9 @@ mod tests {
#[test]
fn mixed_codec_sub_stream_ids_are_distinct() {
// A title mixing AC-3, DTS and LPCM: the sub-id low nibble is the
// POSITIONAL audio-stream number (shared across codecs), OR'd with the
// codec base. So idx 1 (DTS) → 0x89, idx 3 (AC-3) → 0x83 — the real
// wire ids the demux routes on. All distinct (positions are unique).
// A title mixing AC-3, DTS and LPCM: sub-id low nibble is the positional
// audio-stream number OR'd with the codec base (idx 1 DTS → 0x89, idx 3
// AC-3 → 0x83) — the real wire ids the demux routes on, all distinct.
let mut streams = vec![
DvdAudioAttr {
codec: Codec::Ac3,
@@ -1625,10 +1560,8 @@ mod tests {
assert_eq!(attr.language, "fra");
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the DVD-Video IFO spec
// (DVD-Video IFO format; http://dvd.sourceforge.net).
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests, grounded in the DVD-Video IFO spec
// (http://dvd.sourceforge.net).
/// BCD frame-rate flag: bits 7-6 of byte[3]. 0b01 = 25fps (PAL),
/// 0b11 = 29.97fps (NTSC). 0b00/0b10 are "unknown" → frames ignored.
@@ -1938,8 +1871,8 @@ mod tests {
/// Regression test for issue freemkv#25 (NTSC chapter drift).
///
/// Builds a PGC from the real cell table of the DOLL_MASTERS_7 disc in the
/// bug report and checks the chapter marks against MakeMKV's values. Before
/// Builds a PGC from the real cell table of the disc in the
/// bug report and checks the chapter marks against the reference tool's values. Before
/// the fix, chapter 14 landed 4.019 s early; the drift was proportional to
/// elapsed time (0.1%), so a short synthetic fixture would not have caught it.
#[test]
@@ -1987,7 +1920,7 @@ mod tests {
}
let title = parse_pgc(&pgc, 0, pgm.len() as u16).unwrap();
// MakeMKV / source truth, in seconds.
// Reference / source truth, in seconds.
let expected = [
0.0,
62.062,
@@ -2115,7 +2048,7 @@ mod tests {
/// A normal single-angle feature (every cell category 0x00) is never
/// filtered: feature_start_cell == 0, feature_cells == all cells. This is
/// the "Silence of the Lambs" case — the filter must be a no-op.
/// the plain-single-angle-feature case — the filter must be a no-op.
#[test]
fn feature_filter_noop_on_plain_feature() {
let t = DvdTitle {
@@ -2259,10 +2192,8 @@ mod tests {
);
}
// ─────────────────────────────────────────────────────────────────────
// Added: cell-category bit isolation, palette/program-map arithmetic,
// and the malformed-language salvage paths.
// ─────────────────────────────────────────────────────────────────────
/// Each low flag of the cell-category byte comes from its OWN bit and no
/// other: bit3 seamless_play, bit2 interleaved, bit1 stc_discontinuity,
@@ -2572,10 +2503,9 @@ mod tests {
/// `[padding, Y, Cr, Cb]`. Every byte of every entry is distinct here, so
/// a wrong stride, a wrong base or a shifted component shows up.
#[test]
// The loop variable is the DOMAIN VALUE being checked (a palette entry number), not a
// cursor into a collection: it is what the assertion message names and
// what the table is keyed by. `.iter().enumerate()` would rename the
// thing under test to `i` and read worse.
// The loop variable is the DOMAIN VALUE under test (a palette entry number),
// not a collection cursor — it's what the assertion message names, so
// `.iter().enumerate()` would rename the thing under test and read worse.
#[allow(clippy::needless_range_loop)]
fn pgc_palette_entries_read_at_correct_stride() {
let mut pal = [[0u8; 4]; 16];
@@ -2714,16 +2644,8 @@ mod tests {
}
// ── TT_SRPT: the title table (`parse_tt_srpt` / `parse_vmg`) ────────────
//
// This is the table that says which titles a DVD has, and it had no tests
// at all until 1.6.2 — while being at the centre of both DVD stories in
// 1.6.1. The CSS descramble bug destroyed the sector carrying it, so a disc
// enumerating 38 titles produced an image enumerating 10, silently, at exit
// 0. And `dir://` was pulled from the release on 2026-08-05 because a
// folder produced the wrong title list — the same table, reached through a
// synthesized image.
//
// Both were found on real discs. Nothing hermetic covered the table itself.
// This table says which titles a DVD has and had no tests until 1.6.2, despite
// two real-disc 1.6.1 bugs that silently mis-enumerated titles through it.
/// Build a VMG with a TT_SRPT at `tt_srpt_sector` declaring `entries`
/// of `(num_chapters, vts_number, vts_title_num)`.
@@ -2767,11 +2689,9 @@ mod tests {
let at_sector = parse_tt_srpt(&vmg, sector as usize * crate::consts::SECTOR_BYTES)
.expect("reading at the sector offset works");
assert_eq!(at_sector[&1], vec![(3, 1)], "sector {sector}");
// Reading the same pointer as a BYTE offset lands 1/2048th of the
// way in — inside the header here — and yields a different table.
// It does not error: it silently reports the wrong titles, which is
// how 28 dropped titles looked like a scan difference rather than a
// failure.
// Reading the same pointer as a BYTE offset lands 1/2048th of the way
// in and yields a different table without erroring — silently wrong
// titles, which is how dropped titles looked like a scan difference.
let as_bytes = parse_tt_srpt(&vmg, sector as usize).expect("does not error");
assert_ne!(
as_bytes, at_sector,
@@ -2819,11 +2739,9 @@ mod tests {
/// unclamped 65535 is a ~540 MB allocation from a ~800 KB crafted file.
#[test]
fn tt_srpt_clamps_an_absurd_declared_title_count() {
// The fixture must actually CONTAIN more entries than the cap, or the
// walk stops when the buffer runs out and the clamp is never what
// bounded it. An earlier version of this test declared u16::MAX over a
// two-kilobyte fixture: the loop broke on `base + 12 > len` after three
// iterations and the assertion held with the clamp deleted entirely.
// The fixture must actually CONTAIN more entries than the cap, or the walk
// stops when the buffer runs out and the clamp is never exercised — an
// earlier version declared u16::MAX over a 2KB fixture and passed vacuously.
const PRESENT: usize = MAX_TT_SRPT_TITLES + 40;
let entries: Vec<(u16, u8, u8)> = (0..PRESENT)
.map(|i| (1u16, 1u8, (i % 250 + 1) as u8))
+9 -18
View File
@@ -109,19 +109,14 @@ where
return Err(BoundedError::Halted);
}
// Rendezvous channel: the worker sends exactly one value (the
// op's return) and then exits. Capacity-0 means the send blocks
// until we receive — fine on the happy path; on the timeout /
// halt path the receiver is dropped and the worker's send
// returns Err, which the worker ignores.
// Rendezvous channel: worker sends one value then exits. On timeout/halt
// the receiver is dropped, so the worker's send returns Err (ignored).
let (tx, rx) = sync_channel::<R>(0);
let _ = thread::Builder::new()
.name("freemkv-bounded-syscall".into())
.spawn(move || {
// Ignore the send error: if we time out (or get halted)
// before the worker finishes, the receiver is dropped
// and `tx.send` returns Err. Either way, the worker has
// nothing more to do.
// Ignore send error: if we time out/halt before the worker finishes,
// the receiver is dropped and `tx.send` returns Err — nothing more to do.
let _ = tx.send(op());
});
@@ -179,10 +174,8 @@ mod tests {
fn halt_fires_during_wait() {
let halt = Halt::new();
let halt2 = halt.clone();
// Flip the halt from a side thread after ~300 ms — long
// enough that the receive loop has rolled at least one
// 250 ms slice and is sitting in `recv_timeout` again when
// the bit flips.
// Flip halt after ~300ms, long enough that the receive loop has rolled
// at least one 250ms slice and is back in `recv_timeout`.
thread::spawn(move || {
thread::sleep(Duration::from_millis(300));
halt2.cancel();
@@ -196,11 +189,9 @@ mod tests {
#[test]
fn worker_panics() {
// Worker panics → sender drops without sending → recv sees
// Disconnected → WorkerLost. We use an explicit panic in the
// op closure rather than `panic!()` from inside the channel
// machinery; the spawned thread's panic is contained (no
// process abort) because we don't `.join()` it.
// Worker panics → sender drops without sending → recv sees Disconnected →
// WorkerLost. Panic is in the op closure, not the channel machinery; it's
// contained (no process abort) because we don't `.join()` the thread.
let r = bounded_syscall(None, Duration::from_secs(2), || -> u32 {
panic!("intentional test panic");
});
+24 -57
View File
@@ -75,18 +75,15 @@ impl BytePrefetcher {
chunk_bytes: usize,
halt: Option<Halt>,
) -> std::io::Result<Self> {
// A zero-length chunk makes every recycled buffer an empty
// slice; `reader.read(&mut [])` returns Ok(0), which the loop
// below treats as EOF — the consumer would see a clean,
// silent zero-byte stream. Callers pass the downstream
// demuxer's batch size, which is always > 0.
// A zero-length chunk makes every recycled buffer empty; `reader.read(&mut [])`
// returns Ok(0), which the loop below treats as EOF, producing a silent
// zero-byte stream. Callers must pass the downstream demuxer's batch size (> 0).
debug_assert!(chunk_bytes > 0, "BytePrefetcher chunk_bytes must be > 0");
let (tx, rx) = bounded::<Batch>(FORWARD_DEPTH);
let (recycle_tx, recycle_rx) = bounded::<Vec<u8>>(RECYCLE_DEPTH);
// Seed the recycle pool. Without these the first
// `recycle_rx.recv()` would block forever (no consumer has
// returned a buffer yet).
// Seed the recycle pool; otherwise the first `recycle_rx.recv()` blocks
// forever since no consumer has returned a buffer yet.
for _ in 0..RECYCLE_DEPTH {
let _ = recycle_tx.send(vec![0u8; chunk_bytes]);
}
@@ -94,21 +91,13 @@ impl BytePrefetcher {
let producer = std::thread::Builder::new()
.name("freemkv-byte-prefetch".into())
.spawn(move || {
// Wrap the feed loop in catch_unwind so a panic in the inner
// `reader.read` (e.g. a decrypt-on-read slice/arith bug) is NOT
// indistinguishable from a clean finish at the demux boundary. A
// clean exit (EOF, halt, consumer disconnect) returns and drops
// `tx` → the demux loop reads RecvError as EOF (correct). A PANIC
// sends an explicit error sentinel first so the demux loop's
// `Ok(Err(_))` arm fires and propagates a typed error instead of
// converting the dropped channel into a clean `DemuxBatch::Eof`
// that would finalize a TRUNCATED mux while reporting success.
// catch_unwind: a clean exit drops `tx` (demux reads RecvError as EOF),
// but a panic sends an error sentinel first so demux gets a typed error
// instead of finalizing a truncated mux as success.
let body = std::panic::AssertUnwindSafe(|| {
let cancelled = || halt.as_ref().map(|h| h.is_cancelled()).unwrap_or(false);
// Liveness heartbeat: the producer blocks on the recycle and
// forward channels; a stalled consumer or a wedged reader shows
// up as the beat going silent. Total is unknown, so `pos` is
// cumulative bytes read.
// Liveness heartbeat: a stalled consumer or wedged reader shows up
// as the beat going silent. Total is unknown, so `pos` is cumulative.
let mut hb = crate::progress::Heartbeat::new("byte_prefetch");
let mut produced_bytes: u64 = 0;
loop {
@@ -116,10 +105,9 @@ impl BytePrefetcher {
if cancelled() {
return;
}
// Park on the recycle channel, but re-poll halt
// every POLL_INTERVAL: a pure-AtomicBool Halt does
// not disconnect the channel, so a blocking recv()
// would never re-reach the cancel check.
// Re-poll halt every POLL_INTERVAL: a pure-AtomicBool Halt does
// not disconnect the channel, so a blocking recv() would never
// re-reach the cancel check.
let mut buf = loop {
match recycle_rx.recv_timeout(POLL_INTERVAL) {
Ok(b) => break b,
@@ -132,11 +120,9 @@ impl BytePrefetcher {
Err(RecvTimeoutError::Disconnected) => return,
}
};
// Re-expose the full extent. After a short read the
// prior iteration truncated to n < chunk_bytes, so
// this regrows the length back to chunk_bytes
// without reallocating (capacity was fixed at
// construction and never shrinks).
// Regrow to chunk_bytes: a prior short read truncated len to
// n < chunk_bytes. No realloc — capacity was fixed at
// construction and never shrinks.
if buf.len() < chunk_bytes {
buf.resize(chunk_bytes, 0);
} else {
@@ -196,27 +182,12 @@ impl BytePrefetcher {
/// drains `rx`, runs the demuxer in place on each filled buffer,
/// and recycles back through `recycle_tx`.
pub fn into_channels(self) -> (Receiver<Batch>, Sender<Vec<u8>>, PrefetchShell) {
// MOVE the three fields out cleanly — never clone. Each of
// `rx` and `recycle_tx` ends up with exactly ONE live copy:
// the one in the returned tuple. The pre-1.0.0 implementation
// cloned both and then `mem::forget`-ed `self`, leaking the
// originals so an extra live receiver + sender survived
// forever. That defeated the channel-disconnection shutdown:
// when the demux consumer exited early (halt, or a `tx.send`
// error in `demux_thread`), the producer's `recycle_rx.recv()`
// and `tx.send()` never saw all-peers-dropped, so the producer
// never returned and `PrefetchShell::drop`'s `join()` hung.
//
// `ManuallyDrop` + `ptr::read` reads each field out by value
// and suppresses `self`'s own `Drop` (which would otherwise
// double-`join`), leaving NO extra live endpoint behind. This
// is the panic-free equivalent of the `Option::take` approach
// and mirrors `sector::prefetched::into_channels`.
// MOVE the fields out, never clone: the pre-1.0.0 impl cloned + `mem::forget`-ed
// `self`, leaking endpoints that defeated disconnection shutdown (producer join
// hung forever). `ManuallyDrop` + `ptr::read` moves fields out cleanly instead.
let me = std::mem::ManuallyDrop::new(self);
// SAFETY: `me` is `ManuallyDrop`, so none of these fields will
// be dropped by `me`. Each `ptr::read` performs exactly one
// bitwise move out; every field is read exactly once and never
// touched again, so there are no double-frees and no aliasing.
// SAFETY: `me` is `ManuallyDrop`, so none of these fields are dropped by `me`.
// Each `ptr::read` is one bitwise move, read exactly once — no double-frees.
let producer = unsafe { std::ptr::read(&me.producer) };
// SAFETY: `rx` and `recycle_tx` are always `Some` here —
// `into_channels` is the only way to consume a live
@@ -230,13 +201,9 @@ impl BytePrefetcher {
impl Drop for BytePrefetcher {
fn drop(&mut self) {
// Drop channel endpoints BEFORE joining the producer so the
// producer observes SendTimeoutError::Disconnected (forward tx)
// or RecvTimeoutError::Disconnected (recycle rx) and exits
// promptly. Without this, a non-EOF source fills the depth-2
// forward channel and then spins in send_timeout(POLL_INTERVAL)
// forever because rx is never drained, causing join() to
// deadlock.
// Drop channel endpoints BEFORE joining so the producer observes Disconnected
// (send/recv) and exits promptly. Otherwise a non-EOF source fills the forward
// channel and spins in send_timeout forever since rx is never drained.
drop(self.rx.take());
drop(self.recycle_tx.take());
if let Some(h) = self.producer.take() {
+13 -24
View File
@@ -33,7 +33,7 @@
//!
//! On `open()` each platform issues its "sequential access expected"
//! hint so OS-level readahead widens. The hint and the DONTNEED call
//! live in per-OS sibling modules ([`linux::hint_sequential`] et al.)
//! live in per-OS sibling modules (`linux::hint_sequential` et al.)
//! — no inline `#[cfg]` in this file.
//!
//! ## Read-ahead prefetch
@@ -190,13 +190,9 @@ impl SectorSource for FileSectorSource {
) -> Result<usize> {
let count = count as u32;
let bytes = count as usize * SECTOR_BYTES;
// A real check, not a debug_assert: this is a public `SectorSource` impl,
// so an undersized `out` is caller input, and `out[..bytes]` below would
// panic with 'range end index out of range' in release where the assert is
// compiled away. `Drive::read_fua` already carries exactly this guard, with
// a comment recording the same panic being fixed there — this impl was
// simply never given it, and `PrefetchedSectorSource` has a regression test
// for the case that this one lacked.
// A real check, not debug_assert: `out` is caller input to a public
// `SectorSource` impl, and `out[..bytes]` below would panic in release
// where the assert is compiled away. `Drive::read_fua` carries this guard.
if out.len() < bytes {
return Err(Error::DiscRead {
sector: lba as u64,
@@ -215,18 +211,14 @@ impl SectorSource for FileSectorSource {
.read_exact(&mut out[..bytes])
.map_err(|e| Error::IoError { source: e })?;
// Queue the next batch's read with the kernel before the
// caller starts processing what we just returned. readahead()
// is non-blocking — it queues I/O and returns, so the kernel
// pulls those pages into cache while the consumer (decrypt +
// demux + mux) runs. Next read_sectors call hits a warm cache.
// Queue the next batch's read before the caller processes what we returned.
// readahead() is non-blocking; the kernel pulls pages into cache while the
// consumer runs, so the next read_sectors call hits a warm cache.
platform::prefetch(&self.file, offset + bytes as u64, bytes as u64);
// Periodic page-cache eviction on the read side. Without
// this, an 85 GB streaming ISO read pins the entire file in
// the kernel page cache, which starves concurrent writes and
// collapses mux throughput. Mirrors the write-side
// WritebackPipeline's DONTNEED policy.
// Periodic page-cache eviction on the read side: an 85 GB streaming ISO
// read would otherwise pin the whole file in cache, starving concurrent
// writes. Mirrors WritebackPipeline's DONTNEED policy on the write side.
self.bytes_read_since_drop += bytes as u64;
if self.bytes_read_since_drop >= self.drop_chunk_bytes {
let drop_start = self.drop_window_start;
@@ -407,12 +399,9 @@ mod tests {
#[test]
fn drop_chunk_size_env_override() {
// Explicit 8 MiB via env var.
// SAFETY: tests in this crate are single-threaded per the
// default cargo test harness, but std::env::set_var is
// declared `unsafe` since Rust 2024 (it can race with other
// threads / TLS). For a test that runs in-process before any
// FileSectorSource construction this is safe in practice.
// Explicit 8 MiB via env var. SAFETY: set_var is `unsafe` since Rust 2024
// (can race with other threads/TLS), but this test runs single-threaded
// and in-process before any FileSectorSource construction.
unsafe {
std::env::set_var("FREEMKV_READ_DROP_CHUNK_MIB", "8");
}
+3 -11
View File
@@ -101,17 +101,9 @@ pub fn write_image(
on_progress(written);
}
// flush() only pushes the BufWriter's bytes into the kernel via write(2).
// It makes no durability promise at all, so returning Ok here would report
// a finished image while up to several gigabytes of it still sit in the
// page cache. A crash, a power loss, or yanking the removable/network
// volume the image was written to then leaves a truncated or empty file
// that the caller was told was complete.
//
// For a 6-90 GB image that is exactly the failure this crate treats as
// worst: success reported over wrong output. `into_inner` is used rather
// than `flush` so a buffered-write error is surfaced instead of being
// dropped on the floor by BufWriter's Drop.
// flush() only pushes bytes into the kernel via write(2), no durability promise —
// a crash or yanked volume could leave a truncated file reported as complete.
// `into_inner` (not `flush`) also surfaces buffered-write errors, not silently drop them.
let file = out.into_inner().map_err(|e| Error::IoError {
source: e.into_error(),
})?;
+47 -111
View File
@@ -1,13 +1,12 @@
//! Generic bounded producer/consumer pipeline.
//!
//! `Pipeline<I, R>` spawns a single consumer thread, hands it items
//! through a bounded `mpsc::sync_channel`, and joins it on `finish()`.
//! through a bounded `crossbeam_channel`, and joins it on `finish()`.
//! The consumer's behaviour is supplied by a [`Sink`] implementation:
//! `apply` is called once per item, `close` is called once at the end.
//!
//! Call sites in libfreemkv that want a producer/consumer split —
//! sweep (`disc/sweep.rs::SweepSink`) and the file-backed mux highway
//! — are built on this primitive.
//! The file-backed mux highway is built on this primitive. (An earlier
//! sweep sink also used it; sweep moved to freemkv-engine in 1.6.0.)
//!
//! ## Cancellation and error semantics
//!
@@ -170,19 +169,9 @@ fn finish_with_grace<R: Send + 'static>(
}
thread::sleep(POLL_INTERVAL);
}
// Grace expired. CLAIM abandonment, then log and leak. Claiming BEFORE
// dropping the handle guarantees the leaked consumer observes it the moment
// its wedged syscall returns: it then skips any further `apply` and skips
// `close()`, rather than running on to finalise the abandoned output file.
//
// A compare-exchange, not a store, because the consumer may have committed to
// `close()` in the instant between our last `is_finished()` poll and now. It
// then cannot be stopped — the finalise IS happening — so abandoning it would
// report the rip as interrupted while a valid, fully finalised container
// lands on disk. Losing the race means waiting for the result the consumer is
// already producing instead. `AcqRel` pairs with the consumer's own
// compare-exchange and with the `Acquire` loads in its drain loop, so the flag
// is reliably observed even on weak memory models (ARM64/POWER).
// Grace expired. CLAIM abandonment before dropping the handle so the leaked consumer
// skips further `apply`/`close()` once its wedged syscall returns. Compare-exchange,
// not a store: if it already committed to `close()`, wait for its result instead.
if state
.compare_exchange(
state::RUNNING,
@@ -221,11 +210,9 @@ fn finish_with_grace<R: Send + 'static>(
(output will not be finalised)",
FINISH_GRACE_SECS
);
// Dropping `handle` without joining detaches from the thread — the
// consumer keeps running until its kernel call returns or the process
// exits. This is the intentional "leak" documented in
// `finish_with_halt`'s contract; the `abandoned` flag bounds what the
// leaked thread is allowed to do to the output before it exits.
// Dropping `handle` without joining detaches the thread; the consumer keeps
// running until its kernel call returns or the process exits. This is the
// intentional "leak" from `finish_with_halt`'s contract, bounded by `abandoned`.
drop(handle);
Err(leak_err)
}
@@ -365,11 +352,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let mut first_err: Option<Error> = None;
let mut stopped = false;
// Rolling apply-throughput summary. The per-item "apply: OK"
// line was 99% of the mux log; collapse it into a periodic
// summary (count, avg ms, items/s) emitted ~every 5 s while
// debug tracing is on. The individual slow-apply ("took … s")
// STALL events below stay visible — those are signal, not noise.
// Rolling apply-throughput summary: the per-item "apply: OK" line was
// 99% of the mux log, so collapse it into a periodic summary (count,
// avg ms, items/s) every ~5s. Slow-apply STALL events stay visible below.
let mut summary_count: u64 = 0;
let mut summary_nanos: u128 = 0;
let mut summary_since = Instant::now();
@@ -381,13 +366,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline receive: item={}", std::any::type_name::<I>());
}
// If the caller has abandoned us (grace period expired
// after a halt/timeout and the JoinHandle was dropped),
// stop applying items. We keep draining so the producer
// — if it is somehow still alive — never blocks on a
// dead receiver, but we touch the output no further. The
// final post-loop abandonment check returns the error
// and skips `close()`.
// Abandoned (grace expired, JoinHandle dropped): keep draining so a
// still-alive producer never blocks on a dead receiver, but touch
// the output no further. Post-loop check returns error, skips close().
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
continue;
}
@@ -417,11 +398,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
}
first_err = Some(e);
// Publish the failure so the producer can stop
// FEEDING a dead write side instead of only learning
// about it at `finish()` — by which time it has read
// the rest of the disc. `Release` pairs with the
// `Acquire` load in `send_with_halt`.
// Publish the failure so the producer stops feeding a dead
// write side instead of learning at `finish()`, after reading
// the rest of the disc. `Release` pairs with `send_with_halt`.
failed_consumer.store(true, Ordering::Release);
}
}
@@ -473,16 +452,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
);
}
// Final abandonment check: the common leak case is a
// consumer wedged inside `apply` (a blocking write
// syscall). When that syscall finally returns, the
// producer has long since dropped `tx`, so `recv`
// yields `Err` and we fall through to here. If the
// caller abandoned us in the meantime, skip `close()`
// entirely — finalising the output (e.g. writing the
// MKV Cues + patching the segment header) on a file the
// caller already reported as failed is exactly the
// write race we must not run.
// Final abandonment check: a consumer wedged in a blocking `apply` write
// can outlive the producer dropping `tx`, landing here via `recv -> Err`.
// Skip `close()` if abandoned meanwhile — it would race the write.
match first_err {
// No `close()` on this path, so there is nothing to claim —
// just report, unless the caller has already given up on us.
@@ -493,14 +465,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Err(e)
}
}
// CLAIM the finalise. A plain load here left a window in which
// the caller stored `abandoned` AFTER we read it as clear, so
// `close()` ran anyway and finalised (Cues + Segment-size
// patch) an output the caller had already reported as
// interrupted — a truncated rip indistinguishable from a
// complete one. The compare-exchange closes that window: if the
// caller got there first we skip `close()`, and if we get there
// first the caller waits for us instead of abandoning.
// CLAIM the finalise: a plain load could race the caller marking
// `abandoned`, letting `close()` finalise output already reported
// interrupted. Compare-exchange: loser skips `close()`, waits for winner.
None => {
if state_consumer
.compare_exchange(
@@ -645,12 +612,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let end = Instant::now() + deadline;
let mut pending = item;
loop {
// The consumer's `apply` has failed fatally: everything sent from here
// is drained and discarded, so hand the item back at once. Without this
// the producer saw every send succeed (the channel is always being
// drained) and went on reading the whole remaining title — an hour of
// drive time on a UHD — for a write that died on the first frame, only
// learning about it at `finish()`.
// The consumer's `apply` failed fatally: hand the item back now, since
// otherwise sends keep succeeding and the producer reads a whole UHD
// title before learning at `finish()` the write died on frame one.
if self.consumer_failed() {
if debug_enabled() {
tracing::debug!(
@@ -680,9 +644,8 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
}
return Err(pending);
}
// Wait for space-available or halt-check tick, whichever
// is sooner. Crossbeam's send_timeout is kernel-wakeup
// based: the consumer's recv on a saturated channel
// Wait for space-available or halt-check tick, whichever is sooner.
// send_timeout is kernel-wakeup based: recv on a saturated channel
// signals this thread the moment a slot opens up.
let slice = SEND_HALT_CHECK_INTERVAL.min(end.saturating_duration_since(now));
match self.tx.send_timeout(pending, slice) {
@@ -857,14 +820,9 @@ mod tests {
#[test]
fn back_pressure_blocks_sender() {
// depth=2 + 5 sends + 50ms/apply: with the consumer pinned at
// 50 ms per item, the producer can buffer 2 (channel cap) +
// 1 (consumer in flight) = 3 items before sends 4 and 5 must
// block on consumer progress. Wall-clock floor across all 5
// sends is therefore ~2 * 50ms = 100ms (sends 4 and 5 each
// wait roughly one apply-cycle). Use 80 ms as the assertion
// floor to stay above the 50ms-per-item progress floor while
// tolerating CI jitter — it still proves blocking is real.
// depth=2 + 5 sends + 50ms/apply: producer buffers 3 items (2 channel cap +
// 1 in flight) before sends 4 and 5 must block, giving a ~100ms wall-clock
// floor. Assert 80ms to tolerate CI jitter while still proving blocking.
let count = Arc::new(AtomicUsize::new(0));
let sink = SlowSink {
delay: Duration::from_millis(50),
@@ -985,10 +943,9 @@ mod tests {
)
.expect("spawn should succeed");
// Send 10 items. After Stop, subsequent sends may either
// succeed (already buffered) or fail with Err(I) (channel
// closed). Both are valid — we don't assert on the send
// results.
// Send 10 items. After Stop, subsequent sends may either succeed (already
// buffered) or fail with Err(I) (channel closed); both are valid, so we
// don't assert on the send results.
for i in 0..10u64 {
let _ = pipe.send(i);
}
@@ -1087,14 +1044,9 @@ mod tests {
#[test]
fn send_with_halt_returns_item_on_deadline() {
// depth=1 + consumer wedged in apply on the first item, AND
// the channel buffer already loaded with a second item, means
// any further `try_send` sees Full; with a 200 ms deadline and
// no halt fired, send_with_halt must return `Err(item)` within
// roughly the deadline. Synchronising on `started` ensures the
// consumer has actually started its wedged apply BEFORE we
// load the channel-buffer slot — without that, the consumer
// could still drain in a race window.
// depth=1 + wedged consumer + a loaded buffer slot means further `try_send`
// sees Full, so send_with_halt must return `Err(item)` around the 200ms
// deadline. Sync on `started` first so the consumer is wedged before we load the slot.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
@@ -1208,13 +1160,9 @@ mod tests {
matches!(res, Err(Error::Halted)),
"expected Err(Halted), got {res:?}"
);
// Bailed out within the grace period plus a healthy margin.
// The grace spin-poll adds up to FINISH_GRACE_SECS (5s) of
// extra wait for a truly wedged consumer; the test's consumer
// is deliberately never released before this assert so we
// exercise the "grace expires → leak" path. 15s is well under
// the 10-minute JOIN_TIMEOUT backstop and proves the new code
// doesn't block forever.
// Bailed within the grace period plus margin: grace spin-poll adds up to
// FINISH_GRACE_SECS (5s) for the deliberately-unreleased, wedged consumer.
// 15s stays well under the 10-minute JOIN_TIMEOUT, proving it doesn't block forever.
assert!(
elapsed < Duration::from_secs(15),
"halt observation took too long: {elapsed:?}"
@@ -1571,10 +1519,9 @@ mod tests {
halt.cancel();
let start = Instant::now();
// finish_with_halt drops tx (signalling EOF), then observes the
// pre-cancelled halt and enters the grace spin. The consumer
// finishes close() within 500ms, so finish_with_halt must join
// cleanly and return Ok with the correct total.
// finish_with_halt drops tx (EOF), observes the pre-cancelled halt, and
// enters the grace spin; the consumer finishes close() within 500ms, so
// it must join cleanly and return Ok with the correct total.
let res = pipe.finish_with_halt(Some(&halt));
let elapsed = start.elapsed();
@@ -1855,20 +1802,9 @@ mod tests {
// caller genuinely reaches the abandon decision with a close in flight.
let rel = release.clone();
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
//
// These intervals used to be 600 ms against a 300 ms grace, which
// left NO margin: two 300 ms windows end at 600 ms, and the 250 ms
// poll cadence can push the observation later still, so on a loaded
// runner the second window expired first and the caller abandoned —
// failing with Err(Halted) against a race, not a defect.
//
// Scaled up so the jitter is small relative to the intervals: the
// first window ends at ~1.0-1.25 s and the second at ~2.0-2.25 s,
// so releasing at 1.6 s sits well inside the second with roughly
// 350 ms of slack on either side. The ordering under test is
// unchanged; only the margin is.
// Release inside the second grace window. Old 600ms/300ms-grace intervals
// left no margin and flaked on loaded runners; scaled up so windows end
// at ~1.0-1.25s and ~2.0-2.25s, releasing at 1.6s for ~350ms slack.
thread::sleep(Duration::from_millis(1600));
rel.store(true, Ordering::SeqCst);
});
+4 -3
View File
@@ -3,7 +3,7 @@
//! The `libc` crate doesn't expose these symbols across all macOS SDK
//! versions, so we define them locally with values from
//! `/usr/include/sys/fcntl.h`. Two call sites (
//! [`crate::io::writeback_file`] and [`crate::io::sink::preallocate`])
//! [`crate::io::writeback_file`] and `crate::io::sink::preallocate`)
//! need the same constants and `fstore_t` layout — keeping a single
//! source of truth here prevents the two copies from drifting.
//!
@@ -16,8 +16,9 @@ pub(crate) const F_PREALLOCATE: libc::c_int = 42;
/// Anchor preallocation at the current physical EOF.
pub(crate) const F_PEOFPOSMODE: libc::c_int = 3;
/// Prefer a contiguous allocation. Try this first; on `EINVAL` (no
/// contiguous run of that size), fall back to `F_ALLOCATEALL`.
/// Prefer a contiguous allocation. Call sites OR this together with
/// `F_ALLOCATEALL` on the first attempt; on failure they fall back to
/// `F_ALLOCATEALL` alone.
pub(crate) const F_ALLOCATECONTIG: libc::c_uint = 0x0000_0002;
/// Allow non-contiguous allocation. Stronger guarantee than just
+3 -5
View File
@@ -94,11 +94,9 @@ pub(crate) fn open_for_mkv(
return Ok(Box::new(wf));
}
}
// Only Linux differentiates the sink by filesystem type (NFS gets
// the WritebackFile machinery); every other OS always uses
// `LocalFileSink`. Reference `detect` as a value (no call, no
// `statfs` syscall) so it isn't flagged dead on non-Linux while
// still avoiding the wasted probe whose result we'd discard.
// Only Linux differentiates the sink by filesystem type; other OSes always use
// `LocalFileSink`. Reference `detect` as a value (no call/syscall) so it isn't
// flagged dead on non-Linux while avoiding a wasted probe.
#[cfg(not(target_os = "linux"))]
let _ = crate::platform::fs_type::detect;
+3 -5
View File
@@ -18,11 +18,9 @@ pub(super) fn preallocate_impl(file: &File, size_bytes: u64) {
// would wrap a >= 2^63 size to a negative length.
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX) as libc::off_t;
let mut store = Fstore {
// Prefer a contiguous run but accept scattered extents to
// satisfy the full length. Without F_ALLOCATEALL the first
// attempt is best-effort and can return rc=0 with a partial
// allocation, so the fallback below would never fire. Matches
// writeback_file/macos.rs.
// Prefer contiguous but accept scattered extents. Without F_ALLOCATEALL
// the first attempt is best-effort and can return rc=0 with a partial
// allocation, so the fallback below would never fire. Matches writeback_file/macos.rs.
fst_flags: F_ALLOCATECONTIG | F_ALLOCATEALL,
fst_posmode: F_PEOFPOSMODE,
fst_offset: 0,
+15 -25
View File
@@ -56,12 +56,9 @@ impl SocketSink {
/// `("host", 1234)`, a `SocketAddr`, etc.
pub fn connect<A: ToSocketAddrs>(addr: A, sndbuf_bytes: Option<usize>) -> io::Result<Self> {
let stream = TcpStream::connect(addr)?;
// `set_nodelay(true)` keeps small writes (TS packet trains, fMP4
// moof headers) from sitting in Nagle's algorithm until the buffer
// fills. The BufWriter already absorbs syscall overhead; Nagle
// would just add latency without coalescing more. It is a latency
// hint, not a correctness requirement, so a platform that rejects
// TCP_NODELAY must not fail the connect — demote the error.
// `set_nodelay(true)` keeps small writes (TS packet trains, fMP4 moof headers)
// from sitting in Nagle's algorithm; BufWriter already absorbs syscall overhead.
// A latency hint, not a correctness requirement — a rejecting platform must not fail connect.
let _ = stream.set_nodelay(true);
if let Some(n) = sndbuf_bytes {
set_send_buffer(&stream, n)?;
@@ -119,10 +116,9 @@ impl UdpSocketSink {
///
/// `sndbuf_bytes`, when present, is a hint to `SO_SNDBUF`.
pub fn connect<A: ToSocketAddrs>(peer: A, sndbuf_bytes: Option<usize>) -> io::Result<Self> {
// Resolve the peer first so the local bind matches its address
// family. Binding `0.0.0.0:0` (IPv4) and then connecting to an
// IPv6 peer fails with EAFNOSUPPORT, so pick the wildcard that
// matches the resolved family.
// Resolve the peer first so the local bind matches its address family:
// binding `0.0.0.0:0` (IPv4) then connecting to an IPv6 peer fails
// with EAFNOSUPPORT.
let peer_addr = peer
.to_socket_addrs()?
.next()
@@ -164,12 +160,9 @@ impl SequentialSink for UdpSocketSink {
}
}
// ── Platform `SO_SNDBUF` tuning ────────────────────────────────────────────
//
// std's `TcpStream` / `UdpSocket` don't expose `SO_SNDBUF`. We drop to
// libc on Linux + macOS (the libc-dep targets in Cargo.toml). On other
// targets the hint is silently ignored — the socket still works, the
// kernel just picks its own send-buffer size.
// ── Platform `SO_SNDBUF` tuning ─────────────────────────────────────────
// std doesn't expose `SO_SNDBUF`; drop to libc on Linux + macOS. Other
// targets silently ignore the hint — socket still works, kernel picks its own size.
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn set_send_buffer(stream: &TcpStream, bytes: usize) -> io::Result<()> {
@@ -185,10 +178,9 @@ fn set_udp_send_buffer(socket: &UdpSocket, bytes: usize) -> io::Result<()> {
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
fn set_send_buffer(_stream: &TcpStream, _bytes: usize) -> io::Result<()> {
// Non-Linux-non-macOS targets aren't in Cargo.toml's libc dep list;
// silently ignore the hint rather than failing the connect. Callers
// can detect via the lack of an explicit "sndbuf applied" signal
// (not provided, intentionally — this is a hint, not a guarantee).
// Non-Linux-non-macOS targets aren't in Cargo.toml's libc dep list; silently
// ignore the hint rather than failing the connect (intentionally no
// "sndbuf applied" signal — this is a hint, not a guarantee).
Ok(())
}
@@ -265,11 +257,9 @@ mod tests {
});
let mut sink = SocketSink::connect(addr, None).unwrap();
_assert_seq(&mut sink);
// The negative is harder to assert directly (no `is_not<T>`),
// but `SocketSink` does not impl `Seek`, so it can't unify with
// `RandomAccessSink`'s super-bound. The Phase 2 blanket impl
// `impl<T: SequentialSink + Seek> RandomAccessSink for T {}` thus
// excludes it by construction.
// No `is_not<T>` to assert the negative directly, but `SocketSink` doesn't
// impl `Seek`, so it can't unify with `RandomAccessSink`'s super-bound —
// the blanket `impl<T: SequentialSink + Seek> RandomAccessSink for T {}` excludes it.
}
#[test]
+1 -1
View File
@@ -3,7 +3,7 @@
//! queue bounded. On macOS and Windows, a no-op stub.
//!
//! The platform decision lives entirely in this file (the cfg-gated
//! `pub use` below). Callers — and `DiskWriter` itself — are
//! `pub use` below). Callers — and `WritebackFile` itself — are
//! platform-independent.
#[cfg(target_os = "linux")]
+22 -34
View File
@@ -179,18 +179,16 @@ impl WritebackPipeline {
if pos < self.last_flush_pos.saturating_add(self.chunk_bytes) {
return;
}
// Byte offsets are unsigned throughout; the signed cast happens
// only at the libc call boundary where the kernel ABI requires
// `i64`. `saturating_sub` documents and hardens the line-above
// guard that `pos >= last_flush_pos`.
// Byte offsets are unsigned throughout; the signed cast happens only at the
// libc call boundary where the kernel ABI requires `i64`. `saturating_sub`
// hardens the line-above guard that `pos >= last_flush_pos`.
let chunk_off: u64 = self.last_flush_pos;
let chunk_len: u64 = pos.saturating_sub(self.last_flush_pos);
let mut wait_ms: u64 = 0;
let mut fadvise_ms: u64 = 0;
// Async kickoff for the just-completed chunk runs on every
// path (NFS, degraded, normal) — it's nominally non-blocking
// by spec and gives the kernel an early hint that this range
// is ready to flush.
// Async kickoff for the just-completed chunk runs on every path (NFS,
// degraded, normal) — non-blocking by spec, an early hint that this
// range is ready to flush.
let kickoff_rc = unsafe {
libc::sync_file_range(
self.fd,
@@ -211,16 +209,13 @@ impl WritebackPipeline {
}
if let Some((prev_off, prev_len)) = self.pending.take() {
if self.skip_wait() {
// NFS branch (or degraded fallback after a prior
// timeout): the WAIT_AFTER + DONTNEED dance is what
// hangs on NFS — skip it. We still advance `pending`
// so the next call has a stable cycle.
// NFS branch (or degraded fallback after a prior timeout): the
// WAIT_AFTER + DONTNEED dance hangs on NFS — skip it, but still
// advance `pending` so the next call has a stable cycle.
} else {
// Normal local-storage branch with belt-and-braces
// timeout. If WAIT_AFTER hangs > WAIT_AFTER_TIMEOUT
// we mark the pipeline degraded, log a loud error,
// and fall through to the skip path on subsequent
// calls.
// Normal local-storage branch with belt-and-braces timeout: if
// WAIT_AFTER hangs > WAIT_AFTER_TIMEOUT, mark degraded, log loudly,
// and fall through to the skip path on subsequent calls.
match wait_after_with_timeout(self.clone_for_worker(), self.fd, prev_off, prev_len)
{
Some(ms) => {
@@ -238,17 +233,13 @@ impl WritebackPipeline {
self.record_wait(wait_ms);
}
None => {
// Timeout branch: switch to NFS-style skip
// for the rest of the pipeline's life. Do
// NOT call DONTNEED — if WAIT_AFTER hasn't
// Timeout branch: switch to NFS-style skip for the rest of the
// pipeline's life. Do NOT call DONTNEED — if WAIT_AFTER hasn't
// returned, the pages aren't safely flushed.
self.degraded.store(true, Ordering::Relaxed);
// Once degraded we skip DONTNEED, so every subsequent
// chunk's pages stay resident until close — the same
// page-cache exposure profile as NFS. Shrink to the
// floor so that exposure window is as small as the NFS
// path keeps it, instead of whatever the adaptive sizing
// had grown chunk_bytes to (up to 256 MiB).
// Skipping DONTNEED leaves pages resident until close (same
// exposure as NFS), so shrink chunk_bytes to the floor rather
// than whatever adaptive sizing had grown it to (up to 256 MiB).
self.chunk_bytes = CHUNK_BYTES_MIN;
tracing::error!(
target: "mux",
@@ -294,10 +285,9 @@ impl WritebackPipeline {
if self.wait_after_window.len() < ADAPTIVE_WINDOW {
return;
}
// p95 index, derived from the window size so it stays valid if
// ADAPTIVE_WINDOW changes (a hard-coded `[14]` would panic OOB
// for a window <= 14). For the default 16 this is index 15
// (ceil(16 * 95 / 100) - 1 = 15), i.e. the top sample.
// p95 index, derived from window size so it stays valid if ADAPTIVE_WINDOW
// changes (a hard-coded `[14]` would panic OOB for a window <= 14). For the
// default 16 this is index 15, i.e. the top sample.
let mut sorted: Vec<u64> = self.wait_after_window.iter().copied().collect();
sorted.sort_unstable();
let p95_idx = (ADAPTIVE_WINDOW * 95).div_ceil(100).min(ADAPTIVE_WINDOW) - 1;
@@ -442,10 +432,8 @@ fn wait_after_with_timeout(
Err(crate::io::bounded::BoundedError::Timeout)
| Err(crate::io::bounded::BoundedError::Halted) => None,
Err(crate::io::bounded::BoundedError::WorkerLost) => {
// Worker thread spawn failed or panicked before sending.
// Treat as a benign success (no syscall ran) rather than
// a degrade trigger — falling through with elapsed_ms=0
// matches the no-op behaviour.
// Worker thread spawn failed or panicked before sending. Treat as benign
// success (no syscall ran), not a degrade trigger — elapsed_ms=0 matches no-op.
Some(0)
}
}
+3 -4
View File
@@ -17,10 +17,9 @@ use std::time::Duration;
/// Best-effort: a non-zero rc is logged but not propagated, since the
/// caller would just continue with the unreserved file anyway.
pub(super) fn preallocate(file: &File, size_bytes: u64) {
// FALLOC_FL_KEEP_SIZE = 0x01 — keep the reported file size at 0
// (writes grow it normally) while still pre-reserving the extents.
// Clamp to the signed `off_t` range; an unchecked `as i64` cast
// would wrap a >= 2^63 size to a negative length (EINVAL no-op).
// FALLOC_FL_KEEP_SIZE keeps the reported file size at 0 (writes grow it normally)
// while pre-reserving extents. Clamp to `off_t` range; an unchecked `as i64`
// cast would wrap a >= 2^63 size to a negative length (EINVAL no-op).
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX);
let rc = unsafe { libc::fallocate(file.as_raw_fd(), libc::FALLOC_FL_KEEP_SIZE, 0, len) };
tracing::debug!(
+3 -5
View File
@@ -215,11 +215,9 @@ mod tests {
"a halted F_FULLFSYNC must not be reported as a completed sync"
);
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each
// carries its own numeric code through the "E<code>" prefix that
// `From<Error> for io::Error` mints — the only shape `error_code`
// recognises. A bare `ErrorKind` cannot be classified, which is how a
// user cancel here used to read as a hard I/O failure.
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each carries its
// own numeric code via the "E<code>" prefix `From<Error> for io::Error` mints —
// a bare `ErrorKind` can't be classified, so a user cancel used to read as I/O failure.
let lost = bounded_failure_to_result(BoundedError::WorkerLost)
.expect_err("a lost F_FULLFSYNC worker must be an error");
assert!(
+12 -24
View File
@@ -143,8 +143,9 @@ impl WritebackFile {
/// Like [`Self::create`] but pre-reserves `size_bytes` of disk
/// space via the platform's extent-preallocation primitive (Linux
/// `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`, Windows
/// `SetFileValidData` stub). The reported file size is unchanged
/// `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`; Windows has no
/// equivalent, so it is a debug-logged no-op there). The reported
/// file size is unchanged
/// (writes still grow the file naturally) — only the on-disk extent
/// allocation is preallocated, which reduces extent fragmentation
/// on large sequential writes (mux output, especially on slow
@@ -228,18 +229,13 @@ impl Write for WritebackFile {
impl Seek for WritebackFile {
fn seek(&mut self, from: SeekFrom) -> io::Result<u64> {
let p = self.file.seek(from)?;
// Only treat seeks that actually move the position as
// boundaries — sweep does a redundant `seek(Current(pos))`
// before every write, and we don't want that to drain the
// pipeline on every iteration.
// Only treat seeks that actually move the position as boundaries — sweep does
// a redundant `seek(Current(pos))` before every write, which shouldn't drain
// the pipeline on every iteration.
if p != self.pos {
// Diagnostic for the NFS mux hang: the MKV format requires
// the muxer to seek back occasionally (cluster size
// patching, Cues index write, Segment header backpatch).
// Each such seek invalidates the writeback chunk tracking
// and forces a finalize → WAIT_AFTER on the in-flight
// chunk. Logging the seek delta lets us correlate hang
// offsets with specific muxer operations.
// Diagnostic for the NFS mux hang: MKV requires occasional backward seeks
// (cluster patching, Cues, Segment backpatch), each invalidating writeback
// chunk tracking. Logging the delta correlates hang offsets with muxer ops.
let from_pos = self.pos;
let to_pos = p;
let delta: i64 = (to_pos as i64).wrapping_sub(from_pos as i64);
@@ -274,17 +270,9 @@ impl super::sink::RandomAccessSink for WritebackFile {}
impl Drop for WritebackFile {
fn drop(&mut self) {
// Run the pipeline's tail finalize so the last in-flight chunk
// gets its `WAIT_AFTER` + `posix_fadvise(DONTNEED)`. Without
// this, callers that drop a `WritebackFile` without calling
// `sync_all` (panic, early-return, idiomatic `let _ = w;`)
// leave the trailing chunk in cache; the kernel still flushes
// on close, but the bounded-cache invariant fails at the tail.
// We deliberately do *not* call `self.file.sync_all()` here —
// close already triggers a flush, and an `fsync` from `Drop`
// would silently swallow its `io::Error` anyway. `finalize` is
// idempotent so an explicit `sync_all` followed by drop is
// still safe.
// Run the pipeline's tail finalize (WAIT_AFTER + DONTNEED); otherwise a drop
// without `sync_all` leaves the trailing chunk in cache. No `self.file.sync_all()`
// here — `Drop`-triggered fsync would swallow errors; `finalize` is idempotent.
self.pipeline.finalize();
}
}
+26 -67
View File
@@ -381,12 +381,9 @@ pub fn resolve_and_apply_traced(
let mut trace = crate::aacs::trace::ResolutionTrace::new();
// The FIRST source failure seen, if any. A source that returns `Err` did not
// answer "no key for this disc" — it could not answer at all — and that
// reason is stamped onto `disc.aacs_error` below so the decrypt gate reports
// THAT instead of the generic `NoDiscKey`. First-wins (not last) so the
// ordered sources' most-preferred failure is the one the operator is told
// about, matching the first-valid-wins rule for successes.
// The FIRST source failure seen, if any. An `Err` means the source could not
// answer at all (not "no key"), so its reason is stamped onto `disc.aacs_error`
// below, reporting the ordered sources' most-preferred failure to the operator.
let mut source_failure: Option<crate::error::Error> = None;
// The ctx parses Unit_Key_RO.inf at the stride for `inputs.version` (the
@@ -430,26 +427,9 @@ pub fn resolve_and_apply_traced(
outcome: KeyOutcome::NoKey,
});
}
// The source could NOT answer — it was unreachable, it errored, or it
// refused. Nothing is known about whether a key exists, so the path
// is EMPTY: recording `NoEntry` here is exactly the conflation that
// made a seven-hour run of HTTP 502s render as
// `key: online > no entry > NO KEY` + `E7022 No key source has a
// decryption key for this disc`, and sent operators hunting for a VUK
// that was never missing.
//
// The reason itself rides out on `disc.aacs_error` (below), the
// channel `Disc::ensure_decryptable_keys` already reads for the
// E7017-vs-E7022 split — so the decrypt gate raises the SOURCE's code
// (`KeyServiceUnavailable` / `KeyServiceUnauthorized` /
// `KeyServiceRateLimited`) instead of the generic `NoDiscKey`.
//
// `KeyOutcome` deliberately gains no variant: it is matched
// exhaustively by every front-end's trace renderer (freemkv's
// `pipe::render_resolution_trace`, autorip's
// `keysource::render_resolution_trace`), and this fix must not turn
// into a breaking change across four repos to say something the error
// code already says precisely.
// The source could NOT answer (unreachable/errored/refused), so path is
// EMPTY, not `NoEntry` — conflating the two once rendered 502s as "no
// key" and misled operators. The real reason rides on `disc.aacs_error`.
Err(e) => {
if source_failure.is_none() {
source_failure = Some(e);
@@ -462,10 +442,9 @@ pub fn resolve_and_apply_traced(
}
}
}
// Nothing resolved. If a source FAILED rather than answered, stamp that
// reason onto the disc so the decrypt gate can report it — but never clobber
// a reason the scan already captured (e.g. `AacsVidUnavailable`), which is
// closer to the disc itself than a source outage is.
// Nothing resolved. If a source FAILED rather than answered, stamp that reason
// onto the disc for the decrypt gate — but never clobber a reason the scan
// already captured (e.g. `AacsVidUnavailable`), which is closer to the disc.
if let Some(e) = source_failure
&& disc.aacs_error.is_none()
{
@@ -573,22 +552,9 @@ pub fn key_fetch(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
) -> crate::sector::KeyFetch {
// One driver behind both operations: rebuild the sources, inject `samples`
// as the ctx's content samples, run `drive` (the per-kind fetch), map the
// resolved UnitKeys to raw keys. Memoized by the fingerprint of the sample
// batch: the resolved keys are disc-level (a clip's index / CPS keys are
// identical for every title that references it), so the first batch resolves
// over the network and every repeat is answered from the cache with no
// request. A GENUINELY-empty reply (every source ran and none held the key)
// is cached too — the key the service lacks for a batch won't appear on a
// re-ask, so re-hitting the network buys nothing. But an empty reply caused
// by a source FAILURE (network down, source unreachable) is NOT cached: that
// is a transient miss, and caching it would permanently drop a unit that
// could be recovered once the source recovers — the `errored` flag on
// `FetchOutcome` draws exactly that line. Each operation gets its OWN cache:
// a base batch and a forensic anchor never collide, and the same bytes could
// legitimately resolve differently per op.
// The per-kind driver: `drive_unit_keys` or `drive_fmts_indexes`.
// One driver behind both operations, memoized per sample-batch fingerprint
// (keys are disc-level, so repeats hit cache). A genuinely-empty reply is
// cached, but a source FAILURE is not — a transient miss (see `errored`).
type FetchDriver = fn(&[Box<dyn KeySource>], &dyn ResolveCtx) -> FetchOutcome;
fn make_op(
inputs: DiscInputs,
@@ -614,9 +580,8 @@ pub fn key_fetch(
let mut di = inputs.clone();
di.samples = samples.to_vec();
// Parse Unit_Key_RO.inf at the disc's OWN stride (carried on `inputs`):
// an online /decode reply that returns a VUK (not a terminal UK) then
// derives unit keys from `enc_title_keys`, which a V10 disc parses at
// the 48-byte stride — hardcoding the V20 stride here corrupted them.
// a V10 disc parses `enc_title_keys` at the 48-byte stride, so
// hardcoding the V20 stride here would corrupt the derived unit keys.
let ctx = DiscInputsCtx::new(&di);
let outcome = drive(&sources, &ctx);
let keys: Vec<[u8; 16]> = outcome.keys.into_iter().map(|u| u.key).collect();
@@ -666,11 +631,9 @@ pub fn read_encrypted_units(
) -> Vec<Vec<u8>> {
use crate::aacs::content::{ALIGNED_UNIT_LEN, ALIGNED_UNIT_SECTORS, aacs_unit_encrypted};
const CHUNK_UNITS: u32 = 15; // 45 sectors/read — under the drive transfer cap
// Probe several evenly-spaced points across EACH extent rather than only the
// midpoint-and-forward: a title whose encrypted feature starts late, or whose
// midpoint lands in a clear nav stretch, must STILL yield scrambled samples.
// Empty samples make `Disc::decrypt_with` skip wrong-key validation, so a
// real encrypted title returning nothing here is a silent wrong-key hazard.
// Probe several evenly-spaced points across EACH extent, not just midpoint
// forward: a title starting late or landing in a clear nav stretch must still
// yield samples — empty samples make `decrypt_with` skip wrong-key validation.
const PROBES_PER_EXTENT: u32 = 8;
let mut out: Vec<Vec<u8>> = Vec::new();
@@ -687,20 +650,17 @@ pub fn read_encrypted_units(
continue;
}
let units_this = CHUNK_UNITS.min(total_units - unit);
// Saturate: start_lba comes from attacker-controlled UDF/MPLS
// extents; a malformed extent near u32::MAX would otherwise panic
// (debug) or wrap to a wrong LBA (release). An over-capacity LBA then
// fails cleanly via the read_sectors().is_err() skip below.
// Saturate: start_lba comes from attacker-controlled UDF/MPLS extents;
// near u32::MAX it would otherwise panic (debug) or wrap (release).
// An over-capacity LBA fails cleanly via the is_err() skip below.
let lba = ext
.start_lba
.saturating_add(unit.saturating_mul(ALIGNED_UNIT_SECTORS));
let count = (units_this * ALIGNED_UNIT_SECTORS) as u16;
let mut buf = vec![0u8; count as usize * 2048];
// `false` = no recovery retries; the reader is the raw drive/file
// (no decrypt decorator), so these are the on-disc encrypted bytes. A
// read error at one probe skips THAT probe only — it must not abandon
// the rest of the extent (the old `break` blinded the sampler on a
// single transient miss).
// `false` = no recovery retries; reader is the raw drive/file (no
// decrypt decorator). A read error skips only THAT probe — it must not
// abandon the rest of the extent (the old `break` blinded the sampler).
if reader.read_sectors(lba, count, &mut buf, false).is_err() {
continue;
}
@@ -1298,10 +1258,9 @@ mod tests {
use crate::error::Result;
use crate::sector::SectorSource;
// Even units: CPI-clear (byte0 & 0xC0 == 0) AND sync-destroyed (no 0x47).
// Odd units: CPI-set (byte0 = 0xC0) with a scrambled body.
// Neither has clean TS syncs, so `is_clean` is FALSE for BOTH;
// `aacs_unit_encrypted` flags only the odd units.
// Even units: CPI-clear, sync-destroyed. Odd units: CPI-set, scrambled body.
// Neither has clean TS syncs (`is_clean` is FALSE for both), but
// `aacs_unit_encrypted` must flag only the odd (CPI-set) units.
struct MixSource {
ext_start: u32,
total_units: u32,
+18 -25
View File
@@ -47,7 +47,9 @@ pub struct DiscMetadata {
pub titles: BTreeMap<String, String>,
/// First-line / short description, per lang
pub descriptions: BTreeMap<String, String>,
/// Disc N of M for box sets (None if not a box set)
/// Disc N of M, returned whenever both `<di:discNumber>` and
/// `<di:numSets>` are present and sane (including `(1, 1)` for a
/// single-disc release); `None` if either is missing or invalid.
pub disc_number: Option<(u32, u32)>,
}
@@ -76,10 +78,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<DiscMetadata>
let Some(lang) = lang_code_from_filename(&entry.name) else {
continue;
};
// entry.size is attacker-controlled UDF metadata and flows into
// a Vec::with_capacity in read_file. A real BDMV bdmt XML is a
// few KB; cap well above that so a crafted multi-GB size can't
// trigger a huge allocation before any parsing.
// entry.size is attacker-controlled and flows into a Vec::with_capacity
// in read_file. Cap well above a real few-KB bdmt XML so a crafted
// multi-GB size can't trigger a huge allocation before parsing.
if !bdmt_size_acceptable(entry.size) {
continue;
}
@@ -178,11 +179,9 @@ fn looks_like_xml(s: &str) -> bool {
/// helpers are case- and namespace-insensitive, so callers pass the
/// bare local name (no `di:` prefix).
fn extract_title(xml_text: &str) -> Option<String> {
// Order matches the module-level convention: <di:name> first
// (Paramount-style), then <di:title>, then the nested
// tableOfContents/titleName form.
// xml::text already trims its result, so an empty string after
// extraction means a genuinely empty element.
// Priority: <di:name> (Paramount-style), then <di:title>, then nested
// tableOfContents/titleName. xml::text trims, so an empty string here
// means a genuinely empty element.
for tag in ["name", "title"] {
if let Some(s) = xml::text(xml_text, tag)
&& !s.is_empty()
@@ -351,10 +350,9 @@ mod tests {
#[test]
fn multiple_languages_keyed_correctly() {
// Simulate driving parse_bdmt_xml from two synthetic XML
// blobs and aggregating into DiscMetadata the same way parse()
// would. This exercises the BTreeMap key handling without
// needing a UdfFs.
// Drive parse_bdmt_xml from two synthetic XML blobs and aggregate
// into DiscMetadata like parse() would, exercising BTreeMap key
// handling without needing a UdfFs.
let eng_xml = r#"<discInfo xmlns:di="urn:BDA:bdmv;disclibmeta">
<di:name>Aurora Drift</di:name>
</discInfo>"#;
@@ -395,12 +393,9 @@ mod tests {
#[test]
fn malformed_xml_returns_none() {
// Random gibberish has no recognizable title element. We
// document the contract: parse_bdmt_xml returns None, and
// parse() (the caller) skips the file. Aggregating across
// zero files leaves DiscMetadata::default() — which parse()
// surfaces as None to its caller. Either is documented as
// acceptable per the module spec.
// Gibberish has no recognizable title element: parse_bdmt_xml returns
// None and parse() skips the file. Zero files aggregated leaves
// DiscMetadata::default(), which parse() surfaces as None.
let bad = "this is not xml &&& <<< nope";
assert!(parse_bdmt_xml(bad).is_none());
@@ -411,11 +406,9 @@ mod tests {
#[test]
fn description_with_only_child_xml_is_dropped() {
// Real-world bug: <di:description> contained only
// <di:thumbnail/> child elements with no actual prose. The
// previous parser surfaced the raw XML fragment as the
// description string. Now we reject candidates that begin
// with `<`.
// Real-world bug: <di:description> contained only <di:thumbnail/>
// children with no prose, and the old parser surfaced the raw XML
// fragment as the description. Now candidates starting with `<` are rejected.
let xml = r#"<discInfo>
<di:name>Skyline Run</di:name>
<di:description>
+109 -67
View File
@@ -10,10 +10,9 @@
//! for any label parser that needs structured access to .class files
//! inside a `/BDMV/JAR/<x>.jar`.
// Foundation module — public API is staged for `labels::deluxe` (which
// will exercise the bytecode walker) and `labels::dbp`'s refactor onto
// the constant-pool iterator. The dead-code allow comes off as those
// callers land. Tests below cover the API in isolation.
// Foundation module — public API is staged for `labels::deluxe` (bytecode
// walker) and `labels::dbp`'s refactor onto the constant-pool iterator.
// The dead-code allow comes off once those callers land.
#![allow(dead_code)]
const CLASS_MAGIC: u32 = 0xCAFEBABE;
@@ -32,10 +31,9 @@ pub enum Error {
BadInstruction { pc: usize, opcode: u8 },
}
// No Display/std::error::Error impl: this is a crate-internal, typed error
// used only for `match`/`?` within the label parsers (callers discard it via
// `let Ok(_) = ... else continue`). Per the library's zero-English rule there
// is no user-facing text; the variant fields carry the structured detail.
// No Display/std::error::Error impl: crate-internal typed error for `match`/`?`
// within label parsers. Per the zero-English rule there is no user-facing
// text; variant fields carry the structured detail.
pub type Result<T> = std::result::Result<T, Error>;
@@ -155,7 +153,8 @@ impl ConstantPool {
}
/// For `ldc` / `ldc_w` operands: resolve a constant-pool index to a
/// best-effort string. Supports Utf8, String, Integer, Class.
/// best-effort string. Supports Utf8, String, Integer, Float, Long,
/// Double, and Class.
pub fn load_constant_display(&self, index: u16) -> Option<String> {
Some(match self.get(index)? {
CpInfo::Utf8(s) => format!("utf8:{:?}", s),
@@ -588,7 +587,7 @@ impl Instruction<'_> {
/// ldc_w, ldc2_w, new, getstatic, putstatic, getfield, putfield,
/// invokevirtual, invokespecial, invokestatic, invokeinterface,
/// invokedynamic, checkcast, instanceof, anewarray, multianewarray,
/// ldc with cp index in operands[0] — return the index. Returns
/// ldc with cp index in `operands[0]` — return the index. Returns
/// None for opcodes whose operand is not a CP index.
pub fn cp_index(&self) -> Option<u16> {
match self.opcode {
@@ -657,10 +656,9 @@ fn instruction_size(code: &[u8], pc: usize) -> Option<usize> {
if high < low {
return None;
}
// `high - low + 1` can overflow i32 for adversarial bytecode
// (e.g. low=i32::MIN/high=0, or low=0/high=i32::MAX), so widen
// to i64 before adding. The product and final sum are saturating
// so they cannot overflow usize on a 32-bit target either.
// `high - low + 1` can overflow i32 for adversarial bytecode, so widen
// to i64 before adding; the product/sum below saturate so they can't
// overflow usize on a 32-bit target either.
let entries = (high as i64 - low as i64 + 1) as u64;
let table_bytes = entries.saturating_mul(4);
let base = (padded_start - pc + 12) as u64;
@@ -989,14 +987,9 @@ impl<'a> Reader<'a> {
}
fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> {
// `n` is attacker-supplied: it comes from a JVMS `u4` attribute_length
// / code_length (§4.7, §4.7.3) or a `u2` Utf8 length (§4.4.7). Unlike
// the fixed-width readers above, whose `self.pos + k` cannot leave the
// buffer's own address range, `self.pos + n` can wrap — on a 32-bit
// target a `u4` length near 0xFFFF_FFFF plus a non-zero `pos` panics
// in debug and in release wraps to a SMALL end offset that passes the
// bounds check, after which the slice index itself panics. Checked, so
// an out-of-range length is the EOF error it always should have been.
// `n` is attacker-supplied (a JVMS u4/u2 length), so unlike the fixed-width
// readers above, `self.pos + n` can wrap on a 32-bit target and pass a naive
// bounds check. checked_add turns an out-of-range length into an EOF error.
let Some(end) = self.pos.checked_add(n) else {
return Err(Error::UnexpectedEof { needed });
};
@@ -1130,10 +1123,9 @@ mod tests {
#[test]
fn instruction_size_tableswitch_overflow_does_not_panic() {
// Adversarial low/high spanning the full i32 range. `high - low + 1`
// overflows i32; the widened i64 count then saturates the byte
// products. Must return a value (possibly None on a 32-bit usize)
// without panicking.
// Adversarial low/high spanning the full i32 range overflows `high - low + 1`
// in i32; the widened i64 count then saturates the byte products. Must
// return a value (possibly None on a 32-bit usize) without panicking.
for (low, high) in [
(i32::MIN, 0i32),
(0i32, i32::MAX),
@@ -1225,14 +1217,9 @@ mod tests {
assert_eq!(i.cp_index(), None);
}
// ── Robustness smoke tests ──────────────────────────────────────────────
//
// ClassFile::parse must NEVER panic on adversarial input, only
// return Err. These tests feed a battery of malformed byte
// sequences and assert Err results — they're the lightweight
// alternative to a full cargo-fuzz target (which would need
// nightly + separate crate). If we adopt cargo-fuzz later, these
// tests stay as deterministic regression cases.
// ── Robustness smoke tests ── ClassFile::parse must NEVER panic on
// adversarial input, only return Err. Lightweight alternative to a full
// cargo-fuzz target; stays useful as deterministic regression cases.
/// Tiny pseudo-random byte generator — deterministic + reproducible
/// without needing a `rand` dep. xorshift64*; good enough for
@@ -1306,10 +1293,9 @@ mod tests {
#[test]
fn parse_does_not_panic_on_random_bytes() {
// 200 deterministic-pseudo-random byte buffers of varying
// lengths. The contract: never panic, only return Err (or in
// the vanishingly unlikely case of a coincidentally-valid
// buffer, Ok — we don't assert one or the other).
// 200 deterministic pseudo-random buffers of varying lengths.
// Contract: never panic, only return Err (or Ok in the vanishingly
// unlikely coincidentally-valid case — we don't assert which).
let mut state: u64 = 0xDEADBEEF_DEADBEEF;
for _ in 0..200 {
let len = (xorshift(&mut state) % 256) as usize;
@@ -1325,10 +1311,9 @@ mod tests {
#[test]
fn parse_does_not_panic_on_valid_magic_random_tail() {
// 100 buffers that start with the magic + plausible
// minor/major but have garbage afterwards. These are the
// most adversarial — they pass the magic check and then
// exercise every other parser path.
// 100 buffers with valid magic + plausible minor/major but garbage
// afterwards — the most adversarial, passing the magic check then
// exercising every other parser path.
let mut state: u64 = 0xCAFEBABE_DEADBEEF;
for _ in 0..100 {
let mut buf = vec![0xCA, 0xFE, 0xBA, 0xBE, 0x00, 0x00, 0x00, 0x34];
@@ -1389,14 +1374,9 @@ mod tests {
}
}
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness
//
// These exercise plain data accessors on an already-parsed pool
// (built via the test-only `from_entries` constructor) — not the
// untrusted-bytes parsing path, just "does the right variant map to
// the right Option value."
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness: exercise data accessors
// on an already-parsed pool (test-only `from_entries` ctor), checking each
// variant maps to the right Option value.
fn sample_pool() -> ConstantPool {
// index: 0=Empty (reserved), 1=Utf8("Hello"), 2=Integer(42),
@@ -1547,14 +1527,9 @@ mod tests {
assert_ne!(cf.member_descriptor(&m), Some("doStuff"));
}
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness
//
// Mirrors `slice_boundary_is_inclusive_of_the_final_byte`: an
// exact-fit read must succeed, one byte short must fail. Plus
// positive-value tests so a scrambled byte assembly (not just an
// out-of-bounds read) would be caught.
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness: an exact-fit read
// succeeds, one byte short fails, plus positive-value tests so a
// scrambled byte assembly (not just OOB) is caught.
#[test]
fn u16_boundary_is_inclusive_of_the_final_byte() {
@@ -1656,10 +1631,9 @@ mod tests {
#[test]
fn constant_pool_long_entry_occupies_two_slots_via_real_parse() {
// Real class-file bytes (not the `from_entries` synthetic ctor):
// magic + minor/major + cp_count=4 + tag=5 (Long, 8-byte payload
// at index 1, reserved slot at index 2) + tag=1 (Utf8 at index 3)
// + empty access_flags/this/super/interfaces/fields/methods/attrs.
// Real class-file bytes (not the from_entries ctor): magic, minor/major,
// cp_count=4, tag=5 (Long, payload at index 1, reserved slot 2),
// tag=1 (Utf8 at index 3), then empty header tail sections.
let mut buf = vec![
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, // minor
@@ -1691,12 +1665,9 @@ mod tests {
}
}
// -----------------------------------------------------------------
// instruction_size: tableswitch/lookupswitch with non-degenerate
// low/high/npairs (the existing tests only cover low==high==0 and
// npairs==0, which can't distinguish `-` from `+` in the entry-count
// arithmetic).
// -----------------------------------------------------------------
// low/high/npairs — existing tests only cover the all-zero case, which
// can't distinguish `-` from `+` in the entry-count arithmetic.
#[test]
fn instruction_size_tableswitch_non_degenerate_range() {
@@ -1723,4 +1694,75 @@ mod tests {
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
// ── Malformed-input hardening: every count/length/index is attacker- ──────
// controlled, so an oversized or lying field must yield Err/None, never a
// panic or an unbounded allocation.
/// A `constant_pool_count` far larger than the data behind it must fail
/// cleanly (running out of bytes), not pre-allocate gigabytes or panic. The
/// count is a u16, so `Vec::with_capacity` is bounded at 65535 regardless.
#[test]
fn parse_rejects_oversized_constant_pool_count() {
let bytes = [
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, 0x00, 0x00, // minor / major
0xFF, 0xFF, // constant_pool_count = 65535, but no entries follow
];
assert!(matches!(
ClassFile::parse(&bytes),
Err(Error::UnexpectedEof { .. })
));
}
/// An oversized `interfaces_count` / `fields_count` with no data behind it
/// must fail on the first missing element, not panic.
#[test]
fn parse_rejects_oversized_interface_count() {
let bytes = [
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, 0x00, 0x00, // minor / major
0x00, 0x01, // constant_pool_count = 1 (no entries)
0x00, 0x00, // access_flags
0x00, 0x00, // this_class
0x00, 0x00, // super_class
0xFF, 0xFF, // interfaces_count = 65535, none follow
];
assert!(matches!(
ClassFile::parse(&bytes),
Err(Error::UnexpectedEof { .. })
));
}
/// Out-of-range constant-pool indices resolve to `None` from every
/// accessor a label parser drives — no slice panic.
#[test]
fn constant_pool_out_of_range_index_is_none_everywhere() {
let pool = sample_pool();
let oob = 9999u16;
assert!(pool.get(oob).is_none());
assert!(pool.utf8(oob).is_none());
assert!(pool.class_name(oob).is_none());
assert!(pool.string(oob).is_none());
assert!(pool.integer(oob).is_none());
assert!(pool.member_ref(oob).is_none());
assert!(pool.load_constant_display(oob).is_none());
}
/// A `Code` attribute whose declared `code_length` runs past the attribute
/// body must be rejected, not read out of bounds.
#[test]
fn code_attribute_rejects_length_past_its_body() {
// max_stack(2) + max_locals(2) + code_length(4) then a code_length that
// exceeds the remaining bytes.
let mut info = Vec::new();
info.extend_from_slice(&0u16.to_be_bytes()); // max_stack
info.extend_from_slice(&0u16.to_be_bytes()); // max_locals
info.extend_from_slice(&0xFFFF_FFFFu32.to_be_bytes()); // code_length: absurd
info.extend_from_slice(&[0x00, 0x01]); // only 2 code bytes present
assert!(matches!(
parse_code_attribute(&info),
Err(Error::UnexpectedEof { .. })
));
}
}
+3 -4
View File
@@ -116,10 +116,9 @@ impl ClpiVsMplsAudit {
/// merged view. Missing files (read errors, parse failures) are
/// silently skipped — this is diagnostic, not correctness-critical.
pub fn audit(reader: &mut dyn SectorSource, udf: &UdfFs) -> ClpiVsMplsAudit {
// Aggregate by PID across all CLPI files. If a PID appears in
// multiple clips (typical — main movie clip + trailers reference
// the same audio stream PIDs), first encountered wins (they should
// all agree per BD spec).
// Aggregate by PID across all CLPI files. A PID commonly repeats across
// clips (main movie + trailers sharing audio PIDs); first encountered
// wins, since they should all agree per BD spec.
let mut clpi_by_pid: BTreeMap<u16, (u8, String)> = BTreeMap::new();
if let Some(dir) = udf.find_dir("/BDMV/CLIPINF") {
let names: Vec<String> = dir
+4 -7
View File
@@ -101,10 +101,8 @@ fn assign_stream_numbers(
const NUMBER_SPACE_END: u32 = u16::MAX as u32 + 1;
// Numbers already claimed by the map, per type. A map value of 0 is NOT a
// claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable.
// Treat 0 as "unmapped" here (defense in depth — parse_playback_config also
// filters it) so such a stream gets a real synthesized number instead of an
// orphan 0 that collides with / shadows a genuine stream 1.
// claim (apply_labels binds 1-based numbers, so 0 is unmatchable); treat it
// as unmapped so the stream gets a real number instead of colliding with 1.
let mut taken_audio: Vec<u16> = Vec::new();
let mut taken_sub: Vec<u16> = Vec::new();
for info in infos {
@@ -646,9 +644,8 @@ mod tests {
assert_eq!(map.get("sub1").copied(), Some(3));
}
/// Spec: high confidence is returned when streamproperties.xml is fully
/// structured (no fallback). This is the Criterion parser's claim.
/// Mutation: change to ParseResult::medium → confidence assertion fails.
/// Spec: `LangInfoID` values are lowercased so they match
/// `apply_labels`' lookup (an uppercase "ENG" must parse as "eng").
#[test]
fn parse_stream_infos_language_lowercased() {
// LangInfoID values must be lowercased so they match apply_labels' lookup.
+15 -23
View File
@@ -56,11 +56,9 @@ fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
label.name = mb_match.name.clone();
}
}
// Append any menu_base-only stream (present in mb but not in ls by
// (stream_type, stream_number)). Without this the both-files path
// silently drops streams the menu_base-only path would have emitted:
// language_streams is authoritative for type/purpose but is not
// necessarily a superset of menu_base.
// Append menu_base-only streams (present in mb but not in ls by
// (stream_type, stream_number)); language_streams is authoritative for
// type/purpose but not necessarily a superset of menu_base.
for mb_label in mb {
let already = result.iter().any(|l| {
l.stream_type == mb_label.stream_type && l.stream_number == mb_label.stream_number
@@ -197,11 +195,9 @@ fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
"csp" | "cs" | "lsp" | "ls" | "cf" | "pf" | "bp" | "pp" => {
variant_code = variant.clone();
}
// Everything else: defer to vocab::codec as the single
// source of codec-name truth. If it recognizes the token
// (returns something other than the input) it's a known
// codec — store the canonical name. Otherwise it's an
// unknown token, stored as-is.
// Everything else: defer to vocab::codec as the single source of
// codec-name truth — a known token gets its canonical name,
// otherwise it's stored as-is.
_ => codec_hint = vocab::codec(&variant).to_string(),
}
}
@@ -222,10 +218,9 @@ fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
labels
}
// NOTE: `parse_menu_base` / `parse_menu_base_text` are defined just below this
// module and structurally belong above it. They are left in place (with the
// lint allowed) rather than relocated here — a ~120-line block move that is
// safer to do as its own focused change than inline.
// NOTE: `parse_menu_base` / `parse_menu_base_text` structurally belong above this
// module but are left below (lint allowed) — a ~120-line move is safer as its
// own focused change.
#[allow(clippy::items_after_test_module)]
#[cfg(test)]
mod tests {
@@ -311,10 +306,9 @@ mod tests {
#[test]
fn dual_flag_entry_resolves_to_audio_with_no_subtitle_qualifier() {
// An entry tripping BOTH flags (audio_ prefix sets is_audio,
// class "SubtitleButton" sets is_subtitle). Audio wins the type,
// and the subtitle qualifier (SDH) must NOT be carried onto the
// resulting Audio label. Regression for the type/qualifier split.
// An entry tripping BOTH flags (audio_ prefix sets is_audio, class
// "SubtitleButton" sets is_subtitle): audio wins the type, and the
// subtitle qualifier (SDH) must NOT carry onto the Audio label.
let labels = parse_props(
"audio_1.class=SubtitleButton\n\
audio_1.streamNumber=6\n\
@@ -885,11 +879,9 @@ fn parse_menu_base_text(text: &str) -> Vec<StreamLabel> {
let name = props.get("name").cloned().unwrap_or_default();
// Purpose: ask vocab first (word-boundary matched — avoids the
// "Commenter" false positive the prior `name.contains("comment")`
// had). Then fall back to the structural prefix check
// (`audio_commentary.foo`-style keys group commentary streams
// regardless of display name).
// Ask vocab first (word-boundary matched, avoiding the "Commenter"
// false positive of the old `name.contains("comment")`), then fall
// back to the structural prefix check (`audio_commentary.foo`-style).
let purpose = match vocab::purpose(&name) {
LabelPurpose::Normal if prefix_is_commentary(prefix) => LabelPurpose::Commentary,
p => p,
+3 -5
View File
@@ -67,11 +67,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
// BTreeMap so we keep the highest-numbered (last-written) label
// for each stream slot deterministic across runs. The same
// TextField,Audio1,... string can appear in multiple classes
// (button-state variants, localization fallbacks). Last write
// wins — they should all agree, but the structure is defensive.
// BTreeMap keeps the last-written label per stream slot deterministic.
// The same TextField,Audio1,... string can appear in multiple classes
// (button-state variants, fallbacks); last write should agree, but wins defensively.
let mut audios: BTreeMap<u16, String> = BTreeMap::new();
let mut subs: BTreeMap<u16, String> = BTreeMap::new();
+447 -207
View File
@@ -48,16 +48,34 @@
//!
//! ## Confidence
//!
//! [`parse`] returns `Some(ParseResult::medium(labels))` when Phases A
//! through D produce at least one stream — `Medium` because the
//! signal-to-label mapping is heuristic. `None` when the disc isn't
//! Deluxe-authored or when decoding produces zero streams (a
//! recognized-but-broken state that the analyzer still surfaces via
//! `parsers_detected`).
//! [`parse`] returns `Some(ParseResult::high(labels))` when Phases A
//! through D produce at least one stream: the master enums matched their
//! framework-stable fingerprints (a strong ordered-prefix signature), the
//! binding class decoded, and at least one per-stream binding resolved to a
//! real (language, purpose, codec) tuple — the schema was fully recovered,
//! not guessed. `None` when the disc isn't Deluxe-authored or when decoding
//! produces zero streams (a recognized-but-broken state that the analyzer
//! still surfaces via `parsers_detected`).
//!
//! ## Confirmed studio variants
//!
//! - **Universal** (`studio="uni"`): Language enum `pd` (65 values,
//! `English, French, Spanish, Dutch, …`), Purpose enum `lp`
//! (`Normal, Commentary, PiP, Trivia, Descriptive, Score`), audio binding
//! `np.<init>(I, Lpd;, Llp;, Lorg/bluray/ti/CodingType;)`, subtitle binding
//! `wb.<init>(I, Lpd;, Llp;, Lmi;)`, all built in one binding class's
//! `<clinit>` alongside a title-wrapper object whose constructor takes the
//! per-stream arrays (filtered out by the array-parameter guard in the
//! decoder). SDH/RNIB is encoded as a distinct Language VALUE
//! ("English SDH", "English RNIB"), recovered into the qualifier from the
//! name. Grounded on fixtures captured from a real Universal Blu-ray release.
//! - **Disney/Warner**: 70-value Language enum, same binding shape; the
//! original corpus this parser was written against.
use super::class_reader::{
AASTORE, BIPUSH, ClassFile, CodeAttribute, ConstantPool, CpInfo, GETSTATIC, ICONST_0, ICONST_1,
ICONST_2, ICONST_3, ICONST_4, ICONST_5, ICONST_M1, INVOKESPECIAL, LDC, LDC_W, NEW, SIPUSH,
ICONST_2, ICONST_3, ICONST_4, ICONST_5, ICONST_M1, INVOKESPECIAL, LDC, LDC_W, NEW, PUTSTATIC,
SIPUSH,
};
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, jar, vocab};
use crate::sector::SectorSource;
@@ -65,10 +83,9 @@ use crate::udf::UdfFs;
use std::collections::{BTreeMap, HashMap, HashSet};
pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
// The real signal is `com/bydeluxe/` inside a top-level jar's central
// directory. With a reader in detect we check it directly (cheap
// central-directory scan, no bytecode walk) so this parser claims only
// Deluxe discs; `parse()` repeats the check.
// Real signal is `com/bydeluxe/` in a jar's central directory: a cheap
// scan, no bytecode walk, so this parser claims only Deluxe discs.
// `parse()` repeats the check.
jar::for_each_jar(reader, udf, |_entry, archive| {
jar::has_path_prefix(archive, "com/bydeluxe/").then_some(())
})
@@ -76,6 +93,11 @@ pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
}
pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult> {
// Per-studio dispatch signal: Deluxe authored the framework for several
// studios, each shipping `/BDMV/JAR/<n>/config.xml` naming the studio.
// Matching below is studio-agnostic, so this is informational only (logged).
let studio = detect_studio(reader, udf);
jar::for_each_jar(reader, udf, |entry_name, archive| {
if !jar::has_path_prefix(archive, "com/bydeluxe/") {
return None;
@@ -103,10 +125,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// Build a fast-lookup table for Phase D's bytecode decoder.
let master_table = MasterEnumTable::from(&enums);
// Phase C — find ALL binding-class candidates (audio + subtitle
// are often split across two classes on Deluxe). Each gets its
// own `<clinit>` walk; constructions union into a single
// stream list for interpret_streams.
// Phase C — find ALL binding-class candidates (audio + subtitle often
// split across two classes on Deluxe). Each gets its own `<clinit>`
// walk; constructions union into a single stream list.
let binding_classes = find_binding_classes(archive, &master_table.class_name_set());
if binding_classes.is_empty() {
tracing::info!(
@@ -150,24 +171,80 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
tracing::info!(
jar = ?entry_name,
studio = ?studio,
audio = labels.iter().filter(|l| l.stream_type == StreamLabelType::Audio).count(),
subtitle = labels.iter().filter(|l| l.stream_type == StreamLabelType::Subtitle).count(),
"deluxe emitted labels",
);
// Medium confidence: Phase D's signal-to-label mapping is a
// documented heuristic until corpus-disc bytecode confirms
// the exact binding pattern.
Some(ParseResult::medium(labels))
// High confidence: master enums matched, binding class decoded, and at
// least one binding resolved to a real (language, purpose, codec) tuple.
// Un-named STN slots are back-filled from the MPLS floor by the registry.
Some(ParseResult::high(labels))
})
}
/// Read the studio identifier from a Deluxe disc's `config.xml`
/// (`<TitleConfig studio="uni"…>`), if present. The file lives under a
/// numbered subdirectory of `/BDMV/JAR/` (e.g. `/BDMV/JAR/99999/config.xml`),
/// so this scans the subdirectories for the first one that carries it.
///
/// Returns the lowercased studio token (`"uni"`, `"fox"`, `"wb"`, …) or `None`
/// when no readable `config.xml` names a studio.
fn detect_studio(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<String> {
let dir = udf.find_dir("/BDMV/JAR")?;
for entry in &dir.entries {
if !entry.is_dir {
continue;
}
let path = format!("/BDMV/JAR/{}/config.xml", entry.name);
let Ok(bytes) = udf.read_file(reader, &path) else {
continue;
};
if let Some(studio) = parse_studio_attr(&bytes) {
return Some(studio);
}
}
None
}
/// Extract the `studio="…"` attribute value from a `config.xml` byte buffer.
/// Deliberately a tolerant substring scan rather than a full XML parse: the
/// file is tiny attacker-controlled disc metadata and the only field of
/// interest is this one attribute. Rejects an empty or implausibly long value.
fn parse_studio_attr(xml: &[u8]) -> Option<String> {
const MAX_STUDIO_LEN: usize = 32;
let text = std::str::from_utf8(xml).ok()?;
let anchor = text.find("studio")?;
let after_key = &text[anchor + "studio".len()..];
// Skip whitespace / '=' up to the opening quote.
let open = after_key.find(['"', '\''])?;
let quote = after_key.as_bytes()[open];
let rest = &after_key[open + 1..];
let close = rest.find(quote as char)?;
let val = rest[..close].trim();
if val.is_empty() || val.len() > MAX_STUDIO_LEN {
return None;
}
Some(val.to_ascii_lowercase())
}
/// One identified master enum class.
#[derive(Debug)]
pub(crate) struct MasterEnum {
/// Obfuscated class name (e.g. `be.class`, `aw.class`).
pub class_name: String,
/// Ordinal → string-value mapping, in declaration order.
/// Ordinal → string-value mapping, in declaration order (the `ldc`
/// operands of each enum constant's construction).
pub values: Vec<String>,
/// Ordinal → static-field name, in declaration order (the `putstatic`
/// target that stores each enum constant). These are the obfuscated
/// names (`a`, `b`, `c`, …) that the binding class references via
/// `getstatic`, so Phase D resolves a `getstatic <enum>.<field>` by
/// looking the field name up here. Empty (falls back to value-keyed
/// resolution) only for synthetically-built test enums; real enums
/// captured from bytecode always populate it. See
/// [`clinit_enum_field_names`] and [`MasterEnumTable::from`].
pub fields: Vec<String>,
}
/// Fingerprints we use to identify each master enum class. The
@@ -180,6 +257,14 @@ struct Fingerprint {
label: &'static str,
prefix: &'static [&'static str],
expected_count: usize,
/// Half-width of the accepted count window around `expected_count`.
/// The ordered `prefix` is the real discriminator (four-plus exact
/// display strings in declaration order is not something a non-enum
/// class reproduces), so the count check only guards against a wildly
/// different class; the window is per-fingerprint because studios ship
/// different-sized enums (Universal's Language enum has 65 values,
/// Disney/Warner's has 70).
count_tolerance: usize,
}
const FINGERPRINTS: &[Fingerprint] = &[
@@ -187,26 +272,32 @@ const FINGERPRINTS: &[Fingerprint] = &[
label: "Language",
prefix: &["English", "French", "Spanish", "Dutch"],
expected_count: 70,
// Universal = 65, Disney/Warner = 70; widen to span both plus drift.
count_tolerance: 8,
},
Fingerprint {
label: "Purpose",
prefix: &["Normal", "Commentary", "PiP", "Trivia"],
expected_count: 8,
count_tolerance: LDC_COUNT_TOLERANCE,
},
Fingerprint {
label: "VideoFormat",
prefix: &["HD", "HDR10 Plus", "HD Dolby"],
expected_count: 7,
count_tolerance: LDC_COUNT_TOLERANCE,
},
Fingerprint {
label: "Region",
prefix: &["USA_D1", "LIC1", "LIC2", "LIC3"],
expected_count: 22,
count_tolerance: LDC_COUNT_TOLERANCE,
},
Fingerprint {
label: "Studio",
prefix: &["Disney", "Marvel", "Pixar"],
expected_count: 6,
count_tolerance: LDC_COUNT_TOLERANCE,
},
];
@@ -301,19 +392,22 @@ impl CandidatePool {
/// Phase A. Walk every `.class` in `archive`, identify the master
/// enums by `<clinit>` ldc-sequence fingerprint. Returns a vector of
/// `(label, MasterEnum)` — at most one match per fingerprint label.
pub(crate) fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str, MasterEnum)> {
// First pass: collect every class's <clinit> ldc string sequence.
fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str, MasterEnum)> {
// First pass: collect every class's <clinit> ldc strings, keyed by the JVM
// INTERNAL name (`this_class`), not the zip entry name — binding classes
// reference enum constants as `getstatic <internal>.f`.
let mut pool = CandidatePool::default();
jar::for_each_class(archive, |class_name, class| {
jar::for_each_class(archive, |zip_name, class| {
let Some(ldcs) = clinit_ldc_strings(class) else {
return;
};
if ldcs.is_empty() {
return;
}
if !pool.insert(class_name, ldcs) {
let key = class.this_class_name().unwrap_or(zip_name);
if !pool.insert(key, ldcs) {
tracing::debug!(
class = class_name,
class = ?key,
classes = pool.by_class.len(),
bytes = pool.bytes,
"deluxe: candidate pool aggregate cap hit, dropping class"
@@ -331,7 +425,7 @@ pub(crate) fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str
continue;
}
let count = ldcs.len();
if count.abs_diff(fp.expected_count) > LDC_COUNT_TOLERANCE {
if count.abs_diff(fp.expected_count) > fp.count_tolerance {
continue;
}
// Prefer exact-count match; otherwise first hit wins.
@@ -345,7 +439,84 @@ pub(crate) fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str
}
}
if let Some((class_name, values)) = best {
out.push((fp.label, MasterEnum { class_name, values }));
out.push((fp.label, class_name, values));
}
}
if out.is_empty() {
return Vec::new();
}
// Second pass: capture obfuscated `putstatic` field names for matched
// classes so Phase D can resolve `getstatic <enum>.<field>` to an ordinal.
// Targeted re-walk of only the matched classes, not the whole jar again.
let want: HashSet<&str> = out.iter().map(|(_, name, _)| name.as_str()).collect();
let mut fields_by_class: HashMap<String, Vec<String>> = HashMap::new();
jar::for_each_class(archive, |zip_name, class| {
let iname = class.this_class_name().unwrap_or(zip_name);
if want.contains(iname) && !fields_by_class.contains_key(iname) {
fields_by_class.insert(iname.to_string(), clinit_enum_field_names(class));
}
});
out.into_iter()
.map(|(label, class_name, values)| {
let fields = fields_by_class.remove(&class_name).unwrap_or_default();
(
label,
MasterEnum {
class_name,
values,
fields,
},
)
})
.collect()
}
/// Collect the static-field names an enum class's `<clinit>` stores its
/// own instances into, in declaration order — the ordinal → field-name
/// mapping that mirrors [`clinit_ldc_strings`]'s ordinal → value mapping.
///
/// An obfuscated Deluxe enum constant compiles to
/// `new E; dup; ldc "Value"; invokespecial E.<init>(…); putstatic E.<field>`,
/// so the `putstatic` whose owning class AND field descriptor are both this
/// class's own type names the field that ordinal. The binding class then
/// references that constant as `getstatic E.<field>`; without this mapping
/// the resolver in [`MasterEnumTable`] cannot turn the obfuscated field name
/// back into an ordinal.
///
/// Bounded by [`MAX_CLINIT_LDC_STRINGS`] like the value walk: a `.class`
/// gated only by a path prefix can inflate to the read ceiling, and each
/// retained field name is a heap `String`.
fn clinit_enum_field_names(class: &super::class_reader::ClassFile) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let Some(self_name) = class.this_class_name() else {
return out;
};
let self_descriptor = format!("L{self_name};");
for m in &class.methods {
if class.member_name(m) != Some("<clinit>") {
continue;
}
let Some(code) = m.code(&class.constant_pool) else {
continue;
};
for insn in code.instructions() {
if insn.opcode != PUTSTATIC {
continue;
}
let Some(idx) = insn.cp_index() else { continue };
let Some(member) = class.constant_pool.member_ref(idx) else {
continue;
};
// Only the enum's own singleton fields (`Lself;` typed, owned by
// this class) — skip auxiliary statics (`$VALUES` arrays, counters).
if member.class_name == self_name && member.descriptor == self_descriptor {
if out.len() >= MAX_CLINIT_LDC_STRINGS {
break;
}
out.push(member.name.to_string());
}
}
}
out
@@ -396,7 +567,7 @@ fn clinit_ldc_strings(class: &super::class_reader::ClassFile) -> Option<Vec<Stri
|| out_bytes.saturating_add(s.len()) > MAX_CLINIT_LDC_BYTES
{
tracing::debug!(
class = class.this_class_name().unwrap_or(""),
class = ?class.this_class_name().unwrap_or(""),
strings = out.len(),
bytes = out_bytes,
"deluxe: clinit ldc collection hit cap, truncating"
@@ -425,12 +596,6 @@ fn ldcs_match_prefix(ldcs: &[String], prefix: &[&str]) -> bool {
// ── Phase C: find the binding class ─────────────────────────────────────────
/// Phase C: identify the class that builds the per-stream label table.
/// That class has the highest count of `getstatic` operations whose
/// owning class is one of the master enum classes we identified in
/// Phase A. Returns the class name + the count (useful for the
/// analyzer / corpus regression).
///
/// Threshold: requires at least `MIN_GETSTATIC` matches to consider a
/// class a binding candidate. Empirically the binding class on a
/// typical disc has 50+ such getstatic references (one per slot ×
@@ -447,7 +612,7 @@ fn ldcs_match_prefix(ldcs: &[String], prefix: &[&str]) -> bool {
/// A disc that splits the table commonly has one audio binding class
/// with the most getstatic refs and a subtitle binding class with
/// somewhat fewer; both share the master Language + Purpose enums.
pub(crate) fn find_binding_classes(
fn find_binding_classes(
archive: &mut jar::Jar,
master_enum_classes: &HashSet<&str>,
) -> Vec<(String, usize)> {
@@ -563,16 +728,15 @@ const MAX_CONSTRUCTIONS: usize = 4096;
/// Phase D entry point: find the binding class in `archive`, run the
/// bytecode walker against its `<clinit>`, return one `Construction`
/// per `new X / invokespecial X.<init>` sequence.
pub(crate) fn decode_binding(
fn decode_binding(
archive: &mut jar::Jar,
binding_class_name: &str,
master: &MasterEnumTable,
) -> Vec<Construction> {
let target_name = binding_class_name.to_string();
// Short-circuit on the name match: try_each_class stops iterating
// (and stops decompressing/parsing remaining .class entries) as soon
// as the closure returns Some, instead of walking the whole jar past
// the target.
// Short-circuit on the name match: try_each_class stops iterating (and
// decompressing/parsing remaining .class entries) once the closure
// returns Some, instead of walking the whole jar past the target.
jar::try_each_class(archive, |class_name, class| {
if class_name != target_name {
return None;
@@ -728,10 +892,8 @@ impl<'a> BindingDecoder<'a> {
.cp_index()
.and_then(|i| self.pool.member_ref(i))
.map(|m| {
// Three-way resolution:
// 1. org.bluray.ti.CodingType.X → CodingType(X)
// 2. master-enum classname.X → EnumRef(kind, ord)
// 3. anything else → Unknown
// Resolution: CodingType.X → CodingType(X); master-enum
// classname.X → EnumRef(kind, ord); else Unknown.
if m.class_name == BD_CODING_TYPE_CLASS {
StackVal::CodingType(m.name.to_string())
} else if let Some((kind, ord)) = self.master.resolve(m.class_name, m.name)
@@ -744,19 +906,20 @@ impl<'a> BindingDecoder<'a> {
.unwrap_or(StackVal::Unknown);
self.push(val);
}
// invokespecial X.<init>(...) — pop args per descriptor.
// If the object on the stack underneath the args is a
// NewObj of class X (set by an earlier `new X / dup`),
// emit a Construction.
// invokespecial X.<init>(...) — pop args per descriptor. If the
// object underneath the args is a NewObj of class X (set by an
// earlier `new X / dup`), emit a Construction.
INVOKESPECIAL => {
let Some(idx) = insn.cp_index() else { return };
let Some(member) = self.pool.member_ref(idx) else { return };
let arg_count = parse_method_arg_count(member.descriptor);
// Pop args off the symbolic stack.
// A per-stream binding ctor takes only scalars/enum refs, never an
// array; a ctor with an array param is a container/title wrapper and
// must not be recorded as a stream binding (stack still unwinds below).
let is_container = member.descriptor.contains('[');
if self.stack.len() < arg_count + 1 {
// Stack-machine drift — bail on this construction
// (but don't panic; the walker tolerates malformed
// input by best-effort).
// Stack-machine drift: bail on this construction without
// panicking; the walker tolerates malformed input best-effort.
self.stack.clear();
return;
}
@@ -767,7 +930,8 @@ impl<'a> BindingDecoder<'a> {
// operates on. For our pattern it's NewObj(X).
let receiver = self.stack.pop().unwrap_or(StackVal::Unknown);
if let StackVal::NewObj(name) = receiver
&& name == member.class_name {
&& name == member.class_name
&& !is_container {
// Bounded by MAX_CONSTRUCTIONS: an unbounded push here
// is ~1 GiB reachable from a crafted `<clinit>`.
if self.constructions.len() >= MAX_CONSTRUCTIONS {
@@ -806,6 +970,13 @@ impl<'a> BindingDecoder<'a> {
self.stack.pop();
self.stack.pop();
}
// anewarray / newarray — pop count, push array ref. Modelled (not
// left to `_`) so array constructions inside a container's arg list
// keep the symbolic stack aligned for per-stream bindings built alongside.
0xBD /* anewarray */ | 0xBC /* newarray */ => {
self.stack.pop();
self.push(StackVal::Unknown);
}
// aastore — array store consumes 3 slots (arrayref, index, value).
AASTORE => {
for _ in 0..3 {
@@ -820,10 +991,9 @@ impl<'a> BindingDecoder<'a> {
self.stack.pop();
self.stack.pop();
}
// Branches / returns / unhandled — clear stack as a
// conservative resync. Binding `<clinit>` is straight-
// line code in practice, so we rarely hit these on the
// verified pattern.
// Branches/returns/unhandled — clear stack as a conservative resync.
// Binding `<clinit>` is straight-line code in practice, so these
// are rarely hit on the verified pattern.
0xA7 /* goto */ | 0xB1 /* return */ => {
self.stack.clear();
}
@@ -890,8 +1060,15 @@ impl MasterEnumTable {
let mut by_class = HashMap::new();
let mut by_kind = HashMap::new();
for (kind, m) in enums {
let field_map: HashMap<String, u16> = m
.values
// Binding classes reference constants by obfuscated field name
// (`getstatic <enum>.a`), so the resolver keys on captured `putstatic`
// field names (`m.fields`); a synthetic test enum falls back to values.
let keys: &[String] = if m.fields.is_empty() {
&m.values
} else {
&m.fields
};
let field_map: HashMap<String, u16> = keys
.iter()
.enumerate()
.map(|(i, v)| (v.clone(), i as u16))
@@ -902,7 +1079,7 @@ impl MasterEnumTable {
MasterEnumTable { by_class, by_kind }
}
pub(crate) fn class_name_set(&self) -> HashSet<&str> {
fn class_name_set(&self) -> HashSet<&str> {
self.by_class.keys().map(String::as_str).collect()
}
@@ -984,16 +1161,9 @@ fn interpret_streams(constructions: &[Construction], master: &MasterEnumTable) -
}
let Some(lang_ord) = lang_ord else {
// No language resolved. If the construction is still recognisably
// a stream binding it OCCUPIES its STN slot and must advance the
// counter — there is just nothing to label. Numbering only the
// slots that resolve renumbers the rest 1..N and lands every
// surviving label on the wrong stream.
//
// `saturating_add` is safe here where it would not be on the
// emitting path below: no label is produced, so parking the
// counter at `u16::MAX` binds nothing. The next slot that DOES
// resolve hits the `checked_add` guard and stops emission.
// A recognisable stream binding still occupies its STN slot and must
// advance the counter, or renumbering skews every surviving label.
// `saturating_add` is safe here since no label is produced.
match slot_kind(c, coding_type.is_some(), &slot_kinds) {
Some(StreamLabelType::Audio) => audio_idx = audio_idx.saturating_add(1),
Some(StreamLabelType::Subtitle) => sub_idx = sub_idx.saturating_add(1),
@@ -1013,13 +1183,9 @@ fn interpret_streams(constructions: &[Construction], master: &MasterEnumTable) -
.map(str::to_string)
.unwrap_or_default();
// Neither `+= 1` (panics in debug, wraps in release) nor
// `saturating_add` is correct here. Saturation is what turned an
// overflow guard into a non-terminating loop in `criterion`, and here
// it would peg every stream past 65535 at the SAME number — silently
// mislabelling tracks, since `apply_labels` binds on
// `(type, stream_number)`. The 1-based u16 numbering space is a hard
// ceiling, so exhausting it stops label emission instead.
// Neither `+= 1` nor `saturating_add` is correct: saturation caused a
// non-terminating loop in `criterion` and would peg every overflowing
// stream at the SAME number. Exhausting the 1-based u16 space stops emission.
let (stream_type, stream_number) = if coding_type.is_some() {
let Some(n) = audio_idx.checked_add(1) else {
tracing::warn!(
@@ -1051,16 +1217,22 @@ fn interpret_streams(constructions: &[Construction], master: &MasterEnumTable) -
None => (String::new(), String::new()),
};
let (purpose, qualifier) = match purpose_ord {
let (purpose, mut qualifier) = match purpose_ord {
Some(o) => deluxe_purpose_to_label(o),
None => (LabelPurpose::Normal, LabelQualifier::None),
};
// Some frameworks (Universal) encode SDH/RNIB as a distinct Language
// VALUE ("English SDH") rather than in the Purpose enum. When the
// purpose left the qualifier unset, recover it from the language name.
if qualifier == LabelQualifier::None {
qualifier = vocab::qualifier(&lang_value);
}
if let Some(hint) = stream_idx_hint {
tracing::debug!(
disc_stream_idx = hint,
lang = %language,
binding = %c.binding_type,
lang = ?language,
binding = ?c.binding_type,
"deluxe interpret_streams: disc-authored stream index (not used for stream_number; preserved for diagnostic)"
);
}
@@ -1200,14 +1372,9 @@ fn deluxe_purpose_to_label(ordinal: u16) -> (LabelPurpose, LabelQualifier) {
mod tests {
use super::*;
// ── Raw .class / .jar fixture builders ──────────────────────────────────
//
// `identify_master_enums`, `find_binding_classes` and `decode_binding`
// operate on `jar::Jar` (a real `ZipArchive`), not on the in-memory
// `ClassFile` struct the rest of this module's tests build directly (see
// `class_with_clinit`). To exercise them we need real serialized
// `.class` bytes inside a real (stored, uncompressed) zip — this is the
// inverse of `ClassFile::parse` / JVMS §4.
// Raw .class/.jar fixture builders: `identify_master_enums`, `find_binding_classes`
// and `decode_binding` operate on `jar::Jar` (a real `ZipArchive`), not the
// in-memory `ClassFile` struct other tests build, so these need real `.class` bytes.
/// Serialize a constant pool (no Long/Double entries — those need the
/// post-slot `Empty` padding this helper doesn't handle) to the on-disk
@@ -1568,10 +1735,8 @@ mod tests {
#[test]
fn fingerprints_cover_documented_enums() {
// Lock the fingerprint roster — if someone adds/removes a
// fingerprint, this test forces them to think about it. The
// 5 documented enums (Language, Purpose, VideoFormat, Region,
// Studio) all need to be here. Codec is structural (separate
// Lock the fingerprint roster so adding/removing one is deliberate.
// All 5 documented enums must be here; Codec is structural (separate
// path), not fingerprinted by ldc prefix.
let labels: Vec<&str> = FINGERPRINTS.iter().map(|fp| fp.label).collect();
assert_eq!(
@@ -1582,10 +1747,9 @@ mod tests {
#[test]
fn fingerprint_prefixes_nonempty_and_under_expected_count() {
// Each prefix must be non-empty and shorter than expected_count
// (so the count gives ADDITIONAL signal beyond the prefix
// match). If a prefix is as long as expected_count there's no
// counting benefit.
// Each prefix must be non-empty and shorter than expected_count so the
// count gives additional signal beyond the prefix match; a prefix as
// long as expected_count has no counting benefit.
for fp in FINGERPRINTS {
assert!(!fp.prefix.is_empty(), "{} has empty prefix", fp.label);
assert!(
@@ -1600,10 +1764,9 @@ mod tests {
#[test]
fn identify_master_enums_matches_purpose_fingerprint() {
// Exact match: 8 ldcs, first 4 = the Purpose prefix, count ==
// expected_count exactly (abs_diff == 0). A decoy class with the
// same prefix but a wildly different count must be rejected and
// must NOT win over the exact match.
// Exact match: 8 ldcs, first 4 = Purpose prefix, count == expected_count
// exactly. A decoy class with the same prefix but a wildly different
// count must be rejected and must NOT win over the exact match.
let good = class_with_ldc_strings(
"GoodPurpose",
&[
@@ -1635,7 +1798,10 @@ mod tests {
.iter()
.find(|(label, _)| *label == "Purpose")
.unwrap_or_else(|| panic!("Purpose fingerprint not matched: {enums:?}"));
assert_eq!(purpose.1.class_name, "com/bydeluxe/Good.class");
// The identified class is keyed by its JVM INTERNAL name (`this_class`,
// here "GoodPurpose"), NOT the zip entry name — that internal name is
// what the binding class's `getstatic` operands reference.
assert_eq!(purpose.1.class_name, "GoodPurpose");
assert_eq!(purpose.1.values.len(), 8);
assert_eq!(purpose.1.values[0], "Normal");
assert_eq!(purpose.1.values[7], "NoForcedDescriptive");
@@ -1699,10 +1865,9 @@ mod tests {
#[test]
fn identify_master_enums_accepts_count_at_the_tolerance_boundary() {
// abs_diff(expected_count, count) == LDC_COUNT_TOLERANCE (4) exactly
// must still be accepted (`> tolerance` rejects, so `== tolerance`
// is the last accepted value). This is the boundary `327:50`
// mutants (`>` -> `==`/`<`/`>=`) disagree on.
// abs_diff(expected_count, count) == LDC_COUNT_TOLERANCE (4) exactly must
// still be accepted (`> tolerance` rejects, so `== tolerance` is the last
// accepted value) — the boundary `>` vs `==`/`<`/`>=` mutants disagree on.
let mut values: Vec<&str> = vec!["Normal", "Commentary", "PiP", "Trivia"];
let filler: Vec<String> = (0..8).map(|i| format!("Filler{i}")).collect(); // 4+8=12, diff=4
values.extend(filler.iter().map(String::as_str));
@@ -1719,9 +1884,8 @@ mod tests {
#[test]
fn identify_master_enums_finds_nothing_without_com_bydeluxe_signal() {
// No FINGERPRINTS-matching class in the jar at all -> empty result
// (kills the `vec![]` mutant only vacuously if paired with the
// positive tests above proving non-emptiness on a real match).
// No FINGERPRINTS-matching class in the jar -> empty result (kills the
// `vec![]` mutant only vacuously, paired with positive tests above).
let unrelated = class_with_ldc_strings("Unrelated", &["Foo", "Bar"]);
let zip = build_zip(&[("x/Unrelated.class", unrelated)]);
let mut archive = open_jar(zip);
@@ -1748,16 +1912,8 @@ mod tests {
#[test]
fn find_binding_classes_picks_top_candidates_above_threshold() {
// Class A: 100 getstatic refs (the top / binding class). B: 45
// (>40% of top, kept). F: 40 (EXACTLY the 40% threshold — pins
// both the `(top_count * 2) / 5` arithmetic and the `>=`
// comparison: any of the `460`/`461` arithmetic mutants shift
// the threshold away from exactly 40, and a `>= -> <` mutant at
// 461 would drop this exact-boundary entry). E: 39 (just BELOW
// the true 40% threshold — a mutant that shrinks the threshold
// below 39 would wrongly keep this). C: 10 (well below, always
// dropped). D: 3 — below MIN_GETSTATIC(4), never even a raw
// candidate.
// A: 100 refs (top). B: 45 (>40% of top, kept). F: 40 (EXACTLY the 40%
// threshold). E: 39 (just below, dropped). D: 3 (below MIN_GETSTATIC(4)).
let master_classes: HashSet<&str> = ["LanguageEnum"].into_iter().collect();
let a = class_with_getstatic_refs("A", "LanguageEnum", 100);
let b = class_with_getstatic_refs("B", "LanguageEnum", 45);
@@ -1803,15 +1959,9 @@ mod tests {
#[test]
fn decode_binding_finds_named_class_and_stops_at_first_match() {
// `decode_binding` matches by the Jar entry path (the same string
// `find_binding_classes` returns), not by the class's own
// `this_class` name. Two entries: the target path carries a real
// `new AudioSlot; dup; getstatic; invokespecial` construction; a
// decoy at a different path carries none (and, being pure
// getstatic/pop, would also match nothing if walked). If the name
// comparison is broken (`!=` mutated to `==`), decode_binding would
// either never match the real target (empty result) or would match
// and decode the WRONG entry.
// Matches by Jar entry path, not `this_class` name. Target path has a
// real construction; decoy has none. A broken path comparison would
// match nothing or decode the WRONG entry.
let target = class_with_simple_construction("Ignored");
let decoy = class_with_getstatic_refs("Ignored2", "LanguageEnum", 2);
let zip = build_zip(&[
@@ -1917,12 +2067,9 @@ mod tests {
#[test]
fn clinit_ldc_string_count_is_capped() {
// A `.class` gated only by a `com/bydeluxe/` path prefix deflates from
// ~100 KB up to the 64 MiB MAX_CLASS_BYTES ceiling, giving ~33M 2-byte
// `ldc` instructions. Every resolved operand is retained as an owned
// String, and identify_master_enums keeps the whole vector per class in
// a HashMap — so the allocation scales with the DECOMPRESSED size while
// the only byte cap is on the compressed disc file.
// A `.class` gated only by path prefix deflates up to the 64 MiB
// MAX_CLASS_BYTES ceiling (~33M 2-byte `ldc` instructions, each retained
// as an owned String), so allocation scales with decompressed size.
const N: usize = 200_000;
let mut code = Vec::with_capacity(N * 2);
for _ in 0..N {
@@ -1931,12 +2078,9 @@ mod tests {
}
let class = class_with_clinit(ldc_pool("English"), 2, &code);
let ldcs = clinit_ldc_strings(&class).expect("<clinit> present");
// Asserted against a LITERAL, not against MAX_CLINIT_LDC_STRINGS. A test
// that compares the result to the very constant under test passes
// vacuously the moment someone raises that constant — which is the most
// likely future regression here, and exactly the tautology class this
// audit has now found six times. 8192 is double the current cap, so this
// still allows the cap to be tuned, but not removed.
// Asserted against a LITERAL, not MAX_CLINIT_LDC_STRINGS: comparing to the
// constant under test passes vacuously once someone raises it. 8192 is
// double the current cap, allowing the cap to be tuned but not removed.
assert!(
ldcs.len() <= 8192,
"retained {} ldc strings from {N} ldc instructions — the cap is not \
@@ -1973,12 +2117,9 @@ mod tests {
#[test]
fn clinit_ldc_string_bytes_boundary_matches_256kib_not_1280() {
// `MAX_CLINIT_LDC_BYTES = 256 * 1024` (262144). A `* -> +` mutant at
// that computation collapses the cap to `256 + 1024` (1280) — 205x
// smaller. 1000-byte strings make the two cap values discriminate
// sharply: correct code retains 262 of them (262000 bytes, the
// 263rd would push to 263000 > 262144); the mutant retains only 1
// (the 2nd would push to 2000 > 1280).
// `MAX_CLINIT_LDC_BYTES = 256 * 1024`. A `* -> +` mutant collapses the
// cap to `256 + 1024` (1280), 205x smaller. 1000-byte strings discriminate
// sharply: correct code retains 262 (262000 bytes); mutant retains only 1.
const N: usize = 400;
let one = "x".repeat(1000);
let mut code = Vec::with_capacity(N * 2);
@@ -2218,10 +2359,9 @@ mod tests {
#[test]
fn binding_decoder_stack_is_bounded_by_max_stack() {
// ~67M single-byte `iconst_0` fit in a 64 MiB decompressed class, and
// each pushes a StackVal onto a Vec with no depth limit (~2 GiB). The
// Code attribute's own max_stack is parsed and must be honoured: JVMS
// 4.7.3 requires the operand stack never exceed it.
// ~67M single-byte `iconst_0` fit in a 64 MiB decompressed class, each
// pushing a StackVal with no depth limit (~2 GiB). Code's max_stack must
// be honoured: JVMS 4.7.3 requires the operand stack never exceed it.
const MAX_STACK: u16 = 4;
let code = vec![ICONST_0; 200_000];
let pool = build_simple_pool();
@@ -2290,30 +2430,19 @@ mod tests {
let m = MasterEnum {
class_name: "LanguageEnum".into(),
values: vec!["English".into(), "French".into(), "Spanish".into()],
// Empty: this synthetic enum resolves by value (the fallback in
// `MasterEnumTable::from`), so the resolve tests below key on
// "English"/"French"/"Spanish" directly.
fields: Vec::new(),
};
MasterEnumTable::from(&[("Language", m)])
}
#[test]
fn decode_binding_class_finds_the_clinit_method_and_emits_its_construction() {
// decode_binding_class wraps BindingDecoder over every method literally
// named "<clinit>" on the class. Exercises the method-selection
// (`member_name(m) != Some("<clinit>")`) and per-method-union
// truncation (`room == 0`) logic that decode_binding_class adds on
// top of the already-tested BindingDecoder::step/run.
//
// Pool layout (must hold "<clinit>"/"()V"/"Code" at 1/2/3 per
// `class_with_clinit`'s contract, while ALSO matching the fixed cp
// indices — 6/8/12 — the reused `new AudioSlot; dup; getstatic;
// invokespecial` bytecode below references):
// 1 Utf8 "<clinit>" 2 Utf8 "()V" 3 Utf8 "Code"
// 4 Utf8 "LanguageEnum" 5 Class->4
// 6 Fieldref{class:5,nat:9} 7 Utf8 "English"
// 8 Class->10 (AudioSlot) 9 NameAndType{name:7,desc:11}
// 10 Utf8 "AudioSlot" 11 Utf8 "LLanguageEnum;"
// 12 Methodref{class:8,nat:13}
// 13 NameAndType{name:14,desc:15}
// 14 Utf8 "<init>" 15 Utf8 "(LLanguageEnum;)V"
// Exercises method-selection and per-method-union truncation logic.
// Pool must hold "<clinit>"/"()V"/"Code" at 1/2/3 per `class_with_clinit`,
// and match cp indices 6/8/12 used by the reused construction bytecode.
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("<clinit>".into()),
@@ -2370,11 +2499,8 @@ mod tests {
#[test]
fn binding_decoder_recognizes_simple_construction() {
// Synthetic <clinit>:
// new AudioSlot (cp idx 8 -> Class -> Utf8 "AudioSlot")
// dup
// getstatic Lang.Eng (cp idx 6 -> Fieldref)
// invokespecial AS.<init>(LLanguageEnum;)V (cp idx 12)
// Synthetic <clinit>: new AudioSlot (cp 8); dup; getstatic Lang.Eng (cp
// 6); invokespecial AS.<init>(LLanguageEnum;)V (cp 12).
let code: Vec<u8> = vec![
NEW,
0,
@@ -2412,12 +2538,8 @@ mod tests {
#[test]
fn binding_decoder_handles_iconst_and_bipush() {
// <clinit> with an int push before the construction:
// iconst_1
// new AudioSlot; dup; getstatic Lang.Eng; invokespecial AS.<init>(LLanguageEnum;)V
// pop (drops the constructed object)
// bipush 42
// pop
// <clinit>: iconst_1; new AudioSlot; dup; getstatic Lang.Eng;
// invokespecial AS.<init>(LLanguageEnum;)V; pop; bipush 42; pop.
let code: Vec<u8> = vec![
ICONST_1,
NEW,
@@ -2451,13 +2573,9 @@ mod tests {
#[test]
fn binding_decoder_dup_duplicates_top_of_stack() {
// JVMS §3.11.7 `dup` (0x59): duplicate the top stack value. Checked
// directly on `decoder.stack` (not via emitted Constructions, which
// a single `new X; dup; invokespecial` sequence can satisfy either
// way — the leftover copy `dup` is responsible for only matters
// once something ELSE consumes it afterward). `new AudioSlot; dup`
// with no invokespecial must leave exactly two NewObj("AudioSlot")
// entries.
// JVMS §3.11.7 `dup` (0x59): duplicate top stack value, checked directly
// on `decoder.stack`. `new AudioSlot; dup` with no invokespecial must
// leave exactly two NewObj entries.
let pool = build_simple_pool();
let master = lang_enum_master();
let code: Vec<u8> = vec![NEW, 0, 8, 0x59 /* dup */];
@@ -2504,12 +2622,9 @@ mod tests {
#[test]
fn binding_decoder_invokevirtual_pops_receiver_plus_args() {
// JVMS §6.5 `invokevirtual`/`invokeinterface` pop the receiver
// PLUS the descriptor's args (`extra = 1` for opcodes 0xB6/0xB9);
// `invokestatic` (0xB8) pops ONLY the args (no receiver). Each
// case below pushes exactly `to_pop` placeholder ints and checks
// the stack is fully drained — a wrong `extra`/`arg_count+extra`
// computation leaves a wrong number of leftovers.
// JVMS §6.5: invokevirtual/invokeinterface pop receiver PLUS args
// (`extra = 1` for 0xB6/0xB9); invokestatic (0xB8) pops ONLY args. Each
// case pushes exactly `to_pop` placeholders and checks full drain.
let master = lang_enum_master();
let run_stack_len = |opcode: u8, descriptor: &str, n_pushes: usize| -> usize {
let pool = call_ref_pool(descriptor);
@@ -2555,10 +2670,9 @@ mod tests {
0,
"invokestatic must pop exactly the arg count, no receiver"
);
// invokestatic with a leftover value UNDER the args: only the args
// are popped, the leftover survives. Distinguishes a `>` mutant at
// the `len < to_pop` guard (which would incorrectly `clear()` the
// whole stack here instead of leaving the leftover).
// invokestatic with a leftover value UNDER the args: only the args pop,
// the leftover survives. Distinguishes a `>` mutant at the `len < to_pop`
// guard (which would wrongly `clear()` the whole stack).
assert_eq!(
run_stack_len(0xB8, "(I)V", 2), // 1 leftover + 1 real arg pushed
1,
@@ -2568,11 +2682,9 @@ mod tests {
#[test]
fn binding_decoder_invoke_family_defensively_clears_on_stack_underflow() {
// If the symbolic stack has FEWER entries than the call needs to
// pop (malformed/adversarial bytecode, or earlier drift), the
// decoder must defensively clear rather than underflow-subtract
// (`len - to_pop` with `len < to_pop` would panic on the `usize`
// subtraction).
// If the symbolic stack has FEWER entries than the call needs to pop
// (malformed bytecode or earlier drift), the decoder must defensively
// clear rather than underflow-subtract (`len - to_pop` would panic).
let pool = call_ref_pool("(II)V"); // needs to_pop = 2
let code: Vec<u8> = vec![ICONST_0, 0xB8, 0, 6]; // only 1 value on stack
let master = lang_enum_master();
@@ -2616,13 +2728,9 @@ mod tests {
#[test]
fn binding_decoder_int_push_opcodes_produce_the_right_value() {
// JVMS §3.11.3: iconst_<i> pushes exactly i (i in -1..=5); bipush
// sign-extends its i8 operand; sipush sign-extends its i16 operand;
// ldc of a CONSTANT_Integer pushes that constant. Each is checked
// as the sole arg of `new AudioSlot; dup; <push>; invokespecial
// AudioSlot.<init>(I)V` so a wrong (or absent, if the opcode's match
// arm were deleted) push shows up as a wrong (or missing/Unknown)
// arg value, not just "some construction happened".
// JVMS §3.11.3: iconst_<i> pushes exactly i; bipush/sipush sign-extend
// their operand; ldc of a CONSTANT_Integer pushes that constant. Each is
// checked as the sole ctor arg so a wrong/missing push arm is visible.
let cases: Vec<(&str, Vec<u8>, i32)> = vec![
("iconst_m1", vec![ICONST_M1], -1),
("iconst_0", vec![ICONST_0], 0),
@@ -2982,4 +3090,136 @@ mod tests {
assert!(set.contains("LanguageEnum"));
assert_eq!(set.len(), 1);
}
// ── Real-disc fixtures: Universal (studio="uni") — verbatim `.class` files from
// a real `/BDMV/JAR/00000.jar`, exercising Phase A→D against real obfuscated
// bytecode: pd=Language enum, lp=Purpose enum, tl=binding class (np/wb/oq).
const UNI_PD_CLASS: &[u8] = include_bytes!("testdata/deluxe_uni/pd.class");
const UNI_LP_CLASS: &[u8] = include_bytes!("testdata/deluxe_uni/lp.class");
const UNI_TL_CLASS: &[u8] = include_bytes!("testdata/deluxe_uni/tl.class");
fn parse_fixture(bytes: &[u8]) -> super::super::class_reader::ClassFile {
super::super::class_reader::ClassFile::parse(bytes).expect("fixture .class parses")
}
#[test]
fn universal_language_enum_pd_matches_the_language_fingerprint() {
let pd = parse_fixture(UNI_PD_CLASS);
let values = clinit_ldc_strings(&pd).expect("pd has a <clinit>");
// 65-value Universal Language enum — the count the old 70±4 window
// wrongly rejected.
assert_eq!(values.len(), 65);
assert_eq!(&values[..4], &["English", "French", "Spanish", "Dutch"]);
assert!(
ldcs_match_prefix(&values, FINGERPRINTS[0].prefix),
"pd must match the Language prefix"
);
assert!(
values.len().abs_diff(FINGERPRINTS[0].expected_count)
<= FINGERPRINTS[0].count_tolerance,
"65 must fall inside the Language count window (regression guard on \
the widened tolerance)"
);
}
#[test]
fn universal_enum_field_names_map_ordinals_to_obfuscated_fields() {
let pd = parse_fixture(UNI_PD_CLASS);
let fields = clinit_enum_field_names(&pd);
// One putstatic per enum constant, in declaration order: a,b,c,d,…
assert_eq!(fields.len(), 65);
assert_eq!(&fields[..4], &["a", "b", "c", "d"]);
}
/// Build the master table the way `parse` would, but from the fixtures.
fn universal_master() -> MasterEnumTable {
let pd = parse_fixture(UNI_PD_CLASS);
let lp = parse_fixture(UNI_LP_CLASS);
let pd_enum = MasterEnum {
class_name: pd.this_class_name().unwrap().to_string(),
values: clinit_ldc_strings(&pd).unwrap(),
fields: clinit_enum_field_names(&pd),
};
let lp_enum = MasterEnum {
class_name: lp.this_class_name().unwrap().to_string(),
values: clinit_ldc_strings(&lp).unwrap(),
fields: clinit_enum_field_names(&lp),
};
MasterEnumTable::from(&[("Language", pd_enum), ("Purpose", lp_enum)])
}
#[test]
fn universal_binding_class_decodes_real_per_stream_labels() {
let tl = parse_fixture(UNI_TL_CLASS);
let master = universal_master();
let constructions = decode_binding_class(&tl, &master);
assert!(
!constructions.is_empty(),
"tl.<clinit> must yield per-stream constructions"
);
// The title-wrapper `oq` takes array parameters and must be filtered:
// every retained construction is a scalar/enum-only binding.
// (np = 4 args incl. CodingType; wb = 4-5 args incl. `mi`.)
let labels = interpret_streams(&constructions, &master);
assert!(!labels.is_empty(), "must emit at least one stream label");
let audio: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.collect();
let subs: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.collect();
assert!(!audio.is_empty(), "expected audio labels");
assert!(!subs.is_empty(), "expected subtitle labels");
// Real languages recovered via the obfuscated-field-name resolver.
let langs: std::collections::HashSet<&str> =
labels.iter().map(|l| l.language.as_str()).collect();
assert!(langs.contains("eng"), "English audio/subtitle expected");
assert!(
langs.contains("spa") || langs.contains("fra"),
"expected at least one non-English language (Spanish/French)"
);
// The display name is the disc's OWN label, not just an ISO code.
assert!(
labels.iter().any(|l| l.name == "English"),
"expected the disc-authored display name 'English'"
);
// Audio bindings carry a decoded BD-J CodingType → codec hint.
assert!(
audio.iter().any(|l| l.codec_hint == "Dolby Digital"),
"DOLBY_AC3_AUDIO must translate to 'Dolby Digital'; got {:?}",
audio.iter().map(|l| &l.codec_hint).collect::<Vec<_>>()
);
// Subtitle bindings (`wb`, no CodingType) carry no codec hint.
assert!(
subs.iter().all(|l| l.codec_hint.is_empty()),
"subtitle codec hints should be empty (PG implied)"
);
}
#[test]
fn universal_config_xml_studio_attribute_is_parsed() {
// A real config.xml from a Universal Blu-ray release's
// /BDMV/JAR/99999/config.xml.
let xml = br#"<TitleConfig studio="uni" >
<Type><Rental>-</Rental><Single>-</Single></Type>
</TitleConfig>"#;
assert_eq!(parse_studio_attr(xml).as_deref(), Some("uni"));
// Single-quoted and whitespace-padded variants.
assert_eq!(
parse_studio_attr(b"<TitleConfig studio = 'fox'>").as_deref(),
Some("fox")
);
// Missing / empty attribute → None (never panics).
assert_eq!(parse_studio_attr(b"<TitleConfig>"), None);
assert_eq!(parse_studio_attr(b"studio=\"\""), None);
assert_eq!(parse_studio_attr(&[0xff, 0xfe, 0x00]), None);
}
}
+522
View File
@@ -0,0 +1,522 @@
//! Fox — loose `/BDMV/JAR/<id>/dcx.xml` plain-XML manifest.
//!
//! Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
//! human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
//! under `<disc>` it lists every playlist the disc plays. The main-feature
//! playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
//! language, editorial purpose and forced/SDH state outright — everything a
//! label parser wants, in attributes, with no bytecode to walk.
//!
//! NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
//! is one authoring house's internal metadata that happens to press onto the
//! disc. Every field meaning below was read off a real disc; treat an
//! unfamiliar value as unknown rather than guessing.
//!
//! Confirmed schema (a real Fox release, `/BDMV/JAR/05001/dcx.xml`):
//!
//! ```xml
//! <dcx>
//! <disc>
//! <playlist id="00001" lang="eng" name="topmenu"/>
//! ...
//! <playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
//! <audio id="01" lang="eng" type="feature"/>
//! <audio id="02" lang="eng" type="rnib"/>
//! <audio id="03" lang="spa" dial="lat" type="feature"/>
//! ...
//! <subtitle id="01" lang="eng" type="feature" form="sdh"/>
//! <subtitle id="02" lang="spa" dial="lat" type="embed"/>
//! ...
//! <subtitle id="11" lang="eng" type="text"/>
//! <properties> ...chapter marks... </properties>
//! </playlist>
//! <playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
//! </disc>
//! </dcx>
//! ```
//!
//! Field meanings (read off the disc, not documented):
//! * `<audio type>`: `feature` = a normal program track; `rnib` = a
//! descriptive/narration track ("Royal National Institute of Blind People"
//! described-video), mapped to [`LabelPurpose::Descriptive`]. A `type`
//! containing "comment" maps to [`LabelPurpose::Commentary`].
//! * `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
//! hearing track ([`LabelQualifier::Sdh`]).
//! * `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
//! track ([`LabelQualifier::Forced`]); `feature`/`text` are full tracks.
//! * `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
//! (audio ids 01..N, subtitle ids 01..N independently), matching the vendor
//! `stream_number` convention the ordinal binder in [`super::apply_labels`]
//! reads.
//!
//! The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
//! so extraction is scoped to a single `<playlist name="feature">` element —
//! never a document-wide `<audio>` scan, which would merge the regional
//! `00800` (eng) and `00801` (jpn) tables into one and collide their slots.
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorSource;
use crate::udf::UdfFs;
/// Detect a Fox disc.
///
/// Primary, cheap signal: a loose `dcx.xml` under some `/BDMV/JAR/<id>/`
/// subdirectory — the manifest [`parse`] reads. Checked first because it is a
/// directory walk with no sector reads.
///
/// Secondary signal: a `com/foxbd/` prefix in a top-level BD-J jar, which newer
/// Fox discs (later franchise sequels) carry instead of a loose `dcx.xml`. Recognising
/// it attributes the disc to Fox rather than the generic deluxe stub even
/// though [`parse`] cannot decode that bytecode form yet (see the Phase 2 note
/// at the bottom of this file); such a disc detects here but yields no labels,
/// so a lower-confidence parser still wins.
pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
if super::jar_file_exists(udf, "dcx.xml") {
return true;
}
super::jar::for_each_jar(reader, udf, |_, jar| {
super::jar::has_path_prefix(jar, "com/foxbd/").then_some(())
})
.is_some()
}
pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult> {
let data = super::read_jar_file(reader, udf, "dcx.xml")?;
let text = std::str::from_utf8(&data).ok()?;
let labels = labels_from_dcx(text);
if labels.is_empty() {
return None;
}
// High confidence: the manifest is fully structured and every field
// extracted here has its meaning fixed by real discs.
let mut result = ParseResult::high(labels);
// Surface the feature playlist's authoring id (e.g. "00800") so title
// selection can prefer the disc's own feature over a size-inflated decoy —
// the same signal `paramount::parse` provides.
result.feature_playlist = feature_playlist_id(text).map(|id| super::FeaturePlaylistHint {
playlist_id: id.parse::<u16>().ok(),
filename: Some(format!("{id}.mpls")),
});
Some(result)
}
/// Build the stream labels from a `dcx.xml` document. Split out from [`parse`]
/// so the schema mapping is unit-testable without a `SectorSource`/`UdfFs`.
///
/// Scoped to ONE feature playlist element (see the module note): the richest
/// `<playlist name="feature">`. Its nested `<audio>`/`<subtitle>` children are
/// the vendor's STN table for that title.
pub(crate) fn labels_from_dcx(text: &str) -> Vec<StreamLabel> {
let Some(feature) = select_feature_playlist(text) else {
return Vec::new();
};
let mut labels = Vec::new();
// Audio streams.
let mut from = 0;
while let Some((s, e)) = xml::find_element(feature, "audio", from) {
let el = &feature[s..e];
from = e;
let Some(stream_number) = stream_number_from_id(&xml::attr(el, "id")) else {
continue;
};
let Some(language) = xml::attr(el, "lang") else {
continue;
};
let ty = xml::attr(el, "type")
.unwrap_or_default()
.to_ascii_lowercase();
labels.push(StreamLabel {
stream_id: None,
stream_number,
stream_type: StreamLabelType::Audio,
language: normalize_language(&language),
name: String::new(),
purpose: audio_purpose(&ty),
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
});
}
// Subtitle streams.
let mut from = 0;
while let Some((s, e)) = xml::find_element(feature, "subtitle", from) {
let el = &feature[s..e];
from = e;
let Some(stream_number) = stream_number_from_id(&xml::attr(el, "id")) else {
continue;
};
let Some(language) = xml::attr(el, "lang") else {
continue;
};
let ty = xml::attr(el, "type")
.unwrap_or_default()
.to_ascii_lowercase();
let form = xml::attr(el, "form")
.unwrap_or_default()
.to_ascii_lowercase();
labels.push(StreamLabel {
stream_id: None,
stream_number,
stream_type: StreamLabelType::Subtitle,
language: normalize_language(&language),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: subtitle_qualifier(&ty, &form),
codec_hint: String::new(),
variant: String::new(),
});
}
labels
}
/// Pick the disc's primary feature playlist element (the full `<playlist ...>
/// ... </playlist>` text). Among all `<playlist name="feature">` elements —
/// Fox presses one per regional variant (`00800` eng, `00801` jpn) — the one
/// carrying the most `<audio>`/`<subtitle>` streams wins, ties to the first.
///
/// Returning ONE element is the whole point: each feature playlist is its own
/// STN table with its own 1-based slot ids, so merging two would put two
/// different streams on slot `01`. The richest table is the fullest label set
/// and anchors to its matching title by language sequence in
/// [`super::apply_labels`].
fn select_feature_playlist(text: &str) -> Option<&str> {
let mut best: Option<&str> = None;
let mut best_streams = 0usize;
let mut from = 0;
while let Some((s, e)) = xml::find_element(text, "playlist", from) {
let element = &text[s..e];
from = e;
// `name` is read from the element; the opening tag's `name="feature"`
// is the first `name=` in document order, ahead of any nested child
// (nested `<audio>`/`<subtitle>` carry no `name`).
let is_feature =
xml::attr(element, "name").is_some_and(|n| n.eq_ignore_ascii_case("feature"));
if !is_feature {
continue;
}
let streams = count_elements(element, "audio") + count_elements(element, "subtitle");
// First feature wins the tie (matches the registry's first-wins rule);
// a strictly richer table displaces it. `best.is_none()` so a feature
// with zero nested streams is still selected — its id feeds the hint.
if best.is_none() || streams > best_streams {
best_streams = streams;
best = Some(element);
}
}
best
}
/// The authoring id (digits only, e.g. "00800") of the selected feature
/// playlist, or `None` when no feature playlist / no id is present.
fn feature_playlist_id(text: &str) -> Option<String> {
let feature = select_feature_playlist(text)?;
let id = xml::attr(feature, "id")?;
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
Some(digits)
}
/// Count `<tag>` elements inside `element`.
fn count_elements(element: &str, tag: &str) -> usize {
let mut n = 0;
let mut from = 0;
while let Some((_, e)) = xml::find_element(element, tag, from) {
n += 1;
from = e;
}
n
}
/// Parse a nested stream `id` (e.g. "01", "11") into its 1-based STN slot.
/// Digits only, so a stray quote/space cannot poison it; `None` (skip the
/// stream) when it names no slot — 0 is the module's `NO_STN_SLOT` sentinel and
/// cannot be bound by the ordinal path.
fn stream_number_from_id(id: &Option<String>) -> Option<u16> {
let id = id.as_deref()?;
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
digits.parse::<u16>().ok().filter(|&n| n != 0)
}
/// Map an `<audio type>` value to a [`LabelPurpose`]. `rnib` is described-video
/// (descriptive/narration); a `comment*` value is commentary; everything else
/// (`feature`, unknown) is a normal program track.
fn audio_purpose(ty: &str) -> LabelPurpose {
if ty == "rnib" {
LabelPurpose::Descriptive
} else if ty.contains("comment") {
LabelPurpose::Commentary
} else {
LabelPurpose::Normal
}
}
/// Map a `<subtitle>`'s `form`/`type` to a [`LabelQualifier`]. `form="sdh"`
/// takes precedence (an SDH track is the editorially meaningful flag);
/// otherwise `type="embed"` is a dedicated forced/embedded track. `feature`
/// and `text` full tracks carry no qualifier.
fn subtitle_qualifier(ty: &str, form: &str) -> LabelQualifier {
if form == "sdh" {
LabelQualifier::Sdh
} else if ty == "embed" {
LabelQualifier::Forced
} else {
LabelQualifier::None
}
}
/// Trim + lowercase the raw ISO 639-2 code (`"eng"`, `"fra"`, ...).
fn normalize_language(raw: &str) -> String {
raw.trim().to_ascii_lowercase()
}
// Phase 2 (design only, not implemented): newer Fox discs ship no loose `dcx.xml`,
// wrapping the same per-stream data in `com/foxbd` BD-J `.class` bytecode instead. A
// follow-on parser would reuse `super::class_reader`/`jar` (as `dbp`/`deluxe` do).
#[cfg(test)]
mod tests {
use super::*;
/// A real Fox-release manifest, reduced only by truncating the giant
/// chapter-mark `<properties>` blocks (which carry no stream labels). Every
/// `<audio>`/`<subtitle>` element and both regional feature playlists are
/// verbatim from a real disc's `/BDMV/JAR/05001/dcx.xml`, so this fixture exercises the
/// exact bytes production sees: feature selection across two `name="feature"`
/// playlists, the nested-scope rule, and the rnib/sdh/embed flags.
const FOX_DCX_SAMPLE: &str = r#"<dcx>
<disc>
<properties region="ABC" regioncheckon="false" parentallevel="PG" hdronlydisc="true" bootstrap.bdjo="88888"
vstaskenabled="false" defaultversion="1" topmenushowmarkid="0" topmenuloopmarkid="0"/>
<playlist id="00001" lang="eng" name="topmenu"/>
<playlist id="00100" lang="eng" name="foxlogo"/>
<playlist id="00300" lang="eng" name="copyright"/> <!-- English -->
<playlist id="00600" name="black" lang="eng"/>
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
<audio id="04" lang="fra" dial="par" type="feature"/>
<audio id="05" lang="dan" type="feature"/>
<audio id="06" lang="nld" type="feature"/>
<audio id="07" lang="fin" type="feature"/>
<audio id="08" lang="deu" type="feature"/>
<audio id="09" lang="ita" type="feature"/>
<audio id="10" lang="nor" type="feature"/>
<audio id="11" lang="swe" type="feature"/>
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
<subtitle id="03" lang="fra" dial="par" type="embed"/>
<subtitle id="04" lang="dan" type="embed"/>
<subtitle id="05" lang="nld" type="embed"/>
<subtitle id="06" lang="fin" type="feature"/>
<subtitle id="07" lang="deu" type="embed"/>
<subtitle id="08" lang="ita" type="embed"/>
<subtitle id="09" lang="nor" type="embed"/>
<subtitle id="10" lang="swe" type="embed"/>
<subtitle id="11" lang="eng" type="text"/>
<properties>
<entry.marks ids="0,2,4,6,8,10"/>
<playlist.marks timecodes="00:00:00:00,00:03:58:18"/>
</properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="jpn" type="feature"/>
<subtitle id="01" lang="jpn" type="feature"/>
<subtitle id="02" lang="eng" type="feature" form="sdh"/>
<subtitle id="03" lang="jpn" type="text"/>
<subtitle id="04" lang="eng" type="text"/>
<properties>
<entry.marks ids="0,2,4"/>
</properties>
</playlist>
</disc>
</dcx>"#;
fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.collect()
}
fn subs(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.collect()
}
/// The primary feature is the richer `00800` (eng) table, not `00801`
/// (jpn). Both are `name="feature"`; selection must pick by stream count,
/// and the id we surface for the (future) FeaturePlaylistHint is `00800`.
#[test]
fn selects_richest_feature_playlist_and_its_id() {
assert_eq!(
feature_playlist_id(FOX_DCX_SAMPLE),
Some("00800".to_string())
);
}
/// Full real-disc parse: the 00800 audio table. Eleven tracks, id order =
/// STN slot, and slot 2 (`eng rnib`) is the descriptive/narration track.
#[test]
fn fox_dcx_audio_labels() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let a = audio(&labels);
assert_eq!(
a.len(),
11,
"00800 has eleven audio tracks, 00801 not merged"
);
// Slots and languages, in id order.
let got: Vec<(u16, &str, LabelPurpose)> = a
.iter()
.map(|l| (l.stream_number, l.language.as_str(), l.purpose))
.collect();
assert_eq!(
got,
vec![
(1, "eng", LabelPurpose::Normal),
(2, "eng", LabelPurpose::Descriptive), // rnib = described video
(3, "spa", LabelPurpose::Normal),
(4, "fra", LabelPurpose::Normal),
(5, "dan", LabelPurpose::Normal),
(6, "nld", LabelPurpose::Normal),
(7, "fin", LabelPurpose::Normal),
(8, "deu", LabelPurpose::Normal),
(9, "ita", LabelPurpose::Normal),
(10, "nor", LabelPurpose::Normal),
(11, "swe", LabelPurpose::Normal),
]
);
// Vendor labels: no StreamId, they bind by ordinal STN slot.
assert!(a.iter().all(|l| l.stream_id.is_none()));
}
/// Full real-disc parse: the 00800 subtitle table. `form="sdh"` → Sdh,
/// `type="embed"` → Forced, `feature`/`text` → no qualifier.
#[test]
fn fox_dcx_subtitle_labels() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let s = subs(&labels);
assert_eq!(s.len(), 11, "00800 has eleven subtitle tracks");
let got: Vec<(u16, &str, LabelQualifier)> = s
.iter()
.map(|l| (l.stream_number, l.language.as_str(), l.qualifier))
.collect();
assert_eq!(
got,
vec![
(1, "eng", LabelQualifier::Sdh), // feature + form=sdh
(2, "spa", LabelQualifier::Forced), // embed
(3, "fra", LabelQualifier::Forced),
(4, "dan", LabelQualifier::Forced),
(5, "nld", LabelQualifier::Forced),
(6, "fin", LabelQualifier::None), // feature
(7, "deu", LabelQualifier::Forced),
(8, "ita", LabelQualifier::Forced),
(9, "nor", LabelQualifier::Forced),
(10, "swe", LabelQualifier::Forced),
(11, "eng", LabelQualifier::None), // text
]
);
}
/// The nested-scope rule: audio/subtitle come from ONE feature playlist,
/// never a document-wide scan that would merge 00800 and 00801 and collide
/// their `id="01"` slots. If scoping regressed, slot 1 audio would appear
/// twice and the count would jump past 11.
#[test]
fn does_not_merge_regional_feature_playlists() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let a = audio(&labels);
// Exactly one audio label per STN slot 1..=11.
let slot1: Vec<_> = a.iter().filter(|l| l.stream_number == 1).collect();
assert_eq!(slot1.len(), 1, "one stream on slot 1, not one per playlist");
assert_eq!(audio(&labels).len() + subs(&labels).len(), 22);
}
/// The `rnib` described-video mapping in isolation, plus the commentary
/// path (no track in the sample manifest uses it, so it is pinned synthetically).
#[test]
fn audio_purpose_mapping() {
assert_eq!(audio_purpose("feature"), LabelPurpose::Normal);
assert_eq!(audio_purpose("rnib"), LabelPurpose::Descriptive);
assert_eq!(audio_purpose("commentary"), LabelPurpose::Commentary);
assert_eq!(
audio_purpose("director-commentary"),
LabelPurpose::Commentary
);
assert_eq!(audio_purpose("unknown"), LabelPurpose::Normal);
}
/// `form="sdh"` outranks `type`; `embed` is forced; full tracks are None.
#[test]
fn subtitle_qualifier_mapping() {
assert_eq!(subtitle_qualifier("feature", "sdh"), LabelQualifier::Sdh);
assert_eq!(subtitle_qualifier("embed", ""), LabelQualifier::Forced);
assert_eq!(subtitle_qualifier("feature", ""), LabelQualifier::None);
assert_eq!(subtitle_qualifier("text", ""), LabelQualifier::None);
// An SDH embedded track (hypothetical) still flags SDH, the richer fact.
assert_eq!(subtitle_qualifier("embed", "sdh"), LabelQualifier::Sdh);
}
/// `id="NN"` becomes the 1-based STN slot; a missing/zero id names no slot.
#[test]
fn stream_number_from_id_digits_only() {
assert_eq!(stream_number_from_id(&Some("01".into())), Some(1));
assert_eq!(stream_number_from_id(&Some("11".into())), Some(11));
assert_eq!(stream_number_from_id(&Some(" 07 ".into())), Some(7));
assert_eq!(stream_number_from_id(&Some("00".into())), None);
assert_eq!(stream_number_from_id(&None), None);
assert_eq!(stream_number_from_id(&Some("".into())), None);
}
// ── Negative detection ─────────────────────────────────────────────────
/// A document that is not a Fox feature manifest yields no labels. Three
/// shapes: empty, a `<dcx>` whose playlists are all non-feature menu/logo
/// clips (no `name="feature"`), and unrelated XML. None must be mistaken
/// for a Fox feature table.
#[test]
fn non_feature_manifest_yields_no_labels() {
assert!(labels_from_dcx("").is_empty());
assert!(labels_from_dcx("<not-dcx><playlist name=\"x\"/></not-dcx>").is_empty());
let menus_only = r#"<dcx><disc>
<playlist id="00001" lang="eng" name="topmenu"/>
<playlist id="00100" lang="eng" name="foxlogo"/>
<playlist id="00600" name="black" lang="eng"/>
</disc></dcx>"#;
assert!(
labels_from_dcx(menus_only).is_empty(),
"no <playlist name=\"feature\"> → nothing to label"
);
assert_eq!(feature_playlist_id(menus_only), None);
}
/// A feature playlist with no nested streams (an authoring edge) produces
/// no labels rather than a spurious empty-slot entry.
#[test]
fn feature_without_streams_yields_no_labels() {
let doc = r#"<dcx><disc>
<playlist id="00800" lang="eng" name="feature" durs="7628"/>
</disc></dcx>"#;
assert!(labels_from_dcx(doc).is_empty());
// ...but the id is still recoverable for the feature hint.
assert_eq!(feature_playlist_id(doc), Some("00800".to_string()));
}
}
+2 -2
View File
@@ -74,12 +74,12 @@ pub fn has_path_prefix(archive: &Jar, prefix: &str) -> bool {
}
/// Iterate every `.class` entry in the jar, parse it with
/// [`class_reader`], and call `f` with `(entry_name, &ClassFile)`.
/// [`class_reader`](super::class_reader), and call `f` with `(entry_name, &ClassFile)`.
///
/// Entries that fail to read or parse are silently skipped — this is
/// label-extraction code, robustness matters more than completeness.
/// Callers that need to know which classes failed should use the
/// lower-level [`class_reader`] API directly.
/// lower-level [`class_reader`](super::class_reader) API directly.
pub fn for_each_class<F>(archive: &mut Jar, mut f: F)
where
F: FnMut(&str, &ClassFile),
+188 -176
View File
@@ -15,7 +15,8 @@ pub mod clpi_audit;
mod criterion;
mod ctrm;
mod dbp;
mod deluxe;
pub(crate) mod deluxe;
pub(crate) mod fox;
pub(crate) mod jar;
mod mpls_universal;
mod paramount;
@@ -118,15 +119,12 @@ pub enum LabelQualifier {
}
// ── Parser registry ────────────────────────────────────────────────────────
//
// Each entry: (name, detect_fn, parse_fn). Order = tiebreaker only —
// the registry picks the highest-confidence parse result, falling back
// to array order on confidence ties.
// Each entry: (name, detect_fn, parse_fn). Order = tiebreaker only — the
// registry picks the highest-confidence parse result, falling back to order.
// `detect` takes the reader too, so a parser can look INSIDE a jar's central
// directory (real vendor-prefix / project-file check) rather than firing on
// "any jar present". Precise detection is what lets the registry scale to many
// parsers without cross-parser collisions.
// "any jar present", letting the registry scale without cross-parser collisions.
type DetectFn = fn(&mut dyn SectorSource, &UdfFs) -> bool;
type ParseFn = fn(&mut dyn SectorSource, &UdfFs) -> Option<ParseResult>;
@@ -149,6 +147,38 @@ pub enum Confidence {
High,
}
/// The disc's own authoring-designated main-feature playlist, surfaced by a
/// parser that reads the menu/authoring metadata (e.g. Paramount `playlists.xml`
/// carries `<playlist name="Feature" id="00222"/>`). Consumed by title
/// selection (`Disc::main_feature_order`) as a size-independent anti-decoy
/// signal — a playlist obfuscation trick can inflate a streamless decoy past
/// the real feature on size/duration, but it can't forge the authoring id.
/// Best-effort: an unknown field is `None`.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct FeaturePlaylistHint {
/// Playlist number (e.g. 222 for `00222.mpls`) when the authoring id parses.
pub playlist_id: Option<u16>,
/// Playlist filename (e.g. "00222.mpls") when derivable.
pub filename: Option<String>,
}
impl FeaturePlaylistHint {
/// Whether this hint designates the playlist a title carries. Matches on
/// the numeric id OR the filename (case-insensitive), so a hint that pins
/// only one still binds.
pub fn matches(&self, title_playlist_id: u16, title_playlist: &str) -> bool {
if self.playlist_id == Some(title_playlist_id) {
return true;
}
matches!(&self.filename, Some(f) if f.eq_ignore_ascii_case(title_playlist))
}
/// Whether the hint carries no usable identity.
pub fn is_empty(&self) -> bool {
self.playlist_id.is_none() && self.filename.is_none()
}
}
/// Successful parser result. `None` from `parse()` still means "this
/// isn't my disc" (no labels at all); `Some(ParseResult { labels, .. })`
/// with `labels.is_empty()` is also a "no labels" case but reachable
@@ -158,6 +188,9 @@ pub enum Confidence {
pub struct ParseResult {
pub labels: Vec<StreamLabel>,
pub confidence: Confidence,
/// The disc's authoring-named feature playlist, when this parser read it
/// from the menu metadata. `None` for parsers that don't expose one.
pub feature_playlist: Option<FeaturePlaylistHint>,
}
impl ParseResult {
@@ -167,6 +200,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::High,
feature_playlist: None,
}
}
@@ -175,6 +209,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::Medium,
feature_playlist: None,
}
}
@@ -185,6 +220,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::Low,
feature_playlist: None,
}
}
}
@@ -194,19 +230,18 @@ const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
("criterion", criterion::detect, criterion::parse),
("pixelogic", pixelogic::detect, pixelogic::parse),
("ctrm", ctrm::detect, ctrm::parse),
// dbp and deluxe now detect via the real `com/<vendor>/` central-directory
// prefix (reader-backed), so they claim only their own discs. Order between
// them is the tiebreaker on equal confidence; dbp goes first because its
// parse path is cheaper (constant-pool iteration vs. deluxe's bytecode
// walking).
// dbp and deluxe detect via the real `com/<vendor>/` central-directory
// prefix (reader-backed), so they claim only their own discs. dbp goes
// first on ties: its parse path is cheaper (constant-pool vs. bytecode).
("dbp", dbp::detect, dbp::parse),
("deluxe", deluxe::detect, deluxe::parse),
// Universal MPLS fallback. Returns Confidence::Low so framework
// parsers always win when they match. Closes the "no framework
// matched" gap (e.g. HDMV-only discs) with spec-derived language
// + base codec for every stream the playlist references. Runs
// last in registry order so it's only the chosen parser when
// nothing else fired.
// Fox — older discs ship a loose plain-XML `/BDMV/JAR/<id>/dcx.xml` manifest
// with per-stream language/purpose/qualifier (High confidence). Newer Fox
// wraps this in `com/foxbd` bytecode (fox.rs Phase 2); detected, not yet decoded.
("fox", fox::detect, fox::parse),
// Universal MPLS fallback. Returns Confidence::Low so framework parsers
// always win when they match. Closes the "no framework matched" gap (e.g.
// HDMV-only discs) with spec-derived language + codec for every stream.
(
"mpls_universal",
mpls_universal::detect,
@@ -214,20 +249,27 @@ const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
),
// Menu-graphic filename language hints (Low). AFTER mpls_universal so the
// richer spec-derived floor wins the Low tie whenever it produces anything;
// this only becomes the chosen parser when even MPLS yields nothing but the
// menu artwork still names its languages. A last-resort language source.
// only chosen when even MPLS yields nothing but the menu art still names languages.
("png_filenames", png_filenames::detect, png_filenames::parse),
];
/// Search disc for config files, extract labels, apply to streams.
/// This is 100% optional — if anything fails, streams are untouched.
pub fn apply(reader: &mut dyn SectorSource, udf: &UdfFs, titles: &mut [DiscTitle]) {
let labels = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| extract(reader, udf)))
.unwrap_or_default();
if labels.is_empty() {
return;
/// Returns the disc's authoring-designated feature playlist when a parser
/// surfaced one (`None` otherwise), so the caller can bias title selection
/// toward it. Label binding onto streams is a side effect on `titles`.
pub fn apply(
reader: &mut dyn SectorSource,
udf: &UdfFs,
titles: &mut [DiscTitle],
) -> Option<FeaturePlaylistHint> {
let (labels, feature_playlist) =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| extract(reader, udf)))
.unwrap_or_default();
if !labels.is_empty() {
apply_labels(&labels, titles);
}
apply_labels(&labels, titles);
feature_playlist.filter(|h| !h.is_empty())
}
/// Minimum number of streams of one type a title must carry before its
@@ -430,10 +472,9 @@ fn anchor_score(
pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
use std::collections::HashMap;
// `(clip id, PID) -> index into `labels``, harvested from the anchor
// title of each stream type. Keyed by clip because a PID is only unique
// within one clip: two unrelated .m2ts files both open their audio at
// 0x1100.
// `(clip id, PID) -> index into `labels``, harvested from the anchor title
// of each stream type. Keyed by clip because a PID is only unique within
// one clip: two unrelated .m2ts files both open their audio at 0x1100.
let mut pid_map: HashMap<(&str, u16), usize> = HashMap::new();
let mut anchors: [Option<usize>; 2] = [None; 2];
for stream_type in [StreamLabelType::Audio, StreamLabelType::Subtitle] {
@@ -450,15 +491,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
}) else {
continue;
};
// ONLY the anchor's first clip. `disc::bluray` builds a title's
// stream list from `play_items[0]`'s STN table, so that is the only
// clip in which these PIDs were ever observed — the same clip tier 3
// keys its derived ids on (`clip0`, below). Recording the fact
// against every clip the anchor plays claims knowledge of stream
// tables never read: a sibling playlist over a LATER clip then binds
// the anchor's editorial label onto whichever stream of that clip
// reuses the PID, which is a different physical stream (PIDs are
// unique only within a clip).
// ONLY the anchor's first clip: `disc::bluray` builds streams from
// `play_items[0]`'s STN table, the only clip these PIDs were seen in.
// A later clip could reuse the PID for a different stream otherwise.
if let Some(clip) = title.clips.first() {
pid_map.insert((clip.clip_id.as_str(), *pid), pos);
}
@@ -477,12 +512,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
.map(|((c, p), v)| ((c.to_string(), p), v))
.collect();
// Labels that name their own stream, indexed by that name. No anchor, no
// sequence and no ordinal is involved in reaching one of these: the id was
// read from the same STN / ProgramInfo table `disc::bluray` built the
// stream from, so equality here is the same elementary stream by
// construction. Later duplicates lose, matching the first-wins rule the
// rest of this module uses.
// Labels that name their own stream, indexed by that name. The id was read
// from the same STN / ProgramInfo table `disc::bluray` built the stream
// from, so equality here is the same elementary stream by construction.
let by_id: HashMap<&StreamId, usize> = labels
.iter()
.enumerate()
@@ -518,9 +550,8 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
.unwrap_or_default();
// Resolve one stream to (label, authoritative). Authoritative means the
// label is known to belong to THIS stream rather than guessed onto it:
// tiers 1-3 of the doc comment above. Only tier 4, the bare ordinal, is
// not.
// label is known to belong to THIS stream rather than guessed onto it
// (tiers 1-3 of the doc comment above); only tier 4, the bare ordinal, isn't.
let resolve = |stream_type: StreamLabelType, idx: u16, pid: u16, lang: &str| {
if anchors[type_tag(stream_type) as usize] == Some(title_idx)
&& let Some(l) = label_at(labels, stream_type, idx)
@@ -556,15 +587,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
// Structured fields — callers translate purpose to UI text.
a.purpose = label.purpose;
// Codec descriptor: trust the parser's `codec_hint` ONLY
// when it's consistent with the stream's actual codec — it
// may legitimately be richer (e.g. "Dolby Atmos" on a TrueHD
// stream, which the raw spec codec can't express). If the
// hint CONTRADICTS the stream (a mis-bound / shuffled label,
// e.g. "AC-3 2.0" on a TrueHD track, or "TrueHD" on a DD+
// track), discard it and derive the descriptor from the
// stream itself — that's correct per-stream and can never be
// shuffled. An empty hint is left for `fill_defaults`.
// Trust the parser's `codec_hint` ONLY when consistent with the
// stream's actual codec (it may be richer, e.g. "Dolby Atmos" on
// TrueHD); if it CONTRADICTS, derive from the stream itself.
let codec_desc = if label.codec_hint.is_empty() {
// No codec hint — leave for fill_defaults.
String::new()
@@ -606,10 +631,9 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
sub_idx += 1;
if let Some((label, authoritative)) =
resolve(StreamLabelType::Subtitle, sub_idx, s.pid, &s.language)
// A subtitle label carries nothing but the qualifier,
// so an unverifiable one is all risk and no gain: off
// the authoritative path, require the label and the
// stream to state the same language.
// A subtitle label carries nothing but the qualifier, so an
// unverifiable one is all risk, no gain: off the authoritative
// path, require the label and stream to state the same language.
&& (authoritative
|| languages_agree(&label.language, &s.language))
{
@@ -744,19 +768,17 @@ fn codec_hint_consistent(hint: &str, codec: &crate::disc::Codec) -> bool {
let says_dts = !says_dts_ma && !says_dts_hr && h.contains("dts");
let says_lpcm = h.contains("lpcm") || h.contains("pcm");
let says_atmos = h.contains("atmos");
// DTS:X is an object-audio extension carried on a DTS-HD MA (or HR)
// core, exactly as Atmos rides TrueHD / DD+. The spec Codec enum has
// no DtsX variant, so a correctly-authored DTS:X hint must be judged
// consistent with its DtsHdMa/DtsHdHr carrier rather than discarded.
// DTS:X is an object-audio extension carried on a DTS-HD MA (or HR) core,
// as Atmos rides TrueHD/DD+. The spec Codec enum has no DtsX variant, so a
// correctly-authored hint must be judged consistent with its carrier.
let says_dtsx = h.contains("dts:x") || h.contains("dts-x") || h.contains("dtsx");
let names_family =
says_truehd || says_ddp || says_ac3 || says_dts_ma || says_dts_hr || says_dts || says_lpcm;
// Pure-editorial hint (no codec family named) isn't asserting a codec →
// consistent. "Atmos" alone implies a lossless carrier (TrueHD or DD+).
// ("DTS:X" always also matches the "dts" family above, so it never
// reaches this branch — it is handled in the DtsHdMa/DtsHdHr arms.)
// Pure-editorial hint (no codec family named) isn't asserting a codec, so
// it's consistent. "Atmos" alone implies a lossless carrier (TrueHD/DD+).
// ("DTS:X" always matches "dts" above, so it never reaches this branch.)
if !names_family {
return if says_atmos {
matches!(codec, Codec::TrueHd | Codec::Ac3Plus)
@@ -864,7 +886,10 @@ fn generate_audio_label_inner(
}
}
fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
fn extract(
reader: &mut dyn SectorSource,
udf: &UdfFs,
) -> (Vec<StreamLabel>, Option<FeaturePlaylistHint>) {
let mut candidates: Vec<(&'static str, ParseResult)> = Vec::new();
for (name, detect, parse) in PARSERS {
if !detect(reader, udf) {
@@ -879,16 +904,11 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
}
candidates.push((name, result));
}
// One tie-break rule, one implementation. `select_result` owns it and
// carries a regression test for a past bug where the LAST equal-confidence
// parser won instead of the first. `extract` — the path that actually
// ships — used to re-derive the same rule inline with a hand-rolled `>`
// scan and had no test of its own, so that fixed bug could have silently
// recurred here. Array order encodes a trust ordering (the hand-vetted
// parsers are registered ahead of the ones that detect on any BD-J disc),
// so "first wins on a tie" is load-bearing, not incidental.
// One tie-break rule, one implementation. `select_result` regression-tests a
// past bug where the LAST equal-confidence parser won instead of the first.
// Array order encodes trust (hand-vetted parsers before generic BD-J ones).
let best = select_result(&candidates).map(|(n, r)| (*n, r.clone()));
let (name, mut labels) = match best {
let (name, mut labels, feature_playlist) = match best {
Some((n, r)) => {
tracing::info!(
parser = n,
@@ -896,35 +916,29 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
label_count = r.labels.len(),
"label parser selected",
);
(n, r.labels)
(n, r.labels, r.feature_playlist)
}
None => {
tracing::info!("no label parser matched");
return Vec::new();
return (Vec::new(), None);
}
};
// The MPLS floor: framework parsers under-yield on multi-track discs
// because their authoring layer only ships editorial labels for
// "interesting" streams (Director's Cut, Atmos, SDH) and leaves the rest as
// plain numbered slots. MPLS sees every stream a playlist references, and
// names each one, so merging it in gives the user every track even when
// only the "interesting" ones have editorial names. Skipped when
// mpls_universal was itself the chosen parser (its labels ARE the labels).
// The MPLS floor: framework parsers under-yield on multi-track discs, only
// shipping editorial labels for "interesting" streams (Atmos, SDH). MPLS
// names every stream a playlist references, filling the gaps.
if name != "mpls_universal"
&& let Some(mpls_result) = mpls_universal::parse(reader, udf)
{
merge_mpls_floor(&mut labels, &mpls_result.labels);
}
// CLPI orphan streams: PIDs in /BDMV/CLIPINF/*.clpi ProgramInfo that no
// MPLS playlist references. Empirically a small fraction of streams are
// CLPI-only — physically on disc, not menu-reachable. They too are
// appended under the id they name, so a title reaches one only if it
// actually carries that stream.
// CLPI orphan streams: PIDs in /BDMV/CLIPINF/*.clpi ProgramInfo that no MPLS
// playlist references (physically on disc, not menu-reachable). Appended
// under the id they name, so a title reaches one only if it carries it.
let _orphans_added = append_clpi_orphans(&mut labels, reader, udf);
labels
(labels, feature_playlist)
}
/// Merge the MPLS-derived floor into a framework parser's label list.
@@ -1018,12 +1032,9 @@ fn append_clpi_orphans(
udf: &UdfFs,
) -> usize {
use crate::consts::coding_type as c;
// Two exclusions, one exact and one fuzzy. Exact: a stream some label
// already NAMES is not an orphan, whatever it looks like. Fuzzy: the
// pre-existing (type, language, codec_hint) test, kept because it is what
// bounds this list to a handful of entries per disc rather than one per
// stream per clip; it can only ever drop a candidate, and an orphan that
// never binds costs nothing when it is dropped.
// Two exclusions: exact (a stream some label already NAMES is not an orphan)
// and fuzzy (the pre-existing (type, language, codec_hint) test, kept to
// bound this list to a handful of entries per disc, not one per stream).
use std::collections::HashSet;
let named: HashSet<&StreamId> = labels.iter().filter_map(|l| l.stream_id.as_ref()).collect();
let existing: HashSet<(StreamLabelType, String, String)> = labels
@@ -1063,10 +1074,9 @@ fn append_clpi_orphans(
if !seen_pids.insert(s.pid) {
continue;
}
// Translate CLPI coding_type → label stream_type.
// 0x90 = Presentation Graphics (PG subtitle). 0x91 =
// Interactive Graphics (BD-J menu overlay), NOT a user-facing
// subtitle — skip it, matching the MPLS path which drops IG.
// Translate CLPI coding_type → label stream_type. 0x90 = Presentation
// Graphics (PG subtitle); 0x91 = Interactive Graphics (BD-J menu
// overlay), NOT a user-facing subtitle — skip it, matching MPLS.
let stype = match s.coding_type {
c::LPCM..=c::DTS_HD_MA | c::AC3_PLUS_SECONDARY | c::DTS_HD_SECONDARY => {
StreamLabelType::Audio
@@ -1178,16 +1188,13 @@ pub fn analyze(reader: &mut dyn SectorSource, udf: &UdfFs) -> LabelAnalysis {
};
// The MPLS floor: same merge as `extract()`. Skipped when MPLS was itself
// the chosen parser (its labels ARE the labels).
//
// Note this diagnostic path stops here — `extract()` also appends the CLPI
// orphans, which `analyze` has never reported.
// the chosen parser (its labels ARE the labels). This diagnostic path stops
// here — `extract()` also appends the CLPI orphans, which `analyze` never does.
let gap_fill_added = if parser.is_some() && parser != Some("mpls_universal") {
let before = labels.len();
// Re-run MPLS unconditionally — we only ran framework parsers
// above (we want to know which one to pick), and in the
// common case where MPLS would have detected but wasn't
// chosen we still need its labels for the merge.
// Re-run MPLS unconditionally: we only ran framework parsers above (to
// know which to pick), and in the common case where MPLS would have
// detected but wasn't chosen we still need its labels for the merge.
if let Some(mpls_result) = mpls_universal::parse(reader, udf) {
merge_mpls_floor(&mut labels, &mpls_result.labels);
}
@@ -1205,10 +1212,9 @@ pub fn analyze(reader: &mut dyn SectorSource, udf: &UdfFs) -> LabelAnalysis {
);
}
// bdmt runs independently of the parser registry: it's disc-level
// metadata (localized titles, box-set position), not per-stream
// labels, so the "highest confidence wins" logic doesn't apply.
// Always run if detected; surface result as a separate field.
// bdmt runs independently of the parser registry: it's disc-level metadata
// (localized titles, box-set position), not per-stream labels, so "highest
// confidence wins" doesn't apply. Always run if detected, as a separate field.
let disc_metadata = if bdmt::detect(udf) {
bdmt::parse(reader, udf)
} else {
@@ -1263,10 +1269,9 @@ fn collect_chapter_summary(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<Ch
if chapter_count == 0 {
continue;
}
// Duration: sum of (out_time - in_time) across play items,
// each in 45kHz PTS ticks → seconds. Approximates the disc
// module's per-title duration; we don't claim sample accuracy
// here, just enough to identify "the long one" (main movie).
// Duration: sum of (out_time - in_time) across play items, in 45kHz PTS
// ticks. Approximates the disc module's per-title duration; not
// sample-accurate, just enough to identify "the long one" (main movie).
let duration_ticks: u64 = playlist
.play_items
.iter()
@@ -1320,7 +1325,7 @@ pub struct LabelAnalysis {
/// tool.
pub gap_fill_added: usize,
/// Per-playlist chapter summary: `(playlist_filename, chapter_count, duration_secs)`.
/// Sourced from MPLS PlaylistMark entries with `mark_type ≤ 1`
/// Sourced from MPLS PlaylistMark entries with `mark_type == 1`
/// (chapter entries). Ordered by playlist filename. Empty if no
/// MPLS files have parseable marks, or the disc isn't Blu-ray.
pub chapter_summary: Vec<ChapterSummary>,
@@ -1526,15 +1531,16 @@ mod registry_tests {
"ctrm",
"dbp",
"deluxe",
"fox",
"mpls_universal",
"png_filenames",
],
"PARSERS array order changed — file-presence/reader-gated High \
parsers (paramount/criterion/pixelogic/ctrm) stay first; dbp + \
deluxe (now real com/<vendor>/ prefix detect) stay before \
mpls_universal; mpls_universal stays the universal Low fallback; \
png_filenames (Low, language-only hint) stays LAST so MPLS wins \
the Low tie whenever it produces anything."
deluxe (now real com/<vendor>/ prefix detect) and fox (dcx.xml / \
com/foxbd) stay before mpls_universal; mpls_universal stays the \
universal Low fallback; png_filenames (Low, language-only hint) \
stays LAST so MPLS wins the Low tie whenever it produces anything."
);
}
@@ -1556,9 +1562,37 @@ mod registry_tests {
ParseResult {
labels: vec![one_label()],
confidence: conf,
feature_playlist: None,
}
}
#[test]
fn feature_playlist_hint_matches_on_id_or_filename() {
let h = FeaturePlaylistHint {
playlist_id: Some(222),
filename: Some("00222.mpls".into()),
};
assert!(h.matches(222, "99999.mpls"), "numeric id match");
assert!(
h.matches(1, "00222.MPLS"),
"filename match is case-insensitive"
);
assert!(
!h.matches(1, "00001.mpls"),
"neither id nor filename → no match"
);
let id_only = FeaturePlaylistHint {
playlist_id: Some(5),
filename: None,
};
assert!(id_only.matches(5, "anything.mpls"));
assert!(!id_only.matches(6, "00005.mpls"));
assert!(FeaturePlaylistHint::default().is_empty());
assert!(!h.is_empty());
}
/// `select_result` must pick the highest-confidence non-empty result
/// and, on a confidence tie, the FIRST in array order (regression for the
/// old `analyze()` `max_by(...then(Equal))` no-op that picked the LAST).
@@ -1592,6 +1626,7 @@ mod registry_tests {
let empty = ParseResult {
labels: Vec::new(),
confidence: Confidence::High,
feature_playlist: None,
};
// High-confidence but empty must be skipped in favour of a
// non-empty lower-confidence result.
@@ -1608,18 +1643,14 @@ mod registry_tests {
#[test]
fn parsers_registry_all_entries_populated() {
for (name, detect, parse) in PARSERS {
// Function pointers can't be Null in safe Rust, so the
// assertion is just that the array entry was constructed
// — which the iter above already implies. The test
// exists to fail compile if someone changes the tuple
// shape (e.g. adds a 4th field) without updating callers,
// and as a marker for "these parsers exist."
// Function pointers can't be null in safe Rust, so this just fails to
// compile if the tuple shape changes (e.g. a 4th field is added)
// without updating callers; it also marks "these parsers exist."
let _ = (name, detect, parse);
}
// The loop above touches every registry entry. The non-empty
// invariant is covered separately by `parsers_registry_order_locked`,
// whose assert_eq! on the expected order fails if PARSERS is empty.
// This test fails to compile if the tuple shape changes.
// Non-empty invariant is covered separately by
// `parsers_registry_order_locked`, whose assert_eq! on the expected
// order fails if PARSERS is empty.
}
}
@@ -1779,11 +1810,9 @@ mod gap_fill_tests {
#[test]
fn orphan_append_skips_matching_type_lang_codec_tuples() {
// If a "would-be orphan" actually shares (type, lang, codec) with a
// label we already have, drop it — the user-facing rendering would be a
// confusing duplicate. This fuzzy test is what bounds the orphan list
// to a handful of entries per disc; it is a population rule only, and
// no longer decides where anything binds.
// If a "would-be orphan" shares (type, lang, codec) with a label we
// already have, drop it to avoid a confusing duplicate. This fuzzy
// test bounds the orphan list; it's a population rule, not a binding rule.
let labels = [
label(StreamLabelType::Audio, 1, "eng", "TrueHD"),
label(StreamLabelType::Audio, 2, "fra", "AC-3"),
@@ -1832,10 +1861,8 @@ mod gap_fill_tests {
}
// ── apply() integration tests ──────────────────────────────────────────────
//
// End-to-end coverage for the apply_labels + fill_defaults pipeline
// without needing a SectorSource / UdfFs. Synthetic DiscTitle +
// StreamLabel inputs, assert on the resulting Stream field values.
// End-to-end coverage for apply_labels + fill_defaults without needing a
// SectorSource / UdfFs. Synthetic DiscTitle + StreamLabel inputs.
#[cfg(test)]
mod apply_tests {
@@ -1952,15 +1979,9 @@ mod apply_tests {
subtitle(0x12A2, "fra"),
],
)];
// Capture through the crate's ONE global `tracing` subscriber:
// `testlog::capture` installs it exactly once and routes each event to a
// thread-local sink. A process-wide `set_global_default` HERE instead
// would poison every other test's callsite interest cache for the rest of
// the binary — `tracing` caches interest GLOBALLY — and only the first
// `set_global_default` in a process takes effect anyway. The shared
// subscriber answers interest for every callsite and isolates concurrent
// captures per thread, which is the invariant these log-accounting
// assertions rely on.
// `testlog::capture` installs the crate's ONE global `tracing` subscriber,
// routing events to a thread-local sink; `tracing` caches interest
// globally, so a per-test subscriber would poison other tests' callsites.
let ((), events) = crate::testlog::capture(|| {
apply_labels(&labels, &mut titles);
});
@@ -2131,10 +2152,9 @@ mod apply_tests {
#[test]
fn apply_keeps_consistent_dtsx_hint_on_dts_hd_ma() {
// DTS:X rides a DTS-HD MA core just as Atmos rides TrueHD. A
// correctly-authored "DTS:X" hint on a DtsHdMa stream is richer
// than the spec codec yet consistent, so it's kept verbatim —
// not discarded and regenerated to "DTS-HD Master Audio".
// DTS:X rides a DTS-HD MA core just as Atmos rides TrueHD. A correctly
// authored "DTS:X" hint on a DtsHdMa stream is richer than the spec
// codec yet consistent, so it's kept verbatim, not regenerated.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::DtsHdMa,
@@ -2169,10 +2189,9 @@ mod apply_tests {
#[test]
fn apply_normalizes_plain_consistent_hint_to_marketing() {
// A French DD+ track: a DD+ stream whose hint "AC-3+ 5.1" is correct
// but short-form. A sibling DD+ track that fell back uses the marketing
// form — keeping the short form here would read inconsistently, so a
// plain (non-richer) consistent hint is normalized to the stream's own.
// A DD+ stream whose hint "AC-3+ 5.1" is correct but short-form; a sibling
// fallback track uses the marketing form, so a plain (non-richer)
// consistent hint is normalized to the stream's own to stay consistent.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::Ac3Plus,
@@ -2918,13 +2937,8 @@ mod apply_tests {
}
// ── codec_hint_consistent: chained-OR boundary hardening ────────────────
//
// The family-detection booleans are built from chains of `h.contains(..)
// || h.contains(..) || ...` synonym checks. Each test below isolates ONE
// synonym clause (a hint string that matches that clause and NO other
// clause in the same chain) so a `||` -> `&&` flip at that specific
// position changes the family verdict — and, downstream, whether the
// codec match arm returns the spec-correct answer.
// The family-detection booleans chain `h.contains(..) || ...` synonym checks.
// Each test isolates ONE clause so a `||` -> `&&` flip there changes the verdict.
/// Isolates the `"true hd"` (space form) synonym in `says_truehd`,
/// which mutant testing hit at 396:44's `||`. If that `||` is
@@ -3233,10 +3247,9 @@ mod apply_tests {
&["00082", "00090"],
vec![subtitle(0x1200, "eng"), subtitle(0x1201, "fra")],
),
// A sibling playlist over the anchor's SECOND clip. Its subtitle
// reuses PID 0x1200 — PIDs are only unique within a clip — and it
// is Spanish, so nothing about the anchor's English SDH slot
// describes it.
// Sibling playlist over the anchor's SECOND clip; reuses PID 0x1200
// (PIDs are only unique within a clip) but is Spanish, so the
// anchor's English SDH slot doesn't describe it.
title_on_clips("00451.mpls", &["00090"], vec![subtitle(0x1200, "spa")]),
];
apply_labels(&labels, &mut titles);
@@ -3269,10 +3282,9 @@ mod apply_tests {
/// Mutation: as above — move `by_id` after the ordinal fallback.
#[test]
fn a_vendor_codec_claim_does_not_follow_the_ordinal_onto_a_bonus_clip() {
// The measured shape: this framework states no codec_hint at all and
// puts its descriptor in `name`, which `apply_labels` falls back to
// verbatim. Nothing about the stream is consulted on that path, so the
// codec-consistency guard never runs and cannot catch the mis-binding.
// This framework states no codec_hint and puts its descriptor in `name`,
// which `apply_labels` falls back to verbatim, so the codec-consistency
// guard never runs and cannot catch the mis-binding.
let feature_audio = StreamLabel {
name: "English Dolby Atmos".into(),
..audio_label(1, "eng", "", "")
+22 -42
View File
@@ -78,11 +78,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
// MPLS gives language + codec but never editorial info. Low confidence
// means framework parsers always win when they match; MPLS is only
// chosen when nothing else fired — the universal-fallback role we want.
Some(ParseResult::low(labels))
}
@@ -93,7 +91,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// needing a synthetic on-disc UDF image.
///
/// Identity is `(clip, PID)` — what the STN entry states — and it is both the
/// dedup key and the label's [`StreamId`]. A stream twenty playlists list is
/// dedup key and the label's [`StreamId`](super::StreamId). A stream twenty playlists list is
/// one label; two clips that both open their first audio at 0x1100 are two.
/// This replaced a disc-global dense counter that numbered surviving entries
/// 1, 2, 3, … in playlist-directory order: that number was not an STN slot in
@@ -104,25 +102,16 @@ fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
let mut seen: HashSet<super::StreamId> = HashSet::new();
for playlist in playlists {
// `Playlist::streams` is the FIRST play item's STN table, so every
// entry here is a stream of that play item's clip — the same clip
// `disc::bluray` records as the title's `clips[0]`. That pairing is
// what makes the PID an identity rather than a 16-bit number.
//
// Streams cannot be non-empty without a play item to have read them
// from, so the empty case is unreachable on a real disc; entries we
// cannot identify are skipped rather than emitted as unbindable
// labels.
// `Playlist::streams` is the FIRST play item's STN table, so every entry
// here is a stream of that play item's clip, matching `disc::bluray`'s
// `clips[0]` — that pairing is what makes the PID an identity.
let Some(clip_id) = playlist.play_items.first().map(|pi| pi.clip_id.clone()) else {
continue;
};
// 1-based STN slot within THIS playlist's table, per type — the
// `stream_number` field's documented meaning, counted the same way
// `disc::bluray` counts the stream list it builds from these entries.
// Nothing binds through it (these labels bind by id); it is stated
// truthfully rather than invented so that a reader of the label list
// sees where on its own playlist each stream sits.
// 1-based STN slot within THIS playlist's table, per type — matches how
// `disc::bluray` counts the same entries. Nothing binds through it (labels
// bind by id); it's stated truthfully so the label list shows real slots.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
@@ -201,13 +190,9 @@ fn label_type_for(entry: &crate::mpls::StreamEntry) -> Option<StreamLabelType> {
}
fn has_mpls_extension(name: &str) -> bool {
// Case-insensitive ".mpls" suffix. Some discs use uppercase,
// some lowercase; UDF filenames preserve case but we don't.
//
// UDF names are decoded via from_utf8_lossy, so a multi-byte
// replacement char (EF BF BD) can straddle byte index n-5; a raw
// byte slice there panics on a non-char-boundary. `ends_with` on a
// lowercased copy is char-boundary-safe and still case-insensitive.
// Case-insensitive ".mpls" suffix (discs use both cases). UDF names are
// decoded via from_utf8_lossy, so a multi-byte replacement char can straddle
// byte index n-5; `ends_with` on a lowercased copy avoids that boundary panic.
name.len() >= 5 && name.to_ascii_lowercase().ends_with(".mpls")
}
@@ -573,11 +558,9 @@ mod tests {
assert_eq!(id("fra"), Some(("00001".into(), 0x1101)));
assert_eq!(id("deu"), Some(("00001".into(), 0x1102)));
// `stream_number` is the entry's slot in ITS OWN playlist's STN table
// — deu is pl2's second audio, so 2, not "the third distinct stream
// seen while scanning the disc". It used to be the latter: a dense
// disc-global counter that named no table anyone could count against,
// handed to a binder that reads the field as an STN slot.
// `stream_number` is the entry's slot in ITS OWN playlist's STN table —
// deu is pl2's second audio, so 2, not "third distinct stream on disc".
// (Used to be a dense disc-global counter fed to a binder expecting a slot.)
let num = |lang: &str| {
labels
.iter()
@@ -735,10 +718,8 @@ mod tests {
#[test]
fn secondary_audio_becomes_audio_label() {
// stream_type 5 = secondary audio. The conversion should
// still produce an Audio label (the registry's apply path
// can ignore secondary if it wants — this module just
// surfaces what's there).
// stream_type 5 = secondary audio; conversion still produces an Audio
// label (the registry's apply path can ignore secondary if it wants).
let mut sec = audio_entry(0x1A00, 0x83, 3, 1, "eng");
sec.stream_type = 5;
sec.secondary = true;
@@ -875,11 +856,10 @@ mod tests {
#[test]
fn normalize_language_lowercases_and_trims() {
assert_eq!(
super::super::mpls_universal::language_display_name(&{
let trimmed = " ENG ".trim().to_ascii_lowercase();
// feed through production normalize_language logic
trimmed
}),
// trim + lowercase, mirroring production normalize_language
super::super::mpls_universal::language_display_name(
&" ENG ".trim().to_ascii_lowercase(),
),
"English"
);
}
+33 -38
View File
@@ -42,10 +42,27 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
if labels.is_empty() {
return None;
}
// Surface the feature playlist's identity so title selection can prefer
// it over a size-inflated decoy. The id is 5-digit zero-padded on BD
// (matching `NNNNN.mpls`); keep only digits to resist a stray quote/space.
let feature_playlist = super::xml::attr(&feature, "id").and_then(|id| {
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
Some(super::FeaturePlaylistHint {
playlist_id: digits.parse::<u16>().ok(),
filename: Some(format!("{digits}.mpls")),
})
});
// High confidence: this format is fully structured and we extract
// every field whose meaning the corpus establishes. "Documented" would
// be the wrong word — see the module note; nothing about it is.
Some(ParseResult::high(labels))
let mut result = ParseResult::high(labels);
result.feature_playlist = feature_playlist;
Some(result)
}
/// One cell of the `forced_sub` CSV.
@@ -183,32 +200,14 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse audio streams
if let Some(aud) = xml::attr(feature, "aud") {
// aud_com1_idx is a trimmed, comma-separated list of CSV positions
// (some authoring tools emit whitespace, and multiple commentary
// tracks are possible) — symmetric with sub_com1_idx below.
// A HashSet, not a Vec: `com_indices` is parsed straight out of an
// attacker-controlled attribute with no length bound and was scanned
// linearly once per stream, so `aud="..."` and `aud_com1_idx="..."`
// both grown large make this quadratic in the size of one XML file.
// Membership is the only operation performed on it.
// aud_com1_idx: trimmed CSV positions, symmetric with sub_com1_idx.
// HashSet not Vec: parsed from an attacker-controlled attribute and
// only membership-tested; a Vec would make large inputs quadratic.
let com_indices = com_indices(xml::attr(feature, "aud_com1_idx"));
// The CSV *is* the STN list: one cell per stream, in stream order,
// and `aud_com1_idx` is a 0-based index into those same cells. So
// `stream_number` is the cell's own 1-based position — NOT a counter
// that only advances on cells carrying a language.
//
// A cell with an empty language still occupies its STN slot; it just
// has nothing to label. Renumbering the surviving cells 1..N shifts
// every label behind an empty cell one slot forward, which is how a
// marker authored for one stream ends up written onto the stream in
// front of it (see the subtitle side, where the marker is `forced`).
//
// `u16::try_from` rather than `saturating_add`: past the 1-based u16
// numbering space every cell would collapse onto `u16::MAX`, binding
// several streams to one label. Stop emitting instead. Unreachable on
// real media — the BD STN_table admits at most 32 primary audio
// streams per playlist.
// The CSV *is* the STN list, so `stream_number` is each cell's 1-based
// position, not a counter over labeled cells (empty cells still occupy
// their slot). `u16::try_from`: stop rather than wrap onto `u16::MAX`.
for (i, lang) in aud.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
@@ -244,11 +243,9 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// parsed input, membership-only use, linear scan once per stream.
let com_indices = com_indices(xml::attr(feature, "sub_com1_idx"));
// As with audio: the cell position IS the STN slot. `forced_sub` and
// `sub_com1_idx` are indexed against those same cells, so an empty
// cell must not renumber the cells behind it — a forced marker
// authored for one PG slot would otherwise be written onto an
// earlier, full-dialogue subtitle track.
// As with audio: the cell position IS the STN slot, indexed by
// `forced_sub`/`sub_com1_idx`, so an empty cell must not renumber
// the rest — else a forced marker lands on the wrong subtitle track.
for (i, lang) in sub.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
@@ -602,10 +599,9 @@ mod tests {
#[test]
fn aud_com1_idx_trimmed_and_multivalue() {
// Whitespace around the index, and a multi-value list, must both
// resolve. com index is positional against the raw CSV, so with
// an empty slot at position 1, " 2 " marks the 'fra' track
// (CSV index 2, STN slot 3) as commentary.
// Whitespace and multi-value lists must both resolve; com index is
// positional against the raw CSV, so with an empty slot at position 1,
// " 2 " marks 'fra' (CSV index 2, STN slot 3) as commentary.
let feature = r#"<playlist aud="eng,,fra" aud_com1_idx=" 2 " />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -764,10 +760,9 @@ mod tests {
/// Mutation: cast `i + 1` to u16 → stream numbers wrap to 0, skipping apply.
#[test]
fn audio_stream_number_never_wraps() {
// 65535 audio tracks is impossible on a real disc but the parser must
// not panic or produce 0. Build a comma-separated list of 65535 "eng"s.
// We only run the number-assignment logic via labels_from_feature.
// Limit: CSV with 300 slots is sufficient to test the counter.
// 65535 tracks is impossible on a real disc but must not panic/produce 0.
// 300 slots is sufficient to exercise the number-assignment logic
// via labels_from_feature without building the full 65535-entry CSV.
let aud: String = (0..300).map(|_| "eng").collect::<Vec<_>>().join(",");
let feature = format!(r#"<playlist name="Feature" aud="{}" />"#, aud);
let labels = labels_from_feature(&feature);
+47 -126
View File
@@ -111,11 +111,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// shortest meaningful run is 4 chars (lang + underscore).
let strings = text::extract_ascii_strings(&data, 4);
// Tracked across all parse_token calls in this run: which uncatalogued
// token components did the blob contain? If any, we downgrade confidence
// to Medium — the labels are still valid but the disc surfaced something
// we don't catalogue — and report them once (see `UnknownParts`). Parsing
// is single-threaded and sequential, so a plain owned collector suffices.
// Collects uncatalogued token components; if any, confidence downgrades to Medium
// and they're reported once (see `UnknownParts`). Sequential parse, so a plain
// owned collector suffices.
let mut unknown = UnknownParts::default();
let labels = assign_labels(&strings, &mut unknown);
@@ -133,7 +131,11 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
} else {
Confidence::Medium
};
Some(ParseResult { labels, confidence })
Some(ParseResult {
labels,
confidence,
feature_playlist: None,
})
}
/// Walk the extracted token strings of the feature section and emit a
@@ -141,14 +143,9 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// from `parse` so the section/numbering logic is unit-testable without
/// a `SectorSource`/`UdfFs`.
fn assign_labels(strings: &[String], unknown: &mut UnknownParts) -> Vec<StreamLabel> {
// The authoritative per-feature stream list lives in the `FPL_`
// (FeaturePLaylist) section, in STN order. `SEG_*` entries are menu
// segments (intros, logos, disclaimers, previews) that can also carry
// stray stream tokens — e.g. a `SEG_MainFeature` preview segment that
// lists only a commentary track. Anchoring on such a segment grabs the
// wrong streams and misnumbers them. So when the project ships any
// `FPL_` playlist, anchor exclusively on it; only fall back to
// `SEG_MainFeature` on discs that have no `FPL_` section at all.
// The authoritative per-feature stream list lives in `FPL_` (FeaturePlaylist), in
// STN order. `SEG_*` menu segments can carry stray tokens, so anchor on `FPL_` when
// present; fall back to `SEG_MainFeature` only if no `FPL_` section exists.
let has_fpl = strings.iter().any(|s| s.starts_with("FPL_"));
let mut labels = Vec::new();
@@ -191,27 +188,9 @@ fn assign_labels(strings: &[String], unknown: &mut UnknownParts) -> Vec<StreamLa
continue;
}
// Second way a section ends, and the only one some discs give us.
//
// The `SEG_`/`SF_`/`FPL_` markers above are section NAMES, and not every
// section is named: a project's trailing sections (the per-language
// notice, disclaimer and dub-credit cards) are emitted with a plain
// clip name, or none at all. When the feature playlist is the last
// NAMED section in the blob — which it is on most of the corpus — the
// name-only rule never fires again and the walk swallows the whole
// tail of the file as if it were more of the feature's stream list.
// Those trailing clip names are per-language (`{lang3}_{card}`), so
// they pass `is_stream_token` and each one silently advances an STN
// counter; the ones whose card name collides with a catalogued
// component (an `AC` card reads as the AC-3 codec) even emit a label,
// for an STN slot the feature playlist does not have.
//
// What every section does have, named or not, is a stream list that
// OPENS with its video slots. So a `Video Stream N` entry that repeats
// one this section already listed cannot be another slot in this
// section — it is the first entry of the next one, and this section is
// over. Distinct video entries are kept (a section may legitimately
// list a secondary video stream alongside the primary).
// Second section-end signal (the only one some discs give): every section's stream
// list OPENS with its video slots, so a `Video Stream N` repeating one this section
// listed marks the next section's start. Guards against swallowing trailing cards.
if s.starts_with("Video Stream") {
if video_slots.contains(&s.as_str()) || video_slots.len() >= MAX_VIDEO_SLOTS {
break;
@@ -226,26 +205,9 @@ fn assign_labels(strings: &[String], unknown: &mut UnknownParts) -> Vec<StreamLa
break;
}
// Every entry in the section's stream list occupies one STN slot,
// whether or not it carries anything worth labelling. Pixelogic lists
// a slot as an editorial `{lang}_{codec|purpose|region}_…` token, or as
// a bare `Audio Stream N` / `PG Stream N` placeholder when no editorial
// label was authored. Both must advance the per-type counter, and so
// must an editorial token the grammar cannot classify — a
// region-only token (`fra_CF_`, `spa_LS_`: REGIONS sets `variant` but
// neither `is_audio` nor `is_subtitle`, so `parse_token_inner` returns
// `None`) or one whose only distinguishing component is uncatalogued
// (`jpn_DUB_`). Numbering only the slots that PARSE renumbers the rest
// 1..N and lands every surviving label on the wrong stream: on
// one observed UHD feature the seven `*_TXT_FOR_` forced tokens at PG
// STN 11-16/18 collapsed onto STN 2-8, flagging that disc's FULL
// subtitle tracks `forced` and leaving the real forced-narrative
// tracks unflagged.
//
// A slot the grammar cannot classify carries no stream type either, so
// it advances the list currently being enumerated: pixelogic sections
// run video → audio → PG, and `domain` follows the last slot whose type
// WAS known (a typed token or a typed placeholder), starting at Audio.
// Every stream-list entry occupies one STN slot (editorial token, bare placeholder,
// or unclassifiable token) and ALL must advance the per-type counter, else labels
// renumber onto wrong streams. Unclassifiable slots follow `domain` (video→audio→PG).
if let Some(kind) = placeholder_kind(s) {
domain = kind;
match kind {
@@ -365,12 +327,9 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
if raw_part.is_empty() {
continue;
}
// Token components are spec-uppercase (codec IDs, ADES/ACOM/SDH,
// region codes). vocab elsewhere is deliberately case-insensitive,
// so normalize each component to uppercase before the gate to
// avoid silently dropping a lowercase-authored token (which would
// fall through to the unknown branch and, with no is_audio/
// is_subtitle set, get the whole stream discarded below).
// Token components are spec-uppercase (codec IDs, ADES/ACOM/SDH, region codes).
// Normalize each to uppercase before the gate so a lowercase-authored token isn't
// silently dropped through the unknown branch (no is_audio/is_subtitle set).
let part_up = raw_part.to_ascii_uppercase();
let part = part_up.as_str();
if AUDIO_CODECS.contains(&part) {
@@ -395,37 +354,15 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
} else if part == "STRI" || part == "TXT" {
is_subtitle = true;
} else if part == "FOR" {
// `FOR` (forced) is a subtitle-domain qualifier. A token whose
// only non-language component is FOR (e.g. `eng_FOR_`) would
// otherwise classify as neither audio nor subtitle and be
// dropped at the `!is_audio && !is_subtitle` guard below. Treat
// a forced marker as a subtitle signal so the stream survives.
// `FOR` (forced) is a subtitle-domain qualifier. Treat it as a subtitle signal
// so a token whose only non-language component is FOR (e.g. `eng_FOR_`) isn't
// dropped at the `!is_audio && !is_subtitle` guard below.
qualifier = LabelQualifier::Forced;
is_subtitle = true;
} else if part == "DUB" {
// `DUB` names the forced-narrative subtitle track authored to
// accompany that language's DUBBED audio presentation — the
// signs/on-screen-text pass a viewer needs when the dialogue is
// already dubbed. Same editorial class as `*_TXT_FOR_`, spelled
// differently by some authoring runs.
//
// Evidence (two independent discs in the corpus): the token
// appears only inside the PG list, embedded in an otherwise
// contiguous run of `{lang}[_{region}]_TXT_FOR_` siblings — one
// forced-narrative slot per localized language — and takes the
// slot where that language's forced entry belongs. The stream it
// lands on is a sparse PG track (sub-megabyte, a few dozen
// display sets), the signature of a forced-narrative pass rather
// than a full subtitle track.
//
// Deliberately NOT added to `vocab::qualifier`: that maps
// free-form ENGLISH label text, where a bare word "dub" means
// dubbed AUDIO. The forced-subtitle reading is specific to this
// vendor's token grammar, so it stays token-local here.
//
// Like `FOR`, this is a subtitle-domain signal: a token whose
// only non-language component is `DUB` must still classify as a
// subtitle or the `!is_audio && !is_subtitle` guard drops it.
// `DUB` = forced-narrative subtitle for a language's dubbed audio (same class as
// `*_TXT_FOR_`). Token-local, NOT in `vocab::qualifier` (English "dub" = dubbed
// AUDIO). Like FOR, a subtitle-domain signal so the guard keeps the stream.
qualifier = LabelQualifier::Forced;
is_subtitle = true;
} else if REGIONS.contains(&part) {
@@ -433,15 +370,10 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
} else if part.starts_with("PGSTREAM") {
is_subtitle = true;
} else {
// Unknown token component — skip this single part rather
// than discarding the entire stream record. Pre-refactor
// behavior was `return None` here, which silently dropped
// any stream containing a single uncatalogued token (e.g.
// a new codec ID or framework variant). Better to surface
// what we know than discard a whole stream over one part,
// but flag the parse as Medium-confidence so callers know
// some data was elided.
tracing::debug!(part = %part, "pixelogic: unrecognized token component, skipping");
// Unknown token component — skip this single part rather than discarding the
// whole stream record (pre-refactor `return None` dropped streams over one
// uncatalogued token). Recorded so the parse downgrades to Medium confidence.
tracing::debug!(part = ?part, "pixelogic: unrecognized token component, skipping");
if let Some(acc) = unknown.as_deref_mut() {
acc.record(part);
}
@@ -452,12 +384,9 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
return None;
}
// Tie-break for tokens that signal both domains (e.g. `eng_MLP_SDH_`
// sets is_audio via the codec and is_subtitle via SDH). An audio
// codec hint is the stronger, audio-domain signal, so prefer Audio
// when one is present (keeps the parsed codec_hint instead of
// discarding it); otherwise file as Subtitle. Pure-subtitle and
// pure-audio tokens are unaffected.
// Tie-break for tokens signalling both domains (e.g. `eng_MLP_SDH_`: is_audio via codec,
// is_subtitle via SDH). An audio codec is the stronger signal, so prefer Audio when
// present (keeps codec_hint); otherwise Subtitle. Pure tokens are unaffected.
let has_audio_codec = is_audio && !codec.is_empty();
let stream_type = if is_subtitle && !has_audio_codec {
StreamLabelType::Subtitle
@@ -624,11 +553,9 @@ mod tests {
#[test]
fn parse_token_components_are_case_insensitive() {
// Regression for the case-sensitive gate: a lowercase codec/
// qualifier component must classify identically to uppercase
// rather than falling through to the unknown branch and getting
// the whole stream dropped. The ISO 639-2 lang prefix is still
// required lowercase.
// Regression for the case-sensitive gate: a lowercase codec/qualifier component must
// classify identically to uppercase, not fall through to the unknown branch and drop
// the stream. The ISO 639-2 lang prefix is still required lowercase.
let l = parse_token_inner("eng_mlp_", None).unwrap();
assert_eq!(l.stream_type, StreamLabelType::Audio);
assert_eq!(l.codec_hint, "TrueHD");
@@ -653,11 +580,9 @@ mod tests {
#[test]
fn assign_labels_numbers_commentary_behind_placeholders() {
// Observed case: the FPL_MainFeature playlist lists three unlabelled main
// audio tracks as `Audio Stream N` placeholders, then a lone
// `eng_ACOM_` commentary at STN slot 4. The commentary must land on
// audio #4, not collapse onto #1 (which would tag the main feature
// track as commentary).
// Observed case: FPL_MainFeature lists three unlabelled `Audio Stream N`
// placeholders, then a lone `eng_ACOM_` commentary at STN slot 4. It must land on
// audio #4, not collapse onto #1 (which would tag the main track as commentary).
let mut flag = UnknownParts::default();
let tokens = strs(&[
"FPL_MainFeature",
@@ -759,10 +684,9 @@ mod tests {
#[test]
fn assign_labels_prefers_fpl_over_seg_mainfeature() {
// A `SEG_MainFeature` menu/preview segment carries a stray commentary
// token, but the real playlist is `FPL_MainFeature`. When an FPL_
// section exists, the SEG_ one must be ignored as an anchor — so we
// number from the FPL playlist, putting the commentary at slot 2.
// A `SEG_MainFeature` segment carries a stray commentary token, but the real
// playlist is `FPL_MainFeature`. When an FPL_ section exists the SEG_ one is ignored
// as anchor, so numbering comes from the FPL playlist (commentary at slot 2).
let mut flag = UnknownParts::default();
let tokens = strs(&[
"SEG_MainFeature",
@@ -903,8 +827,8 @@ mod tests {
}
/// Spec: is_audio wins over is_subtitle when codec explicitly identified.
/// Tests the commentary audio case — `ACOM` sets both is_audio purpose and SDH-only-is-subtitle:
/// MLP codec wins → Audio.
/// `SDH` alone would suggest a subtitle, but an explicit `AC3` codec token
/// wins the tiebreak → Audio.
/// Mutation: flip the tie-break → Subtitle returned when codec present.
#[test]
fn parse_token_codec_always_wins_type_tiebreak() {
@@ -1070,12 +994,9 @@ mod tests {
],
"only the feature playlist's own slots are labelled"
);
// A card's name is emitted BEFORE its own section's video slot, so a
// forward-only walk meets the first one while still nominally inside
// the feature section and counts it. That residue is one entry, at the
// tail of a list nothing follows in — it cannot renumber any label,
// only cost the parse its High confidence. Every card behind it is
// past the boundary and never seen. Pinned rather than papered over.
// A card's name is emitted BEFORE its section's video slot, so a forward-only walk
// counts the first one while still nominally inside the feature section. That lone
// tail entry can't renumber any label, only cost High confidence. Pinned, not hidden.
assert_eq!(
flag.seen.iter().map(String::as_str).collect::<Vec<_>>(),
vec!["WARNING"],
+1 -1
View File
@@ -7,7 +7,7 @@
//! of menu languages the disc ships.
//!
//! This is a language-only hint (no per-stream purpose/codec), so it runs at
//! [`Confidence::Low`] — it never displaces a real framework parser, and it
//! [`Confidence::Low`](super::Confidence::Low) — it never displaces a real framework parser, and it
//! sits at the same tier as the MPLS floor. It is here so the pattern is a
//! first-class, testable parser that keeps picking up discs as the corpus
//! grows, rather than lost logic. Detection is precise: it fires only on the
Binary file not shown.
Binary file not shown.
Binary file not shown.
+7 -8
View File
@@ -454,7 +454,8 @@ pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
/// Classify a free-form English label string into a [`LabelPurpose`].
///
/// Recognized keywords (case-insensitive, word-boundary matched):
/// Recognized keywords (case-insensitive; single-word keywords are
/// word-boundary matched, multi-word phrases are substring matched):
/// - "commentary", "director's commentary" → `Commentary`
/// - "descriptive", "description", "audio description", "described" → `Descriptive`
/// - "score", "music only" → `Score`
@@ -462,7 +463,8 @@ pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
/// - anything else → `Normal`
///
/// Word-boundary matching means "Commentary track" matches but
/// "Commenter Pro audio" does not.
/// "Commenter Pro audio" does not. Multi-word phrases like "audio
/// description" and "music only" are matched as plain substrings.
pub fn purpose(text: &str) -> LabelPurpose {
let lower = text.to_lowercase();
// Multi-word compounds first — they're more specific.
@@ -531,12 +533,9 @@ fn has_word(haystack: &str, needle: &str) -> bool {
if needle.is_empty() {
return false;
}
// Boundary check is char-aware (not byte-level): a non-ASCII letter
// adjacent to the match (e.g. an accented or CJK char, which is
// multiple UTF-8 bytes) is alphanumeric and so is NOT a boundary,
// preventing false positives like "sdh" inside "cafésch". Needles
// are ASCII tokens, so a byte-offset match aligns with char
// boundaries in `haystack`.
// Char-aware, not byte-level: an accented/CJK char adjacent to the match is
// alphanumeric and so NOT a boundary, preventing false positives like "sdh"
// inside "cafésch". Needles are ASCII, so byte offsets align with char bounds.
for (idx, _) in haystack.match_indices(needle) {
// Char immediately before the match.
let before_is_alnum = haystack[..idx]
+8 -13
View File
@@ -110,10 +110,9 @@ pub fn attr(element: &str, name: &str) -> Option<String> {
/// prior behavior in criterion.rs).
pub fn text(xml: &str, tag: &str) -> Option<String> {
let (_open_end, body_start) = find_open_tag(xml, tag, 0)?;
// For self-closing tags, body_start is past `/>` and there is no
// content. Detect with a *byte* comparison: slicing `&xml[..]` two
// bytes back can land inside a multi-byte UTF-8 char and panic
// (untrusted on-disc XML), but indexing the byte slice never does.
// For self-closing tags, body_start is past `/>` with no content. Detect via
// a *byte* comparison: slicing `&xml[..]` two bytes back can land inside a
// multi-byte UTF-8 char and panic on untrusted XML; byte indexing never does.
let b = xml.as_bytes();
if body_start >= 2 && b[body_start - 2] == b'/' && b[body_start - 1] == b'>' {
return Some(String::new());
@@ -462,10 +461,9 @@ mod tests {
#[test]
fn text_multibyte_before_self_close_does_not_panic() {
// A multi-byte UTF-8 char ending right before the `/>` used to
// panic on a non-char-boundary str slice in `text()`. The
// byte-level self-closing check must handle it cleanly.
// 'é' (0xC3 0xA9) directly precedes the `/>`.
// A multi-byte UTF-8 char ('é' = 0xC3 0xA9) right before `/>` used to
// panic on a non-char-boundary str slice in `text()`; the byte-level
// self-closing check must handle it cleanly.
assert_eq!(text("<x>é</x>", "x"), Some("é".into()));
// Self-closing form with a multi-byte char in an attr value.
assert_eq!(text(r#"<x a="é"/>"#, "x"), Some("".into()));
@@ -636,11 +634,8 @@ mod tests {
}
// ── Malformed / truncated input (untrusted on-disc XML) ────────────────
//
// These scrapers run on XML lifted out of BD-J jar entries, which is
// attacker-controllable. Every scan in this module must terminate and
// stay in bounds on truncated or unbalanced input rather than panic.
// XML 1.0 §2.3 defines the Name production these boundary rules model.
// These scrapers run on attacker-controllable BD-J jar XML, so every scan
// must terminate and stay in bounds on truncated input (XML 1.0 §2.3).
/// A quoted attribute value that is never closed must terminate the
/// scan at EOF rather than reading past the end of the buffer.
+25 -81
View File
@@ -101,6 +101,7 @@ pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFI
pub(crate) const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs;
pub(crate) mod bdnav;
pub(crate) mod clpi;
pub mod consts;
pub mod css;
@@ -109,7 +110,7 @@ pub mod diag;
pub mod dirimage;
pub mod disc;
pub mod drive;
pub mod dvdnav;
pub(crate) mod dvdnav;
pub mod error;
pub mod event;
pub mod halt;
@@ -135,65 +136,43 @@ pub(crate) mod udf;
pub(crate) mod unlock_bridge;
// ─── Drive lifecycle ────────────────────────────────────────────────────────
//
// `Drive::open(path)` → `wait_ready()` → `init()` → `Disc::scan()`. `Drive`
// owns the SCSI session; `DriveCapture` etc. let advanced callers introspect
// drive identity / profile data for sharing.
// `Drive::open(path)` → `wait_ready()` → `init()` → `Disc::scan()`. `Drive` owns
// the SCSI session; `DriveCapture` etc. expose drive identity/profile data.
pub use drive::capture::{
CapturedFeature, DriveCapture, capture_drive_data, mask_bytes, mask_string,
};
pub use drive::{Drive, DriveStatus, extract_scsi_context, find_drive};
// ─── Disc session (drive open + SCSI bring-up hoist) ─────────────────────────
//
// One entry point that opens a drive and brings the transport up, so consumers
// stop hand-rolling `open → wait_ready → init → probe_disc → identify → scan`.
// Owns the `Drive` by value; forwards consumer-built key material into
// `ScanOptions` (the library derives no certs — see `KeySpec`).
// One entry point opens a drive, brings transport up, and forwards caller-built
// key material into `ScanOptions` (the library derives no certs; see `KeySpec`).
pub use session::{
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_dir,
scan_iso,
};
// ─── Errors ─────────────────────────────────────────────────────────────────
//
// All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a
// numeric `code()`; **no English text in the library** — applications map
// codes to localized messages. See `error.rs` for the full taxonomy.
// All fallible APIs return `Result<T, Error>`; `Error` is a typed enum with a
// numeric `code()` — no English text in the library; see `error.rs` for taxonomy.
pub use error::{
Error, Result, error_code, is_disc_level_no_key, is_halt, is_skippable_title_stub,
};
// ─── Cooperative cancellation ───────────────────────────────────────────────
//
// One-bit cooperative cancellation token, shared by every long-running loop —
// libfreemkv's mux, and the recovery passes (sweep/patch) that now live in the
// freemkv-engine crate. Clone it cheaply; pass it by value into each component;
// poll `is_cancelled()` inside the loop body.
// One-bit cancellation token shared by every long-running loop (mux, and the
// engine's sweep/patch passes); clone cheaply, poll `is_cancelled()` in loops.
pub use halt::Halt;
// Generic bounded producer/consumer primitive used by the mux pipeline (and,
// via this re-export, by the engine's sweep/patch recovery passes) to overlap
// reads with writes via a dedicated consumer thread.
// `Pipeline::spawn(name, depth, sink)` spawns a named consumer; `pipe.send(item)`
// pushes one item with back-pressure; `pipe.finish()` joins the
// consumer and surfaces its `close()` output. Callers implement `Sink`
// to define per-item behaviour and end-of-stream finalisation.
//
// `DEFAULT_PIPELINE_DEPTH` (=4) is for callers without specific needs;
// most should use WRITE_PIPELINE_DEPTH instead.
// Patch uses `WRITE_THROUGH_DEPTH` (=1). Returning `Flow::Stop` from
// `apply` ends the consumer cleanly (still calls `close()`).
// Bounded producer/consumer primitive used by mux (and engine's sweep/patch) to
// overlap reads with writes via a consumer thread. `Sink::apply` defines
// per-item behavior; `Flow::Stop` ends the consumer cleanly, still calling `close()`.
pub use io::pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
// ─── Bounded-cache buffered file writer ─────────────────────────────────────
//
// Drop-in `std::fs::File` replacement used everywhere the lib writes large
// sequential output (mux, extract, sweep, patch) — drains dirty pages
// continuously instead of bursting. General I/O infra, not recovery policy;
// promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too.
// Drop-in `std::fs::File` replacement for large sequential output (mux, extract,
// sweep, patch); drains dirty pages continuously. `pub` for freemkv-engine reuse.
pub use io::WritebackFile;
/// Write an image-level source out as a sector image — what an `iso://`
/// DESTINATION means for any source that is not a physical drive. Drive sources
@@ -206,32 +185,17 @@ pub use event::{BatchSizeReason, Event, EventKind};
pub use identity::DriveId;
// ─── Unlock seam ────────────────────────────────────────────────────────────
//
// Drive/disc unlocking (removing bus encryption — firmware, AACS cert, CSS
// bus-auth) lives entirely in the `freemkv-unlock` crate. libfreemkv consumes
// it through the private `unlock_bridge` and exposes nothing of it: clients are
// oblivious to unlockers, exactly as they are to the SCSI layer. There is no
// public unlock surface to import.
// Drive/disc unlocking (firmware, AACS cert, CSS bus-auth) lives entirely in
// `freemkv-unlock`; libfreemkv consumes it via `unlock_bridge` and exposes nothing.
// ─── Decryption (AACS / CSS) ────────────────────────────────────────────────
//
// `Disc::scan()` resolves keys and stores them on `Disc`; in most flows you
// don't touch `DecryptKeys` directly — `DiscStream::new(reader, title, keys, …)`
// accepts whatever `Disc::decrypt_keys()` returned. `decrypt_sectors()` is
// for callers that operate on raw sector buffers (e.g. ISO patching).
// `Disc::scan()` resolves keys onto `Disc`; `DiscStream::new(...)` consumes
// them directly. `decrypt_sectors()` is for raw sector buffers (ISO patching).
pub use decrypt::{AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_threads};
// ─── Disc structure ─────────────────────────────────────────────────────────
//
// `Disc::scan()` produces a fully-populated `Disc` (titles, streams, AACS
// state). `Disc::identify()` is the fast path — UDF only, no playlist parse,
// for displaying disc name + format quickly while a full scan runs in the
// background. The codec / channel / resolution enums are the canonical
// structured representation; never compare against display strings.
// Note: `disc::Stream` here is the codec enum (audio / video / sub kind)
// — not the `pes::Stream` trait re-exported below as `PesStream`. Two
// different concepts, the same short name; the trait gets the `Pes`
// prefix at the crate root to keep both addressable.
// `Disc::scan()` fully populates `Disc`; `Disc::identify()` is a UDF-only fast path
// for name/format display. Codec enums are canonical; never compare display strings.
pub use dirimage::DirImage;
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
@@ -242,23 +206,8 @@ pub use disc::{
pub use keysource::{DiscInputs, KeySource, read_encrypted_units, resolve_and_apply};
// ─── Streams ────────────────────────────────────────────────────────────────
//
// All stream types implement `pes::Stream` — read PES frames from a source,
// write PES frames to a sink. Pick the right type at construction:
//
// - `DiscStream` — physical drive or ISO (any `SectorSource`). Read-only.
// - `MkvStream` — Matroska container. Read on `open()`, write on `create()`.
// - `M2tsStream` — Blu-ray Transport Stream. Write-only sink (`create()`).
// - `NetworkStream` — TCP. Read on `listen()`, write on `connect()`.
// - `NullStream` — write-only black-hole sink. Useful for benchmarks.
// - `StdioStream` — pipe to/from stdin/stdout. Read or write.
//
// Most consumers use the URL resolvers (`input()` / `output()`) which pick
// the right type from a scheme:// URL. Direct construction is for callers
// that need to wire custom readers (e.g. autorip's drive-session reuse).
// The trait is re-exported as `PesStream` here to disambiguate from
// `disc::Stream` (the codec-kind enum re-exported above), which would
// otherwise collide at the crate root.
// All types implement `pes::Stream` (re-exported `PesStream` to avoid colliding
// with `disc::Stream`). Prefer `input()`/`output()` URL resolvers over direct construction.
pub use pes::PesFrame;
pub use pes::Stream as PesStream;
@@ -274,13 +223,8 @@ pub use mux::{Medium, SourceInfo};
pub use mux::{Mp4FitReport, Mp4Sink, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
// `ScsiTransport` is the platform-abstraction trait Drive uses; expose for
// out-of-tree platform backends. `SectorSource` / `SectorSink` are the
// direction-typed read/write traits; `FileSectorSource` and `FileSectorSink`
// are the ISO-on-disk implementations. [`DecryptingSectorSource`] is the
// single decrypt-on-read decorator (AACS / CSS / none) — wrap any
// `SectorSource` to get plaintext sectors out.
// `ScsiTransport` is the platform trait Drive uses, exposed for out-of-tree
// backends. `DecryptingSectorSource` wraps any `SectorSource` to decrypt (AACS/CSS).
pub use mux::build_iso_pipeline;
pub use mux::resolve_mux_key_map;
pub use mux::select::{PidFilter, StreamSelection};
+47 -57
View File
@@ -4,7 +4,7 @@
//! Contains play items (clips) with in/out timestamps,
//! stream info (video, audio, subtitle tracks).
//!
//! Reference: https://github.com/lw/BluRay/wiki/MPLS
//! Format is documented in the BD-ROM PlayList (MPLS) specification.
use crate::error::{Error, Result};
@@ -146,17 +146,16 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
}
let clip_id = String::from_utf8_lossy(&item[0..5]).to_string();
let connection_condition = item[9] & 0x0F;
// After clip_id[0..5] and codec_id[5..9], item[9] is fully reserved;
// item[10] holds is_multi_angle (bit 4) plus connection_condition in its
// low nibble. stc_id follows at item[11] and in_time at item[12..16].
let connection_condition = item[10] & 0x0F;
let in_time = u32::from_be_bytes([item[12], item[13], item[14], item[15]]);
let out_time = u32::from_be_bytes([item[16], item[17], item[18], item[19]]);
// Parse STN table from the first play item
// PlayItem layout after out_time:
// [20:28] UO_mask_table (8 bytes)
// [28] misc flags (1 byte)
// [29] still_mode (1 byte)
// [30:32] still_time (2 bytes)
// [32:] STN_table
// Parse STN table from the first play item. PlayItem layout after out_time:
// UO_mask_table(8) + misc flags(1) + still_mode(1) + still_time(2), then
// STN_table starting at STN_OFFSET.
const STN_OFFSET: usize = 32;
if item_idx == 0 && item.len() > STN_OFFSET + 16 {
// STN header: length(2) + reserved(2) + counts(8) + reserved(4) = 16 bytes
@@ -235,11 +234,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
entry.stream_type = 6;
entry.secondary = true;
streams.push(entry);
// Skip extra ref bytes (audio refs + PG refs).
// Use `next < item.len()` to match the sibling secondary
// blocks; the inner `after_arefs < item.len()` re-guards
// the second read, so the stricter `+2` only mis-aligned
// spos when the aref count sits in the last 1-2 bytes.
// Skip extra ref bytes (audio refs + PG refs). `next < item.len()`
// matches the sibling secondary blocks; the inner
// `after_arefs < item.len()` re-guards the second read near the end.
if next < item.len() {
let n_arefs = item[next] as usize;
let after_arefs = next + 2 + n_arefs + (n_arefs % 2);
@@ -367,13 +364,9 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
return None;
}
// PID location depends on the stream-entry type (BD spec stream_entry()):
// type 1 (stream in the PlayItem's Clip): PID at +2
// type 2 (stream in a SubPath SubClip): +subpath_id(1)+subclip_id(1) → PID at +4
// type 3 / 4 (SubPath clip; type 4 = Dolby Vision +subpath_id(1) → PID at +3
// enhancement layer, e.g. PID 0x1015):
// Previously only type 1 was handled, so the DV EL (type 4) and any
// sub-path stream fell through to PID 0 and were dropped by the mux.
// PID location depends on stream-entry type (BD spec stream_entry()): type 1
// (PlayItem's Clip) → +2; type 2 (SubPath SubClip) → +4; type 3/4 (SubPath
// clip / DV enhancement layer) → +3. Previously only type 1 was handled.
let pid_off = match item[pos + 1] {
STREAM_ENTRY_PLAYITEM_CLIP => 2,
STREAM_ENTRY_SUBPATH_SUBCLIP => 4,
@@ -410,11 +403,9 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
let mut color_space_val = 0u8;
let mut language = String::new();
// `stream_type` here is the STN category passed by the caller, which is
// only ever a primary category (VIDEO/AUDIO/PG_SUBTITLE/IG). Secondary
// audio/video and the DV enhancement layer are parsed through their
// matching primary category (identical attribute layout) and re-tagged by
// the caller after this returns, so there are no secondary arms here.
// `stream_type` is the STN category from the caller — always a primary category
// (VIDEO/AUDIO/PG_SUBTITLE/IG). Secondary audio/video and the DV enhancement layer
// share the primary's attribute layout and are re-tagged by the caller afterward.
match stream_type {
STREAM_CATEGORY_VIDEO => {
// Video: coding_type(1) + format_rate(1) + [hdr_info(1) if HEVC]
@@ -533,10 +524,12 @@ mod tests {
item.extend_from_slice(*clip_id);
// [5..9] codec_id ("M2TS")
item.extend_from_slice(b"M2TS");
// [9] connection_condition in low nibble
// [9] fully reserved
item.push(0);
// [10] is_multi_angle (bit 4) + connection_condition (low nibble)
item.push(*conn & 0x0F);
// [10..12] reserved
item.extend_from_slice(&[0u8; 2]);
// [11] stc_id / reserved
item.push(0);
// [12..16] in_time
item.extend_from_slice(&in_time.to_be_bytes());
// [16..20] out_time
@@ -964,12 +957,9 @@ mod tests {
#[test]
fn mark_type_read_from_correct_offset() {
// Regression for the mark_type off-by-one: each PlayListMark entry is
// reserved(1) + mark_type(1) + .... The parser must read byte[1], not
// byte[0]. build_mpls_with_marks writes reserved=0 at byte[0] and the
// mark_type at byte[1], so a parser that read byte[0] would see 0 for
// every mark. Use distinct non-zero, non-1 types to make the offset
// error unmistakable.
// Regression for the mark_type off-by-one: PlayListMark is reserved(1)+mark_type(1)+...,
// so the parser must read byte[1], not byte[0]. build_mpls_with_marks sets reserved=0
// and mark_type=byte[1]; distinct non-zero/non-1 types make an offset error unmistakable.
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
let marks = vec![
TestMark {
@@ -1022,10 +1012,7 @@ mod tests {
assert_eq!(playlist.marks.len(), 0);
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests below. Grounded in the BD-ROM MPLS spec
// (https://github.com/lw/BluRay/wiki/MPLS) byte layout.
// ─────────────────────────────────────────────────────────────────────
// ─── Added hardening tests below, grounded in the BD-ROM MPLS spec byte layout ───
/// Header guard: parse() requires `playlist_start + 10 <= data.len()`
/// before reading the PlayList header (num_play_items at pl[6..8]).
@@ -1039,22 +1026,24 @@ mod tests {
assert!(parse(&data).is_err());
}
/// Spec: connection_condition is the LOW nibble of PlayItem byte[9]
/// (high nibble is reserved/flags). A byte 0xF5 must yield 5, not 0xF5.
/// connection_condition is the LOW nibble of PlayItem byte[10] (the high
/// nibble carries reserved bits and the is_multi_angle flag at bit 4).
/// byte[9] is entirely reserved and must NOT be read. A byte[10] of 0xF5
/// must yield 5, and a non-zero byte[9] must not leak into the value.
#[test]
fn connection_condition_is_low_nibble_only() {
// Build a custom item where byte[9] = 0xF5 (high nibble set).
// build_mpls masks with &0x0F when writing, so write raw to verify
// the PARSER masks. We patch the item byte directly after building.
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
let mut data = build_mpls(
&[(b"00001", 0, 0, 9000000)],
(1, 0, 0, 0, 0, 0, 0, 0),
&[video],
);
// Locate PlayItem byte[9]: header(40) + pl_header(10) + item_len(2) + 9.
let conn_idx = 40 + 10 + 2 + 9;
data[conn_idx] = 0xF5;
// PlayItem base: header(40) + pl_header(10) + item_len(2).
let item_base = 40 + 10 + 2;
// byte[9] fully reserved — set it to prove it does not leak in.
data[item_base + 9] = 0xFF;
// byte[10] high nibble set — parser must mask to the low nibble.
data[item_base + 10] = 0xF5;
let pl = parse(&data).expect("should parse");
assert_eq!(pl.play_items[0].connection_condition, 0x05);
}
@@ -1203,11 +1192,9 @@ mod tests {
/// PG, and the PG must keep its correct PID (proving IG advanced spos).
#[test]
fn ig_consumed_but_not_retained_and_dv_after_aligned() {
// STN parse order is video, audio, PG, IG, sec_audio, sec_video,
// pip_pg, DV. The IG entry must be consumed (advancing spos) but
// never retained. To PROVE IG advanced the cursor, place a Dolby
// Vision EL after the IG: if IG didn't advance spos, the DV parse
// would land on the IG bytes and read the wrong PID.
// STN parse order is video, audio, PG, IG, sec_audio, sec_video, pip_pg, DV.
// The IG entry must be consumed (advancing spos) but never retained; placing
// a DV EL right after IG proves this — a wrong spos would misread the DV PID.
let video = build_stream_entry_video(0x1011, 0x24, 8, 1, Some(0x12));
let ig = build_stream_entry_pg(0x1400, 0x91, b"eng"); // IG entry bytes
let dv = build_stream_entry_video(0x1015, 0x24, 8, 1, Some(0x12)); // DV EL
@@ -1399,8 +1386,9 @@ mod tests {
let mut item = Vec::new();
item.extend_from_slice(b"00009");
item.extend_from_slice(b"M2TS");
item.push(0x01); // connection_condition
item.extend_from_slice(&[0u8; 2]);
item.push(0); // [9] reserved
item.push(0x01); // [10] connection_condition (low nibble)
item.push(0); // [11] stc_id / reserved
item.extend_from_slice(&90000u32.to_be_bytes()); // in_time
item.extend_from_slice(&180000u32.to_be_bytes()); // out_time
item.resize(32, 0); // pad through STN_OFFSET; item.len()==32 so no STN
@@ -1456,13 +1444,15 @@ mod tests {
}
/// The 20 bytes a PlayItem needs for clip_id(5) + codec_id(4) +
/// connection_condition(1) + reserved(2) + IN_time(4) + OUT_time(4).
/// reserved(1) + connection_condition byte(1) + stc_id/reserved(1) +
/// IN_time(4) + OUT_time(4).
fn play_item_20(clip: &[u8; 5], cc: u8, in_t: u32, out_t: u32) -> Vec<u8> {
let mut it = Vec::new();
it.extend_from_slice(clip);
it.extend_from_slice(b"M2TS");
it.push(cc);
it.extend_from_slice(&[0u8; 2]);
it.push(0); // [9] reserved
it.push(cc); // [10] connection_condition (low nibble)
it.push(0); // [11] stc_id / reserved
it.extend_from_slice(&in_t.to_be_bytes());
it.extend_from_slice(&out_t.to_be_bytes());
assert_eq!(it.len(), 20);
+26 -51
View File
@@ -245,11 +245,8 @@ impl AuAssembler {
dts,
source,
});
// Backstop: the `buf`-size cap prunes marks only when bytes accumulate.
// A run of zero-length (or start-code-free) timed fragments grows no
// bytes, so bound the deque directly — drop the oldest (stalest) mark,
// which belongs to an already-emitted or lost AU. A real AU spans far
// fewer fragments than this cap.
// Backstop: zero-length/start-code-free fragments grow no bytes, so the
// `buf`-size cap never prunes marks. Bound the deque directly instead.
if self.marks.len() > MAX_MARKS {
self.marks.pop_front();
}
@@ -318,12 +315,9 @@ impl AuAssembler {
}
let end_abs = self.base + end as u64;
// The AU's own timing/source: take the FIRST Some of each field
// across every mark in this AU's range [base, end_abs), independently
// — one PES fragment may carry the source while a later fragment of
// the same AU carries the PTS (and vice versa), so reading only the
// front mark would drop the other field. This restores the semantics
// of the pre-consolidation separate pts/source mark deques.
// Take the first Some of each field independently across marks in
// [base, end_abs): one fragment may carry source while another carries
// PTS, so reading only the front mark would drop a field.
let (mut pts, mut dts, mut source) = (None, None, None);
while self.marks.front().is_some_and(|m| m.off < end_abs) {
let m = self.marks.pop_front().unwrap();
@@ -399,10 +393,8 @@ impl AuAssembler {
}
let data = self.buf[..end].to_vec();
self.buf.drain(..end);
// Shrink toward what this AU actually needed (the tail plus room for
// another AU of about this size). Only the short tail is copied, and it
// brings `cap` back under the `2*end` threshold so the next AU of this
// size hands over instead of copying.
// Shrink toward this AU's actual need so `cap` drops under the `2*end`
// threshold and the next similarly-sized AU hands over instead of copying.
self.buf.shrink_to(end.max(tail_len));
return data;
}
@@ -447,10 +439,9 @@ impl AuAssembler {
fn au_boundary_resumable(&mut self) -> Option<usize> {
match self.mode {
Mode::StartCode(marker) => {
// Stateless: the AU ends at the next delimiter after the opener at
// buf[0]. Resume from the furthest searched offset (never before 4,
// to skip the opening delimiter). find_start_code needs 4 bytes, so
// back up 3 to catch a code straddling the previous buffer end.
// Resume from the furthest searched offset (never before 4, to skip the
// opening delimiter at buf[0]); find_start_code needs 4 bytes, so back
// up 3 to catch a code straddling the previous buffer end.
let from = self.scan_pos.max(4);
match find_start_code(&self.buf, from, marker) {
Some(e) => Some(e),
@@ -683,10 +674,8 @@ mod tests {
#[test]
fn au_merges_pts_and_source_from_different_fragments() {
// One fragment of an AU may carry the source stamp while a later fragment
// of the SAME AU carries the PTS (each PES gets a source; only the anchor
// gets a PTS). The AU must keep BOTH — reading only the front mark would
// drop whichever field the first fragment lacked.
// One fragment may carry the source stamp while a later fragment of the same
// AU carries the PTS; the AU must keep both, not just the front mark's field.
let src = crate::pes::SourcePos::at_byte(4242);
let mut a = AuAssembler::for_codec(Codec::H264);
let full = au(0xAB, 80);
@@ -745,10 +734,8 @@ mod tests {
#[test]
fn vc1_i_frame_keeps_its_preceding_seq_and_entry_headers() {
// An I-frame AU is [seq 0x0F][entry 0x0E][frame 0x0D][slices]; a following
// P-frame is just [frame 0x0D][slices]. A plain 0x0D split would strand the
// seq/entry headers on the P-frame's AU — the decode bug. The VC-1 mode must
// group them with the I-frame that follows them.
// A plain 0x0D split would strand the seq/entry headers on the following
// P-frame's AU (a decode bug); VC-1 mode must group them with the I-frame.
let mut a = AuAssembler::for_codec(Codec::Vc1);
let mut iframe = bdu(VC1_SEQ, 0xAA, 8);
iframe.extend(bdu(VC1_ENTRY, 0xBB, 6));
@@ -875,11 +862,8 @@ mod tests {
#[test]
fn resumable_boundary_matches_from_scratch_across_all_fragmentations() {
// The incremental scan_pos cursor must produce byte-identical AUs to a
// whole-buffer rescan, at EVERY fragment granularity (this is what makes
// the O(n) resume equivalent to the old O(n^2) from-scratch scan). Build a
// multi-AU stream per codec, reassemble it fed 1 byte at a time up to
// whole, and require one canonical result.
// The incremental scan_pos cursor must match a whole-buffer rescan at every
// fragment granularity, from 1 byte at a time up to whole (O(n) vs O(n^2)).
let h264 = {
let mut s = au(0x11, 40); // AU1 (AUD + payload)
s.extend(au(0x22, 70)); // AU2
@@ -971,11 +955,9 @@ mod tests {
"an ordinary AU at the head of a clean run is continuous"
);
// Now start-code-free junk past the cap. The FIRST over-cap run still
// has the next AU's delimiter at buf[0], so it is force-flushed as an
// (over-long) access unit — nothing is discarded and nothing is lost.
// Only once the buffer holds no opener at all does the backstop throw
// bytes away, which is the case this test is about.
// The first over-cap run still has a delimiter at buf[0], so it force-flushes
// as an over-long AU with nothing lost. Only once no opener remains does the
// backstop discard bytes, which is what this test covers.
let junk = vec![0xAB; MAX_AU_BUFFER + 4096];
a.push(&junk, Some(2000), None, None, false);
a.push(&junk, Some(2100), None, None, false);
@@ -1118,10 +1100,8 @@ mod tests {
before,
"the emitted AU must own the buffer's allocation (no whole-frame copy)"
);
// The replacement buffer keeps room for another AU of about this size, so
// the next AU does not re-grow — but it is NOT pinned to the OLD capacity,
// which would make `buf` a permanent high-water mark and send every later
// smaller AU down the copy path (see
// Must not pin to the old capacity, or `buf` becomes a permanent high-water
// mark sending every later smaller AU down the copy path (see
// `handover_survives_a_large_au_instead_of_copying_every_later_one`).
assert!(
a.buf.capacity() >= au1.len(),
@@ -1149,10 +1129,9 @@ mod tests {
#[test]
fn handover_survives_a_large_au_instead_of_copying_every_later_one() {
let mut a = AuAssembler::for_codec(Codec::H264);
// Production shape: BD-TS aligns one access unit per PES, so each `push`
// carries about one AU and the buffer holds ~one AU at a time. One large AU
// (the IDR) followed by a run of much smaller ones (P/B frames). Each AU is
// pushed with the NEXT AU's opener so the previous one closes.
// BD-TS aligns one AU per PES, so each `push` carries about one AU. One large
// AU (IDR) is followed by smaller ones (P/B); each push includes the next
// AU's opener so the previous one closes.
const SMALL: usize = 64 * 1024;
let mut pending = au(0x11, 2 * 1024 * 1024);
for i in 0..20u8 {
@@ -1172,12 +1151,8 @@ mod tests {
}
// ── AU-opener detection: the per-mode start-code rule ─────────────────
//
// `au_opener_from` is the SECOND implementation of a rule each codec parser
// also encodes (h264 `NAL_AUD`, hevc `NAL_AUD`, vc1 `SC_*`, mpeg2
// `PICTURE_CODE`/`SEQ_HEADER_CODE`/`GOP_CODE`). Two independent copies of one
// rule drift; these cases pin this copy to the normative byte values and to
// the codes that are explicitly NOT openers, so a drift shows up here.
// `au_opener_from` duplicates a rule each codec parser also encodes; these
// cases pin it to the normative byte values so a drift between copies shows up.
/// The opener offset must be the position of the real start code, never a
/// fixed 0. A constant `Some(0)` makes every pre-sync run of junk bytes look
+97 -188
View File
@@ -33,12 +33,12 @@ use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// Sample rates indexed by fscod (0=48kHz, 1=44.1kHz, 2=32kHz). fscod=3 is
/// reserved in AC-3; in E-AC-3 it signals "fscod2" (reduced rates: 24/22.05/16
/// kHz, selected by byte-4 bits [5:4]). `frame_sample_rate` decodes fscod2 in
/// kHz, selected by byte-4 bits `[5:4]`). `frame_sample_rate` decodes fscod2 in
/// the E-AC-3 case; this table's index-3 entry (48 kHz) is only the fallback
/// when the header is too short to read fscod2.
const SAMPLE_RATES: [u32; 4] = [48_000, 44_100, 32_000, 48_000];
/// E-AC-3 reduced sample rates indexed by fscod2 (byte-4 bits [5:4]), used when
/// E-AC-3 reduced sample rates indexed by fscod2 (byte-4 bits `[5:4]`), used when
/// fscod==3. Index 3 is reserved; we fall back to 48 kHz for it.
const EAC3_REDUCED_RATES: [u32; 4] = [24_000, 22_050, 16_000, 48_000];
@@ -247,18 +247,16 @@ impl Ac3Parser {
{
self.frames_scanned += 1;
}
// Decodability gate: a syncframe with an out-of-range bsid (> 16) or
// a failed native CRC (payload corruption) poisons the access unit it
// belongs to — a dependent substream is useless without its parent and
// vice versa, so the whole access unit is dropped as one silence gap.
// Decodability gate: an out-of-range bsid (> 16) or a failed native CRC
// poisons the whole access unit — a dependent substream is useless
// without its parent and vice versa — so it's dropped as one silence gap.
let reason = ac3_drop_reason(&self.tally, frame, bsid);
if substream_role(remaining, bsid) == SubstreamRole::Extends {
match pending.as_mut() {
// A dependent substream — or an additional independent
// substream (`substreamid` 1..7) — extends the frame set it
// directly follows. Requiring byte contiguity keeps skipped
// junk out of the emitted access unit.
// A dependent (or additional independent, substreamid 1..7)
// substream extending the frame set it directly follows; byte
// contiguity is required to keep skipped junk out of the AU.
Some(au) if au.end == start => {
au.end = start + frame_size;
if au.drop_reason.is_none() {
@@ -266,18 +264,9 @@ impl Ac3Parser {
}
self.saw_extension = true;
}
// A substream with no open access unit — a mid-frame-set
// resync, or a stream whose first syncframe is not
// `substreamid` 0 (Annex E orders a frame set independent
// substream 0 first, so joining mid-set is exactly the case
// here). It belongs to a frame set whose mandatory
// `substreamid`-0 substream was never seen, so it is neither
// decodable on its own (a dependent substream) nor timeable
// (an additional independent substream carries the frame set's
// time period, not its own PTS). Skip it rather than ship a
// fragment a decoder cannot use and re-time the whole
// timeline around; resync at the next `substreamid`-0
// substream. No PTS advance: it carries no duration.
// No open access unit: mid-frame-set resync, or a stream whose
// mandatory substreamid-0 substream was never seen. Neither
// decodable alone nor timeable, so skip it with no PTS advance.
_ => {
tracing::debug!(
target: "mux",
@@ -289,9 +278,8 @@ impl Ac3Parser {
if let Some(au) = pending.take() {
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
// First access unit that starts in this PES's own bytes: adopt
// this PES's timestamp so a genuine PTS jump is followed instead
// of the running cadence drifting past it.
// First AU starting in this PES's own bytes: adopt its timestamp so
// a genuine PTS jump is followed, not drifted past.
if let Some(a) = &anchor
&& start >= a.at
{
@@ -317,26 +305,18 @@ impl Ac3Parser {
scanned_to = pos;
}
// Close or HOLD the trailing access unit. The rest of its frame set — its
// dependent substreams and any additional independent substreams — may
// still be in the next PES, so an access unit that can still grow is
// held: the carry-over rewinds to its first byte and the whole access
// unit is re-scanned (and only then counted/emitted) next call. An
// E-AC-3 `substreamid`-0 substream can always gain more of its frame set;
// a legacy AC-3 syncframe only in the AC-3-core + E-AC-3-dependent
// arrangement, so it is held only once this track has actually shown a
// substream that extends an access unit — a plain AC-3 track keeps
// emitting every frame in-call.
// Close or HOLD the trailing access unit: its frame set may still continue
// in the next PES, so a growable AU is held and re-scanned next call, but
// only once this track has shown a substream extending an AU (E-AC-3).
let mut hold_from = None;
let mut held_out = None;
if let Some(au) = pending {
if !at_eos && (au.bsid >= 11 || self.saw_extension) {
frame_pts_ns = au.pts_ns;
hold_from = Some(au.start);
// Everything below `scanned_to` has been searched already, and
// the access unit's own frames have been sized and CRC-gated;
// record both, rebased onto the carry-over (which starts at
// `au.start`), so the next call resumes instead of redoing it.
// Everything below `scanned_to` is already searched and CRC-gated;
// record it rebased onto the carry-over (starting at `au.start`)
// so the next call resumes instead of redoing the work.
held_out = Some(HeldAu {
end: au.end - au.start,
scanned_to: scanned_to.max(au.end) - au.start,
@@ -349,23 +329,15 @@ impl Ac3Parser {
}
}
// Keep unconsumed data for the next call. `pos` is the start of the
// last unprocessed search region. On the `start + frame_size > len`
// break it sits exactly at the straddling frame's syncword; on the
// `remaining.len() < 6` break it is the value from the top of that
// iteration, with the syncword possibly sitting after some pre-sync
// junk — so the re-scan below (from `pos`, NOT a recomputed sync) is
// required to locate the carry-over syncword. Carry from `pos`, NOT
// from the next syncword: discarding bytes between `pos` and the next
// sync would drop the partial frame we are deliberately keeping across
// the boundary. A held access unit wins: it starts before `pos`.
// Keep unconsumed data for the next call: re-scan from `pos`, not a
// recomputed sync, to avoid dropping the partial frame kept across the
// PES boundary. Held AU wins (its start is before `pos`).
let keep_from = match hold_from {
Some(h) => h,
None if pos < data.len() => {
// A syncword at/after `pos` marks the carry-over start (anything
// before it is junk with no sync). With no full sync, retain the
// whole tail — including a lone trailing 0x0B that may be the first
// half of a syncword split across the PES boundary.
// A syncword at/after `pos` marks the carry-over start. With no full
// sync, retain the whole tail, including a lone trailing 0x0B that
// may be the first half of a syncword split across the boundary.
match find_ac3_sync(&data[pos..]) {
Some(o) => pos + o,
None if data.last() == Some(&0x0B) => data.len() - 1,
@@ -568,17 +540,9 @@ fn ac3_drop_reason(
impl CodecParser for Ac3Parser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
// B1: a concealed/lost gap means the bytes held in `buf` are a TRUNCATED
// frame. Appending the post-gap bytes would splice them into one corrupt
// frame (wrong frame_size, bad CRC → "exponent out of range" / garbage).
// Drop the partial and resync on the next syncword — a clean single-frame
// gap instead of a frankenstein frame. (The video parsers carry this via
// the ResyncGate; audio has no inter-frame refs, so dropping the spliced
// partial is the whole fix.)
//
// Handle the discontinuity BEFORE the empty-data guard so the signal can
// never be stranded by an empty post-gap PES (the demuxer only emits
// non-empty PES today; this is defensive for any future caller).
// B1: a concealed/lost gap means `buf` holds a TRUNCATED frame; appending
// post-gap bytes would splice a corrupt frame, so drop the partial and
// resync on the next syncword. Handled before the empty-data guard.
if pes.discontinuity {
self.acc.clear();
// The held access unit's bytes went with it.
@@ -588,35 +552,19 @@ impl CodecParser for Ac3Parser {
return Vec::new();
}
// This PES's timestamp applies to the first access unit that STARTS in
// its own bytes; anything already buffered began in an earlier PES and
// keeps the running cadence (`flush_pts_ns`). Each subsequent access unit
// in the same call advances by the previous one's duration, so a PES that
// carries several access units stamps a monotonically increasing PTS
// instead of the same PES timestamp on all of them (which collapses their
// timecodes and drifts A/V).
//
// A PES with no PTS (rare for audio, but legal — and the case demuxers
// guard at a post-gap continuation) must NOT reset the timeline to 0: with
// no anchor the running cadence simply continues. The discontinuity-
// carrying PES is a PUSI with a PTS in practice, so this is
// defense-in-depth.
// This PES's timestamp applies to the first AU that STARTS in its own bytes;
// later AUs advance by the previous one's duration (avoiding A/V drift). A
// PES with no PTS must NOT reset the timeline: `flush_pts_ns` just continues.
let carry_len = self.acc.len();
let anchor = pes.pts.map(|p| PtsAnchor {
at: carry_len,
pts_ns: pts_to_ns(p),
});
// Prepend leftover from previous PES, then take the whole buffer into a
// local so the scanner can call `self.tally` (the bytes are no longer
// borrowed from `self`). The unconsumed tail is written back at the end.
// Prepend leftover from the previous PES, then take the scratch buffer out
// of `self` so the scanner can borrow `self.tally`; it's put back at the
// end, keeping its capacity. There is no early return after this point.
self.acc.push(pes);
// Copy the working bytes out so the scanner can borrow `self.tally`;
// the buffer keeps its marks, so the unconsumed tail stays attributed
// to the packet that carried it.
// Take the scratch OUT of `self` so the scanner can borrow `self.tally`
// while reading it; it is put back at the end of the call, keeping its
// capacity for the next PES. There is no early return after this point.
let mut buf = std::mem::take(&mut self.scratch);
buf.clear();
buf.extend_from_slice(self.acc.as_slice());
@@ -643,20 +591,16 @@ impl CodecParser for Ac3Parser {
);
self.acc.clear();
self.held = None;
// Advance the cadence, as both sibling branches below do, so the
// three paths out of this block cannot disagree. Defensive: no
// input reaching this parser was found that both parses frames and
// leaves a residue this large, so the stale-cadence bug this
// prevents is not currently reachable and has no regression test.
// Advance the cadence, as the other two paths out of this block do,
// so all three cannot disagree. Defensive: no input parsing frames
// and leaving a residue this large is currently reachable.
self.flush_pts_ns = frame_pts_ns;
} else {
self.acc.drain(keep_from);
self.held = still_held;
// The carried bytes, when later completed and emitted (next call
// or by flush() at EOS), are timed at the PTS the scanner reached
// here: the PTS of the next access unit in presentation order, or
// — for a HELD access unit — that access unit's own PTS, so the
// hold never shifts it.
// Carried bytes, when later completed and emitted, are timed at the
// PTS the scanner reached here — the next AU's PTS, or for a HELD
// AU, its own PTS, so the hold never shifts it.
self.flush_pts_ns = frame_pts_ns;
}
} else {
@@ -675,11 +619,9 @@ impl CodecParser for Ac3Parser {
}
fn flush(&mut self) -> Vec<Frame> {
// Drain the carry-over buffer at EOS: a complete access unit (including
// one held back waiting for a possible dependent substream) may sit there
// with no following PES to close it, and without this drain the last
// ~32 ms of audio is lost. `at_eos` closes the trailing access unit
// instead of holding it; a partial/garbage tail yields nothing.
// Drain the carry-over at EOS: an AU (possibly held for a dependent
// substream) may sit there with no following PES to close it, else the
// last ~32ms of audio is lost. `at_eos` closes it instead of holding.
let buf = self.acc.as_slice().to_vec();
let marks = self.acc.marks_snapshot();
self.acc.clear();
@@ -720,7 +662,7 @@ fn eac3_samples_per_frame(data: &[u8]) -> u32 {
/// Sample rate (Hz) of an AC-3/E-AC-3 frame from its fscod field (byte 4 bits
/// 7-6). For E-AC-3 (`bsid >= 11`) an fscod of 3 selects a reduced rate via
/// fscod2 (byte 4 bits [5:4]); decoding it keeps the frame duration correct
/// fscod2 (byte 4 bits `[5:4]`); decoding it keeps the frame duration correct
/// instead of mistiming reduced-rate frames at 48 kHz (A/V drift).
fn frame_sample_rate(data: &[u8], bsid: u8) -> u32 {
if data.len() < 5 {
@@ -867,7 +809,7 @@ pub(crate) fn ac3_frame_size(data: &[u8]) -> usize {
}
}
/// AC-3 frame size table: [frmsizecod] -> [48kHz words, 44.1kHz words, 32kHz words]
/// AC-3 frame size table: `[frmsizecod]` -> `[48kHz words, 44.1kHz words, 32kHz words]`
const AC3_FRAME_SIZES: [[usize; 3]; 38] = [
[64, 69, 96],
[64, 70, 96],
@@ -968,11 +910,9 @@ mod tests {
"the buffer must be handed back to the parser, not dropped"
);
// The discriminator: a LARGE packet, then a small one. A reused buffer
// keeps the large capacity; a fresh `Vec` sized to each packet drops
// back to the small one. Feeding equal-sized packets cannot tell those
// apart — the first version of this test did exactly that and a
// to_vec()-per-call implementation passed it.
// Discriminator: LARGE packet then small. A reused buffer keeps the large
// capacity; a fresh `Vec` drops to the small size. Equal-sized packets
// can't tell those apart — an earlier to_vec()-per-call bug passed that test.
let big = [frame.clone(), frame.clone(), frame.clone(), frame.clone()].concat();
parser.parse(&PesPacket {
source: None,
@@ -1070,11 +1010,9 @@ mod tests {
#[test]
fn discontinuity_drops_truncated_partial() {
// B1: a partial AC-3 frame is buffered, then a concealed gap arrives
// (PES marked discontinuity) carrying a fresh complete frame. The
// truncated partial must be DROPPED, not spliced — otherwise the parser
// emits one corrupt frame built from [stale partial | head of fresh] and
// strands the tail (decoders report "incomplete frame" / wrong sync).
// B1: a partial frame is buffered, then a discontinuity-marked PES brings a
// fresh complete frame. The partial must be DROPPED, not spliced, or the
// parser emits one corrupt frame from [stale partial | head of fresh].
let mut parser = Ac3Parser::new();
let frame_data = make_ac3_frame(0, 2); // 160 bytes, starts with 0x0B77
@@ -1218,11 +1156,9 @@ mod tests {
#[test]
fn buffer_stays_bounded_across_many_garbage_pes() {
// The carry-over buffer must never grow without bound. Feed
// many large PES packets that contain no usable frame and assert the
// retained buffer stays tiny — carry-from-`pos` drops all pre-sync junk,
// and a never-completing frame is bounded by the 8192-byte frame cap and
// the MAX_AC3_BUF resync guard.
// The carry-over buffer must never grow without bound: carry-from-`pos`
// drops pre-sync junk, and a never-completing frame is bounded by the
// 8192-byte frame cap and the MAX_AC3_BUF resync guard.
let mut parser = Ac3Parser::new();
for i in 0..256 {
// Vary the trailing byte so we also exercise the lone-0x0B retain.
@@ -1393,10 +1329,9 @@ mod tests {
#[test]
fn eac3_fscod2_reduced_rate_duration() {
// E-AC-3 with fscod==3 (reduced rate) and fscod2==0 → 24 kHz, not 48.
// bsid>=11 selects the E-AC-3 path. When fscod==3 the block count is
// fixed at 6 → 1536 samples. Byte 4 layout: fscod(2)|fscod2(2)|...
// fscod=3 (0b11), fscod2=0 (0b00) → byte4 = 0b1100_0000 = 0xC0.
// E-AC-3 with fscod==3 (reduced rate) and fscod2==0 → 24 kHz, not 48; block
// count is then fixed at 6 → 1536 samples. Byte 4: fscod(2)|fscod2(2)|...,
// so fscod=3 (0b11), fscod2=0 (0b00) → byte4 = 0b1100_0000 = 0xC0.
let data = [0x0B, 0x77, 0x00, 0x00, 0xC0, 16 << 3];
let bsid = get_bsid(&data);
assert!(bsid >= 11, "test frame is E-AC-3");
@@ -1637,11 +1572,9 @@ mod tests {
#[test]
fn eac3_frame_at_min_frame_bytes_passes_sizing_then_crc_gate() {
// The smallest frame the SIZING layer accepts is MIN_FRAME_BYTES = 6
// (frmsiz=2). A synthetic all-zero 6-byte frame passes sizing (so it
// reaches the decodability gate — proven by it being COUNTED as a drop,
// not silently size-skipped) but fails the CRC gate and is dropped; the
// following real AC-3 frame (valid CRC) is emitted.
// Smallest frame SIZING accepts is MIN_FRAME_BYTES = 6 (frmsiz=2). An
// all-zero 6-byte frame passes sizing (proven COUNTED as a drop, not
// size-skipped) but fails CRC; the following real AC-3 frame is emitted.
let mut parser = Ac3Parser::new();
// 0x0B 0x77 | byte2=0 byte3=2 (frmsiz=2 → 6 bytes) | byte4=0 | byte5 bsid
let mut data = vec![0x0B, 0x77, 0x00, 0x02, 0x00, 16 << 3];
@@ -1659,10 +1592,9 @@ mod tests {
#[test]
fn eac3_max_frmsiz_frame_within_window_accepted() {
// E-AC-3 frmsiz is an 11-bit field (3 bits of byte2 + 8 bits of byte3),
// so its maximum value is 0x7FF = 2047 → (2048)*2 = 4096 bytes, which is
// inside the MIN_FRAME_BYTES..=8192 accept window and, with a valid CRC,
// must be emitted.
// E-AC-3 frmsiz is an 11-bit field, so its max value 0x7FF = 2047 →
// (2048)*2 = 4096 bytes, inside the MIN_FRAME_BYTES..=8192 accept window
// and, with a valid CRC, must be emitted.
let mut parser = Ac3Parser::new();
let mut frame = vec![0u8; 4096];
frame[0] = 0x0B;
@@ -1671,9 +1603,8 @@ mod tests {
frame[3] = 0xFF; // frmsiz low → 0x7FF = 2047 → 4096 bytes
frame[5] = 16 << 3; // bsid 16 (E-AC-3)
finalize_ac3_crc(&mut frame); // pass the decodability gate
// The trailing E-AC-3 access unit is HELD at the end of the call (a
// dependent substream may follow in the next PES), so it is closed by
// flush() at EOS rather than in-call. Content and size are unchanged.
// Trailing E-AC-3 AU is HELD at end of call (a dependent substream may
// follow), so it's closed by flush() at EOS, not in-call.
let mut f = parser.parse(&make_eac3_pes(frame));
f.extend(parser.flush());
assert_eq!(f.len(), 1, "4096-byte E-AC-3 frame within window accepted");
@@ -1720,10 +1651,9 @@ mod tests {
#[test]
fn flush_rejects_frame_extending_past_buffer() {
// A buffered sync whose decoded frame size exceeds the buffered bytes
// must be dropped by flush (never emit fewer bytes than the size field
// declares). Build a real AC-3 header (160-byte frame) but only buffer
// 100 bytes.
// A buffered sync whose decoded frame size exceeds the buffered bytes must
// be dropped by flush (never emit fewer bytes than declared). Build a real
// 160-byte AC-3 frame but only buffer 100 bytes.
let mut parser = Ac3Parser::new();
let frame = make_ac3_frame(0, 2); // sizes to 160
parser.acc.seed(&frame[..100]);
@@ -1784,11 +1714,9 @@ mod tests {
#[test]
fn acmod_channels_stereo_2_0_no_lfe() {
// acmod=2 (2/0 L,R), no LFE → 2 channels. Verifies the channel count is
// read from the AC-3 bitstream's acmod, independent of any IFO claim.
// (A disc whose IFO lists 5.1 but where the wrong physical substream is
// selected is a separate stream-SELECTION bug, not this label path —
// tracked for rc.5.2.)
// acmod=2 (2/0 L,R), no LFE → 2 channels; verifies the channel count comes
// from the AC-3 bitstream's acmod, independent of any IFO claim (a disc
// IFO/substream mismatch is a separate SELECTION bug, tracked for rc.5.2).
assert_eq!(acmod_channels(&make_bsi(2, false)), Some(2));
}
@@ -1896,9 +1824,8 @@ mod tests {
#[test]
fn crc_drop_preserves_pts_sync_no_shift() {
// THE INVARIANT: dropping a corrupt frame must not shift the audio after
// it. good / corrupt / good in one PES — the corrupt frame is dropped but
// the trailing clean frame keeps the EXACT PTS it would have had with no
// drop (base + 2 frame durations): a silence gap, not a shift.
// it. good/corrupt/good in one PES — the trailing clean frame keeps the
// EXACT PTS it would have had with no drop: a silence gap, not a shift.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2); // f0
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2)); // dropped
@@ -1968,13 +1895,9 @@ mod tests {
#[test]
fn eac3_independent_plus_dependent_is_one_access_unit() {
// THE FIX: per ETSI TS 102 366 Annex E an access unit is the independent
// substream (strmtyp 0) plus every dependent substream (strmtyp 1) that
// follows it — the 7.1 Dolby Digital Plus arrangement. Both syncframes
// must emerge as ONE frame carrying the INDEPENDENT substream's PTS and
// exactly ONE frame duration (previously each syncframe was emitted as
// its own access unit and each advanced the clock, doubling the timeline
// and handing decoders a parentless dependent substream).
// THE FIX: per ETSI TS 102 366 Annex E an AU is the independent substream
// plus every dependent substream that follows it — both must emerge as ONE
// frame with the independent PTS and ONE duration (not double it).
let mut parser = Ac3Parser::new();
let indep = make_eac3_frame(0, 0, 160);
let dep = make_eac3_frame(1, 0, 96);
@@ -2014,10 +1937,9 @@ mod tests {
#[test]
fn eac3_grouped_timeline_is_not_doubled() {
// Two complete access units (independent + dependent each) in one PES,
// followed by a third independent substream that closes the second.
// The emitted PTS cadence must be one frame duration per access unit —
// the 2x-runtime / A-V-drift symptom of ungrouped substreams.
// Two complete access units (independent + dependent each) in one PES, plus
// a third independent substream closing the second. PTS cadence must be
// one frame duration per AU — else the 2x-runtime / A-V-drift symptom.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..2 {
@@ -2038,10 +1960,9 @@ mod tests {
#[test]
fn plain_ac3_frames_are_not_grouped_or_delayed() {
// NO REGRESSION for legacy AC-3 (bsid < 11): it has no substream
// structure (byte 2 is crc1, not strmtyp), so every syncframe is a
// complete access unit, emitted in the SAME call — never merged with its
// neighbour and never held back for a dependent that cannot exist.
// NO REGRESSION for legacy AC-3 (bsid < 11): it has no substream structure
// (byte 2 is crc1, not strmtyp), so every syncframe is a complete AU,
// emitted in the SAME call — never merged, never held for a dependent.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..3 {
@@ -2060,10 +1981,9 @@ mod tests {
#[test]
fn eac3_access_unit_split_across_pes_is_grouped() {
// The access unit boundary is only known at the NEXT independent
// substream, so a trailing independent substream is held across the PES
// boundary: the dependent half arriving in the next PES still joins it,
// and the AU keeps the FIRST PES's PTS (its independent substream's).
// The AU boundary is only known at the NEXT independent substream, so a
// trailing one is held across the PES boundary: the dependent half in the
// next PES still joins it, keeping the FIRST PES's PTS.
let mut parser = Ac3Parser::new();
let indep = make_eac3_frame(0, 0, 160);
let dep = make_eac3_frame(1, 0, 96);
@@ -2174,14 +2094,9 @@ mod tests {
#[test]
fn eac3_additional_independent_substream_stays_in_the_frame_set() {
// THE FIX: an Annex E frame set is independent substream 0 (mandatory,
// first) with its dependents, then the OPTIONAL additional independent
// substreams 1..7 with theirs — ALL covering the SAME time period. An
// additional independent substream (an associated / commentary service)
// therefore belongs to the frame set already open; keying the access-unit
// boundary on `strmtyp` alone made it close the AU and advance the running
// PTS a SECOND time over the same 32 ms, doubling the timeline (~1 s of
// A/V drift per second of audio).
// THE FIX: a frame set is substream 0 plus dependents, then OPTIONAL
// additional independent substreams 1..7 — ALL the SAME time period.
// Keying the AU boundary on `strmtyp` alone doubled the PTS advance.
let mut parser = Ac3Parser::new();
let ind0 = make_eac3_frame(0, 0, 160); // main programme
let dep0 = make_eac3_frame(1, 0, 96); // its dependent (7.1 extension)
@@ -2228,13 +2143,9 @@ mod tests {
#[test]
fn eac3_stream_joined_mid_frame_set_resyncs_at_substreamid_0() {
// A stream whose first syncframe is an ADDITIONAL independent substream
// (here substreamid 3) joined a frame set whose mandatory substreamid-0
// substream was never seen. Mirroring the orphan-dependent rule, it is
// skipped: it carries the frame set's time period rather than its own, so
// emitting it as an access unit would invent a period for audio whose
// main programme is missing. Grouping resyncs at the next substreamid-0
// substream, and the orphan consumes none of the timeline.
// A stream whose first syncframe is ADDITIONAL substream (substreamid 3)
// joined a set whose mandatory substreamid-0 was never seen. It's skipped
// (carries the set's time period, not its own); resyncs at substreamid-0.
let mut parser = Ac3Parser::new();
let orphan = make_eac3_frame(0, 3, 128);
let orphan_dep = make_eac3_frame(1, 3, 96);
@@ -2291,10 +2202,9 @@ mod tests {
#[test]
fn corrupt_dependent_substream_drops_the_whole_access_unit() {
// A dependent substream that fails its native CRC poisons the access unit
// it belongs to: emitting the independent half alone would ship an AU a
// decoder must reassemble from a corrupt pair. The drop accounts for one
// frame duration, so the following AU keeps its true PTS (gap, not shift).
// A dependent substream failing its native CRC poisons the whole AU:
// emitting the independent half alone ships a pair no decoder can
// reassemble. The drop still accounts for one duration (gap, not shift).
let mut parser = Ac3Parser::new();
let mut dep = make_eac3_frame(1, 0, 96);
dep[20] ^= 0xFF; // break the dependent substream's CRC
@@ -2516,10 +2426,9 @@ mod tests {
let mut data = Vec::new();
for _ in 0..CORRUPT_AUS {
let mut f = eac3_substream_frame(0, 0);
// Corrupt a payload byte AFTER the CRC was finalized: the header
// (sizing, strmtyp/substreamid, bsid) stays intact so the frame is
// still parsed as a whole access unit and fails only the CRC —
// a VERIFIED drop, the only kind that feeds the poison verdict.
// Corrupt a payload byte AFTER CRC finalization: header stays intact so
// the frame is still parsed whole and fails only CRC — a VERIFIED
// drop, the only kind that feeds the poison verdict.
f[100] ^= 0xFF;
assert!(!frame_crc_ok(&f), "the fixture frame must fail its CRC");
data.extend_from_slice(&f);
+3 -7
View File
@@ -50,13 +50,9 @@ fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
// The floor is the header the frame SAYS it carries, not a constant.
// protection_absent (byte1 bit0) clear means a 16-bit crc_check follows the
// 7-byte fixed+variable header, so the frame cannot be shorter than 9 —
// aac_frame_length counts the header and the CRC, not just the payload.
// Comparing against a flat 7 let a CRC-present frame declaring 7 or 8
// through as structurally Valid, and the muxer then carried a frame whose
// own header says it is impossible.
// Floor depends on protection_absent: CRC-present (byte1 bit0 clear) adds a
// 16-bit crc_check after the 7-byte header, so the min is 9, not a flat 7 —
// else a CRC frame declaring length 7-8 wrongly passed as Valid.
let header_bytes = if data[1] & 0x01 == 0 { 9 } else { 7 };
if frame_length < header_bytes {
return AdtsVerdict::Invalid;
+4 -8
View File
@@ -3,7 +3,7 @@
//! [`PictureInfo`] is the single per-frame carrier of coding signals that the
//! muxer (and any downstream index/diagnostic) reads WITHOUT branching on the
//! codec. Each codec's parser decodes its own bitstream once and folds the raw
//! signals into a [`CodingDetail`] variant; consumers then call ONLY the
//! signals into a [`CodingDetail`](crate::mux::codec::coding::CodingDetail) variant; consumers then call ONLY the
//! codec-agnostic accessors ([`coding_type`](PictureInfo::coding_type),
//! [`field_order`](PictureInfo::field_order), [`nb_fields`](PictureInfo::nb_fields),
//! [`progressive`](PictureInfo::progressive)). The accessor surface is fixed:
@@ -181,13 +181,9 @@ impl PictureInfo {
match self.detail {
CodingDetail::Mpeg2(m) => {
if !m.frame_picture {
// A single field picture is inherently interlaced. Which
// field it actually codes is given by picture_structure
// (top/bottom), not by top_field_first — §6.3.10 constrains
// top_field_first to 0 for field pictures, so it is not the
// spec source here. picture_structure is not retained on
// this carrier, so top_field_first is used only as the lone
// field hint available (best-effort, not spec-derived).
// Field pictures are coded top/bottom by picture_structure, not
// top_field_first (§6.3.10 forces it to 0 here). picture_structure
// isn't retained on this carrier, so this is a best-effort hint.
Some(if m.top_field_first {
FieldOrder::Tff
} else {
+6 -12
View File
@@ -99,24 +99,18 @@ mod tests {
#[test]
fn crc16_mlp_known_vector_check_bytes() {
// Independent known-answer for CRC-16 poly 0x002D, init 0, MSB-first over
// the catalogue string "123456789" is 0x4FF7 — computed by a separate
// reference implementation (NOT by crc16_mlp), so a wrong polynomial or
// shift direction here fails this test even though every truehd fixture
// (which derives its trailer from crc16_mlp itself) would still pass.
// Independent known-answer (poly 0x002D, init 0, MSB-first) from a separate
// reference implementation, not crc16_mlp itself — catches a wrong
// polynomial/shift that truehd fixtures (self-derived) would miss.
assert_eq!(crc16_mlp(b"123456789"), 0x4FF7);
assert_eq!(crc16_mlp(&[0x00, 0x01, 0x02, 0x03]), 0x5E26);
}
#[test]
fn crc16_mlp_residue_property_holds() {
// Appending the big-endian CRC zeroes the residue over message+crc.
// This is a property of the CRC itself, pinned here so a change to the
// polynomial or the bit order is caught. It is NOT how the TrueHD
// caller validates a major sync: `truehd::mlp_major_sync_crc_ok` does a
// swap-and-XOR compare against the little-endian trailer word. (This
// comment used to claim the caller relied on the residue, and named a
// `truehd::mlp_major_sync_ok` that does not exist.)
// Appending the big-endian CRC zeroes the residue (pinned to catch poly/bit
// changes). NOT how TrueHD validates — mlp_major_sync_crc_ok instead does a
// swap-and-XOR compare against the little-endian trailer word.
let msg = [0xF8u8, 0x72, 0x6F, 0xBA];
let c = crc16_mlp(&msg);
let mut framed = msg.to_vec();
+1 -1
View File
@@ -59,7 +59,7 @@ impl DropTally {
}
/// Whether the track has been judged too damaged to mux. Once `true`, the
/// caller should drop every remaining AU (passing them to [`record_drop`]
/// caller should drop every remaining AU (passing them to [`Self::record_drop`]
/// with a poison reason) rather than emit them.
pub(crate) fn is_poisoned(&self) -> bool {
self.poisoned
+123 -257
View File
@@ -45,7 +45,7 @@ pub struct DtsParser {
/// Keep/drop bookkeeping for the decodability gate: counts, per-drop and
/// aggregate logging, and the whole-track poison fallback. A dropped AU is
/// NEVER emitted, but the PTS clock is still advanced across it (see
/// [`stamp_pts`] usage) so every SURVIVING AU keeps the exact timestamp it
/// [`Self::stamp_pts`] usage) so every SURVIVING AU keeps the exact timestamp it
/// would have had — a drop becomes a silence gap, never a shift.
tally: super::dropgate::DropTally,
}
@@ -120,7 +120,7 @@ impl DtsParser {
}
/// Stamp an access unit's PTS. `front` is the AU's own core-PES PTS (from
/// [`front_pts`]); `dur_ns` its decoded duration.
/// [`Self::front_pts`]); `dur_ns` its decoded duration.
///
/// The model matches the (correct) AC-3 path: **re-base to each PES's own
/// container timestamp, and advance by one frame duration ONLY within a run
@@ -201,16 +201,9 @@ const PTS_UNSET: i64 = -1;
impl CodecParser for DtsParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
// B1: a concealed/lost gap means the buffered DTS access unit is
// TRUNCATED. Splicing post-gap bytes onto it corrupts the core/extension
// framing (→ "Failed to decode block code(s)" / "Invalid data found").
// Drop the partial AU and its PTS marks; the next PES re-bases a fresh
// unit. (Audio has no inter-frame refs — dropping the spliced partial is
// the whole fix; the video ResyncGate handles video.)
//
// Handle the discontinuity BEFORE the empty-data guard so the signal can
// never be stranded by an empty post-gap PES (defensive; the demuxer only
// emits non-empty PES today).
// B1: a concealed/lost gap means the buffered DTS AU is TRUNCATED; splicing
// post-gap bytes onto it corrupts the framing. Drop the partial AU and its
// PTS marks (DTS frames independently resync, unlike TrueHD/MLP).
if pes.discontinuity {
self.acc.clear();
self.pending_pts = PTS_UNSET;
@@ -222,10 +215,6 @@ impl CodecParser for DtsParser {
if pes.data.is_empty() {
return Vec::new();
}
// A PES with no PTS (rare for audio, but legal — the case demuxers
// guard at a post-gap continuation) must NOT reset the timeline to 0;
// continue from the most recent known base. Defense-in-depth: the
// discontinuity-carrying PES is a PUSI with a PTS in practice.
// A PES with no PTS (rare for audio, but legal) must NOT reset the
// timeline to 0 — continue from the most recent known base.
let pts_ns = super::pesbuf::PesFacts::of(pes)
@@ -236,34 +225,15 @@ impl CodecParser for DtsParser {
0
});
// On Blu-ray, a DTS-HD MA/HRA access unit is a DTS core frame
// (sync 0x7FFE8001) followed by one or more DTS extension substreams
// (sync 0x64582025). The m2ts demuxer hands those out as SEPARATE PES
// packets on the same PID — the core in one PES, then the extension
// substreams in following PES packets (with their own, later PTS). The
// lossless audio lives entirely in the extension substreams, so an
// access unit is only complete once all of its trailing extensions
// have been buffered. We assemble across PES boundaries here: an access
// unit runs from its core sync up to (but not including) the NEXT core
// sync. Emitting on the core boundary keeps the core + every following
// extension substream together (the lossless data), instead of the
// old per-PES emit that dropped the extension PES packets and
// downgraded the track to lossy DTS core (the lossy-core
// downgrade bug). The PTS is the core frame's PTS, captured when the unit began.
// Capture the access unit's PTS base on a fresh buffer, or whenever a
// prior forced (safety-valve) flush left it invalidated — in the
// forced case the bytes still in `buf` are not a real core frame, so
// the first PES to arrive after the flush carries the correct base.
// Blu-ray DTS-HD MA/HRA: core frame + extension substreams (lossless data)
// arrive in SEPARATE, later PES packets, so assemble core-to-next-core
// (else extensions get dropped = lossy core) and capture PTS base fresh.
if self.acc.is_empty() || self.pending_pts == PTS_UNSET {
self.pending_pts = pts_ns;
}
// Mark where THIS PES's bytes begin in the buffer, with its PTS. The
// emitted access unit takes the PTS of the PES covering its first byte
// (see `front_pts`), so an AU whose core arrived in an earlier PES keeps
// that core's timestamp even when its extensions / the following core
// arrive (with a later PTS) in this same parse() call.
// `pts_ns` is this packet's own timestamp, or the carried-forward base
// when it had none; the source offset is always this packet's.
// Mark where THIS PES's bytes begin, with its PTS: the emitted AU takes
// the PTS of the PES covering its first byte (see `front_pts`), so an AU
// whose core arrived earlier keeps that timestamp over later extensions.
self.acc.push_with(
&pes.data,
super::pesbuf::PesFacts::of(pes).with_pts_ns(pts_ns),
@@ -299,15 +269,9 @@ impl CodecParser for DtsParser {
break;
}
let core_size = dts_core_frame_size(self.acc.as_slice());
// `dts_core_frame_size` returns a 14-bit `fsize + 1`, so it is
// always in [1, 16384]; the bare `== 0` / `> MAX_AU_BYTES` checks
// can never fire. A real DTS core frame is at least
// MIN_CORE_FRAME_BYTES (96, the ETSI spec floor), so any decoded
// size below that came from a false/corrupt sync. Reject it (drain
// the 4 syncword bytes and resync) instead of letting a tiny bogus
// size close the current access unit at a junk boundary and drop the
// trailing extension substreams. The `> MAX_AU_BYTES` upper bound is
// kept as a harmless guard.
// A real core frame is at least MIN_CORE_FRAME_BYTES (96, ETSI floor);
// a smaller size came from a false sync, so reject it (drain 4 bytes,
// resync) rather than close the AU at a junk boundary.
if !(MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&core_size) {
// Bogus core sync — skip past it and resync.
self.drain_front(4);
@@ -317,39 +281,24 @@ impl CodecParser for DtsParser {
break; // core frame not fully buffered yet — wait
}
// The access unit ends at the next *valid* core sync. The search
// begins after this core's syncword so we don't re-match it.
// Anything between the core and that next sync is this unit's
// extension substream(s) — which can themselves contain byte
// sequences matching the core syncword, so a raw `find_sync` match
// is not enough: a candidate is only a real boundary if its decoded
// core size is plausible. `next_core_boundary` skips bogus matches.
//
// `forced` distinguishes a real next-core boundary from a forced
// safety-valve flush. On a forced flush the access unit was NOT
// closed by a new core sync, so the bytes following it are not a
// fresh core frame and the current PES's PTS (which on a forced
// flush is an extension-substream PES, carrying its own later
// timestamp) must NOT become the next unit's PTS base.
// The AU ends at the next *valid* core sync; extension bytes can contain
// false syncword matches, so `next_core_boundary` validates decoded size.
// `forced` marks a safety-valve flush whose PTS must NOT become the base.
let mut forced = false;
let (au_end, ext_clean) = match next_core_boundary(self.acc.as_slice(), core_size) {
NextCore::Found { end, ext_clean } => (end, ext_clean),
NextCore::NeedMore if self.acc.len() <= MAX_AU_BYTES => break,
NextCore::NeedMore => {
// A candidate boundary exists but is not fully buffered. Normally
// we wait for more PES; but once the buffer exceeds the AU cap,
// apply the same force-flush safety valve as `None` so a crafted
// stream that keeps a boundary perpetually incomplete can't grow
// `buf` without bound (the `break` above never reaches it).
// A candidate boundary isn't fully buffered; normally wait, but
// past the AU cap apply the same force-flush as `None` so a
// crafted stream can't grow `buf` without bound.
forced = true;
(self.acc.len(), true)
}
NextCore::None => {
// No next core sync buffered yet. The trailing extension
// substream PES packets may still be arriving, so WAIT for
// them rather than emit a core-only (lossy) frame — unless
// the buffer has grown unreasonably large, in which case
// emit what we have to guarantee forward progress.
// No next core sync yet: trailing extension PES packets may
// still arrive, so WAIT rather than emit a lossy core-only
// frame, unless the buffer has grown unreasonably large.
if self.acc.len() <= MAX_AU_BYTES {
break;
}
@@ -358,26 +307,18 @@ impl CodecParser for DtsParser {
}
};
// Damaged source encoding: when the extension boundary was GARBAGE
// (not any DTS sync — `ext_clean == false`), the extension bytes for
// this AU are corrupt and would make the decoder cascade "DSYNC check
// failed" / "Read past end of XLL band data". Emit the clean DTS core
// ALONE (a decodable, lossy frame) and still drain past the garbage to
// the next core — a perfect mux drops the bad frame's corrupt part
// rather than shipping it. A recognized-but-unsizeable extension
// (`ext_clean == true`) is preserved in full (lossless).
// When the extension boundary is GARBAGE (`ext_clean == false`), emit
// the clean core ALONE (lossy) and drain past it to the next core; an
// unsizeable-but-recognized extension is kept in full (lossless).
let emit_end = if ext_clean { au_end } else { core_size };
let au: Vec<u8> = self.acc.as_slice()[..emit_end].to_vec();
// The AU's own core PES PTS (the PES covering its first byte, even if
// that PES preceded the one(s) carrying its extensions or the next
// core), stamped monotonically: honored when it advances past the
// running clock (UHD one-AU-per-PES), but never allowed to collide
// with the previous AU when several cores share ONE PES (DVD).
// The AU's own core PES PTS, stamped monotonically: honored when it
// advances past the running clock (UHD, one AU per PES), but never
// allowed to collide with the previous AU (DVD, several cores per PES).
let dur_ns = dts_core_duration_ns(&au) as i64;
// Advance the PTS clock for this AU BEFORE the decodability gate, so
// a dropped AU still advances the timeline exactly as an emitted one
// would: the following AU keeps its true PTS and the drop is a gap,
// never a shift. `emit_or_drop` decides whether to actually push it.
// Advance the PTS clock BEFORE the decodability gate, so a dropped AU
// still advances the timeline like an emitted one (gap, not shift).
// `emit_or_drop` decides whether to actually push it.
let au_pts = self.stamp_pts(self.front_pts(), dur_ns);
// Read BEFORE draining: after the drain the front is the NEXT
// unit's packet, not this one's.
@@ -389,10 +330,8 @@ impl CodecParser for DtsParser {
// marker survives. Track it so the fallback stays sensible.
self.pending_pts = self.front_pts();
if forced {
// Safety-valve flush: the next access unit's real core PES has
// not arrived. Invalidate the PTS so the next PES sets it
// regardless of buffer state, rather than inheriting this
// (non-core) PES's timestamp.
// Safety-valve flush: the next AU's real core PES hasn't arrived,
// so invalidate the PTS rather than inherit this non-core PES's.
self.pending_pts = PTS_UNSET;
}
}
@@ -553,10 +492,8 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
pos += sz; // skip the whole extension substream precisely
}
// A real extension sync we couldn't size (truncated header /
// unsupported sub-form) — heuristic fallback, but the region IS a
// recognized extension, so keep it (ext_clean = true). This path
// is load-bearing for valid streams.
// A real extension sync we couldn't size (truncated/unsupported):
// heuristic fallback, but the region IS recognized, so keep it.
_ => return scan_for_next_core(buf, pos, true),
}
} else if buf[pos..].starts_with(&DTS_CORE_SYNC) {
@@ -574,10 +511,9 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
return scan_for_next_core(buf, pos, true); // implausible core — recognized sync, keep
} else {
// GARBAGE at the extension boundary — neither a core sync nor a
// DTS-HD extension sync. This is damaged source encoding: the
// extension region is corrupt. Mark ext_clean = false so the caller
// emits the clean core alone and drops the garbage.
// GARBAGE at the extension boundary — neither a core nor extension
// sync, so the extension region is corrupt: mark ext_clean = false so
// the caller emits the clean core alone and drops the garbage.
return scan_for_next_core(buf, pos, false);
}
}
@@ -748,14 +684,9 @@ impl DropReason {
fn core_header_drop_reason(au: &[u8]) -> Option<DropReason> {
let mut r = BitReader::new(au.get(SYNCWORD_BYTES..)?);
// FTYPE: 1 = NORMAL frame, 0 = TERMINATION frame (the last frame of the
// stream). Per ETSI TS 102 114: a normal frame must carry a full 32-sample
// PCM block, so a DEFICIT_SAMPLE_COUNT other than the full block marks a
// terminating/short frame (the spec's deficit semantics, which
// on `normal_frame`) — the deficit-sample field must equal 32 ONLY for a
// normal frame. A termination frame legitimately carries fewer samples and
// is fully decodable; dropping it would silence the last frame of every
// stream that ends on one (a guaranteed per-track loss on real discs).
// FTYPE: 1 = NORMAL frame, 0 = TERMINATION frame. Per ETSI TS 102 114 the
// deficit-sample field must equal 32 ONLY for a normal frame; gating a
// termination frame on it would silence the last frame of every stream.
let normal_frame = r.read_bit()? == 1;
let deficit_samples = r.read_bits(5)? + 1;
if normal_frame && deficit_samples != DTS_PCMBLOCK_SAMPLES {
@@ -779,11 +710,9 @@ fn core_header_drop_reason(au: &[u8]) -> Option<DropReason> {
return Some(DropReason::SampleRate);
}
let _br_code = r.read_bits(5)?;
// Reserved bit. Both reference decoders SKIP this field rather than reject
// on it — the field is reserved in ETSI TS 102 114 and skipped ("Reserved
// field"). A frame that sets it is still fully decodable, so rejecting it
// was a false-drop that silenced any real stream whose encoder set the bit.
// Read past it without gating (never reject a decodable frame).
// Reserved bit (ETSI TS 102 114): both reference decoders SKIP it rather
// than reject on it, so read past without gating — a frame setting it is
// still fully decodable.
let _reserved = r.read_bit()?;
// drc, ts, aux, hdcd (1 each) → ext_audio_type (3) → ext_present, aspf (1 each).
r.skip_bits(4)?;
@@ -829,11 +758,9 @@ mod tests {
let fsize = size - 1;
let mut data = vec![0u8; size];
data[0..4].copy_from_slice(&DTS_CORE_SYNC);
// byte4: FTYPE(1) SHORT(5) CPF(0) NBLKS-high(0). FTYPE = 1 = a NORMAL
// frame (the common real-stream case); SHORT = 31 makes deficit_samples
// = 32 = DTS_PCMBLOCK_SAMPLES, which the decodability gate (per ETSI TS
// 102 114) requires of a normal frame. NBLKS high bit (byte4 bit0) stays
// 0 for NBLKS = 15. (0x80 | (31 << 2) = 0xFC.)
// byte4: FTYPE(1) SHORT(5) CPF(0) NBLKS-high(0). FTYPE=1 (NORMAL frame);
// SHORT=31 makes deficit_samples=32=DTS_PCMBLOCK_SAMPLES, required by the
// decodability gate for a normal frame. (0x80 | (31 << 2) = 0xFC.)
data[4] = 0x80 | (31u8 << 2);
// NBLKS = 15 → (15+1)*32 = 512 samples/frame (the DVD/UHD DTS-core norm).
// NBLKS is byte4 bit0 + byte5 bits7-2; here byte4 bit0 = 0, byte5 = 15<<2.
@@ -931,11 +858,9 @@ mod tests {
// EXSS size parse round-trips.
assert_eq!(exss_frame_size(&make_exss(600, None)), Some(600));
// AU = core(512) + a REAL EXSS substream whose XLL payload embeds a DTS
// core syncword decoding to a plausible size (512). The heuristic-only
// framer would split here and truncate the lossless extension (the
// Dunkirk "Failed to decode block code(s)" decoder-failure class).
// Precise EXSS sizing spans the whole extension to the REAL next core.
// AU = core(512) + a REAL EXSS substream whose XLL payload embeds a false
// DTS core syncword (plausible size 512). Precise EXSS sizing must span
// the whole extension to the REAL next core, or the extension gets split.
let core = make_dts_core(512);
let exss = make_exss(600, Some(40));
let next = make_dts_core(512);
@@ -954,10 +879,9 @@ mod tests {
#[test]
fn garbage_extension_emits_core_only_but_valid_ext_is_kept() {
// Damaged source: a valid core, then GARBAGE (no core sync, no extension
// sync) where the extension belongs, then the next core. The framer must
// mark this boundary ext_clean=false and the parser must emit the clean
// 512-byte CORE alone (dropping the garbage), draining to the next core.
// Damaged source: valid core, then GARBAGE (no core/extension sync) where
// the extension belongs, then the next core. Must mark ext_clean=false and
// emit the clean 512-byte CORE alone, draining past the garbage.
let core = make_dts_core(512);
let garbage = vec![0xE4, 0x3F, 0xE3, 0x90, 0xCC, 0x6C]; // real head bytes from a damaged stream
let mut garbage = garbage;
@@ -1039,12 +963,9 @@ mod tests {
#[test]
fn discontinuity_drops_truncated_partial() {
// B1: a partial DTS core is buffered, then a concealed gap (PES marked
// discontinuity) carries a fresh core. The truncated partial must be
// DROPPED — splicing it makes the framer emit a corrupt sub-core-length
// AU (the Dunkirk "Failed to decode block code(s)" decoder-failure
// class) and strands the rest. With the fix the post-gap core is the
// only AU, and it carries the post-gap PTS (not the stale pre-gap one).
// B1: a partial core is buffered, then a discontinuity-marked PES carries
// a fresh one. The partial must be DROPPED, not spliced; the post-gap
// core is the only AU emitted, with the post-gap PTS.
let mut parser = DtsParser::new();
// PES 1: first half of a 512-byte core (no boundary marker).
@@ -1082,10 +1003,9 @@ mod tests {
#[test]
fn two_cores_back_to_back_advance_within_one_pes() {
// Both cores arrive in ONE PES — the DVD layout. AU1 keeps the PES PTS;
// AU2 must ADVANCE by one frame duration to stay monotonic. Reusing the
// single PES PTS for both was the "non monotonically increasing dts to
// muxer" bug (fixed for 1.2.1).
// Both cores arrive in ONE PES (DVD layout). AU1 keeps the PES PTS; AU2
// must ADVANCE by one frame duration to stay monotonic — reusing the same
// PES PTS for both was the "non monotonically increasing dts" bug (1.2.1).
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512);
stream.extend_from_slice(&make_dts_core(640));
@@ -1110,12 +1030,9 @@ mod tests {
#[test]
fn two_aus_flushed_in_one_call_keep_their_own_pts() {
// The real-stream trigger: core1 in PES A (pts 100), then core2 + core3
// arrive in a LATER PES B (pts 200). Processing PES B closes both AU1
// (core1) and AU2 (core2) in a single parse() call. AU2's core arrived
// in PES A region? No — here AU2 (core2) is in PES B, so it should be
// 200. The defect to guard is AU1 NOT being overwritten to 200, and AU2
// not inheriting an unrelated timestamp.
// core1 in PES A (pts 100), then core2 + core3 in a LATER PES B (pts 200).
// Processing PES B closes both AU1 (core1) and AU2 (core2, in PES B, so
// 200) in one parse() call. Guard: AU1 must NOT be overwritten to 200.
let mut parser = DtsParser::new();
// PES A: just core1 (held — no following core yet).
@@ -1150,10 +1067,9 @@ mod tests {
#[test]
fn second_au_with_core_in_earlier_pes_keeps_that_pts() {
// core1 + core2 both arrive in PES A (pts 100); core3 arrives in PES B
// (pts 200). When PES B closes AU2 (core2, whose core was in PES A), AU2
// must keep PES A's 100 — the bug was AU2 inheriting the closing PES's
// 200.
// core1+core2 arrive in PES A (pts 100); core3 in PES B (pts 200). When PES
// B closes AU2 (core2, whose core was in PES A), AU2 must keep PES A's
// 100 — the bug was AU2 inheriting the closing PES's 200.
let mut parser = DtsParser::new();
// PES A: core1 + core2. AU1 (core1) emits immediately (core2 boundary);
@@ -1164,10 +1080,9 @@ mod tests {
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000), "AU1 PES A PTS");
// PES B (realistically later): core3 — closes AU2 (core2). AU2's core
// was in PES A, so it is PES A's 2nd frame: it advances one frame
// duration from AU1 (still on PES A's timeline, and monotonic) — it does
// NOT inherit the closing PES B PTS.
// PES B: core3 closes AU2 (core2). AU2's core was in PES A, so it's PES
// A's 2nd frame — advances one duration from AU1, and does NOT inherit
// the closing PES B PTS.
let f2 = parser.parse(&make_pes(make_dts_core(640), Some(190000)));
assert_eq!(f2.len(), 1);
assert_eq!(f2[0].data.len(), 600, "AU2 = core2");
@@ -1189,10 +1104,9 @@ mod tests {
#[test]
fn dvd_many_cores_one_pes_are_strictly_monotonic() {
// Punisher-DVD reproduction: a single PES carrying SEVERAL DTS core
// frames (the DVD packing) must emit STRICTLY-increasing PTSs. The old
// code stamped every AU with the one PES PTS, which a muxer rejects as
// "non monotonically increasing dts to muxer: X >= X".
// Punisher-DVD repro: a PES with SEVERAL DTS cores must emit STRICTLY
// increasing PTSs; the old code stamped every AU with the one PES PTS,
// rejected by muxers as "non monotonically increasing dts: X >= X".
let mut parser = DtsParser::new();
let mut stream = Vec::new();
for _ in 0..6 {
@@ -1238,11 +1152,9 @@ mod tests {
#[test]
fn dts_core_sfreq_table_matches_the_dca_spec() {
// Lock the SFREQ → sample-rate table to the authoritative values in
// ETSI TS 102 114 Table 6-4. The high-rate triad in particular —
// 48 k / 96 k / 192 k at indices 13/14/15 — must not
// be shifted; a wrong entry would compute an N× frame duration and
// reintroduce PTS drift on a 96/192 kHz DTS stream.
// Lock the SFREQ → sample-rate table to ETSI TS 102 114 Table 6-4. The
// high-rate triad (48/96/192k at indices 13/14/15) must not shift; a wrong
// entry computes an N× frame duration, reintroducing PTS drift.
let mut core = make_dts_core(512);
let set_sfreq = |c: &mut [u8], idx: u8| c[8] = (c[8] & !0x3C) | ((idx & 0x0F) << 2);
for (idx, want) in [
@@ -1269,19 +1181,9 @@ mod tests {
#[test]
fn new_pes_rebases_to_its_own_pts_no_drift() {
// Regression for the drift bug: a global running clock overshot a
// feature-long DTS track by minutes (2h44 for a 2h03 film). When a NEW
// PES arrives whose PTS is BEHIND where accumulated frame durations
// would put a running clock, the AU must re-base to that PES's OWN
// timestamp — tracking the container, not drifting ahead of it.
//
// The re-base can make one emitted PTS sit just below the previous AU's
// (a fresh PES whose PTS lands under the within-PES cursor). That is
// CORRECT here and is NOT a muxer defect: the parser reports the true
// container timestamps, and the mkv muxer applies the strictly-monotonic
// per-track nudge to AUDIO at emit time (`mkv::block_ts` / `monotonic_ts`,
// tested in `mkv.rs`), so the written block DTS is always monotonic. The
// alternative — clamping in the parser — is what reintroduced the drift.
// Drift-bug regression: a global running clock overshot a track by minutes.
// A NEW PES BEHIND the running clock must re-base to its OWN timestamp;
// the mkv muxer nudges AUDIO monotonic at emit — clamping here caused it.
let mut parser = DtsParser::new();
// PES A: core1 + core2 (2 frames), pts 90000.
let mut pes_a = make_dts_core(512);
@@ -1325,12 +1227,9 @@ mod tests {
#[test]
fn keeps_dts_hd_extension_in_separate_pes_packets() {
// The real Blu-ray layout (ground-truthed on real UHD discs): the DTS core
// arrives in one PES, then its DTS-HD MA extension substreams arrive
// in SEPARATE following PES packets on the same PID. The parser must
// stitch core + all trailing extensions into one access unit — not
// emit a core-only (lossy) frame and drop the extension PES packets
// (the lossy-core downgrade bug).
// Real Blu-ray layout: the DTS core arrives in one PES, then its DTS-HD MA
// extensions arrive in SEPARATE following PES packets. Must stitch core +
// all trailing extensions into one AU, not emit lossy core-only frames.
let mut parser = DtsParser::new();
// Frame 1: core (512) + two extension substreams (256 + 200).
@@ -1411,11 +1310,9 @@ mod tests {
#[test]
fn bogus_tiny_core_sync_does_not_split_or_drop_real_au() {
// A real core frame followed by an extension substream that happens to
// contain a false core sync whose fsize decodes tiny. The bogus sync
// must NOT close the real access unit early (dropping the rest of the
// extension) nor emit a junk few-byte frame — it must be skipped, and
// the whole core + extension preserved as one access unit.
// A real core followed by an extension containing a false core sync whose
// fsize decodes tiny. The bogus sync must NOT close the AU early or emit
// a junk frame — skip it and preserve core + extension as one AU.
let mut parser = DtsParser::new();
// Frame 1: core(512) + an extension whose body embeds a bogus tiny
@@ -1452,10 +1349,8 @@ mod tests {
#[test]
fn sub_spec_core_size_is_rejected_as_false_sync() {
// A core sync whose decoded fsize+1 lands in [CORE_HEADER_MIN_BYTES,
// MIN_CORE_FRAME_BYTES) — i.e. a "frame" smaller than the 96-byte ETSI
// spec minimum — is a false sync inside extension payload and must NOT
// close an access unit. Pick a decoded size of 64 (well inside the old
// 10..96 false-positive window the raised floor now rejects).
// MIN_CORE_FRAME_BYTES) — smaller than the 96-byte ETSI spec minimum — is
// a false sync and must NOT close an AU. 64 sits in the raised reject window.
let false_size = 64usize;
assert!(
(CORE_HEADER_MIN_BYTES..MIN_CORE_FRAME_BYTES).contains(&false_size),
@@ -1488,10 +1383,9 @@ mod tests {
#[test]
fn forced_emit_does_not_corrupt_next_au_pts() {
// When the buffer exceeds MAX_AU_BYTES with no next core sync, the
// parser force-emits for forward progress. The current PES at that
// point is an extension-substream PES (later PTS). The forced path must
// NOT make that extension PTS the base of the NEXT access unit.
// When the buffer exceeds MAX_AU_BYTES with no next core sync, the parser
// force-emits for forward progress; the current PES is an extension PES
// (later PTS). The forced path must NOT base the NEXT AU on that PTS.
let mut parser = DtsParser::new();
// Core PES at the real PTS, then a giant extension (no next core) that
@@ -1533,13 +1427,9 @@ mod tests {
#[test]
fn needmore_past_cap_force_flushes_to_bound_buffer() {
// A crafted DTS-HD stream whose extension substream declares a size
// larger than what is (ever) buffered keeps `next_core_boundary` in a
// sustained NeedMore state (a candidate boundary that is never fully
// buffered). Once `buf` exceeds MAX_AU_BYTES the NeedMore force-flush
// safety valve must fire — mirroring the None arm — so the buffer can't
// grow without bound. WITHOUT the guard the parser would `break` and
// retain everything, emitting nothing.
// A crafted stream whose extension declares a size larger than ever
// buffered keeps `next_core_boundary` in sustained NeedMore; past
// MAX_AU_BYTES the force-flush valve must fire, or `buf` grows forever.
let mut parser = DtsParser::new();
let core = make_dts_core(512);
@@ -1586,10 +1476,9 @@ mod tests {
#[test]
fn core_frame_size_bit_layout() {
// fsize is 14 bits at bits 46-59: byte5[1:0] (high 2), byte6 (mid 8),
// byte7[7:4] (low 4). Returned value is fsize + 1 (on-wire length-1).
// Set fsize = 0x1FFF (= 8191): byte5 low2 = 0b01, byte6 = 0xFF,
// byte7 high4 = 0xF (0xF0). (1<<12)|(0xFF<<4)|0xF = 0x1FFF → size 8192.
// fsize is 14 bits at bits 46-59: byte5[1:0] (high2), byte6 (mid8),
// byte7[7:4] (low4); returned value is fsize + 1 (on-wire length-1).
// (1<<12)|(0xFF<<4)|0xF = 0x1FFF → size 8192.
let mut d = vec![0u8; CORE_HEADER_MIN_BYTES];
d[5] = 0x01;
d[6] = 0xFF;
@@ -1634,10 +1523,8 @@ mod tests {
#[test]
fn dts_core_samples_reads_nblks_high_bit_from_byte4_bit0() {
// NBLKS is 7 bits: byte4 bit0 is the HIGH bit (<<6), byte5>>2 the low 6.
// With byte4 bit0 set and byte5 = 0, nblks = 64 (not 0), so a `<<` -> `>>`
// typo on the byte4 contribution (which always yields 0, since a 1-bit
// value has nothing to shift right into) collapses samples from 2080
// down to 32.
// With byte4 bit0 set and byte5 = 0, nblks = 64 (not 0); a `<<` -> `>>`
// typo on byte4 (always 0) would collapse samples from 2080 to 32.
let mut d = vec![0u8; CORE_HEADER_MIN_BYTES];
d[4] = 0x01;
d[5] = 0x00;
@@ -1646,13 +1533,9 @@ mod tests {
#[test]
fn dts_core_samples_and_sample_rate_decode_at_exact_header_min_length() {
// The length guard is `data.len() < CORE_HEADER_MIN_BYTES`; exactly
// CORE_HEADER_MIN_BYTES (10) bytes must still be read as a real header
// (index 8, SFREQ, is in bounds), not treated as truncated. Use NBLKS
// and SFREQ values that differ from the truncated-header fallbacks
// (512 samples, 48 kHz) so a `<` -> `<=` typo that takes the fallback
// path one byte early is visible in the output, not masked by a
// coincidental match.
// The guard is `data.len() < CORE_HEADER_MIN_BYTES`; exactly 10 bytes must
// still read as a real header (index 8, SFREQ, is in bounds). Use non-
// fallback NBLKS/SFREQ so a `<` -> `<=` typo is visible, not masked.
let mut d = vec![0u8; CORE_HEADER_MIN_BYTES];
d[4] = 0x00;
d[5] = 0x00; // nblks = 0 -> 32 samples, not the 512 fallback
@@ -1664,11 +1547,9 @@ mod tests {
#[test]
fn dts_core_samples_and_sample_rate_decode_past_header_min_length_too() {
// A `<` -> `>` typo on the same guard takes the FALLBACK path once
// `data.len()` exceeds CORE_HEADER_MIN_BYTES instead of never — i.e.
// every real, normally-sized core frame (always well past 10 bytes)
// would silently report the 512-sample/48kHz fallback. Use a buffer
// twice the minimum with non-fallback NBLKS/SFREQ values.
// A `<` -> `>` typo on the same guard takes the FALLBACK path once len
// exceeds CORE_HEADER_MIN_BYTES — every real core frame would silently
// report the 512-sample/48kHz fallback. Use a buffer twice the minimum.
let mut d = vec![0u8; CORE_HEADER_MIN_BYTES * 2];
d[4] = 0x00;
d[5] = 0x00; // nblks = 0 -> 32 samples
@@ -1728,10 +1609,8 @@ mod tests {
#[test]
fn next_core_needs_more_when_candidate_header_truncated() {
// A second core sync appears but fewer than CORE_HEADER_MIN_BYTES follow
// it, so its size can't be judged → the access unit can't be closed yet
// (NeedMore → parse() breaks and waits). Build core(512) + a bare 2nd
// sync with only the 4 syncword bytes buffered (< CORE_HEADER_MIN_BYTES
// after it), so the candidate can't be validated.
// it, so its size can't be judged (NeedMore → parse() breaks and waits).
// Build core(512) + a bare 2nd sync with only the 4 syncword bytes.
let mut parser = DtsParser::new();
let mut data = make_dts_core(512);
data.extend_from_slice(&DTS_CORE_SYNC); // 2nd sync, header truncated
@@ -1750,9 +1629,8 @@ mod tests {
#[test]
fn multiple_false_syncs_in_extension_all_skipped() {
// An extension body containing SEVERAL byte sequences that match the core
// syncword but decode to sub-spec sizes must ALL be skipped; the AU is
// closed only at the next real core. Guards the loop in
// An extension body with SEVERAL byte sequences matching the core syncword
// but decoding to sub-spec sizes must ALL be skipped; guards the loop in
// next_core_boundary that advances `from = pos + 4` past each false sync.
let mut parser = DtsParser::new();
let mut ext = make_dts_ext(400);
@@ -1977,10 +1855,8 @@ mod tests {
#[test]
fn drop_preserves_av_sync_no_shift() {
// THE INVARIANT: dropping an undecodable AU must never shift the audio
// that follows. A good/bad/good run in ONE PES — the bad middle core is
// dropped, but the trailing good core must keep the EXACT PTS it would
// have had with no drop (base + 2 frame durations), so the drop is a
// silence gap, not a shift.
// that follows. good/bad/good in ONE PES: the trailing good core keeps
// the EXACT PTS it would have had with no drop — a gap, not a shift.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512); // c1: good
stream.extend_from_slice(&make_bad_dts_core(512)); // c2: undecodable
@@ -2043,10 +1919,8 @@ mod tests {
}
#[test]
// The loop variable is the DOMAIN VALUE being checked (a DTS SFREQ code), not a
// cursor into a collection: it is what the assertion message names and
// what the table is keyed by. `.iter().enumerate()` would rename the
// thing under test to `i` and read worse.
// The loop variable is the DOMAIN VALUE checked (a DTS SFREQ code), not a
// collection cursor: `.iter().enumerate()` would rename it to `i` and read worse.
#[allow(clippy::needless_range_loop)]
fn sr_validity_table_marks_reserved_codes() {
// The core-header sample-rate validity table must have ZERO (reject) at
@@ -2083,11 +1957,9 @@ mod tests {
d[5] = (d[5] & 0x03) | (14u8 << 2);
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmBlocks));
// audio_mode reserved (>= 16): AMODE = byte7 bits3-0 (high 4) + byte8
// bits7-6. Set AMODE high nibble to 0xF → audio_mode = 60, a genuinely
// RESERVED code (16-63) a decoder rejects. (Codes 10-15 are LEGAL
// multichannel layouts and must NOT be dropped — see
// legal_multichannel_amode_is_not_dropped.)
// audio_mode reserved (>= 16): AMODE = byte7 bits3-0 + byte8 bits7-6. Set
// high nibble to 0xF → audio_mode = 60, a genuinely RESERVED code a
// decoder rejects (10-15 are LEGAL, see legal_multichannel_amode_is_not_dropped).
let mut d = good.clone();
d[7] |= 0x0F;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::Amode));
@@ -2112,12 +1984,9 @@ mod tests {
#[test]
fn legal_multichannel_amode_is_not_dropped() {
// ETSI TS 102 114 §5.3.1: AMODE is a 6-bit field with 16 LEGAL
// channel-arrangement codes (0-15); only 16-63 are reserved. ETSI TS 102
// 114 §5.3.1's per-AMODE channel counts cover all 16, confirming codes
// 10-15 are decodable 6/7/8-channel layouts. The decodability gate must
// KEEP them — dropping a spec-legal multichannel core silences audio the
// recover-100% goal must preserve.
// ETSI TS 102 114 §5.3.1: AMODE is 6 bits with 16 LEGAL codes (0-15), only
// 16-63 reserved; codes 10-15 are decodable 6/7/8-channel layouts. The
// gate must KEEP them — dropping a legal multichannel core silences audio.
fn set_amode(core: &mut [u8], amode: u32) {
// audio_mode = (byte7 & 0x0F) << 2 | (byte8 >> 6).
core[7] = (core[7] & 0xF0) | ((amode >> 2) & 0x0F) as u8;
@@ -2156,10 +2025,9 @@ mod tests {
#[test]
fn termination_frame_with_small_deficit_is_kept() {
// ETSI TS 102 114 (a normal frame carries a full 32-sample PCM block):
// a TERMINATION frame (FTYPE=0) may legally carry fewer than 32 deficit
// samples and is fully decodable. It must NOT be dropped — dropping the
// last frame of a stream silences real audio (recover-100% violation).
// ETSI TS 102 114: a normal frame carries a full 32-sample PCM block, but
// a TERMINATION frame (FTYPE=0) may legally carry fewer and is decodable.
// It must NOT be dropped — that would silence the last frame of a stream.
let mut core = make_dts_core(512);
set_ftype_short(&mut core, false, 10); // termination, deficit = 11 (< 32)
assert_eq!(
@@ -2196,11 +2064,9 @@ mod tests {
#[test]
fn reserved_bit_set_is_not_dropped() {
// The bit after RATE is a RESERVED field that both reference decoders
// SKIP (a reserved field per ETSI TS 102 114) — they
// never reject a frame that sets it. Rejecting was a false-drop that
// silenced any real stream whose encoder set the bit. Setting it (byte9
// bit4) on an otherwise-valid core must leave it KEPT.
// The bit after RATE is RESERVED (ETSI TS 102 114); both reference
// decoders SKIP it and never reject on it. Setting it (byte9 bit4) on an
// otherwise-valid core must leave it KEPT.
let mut core = make_dts_core(512);
assert_eq!(core_header_drop_reason(&core), None, "baseline decodable");
core[9] |= 0x10; // set the reserved bit
+6 -15
View File
@@ -68,17 +68,9 @@ impl CodecParser for DvdSubParser {
let mut out = Vec::new();
// A PES carrying a real PTS is the START of a new SPU; continuations of
// an in-progress SPU carry no PTS (the PS demuxer leaves `pts` None when
// the PES has no PTS field — see the module doc). PTS is therefore the
// authoritative SPU-boundary signal, NOT merely `pending.is_some()`.
//
// Append-as-continuation ONLY when this PES has no PTS. When it has a
// PTS but a stale `pending` is still open (a lost continuation, or a
// corrupt/oversized declared SPU_size that real data never reaches),
// force-emit the stuck unit truncated and fall through to start a fresh
// SPU from this PES. Without this, one bad SPU_size would swallow every
// later subtitle until EOF — exactly the damaged-disc case we target.
// PTS present == start of a new SPU (continuations carry none); a PTS
// while `pending` is still open (lost continuation/corrupt SPU_size)
// force-emits it truncated rather than swallowing all later SPUs.
if pes.pts.is_none() {
if self.pending.is_some() {
// Continuation: append, bounded by MAX_SPU_BYTES.
@@ -715,10 +707,9 @@ mod tests {
#[test]
fn no_pts_sized_segment_without_pending_starts_new_spu() {
// A no-PTS segment with no pending but a valid SPU_size (>= 2) and an
// incomplete length begins a fresh pending SPU (the demuxer may have
// dropped the PTS, but the size field is authoritative for boundary).
// declared = 16, only 3 bytes present → held pending, no emit.
// No-PTS segment, no pending, valid SPU_size (>= 2), incomplete length:
// starts a fresh pending SPU (size field is authoritative for boundary
// even if the demuxer dropped the PTS). declared=16, 3 bytes → held.
let mut parser = DvdSubParser::new(None);
let f = parser.parse(&make_pes(vec![0x00, 0x10, 0xAA], None));
assert!(f.is_empty(), "incomplete sized segment held, not emitted");

Some files were not shown because too many files have changed in this diff Show More