Compare commits

...
61 Commits
Author SHA1 Message Date
Matthew Jackson 3f03c6db06 v1.6.11: bump version (freemkv-unlock git-pinned for the tag) 2026-08-25 23:17:07 -07:00
Matthew Jackson 562bb8bee5 docs: neutralize tool-comparison and technique wording (opsec) 2026-08-25 22:57:01 -07:00
Matthew Jackson 7c13ed32b5 fix: align drive-prep unlocker token with unlock rename; neutralize product name 2026-08-25 22:54:38 -07:00
Matthew Jackson 574aa2c506 docs: neutralize retail title references (opsec) 2026-08-25 22:45:57 -07:00
Matthew Jackson 3912762b52 docs: neutralize comment provenance and external references (opsec) 2026-08-25 22:31:19 -07:00
Matthew Jackson 028768e71b docs: 1.6.11 changelog 2026-08-25 22:08:48 -07:00
Matthew Jackson bf4e5ac395 docs: standardize README badges (License, CI, codecov) 2026-08-25 20:26:28 -07:00
Matthew Jackson 11948d513b nav: taint control-transfer decisions only, and track SPRM-laundered GPRMs
Refine the First-Play SPRM-taint abstain to fix two defects:

- Over-abstention: a tainted compare that guards a SetGPRM store (not a
  control transfer) no longer forces abstention. The predicate only
  decides whether the store runs; a following unconditional dispatch is
  still resolvable. Taint now gates only JumpTT/Goto (the control
  transfers the resolver follows); a store under a tainted guard taints
  its destination GPRM (unknown post-state) instead of abstaining.

- Under-abstention: an SPRM laundered through a GPRM (SetGPRM g = SPRM,
  then a later GPRM-only compare on g) previously evaded taint and let
  the resolver commit to the cold-machine arm. Add per-GPRM taint bits:
  a store propagates its source's taint (SPRM or tainted GPRM) and a
  concrete/immediate store clears it; eval marks a compare tainted when
  it reads a tainted GPRM.

Adds red-before-green tests for both plus taint-clear guards.
2026-08-25 20:26:03 -07:00
Matthew Jackson 7d5c0a5c41 Fix round-3 audit findings: VM operand decode, MPLS field offset, log escaping, nav SPRM abstain, tests
1. dvdnav/vmcmd: the register operand of the if_v1/if_v3 compare layouts is the
   LOW byte of the reg-or-immediate field (b[5] / b[7]), not the high byte
   (b[4] / b[6]). Fix both layouts and their comment; add register-mode KATs
   with distinct high/low bytes.

2. mpls: connection_condition is the low nibble of PlayItem byte[10]
   (byte[9] is fully reserved), corroborated by the byte-11 stc_id and
   byte-12 in_time reads. Relocate every fixture's encoded value and update the
   regression test to assert the reserved byte does not leak.

3. labels/deluxe: escape the two disc-derived class-name tracing fields via the
   Debug path, matching the other log sites.

4. disc/dvd: add an integration test that drives scan_dvd_titles with a resolved
   First-Play nav target so the 1-based promotion join is verified end-to-end.

5. bdnav/vm: add unit tests for the SET/SWAP immediate gate — full register
   exchange, and that an immediate-flagged operand is never a write target.

6. dvdnav/nav: honor the documented contract — a First-Play compare that reads a
   system parameter is undecidable at scan time, so eval now taints on SPRM reads
   and the executor abstains rather than committing to the cold-power-on arm.
   GPRM reads never taint.

7. labels/deluxe: narrow four now-internal helpers from pub(crate) to private.
2026-08-25 20:26:03 -07:00
Matthew Jackson 1ede962aec ci(coverage): authenticate Codecov upload with org CODECOV_TOKEN 2026-08-25 19:32:30 -07:00
Matthew Jackson 014ea870a4 ci(coverage): add --ignore-run-fail so instrumentation flake never blocks upload 2026-08-25 19:04:41 -07:00
Matthew Jackson bdd2876d78 ci: add non-required Codecov coverage job, config, and badge
Add an additive coverage job to CI that generates lcov via cargo llvm-cov
and uploads to Codecov (tokenless). Mirrors the test job's pinned 1.94
toolchain and dual freemkv-unlock checkout for the path dep. Informational
only: no gate depends on it. Adds codecov.yml and a README badge.
2026-08-25 18:54:23 -07:00
Matthew Jackson 44de7a8d12 harness: fuzz the Deluxe bytecode decoder directly, not through a dead zip open
Audit round-2 (fix-adversary): the round-1 deluxe sweep fed the zip local-file-
header magic, but ZipArchive::new validates the end-of-central-directory at the
buffer's end — which the byte generators (front magic + noise) can never
synthesize — so the archive open always failed and decode_binding was never
reached: false coverage for the very bytecode-decoder panic path it guarded.
Drive decode_binding_class directly on a parsed .class with an empty master-enum
table instead, so the symbolic stack machine is actually exercised.
2026-08-25 18:45:28 -07:00
Matthew Jackson 552076a943 disc/labels/harness: audit round-1 batch — DoS bound, fuzz coverage, API surface
Continuing the /codeaudit HEAD..v1.6.10 round:

- disc/mod.rs rank_titles: bound the O(n^2 * clips) composite scan to a
  sub-second work budget (both title count and per-title clip count are
  untrusted disc metadata); above it, skip composite classification and degrade
  to the has-video + canonical order. Reorder the per-pair checks so the O(len)
  is_subset runs only after the cheap video/size gates. (regression test)
- bdnav/vm.rs SWAP: honour the imm_op1/imm_op2 store gates so a SWAP to an
  immediate operand does not corrupt a register.
- harness.rs: fuzz the previously-uncovered new parsers — the Deluxe bytecode
  decoder, the Fox dcx.xml scanner, and the combined bdnav index+mobj+VM resolve
  path (labels::{deluxe,fox} and bdnav::vm made pub(crate) for in-crate fuzzing).
- dvdnav: pub(crate) the module + submodules (was pub) to match bdnav and stop
  leaking the DVD IFO/VM internals as public API; drop an unused re-export.
- labels: escape disc-derived strings in diagnostics (Debug not Display) so a
  hostile disc cannot inject terminal control sequences into logs.
2026-08-25 18:33:37 -07:00
Matthew Jackson 3ecbeb7eba disc/bdnav: audit round-1 — VM title/PSR faithfulness + main_title never-panics
From the /codeaudit HEAD..v1.6.10 round (each verified against the libbluray
HDMV VM reference, and design-refuted SOUND on two models):

- bdnav/vm.rs title_obj: resolve First Play via the spec sentinel 0xFFFF (was
  num_titles+1). A JumpTitle 0xFFFF now reaches First Play, and an out-of-range
  JumpTitle(N+1) correctly abstains instead of mis-jumping there.
- bdnav/vm.rs wr: refuse stores to PSR-tagged registers (matches _store_reg),
  so an illegal SET/SWAP to a PSR is a no-op and later compares agree with a
  real player instead of diverging.
- disc/profile.rs main_title(): return Option<&TitleProfile> instead of indexing
  titles[0], which panicked on a disc that scanned to zero titles.

Regression tests added, each shown red-before-green.
2026-08-25 18:16:17 -07:00
Matthew Jackson b405154fd0 ci: run the consumers job at the highest dependent MSRV (1.98)
The MSRV split raised the dependents above libfreemkv's own floor — the
bins (freemkv, autorip, bdemu) are 1.98 and freemkv-keysources is 1.97 —
so the consumers job could no longer build them at libfreemkv's 1.94
toolchain (`rustc 1.94.0 is not supported by ... freemkv-keysources
requires rustc 1.97`), leaving dev red since the MSRV change. Pin that one
job to 1.98, which satisfies every dependent's floor. libfreemkv's own
lib/lint/test jobs stay at its 1.94 MSRV.
2026-08-25 13:49:35 -07:00
Matthew Jackson 8a630c8839 disc: payload-plausibility floor on nav/authoring feature promotion
A nav- or authoring-resolved playlist could promote a video-bearing but
negligible-payload branch above the substantial feature it sits beside:
Spotlight (2015) UHD navigates to a Dolby-Vision seamless-branch playlist
(00001, 0.4 GB across 102 tiny clips) instead of the 91.5 GB single-clip
feature (00002). A player plays the branch; a rip wants the feature.

Require a nav/authoring-promoted title to also carry at least 10% of the
largest video-bearing title's bytes; otherwise selection falls through to
the composite/size ranking, which recovers the real feature. Adds the
Spotlight-shaped regression test.

Verified against the 88-disc hoard: title selection is identical to the
prior release on every disc except the one it fixes (Spider-Man 3 UHD,
00245 wrapper decoy -> 00001 feature).

Also clears three 1.97 clippy lints in the HDMV VM and drops two
reference-implementation filename citations from the bdnav module docs.
2026-08-25 13:44:04 -07:00
Matthew Jackson 6c00090308 labels/fox: wire the extracted feature-playlist id into the selection hint 2026-08-25 13:01:25 -07:00
Matthew Jackson 7f9f5a9d3e Merge branch 'label-fox' into integrate-1.6.11 2026-08-25 13:00:06 -07:00
Matthew Jackson b3c23c19ee Merge branch 'label-deluxe' into integrate-1.6.11 2026-08-25 12:59:54 -07:00
Matthew Jackson f3923cf587 Merge branch 'dvd-nav' into integrate-1.6.11 2026-08-25 12:59:48 -07:00
Matthew Jackson 002912abb7 dvdnav: resolve DVD main feature by following First-Play navigation
Implement the DVD navigation feature resolver (#40): read the disc's own
First-Play program out of VMGI (VIDEO_TS.IFO) and execute it on a bounded
register machine to identify which title a player would land on, then prefer
that title in DVD main-feature selection.

- src/dvdnav/nav.rs: new module.
  * Parses the VMGI First-Play PGC command table (FP_PGC pointer at VMGI+0x84,
    PGC command-table pointer at PGC+0xE4, pre-command list) and TT_SRPT
    (title -> VTS/VTS_TTN, pointer at VMGI+0xC4).
  * Executes the pre-command list with the existing vmcmd decoder over 16 GPRMs
    + SPRMs from a cold machine: SetGPRM (mov/add/sub/mul/div/mod/and/or/xor),
    compare predicates, Goto, Break/Exit, and JumpTT (title dispatch). Menu /
    sub-domain jumps and intra-domain links are treated as undecidable.
  * resolve_from_vmg(&[u8]) is pure and total; resolve_main_title(reader, udf)
    reads the IFO and returns the resolved (title, vtsn, vts_ttn) or None.
  * Every offset/count is bounds-checked; a step budget bounds non-convergence.

- src/disc/dvd.rs: scan_dvd_titles now resolves the nav main title and returns
  its playlist_id alongside the titles; test call sites updated for the tuple.
- src/disc/mod.rs: the DVD nav pick feeds sort_titles_by_main_feature as
  nav_feature, the same seam the BD path uses, guarded to a video-bearing
  title. None -> today's leading-cell heuristic stands (strict improvement).
- src/harness.rs: add resolve_from_vmg to the never-panic sweep.

Tests: 11 nav unit tests (unconditional/conditional/set-then-dispatch convergence,
menu/empty/out-of-range/self-loop/truncation/overflow abstention) built to the
IFO spec using the real First-Play command bytes from the existing decoder KATs
(unconditional JumpTT 1; menu entry). Full suite green on 1.94.
2026-08-25 12:55:34 -07:00
Matthew Jackson b5fd193d54 disc: add normalized DiscProfile output surface
Add a thin, format-agnostic typed VIEW over the scanned Disc model so
downstream reads `profile.titles[i].subtitles[j].forced` (and analogous
audio/video flags) for any disc with no per-format conditionals.

New src/disc/profile.rs:
- DiscProfile / TitleProfile / VideoTrack / AudioTrack / SubtitleTrack
  (serde Serialize + Deserialize), re-exported from the disc module.
- DiscProfile::from_disc / From<&Disc>, Disc::profile(), and accessors
  (DiscProfile::main_title, TitleProfile::video/audio/subtitles).
- Splits each title's Stream enum into three typed vectors; flattens
  normalized fields (language default "und", forced/sdh from the subtitle
  qualifier, commentary/descriptive from the audio purpose, label -> name).
- Hoists derived flags so downstream never recomputes: main_title (the
  pre-sorted main feature at index 0), is_main, and the default
  audio/video track (first non-secondary, keep only the first), subtitle
  default always false.

Make the profile types the single serialization source of truth: the
json:// sink now serializes TitleProfile instead of hand-building JSON,
so json:// and any downstream share one schema. Stream arrays are split
by kind and qualifier/purpose decomposed into booleans; json://-only
extras (clips, full chapter_marks list) are spliced back on. meta_sink
tests reconciled to the new schema.

Tests: profile normalization across BdTs + MpegPs titles, typed-vector
split, populated fields, hoisted main/is_main/default flags,
qualifier/purpose mapping, and a serde round-trip.
2026-08-25 12:43:24 -07:00
Matthew Jackson 49011d577f labels/deluxe: decode real per-stream labels, emit High (Universal + others)
The Deluxe parser detected com/bydeluxe discs but emitted nothing, so
mpls_universal won at Low. Grounding the decoder on real Universal discs
(Fast Five / F9 / Bourne, studio="uni") surfaced three defects that
kept the full Phase A-D pipeline from ever resolving a stream on any
real disc, all fixed here:

1. Master enums were keyed by the ZIP ENTRY name ("pd.class") but the
   binding class references enum constants by JVM INTERNAL name
   (getstatic pd.<field>), so Phase C's getstatic count never matched
   the identified enums and found zero binding classes. Key master enums
   by this_class_name instead.
2. MasterEnumTable resolved getstatic operands by enum VALUE string
   ("English") but the operand is the obfuscated static-FIELD name
   ("a"), so every language/purpose ordinal came back unresolved.
   Capture the ordered putstatic field names (clinit_enum_field_names)
   and key the resolver on them.
3. The Language-enum fingerprint demanded 70+-4 values; Universal's
   enum has 65, so Phase A rejected it outright. Give each fingerprint
   its own count window (Language widened to span 65 and 70); the
   ordered 4+-string prefix stays the real discriminator.

Also: model anewarray/newarray so array construction keeps the symbolic
stack aligned, and skip constructors with an array parameter so the
Universal title-wrapper object (oq.<init>(...,[Lnp;,[Lwb;,...)) is not
mistaken for a subtitle binding. Recover SDH/RNIB from the language
display name (Universal encodes it as a distinct Language value, not a
Purpose). Add config.xml <TitleConfig studio="..."> detection. Raise
confidence to High when the binding table decodes >=1 stream; the
mpls_universal floor still back-fills un-named STN slots.

Fixtures: verbatim pd/lp/tl .class files from FastFive.iso decode to
English/French/Spanish + English Commentary (Dolby Digital) audio and
per-title eng/spa/fra subtitles. Validated end to end: labels-analyze on
FastFive/F9/Bourne now reports parser=deluxe High with real per-stream
languages, names, purposes and codecs instead of mpls_universal Low.
2026-08-25 12:25:38 -07:00
Matthew Jackson f8e39abe54 labels: harden class_reader/jar as first-class .class format readers
Treat Java bytecode as a documented format we READ (never execute), a
peer of mpls/clpi. The class reader's bounds checking was already sound
(typed Reader with checked u8/u16/u32/u64 + slice, saturating switch
arithmetic, iterator halt-on-truncation); this adds the untrusted-input
entry points the never-panic harness was missing and the malformed-input
regression tests the format warranted.

- harness: add class_reader (magic CAFEBABE) and jar (zip PK header)
  sweep targets. The class_reader target also drives the accessors a
  label parser exercises (this_class_name, member Code walk, cp_index,
  member_ref, load_constant_display) so their indexing is fuzzed, not
  just the structural parse. The jar target opens the sweep bytes as a
  zip and walks every .class through for_each_class exactly as deluxe/dbp
  do.
- class_reader tests: oversized constant_pool_count and interfaces_count
  reject with UnexpectedEof (u16 counts keep with_capacity bounded);
  out-of-range cp indices return None from every accessor; a Code
  attribute whose declared code_length runs past its body is rejected.
2026-08-25 12:25:19 -07:00
Matthew Jackson 65d719ee5d disc: wire bdnav as the top nav-feature tier in main-feature selection
scan_with runs the bdnav VM for BDMV discs and feeds the video-gated result
as the new top MAIN_FEATURE_ORDER key (nav-feature > authoring-feature >
standalone > has-video). A nav result is only honoured for a video-bearing
title, so a mis-resolve can never reopen the decoy hole; when the VM abstains
(BD-J, non-convergence) the structural/heuristic order decides exactly as
before. Adds nav selection tests and updates the diag decision reason.
2026-08-25 12:19:16 -07:00
Matthew Jackson c3bdbdcfc9 bdnav: faithful HDMV navigation VM (index.bdmv + MovieObject.bdmv)
Add src/bdnav: a libbluray-faithful, bounded, never-panic HDMV navigation VM
that runs First-Play with correct power-on PSR/GPR state to resolve the
playlist the disc itself plays as the feature. One generic VM handles the
obfuscated Sony dispatcher and every other HDMV disc with no vendor
special-casing; it abstains (None) on the BD-J boundary, malformed data, or
non-convergence. index/mobj parsers added to the never-panic harness.
2026-08-25 12:19:16 -07:00
Matthew Jackson 3f4e69f7b8 labels: add Fox dcx.xml vendor parser
Older Fox discs ship a loose plain-XML manifest at
/BDMV/JAR/<id>/dcx.xml (root <dcx>) that names every feature stream's
language, editorial purpose and forced/SDH state in attributes — fully
parseable and previously unowned (2 Fox discs matched no parser, 11 were
mis-owned by the deluxe stub).

New src/labels/fox.rs:
  - detect(): loose dcx.xml under /BDMV/JAR/*/ (cheap dir walk), falling
    back to a com/foxbd/ jar central-directory prefix so newer Fox discs
    are attributed to Fox even though their bytecode form isn't decoded
    yet (Phase 2, sketched in-file).
  - parse(): scopes extraction to one <playlist name="feature"> element
    (the richest, to avoid merging regional 00800/00801 STN tables),
    reusing labels/xml.rs. audio type=rnib -> Descriptive, comment* ->
    Commentary; subtitle form=sdh -> Sdh, type=embed -> Forced; nested
    id="NN" -> 1-based per-type STN slot. High confidence.
  - Feature playlist id recorded via tracing::debug for now; dev's
    ParseResult has no feature_playlist field (that FeaturePlaylistHint
    slot lives on selection-hardening). An in-code note documents wiring
    it at reconciliation, matching paramount::parse.

Registered ahead of mpls_universal, alongside the High-confidence vendor
parsers; registry-order lock test updated.

Tests embed the real Life of Pi dcx.xml (/BDMV/JAR/05001), asserting the
00800 feature selection + id, all 11 audio labels (incl. the eng rnib
descriptive track) and 11 subtitle labels (sdh/forced/none), the
no-cross-playlist-merge scope rule, plus negative-detect cases.
2026-08-25 12:09:37 -07:00
Matthew Jackson 89b03610e7 disc: demote play-all/wrapper composites in main-feature selection
The has-video gate alone fixed Spider-Man 3 UHD only by accident: the
00245 decoy = [bumper][00001 feature clip][outro], and streams come from
PlayItem 0 (the bumper) only, so it reported v=0. That is fragile — a
video-bearing composite [feature][outro] still beat the feature on size
(audit R2), and a real feature with a non-video first PlayItem would be
wrongly disqualified (R6).

Add structural clip-set composite detection: a title whose clip set
properly contains another substantial video title's whole clip set is a
play-all/wrapper composite and is demoted below the standalone title it
wraps — independent of stream counts, physical size, and disc capacity
(which is 0/unknown when READ CAPACITY fails, disabling the oversize
gate). Replace the has-video disqualification with the more-permissive
has_probable_video (real video, or size+duration+clips consistent with
video) so a bumper-lead feature is never disqualified while streamless
menus still are. Corroborate the authoring hint by duration (a hint at a
short branch/trailer can't override the feature) and require real video
streams for it. Precedence: authoring -> standalone-over-composite ->
has-video -> canonical (with a reserved top slot for the future BD-nav
key). Unify the session.rs key-sampling/verdict pickers and
image_crack_extents on has_probable_video.

Tests: video-bearing composite (T2), oversize composite with unknown
capacity (T3), short-branch hint not honoured (T4), bumper-lead feature
not disqualified (T6), all-streamless deterministic + single-source pick
(T7), equal-clip twins not composite, video-only title. T2/T3/T6 fail
under the prior has-video-only logic.
2026-08-25 11:54:04 -07:00
Matthew Jackson a7cbb3afcd disc: harden main-feature selection against streamless decoys
Main-feature selection ranked purely on physical size -> duration ->
audio (canonical_title_order), with no requirement that the chosen title
carry a decodable video stream. A playlist-obfuscation decoy can inflate
a streamless playlist past the real feature on size AND duration (Spider-
Man 3 UHD: 00245.mpls, 78.4GB/8350s/v=0 beat 00001.mpls, 76.5GB/8038s/
v=1), so the mux ran a title with zero streams -> zero frames -> E6008.

- Add Disc::main_feature_order layering two gates atop canonical_title_order:
  authoring-feature (a title the disc's menu authoring designates AND that
  has video) wins outright; then has-video demotes any streamless title
  below every title that has video. scan_with re-sorts with it after labels
  run so titles[0] is the selected feature. New MAIN_FEATURE_ORDER_KEYS +
  DiscTitle::has_video().
- Surface the authoring feature-playlist id: labels::FeaturePlaylistHint,
  captured from Paramount playlists.xml <playlist name="Feature" id=..>,
  threaded through ParseResult/extract; labels::apply now returns it.
- Unify pickers so key-sampling and the structural verdict (session.rs) and
  image_crack_extents prefer the has-video feature, not a streamless decoy.
- diag: decision row logs video= count + main_feature_order reason and a
  tag=decision.demoted line per no-video title (closes the field-diagnosis
  gap this incident exposed).

Tests: main_feature_order_demotes_streamless_decoy (pre-gate vs post-gate
contrast), _honours_authoring_hint, authoring_hint_never_promotes_a_
streamless_title, feature_playlist_hint_matches_on_id_or_filename; diag
reason test updated. QA green on 1.94 (fmt/clippy -D warnings/test).
2026-08-25 11:23:44 -07:00
Matthew Jackson f75766b4d5 chore: lower MSRV to 1.94, extend disc provenance tests, tidy comments
Lower the crate MSRV from 1.97 to 1.94 -- the lowest toolchain on which
`cargo build`, `cargo clippy --all-targets -D warnings`, and
`cargo test --tests` all pass clean. Dependencies floor the build at 1.90
(freemkv-unlock 1.90, zip 1.88), but clippy is only warning-clean from
1.94 (older clippy has a `duplicated_attributes` false positive on
`#![cfg(test)]` and flags a now-documented `let_and_return`). Update
`rust-version` and every `dtolnay/rust-toolchain` CI pin to 1.94.0, and
add a CHANGELOG Unreleased entry.

Extend the byte-provenance coverage added in f7d38ca (currently one
TS-path test) with two PS-path tests driving the real `Mpeg2Parser`:
- `ps_stream_stamps_source_provenance`: the PS path forwards `ps.source`
  onto `PesPacket`, so demuxed frames carry `Some(SourcePos)`.
- `ps_stream_carries_feed_base_across_reads`: the cumulative `fed_bytes`
  base carries ACROSS read buffers -- a frame decoded from the second
  sector is stamped `source.byte >= 2048`, not reset per read.

Also record the f7d38ca provenance fix in the CHANGELOG, inline a
`let_and_return` test binding, and trim a redundant sentence from
`PsDemuxer::feed_at`'s doc comment.
2026-08-25 10:35:47 -07:00
dependabot[bot] 1c313fe882 Bump actions/checkout from 5 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-24 08:52:57 -07:00
Pascal Harprecht f7d38caf0b mux/disc: pass feed base offset to demuxer to stamp byte provenance
On direct disc reads (`disc://...`), `DiscStream` called `demuxer.feed(&self.read_buf[..bytes])`
instead of `feed_at(buf_base, ...)`. Without a running feed base offset, demuxed `PesPacket`s
had `source = None`, stripping byte provenance from downstream `PesFrame`s.

Under a `SeamPlan`, when `src_byte` is `None`, `SeamPlan::place` falls back to a timestamp
heuristic across clips. On multi-clip playlists where a subsequent clip starts at the same
base timestamp as the initial clip (`in_time = 600s`), B-frame PTS reorder dips (where
`raw_ns < last` and `|raw_ns - next_in| <= 250ms`) falsely triggered a `stepped_back` clip
transition on the first reordered frame. Because the subsequent clip's span was short (e.g. 1s),
all subsequent video frames across the rest of the title were rejected and dropped as outside
clip marks. Audio tracks (which have monotonic PTS without reordering) were unaffected.

Track `fed_bytes` in `DiscStream` and route through `feed_at` for both TS and PS demuxers,
forwarding `ps.source` to `PesPacket` on the PS path.

Test: `mux::disc::tests::stream_surface_tests::disc_stream_stamps_source_provenance`
2026-08-24 08:52:19 -07:00
Matthew Jackson e98e98c8fd TrueHD: resync forward to next major sync on a source discontinuity
MLP/TrueHD carries predictor + restart state across access units, so after a source transport-stream discontinuity (continuity-counter gap) the parser was splicing post-gap audio mid-stream, decoding against stale state and producing a decoder-choking seam (a burst of "restart header sync incorrect" / "Invalid blocksize" errors that could flag the whole track corrupt).

Honor the source's own gap signal: drop forward to the next CRC-valid major-sync AU (a clean re-init point) before resuming, guarded so it never arms before a validated baseline exists (no stream-head whole-track drop). Mirrors the video keyframe resync and the ResyncGate design. Corrects the "audio has no inter-frame refs" comments in truehd/dts/resync. Adds a regression test.
2026-08-23 16:30:29 -07:00
Matthew Jackson 7c085db133 restore freemkv-unlock path dep for local dev (post-v1.6.9) 2026-08-22 08:34:31 -07:00
Matthew Jackson 587e26353c v1.6.9: bump version (freemkv-unlock git-pinned for the tag) 2026-08-22 08:34:26 -07:00
Matthew Jackson 3eb3e649b1 changelog: 1.6.9 unified-release entry 2026-08-22 08:24:55 -07:00
Matthew Jackson b01505983c restore freemkv-unlock path dep for local dev (post-v1.6.8) 2026-08-21 12:54:53 -07:00
Matthew Jackson 87bc07dad8 v1.6.8: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 12:54:48 -07:00
Matthew Jackson 743c5830b6 docs: 1.6.8 changelog (unified release) 2026-08-21 12:47:34 -07:00
Matthew Jackson 80c8e4ca24 ci: require a CHANGELOG entry for the crate version on main
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
2026-08-21 11:23:45 -07:00
Matthew Jackson f49e79c198 restore freemkv-unlock path dep for local dev (post-v1.6.7) 2026-08-21 09:31:40 -07:00
Matthew Jackson 231cbf3d76 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 21c7b12e2b docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 799039df21 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
Matthew Jackson 497b7726dc v1.6.6: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 20:03:44 -07:00
Matthew Jackson 94b549a9d8 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson fc2d35a19c v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew Jackson 3c51190c32 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson dfe0390445 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 8678a2bf94 drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson d953b46023 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 443ac6bcb1 Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 794393f24e CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson c6e2614c4c Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson b02c402dae CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 128e3bb120 Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson dc100f1c80 Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 286b5d4527 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson d9f3f82c65 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson e30209091e restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
72 changed files with 12633 additions and 1133 deletions
+15
View File
@@ -46,3 +46,18 @@ updates:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+72 -5
View File
@@ -11,6 +11,32 @@ on:
pull_request:
jobs:
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
# forgot its changelog entry must fail here rather than ship a blank release.
# Hard-fails on main (and PRs targeting main); warns elsewhere so an
# in-progress dev/qa branch whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
set -euo pipefail
version=$(grep -m1 '^version = ' Cargo.toml | sed -E 's/.*"([^"]+)".*/\1/')
if grep -qE "^## \[${version}\]" CHANGELOG.md; then
echo "CHANGELOG.md has an entry for ${version}."
exit 0
fi
msg="CHANGELOG.md has no '## [${version}]' entry"
if [ "${GITHUB_REF_NAME:-}" = "main" ] || [ "${GITHUB_BASE_REF:-}" = "main" ]; then
echo "::error::${msg} — main requires a changelog entry for the released version."
exit 1
fi
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
runs-on: ubuntu-latest
steps:
@@ -31,7 +57,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
@@ -69,7 +95,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -109,7 +135,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -149,7 +175,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -204,7 +230,10 @@ jobs:
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
# The consumers span a higher MSRV than libfreemkv itself (the bins are
# 1.98, keysources 1.97), so build them with a toolchain that satisfies the
# HIGHEST dependent floor — libfreemkv's own 1.94 cannot compile them.
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
@@ -227,3 +256,41 @@ jobs:
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
# ── Coverage (non-required, additive) ───────────────────────────────────────
#
# Reports line coverage to Codecov. Deliberately NOT wired into any release
# gate: it is informational only (see codecov.yml) and no other job `needs:`
# it. Mirrors the `test` job's layout exactly — same pinned 1.94 toolchain,
# same dual checkout so the `../freemkv-unlock` path dep resolves, same cache
# workspace and working-directory. cargo-llvm-cov comes as a prebuilt binary
# from taiki-e/install-action, so it is toolchain-independent.
#
# No `--locked`: like the fmt/clippy/test jobs above, this crate's Cargo.lock
# is gitignored, so there is nothing to lock against on a fresh runner.
coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
slug: freemkv/libfreemkv
fail_ci_if_error: false
+3 -3
View File
@@ -79,7 +79,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -100,7 +100,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
@@ -125,7 +125,7 @@ jobs:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
+2 -2
View File
@@ -29,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# gated by the local precommit (same Rust 1.94, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -38,7 +38,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@1.94.0
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
+212
View File
@@ -1,5 +1,217 @@
# Changelog
## [Unreleased]
## [1.6.11] — 2026-08-26
### Added
- Navigation-driven main-feature selection: selection now follows the disc's
own on-disc menu/navigation logic — a Blu-ray HDMV navigation VM and a DVD
First-Play resolver — to reach the main feature, instead of guessing by
title size alone.
- Clip-set composite/decoy detection, so a title assembled from a composite
or decoy clip set is recognized as such during selection.
- A feature-payload floor, giving main-feature selection a minimum-size
sanity check.
- Normalized `DiscProfile` output.
- Vendor navigation-label parsing.
- Substantially expanded unit-test coverage across the navigation and
clip-selection paths.
### Fixed
- Main-feature selection no longer picks an obfuscated decoy playlist over
the real feature.
- Two rounds of code-audit hardening: navigation VM operand/field-offset
conformance, navigation now abstains rather than guessing on an
undecidable system-parameter branch, panic-safety fixes, log-escaping
fixes, and expanded fuzz/test coverage.
- Direct disc reads (`disc://…`): `DiscStream` now threads a cumulative feed
base offset into the demuxer (`feed_at`) for both the TS and PS paths and
forwards it onto each `PesPacket`, so demuxed frames carry byte-exact source
provenance (`SourcePos`). Without it, `SeamPlan::place` fell back to a PTS
heuristic that a B-frame PTS reorder dip could misread as a `stepped_back`
clip transition, dropping all subsequent video frames in multi-clip playlists.
### Changed
- Lowered the minimum supported Rust version (MSRV) from 1.97 to 1.94. This is
the lowest toolchain on which `cargo build`, `cargo clippy --all-targets
-D warnings`, and `cargo test --tests` all pass clean; CI toolchain pins were
moved to match. (The crate's dependencies floor `cargo build` at 1.90, but
clippy is only warning-clean from 1.94.)
## [1.6.10] — 2026-08-23
### Fixed
- TrueHD/MLP audio: on a source transport-stream discontinuity (lost or gapped
packets, signalled by a continuity-counter break), the parser now drops forward
to the next major-sync access unit before resuming, instead of splicing the
post-gap audio mid-stream. MLP carries predictor + restart state across access
units, so resuming mid-stream decoded against stale state and produced a
decoder-choking seam — streaming decoders emitted a burst of "restart header
sync incorrect" / "Invalid blocksize" errors and could flag the whole track
corrupt. This mirrors the video path (resync to the next keyframe after a gap)
and how the independent-frame audio codecs (AC-3, DTS, FLAC, AAC) already
recover; the fix is guarded so it never arms before a validated major-sync
baseline exists (no whole-track drop at stream head). The gapped instant itself
is in the source and cannot be recovered, but the muxed track is now
decoder-clean.
## [1.6.9] — 2026-08-22
### Changed
- Version aligned to 1.6.9 for the unified release. No functional changes to
this crate; the release was driven by autorip (automatic per-episode TV
ripping — each episode named `S{NN}E{MM}`, with TMDB runtime-aligned episode
numbering across multi-disc seasons — a Manual Rename option, and a unified
per-disc staging state file — see the autorip 1.6.9 notes).
## [1.6.8] — 2026-08-21
### Changed
- Version aligned to 1.6.8 for the unified release. No functional changes to
this crate; the release was driven by autorip (webhooks now fire per pipeline
stage — Rip / Mux / Move — with the Rip hook firing the moment the drive is
free again, plus a Ripper-tab activity-banner fix so it also shows during
moves — see the autorip 1.6.8 notes).
## [1.6.7] — 2026-08-21
### Changed
- Version aligned to 1.6.7 for the unified release. No functional changes to
this crate; the release was driven by autorip (per-webhook event selection,
a progress bar per moved artifact, and move-queue / webhook-error fixes —
see the autorip 1.6.7 notes).
## [1.6.6] — 2026-08-20
### Changed
- Version aligned to 1.6.6 for the unified release. No functional changes
to this crate; the release was driven by autorip (webhooks may now target
private/LAN addresses — see the autorip 1.6.6 notes).
## [1.6.5] — 2026-08-20
### Fixed
- **On some Blu-ray/UHD titles the picture stopped short of the declared
end.** When a title is stitched across a clip seam, freemkv drops video
after the join until it sees a fresh keyframe it can restart decoding
from. It only recognised one kind of keyframe (an IDR frame), but a
Blu-ray's final segment can open on a different kind of self-contained
frame instead — so on those discs the restart point was never
recognised and every remaining frame to the end of the film was
dropped, losing up to a group-of-pictures-plus tail of picture. That
kind of frame is now recognised as a valid restart point, matching how
the HEVC path already worked, so the tail is kept. HEVC and DVD titles
were never affected.
- **A split HD-DVD feature could be exported as half the film while still
claiming the full running time.** A feature stored across parts (for
example `FEATURE_1.EVO` + `FEATURE_2.EVO`) is composed back into one
title. If a part resolved with no usable data — a zero-length file, or
an extent map that yielded nothing — that part was neither used nor
flagged, so the feature was quietly composed from the surviving parts
alone, still advertising the whole runtime, and exited success with no
log. The missing part is now marked unusable and logged under its own
code (E6019), so half a movie can no longer be presented as a whole one.
- **A Blu-ray title with an unreadable clip could be exported short while
still reporting the full running time.** A scratched or malformed
clip-info (`.clpi`) sector, or a clip whose extents could not be
resolved (a bad sector, a broken allocation chain, an embedded-data
file), was silently skipped — but the title's duration had already been
counted from the playlist, so it shipped short of the runtime it claimed
at exit success, with nothing logged. Previously only one narrow failure
kind counted; now every unresolvable clip drops the title and warns with
the read's own error code. A truly absent optional file (such as the 3D
`.ssif` on a 2D disc) is still treated as benign.
- **Pressing Stop during a disc scan could leave the disc looking like it
holds fewer titles than it does — or none at all.** Once cancelled,
every remaining drive command fails, but the Blu-ray and DVD title
enumerators treated those failures as ordinary skips: the Blu-ray scan
returned a truncated title list and the DVD scan returned zero titles,
both at exit success, so a cancelled scan was indistinguishable from a
disc that simply held that many titles. A cancel is now propagated as an
error out of every enumerator and at every read site, so it can never be
cached, displayed, or ripped from as if it were the real disc. HD-DVD
already behaved this way and is the model the others now follow.
- **A decrypted HD-DVD could be refused with a DVD copy-protection error
(E7023).** HD-DVD and DVD share the same MPEG program-stream container,
and the scramble detector mistook an HD-DVD navigation packet for a
CSS-scrambled sector — so a good, already-decrypted HD-DVD was reported
as carrying an unrecoverable CSS key and hard-failed, sending anyone
triaging it hunting for a missing DVD key on a disc that never had one.
The detector now excludes the structural packet types that CSS never
scrambles, identified from a field that is readable even on ciphertext,
so a decrypted HD-DVD scans clean. The DVD CSS crack is unchanged and a
genuinely scrambled, uncrackable DVD still hard-fails as before, so
ciphertext can never be muxed as plaintext.
- **A file with an allocated-but-never-written region could splice
undefined sectors into the rip.** The UDF reader treated an
allocated-but-unrecorded extent as ordinary content and read whatever
happened to be on those sectors into the output, and it decoded
embedded-data files (which store their content inline, not as a sector
map) as if their bytes were an extent list, pointing the reader at
unrelated sectors. Such extents are now refused when they actually
occupy space, and embedded files are handled as their own case; a
legitimately zero-length file still reads as empty rather than dropping
its title.
- **Audio and subtitle tracks in many languages were all labelled
"undefined."** The language mapping recognised only a handful of codes
and collapsed fifteen others to `und`. Every ISO 639-1 language code is
now mapped, so those tracks carry their real language. The DVD subtitle
colour palette, which was written in the wrong order, is also corrected.
- **If the disc read-ahead thread died mid-rip, a whole title could be
fabricated and reported complete.** A prefetch producer that terminated
returned an end-of-data signal that the reader legitimately read as a
short read and zero-filled, so a failed read could be papered over with
zeros and the pass still reported as successful. It now reports a
distinct source-terminated error instead. Dead-bus drive faults are
likewise classified rather than flattened, so the wedged-drive recovery
path can see them, and the drive now responds to Stop during spin-up and
spin-down instead of staying deaf for up to ~30 seconds.
- **A holed extraction could climb to a clean 100% on the live progress
channel.** The progress feed hardcoded its unreadable-byte count to zero
and counted every zero-filled hole as good data, so a progress-only
consumer saw a damaged extraction finish spotless — even though the
authoritative result was already truthful. The real good/unreadable
split is now threaded through the live channel. Separately, an
unreadable HD-DVD authored clip order is now logged with its error code
instead of being silently discarded before falling back to the
per-clip heuristic.
### Security
- **Bounded several unbounded amplification axes in the HD-DVD and Blu-ray
scanners.** A crafted disc could drive the playlist nesting depth, the
title count, and the clips and chapters per title without limit, and
could force a repeated clip-name fallback probe — each of which alone
left the worst-case scan unbounded. All now carry positional caps (512
titles and clips, roughly ten times any retail disc) and the fallback
probe is memoized. No effect on a well-formed disc.
### Changed
- **How freemkv picks the per-platform drive code was consolidated, with
no change to reading a disc.** The drive layer chose its
operating-system-specific module separately inside each entry point; it
now selects that module once, so a new entry point cannot silently
forget a platform. Purely internal.
## [1.6.4] — 2026-08-15
### Fixed
+3 -3
View File
@@ -1,8 +1,8 @@
[package]
name = "libfreemkv"
version = "1.6.4"
version = "1.6.11"
edition = "2024"
rust-version = "1.97"
rust-version = "1.94"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@@ -26,7 +26,7 @@ aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.4" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.11" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
+2
View File
@@ -1,4 +1,6 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/libfreemkv/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/libfreemkv)
# libfreemkv
+18
View File
@@ -0,0 +1,18 @@
# Coverage is informational only: it never fails CI or blocks a merge/release.
# The `coverage` job in .github/workflows/ci.yml is non-required and no gate
# depends on it.
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
comment: false
ignore:
- "tests"
- "examples"
- "benches"
+5 -5
View File
@@ -258,7 +258,7 @@ pub fn ts_packet_total(unit: &[u8]) -> usize {
/// the freebie `is_clean_ts` skips at packet 0); packs 1 and 2 (offsets 2048 and
/// 4096) are in the encrypted region, so a wrong key garbles them and this returns
/// false (64 bits of discrimination). Validated against real decrypted HD-DVD
/// `.evo` (ANCHORMAN / SHAUN_OF_THE_DEAD): pack starts are exactly 2048-aligned,
/// `.evo` (two real retail titles): pack starts are exactly 2048-aligned,
/// three per aligned unit, at offsets 0 / 2048 / 4096.
///
/// UNVERIFIED-HDDVD-DECRYPT (2 of 2): the pack STRUCTURE here is confirmed on
@@ -799,7 +799,7 @@ mod tests {
// ── Padding-aware IsDecryptable (fragment-tail recovery) ───────────────
//
// A content fragment can end mid-unit, with the disc zero-padding the rest
// of the aligned unit to the next fragment (proven on Dunkirk: ~11 real
// of the aligned unit to the next fragment (proven on a real disc: ~11 real
// video packets + source-zero pad). `decrypt_unit` must accept such a unit
// — its real packets decrypt; the source-zero tail is padding, not content
// — while still REJECTING a unit whose undecryptable tail is non-zero (a
@@ -833,7 +833,7 @@ mod tests {
#[test]
fn fragment_tail_with_source_zero_pad_is_decryptable() {
// 11 real content packets, then source-zero padding (the Dunkirk shape).
// 11 real content packets, then source-zero padding (a real-disc fragment shape).
let key = [0x5Au8; 16];
let mut unit = tail_filled_unit(&key, 11, 0x00);
decrypt_unit(&mut unit, &key);
@@ -1000,7 +1000,7 @@ mod tests {
use crate::disc::ContentFormat;
// Clean HD-DVD `.evo` Program-Stream unit: pack_start_code `00 00 01 BA`
// at each 2048-byte pack boundary (0/2048/4096) — the layout validated
// against real decrypted ANCHORMAN / SHAUN_OF_THE_DEAD `.evo`. Offset 0 is
// against real decrypted retail HD-DVD `.evo`. Offset 0 is
// the clear-seed freebie; the packs at 2048/4096 are encrypted and are
// what actually discriminate a key.
let mut ps = vec![0u8; ALIGNED_UNIT_LEN];
@@ -1211,7 +1211,7 @@ mod tests {
/// 2048-byte sector, and each call built its own AES-128 key schedule, so a
/// 6144-byte aligned unit performed THREE key expansions under the same
/// loop-invariant `read_data_key`. On a 90 GB UHD read on a stock (non-
/// LibreDrive) drive — ~14.6 million aligned units — that is ~29 million
/// firmware-unlocked) drive — ~14.6 million aligned units — that is ~29 million
/// redundant expansions on the per-unit decrypt hot path, for a key that is
/// constant for the whole disc. The counter is incremented inside
/// `crypto::new_cipher`, the single construction site.
+1 -1
View File
@@ -228,7 +228,7 @@ const VTKF_AV_FLG: u8 = 0x80;
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
/// (real multi-CPS-unit HD-DVD titles) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
+149
View File
@@ -0,0 +1,149 @@
//! Parse `/BDMV/index.bdmv` — the BD navigation index (First Play, Top Menu,
//! and the title table), per its documented binary layout. This is read as a
//! documented binary format, never executed: every field is bounds-checked and
//! any malformed input yields `None` (the nav resolver then abstains).
//!
//! Layout: `"INDX"` + version(4) + `indexes_start`(u32 @8) + … ; at
//! `indexes_start`: `index_len`(u32), First-Play object(12), Top-Menu object(12),
//! `num_titles`(u16), then `num_titles` title objects(12 each). Every object's
//! `object_type` is the top two bits of its first byte; for an HDMV object the
//! `id_ref` into `MovieObject.bdmv` is a big-endian u16 at object offset 6.
/// One playback/title object in `index.bdmv`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum PlaybackObj {
/// HDMV title. `id_ref` indexes `MovieObject.bdmv`; `0xffff` means "no
/// object".
Hdmv { id_ref: u16 },
/// BD-J title (a Java Xlet chooses what plays — the nav VM cannot resolve
/// it and must abstain).
BdJ,
/// Unrecognised object type.
Unknown,
}
/// The parsed index: the two entry objects plus the title table (title numbers
/// `1..=titles.len()` map to `titles[i-1]`; title 0 is the Top Menu).
#[derive(Debug, Clone)]
pub(crate) struct Index {
pub first_play: PlaybackObj,
pub top_menu: PlaybackObj,
pub titles: Vec<PlaybackObj>,
}
const OBJ_LEN: usize = 12;
/// Sanity cap on the title count (real discs have well under this).
const MAX_TITLES: usize = 4096;
fn be_u16(d: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*d.get(o)?, *d.get(o + 1)?]))
}
fn be_u32(d: &[u8], o: usize) -> Option<u32> {
Some(u32::from_be_bytes([
*d.get(o)?,
*d.get(o + 1)?,
*d.get(o + 2)?,
*d.get(o + 3)?,
]))
}
/// Parse one 12-byte object starting at `o`. `object_type` is the top two bits
/// of byte 0; the object body starts at byte 4, and an HDMV `id_ref` is the
/// big-endian u16 at body offset +2 (object offset +6).
fn parse_obj(d: &[u8], o: usize) -> Option<PlaybackObj> {
let b0 = *d.get(o)?;
d.get(o + OBJ_LEN - 1)?; // require the whole 12-byte record
Some(match (b0 >> 6) & 0x3 {
1 => PlaybackObj::Hdmv {
id_ref: be_u16(d, o + 6)?,
},
2 => PlaybackObj::BdJ,
_ => PlaybackObj::Unknown,
})
}
/// Parse `index.bdmv`. Returns `None` on any structural problem.
pub(crate) fn parse(d: &[u8]) -> Option<Index> {
if d.get(0..4)? != b"INDX" {
return None;
}
// version at 4..8 ("0100"/"0200"/"0300") is not load-bearing for resolution.
let indexes_start = be_u32(d, 8)? as usize;
// At indexes_start: u32 index_len, First-Play(12), Top-Menu(12), u16 titles.
let mut o = indexes_start.checked_add(4)?;
let first_play = parse_obj(d, o)?;
o = o.checked_add(OBJ_LEN)?;
let top_menu = parse_obj(d, o)?;
o = o.checked_add(OBJ_LEN)?;
let num_titles = be_u16(d, o)? as usize;
o = o.checked_add(2)?;
if num_titles == 0 || num_titles > MAX_TITLES {
return None;
}
let span = num_titles.checked_mul(OBJ_LEN)?;
if o.checked_add(span)? > d.len() {
return None;
}
let mut titles = Vec::with_capacity(num_titles);
for i in 0..num_titles {
titles.push(parse_obj(d, o + i * OBJ_LEN)?);
}
Some(Index {
first_play,
top_menu,
titles,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a 12-byte object record: `object_type` in the top two bits of
/// byte 0, HDMV `id_ref` big-endian at offset 6.
fn hdmv_obj(id_ref: u16) -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = 1 << 6;
b[6..8].copy_from_slice(&id_ref.to_be_bytes());
b
}
fn bdj_obj() -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = 2 << 6;
b
}
fn build(first: [u8; 12], top: [u8; 12], titles: &[[u8; 12]]) -> Vec<u8> {
let indexes_start = 48u32;
let mut d = vec![0u8; indexes_start as usize];
d[0..4].copy_from_slice(b"INDX");
d[4..8].copy_from_slice(b"0300");
d[8..12].copy_from_slice(&indexes_start.to_be_bytes());
d.extend_from_slice(&0u32.to_be_bytes()); // index_len (unused by parser)
d.extend_from_slice(&first);
d.extend_from_slice(&top);
d.extend_from_slice(&(titles.len() as u16).to_be_bytes());
for t in titles {
d.extend_from_slice(t);
}
d
}
#[test]
fn parses_hdmv_first_play_and_titles() {
let d = build(hdmv_obj(0), bdj_obj(), &[bdj_obj(), bdj_obj()]);
let idx = parse(&d).expect("parses");
assert_eq!(idx.first_play, PlaybackObj::Hdmv { id_ref: 0 });
assert_eq!(idx.top_menu, PlaybackObj::BdJ);
assert_eq!(idx.titles.len(), 2);
assert_eq!(idx.titles[0], PlaybackObj::BdJ);
}
#[test]
fn rejects_bad_magic_and_truncation() {
assert!(parse(b"NOPE").is_none());
let d = build(hdmv_obj(3), hdmv_obj(0), &[hdmv_obj(1)]);
// Truncating below the declared title span must yield None, never panic.
assert!(parse(&d[..d.len() - 4]).is_none());
}
}
+147
View File
@@ -0,0 +1,147 @@
//! Parse `/BDMV/MovieObject.bdmv` — the HDMV navigation programs, per their
//! documented binary layout. Read as a documented format, never executed here:
//! bounds-checked, never panics.
//!
//! Layout: `"MOBJ"` + version(4) + reserved… ; `MovieObjects()` at byte 40 is
//! `length`(u32) + reserved(u32) + `num_objects`(u16 @48), then objects from
//! byte 50. Each object is `flags`(1) + reserved(1) + `num_cmds`(u16) followed
//! by `num_cmds` 12-byte navigation commands.
/// One decoded 12-byte navigation command. Field layout (big-endian): byte 0 =
/// `op_cnt(3) grp(2) sub_grp(3)`; byte 1 = `imm_op1(1) imm_op2(1) branch_opt(4)`
/// (low nibble); byte 2 low nibble = `cmp_opt`; byte 3 low 5 bits = `set_opt`;
/// then `dst`(u32) and `src`(u32).
#[derive(Debug, Clone, Copy)]
pub(crate) struct Cmd {
pub op_cnt: u8,
pub grp: u8,
pub sub_grp: u8,
pub imm_op1: bool,
pub imm_op2: bool,
pub branch_opt: u8,
pub cmp_opt: u8,
pub set_opt: u8,
pub dst: u32,
pub src: u32,
}
/// One HDMV navigation program.
#[derive(Debug, Clone)]
pub(crate) struct MovieObject {
pub cmds: Vec<Cmd>,
}
const CMD_LEN: usize = 12;
/// Sanity caps (real discs are far under these; some densely-branched
/// dispatchers run several thousand commands in one object).
const MAX_OBJECTS: usize = 4096;
const MAX_CMDS: usize = 200_000;
fn be_u16(d: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*d.get(o)?, *d.get(o + 1)?]))
}
/// Decode one 12-byte command. Caller guarantees `b.len() == 12`.
fn decode_cmd(b: &[u8]) -> Cmd {
let (b0, b1, b2, b3) = (b[0], b[1], b[2], b[3]);
Cmd {
op_cnt: (b0 >> 5) & 0x7,
grp: (b0 >> 3) & 0x3,
sub_grp: b0 & 0x7,
imm_op1: (b1 >> 7) & 1 == 1,
imm_op2: (b1 >> 6) & 1 == 1,
branch_opt: b1 & 0x0f,
cmp_opt: b2 & 0x0f,
set_opt: b3 & 0x1f,
dst: u32::from_be_bytes([b[4], b[5], b[6], b[7]]),
src: u32::from_be_bytes([b[8], b[9], b[10], b[11]]),
}
}
/// Parse `MovieObject.bdmv` into its programs. Returns `None` on any structural
/// problem.
pub(crate) fn parse(d: &[u8]) -> Option<Vec<MovieObject>> {
if d.get(0..4)? != b"MOBJ" {
return None;
}
let num = be_u16(d, 48)? as usize;
if num > MAX_OBJECTS {
return None;
}
let mut off = 50usize;
let mut objs = Vec::with_capacity(num);
for _ in 0..num {
// Object header: flags(1) + reserved(1) + num_cmds(u16).
let num_cmds = be_u16(d, off + 2)? as usize;
off = off.checked_add(4)?;
if num_cmds > MAX_CMDS {
return None;
}
let span = num_cmds.checked_mul(CMD_LEN)?;
let end = off.checked_add(span)?;
if end > d.len() {
return None;
}
let mut cmds = Vec::with_capacity(num_cmds);
for i in 0..num_cmds {
let s = off + i * CMD_LEN;
cmds.push(decode_cmd(&d[s..s + CMD_LEN]));
}
off = end;
objs.push(MovieObject { cmds });
}
Some(objs)
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
/// Encode one command from its opcode bytes + operands.
pub(crate) fn cmd(b0: u8, b1: u8, b2: u8, b3: u8, dst: u32, src: u32) -> [u8; 12] {
let mut b = [0u8; 12];
b[0] = b0;
b[1] = b1;
b[2] = b2;
b[3] = b3;
b[4..8].copy_from_slice(&dst.to_be_bytes());
b[8..12].copy_from_slice(&src.to_be_bytes());
b
}
pub(crate) fn build(objects: &[&[[u8; 12]]]) -> Vec<u8> {
let mut d = vec![0u8; 50];
d[0..4].copy_from_slice(b"MOBJ");
d[48..50].copy_from_slice(&(objects.len() as u16).to_be_bytes());
for obj in objects {
d.push(0); // flags
d.push(0); // reserved
d.extend_from_slice(&(obj.len() as u16).to_be_bytes());
for c in *obj {
d.extend_from_slice(c);
}
}
d
}
#[test]
fn parses_objects_and_commands() {
// op_cnt=1, grp=BRANCH(0), sub_grp=PLAY(2), branch_opt=PLAY_PL(0), imm dst.
let b0 = (1 << 5) | 2;
let play = cmd(b0, 0x80, 0, 0, 11, 0);
let d = build(&[&[play]]);
let objs = parse(&d).expect("parses");
assert_eq!(objs.len(), 1);
assert_eq!(objs[0].cmds.len(), 1);
assert_eq!(objs[0].cmds[0].dst, 11);
assert_eq!(objs[0].cmds[0].sub_grp, 2);
}
#[test]
fn rejects_truncation() {
let play = cmd((1 << 5) | 2, 0x80, 0, 0, 11, 0);
let d = build(&[&[play]]);
assert!(parse(&d[..d.len() - 3]).is_none());
assert!(parse(b"NOPE").is_none());
}
}
+50
View File
@@ -0,0 +1,50 @@
//! BD/UHD HDMV navigation resolver — "mimic a real player" for main-feature
//! title selection. It reads `/BDMV/index.bdmv` + `/BDMV/MovieObject.bdmv` and
//! runs a faithful, bounded HDMV navigation VM to find the playlist the disc's
//! own First-Play navigation plays as the feature.
//!
//! There is no per-disc special-casing: one correctly-booted VM resolves
//! densely-branched dispatchers as well as simple ones. The honest boundary is
//! BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
//! on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
//! and selection falls back to the structural/heuristic order.
//!
//! Contract: read-only, bounded, and never panics or hard-fails.
pub(crate) mod index;
pub(crate) mod mobj;
pub(crate) mod vm;
use crate::sector::SectorSource;
use crate::udf::UdfFs;
/// Resolve the playlist id the disc's First-Play navigation plays as the
/// feature, restricted to ids the caller marks as feature candidates
/// (`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
/// short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J discs,
/// missing/malformed nav data, or when navigation does not converge on a
/// candidate.
pub(crate) fn resolve_feature(
reader: &mut dyn SectorSource,
udf: &UdfFs,
is_feature_candidate: impl Fn(u16) -> bool,
) -> Option<u16> {
// Belt-and-suspenders: the parsers and VM are written panic-free and bounded,
// but a navigation resolver must NEVER take down a scan — swallow any
// unexpected panic and abstain.
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
resolve_inner(reader, udf, &is_feature_candidate)
}))
.ok()
.flatten()
}
fn resolve_inner(
reader: &mut dyn SectorSource,
udf: &UdfFs,
is_feature_candidate: &dyn Fn(u16) -> bool,
) -> Option<u16> {
let index = index::parse(&udf.read_file(reader, "/BDMV/index.bdmv").ok()?)?;
let mobjs = mobj::parse(&udf.read_file(reader, "/BDMV/MovieObject.bdmv").ok()?)?;
vm::resolve(&index, &mobjs, is_feature_candidate)
}
+474
View File
@@ -0,0 +1,474 @@
//! The HDMV navigation VM — a faithful, bounded re-implementation of the subset
//! of HDMV navigation semantics needed to follow First-Play to the feature
//! `PlayPlayList`. It "mimics a real player": correct power-on register state is
//! what lets a densely-branched dispatcher converge on the feature rather than
//! loop. Never panics; hard step/switch caps.
//!
//! It resolves generically — there is no per-disc special-casing. When
//! First-Play is a BD-J title (the feature is chosen by a Java Xlet), or the
//! program does not reach a `PlayPL` on a caller-approved feature candidate, it
//! abstains (`None`) and selection falls back to the structural/heuristic order.
use super::index::{Index, PlaybackObj};
use super::mobj::MovieObject;
/// Top bit of an operand selects PSR (else GPR).
const PSR_FLAG: u32 = 0x8000_0000;
/// Upper bound on VM steps before declaring non-convergence.
const MAX_STEPS: usize = 1_000_000;
/// Cap on object/title switches — a dispatcher runs in one object; this only
/// bounds pathological jump chains.
const MAX_SWITCHES: usize = 1024;
/// BD player power-on register defaults (indices 0..=61); unlisted = 0. The
/// load-bearing values are PSR4/5 = `0xffff` ("no title/chapter selected") and
/// PSR6/7/8 = 0, which steer dispatcher compares; the rest are set for fidelity
/// with real player behavior.
fn psr_init() -> [u32; 128] {
let mut p = [0u32; 128];
let vals: &[(usize, u32)] = &[
(0, 1),
(1, 0xff),
(2, 0x0fff_0fff),
(3, 1),
(4, 0xffff),
(5, 0xffff),
(10, 0xffff),
(12, 0xff),
(13, 0xff),
(14, 0xffff),
(15, 0xAAAA),
(16, 0x00ff_ffff),
(17, 0x00ff_ffff),
(18, 0x00ff_ffff),
(19, 0xffff),
(20, 2),
(29, 0x0000_0003),
(30, 0x0001_ffff),
(31, 0x0003_0200),
(36, 0xffff),
(37, 0xffff),
(42, 0xffff),
(44, 0xff),
];
for &(i, v) in vals {
p[i] = v;
}
for slot in p.iter_mut().take(62).skip(48) {
*slot = 0xffff_ffff;
}
p
}
struct Vm<'a> {
mobjs: &'a [MovieObject],
index: &'a Index,
gpr: [u32; 4096],
psr: [u32; 128],
}
impl Vm<'_> {
fn rd(&self, val: u32) -> u32 {
if val & PSR_FLAG != 0 {
self.psr[(val & 0x7f) as usize]
} else {
self.gpr[(val & 0xfff) as usize]
}
}
fn wr(&mut self, val: u32, x: u32) {
// Per the HDMV nav spec, a store to a PSR-tagged register is refused;
// only GPR writes take effect. PSR values come only from power-on init,
// never from a nav SET/SWAP operand.
if val & PSR_FLAG == 0 {
self.gpr[(val & 0xfff) as usize] = x;
}
}
fn fetch(&self, imm: bool, raw: u32) -> u32 {
if imm { raw } else { self.rd(raw) }
}
/// Resolve a JumpTitle target: title 0 = Top Menu, `1..=N` = `titles[i-1]`,
/// `0xFFFF` = First Play (BD-ROM title numbering). Any other value is an
/// invalid title reference and abstains (`None`).
fn title_obj(&self, title: u32) -> Option<PlaybackObj> {
let n = self.index.titles.len() as u32;
if title == 0 {
Some(self.index.top_menu)
} else if title <= n {
self.index.titles.get((title - 1) as usize).copied()
} else if title == 0xFFFF {
Some(self.index.first_play)
} else {
None
}
}
}
/// Run First-Play and return the first `PlayPL`/`PlayPL_PM`/`PlayPL_PI` whose
/// playlist id passes `is_feature`. `None` = BD-J boundary, non-convergence, or
/// no approved feature playlist reached.
pub(crate) fn resolve(
index: &Index,
mobjs: &[MovieObject],
is_feature: &dyn Fn(u16) -> bool,
) -> Option<u16> {
// Enter at First-Play when it is an HDMV object that exists; otherwise
// abstain (BD-J boundary, or a `0xffff` "no object"). We deliberately do not
// chase the Top-Menu program here — its "Play Movie" target usually lives in
// the IG-stream button navigation commands inside the menu `.m2ts`, not in
// `MovieObject.bdmv` (future work).
let start = match index.first_play {
PlaybackObj::Hdmv { id_ref } if (id_ref as usize) < mobjs.len() => id_ref as usize,
_ => return None,
};
let mut vm = Vm {
mobjs,
index,
gpr: [0; 4096],
psr: psr_init(),
};
run(&mut vm, start, is_feature)
}
fn run(vm: &mut Vm, mut obj_id: usize, is_feature: &dyn Fn(u16) -> bool) -> Option<u16> {
let mut pc = 0usize;
let mut steps = 0usize;
let mut switches = 0usize;
loop {
steps += 1;
if steps > MAX_STEPS {
return None;
}
let obj = vm.mobjs.get(obj_id)?;
// Ran off the end without a feature PlayPL → abstain.
let c = *obj.cmds.get(pc)?;
let mut npc = pc + 1;
let dst = if c.op_cnt > 0 {
vm.fetch(c.imm_op1, c.dst)
} else {
0
};
let src = if c.op_cnt > 1 {
vm.fetch(c.imm_op2, c.src)
} else {
0
};
match c.grp {
// BRANCH
0 => match c.sub_grp {
// GOTO
0 => match c.branch_opt {
0x01 => npc = dst as usize, // GOTO
0x02 => return None, // BREAK — terminate
_ => {} // NOP / other
},
// JUMP
1 => match c.branch_opt {
0x00 | 0x02 => {
// JumpObject / CallObject (return address not modelled —
// for feature resolution we only follow the play path).
switches += 1;
if switches > MAX_SWITCHES || dst as usize >= vm.mobjs.len() {
return None;
}
obj_id = dst as usize;
pc = 0;
continue;
}
0x01 | 0x03 => {
// JumpTitle / CallTitle — resolve through the index table.
switches += 1;
if switches > MAX_SWITCHES {
return None;
}
match vm.title_obj(dst) {
Some(PlaybackObj::Hdmv { id_ref })
if (id_ref as usize) < vm.mobjs.len() =>
{
obj_id = id_ref as usize;
pc = 0;
continue;
}
// BD-J / unknown / invalid title → abstain.
_ => return None,
}
}
_ => {} // RESUME / other — fall through
},
// PLAY: PlayPL / PlayPL_PI / PlayPL_PM emit a playlist id (the
// rest — Terminate / Link — fall through). A logo/pre-roll whose
// id isn't a feature candidate lets autoplay resume at pc + 1.
2 if matches!(c.branch_opt, 0x00..=0x02) => {
let id = dst as u16;
if is_feature(id) {
return Some(id);
}
}
_ => {}
},
// CMP — skip the next command when the compare is false.
1 => {
let truth = match c.cmp_opt {
0x01 => (dst & !src) == 0,
0x02 => dst == src,
0x03 => dst != src,
0x04 => dst >= src,
0x05 => dst > src,
0x06 => dst <= src,
0x07 => dst < src,
_ => true,
};
if !truth {
npc = pc + 2;
}
}
// SET (sub_grp 0). SETSYSTEM (sub_grp 1) only mutates system PSRs
// whose values the feature path does not branch on — skip it.
2 if c.sub_grp == 0 => {
let r: Option<u32> = match c.set_opt {
0x01 => Some(src), // MOVE
0x02 => {
// SWAP exchanges the two operands; a store to an operand
// flagged immediate is refused, and `wr` already drops
// PSR stores.
if !c.imm_op1 {
vm.wr(c.dst, src);
}
if !c.imm_op2 {
vm.wr(c.src, dst);
}
None
}
0x03 => Some(dst.wrapping_add(src)),
0x04 => Some(dst.saturating_sub(src)),
0x05 => Some(dst.wrapping_mul(src)),
0x06 => Some(dst.checked_div(src).unwrap_or(0xffff_ffff)),
0x07 => Some(dst.checked_rem(src).unwrap_or(0xffff_ffff)),
0x08 => Some(dst), // RND — deterministic stand-in
0x09 => Some(dst & src),
0x0a => Some(dst | src),
0x0b => Some(dst ^ src),
0x0c => Some(dst | (1u32 << (src & 31))),
0x0d => Some(dst & !(1u32 << (src & 31))),
0x0e => Some(dst.wrapping_shl(src & 31)),
0x0f => Some(dst.wrapping_shr(src & 31)),
_ => None,
};
if let Some(r) = r
&& !c.imm_op1
{
vm.wr(c.dst, r);
}
}
_ => {}
}
pc = npc;
}
}
#[cfg(test)]
mod tests {
use super::super::index::{Index, PlaybackObj};
use super::super::mobj::{self, tests::build, tests::cmd};
use super::*;
fn play_pl(id: u16) -> [u8; 12] {
// op_cnt=1, grp=BRANCH(0), sub_grp=PLAY(2), branch_opt=PLAY_PL(0), imm dst.
cmd((1 << 5) | 2, 0x80, 0, 0, id as u32, 0)
}
fn jump_object(obj: u16) -> [u8; 12] {
// grp=BRANCH(0), sub_grp=JUMP(1), branch_opt=JUMP_OBJECT(0), imm dst.
cmd((1 << 5) | 1, 0x80, 0, 0, obj as u32, 0)
}
fn jump_title(title: u16) -> [u8; 12] {
// sub_grp=JUMP(1), branch_opt=JUMP_TITLE(1), imm dst.
cmd((1 << 5) | 1, 0x81, 0, 0, title as u32, 0)
}
fn set_move_gpr(reg: u16, imm: u16) -> [u8; 12] {
// grp=SET(2), sub_grp=SET(0), set_opt=MOVE(1); dst=reg (GPR), imm src.
// op_cnt=2, imm_op2=1 (src immediate), imm_op1=0 (dst is a register).
cmd((2 << 5) | (2 << 3), 0x40, 0, 0x01, reg as u32, imm as u32)
}
fn play_pl_reg(reg: u16) -> [u8; 12] {
// PLAY_PL with dst = GPR[reg] (op_cnt=1, imm_op1=0).
cmd((1 << 5) | 2, 0x00, 0, 0, reg as u32, 0)
}
fn idx(first: PlaybackObj, titles: Vec<PlaybackObj>) -> Index {
Index {
first_play: first,
top_menu: PlaybackObj::BdJ,
titles,
}
}
#[test]
fn resolves_immediate_playpl() {
// First-Play HDMV obj0 → JumpObject 1 → PlayPL 11.
let d = build(&[&[jump_object(1)], &[play_pl(11)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
assert_eq!(resolve(&index, &mobjs, &|id| id == 11), Some(11));
}
#[test]
fn skips_non_feature_playpl_then_returns_feature() {
// Autoplay chain: logo PlayPL 99 (not a candidate) → feature PlayPL 1.
let d = build(&[&[play_pl(99), play_pl(1)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
// Only 1 is a feature candidate; 99 is a logo.
assert_eq!(resolve(&index, &mobjs, &|id| id == 1), Some(1));
}
#[test]
fn abstains_when_first_play_jumps_to_bdj_title() {
// Computed-title-number shape: First-Play computes a title number in a
// GPR and JumpTitles to it; the target title is BD-J → abstain (None).
let d = build(&[&[set_move_gpr(0xFEB & 0xfff, 2), {
// JumpTitle with dst = GPR[0xFEB].
cmd((1 << 5) | 1, 0x01, 0, 0, 0x0000_0FEB, 0)
}]]);
let mobjs = mobj::parse(&d).unwrap();
// title 2 → titles[1] = BD-J.
let index = idx(
PlaybackObj::Hdmv { id_ref: 0 },
vec![PlaybackObj::Hdmv { id_ref: 5 }, PlaybackObj::BdJ],
);
assert_eq!(resolve(&index, &mobjs, &|_| true), None);
}
#[test]
fn abstains_when_first_play_is_bdj() {
let d = build(&[&[play_pl(1)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::BdJ, vec![]);
assert_eq!(resolve(&index, &mobjs, &|_| true), None);
}
#[test]
fn jumptitle_first_play_is_0xffff_not_n_plus_1() {
// BD-ROM title numbering: 0 = Top Menu, 0xFFFF = First Play,
// 1..=N = titles. `N+1` is an INVALID title ref.
let index = idx(
PlaybackObj::Hdmv { id_ref: 7 },
vec![PlaybackObj::Hdmv { id_ref: 1 }],
);
let vm = Vm {
mobjs: &[],
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
assert_eq!(vm.title_obj(0), Some(index.top_menu), "title 0 = Top Menu");
assert_eq!(
vm.title_obj(1),
Some(PlaybackObj::Hdmv { id_ref: 1 }),
"title 1 = titles[0]"
);
assert_eq!(
vm.title_obj(0xFFFF),
Some(PlaybackObj::Hdmv { id_ref: 7 }),
"0xFFFF = First Play (abstained wrongly before the fix)"
);
assert_eq!(
vm.title_obj(2),
None,
"N+1 is an invalid title ref, not First Play (resolved wrongly before the fix)"
);
}
#[test]
fn set_to_psr_is_refused() {
// A SET MOVE to PSR6 must be a no-op (PSR stores are refused by spec).
// Program: MOVE PSR6 <- 5; CMP PSR6 == 5; PlayPL 100 else 200.
// With the write refused, PSR6 stays at its power-on 0, so the compare is
// false and control skips to PlayPL 200. If the illegal PSR write took
// effect, the compare would be true and PlayPL 100 would win.
let set_psr6 = cmd((2 << 5) | (2 << 3), 0x40, 0, 0x01, 0x8000_0006, 5);
let cmp_psr6_eq_5 = cmd((2 << 5) | (1 << 3), 0x40, 0x02, 0, 0x8000_0006, 5);
let d = build(&[&[set_psr6, cmp_psr6_eq_5, play_pl(100), play_pl(200)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
assert_eq!(
resolve(&index, &mobjs, &|id| id == 100 || id == 200),
Some(200),
"PSR store must be refused, so the compare is false and 200 is reached"
);
}
#[test]
fn register_fold_and_cmp_skip_reach_playpl() {
// SET GPR[3]=7; CMP GPR[3]==7 (true → do NOT skip); PlayPL GPR[3].
// grp=CMP(1), op_cnt=2, imm_op2=1 (src=7 immediate), cmp_opt=EQ(2); dst=GPR3.
let cmp_eq_reg = cmd((2 << 5) | (1 << 3), 0x40, 0x02, 0, 3, 0x0000_0007);
let d = build(&[&[set_move_gpr(3, 7), cmp_eq_reg, play_pl_reg(3), play_pl(999)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
// If the compare wrongly skipped, we'd hit PlayPL 999 instead of 7.
assert_eq!(resolve(&index, &mobjs, &|id| id == 7 || id == 999), Some(7));
}
#[test]
fn jumptitle_namespace_differs_from_jumpobject() {
// JumpTitle 1 → titles[0] = HDMV obj 1 (which PlayPLs 42). Proves title
// numbers resolve through the index, not as object indices.
let d = build(&[&[jump_title(1)], &[play_pl(42)]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(
PlaybackObj::Hdmv { id_ref: 0 },
vec![PlaybackObj::Hdmv { id_ref: 1 }],
);
assert_eq!(resolve(&index, &mobjs, &|id| id == 42), Some(42));
}
#[test]
fn swap_exchanges_two_registers() {
// MOVE g0=11; MOVE g1=22; SWAP g0<->g1; PlayPL GPR[0]. SWAP: grp=SET(2),
// sub_grp=SET(0), set_opt=SWAP(2), both operands registers (imm_op*=0).
let swap01 = cmd((2 << 5) | (2 << 3), 0x00, 0, 0x02, 0, 1);
let d = build(&[&[
set_move_gpr(0, 11),
set_move_gpr(1, 22),
swap01,
play_pl_reg(0),
]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
let mut vm = Vm {
mobjs: &mobjs,
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
// After the swap GPR0 holds the old GPR1 (22) and PlayPL plays it.
assert_eq!(run(&mut vm, 0, &|id| id == 22), Some(22));
// Full exchange: GPR1 now holds the old GPR0 (11).
assert_eq!(vm.gpr[1], 11);
}
#[test]
fn swap_immediate_operand_is_not_written_back() {
// Seed GPR[5]=77, then SWAP dst=GPR0 (register) with src=IMMEDIATE 5
// (imm_op1=0, imm_op2=1). The imm gate must refuse using the immediate
// as a write target, so GPR[5] — the register the literal 5 would alias
// — keeps 77; only GPR0 receives the immediate. Dropping either
// `if !c.imm_op1` / `if !c.imm_op2` guard makes an assertion fail.
let swap_imm = cmd((2 << 5) | (2 << 3), 0x40, 0, 0x02, 0, 5);
let d = build(&[&[set_move_gpr(5, 77), swap_imm]]);
let mobjs = mobj::parse(&d).unwrap();
let index = idx(PlaybackObj::Hdmv { id_ref: 0 }, vec![]);
let mut vm = Vm {
mobjs: &mobjs,
index: &index,
gpr: [0; 4096],
psr: psr_init(),
};
// No PlayPL → program runs off the end (None); inspect registers after.
assert_eq!(run(&mut vm, 0, &|_| false), None);
assert_eq!(
vm.gpr[5], 77,
"immediate operand must not be a write target"
);
assert_eq!(vm.gpr[0], 5, "register operand receives the immediate");
}
}
+2 -3
View File
@@ -4,7 +4,7 @@
//! The EP (Entry Point) map provides timestamp → SPN mapping.
//! SPN × 192 = byte offset in the m2ts file.
//!
//! Reference: https://github.com/lw/BluRay/wiki/CLPI
//! Format is documented in the BD-ROM Clip Information (CLPI) specification.
use crate::error::{Error, Result};
@@ -230,8 +230,7 @@ mod tests {
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the BD-ROM CLPI spec
// (https://github.com/lw/BluRay/wiki/CLPI).
// Added hardening tests, grounded in the BD-ROM CLPI spec byte layout.
// ─────────────────────────────────────────────────────────────────────
/// Build a ProgramInfo section. `streams` = Vec<(pid, sci_bytes)>.
+4
View File
@@ -114,6 +114,10 @@ pub mod coding_type {
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
+10 -10
View File
@@ -1,9 +1,9 @@
//! CSS title-key recovery — Frank A. Stevenson's divide-and-conquer attack
//! (1999), implemented from his published cryptanalysis ("Cryptanalysis of
//! Contents Scrambling System"). It recovers the 5-byte CSS title key from a
//! single scrambled DVD sector with no player keys and no disc-key crack, using
//! only known plaintext. Implemented from that public description; nothing here
//! is copied or translated from any particular CSS software.
//! CSS title-key recovery — a divide-and-conquer known-plaintext attack,
//! implemented from the openly published CSS cryptanalysis literature. It
//! recovers the 5-byte CSS title key from a single scrambled DVD sector with no
//! player keys and no disc-key crack, using only known plaintext. Implemented
//! from that public description; nothing here is copied or translated from any
//! particular CSS software.
//!
//! # The cipher this attacks
//!
@@ -41,8 +41,8 @@ const ENCRYPTED_START: usize = 0x80; // byte 128
const SEED_OFFSET: usize = 0x54; // sector seed at bytes 0x54-0x58
const FLAG_BYTE: usize = 0x14;
/// Recover the title key from cipher + known plaintext (the core of Stevenson's
/// attack). `crypted` is the ciphertext starting at sector byte 0x80;
/// Recover the title key from cipher + known plaintext (the core of the
/// recovery). `crypted` is the ciphertext starting at sector byte 0x80;
/// `decrypted` is the matching known plaintext; `seed` is `sector[0x54..0x59]`.
/// On success returns the recovered 5-byte title key; `None` if no LFSR seed
/// reproduces the keystream.
@@ -210,8 +210,8 @@ fn descramble_matches(sector: &[u8], title: &[u8; 5], plain: &[u8]) -> bool {
}
/// Find a repeating pattern just before the encrypted region and assume the
/// plaintext at 0x80 continues it — the known-plaintext step of Stevenson's
/// attack. Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// plaintext at 0x80 continues it — the known-plaintext step of the recovery.
/// Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// with a cycle length in 2..0x2F. If the run is long enough (`plen > 3` and at
/// least two full cycles), the known plaintext at 0x80 is taken to be the
/// periodic run continuing forward, and [`recover_title_key_from_plain`] is
+2 -3
View File
@@ -1,9 +1,8 @@
//! CSS content cipher — an independent implementation of the publicly
//! documented Content Scramble System stream cipher.
//!
//! The algorithm is the one recovered and published in Frank A. Stevenson's
//! 1999 cryptanalysis ("Cryptanalysis of Contents Scrambling System") and
//! described in the open CSS literature. It is implemented here from that public
//! The algorithm is the publicly documented CSS stream cipher, described in the
//! open CSS cryptanalysis literature. It is implemented here from that public
//! description; its constants (see [`super::tables`]) are the cipher's own
//! defined values. Nothing in this file is copied or translated from any
//! particular CSS software.
+263 -61
View File
@@ -2,9 +2,9 @@
//!
//! CSS uses a weak 40-bit LFSR stream cipher (broken since 1999).
//!
//! The title key is recovered keylessly: [`crack_key`] runs the Stevenson
//! known-plaintext attack (see the [`stevenson`] module) on the scrambled
//! data, needing no player keys, disc-key crack, or external key file.
//! The title key is recovered keylessly: [`crack_key`] recovers it directly
//! from the scrambled data (see the [`keyless`] module),
//! needing no player keys, disc-key recovery, or external key file.
//! Sectors are then decrypted with [`descramble_sector`].
//!
//! Usage:
@@ -14,8 +14,8 @@
//! }
//! ```
pub mod keyless;
pub mod lfsr;
pub mod stevenson;
pub(crate) mod tables;
use crate::disc::Extent;
@@ -56,7 +56,7 @@ impl std::fmt::Debug for CssState {
}
/// Recover the CSS title key with no keys, by scanning scrambled sectors and
/// running the Stevenson known-plaintext attack (see the [`stevenson`] module).
/// running the known-plaintext attack (see the [`keyless`] module).
///
/// The crib comes from the periodic-run detector: a scrambled sector's cleartext region
/// (bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
@@ -75,7 +75,7 @@ pub fn crack_key(
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
crack_key_scan(reader, extents, batch_sectors, None).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -88,7 +88,7 @@ pub fn crack_key(
/// scanned extents (`is_scrambled_pack` never true): the content is genuinely
/// plaintext, so proceeding without a key is correct.
/// - [`CrackOutcome::ScrambledUncracked`] — scrambled sectors WERE seen but no
/// key could be recovered (the Stevenson attack found no crackable crib, or
/// key could be recovered (the recovery found no crackable crib, or
/// the scrambled region was unreadable). The content is encrypted; muxing it
/// as plaintext would emit garbage, so callers MUST surface a hard error
/// instead of falling through to "unencrypted" — the per-title
@@ -136,7 +136,7 @@ pub fn crack_key_outcome(
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
) -> CrackOutcome {
crack_key_scan(reader, extents, batch_sectors, halt, true)
crack_key_scan(reader, extents, batch_sectors, halt)
}
/// Resolve a DVD title's CSS descramble key from the reader when the caller
@@ -212,10 +212,6 @@ fn crack_key_scan(
extents: &[Extent],
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
// True only on the INITIAL scan: a fully CSS-locked (`05/6F/03`) result is a
// hard `ScrambledUncracked`. False on the per-VTS re-crack so a lapsed-AGID
// locked read returns None instead of killing a genuinely crackable title.
fail_on_locked: bool,
) -> CrackOutcome {
// Batch the reads: a live optical drive at 1 sector/read is glacial, and the
// crack only needs to FIND one scrambled sector whose 0x80 plaintext matches
@@ -301,6 +297,18 @@ fn crack_key_scan(
let usable = (got / 2048).min(n as usize);
// At least one, so a source returning Ok(0) cannot spin here.
advance = (usable as u32).max(1);
if usable == 0 {
// Nothing was inspected, so the per-sector `tried`
// charge below never runs — but the cursor still moves
// one sector (the `.max(1)` above). Charge that sector
// to the budget, or `tried` stays frozen and the loop
// is bounded only by the disc-declared
// `ext.sector_count`: the anti-grind budget stops
// applying to exactly the misbehaving source it exists
// for. Mirrors the `Err` arm's `tried += n`, which
// likewise charges an uninspected advance.
tried += 1;
}
for s in 0..usable {
tried += 1;
let sect = &buf[s * 2048..(s + 1) * 2048];
@@ -310,7 +318,7 @@ fn crack_key_scan(
// would falsely report ScrambledUncracked (a false E7023).
if is_scrambled_pack(sect) {
saw_scrambled = true;
if let Some(key) = stevenson::crack_title_key(sect) {
if let Some(key) = keyless::crack_title_key(sect) {
return CrackOutcome::Cracked(CssState {
title_key: key,
crack_span,
@@ -344,14 +352,29 @@ fn crack_key_scan(
}
// Budget exhausted / extents walked / early-bailed with no key recovered.
// The disc is ENCRYPTED-but-uncracked (a hard failure on the initial scan)
// when EITHER a scrambled sector was actually seen, OR — on the initial scan
// only (`fail_on_locked`) — every read was CSS-locked (`05/6F/03`), itself
// proof of scrambling. A re-crack (`fail_on_locked` false) stays soft: a
// lapsed-AGID locked read yields None, not a hard fail, so a crackable title
// in another VTS isn't killed. Only a scan that saw neither a scrambled
// sector nor a CSS-lock is genuinely unencrypted.
if saw_scrambled || (saw_locked && fail_on_locked) {
// The disc is ENCRYPTED-but-uncracked (a hard failure) when EITHER a
// scrambled sector was actually seen, OR every read was CSS-locked
// (`05/6F/03`), itself proof of scrambling. Only a scan that saw neither a
// scrambled sector nor a CSS-lock is genuinely unencrypted.
//
// A prior revision made this conditional on a caller-supplied
// `fail_on_locked: bool`, documented as "false on the per-VTS re-crack, so
// a lapsed-AGID locked read returns None instead of killing a genuinely
// crackable title." That parameter never had an observable effect: its
// ONLY non-test caller with `false` was [`crack_key`], whose `Option`
// return collapses `ScrambledUncracked` and `Unencrypted` alike to `None`
// via [`CrackOutcome::into_state`] — so the branch this comment describes
// was unreachable from the moment it was introduced (see the crate's audit
// notes for the git-archaeology). The one production caller that DOES
// observe the `Cracked` / `Unencrypted` / `ScrambledUncracked` split for a
// per-VTS re-crack (`Disc::decrypt_keys_for_title`) has always gone
// through [`crack_key_outcome`], which hardcoded this to always-hard-fail.
// Reconnecting the soft variant there would mean a locked read on a live
// drive gets reported as `Unencrypted` (`title_is_clear = true`) and the
// title is muxed with NO key — precisely the silent-garbage failure mode
// `CrackOutcome` exists to prevent. So the parameter is removed rather
// than revived: the scan is unconditionally hard-fail-on-locked.
if saw_scrambled || saw_locked {
CrackOutcome::ScrambledUncracked
} else {
CrackOutcome::Unencrypted
@@ -402,19 +425,16 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
///
/// # Errors
///
/// [`Error::DecryptFailed`] when a sector's own crib proves the cached key stale
/// and the re-crack from that same sector also fails. CSS has no external key
/// source — the title key comes only from cracking the data — so on a readable
/// sector this is not a missing input, it is recovery failing on data we can
/// see. Emitting the sector anyway means one of two bad outcomes: descrambled
/// with the key its crib just rejected, which yields garbage behind an intact
/// clear header (valid pack start, passes every structural check the PS demuxer
/// applies, corruption confined to the PES payload where nothing looks); or
/// passed through still scrambled, which is ciphertext delivered where plaintext
/// is meant to be. Both are bad data reported as success.
/// Never returns `Err` — the signature is `Result` only to match the decrypt
/// seam it is dispatched from, and the `usize` is that seam's legacy
/// always-zero loss count (see [`crate::decrypt::decrypt_sectors`]).
///
/// This matches the AACS sibling, which returns [`Error::DecryptFailed`] rather
/// than apply a neighbouring CPS unit's key.
/// This section used to document an [`Error::DecryptFailed`] for the case where
/// a sector's crib rejects the cached key and the re-crack from that sector also
/// fails. That behaviour was tried and REVERTED, for the reason set out at the
/// `None =>` arm below: crib mismatch plus crack failure is the signature of a
/// crib FALSE POSITIVE, not of a stale key, and failing there made real discs
/// unrippable. The arm descrambles with the cached key and returns `Ok`.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
// `is_scrambled_pack`, NOT the looser `is_scrambled`. The raw flag test
@@ -434,7 +454,7 @@ pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::erro
if chunk.len() < 2048 || !is_scrambled_pack(chunk) {
continue;
}
let crib = stevenson::attack_crib(chunk);
let crib = keyless::attack_crib(chunk);
// Snapshot the ciphertext (chunk is exactly 2048 here) only when there is
// a crib to validate against, so the common cache-hit path costs no
// per-sector heap allocation.
@@ -449,7 +469,7 @@ pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::erro
// Cached key is stale for this region — restore the ciphertext and
// crack this sector's own key.
chunk.copy_from_slice(&original);
match stevenson::crack_title_key(chunk) {
match keyless::crack_title_key(chunk) {
Some(fresh) => {
*title_key = fresh;
lfsr::descramble_sector(title_key, chunk);
@@ -475,7 +495,7 @@ pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::erro
// either opens a unit or does not, whereas a CSS title key
// is recovered from data whose recoverability varies sector
// by sector. Real DVDs hit this constantly; the change made
// Greenland.iso unrippable and was caught by the real-media
// a real disc unrippable and was caught by the real-media
// acceptance gate, not by any unit test.
lfsr::descramble_sector(title_key, chunk);
}
@@ -529,6 +549,35 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// scrambled feature is made of valid PS packs, so its scrambled sectors still
/// pass this check and still drive `ScrambledUncracked` when no key cracks.
///
/// The pack-start check alone is not enough, and the sector's PES `stream_id`
/// (offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
/// prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
/// streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
/// (`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
/// stream_id even on a scrambled sector. The MPEG-PS structural packets that
/// are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
/// private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
/// because on those the byte at 0x14 is NOT a PES scrambling-control field but
/// raw payload/structure whose bits 4-5 land set by chance.
///
/// This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
/// MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
/// private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
/// set. With no `0x11` gate those nav packs flipped the crack scan's
/// `saw_scrambled` flag on a disc that carries no CSS at all; the crack then
/// found no key (there is none) and the scan returned `ScrambledUncracked`,
/// hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
/// `0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
/// can act on (the recovery only recovers a key from a scrambled ES
/// pack), so a decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
///
/// This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
/// real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
/// private_stream_1 (`0xBD`) packs, none of which are excluded, so its
/// scrambled sectors still set `saw_scrambled` and still drive
/// `ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
/// so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
///
/// The DESCRAMBLE path gates on this same function — [`descramble_sector`] and
/// [`descramble_region`] both call it, not the raw flag test — because the raw
/// test does not merely mis-skip a sector there: it descrambles one that was
@@ -537,7 +586,14 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// lost 1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38
/// titles became 10 — silently, at exit 0. One gate, both paths.
pub fn is_scrambled_pack(sector: &[u8]) -> bool {
sector.len() >= 2048 && sector[0x00..0x04] == PACK_START && (sector[0x14] >> 4) & 0x03 != 0
use crate::consts::pes_stream_id::{PADDING_STREAM, PRIVATE_STREAM_2, SYSTEM_HEADER};
sector.len() >= 2048
&& sector[0x00..0x04] == PACK_START
&& !matches!(
sector[0x11],
SYSTEM_HEADER | PADDING_STREAM | PRIVATE_STREAM_2
)
&& (sector[0x14] >> 4) & 0x03 != 0
}
#[cfg(test)]
@@ -559,7 +615,7 @@ mod tests {
/// This test exists because round 9 read the same code as "descrambling
/// with a key we just proved stale" and made it `DecryptFailed` to match
/// the AACS path. Real DVDs hit this constantly — the change made
/// Greenland.iso unrippable, and no unit test caught it; the real-media
/// a real disc unrippable, and no unit test caught it; the real-media
/// acceptance gate did. CSS is not AACS: an AACS unit key either opens a
/// unit or does not, whereas a CSS title key is recovered from data whose
/// recoverability varies sector by sector.
@@ -580,11 +636,11 @@ mod tests {
"fixture must be a scrambled sector"
);
assert!(
stevenson::attack_crib(&sector).is_some(),
keyless::attack_crib(&sector).is_some(),
"fixture must yield a crib, or the mismatch branch is never entered"
);
assert!(
stevenson::crack_title_key(&sector).is_none(),
keyless::crack_title_key(&sector).is_none(),
"fixture must be uncrackable, or the failure branch is never entered"
);
@@ -593,7 +649,22 @@ mod tests {
let out = descramble_region(&mut sector, &mut key)
.expect("a crib false positive must NOT fail the rip");
assert_eq!(out, 0, "CSS reports no loss term of its own");
// The "no loss term" contract belongs to the SEAM, not to this
// function: `decrypt_sectors`' `usize` is a legacy always-zero count
// that the CSS arm feeds from here. Asserting `out == 0` on
// `descramble_region` alone only restates its single `Ok(0)` return —
// a body replaced by `Ok(0)` satisfies it just as well. Assert it one
// level up, where the value is actually assembled and returned, so the
// arm dispatch and the plumbing are exercised too.
assert_eq!(out, 0);
let mut seam_sector = sector;
let mut seam_keys = crate::decrypt::DecryptKeys::Css { title_key: key };
assert_eq!(
crate::decrypt::decrypt_sectors(&mut seam_sector, &mut seam_keys, 0)
.expect("a crib false positive must NOT fail the rip at the seam either"),
0,
"CSS reports no loss term of its own through decrypt_sectors"
);
assert_eq!(
key, key_before,
"a failed re-crack must leave the cached key in place — it is still \
@@ -722,6 +793,52 @@ mod tests {
);
}
/// Detection fix: `is_scrambled_pack` must EXCLUDE the MPEG-PS structural
/// packets that CSS never scrambles — system_header (0xBB), padding (0xBE)
/// and private_stream_2 (0xBF, DVD PCI/DSI and HD-DVD `.evo` RDI nav) — by
/// their `stream_id` at offset 0x11. On those packets byte 0x14 is raw
/// payload/structure, not a PES scrambling-control field, so its bits 4-5
/// land set by chance.
///
/// This is the exact decrypted-HD-DVD defect: an `.evo` RDI pack is
/// private_stream_2 (0xBF) with the pack-start code and bits set at 0x14, so
/// the old gate flipped `saw_scrambled` on a CSS-free disc → the crack found
/// no key → `ScrambledUncracked` → E7023 on a good HD-DVD.
///
/// Grounding: `!matches!(sector[0x11], SYSTEM_HEADER | PADDING_STREAM |
/// PRIVATE_STREAM_2)`. Mutation: drop the 0x11 exclusion → the 0xBF nav pack
/// counts as scrambled evidence and the first assert fails. The video and
/// private_stream_1 packs prove the gate does NOT reject a real scrambled
/// DVD sector (the catastrophic direction).
#[test]
fn is_scrambled_pack_excludes_nav_and_structural_stream_ids() {
use crate::consts::pes_stream_id::{
PADDING_STREAM, PRIVATE_STREAM_1, PRIVATE_STREAM_2, SYSTEM_HEADER, VIDEO, VIDEO_MAX,
};
// A pack-start pack with 0x14 scramble bits set, varying only 0x11.
let mut s = vec![0u8; 2048];
s[0x00..0x04].copy_from_slice(&PACK_START);
s[0x14] = 0x30;
for excluded in [SYSTEM_HEADER, PADDING_STREAM, PRIVATE_STREAM_2] {
s[0x11] = excluded;
assert!(
!is_scrambled_pack(&s),
"stream_id {excluded:#04x} is a structural/nav pack CSS never scrambles — \
it must NOT count as scramble evidence (else a decrypted HD-DVD RDI pack → E7023)"
);
}
// A genuinely scramblable elementary-stream pack must STILL register —
// proving the exclusion did not weaken real-DVD CSS detection.
for scramblable in [VIDEO, VIDEO_MAX, PRIVATE_STREAM_1, 0xE2] {
s[0x11] = scramblable;
assert!(
is_scrambled_pack(&s),
"stream_id {scramblable:#04x} is a scramblable ES pack — a real CSS DVD's \
scrambled sector must still be counted, never passed through as plaintext"
);
}
}
// ── crack_key scanning over a mock SectorSource ────────────────────────
/// Records every (lba, count) read; returns a caller-supplied flag byte at
@@ -734,7 +851,7 @@ mod tests {
/// scrambled reads because the bus-auth gate isn't open).
lock_all: bool,
/// When set, the sector at `crackable.0` is served as a full
/// Stevenson-crackable scrambled sector (`crackable.1`, 2048 bytes)
/// crackable scrambled sector (`crackable.1`, 2048 bytes)
/// instead of the uniform `flag_byte` fill. Lets the scan actually
/// reach `CrackOutcome::Cracked` from a synthetic ISO.
crackable: Option<(u32, Vec<u8>)>,
@@ -743,6 +860,11 @@ mod tests {
/// a damaged region. `Some(0)` is the degenerate case that must not
/// spin the scan.
short_read: Option<usize>,
/// PES `stream_id` written at offset 0x11 of each uniform-fill sector.
/// `0x00` (the default) is a scramblable-looking pack; set to
/// private_stream_2 (`0xBF`) to model an HD-DVD `.evo` RDI nav pack,
/// which carries the pack-start code and 0x14 bits but no CSS.
stream_id: u8,
}
impl MockSource {
@@ -754,15 +876,16 @@ mod tests {
lock_all: false,
crackable: None,
short_read: None,
stream_id: 0x00,
}
}
}
/// Build a Stevenson-crackable scrambled sector for `(title_key, seed)`:
/// Build a crackable scrambled sector for `(title_key, seed)`:
/// the cleartext header (0x59..0x80) carries a periodic run that continues
/// across the 0x80 boundary into the encrypted region — the crib
/// `stevenson::crack_title_key` recovers a key from. Mirrors the
/// `synth_periodic_sector` fixture in the stevenson tests but built here
/// `keyless::crack_title_key` recovers a key from. Mirrors the
/// `synth_periodic_sector` fixture in the keyless tests but built here
/// from the crate-internal `scramble_sector`.
fn crackable_sector(title_key: &[u8; 5], seed: &[u8; 5], period: usize) -> Vec<u8> {
const RUN_START: usize = 0x59;
@@ -828,11 +951,14 @@ mod tests {
}
_ => {
// Real DVD video sectors always open with the MPEG-PS
// pack-start code; `is_scrambled_pack` (Fix 3) requires it
// before trusting the 0x14 scramble bits, so the fixture
// must include it for a `flag_byte` of 0x30 to register
// as scrambled.
// pack-start code; `is_scrambled_pack` requires it before
// trusting the 0x14 scramble bits, so the fixture must
// include it for a `flag_byte` of 0x30 to register as
// scrambled. `stream_id` (byte 0x11) defaults to 0x00 (a
// scramblable pack); an HD-DVD RDI nav pack sets it to
// private_stream_2 (0xBF), which the 0x11 exclusion drops.
buf[base..base + 4].copy_from_slice(&PACK_START);
buf[base + 0x11] = self.stream_id;
buf[base + 0x14] = self.flag_byte;
}
}
@@ -860,7 +986,7 @@ mod tests {
start_lba: 100,
sector_count: 4,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None, false);
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().clone();
assert_eq!(
reads,
@@ -882,7 +1008,7 @@ mod tests {
start_lba: 0,
sector_count: 8,
}];
let outcome = crack_key_scan(&mut src, &ext, 4, None, false);
let outcome = crack_key_scan(&mut src, &ext, 4, None);
assert!(
matches!(outcome, CrackOutcome::Unencrypted),
"nothing was read, so nothing scrambled was seen"
@@ -895,6 +1021,39 @@ mod tests {
);
}
/// The 50_000-sector budget must hold whatever the source returns, not
/// only when the source delivers sectors.
///
/// `tried` is incremented ONLY per inspected sector, inside
/// `for s in 0..usable`. An `Ok(0)` inspects nothing, so that loop never
/// runs — yet `advance` is forced to 1 to stop the scan spinning, so the
/// cursor keeps walking. The budget is then never consulted and the scan
/// runs for the extent's full, disc-declared `sector_count`: a misbehaving
/// or adversarial source (an emulated drive, a bridge answering short)
/// converts the anti-grind bound into no bound at all.
///
/// Mutation: delete the `tried` charge in the `usable == 0` arm and this
/// goes red at 60_000 reads.
#[test]
fn a_source_that_returns_zero_sectors_still_obeys_the_scan_budget() {
const MAX_TRIES: usize = 50_000;
let mut src = MockSource::new(0x00);
src.short_read = Some(0);
// Deliberately LARGER than the budget: if the budget is what stops the
// scan, the extent's own length is never reached.
let ext = [crate::disc::Extent {
start_lba: 0,
sector_count: 60_000,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().len();
assert!(
reads <= MAX_TRIES,
"an Ok(0)-returning source must be stopped by the {MAX_TRIES}-sector \
budget, not by the disc-declared extent length; got {reads} reads"
);
}
/// crack_key caps total scanned sectors at 50_000 even when extents are
/// far larger, and counts EVERY scanned sector (clear ones included)
/// toward the budget. With one 200_000-sector extent of clear sectors, it
@@ -943,7 +1102,7 @@ mod tests {
}
/// THE Fix 6 regression: a scan that SEES scrambled sectors (flag set) but
/// recovers no key (the mock's zeroed data has no Stevenson crib) must
/// recovers no key (the mock's zeroed data has no recoverable crib) must
/// return `ScrambledUncracked` — a HARD failure — NOT `Unencrypted`. The
/// old code conflated this with "unencrypted" and muxed scrambled MPEG as
/// plaintext (garbage at exit 0).
@@ -1100,12 +1259,17 @@ mod tests {
);
}
/// MISSING #1 guard: the re-crack path (the `Option`-returning `crack_key`,
/// `fail_on_locked == false`) must NOT hard-fail on a CSS-locked read — it
/// returns `None`. A lapsed-AGID re-crack of another VTS stays soft so a
/// genuinely crackable title isn't killed by a transient locked read.
/// `crack_key` (the `Option`-returning convenience wrapper) collapses
/// `ScrambledUncracked` and `Unencrypted` alike to `None` via
/// [`CrackOutcome::into_state`], so an all-locked scan still reads `None`
/// here even though the scan itself now treats every CSS-lock as a hard
/// `ScrambledUncracked` (see `crack_key_scan`'s removal of the dead
/// `fail_on_locked` parameter). Callers that need to tell "locked/
/// uncrackable" apart from "genuinely clear" must use `crack_key_outcome`,
/// which the `all_locked_synthetic_iso_yields_css_key_missing_signal` test
/// pins directly.
#[test]
fn crack_key_recrack_locked_is_none_not_hard_fail() {
fn crack_key_all_locked_collapses_to_none() {
let mut src = MockSource::new(0x30);
src.lock_all = true;
let extents = [Extent {
@@ -1252,7 +1416,7 @@ mod tests {
/// SCAN-LEVEL CRACKED (audit gap "MockSource never yields a crackable
/// sector"): drive the full `crack_key_scan` over a synthetic ISO whose
/// scan hits a Stevenson-crackable scrambled sector. The outcome must be
/// scan hits a crackable scrambled sector. The outcome must be
/// `CrackOutcome::Cracked` with a key that round-trips the sector, AND the
/// `crack_span` must be recorded as the half-open extent span (the per-VTS
/// routing key the mux path needs). Previously only the leaf crack and the
@@ -1316,7 +1480,7 @@ mod tests {
sector_count: 50,
}];
// Cracked: a real Stevenson-crackable sector in an otherwise clear scan.
// Cracked: a real crackable sector in an otherwise clear scan.
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let mut cracked_src = MockSource::new(0x00);
@@ -1430,6 +1594,44 @@ mod tests {
);
}
/// The decrypted-HD-DVD regression, end to end through `resolve_dvd_title_key`.
/// An `.evo` is MPEG-PS (`ContentFormat::MpegPs`) exactly like a DVD `.vob`
/// and carries `None` keys once decrypted, so it reaches the CSS crack. Its
/// RDI navigation packs are private_stream_2 (0xBF) with the pack-start code
/// and bits set at offset 0x14 — but HD-DVD carries no CSS at all. The 0x11
/// exclusion in `is_scrambled_pack` keeps those nav packs from flipping the
/// scan's `saw_scrambled` gate, so the scan returns `Unencrypted` and the
/// title muxes cleanly instead of hard-failing.
///
/// Catches the mutation of dropping the 0x11 exclusion: without it every 0xBF
/// RDI pack counts as scramble evidence, the crack finds no key (there is
/// none), and the scan returns `ScrambledUncracked` → `CssKeyMissing` (E7023)
/// on a perfectly good HD-DVD — the exact defect a real CI run produced.
#[test]
fn resolve_dvd_title_key_decrypted_hddvd_rdi_packs_scan_clean_no_e7023() {
let mut src = MockSource::new(0x30); // 0x14 bits set…
src.stream_id = crate::consts::pes_stream_id::PRIVATE_STREAM_2; // …but a 0xBF nav pack
let extents = [Extent {
start_lba: 0,
sector_count: 64,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
8,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("a decrypted HD-DVD's RDI nav packs are not CSS — the scan must not hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"no CSS key exists on an HD-DVD; keys must stay None and the title mux clean"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
@@ -1595,8 +1797,8 @@ mod tests {
/// PER-VTS RE-CRACK SUCCESS (audit gap "success path missing"): the prior
/// re-crack test only covered the locked→None path. Here a re-crack
/// (`crack_key`, `fail_on_locked == false`) over a DIFFERENT VTS's extents
/// finds that VTS's own crackable sector and returns a `CssState` whose
/// (`crack_key`) over a DIFFERENT VTS's extents finds that VTS's own
/// crackable sector and returns a `CssState` whose
/// `crack_span` matches the new extents — proving a key cracked for one VTS
/// is genuinely re-derived (not reused) for another.
#[test]
+18 -5
View File
@@ -849,10 +849,10 @@ mod tests {
);
}
/// Build a Stevenson-crackable scrambled CSS sector for `title_key` (mirrors
/// Build a crackable scrambled CSS sector for `title_key` (mirrors
/// `crackable_sector` in the css::mod tests): a periodic run in the clear
/// header continues past 0x80 into the encrypted region, so
/// `stevenson::crack_title_key` recovers the key. Distinct `seed` values give
/// `keyless::crack_title_key` recovers the key. Distinct `seed` values give
/// two sectors different cribs, standing in for two VOB regions.
fn crackable_css_sector(title_key: &[u8; 5], seed: &[u8; 5]) -> Vec<u8> {
const RUN_START: usize = 0x59;
@@ -1157,12 +1157,12 @@ mod tests {
// Precondition: each sector must be crackable on its own (the rekey
// depends on it). If this fails the fixture, not the path, is at fault.
assert_eq!(
crate::css::stevenson::crack_title_key(&s0),
crate::css::keyless::crack_title_key(&s0),
Some(key_a),
"fixture s0 must crack to key_a standalone"
);
assert_eq!(
crate::css::stevenson::crack_title_key(&s1),
crate::css::keyless::crack_title_key(&s1),
Some(key_b),
"fixture s1 must crack to key_b standalone"
);
@@ -1899,11 +1899,24 @@ mod tests {
/// below its own range start, or a zero unit size, are both map bugs — they
/// must return a defined answer rather than panicking on debug overflow or
/// dividing by zero.
///
/// Every case here asserts the DEFINED answer, not merely the absence of a
/// panic. The zero-unit-size case used to be written
/// `assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
/// accepts both answers and so pinned nothing at all: the guards could
/// invert and it would still pass. The answer is knowable —
/// `saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
/// is even — so pin it.
#[test]
fn phase_gate_does_not_panic_on_a_malformed_map() {
use super::{Phase, unit_is_our_phase};
// Unit below its own range start: saturating_sub clamps to 0, and unit
// 0 is even.
assert!(unit_is_our_phase(10, 100, 3, Phase::Even));
assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true);
// Zero unit size: max(1) makes the divisor 1, so the index is the raw
// offset 70 — even.
assert!(unit_is_our_phase(100, 30, 0, Phase::Even));
// Both malformations at once: offset 0 over divisor 1 is unit 0, even.
assert!(unit_is_our_phase(5, 5, 0, Phase::Even));
}
}
+29 -7
View File
@@ -472,14 +472,33 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!(
target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason={}",
size={}B clips={} video={} reason={}",
main.playlist,
main.duration_secs,
main.size_bytes,
main.clips.len(),
main.video_streams().count(),
main_feature_reason(),
);
}
// Titles disqualified from main-feature candidacy for carrying no video —
// the streamless obfuscation decoys the `has-video` gate demotes. Logged so
// a field bug report shows WHY a long/large playlist was not picked (the
// gap that let an undiagnosable E6008 loop on a real UHD title's decoy
// playlist `00245`).
for (ti, t) in disc.titles.iter().enumerate() {
if !t.has_video() {
tracing::debug!(
target: DIAG,
"tag=decision.demoted title_idx={} playlist={:?} dur={:.1}s size={}B reason=no-video",
ti,
t.playlist,
t.duration_secs,
t.size_bytes,
);
}
}
}
/// The `reason=` token on the main-feature decision row.
@@ -491,10 +510,12 @@ pub fn dump_disc(disc: &Disc) {
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
let keys: Vec<&str> = Disc::MAIN_FEATURE_ORDER_KEYS
.iter()
.chain(Disc::CANONICAL_TITLE_ORDER_KEYS.iter())
.copied()
.collect();
format!("main_feature_order({})", keys.join(", "))
}
fn dump_aacs(disc: &Disc) {
@@ -670,8 +691,9 @@ mod tests {
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
reason,
"main_feature_order(nav-feature, authoring-feature, standalone, has-video, fits-disc, largest-size, longest, richest-audio)",
"the reason must name the selection keys (authoring, standalone-over-composite, and has-video gates, then the physical keys) in priority order"
);
}
+44 -9
View File
@@ -17,18 +17,31 @@ struct Scratch(PathBuf);
impl Scratch {
fn new(tag: &str) -> Self {
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{:?}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
let p = Self::unique_path(tag);
std::fs::create_dir_all(&p).unwrap();
Self(p)
}
/// The path a scratch dir takes — kept separate from directory creation so
/// its uniqueness is testable without a syscall between draws. A monotonic
/// counter, NOT a timestamp, is what guarantees it: `SystemTime::now()`
/// resolves to only a MICROSECOND on macOS, so two of the seven parallel
/// tests that share the "bdmv" tag routinely read the same value within one
/// microsecond, collide on the same directory, and the first to finish
/// `remove_dir_all`s it out from under the others' reads — an intermittent
/// `read_sectors` ENOENT. The counter is unique regardless of clock
/// granularity; pairing it with the pid keeps it unique across test-binary
/// processes.
fn unique_path(tag: &str) -> PathBuf {
static SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let uniq = SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{uniq}",
std::process::id()
));
p
}
fn path(&self) -> &Path {
&self.0
}
@@ -69,6 +82,28 @@ fn bdmv_scratch() -> (Scratch, Vec<u8>, Vec<u8>) {
(s, index, clip)
}
/// Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
/// tag and run in parallel; if two land the same path, the first to drop
/// `remove_dir_all`s it out from under the other's reads, which surfaced as an
/// intermittent `read_sectors` failure. Regression: with the old
/// `SystemTime::now()` name this is RED, because macOS's clock advances only
/// per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
/// between them to nudge the clock, mirroring the real cross-thread collision —
/// hands back the same value many times over. The counter-based name is unique
/// regardless of clock granularity.
#[test]
fn scratch_paths_are_unique_even_at_clock_resolution() {
let paths: Vec<PathBuf> = (0..1000).map(|_| Scratch::unique_path("uniq")).collect();
let distinct: std::collections::HashSet<&PathBuf> = paths.iter().collect();
assert_eq!(
distinct.len(),
paths.len(),
"every Scratch must own a distinct path; a timestamp-based name collides \
under macOS's microsecond clock and lets one parallel test delete \
another's files mid-read"
);
}
// ── The de-risking spike ────────────────────────────────────────────────────
/// THE load-bearing assertion of the whole design: metadata synthesized here
+908 -120
View File
File diff suppressed because it is too large Load Diff
+300 -26
View File
@@ -7,15 +7,50 @@ use crate::udf;
impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
///
/// Returns the scanned titles plus, when the disc's own First-Play
/// navigation deterministically dispatches to a title, that title's
/// `playlist_id` — the nav-resolved main feature. The caller feeds it into
/// `sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the
/// BD path uses. It is `None` (→ existing ranking heuristics stand) whenever
/// the navigation is menu-only, malformed, or does not converge.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Vec<DiscTitle> {
halt: Option<&crate::halt::Halt>,
) -> Result<(Vec<DiscTitle>, Option<u16>)> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(_) => return Vec::new(),
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok((Vec::new(), None)),
};
// Follow the disc's First-Play navigation the way a player would, to see
// which title it selects as the feature (issue #40). Read-only, and any
// parse error / interactive-menu entry / non-convergence returns `None`
// — a strict improvement: it never removes a title, only re-ranks toward
// the nav-chosen one. The `(vtsn, vts_ttn)` it yields is matched to the
// scanned title below to recover its `playlist_id`.
let nav_target = crate::dvdnav::resolve_main_title(reader, udf_fs);
let mut nav_feature: Option<u16> = None;
let mut titles = Vec::new();
let mut title_number: u16 = 0;
@@ -109,6 +144,17 @@ impl Disc {
for (vts_title_idx, dvd_title) in ts.titles.iter().enumerate() {
title_number += 1;
// If the nav executor picked this title (identified by its VTS
// number and 1-based position within the set — the same
// `vts_ttn` convention used below), remember its global
// `playlist_id` so the caller can promote it in ranking.
if let Some(rt) = nav_target
&& rt.vtsn == ts.vts_number
&& rt.vts_ttn as usize == vts_title_idx + 1
{
nav_feature = Some(title_number);
}
// Diagnostic dump (--log-level 3): per-cell category table +
// chapter map for this title, BEFORE lowering drops the
// per-cell IFO detail. No-op unless freemkv::diag is enabled.
@@ -116,7 +162,7 @@ impl Disc {
// Feature start cell. Prefer the DVD nav-VM resolver, which
// PARKED (#40, menu-at-start playback). The "menu at the start"
// symptom (e.g. SOTL) was a sector-mapping fault — the absolute
// symptom (seen on a real disc) was a sector-mapping fault — the absolute
// VOB rebase in `ifo::parse_vts` (`vob_start_sector =
// file_start_lba + vtstt_vobs`) — NOT a navigation problem, so
// feature-start resolution is unnecessary for correct rips. The
@@ -243,7 +289,14 @@ impl Disc {
}
}
titles
// Polled again AFTER the loop: a cancel raised during the IFO reads
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok((titles, nav_feature))
}
}
@@ -528,6 +581,119 @@ mod tests {
// Tests
// ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|(ts, _)| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|(ts, _)| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test]
@@ -535,7 +701,12 @@ mod tests {
let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]);
assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0
.is_empty()
);
}
/// Single VTS, single title, one cell. Extent absolute LBA =
@@ -571,7 +742,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -597,7 +770,7 @@ mod tests {
/// before the title VOB. Cell `first_sector` values are relative to
/// `vtstt_vobs`; reading 0xC0 prepended the menu and shifted every extent
/// back by `vtstt_vobs - vtsm_vobs`, so the rip opened on the parental
/// prompt instead of the movie (Greenland NTSC R1: vtsm=44, vtstt=3640).
/// prompt instead of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
///
/// Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); we additionally stamp
/// a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve from 3640.
@@ -627,7 +800,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -647,7 +822,7 @@ mod tests {
);
}
/// ABSOLUTE-REBASE regression (THESILENCEOFTHELAMBS / Greenland fix):
/// ABSOLUTE-REBASE regression (real-disc menu-at-start fix):
/// `ifo::parse_vts` now sets `vob_start_sector = file_start_lba(IFO) +
/// vtstt_vobs`, so an extent's `start_lba` must equal the sum of THREE
/// independent terms — the IFO file's absolute on-disc LBA, the
@@ -685,7 +860,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -743,7 +920,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -784,7 +963,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let v = t
.streams
.iter()
@@ -839,7 +1020,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let v = t
.streams
.iter()
@@ -900,7 +1083,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -911,7 +1096,7 @@ mod tests {
.collect();
assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "en");
assert_eq!(audios[0].language, "eng");
assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -970,7 +1155,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1027,7 +1214,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1040,7 +1229,7 @@ mod tests {
// Languages preserved in order.
assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["en", "fr", "de"]
vec!["eng", "fra", "deu"]
);
// PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1087,7 +1276,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
let sub = t
.streams
.iter()
@@ -1097,7 +1288,7 @@ mod tests {
})
.expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "en");
assert_eq!(sub.language, "eng");
assert!(
sub.codec_data.is_some(),
"non-zero palette must yield codec_data"
@@ -1141,7 +1332,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1154,6 +1347,79 @@ mod tests {
assert_eq!(titles[1].extents[0].start_lba, 10200); // 10000 + 200
}
/// Stamp a First-Play PGC into a VMG whose pre-command list is a single
/// unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
/// title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
/// TT_SRPT pointers `build_vmg` already writes.
fn stamp_first_play_jumptt(vmg: &mut [u8], ttn: u8) {
const FP_PGC_PTR: usize = 0x84; // u32 byte offset of the First-Play PGC
const CMD_TBL_PTR: usize = 0xE4; // u16 cmd-table offset (rel to PGC)
let fp_pgc: u32 = 0x200;
let cmd_tbl_rel: u16 = 0x100;
vmg[FP_PGC_PTR..FP_PGC_PTR + 4].copy_from_slice(&fp_pgc.to_be_bytes());
let pgc = fp_pgc as usize;
vmg[pgc + CMD_TBL_PTR..pgc + CMD_TBL_PTR + 2].copy_from_slice(&cmd_tbl_rel.to_be_bytes());
let tbl = pgc + cmd_tbl_rel as usize;
vmg[tbl..tbl + 2].copy_from_slice(&1u16.to_be_bytes()); // nr_of_pre = 1
// JumpTT command (type 1, direct=1, cmd=2): ttn in byte 5.
vmg[tbl + 8..tbl + 16].copy_from_slice(&[0x30, 0x02, 0, 0, 0, ttn, 0, 0]);
}
/// The DVD First-Play nav promotion branch (issue #40): when
/// `resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
/// must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
/// the matching `vts_number` — to the running `title_number`, and surface it
/// as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
/// the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
/// the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
/// join or a vtsn/vts_ttn field swap would promote the wrong title or none.
#[test]
fn scan_dvd_titles_nav_promotes_first_play_target() {
let mut disc = MemDisc::new();
// TT_SRPT: title 1 → VTS 1 (in-set title 1); title 2 → VTS 2 (in-set 1).
let mut vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
stamp_first_play_jumptt(&mut vmg, 2); // First-Play → JumpTT title 2
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
let (titles, nav_feature) = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// Sanity: the resolver reached the branch with the VTS-2 target.
assert_eq!(
crate::dvdnav::resolve_main_title(&mut disc, &udf),
Some(crate::dvdnav::nav::ResolvedTitle {
title: 2,
vtsn: 2,
vts_ttn: 1,
})
);
// The promotion mapped that target to the second scanned title.
assert_eq!(nav_feature, Some(2));
assert_eq!(titles[1].playlist_id, 2);
}
/// chapter_times from the IFO become Chapter entries with ordinal
/// names (dvd.rs maps chapter_times → Chapter{time_secs, chapter_name}).
#[test]
@@ -1178,7 +1444,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1263,7 +1531,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1287,7 +1557,7 @@ mod tests {
/// Conservative guard end-to-end: a normal feature (every cell category
/// 0x00) is muxed in full — the filter drops nothing and chapters are
/// unshifted. This is the "Silence of the Lambs" case.
/// unshifted. This is the plain-single-angle-feature case.
#[test]
fn scan_dvd_titles_plain_feature_untouched() {
let mut disc = MemDisc::new();
@@ -1315,7 +1585,9 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1358,7 +1630,9 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.0;
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
+7 -7
View File
@@ -1,6 +1,6 @@
//! Physical AC-3 sub-stream probing for DVD audio routing.
//!
//! ## Why this exists (Silence-of-the-Lambs wrong-substream bug)
//! ## Why this exists (a real-disc wrong-substream bug)
//!
//! A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
//! scan assigns each declared stream a `private_stream_1` sub-stream id purely
@@ -8,7 +8,7 @@
//! `0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
//! physical sub-stream order on the wire matches the IFO declaration order.
//!
//! On some discs it does NOT. The R2 PAL "The Silence of the Lambs" feature
//! On some discs it does NOT. A real R2 PAL DVD release's feature
//! declares ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but
//! the physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
//! `0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
@@ -34,7 +34,7 @@ use std::collections::BTreeMap;
/// a DVD feature opens with logos/warnings whose audio is frequently a thin 2.0
/// bed on the FIRST sub-stream only — the other physical `0x8x` sub-streams and
/// the main 5.1 mix do not appear until a sector or two further in. 512 sectors
/// (1 MiB) was too short: on Greenland it saw ONLY `0x80`, and only its opening
/// (1 MiB) was too short: on one real disc it saw ONLY `0x80`, and only its opening
/// 2.0 frames. 1024 sectors (2 MiB) reliably contains at least one frame of
/// every physical AC-3 sub-stream AND enough of `0x80` to reach its 5.1 frames.
/// Still bounded so a live drive is never hammered (see the project "don't
@@ -54,7 +54,7 @@ const PROBE_SECTORS: u16 = 1024;
/// The first frame of a sub-stream at the head of a feature is NOT
/// representative. A DVD opens with logos/warnings, and the main `0x80`
/// sub-stream there frequently carries a thin 2.0 bed before transitioning to
/// its real 5.1 main mix a fraction of a second later (observed on Greenland:
/// its real 5.1 main mix a fraction of a second later (observed on a real disc:
/// `0x80`'s first frames are acmod=2 → 2 channels, then it becomes acmod=7+lfe →
/// 6 channels within the same 2 MiB window). Recording only the FIRST frame read
/// `0x80=2` and missed the 5.1 entirely, defeating the channel-match routing.
@@ -347,7 +347,7 @@ mod tests {
assert_eq!(probed.get(&0x81), Some(&6), "0x81 is the 5.1 main mix");
}
/// GREENLAND regression — the probe must read each sub-stream's TRUE
/// Real-disc regression — the probe must read each sub-stream's TRUE
/// (max-mix) channel count, not be poisoned by an unrepresentative head
/// frame, and must NOT cross-contaminate between sub-streams.
///
@@ -590,7 +590,7 @@ mod tests {
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// End-to-end `probe_and_remap`: a real-disc-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
@@ -603,7 +603,7 @@ mod tests {
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
fn probe_and_remap_reroutes_swapped_substream_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
+265 -31
View File
@@ -107,8 +107,10 @@ struct PlannedFile {
size: u64,
/// Inline (ICB-embedded) data, if any. When `Some`, `extents` is empty.
inline: Option<Vec<u8>>,
/// Absolute disc extents `(abs_lba, byte_len)`.
extents: Vec<(u32, u32)>,
/// Absolute disc extents, each carrying whether it was ever RECORDED (an
/// ECMA-167 4/14.14.1.1 type-1 extent is allocated but not recorded: it
/// occupies the file's byte space and its contents are zeros).
extents: Vec<crate::udf::AbsExtent>,
}
impl Disc {
@@ -144,7 +146,7 @@ impl Disc {
let fs = udf::read_filesystem(reader)?;
let mut planned: Vec<PlannedFile> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
let mut seen_hosts: std::collections::HashMap<PathBuf, String> =
let mut seen_hosts: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
plan_tree(
reader,
@@ -235,6 +237,9 @@ impl Disc {
let mut result = ExtractResult::default();
let total_bytes = required;
let mut done_bytes: u64 = 0;
// Cumulative zero-filled-unreadable bytes, so the live progress channel
// can report the good/unreadable split instead of pinning unreadable at 0.
let mut done_unreadable: u64 = 0;
// CSS per-VTS key cache; only consulted for CSS discs.
let is_css = matches!(base_keys, DecryptKeys::Css { .. });
@@ -269,8 +274,15 @@ impl Disc {
// non-tolerate), so extract_one_file already zero-filled it and
// counted it in bytes_unreadable — one 'lost' bucket covers both
// media damage and decrypt failure.
let (fr, halted) =
extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?;
let (fr, halted) = extract_one_file(
&mut dec,
dest,
pf,
total_bytes,
&mut done_bytes,
&mut done_unreadable,
opts,
)?;
result.bytes_good = result.bytes_good.saturating_add(fr.bytes_good);
result.bytes_unreadable = result.bytes_unreadable.saturating_add(fr.bytes_unreadable);
@@ -287,8 +299,8 @@ impl Disc {
Ok(result)
}
/// Resolve the CSS title key for a VTS group by cracking from its title
/// VOB extents (keyless Stevenson attack). The disc-wide key is reused when
/// Resolve the CSS title key for a VTS group by recovering it from the
/// group's scrambled title-VOB sectors. The disc-wide key is reused when
/// it already covers this VTS (single-VTS discs, or this VTS's span). The
/// reader is borrowed from the decrypting decorator (its inner source).
///
@@ -339,10 +351,14 @@ impl Disc {
let mut extents: Vec<crate::disc::Extent> = Vec::new();
for pf in files {
for &(abs_lba, byte_len) in &pf.extents {
// Unrecorded extents hold no bytes the VOB ever wrote, so they can
// carry no scrambled sector for the crack to work from — feeding
// them in spends the shared sector budget on media that is not part
// of the title.
for ext in pf.extents.iter().filter(|e| e.recorded) {
extents.push(crate::disc::Extent {
start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
start_lba: ext.lba,
sector_count: (ext.len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
});
}
}
@@ -379,9 +395,22 @@ impl Disc {
crate::css::CrackOutcome::Unencrypted => Ok(base_keys.clone()),
// Scrambled sectors WERE seen and no key came out. Reusing the
// disc-wide key here writes corrupt PES behind an intact header and
// reports a complete extract at exit 0. Skippable per title, which
// is why this is the per-title code and not the disc-level one — a
// sibling VTS may still crack.
// reports a complete extract at exit 0.
//
// `CssKeyMissing` is the per-TITLE code rather than the disc-level
// `CssNoDiscKey` because the two are treated differently elsewhere:
// it is what `error::is_skippable_title_stub` matches, and the MUX
// path really does skip a title on it and carry on with the rest of
// the disc.
//
// EXTRACT does not skip. This error is `?`-propagated by the only
// caller, out of the loop over every planned file, so one
// uncrackable VTS aborts the whole extract — including files from
// sibling VTS groups that would have cracked. That is deliberate:
// aborting loudly is the safe failure here, and pinned by
// `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`.
// Do not read the code choice as a promise that extraction
// continues past this point.
crate::css::CrackOutcome::ScrambledUncracked => Err(Error::CssKeyMissing),
}
}
@@ -429,7 +458,7 @@ fn plan_tree(
is_root: bool,
files: &mut Vec<PlannedFile>,
dirs: &mut Vec<PathBuf>,
seen_hosts: &mut std::collections::HashMap<PathBuf, String>,
seen_hosts: &mut std::collections::HashMap<String, String>,
) -> Result<()> {
for entry in &dir.entries {
if entry.name.is_empty() {
@@ -447,13 +476,48 @@ fn plan_tree(
let safe = sanitize_component(&entry.name)?;
let child_rel = host_rel.join(&safe);
let child_disc = format!("{disc_path}/{}", entry.name);
// Collision: two distinct disc paths → same host path.
if let Some(prev) = seen_hosts.insert(child_rel.clone(), child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: child_rel.to_string_lossy().into_owned(),
});
// Collision: two distinct disc paths → same host FILE. The key must
// model the HOST's namespace, not the disc's, and the two differ twice
// over:
//
// * CASE. macOS APFS and Windows NTFS are case-insensitive by default,
// so `Movie` and `movie` are one file there. Keyed by the
// case-preserving path they were two entries, nothing collided, and
// the second extraction overwrote the first with both files still
// reported `complete: true`.
// * The `.partial` SUFFIX. `extract_one_file` streams through
// `<final>.partial` before renaming, so a file's temp path lives in
// the same namespace as every other file's final path: a disc
// holding both `X` and `X.partial` planned two distinct final names,
// and extracting `X` then truncated the real `X.partial`.
//
// Only files get the `.partial` alias — directories are created
// directly and never stream through a temp name — but it is checked
// against every entry's primary key, so a directory `X.partial` beside
// a file `X` is caught too. The alias stores the OWNING file's disc
// path, so the `prev != child_disc` test keeps its meaning: a repeated
// identical disc path is not a collision, two different ones are.
//
// The fold is `to_lowercase`, which closes the ASCII and simple-Unicode
// case classes. It is NOT full case folding, and it does NOT normalize:
// APFS also unifies NFC/NFD, so `é` recorded as U+00E9 and as
// `e`+U+0301 remain two keys here and one file there. That residual is
// a false NEGATIVE (a missed collision, never a spurious one), so this
// is a strict improvement rather than a complete model of the host.
let mut register = |key: PathBuf| -> Result<()> {
let folded = key.to_string_lossy().to_lowercase();
if let Some(prev) = seen_hosts.insert(folded, child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: key.to_string_lossy().into_owned(),
});
}
Ok(())
};
register(child_rel.clone())?;
if !entry.is_dir {
register(with_partial_suffix(&child_rel))?;
}
if entry.is_dir {
dirs.push(child_rel.clone());
@@ -499,6 +563,7 @@ fn extract_one_file<S: SectorSource>(
pf: &PlannedFile,
total_bytes: u64,
done_bytes: &mut u64,
done_unreadable: &mut u64,
opts: &ExtractOptions,
) -> Result<(FileResult, bool)> {
let final_path = dest.join(&pf.host_rel);
@@ -528,16 +593,53 @@ fn extract_one_file<S: SectorSource>(
finalize_file(writer, &partial_path, pf.size, &final_path)?;
fr.complete = true;
*done_bytes = done_bytes.saturating_add(pf.size);
report(opts, *done_bytes, total_bytes);
report(opts, *done_bytes, *done_unreadable, total_bytes);
return Ok((fr, false));
}
let mut written: u64 = 0;
let mut buf = vec![0u8; READ_BATCH_SECTORS as usize * SECTOR_BYTES];
'extents: for &(abs_lba, byte_len) in &pf.extents {
'extents: for &crate::udf::AbsExtent {
lba: abs_lba,
len: byte_len,
recorded,
} in &pf.extents
{
if written >= pf.size {
break;
}
// ECMA-167 4/14.14.1.1 type 1: allocated but NOT recorded. The extent
// is part of the file's byte space and its contents are defined to be
// zeros, so write the zeros WITHOUT reading the media — those sectors
// hold nothing this file ever wrote (on an AACS disc, ciphertext that
// decrypts to noise). This mirrors `UdfFs::read_file_limited`, which
// takes the same decision from the same flag. Skipping the extent
// entirely instead would slide every later extent's bytes down by the
// hole's length.
if !recorded {
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64);
let hole_bytes = (sectors * SECTOR_BYTES_U64).min(pf.size.saturating_sub(written));
let mut left = hole_bytes;
for b in buf.iter_mut() {
*b = 0;
}
while left > 0 {
let n = left.min(buf.len() as u64) as usize;
write_all(&mut writer, &buf[..n], &partial_path)?;
written = written.saturating_add(n as u64);
*done_bytes = done_bytes.saturating_add(n as u64);
left -= n as u64;
}
fr.bytes_good = fr.bytes_good.saturating_add(hole_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
if opts.cancelled(cont) {
return Ok((fr, true));
}
if written >= pf.size {
break 'extents;
}
continue;
}
// Anchor AACS unit alignment at THIS extent's start (clip-anchored
// gate, not absolute LBA 0 and NOT the file's first extent). A file
// may span multiple extents (fragmented / Long-AD / continuation ICB
@@ -577,10 +679,11 @@ fn extract_one_file<S: SectorSource>(
}
write_all(&mut writer, &buf[..usable], &partial_path)?;
fr.bytes_unreadable = fr.bytes_unreadable.saturating_add(usable as u64);
*done_unreadable = done_unreadable.saturating_add(usable as u64);
}
written = written.saturating_add(usable as u64);
*done_bytes = done_bytes.saturating_add(usable as u64);
let cont = report(opts, *done_bytes, total_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
sector_off += batch;
if opts.cancelled(cont) {
// Leave the `.partial`; do NOT rename. The aggregate run
@@ -693,15 +796,20 @@ fn finalize_file(
/// Emit a progress report. Returns `true` to continue, `false` if the sink
/// requested an early stop (or there is no sink — always continue).
fn report(opts: &ExtractOptions, done: u64, total: u64) -> bool {
fn report(opts: &ExtractOptions, done: u64, unreadable: u64, total: u64) -> bool {
match opts.progress {
Some(p) => {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Mux,
work_done: done,
work_total: total,
bytes_good_total: done,
bytes_unreadable_total: 0,
// `done` counts good AND zero-filled-unreadable bytes together;
// split them so a consumer driven only by the live progress
// channel sees a holed extraction as holed rather than as a clean
// climb to 100%. The final ExtractResult already carries the true
// split — this used to pin unreadable at 0 and call every byte good.
bytes_good_total: done.saturating_sub(unreadable),
bytes_unreadable_total: unreadable,
bytes_pending_total: 0,
bytes_retryable_total: 0,
bytes_total_disc: total,
@@ -1185,6 +1293,17 @@ mod tests {
s
}
/// Build a file ICB whose FIRST short AD is an ECMA-167 4/14.14.1.1 type-1
/// (allocated, NOT recorded) extent and whose second is ordinary recorded
/// data. Both are `sectors_each` sectors long.
fn build_hole_then_data_icb(sectors_each: u32, hole_lba: u32, data_lba: u32) -> [u8; 2048] {
let mut s = build_two_extent_icb(sectors_each, hole_lba, data_lba);
let len = sectors_each * SECTOR_BYTES as u32;
// Re-stamp AD #0 with extent type 1 in bits 30..31 of the length field.
s[216..220].copy_from_slice(&(0x4000_0000u32 | (len & 0x3FFF_FFFF)).to_le_bytes());
s
}
/// Encrypt the clear unit from `clear_aacs_unit(tag)` under `unit_key` so
/// `aacs::content::decrypt_unit` recovers it cleanly (zero decrypt loss).
/// `tag` distinguishes two units' payloads.
@@ -1386,7 +1505,8 @@ mod tests {
}
/// A CSS-scrambled title VOB is descrambled on extraction: the producer
/// cracks the per-VTS key (keyless Stevenson) and the output VOB is plain.
/// recovers the per-VTS key from the scrambled sectors and the output VOB
/// is plain.
#[test]
fn css_title_vob_is_descrambled() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
@@ -1598,6 +1718,62 @@ mod tests {
assert!(matches!(err, Error::DirNameCollision { .. }));
}
/// Two disc names differing only by CASE are one host file on macOS APFS
/// and Windows NTFS, both case-insensitive by default. Keyed by the
/// case-preserving path, the collision map sees two entries, raises
/// nothing, and the second file extracted overwrites the first — while both
/// `PlannedFile`s report `complete: true`. Silent data loss reported as a
/// clean extract is the one outcome this crate must never produce, so the
/// host-equivalence key has to model the host's namespace, not the disc's.
#[test]
fn names_differing_only_by_case_are_a_collision() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("Movie", 30, 31, b"a".to_vec(), false),
file("movie", 32, 33, b"b".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("case_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err("two names that fold to one host file must collide");
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A file's in-flight `.partial` path shares the host namespace with every
/// other planned file's FINAL path. A disc carrying both `X` and
/// `X.partial` plans two distinct final names, so nothing collides — but
/// extracting `X` writes through `X.partial`, the same host path the other
/// file owns. Whichever lands second truncates the other, and both entries
/// are still reported complete.
#[test]
fn a_files_partial_path_colliding_with_another_files_final_name_is_an_error() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("X", 30, 31, b"aaaa".to_vec(), false),
file("X.partial", 32, 33, b"bbbb".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("partial_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err(
"X's temp path IS X.partial's final path — one host file for two \
disc files, so it must be refused up front",
);
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A non-empty target dir is refused without `--force`, and accepted with.
#[test]
fn non_empty_target_requires_force() {
@@ -1727,6 +1903,63 @@ mod tests {
);
}
/// An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED:
/// the space belongs to the file and occupies its byte range, but nothing
/// was ever written there, and the standard defines its contents as zeros.
/// `read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
/// `read_file_limited` honours it by emitting zeros WITHOUT touching the
/// media. The tree extractor reads the same ICBs and must agree: reading
/// those sectors returns whatever the media happens to hold there — on an
/// AACS disc, ciphertext that decrypts to noise — and writes it into the
/// extracted file as if the disc had recorded it.
///
/// The hole here is filled with a recognisable non-zero pattern, so
/// "emitted zeros" and "read the media" are told apart by CONTENT.
#[test]
fn extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it() {
const SECTORS_EACH: u32 = 1;
const HOLE: u32 = 5000;
const DATA: u32 = 5004;
let hole_bytes = vec![0xEEu8; SECTOR_BYTES];
let data_bytes = vec![0x5Au8; SECTOR_BYTES];
// The file's byte space: the hole's zeros FIRST, then the real data.
let mut expect = vec![0u8; SECTOR_BYTES];
expect.extend_from_slice(&data_bytes);
let mut disc = MemDisc::new();
build_udf_skeleton(&mut disc, 10);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "INDEX.BDMV", 42, false, false);
disc.put(
PART_START + 42,
build_hole_then_data_icb(SECTORS_EACH, HOLE, DATA),
);
disc.put_bytes(PART_START + HOLE, &hole_bytes);
disc.put_bytes(PART_START + DATA, &data_bytes);
disc.put(PART_START + 10, build_dir_icb(11, root_fids.len() as u32));
disc.put_bytes(PART_START + 11, &root_fids);
let out = TmpDir::new("unrecorded_extent");
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract");
let got = read_out(out.path(), "INDEX.BDMV").expect("file written");
assert_eq!(
got, expect,
"an unrecorded extent contributes zeros to the file, never the \
bytes that happen to sit on those sectors"
);
assert_eq!(
res.bytes_unreadable, 0,
"a hole is not a read failure — nothing was attempted"
);
assert!(res.complete);
}
/// Focused alignment-computation check underpinning the per-extent fix:
/// when each extent anchors its OWN start as the unit base, the extent's
/// own batch starts are always unit-aligned; anchoring a later extent
@@ -1862,8 +2095,8 @@ mod tests {
let seed = [0x11u8, 0x22, 0x33, 0x44, marker];
let mut plain = vec![0u8; 2048];
// The crack scan's hardened `is_scrambled_pack` gate requires the
// MPEG-PS pack-start signature before it will even ATTEMPT a
// Stevenson crack (see `css::is_scrambled_pack`) -- without it,
// MPEG-PS pack-start signature before it will even ATTEMPT a key
// crack (see `css::is_scrambled_pack`) -- without it,
// `resolve_vts_key` silently falls back to `base_keys` for BOTH
// VTS groups regardless of which extents were gathered, masking
// this exact regression.
@@ -1971,7 +2204,8 @@ mod tests {
};
// Scrambled — the pack header and the 0x14 flag are set, so the scan
// SEES ciphertext — but the cleartext region carries no periodic run,
// so the Stevenson crib never forms and no key is recoverable.
// so the repeating crib the key recovery relies on never forms and no
// key is recoverable.
let uncrackable = {
let mut sect = vec![0u8; 2048];
sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
+1781 -59
View File
File diff suppressed because it is too large Load Diff
+1014 -32
View File
File diff suppressed because it is too large Load Diff
+505
View File
@@ -0,0 +1,505 @@
//! Normalized, format-agnostic disc profile — a thin typed VIEW over the scanned
//! [`Disc`] model.
//!
//! The scan already normalizes every source (DVD/BD/UHD/HD-DVD) into the same
//! [`Disc`] / [`DiscTitle`] / [`Stream`] shapes, carrying language / forced /
//! qualifier / purpose / secondary / label per stream. This module hoists those
//! into a flat, serde-friendly surface so a consumer reads
//! `profile.titles[i].subtitles[j].forced` (and the analogous audio/video flags)
//! for ANY disc with no per-format conditionals.
//!
//! Everything here is DERIVED from the model at construction: the enum `Stream`
//! is split into three typed vectors, the `qualifier` / `purpose` enums are
//! decomposed into booleans, and the "default track" selection is precomputed so
//! downstream never recomputes it. Every field is always populated — a sensible
//! default (`"und"` language, empty `name`, `false` flag) stands in where the
//! model has nothing, so consumers never handle a bare `Option`.
use serde::{Deserialize, Serialize};
use super::{
AudioStream, Disc, DiscFormat, DiscTitle, LabelPurpose, LabelQualifier, Stream, SubtitleStream,
VideoStream,
};
/// A disc's complete normalized profile: identity plus every title's typed
/// track breakdown, with the main-feature selection hoisted to `main_title`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct DiscProfile {
/// Disc container family (`"bluray"`, `"uhd"`, `"dvd"`, `"hddvd"`, …).
pub format: String,
/// Best available display name (disc metadata title, else volume id).
pub disc_name: String,
/// Stable disc identifier (the UDF volume identifier).
pub disc_id: String,
/// Index into `titles` of the selected main feature. The scan pre-sorts
/// titles so `titles[0]` is the main feature, so this is `0` whenever any
/// title exists.
pub main_title: usize,
/// Every title, in the scan's main-feature order.
pub titles: Vec<TitleProfile>,
}
/// One title's normalized profile: identity, duration/size, chapter count, the
/// main-feature flag, and its streams split into typed per-kind vectors.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct TitleProfile {
/// Position of this title within [`DiscProfile::titles`].
pub index: usize,
/// Playlist / program identifier (e.g. `"00800.mpls"`).
pub playlist: String,
/// Duration in seconds.
pub duration_secs: f64,
/// Total size in bytes.
pub size_bytes: u64,
/// Number of chapter points.
pub chapters: usize,
/// Whether this is the disc's main feature (`index == main_title`).
pub is_main: bool,
/// Video tracks, in declared order.
pub video: Vec<VideoTrack>,
/// Audio tracks, in declared order.
pub audio: Vec<AudioTrack>,
/// Subtitle tracks, in declared order.
pub subtitles: Vec<SubtitleTrack>,
}
/// A normalized video track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct VideoTrack {
/// Compact codec id (e.g. `"hevc"`, `"h264"`, `"vc1"`, `"mpeg2"`).
pub codec: String,
/// Resolution label (e.g. `"2160p"`, `"1080p"`, `"576i"`).
pub resolution: String,
/// HDR format id (e.g. `"sdr"`, `"hdr10"`, `"hdr10+"`, `"dv"`, `"hlg"`).
pub hdr: String,
/// Frame-rate label (e.g. `"23.976"`, `"25"`).
pub frame_rate: String,
/// Whether this is the title's default video track (first non-secondary).
pub default: bool,
/// Whether this is a secondary stream (PiP / dependent view).
pub secondary: bool,
}
/// A normalized audio track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct AudioTrack {
/// ISO 639-2 language code, `"und"` when the source stated none.
pub language: String,
/// Compact codec id (e.g. `"truehd"`, `"dtshd_ma"`, `"ac3"`).
pub codec: String,
/// Channel layout label (e.g. `"stereo"`, `"5.1"`, `"unknown"`).
pub channels: String,
/// Whether this is the title's default audio track (first non-secondary).
pub default: bool,
/// Editorial purpose flag: commentary track.
pub commentary: bool,
/// Editorial purpose flag: descriptive / audio-description track.
pub descriptive: bool,
/// Codec / variant text label; empty when the source stated none.
pub name: String,
}
/// A normalized subtitle track.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SubtitleTrack {
/// ISO 639-2 language code, `"und"` when the source stated none.
pub language: String,
/// Compact codec id (e.g. `"pgs"`, `"dvdsub"`).
pub codec: String,
/// Forced-narrative flag.
pub forced: bool,
/// SDH (subtitles for the deaf / hard-of-hearing) flag.
pub sdh: bool,
/// Whether this is the title's default subtitle track. Always `false`: the
/// mux never marks a subtitle default, and the normalized view mirrors that.
pub default: bool,
/// Track name; empty (the subtitle model carries no free-text label).
pub name: String,
}
/// Container family as a compact, stable id. Kept here (not on [`DiscFormat`])
/// because it is a serialization concern of this view.
fn format_id(format: DiscFormat) -> &'static str {
match format {
DiscFormat::Uhd => "uhd",
DiscFormat::Fmts => "uhd_fmts",
DiscFormat::BluRay => "bluray",
DiscFormat::HdDvd => "hddvd",
DiscFormat::Dvd => "dvd",
DiscFormat::Unknown => "unknown",
}
}
/// `"und"` for an empty language code, else the code unchanged. Mirrors the
/// muxer's `language_or_und`: the one place a source with no language table has
/// to fall back to the ISO 639-2 "undetermined" code.
fn language_or_und(lang: &str) -> String {
if lang.is_empty() {
"und".to_string()
} else {
lang.to_string()
}
}
impl VideoTrack {
fn from_stream(v: &VideoStream, default: bool) -> Self {
Self {
codec: v.codec.id().to_string(),
resolution: v.resolution.to_string(),
hdr: v.hdr.id().to_string(),
frame_rate: v.frame_rate.to_string(),
default,
secondary: v.secondary,
}
}
}
impl AudioTrack {
fn from_stream(a: &AudioStream, default: bool) -> Self {
Self {
language: language_or_und(&a.language),
codec: a.codec.id().to_string(),
channels: a.channels.to_string(),
default,
commentary: a.purpose == LabelPurpose::Commentary,
descriptive: a.purpose == LabelPurpose::Descriptive,
name: a.label.clone(),
}
}
}
impl SubtitleTrack {
fn from_stream(s: &SubtitleStream) -> Self {
Self {
language: language_or_und(&s.language),
codec: s.codec.id().to_string(),
// The authoritative forced flag (`forced`, set by the content probe /
// STN) OR the label-derived qualifier — either is sufficient.
forced: s.forced || s.qualifier == LabelQualifier::Forced,
sdh: s.qualifier == LabelQualifier::Sdh,
// The mux never defaults a subtitle track; mirror that here.
default: false,
name: String::new(),
}
}
}
impl TitleProfile {
/// Build a title's profile. `index` / `is_main` are disc-level facts the
/// caller supplies (see [`DiscProfile::from_disc`]); the streams are split
/// and the per-kind default track is hoisted here.
pub fn from_title(title: &DiscTitle, index: usize, is_main: bool) -> Self {
let mut video = Vec::new();
let mut audio = Vec::new();
let mut subtitles = Vec::new();
// "First non-secondary is the default; keep only the first" — the same
// rule the Matroska path applies (`is_default = !secondary`, then only
// the first video and first audio survive as default).
let mut video_default_taken = false;
let mut audio_default_taken = false;
for s in &title.streams {
match s {
Stream::Video(v) => {
let default = !v.secondary && !video_default_taken;
video_default_taken |= default;
video.push(VideoTrack::from_stream(v, default));
}
Stream::Audio(a) => {
let default = !a.secondary && !audio_default_taken;
audio_default_taken |= default;
audio.push(AudioTrack::from_stream(a, default));
}
Stream::Subtitle(t) => subtitles.push(SubtitleTrack::from_stream(t)),
}
}
Self {
index,
playlist: title.playlist.clone(),
duration_secs: title.duration_secs,
size_bytes: title.size_bytes,
chapters: title.chapters.len(),
is_main,
video,
audio,
subtitles,
}
}
/// The title's video tracks.
pub fn video(&self) -> &[VideoTrack] {
&self.video
}
/// The title's audio tracks.
pub fn audio(&self) -> &[AudioTrack] {
&self.audio
}
/// The title's subtitle tracks.
pub fn subtitles(&self) -> &[SubtitleTrack] {
&self.subtitles
}
}
impl DiscProfile {
/// Build the normalized profile from a scanned [`Disc`].
pub fn from_disc(disc: &Disc) -> Self {
// The scan pre-sorts titles so `titles[0]` is the selected main feature
// (`sort_titles_by_main_feature`), so the main title is index 0 whenever
// any title exists.
let main_title = 0;
let has_titles = !disc.titles.is_empty();
let titles = disc
.titles
.iter()
.enumerate()
.map(|(i, t)| TitleProfile::from_title(t, i, has_titles && i == main_title))
.collect();
Self {
format: format_id(disc.format).to_string(),
disc_name: disc
.meta_title
.clone()
.unwrap_or_else(|| disc.volume_id.clone()),
disc_id: disc.volume_id.clone(),
main_title,
titles,
}
}
/// The selected main-feature title, or `None` when the disc scanned to zero
/// titles (a data-only image with no /BDMV, /HVDVD_TS or /VIDEO_TS). Never
/// panics — mirrors the `Option`-returning main-feature accessors elsewhere
/// (`titles.first()`, `dvdnav::resolve_main_title`).
pub fn main_title(&self) -> Option<&TitleProfile> {
self.titles.get(self.main_title)
}
}
impl From<&Disc> for DiscProfile {
fn from(disc: &Disc) -> Self {
DiscProfile::from_disc(disc)
}
}
impl Disc {
/// This disc's normalized, format-agnostic [`DiscProfile`].
pub fn profile(&self) -> DiscProfile {
DiscProfile::from_disc(self)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{
AudioChannels, AudioStream, Codec, ColorSpace, ContentFormat, DiscRegion, DiscTitle,
FrameRate, HdrFormat, Resolution, SampleRate, Stream, SubtitleStream, VideoStream,
};
/// A minimal titleless [`Disc`] for the disc-level tests.
fn test_disc() -> Disc {
Disc {
volume_id: String::new(),
meta_title: None,
format: DiscFormat::BluRay,
capacity_sectors: 0,
capacity_bytes: 0,
layers: 1,
titles: Vec::new(),
region: DiscRegion::Free,
aacs: None,
css: None,
encrypted: false,
aacs_error: None,
css_error: None,
content_format: ContentFormat::BdTs,
}
}
#[test]
fn main_title_is_none_on_empty_disc() {
// A data-only image (no /BDMV, /HVDVD_TS, /VIDEO_TS) scans to zero
// titles; main_title() must return None, not panic on titles[0].
let profile = DiscProfile::from_disc(&test_disc());
assert!(profile.titles.is_empty());
assert!(profile.main_title().is_none());
}
fn video(codec: Codec, res: Resolution, secondary: bool) -> Stream {
Stream::Video(VideoStream {
pid: 0x1011,
codec,
resolution: res,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
display_aspect: None,
secondary,
label: String::new(),
measured_cicp: None,
})
}
fn audio(lang: &str, secondary: bool, purpose: LabelPurpose, label: &str) -> Stream {
Stream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Stereo,
language: lang.into(),
sample_rate: SampleRate::S48,
secondary,
purpose,
label: label.into(),
})
}
fn subtitle(lang: &str, forced: bool, qualifier: LabelQualifier) -> Stream {
Stream::Subtitle(SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: lang.into(),
forced,
qualifier,
codec_data: None,
})
}
/// A BD transport-stream title with a mix of streams: two audio tracks
/// (main + commentary), two subtitles (forced + SDH), a video track.
fn bdts_title() -> DiscTitle {
let mut t = DiscTitle::empty();
t.content_format = ContentFormat::BdTs;
t.playlist = "00800.mpls".into();
t.duration_secs = 7200.0;
t.size_bytes = 30_000_000_000;
t.streams = vec![
video(Codec::Hevc, Resolution::R2160p, false),
audio("eng", false, LabelPurpose::Normal, "Dolby TrueHD 5.1"),
audio("eng", true, LabelPurpose::Commentary, "Director"),
subtitle("eng", true, LabelQualifier::Forced),
subtitle("eng", false, LabelQualifier::Sdh),
];
t
}
/// A DVD (MPEG program stream) title: two audio tracks (both non-secondary,
/// second is descriptive), one language-less subtitle, one video track.
fn dvd_title() -> DiscTitle {
let mut t = DiscTitle::empty();
t.content_format = ContentFormat::MpegPs;
t.playlist = "VTS_01".into();
t.duration_secs = 5400.0;
t.size_bytes = 4_000_000_000;
t.streams = vec![
video(Codec::Mpeg2, Resolution::R576i, false),
audio("eng", false, LabelPurpose::Normal, ""),
audio("eng", false, LabelPurpose::Descriptive, ""),
subtitle("", false, LabelQualifier::None),
];
t
}
#[test]
fn splits_streams_into_typed_vectors_across_formats() {
for title in [bdts_title(), dvd_title()] {
let p = TitleProfile::from_title(&title, 0, true);
assert_eq!(p.video.len(), 1, "one video track");
assert_eq!(p.audio.len(), 2, "two audio tracks");
assert_eq!(p.subtitles.len(), 1.max(p.subtitles.len()));
// Accessors return the same vectors.
assert_eq!(p.video(), p.video.as_slice());
assert_eq!(p.audio(), p.audio.as_slice());
assert_eq!(p.subtitles(), p.subtitles.as_slice());
}
}
#[test]
fn hoists_default_flags_first_non_secondary() {
let p = TitleProfile::from_title(&bdts_title(), 0, true);
// First (non-secondary) video/audio is the default.
assert!(p.video[0].default, "first video defaults");
assert!(p.audio[0].default, "first non-secondary audio defaults");
// The commentary track is secondary → never default.
assert!(
!p.audio[1].default,
"secondary/commentary audio not default"
);
// Subtitles are never default.
assert!(p.subtitles.iter().all(|s| !s.default));
}
#[test]
fn second_default_cleared_when_two_non_secondary_audio() {
// DVD title: BOTH audio tracks are non-secondary. Only the first keeps
// the default flag (mirrors the muxer's "keep only first default").
let p = TitleProfile::from_title(&dvd_title(), 0, true);
assert!(p.audio[0].default, "first audio default");
assert!(
!p.audio[1].default,
"second non-secondary audio default cleared"
);
assert!(p.audio[1].descriptive, "second audio is descriptive");
assert!(!p.audio[1].commentary);
}
#[test]
fn maps_qualifier_and_purpose_and_language_defaults() {
let p = TitleProfile::from_title(&bdts_title(), 0, true);
// Audio purpose → commentary/descriptive booleans.
assert!(!p.audio[0].commentary && !p.audio[0].descriptive);
assert!(p.audio[1].commentary);
assert_eq!(p.audio[1].name, "Director", "label maps to name");
// Subtitle qualifier → forced/sdh booleans.
assert!(p.subtitles[0].forced && !p.subtitles[0].sdh);
assert!(p.subtitles[1].sdh && !p.subtitles[1].forced);
// Language default: DVD title's blank subtitle language → "und".
let dvd = TitleProfile::from_title(&dvd_title(), 0, true);
assert_eq!(dvd.subtitles[0].language, "und");
assert_eq!(dvd.audio[0].language, "eng");
// Codec/resolution/hdr flattened as compact ids/labels.
assert_eq!(p.video[0].codec, "hevc");
assert_eq!(p.video[0].resolution, "2160p");
assert_eq!(p.video[0].hdr, "hdr10");
assert_eq!(dvd.video[0].codec, "mpeg2");
}
#[test]
fn from_disc_hoists_main_and_is_main() {
let mut disc = test_disc();
disc.format = DiscFormat::BluRay;
disc.meta_title = Some("SOME MOVIE".into());
disc.volume_id = "VOL_ID".into();
disc.titles = vec![bdts_title(), dvd_title()];
let profile = disc.profile();
assert_eq!(profile.format, "bluray");
assert_eq!(profile.disc_name, "SOME MOVIE");
assert_eq!(profile.disc_id, "VOL_ID");
assert_eq!(profile.main_title, 0);
assert!(profile.titles[0].is_main, "titles[0] is the main feature");
assert!(!profile.titles[1].is_main);
assert_eq!(profile.titles[1].index, 1);
assert_eq!(profile.main_title().unwrap().playlist, "00800.mpls");
}
#[test]
fn disc_name_falls_back_to_volume_id() {
let mut disc = test_disc();
disc.meta_title = None;
disc.volume_id = "PLAIN_VOLUME".into();
assert_eq!(disc.profile().disc_name, "PLAIN_VOLUME");
}
#[test]
fn serde_round_trip() {
let mut disc = test_disc();
disc.format = DiscFormat::Dvd;
disc.volume_id = "RT".into();
disc.titles = vec![bdts_title(), dvd_title()];
let profile = disc.profile();
let json = serde_json::to_string(&profile).expect("serialize");
let back: DiscProfile = serde_json::from_str(&json).expect("deserialize");
assert_eq!(profile, back, "profile round-trips through serde");
}
}
+425 -60
View File
@@ -8,6 +8,17 @@ pub fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
// A failed `ioctl(SG_IO)` and a vanished device never produced a SCSI
// reply at all — they are dead-bus faults, not recoverable bad sectors.
// `Error::is_scsi_transport_failure` (error.rs) already declares both
// variants transport failures so sweep / patch / fill_extents abort the
// pass instead of zero-filling against a wedged device; callers that
// flatten an error through here (Drive::read_one, DiscStream::fill_extents,
// freemkv-engine's recovery sweep/patch) would otherwise collapse them to
// status 0 and silently destroy that classification.
Error::IoError { .. } | Error::DeviceNotFound { .. } => {
(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None)
}
_ => (0, None),
}
}
@@ -24,6 +35,15 @@ pub(crate) mod macos;
#[cfg(windows)]
pub(crate) mod windows;
// Pick the platform module ONCE, here, so the cross-platform entry points below
// dispatch through `platform::…` with no per-function `#[cfg]` in their bodies.
#[cfg(target_os = "linux")]
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
pub(crate) use macos as platform;
#[cfg(windows)]
pub(crate) use windows as platform;
use crate::error::{Error, Result};
use crate::event::Event;
use crate::identity::DriveId;
@@ -279,22 +299,33 @@ impl Drive {
for attempt in 0..60u64 {
hb.tick(attempt, 60);
let mut buf = [0u8; 0];
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
// `checked_exec`, not a bare `execute`. This poll was the ONE
// drive path that talked to the transport directly, and it is the
// longest-running one: 60 x 500 ms = ~30 s. It never read
// `self.halt`, so an operator Stop pressed while a cold drive spun
// up was ignored for up to half a minute while every other path in
// this file returned `Halted` at its next command boundary. A Stop
// that does nothing for 30 s is indistinguishable from a hung app.
//
// A TUR that FAILS is the ordinary not-ready-yet answer and must
// keep the loop going — only `Halted` aborts it.
match self.checked_exec(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
}
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => {}
}
std::thread::sleep(std::time::Duration::from_millis(500));
// Halt-aware backoff: the flag can also flip DURING the 500 ms
// gap, which is where most of the 30 s is actually spent.
sleep_until_halted(&self.halt, std::time::Duration::from_millis(500))?;
}
tracing::warn!(
target: "freemkv::drive",
@@ -504,9 +535,10 @@ impl Drive {
crate::unlock_bridge::run_features(self.scsi.as_mut(), &self.drive_id);
let r: Result<()> = match unlock_res {
Ok(unlocked) => {
// Record WHICH drive-prep unlocker actually ran — "LibreDrive"
// (MediaTek) or "Renesas" — not the ld-only identity lookup, so a
// Renesas drive reports itself honestly rather than as nothing.
// Record WHICH drive-prep unlocker actually ran — "MT1959" (the
// drive-firmware bus-unlock route) or "Renesas" (the cert route) —
// not the ld-only identity lookup, so a Renesas drive reports
// itself honestly rather than as nothing.
self.unlocker_name = Some(matched.to_string());
// Stash the OEM Volume ID the unlocker returned for the AACS
// handshake phase (do_handshake reads it via `oem_vid()`). A
@@ -926,11 +958,34 @@ impl Drive {
// range NonTrimmed and retries (a loud miss, never a silent commit).
// The sector/file path enforces the same invariant in
// sector/prefetched.rs; this is the live-drive counterpart.
Ok(_) => Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
}),
//
// SAY SO. The refusal was correct and completely silent: the
// sibling `Err(e)` arm below warns with lba/count/status, this one
// logged nothing at all, so a drive that residual-underruns on
// GOOD status was indistinguishable in the logs from a scratched
// disc — two different populations (replace the drive vs. clean
// the disc) collapsed into one. Absence of a log is itself the
// defect here; the read still fails either way.
//
// `scsi_status`/`sense` are deliberately absent from the line
// rather than faked: the command SUCCEEDED, so there is no sense
// data to report. `transferred` vs `expected` is the whole signal.
Ok(result) => {
tracing::warn!(
target: "freemkv::drive",
lba,
count,
transferred = result.bytes_transferred,
expected = count as usize * 2048,
code = crate::error::E_DISC_READ,
"READ(10) returned GOOD status with a residual underrun; refusing the short transfer"
);
Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
})
}
Err(Error::Halted) => Err(Error::Halted),
Err(e) => {
let (status, sense) = extract_scsi_context(&e);
@@ -1086,14 +1141,25 @@ impl Drive {
let stop = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x00, 0]; // START=0, LOEJ=0 → spin down
let start = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x01, 0]; // START=1, LOEJ=0 → spin up
let mut buf = [0u8; 0];
self.scsi
.as_mut()
.execute(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS));
self.scsi
.as_mut()
.execute(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS));
// `checked_exec` + `sleep_until_halted`, not `execute` + a blind
// `thread::sleep`. This routine is ~15 s of deliberate waiting
// (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) issued from the
// recovery path, i.e. exactly when a run is going badly and the
// operator is most likely to press Stop. Both commands bypassed the
// halt flag and both sleeps were unconditional, so the whole cycle was
// deaf: Stop appeared to hang. Nothing here is un-interruptible — a
// half-finished spin cycle leaves the drive spun down, which the next
// command spins back up.
self.checked_exec(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS),
)?;
self.checked_exec(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS),
)?;
Ok(())
}
@@ -1299,11 +1365,18 @@ pub(crate) fn decode_read_capacity(buf: &[u8; 8], bytes_transferred: usize) -> R
last_lba.checked_add(1).ok_or(Error::DiscCapacityOverflow)
}
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
/// to true. Kept for the unit tests that cover the slicing behaviour;
/// production code paths no longer sleep on the recovery hot path
/// (recovery loop removed in 0.13.6).
#[cfg(test)]
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping to
/// true, returning [`Error::Halted`].
///
/// This was `#[cfg(test)]` for two releases, kept alive only by the four unit
/// tests below, while the two production paths that actually sleep — the
/// `wait_ready` poll backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle
/// pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
/// `std::thread::sleep` and so were DEAF to the operator's Stop for ~30 s and
/// ~15 s respectively. Every other drive path returns `Halted` at the next
/// `checked_exec` boundary; a Stop pressed during spin-up simply did nothing
/// visible until the poll ran out. The primitive existed; the sleeping code
/// just did not call it. It is production code again.
fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<()> {
const SLICE: std::time::Duration = std::time::Duration::from_millis(100);
let deadline = std::time::Instant::now() + total;
@@ -1322,18 +1395,7 @@ fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<(
/// Internal: discover drive paths + IDs without opening full Drive objects.
fn discover_drives() -> Vec<(String, DriveId)> {
#[cfg(target_os = "linux")]
{
linux::find_drives()
}
#[cfg(target_os = "macos")]
{
macos::find_drives()
}
#[cfg(windows)]
{
windows::find_drives()
}
platform::find_drives()
}
/// Structured outcome of [`resolve_device`] — a machine-readable signal
@@ -1353,20 +1415,14 @@ pub enum DeviceResolution {
/// Resolve a device path to its raw SCSI device. Returns the resolved
/// path plus a structured [`DeviceResolution`] signal describing whether
/// any substitution happened; the application layer maps that to UX text.
///
/// Staged, not yet wired: the cross-platform dispatch is kept ready for the
/// caller that will consume it, so the per-platform implementations below it
/// (and their tests) stay live. `allow(dead_code)` marks that deliberately —
/// this is not an accidental orphan.
#[allow(dead_code)]
pub(crate) fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
#[cfg(target_os = "linux")]
{
linux::resolve_device(path)
}
#[cfg(target_os = "macos")]
{
macos::resolve_device(path)
}
#[cfg(windows)]
{
windows::resolve_device(path)
}
platform::resolve_device(path)
}
#[cfg(test)]
@@ -1936,6 +1992,90 @@ mod command_tests {
assert!(err.scsi_sense().is_none());
}
/// `extract_scsi_context` must map the two non-SCSI dead-bus faults
/// (a failed `ioctl(SG_IO)` → `Error::IoError`, a vanished fd →
/// `Error::DeviceNotFound`) to the 0xFF TRANSPORT_FAILURE sentinel, not
/// to 0x00. Everything else keeps the (0, None) catch-all.
/// Spec: `Error::is_scsi_transport_failure` (error.rs) declares both
/// variants transport failures so sweep/patch/fill_extents abort
/// the pass instead of zero-filling against a wedged device.
/// Mutation: returning (0, None) here flattens IoError into
/// `DiscRead { status: Some(0) }`, which is_scsi_transport_failure
/// rejects — a wedged USB bridge zero-fills the whole title.
#[test]
fn extract_scsi_context_maps_dead_bus_faults_to_transport_failure() {
let (status, sense) = extract_scsi_context(&Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a failed ioctl(SG_IO) is a transport-layer fault"
);
assert!(sense.is_none(), "no SCSI reply means no sense data");
let (status, sense) = extract_scsi_context(&Error::DeviceNotFound {
path: "/dev/sg9".into(),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a vanished device is a transport-layer fault"
);
assert!(sense.is_none());
// Control: the catch-all still yields (0, None) for unrelated errors,
// so the two asserts above are about these variants specifically.
assert_eq!(extract_scsi_context(&Error::Halted), (0, None));
// Control: real SCSI replies still pass their own status through.
let s = crate::scsi::ScsiSense {
sense_key: 3,
asc: 0x11,
ascq: 0x00,
};
assert_eq!(
extract_scsi_context(&Error::ScsiError {
opcode: 0x28,
status: 0x02,
sense: Some(s),
}),
(0x02, Some(s))
);
}
/// End-to-end: an `Error::IoError` raised by the transport must still be
/// classified as a transport failure after `Drive::read` flattens it into
/// `Error::DiscRead`. Without the `extract_scsi_context` mapping the
/// variant is destroyed (status becomes `Some(0)`) and the caller treats a
/// dead bus as a recoverable bad sector.
#[test]
fn read_io_error_surfaces_as_transport_failure_not_a_bad_sector() {
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
},
}));
let mut buf = vec![0u8; 2048];
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a wedged bus must abort the pass, not zero-fill: got {err:?}"
);
// The same for a device that vanished mid-read.
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::DeviceNotFound {
path: "/dev/sg9".into(),
},
}));
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a vanished device must abort the pass: got {err:?}"
);
}
#[test]
fn read_returns_halted_before_dispatch_without_touching_transport() {
// When the halt flag is set, checked_exec returns Halted BEFORE
@@ -2589,6 +2729,231 @@ mod command_tests {
);
}
/// A Stop pressed before the poll starts must be answered at once.
///
/// `wait_ready` was the ONE drive path that called
/// `self.scsi.as_mut().execute(..)` instead of `checked_exec`, and its
/// 60 x 500 ms loop never read `self.halt`. A cancel during spin-up was
/// therefore ignored for ~30 s — the test above measures exactly how long
/// — while every other drive path returns `Halted` at its next command
/// boundary. Thirty seconds of a dead Stop button is indistinguishable
/// from a hung application.
///
/// Mutation: restoring the bare `execute` makes this run the full poll and
/// return `DeviceNotReady`, failing both assertions.
#[test]
fn wait_ready_returns_halted_when_stopped_before_the_poll() {
struct NeverReady;
impl ScsiTransport for NeverReady {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(NeverReady));
d.halt();
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop is the operator, not a drive that failed to spin up: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"the poll must abandon immediately, not run its ~30 s course"
);
}
/// ...and a Stop pressed PART WAY THROUGH the poll must be answered at the
/// next command boundary, not at the end of the 30 s.
///
/// The flag is flipped by the transport itself on its third TEST UNIT
/// READY, which is deterministic (no wall-clock race): `checked_exec`
/// re-reads the flag after the command completes, so the loop must exit on
/// attempt three of sixty.
#[test]
fn wait_ready_returns_halted_when_stopped_during_the_poll() {
struct StopsOnThirdPoll {
halt: Arc<AtomicBool>,
seen: usize,
}
impl ScsiTransport for StopsOnThirdPoll {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
self.seen += 1;
if self.seen == 3 {
self.halt.store(true, Ordering::Relaxed);
}
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(StopsOnThirdPoll {
halt: Arc::new(AtomicBool::new(false)),
seen: 0,
}));
// Hand the transport the drive's OWN flag, so setting it is exactly
// what an operator's Stop does.
let flag = d.halt_flag();
d.scsi = Box::new(StopsOnThirdPoll {
halt: flag,
seen: 0,
});
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a mid-poll Stop must surface as Halted, not as DeviceNotReady \
after the full 30 s: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"must exit on the third poll, not the sixtieth"
);
}
/// `spin_cycle` must not be deaf to Stop for its ~15 s of deliberate
/// waiting.
///
/// Both START STOP UNIT commands went out through a bare `execute` and
/// both pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) were plain
/// `std::thread::sleep`, so the whole routine ignored the halt flag. It
/// runs from the RECOVERY path — precisely when a run is going badly and
/// the operator is most likely to press Stop.
///
/// The flag is set BEFORE the call, so this is deterministic: the very
/// first `checked_exec` must refuse.
///
/// Mutation: restoring either bare `execute` lets the first command
/// through and the test then waits out a real 5 s sleep, failing the
/// elapsed bound.
#[test]
fn spin_cycle_returns_halted_when_stopped_before_it_starts() {
let RecordingHarness {
drive: mut d,
cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
d.halt();
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop must abort the spin cycle: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"no sleep may run after the flag is set"
);
assert!(
cdb.lock().unwrap().is_empty(),
"not one START STOP UNIT may be issued after Stop"
);
}
/// ...and a Stop that lands DURING the spin-down pause must wake it.
///
/// This is the half a `checked_exec` alone cannot fix: the flag flips
/// while the thread is parked inside the 5 s `SPIN_DOWN_IDLE_SECS` sleep,
/// so only a halt-aware sleep can observe it. `sleep_until_halted` — which
/// already lived in this file with four tests, marked `#[cfg(test)]` and
/// called from nowhere — wakes within ~100 ms.
///
/// The margin is deliberately enormous (≈0.3 s against 5 s) so the bound
/// is not a wall-clock race: a plain `thread::sleep` cannot come in under
/// two seconds, and the halt-aware one cannot take that long.
#[test]
fn spin_cycle_wakes_from_its_spin_down_pause_when_stopped() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
let flag = d.halt_flag();
let stopper = std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(200));
flag.store(true, Ordering::Relaxed);
});
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
stopper.join().expect("stopper thread");
assert!(
matches!(r, Err(Error::Halted)),
"a Stop during the spin-down pause must surface as Halted: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"the pause must be halt-aware, not a blind {SPIN_DOWN_IDLE_SECS}s \
thread::sleep; took {:?}",
t0.elapsed()
);
}
/// A READ(10) that completes with GOOD status but a residual underrun is
/// correctly refused — and must SAY SO.
///
/// The refusal itself already existed and is right: the tail of the buffer
/// still holds stale bytes, so committing them would be silent
/// corruption. What was missing is any trace of it. The sibling `Err(e)`
/// arm warns with lba/count/status; this arm logged NOTHING, so a drive
/// that residual-underruns on GOOD status produced exactly the same
/// journal as a scratched disc — two populations with opposite remedies
/// (replace the drive vs. clean the disc) collapsed into one, with the
/// operator sent after the wrong one.
///
/// Mutation: deleting the `tracing::warn!` leaves no event; logging a
/// different code, or dropping the transferred/expected pair that is the
/// entire signal, fails the field assertions.
#[test]
fn read_logs_a_good_status_short_transfer() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(1024)); // half of one sector
let mut buf = vec![0u8; 2048];
let (r, events) = crate::testlog::capture(|| d.read(7, 1, &mut buf, false));
assert!(
matches!(r, Err(Error::DiscRead { sector: 7, .. })),
"a short transfer stays a failed read: {r:?}"
);
let line = events
.iter()
.find(|e| e.target == "freemkv::drive" && e.field("transferred").is_some())
.unwrap_or_else(|| {
panic!("a silently refused short transfer is the defect; got {events:?}")
});
assert_eq!(line.level, tracing::Level::WARN);
assert_eq!(line.field("lba"), Some("7"));
assert_eq!(line.field("transferred"), Some("1024"));
assert_eq!(
line.field("expected"),
Some("2048"),
"the underrun is only legible as transferred-vs-expected"
);
assert_eq!(
line.field("code"),
Some(crate::error::E_DISC_READ.to_string().as_str())
);
}
// ── report_key / mode_sense / read_buffer empty-vs-some ─────────
#[test]
+7 -4
View File
@@ -8,11 +8,14 @@
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against real discs.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
//! that resolves the start cell build on top of this.
//! Current contents: [`vmcmd`] — the VM command decoder; [`nav`] — the
//! First-Play navigation executor that resolves which title the disc's own
//! navigation selects as the feature.
pub mod vmcmd;
pub(crate) mod nav;
pub(crate) mod vmcmd;
pub(crate) use nav::resolve_main_title;
use crate::sector::SectorSource;
+744
View File
@@ -0,0 +1,744 @@
//! DVD-Video navigation executor — the DVD arm of "mimic a real player" title
//! selection (issue #40).
//!
//! A player enters a disc at the **First-Play PGC** (FP_PGC): a short program of
//! VM commands (parsed by [`super::vmcmd`]) that runs before any menu draws and
//! typically dispatches straight to the feature title. This module reads that
//! program out of `VIDEO_TS.IFO` (VMGI), executes its pre-command list on a
//! minimal register machine, and — when the program deterministically reaches a
//! title dispatch (`JumpTT`) — maps the selected VMG title through the title
//! search pointer table (`TT_SRPT`) to its `(VTS number, title-in-set)`
//! coordinates. That is the title a player would land on, so DVD title
//! selection can prefer it over the structural leading-cell heuristic.
//!
//! Contract: this is a strict improvement over the fallback. Any parse error,
//! interactive/menu-only entry, or non-convergence yields `None`, and the caller
//! keeps today's behaviour. Every table count and offset is attacker-controlled,
//! so all arithmetic is checked and all reads are bounds-guarded — the entry
//! point [`resolve_from_vmg`] never panics on any input (exercised by the
//! never-panic harness).
use super::vmcmd::{self, Compare, Instr};
use crate::consts::SECTOR_BYTES;
use crate::sector::SectorSource;
use crate::udf::UdfFs;
// ── VMGI management-table byte offsets into VIDEO_TS.IFO (DVD-Video spec) ─────
//
// The VMGI_MAT header carries these fixed offsets. The values they hold are
// per-disc; the offsets are constant.
const VMGI_MAGIC: &[u8; 12] = b"DVDVIDEO-VMG";
/// u32 **byte** offset (from the start of VIDEO_TS.IFO) of the First-Play PGC.
const VMGI_FP_PGC_PTR: usize = 0x84;
/// u32 **sector** offset (from the start of VIDEO_TS.IFO) of TT_SRPT.
const VMGI_TT_SRPT_PTR: usize = 0xC4;
/// Within a PGC, the command-table pointer is a u16 at `PGC + 0xE4` giving the
/// command table's byte offset relative to the PGC start (DVD-Video PGC layout;
/// the same layout `ifo::parse_pgc` reads for cell/program tables).
const PGC_CMD_TBL_PTR: usize = 0xE4;
/// Upper bound on commands executed before declaring non-convergence. A
/// conformant First-Play routine reaches a title in a handful of steps; the cap
/// stops a crafted command list (self-`Goto`, mutual jumps) from spinning.
const STEP_BUDGET: usize = 1024;
/// Maximum commands honoured in one PGC command list. The DVD-Video VM caps a
/// list at 128 pre / 128 post / 128 cell commands; the on-disc count is an
/// untrusted u16, so it is clamped to this format maximum.
const MAX_CMDS: usize = 128;
/// Maximum TT_SRPT entries honoured — the DVD-Video 99-title format maximum
/// (mirrors `ifo::MAX_TT_SRPT_TITLES`; the on-disc count is an untrusted u16).
const MAX_TT_SRPT_TITLES: usize = 99;
/// The title the First-Play navigation selects, in VMG coordinates.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResolvedTitle {
/// 1-based VMG title number (the TT_SRPT index the `JumpTT` named).
pub title: u8,
/// 1-based VTS (title set) number the title lives in.
pub vtsn: u8,
/// 1-based title-within-set number (TT_SRPT `VTS_TTN`).
pub vts_ttn: u8,
}
// ── Bounds-guarded big-endian reads (return None past the end) ────────────────
#[inline]
fn u16_at(b: &[u8], o: usize) -> Option<u16> {
Some(u16::from_be_bytes([*b.get(o)?, *b.get(o.checked_add(1)?)?]))
}
#[inline]
fn u32_at(b: &[u8], o: usize) -> Option<u32> {
Some(u32::from_be_bytes([
*b.get(o)?,
*b.get(o.checked_add(1)?)?,
*b.get(o.checked_add(2)?)?,
*b.get(o.checked_add(3)?)?,
]))
}
/// A register operand index names a system parameter (SPRM) when it carries the
/// `0x80` bias — matching [`Vm::reg`]'s split of GPRM (`< 128`) vs SPRM
/// (`>= 128`). Only SPRM reads are session-specific and therefore taint a
/// compare; GPRM reads do not.
#[inline]
fn is_sprm(idx: u8) -> bool {
idx >= 128
}
// ── Minimal navigation register machine ──────────────────────────────────────
/// The subset of the DVD-Video VM register file the First-Play resolver needs:
/// 16 general parameters (GPRMs) and the system parameters (SPRMs). Execution
/// starts from a cold machine (all zero) — the same starting point a player has
/// before any user interaction. The SPRMs a First-Play routine might branch on
/// (region, parental level, language) are player-/session-specific and unknown
/// at scan time; leaving them zero means the resolver only commits to a title
/// when the program reaches one *without* depending on them (an unconditional
/// dispatch, or a conditional whose zero-register path a player also takes).
/// When a branch genuinely depends on an unknown SPRM the routine is treated as
/// undecidable and the caller falls back — see the module contract.
struct Vm {
gprm: [u16; 16],
sprm: [u16; 24],
/// Per-GPRM taint: set when the register holds a value derived from a system
/// parameter (SPRM). An SPRM read taints a compare directly ([`is_sprm`]);
/// this bit tracks an SPRM *laundered through* a GPRM store so a later
/// GPRM-only compare that reads it is still recognised as undecidable.
gprm_tainted: [bool; 16],
}
impl Vm {
fn new() -> Self {
Self {
gprm: [0; 16],
sprm: [0; 24],
gprm_tainted: [false; 16],
}
}
/// Whether a register *operand read* is session-specific: an SPRM read is
/// always undecidable at scan time; a GPRM read is undecidable only when the
/// register currently holds an SPRM-derived (tainted) value.
fn reg_tainted(&self, idx: u8) -> bool {
if is_sprm(idx) {
true
} else {
self.gprm_tainted[(idx & 0x0F) as usize]
}
}
/// Mark a GPRM's post-state as unknown. Used when a store to it was guarded
/// by an undecidable (tainted) predicate: whether the write happened is
/// itself unknown, so the destination is conservatively tainted.
fn taint_gprm(&mut self, reg: u8) {
self.gprm_tainted[(reg & 0x0F) as usize] = true;
}
/// Read a register operand. Per the VM command model, indices `< 16` are
/// GPRMs and `>= 128` are SPRMs (0-23 after the `0x80` bias); anything out
/// of range reads as 0 rather than panicking.
fn reg(&self, idx: u8) -> u16 {
if idx >= 128 {
self.sprm.get((idx - 128) as usize).copied().unwrap_or(0)
} else {
self.gprm.get((idx & 0x0F) as usize).copied().unwrap_or(0)
}
}
/// Evaluate a compare predicate (op codes per [`Compare`]). Returns the
/// predicate result together with a *taint* flag that is set when either
/// operand read a system parameter (SPRM). An SPRM's value is
/// player-/session-specific (region, parental level, language) and unknown
/// at scan time, so a branch decision that depends on one is undecidable —
/// the caller abstains rather than commit to the cold-power-on arm. A GPRM
/// read taints only when that register was itself loaded from an SPRM
/// (tracked by [`Vm::gprm_tainted`]); a GPRM holding a program-established
/// value does not taint.
fn eval(&self, c: &Compare) -> (bool, bool) {
let mut tainted = self.reg_tainted(c.lhs_reg);
let l = self.reg(c.lhs_reg);
let r = if c.immediate {
c.imm
} else {
tainted |= self.reg_tainted(c.rhs_reg);
self.reg(c.rhs_reg)
};
let result = match c.op {
1 => (l & r) != 0,
2 => l == r,
3 => l != r,
4 => l >= r,
5 => l > r,
6 => l <= r,
7 => l < r,
_ => false,
};
(result, tainted)
}
/// Apply a `SetGPRM`. Only the arithmetic/logic set-ops that a First-Play
/// routine uses to prepare a dispatch are modelled; unmodelled ops
/// (swap/rnd) leave the register unchanged (best-effort, never panic).
fn set(&mut self, reg: u8, op: u8, immediate: bool, imm: u16, src: u8) {
let idx = (reg & 0x0F) as usize;
let v = if immediate { imm } else { self.reg(src) };
// The value's taint follows its inputs: an immediate is concrete
// (untainted); a register source carries its own taint ([`reg_tainted`]
// covers both a direct SPRM read and a laundered GPRM).
let src_tainted = !immediate && self.reg_tainted(src);
let cur = self.gprm[idx];
let cur_tainted = self.gprm_tainted[idx];
// Each arm sets (new value, new taint). `mov` overwrites, so its taint is
// the source's alone — this is what CLEARS a prior taint on a concrete
// store. The accumulating ops fold the source into the current value, so
// their taint is the union. Unmodelled ops leave value and taint intact.
let (nv, nt) = match op {
1 => (v, src_tainted), // mov
3 => (cur.wrapping_add(v), cur_tainted || src_tainted), // add
4 => (cur.wrapping_sub(v), cur_tainted || src_tainted), // sub
5 => (cur.wrapping_mul(v), cur_tainted || src_tainted), // mul
6 => (
cur.checked_div(v).unwrap_or(cur),
cur_tainted || src_tainted,
), // div (÷0 → unchanged)
7 => (
cur.checked_rem(v).unwrap_or(cur),
cur_tainted || src_tainted,
), // mod (÷0 → unchanged)
9 => (cur & v, cur_tainted || src_tainted), // and
10 => (cur | v, cur_tainted || src_tainted), // or
11 => (cur ^ v, cur_tainted || src_tainted), // xor
_ => (cur, cur_tainted), // swap/rnd/unmodelled
};
self.gprm[idx] = nv;
self.gprm_tainted[idx] = nt;
}
}
// ── VMGI structure readers ───────────────────────────────────────────────────
/// Extract the First-Play PGC pre-command list from VMGI bytes. Returns `None`
/// when there is no First-Play PGC, no command table, or the table is out of
/// bounds — all of which mean "nothing to execute" → fall back.
fn fp_pre_commands(vmg: &[u8]) -> Option<Vec<[u8; 8]>> {
let pgc = u32_at(vmg, VMGI_FP_PGC_PTR)? as usize;
if pgc == 0 {
return None; // no First-Play PGC on this disc
}
let cmd_ptr = u16_at(vmg, pgc.checked_add(PGC_CMD_TBL_PTR)?)? as usize;
if cmd_ptr == 0 {
return None; // First-Play PGC carries no command table
}
let tbl = pgc.checked_add(cmd_ptr)?;
// Command table header: nr_of_pre (u16) at +0, then pre-commands at +8.
let nr_pre = (u16_at(vmg, tbl)? as usize).min(MAX_CMDS);
let base = tbl.checked_add(8)?;
let mut cmds = Vec::with_capacity(nr_pre);
for i in 0..nr_pre {
let o = base.checked_add(i.checked_mul(8)?)?;
if o.checked_add(8)? > vmg.len() {
break; // truncated table — execute what parsed
}
let mut c = [0u8; 8];
c.copy_from_slice(&vmg[o..o + 8]);
cmds.push(c);
}
Some(cmds)
}
/// Parse TT_SRPT into a per-title `(vtsn, vts_ttn)` list indexed by 1-based VMG
/// title number. This is the mapping `JumpTT` needs: a title number → the VTS
/// title-set coordinates that identify the corresponding scanned title.
fn tt_srpt_titles(vmg: &[u8]) -> Option<Vec<(u8, u8)>> {
let sector = u32_at(vmg, VMGI_TT_SRPT_PTR)? as usize;
let base = sector.checked_mul(SECTOR_BYTES)?;
let n = (u16_at(vmg, base)? as usize).min(MAX_TT_SRPT_TITLES);
let entries = base.checked_add(8)?;
let mut out = Vec::with_capacity(n);
for i in 0..n {
// Title entries are 12 bytes: VTSN at +6, VTS_TTN at +7.
let e = entries.checked_add(i.checked_mul(12)?)?;
if e.checked_add(12)? > vmg.len() {
break; // truncated — map what parsed
}
out.push((vmg[e + 6], vmg[e + 7]));
}
Some(out)
}
// ── First-Play executor ──────────────────────────────────────────────────────
/// Execute the First-Play pre-command list and return the VMG title number it
/// dispatches to, if it reaches one deterministically. Returns `None` for a
/// program that ends in a menu/sub-domain, links within the menu domain, exits,
/// or does not converge inside the step budget — every case the caller falls
/// back on.
fn run_first_play(cmds: &[[u8; 8]]) -> Option<u8> {
let mut vm = Vm::new();
let mut pc = 0usize;
let mut budget = STEP_BUDGET;
while pc < cmds.len() {
if budget == 0 {
return None; // non-convergence guard
}
budget -= 1;
let cmd = vmcmd::decode(&cmds[pc]);
// A guarded command's predicate decides whether its instruction runs.
// `taken` is the predicate result; `tainted` marks it undecidable at scan
// time (it read a system parameter, directly or laundered through a
// GPRM). A command with no predicate always runs and is never tainted.
let (taken, tainted) = match cmd.compare {
Some(cmp) => vm.eval(&cmp),
None => (true, false),
};
// A decidable false predicate skips this line — no effect, taint
// unchanged. When the predicate is tainted we cannot decide `taken`, so
// we do not skip; the per-instruction handling below is conservative.
if !tainted && !taken {
pc += 1;
continue;
}
match cmd.instr {
// Title dispatch — a control transfer, the answer we are looking for.
// Abstain only when an undecidable predicate gates it: committing
// would pick the cold-power-on arm of an SPRM-dependent decision.
Instr::JumpTt { ttn } => {
if tainted {
return None;
}
return Some(ttn);
}
// Control flow within the pre list (Goto lines are 1-based) — also a
// control transfer, so an undecidable guard abstains.
Instr::Goto { line } => {
if tainted {
return None;
}
let l = line as usize;
if l == 0 || l > cmds.len() {
return None;
}
pc = l - 1;
continue;
}
// Register preparation before a dispatch — a store, not a control
// transfer, so a tainted guard does NOT abstain. If the guard is
// undecidable we do not know whether the store ran, so the
// destination's post-state is unknown → taint it. Otherwise apply the
// store (which itself propagates or clears the destination's taint).
Instr::SetGprm {
reg,
op,
immediate,
imm,
src,
} => {
if tainted {
vm.taint_gprm(reg);
} else {
vm.set(reg, op, immediate, imm, src);
}
}
// No-effect (for resolution) instructions: keep executing. SetSystem
// writes an SPRM (never a GPRM in this decoder), and SPRM reads are
// caught at read time, so no GPRM taint is needed here.
Instr::Nop | Instr::SetSystem => {}
// Everything else leaves the deterministically-followable path:
// Break/Exit end the pre list with no title selected; the JumpSS
// sub-domain jumps and the intra-domain Link ops land in a menu or
// depend on an interactive button selection that cannot be resolved
// statically. Abstain → caller falls back.
_ => return None,
}
pc += 1;
}
None
}
// ── Public entry points ──────────────────────────────────────────────────────
/// Resolve the main-feature title from raw VMGI (`VIDEO_TS.IFO`) bytes.
///
/// Pure and total: any malformed/hostile input returns `None`, never panics.
/// This is the harness entry point.
pub fn resolve_from_vmg(vmg: &[u8]) -> Option<ResolvedTitle> {
if vmg.len() < 0xC8 || &vmg[0..12] != VMGI_MAGIC {
return None;
}
let cmds = fp_pre_commands(vmg)?;
let ttn = run_first_play(&cmds)?;
if ttn == 0 {
return None; // title 0 is not addressable
}
let titles = tt_srpt_titles(vmg)?;
let (vtsn, vts_ttn) = titles.get((ttn as usize) - 1).copied()?;
if vtsn == 0 {
return None; // invalid TT_SRPT entry
}
Some(ResolvedTitle {
title: ttn,
vtsn,
vts_ttn,
})
}
/// Resolve the main-feature title by reading the disc's VMGI and following its
/// First-Play navigation. Returns `None` (→ caller keeps the heuristic) when the
/// IFO cannot be read or the navigation does not deterministically reach a
/// title.
pub fn resolve_main_title(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ResolvedTitle> {
let vmg = udf.read_file(reader, "/VIDEO_TS/VIDEO_TS.IFO").ok()?;
let resolved = resolve_from_vmg(&vmg)?;
tracing::debug!(
target: "freemkv::dvdnav",
title = resolved.title,
vtsn = resolved.vtsn,
vts_ttn = resolved.vts_ttn,
"nav resolved main-feature title from First-Play program"
);
Some(resolved)
}
#[cfg(test)]
mod tests {
use super::*;
// ── Fixture builders ────────────────────────────────────────────────────
/// Assemble a VMGI (`VIDEO_TS.IFO`) image with a First-Play PGC whose
/// pre-command list is `pre`, and a TT_SRPT at sector `tt_srpt_sector`
/// mapping each title to `(vtsn, vts_ttn)`.
///
/// Layout: the fixed VMGI header (magic + the FP_PGC and TT_SRPT pointers),
/// then the FP_PGC (with a command table holding `pre`), then the TT_SRPT at
/// its sector. Kept minimal but spec-shaped so the same code path a real
/// disc takes is exercised.
fn build_vmgi(pre: &[[u8; 8]], tt_srpt_sector: u32, titles: &[(u8, u8)]) -> Vec<u8> {
// Place the FP_PGC at a fixed byte offset past the header.
let fp_pgc_off: u32 = 0x400;
// Command table sits 0x100 bytes into the PGC.
let cmd_tbl_rel: u16 = 0x100;
let mut v = vec![0u8; 0x800];
v[0..12].copy_from_slice(VMGI_MAGIC);
v[VMGI_FP_PGC_PTR..VMGI_FP_PGC_PTR + 4].copy_from_slice(&fp_pgc_off.to_be_bytes());
v[VMGI_TT_SRPT_PTR..VMGI_TT_SRPT_PTR + 4].copy_from_slice(&tt_srpt_sector.to_be_bytes());
// FP_PGC: command-table pointer at +0xE4.
let pgc = fp_pgc_off as usize;
v[pgc + PGC_CMD_TBL_PTR..pgc + PGC_CMD_TBL_PTR + 2]
.copy_from_slice(&cmd_tbl_rel.to_be_bytes());
// Command table: nr_of_pre at +0, pre-commands from +8.
let tbl = pgc + cmd_tbl_rel as usize;
v[tbl..tbl + 2].copy_from_slice(&(pre.len() as u16).to_be_bytes());
for (i, c) in pre.iter().enumerate() {
let o = tbl + 8 + i * 8;
v[o..o + 8].copy_from_slice(c);
}
// TT_SRPT at its sector: count at +0, 12-byte entries from +8.
let base = tt_srpt_sector as usize * SECTOR_BYTES;
let need = base + 8 + titles.len() * 12;
if v.len() < need {
v.resize(need, 0);
}
v[base..base + 2].copy_from_slice(&(titles.len() as u16).to_be_bytes());
for (i, &(vtsn, vts_ttn)) in titles.iter().enumerate() {
let e = base + 8 + i * 12;
v[e + 6] = vtsn;
v[e + 7] = vts_ttn;
}
v
}
fn h(s: &str) -> [u8; 8] {
let v: Vec<u8> = (0..8)
.map(|i| u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).unwrap())
.collect();
v.try_into().unwrap()
}
// ── Executor: convergence to a title ────────────────────────────────────
/// Unconditional-dispatch shape: First-Play is an unconditional `JumpTT 1`,
/// and TT_SRPT maps title 1 to the feature title set (here VTS_02, title 1).
/// The resolver must return that title.
#[test]
fn first_play_jumptt_1_resolves_to_its_title_set() {
// 3002...0001 = JumpTT ttn=1.
let vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1), (3, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// A First-Play that sets up a register and then unconditionally dispatches
/// still resolves (the SetGPRM is a no-op for an unconditional JumpTT).
#[test]
fn setgprm_then_jumptt_resolves() {
let vmgi = build_vmgi(
&[h("7100000603e80000"), h("3002000000020000")],
1,
&[(1, 1), (4, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 2,
vtsn: 4,
vts_ttn: 1
})
);
}
/// A conditional dispatch whose predicate is true on the cold (zero)
/// machine is taken. `if g0 == 0 -> JumpTT 1` with g0 defaulting to 0.
#[test]
fn conditional_jumptt_true_on_cold_machine_is_taken() {
// 30 22: jump with EQ compare (op=2), operands are registers b6/b7 both
// 0 (g0 == g0) → true; JumpTT ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000010000")], 1, &[(5, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 5,
vts_ttn: 1
})
);
}
/// A conditional dispatch that is false falls through to the next line's
/// unconditional dispatch.
#[test]
fn false_conditional_falls_through_to_next_dispatch() {
// line 1: 30 32 with GT compare (op=3 = !=) g0(b6=1?) ... build a clearly
// false compare: if g0 != g0 -> JumpTT 9 (false, skipped);
// line 2: unconditional JumpTT 1.
let vmgi = build_vmgi(
&[h("3032000000090000"), h("3002000000010000")],
1,
&[(7, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 7,
vts_ttn: 1
})
);
}
// ── Executor: abstention (→ caller falls back) ──────────────────────────
/// Menu-entry shape: First-Play jumps into a VTS menu (`JumpSS VTSM root`).
/// That is an interactive menu, not a static dispatch — the resolver must
/// abstain so the caller keeps the leading-cell heuristic.
#[test]
fn first_play_into_menu_abstains() {
// 3006...0183 = JumpSS VTSM root.
let vmgi = build_vmgi(&[h("3006000101830000")], 1, &[(3, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A dispatch gated on a system parameter (SPRM) is undecidable at scan
/// time — the resolver must abstain rather than commit to the cold-power-on
/// arm. Here `if SPRM0 == g0 -> JumpTT 1` is *true* on the cold machine
/// (both read 0), so without SPRM taint-tracking the resolver would wrongly
/// return title 1; with it, the SPRM read taints the branch and it abstains.
#[test]
fn sprm_gated_dispatch_abstains() {
// 30 22: jump with EQ compare (op=2); if_v2 operands are registers
// b6/b7. b6=0x80 = SPRM0 (session-specific), b7=0x00 = g0. ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000018000")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// The mirror of the SPRM case: a compare that reads only GPRMs is decidable
/// and must NOT over-abstain. `if g0 == g1 -> JumpTT 1` is true on the cold
/// machine and resolves to title 1 (no SPRM operand → no taint).
#[test]
fn gprm_only_compare_does_not_taint() {
// 30 22: jump EQ; b6=0x00 = g0, b7=0x01 = g1 (both 0 on the cold
// machine → equal). ttn=b5=1.
let vmgi = build_vmgi(&[h("3022000000010001")], 1, &[(6, 1)]);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 6,
vts_ttn: 1
})
);
}
/// Defect-1 regression (over-abstention): a store guarded by an SPRM-tainted
/// compare does NOT gate a control transfer, so it must not abstain. Here
/// `if SPRM0 == g0 -> g0 = 5` (a guarded store) is followed by an
/// UNCONDITIONAL `JumpTT 1`. The tainted predicate only decides whether the
/// store runs; the dispatch that follows is SPRM-independent, so the resolver
/// must resolve to title 1.
#[test]
fn sprm_guarded_store_then_unconditional_dispatch_resolves() {
// line 0: 71 20 | lhs=b2=0x80 (SPRM0), cmp EQ(op=2, register), rhs=b7=0
// (g0) → tainted guard; SetGPRM g0 = imm(bytes4-5)=5 (mov).
// line 1: unconditional JumpTT 1.
let vmgi = build_vmgi(
&[h("7120800000050000"), h("3002000000010000")],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// Defect-2 regression (under-abstention): an SPRM laundered through a GPRM
/// must still taint a later branch. `SetGPRM g0 = SPRM20` (register source,
/// index >= 128) followed by `if g0 == g1 -> JumpTT 1` — the compare reads
/// only GPRMs, but g0 now holds an SPRM-derived value, so the decision is
/// undecidable and the resolver must abstain.
#[test]
fn sprm_laundered_through_gprm_abstains() {
// line 0: 61 00 | SetGPRM g0 = SPRM20 (mov, register src=b5=0x94).
// line 1: 30 22 | if g0 == g1 -> JumpTT 1 (both 0 on cold machine).
let vmgi = build_vmgi(
&[h("6100000000940000"), h("3022000000010001")],
1,
&[(2, 1)],
);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// Guard: an unconditional immediate store CLEARS a prior taint (no sticky
/// latch). Taint g0 from an SPRM, overwrite it with an immediate, then branch
/// on g0 — the branch is now decidable and must resolve, not abstain.
#[test]
fn unconditional_immediate_store_clears_taint() {
// line 0: SetGPRM g0 = SPRM20 (taints g0).
// line 1: SetGPRM g0 = imm 0 (mov, immediate) → clears taint, g0 = 0.
// line 2: if g0 == g1 -> JumpTT 1 (both 0) → resolves.
let vmgi = build_vmgi(
&[
h("6100000000940000"),
h("7100000000000000"),
h("3022000000010001"),
],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// Guard: copying an UNtainted GPRM does not taint the destination. `g0 = g1`
/// (register source, g1 untainted) then `if g0 == g2 -> JumpTT 1` must
/// resolve (no SPRM ever entered g0).
#[test]
fn gprm_copy_of_untainted_gprm_does_not_taint() {
// line 0: 61 00 | SetGPRM g0 = g1 (mov, register src=b5=1).
// line 1: 30 22 | if g0 == g2 -> JumpTT 1 (both 0) → resolves.
let vmgi = build_vmgi(
&[h("6100000000010000"), h("3022000000010002")],
1,
&[(2, 1)],
);
assert_eq!(
resolve_from_vmg(&vmgi),
Some(ResolvedTitle {
title: 1,
vtsn: 2,
vts_ttn: 1
})
);
}
/// An empty First-Play pre list selects no title.
#[test]
fn empty_first_play_abstains() {
let vmgi = build_vmgi(&[], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A JumpTT whose title number is past the TT_SRPT table resolves to no
/// coordinates (→ abstain) rather than indexing out of bounds.
#[test]
fn jumptt_past_tt_srpt_abstains() {
// JumpTT ttn=9 but TT_SRPT declares only 1 title.
let vmgi = build_vmgi(&[h("3002000000090000")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
/// A self-referential `Goto` cannot spin forever — the step budget stops it
/// and the resolver abstains.
#[test]
fn self_goto_hits_budget_and_abstains() {
// Goto line 1 (0-special sub 1), byte7 = 01.
let vmgi = build_vmgi(&[h("0001000000000001")], 1, &[(2, 1)]);
assert_eq!(resolve_from_vmg(&vmgi), None);
}
// ── Robustness ──────────────────────────────────────────────────────────
#[test]
fn bad_magic_abstains() {
let mut vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
vmgi[0] = b'X';
assert_eq!(resolve_from_vmg(&vmgi), None);
}
#[test]
fn truncated_and_short_inputs_never_panic() {
// Below the header minimum, and a valid header truncated at every length.
assert_eq!(resolve_from_vmg(&[]), None);
assert_eq!(resolve_from_vmg(VMGI_MAGIC), None);
let full = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
for len in 0..full.len() {
let _ = resolve_from_vmg(&full[..len]);
}
}
/// A TT_SRPT pointer that would overflow when scaled to a byte offset must
/// be rejected, not wrap. Build a normal fixture, then overwrite just the
/// TT_SRPT sector pointer with the extreme value (building a real table at
/// that sector would allocate terabytes in the fixture, not the resolver).
#[test]
fn overflowing_tt_srpt_sector_abstains() {
let mut vmgi = build_vmgi(&[h("3002000000010000")], 1, &[(2, 1)]);
vmgi[VMGI_TT_SRPT_PTR..VMGI_TT_SRPT_PTR + 4].copy_from_slice(&u32::MAX.to_be_bytes());
assert_eq!(resolve_from_vmg(&vmgi), None);
}
}
+36 -17
View File
@@ -163,9 +163,11 @@ fn be16(b: &[u8; 8], o: usize) -> u16 {
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
// v1 (special + link): lhs reg = b[3]; rhs imm = bytes4-5 / rhs reg = b[4].
// v1 (special + link): lhs reg = b[3]; rhs imm = bytes4-5 / rhs reg = b[5]
// (the register operand is the low byte of the reg-or-immediate field).
// v2 (jump + system-set): lhs reg = b[6]; rhs reg = b[7] (registers only).
// v3 (set-GPRM): lhs reg = b[2]; rhs imm = bytes6-7 / rhs reg = b[6].
// v3 (set-GPRM): lhs reg = b[2]; rhs imm = bytes6-7 / rhs reg = b[7]
// (the register operand is the low byte of the reg-or-immediate field).
fn if_v1(b: &[u8; 8]) -> Option<Compare> {
let op = (b[1] >> 4) & 7;
(op != 0).then(|| Compare {
@@ -173,7 +175,7 @@ fn if_v1(b: &[u8; 8]) -> Option<Compare> {
lhs_reg: b[3],
immediate: b[1] >> 7 != 0,
imm: be16(b, 4),
rhs_reg: b[4],
rhs_reg: b[5],
})
}
fn if_v2(b: &[u8; 8]) -> Option<Compare> {
@@ -193,7 +195,7 @@ fn if_v3(b: &[u8; 8]) -> Option<Compare> {
lhs_reg: b[2],
immediate: b[1] >> 7 != 0,
imm: be16(b, 6),
rhs_reg: b[6],
rhs_reg: b[7],
})
}
@@ -295,16 +297,16 @@ mod tests {
v.try_into().unwrap()
}
// KATs taken from the real SOTL / Greenland discs (decoded in the PoC).
// KATs taken from real-world First-Play command streams.
#[test]
fn greenland_first_play_is_jumptt_1() {
fn first_play_jumptt_1_decodes() {
let c = decode(&h("3002000000010000"));
assert_eq!(c.instr, Instr::JumpTt { ttn: 1 });
assert!(c.compare.is_none());
}
#[test]
fn sotl_first_play_is_jumpss_vtsm_root() {
fn first_play_jumpss_vtsm_root_decodes() {
// 30 06 ... byte5=0x83 -> sub 2 (VTSM), vts=byte4=1, menu=byte5&0xF=3 (root)
let c = decode(&h("3006000101830000"));
assert_eq!(
@@ -318,8 +320,9 @@ mod tests {
}
#[test]
fn sotl_title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (cell 2 = the 5:02 start)
fn title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (the cell where the
// feature begins).
let c = decode(&h("20a6000000020002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
@@ -330,7 +333,7 @@ mod tests {
}
#[test]
fn sotl_root_button_is_linkpgcn_37() {
fn root_button_is_linkpgcn_37() {
assert_eq!(
decode(&h("2004000000000025")).instr,
Instr::LinkPgcn { pgcn: 37 }
@@ -338,7 +341,7 @@ mod tests {
}
#[test]
fn greenland_scene_button_is_linkpgn() {
fn scene_button_is_linkpgn() {
assert_eq!(
decode(&h("2006000000001401")).instr,
Instr::LinkPgn { pgn: 1 }
@@ -354,7 +357,7 @@ mod tests {
#[test]
fn setgprm_immediate_mov() {
// SOTL First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
// First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
match decode(&h("7100000603e80000")).instr {
Instr::SetGprm {
reg,
@@ -382,16 +385,32 @@ mod tests {
);
}
// if_version_1 register compare: rhs register is byte4 (not byte5).
// if_version_1 register compare: the register operand is the LOW byte of the
// reg-or-immediate field (byte5), NOT the high byte (byte4). Distinct values
// in byte4 (0xAA) vs byte5 (0x05) pin the field down.
#[test]
fn link_register_compare_rhs_is_byte4() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN
let c = decode(&h("2026000304000002"));
fn link_register_compare_rhs_is_low_byte() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN (pgn=byte7=2)
let c = decode(&h("20260003aa050002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 3);
assert_eq!(cmp.rhs_reg, 4);
assert_eq!(cmp.rhs_reg, 5); // low byte of bytes4-5, not the 0xAA high byte
}
// if_version_3 (set-GPRM) register compare: the register operand is the LOW
// byte of the reg-or-immediate field (byte7), NOT the high byte (byte6).
// Distinct values in byte6 (0xBB) vs byte7 (0x07) pin the field down.
#[test]
fn setgprm_register_compare_rhs_is_low_byte() {
// 71 20: set-GPRM (type 3), cmp=EQ(2), dircmp=0(register); lhs=byte2=2.
let c = decode(&h("712002000000bb07"));
let cmp = c.compare.expect("conditional");
assert_eq!(cmp.op, 2);
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 2);
assert_eq!(cmp.rhs_reg, 7); // low byte of bytes6-7, not the 0xBB high byte
}
// if_version_2 jump compare: both operands are registers in byte6 / byte7.
+238 -126
View File
@@ -45,6 +45,7 @@ pub const E_INVALID_CDB_LENGTH: u16 = 4001;
// I/O (5xxx)
pub const E_IO_ERROR: u16 = 5000;
pub const E_SOURCE_TERMINATED: u16 = 5001;
// Disc format (6xxx)
pub const E_DISC_READ: u16 = 6000;
@@ -63,6 +64,22 @@ pub const E_SELECTION_PID_UNKNOWN: u16 = 6014;
pub const E_UDF_BUFFER_TOO_SMALL: u16 = 6012;
pub const E_UDF_NOT_FILESYSTEM: u16 = 6013;
pub const E_IMAGE_TRUNCATED: u16 = 6015;
pub const E_UDF_AD_CHAIN_TOO_LONG: u16 = 6016;
pub const E_UDF_UNRECORDED_EXTENT: u16 = 6017;
pub const E_UDF_EMBEDDED_DATA: u16 = 6018;
/// A file that EXISTS and whose allocation descriptors resolved without error,
/// yet yields not one usable extent: an empty AD list, or a list every entry of
/// which is zero-length or points at LBA 0. Reported by the HD-DVD clip
/// resolver (`disc::hddvd`).
///
/// It has no [`Error`] variant on purpose. `UdfFs::file_extents` returns
/// `Ok(vec![])` here rather than failing — the emptiness is only a defect in
/// the eye of a caller that needs bytes — so the condition is DETECTED by the
/// caller, not returned to it. The code exists so that detection can be
/// accounted in the log with something other than a neighbouring error's code:
/// logging it as E6017 would file a zero-length AD list as an authoring hole
/// and send whoever triages it at the wrong population.
pub const E_UDF_NO_USABLE_EXTENT: u16 = 6019;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -393,6 +410,18 @@ pub enum Error {
UdfNotFound {
path: String,
},
/// The file's ICB allocation list contains an unrecorded (ECMA-167
/// 4/14.14.1.1 type-1/type-2) extent: space allocated to the file at that
/// location but never written, so its true content there is zeros while
/// the media holds something else.
///
/// Raised by [`crate::udf::UdfFs::file_extents`] because a
/// `(lba, sector_count)` read plan cannot express a hole — reading it
/// splices undefined sectors into the rip as content, and dropping it
/// slides every later extent's byte space.
UdfUnrecordedExtent {
path: String,
},
/// The reader was addressable but the bytes are structurally NOT a UDF
/// filesystem — a deterministic tag/format mismatch (e.g. no Anchor Volume
/// Descriptor Pointer at sector 256, no partition descriptor, no File Set
@@ -405,6 +434,32 @@ pub enum Error {
/// 2048-byte sector. A contract violation on the public reader API —
/// returned instead of panicking on the slice.
UdfBufferTooSmall,
/// A file's allocation-descriptor continuation chain did not end within the
/// hop budget the UDF reader allows.
///
/// The budget exists so a crafted or corrupt disc cannot loop the reader
/// forever. Hitting it is NOT the end of the chain: the extents beyond that
/// point are unknown, so the extent list in hand describes only part of the
/// file. Returning that list would let a caller zero-pad the remainder to
/// the declared size and report a mostly-empty file as a complete
/// extraction, so the read fails instead.
UdfAdChainTooLong,
/// A file's ICB declares its data EMBEDDED inline (ECMA-167 4/14.6.8
/// allocation-descriptor type 3), so it has no out-of-line extents at all.
///
/// Returned only when a caller asked for a read plan over such a file.
/// The bytes in the allocation-descriptor field are then the file's own
/// CONTENT, not descriptors, so decoding them as (length, LBA) pairs
/// manufactures extents out of arbitrary data and points the reader at
/// unrelated sectors — a rip that completes at rc=0 carrying whatever
/// happened to be there. `read_directory` already refuses the same shape
/// for directories; this is the file half of that decision.
///
/// A file that legitimately stores its data this way is tiny (an ICB caps
/// it at well under 2 KiB — the AACS `*.inf` key files are the usual
/// case), and the callers that expect one read it via `read_inline_data`
/// long before extents are ever requested. A stream file cannot be one.
UdfEmbeddedData,
DiscTitleRange {
index: usize,
count: usize,
@@ -733,6 +788,19 @@ pub enum Error {
/// it silently leaves encrypted. The producer surfaces this rather
/// than emit still-encrypted bytes.
ExtentNotUnitAligned,
/// A [`crate::sector::SectorSource`] that feeds its reads from a
/// producer thread has terminated for good — the thread exited after
/// an error or before delivering the extents it was given — so it can
/// never return another byte.
///
/// It exists because the alternative answer is a lie: a dead source
/// that reports `Ok(0)` is indistinguishable from end-of-stream, and
/// `DiscStream::fill_extents` legitimately reads a short count as a
/// skippable hole — zero-filling and advancing over every remaining
/// sector of the title and still returning success. Unlike a bad
/// sector, this condition cannot be retried at a smaller size or
/// skipped past, so every consumer must abort the pass on it.
SourceTerminated,
/// An MPEG-TS packet under construction violated the 188-byte fixed
/// size (over-long adaptation field, overflowing payload, or a
/// short/mis-assembled packet). Indicates a muxer invariant break,
@@ -850,13 +918,17 @@ impl Error {
Error::ScsiError { .. } => E_SCSI_ERROR,
Error::InvalidCdbLength { .. } => E_INVALID_CDB_LENGTH,
Error::IoError { .. } => E_IO_ERROR,
Error::SourceTerminated => E_SOURCE_TERMINATED,
Error::DiscRead { .. } => E_DISC_READ,
Error::Halted => E_HALTED,
Error::MplsParse => E_MPLS_PARSE,
Error::ClpiParse => E_CLPI_PARSE,
Error::UdfNotFound { .. } => E_UDF_NOT_FOUND,
Error::UdfUnrecordedExtent { .. } => E_UDF_UNRECORDED_EXTENT,
Error::UdfNotFilesystem => E_UDF_NOT_FILESYSTEM,
Error::UdfBufferTooSmall => E_UDF_BUFFER_TOO_SMALL,
Error::UdfAdChainTooLong => E_UDF_AD_CHAIN_TOO_LONG,
Error::UdfEmbeddedData => E_UDF_EMBEDDED_DATA,
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::ShortImageRead { .. } => E_SHORT_IMAGE_READ,
Error::EmptyImage => E_EMPTY_IMAGE,
@@ -1063,6 +1135,7 @@ impl std::fmt::Display for Error {
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::UdfUnrecordedExtent { path } => write!(f, "E{}: {}", self.code(), path),
Error::SeamPlanDroppedMost { dropped, written } => {
write!(f, "E{} {dropped}/{written}", self.code())
}
@@ -1128,7 +1201,21 @@ impl std::error::Error for Error {
impl From<std::io::Error> for Error {
fn from(e: std::io::Error) -> Self {
Error::IoError { source: e }
// If this `io::Error` is one WE produced (`From<Error> for io::Error`
// carries the typed value in its boxed payload), give the original
// back instead of burying it in `Error::IoError`. That wrapper is not
// neutral: `is_scsi_transport_failure` treats `IoError` as a
// transport-layer fault (dead bus / wedged bridge), so re-wrapping a
// round-tripped `DiscRead` MEDIUM ERROR turned a skippable bad sector
// into a pass-aborting bridge wedge — the exact inverse of what that
// arm exists for. Any error that crosses a thread boundary as an
// `io::Error` (the prefetch channel's `Batch`) keeps its
// classification, its SCSI status, and its sense data.
match e.downcast::<Error>() {
Ok(typed) => typed,
// A genuine OS/`std` error — the `IoError` wrapper is correct here.
Err(io) => Error::IoError { source: io },
}
}
}
@@ -1143,7 +1230,6 @@ impl From<Error> for std::io::Error {
return source;
}
let code = e.code();
let msg = e.to_string();
// Map our error categories to io::ErrorKind
let kind = match code {
// Device access-denied semantics map to PermissionDenied;
@@ -1244,7 +1330,13 @@ impl From<Error> for std::io::Error {
E_DIR_IMAGE_FILE_CHANGED => std::io::ErrorKind::InvalidData,
_ => std::io::ErrorKind::Other,
};
std::io::Error::new(kind, msg)
// Carry the typed value itself as the payload rather than its
// stringification. `Display` is unchanged (`io::Error` delegates to the
// boxed error, whose `Display` is the same `E<code>[: …]` string), so
// `error_code` and every consumer built on it are unaffected — but the
// typed error now SURVIVES the conversion and `From<io::Error> for
// Error` can hand it back intact.
std::io::Error::new(kind, e)
}
}
@@ -1263,13 +1355,14 @@ pub type Result<T> = std::result::Result<T, Error>;
/// removing comes back.
///
/// [`From<Error> for io::Error`] is the ONLY path from a typed [`Error`] to an
/// `io::Error` in this crate, and it stringifies (`io::Error::new(kind, msg)`
/// where `msg` is the `Error`'s `E<code>[: …]` [`Display`](std::fmt::Display)
/// string) rather than boxing the typed value — no code path constructs an
/// `io::Error` that still holds a `crate::error::Error` via `get_ref`. So the
/// only recognised shape is the round-tripped `E<code>` message prefix.
/// `io::Error` in this crate. It boxes the typed value as the payload
/// (`io::Error::new(kind, e)`), whose [`Display`](std::fmt::Display) is the
/// same `E<code>[: …]` string the stringifying version produced — so this
/// parse is unaffected, and `From<io::Error> for Error` can additionally
/// `downcast` the payload back to the exact typed error. Errors that did NOT
/// come from this crate carry no `E<code>` prefix and yield `None`.
pub fn error_code(e: &std::io::Error) -> Option<u16> {
// Round-tripped: `From<Error> for io::Error` stringifies as "E<code>[: …]".
// Round-tripped: `From<Error> for io::Error` renders as "E<code>[: …]".
let s = e.to_string();
let digits = s.strip_prefix('E')?;
let end = digits
@@ -1398,6 +1491,22 @@ impl Error {
)
}
/// True if the read SOURCE itself is gone, as opposed to one range of
/// media being unreadable. Kept separate from
/// [`is_scsi_transport_failure`](Self::is_scsi_transport_failure) —
/// which is about the bus/bridge and drives "power-cycle the drive"
/// advice — because a terminated producer thread is neither a wedged
/// bridge nor a bad sector, and reporting it as SCSI status 0xFF would
/// be a fabricated status byte.
///
/// What it shares with a transport failure is the only thing the read
/// loops need to know: retrying smaller or skipping ahead cannot
/// recover anything, so the pass must abort rather than fabricate
/// zeros for the rest of the title.
pub fn is_source_terminated(&self) -> bool {
matches!(self, Error::SourceTerminated)
}
/// True if this error indicates bridge degradation — the SCSI status
/// is neither GOOD (0x00), CHECK CONDITION (0x02), nor transport failure
/// (0xFF). Observed on the Initio INIC-1618L USB bridge preceding a full
@@ -1804,126 +1913,129 @@ mod tests {
// ── New comprehensive tests ────────────────────────────────────────────────
/// Every `pub const E_*` code declared in this file, as
/// `(name, value)`, PARSED OUT OF THE SOURCE at compile time.
///
/// WHY THIS IS PARSED AND NOT LISTED. The uniqueness test below used to
/// carry a hand-maintained `vec![]` of the constants, with a doc comment
/// claiming it "pins all code assignments". It did not. At the time this
/// was written the file declared **127** `pub const E_*` and the vector
/// named **109** of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
/// `E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`,
/// `E_DIR_NAME_COLLISION`, `E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`,
/// `E_DIR_RAW_REJECTED`, `E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`,
/// `E_DRIVE_INQUIRY_SHORT`, `E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`,
/// `E_SEAM_PLAN_DROPPED_MOST`, `E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`,
/// `E_SOURCE_TERMINATED`, `E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were
/// outside the guarantee entirely, so a new variant colliding with any of
/// them passed green. Worse, an earlier audit READ that doc comment and
/// trusted it while assigning new codes.
///
/// The defect is not the eighteen omissions, it is that the list is
/// hand-maintained at all: adding a constant and forgetting the vector is
/// a silent no-op, which is the definition of a guarantee that decays.
/// Deriving the list from the declarations makes forgetting impossible —
/// the only way to escape the check is to stop declaring the constant.
///
/// `include_str!` of this very file is the cheapest seam that does that:
/// no `build.rs`, no proc macro, no dependency, and — being `#[cfg(test)]`
/// — not one byte of the source embedded in a release build.
fn declared_error_codes() -> Vec<(&'static str, u16)> {
const SRC: &str = include_str!("error.rs");
SRC.lines()
.filter_map(|line| {
// Only real declarations at column 0. A retired code is
// recorded as `// 2001: burned/retired`, which carries no `pub
// const` and is therefore correctly invisible here.
let decl = line.strip_prefix("pub const ")?;
let (name, value) = decl.split_once(": u16 = ")?;
if !name.starts_with("E_") {
return None;
}
let value = value.strip_suffix(';')?;
Some((
name,
value
.parse::<u16>()
.unwrap_or_else(|_| panic!("non-literal error code for `{name}`")),
))
})
.collect()
}
/// The parser above must actually FIND the declarations, and read their
/// values correctly. Without this, a `declared_error_codes` that returned
/// an empty vector would make the uniqueness test below pass vacuously —
/// exactly the failure mode (a guarantee that is really a no-op) this
/// whole change exists to remove.
///
/// Mutation: a `strip_prefix` typo, an off-by-one in the name slice, or a
/// parser that drops the last line fails here.
#[test]
fn declared_error_codes_parses_the_declarations_it_claims_to() {
let declared = declared_error_codes();
// Independent count of the declarations, computed a different way
// from the parser under test.
let expected = include_str!("error.rs")
.lines()
.filter(|l| l.starts_with("pub const E_"))
.count();
assert_eq!(
declared.len(),
expected,
"the parser must see every `pub const E_*` line"
);
assert!(
expected >= 120,
"sanity floor: this file declares well over a hundred codes, got {expected}"
);
// Names and values must match the compiled constants, so a parser that
// mis-slices the name or mis-reads the digits cannot pass.
for (name, value) in [
("E_DEVICE_NOT_FOUND", E_DEVICE_NOT_FOUND),
("E_UDF_UNRECORDED_EXTENT", E_UDF_UNRECORDED_EXTENT),
("E_KEYDB_PARSE", E_KEYDB_PARSE),
] {
assert!(
declared.contains(&(name, value)),
"`{name}` = {value} not parsed out of the source; got {:?}",
declared
.iter()
.filter(|(n, _)| *n == name)
.collect::<Vec<_>>()
);
}
// A comment-only retired code must NOT be picked up as a constant.
assert!(
!declared.iter().any(|(n, _)| n.is_empty()),
"no empty names"
);
}
/// Every published error code constant must be unique.
/// This pins all code assignments: a new variant that accidentally reuses
/// an existing code will make this test fail.
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating E_KEYDB_CONNECT) fails here.
///
/// The set is derived from the declarations by
/// [`declared_error_codes`], so — unlike the hand-kept vector this
/// replaced — a newly added constant is covered the moment it is
/// written, with no second edit to remember.
///
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating
/// E_KEYDB_CONNECT) fails here, and so now does the same collision on any
/// of the eighteen codes the old hand-kept list had never heard of.
#[test]
fn all_error_code_constants_are_unique() {
let mut codes = vec![
E_DEVICE_NOT_FOUND,
E_DEVICE_PERMISSION,
E_DEVICE_NOT_READY,
E_DEVICE_RESET_FAILED,
E_SCSI_INTERFACE_UNAVAILABLE,
E_DEVICE_LOCKED,
E_IOKIT_PLUGIN_FAILED,
E_UNSUPPORTED_DRIVE,
E_PROFILE_PARSE,
E_UNSUPPORTED_PLATFORM,
E_PLATFORM_NOT_IMPLEMENTED,
E_UNLOCK_FAILED,
E_SIGNATURE_MISMATCH,
E_SCSI_ERROR,
E_INVALID_CDB_LENGTH,
E_IO_ERROR,
E_DISC_READ,
E_MPLS_PARSE,
E_CLPI_PARSE,
E_UDF_NOT_FOUND,
E_DISC_TITLE_RANGE,
E_IFO_PARSE,
E_MKV_INVALID,
E_NO_STREAMS,
E_HALTED,
E_MAPFILE_INVALID,
E_IMAGE_TRUNCATED,
E_UDF_BUFFER_TOO_SMALL,
E_UDF_NOT_FILESYSTEM,
E_AACS_NO_KEYS,
E_AACS_CERT_SHORT,
E_AACS_AGID_ALLOC,
E_AACS_CERT_REJECTED,
E_AACS_CERT_READ,
E_AACS_CERT_VERIFY,
E_AACS_KEY_READ,
E_AACS_KEY_REJECTED,
E_AACS_KEY_VERIFY,
E_AACS_VID_READ,
E_AACS_VID_MAC,
E_AACS_DATA_KEY,
E_DECRYPT_FAILED,
E_CSS_AUTH_FAILED,
E_AACS_HOST_CERT_REJECTED,
E_AACS_RAW_READ_UNSUPPORTED,
E_AACS_VID_UNAVAILABLE,
E_AACS_MK_UNAVAILABLE,
E_AACS_VUK_NOT_IN_KEYDB,
E_DRIVE_PROFILE_MISSING,
E_VID_CDB_UNAVAILABLE,
E_NO_DISC_KEY,
E_CSS_KEY_MISSING,
E_CSS_NO_DISC_KEY,
E_KEY_SERVICE_UNAVAILABLE,
E_KEY_SERVICE_UNAUTHORIZED,
E_KEY_SERVICE_RATE_LIMITED,
E_AACS_NO_HOST_CERT,
E_AACS_BUS_KEY_UNAVAILABLE,
E_FMTS_KEY_MISSING,
E_KEYDB_CONNECT,
E_KEYDB_HTTP,
E_KEYDB_INVALID,
E_KEYDB_WRITE,
E_KEYDB_PARSE,
E_KEYDB_LOAD,
E_KEYDB_UNSUPPORTED_SCHEME,
E_KEYDB_TOO_MANY_REDIRECTS,
E_STREAM_READ_ONLY,
E_STREAM_WRITE_ONLY,
E_STREAM_URL_INVALID,
E_STREAM_URL_MISSING_PATH,
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MUX_HEADER_BUFFER_EXCEEDED,
E_MKV_LACING_INVALID,
E_MKV_SOURCE_INVALID,
E_MKV_UNENCODABLE,
E_MP4_NO_VIDEO_TRACK,
E_MP4_INVALID,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
E_ISO_TOO_LARGE,
E_NO_METADATA,
E_DISC_URL_NOT_DIRECT,
E_HEVC_PARAM_PARSE,
E_MUX_TRACK_RANGE,
E_FMP4_UNIMPLEMENTED,
E_DEMUX_THREAD_PANICKED,
E_PIPELINE_JOIN_TIMEOUT,
E_PIPELINE_CONSUMER_PANICKED,
E_SWEEP_CONSUMER_GONE,
E_PIPELINE_CONSUMER_GONE,
E_DISC_CAPACITY_OVERFLOW,
E_M2TS_PACKET_MALFORMED,
E_EXTENT_NOT_UNIT_ALIGNED,
E_DISC_CAPACITY_MALFORMED,
E_DIR_IMAGE_SSIF_UNSUPPORTED,
E_DIR_IMAGE_PLACEMENT,
E_DIR_IMAGE_ENCRYPTED,
E_DIR_IMAGE_UNSUPPORTED_TREE,
E_DIR_IMAGE_FILE_CHANGED,
E_DIR_IMAGE_TOO_LARGE,
];
let original_len = codes.len();
codes.sort();
codes.dedup();
assert_eq!(
codes.len(),
original_len,
"duplicate error code constants detected — check error.rs"
let mut by_code: std::collections::BTreeMap<u16, Vec<&str>> =
std::collections::BTreeMap::new();
for (name, value) in declared_error_codes() {
by_code.entry(value).or_default().push(name);
}
let dupes: Vec<_> = by_code
.iter()
.filter(|(_, names)| names.len() > 1)
.collect();
assert!(
dupes.is_empty(),
"duplicate error code constants detected — check error.rs: {dupes:?}"
);
}
+120
View File
@@ -180,6 +180,31 @@ fn clpi_parse_never_panics() {
});
}
#[test]
fn bdnav_index_parse_never_panics() {
sweep("bdnav_index", b"INDX", |b| {
let _ = crate::bdnav::index::parse(b);
});
}
#[test]
fn bdnav_mobj_parse_never_panics() {
sweep("bdnav_mobj", b"MOBJ", |b| {
let _ = crate::bdnav::mobj::parse(b);
});
}
#[test]
fn dvd_nav_resolve_never_panics() {
// The DVD First-Play navigation resolver parses VMGI (`VIDEO_TS.IFO`) —
// FP_PGC command table + TT_SRPT, all attacker-controlled offsets/counts —
// and executes an untrusted command list on a bounded VM. Magic is the VMGI
// signature so the sweep reaches past the header guard into the executor.
sweep("dvd_nav", b"DVDVIDEO-VMG", |b| {
let _ = crate::dvdnav::nav::resolve_from_vmg(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
@@ -189,6 +214,101 @@ fn udf_name_parse_never_panics() {
});
}
#[test]
fn class_reader_parse_never_panics() {
// Java `.class` files are a documented format we READ (never execute) to
// recover BD-J stream labels — a peer of mpls/clpi here. Every count,
// length and constant-pool index in the file is attacker-controlled disc
// metadata, so the parser must reject malformed input with `Err`, never
// panic. Magic is the class-file magic `0xCAFEBABE`.
sweep("class_reader", &[0xCA, 0xFE, 0xBA, 0xBE], |b| {
if let Ok(class) = crate::labels::class_reader::ClassFile::parse(b) {
// Exercise the accessors a label parser drives, so their indexing
// is fuzzed too, not just the structural parse.
let _ = class.this_class_name();
let _ = class.super_class_name();
for m in &class.methods {
let _ = class.member_name(m);
if let Some(code) = m.code(&class.constant_pool) {
for insn in code.instructions() {
let _ = insn.cp_index();
if let Some(idx) = insn.cp_index() {
let _ = class.constant_pool.get(idx);
let _ = class.constant_pool.member_ref(idx);
let _ = class.constant_pool.load_constant_display(idx);
}
}
}
}
}
});
}
#[test]
fn jar_class_walk_never_panics() {
// The jar layer opens a `/BDMV/JAR/<x>.jar` (a zip) and walks every
// `.class` inside it, handing each to the class reader. Feed the sweep's
// bytes as a zip archive; when they parse as one, walk the classes exactly
// as `deluxe`/`dbp` do. Neither the zip open nor the class walk may panic.
// Magic is the zip local-file-header signature `PK\x03\x04`.
sweep("jar", &[0x50, 0x4B, 0x03, 0x04], |b| {
if let Ok(mut archive) = zip::ZipArchive::new(std::io::Cursor::new(b.to_vec())) {
crate::labels::jar::for_each_class(&mut archive, |_name, _class| {});
}
});
}
#[test]
fn deluxe_bytecode_decode_never_panics() {
// The Deluxe BD-J label parser runs a symbolic stack machine over `<clinit>`
// bytecode (`decode_binding_class`). Fuzzing it THROUGH a zip is vacuous —
// `zip::ZipArchive::new` validates the end-of-central-directory record at the
// buffer's end, which the generators (front magic + noise) cannot synthesize,
// so the archive open always fails and the decoder is never reached. Drive
// the decoder DIRECTLY instead: parse the sweep bytes as a `.class` (they also
// exercise the class parser) and run the bytecode decoder over it with an
// empty master-enum table. None may panic on crafted/corrupt bytecode.
let master = crate::labels::deluxe::MasterEnumTable::from(&[] as &[(
&'static str,
crate::labels::deluxe::MasterEnum,
)]);
sweep("deluxe_decode", &[0xCA, 0xFE, 0xBA, 0xBE], |b| {
if let Ok(class) = crate::labels::class_reader::ClassFile::parse(b) {
let _ = crate::labels::deluxe::decode_binding_class(&class, &master);
}
});
}
#[test]
fn fox_dcx_labels_never_panics() {
// Fox's `dcx.xml` is untrusted disc-authored XML; `labels_from_dcx` scans it
// with byte-offset XML helpers documented as UTF-8-boundary-prone. No magic —
// feed lossy UTF-8 so multi-byte boundaries inside tags/attributes are hit.
sweep("fox_dcx", b"<", |b| {
let text = String::from_utf8_lossy(b);
let _ = crate::labels::fox::labels_from_dcx(&text);
});
}
#[test]
fn bdnav_vm_resolve_never_panics() {
// The HDMV navigation VM executes an untrusted `MovieObject.bdmv` command
// list on a bounded VM. Parse the sweep bytes as a MovieObject set and run
// the VM from a First-Play entry — the combined parse+execute path a real
// disc drives, which the per-parser index/mobj sweeps never reach.
use crate::bdnav::index::{Index, PlaybackObj};
sweep("bdnav_vm", b"MOBJ", |b| {
if let Some(mobjs) = crate::bdnav::mobj::parse(b) {
let index = Index {
first_play: PlaybackObj::Hdmv { id_ref: 0 },
top_menu: PlaybackObj::Hdmv { id_ref: 0 },
titles: Vec::new(),
};
let _ = crate::bdnav::vm::resolve(&index, &mobjs, &|_| false);
}
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
+124 -51
View File
@@ -49,7 +49,7 @@ pub struct DvdTitle {
pub cells: Vec<DvdCell>,
/// Chapter start times in seconds (derived from program map + cell times)
pub chapter_times: Vec<f64>,
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cb, Cr].
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cr, Cb].
pub palette: Option<Vec<[u8; 4]>>,
}
@@ -149,8 +149,8 @@ impl DvdTitle {
/// - As a final guard it never returns past the last cell, and never drops
/// when that would leave zero cells.
///
/// For "The Silence of the Lambs" (every feature cell category `0x00`,
/// chapter 1 at 00:00:00) this returns 0 — a no-op — which is the correct
/// For a real disc where every feature cell category is `0x00` and
/// chapter 1 sits at 00:00:00, this returns 0 — a no-op — which is the correct
/// result: the disc's scene-index lives in a separate menu/title PGC, not
/// in leading cells of the feature PGC, so there is nothing to drop here.
pub fn feature_start_cell(&self) -> usize {
@@ -425,6 +425,13 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
for (&vts_number, titles_info) in &title_set_map {
match parse_vts(reader, udf, vts_number, titles_info) {
Ok(ts) => title_sets.push(ts),
// The operator's Stop is not a placeholder title set. Once the
// drive's halt flag is set EVERY command fails with `Halted`, so
// swallowing it here would skip every REMAINING title set in turn
// and return `Ok` with a truncated list — a cancelled scan that
// is indistinguishable from a disc genuinely holding fewer
// titles. `scan_dvd_titles` propagates it to the caller.
Err(Error::Halted) => return Err(Error::Halted),
Err(e) => {
// Some discs carry placeholder TT_SRPT entries for title sets
// that are not really there, so one failure is not fatal. But
@@ -571,7 +578,7 @@ fn parse_vts(
// values are in turn relative to `vtstt_vobs`. To turn them into the absolute
// disc LBAs the reader needs, add the IFO file's own on-disc location (from
// the UDF FS). Without this rebase every extent started `ifo_lba` sectors too
// early — for THESILENCEOFTHELAMBS the feature began at LBA 126 (the VMGI /
// early — on one real disc the feature began at LBA 126 (the VMGI /
// VIDEO_TS.VOB main-menu region) instead of 132886 (VTS_03_1.VOB), so the
// first ~4.5 min of muxed video was the disc's main menu before the stream
// drifted into the movie.
@@ -691,7 +698,16 @@ fn parse_video_attr(data: &[u8]) -> Result<DvdVideoAttr> {
}
/// Parse one audio stream attribute block (8 bytes at `offset`).
fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
/// `pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
/// `parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
/// calls it directly so its language-mapping tests run the real parser over real
/// on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
/// could agree with the muxer while both disagree with the disc. Narrowing this
/// would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
/// that can drift apart, or moving those tests away from the code they exist to
/// pin — both worse than the widened crate-internal visibility, which reaches no
/// public API.
pub(crate) fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let b0 = byte_at(data, offset)?;
let b1 = byte_at(data, offset + 1)?;
@@ -718,25 +734,9 @@ fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let channels = (b1 & 0x07) + 1; // (channels - 1) in low 3 bits of byte 1
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
{
String::from_utf8_lossy(lang_bytes).to_string()
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
// Try to interpret as printable ASCII
let s: String = lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdAudioAttr {
codec,
@@ -785,9 +785,27 @@ fn assign_audio_sub_stream_ids(streams: &mut [DvdAudioAttr]) {
/// Parse one subtitle stream attribute block (6 bytes at `offset`).
fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdSubtitleAttr { language })
}
/// Decode the raw 2-byte on-disc language code shared by the audio and
/// subtitle attribute blocks. A pair of lowercase a-z bytes is taken
/// verbatim (the ISO 639-1 code the DVD-Video spec puts there); an all-zero
/// pair means unspecified (empty string); anything else falls through to an
/// ASCII-alphanumeric salvage — letters (either case) and digits are kept,
/// everything else (control bytes, punctuation, high bytes from a corrupt or
/// hostile disc) is dropped.
///
/// The salvage is deliberately not narrowed to a-z: whatever survives is only
/// ever a lookup key for [`dvd_lang_to_iso639_2`], which degrades anything it
/// does not recognize to `und`, so a stray `X` or `5` costs nothing and cannot
/// reach an output stream as a language code.
fn parse_raw_dvd_lang_bytes(lang_bytes: &[u8]) -> String {
if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
@@ -796,15 +814,40 @@ fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
let s: String = lang_bytes
lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
.collect()
}
}
Ok(DvdSubtitleAttr { language })
/// Convert a DVD IFO audio/subtitle attribute's language code — ISO 639-1
/// (2 lowercase letters) per the DVD-Video spec, or empty when unspecified —
/// to the ISO 639-2 form every downstream consumer (`disc::AudioStream` /
/// `disc::SubtitleStream::language`, and in turn Matroska's `Language`
/// element per RFC 9559 §12 and the MP4 sink's `mdhd` language) requires.
///
/// Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
/// 639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
/// frozen-1988 language list still puts on disc). The narrower
/// `vocab::menu_lang` table is deliberately NOT used here: it exists for
/// Blu-ray menu-graphic filename tokens and knows only 25 languages, so a
/// Region-2 disc's Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian,
/// Hebrew, Estonian, Latvian, Lithuanian and Icelandic tracks would all fold
/// onto `und` together. DVD streams carry an empty `label`, so the language
/// is the only thing distinguishing one subtitle track from the next — a
/// valid code that is identical for six tracks is worse for the user than the
/// invalid one it replaced. Both tables normalize to ISO 639-2/T, so they
/// agree wherever they overlap.
///
/// An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's
/// own "undetermined" value) — a valid element value — rather than passing
/// through an invalid 2-letter code or an empty string. Never guesses.
fn dvd_lang_to_iso639_2(raw: &str) -> String {
crate::labels::vocab::iso639_1_to_iso639_2(raw)
.unwrap_or("und")
.to_string()
}
// ── PGC parser ──────────────────────────────────────────────────────────────
@@ -1016,7 +1059,7 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
times
};
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cb, Cr]
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cr, Cb]
let palette = if pgc_offset + 0xA4 + 64 <= data.len() {
let mut colors = Vec::with_capacity(16);
for i in 0..16 {
@@ -1441,7 +1484,7 @@ mod tests {
data[0] = 0x00;
// b1: bits 2-0=101 (channels-1=5) => 0x05
data[1] = 0x05;
// language "en"
// on-disc language "en" (ISO 639-1) -> parsed as ISO 639-2 "eng"
data[2] = b'e';
data[3] = b'n';
@@ -1449,7 +1492,7 @@ mod tests {
assert_eq!(attr.codec, Codec::Ac3);
assert_eq!(attr.sample_rate, 48000);
assert_eq!(attr.channels, 6);
assert_eq!(attr.language, "en");
assert_eq!(attr.language, "eng");
}
#[test]
@@ -1579,7 +1622,7 @@ mod tests {
assert_eq!(attr.codec, Codec::Dts);
assert_eq!(attr.sample_rate, 96000);
assert_eq!(attr.channels, 2);
assert_eq!(attr.language, "fr");
assert_eq!(attr.language, "fra");
}
// ─────────────────────────────────────────────────────────────────────
@@ -1693,15 +1736,17 @@ mod tests {
}
/// Audio language bytes [offset+2..+4]: when both bytes are 0x00 the
/// language is the empty string (unspecified), per source.
/// on-disc code is unspecified, and `dvd_lang_to_iso639_2` maps that to
/// the valid ISO 639-2 "und" (undetermined) rather than an empty string,
/// which is not a legal Matroska `Language` element value.
#[test]
fn audio_attr_zero_language_is_empty() {
fn audio_attr_zero_language_becomes_und() {
let mut data = vec![0u8; 8];
data[0] = 0x00;
data[2] = 0x00;
data[3] = 0x00;
let attr = parse_audio_attr(&data, 0).unwrap();
assert_eq!(attr.language, "");
assert_eq!(attr.language, "und");
}
/// Audio sample_rate flag (b0>>3 & 0x03): 0=48kHz, 1=96kHz, else 48kHz.
@@ -1722,11 +1767,11 @@ mod tests {
data[2] = b'd';
data[3] = b'e';
let attr = parse_subtitle_attr(&data, 0).unwrap();
assert_eq!(attr.language, "de");
assert_eq!(attr.language, "deu");
let zero = vec![0u8; 6];
let attr2 = parse_subtitle_attr(&zero, 0).unwrap();
assert_eq!(attr2.language, "");
assert_eq!(attr2.language, "und");
}
/// assign_audio_sub_stream_ids: MP1/MP2 and other non-private-stream-1
@@ -1812,8 +1857,8 @@ mod tests {
assert_eq!(title.cells[1].first_sector, 20);
}
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cb, Cr].
/// A palette with at least one non-zero Y/Cb/Cr is returned as Some;
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cr, Cb].
/// A palette with at least one non-zero Y/Cr/Cb is returned as Some;
/// an all-zero palette returns None (source filters empty palettes).
#[test]
fn pgc_palette_present_and_empty() {
@@ -1833,14 +1878,14 @@ mod tests {
assert!(title2.palette.is_none());
}
/// parse_pgc palette layout: each color is [padding, Y, Cb, Cr] and the
/// parse_pgc palette layout: each color is [padding, Y, Cr, Cb] and the
/// "non-empty" test ignores the padding byte (index 0). A palette whose
/// ONLY non-zero bytes are padding must still be treated as empty (None).
#[test]
fn pgc_palette_padding_only_is_empty() {
let mut pgc = vec![0u8; 0xEA];
pgc[0x03] = 0;
// Set padding byte (index 0) of color 0 non-zero, but Y/Cb/Cr zero.
// Set padding byte (index 0) of color 0 non-zero, but Y/Cr/Cb zero.
pgc[0xA4] = 0xFF;
let title = parse_pgc(&pgc, 0, 1).unwrap();
assert!(
@@ -1893,8 +1938,8 @@ mod tests {
/// Regression test for issue freemkv#25 (NTSC chapter drift).
///
/// Builds a PGC from the real cell table of the DOLL_MASTERS_7 disc in the
/// bug report and checks the chapter marks against MakeMKV's values. Before
/// Builds a PGC from the real cell table of the disc in the
/// bug report and checks the chapter marks against the reference tool's values. Before
/// the fix, chapter 14 landed 4.019 s early; the drift was proportional to
/// elapsed time (0.1%), so a short synthetic fixture would not have caught it.
#[test]
@@ -1942,7 +1987,7 @@ mod tests {
}
let title = parse_pgc(&pgc, 0, pgm.len() as u16).unwrap();
// MakeMKV / source truth, in seconds.
// Reference / source truth, in seconds.
let expected = [
0.0,
62.062,
@@ -2070,7 +2115,7 @@ mod tests {
/// A normal single-angle feature (every cell category 0x00) is never
/// filtered: feature_start_cell == 0, feature_cells == all cells. This is
/// the "Silence of the Lambs" case — the filter must be a no-op.
/// the plain-single-angle-feature case — the filter must be a no-op.
#[test]
fn feature_filter_noop_on_plain_feature() {
let t = DvdTitle {
@@ -2278,10 +2323,13 @@ mod tests {
/// two bytes at +2. Only a pair of lowercase a-z bytes is taken verbatim;
/// anything else falls through to the ASCII-alphanumeric salvage, which
/// keeps only the usable characters. A byte outside a-z must never end up
/// in the language string.
/// in the raw salvaged string. This exercises `parse_raw_dvd_lang_bytes`
/// directly — the byte-level salvage step — separately from the ISO
/// 639-1 -> 639-2 mapping `parse_audio_attr`/`parse_subtitle_attr` apply
/// on top (see `dvd_two_letter_and_malformed_language_becomes_iso639_2`).
#[test]
fn language_code_rejects_non_lowercase_bytes() {
// (byte0, byte1, expected language)
// (byte0, byte1, expected raw salvage)
let cases: [(u8, u8, &str); 8] = [
(b'e', b'n', "en"), // both in range → verbatim
(0x21, b'n', "n"), // '!' is below 'a'
@@ -2292,6 +2340,31 @@ mod tests {
(0x00, b'E', "E"), // only the first byte is zero
(0x00, 0x00, ""), // both zero → unset
];
for (b0, b1, want) in cases {
assert_eq!(
parse_raw_dvd_lang_bytes(&[b0, b1]),
want,
"raw language salvage for ({b0:#04x}, {b1:#04x})"
);
}
}
/// The full pipeline `parse_audio_attr`/`parse_subtitle_attr` apply on
/// top of the raw salvage: a valid ISO 639-1 code maps to its ISO 639-2
/// equivalent, and anything the raw salvage does NOT produce a mapped
/// code for (empty, or a single leftover letter from a malformed byte
/// pair) degrades to "und" — never an invalid 2-letter/1-letter code and
/// never an empty string, both of which are illegal Matroska `Language`
/// element values (RFC 9559 §12).
#[test]
fn dvd_two_letter_and_malformed_language_becomes_iso639_2() {
// (byte0, byte1, expected final language)
let cases: [(u8, u8, &str); 4] = [
(b'e', b'n', "eng"), // valid ISO 639-1 → mapped
(0x21, b'n', "und"), // malformed → salvage "n", unmapped → und
(b'E', 0x00, "und"), // malformed → salvage "E", unmapped → und
(0x00, 0x00, "und"), // unspecified → und
];
for (b0, b1, want) in cases {
let mut audio = vec![0u8; 8];
audio[2] = b0;
@@ -2496,7 +2569,7 @@ mod tests {
}
/// The subtitle palette is 16 entries of 4 bytes at PGC+0xA4, each
/// `[padding, Y, Cb, Cr]`. Every byte of every entry is distinct here, so
/// `[padding, Y, Cr, Cb]`. Every byte of every entry is distinct here, so
/// a wrong stride, a wrong base or a shifted component shows up.
#[test]
// The loop variable is the DOMAIN VALUE being checked (a palette entry number), not a
@@ -2524,7 +2597,7 @@ mod tests {
}
}
/// A palette is "present" when ANY of Y, Cb or Cr is non-zero in ANY
/// A palette is "present" when ANY of Y, Cr or Cb is non-zero in ANY
/// entry — a single non-zero chroma component is enough. Only the
/// padding byte [0] is ignored.
#[test]
+71
View File
@@ -1723,4 +1723,75 @@ mod tests {
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
// ── Malformed-input hardening: every count/length/index is attacker- ──────
// controlled, so an oversized or lying field must yield Err/None, never a
// panic or an unbounded allocation.
/// A `constant_pool_count` far larger than the data behind it must fail
/// cleanly (running out of bytes), not pre-allocate gigabytes or panic. The
/// count is a u16, so `Vec::with_capacity` is bounded at 65535 regardless.
#[test]
fn parse_rejects_oversized_constant_pool_count() {
let bytes = [
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, 0x00, 0x00, // minor / major
0xFF, 0xFF, // constant_pool_count = 65535, but no entries follow
];
assert!(matches!(
ClassFile::parse(&bytes),
Err(Error::UnexpectedEof { .. })
));
}
/// An oversized `interfaces_count` / `fields_count` with no data behind it
/// must fail on the first missing element, not panic.
#[test]
fn parse_rejects_oversized_interface_count() {
let bytes = [
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, 0x00, 0x00, // minor / major
0x00, 0x01, // constant_pool_count = 1 (no entries)
0x00, 0x00, // access_flags
0x00, 0x00, // this_class
0x00, 0x00, // super_class
0xFF, 0xFF, // interfaces_count = 65535, none follow
];
assert!(matches!(
ClassFile::parse(&bytes),
Err(Error::UnexpectedEof { .. })
));
}
/// Out-of-range constant-pool indices resolve to `None` from every
/// accessor a label parser drives — no slice panic.
#[test]
fn constant_pool_out_of_range_index_is_none_everywhere() {
let pool = sample_pool();
let oob = 9999u16;
assert!(pool.get(oob).is_none());
assert!(pool.utf8(oob).is_none());
assert!(pool.class_name(oob).is_none());
assert!(pool.string(oob).is_none());
assert!(pool.integer(oob).is_none());
assert!(pool.member_ref(oob).is_none());
assert!(pool.load_constant_display(oob).is_none());
}
/// A `Code` attribute whose declared `code_length` runs past the attribute
/// body must be rejected, not read out of bounds.
#[test]
fn code_attribute_rejects_length_past_its_body() {
// max_stack(2) + max_locals(2) + code_length(4) then a code_length that
// exceeds the remaining bytes.
let mut info = Vec::new();
info.extend_from_slice(&0u16.to_be_bytes()); // max_stack
info.extend_from_slice(&0u16.to_be_bytes()); // max_locals
info.extend_from_slice(&0xFFFF_FFFFu32.to_be_bytes()); // code_length: absurd
info.extend_from_slice(&[0x00, 0x01]); // only 2 code bytes present
assert!(matches!(
parse_code_attribute(&info),
Err(Error::UnexpectedEof { .. })
));
}
}
+408 -38
View File
@@ -48,16 +48,34 @@
//!
//! ## Confidence
//!
//! [`parse`] returns `Some(ParseResult::medium(labels))` when Phases A
//! through D produce at least one stream — `Medium` because the
//! signal-to-label mapping is heuristic. `None` when the disc isn't
//! Deluxe-authored or when decoding produces zero streams (a
//! recognized-but-broken state that the analyzer still surfaces via
//! `parsers_detected`).
//! [`parse`] returns `Some(ParseResult::high(labels))` when Phases A
//! through D produce at least one stream: the master enums matched their
//! framework-stable fingerprints (a strong ordered-prefix signature), the
//! binding class decoded, and at least one per-stream binding resolved to a
//! real (language, purpose, codec) tuple — the schema was fully recovered,
//! not guessed. `None` when the disc isn't Deluxe-authored or when decoding
//! produces zero streams (a recognized-but-broken state that the analyzer
//! still surfaces via `parsers_detected`).
//!
//! ## Confirmed studio variants
//!
//! - **Universal** (`studio="uni"`): Language enum `pd` (65 values,
//! `English, French, Spanish, Dutch, …`), Purpose enum `lp`
//! (`Normal, Commentary, PiP, Trivia, Descriptive, Score`), audio binding
//! `np.<init>(I, Lpd;, Llp;, Lorg/bluray/ti/CodingType;)`, subtitle binding
//! `wb.<init>(I, Lpd;, Llp;, Lmi;)`, all built in one binding class's
//! `<clinit>` alongside a title-wrapper object whose constructor takes the
//! per-stream arrays (filtered out by the array-parameter guard in the
//! decoder). SDH/RNIB is encoded as a distinct Language VALUE
//! ("English SDH", "English RNIB"), recovered into the qualifier from the
//! name. Grounded on fixtures captured from a real Universal Blu-ray release.
//! - **Disney/Warner**: 70-value Language enum, same binding shape; the
//! original corpus this parser was written against.
use super::class_reader::{
AASTORE, BIPUSH, ClassFile, CodeAttribute, ConstantPool, CpInfo, GETSTATIC, ICONST_0, ICONST_1,
ICONST_2, ICONST_3, ICONST_4, ICONST_5, ICONST_M1, INVOKESPECIAL, LDC, LDC_W, NEW, SIPUSH,
ICONST_2, ICONST_3, ICONST_4, ICONST_5, ICONST_M1, INVOKESPECIAL, LDC, LDC_W, NEW, PUTSTATIC,
SIPUSH,
};
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, jar, vocab};
use crate::sector::SectorSource;
@@ -76,6 +94,13 @@ pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
}
pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult> {
// Per-studio dispatch signal. Deluxe authored the framework for several
// studios (Universal `studio="uni"`, Fox, WB, Disney/Pixar); each ships a
// `/BDMV/JAR/<n>/config.xml` naming the studio. The structural enum/binding
// matching below is studio-agnostic, so this is currently informational
// (logged, and available for future per-studio special-casing), not a gate.
let studio = detect_studio(reader, udf);
jar::for_each_jar(reader, udf, |entry_name, archive| {
if !jar::has_path_prefix(archive, "com/bydeluxe/") {
return None;
@@ -85,16 +110,16 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let enums = identify_master_enums(archive);
if enums.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: com/bydeluxe/ present but no master enum fingerprint matched"
);
return None;
}
for (label, m) in &enums {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
enum = %label,
class = %m.class_name,
class = ?m.class_name,
count = m.values.len(),
"deluxe master enum identified",
);
@@ -110,15 +135,15 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let binding_classes = find_binding_classes(archive, &master_table.class_name_set());
if binding_classes.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: no binding class found (no class has enough getstatic refs to master enums)"
);
return None;
}
for (name, count) in &binding_classes {
tracing::info!(
jar = %entry_name,
binding_class = %name,
jar = ?entry_name,
binding_class = ?name,
getstatic_count = count,
"deluxe binding class candidate",
);
@@ -138,7 +163,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
if streams.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: binding classes found but produced 0 decoded streams"
);
return None;
@@ -149,25 +174,83 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
studio = ?studio,
audio = labels.iter().filter(|l| l.stream_type == StreamLabelType::Audio).count(),
subtitle = labels.iter().filter(|l| l.stream_type == StreamLabelType::Subtitle).count(),
"deluxe emitted labels",
);
// Medium confidence: Phase D's signal-to-label mapping is a
// documented heuristic until corpus-disc bytecode confirms
// the exact binding pattern.
Some(ParseResult::medium(labels))
// High confidence: the master enums matched their framework-stable
// fingerprints, the binding class decoded its `<clinit>`, and at least
// one per-stream binding resolved to a real (language, purpose, codec)
// tuple. Un-named STN slots are still back-filled from the MPLS floor
// by the registry (`merge_mpls_floor`).
Some(ParseResult::high(labels))
})
}
/// Read the studio identifier from a Deluxe disc's `config.xml`
/// (`<TitleConfig studio="uni"…>`), if present. The file lives under a
/// numbered subdirectory of `/BDMV/JAR/` (e.g. `/BDMV/JAR/99999/config.xml`),
/// so this scans the subdirectories for the first one that carries it.
///
/// Returns the lowercased studio token (`"uni"`, `"fox"`, `"wb"`, …) or `None`
/// when no readable `config.xml` names a studio.
fn detect_studio(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<String> {
let dir = udf.find_dir("/BDMV/JAR")?;
for entry in &dir.entries {
if !entry.is_dir {
continue;
}
let path = format!("/BDMV/JAR/{}/config.xml", entry.name);
let Ok(bytes) = udf.read_file(reader, &path) else {
continue;
};
if let Some(studio) = parse_studio_attr(&bytes) {
return Some(studio);
}
}
None
}
/// Extract the `studio="…"` attribute value from a `config.xml` byte buffer.
/// Deliberately a tolerant substring scan rather than a full XML parse: the
/// file is tiny attacker-controlled disc metadata and the only field of
/// interest is this one attribute. Rejects an empty or implausibly long value.
fn parse_studio_attr(xml: &[u8]) -> Option<String> {
const MAX_STUDIO_LEN: usize = 32;
let text = std::str::from_utf8(xml).ok()?;
let anchor = text.find("studio")?;
let after_key = &text[anchor + "studio".len()..];
// Skip whitespace / '=' up to the opening quote.
let open = after_key.find(['"', '\''])?;
let quote = after_key.as_bytes()[open];
let rest = &after_key[open + 1..];
let close = rest.find(quote as char)?;
let val = rest[..close].trim();
if val.is_empty() || val.len() > MAX_STUDIO_LEN {
return None;
}
Some(val.to_ascii_lowercase())
}
/// One identified master enum class.
#[derive(Debug)]
pub(crate) struct MasterEnum {
/// Obfuscated class name (e.g. `be.class`, `aw.class`).
pub class_name: String,
/// Ordinal → string-value mapping, in declaration order.
/// Ordinal → string-value mapping, in declaration order (the `ldc`
/// operands of each enum constant's construction).
pub values: Vec<String>,
/// Ordinal → static-field name, in declaration order (the `putstatic`
/// target that stores each enum constant). These are the obfuscated
/// names (`a`, `b`, `c`, …) that the binding class references via
/// `getstatic`, so Phase D resolves a `getstatic <enum>.<field>` by
/// looking the field name up here. Empty (falls back to value-keyed
/// resolution) only for synthetically-built test enums; real enums
/// captured from bytecode always populate it. See
/// [`clinit_enum_field_names`] and [`MasterEnumTable::from`].
pub fields: Vec<String>,
}
/// Fingerprints we use to identify each master enum class. The
@@ -180,6 +263,14 @@ struct Fingerprint {
label: &'static str,
prefix: &'static [&'static str],
expected_count: usize,
/// Half-width of the accepted count window around `expected_count`.
/// The ordered `prefix` is the real discriminator (four-plus exact
/// display strings in declaration order is not something a non-enum
/// class reproduces), so the count check only guards against a wildly
/// different class; the window is per-fingerprint because studios ship
/// different-sized enums (Universal's Language enum has 65 values,
/// Disney/Warner's has 70).
count_tolerance: usize,
}
const FINGERPRINTS: &[Fingerprint] = &[
@@ -187,26 +278,32 @@ const FINGERPRINTS: &[Fingerprint] = &[
label: "Language",
prefix: &["English", "French", "Spanish", "Dutch"],
expected_count: 70,
// Universal = 65, Disney/Warner = 70; widen to span both plus drift.
count_tolerance: 8,
},
Fingerprint {
label: "Purpose",
prefix: &["Normal", "Commentary", "PiP", "Trivia"],
expected_count: 8,
count_tolerance: LDC_COUNT_TOLERANCE,
},
Fingerprint {
label: "VideoFormat",
prefix: &["HD", "HDR10 Plus", "HD Dolby"],
expected_count: 7,
count_tolerance: LDC_COUNT_TOLERANCE,
},
Fingerprint {
label: "Region",
prefix: &["USA_D1", "LIC1", "LIC2", "LIC3"],
expected_count: 22,
count_tolerance: LDC_COUNT_TOLERANCE,
},
Fingerprint {
label: "Studio",
prefix: &["Disney", "Marvel", "Pixar"],
expected_count: 6,
count_tolerance: LDC_COUNT_TOLERANCE,
},
];
@@ -301,19 +398,26 @@ impl CandidatePool {
/// Phase A. Walk every `.class` in `archive`, identify the master
/// enums by `<clinit>` ldc-sequence fingerprint. Returns a vector of
/// `(label, MasterEnum)` — at most one match per fingerprint label.
pub(crate) fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str, MasterEnum)> {
// First pass: collect every class's <clinit> ldc string sequence.
fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str, MasterEnum)> {
// First pass: collect every class's <clinit> ldc string sequence, keyed by
// the class's JVM INTERNAL name (`this_class`), NOT the zip entry name.
// The binding class references an enum constant as `getstatic <internal>.f`
// (e.g. `pd`, or `com/foo/pd`), so the master enum has to be identified by
// that same internal name or Phase C's getstatic count never matches it.
// The zip entry name (`pd.class`) is a different namespace, used only to
// locate a class for decoding.
let mut pool = CandidatePool::default();
jar::for_each_class(archive, |class_name, class| {
jar::for_each_class(archive, |zip_name, class| {
let Some(ldcs) = clinit_ldc_strings(class) else {
return;
};
if ldcs.is_empty() {
return;
}
if !pool.insert(class_name, ldcs) {
let key = class.this_class_name().unwrap_or(zip_name);
if !pool.insert(key, ldcs) {
tracing::debug!(
class = class_name,
class = ?key,
classes = pool.by_class.len(),
bytes = pool.bytes,
"deluxe: candidate pool aggregate cap hit, dropping class"
@@ -331,7 +435,7 @@ pub(crate) fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str
continue;
}
let count = ldcs.len();
if count.abs_diff(fp.expected_count) > LDC_COUNT_TOLERANCE {
if count.abs_diff(fp.expected_count) > fp.count_tolerance {
continue;
}
// Prefer exact-count match; otherwise first hit wins.
@@ -345,7 +449,85 @@ pub(crate) fn identify_master_enums(archive: &mut jar::Jar) -> Vec<(&'static str
}
}
if let Some((class_name, values)) = best {
out.push((fp.label, MasterEnum { class_name, values }));
out.push((fp.label, class_name, values));
}
}
if out.is_empty() {
return Vec::new();
}
// Second pass: capture the obfuscated `putstatic` field names for the
// classes we matched, so Phase D can resolve `getstatic <enum>.<field>`
// to an ordinal. This is a targeted re-walk of only the matched enum
// classes (at most one per fingerprint), not the whole jar again.
let want: HashSet<&str> = out.iter().map(|(_, name, _)| name.as_str()).collect();
let mut fields_by_class: HashMap<String, Vec<String>> = HashMap::new();
jar::for_each_class(archive, |zip_name, class| {
let iname = class.this_class_name().unwrap_or(zip_name);
if want.contains(iname) && !fields_by_class.contains_key(iname) {
fields_by_class.insert(iname.to_string(), clinit_enum_field_names(class));
}
});
out.into_iter()
.map(|(label, class_name, values)| {
let fields = fields_by_class.remove(&class_name).unwrap_or_default();
(
label,
MasterEnum {
class_name,
values,
fields,
},
)
})
.collect()
}
/// Collect the static-field names an enum class's `<clinit>` stores its
/// own instances into, in declaration order — the ordinal → field-name
/// mapping that mirrors [`clinit_ldc_strings`]'s ordinal → value mapping.
///
/// An obfuscated Deluxe enum constant compiles to
/// `new E; dup; ldc "Value"; invokespecial E.<init>(…); putstatic E.<field>`,
/// so the `putstatic` whose owning class AND field descriptor are both this
/// class's own type names the field that ordinal. The binding class then
/// references that constant as `getstatic E.<field>`; without this mapping
/// the resolver in [`MasterEnumTable`] cannot turn the obfuscated field name
/// back into an ordinal.
///
/// Bounded by [`MAX_CLINIT_LDC_STRINGS`] like the value walk: a `.class`
/// gated only by a path prefix can inflate to the read ceiling, and each
/// retained field name is a heap `String`.
fn clinit_enum_field_names(class: &super::class_reader::ClassFile) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let Some(self_name) = class.this_class_name() else {
return out;
};
let self_descriptor = format!("L{self_name};");
for m in &class.methods {
if class.member_name(m) != Some("<clinit>") {
continue;
}
let Some(code) = m.code(&class.constant_pool) else {
continue;
};
for insn in code.instructions() {
if insn.opcode != PUTSTATIC {
continue;
}
let Some(idx) = insn.cp_index() else { continue };
let Some(member) = class.constant_pool.member_ref(idx) else {
continue;
};
// Only the enum's own singleton fields (`Lself;` typed, owned by
// this class) — skip auxiliary statics (`$VALUES` arrays, counters).
if member.class_name == self_name && member.descriptor == self_descriptor {
if out.len() >= MAX_CLINIT_LDC_STRINGS {
break;
}
out.push(member.name.to_string());
}
}
}
out
@@ -396,7 +578,7 @@ fn clinit_ldc_strings(class: &super::class_reader::ClassFile) -> Option<Vec<Stri
|| out_bytes.saturating_add(s.len()) > MAX_CLINIT_LDC_BYTES
{
tracing::debug!(
class = class.this_class_name().unwrap_or(""),
class = ?class.this_class_name().unwrap_or(""),
strings = out.len(),
bytes = out_bytes,
"deluxe: clinit ldc collection hit cap, truncating"
@@ -447,7 +629,7 @@ fn ldcs_match_prefix(ldcs: &[String], prefix: &[&str]) -> bool {
/// A disc that splits the table commonly has one audio binding class
/// with the most getstatic refs and a subtitle binding class with
/// somewhat fewer; both share the master Language + Purpose enums.
pub(crate) fn find_binding_classes(
fn find_binding_classes(
archive: &mut jar::Jar,
master_enum_classes: &HashSet<&str>,
) -> Vec<(String, usize)> {
@@ -563,7 +745,7 @@ const MAX_CONSTRUCTIONS: usize = 4096;
/// Phase D entry point: find the binding class in `archive`, run the
/// bytecode walker against its `<clinit>`, return one `Construction`
/// per `new X / invokespecial X.<init>` sequence.
pub(crate) fn decode_binding(
fn decode_binding(
archive: &mut jar::Jar,
binding_class_name: &str,
master: &MasterEnumTable,
@@ -752,6 +934,17 @@ impl<'a> BindingDecoder<'a> {
let Some(idx) = insn.cp_index() else { return };
let Some(member) = self.pool.member_ref(idx) else { return };
let arg_count = parse_method_arg_count(member.descriptor);
// A per-stream binding constructor takes only scalars and enum
// references — never an array. A constructor with an array
// parameter is a container/title wrapper (e.g. Universal's
// title object `oq.<init>(…, [Lnp;, [Lwb;, [Loq;, [J)`, which
// holds the per-stream arrays) and must NOT be recorded as a
// stream binding: it carries a Language (the title's primary
// language) but is not itself a stream, and emitting it would
// land a spurious label in the subtitle list. The stack is
// still unwound below so the following real bindings stay in
// sync; only the `Construction` push is suppressed.
let is_container = member.descriptor.contains('[');
// Pop args off the symbolic stack.
if self.stack.len() < arg_count + 1 {
// Stack-machine drift — bail on this construction
@@ -767,7 +960,8 @@ impl<'a> BindingDecoder<'a> {
// operates on. For our pattern it's NewObj(X).
let receiver = self.stack.pop().unwrap_or(StackVal::Unknown);
if let StackVal::NewObj(name) = receiver
&& name == member.class_name {
&& name == member.class_name
&& !is_container {
// Bounded by MAX_CONSTRUCTIONS: an unbounded push here
// is ~1 GiB reachable from a crafted `<clinit>`.
if self.constructions.len() >= MAX_CONSTRUCTIONS {
@@ -806,6 +1000,15 @@ impl<'a> BindingDecoder<'a> {
self.stack.pop();
self.stack.pop();
}
// anewarray / newarray — pop the count, push the array reference.
// Modelled (rather than left to the `_` no-op) so the array
// constructions inside a container's argument list keep the
// symbolic stack aligned for the per-stream bindings built
// alongside them.
0xBD /* anewarray */ | 0xBC /* newarray */ => {
self.stack.pop();
self.push(StackVal::Unknown);
}
// aastore — array store consumes 3 slots (arrayref, index, value).
AASTORE => {
for _ in 0..3 {
@@ -890,8 +1093,18 @@ impl MasterEnumTable {
let mut by_class = HashMap::new();
let mut by_kind = HashMap::new();
for (kind, m) in enums {
let field_map: HashMap<String, u16> = m
.values
// The binding class references enum constants by their obfuscated
// static-field name (`getstatic <enum>.a`), so the resolver map is
// keyed on the `putstatic` field names captured alongside the
// values (`m.fields`), NOT on the value strings. A synthetic test
// enum with no captured field names falls back to keying on the
// values themselves, which is how those tests reference it.
let keys: &[String] = if m.fields.is_empty() {
&m.values
} else {
&m.fields
};
let field_map: HashMap<String, u16> = keys
.iter()
.enumerate()
.map(|(i, v)| (v.clone(), i as u16))
@@ -902,7 +1115,7 @@ impl MasterEnumTable {
MasterEnumTable { by_class, by_kind }
}
pub(crate) fn class_name_set(&self) -> HashSet<&str> {
fn class_name_set(&self) -> HashSet<&str> {
self.by_class.keys().map(String::as_str).collect()
}
@@ -1051,16 +1264,24 @@ fn interpret_streams(constructions: &[Construction], master: &MasterEnumTable) -
None => (String::new(), String::new()),
};
let (purpose, qualifier) = match purpose_ord {
let (purpose, mut qualifier) = match purpose_ord {
Some(o) => deluxe_purpose_to_label(o),
None => (LabelPurpose::Normal, LabelQualifier::None),
};
// Some frameworks (notably Universal) do not carry SDH/RNIB in the
// Purpose enum at all — they encode it as a distinct Language enum
// VALUE ("English SDH", "English RNIB"). When the purpose left the
// qualifier unset, recover it from the language display name so those
// tracks are still flagged.
if qualifier == LabelQualifier::None {
qualifier = vocab::qualifier(&lang_value);
}
if let Some(hint) = stream_idx_hint {
tracing::debug!(
disc_stream_idx = hint,
lang = %language,
binding = %c.binding_type,
lang = ?language,
binding = ?c.binding_type,
"deluxe interpret_streams: disc-authored stream index (not used for stream_number; preserved for diagnostic)"
);
}
@@ -1635,7 +1856,10 @@ mod tests {
.iter()
.find(|(label, _)| *label == "Purpose")
.unwrap_or_else(|| panic!("Purpose fingerprint not matched: {enums:?}"));
assert_eq!(purpose.1.class_name, "com/bydeluxe/Good.class");
// The identified class is keyed by its JVM INTERNAL name (`this_class`,
// here "GoodPurpose"), NOT the zip entry name — that internal name is
// what the binding class's `getstatic` operands reference.
assert_eq!(purpose.1.class_name, "GoodPurpose");
assert_eq!(purpose.1.values.len(), 8);
assert_eq!(purpose.1.values[0], "Normal");
assert_eq!(purpose.1.values[7], "NoForcedDescriptive");
@@ -2290,6 +2514,10 @@ mod tests {
let m = MasterEnum {
class_name: "LanguageEnum".into(),
values: vec!["English".into(), "French".into(), "Spanish".into()],
// Empty: this synthetic enum resolves by value (the fallback in
// `MasterEnumTable::from`), so the resolve tests below key on
// "English"/"French"/"Spanish" directly.
fields: Vec::new(),
};
MasterEnumTable::from(&[("Language", m)])
}
@@ -2982,4 +3210,146 @@ mod tests {
assert!(set.contains("LanguageEnum"));
assert_eq!(set.len(), 1);
}
// ── Real-disc fixtures: Universal (studio="uni") ─────────────────────────
//
// Captured from a real Universal Blu-ray release's `/BDMV/JAR/00000.jar`
// (`com/bydeluxe/…`).
// These are verbatim, unmodified `.class` files — the format we READ, never
// execute — exercising the full Phase A→D pipeline against real obfuscated
// Deluxe bytecode rather than synthetic fixtures.
//
// pd.class = Language enum (65 values: English, French, Spanish, Dutch …)
// lp.class = Purpose enum (Normal, Commentary, PiP, Trivia, Descriptive, Score)
// tl.class = binding class: audio `np.<init>(I,Lpd;,Llp;,LCodingType;)`,
// subtitle `wb.<init>(I,Lpd;,Llp;,Lmi;)`, and a title-wrapper
// `oq.<init>(…,[Lnp;,[Lwb;,[Loq;,[J)` that must NOT leak a label.
const UNI_PD_CLASS: &[u8] = include_bytes!("testdata/deluxe_uni/pd.class");
const UNI_LP_CLASS: &[u8] = include_bytes!("testdata/deluxe_uni/lp.class");
const UNI_TL_CLASS: &[u8] = include_bytes!("testdata/deluxe_uni/tl.class");
fn parse_fixture(bytes: &[u8]) -> super::super::class_reader::ClassFile {
super::super::class_reader::ClassFile::parse(bytes).expect("fixture .class parses")
}
#[test]
fn universal_language_enum_pd_matches_the_language_fingerprint() {
let pd = parse_fixture(UNI_PD_CLASS);
let values = clinit_ldc_strings(&pd).expect("pd has a <clinit>");
// 65-value Universal Language enum — the count the old 70±4 window
// wrongly rejected.
assert_eq!(values.len(), 65);
assert_eq!(&values[..4], &["English", "French", "Spanish", "Dutch"]);
assert!(
ldcs_match_prefix(&values, FINGERPRINTS[0].prefix),
"pd must match the Language prefix"
);
assert!(
values.len().abs_diff(FINGERPRINTS[0].expected_count)
<= FINGERPRINTS[0].count_tolerance,
"65 must fall inside the Language count window (regression guard on \
the widened tolerance)"
);
}
#[test]
fn universal_enum_field_names_map_ordinals_to_obfuscated_fields() {
let pd = parse_fixture(UNI_PD_CLASS);
let fields = clinit_enum_field_names(&pd);
// One putstatic per enum constant, in declaration order: a,b,c,d,…
assert_eq!(fields.len(), 65);
assert_eq!(&fields[..4], &["a", "b", "c", "d"]);
}
/// Build the master table the way `parse` would, but from the fixtures.
fn universal_master() -> MasterEnumTable {
let pd = parse_fixture(UNI_PD_CLASS);
let lp = parse_fixture(UNI_LP_CLASS);
let pd_enum = MasterEnum {
class_name: pd.this_class_name().unwrap().to_string(),
values: clinit_ldc_strings(&pd).unwrap(),
fields: clinit_enum_field_names(&pd),
};
let lp_enum = MasterEnum {
class_name: lp.this_class_name().unwrap().to_string(),
values: clinit_ldc_strings(&lp).unwrap(),
fields: clinit_enum_field_names(&lp),
};
MasterEnumTable::from(&[("Language", pd_enum), ("Purpose", lp_enum)])
}
#[test]
fn universal_binding_class_decodes_real_per_stream_labels() {
let tl = parse_fixture(UNI_TL_CLASS);
let master = universal_master();
let constructions = decode_binding_class(&tl, &master);
assert!(
!constructions.is_empty(),
"tl.<clinit> must yield per-stream constructions"
);
// The title-wrapper `oq` takes array parameters and must be filtered:
// every retained construction is a scalar/enum-only binding.
// (np = 4 args incl. CodingType; wb = 4-5 args incl. `mi`.)
let labels = interpret_streams(&constructions, &master);
assert!(!labels.is_empty(), "must emit at least one stream label");
let audio: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.collect();
let subs: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.collect();
assert!(!audio.is_empty(), "expected audio labels");
assert!(!subs.is_empty(), "expected subtitle labels");
// Real languages recovered via the obfuscated-field-name resolver.
let langs: std::collections::HashSet<&str> =
labels.iter().map(|l| l.language.as_str()).collect();
assert!(langs.contains("eng"), "English audio/subtitle expected");
assert!(
langs.contains("spa") || langs.contains("fra"),
"expected at least one non-English language (Spanish/French)"
);
// The display name is the disc's OWN label, not just an ISO code.
assert!(
labels.iter().any(|l| l.name == "English"),
"expected the disc-authored display name 'English'"
);
// Audio bindings carry a decoded BD-J CodingType → codec hint.
assert!(
audio.iter().any(|l| l.codec_hint == "Dolby Digital"),
"DOLBY_AC3_AUDIO must translate to 'Dolby Digital'; got {:?}",
audio.iter().map(|l| &l.codec_hint).collect::<Vec<_>>()
);
// Subtitle bindings (`wb`, no CodingType) carry no codec hint.
assert!(
subs.iter().all(|l| l.codec_hint.is_empty()),
"subtitle codec hints should be empty (PG implied)"
);
}
#[test]
fn universal_config_xml_studio_attribute_is_parsed() {
// A real config.xml from a Universal Blu-ray release's
// /BDMV/JAR/99999/config.xml.
let xml = br#"<TitleConfig studio="uni" >
<Type><Rental>-</Rental><Single>-</Single></Type>
</TitleConfig>"#;
assert_eq!(parse_studio_attr(xml).as_deref(), Some("uni"));
// Single-quoted and whitespace-padded variants.
assert_eq!(
parse_studio_attr(b"<TitleConfig studio = 'fox'>").as_deref(),
Some("fox")
);
// Missing / empty attribute → None (never panics).
assert_eq!(parse_studio_attr(b"<TitleConfig>"), None);
assert_eq!(parse_studio_attr(b"studio=\"\""), None);
assert_eq!(parse_studio_attr(&[0xff, 0xfe, 0x00]), None);
}
}
+537
View File
@@ -0,0 +1,537 @@
//! Fox — loose `/BDMV/JAR/<id>/dcx.xml` plain-XML manifest.
//!
//! Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
//! human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
//! under `<disc>` it lists every playlist the disc plays. The main-feature
//! playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
//! language, editorial purpose and forced/SDH state outright — everything a
//! label parser wants, in attributes, with no bytecode to walk.
//!
//! NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
//! is one authoring house's internal metadata that happens to press onto the
//! disc. Every field meaning below was read off a real disc; treat an
//! unfamiliar value as unknown rather than guessing.
//!
//! Confirmed schema (a real Fox release, `/BDMV/JAR/05001/dcx.xml`):
//!
//! ```xml
//! <dcx>
//! <disc>
//! <playlist id="00001" lang="eng" name="topmenu"/>
//! ...
//! <playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
//! <audio id="01" lang="eng" type="feature"/>
//! <audio id="02" lang="eng" type="rnib"/>
//! <audio id="03" lang="spa" dial="lat" type="feature"/>
//! ...
//! <subtitle id="01" lang="eng" type="feature" form="sdh"/>
//! <subtitle id="02" lang="spa" dial="lat" type="embed"/>
//! ...
//! <subtitle id="11" lang="eng" type="text"/>
//! <properties> ...chapter marks... </properties>
//! </playlist>
//! <playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
//! </disc>
//! </dcx>
//! ```
//!
//! Field meanings (read off the disc, not documented):
//! * `<audio type>`: `feature` = a normal program track; `rnib` = a
//! descriptive/narration track ("Royal National Institute of Blind People"
//! described-video), mapped to [`LabelPurpose::Descriptive`]. A `type`
//! containing "comment" maps to [`LabelPurpose::Commentary`].
//! * `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
//! hearing track ([`LabelQualifier::Sdh`]).
//! * `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
//! track ([`LabelQualifier::Forced`]); `feature`/`text` are full tracks.
//! * `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
//! (audio ids 01..N, subtitle ids 01..N independently), matching the vendor
//! `stream_number` convention the ordinal binder in [`super::apply_labels`]
//! reads.
//!
//! The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
//! so extraction is scoped to a single `<playlist name="feature">` element —
//! never a document-wide `<audio>` scan, which would merge the regional
//! `00800` (eng) and `00801` (jpn) tables into one and collide their slots.
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorSource;
use crate::udf::UdfFs;
/// Detect a Fox disc.
///
/// Primary, cheap signal: a loose `dcx.xml` under some `/BDMV/JAR/<id>/`
/// subdirectory — the manifest [`parse`] reads. Checked first because it is a
/// directory walk with no sector reads.
///
/// Secondary signal: a `com/foxbd/` prefix in a top-level BD-J jar, which newer
/// Fox discs (later franchise sequels) carry instead of a loose `dcx.xml`. Recognising
/// it attributes the disc to Fox rather than the generic deluxe stub even
/// though [`parse`] cannot decode that bytecode form yet (see the Phase 2 note
/// at the bottom of this file); such a disc detects here but yields no labels,
/// so a lower-confidence parser still wins.
pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
if super::jar_file_exists(udf, "dcx.xml") {
return true;
}
super::jar::for_each_jar(reader, udf, |_, jar| {
super::jar::has_path_prefix(jar, "com/foxbd/").then_some(())
})
.is_some()
}
pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult> {
let data = super::read_jar_file(reader, udf, "dcx.xml")?;
let text = std::str::from_utf8(&data).ok()?;
let labels = labels_from_dcx(text);
if labels.is_empty() {
return None;
}
// High confidence: the manifest is fully structured and every field
// extracted here has its meaning fixed by real discs.
let mut result = ParseResult::high(labels);
// Surface the feature playlist's authoring id (e.g. "00800") so title
// selection can prefer the disc's own feature over a size-inflated decoy —
// the same signal `paramount::parse` provides.
result.feature_playlist = feature_playlist_id(text).map(|id| super::FeaturePlaylistHint {
playlist_id: id.parse::<u16>().ok(),
filename: Some(format!("{id}.mpls")),
});
Some(result)
}
/// Build the stream labels from a `dcx.xml` document. Split out from [`parse`]
/// so the schema mapping is unit-testable without a `SectorSource`/`UdfFs`.
///
/// Scoped to ONE feature playlist element (see the module note): the richest
/// `<playlist name="feature">`. Its nested `<audio>`/`<subtitle>` children are
/// the vendor's STN table for that title.
pub(crate) fn labels_from_dcx(text: &str) -> Vec<StreamLabel> {
let Some(feature) = select_feature_playlist(text) else {
return Vec::new();
};
let mut labels = Vec::new();
// Audio streams.
let mut from = 0;
while let Some((s, e)) = xml::find_element(feature, "audio", from) {
let el = &feature[s..e];
from = e;
let Some(stream_number) = stream_number_from_id(&xml::attr(el, "id")) else {
continue;
};
let Some(language) = xml::attr(el, "lang") else {
continue;
};
let ty = xml::attr(el, "type")
.unwrap_or_default()
.to_ascii_lowercase();
labels.push(StreamLabel {
stream_id: None,
stream_number,
stream_type: StreamLabelType::Audio,
language: normalize_language(&language),
name: String::new(),
purpose: audio_purpose(&ty),
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
});
}
// Subtitle streams.
let mut from = 0;
while let Some((s, e)) = xml::find_element(feature, "subtitle", from) {
let el = &feature[s..e];
from = e;
let Some(stream_number) = stream_number_from_id(&xml::attr(el, "id")) else {
continue;
};
let Some(language) = xml::attr(el, "lang") else {
continue;
};
let ty = xml::attr(el, "type")
.unwrap_or_default()
.to_ascii_lowercase();
let form = xml::attr(el, "form")
.unwrap_or_default()
.to_ascii_lowercase();
labels.push(StreamLabel {
stream_id: None,
stream_number,
stream_type: StreamLabelType::Subtitle,
language: normalize_language(&language),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: subtitle_qualifier(&ty, &form),
codec_hint: String::new(),
variant: String::new(),
});
}
labels
}
/// Pick the disc's primary feature playlist element (the full `<playlist ...>
/// ... </playlist>` text). Among all `<playlist name="feature">` elements —
/// Fox presses one per regional variant (`00800` eng, `00801` jpn) — the one
/// carrying the most `<audio>`/`<subtitle>` streams wins, ties to the first.
///
/// Returning ONE element is the whole point: each feature playlist is its own
/// STN table with its own 1-based slot ids, so merging two would put two
/// different streams on slot `01`. The richest table is the fullest label set
/// and anchors to its matching title by language sequence in
/// [`super::apply_labels`].
fn select_feature_playlist(text: &str) -> Option<&str> {
let mut best: Option<&str> = None;
let mut best_streams = 0usize;
let mut from = 0;
while let Some((s, e)) = xml::find_element(text, "playlist", from) {
let element = &text[s..e];
from = e;
// `name` is read from the element; the opening tag's `name="feature"`
// is the first `name=` in document order, ahead of any nested child
// (nested `<audio>`/`<subtitle>` carry no `name`).
let is_feature =
xml::attr(element, "name").is_some_and(|n| n.eq_ignore_ascii_case("feature"));
if !is_feature {
continue;
}
let streams = count_elements(element, "audio") + count_elements(element, "subtitle");
// First feature wins the tie (matches the registry's first-wins rule);
// a strictly richer table displaces it. `best.is_none()` so a feature
// with zero nested streams is still selected — its id feeds the hint.
if best.is_none() || streams > best_streams {
best_streams = streams;
best = Some(element);
}
}
best
}
/// The authoring id (digits only, e.g. "00800") of the selected feature
/// playlist, or `None` when no feature playlist / no id is present.
fn feature_playlist_id(text: &str) -> Option<String> {
let feature = select_feature_playlist(text)?;
let id = xml::attr(feature, "id")?;
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
Some(digits)
}
/// Count `<tag>` elements inside `element`.
fn count_elements(element: &str, tag: &str) -> usize {
let mut n = 0;
let mut from = 0;
while let Some((_, e)) = xml::find_element(element, tag, from) {
n += 1;
from = e;
}
n
}
/// Parse a nested stream `id` (e.g. "01", "11") into its 1-based STN slot.
/// Digits only, so a stray quote/space cannot poison it; `None` (skip the
/// stream) when it names no slot — 0 is the module's `NO_STN_SLOT` sentinel and
/// cannot be bound by the ordinal path.
fn stream_number_from_id(id: &Option<String>) -> Option<u16> {
let id = id.as_deref()?;
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
digits.parse::<u16>().ok().filter(|&n| n != 0)
}
/// Map an `<audio type>` value to a [`LabelPurpose`]. `rnib` is described-video
/// (descriptive/narration); a `comment*` value is commentary; everything else
/// (`feature`, unknown) is a normal program track.
fn audio_purpose(ty: &str) -> LabelPurpose {
if ty == "rnib" {
LabelPurpose::Descriptive
} else if ty.contains("comment") {
LabelPurpose::Commentary
} else {
LabelPurpose::Normal
}
}
/// Map a `<subtitle>`'s `form`/`type` to a [`LabelQualifier`]. `form="sdh"`
/// takes precedence (an SDH track is the editorially meaningful flag);
/// otherwise `type="embed"` is a dedicated forced/embedded track. `feature`
/// and `text` full tracks carry no qualifier.
fn subtitle_qualifier(ty: &str, form: &str) -> LabelQualifier {
if form == "sdh" {
LabelQualifier::Sdh
} else if ty == "embed" {
LabelQualifier::Forced
} else {
LabelQualifier::None
}
}
/// Trim + lowercase the raw ISO 639-2 code (`"eng"`, `"fra"`, ...).
fn normalize_language(raw: &str) -> String {
raw.trim().to_ascii_lowercase()
}
// ── Phase 2 (design only, not implemented here) ─────────────────────────────
//
// Newer Fox discs (later franchise sequels, surveyed as 11 discs currently mis-owned by
// the deluxe stub) ship NO loose `dcx.xml`. They wrap the same per-stream data
// inside `com/foxbd` BD-J `.class` bytecode. The follow-on parser would reuse
// `super::class_reader` the way `dbp`/`deluxe` already do:
//
// * `detect` already recognises the `com/foxbd/` central-directory prefix
// (above), so Phase 2 only adds a parse path.
// * Iterate the jar's classes via `super::jar::for_each_class` and read the
// constant pool (`ClassFile`), which holds the authored strings verbatim:
// - `FeatureIntroPlaylist.play: PLAYLIST ID:` → the feature playlist id
// (→ the same `FeaturePlaylistHint` the dcx path should wire up),
// - `RNIB` → a descriptive/narration audio track (→ Descriptive),
// - `primaryAudioId=` and the parallel subtitle-id constants → the STN
// slot numbering, mirroring the `id="NN"` mapping here.
// * Emit the identical `StreamLabel` shape at High confidence so the two Fox
// forms are indistinguishable downstream.
#[cfg(test)]
mod tests {
use super::*;
/// A real Fox-release manifest, reduced only by truncating the giant
/// chapter-mark `<properties>` blocks (which carry no stream labels). Every
/// `<audio>`/`<subtitle>` element and both regional feature playlists are
/// verbatim from a real disc's `/BDMV/JAR/05001/dcx.xml`, so this fixture exercises the
/// exact bytes production sees: feature selection across two `name="feature"`
/// playlists, the nested-scope rule, and the rnib/sdh/embed flags.
const FOX_DCX_SAMPLE: &str = r#"<dcx>
<disc>
<properties region="ABC" regioncheckon="false" parentallevel="PG" hdronlydisc="true" bootstrap.bdjo="88888"
vstaskenabled="false" defaultversion="1" topmenushowmarkid="0" topmenuloopmarkid="0"/>
<playlist id="00001" lang="eng" name="topmenu"/>
<playlist id="00100" lang="eng" name="foxlogo"/>
<playlist id="00300" lang="eng" name="copyright"/> <!-- English -->
<playlist id="00600" name="black" lang="eng"/>
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
<audio id="04" lang="fra" dial="par" type="feature"/>
<audio id="05" lang="dan" type="feature"/>
<audio id="06" lang="nld" type="feature"/>
<audio id="07" lang="fin" type="feature"/>
<audio id="08" lang="deu" type="feature"/>
<audio id="09" lang="ita" type="feature"/>
<audio id="10" lang="nor" type="feature"/>
<audio id="11" lang="swe" type="feature"/>
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
<subtitle id="03" lang="fra" dial="par" type="embed"/>
<subtitle id="04" lang="dan" type="embed"/>
<subtitle id="05" lang="nld" type="embed"/>
<subtitle id="06" lang="fin" type="feature"/>
<subtitle id="07" lang="deu" type="embed"/>
<subtitle id="08" lang="ita" type="embed"/>
<subtitle id="09" lang="nor" type="embed"/>
<subtitle id="10" lang="swe" type="embed"/>
<subtitle id="11" lang="eng" type="text"/>
<properties>
<entry.marks ids="0,2,4,6,8,10"/>
<playlist.marks timecodes="00:00:00:00,00:03:58:18"/>
</properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="jpn" type="feature"/>
<subtitle id="01" lang="jpn" type="feature"/>
<subtitle id="02" lang="eng" type="feature" form="sdh"/>
<subtitle id="03" lang="jpn" type="text"/>
<subtitle id="04" lang="eng" type="text"/>
<properties>
<entry.marks ids="0,2,4"/>
</properties>
</playlist>
</disc>
</dcx>"#;
fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.collect()
}
fn subs(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.collect()
}
/// The primary feature is the richer `00800` (eng) table, not `00801`
/// (jpn). Both are `name="feature"`; selection must pick by stream count,
/// and the id we surface for the (future) FeaturePlaylistHint is `00800`.
#[test]
fn selects_richest_feature_playlist_and_its_id() {
assert_eq!(
feature_playlist_id(FOX_DCX_SAMPLE),
Some("00800".to_string())
);
}
/// Full real-disc parse: the 00800 audio table. Eleven tracks, id order =
/// STN slot, and slot 2 (`eng rnib`) is the descriptive/narration track.
#[test]
fn fox_dcx_audio_labels() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let a = audio(&labels);
assert_eq!(
a.len(),
11,
"00800 has eleven audio tracks, 00801 not merged"
);
// Slots and languages, in id order.
let got: Vec<(u16, &str, LabelPurpose)> = a
.iter()
.map(|l| (l.stream_number, l.language.as_str(), l.purpose))
.collect();
assert_eq!(
got,
vec![
(1, "eng", LabelPurpose::Normal),
(2, "eng", LabelPurpose::Descriptive), // rnib = described video
(3, "spa", LabelPurpose::Normal),
(4, "fra", LabelPurpose::Normal),
(5, "dan", LabelPurpose::Normal),
(6, "nld", LabelPurpose::Normal),
(7, "fin", LabelPurpose::Normal),
(8, "deu", LabelPurpose::Normal),
(9, "ita", LabelPurpose::Normal),
(10, "nor", LabelPurpose::Normal),
(11, "swe", LabelPurpose::Normal),
]
);
// Vendor labels: no StreamId, they bind by ordinal STN slot.
assert!(a.iter().all(|l| l.stream_id.is_none()));
}
/// Full real-disc parse: the 00800 subtitle table. `form="sdh"` → Sdh,
/// `type="embed"` → Forced, `feature`/`text` → no qualifier.
#[test]
fn fox_dcx_subtitle_labels() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let s = subs(&labels);
assert_eq!(s.len(), 11, "00800 has eleven subtitle tracks");
let got: Vec<(u16, &str, LabelQualifier)> = s
.iter()
.map(|l| (l.stream_number, l.language.as_str(), l.qualifier))
.collect();
assert_eq!(
got,
vec![
(1, "eng", LabelQualifier::Sdh), // feature + form=sdh
(2, "spa", LabelQualifier::Forced), // embed
(3, "fra", LabelQualifier::Forced),
(4, "dan", LabelQualifier::Forced),
(5, "nld", LabelQualifier::Forced),
(6, "fin", LabelQualifier::None), // feature
(7, "deu", LabelQualifier::Forced),
(8, "ita", LabelQualifier::Forced),
(9, "nor", LabelQualifier::Forced),
(10, "swe", LabelQualifier::Forced),
(11, "eng", LabelQualifier::None), // text
]
);
}
/// The nested-scope rule: audio/subtitle come from ONE feature playlist,
/// never a document-wide scan that would merge 00800 and 00801 and collide
/// their `id="01"` slots. If scoping regressed, slot 1 audio would appear
/// twice and the count would jump past 11.
#[test]
fn does_not_merge_regional_feature_playlists() {
let labels = labels_from_dcx(FOX_DCX_SAMPLE);
let a = audio(&labels);
// Exactly one audio label per STN slot 1..=11.
let slot1: Vec<_> = a.iter().filter(|l| l.stream_number == 1).collect();
assert_eq!(slot1.len(), 1, "one stream on slot 1, not one per playlist");
assert_eq!(audio(&labels).len() + subs(&labels).len(), 22);
}
/// The `rnib` described-video mapping in isolation, plus the commentary
/// path (no track in the sample manifest uses it, so it is pinned synthetically).
#[test]
fn audio_purpose_mapping() {
assert_eq!(audio_purpose("feature"), LabelPurpose::Normal);
assert_eq!(audio_purpose("rnib"), LabelPurpose::Descriptive);
assert_eq!(audio_purpose("commentary"), LabelPurpose::Commentary);
assert_eq!(
audio_purpose("director-commentary"),
LabelPurpose::Commentary
);
assert_eq!(audio_purpose("unknown"), LabelPurpose::Normal);
}
/// `form="sdh"` outranks `type`; `embed` is forced; full tracks are None.
#[test]
fn subtitle_qualifier_mapping() {
assert_eq!(subtitle_qualifier("feature", "sdh"), LabelQualifier::Sdh);
assert_eq!(subtitle_qualifier("embed", ""), LabelQualifier::Forced);
assert_eq!(subtitle_qualifier("feature", ""), LabelQualifier::None);
assert_eq!(subtitle_qualifier("text", ""), LabelQualifier::None);
// An SDH embedded track (hypothetical) still flags SDH, the richer fact.
assert_eq!(subtitle_qualifier("embed", "sdh"), LabelQualifier::Sdh);
}
/// `id="NN"` becomes the 1-based STN slot; a missing/zero id names no slot.
#[test]
fn stream_number_from_id_digits_only() {
assert_eq!(stream_number_from_id(&Some("01".into())), Some(1));
assert_eq!(stream_number_from_id(&Some("11".into())), Some(11));
assert_eq!(stream_number_from_id(&Some(" 07 ".into())), Some(7));
assert_eq!(stream_number_from_id(&Some("00".into())), None);
assert_eq!(stream_number_from_id(&None), None);
assert_eq!(stream_number_from_id(&Some("".into())), None);
}
// ── Negative detection ─────────────────────────────────────────────────
/// A document that is not a Fox feature manifest yields no labels. Three
/// shapes: empty, a `<dcx>` whose playlists are all non-feature menu/logo
/// clips (no `name="feature"`), and unrelated XML. None must be mistaken
/// for a Fox feature table.
#[test]
fn non_feature_manifest_yields_no_labels() {
assert!(labels_from_dcx("").is_empty());
assert!(labels_from_dcx("<not-dcx><playlist name=\"x\"/></not-dcx>").is_empty());
let menus_only = r#"<dcx><disc>
<playlist id="00001" lang="eng" name="topmenu"/>
<playlist id="00100" lang="eng" name="foxlogo"/>
<playlist id="00600" name="black" lang="eng"/>
</disc></dcx>"#;
assert!(
labels_from_dcx(menus_only).is_empty(),
"no <playlist name=\"feature\"> → nothing to label"
);
assert_eq!(feature_playlist_id(menus_only), None);
}
/// A feature playlist with no nested streams (an authoring edge) produces
/// no labels rather than a spurious empty-slot entry.
#[test]
fn feature_without_streams_yields_no_labels() {
let doc = r#"<dcx><disc>
<playlist id="00800" lang="eng" name="feature" durs="7628"/>
</disc></dcx>"#;
assert!(labels_from_dcx(doc).is_empty());
// ...but the id is still recoverable for the feature hint.
assert_eq!(feature_playlist_id(doc), Some("00800".to_string()));
}
}
+259 -18
View File
@@ -15,7 +15,8 @@ pub mod clpi_audit;
mod criterion;
mod ctrm;
mod dbp;
mod deluxe;
pub(crate) mod deluxe;
pub(crate) mod fox;
pub(crate) mod jar;
mod mpls_universal;
mod paramount;
@@ -149,6 +150,38 @@ pub enum Confidence {
High,
}
/// The disc's own authoring-designated main-feature playlist, surfaced by a
/// parser that reads the menu/authoring metadata (e.g. Paramount `playlists.xml`
/// carries `<playlist name="Feature" id="00222"/>`). Consumed by title
/// selection (`Disc::main_feature_order`) as a size-independent anti-decoy
/// signal — a playlist obfuscation trick can inflate a streamless decoy past
/// the real feature on size/duration, but it can't forge the authoring id.
/// Best-effort: an unknown field is `None`.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct FeaturePlaylistHint {
/// Playlist number (e.g. 222 for `00222.mpls`) when the authoring id parses.
pub playlist_id: Option<u16>,
/// Playlist filename (e.g. "00222.mpls") when derivable.
pub filename: Option<String>,
}
impl FeaturePlaylistHint {
/// Whether this hint designates the playlist a title carries. Matches on
/// the numeric id OR the filename (case-insensitive), so a hint that pins
/// only one still binds.
pub fn matches(&self, title_playlist_id: u16, title_playlist: &str) -> bool {
if self.playlist_id == Some(title_playlist_id) {
return true;
}
matches!(&self.filename, Some(f) if f.eq_ignore_ascii_case(title_playlist))
}
/// Whether the hint carries no usable identity.
pub fn is_empty(&self) -> bool {
self.playlist_id.is_none() && self.filename.is_none()
}
}
/// Successful parser result. `None` from `parse()` still means "this
/// isn't my disc" (no labels at all); `Some(ParseResult { labels, .. })`
/// with `labels.is_empty()` is also a "no labels" case but reachable
@@ -158,6 +191,9 @@ pub enum Confidence {
pub struct ParseResult {
pub labels: Vec<StreamLabel>,
pub confidence: Confidence,
/// The disc's authoring-named feature playlist, when this parser read it
/// from the menu metadata. `None` for parsers that don't expose one.
pub feature_playlist: Option<FeaturePlaylistHint>,
}
impl ParseResult {
@@ -167,6 +203,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::High,
feature_playlist: None,
}
}
@@ -175,6 +212,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::Medium,
feature_playlist: None,
}
}
@@ -185,6 +223,7 @@ impl ParseResult {
ParseResult {
labels,
confidence: Confidence::Low,
feature_playlist: None,
}
}
}
@@ -201,6 +240,13 @@ const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
// walking).
("dbp", dbp::detect, dbp::parse),
("deluxe", deluxe::detect, deluxe::parse),
// Fox — older discs ship a loose plain-XML `/BDMV/JAR/<id>/dcx.xml`
// manifest (root `<dcx>`) with per-stream language/purpose/qualifier.
// High confidence: fully structured, no guessing. Registered ahead of
// the universal MPLS fallback so its editorial labels win. Newer Fox
// wraps the same data in `com/foxbd` bytecode (see fox.rs Phase 2 note),
// which this parser detects but does not yet decode.
("fox", fox::detect, fox::parse),
// Universal MPLS fallback. Returns Confidence::Low so framework
// parsers always win when they match. Closes the "no framework
// matched" gap (e.g. HDMV-only discs) with spec-derived language
@@ -221,13 +267,21 @@ const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
/// Search disc for config files, extract labels, apply to streams.
/// This is 100% optional — if anything fails, streams are untouched.
pub fn apply(reader: &mut dyn SectorSource, udf: &UdfFs, titles: &mut [DiscTitle]) {
let labels = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| extract(reader, udf)))
.unwrap_or_default();
if labels.is_empty() {
return;
/// Returns the disc's authoring-designated feature playlist when a parser
/// surfaced one (`None` otherwise), so the caller can bias title selection
/// toward it. Label binding onto streams is a side effect on `titles`.
pub fn apply(
reader: &mut dyn SectorSource,
udf: &UdfFs,
titles: &mut [DiscTitle],
) -> Option<FeaturePlaylistHint> {
let (labels, feature_playlist) =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| extract(reader, udf)))
.unwrap_or_default();
if !labels.is_empty() {
apply_labels(&labels, titles);
}
apply_labels(&labels, titles);
feature_playlist.filter(|h| !h.is_empty())
}
/// Minimum number of streams of one type a title must carry before its
@@ -450,13 +504,22 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
}) else {
continue;
};
for clip in &title.clips {
// ONLY the anchor's first clip. `disc::bluray` builds a title's
// stream list from `play_items[0]`'s STN table, so that is the only
// clip in which these PIDs were ever observed — the same clip tier 3
// keys its derived ids on (`clip0`, below). Recording the fact
// against every clip the anchor plays claims knowledge of stream
// tables never read: a sibling playlist over a LATER clip then binds
// the anchor's editorial label onto whichever stream of that clip
// reuses the PID, which is a different physical stream (PIDs are
// unique only within a clip).
if let Some(clip) = title.clips.first() {
pid_map.insert((clip.clip_id.as_str(), *pid), pos);
}
}
tracing::info!(
stream_type = ?stream_type,
playlist = %titles[anchor].playlist,
playlist = ?titles[anchor].playlist,
slots = slots_of(&titles[anchor], stream_type).len(),
"label list anchored to a title by its stream-language sequence",
);
@@ -855,7 +918,10 @@ fn generate_audio_label_inner(
}
}
fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
fn extract(
reader: &mut dyn SectorSource,
udf: &UdfFs,
) -> (Vec<StreamLabel>, Option<FeaturePlaylistHint>) {
let mut candidates: Vec<(&'static str, ParseResult)> = Vec::new();
for (name, detect, parse) in PARSERS {
if !detect(reader, udf) {
@@ -879,7 +945,7 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
// parsers are registered ahead of the ones that detect on any BD-J disc),
// so "first wins on a tie" is load-bearing, not incidental.
let best = select_result(&candidates).map(|(n, r)| (*n, r.clone()));
let (name, mut labels) = match best {
let (name, mut labels, feature_playlist) = match best {
Some((n, r)) => {
tracing::info!(
parser = n,
@@ -887,11 +953,11 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
label_count = r.labels.len(),
"label parser selected",
);
(n, r.labels)
(n, r.labels, r.feature_playlist)
}
None => {
tracing::info!("no label parser matched");
return Vec::new();
return (Vec::new(), None);
}
};
@@ -915,7 +981,7 @@ fn extract(reader: &mut dyn SectorSource, udf: &UdfFs) -> Vec<StreamLabel> {
// actually carries that stream.
let _orphans_added = append_clpi_orphans(&mut labels, reader, udf);
labels
(labels, feature_playlist)
}
/// Merge the MPLS-derived floor into a framework parser's label list.
@@ -1517,15 +1583,16 @@ mod registry_tests {
"ctrm",
"dbp",
"deluxe",
"fox",
"mpls_universal",
"png_filenames",
],
"PARSERS array order changed — file-presence/reader-gated High \
parsers (paramount/criterion/pixelogic/ctrm) stay first; dbp + \
deluxe (now real com/<vendor>/ prefix detect) stay before \
mpls_universal; mpls_universal stays the universal Low fallback; \
png_filenames (Low, language-only hint) stays LAST so MPLS wins \
the Low tie whenever it produces anything."
deluxe (now real com/<vendor>/ prefix detect) and fox (dcx.xml / \
com/foxbd) stay before mpls_universal; mpls_universal stays the \
universal Low fallback; png_filenames (Low, language-only hint) \
stays LAST so MPLS wins the Low tie whenever it produces anything."
);
}
@@ -1547,9 +1614,37 @@ mod registry_tests {
ParseResult {
labels: vec![one_label()],
confidence: conf,
feature_playlist: None,
}
}
#[test]
fn feature_playlist_hint_matches_on_id_or_filename() {
let h = FeaturePlaylistHint {
playlist_id: Some(222),
filename: Some("00222.mpls".into()),
};
assert!(h.matches(222, "99999.mpls"), "numeric id match");
assert!(
h.matches(1, "00222.MPLS"),
"filename match is case-insensitive"
);
assert!(
!h.matches(1, "00001.mpls"),
"neither id nor filename → no match"
);
let id_only = FeaturePlaylistHint {
playlist_id: Some(5),
filename: None,
};
assert!(id_only.matches(5, "anything.mpls"));
assert!(!id_only.matches(6, "00005.mpls"));
assert!(FeaturePlaylistHint::default().is_empty());
assert!(!h.is_empty());
}
/// `select_result` must pick the highest-confidence non-empty result
/// and, on a confidence tie, the FIRST in array order (regression for the
/// old `analyze()` `max_by(...then(Equal))` no-op that picked the LAST).
@@ -1583,6 +1678,7 @@ mod registry_tests {
let empty = ParseResult {
labels: Vec::new(),
confidence: Confidence::High,
feature_playlist: None,
};
// High-confidence but empty must be skipped in favour of a
// non-empty lower-confidence result.
@@ -1907,6 +2003,78 @@ mod apply_tests {
}
}
/// Sentinel embedded in the crafted playlist name below. The capture keeps
/// ONLY fields whose rendered form contains it, so the assertion cannot be
/// fooled by another test's log output.
const LOG_INJECTION_SENTINEL: &str = "FMKV-LOG-INJECTION-PROBE";
/// A playlist name is a raw UDF directory entry — disc-controlled bytes,
/// validated no further than a lossy UTF-8 decode. Logging it through
/// tracing's `%` (Display) sigil writes those bytes VERBATIM, so a crafted
/// `.mpls` filename carrying ANSI escapes or control characters forges
/// terminal output and log structure in any consumer rendering the event
/// (CWE-117). `?` (Debug) escapes them, and `str`'s Debug is exactly the
/// escaping this needs.
///
/// `info!` is not covered by the debug/trace-logging exemption: this fires
/// on an ordinary rip of an ordinary disc.
///
/// Mutation: put `%` back on `playlist` in `apply_labels` and this goes red.
#[test]
fn a_disc_derived_playlist_name_is_escaped_in_the_log_not_written_verbatim() {
// A name whose bytes would clear the line and repaint it.
let evil = format!("\u{1b}[2K\u{1b}[31m{LOG_INJECTION_SENTINEL}\u{7}\u{1b}[0m.mpls");
let labels = vec![
sub_label(1, "eng", LabelQualifier::None),
sub_label(2, "spa", LabelQualifier::None),
sub_label(3, "fra", LabelQualifier::None),
];
let mut titles = vec![title_on_clip(
&evil,
"00294",
vec![
subtitle(0x12A0, "eng"),
subtitle(0x12A1, "spa"),
subtitle(0x12A2, "fra"),
],
)];
// Capture through the crate's ONE global `tracing` subscriber:
// `testlog::capture` installs it exactly once and routes each event to a
// thread-local sink. A process-wide `set_global_default` HERE instead
// would poison every other test's callsite interest cache for the rest of
// the binary — `tracing` caches interest GLOBALLY — and only the first
// `set_global_default` in a process takes effect anyway. The shared
// subscriber answers interest for every callsite and isolates concurrent
// captures per thread, which is the invariant these log-accounting
// assertions rely on.
let ((), events) = crate::testlog::capture(|| {
apply_labels(&labels, &mut titles);
});
let playlist: Vec<&str> = events
.iter()
.filter(|e| e.target.starts_with("libfreemkv::labels"))
.filter_map(|e| e.field("playlist"))
.filter(|v| v.contains(LOG_INJECTION_SENTINEL))
.collect();
assert!(
!playlist.is_empty(),
"the anchoring event must actually have fired, or this test proves \
nothing; captured: {events:?}"
);
for rendered in playlist {
assert!(
!rendered.contains('\u{1b}') && !rendered.contains('\u{7}'),
"a disc-controlled playlist name reached the log with its raw \
control bytes intact: {rendered:?}"
);
assert!(
rendered.contains(LOG_INJECTION_SENTINEL),
"the name must still be legible once escaped: {rendered:?}"
);
}
}
fn sub_label(num: u16, lang: &str, qualifier: LabelQualifier) -> StreamLabel {
StreamLabel {
stream_id: None,
@@ -3103,6 +3271,79 @@ mod apply_tests {
);
}
/// A title that plays SEVERAL clips in order — the shape the anchor's
/// PID facts are harvested from.
fn title_on_clips(playlist: &str, clip_ids: &[&str], streams: Vec<Stream>) -> DiscTitle {
DiscTitle {
playlist: playlist.into(),
clips: clip_ids
.iter()
.map(|id| crate::disc::Clip {
feed_span: None,
clip_id: (*id).into(),
in_time: 0,
out_time: 0,
duration_secs: 3600.0,
source_packets: 0,
})
.collect(),
..title_with(streams)
}
}
/// Spec: the anchor proves a `(clip, PID)` fact only for the clip its
/// stream table was READ FROM — the first play item — never for every clip
/// the anchor happens to play.
///
/// `disc::bluray` builds a title's stream list from `play_items[0]`'s STN
/// table, and tier 3 fifty lines below says exactly that by keying its
/// derived ids on `clip0`. Tier 2's harvest contradicted it: it recorded
/// the anchor's slot PIDs against EVERY clip the anchor plays, so a sibling
/// playlist that plays a LATER clip of the anchor bound the anchor's
/// editorial label onto whatever stream in that clip happens to reuse the
/// PID — a different physical stream, in a different clip, whose own
/// language says so.
///
/// Mutation: harvest over `&title.clips` instead of its first clip — the
/// featurette wears the feature's SDH again.
#[test]
fn an_anchor_proves_pids_only_for_the_clip_its_table_came_from() {
let labels = vec![
sub_label(1, "eng", LabelQualifier::Sdh),
sub_label(2, "fra", LabelQualifier::None),
];
let mut titles = vec![
// The anchor: its stream table is clip 00082's (the first play
// item); it merely CONTINUES into 00090.
title_on_clips(
"00800.mpls",
&["00082", "00090"],
vec![subtitle(0x1200, "eng"), subtitle(0x1201, "fra")],
),
// A sibling playlist over the anchor's SECOND clip. Its subtitle
// reuses PID 0x1200 — PIDs are only unique within a clip — and it
// is Spanish, so nothing about the anchor's English SDH slot
// describes it.
title_on_clips("00451.mpls", &["00090"], vec![subtitle(0x1200, "spa")]),
];
apply_labels(&labels, &mut titles);
assert_eq!(
sub_state(&titles[0]),
vec![
(0x1200, false, LabelQualifier::Sdh),
(0x1201, false, LabelQualifier::None)
],
"the anchor itself keeps the qualifiers the list states for it"
);
assert_eq!(
sub_state(&titles[1]),
vec![(0x1200, false, LabelQualifier::None)],
"a PID in a clip the anchor's table never described is not that \
table's stream 1"
);
}
/// Spec: a vendor codec/variant claim does not follow the ordinal onto a
/// bonus clip that carries a different codec.
///
+4 -5
View File
@@ -875,11 +875,10 @@ mod tests {
#[test]
fn normalize_language_lowercases_and_trims() {
assert_eq!(
super::super::mpls_universal::language_display_name(&{
let trimmed = " ENG ".trim().to_ascii_lowercase();
// feed through production normalize_language logic
trimmed
}),
// trim + lowercase, mirroring production normalize_language
super::super::mpls_universal::language_display_name(
&" ENG ".trim().to_ascii_lowercase(),
),
"English"
);
}
+19 -1
View File
@@ -42,10 +42,28 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
if labels.is_empty() {
return None;
}
// Surface the feature playlist's identity (e.g. `id="00222"`) so title
// selection can prefer it over a size-inflated decoy. The id is 5-digit
// zero-padded on BD (matching the `NNNNN.mpls` filename); keep only the
// digits so a stray attribute quote/space can't poison the parse.
let feature_playlist = super::xml::attr(&feature, "id").and_then(|id| {
let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
Some(super::FeaturePlaylistHint {
playlist_id: digits.parse::<u16>().ok(),
filename: Some(format!("{digits}.mpls")),
})
});
// High confidence: this format is fully structured and we extract
// every field whose meaning the corpus establishes. "Documented" would
// be the wrong word — see the module note; nothing about it is.
Some(ParseResult::high(labels))
let mut result = ParseResult::high(labels);
result.feature_playlist = feature_playlist;
Some(result)
}
/// One cell of the `forced_sub` CSV.
+6 -2
View File
@@ -133,7 +133,11 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
} else {
Confidence::Medium
};
Some(ParseResult { labels, confidence })
Some(ParseResult {
labels,
confidence,
feature_playlist: None,
})
}
/// Walk the extracted token strings of the feature section and emit a
@@ -441,7 +445,7 @@ fn parse_token_inner(s: &str, mut unknown: Option<&mut UnknownParts>) -> Option<
// what we know than discard a whole stream over one part,
// but flag the parse as Medium-confidence so callers know
// some data was elided.
tracing::debug!(part = %part, "pixelogic: unrecognized token component, skipping");
tracing::debug!(part = ?part, "pixelogic: unrecognized token component, skipping");
if let Some(acc) = unknown.as_deref_mut() {
acc.record(part);
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
+326
View File
@@ -219,6 +219,237 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
Some(code)
}
// ── ISO 639-1 → ISO 639-2 ────────────────────────────────────────────────────
/// The complete ISO 639-1 set, paired with its ISO 639-2/**T** (terminological)
/// code. Every two-letter code ISO 639-1 defines appears exactly once.
///
/// /T is the variant the rest of this crate uses — [`lang`] and [`menu_lang`]
/// both normalize to it (`deu` not `ger`, `fra` not `fre`, `zho` not `chi`,
/// `ces`, `nld`, `ell`, `ron`, `slk`, `isl`, `eus`, `hrv`) — so the three
/// tables cannot disagree. `iso639_1_agrees_with_menu_lang` pins that.
///
/// For the 165 codes where 639-2/B and /T are identical this distinction does
/// not arise; it only matters for the 20-odd languages with a distinct
/// bibliographic code.
const ISO_639_1_TO_2: &[(&str, &str)] = &[
("aa", "aar"),
("ab", "abk"),
("ae", "ave"),
("af", "afr"),
("ak", "aka"),
("am", "amh"),
("an", "arg"),
("ar", "ara"),
("as", "asm"),
("av", "ava"),
("ay", "aym"),
("az", "aze"),
("ba", "bak"),
("be", "bel"),
("bg", "bul"),
("bh", "bih"),
("bi", "bis"),
("bm", "bam"),
("bn", "ben"),
("bo", "bod"),
("br", "bre"),
("bs", "bos"),
("ca", "cat"),
("ce", "che"),
("ch", "cha"),
("co", "cos"),
("cr", "cre"),
("cs", "ces"),
("cu", "chu"),
("cv", "chv"),
("cy", "cym"),
("da", "dan"),
("de", "deu"),
("dv", "div"),
("dz", "dzo"),
("ee", "ewe"),
("el", "ell"),
("en", "eng"),
("eo", "epo"),
("es", "spa"),
("et", "est"),
("eu", "eus"),
("fa", "fas"),
("ff", "ful"),
("fi", "fin"),
("fj", "fij"),
("fo", "fao"),
("fr", "fra"),
("fy", "fry"),
("ga", "gle"),
("gd", "gla"),
("gl", "glg"),
("gn", "grn"),
("gu", "guj"),
("gv", "glv"),
("ha", "hau"),
("he", "heb"),
("hi", "hin"),
("ho", "hmo"),
("hr", "hrv"),
("ht", "hat"),
("hu", "hun"),
("hy", "hye"),
("hz", "her"),
("ia", "ina"),
("id", "ind"),
("ie", "ile"),
("ig", "ibo"),
("ii", "iii"),
("ik", "ipk"),
("io", "ido"),
("is", "isl"),
("it", "ita"),
("iu", "iku"),
("ja", "jpn"),
("jv", "jav"),
("ka", "kat"),
("kg", "kon"),
("ki", "kik"),
("kj", "kua"),
("kk", "kaz"),
("kl", "kal"),
("km", "khm"),
("kn", "kan"),
("ko", "kor"),
("kr", "kau"),
("ks", "kas"),
("ku", "kur"),
("kv", "kom"),
("kw", "cor"),
("ky", "kir"),
("la", "lat"),
("lb", "ltz"),
("lg", "lug"),
("li", "lim"),
("ln", "lin"),
("lo", "lao"),
("lt", "lit"),
("lu", "lub"),
("lv", "lav"),
("mg", "mlg"),
("mh", "mah"),
("mi", "mri"),
("mk", "mkd"),
("ml", "mal"),
("mn", "mon"),
("mr", "mar"),
("ms", "msa"),
("mt", "mlt"),
("my", "mya"),
("na", "nau"),
("nb", "nob"),
("nd", "nde"),
("ne", "nep"),
("ng", "ndo"),
("nl", "nld"),
("nn", "nno"),
("no", "nor"),
("nr", "nbl"),
("nv", "nav"),
("ny", "nya"),
("oc", "oci"),
("oj", "oji"),
("om", "orm"),
("or", "ori"),
("os", "oss"),
("pa", "pan"),
("pi", "pli"),
("pl", "pol"),
("ps", "pus"),
("pt", "por"),
("qu", "que"),
("rm", "roh"),
("rn", "run"),
("ro", "ron"),
("ru", "rus"),
("rw", "kin"),
("sa", "san"),
("sc", "srd"),
("sd", "snd"),
("se", "sme"),
("sg", "sag"),
("si", "sin"),
("sk", "slk"),
("sl", "slv"),
("sm", "smo"),
("sn", "sna"),
("so", "som"),
("sq", "sqi"),
("sr", "srp"),
("ss", "ssw"),
("st", "sot"),
("su", "sun"),
("sv", "swe"),
("sw", "swa"),
("ta", "tam"),
("te", "tel"),
("tg", "tgk"),
("th", "tha"),
("ti", "tir"),
("tk", "tuk"),
("tl", "tgl"),
("tn", "tsn"),
("to", "ton"),
("tr", "tur"),
("ts", "tso"),
("tt", "tat"),
("tw", "twi"),
("ty", "tah"),
("ug", "uig"),
("uk", "ukr"),
("ur", "urd"),
("uz", "uzb"),
("ve", "ven"),
("vi", "vie"),
("vo", "vol"),
("wa", "wln"),
("wo", "wol"),
("xh", "xho"),
("yi", "yid"),
("yo", "yor"),
("za", "zha"),
("zh", "zho"),
("zu", "zul"),
];
/// The three two-letter codes ISO 639-1 has since withdrawn, mapped to their
/// replacements. DVD-Video froze its language list on the 1988 edition, so
/// discs authored to the spec carry these spellings and no other table sees
/// them: `iw` Hebrew (now `he`), `in` Indonesian (now `id`), `ji` Yiddish
/// (now `yi`).
const ISO_639_1_DEPRECATED: &[(&str, &str)] = &[("iw", "he"), ("in", "id"), ("ji", "yi")];
/// Map an ISO 639-1 two-letter language code to its ISO 639-2/T three-letter
/// code, accepting the withdrawn DVD-era spellings (`iw`, `in`, `ji`) as
/// aliases for their replacements.
///
/// Covers the WHOLE of ISO 639-1, unlike [`menu_lang`], whose table only spans
/// the languages that show up in Blu-ray menu-graphic filenames. Callers that
/// convert a spec field — a DVD IFO attribute block, say — need the whole set:
/// narrowing it to the menu vocabulary would fold every other language onto
/// one value and make a disc's tracks indistinguishable from each other.
///
/// Case-insensitive and trimmed. Returns `None` for anything that is not an
/// ISO 639-1 code, so callers decide the fallback rather than getting a guess.
pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
let c = code.trim().to_ascii_lowercase();
let c = ISO_639_1_DEPRECATED
.iter()
.find(|(old, _)| *old == c)
.map_or(c.as_str(), |(_, new)| new);
ISO_639_1_TO_2
.iter()
.find(|(two, _)| *two == c)
.map(|(_, three)| *three)
}
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
@@ -789,4 +1020,99 @@ mod tests {
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
/// Structural invariants of `ISO_639_1_TO_2`: it must hold the complete
/// ISO 639-1 set (184 codes), every key a distinct pair of lowercase
/// letters and every value three lowercase letters. A typo'd or duplicated
/// row fails here rather than silently mislabelling a track.
#[test]
fn iso639_1_table_is_complete_and_well_formed() {
assert_eq!(
ISO_639_1_TO_2.len(),
184,
"ISO 639-1 defines 184 two-letter codes; the table must hold all \
of them"
);
let mut keys: Vec<&str> = ISO_639_1_TO_2.iter().map(|(two, _)| *two).collect();
keys.sort_unstable();
let unique = keys.len();
keys.dedup();
assert_eq!(unique, keys.len(), "no ISO 639-1 code may appear twice");
for (two, three) in ISO_639_1_TO_2 {
assert!(
two.len() == 2 && two.bytes().all(|b| b.is_ascii_lowercase()),
"{two:?} is not a two-letter lowercase ISO 639-1 code"
);
assert!(
three.len() == 3 && three.bytes().all(|b| b.is_ascii_lowercase()),
"{three:?} is not a three-letter lowercase ISO 639-2 code"
);
}
// The withdrawn DVD-era spellings resolve, and are not themselves
// rows in the main table (they are aliases, not codes).
for (old, new) in ISO_639_1_DEPRECATED {
assert!(
!ISO_639_1_TO_2.iter().any(|(two, _)| two == old),
"withdrawn code {old:?} must not be a table row"
);
assert_eq!(
iso639_1_to_iso639_2(old),
iso639_1_to_iso639_2(new),
"withdrawn code {old:?} must resolve exactly as {new:?}"
);
}
}
/// The two tables must not disagree. Every two-letter token `menu_lang`
/// accepts has to yield the same ISO 639-2/T code through
/// `iso639_1_to_iso639_2`, so a DVD-sourced language and a Blu-ray
/// menu-label language for the same tongue never produce different
/// `Language` elements.
#[test]
fn iso639_1_agrees_with_menu_lang() {
for (two, three) in ISO_639_1_TO_2 {
if let Some(via_menu) = menu_lang(two) {
assert_eq!(
via_menu, *three,
"menu_lang({two:?}) = {via_menu:?} disagrees with the ISO \
639-1 table's {three:?}"
);
}
}
// Spot-check the /T choice itself, on the languages where /B differs.
for (two, t_code) in [
("de", "deu"),
("fr", "fra"),
("zh", "zho"),
("cs", "ces"),
("nl", "nld"),
("el", "ell"),
("ro", "ron"),
("sk", "slk"),
("is", "isl"),
("hy", "hye"),
("ka", "kat"),
("fa", "fas"),
] {
assert_eq!(
iso639_1_to_iso639_2(two),
Some(t_code),
"the crate standardises on ISO 639-2/T, so {two:?} is \
{t_code:?} and never the bibliographic form"
);
}
}
/// Trimming, case-insensitivity, and the no-guess contract.
#[test]
fn iso639_1_normalizes_input_and_never_guesses() {
assert_eq!(iso639_1_to_iso639_2("RO"), Some("ron"));
assert_eq!(iso639_1_to_iso639_2(" Ro "), Some("ron"));
assert_eq!(iso639_1_to_iso639_2("IW"), Some("heb"));
assert_eq!(iso639_1_to_iso639_2("zz"), None);
assert_eq!(iso639_1_to_iso639_2(""), None);
assert_eq!(iso639_1_to_iso639_2("e"), None);
// A three-letter code is not ISO 639-1 input — that is menu_lang's job.
assert_eq!(iso639_1_to_iso639_2("eng"), None);
}
}
+4 -1
View File
@@ -101,6 +101,7 @@ pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFI
pub(crate) const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs;
pub(crate) mod bdnav;
pub(crate) mod clpi;
pub mod consts;
pub mod css;
@@ -109,7 +110,7 @@ pub mod diag;
pub mod dirimage;
pub mod disc;
pub mod drive;
pub mod dvdnav;
pub(crate) mod dvdnav;
pub mod error;
pub mod event;
pub mod halt;
@@ -129,6 +130,8 @@ pub mod progress;
pub mod scsi;
pub mod sector;
pub mod session;
#[cfg(test)]
pub(crate) mod testlog;
pub(crate) mod udf;
pub(crate) mod unlock_bridge;
+30 -20
View File
@@ -4,7 +4,7 @@
//! Contains play items (clips) with in/out timestamps,
//! stream info (video, audio, subtitle tracks).
//!
//! Reference: https://github.com/lw/BluRay/wiki/MPLS
//! Format is documented in the BD-ROM PlayList (MPLS) specification.
use crate::error::{Error, Result};
@@ -146,7 +146,10 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
}
let clip_id = String::from_utf8_lossy(&item[0..5]).to_string();
let connection_condition = item[9] & 0x0F;
// After clip_id[0..5] and codec_id[5..9], item[9] is fully reserved;
// item[10] holds is_multi_angle (bit 4) plus connection_condition in its
// low nibble. stc_id follows at item[11] and in_time at item[12..16].
let connection_condition = item[10] & 0x0F;
let in_time = u32::from_be_bytes([item[12], item[13], item[14], item[15]]);
let out_time = u32::from_be_bytes([item[16], item[17], item[18], item[19]]);
@@ -533,10 +536,12 @@ mod tests {
item.extend_from_slice(*clip_id);
// [5..9] codec_id ("M2TS")
item.extend_from_slice(b"M2TS");
// [9] connection_condition in low nibble
// [9] fully reserved
item.push(0);
// [10] is_multi_angle (bit 4) + connection_condition (low nibble)
item.push(*conn & 0x0F);
// [10..12] reserved
item.extend_from_slice(&[0u8; 2]);
// [11] stc_id / reserved
item.push(0);
// [12..16] in_time
item.extend_from_slice(&in_time.to_be_bytes());
// [16..20] out_time
@@ -1023,8 +1028,8 @@ mod tests {
}
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests below. Grounded in the BD-ROM MPLS spec
// (https://github.com/lw/BluRay/wiki/MPLS) byte layout.
// Added hardening tests below, grounded in the BD-ROM MPLS spec byte
// layout.
// ─────────────────────────────────────────────────────────────────────
/// Header guard: parse() requires `playlist_start + 10 <= data.len()`
@@ -1039,22 +1044,24 @@ mod tests {
assert!(parse(&data).is_err());
}
/// Spec: connection_condition is the LOW nibble of PlayItem byte[9]
/// (high nibble is reserved/flags). A byte 0xF5 must yield 5, not 0xF5.
/// connection_condition is the LOW nibble of PlayItem byte[10] (the high
/// nibble carries reserved bits and the is_multi_angle flag at bit 4).
/// byte[9] is entirely reserved and must NOT be read. A byte[10] of 0xF5
/// must yield 5, and a non-zero byte[9] must not leak into the value.
#[test]
fn connection_condition_is_low_nibble_only() {
// Build a custom item where byte[9] = 0xF5 (high nibble set).
// build_mpls masks with &0x0F when writing, so write raw to verify
// the PARSER masks. We patch the item byte directly after building.
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
let mut data = build_mpls(
&[(b"00001", 0, 0, 9000000)],
(1, 0, 0, 0, 0, 0, 0, 0),
&[video],
);
// Locate PlayItem byte[9]: header(40) + pl_header(10) + item_len(2) + 9.
let conn_idx = 40 + 10 + 2 + 9;
data[conn_idx] = 0xF5;
// PlayItem base: header(40) + pl_header(10) + item_len(2).
let item_base = 40 + 10 + 2;
// byte[9] fully reserved — set it to prove it does not leak in.
data[item_base + 9] = 0xFF;
// byte[10] high nibble set — parser must mask to the low nibble.
data[item_base + 10] = 0xF5;
let pl = parse(&data).expect("should parse");
assert_eq!(pl.play_items[0].connection_condition, 0x05);
}
@@ -1399,8 +1406,9 @@ mod tests {
let mut item = Vec::new();
item.extend_from_slice(b"00009");
item.extend_from_slice(b"M2TS");
item.push(0x01); // connection_condition
item.extend_from_slice(&[0u8; 2]);
item.push(0); // [9] reserved
item.push(0x01); // [10] connection_condition (low nibble)
item.push(0); // [11] stc_id / reserved
item.extend_from_slice(&90000u32.to_be_bytes()); // in_time
item.extend_from_slice(&180000u32.to_be_bytes()); // out_time
item.resize(32, 0); // pad through STN_OFFSET; item.len()==32 so no STN
@@ -1456,13 +1464,15 @@ mod tests {
}
/// The 20 bytes a PlayItem needs for clip_id(5) + codec_id(4) +
/// connection_condition(1) + reserved(2) + IN_time(4) + OUT_time(4).
/// reserved(1) + connection_condition byte(1) + stc_id/reserved(1) +
/// IN_time(4) + OUT_time(4).
fn play_item_20(clip: &[u8; 5], cc: u8, in_t: u32, out_t: u32) -> Vec<u8> {
let mut it = Vec::new();
it.extend_from_slice(clip);
it.extend_from_slice(b"M2TS");
it.push(cc);
it.extend_from_slice(&[0u8; 2]);
it.push(0); // [9] reserved
it.push(cc); // [10] connection_condition (low nibble)
it.push(0); // [11] stc_id / reserved
it.extend_from_slice(&in_t.to_be_bytes());
it.extend_from_slice(&out_t.to_be_bytes());
assert_eq!(it.len(), 20);
+325 -7
View File
@@ -91,6 +91,18 @@ pub struct Ac3Parser {
/// across the PES boundary because it may be the core of an AC-3-core +
/// E-AC-3-dependent frame set whose remaining substreams are in the next PES.
saw_extension: bool,
/// The access unit held open across the last PES boundary, ALREADY
/// scanned. The carry-over begins at its first byte, so without this the
/// next call re-scans and re-CRCs every syncframe of it from byte 0 — and
/// an access unit that keeps gaining substreams grows to [`MAX_AC3_BUF`]
/// (1 MiB) before the resync guard drops it, which on a ~2 KiB DVD PES is
/// three orders of magnitude of repeated work per packet.
held: Option<HeldAu>,
/// Test-only: syncframes examined (sized + CRC-gated) by
/// `scan_access_units`. Pins the resume above — the property it exists for
/// is a WORK bound, which no frame-level assertion can observe.
#[cfg(test)]
frames_scanned: u64,
}
impl Default for Ac3Parser {
@@ -107,6 +119,9 @@ impl Ac3Parser {
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
saw_extension: false,
held: None,
#[cfg(test)]
frames_scanned: 0,
}
}
@@ -149,20 +164,56 @@ impl Ac3Parser {
anchor: Option<PtsAnchor>,
at_eos: bool,
marks: &[(usize, super::pesbuf::PesFacts)],
) -> (Vec<Frame>, usize, i64) {
held: Option<HeldAu>,
) -> ScanOut {
let mut frames = Vec::new();
let mut pos = 0usize;
// Running PTS for the next access unit to emit in this call.
let mut frame_pts_ns = base_pts_ns;
let mut anchor = anchor;
let mut pending: Option<PendingAu> = None;
// How far this call has proved there is no further syncframe to
// process; carried over so the held access unit's own bytes (and the
// junk after them) are not searched again next call.
let mut scanned_to = 0usize;
// Resume a held access unit instead of re-deriving it. `keep_from` was
// its first byte, so it starts at 0 of this buffer, and every frame in
// it was sized and CRC-gated on the call that built it.
if let Some(h) = held {
let mut drop_reason = h.drop_reason;
// The one verdict that can have changed since: the track may have
// become poisoned while this access unit was held, and a re-scan
// would have picked that up.
if drop_reason.is_none() && self.tally.is_poisoned() {
drop_reason = Some("track-poisoned");
}
pending = Some(PendingAu {
start: 0,
end: h.end,
pts_ns: base_pts_ns,
duration_ns: h.duration_ns,
drop_reason,
bsid: h.bsid,
});
frame_pts_ns = base_pts_ns + h.duration_ns as i64;
pos = h.scanned_to;
scanned_to = h.scanned_to;
}
while pos < data.len() {
let sync = find_ac3_sync(&data[pos..]);
let start = match sync {
Some(offset) => pos + offset,
None => break,
None => {
// No syncword in `data[pos..]` at all: every byte but the
// last is proved sync-free (a syncword is two bytes and the
// second may still arrive).
scanned_to = data.len().saturating_sub(1).max(pos);
break;
}
};
scanned_to = start;
let remaining = &data[start..];
@@ -182,6 +233,7 @@ impl Ac3Parser {
// Invalid/sub-header frame size (e.g. an E-AC-3 frmsiz of 0/1
// sizing to a 2/4-byte fragment) — skip this sync word.
pos = start + 2;
scanned_to = pos;
continue;
}
@@ -191,6 +243,10 @@ impl Ac3Parser {
}
let frame = &data[start..start + frame_size];
#[cfg(test)]
{
self.frames_scanned += 1;
}
// Decodability gate: a syncframe with an out-of-range bsid (> 16) or
// a failed native CRC (payload corruption) poisons the access unit it
// belongs to — a dependent substream is useless without its parent and
@@ -258,6 +314,7 @@ impl Ac3Parser {
}
pos = start + frame_size;
scanned_to = pos;
}
// Close or HOLD the trailing access unit. The rest of its frame set — its
@@ -271,10 +328,22 @@ impl Ac3Parser {
// substream that extends an access unit — a plain AC-3 track keeps
// emitting every frame in-call.
let mut hold_from = None;
let mut held_out = None;
if let Some(au) = pending {
if !at_eos && (au.bsid >= 11 || self.saw_extension) {
frame_pts_ns = au.pts_ns;
hold_from = Some(au.start);
// Everything below `scanned_to` has been searched already, and
// the access unit's own frames have been sized and CRC-gated;
// record both, rebased onto the carry-over (which starts at
// `au.start`), so the next call resumes instead of redoing it.
held_out = Some(HeldAu {
end: au.end - au.start,
scanned_to: scanned_to.max(au.end) - au.start,
duration_ns: au.duration_ns,
drop_reason: au.drop_reason,
bsid: au.bsid,
});
} else {
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
@@ -306,7 +375,12 @@ impl Ac3Parser {
None => data.len(),
};
(frames, keep_from, frame_pts_ns)
ScanOut {
frames,
keep_from,
frame_pts_ns,
held: held_out,
}
}
}
@@ -319,6 +393,36 @@ struct PtsAnchor {
pts_ns: i64,
}
/// What one `scan_access_units` pass produced: the access units it emitted,
/// the offset in the scanned buffer from which bytes must be carried over to
/// the next call, the PTS to stamp on the access unit that begins that
/// carry-over, and — when the trailing access unit was HELD — the state that
/// lets the next call resume rather than re-derive it.
struct ScanOut {
frames: Vec<Frame>,
keep_from: usize,
frame_pts_ns: i64,
held: Option<HeldAu>,
}
/// A trailing access unit held across the PES boundary, already scanned.
/// Offsets are relative to the carry-over, which begins at the access unit's
/// first byte — so the access unit occupies `0..end`.
#[derive(Clone, Copy)]
struct HeldAu {
/// End of the access unit's bytes.
end: usize,
/// How far the scan that built it had searched (`>= end`). Bytes below it
/// hold no further syncframe to process.
scanned_to: usize,
/// Duration contributed by the access unit's `substreamid`-0 substream.
duration_ns: u64,
/// Decodability verdict reached for it so far.
drop_reason: Option<&'static str>,
/// bsid of the substream that opened it.
bsid: u8,
}
/// An access unit (frame set) under construction: `data[start..end]` is the
/// `substreamid`-0 independent substream frame plus every substream appended to it
/// so far — its dependents, and any additional independent substreams 1..7 with
@@ -477,6 +581,8 @@ impl CodecParser for Ac3Parser {
// non-empty PES today; this is defensive for any future caller).
if pes.discontinuity {
self.acc.clear();
// The held access unit's bytes went with it.
self.held = None;
}
if pes.data.is_empty() {
return Vec::new();
@@ -516,8 +622,13 @@ impl CodecParser for Ac3Parser {
buf.extend_from_slice(self.acc.as_slice());
let marks = self.acc.marks_snapshot();
let data = &buf;
let (frames, keep_from, frame_pts_ns) =
self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks);
let held = self.held.take();
let ScanOut {
frames,
keep_from,
frame_pts_ns,
held: still_held,
} = self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks, held);
if keep_from < data.len() {
let tail = &data[keep_from..];
@@ -531,6 +642,7 @@ impl CodecParser for Ac3Parser {
MAX_AC3_BUF
);
self.acc.clear();
self.held = None;
// Advance the cadence, as both sibling branches below do, so the
// three paths out of this block cannot disagree. Defensive: no
// input reaching this parser was found that both parses frames and
@@ -539,6 +651,7 @@ impl CodecParser for Ac3Parser {
self.flush_pts_ns = frame_pts_ns;
} else {
self.acc.drain(keep_from);
self.held = still_held;
// The carried bytes, when later completed and emitted (next call
// or by flush() at EOS), are timed at the PTS the scanner reached
// here: the PTS of the next access unit in presentation order, or
@@ -548,6 +661,7 @@ impl CodecParser for Ac3Parser {
}
} else {
self.acc.clear();
self.held = None;
// Nothing carried, but keep the cadence so a following PES with no
// PTS (no anchor) continues the timeline instead of reusing a stale
// value.
@@ -569,9 +683,10 @@ impl CodecParser for Ac3Parser {
let buf = self.acc.as_slice().to_vec();
let marks = self.acc.marks_snapshot();
self.acc.clear();
let held = self.held.take();
let out = self
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks)
.0;
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks, held)
.frames;
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
@@ -2199,6 +2314,125 @@ mod tests {
);
}
/// A 256-byte E-AC-3 syncframe with a valid CRC. `strmtyp`/`substreamid`
/// go into byte 2 (A/52 Annex E), which is what `substream_role` reads:
/// (0, 0) OPENS an access unit, (1, 0) is a dependent substream that
/// EXTENDS the open one.
fn eac3_substream_frame(strmtyp: u8, substreamid: u8) -> Vec<u8> {
const SIZE: usize = 256;
let frmsiz = SIZE / 2 - 1; // (frmsiz + 1) * 2 == SIZE
let mut f = vec![0u8; SIZE];
f[0] = 0x0B;
f[1] = 0x77;
f[2] = (strmtyp << 6) | (substreamid << 3) | ((frmsiz >> 8) as u8 & 0x07);
f[3] = (frmsiz & 0xFF) as u8;
f[5] = 16 << 3; // bsid 16 → E-AC-3
finalize_ac3_crc(&mut f);
f
}
/// An access unit closes only at the next `substreamid`-0 independent
/// substream, so one that keeps gaining dependent substreams stays OPEN
/// across PES boundaries and its bytes stay in the carry-over. The
/// carry-over must not be re-scanned — and re-CRCed — from the access
/// unit's first byte on every packet: the buffer only stops growing at
/// MAX_AC3_BUF (1 MiB), and a PES on a DVD is about 2 KiB, so re-deriving
/// the held access unit costs work quadratic in the packets fed.
///
/// Measured directly, because a work bound has no frame-level shadow:
/// `frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
/// Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
///
/// Mutation: pass `None` for `held` in `parse` (or drop the `if let
/// Some(h) = held` resume) — the count returns to the quadratic figure.
#[test]
fn a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet() {
const DEPENDENTS: usize = 200;
let mut parser = Ac3Parser::new();
// Opens the access unit.
let emitted = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
emitted.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..DEPENDENTS {
let f = parser.parse(&make_eac3_pes(eac3_substream_frame(1, 0)));
assert!(f.is_empty(), "a dependent substream extends the open unit");
}
let fed = (DEPENDENTS + 1) as u64;
assert!(
parser.frames_scanned <= 2 * fed,
"the scanner examined {} syncframes for {fed} fed — a held access \
unit must be resumed, not re-derived",
parser.frames_scanned
);
// ...and the resume must not have cost correctness: the whole frame
// set is still one access unit, emitted intact at EOS.
let out = parser.flush();
assert_eq!(out.len(), 1, "the frame set is a single access unit");
assert_eq!(
out[0].data.len(),
256 * (DEPENDENTS + 1),
"every substream of the frame set belongs to it"
);
}
/// A concealed gap must drop the HELD access unit, not just the byte
/// buffer.
///
/// `parse` clears `self.acc` on a discontinuity because the buffered bytes
/// are a truncated frame. The held access unit is described by OFFSETS into
/// exactly those bytes, so it has to go with them. Without
/// `self.held = None`, the next packet resumes a HeldAu whose `start`/`end`
/// were computed against the pre-gap buffer but are applied to the
/// unrelated post-gap bytes — splicing audio across the gap at best, and
/// indexing past the end of the new, shorter buffer at worst.
///
/// The two existing discontinuity tests use plain AC-3 (bsid < 11), which
/// never holds an access unit open, so neither of them reaches this reset.
#[test]
fn a_discontinuity_drops_the_held_access_unit_with_its_bytes() {
let mut parser = Ac3Parser::new();
// Open an access unit and extend it, so a HeldAu exists describing
// offsets into a large buffer.
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(0, 0)))
.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..8 {
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(1, 0)))
.is_empty(),
"a dependent substream extends the open unit"
);
}
// The gap. Its post-gap payload is deliberately far SHORTER than the
// held unit's bytes, so a stale HeldAu indexes past its end.
let mut gap = make_eac3_pes(eac3_substream_frame(0, 0));
gap.discontinuity = true;
let _ = parser.parse(&gap);
// Whatever comes out, nothing may carry pre-gap bytes: the truncated
// unit was dropped, so the only access unit that can be emitted is the
// one opened after the gap.
let out = parser.flush();
let total: usize = out.iter().map(|f| f.data.len()).sum();
assert!(
total <= 256,
"a post-gap access unit must not be spliced onto the 9 frames held \
before the gap; got {total} bytes across {} frame(s)",
out.len()
);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
@@ -2248,4 +2482,88 @@ mod tests {
"the unit belongs to the packet its FIRST byte came from"
);
}
/// A track that becomes POISONED while an access unit is held open across a
/// PES boundary must not emit that access unit when it resumes.
///
/// Mutation this catches: deleting (or inverting) the resume-path re-check
/// `if drop_reason.is_none() && self.tally.is_poisoned()` at the top of
/// `scan_access_units`. The verdict on a held access unit is frozen at the
/// moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
/// previous access unit is closed — so the very drop that crosses the
/// poison threshold lands after the next unit's verdict was already taken
/// as `None`. Without the re-check that unit is emitted as an ordinary
/// frame after the track has been judged too damaged to mux: corrupt audio
/// passed through as success, and worse, it is the ONE frame that escapes
/// a whole-track fallback whose entire point is that nothing after the
/// verdict ships.
///
/// Neither existing held-AU test reaches this: both keep a pristine tally.
///
/// The fixture drives the exact interleaving above. `DropTally` poisons
/// once `verified_dropped * 2 > kept + dropped` past the 200-AU gate, so
/// one PES carries 200 CRC-failing `substreamid`-0 syncframes followed by a
/// clean one. Closing corrupt unit #200 (which happens only when the clean
/// frame is reached) latches the poison — after that clean frame's own
/// verdict was computed. Being an E-AC-3 unit that can still gain
/// substreams, it is then HELD across the boundary with `drop_reason:
/// None`, which is precisely the state the re-check exists for.
#[test]
fn a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes() {
// One more than the verdict gate: the Nth close is what poisons.
const CORRUPT_AUS: usize = 200;
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..CORRUPT_AUS {
let mut f = eac3_substream_frame(0, 0);
// Corrupt a payload byte AFTER the CRC was finalized: the header
// (sizing, strmtyp/substreamid, bsid) stays intact so the frame is
// still parsed as a whole access unit and fails only the CRC —
// a VERIFIED drop, the only kind that feeds the poison verdict.
f[100] ^= 0xFF;
assert!(!frame_crc_ok(&f), "the fixture frame must fail its CRC");
data.extend_from_slice(&f);
}
// The clean access unit. Last in the PES, so it is held open.
let clean = eac3_substream_frame(0, 0);
assert!(
frame_crc_ok(&clean),
"the held unit is individually decodable"
);
data.extend_from_slice(&clean);
let emitted = parser.parse(&make_eac3_pes(data));
assert!(
emitted.is_empty(),
"every corrupt unit is dropped and the clean one is held; got {} frame(s)",
emitted.len()
);
// The state the re-check depends on: the track IS poisoned, and the
// held unit was opened before that verdict existed.
assert!(
parser.tally.is_poisoned(),
"fixture must actually cross the whole-track poison threshold"
);
// Resume. The next PES opens a new unit, which closes the held one.
let out = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
out.is_empty(),
"an access unit held across the boundary must not be emitted once \
the track is poisoned; got {} frame(s) totalling {} bytes",
out.len(),
out.iter().map(|f| f.data.len()).sum::<usize>()
);
// ...and nothing may leak at end of stream either.
let tail = parser.flush();
assert!(
tail.is_empty(),
"a poisoned track emits nothing at EOS; got {} frame(s)",
tail.len()
);
assert!(
parser.dropped_frames() > CORRUPT_AUS as u64,
"the held unit must be ACCOUNTED as a drop, not silently discarded"
);
}
}
+18 -6
View File
@@ -29,10 +29,17 @@ pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
}
/// CRC-16 with polynomial 0x002D, init 0, MSB-first, used by the MLP / Dolby
/// TrueHD major-sync header checksum. NOTE: MLP's checksum is the "reversed"
/// scheme — the stored trailer word is the little-endian-read CRC, so this
/// standard CRC must be compared against the stored bytes read big-endian.
/// The caller handles that comparison (see `truehd::mlp_major_sync_ok`).
/// TrueHD major-sync header checksum.
///
/// NOTE: MLP's checksum is the "reversed" scheme. This function emits its two
/// bytes in the OPPOSITE order to a standard little-endian CRC readout, so the
/// caller swaps them back and compares against the stored trailer word read
/// LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
///
/// Comparing big-endian instead is precisely the bug that function was fixed
/// for: it could never validate a real extended major sync, so whole TrueHD
/// tracks were dropped silently. This comment used to prescribe exactly that,
/// and to point at a `truehd::mlp_major_sync_ok` that does not exist.
/// Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
@@ -103,8 +110,13 @@ mod tests {
#[test]
fn crc16_mlp_residue_property_holds() {
// Appending the big-endian CRC zeroes the residue over message+crc — the
// scheme `truehd::mlp_major_sync_ok` relies on.
// Appending the big-endian CRC zeroes the residue over message+crc.
// This is a property of the CRC itself, pinned here so a change to the
// polynomial or the bit order is caught. It is NOT how the TrueHD
// caller validates a major sync: `truehd::mlp_major_sync_crc_ok` does a
// swap-and-XOR compare against the little-endian trailer word. (This
// comment used to claim the caller relied on the residue, and named a
// `truehd::mlp_major_sync_ok` that does not exist.)
let msg = [0xF8u8, 0x72, 0x6F, 0xBA];
let c = crc16_mlp(&msg);
let mut framed = msg.to_vec();
+8 -4
View File
@@ -205,8 +205,12 @@ impl CodecParser for DtsParser {
// TRUNCATED. Splicing post-gap bytes onto it corrupts the core/extension
// framing (→ "Failed to decode block code(s)" / "Invalid data found").
// Drop the partial AU and its PTS marks; the next PES re-bases a fresh
// unit. (Audio has no inter-frame refs — dropping the spliced partial is
// the whole fix; the video ResyncGate handles video.)
// unit. (DTS core/extension frames are independently decodable — every
// frame re-inits on its own syncword — so dropping the truncated partial
// and resuming on the next frame is sufficient here. This is NOT true of
// every audio codec: TrueHD/MLP carries predictor + restart state across
// access units and must additionally drop forward to its next major sync;
// see `codec::truehd`. The video ResyncGate handles inter-coded video.)
//
// Handle the discontinuity BEFORE the empty-data guard so the signal can
// never be stranded by an empty post-gap PES (defensive; the demuxer only
@@ -934,7 +938,7 @@ mod tests {
// AU = core(512) + a REAL EXSS substream whose XLL payload embeds a DTS
// core syncword decoding to a plausible size (512). The heuristic-only
// framer would split here and truncate the lossless extension (the
// Dunkirk "Failed to decode block code(s)" decoder-failure class).
// real-disc "Failed to decode block code(s)" decoder-failure class).
// Precise EXSS sizing spans the whole extension to the REAL next core.
let core = make_dts_core(512);
let exss = make_exss(600, Some(40));
@@ -1042,7 +1046,7 @@ mod tests {
// B1: a partial DTS core is buffered, then a concealed gap (PES marked
// discontinuity) carries a fresh core. The truncated partial must be
// DROPPED — splicing it makes the framer emit a corrupt sub-core-length
// AU (the Dunkirk "Failed to decode block code(s)" decoder-failure
// AU (the real-disc "Failed to decode block code(s)" decoder-failure
// class) and strands the rest. With the fix the post-gap core is the
// only AU, and it carries the post-gap PTS (not the stale pre-gap one).
let mut parser = DtsParser::new();
+60 -11
View File
@@ -159,7 +159,13 @@ impl CodecParser for DvdSubParser {
/// Convert a single YCbCr color to RGB, clamping to [0, 255].
///
/// Input: `[padding, Y, Cb, Cr]` (as stored in DVD IFO PGC data).
/// Input: `[padding, Y, Cr, Cb]` (as stored in DVD IFO PGC data). Note the
/// chroma order: the on-disc PGC CLUT is **Cr before Cb** — byte 2 is Cr and
/// byte 3 is Cb. Reading byte 2 as Cb swaps red and blue on every chromatic
/// entry, and is invisible on the achromatic (white/black/grey, Cb = Cr = 128)
/// entries that dominate real palettes, which is how it survives casual
/// inspection. The order is fixed by the DVD-Video PGC format, not by us.
///
/// Returns `[R, G, B]`.
///
/// Range convention (deliberate): this uses the **full-range (JFIF) BT.601**
@@ -175,8 +181,8 @@ impl CodecParser for DvdSubParser {
/// side in lockstep.
pub fn ycbcr_to_rgb(color: &[u8; 4]) -> [u8; 3] {
let y = color[1] as f64;
let cb = color[2] as f64;
let cr = color[3] as f64;
let cr = color[2] as f64;
let cb = color[3] as f64;
let r = y + 1.402 * (cr - 128.0);
let g = y - 0.344 * (cb - 128.0) - 0.714 * (cr - 128.0);
@@ -198,7 +204,7 @@ fn clamp_u8(v: f64) -> u8 {
/// Format a 16-color YCbCr palette as a VobSub `.idx` header for S_VOBSUB
/// CodecPrivate.
///
/// Each entry is `[padding, Y, Cb, Cr]`. Output is a UTF-8 text block carrying
/// Each entry is `[padding, Y, Cr, Cb]`. Output is a UTF-8 text block carrying
/// the two `.idx` header lines mkvmerge / libvobsub expect:
///
/// ```text
@@ -432,24 +438,26 @@ mod tests {
#[test]
fn ycbcr_to_rgb_clamps_overflow() {
// Y=255, Cr=255 → R would be 255 + 1.402*127 = ~433, should clamp to 255
let color = [0x00, 255, 128, 255];
// Y=255, Cr=255 → R would be 255 + 1.402*127 = ~433, should clamp to 255.
// Cr is byte 2 in the on-disc [pad, Y, Cr, Cb] layout.
let color = [0x00, 255, 255, 128];
let [r, _g, _b] = ycbcr_to_rgb(&color);
assert_eq!(r, 255);
}
#[test]
fn ycbcr_to_rgb_clamps_underflow() {
// Y=0, Cr=0 → R = 0 + 1.402*(0-128) = -179, should clamp to 0
let color = [0x00, 0, 128, 0];
// Y=0, Cr=0 → R = 0 + 1.402*(0-128) = -179, should clamp to 0.
// Cr is byte 2 in the on-disc [pad, Y, Cr, Cb] layout.
let color = [0x00, 0, 0, 128];
let [r, _g, _b] = ycbcr_to_rgb(&color);
assert_eq!(r, 0);
}
#[test]
fn ycbcr_to_rgb_red() {
// Approximate red: Y=82, Cb=90, Cr=240
let color = [0x00, 82, 90, 240];
// Approximate red: Y=82, Cr=240, Cb=90 — on disc as [pad, Y, Cr, Cb].
let color = [0x00, 82, 240, 90];
let [r, g, b] = ycbcr_to_rgb(&color);
// R = 82 + 1.402*(240-128) = 82 + 156.9 ≈ 239
// G = 82 - 0.344*(90-128) - 0.714*(240-128) = 82 + 13.1 - 79.97 ≈ 15
@@ -459,6 +467,46 @@ mod tests {
assert!(b < 30, "B should be low for red, got {}", b);
}
/// On-disc DVD PGC CLUT byte order is `[0, Y, Cr, Cb]` — byte 2 is **Cr**
/// and byte 3 is **Cb**, per the DVD-Video PGC format.
///
/// This fixture uses a real on-disc red entry, so it fails if the two
/// chroma bytes are ever exchanged again. It is deliberately NOT built
/// from this crate's own doc comments: those described the order wrongly
/// for a long time, and the previous version of this test inherited the
/// error from them and therefore could not detect it.
///
/// A saturated RED entry therefore appears on disc as Y=76, Cr=255, Cb=85
/// (full-range BT.601 encoding of RGB #FF0000), i.e. bytes
/// `[0x00, 76, 255, 85]`. Reading byte 2 as Cb and byte 3 as Cr instead
/// turns this entry BLUE, which is the exact user-visible symptom.
///
/// The pre-existing `_white` / `_black` tests cannot catch this: they use
/// Cb = Cr = 128, so exchanging two equal bytes is a literal no-op.
#[test]
fn ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb() {
// On-disc [pad, Y, Cr, Cb] for saturated red.
let on_disc_red = [0x00u8, 76, 255, 85];
let [r, g, b] = ycbcr_to_rgb(&on_disc_red);
assert!(
r > 200 && b < 60,
"on-disc red [0,Y=76,Cr=255,Cb=85] must render red-dominant, \
got R={r} G={g} B={b} (R and B swapped => byte 2/3 are transposed)"
);
assert_eq!([r, g, b], [254, 0, 0], "exact full-range BT.601 red");
// And the converse: a saturated BLUE on-disc entry (Y=29, Cr=107, Cb=255)
// must not come out red.
let on_disc_blue = [0x00u8, 29, 107, 255];
let [r2, g2, b2] = ycbcr_to_rgb(&on_disc_blue);
assert!(
b2 > 200 && r2 < 60,
"on-disc blue [0,Y=29,Cr=107,Cb=255] must render blue-dominant, \
got R={r2} G={g2} B={b2}"
);
}
// ── Palette formatting tests ──────────────────────────────────────────
#[test]
@@ -699,7 +747,8 @@ mod tests {
#[test]
fn ycbcr_blue_channel_clamps_high() {
// B = Y + 1.772*(Cb-128). Y=128, Cb=255 → 128 + 1.772*127 ≈ 353 → clamp 255.
let [_r, _g, b] = ycbcr_to_rgb(&[0x00, 128, 255, 128]);
// Cb is byte 3 in the on-disc [pad, Y, Cr, Cb] layout.
let [_r, _g, b] = ycbcr_to_rgb(&[0x00, 128, 128, 255]);
assert_eq!(b, 255, "blue clamps at 255");
}
+73 -1
View File
@@ -1,7 +1,8 @@
//! H.264 (AVC) elementary stream parser.
//!
//! Extracts SPS and PPS NAL units for MKV codecPrivate.
//! Detects keyframes (IDR slices).
//! Detects keyframes: IDR slices, and open-GOP intra-coded access units (the
//! non-IDR recovery points BD titles use — see the intra promotion in `parse`).
//! Each PES packet = one access unit = one frame.
use super::coding::{CodingType, PictureInfo};
@@ -351,6 +352,26 @@ impl CodecParser for H264Parser {
return Vec::new();
}
// Open-GOP resync anchor: an intra-coded access unit is a keyframe even
// when it is NOT an IDR. BD H.264 titles use open GOPs whose only
// random-access point is a non-IDR I-frame (the recovery point, which an
// encoder MAY additionally tag with a recovery_point SEI). Keying off the
// already-measured picture coding type covers every signalling — a
// recovery_point SEI, or a disc that marks the recovery point only by GOP
// structure with no SEI at all — where SEI parsing alone would miss the
// latter. Without this, the B1 resync gate (`mux/resync.rs`), once armed
// by a concealed-gap / clip-seam discontinuity, never sees a keyframe on
// such a tail and drops every frame to EOF, truncating the video. HEVC
// does not hit this: its open-GOP IRAP (CRA) is a distinct NAL type
// already flagged a keyframe (`mux/codec/hevc.rs`); H.264 has no such NAL
// tag, so the intra coding type is the equivalent signal. Consistent with
// `videomap.rs`'s model that a keyframe IS the intra decode-restart point.
// IDR is still flagged by NAL type in the loop above, so a slice whose
// header failed to parse (`coding_type == None`) is unaffected here.
if coding_type == Some(CodingType::I) {
keyframe = true;
}
// Every keyframe is self-contained: re-assert the active SPS/PPS in-band
// ahead of the slices (even when unchanged vs codecPrivate) so a decoder
// that dropped the set at a reset recovers, and a stale avcC re-apply
@@ -694,6 +715,57 @@ mod tests {
}
}
/// Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
/// keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD open-
/// GOP tail instead of dropping every frame to EOF and truncating the video.
/// BD H.264 titles reach random access at a non-IDR I-frame (the recovery
/// point); HEVC gets this free from the CRA NAL type, H.264 does not, so the
/// measured intra coding type is the signal. Red before the intra promotion
/// in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
#[test]
fn open_gop_intra_access_unit_is_a_keyframe() {
// Annex B AU: one non-IDR coded slice (NAL type 1, nal_ref_idc 3 → 0x61)
// whose header decodes first_mb_in_slice = 0 (ue = "1") and slice_type = 7
// (ue = "0001000"), i.e. an all-I slice. The two ue(v) fields pack into
// one byte: "1" + "0001000" = 0b1000_1000 = 0x88.
let au = vec![0x00, 0x00, 0x01, 0x61, 0x88, 0x00];
let mut parser = H264Parser::new();
let frames = parser.parse(&make_pes(au, Some(0)));
assert_eq!(frames.len(), 1, "one PES access unit → one frame");
assert_eq!(
frames[0].coding.map(|c| c.coding_type()),
Some(CodingType::I),
"the slice header measures an I picture",
);
assert!(
frames[0].keyframe,
"an open-GOP intra access unit is a resync-safe keyframe even without an IDR",
);
}
/// Guard the promotion's precision: a non-intra (P) access unit must NOT be
/// promoted to a keyframe, or the resync gate would resume on a frame that
/// carries inter-references to dropped data and the MKV cue index would gain
/// bogus random-access points.
#[test]
fn inter_coded_access_unit_is_not_a_keyframe() {
// Same NAL type 1, but slice_type = 0 (P): ue "1", so with first_mb = 0
// the byte is "1" + "1" padded = 0b1100_0000 = 0xC0.
let au = vec![0x00, 0x00, 0x01, 0x61, 0xC0, 0x00];
let mut parser = H264Parser::new();
let frames = parser.parse(&make_pes(au, Some(0)));
assert_eq!(frames.len(), 1);
assert_eq!(
frames[0].coding.map(|c| c.coding_type()),
Some(CodingType::P),
"the slice header measures a P picture",
);
assert!(
!frames[0].keyframe,
"an inter-coded access unit must never be a keyframe",
);
}
/// The slice-header parse must remove emulation-prevention bytes first
/// (ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is
/// EBSP, and a 0x03 following 0x00 0x00 is not part of the RBSP.
+1 -1
View File
@@ -1183,7 +1183,7 @@ mod tests {
#[test]
fn opening_au_keeps_disc_pts_and_opening_seq_header_no_zero_floor() {
// SOTL SUB-TASK 2 regression (opening-GOP / still-frame open). A DVD
// Opening-GOP / still-frame open regression. A DVD
// title opens on a VOBU that begins with a sequence header + I-frame; the
// disc stamps that opening I-frame at its REAL (non-zero) timeline PTS,
// not 0. The parser must (a) emit the opening I-frame with that real PTS
+89
View File
@@ -367,6 +367,28 @@ impl CodecParser for TrueHdParser {
// emits non-empty PES today).
if pes.discontinuity {
self.acc.clear();
// Unlike AC-3/DTS (independent frames), MLP/TrueHD carries filter +
// predictor + restart state ACROSS access units. A demux-signalled
// discontinuity (lost TS packets / a concealed source gap) leaves that
// state stale, so resuming mid-stream on the next parity-OK AU decodes
// against broken state and desyncs a conformant decoder ("restart header
// sync incorrect" / "Invalid blocksize"). Arm the same drop-forward the
// corruption path uses: keep dropping until the next CRC-valid major sync
// (a clean re-init point), then resume. This mirrors the video path's
// "resync on the next keyframe" after a gap — the gapped audio can't be
// recovered either way, but the output stays decoder-clean instead of
// shipping a decoder-choking seam.
//
// Only once a validated major sync has established a baseline: before
// that there is no CRC-validatable re-init point to resume on, so arming
// here would collateral-drop the WHOLE track waiting for a major sync we
// cannot yet validate — the exact stream-head catastrophe `au_check`
// guards (a checksum-failed head major sync is kept, not nuked). At
// stream head we therefore keep the current behaviour: drop the truncated
// partial (above) and let the fresh post-gap unit through.
if self.num_substreams.is_some() {
self.resync_pending = true;
}
}
if pes.data.is_empty() {
return Vec::new();
@@ -913,6 +935,73 @@ mod tests {
);
}
#[test]
fn discontinuity_resyncs_forward_to_next_major_sync() {
// A source-signalled discontinuity (lost TS packets / a concealed gap)
// leaves MLP's cross-AU predictor + restart state stale. Resuming
// mid-stream on the next parity-OK AU decodes against broken state and
// desyncs a conformant decoder ("restart header sync incorrect" /
// "Invalid blocksize"). So a discontinuity must arm the SAME drop-forward
// the corruption path uses: drop every post-gap AU until the next
// CRC-valid major sync (the clean re-init point), then resume there.
//
// Regression for a real disc whose transport
// stream has a multi-PID continuity-counter gap at ~6:49: before this
// fix the TrueHD path cleared its buffer and resumed on the next NON-major
// AU, shipping a decoder-choking seam (559 `restart header sync` errors),
// while AC-3/DTS on the same disc were fine because THEIR frames are
// independently decodable. Video already resyncs to a keyframe after a
// gap (see `mux::resync`); this gives TrueHD the equivalent, keyed on the
// codec's own re-init point (a major-sync AU).
let mut parser = TrueHdParser::new();
// Establish a baseline so num_substreams is known and the gate is live.
let mut pre = valid_major_sync();
pre.extend_from_slice(&valid_normal_au());
let pre_frames = parser.parse(&make_pes(pre, Some(90000)));
assert_eq!(pre_frames.len(), 2, "baseline: major sync + one normal AU");
// Post-gap PES: two normal AUs, then a valid major sync, then a normal AU.
// `discontinuity: true` says packets were lost before this PES.
let mut post = valid_normal_au();
post.extend_from_slice(&valid_normal_au());
post.extend_from_slice(&valid_major_sync());
post.extend_from_slice(&valid_normal_au());
let post_pes = PesPacket {
source: None,
pid: 0x1100,
pts: Some(90000 + 4 * 900),
dts: None,
data: post,
discontinuity: true,
};
let mut frames = parser.parse(&post_pes);
frames.extend(parser.flush());
// Only the major sync and the AU after it survive; the two leading
// post-gap normal AUs are dropped for lack of a re-init point.
assert_eq!(
frames.len(),
2,
"resume only at the major sync + what follows it (got {})",
frames.len()
);
assert!(
frames[0].keyframe,
"the first surviving post-gap frame MUST be a major sync (re-init \
point), never a mid-stream AU"
);
assert!(
!frames[1].keyframe,
"the AU after the re-init point is a normal AU"
);
assert_eq!(
parser.dropped_frames(),
2,
"the two post-gap AUs before the major sync are dropped as collateral"
);
}
#[test]
fn crc_failed_head_major_sync_is_kept_not_track_killed() {
// REGRESSION (every TrueHD title muxed after the checksum gate landed): a
+406 -4
View File
@@ -190,6 +190,9 @@ pub struct DiscStream {
/// timestamp cost or `prof_active()` env-var lookup (which takes a
/// process-wide lock).
profiling: bool,
/// Cumulative bytes fed to the demuxer across all read buffers,
/// used to stamp byte-exact provenance (`SourcePos`) onto demuxed packets.
fed_bytes: u64,
/// B1 drop-to-keyframe resync gate, one per stream — mirrors
/// `PipelinedPesStream`. After an AACS-concealed (undecryptable) gap stamps
/// a frame `discontinuity`, the gate drops forward inter-coded video frames
@@ -331,6 +334,7 @@ impl DiscStream {
pid_to_track,
skip_parse: std::env::var_os("FREEMKV_SKIP_PARSE").is_some(),
profiling: std::env::var_os("FREEMKV_PROFILE").is_some(),
fed_bytes: 0,
resync,
is_video,
})
@@ -566,6 +570,20 @@ impl DiscStream {
.into());
}
// The read SOURCE is gone (a prefetch producer thread that
// terminated), not one range of media. Shrinking and retrying at
// the same LBA asks a dead source for data it can never produce,
// and the `skip_errors` branch below would then zero-fill and
// advance over every remaining sector of the title and still
// return success. Abort with the terminal error itself — a
// fabricated SCSI status would be a lie, so this is deliberately
// NOT folded into the transport-failure arm above.
if let Some(e) = res.as_ref().err()
&& e.is_source_terminated()
{
return Err(crate::error::Error::SourceTerminated.into());
}
if (sectors as u32) <= align {
// Bottomed out at one unit (AACS) / one sector (CSS) / the
// extent tail. This is single-pass disc→MKV, which has NO Pass N
@@ -621,6 +639,16 @@ impl DiscStream {
.into());
}
// Same rule as after the first-attempt read: the 60s recovery
// read goes through the same source, so it can be the call
// that discovers the source is dead. Skipping the unit would
// zero-fill the rest of the title as fabricated content.
if let Some(e) = rec.as_ref().err()
&& e.is_source_terminated()
{
return Err(crate::error::Error::SourceTerminated.into());
}
// Recovery read also failed. Skip the WHOLE failed unit or bail.
// Zero-filling and advancing by the full unit keeps
// current_offset unit-aligned, so the next read still begins on a
@@ -825,7 +853,7 @@ impl crate::pes::Stream for DiscStream {
continue;
};
let pes = super::ts::PesPacket {
source: None,
source: ps.source,
pid,
pts: ps.pts.map(|p| p as i64),
dts: ps.dts.map(|d| d as i64),
@@ -899,8 +927,11 @@ impl crate::pes::Stream for DiscStream {
// read_sectors call. The pre-0.18 inline decrypt step
// lived here.
let buf_base = self.fed_bytes;
self.fed_bytes = self.fed_bytes.saturating_add(bytes as u64);
if let Some(ref mut demuxer) = self.ts_demuxer {
let packets = demuxer.feed(&self.read_buf[..bytes]);
let packets = demuxer.feed_at(buf_base, &self.read_buf[..bytes]);
let t2 = self.profiling.then(std::time::Instant::now);
if let (Some(t1), Some(t2)) = (t1, t2) {
prof_tick("feed", t2.duration_since(t1).as_nanos(), 0);
@@ -962,7 +993,7 @@ impl crate::pes::Stream for DiscStream {
prof_tick("consume", t3.duration_since(t2).as_nanos(), 0);
}
} else if let Some(ref mut demuxer) = self.ps_demuxer {
let packets = demuxer.feed(&self.read_buf[..bytes]);
let packets = demuxer.feed_at(buf_base, &self.read_buf[..bytes]);
for ps in &packets {
// Route by the REAL DVD PID (see consume_ps in
// pipelined_stream.rs); the old (sub_id & 0x1F)+1
@@ -998,7 +1029,7 @@ impl crate::pes::Stream for DiscStream {
};
let pes = super::ts::PesPacket {
source: None,
source: ps.source,
pid,
pts: ps.pts.map(|p| p as i64),
dts: ps.dts.map(|d| d as i64),
@@ -1859,6 +1890,130 @@ mod tests {
);
}
/// REGRESSION (round-4 audit): an ordinary MEDIUM ERROR bad sector must
/// keep its identity when it crosses the prefetch producer channel — the
/// same `DiscRead` with its SCSI status, NOT a transport failure.
///
/// `PrefetchedSectorSource::read_sectors` re-wrapped every error that
/// crossed the channel as `Error::IoError`, and `is_scsi_transport_failure`
/// matches `IoError` (the wedged-USB-bridge arm). So a bad sector reached
/// `fill_extents` looking like a dead bus and aborted the pass with a
/// fabricated status 0xFF — the exact inverse of what that short-circuit
/// exists for, and it told the user to power-cycle a healthy drive.
///
/// Asserted on the source, not on a `fill_extents` skip: the producer
/// thread exits for good after sending an error, so nothing downstream of
/// it can genuinely recover the rest of the title (see
/// `dead_prefetch_producer_does_not_silently_zero_fill_the_title`). An
/// assertion that the pass continues could only ever have been satisfied
/// by fabricated zeros.
#[test]
fn bad_sector_keeps_its_identity_across_the_prefetch_channel() {
const COUNT: u32 = 9;
let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let reader = RecordingReader {
capacity: COUNT,
bad_sector: 4,
log: log.clone(),
};
let mut prefetched = crate::sector::PrefetchedSectorSource::new_with_events(
reader,
vec![crate::disc::Extent {
start_lba: 0,
sector_count: COUNT,
}],
8,
1,
None,
None,
)
.expect("spawn producer");
let mut buf = vec![0u8; 8 * 2048];
let err = crate::sector::SectorSource::read_sectors(&mut prefetched, 0, 8, &mut buf, false)
.expect_err("the batch covering the bad sector must fail");
assert!(
!err.is_scsi_transport_failure(),
"a MEDIUM ERROR bad sector is not a dead bus; got {err:?}"
);
assert!(
matches!(
err,
crate::error::Error::DiscRead {
sector: 4,
status: Some(0x02),
..
}
),
"the producer's typed error must survive the channel intact; got {err:?}"
);
}
/// The prefetch producer thread terminates PERMANENTLY on its first read
/// error, so once one bad sector has crossed the channel the source can
/// never deliver another byte. Driving `fill_extents` to exhaustion after
/// that must NOT look like a completed pass: every remaining sector would
/// be fabricated zeros, and DATA LOSS MUST NEVER LOOK LIKE SUCCESS.
///
/// The expectation is the product rule, not the code: a source that is
/// permanently out of data must report that, not answer `Ok(0)` forever —
/// which `commit_read` legitimately reads as an ordinary short read and
/// zero-fills.
#[test]
fn dead_prefetch_producer_does_not_silently_zero_fill_the_title() {
const COUNT: u32 = 30;
let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let reader = RecordingReader {
capacity: COUNT,
bad_sector: 4,
log: log.clone(),
};
let prefetched = crate::sector::PrefetchedSectorSource::new_with_events(
reader,
vec![crate::disc::Extent {
start_lba: 0,
sector_count: COUNT,
}],
8,
1,
None,
None,
)
.expect("spawn producer");
let mut stream = DiscStream::new(
Box::new(prefetched),
synthetic_title(COUNT),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
stream.skip_errors = true;
// Drive the whole title. Bounded so a regression cannot hang the suite.
let mut completed_clean = false;
for _ in 0..(COUNT as usize * 4) {
match stream.fill_extents() {
Ok(true) => continue,
Ok(false) => {
completed_clean = true;
break;
}
Err(_) => break,
}
}
assert!(
!completed_clean,
"the producer died at sector 4, so sectors 4..{COUNT} were never \
read — reporting the pass as complete zero-fills {} of {} bytes \
and calls it success",
stream.lost_bytes,
COUNT as u64 * 2048
);
}
/// AACS unit-alignment skip (the #1 coverage gap). With `unit_align=3`
/// (DecryptKeys::Aacs) and `skip_errors=true`, a single bad mid-extent
/// sector must NOT desync the rest of the title: every `read_sectors`
@@ -2478,6 +2633,253 @@ mod tests {
);
}
#[test]
fn disc_stream_stamps_source_provenance() {
use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream};
use crate::pes::Stream;
// Build 1 TS packet containing a complete PES for PID 0x1011
// 4-byte BD prefix + 188-byte TS packet (PUSI=1, PID=0x1011)
let mut ts_pkt = vec![0x00, 0x00, 0x00, 0x00, 0x47, 0x50, 0x11, 0x10];
// PES header: 00 00 01 E0, length 0x000E (14 bytes), flags 0x80 0x80 (PTS present), header_len 5, PTS 0
ts_pkt.extend_from_slice(&[
0x00, 0x00, 0x01, 0xE0, 0x00, 0x0E, 0x80, 0x80, 0x05, 0x21, 0x00, 0x01, 0x00, 0x01,
]);
// Annex B AU: SPS + PPS + IDR slice
ts_pkt.extend_from_slice(&[
0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0x00, 0x0A, // SPS
0x00, 0x00, 0x00, 0x01, 0x68, 0xCE, 0x38, 0x80, // PPS
0x00, 0x00, 0x00, 0x01, 0x65, 0x88, 0x80, // IDR slice
]);
ts_pkt.resize(192, 0xFF);
let mut sector = ts_pkt;
sector.resize(2048, 0xFF);
struct SectorPayloadReader(Vec<u8>);
impl crate::sector::SectorSource for SectorPayloadReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let n = (count as usize * 2048).min(self.0.len()).min(buf.len());
buf[..n].copy_from_slice(&self.0[..n]);
Ok(n)
}
fn capacity_sectors(&self) -> u32 {
1
}
}
let mut title = synthetic_title(1);
title.streams = vec![crate::disc::Stream::Video(VideoStream {
pid: 0x1011,
codec: Codec::H264,
resolution: Resolution::R1080p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Bt709,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
})];
title.content_format = ContentFormat::BdTs;
let reader = SectorPayloadReader(sector);
let mut s = DiscStream::new(
Box::new(reader),
title,
crate::decrypt::DecryptKeys::None,
1,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
let frame = s.read().unwrap().expect("expected 1 frame");
assert!(
frame.source.is_some(),
"DiscStream must stamp byte provenance (source) on demuxed frames"
);
assert_eq!(frame.source.unwrap().byte, 0);
}
// ── PS-path provenance ────────────────────────────────────────────
/// A 14-byte MPEG-2 PS pack header (`00 00 01 BA …`), no stuffing.
fn ps_pack_header() -> Vec<u8> {
let mut p = vec![0x00, 0x00, 0x01, 0xBAu8];
p.extend_from_slice(&[0x44, 0x00, 0x04, 0x00, 0x04, 0x01, 0x01, 0x89, 0xC3]);
p.push(0xF8); // pack_stuffing_length = 0 (low 3 bits)
p
}
/// One coded MPEG-2 picture: picture header (coding_type, temporal_ref) +
/// a picture-coding extension + slice padding, so the real `Mpeg2Parser`
/// treats it as a complete picture.
fn ps_pic(coding_type: u8, tr: u16) -> Vec<u8> {
let b4 = ((tr >> 2) & 0xFF) as u8;
let b5 = (((tr & 0x03) as u8) << 6) | ((coding_type & 0x07) << 3);
let mut au = vec![0x00, 0x00, 0x01, 0x00u8, b4, b5, 0x00, 0x00];
au.extend_from_slice(&[0x00, 0x00, 0x01, 0xB5u8, 0x80, 0x00, 0x03, 0x80, 0x00]);
au.extend_from_slice(&[0xAA; 32]);
au
}
/// One GOP's ES: seq header + GOP + I/P/B pictures.
fn ps_gop_es() -> Vec<u8> {
let mut es = seq_header(720, 480);
es.extend_from_slice(&[0x00, 0x00, 0x01, 0xB8u8, 0x00, 0x00, 0x00, 0x00]); // GOP header
es.extend_from_slice(&ps_pic(1, 0)); // I (keyframe)
es.extend_from_slice(&ps_pic(2, 2)); // P
es.extend_from_slice(&ps_pic(3, 1)); // B
es
}
/// A bounded video PES (stream_id 0xE0) carrying `es` with a 90 kHz PTS.
fn ps_video_pes(es: &[u8], pts: u64) -> Vec<u8> {
let mut pes = vec![0x00, 0x00, 0x01, 0xE0u8];
let mut body = vec![0x80u8, 0x80, 0x05]; // flags1, flags2 (PTS only), header_data_len
let p = pts & 0x1_FFFF_FFFF;
body.push(0x21 | (((p >> 30) & 0x07) << 1) as u8);
body.push(((p >> 22) & 0xFF) as u8);
body.push((0x01 | (((p >> 15) & 0x7F) << 1)) as u8);
body.push(((p >> 7) & 0xFF) as u8);
body.push((0x01 | ((p & 0x7F) << 1)) as u8);
body.extend_from_slice(es);
let len = body.len() as u16;
pes.extend_from_slice(&len.to_be_bytes());
pes.extend_from_slice(&body);
pes
}
/// Serves a fixed byte image sector-by-sector, mapping LBA to its slice.
struct ImageReader(Vec<u8>);
impl crate::sector::SectorSource for ImageReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let start = lba as usize * 2048;
let want = count as usize * 2048;
for (i, b) in buf[..want].iter_mut().enumerate() {
*b = self.0.get(start + i).copied().unwrap_or(0);
}
Ok(want)
}
fn capacity_sectors(&self) -> u32 {
(self.0.len() / 2048) as u32
}
}
fn mpeg2_video_title(sector_count: u32) -> DiscTitle {
use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream};
let mut t = synthetic_title(sector_count);
t.content_format = ContentFormat::MpegPs;
t.streams = vec![crate::disc::Stream::Video(VideoStream {
pid: 0xE0, // DVD video stream id
codec: Codec::Mpeg2,
resolution: Resolution::R480i,
frame_rate: FrameRate::F29_97,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Smpte170m,
display_aspect: Some((4, 3)),
secondary: false,
label: String::new(),
measured_cicp: None,
})];
t
}
/// The PS path (`ps_demuxer` + `Mpeg2Parser`) must forward `ps.source`
/// onto the `PesPacket` so demuxed frames carry byte provenance — the
/// second half of commit a0d4dd3 (the first half is the TS path above).
#[test]
fn ps_stream_stamps_source_provenance() {
use crate::pes::Stream;
let mut sector = ps_pack_header();
sector.extend_from_slice(&ps_video_pes(&ps_gop_es(), 0));
sector.resize(2048, 0xFF);
let mut s = DiscStream::new(
Box::new(ImageReader(sector)),
mpeg2_video_title(1),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::MpegPs,
false,
None,
)
.unwrap();
let frame = s
.read()
.unwrap()
.expect("expected a frame from the PS path");
assert!(
frame.source.is_some(),
"PS path must forward ps.source so demuxed frames carry provenance"
);
}
/// The feed base offset accumulates ACROSS read buffers: it is not reset
/// per read. With one GOP in sector 0 and another in sector 1, read
/// sector-by-sector (`batch_sectors = 1`), a frame decoded out of the
/// second sector must carry `source.byte >= 2048` — proof the running
/// `fed_bytes` base carried over from the first read rather than restarting
/// at 0. (A per-read reset would stamp both GOPs' frames below 2048.)
#[test]
fn ps_stream_carries_feed_base_across_reads() {
use crate::pes::Stream;
let mut image = ps_pack_header();
image.extend_from_slice(&ps_video_pes(&ps_gop_es(), 0));
image.resize(2048, 0xFF); // sector 0
let mut sector1 = ps_pack_header();
sector1.extend_from_slice(&ps_video_pes(&ps_gop_es(), 3003));
sector1.resize(2048, 0xFF);
image.extend_from_slice(&sector1); // sector 1
let mut s = DiscStream::new(
Box::new(ImageReader(image)),
mpeg2_video_title(2),
crate::decrypt::DecryptKeys::None,
1, // one sector per read → separate feed_at calls per sector
ContentFormat::MpegPs,
false,
None,
)
.unwrap();
let mut max_byte = 0u64;
let mut min_byte = u64::MAX;
while let Some(frame) = s.read().unwrap() {
if let Some(src) = frame.source {
max_byte = max_byte.max(src.byte);
min_byte = min_byte.min(src.byte);
}
}
assert!(
min_byte < 2048,
"a frame from the first sector must be stamped below one sector"
);
assert!(
max_byte >= 2048,
"a frame from the second sector must carry the carried-over base \
(source.byte >= 2048), not a per-read-reset offset"
);
}
// ── AdaptiveBatch ─────────────────────────────────────────────────
/// AACS decrypts whole 3-sector (6144 B) units, so every batch size at or
+72 -177
View File
@@ -6,7 +6,7 @@
//! through [`super::resolve::output`] like the other write-only sinks; the
//! ISO/disc scan that builds the title is all they need.
use crate::disc::{Chapter, DiscTitle, Stream as DiscStream};
use crate::disc::{Chapter, DiscTitle, TitleProfile};
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Write};
@@ -105,150 +105,59 @@ impl Stream for ChaptersSink {
// ── json:// ──────────────────────────────────────────────────────────────────
/// Serialization id for an audio stream's editorial purpose.
fn purpose_id(p: crate::labels::LabelPurpose) -> &'static str {
use crate::labels::LabelPurpose::*;
match p {
Normal => "normal",
Commentary => "commentary",
Descriptive => "descriptive",
Score => "score",
Ime => "ime",
}
}
/// Serialization id for a subtitle stream's qualifier.
fn qualifier_id(q: crate::labels::LabelQualifier) -> &'static str {
use crate::labels::LabelQualifier::*;
match q {
None => "none",
Sdh => "sdh",
DescriptiveService => "descriptive_service",
Forced => "forced",
}
}
/// One stream as JSON — every field the scan resolved, nothing dropped. This is
/// the complete per-stream model (`disc::Stream`), not a summary: consumers get
/// resolution/HDR/aspect for video, channels/sample-rate/purpose for audio, and
/// the qualifier for subtitles, all in machine-readable form.
fn stream_json(s: &DiscStream) -> serde_json::Value {
use super::demux_sink::codec_label;
use serde_json::json;
match s {
DiscStream::Video(v) => {
// Absent dimensions serialise as 0 here; the JSON consumer reads
// this as informational metadata, not as a mux input.
let (w, h) = v.resolution.pixels().unwrap_or((0, 0));
let (fps_num, fps_den) = v.frame_rate.as_fraction();
let mut o = json!({
"kind": "video",
"codec": codec_label(v.codec),
"pid": v.pid,
"resolution": v.resolution.to_string(),
"width": w,
"height": h,
"interlaced": v.resolution.is_interlaced(),
"frame_rate": v.frame_rate.to_string(),
"frame_rate_num": fps_num,
"frame_rate_den": fps_den,
"hdr": v.hdr.id(),
"color_space": v.color_space.id(),
"secondary": v.secondary,
"mvc_dependent": v.is_mvc_dependent(),
});
if let Some((num, den)) = v.display_aspect {
o["display_aspect"] = json!(format!("{num}:{den}"));
}
if let Some(c) = v.measured_cicp {
o["measured_cicp"] = json!({
"matrix": c.matrix,
"transfer": c.transfer,
"primaries": c.primaries,
"range": c.range,
});
}
if !v.label.is_empty() {
o["label"] = json!(v.label);
}
o
}
DiscStream::Audio(a) => {
let mut o = json!({
"kind": "audio",
"codec": codec_label(a.codec),
"pid": a.pid,
"language": a.language,
"channels": a.channels.to_string(),
"sample_rate": a.sample_rate.to_string(),
"secondary": a.secondary,
"purpose": purpose_id(a.purpose),
});
// `AudioChannels::count()` and `SampleRate::hz()` FABRICATE a concrete
// value for the `Unknown` variant (6 channels / 48000 Hz), so calling
// them unconditionally reported a confident 5.1 / 48 kHz for audio
// whose format is genuinely unknown — contradicting the neighbouring
// `channels` / `sample_rate` strings, which honestly say "unknown".
// Omit the numeric key entirely instead: the same guard `mkv.rs`
// applies before writing Channels / SamplingFrequency (there it emits
// 0 so the EBML serializer drops the element). Kept as a guard here
// rather than fixed in `count()`/`hz()` because those return
// non-optional scalars that other callers rely on.
if !matches!(a.channels, crate::disc::AudioChannels::Unknown) {
o["channel_count"] = json!(a.channels.count());
}
if !matches!(a.sample_rate, crate::disc::SampleRate::Unknown) {
o["sample_rate_hz"] = json!(a.sample_rate.hz());
}
if !a.label.is_empty() {
o["label"] = json!(a.label);
}
o
}
DiscStream::Subtitle(t) => json!({
"kind": "subtitle",
"codec": codec_label(t.codec),
"pid": t.pid,
"language": t.language,
"forced": t.forced,
"qualifier": qualifier_id(t.qualifier),
}),
}
}
/// The `json://` document for one title: identity, duration/size, its clips,
/// its complete stream models, and its chapter points. A stable, machine-
/// readable view of one title — the same information the scan resolved, no loss.
/// The `json://` document for one title.
///
/// SINGLE SERIALIZATION SOURCE OF TRUTH: the per-title / per-stream schema comes
/// from the normalized [`TitleProfile`] (the same typed view `Disc::profile`
/// exposes), serialized directly — `json://` and any downstream consumer share
/// one schema rather than a hand-built one that can drift.
///
/// SCHEMA NOTE. This replaces the previous hand-built `streams` array. The
/// stable editorial meaning is preserved, but the shape is normalized: the one
/// `streams` array (tagged with `kind`) becomes three typed arrays (`video` /
/// `audio` / `subtitles`); subtitle `qualifier` becomes the booleans `forced` +
/// `sdh`; audio `purpose` becomes `commentary` + `descriptive`; and per-track
/// `default` is now hoisted (first non-secondary video/audio). Per-stream fields
/// the normalized profile intentionally omits (`pid`, `width`/`height`,
/// `color_space`, `measured_cicp`, `sample_rate`, `channel_count`,
/// `mvc_dependent`) are no longer emitted; they live on the richer scan model,
/// not on this format-agnostic view.
///
/// `json://`-only extras that are NOT part of the normalized profile — the
/// title's `clips` and the full `chapter_marks` list — are spliced back on so
/// those consumers do not regress. (`chapters` from the profile is the marker
/// COUNT; the full list is under `chapter_marks`.) `json://` is a per-title sink
/// with no disc context, so `index` is `0` and `is_main` is `false`; the
/// disc-level [`crate::disc::DiscProfile`] populates those correctly.
pub(crate) fn title_json(title: &DiscTitle) -> serde_json::Value {
use serde_json::json;
let streams: Vec<_> = title.streams.iter().map(stream_json).collect();
let clips: Vec<_> = title
.clips
.iter()
.map(|c| {
json!({
let profile = TitleProfile::from_title(title, 0, false);
// Serializing a plain-scalar struct is infallible; fall back to an empty
// object rather than panic if that ever changes (the create() path then
// surfaces the empty doc as a NoMetadata error).
let mut doc = serde_json::to_value(&profile).unwrap_or_else(|_| json!({}));
doc["format"] = json!(format!("{:?}", title.content_format));
doc["playlist_id"] = json!(title.playlist_id);
doc["clips"] = json!(
title
.clips
.iter()
.map(|c| json!({
"clip_id": c.clip_id,
"duration_secs": c.duration_secs,
"source_packets": c.source_packets,
})
})
.collect();
let chapters: Vec<_> = title
.chapters
.iter()
.enumerate()
.map(|(i, c)| json!({ "n": i + 1, "start_secs": c.time_secs, "name": c.name }))
.collect();
json!({
"playlist": title.playlist,
"playlist_id": title.playlist_id,
"duration_secs": title.duration_secs,
"size_bytes": title.size_bytes,
"format": format!("{:?}", title.content_format),
"clips": clips,
"streams": streams,
"chapters": chapters,
})
}))
.collect::<Vec<_>>()
);
doc["chapter_marks"] = json!(
title
.chapters
.iter()
.enumerate()
.map(|(i, c)| json!({ "n": i + 1, "start_secs": c.time_secs, "name": c.name }))
.collect::<Vec<_>>()
);
doc
}
/// `json://` sink: writes the title's structured metadata at construction; the
@@ -347,28 +256,28 @@ mod tests {
})];
let v = title_json(&t);
assert_eq!(v["playlist"], "MAIN");
let a = &v["streams"][0];
assert_eq!(a["kind"], "audio");
assert_eq!(a["codec"], "TrueHD");
// Normalized profile schema: streams split into typed arrays.
let a = &v["audio"][0];
assert_eq!(a["codec"], "truehd");
assert_eq!(a["language"], "eng");
// Completeness: audio carries channels + sample rate + purpose, not just codec.
assert_eq!(a["channels"], "stereo");
assert_eq!(a["channel_count"], 2);
assert_eq!(a["sample_rate"], "48kHz");
assert_eq!(a["sample_rate_hz"], 48000.0);
assert_eq!(a["purpose"], "normal");
assert_eq!(v["chapters"][1]["n"], 2);
assert_eq!(v["chapters"][1]["start_secs"], 62.5);
assert_eq!(v["chapters"][1]["name"], "2");
// Editorial purpose is decomposed into booleans.
assert!(!a["commentary"].as_bool().unwrap());
assert!(!a["descriptive"].as_bool().unwrap());
// First non-secondary audio is the hoisted default.
assert!(a["default"].as_bool().unwrap());
// Chapter COUNT from the profile; full list under chapter_marks.
assert_eq!(v["chapters"], 2);
assert_eq!(v["chapter_marks"][1]["n"], 2);
assert_eq!(v["chapter_marks"][1]["start_secs"], 62.5);
assert_eq!(v["chapter_marks"][1]["name"], "2");
}
/// An audio stream whose channel layout / sample rate are genuinely unknown
/// must not be reported with a fabricated 5.1 / 48 kHz. `AudioChannels::count()`
/// maps `Unknown` to 6 and `SampleRate::hz()` maps `Unknown` to 48000, so the
/// numeric fields must be omitted rather than computed — otherwise the JSON
/// contradicts its own `channels` / `sample_rate` strings ("unknown").
/// An audio stream whose channel layout is genuinely unknown reports the
/// honest `"unknown"` string (the normalized profile carries the layout label,
/// not a fabricated numeric channel count).
#[test]
fn unknown_audio_layout_omits_fabricated_numeric_fields() {
fn unknown_audio_layout_reports_unknown_channels() {
use crate::disc::{AudioChannels, AudioStream, Codec, DiscTitle};
use crate::disc::{LabelPurpose, SampleRate, Stream as DiscStream};
let mut t = DiscTitle::empty();
@@ -383,21 +292,10 @@ mod tests {
label: String::new(),
})];
let v = title_json(&t);
let a = &v["streams"][0];
// The honest string fields.
let a = &v["audio"][0];
assert_eq!(a["channels"], "unknown");
assert_eq!(a["sample_rate"], "unknown");
// The numeric fields must not assert a value the scan never resolved.
assert!(
a["channel_count"].is_null(),
"unknown channel layout must not report a channel_count, got {}",
a["channel_count"]
);
assert!(
a["sample_rate_hz"].is_null(),
"unknown sample rate must not report a sample_rate_hz, got {}",
a["sample_rate_hz"]
);
// No fabricated numeric channel count is emitted in the profile schema.
assert!(a["channel_count"].is_null());
}
#[test]
@@ -420,15 +318,12 @@ mod tests {
label: String::new(),
measured_cicp: None,
})];
let vid = &title_json(&t)["streams"][0];
assert_eq!(vid["kind"], "video");
let vid = &title_json(&t)["video"][0];
assert_eq!(vid["codec"], "hevc");
assert_eq!(vid["resolution"], "2160p");
assert_eq!(vid["width"], 3840);
assert_eq!(vid["height"], 2160);
assert_eq!(vid["frame_rate"], "23.976");
assert_eq!(vid["frame_rate_num"], 24000);
assert_eq!(vid["hdr"], "hdr10");
assert_eq!(vid["color_space"], "bt2020");
assert!(vid["default"].as_bool().unwrap());
}
fn temp_path(name: &str) -> std::path::PathBuf {
+302 -7
View File
@@ -359,6 +359,22 @@ fn write_hdr10<W: Write + Seek>(w: &mut W, h: &crate::mux::codec::Hdr10Metadata)
Ok(())
}
/// The Matroska `Language` element the muxer writes for a stream whose source
/// reported `lang`. RFC 9559 §12 defines the element as an ISO 639-2 code and
/// gives no meaning to an empty one; the code for "no language stated" is
/// `und`, and that is what a source with no language table (the HD-DVD EVO
/// stream probe, a Blu-ray STN slot with no language bytes) has to emit. The
/// element is written unconditionally by `MkvMuxer::new`, so this is the one
/// place that decides it — a source-side default would have to be repeated in
/// every scanner and would still leave the muxer able to ship an invalid file.
fn language_or_und(lang: &str) -> String {
if lang.is_empty() {
"und".to_string()
} else {
lang.to_string()
}
}
impl MkvTrack {
/// Build a video track from a [`VideoStream`]. Language defaults to `"und"`;
/// colour metadata is derived from the stream's colour space and HDR format
@@ -440,7 +456,7 @@ impl MkvTrack {
//
// rc.5.1 added it (= half the frame period, 20 ms for 576i25) to try
// to fix the Windows-fps report, on the theory that Windows derives
// fps from it. The captured SOTL evidence proves the opposite: with
// fps from it. The captured real-media evidence proves the opposite: with
// FlagInterlaced=1 + DefaultDuration=40 ms + DefaultDecodedFieldDuration=20 ms,
// Windows Explorer reports 12.5 fps (half), and a media analyzer
// flips the track to "Frame rate mode: Variable" with no clean rate. A
@@ -530,7 +546,7 @@ impl MkvTrack {
Self {
track_type: ebml::TRACK_TYPE_AUDIO,
codec_id,
language: a.language.clone(),
language: language_or_und(&a.language),
name,
codec_private: None,
is_default: !a.secondary,
@@ -587,7 +603,7 @@ impl MkvTrack {
Self {
track_type: ebml::TRACK_TYPE_SUBTITLE,
codec_id,
language: s.language.clone(),
language: language_or_und(&s.language),
name: String::new(),
codec_private: s.codec_data.clone(),
is_default: false,
@@ -1095,7 +1111,7 @@ impl<W: Write + Seek> MkvMuxer<W> {
// `field_duration_ns == 0` (see `MkvTrack::video`) so this element is
// NOT written: emitting it (20 ms for 576i25) is exactly what made
// Windows Explorer report 12.5 fps and a media analyzer flip to VFR on
// the captured SOTL rip, while a known-correct rip — which omits it —
// the captured rip, while a known-correct rip — which omits it —
// shows the full 25 fps. The guard below is retained so a non-zero
// value still emits
// a well-formed element for any future caller / round-trip test, but
@@ -3307,6 +3323,285 @@ mod tests {
);
}
/// Read back the value of the FIRST `Language` element in `data` as a
/// UTF-8 string. `LANGUAGE` (0x22B59C) is a 3-byte EBML ID; its size is
/// always a 1-byte VINT for the short strings this writer emits.
fn first_language_value(data: &[u8]) -> &str {
let pos = find_id(data, ebml::LANGUAGE).expect("Language element must be present");
let len = (data[pos + 3] & 0x7F) as usize;
std::str::from_utf8(&data[pos + 4..pos + 4 + len]).unwrap()
}
/// RFC 9559 §12 / the Matroska `Language` element spec restrict the
/// legacy `Language` element to the Matroska language form (ISO 639-2,
/// three lowercase letters), never ISO 639-1 (two letters). The DVD
/// IFO audio-attribute block itself carries a raw ISO 639-1 code (e.g.
/// "en") on disc — `ifo::parse_audio_attr` converts it to ISO 639-2
/// before returning, via `ifo::dvd_lang_to_iso639_2`. This mimics the
/// real DVD pipeline (`disc/dvd.rs`'s `Stream::Audio` construction) end
/// to end: real on-disc IFO bytes -> `ifo::parse_audio_attr` ->
/// `disc::AudioStream` -> `MkvTrack::audio` -> the muxer -> the emitted
/// `Language` element.
#[test]
fn dvd_two_letter_language_becomes_iso_639_2_in_language_element() {
// AC-3 (coding_mode=0), 48 kHz, 6 channels, on-disc language "en" —
// the exact byte layout `ifo::audio_attr_parsing` pins.
let mut attr_bytes = vec![0u8; 8];
attr_bytes[0] = 0x00;
attr_bytes[1] = 0x05;
attr_bytes[2] = b'e';
attr_bytes[3] = b'n';
let attr = crate::ifo::parse_audio_attr(&attr_bytes, 0).unwrap();
assert_eq!(
attr.language, "eng",
"parse_audio_attr must already return the ISO 639-2 form"
);
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"eng",
"a DVD-sourced ISO 639-1 code must be written as its ISO 639-2 \
equivalent in the Matroska Language element, per RFC 9559 §12"
);
}
/// A source that knows no language at all leaves `language` EMPTY, and the
/// muxer writes the `Language` element unconditionally — so an empty
/// string becomes a zero-length `Language` in the shipped file, which is
/// not a Matroska language form (RFC 9559 §12 wants three ISO 639-2
/// letters) and is not the ISO 639-2 code for "unknown" either.
///
/// This is the state every HD-DVD rip is in: `disc::hddvd`'s EVO stream
/// probe has no language table to read and sets `language: String::new()`
/// on every audio stream it finds. The DVD path normalises to "und" in
/// `ifo::parse_audio_attr`; the guard has to exist at the muxer too, which
/// is the one place every source funnels through.
#[test]
fn a_source_with_no_language_emits_und_not_an_empty_language_element() {
// Exactly what `disc::hddvd::probe_evo_streams` builds.
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: Codec::Ac3Plus,
channels: crate::disc::AudioChannels::Surround51,
language: String::new(),
sample_rate: crate::disc::SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let subtitle_stream = crate::disc::SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: String::new(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
};
for track in [
MkvTrack::audio(&audio_stream),
MkvTrack::subtitle(&subtitle_stream),
] {
let track_type = track.track_type;
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"und",
"track type {track_type}: a stream with no known language must \
emit the ISO 639-2 'undetermined' code, never a zero-length \
Language element"
);
}
}
/// An unmapped or absent DVD language code (bytes 0x00 0x00 in the IFO
/// attribute block) must degrade to the valid Matroska "undetermined"
/// code `und`, never to an empty string or a raw 2-letter code — both of
/// which violate the Matroska language form.
#[test]
fn dvd_unmapped_or_empty_language_becomes_und_in_language_element() {
// Empty IFO language bytes (0x00 0x00).
let mut empty_bytes = vec![0u8; 8];
empty_bytes[0] = 0x00; // AC-3, 48k
empty_bytes[1] = 0x05; // 6ch
let empty_attr = crate::ifo::parse_audio_attr(&empty_bytes, 0).unwrap();
assert_eq!(
empty_attr.language, "und",
"IFO zero bytes (unspecified) must resolve to 'und', not empty"
);
// An IFO code with no known ISO 639-1 -> 639-2 mapping (e.g. a
// fictitious "zz").
let mut unmapped_bytes = vec![0u8; 8];
unmapped_bytes[0] = 0x00;
unmapped_bytes[1] = 0x05;
unmapped_bytes[2] = b'z';
unmapped_bytes[3] = b'z';
let unmapped_attr = crate::ifo::parse_audio_attr(&unmapped_bytes, 0).unwrap();
assert_eq!(
unmapped_attr.language, "und",
"an on-disc code with no known ISO 639-1 -> 639-2 mapping must \
degrade to 'und', never pass through raw"
);
for attr in [empty_attr, unmapped_attr] {
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"und",
"an empty or unmapped DVD language code must degrade to 'und', \
never an empty string or an invalid ISO 639-1 code"
);
}
}
/// Build an 8-byte DVD IFO audio-attribute block (AC-3, 48 kHz, 6ch)
/// carrying `code` in the language bytes, run it through the real DVD
/// pipeline (`ifo::parse_audio_attr` -> `disc::AudioStream` ->
/// `MkvTrack::audio` -> the muxer) and return the value that actually
/// lands in the emitted Matroska `Language` element.
fn emitted_language_for_dvd_code(code: &[u8; 2]) -> String {
let mut attr_bytes = vec![0u8; 8];
attr_bytes[0] = 0x00; // AC-3, 48 kHz
attr_bytes[1] = 0x05; // 6 channels
attr_bytes[2] = code[0];
attr_bytes[3] = code[1];
let attr = crate::ifo::parse_audio_attr(&attr_bytes, 0).unwrap();
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
first_language_value(&data).to_string()
}
/// The ISO 639-1 -> ISO 639-2 conversion must cover the WHOLE of ISO
/// 639-1, not just the handful of languages that happen to appear in
/// Blu-ray menu-graphic filenames. A Region-2 disc routinely carries
/// Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian, Hebrew,
/// Estonian, Latvian, Lithuanian, Icelandic and so on; if those all
/// collapse to `und`, every one of a disc's subtitle tracks emits the
/// same `Language` value and nothing else tells them apart (DVD streams
/// carry an empty `label`). A valid-but-identical code is worse for the
/// user than the invalid one it replaced, so each of these must reach the
/// emitted `Language` element as its own correct three-letter code.
#[test]
fn dvd_language_outside_the_menu_vocabulary_is_still_mapped() {
assert_eq!(
emitted_language_for_dvd_code(b"ro"),
"ron",
"Romanian ('ro'), common on Region-2 discs, must reach the \
Matroska Language element as 'ron' — not 'und'"
);
// The rest of the set the menu-label table never knew, one per
// language so a single missing table row fails loudly.
for (code, expected) in [
(b"bg", "bul"),
(b"hr", "hrv"),
(b"sr", "srp"),
(b"sk", "slk"),
(b"sl", "slv"),
(b"he", "heb"),
(b"et", "est"),
(b"lv", "lav"),
(b"lt", "lit"),
(b"is", "isl"),
(b"id", "ind"),
(b"vi", "vie"),
(b"fa", "fas"),
] {
assert_eq!(
emitted_language_for_dvd_code(code),
expected,
"DVD language {:?} must map to {expected:?} in the emitted \
Language element",
std::str::from_utf8(code).unwrap()
);
}
}
/// DVD-Video froze its language list on the 1988 edition of ISO 639-1,
/// which spelled Hebrew `iw`, Indonesian `in` and Yiddish `ji`. Real
/// discs authored to that list carry those bytes, so they must map to the
/// same ISO 639-2 codes as the modern `he` / `id` / `yi` spellings rather
/// than degrading to `und`.
#[test]
fn dvd_era_language_aliases_map_to_the_modern_code() {
assert_eq!(
emitted_language_for_dvd_code(b"iw"),
"heb",
"the DVD-era spelling of Hebrew ('iw') must emit 'heb'"
);
assert_eq!(emitted_language_for_dvd_code(b"in"), "ind");
assert_eq!(emitted_language_for_dvd_code(b"ji"), "yid");
// ...and agree with the modern spellings.
assert_eq!(emitted_language_for_dvd_code(b"he"), "heb");
assert_eq!(emitted_language_for_dvd_code(b"id"), "ind");
assert_eq!(emitted_language_for_dvd_code(b"yi"), "yid");
}
/// Widening the table must not weaken the degradation guarantee: a code
/// that is not ISO 639-1 at all still has to yield exactly `und`, a valid
/// Matroska language value, and never a passed-through two-letter code,
/// an empty string, or a guess.
#[test]
fn unknown_dvd_language_still_yields_exactly_und() {
for code in [b"zz", b"qq", b"xx"] {
assert_eq!(
emitted_language_for_dvd_code(code),
"und",
"a code outside ISO 639-1 must degrade to exactly 'und'"
);
}
// Empty language bytes (0x00 0x00) likewise.
assert_eq!(emitted_language_for_dvd_code(&[0x00, 0x00]), "und");
}
#[test]
fn mkv_forced_flag_on_forced_subtitle() {
use crate::disc::SubtitleStream;
@@ -4147,7 +4442,7 @@ mod tests {
#[test]
fn opening_keyframe_with_nonzero_disc_pts_anchors_base_not_corrupted() {
// SOTL SUB-TASK 2 regression (opening-GOP PTS handling). A DVD title
// Opening-GOP PTS handling regression. A DVD title
// opens on an I-frame the disc stamps at its REAL timeline PTS (here
// ~10 s, a large non-zero value — NOT 0). The muxer must anchor `base` on
// that first kept keyframe so the first cluster's timestamp is 0 (the
@@ -5261,11 +5556,11 @@ mod tests {
#[test]
fn interlaced_576i_omits_default_decoded_field_duration_keeps_full_frame_duration() {
// SOTL SUB-TASK 1 regression. The Windows-fps fix: a 576i25 track must
// Interlaced-fps signalling regression. The Windows-fps fix: a 576i25 track must
// carry the FULL-FRAME DefaultDuration (40 ms → `1/DefaultDuration` = 25
// fps, the only rate every tool trusts) and must NOT emit
// DefaultDecodedFieldDuration. rc.5.1 emitted the 20 ms field duration to
// try to fix Windows; the captured SOTL evidence proved it did the
// try to fix Windows; the captured real-media evidence proved it did the
// opposite (Explorer 12.5 fps, analyzer VFR). A known-correct rip omits
// it (Explorer 25 fps, analyzer CFR). So: frame duration present = 40 ms,
// field duration ABSENT, interlace signalling (FlagInterlaced/FieldOrder)
+84 -1
View File
@@ -12,8 +12,19 @@ use super::{WriteSeek, ebml};
type MkvHeaderResult = io::Result<(crate::disc::DiscTitle, Vec<(u16, Vec<u8>)>, i64, TrackTable)>;
/// Skip `n` bytes on a forward-only reader (no Seek required).
///
/// A skip that runs out of input before `n` bytes is a TRUNCATED element, and is
/// reported the same way `ebml::read_binary_val` reports a truncated body: as
/// `MkvSourceInvalid`. Discarding `io::copy`'s byte count instead made a skip
/// that hit EOF look like a success, so one corrupt size field mid-Clusters
/// drained the rest of the file, the next element header raised
/// `UnexpectedEof`, and `Stream::read` mapped that to `Ok(None)` — half the
/// title missing, `errors = 0`, `completed = true`.
fn skip_bytes(r: &mut impl Read, n: u64) -> io::Result<()> {
io::copy(&mut r.take(n), &mut io::sink())?;
let skipped = io::copy(&mut r.take(n), &mut io::sink())?;
if skipped != n {
return Err(crate::error::Error::MkvSourceInvalid.into());
}
Ok(())
}
@@ -3193,6 +3204,78 @@ mod tests {
assert!(stream.read().unwrap().is_none(), "clean EOF → None");
}
/// A skipped element whose declared size runs PAST the end of the file is a
/// truncated element, exactly like a truncated `read_binary_val` body — and
/// must be reported the same way, as `MkvSourceInvalid`.
///
/// `skip_bytes` used to discard `io::copy`'s returned count, so the skip
/// "succeeded" having drained the rest of the file. The next element header
/// then hit `UnexpectedEof`, which `read()` maps to `Ok(None)` — a clean end
/// of stream. One corrupt size field mid-Clusters therefore threw away every
/// remaining frame of the title and reported `errors = 0`, `complete = true`.
#[test]
fn a_skip_past_eof_is_an_error_not_a_clean_end_of_stream() {
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
// Frame 1 — read normally.
let block = [0x81u8, 0x00, 0x00, 0x80, 0xAA];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block.len() as u64).unwrap();
cluster.extend_from_slice(&block);
// A VOID whose size field is corrupt: it claims 1 MiB, and the file
// holds only the handful of bytes below. This is the "corrupt size
// field mid-Clusters" case.
ebml::write_id(&mut cluster, ebml::VOID).unwrap();
ebml::write_size(&mut cluster, 1024 * 1024).unwrap();
// Frame 2 — the rest of the title, swallowed by the bad skip.
let block2 = [0x81u8, 0x00, 0x01, 0x80, 0xBB];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block2.len() as u64).unwrap();
cluster.extend_from_slice(&block2);
let bytes = mkv_with_track_and_cluster(1, 1, &cluster);
let mut stream = MkvStream::open(Cursor::new(bytes)).unwrap();
assert!(stream.read().unwrap().is_some(), "first frame reads");
let e = match stream.read() {
Err(e) => e,
Ok(None) => panic!(
"a skip that hit EOF was reported as a CLEAN END OF STREAM: the \
rest of the title is gone and the caller sees errors = 0, \
complete = true"
),
Ok(Some(_)) => panic!("the truncated skip must not yield a frame"),
};
assert!(is_mkv_source_invalid(&e), "{e:?}");
}
/// The honest path this fix must not break: a skipped element whose declared
/// size is exactly satisfied by the bytes present is still skipped cleanly,
/// and the genuine EOF that follows is still `Ok(None)`.
#[test]
fn a_fully_satisfied_skip_still_ends_at_a_clean_eof() {
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
// A VOID that is fully present.
ebml::write_id(&mut cluster, ebml::VOID).unwrap();
ebml::write_size(&mut cluster, 8).unwrap();
cluster.extend_from_slice(&[0u8; 8]);
let block = [0x81u8, 0x00, 0x00, 0x80, 0xAA];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block.len() as u64).unwrap();
cluster.extend_from_slice(&block);
let bytes = mkv_with_track_and_cluster(1, 1, &cluster);
let mut stream = MkvStream::open(Cursor::new(bytes)).unwrap();
let f = stream.read().unwrap().expect("the frame after the VOID");
assert_eq!(f.data, vec![0xAA]);
assert!(
stream.read().unwrap().is_none(),
"a genuine EOF at a record boundary is still a clean end"
);
}
// ============================================================
// Block LACING (RFC 9559 §10.3) and TrackNumber→stream routing
// (RFC 9559 §5.1.4.1.1).
+22 -2
View File
@@ -627,16 +627,36 @@ mod tests {
/// accept_from() must reject a connection whose first bytes are NOT the
/// FMKV magic — there is no metadata to drive muxing, so it surfaces
/// NoMetadata rather than proceeding with an empty/garbage title.
///
/// FLAKE FIXED, not the behaviour under test: this used to
/// `shutdown(Shutdown::Both)` the instant the bytes were written. Closing
/// the READ half while the server had not yet read makes the kernel answer
/// the server's in-flight data with an RST, so `accept_from` came back
/// `ConnectionReset` instead of the `InvalidInput` this asserts — rarely
/// when run alone, reproducibly under the loaded concurrent suite, where
/// the server thread is descheduled long enough for the race to open. A
/// logging/protocol assertion that fails at random teaches the next person
/// to re-run until green, which is how a real regression gets waved
/// through.
///
/// Half-closing (`Shutdown::Write`) delivers the same EOF the test needs
/// while leaving the read half open, and blocking on a read until the
/// server drops its end keeps the socket alive for as long as the server
/// is looking at it. The port was already ephemeral (`:0`), so it was
/// never a port collision.
#[test]
fn accept_from_rejects_stream_without_fmkv_header() {
use std::io::Read as _;
use std::io::Write as _;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
let handle = std::thread::spawn(move || {
// Raw non-FMKV bytes (not starting with 'F') then close.
// Raw non-FMKV bytes (not starting with 'F') then EOF.
let mut s = TcpStream::connect(addr).unwrap();
s.write_all(&[0x47u8; 64]).unwrap(); // TS sync bytes, no FMKV magic
s.shutdown(std::net::Shutdown::Both).unwrap();
s.shutdown(std::net::Shutdown::Write).unwrap();
// Park until the server closes, so no RST can overtake the data.
let _ = s.read(&mut [0u8; 1]);
});
let err = match NetworkStream::accept_from(listener) {
Ok(_) => panic!("missing FMKV header must error, not silently accept"),
+261 -17
View File
@@ -17,7 +17,7 @@ use super::codec::startcode::find_start_code;
const PACK_HEADER_ID: u8 = 0xBA;
/// System header start code suffix.
const SYSTEM_HEADER_ID: u8 = 0xBB;
const SYSTEM_HEADER_ID: u8 = crate::consts::pes_stream_id::SYSTEM_HEADER;
/// Program end start code suffix.
const PROGRAM_END_ID: u8 = 0xB9;
@@ -29,7 +29,7 @@ const PRIVATE_STREAM_1: u8 = crate::consts::pes_stream_id::PRIVATE_STREAM_1;
const PRIVATE_STREAM_2: u8 = crate::consts::pes_stream_id::PRIVATE_STREAM_2;
/// Extended stream id (0xFD) — the H.222.0 escape whereby the real stream id is
/// the `stream_id_extension` carried in the PES extension. HD-DVD `.evo` puts its
/// VC-1 video (and HD audio) here (Shaun of the Dead: VC-1 on `0xFD` ext `0x55`);
/// VC-1 video (and HD audio) here (a real HD-DVD title: VC-1 on `0xFD` ext `0x55`);
/// a transport stream never uses it. The elementary-stream bytes follow the PES
/// header exactly like any other PES — only the routing key differs.
const EXTENDED_STREAM_ID: u8 = 0xFD;
@@ -169,6 +169,24 @@ pub struct PsDemuxer {
/// stay byte-identical.
buffer_base: u64,
has_base: bool,
/// Boundary-scan cursor for an unbounded (length-0) PES still waiting for
/// its terminating PS-layer unit: `(buffer offset of the PES start code,
/// buffer offset up to which the search has already proved there is no
/// boundary)`. Both are buffer-relative and are rebased when the buffer
/// drains.
///
/// Without it, every `feed` re-searches the WHOLE accumulated payload from
/// the PES header: the buffer only stops growing at [`MAX_PS_BUFFER`], so a
/// stream that declares an unbounded PES and then never emits a PS-layer
/// start code (a corrupt or crafted VOB) makes the demuxer scan up to 4 MiB
/// per call, quadratic in the bytes fed. Cleared whenever the PES is
/// emitted, so it can never outlive the packet it describes.
pending_scan: Option<(usize, usize)>,
/// Test-only: total bytes examined by `find_ps_boundary`. Pins the cursor
/// above — the property it exists for is a WORK bound, which no
/// packet-level assertion can observe.
#[cfg(test)]
boundary_bytes_scanned: u64,
}
impl Default for PsDemuxer {
@@ -184,6 +202,9 @@ impl PsDemuxer {
buffer: Vec::with_capacity(64 * 1024),
buffer_base: 0,
has_base: false,
pending_scan: None,
#[cfg(test)]
boundary_bytes_scanned: 0,
}
}
@@ -196,9 +217,7 @@ impl PsDemuxer {
/// Like [`feed`](Self::feed) but records the absolute source byte offset of
/// `data[0]`, so every PES this call completes is stamped with a
/// [`crate::pes::SourcePos`]. The provenance-stamping entry point; the
/// highway calls this with each batch's known source offset. The base must
/// be the offset of the FIRST byte appended (i.e. of `data[0]`), which lines
/// up with the current buffer tail.
/// highway calls this with each batch's known source offset.
pub fn feed_at(&mut self, base_offset: u64, data: &[u8]) -> Vec<PsPacket> {
if !self.has_base {
// First base seen: the offset of data[0] is base_offset, and data[0]
@@ -220,6 +239,8 @@ impl PsDemuxer {
// discarded.
let packets = self.extract_packets(true);
self.buffer.clear();
// The buffer the cursor indexes into is gone.
self.pending_scan = None;
packets
}
@@ -288,11 +309,29 @@ impl PsDemuxer {
// start code — the video ES payload is itself full of
// 00 00 01 xx codes that would otherwise cut the PES short.
let end = if pes_packet_len == 0 {
match find_ps_boundary(&self.buffer, sc + 4) {
Some(next) => next,
// Resume where the last call stopped searching for
// THIS PES's terminating unit; anything before that is
// already proved boundary-free.
let from = match self.pending_scan {
Some((pes_at, searched_to)) if pes_at == sc => searched_to,
_ => sc + 4,
};
let (found, searched_to) = find_ps_boundary(&self.buffer, from);
#[cfg(test)]
{
self.boundary_bytes_scanned += searched_to.saturating_sub(from) as u64;
}
match found {
Some(next) => {
self.pending_scan = None;
next
}
// At EOF the rest of the buffer is this PES's
// payload — emit it.
None if flushing => self.buffer.len(),
None if flushing => {
self.pending_scan = None;
self.buffer.len()
}
None => {
// No boundary buffered yet. Normally wait for
// more data, but a corrupt stream could declare
@@ -301,8 +340,10 @@ impl PsDemuxer {
// stops untrusted input forcing unbounded
// allocation. Past the cap, flush what we have.
if self.buffer.len() - sc > MAX_PS_BUFFER {
self.pending_scan = None;
self.buffer.len()
} else {
self.pending_scan = Some((sc, searched_to));
break; // wait for more data
}
}
@@ -338,6 +379,13 @@ impl PsDemuxer {
if self.has_base {
self.buffer_base += pos as u64;
}
// The cursor is a BUFFER offset, so it moves with the drain. A
// pending PES always starts at or after `pos` (the loop broke on
// it, having already consumed everything before it), so neither
// component can underflow.
self.pending_scan = self
.pending_scan
.map(|(pes_at, searched_to)| (pes_at - pos, searched_to - pos));
}
// Trim a start-code-free tail. Every other exit from the loop above
@@ -359,6 +407,10 @@ impl PsDemuxer {
if self.has_base {
self.buffer_base += drop as u64;
}
// A pending PES implies a start code IS in the buffer, so this
// branch cannot run while one is open; drop the cursor anyway
// rather than leave a stale offset behind this drain.
self.pending_scan = None;
}
packets
@@ -380,11 +432,19 @@ const START_CODE_PREFIX_KEEP: usize = 2;
/// PES inside its own payload and re-scan the discarded video bytes as bogus PS
/// units. Restricting the search to PS-layer IDs (>= 0xB9, excluding the video
/// ES codes below it) frames the unbounded PES at the right boundary.
fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
/// Returns `(boundary, searched_to)`. `searched_to` is the offset up to which
/// every byte has been PROVED not to begin a PS-layer boundary start code, so
/// a later call over the same buffer (grown at the tail) may resume there
/// instead of re-scanning the payload from the PES header. When the scan runs
/// off the end, the last two bytes are NOT proved: a `00 00 01` prefix can
/// straddle the next feed's boundary by up to two bytes.
fn find_ps_boundary(data: &[u8], from: usize) -> (Option<usize>, usize) {
let mut pos = from;
while let Some(sc) = find_start_code(data, pos) {
if sc + 3 >= data.len() {
return None;
// A start code whose ID byte has not arrived yet: undecided, so
// the next scan must look at it again.
return (None, sc);
}
let id = data[sc + 3];
if id == PACK_HEADER_ID
@@ -392,11 +452,11 @@ fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
|| id == PROGRAM_END_ID
|| is_pes_stream_id(id)
{
return Some(sc);
return (Some(sc), sc);
}
pos = sc + 4;
}
None
(None, data.len().saturating_sub(2).max(from))
}
/// Check whether a start code byte is a valid PES stream ID that carries payload.
@@ -551,8 +611,8 @@ fn parse_pes_packet(data: &[u8]) -> Option<PsPacket> {
0x80..=0x8F => 4, // AC3/DTS: sub_id + frame_count + access_unit_ptr(2)
// HD-DVD Dolby Digital Plus (E-AC-3): the sub-header is the same
// 4-byte shape as DVD AC-3 — sub_id + number_of_frames(1) +
// first_access_unit_pointer(2). Verified empirically on ANCHORMAN
// EVO: across every 0xC0..=0xC7 packet the 0x0B77 E-AC-3 syncword
// first_access_unit_pointer(2). Verified empirically on a real
// HD-DVD EVO: across every 0xC0..=0xC7 packet the 0x0B77 E-AC-3 syncword
// sits `first_access_unit_pointer` bytes past this 4-byte header
// (the leading bytes are the tail of the previous frame). Stripping
// exactly these 4 bytes on EVERY packet yields a clean, continuous
@@ -1521,7 +1581,7 @@ mod tests {
// Just outside the range.
assert_eq!(dvd_audio_pid(0xBF), None);
assert_eq!(dvd_audio_pid(0xC8), None);
// Four DD+ tracks (ANCHORMAN) get four distinct PIDs.
// Four DD+ tracks (as seen on a real disc) get four distinct PIDs.
let pids: Vec<u16> = (0xC0u8..=0xC3).map(|s| dvd_audio_pid(s).unwrap()).collect();
assert_eq!(pids, vec![0xBDC0, 0xBDC1, 0xBDC2, 0xBDC3]);
// And route through dvd_pid on a private_stream_1 packet.
@@ -1752,7 +1812,191 @@ mod tests {
/// after it — exactly the tail a real feed can end on.
#[test]
fn find_ps_boundary_handles_a_bare_start_code_at_the_buffer_head() {
assert_eq!(find_ps_boundary(&[0x00, 0x00, 0x01], 0), None);
assert_eq!(
find_ps_boundary(&[0x00, 0x00, 0x01], 0),
(None, 0),
"an undecided trailing start code is not proved boundary-free"
);
}
/// An unbounded (length-0) PES is terminated by the next PS-LAYER unit, and
/// until one arrives the payload accumulates in the buffer. The search for
/// that unit must not restart at the PES header on every feed: the buffer
/// only stops growing at `MAX_PS_BUFFER` (4 MiB), and a feed is one read
/// batch (at most 510 sectors ≈ 1 MiB, 60 sectors ≈ 120 KiB on an optical
/// drive), so re-scanning from byte 0 costs work quadratic in the bytes
/// fed — up to 4 MiB of scanning per call, for as long as a corrupt or
/// crafted VOB withholds the boundary. A conformant DVD ends every pack
/// within 2048 bytes and never reaches this state.
///
/// Measured directly, because a work bound has no packet-level shadow:
/// `boundary_bytes_scanned` counts the bytes `find_ps_boundary` examines.
/// 256 chunks x 4 KiB of boundary-free payload is 1 MiB of input;
/// re-scanning from the header on every call examines
/// 4 KiB * 256*257/2 = ~128 MiB.
///
/// Mutation: drop the `Some((pes_at, searched_to)) if pes_at == sc` arm so
/// `from` is always `sc + 4`.
#[test]
fn an_unterminated_pes_is_not_rescanned_from_its_header_every_feed() {
const CHUNKS: usize = 256;
const CHUNK: usize = 4096;
let mut demuxer = PsDemuxer::new();
// Unbounded PES header (length 0), then payload that carries no start
// code at all, so no PS-layer boundary is ever found.
demuxer.feed(&[0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00]);
for _ in 0..CHUNKS {
assert!(
demuxer.feed(&[0xFFu8; CHUNK]).is_empty(),
"no boundary yet, so no PES can be emitted"
);
}
let fed = (CHUNKS * CHUNK) as u64;
assert!(
demuxer.boundary_bytes_scanned <= 2 * fed,
"boundary search examined {} bytes over {fed} bytes of payload — \
the scan must advance with the buffer, not restart at the PES header",
demuxer.boundary_bytes_scanned
);
// ...and the cursor must not have cost correctness: the PES still ends
// at the pack header that finally arrives, with its whole payload.
let pack = [
0x00,
0x00,
0x01,
PACK_HEADER_ID,
0x44,
0x00,
0x04,
0x00,
0x04,
0x01,
0x00,
0x00,
0x03,
0xF8,
];
let packets = demuxer.feed(&pack);
assert_eq!(packets.len(), 1, "the pack header terminates the PES");
assert_eq!(
packets[0].data.len(),
CHUNKS * CHUNK,
"the whole accumulated payload belongs to the PES"
);
}
/// The resume cursor is a BUFFER offset, so it must be rebased when the
/// buffer drains — and this is the only test in which a drain actually
/// happens while a cursor is live.
///
/// Mutations this catches, both halves of
/// `self.pending_scan.map(|(pes_at, searched_to)| (pes_at - pos, searched_to - pos))`
/// in `extract_packets`:
/// * `searched_to - pos` -> `searched_to`: the next call resumes `pos`
/// bytes PAST where the previous scan actually stopped, so that window
/// is never examined. Here the terminating pack header lands inside it
/// and is missed outright — the unbounded PES runs on past its real
/// end, swallowing the following unit. That is a CORRECTNESS failure,
/// not a slow path, and the assertion on the emitted packet catches it.
/// * `pes_at - pos` -> `pes_at`: the stale offset no longer equals the
/// PES's post-drain `sc`, the resume arm stops matching and the search
/// restarts at the PES header. Caught by `boundary_bytes_scanned`,
/// which is why the fixture puts 64 KiB of payload in the SAME chunk
/// that opens the PES: that is exactly the span a restart re-examines,
/// so the mutant roughly doubles the bytes scanned.
///
/// `an_unterminated_pes_is_not_rescanned_from_its_header_every_feed` cannot
/// reach either: it opens the unbounded PES as the very FIRST bytes of the
/// very first feed, so nothing ever drains ahead of it, `pos` stays 0 and
/// the subtraction is a no-op. The comment above it asserts neither
/// component can underflow; nothing exercised the arithmetic at all.
///
/// So this fixture puts COMPLETE PS units — a pack header and a
/// length-bounded PES — ahead of the unbounded video PES *in the same
/// chunk*. The loop consumes them, breaks on the unbounded PES, and drains
/// `pos` bytes with `pending_scan` live: exactly the real DVD shape, where
/// a video PES opens partway through a read batch.
#[test]
fn a_resume_cursor_survives_the_drain_of_units_ahead_of_the_unbounded_pes() {
// Payload fed in the SAME chunk that opens the PES. Large enough that
// re-scanning it is unmistakable in `boundary_bytes_scanned`, and it is
// the exact span the un-rebased `pes_at` mutant re-examines.
const PAYLOAD: usize = 64 * 1024;
// A 14-byte MPEG-2 pack header with pack_stuffing_length 0.
const PACK: [u8; 14] = [
0x00,
0x00,
0x01,
PACK_HEADER_ID,
0x44,
0x00,
0x04,
0x00,
0x04,
0x01,
0x00,
0x00,
0x03,
0xF8,
];
// A length-BOUNDED PES — a complete unit, so the loop consumes it and
// `pos` advances past it before breaking on the unbounded PES.
const BOUNDED_PES: [u8; 11] = [
0x00, 0x00, 0x01, 0xE0, 0x00, 0x05, 0x80, 0x00, 0x00, 0xAA, 0xBB,
];
// The unbounded (length-0) video PES whose scan must be resumed.
const OPEN_PES: [u8; 9] = [0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00];
// Bytes drained ahead of the unbounded PES on the first feed — the
// `pos` the cursor must be rebased by.
const DRAINED: usize = PACK.len() + BOUNDED_PES.len();
let mut demuxer = PsDemuxer::new();
let mut first = PACK.to_vec();
first.extend_from_slice(&BOUNDED_PES);
first.extend_from_slice(&OPEN_PES);
first.extend_from_slice(&[0xFFu8; PAYLOAD]);
let head = demuxer.feed(&first);
assert_eq!(
head.len(),
1,
"the bounded PES ahead of the open one is emitted immediately, \
which is what makes the buffer drain with a cursor live"
);
// The terminating pack arrives at the head of the next feed — i.e.
// within `DRAINED` bytes of where the previous scan stopped, which is
// precisely the window an un-rebased `searched_to` skips over.
assert!(
PACK.len() <= DRAINED,
"the terminating pack must fit inside the window a stale \
`searched_to` would skip, or the mutant survives"
);
let packets = demuxer.feed(&PACK);
assert_eq!(
packets.len(),
1,
"the pack header terminates the open PES; a scan resumed past it \
never sees it and the PES runs on"
);
assert_eq!(
packets[0].data.len(),
PAYLOAD,
"exactly the payload fed belongs to the PES"
);
// Work bound: the payload is proved boundary-free ONCE. Re-scanning it
// after the drain roughly doubles this.
assert!(
demuxer.boundary_bytes_scanned <= (PAYLOAD + 1024) as u64,
"boundary search examined {} bytes over {PAYLOAD} bytes of payload — \
a cursor left un-rebased across the drain never matches the PES's \
new offset, so the scan restarts at the header",
demuxer.boundary_bytes_scanned
);
}
/// The boundary-ID check is a 4-way `||`; a mutant that turns the FIRST
@@ -1763,7 +2007,7 @@ mod tests {
let data = [0x00, 0x00, 0x01, PACK_HEADER_ID, 0xAA];
assert_eq!(
find_ps_boundary(&data, 0),
Some(0),
(Some(0), 0),
"a pack header start code alone must register as a PS-layer boundary"
);
}
+5 -2
View File
@@ -1521,8 +1521,11 @@ fn forensic_clip_extents(
tracing::warn!(target: "freemkv::keysource", "fmts: more than one forensic clip on the disc — segment byte space is ambiguous");
return Ok(None);
}
// Addressing variant: these extents are a byte-space map for the forensic
// segment table (`clip_byte_to_lba`), not a read plan — an unrecorded
// extent must stay in place here or every later segment offset shifts.
let exts: Vec<crate::disc::Extent> = udf
.file_extents(reader, &format!("/BDMV/STREAM/{name}"))
.file_extents_addressing(reader, &format!("/BDMV/STREAM/{name}"))
.map_err(io::Error::from)?
.into_iter()
.filter(|&(lba, sectors)| lba > 0 && sectors > 0)
@@ -4440,7 +4443,7 @@ mod tests {
"00001.fmts",
20,
FMTS_CONTENT_LBA - PART_START,
FMTS_CONTENT_SECTORS * 2048,
u64::from(FMTS_CONTENT_SECTORS) * 2048,
true,
)],
subdirs: Vec::new(),
+9 -3
View File
@@ -11,9 +11,15 @@
//! and resume cleanly there. The gap rounds up to (at most) one GOP — the price
//! of never emitting a dangling reference; it is logged.
//!
//! Audio and subtitle frames are independent (no inter-frame references), so a
//! gap there costs only the single already-dropped frame; the gate is a no-op
//! for non-video tracks (it always admits).
//! Most audio and all subtitle frames are independently decodable (each frame
//! re-inits on its own header), so a gap there costs only the single already-
//! dropped frame and this gate is a no-op for them (it always admits). The one
//! exception is TrueHD/MLP, whose predictor + restart state spans access units:
//! its re-init point is a codec-specific major-sync AU (not a generic keyframe),
//! so it runs the equivalent drop-forward-to-major-sync inside `codec::truehd`
//! rather than through this gate. In short: this gate is NOT keyed on "audio vs
//! video" but on "needs a re-init point after a gap" — video here, TrueHD in its
//! own parser, everything else genuinely independent.
/// Per-track keyframe-resync state. One gate per elementary stream; a video
/// track's gate stays "armed" from a discontinuity until the next keyframe.
+1 -1
View File
@@ -2250,7 +2250,7 @@ mod tests {
/// trimming to `[in, out]` is. Real discs author trailing audio (a fade after
/// the last video frame) PAST the OUT mark in the m2ts; the no-plan inference
/// path kept it, leaving audio seconds past the declared duration — the
/// `The Bourne Supremacy` defect (audio +35.6 s past a single-clip title's
/// audio-drift defect found on a real disc (audio +35.6 s past a single-clip title's
/// end). The plan drops it, and every KEPT frame is placed exactly as the
/// no-plan path would (raw rebased by `−in_ns`, which the MKV muxer already
/// does), so a disc with no out-of-mark content is byte-identical.
+200 -9
View File
@@ -85,6 +85,12 @@ pub struct PrefetchedSectorSource {
///
/// [`capacity_sectors`]: SectorSource::capacity_sectors
total_sectors: u32,
/// Latched the moment a terminal error crosses the channel. The
/// producer NEVER resumes after sending one (every error arm
/// `return`s), so the closed channel that follows is a dead source,
/// not end-of-stream — and `read_sectors` must keep saying so instead
/// of answering `Ok(0)` for the rest of the title.
producer_failed: bool,
}
impl PrefetchedSectorSource {
@@ -300,6 +306,22 @@ impl PrefetchedSectorSource {
return;
}
let sectors_read = (n / 2048) as u32;
// A genuine zero-byte read with no error
// would otherwise spin this loop forever.
// It is not end-of-stream either: the
// extent list still has `remaining`
// sectors to serve, so the inner source
// has quit early. Send a terminal
// sentinel — dropping `tx` here instead
// would reach the consumer as a clean EOF
// and finalize a TRUNCATED title as
// success, exactly as the panic sentinel
// below exists to prevent.
if sectors_read == 0 {
let _ =
tx.send(Err(crate::error::Error::SourceTerminated.into()));
return;
}
buf.truncate(n);
bytes_read_total = bytes_read_total.saturating_add(n as u64);
if let Some(ref f) = event_fn {
@@ -313,12 +335,6 @@ impl PrefetchedSectorSource {
if tx.send(Ok(buf)).is_err() {
return; // consumer dropped
}
// A genuine zero-byte read with no error would
// otherwise spin this loop forever; treat it
// as end-of-source.
if sectors_read == 0 {
return;
}
offset = offset.saturating_add(sectors_read);
}
Err(e) => {
@@ -344,6 +360,7 @@ impl PrefetchedSectorSource {
recycle_tx,
producer: Some(producer),
total_sectors,
producer_failed: false,
})
}
@@ -486,8 +503,29 @@ impl SectorSource for PrefetchedSectorSource {
let _ = self.recycle_tx.send(filled);
Ok(n)
}
Ok(Err(e)) => Err(crate::error::Error::IoError { source: e }),
// Channel closed (producer finished or panicked).
// Recover the producer's TYPED error rather than blanket-wrapping
// it as `Error::IoError`. That wrapper is a classification, not a
// container: `is_scsi_transport_failure` matches `IoError`, so a
// wrapped MEDIUM ERROR bad sector reached `fill_extents` looking
// like a wedged bridge and aborted the pass instead of being
// skipped under `skip_errors`. `From<io::Error> for Error`
// downcasts the boxed payload back to the exact variant the
// producer sent (status + sense intact); a genuine OS-level
// `io::Error` — the real dead-bus case — still becomes `IoError`.
Ok(Err(e)) => {
// The producer `return`s after every error it sends, so
// this is also the moment the source dies. Latch it: the
// closed channel that follows must not read as EOF.
self.producer_failed = true;
Err(crate::error::Error::from(e))
}
// Channel closed. Clean EOF only if the producer never
// signalled a failure — otherwise it exited without
// delivering the rest of the extents, and answering `Ok(0)`
// would let `fill_extents` mistake a dead source for a short
// read, zero-fill every remaining sector of the title and
// still report the pass as complete.
Err(_) if self.producer_failed => Err(crate::error::Error::SourceTerminated),
Err(_) => Ok(0),
}
}
@@ -1214,7 +1252,12 @@ mod tests {
/// an error (not Ok(0)/EOF), and its ErrorKind must survive the
/// round-trip through the channel. Grounding: the producer's
/// `Err(e) => tx.send(Err(e.into()))` arm, and `read_sectors`'
/// `Ok(Err(e)) => Err(IoError{source:e})`.
/// `Ok(Err(e)) => { self.producer_failed = true; Err(Error::from(e)) }`.
///
/// That arm recovers the producer's TYPED error by downcast rather than
/// blanket-wrapping it as `Error::IoError`, so the kind survives; the
/// `producer_failed` latch it also sets is what turns the channel close
/// that follows into `SourceTerminated` instead of a clean EOF.
#[test]
fn reader_error_propagates_with_kind() {
with_watchdog(Duration::from_secs(10), || {
@@ -1235,6 +1278,63 @@ mod tests {
});
}
/// An inner source that answers a mid-extent read with `Ok(0)` has quit
/// early: the extent list still has sectors to serve. The producer must
/// say so, not simply drop `tx` — a closed channel reads as clean
/// end-of-stream, and `DiscStream::fill_extents` then fabricates zeros
/// for every remaining sector of the title and reports the pass
/// complete. Same rule as the panic sentinel: a truncated title must
/// never be finalized as success.
struct QuitsEarlySource;
impl SectorSource for QuitsEarlySource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Ok(0)
}
fn capacity_sectors(&self) -> u32 {
9
}
}
#[test]
fn inner_source_quitting_early_is_not_reported_as_end_of_stream() {
with_watchdog(Duration::from_secs(10), || {
let extents = vec![Extent {
start_lba: 0,
sector_count: 9,
}];
let mut pf =
PrefetchedSectorSource::new(QuitsEarlySource, extents, 3, None).expect("spawn");
let mut buf = vec![0u8; 3 * 2048];
let mut last = pf.read_sectors(0, 3, &mut buf, false);
// Whatever the first answer, no call may ever settle on a clean
// `Ok(0)`: 9 sectors were promised and none were delivered.
for _ in 0..4 {
if last.is_err() {
break;
}
assert_eq!(
*last.as_ref().unwrap(),
0,
"the source delivered no bytes, so nothing can be Ok(n>0)"
);
last = pf.read_sectors(0, 3, &mut buf, false);
}
let err =
last.expect_err("an undelivered extent list must surface as an error, not as EOF");
assert!(
err.is_source_terminated(),
"the source is gone for good — retrying or skipping cannot \
recover anything; got {err:?}"
);
});
}
/// A read returning a byte count that is not a whole number of
/// sectors (n % 2048 != 0) must be rejected — never truncated and
/// advanced, which would split a sector and hand decrypt a partial
@@ -1472,4 +1572,95 @@ mod tests {
assert_eq!(got.len(), 30 * 2048, "all 10 extents must be drained");
});
}
/// Source whose every read fails with a `DiscRead` carrying the given
/// SCSI status (and optional sense) — an ordinary MEDIUM ERROR bad
/// sector (0x02 + 03/11/00) or the transport-failure sentinel (0xFF).
struct FailingSource {
status: u8,
sense: Option<crate::scsi::ScsiSense>,
}
impl SectorSource for FailingSource {
fn capacity_sectors(&self) -> u32 {
9999
}
fn read_sectors(
&mut self,
lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(self.status),
sense: self.sense,
})
}
}
fn read_one_err(status: u8, sense: Option<crate::scsi::ScsiSense>) -> crate::error::Error {
let extents = vec![Extent {
start_lba: 100,
sector_count: 9,
}];
let mut pf = PrefetchedSectorSource::new(FailingSource { status, sense }, extents, 3, None)
.expect("spawn");
let mut buf = vec![0u8; 3 * 2048];
pf.read_sectors(100, 3, &mut buf, false)
.expect_err("the producer's read failure must surface")
}
/// REGRESSION: an ordinary MEDIUM ERROR bad sector that crosses the
/// prefetch channel must NOT be classified as a SCSI transport failure.
///
/// `read_sectors` used to re-wrap EVERY channel error as
/// `Error::IoError { source }`, and `is_scsi_transport_failure` matches
/// `IoError` (the wedged-USB-bridge arm). So a skippable bad sector
/// arrived at `DiscStream::fill_extents` looking like a dead bus and
/// aborted the whole pass instead of honouring `skip_errors`.
#[test]
fn bad_sector_across_channel_is_not_a_transport_failure() {
with_watchdog(Duration::from_secs(10), || {
let sense = crate::scsi::ScsiSense {
sense_key: 0x03,
asc: 0x11,
ascq: 0x00,
};
let err = read_one_err(crate::scsi::SCSI_STATUS_CHECK_CONDITION, Some(sense));
assert!(
!err.is_scsi_transport_failure(),
"a MEDIUM ERROR bad sector must stay a bad sector across the \
prefetch channel, got {err:?}"
);
// The classification survives because the typed variant does.
assert!(
matches!(err, crate::error::Error::DiscRead { status: Some(s), .. } if s == 0x02),
"expected the producer's DiscRead to survive the channel, got {err:?}"
);
assert_eq!(
err.scsi_sense().map(|s| (s.sense_key, s.asc, s.ascq)),
Some((0x03, 0x11, 0x00)),
"the drive's sense triple must survive the channel"
);
});
}
/// OPPOSITE-DIRECTION CONTROL: a genuine transport failure (status 0xFF,
/// wedged USB bridge) crossing the same channel MUST still classify as a
/// transport failure, so `fill_extents` / sweep keep aborting the pass
/// instead of zero-filling every read against a dead bus.
#[test]
fn transport_failure_across_channel_still_classifies() {
with_watchdog(Duration::from_secs(10), || {
let err = read_one_err(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None);
assert!(
err.is_scsi_transport_failure(),
"a 0xFF transport failure must remain one across the prefetch \
channel, got {err:?}"
);
});
}
}
+28 -2
View File
@@ -91,7 +91,20 @@ pub fn resolve_keys_for(
inputs.samples = if src_vec.is_empty() {
Vec::new()
} else {
match disc.titles.iter().max_by_key(|t| t.size_bytes).cloned() {
// Sample encrypted units from the LARGEST title that has video — the
// main feature carries the most units to validate a key against, while a
// size-inflated STREAMLESS decoy (the biggest title on some obfuscated
// UHDs) would waste the read on the wrong title. Falls back to the
// largest title outright when none has video (title-less/empty discs and
// the pre-scan sampling path).
match disc
.titles
.iter()
.filter(|t| t.has_probable_video())
.max_by_key(|t| t.size_bytes)
.or_else(|| disc.titles.iter().max_by_key(|t| t.size_bytes))
.cloned()
{
Some(title) => read_encrypted_units(reader, &title, MIN_SAMPLE_UNITS),
None => Vec::new(),
}
@@ -451,7 +464,7 @@ const AACS_PROBE_UNITS: usize = 8;
/// The extra step over `scan_iso` is the encryption verdict.
/// `Disc::scan_with` decides `encrypted` STRUCTURALLY, from the presence of an
/// `/AACS` or `/BDMV/AACS` directory (see `disc::aacs_dir_present`). For the common
/// case — a MakeMKV-style backup, which strips `AACS/` — that already gives the
/// case — a typical disc backup, which strips `AACS/` — that already gives the
/// right answer, and `DecryptKeys::None` is a pass-through. But a folder copied
/// verbatim from a decrypted disc keeps `AACS/`, and the tree shape then claims
/// encryption over content that is already in the clear: the rip would fail
@@ -545,14 +558,27 @@ fn probe_folder_encryption(reader: &mut dyn SectorSource, disc: &Disc) -> Result
// genuine ciphertext reads as clear and the mux writes it out as video at
// exit 0. `is_unit_aligned` cannot catch it, because it measures against
// this same wrong base.
// Probe the first extent of the LARGEST title that has video — the movie —
// not merely the largest-by-sector-count title, which on an obfuscated disc
// can be a streamless decoy whose base would mismeasure the ciphertext
// alignment for the actual feature. Falls back to the largest title when
// none has video.
let Some(extent) = disc
.titles
.iter()
.filter(|t| t.has_probable_video())
.max_by_key(|t| {
t.extents
.iter()
.fold(0u64, |a, e| a.saturating_add(e.sector_count as u64))
})
.or_else(|| {
disc.titles.iter().max_by_key(|t| {
t.extents
.iter()
.fold(0u64, |a, e| a.saturating_add(e.sector_count as u64))
})
})
.and_then(|t| t.extents.first())
else {
// No content to judge. A folder with an AACS directory and no titles
+226
View File
@@ -0,0 +1,226 @@
//! Test-only capture of `tracing` events, so the crate's logging contract is
//! ENFORCED rather than merely commented.
//!
//! # Why this exists
//!
//! The error contract is "Account / Log / Classify", and the round-2 audit
//! found the third leg unverifiable: three separate sites carry long comments
//! insisting they log **the error's OWN code, not a fixed one** — because
//! flattening a scratched sector (E6000) or an over-long allocation-descriptor
//! chain (E6016) into E6017 sends whoever triages them after authoring holes
//! and hides the population that actually exists. Nothing tested that. Putting
//! a literal back at `bluray.rs`'s or `hddvd.rs`'s warn sites broke no test, so
//! the guarantee was a convention one careless edit away from being false. Two
//! of those very sites were changed in round 1, and a third still carried a
//! hardcoded `code = 6017`.
//!
//! Likewise "absence of a log is itself a bug": a refusal that returns the
//! right error but says nothing produces the wrong population downstream (a
//! residual-underrunning drive is indistinguishable from a scratched disc).
//! That is only checkable by looking at what was emitted.
//!
//! # Why a hand-rolled subscriber and not `tracing-subscriber`
//!
//! Same posture as [`crate::harness`]: this crate has exactly one
//! dev-dependency on purpose. `tracing-subscriber` would pull a tree of them to
//! do what forty lines of [`tracing::Subscriber`] does here. The capture is
//! installed with [`tracing::subscriber::with_default`], which is
//! THREAD-LOCAL — so it composes with `cargo test`'s parallel harness and two
//! capturing tests cannot see each other's events.
//!
//! Field values are stringified through [`std::fmt::Debug`]/`Display` because
//! that is all the visitor API offers without a typed schema; tests compare
//! against the string form of the expected constant, which is exactly the
//! comparison that catches a hardcoded code.
#![cfg(test)]
use std::sync::{Arc, Mutex};
/// One captured `tracing` event: its target, level, message and fields.
#[derive(Debug, Clone)]
pub(crate) struct CapturedEvent {
pub target: String,
pub level: tracing::Level,
/// Every field, in emission order, stringified. The implicit `message`
/// field (the format string) is included under the name `message`.
pub fields: Vec<(String, String)>,
}
impl CapturedEvent {
/// The stringified value of `name`, or `None` if the event has no such
/// field.
pub fn field(&self, name: &str) -> Option<&str> {
self.fields
.iter()
.find(|(k, _)| k == name)
.map(|(_, v)| v.as_str())
}
/// The event's message (the `tracing` format string), or `""`.
pub fn message(&self) -> &str {
self.field("message").unwrap_or("")
}
}
#[derive(Default)]
struct Visitor(Vec<(String, String)>);
impl tracing::field::Visit for Visitor {
fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) {
self.0
.push((field.name().to_string(), format!("{value:?}")));
}
fn record_str(&mut self, field: &tracing::field::Field, value: &str) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_u64(&mut self, field: &tracing::field::Field, value: u64) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_i64(&mut self, field: &tracing::field::Field, value: i64) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_bool(&mut self, field: &tracing::field::Field, value: bool) {
self.0.push((field.name().to_string(), value.to_string()));
}
}
type Sink = Arc<Mutex<Vec<CapturedEvent>>>;
thread_local! {
/// The sink for a capture ACTIVE ON THIS THREAD, if any. Thread-local so
/// concurrent captures across `cargo test`'s parallel harness never see
/// each other's events, and so a non-capturing thread simply has `None`.
static SINK: std::cell::RefCell<Option<Sink>> = const { std::cell::RefCell::new(None) };
}
/// The ONE process-wide subscriber. Installed once and left installed; it is
/// offered every event and records into whichever thread's sink is active,
/// dropping the event when none is.
struct Capture;
impl tracing::Subscriber for Capture {
// `sometimes`, deliberately NOT the default `always`/`never`. A cacheable
// interest lets `tracing`'s GLOBAL per-callsite cache short-circuit a
// callsite to "off", and under `cargo test`'s parallel harness a cache
// rebuild triggered by ANY other thread can leave it there while a capture
// is live — the capture then observes NOTHING. That is the race that made
// `parse_playlist_unreadable_clip_icb_yields_no_title` flake ~1 run in 15.
// `sometimes` forces `enabled` to be consulted on the emitting thread for
// every event, so a capture always sees its own.
fn register_callsite(
&self,
_meta: &'static tracing::Metadata<'static>,
) -> tracing::subscriber::Interest {
tracing::subscriber::Interest::sometimes()
}
fn enabled(&self, _metadata: &tracing::Metadata<'_>) -> bool {
SINK.with(|s| s.borrow().is_some())
}
// Spans are irrelevant here — nothing in this crate asserts on span
// structure, only on events — so they get a constant id and no storage.
fn new_span(&self, _span: &tracing::span::Attributes<'_>) -> tracing::span::Id {
tracing::span::Id::from_u64(1)
}
fn record(&self, _span: &tracing::span::Id, _values: &tracing::span::Record<'_>) {}
fn record_follows_from(&self, _span: &tracing::span::Id, _follows: &tracing::span::Id) {}
fn event(&self, event: &tracing::Event<'_>) {
SINK.with(|s| {
if let Some(sink) = s.borrow().as_ref() {
let mut v = Visitor::default();
event.record(&mut v);
let meta = event.metadata();
sink.lock().expect("capture mutex").push(CapturedEvent {
target: meta.target().to_string(),
level: *meta.level(),
fields: v.0,
});
}
});
}
fn enter(&self, _span: &tracing::span::Id) {}
fn exit(&self, _span: &tracing::span::Id) {}
}
/// Install the single global capturing subscriber, exactly once.
fn install() {
static INSTALLED: std::sync::OnceLock<()> = std::sync::OnceLock::new();
INSTALLED.get_or_init(|| {
// This is the crate's ONLY `set_global_default`; a second call cannot
// happen. Tolerate it via `.ok()` rather than panic if that ever
// changes — capture then no-ops, which the non-empty assertions catch.
let _ = tracing::subscriber::set_global_default(Capture);
});
}
/// Run `f` with every `tracing` event it emits ON THIS THREAD captured.
///
/// Returns `f`'s value alongside the events, in emission order.
///
/// # Why one global subscriber, not scoped `with_default`
///
/// `tracing`'s per-callsite INTEREST CACHE is GLOBAL, but `with_default` is
/// thread-local. Under `cargo test`'s parallel harness a rebuild of that cache
/// — triggered by ANY other thread registering any callsite for the first time
/// — re-evaluates the target callsite against the global dispatcher, which a
/// scoped subscriber is NOT part of, and can leave it cached "off" while a
/// capture is live, so the capture observes NOTHING. Serialising captures
/// against each other does not help: the poisoning thread is not itself
/// capturing. `parse_playlist_unreadable_clip_icb_yields_no_title` flaked ~1
/// run in 15 on exactly this — an empty event list.
///
/// The robust shape is a single subscriber installed globally for the whole
/// run, whose `register_callsite` returns `sometimes` (so no callsite is ever
/// hard-cached) and which routes each event to the emitting thread's own sink.
/// No scoped-dispatcher transitions, no cross-thread cache race, and concurrent
/// captures on different threads stay isolated by the thread-local sink.
pub(crate) fn capture<T>(f: impl FnOnce() -> T) -> (T, Vec<CapturedEvent>) {
install();
let sink: Sink = Arc::default();
// Save/restore any outer sink so a nested capture on one thread still works.
let prev = SINK.with(|s| s.borrow_mut().replace(sink.clone()));
let out = f();
SINK.with(|s| *s.borrow_mut() = prev);
let events = std::mem::take(&mut *sink.lock().expect("capture mutex"));
(out, events)
}
#[cfg(test)]
mod tests {
use super::*;
/// The capture must actually see events and their field VALUES — if it
/// silently recorded nothing, every logging assertion built on it would
/// pass vacuously, which is worse than having no harness at all.
///
/// Mutation: an `enabled()` returning `false`, or an `event()` that drops
/// the visitor's fields, fails here.
#[test]
fn capture_records_target_level_and_fields() {
let ((), events) = capture(|| {
tracing::warn!(target: "freemkv::testlog", code = 6017u16, clip = ?"A.EVO", "E6017");
});
assert_eq!(events.len(), 1, "exactly one event: {events:?}");
assert_eq!(events[0].target, "freemkv::testlog");
assert_eq!(events[0].level, tracing::Level::WARN);
assert_eq!(events[0].field("code"), Some("6017"));
assert_eq!(events[0].field("clip"), Some("\"A.EVO\""));
assert_eq!(events[0].message(), "E6017");
}
/// A field that is absent must read as `None`, not as an empty string — an
/// assertion of the shape `field("code") == Some(..)` has to be able to
/// fail when the site stops logging the code at all.
#[test]
fn missing_field_is_none_and_capture_is_scoped() {
let ((), events) = capture(|| tracing::warn!(target: "freemkv::testlog", "no fields"));
assert_eq!(events[0].field("code"), None);
// Emitted outside `capture`, so it must not appear in a later capture.
tracing::warn!(target: "freemkv::testlog", code = 1u16, "outside");
let ((), later) = capture(|| {});
assert!(
later.is_empty(),
"capture is scoped to its closure: {later:?}"
);
}
}
+770 -133
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -104,7 +104,7 @@ pub(crate) fn map_host_certs(certs: &[crate::aacs::types::HostCert]) -> Vec<fu::
/// Result of a capability dispatch: `(matched_name, result)`. `matched_name` is
/// the unlocker that handled it (or `""` if none did) — lets the caller record
/// WHICH unlocker ran (e.g. `LibreDrive` vs `Renesas`), distinct from the ld-only
/// WHICH unlocker ran (e.g. `MT1959` vs `Renesas`), distinct from the ld-only
/// identity lookup [`unlocker_name`]. Iterating stops at the first unlocker whose
/// capability method returns anything other than `NotApplicable` — i.e. an actual
/// unlock (`Ok`) OR a real failure such as a dead bus (`Err(Transport)`), which
+8 -8
View File
@@ -559,18 +559,18 @@ fn css_roundtrip_multiple_keys() {
}
}
// ── CSS Stevenson attack tests ─────────────────────────────────────────────
// ── CSS keyless recovery tests ─────────────────────────────────────────────
/// Attempt the Stevenson attack on synthetically scrambled sectors.
/// Attempt the keyless recovery on synthetically scrambled sectors.
///
/// The CSS cipher on real DVDs stores ciphertext through a TAB1 output
/// layer that the Stevenson attack depends on. Synthetically scrambled
/// layer that the keyless recovery depends on. Synthetically scrambled
/// sectors (produced by calling descramble_sector on plaintext) may not
/// exhibit this relationship, so the attack is not guaranteed to converge
/// on synthetic data. This test verifies that when the attack DOES return
/// a key, that key correctly descrambles the sector.
#[test]
fn css_stevenson_attack_validates_cracked_key() {
fn css_keyless_recovery_validates_cracked_key() {
let candidates: &[([u8; 5], [u8; 5])] = &[
(
[0x42, 0x13, 0x37, 0xBE, 0xEF],
@@ -617,7 +617,7 @@ fn css_stevenson_attack_validates_cracked_key() {
css::lfsr::descramble_sector(key, &mut sector);
sector[0x14] = 0x30;
let cracked = css::stevenson::crack_title_key(&sector);
let cracked = css::keyless::crack_title_key(&sector);
if let Some(cracked_key) = cracked {
let mut test = sector.clone();
@@ -635,7 +635,7 @@ fn css_stevenson_attack_validates_cracked_key() {
any_cracked = true;
eprintln!(
"Stevenson attack succeeded: key={:02X?} seed={:02X?} cracked={:02X?}",
"keyless recovery succeeded: key={:02X?} seed={:02X?} cracked={:02X?}",
key, seed, cracked_key
);
}
@@ -643,7 +643,7 @@ fn css_stevenson_attack_validates_cracked_key() {
if !any_cracked {
eprintln!(
"Stevenson attack did not converge on any synthetic key/seed pair. \
"keyless recovery did not converge on any synthetic key/seed pair. \
This is expected: synthetic sectors lack the TAB1 output encoding \
present in real CSS-encrypted DVD sectors."
);
@@ -696,7 +696,7 @@ fn css_recover_title_key_with_exact_plaintext() {
sector[0x14] = 0x30;
// Recover with exact known plaintext
let recovered = css::stevenson::recover_title_key(&sector, &pes_header);
let recovered = css::keyless::recover_title_key(&sector, &pes_header);
if let Some(rkey) = recovered {
let mut test = sector.clone();
+1 -1
View File
@@ -38,7 +38,7 @@ fn css_decrypt_of_an_uncrackable_sector_still_descrambles() {
// and the cached key stays the best available evidence.
//
// This test previously asserted DecryptFailed, matching a round-9 change
// that made real DVDs unrippable (Greenland.iso). CSS is not AACS: an AACS
// that made real DVDs unrippable. CSS is not AACS: an AACS
// unit key either opens a unit or does not, whereas a CSS title key is
// recovered from data whose recoverability varies sector by sector.
let mut sector = vec![0xFFu8; 2048];