19 Commits
Author SHA1 Message Date
Matthew Jackson 767d205fd4 mux: document that the Url arm selects via InputOptions.selection
CI / lint (push) Failing after 1m19s
CI / test (push) Failing after 1m20s
leak-guard / leak-guard (push) Successful in 10s
CI / check-macos (push) Has been cancelled
CI / check-windows (push) Has been cancelled
The Url mux path prunes streams inside input() via InputOptions.selection;
MuxOptions.selection only applies on the File/Session arms. A Url-source caller
must set InputOptions.selection — noting it so a future caller does not put the
selection on MuxOptions and silently keep every track (the bug the GUI hit).
2026-07-28 18:40:47 -07:00
Matthew Jackson bbb5a953f1 1.6.0: remove recovery strategy (moved to freemkv-engine) + trim dead surface
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).

- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
  Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
  classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
  three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
  READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
  consumer), and the DriveSpeed enum (its one live use — set max drive
  speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
  decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
2026-07-28 15:35:19 -07:00
MattJackson 3f238ab3dc changelog: 1.6.0 section (layering API, mux fixes, engine relocation, stream selection); date 1.5.2 2026-07-28 13:54:42 -07:00
MattJackson ec62618bde mux: end-to-end test that stream selection drops excluded-PID frames
A title declaring two audio PIDs, pruned to one via StreamSelection::apply
before build_iso_pipeline, must never surface a frame from the excluded PID.
Proves the declaration-driven seam end-to-end through the full highway (read ->
demux -> codec parse): the demuxer is built from the pruned title.streams, so
the excluded PID is untracked and skipped, and both track headers and frames
follow the pruned list. Builds on the existing synthetic-TS harness.
2026-07-28 13:36:36 -07:00
MattJackson 15111d544b disc: add DiscTitle audio_streams/subtitle_streams/video_streams accessors
Typed iterators over the audio/subtitle/video streams, cleaner than matching on
the Stream enum for the common iterate-the-tracks case (stream selection, the
desktop UI info panel, disc-info). Additive; all lib tests pass on 1.86.
2026-07-28 13:35:12 -07:00
MattJackson 19b646e012 mux: StreamSelection primitive + apply sites for per-title stream selection
The demux pipeline is declaration-driven off DiscTitle.streams (build_demux_state,
DiscStream::new, and the MKV writer all key off that list), so 'which streams to
keep' is already a pipeline capability with no public knob. This adds the knob:

- mux/select.rs: StreamSelection { audio, subtitle: PidFilter::All | Only(Vec<u16>) }
  + apply(&mut DiscTitle): keep Video always, keep Audio/Subtitle whose PID the
  filter lists, prune the rest (and the parallel codec_privates in lockstep);
  error SelectionPidUnknown on a listed PID absent from the title (fail loud, not
  a silently-missing track). Pure; 6 unit tests. Re-exported at crate root.
- Error::SelectionPidUnknown (E6014).
- MuxOptions gains  (+ derives Default now) applied in mux_stream's
  Iso/Session arms before the highway/DiscStream builds demux state (and before
  probe_and_remap's DVD AC-3 PID rewrite). InputOptions gains  applied
  in input()'s iso arm right after the title-index bounds check.

PIDs not languages -- language->PID is caller/engine policy. Default All/All is a
no-op (apply gated on !is_all()), so the no-selection path is byte-identical:
nothing below the title-finalization line changes (ts/ps/demux_thread/
pipelined_stream/mkv/disc untouched). All 2488 lib tests pass on 1.86.

Design: freemkv-private/audit/engine-split/STREAM-SELECTION-DESIGN.md (Fable).
2026-07-28 13:14:40 -07:00
MattJackson 17694af625 error: add is_disc_level_no_key classifier (re-exported)
Distinguishes a WHOLE-DISC key failure (E_NO_DISC_KEY / E_KEYDB_LOAD /
E_AACS_NO_KEYS -- every title fails identically) from a per-title skippable
stub. The engine's multi-title loop uses it to fail-fast on the first no-key
title instead of iterating all N. Additive.
2026-07-28 12:45:58 -07:00
MattJackson 828f5c0192 error: re-export is_halt + is_skippable_title_stub at crate root
The engine's multi-title rip loop classifies per-title mux failures (halt vs
skippable stub vs hard) using these typed classifiers instead of E-code string
matching. Additive; no behavior change.
2026-07-28 12:31:40 -07:00
MattJackson 075223a4bd disc: promote locate_ranges to pub (engine multipass reads it)
Small pub promotion + fmt one-lining. The relocated multipass progress
reporting in freemkv-engine needs locate_ranges externally. No behavior change.
2026-07-28 12:09:38 -07:00
MattJackson 94ac4a442a drive: promote extract_scsi_context to pub
Small pure error->(status,sense) introspection helper the relocated
sweep/patch will need externally. Same category as the prior WritebackFile/
resolve_content_key_map promotions -- infra, not policy. No behavior change.
2026-07-28 11:41:49 -07:00
MattJackson f468237279 io: promote WritebackFile to pub
Bounded-cache buffered File replacement used across mux/extract/sweep/patch
-- general I/O infrastructure, not recovery policy. freemkv-engine's
relocated sweep/patch need to construct it directly once those methods
leave this crate. No behavior change.
2026-07-28 11:36:43 -07:00
MattJackson 227545fabc scsi: promote SenseFamily to a lib-level SCSI-fact primitive
Moved SenseFamily::from_sense_key + is_wedge_family from disc/read_error.rs
into scsi/mod.rs (with its own tests) and re-exported at the crate root.
This is pure SCSI sense-code classification -- objective hardware fact, zero
recovery-policy opinion -- so it belongs in the library primitives, unlike
the retry-DECISION state machine (ReadCtx/PassSummary/ReadAction/
handle_read_error) built on top of it, which is freemkv's specific recovery
strategy and is moving to freemkv-engine next.

disc/read_error.rs and disc/section_recover.rs now import SenseFamily from
crate::scsi instead of defining/re-exporting their own copy. No behavior
change. Precommit green on Rust 1.86 (fmt+clippy+test).
2026-07-28 11:32:13 -07:00
MattJackson 05fd7fcbfa disc: promote resolve_content_key_map + encrypted_content_ranges to pub
The upcoming freemkv-engine crate needs both to build the multipass
sweep/patch recovery strategy externally over Disc's public API. Everything
else sweep/patch touch on Disc was already pub; these were the only two
gaps. No behavior change -- visibility only.
2026-07-28 11:26:37 -07:00
Matthew Jackson 967d0ac77e mux: fix DTS core-header false-drops + close TrueHD/mux gate coverage
DTS core decodability gate (core_header_drop_reason) — full ETSI TS 102 114
spec-conformance sweep against ffmpeg ff_dca_parse_core_frame_header and
dcadec parse_frame_header:

- deficit_samples: only require ==32 for NORMAL frames (FTYPE==1). A
  TERMINATION frame (FTYPE==0, the last frame of a stream) legitimately
  carries fewer and is fully decodable; the old unconditional check dropped
  it on every stream that ends on one — a guaranteed per-track silence gap.
  Matches ffmpeg (normal_frame && deficit != DCA_PCMBLOCK_SAMPLES) and
  dcadec (branches on normal_frame).
- reserved bit (after RATE): both reference decoders SKIP it (ffmpeg
  skip_bits1, dcadec bits_skip1 "Reserved field") and never reject on it.
  Rejecting was a false-drop that silenced any real stream whose encoder
  set the bit. Relaxed to read-and-discard; DropReason::ReservedBit removed.

Swept and confirmed spec-correct as-is (no change): npcmblocks multiple-of-8,
frame_size>=96, audio_mode>=16 (ffmpeg-permissive), sample-rate validity
table (matches avpriv_dca_sample_rates incl 96k/192k at 14/15), LFE flag==3
invalid, PCMR bits table (matches dcadec sample_res {16,16,20,20,0,24,24,0}).
Bit-read order verified field-by-field against dcadec. bit_rate is left
unvalidated (lenient, never-false-drop direction) as before.

Tests: termination frame with small deficit is kept; normal frame with bad
deficit is dropped; reserved-bit-set frame is kept. make_bad_dts_core now
uses an invalid LFE flag (duration-neutral) instead of the relaxed reserved
bit.

TrueHD: add coverage for the EXTENDED major-sync header CRC path (ms[25]&1,
mshdr=28+2+2n) — previously zero-tested, the exact path a shipped endianness
bug once used to silently drop whole 7.1/Atmos tracks. Trailer is an
independently-computed oracle (separate CRC-16/0x2D, anchored to the 0x4FF7
catalogue value, NOT crc16_mlp), stored little-endian; test asserts accept,
body-corruption reject, and big-endian-trailer reject.

mux driver: extract the finish completion mapping into pure mux_run_completed
so the finalize_failed -> completed=false branch (reachable only via real
write-thread wedge timing) is unit-tested; add an out-of-range
MuxInput::Session title_index test asserting a clean Error::MuxTrackRange
(E9011) instead of a panic.
2026-07-24 10:28:21 -07:00
Matthew Jackson eb0aa3556e Fix read-fault misclassification, DTS AMODE channel table, and untestable guards
- resolve_fmts_key_map: distinguish a genuinely-not-FMTS disc from a
  transient live-drive read fault. read_filesystem now returns the new
  Error::UdfNotFilesystem for a deterministic tag/format mismatch (no AVDP,
  no partition descriptor, no FSD); resolve maps only UdfNotFilesystem (fs)
  and UdfNotFound (.tbl absent) to Ok(None), and PROPAGATES DiscRead / other
  I/O faults so a marginal AACS 2.1 disc fails loud instead of silently
  dropping forensic content under a base-Unit-Key-only map.

- DTS_AMODE_CH (mp4/audio.rs): extend 10→16 entries
  {1,2,2,2,2,3,3,4,4,5,6,6,6,7,8,8} (ff_dca_channels / ETSI TS 102 114) so
  the spec-legal high AMODEs that now pass the decodability gate declare
  their true channelcount (AMODE 13→7, 14/15→8) instead of a truncated 6.

- session.rs resolve_keys "called before scan" guard is now testable:
  from_parts_for_test takes Option<Disc>; added a test that a disc-less
  session returns a clean DeviceNotReady Err rather than panicking.

- mp4/read.rs: a track with samples but a missing/malformed stts (mandatory
  per ISO/IEC 14496-12) is dropped rather than emitting all-zero timestamps,
  matching the existing stco/stsc guards; all-tracks-dropped → Mp4Invalid.

- Remove the inert MuxInput::Iso.key_map field (the Iso path re-derives its
  map inside build_iso_pipeline); the live path keeps Live.key_map.

All four fixes are mutation-verified.
2026-07-24 09:59:33 -07:00
Matthew Jackson 7ba43d9c02 mux: distinguish FMTS phase-probe read fault from wrong key; cover multi-CPS
Fix 1 (correctness): resolve_fmts_key_map's per-index phase probe read a
single representative segment via read_unit (whose read_sectors(...).ok()?
swallows read errors into None) with no fault fallback. A transient live-drive
read fault (e.g. NOT READY 2/04/3E on the BU40N) made every probe read return
None, giving even==0 && odd==0 — indistinguishable from a genuine wrong key —
so resolve_tie_phase returned FmtsKeyMissing and aborted the entire multi-title
rip even though the forensic index keys were valid and in hand.

Extract the probe into probe_index_phase, which returns Phase / WrongKey /
ReadFault. It mirrors the anchor loop's tolerance: it tries multiple same-index
segments and only concludes WrongKey once a read actually succeeded and decrypted
to no clean parity. If every read of every same-index segment faults it returns
ReadFault; the caller then leaves the phase unresolved so the range-builder
defaults to Phase::All (decrypt both parities, demux drops the garbled alternate
half) instead of aborting. A wrong key can never be masked as a read fault:
ReadFault requires that not a single read succeeded, so zero decrypt evidence.

Fix 2 (test coverage): resolve_mux_key_map's multi-CPS branch was only exercised
with an all-zero source, so pick(), the KeyFetch cold path, and the fail-loud
DecryptFailed guard never ran. Add tests over real AACS ciphertext (via
aacs_encrypt_unit_for_test) covering pick() selecting the correct pool index,
the DecryptFailed guard firing when a clean sample matches no held/fetched key,
and the fetch cold path recovering a missing unit key. Mutation-verified.

Fix 3 (doc): move the reader_event_fn EventKind->MuxEvents mapping doc off
session_mux_keys onto reader_event_fn.
2026-07-24 09:28:09 -07:00
Matthew Jackson bf9a69ec80 mux: thread halt into live AACS key-map resolution; cover Session arm
Round-2 follow-ups to 6d6e60f (inline base-map resolve on the live
single-pass Session/Live mux arms).

Fix 1 (halt threading) — the inline resolve chain sampled ciphertext off
the LIVE drive with no cancel token, so an operator /api/stop during key
resolution was not honored (the FMTS probe can issue hundreds of reads,
each able to stall to the 60s SCSI recovery timeout — violating the
"don't hammer a struggling live drive" rule). Add an optional
`halt: Option<&Halt>` to `resolve_mux_key_map`, `resolve_fmts_key_map`,
`resolve_inline_base_map`, and `Disc::resolve_content_key_map`, and poll
it at each loop boundary (FMTS anchor + per-index probe loops, multi-CPS
extent loop) — returning Err(Halted) promptly. Live/Session arms pass the
driver's halt; sweep/patch pass their own token (via Halt::from_arc);
file-backed probe/ISO callers pass None. Tested with a pre-cancelled halt
(Err Halted, no extent sampling) and a None-halt no-abort case;
mutation-verified (dropping the extent-loop check → Ok, not Err).

Fix 2 (Session-arm coverage) — the MuxInput::Session arm ran the same
resolve→install→decrypt sequence as Live but had NO end-to-end test
(DiscSession only exposed open(), which needs live hardware). Add a
#[cfg(test)] DiscSession::from_parts_for_test (injected reader + scanned
disc, no Drive), an end-to-end AACS decrypt test through the Session arm
(mutation-verified: dropping with_key_map → mux aborts), and a
missing-reader clean-error (not panic) test.

Fix 3 (cleanups) — io_error_code: remove the unreachable typed-Error
downcast branch (From<Error> for io::Error stringifies; no path builds an
io::Error holding a typed Error), keeping the stringify parse is_halt /
is_skippable_title_stub rely on. Add a resolve_keys_for test covering the
largest-title sampling branch. Document the patch wedge-exit coverage gap
(TODO) in passn_handler_ab.rs.
2026-07-24 09:04:19 -07:00
Matthew Jackson 6d6e60fdf8 mux: resolve+install AACS key map on live single-pass (Session/Live)
Under the map-only decrypt model an AACS DecryptingSectorSource decrypts
nothing until a key map is installed; with no map the AACS arm fails loud
with DecryptFailed on the first content unit. The two inline live-mux arms
in mux_stream did not install one:

  - MuxInput::Session (freemkv `rip disc://…mkv`) installed NO map at all.
  - MuxInput::Live (autorip non-FMTS single-pass) installed only a
    caller-supplied forensic FMTS map, which is None for a plain AACS disc.

So EVERY plain AACS Blu-ray/UHD ripped via the live single-pass path failed
DecryptFailed on the first content read. This predates the mux_stream
refactor: the bug was introduced with the map-only decrypt model, and the
pre-refactor CLI likewise built DiscStream::new without with_key_map.

Fix: add resolve_inline_base_map, the inline counterpart to what
build_iso_pipeline does for the file highway. Both arms now resolve the
AACS map off the reader (borrow to sample, then move into DiscStream) and
install it via with_key_map before any read. DVD/CSS keeps DecryptKeys::None
(DiscStream's per-title CSS crack owns it); clear/raw resolve to no map.
Session passes session.key_fetch() so a multi-CPS/orphan unit can still be
recovered; a caller-supplied FMTS map (autorip) is used verbatim, never
re-resolved.

Tests: an end-to-end MuxInput::Live mux over a genuinely-AACS-encrypted
synthetic unit now decrypts and finalises (mutation-verified: dropping the
resolve/install makes the mux abort). Adds a pub(crate) test-only AACS
encrypt helper so the mux test can build a real encrypted fixture, and a
gating test for resolve_inline_base_map (AACS→map, CSS/clear/raw→none).
2026-07-24 08:34:51 -07:00
Matthew Jackson 661ab138c6 Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just
  0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements;
  only 16-63 are reserved. ffmpeg's ff_dca_channels[16] confirms 10-15 are
  decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal
  multichannel core frames as undecodable, silencing recoverable audio. Add a
  regression test (literal 0..16 range) that fails if the bound reverts to 10.

- keysource: only memoize a NEGATIVE (empty) key-fetch result when every source
  genuinely ran and none held the key — never when a source Err'd (network down,
  unreachable). A transient outage was being cached as a permanent "no key" for
  the fingerprint, permanently dropping a unit that could be recovered once the
  source came back. Thread an `errored` flag out of the drivers and gate the
  cache insert on it. Tests cover both the recover-after-outage case and that a
  genuine absence is still memoized.

- pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS
  display sets through the full demux -> parse -> observe -> apply path, both a
  forced verdict landing and a non-forced verdict clearing a vendor flag.

- mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD).

- scan_iso test: add independent fixture expectations (volume id) so the parity
  test is no longer purely tautological against a re-run of the same composition.
2026-07-24 08:32:37 -07:00
162 changed files with 6610 additions and 62560 deletions
-63
View File
@@ -1,63 +0,0 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+10 -188
View File
@@ -2,228 +2,50 @@ name: CI
on: on:
push: push:
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed branches: [main]
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
pull_request: pull_request:
jobs: jobs:
lint: lint:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
with: - uses: dtolnay/rust-toolchain@1.86.0
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with: with:
components: clippy, rustfmt components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check - run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked # libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed # would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip, # lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked. # bdemu) track Cargo.lock and DO use --locked.
# --all-targets so TEST code is linted too. Without it this crate — the - run: cargo clippy -- -D warnings
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
working-directory: libfreemkv
test: test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
with: - uses: dtolnay/rust-toolchain@1.86.0
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests - run: cargo test --tests
working-directory: libfreemkv
check-macos: check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest runs-on: macos-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
with: - uses: dtolnay/rust-toolchain@1.86.0
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo check - run: cargo check
working-directory: libfreemkv
check-windows: check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest runs-on: windows-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
with: - uses: dtolnay/rust-toolchain@1.86.0
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
# Build the tests (not just `cargo check`): catches errors in test # Build the tests (not just `cargo check`): catches errors in test
# code and forces full codegen of the Windows-only SPTI transport # code and forces full codegen of the Windows-only SPTI transport
# (src/scsi/windows.rs), which never compiles on the Linux/macOS dev # (src/scsi/windows.rs), which never compiles on the Linux/macOS dev
# hosts. We don't `cargo test` here — the suite needs no drive but the # hosts. We don't `cargo test` here — the suite needs no drive but the
# extra build is the value; running tests is covered by the Linux job. # extra build is the value; running tests is covered by the Linux job.
- run: cargo build --tests - run: cargo build --tests
working-directory: libfreemkv
# ── Did this change break anything downstream? ──────────────────────────────
#
# Every job above proves libfreemkv builds. None proved its DEPENDENTS do,
# and that gap is real: an engine signature change broke autorip today and
# went unnoticed because consumer CI only fires on a push to that consumer.
# libfreemkv sits below all five of them, so a break here is worth strictly
# more than a break anywhere else in the project.
#
# `cargo check --all-targets` only — each dependent owns its own behaviour
# and has its own suite. The question here is just "does everything built on
# me still compile against this commit".
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo test --tests
working-directory: freemkv
- run: cargo test --tests
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
leak-guard: leak-guard:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Compute commit range - name: Compute commit range
-133
View File
@@ -1,133 +0,0 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
on:
push:
branches: [qa]
workflow_dispatch:
jobs:
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
working-directory: libfreemkv
+6 -11
View File
@@ -4,11 +4,6 @@ on:
push: push:
tags: tags:
- 'v*' - 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions: permissions:
contents: write contents: write
@@ -17,7 +12,7 @@ jobs:
verify: verify:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
- name: Verify Cargo.toml version matches tag - name: Verify Cargo.toml version matches tag
run: | run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')" CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -29,7 +24,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the # Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already # publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers # gated by the local precommit (same Rust 1.86, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building # (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish # the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path. # below must NOT sit on their critical path.
@@ -37,8 +32,8 @@ jobs:
needs: verify needs: verify
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.97.0 - uses: dtolnay/rust-toolchain@1.86.0
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked # libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner). # would always fail (no lockfile to lock against on a fresh runner).
@@ -56,8 +51,8 @@ jobs:
needs: verify needs: verify
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v5
- name: Create GitHub Release - name: Create GitHub Release
uses: softprops/action-gh-release@v3 uses: softprops/action-gh-release@v2
with: with:
generate_release_notes: true generate_release_notes: true
+36
View File
@@ -0,0 +1,36 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push
-9
View File
@@ -14,12 +14,3 @@ scratch/
# internal agent context — never publish (path AND dir; leak-guard blocks both) # internal agent context — never publish (path AND dir; leak-guard blocks both)
CLAUDE.md CLAUDE.md
.claude/ .claude/
# Nightly harness output. Written into the repo it audits, and it embeds
# absolute paths from the machine that ran it — which must never reach a public
# repo. Ignored rather than relocated so a run from any working copy is safe.
.nightly/
# cargo-mutants working output: large, machine-specific, never committed
mutants.out/
mutants.out.old/
+931 -734
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -36,7 +36,7 @@ This Code of Conduct applies within all community spaces, and also applies when
## Enforcement ## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at matthew@pq.io. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident. All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+14 -9
View File
@@ -1,8 +1,8 @@
[package] [package]
name = "libfreemkv" name = "libfreemkv"
version = "1.6.6" version = "1.6.0"
edition = "2024" edition = "2024"
rust-version = "1.97" rust-version = "1.86"
license = "MIT" license = "MIT"
description = "Open source raw disc access library for optical drives" description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv" repository = "https://github.com/freemkv/libfreemkv"
@@ -22,22 +22,27 @@ codegen-units = 1
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
serde_json = "1" serde_json = "1"
sha1 = "0.10" sha1 = "0.10"
aes = "0.9" sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
# Interim path dep for local cross-repo dev; the release script re-pins this to # Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so # `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path). # the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.5" } freemkv-unlock = { path = "../freemkv-unlock" }
rand = "0.10" num-bigint = "0.4"
zip = { version = "8", default-features = false, features = ["deflate"] } num-traits = "0.2"
base64 = "0.23" num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion # Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL` # bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real # — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied # parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags). # attribute order, self-closing tags).
roxmltree = "0.20" roxmltree = "0.20"
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute # Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# and friends). Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip) # under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log. # wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1" tracing = "0.1"
+42 -9
View File
@@ -55,15 +55,48 @@ output.finish()?;
### Multi-pass recovery rip ### Multi-pass recovery rip
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
ddrescue mapfile, damage classification and the multipass loop now live in the forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`. loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md).
libfreemkv keeps the layers underneath: the raw single-shot read ```rust
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's use libfreemkv::{SweepOptions, PatchOptions};
strategy is built on. The dependency runs engine → libfreemkv, so this crate use libfreemkv::disc::{mapfile, mapfile_path_for};
cannot call into it; front-ends get recovery from the engine directly. See use std::path::Path;
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
## What It Does ## What It Does
@@ -81,7 +114,7 @@ cannot call into it; front-ends get recovery from the engine directly. See
| Stream | Input | Output | Transport | | Stream | Input | Output | Transport |
|--------|-------|--------|-----------| |--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI | | DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) | | IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| MkvStream | Yes | Yes | Matroska container | | MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header | | M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header | | NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
-22
View File
@@ -1,22 +0,0 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
+3 -3
View File
@@ -141,8 +141,8 @@ If your machine has a free SATA port, use it.
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details. freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry. - **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window. - **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block. - **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected. This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
@@ -302,7 +302,7 @@ Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not adver
When something goes wrong during a rip, gather this information before filing an issue: When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`. 1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info disc://`. 2. **Drive model:** from the drive label, or from `freemkv info`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA. 3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`. 4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash. 5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
+8 -15
View File
@@ -22,7 +22,7 @@ fn main() {
&target_arch // x86_64 → x86_64 &target_arch // x86_64 → x86_64
}; };
let cc_status = std::process::Command::new("cc") std::process::Command::new("cc")
.args([ .args([
"-arch", "-arch",
clang_arch, clang_arch,
@@ -38,19 +38,12 @@ fn main() {
"-O2", "-O2",
]) ])
.status() .status()
.expect("failed to spawn cc for macos_shim.c"); .expect("failed to compile macos_shim.c");
// `.status()` succeeding only means the process RAN. A real compile error
// exits non-zero, and ignoring that left no object file, which surfaced
// much later as an unexplained link failure against a missing symbol. The
// shim is macOS-only and is neither linted nor compiled on the other two
// platforms, so a mistake in it has exactly one chance to be noticed.
assert!(cc_status.success(), "cc failed to compile macos_shim.c");
let ar_status = std::process::Command::new("ar") std::process::Command::new("ar")
.args(["rcs", &lib, &obj]) .args(["rcs", &lib, &obj])
.status() .status()
.expect("failed to spawn ar"); .expect("failed to create static lib");
assert!(ar_status.success(), "ar failed to create the static lib");
println!("cargo:rustc-link-search=native={out_dir}"); println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi"); println!("cargo:rustc-link-lib=static=macos_scsi");
@@ -84,10 +77,10 @@ fn emit_git_suffix() {
// Re-run when HEAD (or the branch it points at) moves so the stamp stays // Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild. // current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD"); println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD") if let Ok(head) = std::fs::read_to_string(".git/HEAD") {
&& let Some(ref_path) = head.strip_prefix("ref: ") if let Some(ref_path) = head.strip_prefix("ref: ") {
{ println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim()); }
} }
} }
+1 -1
View File
@@ -12,7 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------| |----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support | | [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed | | [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) | | [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake | | [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc | | [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types | | [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
+16 -27
View File
@@ -65,38 +65,27 @@ if disc.encrypted {
} }
} }
// Read content -- decryption is applied on read by the DiscStream decorator. // Read content -- decryption is automatic
// Live disc does NOT go through the URL resolver: `input("disc://...")` returns let mut reader = disc.open_title(&mut session, 0).unwrap();
// Error::DiscUrlNotDirect by design. while let Some(unit) = reader.read_unit().unwrap() {
let keys = disc.decrypt_keys(); // decrypted content
let mut stream = DiscStream::new(
Box::new(drive),
disc.titles[0].clone(),
keys,
batch_sectors,
disc.titles[0].content_format,
false, // raw: false → decrypt on read
None, // halt
)?;
while let Ok(Some(frame)) = stream.read() {
// decrypted PES frames
} }
``` ```
The application never calls decryption functions and never manages the The application never touches keys, never calls decryption functions, and never
drive-level handshake. It DOES own key resolution — see below. manages handshakes. All of that is internal to `Disc::scan()` and the content
reader.
### Key resolution is the caller's job ### KEYDB Location
`libfreemkv` is **lookup-free: it resolves no keys and reads no keydb.** There is `ScanOptions` controls where the keydb is loaded from. If no explicit path is
no `ScanOptions::with_keydb`, and `ScanOptions` has no keydb field — its only set, the library checks the standard config locations. To specify an explicit
scan input is the optional drive credentials for the live-drive authenticated path:
handshake.
The caller resolves a key out-of-band through a key source and applies it with ```rust
[`Disc::decrypt_with`]. `freemkv-keysources` is the crate that implements the let opts = ScanOptions::with_keydb("/path/to/keydb.cfg");
keydb and key-server sources; `ScanOptions::key_sources` takes them as let disc = Disc::scan(&mut session, &opts).unwrap();
`Box<dyn KeySource>`. ```
### AacsState ### AacsState
@@ -108,7 +97,7 @@ After a successful scan, `disc.aacs` contains an `AacsState`:
| `bus_encryption` | `bool` | Whether bus encryption is active | | `bus_encryption` | `bool` | Whether bus encryption is active |
| `mkb_version` | `Option<u32>` | MKB version from disc | | `mkb_version` | `Option<u32>` | MKB version from disc |
| `disc_hash` | `String` | Identifier for the disc's key-input files | | `disc_hash` | `String` | Identifier for the disc's key-input files |
| `key_source` | `KeyOrigin` | How the disc's key was resolved | | `key_source` | `KeySource` | How the disc's key was resolved |
## keydb.cfg ## keydb.cfg
+7 -10
View File
@@ -170,20 +170,17 @@ libfreemkv/src/
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer) │ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline │ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read) ├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), eject │ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution │ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution
│ ├── linux.rs Linux drive discovery │ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery │ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery │ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, per-format parsing) ├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
│ ├── mod.rs Disc struct, scan, titles, formats │ ├── mod.rs Disc struct, scan, titles, formats; Disc::copy + Disc::sweep (Pass 1)
│ ├── bluray.rs Blu-ray / UHD scanning (MPLS/CLPI-driven) │ ├── sweep.rs Pass 1 internal helpers (pub(super))
│ ├── dvd.rs DVD-Video scanning (IFO-driven) │ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── hddvd.rs HD-DVD scanning │ ├── mapfile.rs ddrescue-format mapfile
── extract.rs Per-extent content extraction ── read_error.rs ReadCtx / ReadAction state machine
│ ├── encrypt.rs Encrypted-range mapping for content reads
│ ├── dvd_audio_probe.rs DVD audio-stream probing
│ └── pgs_forced_probe.rs PGS forced-subtitle probing
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI) ├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── unlock.rs Unlocker trait + registry (pluggable unlock seam) ├── unlock.rs Unlocker trait + registry (pluggable unlock seam)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt) ├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
+2 -3
View File
@@ -96,13 +96,12 @@ After open:
- `init()` -- routes to the matching registered unlocker (if any); otherwise - `init()` -- routes to the matching registered unlocker (if any); otherwise
a no-op and the cert handshake carries the disc a no-op and the cert handshake carries the disc
- `probe_disc()` -- probe disc surface for optimal speeds - `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (`READ_TIMEOUT_MS` 10 s vs. `READ_RECOVERY_TIMEOUT_MS` 60 s) - `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `wait_ready()` -- wait for disc insertion - `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray - `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1 Recovery is layered above `Drive::read`, not inside it. Layer 1
(`freemkv_engine::recovery::patch`, in the engine crate) handles bad-range (`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry) request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
+5 -15
View File
@@ -69,23 +69,13 @@ Each stream PID entry header (14 bytes):
``` ```
Offset Size Field Offset Size Field
------ ---- ----- ------ ---- -----
2 2 stream_PID (byte-aligned) 0 2 Stream PID
4 10 Bit-packed block, 80 bits total (see below) 2 2 Reserved + EP stream type
4 2 Number of coarse entries
6 4 Number of fine entries (note: 32-bit, can be large)
10 4 EP map start offset (relative to EP map start)
``` ```
The stream PID entry is **not** byte-aligned past `stream_PID`. Bytes 4..14 are one
80-bit packed field, read as a `u64` plus a trailing `u16`:
Bits Width Field
---- ----- -----
0-9 10 reserved
10-13 4 EP_stream_type
14-29 16 num_EP_coarse
30-47 18 num_EP_fine
48-79 32 EP_map_start_address (relative to the EP map start)
Note `num_EP_fine` is **18 bits**, not 32. See `parse_cpi` in `src/clpi.rs`.
libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking. libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking.
### Two-Level Index ### Two-Level Index
+2 -2
View File
@@ -79,7 +79,7 @@ Insert disc
│ Or: read sectors → decrypt → raw bytes (for ISO output) │ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the │ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer │ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — freemkv_engine::recovery::patch re-runs against the mapfile. │ 1 above this — Disc::patch re-runs against the mapfile.
PES frames → output stream (MKV, M2TS, network, etc.) PES frames → output stream (MKV, M2TS, network, etc.)
@@ -123,7 +123,7 @@ output.finish()?;
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake | | aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher | | css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) | | decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan (sweep/patch/mapfile moved to freemkv-engine in 1.6.0) | | disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| labels/ | -- | BD-J stream labels (5 format parsers) | | labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) | | mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits | | pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
+3 -4
View File
@@ -58,16 +58,15 @@ selects the per-CDB timeout:
| `recovery` | Timeout | Used by | | `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------| |------------|----------|------------------------------------------|
| `false` | 10 s | `freemkv_engine::recovery::sweep` fast skip-forward pass, `DiscStream::fill_extents` | | `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 60 s | `freemkv_engine::recovery::patch` retry pass over the mapfile | | `true` | 30 s | `Disc::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately. On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation. There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`: Recovery is layered above `Drive::read`:
- **Layer 1 — `freemkv_engine::recovery::patch`** (in the engine crate, not - **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
here) loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range. `read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on - **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read failure, retries at the same LBA, and probes back up on a clean-read
+168 -64
View File
@@ -1,38 +1,136 @@
# Rip recovery — what libfreemkv owns # Rip recovery — three-layer architecture
**Recovery strategy moved OUT of this crate in 1.6.0.** The forward sweep, the `libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
targeted retry pass, the ddrescue mapfile, damage classification and the discs: a fast forward sweep that tolerates read failures, in-loop request-size
multipass loop now live in the **`freemkv-engine`** crate as adaptation that survives transient drive trouble without bailing, and targeted
`freemkv_engine::recovery::{copy, sweep, patch}`. The dependency runs retry passes against a persistent bad-range map. The stream pipeline
engine → libfreemkv, so this crate cannot call into the engine; front-ends (`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
(`freemkv` CLI, autorip) get recovery from the engine directly. the mux stage never touches the drive.
What stayed here are the two layers underneath the strategy: the single-shot Recovery is layered cleanly. Each layer has one responsibility and does not
read primitive, and the in-stream request-size adaptation that sits in front of reach into the others.
it. This document covers those, plus the design constraints they encode — the
constraints are the reason the strategy above them looks the way it does, so
they belong with the code that enforces them.
For the strategy itself — damage-jump thresholds, pass ordering, mapfile status
state machine, wedge detection — read `freemkv-engine/src/recovery/`.
| Layer | Where it lives | What it does | | Layer | Where it lives | What it does |
|-------|---------------|--------------| |-------|---------------|--------------|
| 1 — Bad-range retry | **`freemkv-engine`** (`recovery::patch`) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. | | 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. | | 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` in `src/mux/disc.rs` | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. | | 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
Layer 2 also translates drive facts: [`SenseFamily`](../src/scsi/mod.rs) The library exposes flat verbs; the caller drives the multipass loop. Autorip
classifies SCSI sense data into the categories the engine's strategy routes on runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
(marginal vs. hardware vs. not-ready). Getting that classification wrong clean or the configured retry budget is exhausted, then hands the ISO off to
silently misroutes recovery, which is why it lives next to the transport rather the mux pipeline. The `freemkv` CLI does the same shape, but as of 1.6.0 the
than in the strategy. loop itself (including the multi-title rip loop) lives one layer up, in the
shared `freemkv-engine` crate, with a terminal-output progress sink plugged
into it as the `Sink`. Layer 3 runs inside any consumer of `DiscStream`
(direct PES pipeline, ISO playback, etc.) without caller involvement.
Layer 3 runs inside any consumer of `DiscStream` — direct PES pipeline, ISO Three primitives compose the disc-side flow:
playback — without caller involvement, and applies whether or not the engine's
recovery is in play.
## In-stream — adaptive batch halving (`DiscStream::fill_extents`) | Primitive | What it does |
|---------------------------|-----------------------------------------------------------------------|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
## Data model
### Mapfile
Format: [ddrescue](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)-compatible
plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
so a crashed rip loses at most one block.
```
# Rescue Logfile. Created by libfreemkv v0.13.6
# Current pos / status / pass / pass_time
0x000000000 ? 1 0
# pos size status
0x000000000 0x12a35d000 +
0x12a35d000 0x000003000 -
0x12a360000 0x009c4a000 +
0x12d00a000 0x000064000 *
```
Status characters match ddrescue:
| Char | Meaning |
|------|----------------------------------------------------|
| `?` | Not yet attempted |
| `*` | Fast-pass failed; needs edge-trim |
| `/` | Trimmed; interior needs sector scrape |
| `-` | Unreadable this session |
| `+` | Finished (good) |
Position and size are hex byte offsets into the ISO.
### `SweepOptions` and `PatchOptions`
The library no longer dispatches between sweep and patch internally — the
caller picks the verb explicitly per pass. The two option structs are flat
and have no overlap:
```rust
SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true, // damage-jump + zero-fill on read failure
progress: Some(&reporter),
halt: Some(flag),
}
PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true, // walk bad ranges high → low LBA
wedged_threshold: 50,
progress: Some(&reporter),
halt: Some(flag),
}
```
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
- No mapfile → `sweep` (fresh Pass 1).
- Mapfile with `?` ranges → `sweep` with `resume: true`.
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
- Mapfile clean → done; no further pass needed.
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
lines of `Mapfile::stats()` checks.
## Algorithm
### Pass 1 — fast sweep (`Disc::sweep`)
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
**damage-jump**: skip ahead by `1024×batch×multiplier` sectors (64 MB base for
UHD). Double the multiplier on each jump (64→128→256→512 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
When a consumer reads a `DiscStream` directly (no ISO intermediate), When a consumer reads a `DiscStream` directly (no ISO intermediate),
`fill_extents` runs an adaptive sizer in front of `Drive::read`: `fill_extents` runs an adaptive sizer in front of `Drive::read`:
@@ -46,53 +144,59 @@ When a consumer reads a `DiscStream` directly (no ISO intermediate),
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return `EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
`Err(DiscRead)`. `Err(DiscRead)`.
This exists so a transient single-sector glitch inside a 32-sector batch can be This is layer 3. It exists so a transient single-sector glitch in a 32-sector
isolated and read individually without the caller implementing retry logic. See batch can be isolated and read individually without the caller needing to
[`src/event.rs`](../src/event.rs) for the emitted events. implement retry logic.
## Design choices ## Design choices
These are constraints on the read path, enforced here and relied on by the **`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
engine's strategy. no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
handles ranges, layer 3 handles request size, neither touches the
wedge-prone reset path.
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset, no **No `MODE SELECT` to disable drive retries.** Neither ddrescue
eject cycle. The `recovery` flag controls only the per-CDB timeout (10 s vs. nor any consumer ripper does this. Drive firmware has access to raw analog signal, laser
60 s); on any failure it returns `Err(DiscRead)` immediately. Inline recovery power control, and drive-specific ECC tuning that userspace can't replicate —
(5× gentle retry → close + SCSI reset + reopen → 5× more) was removed in disabling it throws away recovery headroom on marginal sectors. We fail fast
0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale: the inline via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
reset on the LG BU40N (Initio USB-SATA bridge) wedged drive firmware below the in pass 2 / patch.
bridge without ever recovering a sector, and the gentle-retry phase produced
long stretches of 0 KB/s with nothing to show for it. Recovery responsibility
is layered instead: layer 1 handles ranges, layer 3 handles request size,
neither touches the wedge-prone reset path.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue nor any **No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
consumer ripper does this. Drive firmware has access to raw analog signal, trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
laser power control and drive-specific ECC tuning that userspace cannot `SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
replicate — disabling it throws away recovery headroom on marginal sectors. The The macOS reset (which had been a no-op) was removed entirely. The top-level
fast pass fails quickly via short SG_IO timeouts and lets the firmware work the `scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
long timeout during retry. also removed (no callers). The remaining `Drive::reset()` is only invoked
explicitly by callers that need an eject-cycle escape hatch — it is never
reached from a read path.
**No SCSI reset from any read path.** There is no reset escape hatch on **ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
`Drive` at all: the `SG_SCSI_RESET` ioctl and STOP/START UNIT escalation went in mux stage reads the ISO via `FileSectorSource`. For single-pass (no retries),
0.13.6, the macOS reset (always a no-op) was removed entirely, and the this adds ~2-3 min (local disk mux) but gains resumability across crashes,
top-level `scsi::reset()` wrappers went with their last callers. The only
remaining reset is a Windows-specific device-level helper in
[`src/scsi/windows.rs`](../src/scsi/windows.rs), never reached from a read.
**ISO intermediate, even for single-pass.** The engine's Pass 1 always writes
an ISO, and the mux stage reads it back via `FileSectorSource`. For a
no-retry rip this costs a few minutes but buys resumability across crashes,
re-muxability without re-ripping, and a persistent forensic artifact. Callers re-muxability without re-ripping, and a persistent forensic artifact. Callers
who need pure speed can bypass it with `DiscStream::new(Box::new(drive), …)` who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
nothing forbids it, and layer 3 still applies there. directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
still applies there.
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
coalesce on adjacent same-status ranges so files stay small.
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
ddrescue's but `patch` collapses the distinction — it just tries every
non-finished range with the long timeout. Future work can specialize (trim vs.
scrape vs. retry with direction reversal) if there's measured benefit.
## References ## References
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html) - [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html) - [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
- Recovery strategy and mapfile: `freemkv-engine/src/recovery/` - Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
- In this crate: [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`),
[`src/scsi/mod.rs`](../src/scsi/mod.rs) (`SenseFamily`),
[`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`),
[`src/event.rs`](../src/event.rs) (progress events).
+1 -1
View File
@@ -114,7 +114,7 @@ The `read_filesystem()` function in `src/udf.rs` follows the pointer chain above
2. Scans sectors 32-63 for the Partition Descriptor and Logical Volume Descriptor. 2. Scans sectors 32-63 for the Partition Descriptor and Logical Volume Descriptor.
3. If two partition maps exist and the second is Type 2, reads the metadata file ICB at partition_start to find metadata_start. 3. If two partition maps exist and the second is Type 2, reads the metadata file ICB at partition_start to find metadata_start.
4. Reads the FSD at metadata_start, extracts the root directory ICB LBA. 4. Reads the FSD at metadata_start, extracts the root directory ICB LBA.
5. Calls `read_directory()` recursively (max depth `MAX_DIR_DEPTH` = 8) to build the full file tree. 5. Calls `read_directory()` recursively (max depth 3) to build the full file tree.
Each directory read involves two sector reads: one for the ICB, then one or more for the directory data. File sizes are read from info_length in each file's ICB. Each directory read involves two sector reads: one for the ICB, then one or more for the directory data. File sizes are read from info_length in each file's ICB.
+83
View File
@@ -0,0 +1,83 @@
// Minimal ISO dumper — find exact stall point
use libfreemkv::Drive;
use std::io::{BufWriter, Write};
use std::path::Path;
use std::time::Instant;
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("Usage: iso_dump <device> <output>");
std::process::exit(1);
}
let mut drive = Drive::open(Path::new(&args[1])).unwrap();
drive.wait_ready().unwrap();
let _ = drive.init();
let _ = drive.probe_disc();
// AACS handshake — required to read past the protected area
eprint!("Scanning disc... ");
let _ = libfreemkv::Disc::scan(&mut drive, &libfreemkv::ScanOptions::default());
eprintln!("OK");
let cap = drive.read_capacity().unwrap();
let batch = libfreemkv::disc::detect_max_batch_sectors(drive.device_path());
eprintln!("Device: {} | {} sectors | batch {}", args[1], cap, batch);
let file = std::fs::File::create(&args[2]).unwrap();
let mut w = BufWriter::with_capacity(4 * 1024 * 1024, file);
let mut buf = vec![0u8; batch as usize * 2048];
let mut lba: u32 = 0;
let start = Instant::now();
let mut last = Instant::now();
let mut bytes: u64 = 0;
let mut last_bytes: u64 = 0;
while lba < cap {
let count = ((cap - lba) as u16).min(batch);
let n = count as usize * 2048;
// Tiny yield between reads — test if pacing prevents firmware throttle
std::thread::yield_now();
let t0 = Instant::now();
let ok = drive.read(lba, count, &mut buf[..n], true).is_ok();
let read_ms = t0.elapsed().as_millis();
// Flag slow reads
if read_ms > 2000 {
eprintln!("\n SLOW READ: LBA {} took {}ms (ok={})", lba, read_ms, ok);
}
if !ok {
buf[..n].fill(0);
}
w.write_all(&buf[..n]).unwrap();
lba += count as u32;
bytes += n as u64;
if last.elapsed().as_millis() >= 1000 {
let delta = bytes - last_bytes;
let speed = delta as f64 / last.elapsed().as_secs_f64() / 1_048_576.0;
let avg = bytes as f64 / start.elapsed().as_secs_f64() / 1_048_576.0;
let pct = bytes as f64 / (cap as f64 * 2048.0) * 100.0;
eprint!(
"\r {:.1}% LBA {} | {:.0} MB/s (avg {:.0}) | {:.1} GB ",
pct,
lba,
speed,
avg,
bytes as f64 / 1e9
);
last_bytes = bytes;
last = Instant::now();
}
}
w.flush().unwrap();
eprintln!(
"\nDone: {:.1} GB in {:.0}s",
bytes as f64 / 1e9,
start.elapsed().as_secs_f64()
);
}
+69 -372
View File
@@ -4,11 +4,11 @@
#[cfg(test)] #[cfg(test)]
use aes::Aes128; use aes::Aes128;
#[cfg(test)] #[cfg(test)]
use aes::cipher::{Array, KeyInit}; use aes::cipher::{KeyInit, generic_array::GenericArray};
use super::crypto::{aes_cbc_decrypt, aes_cbc_encrypt, aes_ecb_encrypt}; use super::crypto::{aes_cbc_decrypt, aes_ecb_encrypt};
// Only this module's test fixtures build CBC ciphertext by hand now — the // Available at module scope for this module's test fixtures (they reference
// production paths get the IV from `aes_cbc_encrypt` / `aes_cbc_decrypt`. // `super::AACS_IV` when building CBC ciphertext directly); test-only.
#[cfg(test)] #[cfg(test)]
use super::crypto::AACS_IV; use super::crypto::AACS_IV;
@@ -41,8 +41,7 @@ pub const ALIGNED_UNIT_SECTORS: u32 = (ALIGNED_UNIT_LEN / SECTOR_BYTES) as u32;
/// underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`, mis-reports the /// underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`, mis-reports the
/// alignment (e.g. `lba == unit_base - 1` would falsely read as aligned). /// alignment (e.g. `lba == unit_base - 1` would falsely read as aligned).
pub fn is_unit_aligned(lba: u32, unit_base: u32) -> bool { pub fn is_unit_aligned(lba: u32, unit_base: u32) -> bool {
lba.saturating_sub(unit_base) lba.saturating_sub(unit_base) % ALIGNED_UNIT_SECTORS == 0
.is_multiple_of(ALIGNED_UNIT_SECTORS)
} }
use crate::consts::SECTOR_BYTES; use crate::consts::SECTOR_BYTES;
@@ -147,14 +146,8 @@ pub fn aacs_unit_needs_decrypt(unit: &[u8], format: crate::disc::ContentFormat)
} }
/// Minimum synced content packets that PROVE a key opened a unit. Four `0x47` /// Minimum synced content packets that PROVE a key opened a unit. Four `0x47`
/// syncs are 32 bits of MPEG-TS structure, but the per-UNIT false-pass risk is /// syncs 32 bits of MPEG-TS structure ≈ 1-in-4-billion that a wrong key (uniform
/// NOT 2^-32: `is_clean_ts` accepts ANY four of the ~31 encrypted packets in a /// AES noise, `0x47` at 1/256 per packet) fakes it. It is an ABSOLUTE proof floor,
/// 6144-byte aligned unit, so for a wrong key (uniform AES noise, `0x47` at 1/256
/// per packet) it is ≈ C(31,4)·256^-4 ≈ 7e-6, i.e. ~1e-5 — the figure
/// [`is_clean_ts`]'s own doc below states. 1-in-4-billion is the probability for
/// four SPECIFIC packets and overstates the margin by ~4000x; at
/// `KEY_PROOF_PACKETS = 3` the per-unit rate is ≈ C(31,3)·256^-3 ≈ 2.6e-4, so do
/// NOT lower it on the strength of slack that is not there. It is an ABSOLUTE proof floor,
/// NOT a proportion — a unit the key opened but whose content is bad-encoded /// NOT a proportion — a unit the key opened but whose content is bad-encoded
/// (many non-conforming packets) is proven by ANY four good packets, not rejected /// (many non-conforming packets) is proven by ANY four good packets, not rejected
/// for the bad ones. /// for the bad ones.
@@ -178,16 +171,8 @@ pub fn is_clean(unit: &[u8], format: crate::disc::ContentFormat) -> bool {
/// Structural "does this unit carry enough valid MPEG-TS to prove a key opened /// Structural "does this unit carry enough valid MPEG-TS to prove a key opened
/// it?" — the Transport-Stream arm of [`is_clean`]. It is /// it?" — the Transport-Stream arm of [`is_clean`]. It is
/// NOT a decryption verdict: [`decrypt_unit`] applies a key (that is /// NOT a decryption verdict: [`decrypt_unit`] applies a key (that is
/// "decrypt"); whether the plaintext is clean TS is this SEPARATE question. /// "decrypt"); whether the plaintext is clean TS is this SEPARATE question. The
/// /// mux never calls this — TS validity is a muxer concern, never a decrypt result.
/// The mux is its PRINCIPAL consumer for `BdTs` discs, reaching it through
/// [`is_clean`]: `mux::resolve`'s multi-CPS `pick` closure selects a unit key by
/// it, `probe_index_phase` reports each FMTS index's interleave parity by it, and
/// `decrypt::decrypt_sectors_mapped` uses it as the forensic-range verify net.
/// (The doc used to say "the mux never calls this", which invited a maintainer to
/// tighten or loosen the proof rule below believing only whole-disc read
/// verification was affected — while it in fact changes which unit key a
/// multi-CPS disc muxes with and which phase an FMTS index is muxed at.)
/// ///
/// Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the /// Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the
/// ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed, so /// ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed, so
@@ -341,43 +326,19 @@ pub fn decrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) {
} }
} }
/// Encrypt one AACS aligned unit (6144 bytes) IN PLACE — the exact inverse of /// Test-only inverse of [`decrypt_unit`]: encrypt a clear aligned unit under
/// [`decrypt_unit`], for authoring an encrypted disc image (and for building /// `unit_key` and set the CPI-encrypted flag (top 2 bits of byte 0) so the unit
/// genuinely-encrypted read-path fixtures). /// reads as encrypted under [`aacs_unit_encrypted`]. Exposed `pub(crate)` for
/// /// cross-module mux tests (the mux `driver.rs` builds a genuinely-AACS-encrypted
/// PURE, on the same terms as `decrypt_unit`: it applies the key and nothing else. /// fixture to prove the live/session decrypt path installs its key map). Uses
/// It does NOT set the encrypted flag, because where that flag lives is /// only the module-scope primitives so it stays in lock-step with `decrypt_unit`.
/// container-specific (CPI bits in byte 0 for BD-TS, elsewhere for HD-DVD-PS) and #[cfg(test)]
/// keeping it out is what lets this stay container-agnostic. The only guard is the pub(crate) fn aacs_encrypt_unit_for_test(unit: &mut [u8], unit_key: &[u8; 16]) {
/// length check, since the crypto is defined only over a whole 6144-byte unit.
///
/// **Set the encrypted flag BEFORE calling, never after.** Bytes 0..16 are the key
/// seed and are left in plaintext, so the Block Key derives from them: mutating any
/// header byte after encrypting changes the seed a decryptor will derive from and
/// silently yields garbage. The caller's order must be flag, then encrypt.
///
/// Block Key = AES-128E(Kcu, seed) ⊕ seed, then AES-128-CBC **encrypt** bytes
/// 16..6144 under the AACS IV — the forward direction of the same construction
/// `decrypt_unit` documents, sharing its module-scope primitives so the two cannot
/// drift apart.
///
/// Note one deliberate asymmetry: `decrypt_unit` restores all-zero-on-disc source
/// padding packets to zero. This does not, and need not — an all-zero plaintext
/// packet encrypts to ciphertext that is not all-zero, so it is not mistaken for
/// padding on the way back and the round trip is still exact. Authoring that wants
/// true source-zero padding leaves those packets unencrypted instead.
///
/// Returns `false` — encrypting nothing — when `unit` is shorter than
/// [`ALIGNED_UNIT_LEN`]. That case MUST be checked: the caller has already set the
/// container's encrypted flag by then (this function's contract requires it), so
/// ignoring the result leaves a unit marked encrypted while still carrying
/// plaintext, which is the worst possible outcome for an authoring tool.
#[must_use = "returns false when the slice is too short to encrypt, leaving \
plaintext behind a flag that already says 'encrypted'"]
pub fn encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) -> bool {
if unit.len() < ALIGNED_UNIT_LEN { if unit.len() < ALIGNED_UNIT_LEN {
return false; return;
} }
// Set CPI bits BEFORE key derivation so the recovered plaintext header matches.
unit[0] |= 0xC0;
let mut header = [0u8; 16]; let mut header = [0u8; 16];
header.copy_from_slice(&unit[..16]); header.copy_from_slice(&unit[..16]);
let derived = aes_ecb_encrypt(unit_key, &header); let derived = aes_ecb_encrypt(unit_key, &header);
@@ -385,30 +346,31 @@ pub fn encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) -> bool {
for i in 0..16 { for i in 0..16 {
k[i] = derived[i] ^ header[i]; k[i] = derived[i] ^ header[i];
} }
// CBC-encrypt bytes 16.. under the fixed AACS IV — the exact forward of the // CBC-encrypt bytes 16.. under the fixed AACS IV (forward of `aes_cbc_decrypt`).
// `aes_cbc_decrypt` call in `decrypt_unit`, and one key expansion for the whole let mut prev = AACS_IV;
// unit rather than one per 16-byte block. let num_blocks = (ALIGNED_UNIT_LEN - 16) / 16;
aes_cbc_encrypt(&k, &mut unit[16..ALIGNED_UNIT_LEN]); for i in 0..num_blocks {
true let off = 16 + i * 16;
let mut block = [0u8; 16];
for j in 0..16 {
block[j] = unit[off + j] ^ prev[j];
}
let enc = aes_ecb_encrypt(&k, &block);
unit[off..off + 16].copy_from_slice(&enc);
prev.copy_from_slice(&enc);
}
} }
/// Remove bus encryption from an aligned unit (AACS 2.0 / UHD). /// Remove bus encryption from an aligned unit (AACS 2.0 / UHD).
/// Bus encryption uses read_data_key, decrypting bytes 16..2048 of each 2048-byte sector. /// Bus encryption uses read_data_key, decrypting bytes 16..2048 of each 2048-byte sector.
pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) { pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
// Expand the key schedule ONCE for the whole unit. `read_data_key` is
// loop-invariant here (and constant for the entire disc), but calling
// `aes_cbc_decrypt` per sector rebuilt the AES-128 schedule per sector — three
// expansions per 6144-byte aligned unit, i.e. ~29 million redundant expansions
// over a 90 GB read on a stock drive, on the per-unit decrypt hot path.
// Measured by `decrypt_bus_expands_the_read_data_key_once_per_unit`.
let cipher = crate::aacs::crypto::new_cipher_for(read_data_key);
for sector_start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) { for sector_start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
if sector_start + SECTOR_BYTES > unit.len() { if sector_start + SECTOR_BYTES > unit.len() {
break; break;
} }
// First 16 bytes of each sector are plaintext // First 16 bytes of each sector are plaintext
crate::aacs::crypto::cbc_decrypt_blocks( aes_cbc_decrypt(
&cipher, read_data_key,
&mut unit[sector_start + 16..sector_start + SECTOR_BYTES], &mut unit[sector_start + 16..sector_start + SECTOR_BYTES],
); );
} }
@@ -418,105 +380,7 @@ pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
mod tests { mod tests {
use super::super::crypto::aes_ecb_decrypt; use super::super::crypto::aes_ecb_decrypt;
use super::*; use super::*;
use aes::cipher::BlockCipherEncrypt; // test fixtures build ciphertext directly use aes::cipher::BlockEncrypt; // test fixtures build ciphertext directly
/// [`encrypt_unit`] is the exact inverse of [`decrypt_unit`]: whatever an
/// authoring caller encrypts, the read path must recover byte-for-byte.
///
/// Mutation: drop the trailing `⊕ header` from either function's Block Key
/// derivation, or swap `AACS_IV` for zeroes in one of them, and the two stop
/// agreeing -> this fails.
#[test]
fn encrypt_unit_is_the_exact_inverse_of_decrypt_unit() {
let key = [0x3Cu8; 16];
// Content with no all-zero packets: every byte position exercised.
let mut clear: Vec<u8> = (0..ALIGNED_UNIT_LEN)
.map(|i| (i * 7 % 251 + 1) as u8)
.collect();
// The encrypted flag belongs to the caller and must be set BEFORE the
// crypto, since bytes 0..16 are the key seed.
clear[0] |= 0xC0;
let mut unit = clear.clone();
assert!(
encrypt_unit(&mut unit, &key),
"a full-length unit must encrypt"
);
assert_ne!(
unit[16..],
clear[16..],
"the payload must actually be enciphered"
);
assert_eq!(
unit[..16],
clear[..16],
"the 16-byte seed stays plaintext on disc"
);
decrypt_unit(&mut unit, &key);
assert_eq!(unit, clear, "round trip must be byte-exact");
}
/// The documented padding asymmetry actually holds: `decrypt_unit` restores
/// all-zero-ON-DISC packets to zero, but an all-zero PLAINTEXT packet enciphers
/// to non-zero bytes, so it is not mistaken for padding and still round-trips.
/// This is the one place the two functions are deliberately not symmetric, so
/// the claim is worth pinning rather than asserting in prose alone.
/// A slice too short to encrypt must SAY so. The caller has already set the
/// container's encrypted flag by the time it calls this (the contract requires
/// flag-before-crypto, since the header is the key seed), so a silent no-op
/// leaves a unit advertised as encrypted while still carrying plaintext.
#[test]
fn encrypt_unit_reports_a_slice_too_short_to_encrypt() {
let key = [0x11u8; 16];
let mut short = vec![0u8; ALIGNED_UNIT_LEN - 1];
short[0] |= 0xC0; // the caller already flagged it encrypted
let before = short.clone();
assert!(
!encrypt_unit(&mut short, &key),
"a short slice must report false, not silently succeed"
);
assert_eq!(
short, before,
"a refused encrypt must leave the buffer untouched"
);
// Exactly ALIGNED_UNIT_LEN is the boundary and must succeed.
let mut exact = vec![0u8; ALIGNED_UNIT_LEN];
exact[0] |= 0xC0;
assert!(encrypt_unit(&mut exact, &key), "a full unit must encrypt");
}
#[test]
fn encrypt_unit_round_trips_all_zero_plaintext_packets() {
let key = [0xA5u8; 16];
let mut clear = vec![0u8; ALIGNED_UNIT_LEN];
clear[0] |= 0xC0; // flag before crypto
// Give packet 0 some content; leave every later packet entirely zero.
for (i, b) in clear[16..192].iter_mut().enumerate() {
*b = (i % 255 + 1) as u8;
}
let mut unit = clear.clone();
assert!(
encrypt_unit(&mut unit, &key),
"a full-length unit must encrypt"
);
// No later packet may encipher to all-zero, or decrypt would treat it as
// source padding and the asymmetry would bite.
for p in 1..ALIGNED_UNIT_LEN / BD_SOURCE_PACKET_BYTES {
let off = p * BD_SOURCE_PACKET_BYTES;
assert!(
unit[off..off + BD_SOURCE_PACKET_BYTES]
.iter()
.any(|&b| b != 0),
"packet {p} enciphered to all-zero, which decrypt reads as padding"
);
}
decrypt_unit(&mut unit, &key);
assert_eq!(unit, clear, "zero-payload packets must round trip exactly");
}
#[test] #[test]
fn test_aes_ecb_roundtrip() { fn test_aes_ecb_roundtrip() {
@@ -686,11 +550,23 @@ mod tests {
let original = vec![0x42u8; 128]; // 8 blocks let original = vec![0x42u8; 128]; // 8 blocks
let mut data = original.clone(); let mut data = original.clone();
// Encrypt with the REAL production primitive. This test previously // Encrypt with CBC manually (forward direction)
// defined a local `fn aes_cbc_encrypt` that SHADOWED it, so it round- fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
// tripped a copy of the algorithm against itself and never exercised let cipher = Aes128::new(GenericArray::from_slice(key));
// `crypto::aes_cbc_encrypt` at all — a mutation to the shipped function let mut prev = super::AACS_IV;
// could not fail it. let num_blocks = data.len() / 16;
for i in 0..num_blocks {
let offset = i * 16;
for j in 0..16 {
data[offset + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&data[offset..offset + 16]);
cipher.encrypt_block(&mut block);
data[offset..offset + 16].copy_from_slice(&block);
prev.copy_from_slice(&data[offset..offset + 16]);
}
}
aes_cbc_encrypt(&key, &mut data); aes_cbc_encrypt(&key, &mut data);
assert_ne!(data, original); // should be different after encrypt assert_ne!(data, original); // should be different after encrypt
@@ -725,7 +601,7 @@ mod tests {
} }
// CBC encrypt bytes 16..6143 // CBC encrypt bytes 16..6143
let cipher = Aes128::new(&encrypt_key.into()); let cipher = Aes128::new(GenericArray::from_slice(&encrypt_key));
let mut prev = AACS_IV; let mut prev = AACS_IV;
let num_blocks = (ALIGNED_UNIT_LEN - 16) / 16; let num_blocks = (ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..num_blocks { for i in 0..num_blocks {
@@ -733,9 +609,7 @@ mod tests {
for j in 0..16 { for j in 0..16 {
plain[off + j] ^= prev[j]; plain[off + j] ^= prev[j];
} }
let mut chunk = [0u8; 16]; let mut block = GenericArray::clone_from_slice(&plain[off..off + 16]);
chunk.copy_from_slice(&plain[off..off + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.encrypt_block(&mut block); cipher.encrypt_block(&mut block);
plain[off..off + 16].copy_from_slice(&block); plain[off..off + 16].copy_from_slice(&block);
prev.copy_from_slice(&plain[off..off + 16]); prev.copy_from_slice(&plain[off..off + 16]);
@@ -778,11 +652,7 @@ mod tests {
fn aacs_encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) { fn aacs_encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) {
// Delegate to the module-scope `pub(crate)` helper (the single encrypt // Delegate to the module-scope `pub(crate)` helper (the single encrypt
// implementation, shared with the mux `driver.rs` decrypt test). // implementation, shared with the mux `driver.rs` decrypt test).
unit[0] |= 0xC0; super::aacs_encrypt_unit_for_test(unit, unit_key);
assert!(
super::encrypt_unit(unit, unit_key),
"a full-length unit must encrypt"
);
} }
/// Build a clear aligned unit with TS sync bytes at offset 4 + k*192. /// Build a clear aligned unit with TS sync bytes at offset 4 + k*192.
@@ -877,7 +747,7 @@ mod tests {
// ── Defect-tolerant "did a key OPEN this unit?" verdict ───────────────── // ── Defect-tolerant "did a key OPEN this unit?" verdict ─────────────────
// //
// The authored-bad-packet bug: a commercial disc carries the odd bad TS // The Bourne-UHD bug: a commercial disc carries the odd authored-bad TS
// packet (a pressing/encoding defect, or an AACS 2.1 forensic-variant frame) // packet (a pressing/encoding defect, or an AACS 2.1 forensic-variant frame)
// — one non-conforming packet inside an otherwise perfectly-decrypted 6144 // — one non-conforming packet inside an otherwise perfectly-decrypted 6144
// unit. The OLD strict per-packet acceptance rejected the WHOLE unit over // unit. The OLD strict per-packet acceptance rejected the WHOLE unit over
@@ -1205,100 +1075,21 @@ mod tests {
assert_eq!(aes_ecb_decrypt(&key, &expected), pt); assert_eq!(aes_ecb_decrypt(&key, &expected), pt);
} }
// ── decrypt_bus: one key schedule per unit, not one per sector ─────────
/// MEASURED, not reasoned: `decrypt_bus` called `aes_cbc_decrypt` once per
/// 2048-byte sector, and each call built its own AES-128 key schedule, so a
/// 6144-byte aligned unit performed THREE key expansions under the same
/// loop-invariant `read_data_key`. On a 90 GB UHD read on a stock (non-
/// LibreDrive) drive — ~14.6 million aligned units — that is ~29 million
/// redundant expansions on the per-unit decrypt hot path, for a key that is
/// constant for the whole disc. The counter is incremented inside
/// `crypto::new_cipher`, the single construction site.
#[test]
fn decrypt_bus_expands_the_read_data_key_once_per_unit() {
use crate::aacs::crypto::KEY_EXPANSIONS;
let mut unit = clear_unit();
let rdk = [0x4Eu8; 16];
KEY_EXPANSIONS.with(|c| c.set(0));
decrypt_bus(&mut unit, &rdk);
let n = KEY_EXPANSIONS.with(|c| c.get());
assert_eq!(
n, 1,
"one aligned unit under one read_data_key must expand the schedule \
exactly once, not once per 2048-byte sector"
);
}
/// The single-expansion refactor must be byte-identical: bus encryption
/// ([C] §4.2) covers bytes 16..2048 of every 2048-byte sector, so a
/// three-sector aligned unit round-trips through the forward direction
/// sector by sector and `decrypt_bus` must recover it exactly.
#[test]
fn decrypt_bus_roundtrips_every_sector_region() {
let rdk = [0x91u8; 16];
let original = clear_unit();
let mut unit = original.clone();
// Forward direction, region by region — the inverse of decrypt_bus.
for start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
crate::aacs::crypto::aes_cbc_encrypt(&rdk, &mut unit[start + 16..start + SECTOR_BYTES]);
}
assert_ne!(
&unit[16..64],
&original[16..64],
"the forward direction must have changed the bytes"
);
decrypt_bus(&mut unit, &rdk);
assert_eq!(
unit.as_slice(),
original.as_slice(),
"decrypt_bus must invert the per-sector bus encryption exactly"
);
}
// ── CBC decrypt: first-block uses fixed AACS IV ──────────────────────── // ── CBC decrypt: first-block uses fixed AACS IV ────────────────────────
/// The published `iv0` bytes, INDEPENDENT of the production constant.
///
/// [C] §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation.
/// Both IV tests below used to compute their expected value from
/// `crypto::AACS_IV` itself, so the constant was asserted against itself and
/// NOTHING in the suite pinned its bytes: swapping `AACS_IV` for `[0u8; 16]`
/// left both tests passing (one builds its ciphertext with the same value and
/// the other cancels the change in a triple XOR) while every real AACS disc
/// decrypted to noise — block 0 of every 6128-byte aligned unit and of every
/// bus-encrypted sector XORed with the wrong IV. This literal is the
/// independent witness the tests assert against.
const IV0_PUBLISHED: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F,
0x78,
];
/// Pins the fixed AACS CBC IV ([C] §2.1.2 `iv0`) against a literal, so a
/// change to `crypto::AACS_IV` fails HERE rather than silently shipping.
#[test]
fn aacs_iv_matches_published_iv0() {
assert_eq!(
AACS_IV, IV0_PUBLISHED,
"the fixed AACS CBC IV must be the published iv0"
);
}
#[test] #[test]
fn cbc_decrypt_first_block_xors_aacs_iv() { fn cbc_decrypt_first_block_xors_aacs_iv() {
// CBC: P[0] = AES-D(K, C[0]) XOR IV, and the IV is the fixed AACS // CBC: P[0] = AES-D(K, C[0]) XOR IV, and the IV is the fixed AACS
// constant (not zero). Encrypt a single block forward with the PUBLISHED // constant (not zero). Encrypt a single block forward with IV, then
// iv0 literal, then confirm aes_cbc_decrypt recovers it — proving the IV // confirm aes_cbc_decrypt recovers it — proving the IV used on block
// the production code uses on block 0 is exactly that value. Building the // 0 is exactly AACS_IV. A mutation that swaps AACS_IV for [0u8;16]
// fixture from `IV0_PUBLISHED` rather than from `AACS_IV` is what makes // makes the recovered block wrong.
// the claim real: a mutation that swaps AACS_IV for [0u8;16] now makes the
// recovered block wrong.
let key = [0x24u8; 16]; let key = [0x24u8; 16];
let plain = [0x5Au8; 16]; let plain = [0x5Au8; 16];
// Forward CBC for one block: C = AES-E(K, P XOR IV). // Forward CBC for one block: C = AES-E(K, P XOR IV).
let mut x = plain; let mut x = plain;
for j in 0..16 { for j in 0..16 {
x[j] ^= IV0_PUBLISHED[j]; x[j] ^= AACS_IV[j];
} }
let ct = aes_ecb_encrypt(&key, &x); let ct = aes_ecb_encrypt(&key, &x);
let mut buf = ct; let mut buf = ct;
@@ -1327,13 +1118,10 @@ mod tests {
// * Blocks 1..=3 are independent of the IV — they MUST equal the NIST // * Blocks 1..=3 are independent of the IV — they MUST equal the NIST
// plaintext byte-for-byte (P[i] = AES-D(K, C[i]) XOR C[i-1]). This // plaintext byte-for-byte (P[i] = AES-D(K, C[i]) XOR C[i-1]). This
// pins the real reverse-order CBC chaining against a published KAT. // pins the real reverse-order CBC chaining against a published KAT.
// * Block 0 = AES-D(K, C[0]) XOR iv0 = NIST_PT[0] XOR NIST_IV XOR iv0 — // * Block 0 = AES-D(K, C[0]) XOR AACS_IV = NIST_PT[0] XOR NIST_IV
// the documented IV substitution. Asserting this exact relation pins // XOR AACS_IV — the documented IV substitution. Asserting this exact
// both the AES decrypt of C[0] AND that block 0 uses iv0. The expected // relation pins both the AES decrypt of C[0] AND that block 0 uses
// value is built from the `IV0_PUBLISHED` literal, NOT from // AACS_IV (a swap to [0u8;16] or a chaining bug fails it).
// `crypto::AACS_IV`: computing it from the production constant made
// the change cancel out of the triple XOR, so a swap to [0u8;16] still
// passed. It now fails.
let key = [ let key = [
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF, 0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C, 0x4F, 0x3C,
@@ -1373,7 +1161,7 @@ mod tests {
// Block 0: NIST_PT[0] XOR NIST_IV XOR AACS_IV (the fixed-IV substitution). // Block 0: NIST_PT[0] XOR NIST_IV XOR AACS_IV (the fixed-IV substitution).
let mut expected_block0 = [0u8; 16]; let mut expected_block0 = [0u8; 16];
for i in 0..16 { for i in 0..16 {
expected_block0[i] = nist_plaintext[i] ^ nist_iv[i] ^ IV0_PUBLISHED[i]; expected_block0[i] = nist_plaintext[i] ^ nist_iv[i] ^ AACS_IV[i];
} }
assert_eq!( assert_eq!(
&buf[0..16], &buf[0..16],
@@ -1498,7 +1286,7 @@ mod tests {
let plain = unit.clone(); let plain = unit.clone();
// Forward: CBC-encrypt unit[s+16 .. s+2048] per sector under AACS IV. // Forward: CBC-encrypt unit[s+16 .. s+2048] per sector under AACS IV.
let cipher = Aes128::new(&rdk.into()); let cipher = Aes128::new(GenericArray::from_slice(&rdk));
for s in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) { for s in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
let mut prev = AACS_IV; let mut prev = AACS_IV;
let body = s + 16; let body = s + 16;
@@ -1509,9 +1297,7 @@ mod tests {
for j in 0..16 { for j in 0..16 {
unit[off + j] ^= prev[j]; unit[off + j] ^= prev[j];
} }
let mut chunk = [0u8; 16]; let mut blk = GenericArray::clone_from_slice(&unit[off..off + 16]);
chunk.copy_from_slice(&unit[off..off + 16]);
let mut blk: Array<u8, _> = chunk.into();
cipher.encrypt_block(&mut blk); cipher.encrypt_block(&mut blk);
unit[off..off + 16].copy_from_slice(&blk); unit[off..off + 16].copy_from_slice(&blk);
prev.copy_from_slice(&unit[off..off + 16]); prev.copy_from_slice(&unit[off..off + 16]);
@@ -1568,95 +1354,6 @@ mod tests {
assert_eq!(ts_sync_count(&unit), 1); assert_eq!(ts_sync_count(&unit), 1);
} }
// ── the encrypted-flag readers ────────────────────────────────────────
/// `aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the
/// guard that stops a truncated encrypted fragment from being emitted as
/// clear content. It reads ONLY the two Copy Permission Indicator bits
/// ([BD] §3.10.2, byte 0 bits 6-7); the remaining six bits are
/// `TP_extra_header` arrival-timestamp bits and carry no encryption
/// meaning.
///
/// Both failure directions are damaging and silent: a reader that answers
/// "encrypted" for a clear fragment discards good content, and one that
/// answers "clear" for an encrypted fragment writes ciphertext into the
/// output as if it were video.
#[test]
fn aacs_unit_seed_encrypted_reads_only_the_two_cpi_bits() {
use crate::disc::ContentFormat::BdTs;
// CPI bits clear → NOT encrypted, whatever the ATS bits say.
for ats in 0u8..=0x3F {
assert!(
!aacs_unit_seed_encrypted(&[ats], BdTs),
"byte0={ats:#04x} has both CPI bits clear → not encrypted"
);
}
// Either CPI bit set → encrypted, whatever the ATS bits say.
for &cpi in &[0x40u8, 0x80, 0xC0] {
assert!(
aacs_unit_seed_encrypted(&[cpi], BdTs),
"byte0={cpi:#04x} has a CPI bit set → encrypted"
);
assert!(
aacs_unit_seed_encrypted(&[cpi | 0x3F], BdTs),
"ATS bits must not change the answer"
);
}
// Too short to hold the flag → false rather than a panic.
assert!(!aacs_unit_seed_encrypted(&[], BdTs));
}
/// The MpegPs (HD-DVD `.evo`) side reads `PES_scrambling_control` at its own
/// fixed offset, and a fragment shorter than that offset must be reported
/// clear rather than panic.
#[test]
fn aacs_unit_seed_encrypted_reads_the_ps_scramble_flag_or_says_clear() {
use crate::disc::ContentFormat::MpegPs;
let mut frag = vec![0u8; PS_SCRAMBLE_OFF + 1];
assert!(!aacs_unit_seed_encrypted(&frag, MpegPs), "flag byte zero");
frag[PS_SCRAMBLE_OFF] = PS_SCRAMBLE_MASK;
assert!(aacs_unit_seed_encrypted(&frag, MpegPs), "flag byte set");
// Bits outside the mask are not the scrambling control.
frag[PS_SCRAMBLE_OFF] = !PS_SCRAMBLE_MASK;
assert!(!aacs_unit_seed_encrypted(&frag, MpegPs), "outside the mask");
// A fragment that stops short of the flag byte is not classifiable.
assert!(!aacs_unit_seed_encrypted(&frag[..PS_SCRAMBLE_OFF], MpegPs));
}
/// `aacs_unit_encrypted` is the AUTHORITATIVE gate and requires a WHOLE
/// 6144-byte aligned unit: on anything shorter the flag byte is not
/// guaranteed to be the unit's, so it must answer `false` and leave the
/// partial-unit case to `aacs_unit_seed_encrypted`. A reversed length guard
/// would both classify fragments off arbitrary mid-stream bytes and, on an
/// empty slice, index out of bounds.
#[test]
fn aacs_unit_encrypted_requires_a_whole_aligned_unit() {
use crate::disc::ContentFormat::BdTs;
// A short buffer whose byte 0 has the CPI bits set is still NOT a unit.
let mut short = vec![0u8; ALIGNED_UNIT_LEN - 1];
short[0] = 0xC0;
assert!(
!aacs_unit_encrypted(&short, BdTs),
"a sub-unit buffer must not be classified"
);
assert!(!aacs_unit_encrypted(&[], BdTs), "empty must not index");
// Exactly one aligned unit IS classified.
let mut unit = vec![0u8; ALIGNED_UNIT_LEN];
unit[0] = 0xC0;
assert!(
aacs_unit_encrypted(&unit, BdTs),
"a full unit with CPI set is encrypted"
);
unit[0] = 0x00;
assert!(!aacs_unit_encrypted(&unit, BdTs), "CPI clear is not");
}
#[test] #[test]
fn ts_packet_total_for_various_lengths() { fn ts_packet_total_for_various_lengths() {
// total = len / 192 (BD-TS packet size). Pin a few lengths. // total = len / 192 (BD-TS packet size). Pin a few lengths.
+9 -175
View File
@@ -8,44 +8,17 @@
//! content / keys / variant modules. //! content / keys / variant modules.
use aes::Aes128; use aes::Aes128;
use aes::cipher::{Array, BlockCipherDecrypt, BlockCipherEncrypt, KeyInit}; use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit, generic_array::GenericArray};
/// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`). /// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`).
pub(crate) const AACS_IV: [u8; 16] = [ pub(crate) const AACS_IV: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78, 0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78,
]; ];
// Per-thread count of AES-128 key schedules built through `new_cipher`.
// Test-only instrumentation: an AES-128 key expansion is 10 round-key
// derivations, and the CBC helpers here run on the per-aligned-unit decrypt hot
// path of a whole disc read, so "how many times was the schedule built for one
// loop-invariant key" is a property worth asserting rather than reasoning about.
// THREAD-LOCAL, not a global atomic: `cargo test` runs tests concurrently, so a
// shared counter would see every other test's expansions. See
// `content::tests::decrypt_bus_expands_the_read_data_key_once_per_unit`.
#[cfg(test)]
thread_local! {
pub(crate) static KEY_EXPANSIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Build an AES-128 key schedule for a caller that will drive
/// [`cbc_decrypt_blocks`] over several regions under one key.
pub(crate) fn new_cipher_for(key: &[u8; 16]) -> Aes128 {
new_cipher(key)
}
/// Build an AES-128 key schedule. The single construction site for the CBC
/// helpers, so [`KEY_EXPANSIONS`] can count them under test.
fn new_cipher(key: &[u8; 16]) -> Aes128 {
#[cfg(test)]
KEY_EXPANSIONS.with(|c| c.set(c.get() + 1));
Aes128::new(&(*key).into())
}
/// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`). /// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`).
pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] { pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(&(*key).into()); let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block: Array<u8, _> = (*data).into(); let mut block = GenericArray::clone_from_slice(data);
cipher.encrypt_block(&mut block); cipher.encrypt_block(&mut block);
let mut out = [0u8; 16]; let mut out = [0u8; 16];
out.copy_from_slice(&block); out.copy_from_slice(&block);
@@ -54,81 +27,25 @@ pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
/// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`). /// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`).
pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] { pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(&(*key).into()); let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block: Array<u8, _> = (*data).into(); let mut block = GenericArray::clone_from_slice(data);
cipher.decrypt_block(&mut block); cipher.decrypt_block(&mut block);
let mut out = [0u8; 16]; let mut out = [0u8; 16];
out.copy_from_slice(&block); out.copy_from_slice(&block);
out out
} }
/// AES-128-CBC ENCRYPT in place under the fixed [`AACS_IV`] — the forward /// AES-128-CBC decrypt in-place with the fixed AACS IV. [C] §2.1.2 (`AES-128CBCD`).
/// direction of [`aes_cbc_decrypt`], and its exact inverse. [C] §2.1.2
/// (`AES-128CBCE`).
///
/// Precondition: `data.len()` is a multiple of 16; the assert
/// documents/enforces that contract.
///
/// Constructs the cipher ONCE for the whole slice. Driving this from the
/// single-block [`aes_ecb_encrypt`] instead rebuilds the AES key schedule per
/// 16-byte block, which for a 6144-byte aligned unit is 383 redundant key
/// expansions.
pub(crate) fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len().is_multiple_of(16),
"aes_cbc_encrypt requires a block-aligned slice"
);
let cipher = new_cipher(key);
let num_blocks = data.len() / 16;
let mut prev = AACS_IV;
// Forward order: each block is XORed with the PRECEDING ciphertext block.
for i in 0..num_blocks {
let offset = i * 16;
let mut block = [0u8; 16];
for j in 0..16 {
block[j] = data[offset + j] ^ prev[j];
}
let mut ga: Array<u8, _> = block.into();
cipher.encrypt_block(&mut ga);
data[offset..offset + 16].copy_from_slice(&ga);
prev.copy_from_slice(&ga);
}
}
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. [C] §2.1.2
/// (`AES-128CBCD`).
/// ///
/// Precondition: `data.len()` is a multiple of 16. Any trailing partial /// Precondition: `data.len()` is a multiple of 16. Any trailing partial
/// block is silently ignored; all callers pass aligned regions (6128 and /// block is silently ignored; all callers pass aligned regions (6128 and
/// 2032 bytes), and the assert documents/enforces that contract. /// 2032 bytes), and the assert documents/enforces that contract.
///
/// (This doc block was orphaned onto `aes_cbc_encrypt` above when that function
/// was inserted directly after it with no separating blank line, so rustdoc
/// rendered the crate's only forward-direction AACS primitive as "decrypt" and
/// cited the spec's DECRYPT clause for it, while this function had no doc at
/// all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in `content.rs` pins
/// the directions behaviourally so a maintainer 'fixing' the contradiction by
/// swapping the two bodies fails the suite instead of shipping a second
/// decryptor behind an already-set encrypted flag.)
pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) { pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!( debug_assert!(
data.len().is_multiple_of(16), data.len() % 16 == 0,
"aes_cbc_decrypt requires a block-aligned slice" "aes_cbc_decrypt requires a block-aligned slice"
); );
cbc_decrypt_blocks(&new_cipher(key), data); let cipher = Aes128::new(GenericArray::from_slice(key));
}
/// AES-128-CBC decrypt in place under the fixed [`AACS_IV`] with an ALREADY
/// EXPANDED key schedule.
///
/// Split out of [`aes_cbc_decrypt`] so a caller that decrypts several regions
/// under one loop-invariant key expands the schedule once. `decrypt_bus`
/// ([`super::content::decrypt_bus`]) is that caller: bus encryption
/// ([C] §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// 2048-byte sector, so a 6144-byte aligned unit is three regions under one
/// `read_data_key` — three key schedules where one suffices, on the per-unit
/// decrypt hot path of a whole 90 GB read.
pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
let num_blocks = data.len() / 16; let num_blocks = data.len() / 16;
// Process blocks in reverse to avoid clobbering ciphertext needed for XOR // Process blocks in reverse to avoid clobbering ciphertext needed for XOR
for i in (0..num_blocks).rev() { for i in (0..num_blocks).rev() {
@@ -140,9 +57,7 @@ pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
p.copy_from_slice(&data[(i - 1) * 16..i * 16]); p.copy_from_slice(&data[(i - 1) * 16..i * 16]);
p p
}; };
let mut chunk = [0u8; 16]; let mut block = GenericArray::clone_from_slice(&data[offset..offset + 16]);
chunk.copy_from_slice(&data[offset..offset + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.decrypt_block(&mut block); cipher.decrypt_block(&mut block);
for j in 0..16 { for j in 0..16 {
data[offset + j] = block[j] ^ prev[j]; data[offset + j] = block[j] ^ prev[j];
@@ -185,84 +100,3 @@ pub(crate) fn aesg3(key: &[u8; 16], inc: u8) -> [u8; 16] {
} }
out out
} }
#[cfg(test)]
mod tests {
use super::*;
/// The AACS-G3 seed `s0`, transcribed independently from [C] §3.2.2 rather
/// than read from [`AESG3_SEED`] — a test that sourced the seed from the
/// production constant would assert that constant against itself and would
/// still pass if it were edited.
const S0: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B,
0xD9,
];
/// An arbitrary non-degenerate key. Nothing about it is secret or special;
/// the AES-G3 relation holds for every key, and a constant-returning body
/// cannot satisfy it for any.
const K: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2, 0xE1,
0xF0,
];
/// `aesg3` is the node function of the AACS subset-difference tree: every
/// Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every
/// descent step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that
/// returned a fixed block would make every device key in the crate derive
/// the SAME Processing Key, and a `^` that became `|` or `&` would derive a
/// wrong-but-plausible one — in both cases the MKB walk simply stops
/// finding Media Keys, with no error to say why.
///
/// Pinned through the spec relation rather than a re-implementation:
/// [C] §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `s = s0 + inc` (added into the last seed byte), so applying the
/// FORWARD primitive [`aes_ecb_encrypt`] — a different function from the
/// one under test — to `aesg3(k, inc) XOR s` must reproduce `s` exactly.
#[test]
fn aesg3_inverts_to_the_spec_seed_under_aes_encrypt() {
for inc in 0u8..=2 {
let mut seed = S0;
seed[15] = seed[15].wrapping_add(inc);
let out = aesg3(&K, inc);
// out == AES-128D(K, seed) XOR seed, so out XOR seed is the raw
// decryption and re-encrypting it must land back on the seed.
let mut pre = [0u8; 16];
for i in 0..16 {
pre[i] = out[i] ^ seed[i];
}
assert_eq!(
aes_ecb_encrypt(&K, &pre),
seed,
"AES-G3 inc={inc} must satisfy out = AES-128D(k, s0+inc) XOR (s0+inc)"
);
}
}
/// The Triple Generator's three outputs ([C] §3.2.2: left = inc 0, the
/// Processing Key = inc 1, right = inc 2) are the two child node keys and
/// the Processing Key of ONE tree node. They must be three different keys —
/// if `inc` were ignored, a descent would revisit its own parent and the
/// walk would derive the same key at every level of the tree.
#[test]
fn aesg3_yields_three_distinct_subkeys_for_the_three_increments() {
let left = aesg3(&K, 0);
let pk = aesg3(&K, 1);
let right = aesg3(&K, 2);
assert_ne!(left, pk, "left child and Processing Key must differ");
assert_ne!(pk, right, "Processing Key and right child must differ");
assert_ne!(left, right, "left and right children must differ");
}
/// Distinct parent keys must yield distinct subkeys — the tree would
/// collapse otherwise.
#[test]
fn aesg3_separates_distinct_parent_keys() {
let mut other = K;
other[0] ^= 0x01;
assert_ne!(aesg3(&K, 1), aesg3(&other, 1));
}
}
-1044
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -189,7 +189,7 @@ mod tests {
let unit_packets = (ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32; // 32 let unit_packets = (ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32; // 32
// Start so the unit covers [80, 80+31] = [80, 111]: overlaps at 100. // Start so the unit covers [80, 80+31] = [80, 111]: overlaps at 100.
let off = 80u64 * SOURCE_PACKET_LEN; let off = 80u64 * SOURCE_PACKET_LEN;
assert!(80 + unit_packets > 100, "sanity: unit tails into seg"); assert!(80 + unit_packets - 1 >= 100, "sanity: unit tails into seg");
assert_eq!( assert_eq!(
unit_disposition(off, &segs, Some(5)), unit_disposition(off, &segs, Some(5)),
UnitDisposition::Index(5) UnitDisposition::Index(5)
+1 -382
View File
@@ -5,6 +5,7 @@
use super::mkb::*; use super::mkb::*;
/// Parsed Unit_Key_RO.inf file. /// Parsed Unit_Key_RO.inf file.
#[derive(Debug)]
pub struct UnitKeyFile { pub struct UnitKeyFile {
/// Disc hash (SHA1 of the entire file) — used as KEYDB lookup key /// Disc hash (SHA1 of the entire file) — used as KEYDB lookup key
pub disc_hash: [u8; 20], pub disc_hash: [u8; 20],
@@ -22,29 +23,6 @@ pub struct UnitKeyFile {
pub title_cps_unit: Vec<u16>, pub title_cps_unit: Vec<u16>,
} }
/// Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
/// the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
/// — plus the disc hash they are looked up by. A derived `Debug` printed every key
/// byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
/// message, a future `tracing::debug!` in this module) leaked them. Only
/// non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
impl std::fmt::Debug for UnitKeyFile {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKeyFile")
// The disc hash is the public keydb lookup key, printed as hex the
// same way `DiscEntry` prints its own — never as raw bytes.
.field("disc_hash", &disc_hash_hex(&self.disc_hash))
.field("app_type", &self.app_type)
.field("num_bdmv_dir", &self.num_bdmv_dir)
.field("use_skb_mkb", &self.use_skb_mkb)
.field("version", &self.version)
.field("encrypted_keys", &"<redacted>")
.field("encrypted_keys_len", &self.encrypted_keys.len())
.field("title_cps_unit", &self.title_cps_unit)
.finish()
}
}
/// Compute disc hash (SHA1 of Unit_Key_RO.inf content). /// Compute disc hash (SHA1 of Unit_Key_RO.inf content).
pub fn disc_hash(data: &[u8]) -> [u8; 20] { pub fn disc_hash(data: &[u8]) -> [u8; 20] {
use sha1::{Digest, Sha1}; use sha1::{Digest, Sha1};
@@ -519,363 +497,4 @@ mod vtkf_tests {
// Same as applying the shared unwrap directly to the stored enc key. // Same as applying the shared unwrap directly to the stored enc key.
assert_eq!(derived, super::super::derive::decrypt_unit_key(&vuk, &enc)); assert_eq!(derived, super::super::derive::decrypt_unit_key(&vuk, &enc));
} }
/// `UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
/// printed every byte; the hand-written impl must not. Sentinel key byte
/// 0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
/// same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
/// `#[derive(Debug)]` back fails this.
#[test]
fn unit_key_file_debug_is_redacted() {
let f = UnitKeyFile {
disc_hash: [0xD5; 20],
app_type: 1,
num_bdmv_dir: 1,
use_skb_mkb: false,
version: AacsVersion::V20,
encrypted_keys: vec![(0, [0xD5; 16]), (1, [0xD5; 16])],
title_cps_unit: vec![0, 1],
};
let dbg = format!("{f:?}");
assert!(
!dbg.contains("213"),
"UnitKeyFile Debug leaked key bytes (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"UnitKeyFile Debug missing redaction marker: {dbg}"
);
// Non-secret shape is still useful for diagnostics.
assert!(dbg.contains("encrypted_keys_len: 2"), "{dbg}");
}
}
#[cfg(test)]
mod read_mkb_tests {
use super::*;
use crate::scsi::{DataDirection, SCSI_READ_DISC_STRUCTURE, ScsiResult, ScsiTransport};
/// A drive that answers READ DISC STRUCTURE format 0x83 from a scripted set
/// of packs and records every CDB it was handed.
struct MkbDrive {
/// One entry per pack: the pack's MKB payload bytes.
packs: Vec<Vec<u8>>,
cdbs: Vec<Vec<u8>>,
}
impl ScsiTransport for MkbDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
self.cdbs.push(cdb.to_vec());
// Pack number is carried in the CDB address field (bytes 2..6),
// MMC-6 READ DISC STRUCTURE.
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]) as usize;
let body = self.packs.get(pack).cloned().unwrap_or_default();
// Header: BE16 data length (counts the 2 header bytes that follow
// it plus the payload), reserved byte, pack count, then payload.
let data_len = body.len() + 2;
data[0..2].copy_from_slice(&(data_len as u16).to_be_bytes());
data[2] = 0x00;
data[3] = self.packs.len() as u8;
data[4..4 + body.len()].copy_from_slice(&body);
Ok(ScsiResult {
status: 0,
bytes_transferred: 4 + body.len(),
sense: [0u8; 32],
})
}
}
/// `read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
/// downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
/// Media Key ladder) consumes exactly what it returns. An empty return is
/// not a benign "no MKB" — it is a total read failure reported as success,
/// and every derivation then fails with a key-not-found code that points
/// the operator at their keydb rather than at the drive.
///
/// This pins the CONTENT: the concatenated payload of all packs, in pack
/// order, byte for byte.
#[test]
fn read_mkb_from_drive_returns_the_concatenated_pack_payload() {
let pack0: Vec<u8> = (0..600u32).map(|i| (i % 251) as u8).collect();
let pack1: Vec<u8> = (0..300u32).map(|i| (i % 253) as u8 ^ 0xA5).collect();
let mut drive = MkbDrive {
packs: vec![pack0.clone(), pack1.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
let mut expected = pack0.clone();
expected.extend_from_slice(&pack1);
assert_eq!(
mkb.len(),
expected.len(),
"every pack's payload must be concatenated, none dropped"
);
assert!(
mkb == expected,
"MKB bytes must be the drive's payload in pack order; first \
mismatch at {:?}",
(0..expected.len()).find(|&i| mkb[i] != expected[i])
);
// MMC-6 READ DISC STRUCTURE with the AACS MKB format code, one command
// per pack, pack number in the address field.
assert_eq!(drive.cdbs.len(), 2, "one command per declared pack");
for (i, cdb) in drive.cdbs.iter().enumerate() {
assert_eq!(cdb[0], SCSI_READ_DISC_STRUCTURE, "opcode");
assert_eq!(cdb[7], 0x83, "AACS MKB disc-structure format code");
assert_eq!(
u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]),
i as u32,
"pack {i} must be requested by number"
);
}
}
/// The CDB is what the drive actually acts on, and every byte of it is
/// load-bearing: a wrong format code returns a different disc structure
/// entirely, and a wrong allocation length truncates the pack. The existing
/// test above pins the opcode, the format code and the pack number; this
/// pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
///
/// Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
/// `[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
/// (BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
/// BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
#[test]
fn read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack() {
let mut drive = MkbDrive {
packs: vec![vec![0x11u8; 64], vec![0x22u8; 64], vec![0x33u8; 64]],
cdbs: Vec::new(),
};
read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(drive.cdbs.len(), 3, "one command per declared pack");
for (pack, cdb) in drive.cdbs.iter().enumerate() {
let p = pack as u32;
let expected: [u8; 12] = [
SCSI_READ_DISC_STRUCTURE,
0x01,
(p >> 24) as u8,
(p >> 16) as u8,
(p >> 8) as u8,
p as u8,
0x00,
0x83, // AACS MKB disc-structure format
0x80, // allocation length 32772 = 0x8004, high byte
0x04, // …low byte
0x00,
0x00,
];
assert_eq!(
cdb.as_slice(),
&expected[..],
"CDB for pack {pack} must match the MMC-6 READ DISC STRUCTURE layout"
);
}
}
/// A pack payload filling the FULL 32768-byte window must come back whole.
/// The `len > 0 && len <= 32768` bound is what stands between a maximal
/// pack and a silently dropped one, and the small payloads used elsewhere
/// in this module never reach it.
#[test]
fn read_mkb_from_drive_accepts_a_full_size_pack() {
let full: Vec<u8> = (0..32768u32).map(|i| (i % 251) as u8).collect();
let other: Vec<u8> = (0..32768u32).map(|i| (i % 241) as u8 ^ 0x5A).collect();
// TWO maximal packs: the first-pack read and the per-pack loop carry
// separate bounds, so both must accept a full-window payload.
let mut drive = MkbDrive {
packs: vec![full.clone(), other.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
65536,
"neither maximal pack may be dropped at the size bound"
);
let mut expected = full.clone();
expected.extend_from_slice(&other);
assert!(mkb == expected, "both maximal packs' bytes must be intact");
}
/// A pack that declares only the 2-byte header and NO payload contributes
/// nothing, and must not push a phantom byte into the MKB — an off-by-one
/// at the zero-length boundary corrupts every following pack's alignment.
#[test]
fn read_mkb_from_drive_zero_length_pack_contributes_nothing() {
let mut drive = MkbDrive {
packs: vec![Vec::new(), vec![0xABu8; 32]],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
32,
"an empty pack adds no bytes; only pack 1's payload is present"
);
assert!(mkb == vec![0xABu8; 32], "and the bytes are pack 1's");
}
/// A drive that DECLARES more payload than it returned must not be
/// believed. The BE16 length in the response header is drive-supplied data:
/// a firmware bug, a short transfer, or a hostile device can put a value in
/// it that runs past the 32772-byte buffer. Copying `len` bytes on that word
/// alone panics the rip thread mid-scan.
///
/// Both the first-pack read and the per-pack loop carry the same bound, so
/// both are exercised here: the over-declared pack contributes nothing and
/// the honest pack still comes through.
#[test]
fn read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds() {
/// Pack 0 is honest; pack 1 declares a 60000-byte payload it never sent.
struct LyingDrive {
honest: Vec<u8>,
}
impl ScsiTransport for LyingDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]);
data[3] = 2; // two packs declared
if pack == 0 {
let dl = self.honest.len() + 2;
data[0..2].copy_from_slice(&(dl as u16).to_be_bytes());
data[4..4 + self.honest.len()].copy_from_slice(&self.honest);
} else {
// A length far beyond the 32772-byte response buffer.
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
}
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let honest = vec![0xC7u8; 256];
let mut drive = LyingDrive {
honest: honest.clone(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("an over-declared pack is not an error");
assert_eq!(
mkb.len(),
honest.len(),
"only the honest pack's bytes may be taken; the over-declared pack \
contributes nothing and must not be read past the buffer"
);
assert!(mkb == honest, "and those bytes are pack 0's");
}
/// The same over-declaration on the FIRST pack, which uses a separate bound
/// from the loop's.
#[test]
fn read_mkb_from_drive_ignores_a_first_pack_declaring_more_than_the_buffer() {
struct LyingFirst;
impl ScsiTransport for LyingFirst {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
data[3] = 1;
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut LyingFirst).expect("not an error");
assert!(
mkb.is_empty(),
"a first pack declaring more than the buffer holds yields no bytes"
);
}
/// A single-pack disc still yields that pack's bytes — the common case, and
/// the one where a body returning an empty vector looks most plausible.
#[test]
fn read_mkb_from_drive_returns_a_single_packs_payload() {
let pack: Vec<u8> = (0..1024u32).map(|i| (i * 7 % 256) as u8).collect();
let mut drive = MkbDrive {
packs: vec![pack.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(mkb.len(), pack.len(), "single pack payload length");
assert!(mkb == pack, "single pack payload bytes");
}
/// A drive that reports a header-only response (`data_len < 2`) has no MKB
/// to give. That must be an EMPTY vec, not a partial one — the distinction
/// matters because the AACS paths treat a non-empty MKB as parseable.
#[test]
fn read_mkb_from_drive_empty_response_is_empty() {
struct NoMkb;
impl ScsiTransport for NoMkb {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&0u16.to_be_bytes());
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut NoMkb).expect("no-MKB drive still returns Ok");
assert!(
mkb.is_empty(),
"a header-only response carries no MKB bytes"
);
}
/// A transport failure on the FIRST pack must propagate as an error — the
/// MKB is the root of the whole AACS ladder, so an unreadable one cannot be
/// downgraded to "an MKB with no records".
#[test]
fn read_mkb_from_drive_propagates_the_first_pack_failure() {
struct DeadDrive;
impl ScsiTransport for DeadDrive {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
Err(crate::error::Error::ScsiError {
opcode: SCSI_READ_DISC_STRUCTURE,
status: 0x02,
sense: None,
})
}
}
assert!(
read_mkb_from_drive(&mut DeadDrive).is_err(),
"an unreadable MKB must surface as an error, not an empty MKB"
);
}
} }
-105
View File
@@ -432,109 +432,4 @@ mod tests {
"trim keeps the framed records, dropping the end marker and padding" "trim keeps the framed records, dropping the end marker and padding"
); );
} }
// ── BE24 length field: all THREE bytes ────────────────────────────────
/// The record length is a big-endian **24-bit** field, so the high byte
/// carries lengths of 64 KiB and up. The MKB records that matter most are
/// exactly that size — a real UHD cvalue table is `46_101 * 16` bytes and a
/// `0x2d` variant record is ~92 KiB — so a walker that dropped the high
/// byte would mis-frame every record of a real MKB from the first big one
/// onward, and every downstream key lookup would read the wrong bytes.
///
/// (The pre-existing high-byte test used total length `0x0110`, whose high
/// byte is ZERO — it exercised the middle byte only. This one puts a
/// non-zero value in the high byte.)
#[test]
fn mkb_records_honors_the_high_byte_of_the_be24_length() {
const TOTAL: usize = 0x0001_0004; // 65_540 — high byte 0x01
let mut mkb = vec![REC_VKD_TABLE, 0x01, 0x00, 0x04];
mkb.resize(TOTAL, 0xAB);
// A second record follows, so a walker that mis-read the length would
// frame a different number of records rather than merely a short one.
mkb.extend(rec(REC_TYPE_AND_VERSION, &[0x11; 8]));
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the big record must be framed as ONE record");
assert_eq!(
recs[0].rec_len, TOTAL,
"rec_len must include the high BE24 byte"
);
assert_eq!(recs[0].body.len(), TOTAL - 4);
assert_eq!(
recs[1].rec_type, REC_TYPE_AND_VERSION,
"the following record must start where the big one ends"
);
}
// ── Header-only records and the exact end marker ──────────────────────
/// `rec_len == 4` is a well-formed HEADER-ONLY record (the minimum the
/// walker accepts), including one sitting at the very end of the buffer
/// with no bytes after it. Rejecting either — the `pos + 4` bound or the
/// `rec_len < 4` floor being off by one — silently drops the MKB's last
/// record, and "the record isn't there" is indistinguishable from "the disc
/// doesn't carry it".
#[test]
fn mkb_records_yields_a_header_only_record_at_the_buffer_end() {
let mut mkb = rec(REC_TYPE_AND_VERSION, &[0xAA, 0xBB]);
mkb.extend([REC_VKD_TABLE, 0x00, 0x00, 0x04]); // 4-byte, empty body, at EOF
assert_eq!(
mkb.len(),
10,
"sanity: the last record ends at the buffer end"
);
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the trailing header-only record is a record");
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].rec_len, 4);
assert!(recs[1].body.is_empty());
}
/// ONLY the exact `00 00 00 00` marker ends the walk. A record whose TYPE
/// happens to be `0x00` but which declares a real length is a record, not
/// the end of the MKB — stopping there would truncate everything after it,
/// including the cvalue and verify records the key derivation needs.
#[test]
fn mkb_records_stops_only_on_the_all_zero_end_marker() {
// A type-0 record of length 8, then a normal record, then the marker.
let mut mkb = vec![0x00, 0x00, 0x00, 0x08, 1, 2, 3, 4];
mkb.extend(rec(REC_VKD_TABLE, &[0x55; 16]));
mkb.extend([0x00, 0x00, 0x00, 0x00]); // the real end marker
mkb.extend(rec(0x99, &[0xFF; 4])); // past the marker: not walked
let recs = walk_mkb(&mkb);
assert_eq!(
recs.len(),
2,
"a type-0 record with a non-zero length is a record, not the end"
);
assert_eq!(recs[0].rec_type, 0x00);
assert_eq!(recs[0].rec_len, 8);
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].body, vec![0x55; 16]);
}
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim ([C] §3.2.5.1.1
/// Table 3-2), including a value this build does not recognise — the caller
/// uses it to tell "unknown MKB generation" from "no Type record at all".
/// All four bytes must come from the record body; reading any of them from
/// the wrong offset yields a type that silently classifies as a different
/// AACS generation.
///
/// The recognised constants all share bytes with the `0x10` record-type
/// header byte (e.g. `MKB_21_CATEGORY_C` is `48 15 10 03`), so this uses a
/// value with four distinct bytes, none of them `0x10`.
#[test]
fn mkb_type_raw_reads_all_four_body_bytes() {
const RAW: u32 = 0xDEAD_BEEF;
let mkb = type_and_version(RAW, 7);
assert_eq!(
mkb_type_raw(&mkb),
Some(RAW),
"every byte of the MKBType field must come from the record body"
);
assert_eq!(mkb_version(&mkb), Some(7));
}
} }
-54
View File
@@ -313,60 +313,6 @@ mod tests {
); );
} }
/// The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required —
/// the discovery is a conjunction, not a disjunction.
///
/// The existing fixtures only ever present a directory that satisfies both
/// (`AAC!` with `MKBROM.AACS`) alongside one that satisfies neither
/// (`AAC!_BAK` — which contains `MKBROM.AACS` but is ALSO reached only after
/// the real dir), so either half of the conjunction could be dropped and the
/// same directory would still be found. Here a directory satisfies the name
/// half and NOT the contents half: it must not be picked.
///
/// If it were, the HD DVD path would resolve `MKBROM.AACS`,
/// `CONTENT_CERT.AACS` and the title-key file under a directory that holds
/// none of them — the disc reports "no AACS key files" and never rips.
#[test]
fn a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
// Ends in '!' — but carries no MKBROM.AACS, so it is not the
// HD DVD AACS directory.
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTKF090.AACS", 102, 5200, 2048, true),
file("CONTENT_CERT.AACS", 103, 5300, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
super::find_hddvd_aacs_dir(&udf).is_none(),
"a '!' directory without MKBROM.AACS is not the AACS directory"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
],
"no HD DVD candidates may be appended from a directory that was \
never identified as the AACS directory"
);
}
#[test] #[test]
fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() { fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() {
use crate::udf::fixture::*; use crate::udf::fixture::*;
-47
View File
@@ -197,17 +197,6 @@ mod tests {
} }
} }
/// A host cert whose (non-secret) certificate body and private key are both
/// filled with `byte`, so a cert is identifiable in an aggregated list.
fn cert(byte: u8) -> HostCert {
HostCert {
private_key: [byte; 20],
certificate: vec![byte; 92],
private_key_v2: None,
certificate_v2: None,
}
}
fn dk(byte: u8, node: u16) -> DeviceKey { fn dk(byte: u8, node: u16) -> DeviceKey {
DeviceKey { DeviceKey {
key: [byte; 16], key: [byte; 16],
@@ -368,42 +357,6 @@ mod tests {
assert_eq!(got.disc_hash, "vid-a"); assert_eq!(got.disc_hash, "vid-a");
} }
/// `Providers::host_certs` is the union across the provider array. It is not
/// wired into the handshake today (see the module docs), so nothing else in
/// the crate would notice a body that dropped every cert on the floor — and
/// the day it IS wired in, a silently-empty cert list means the drive AACS
/// authentication finds no host certificate to present and every disc fails
/// to open, with no indication that the caller's certs were discarded.
///
/// Unlike the bulk key unions this one does NOT dedup (HostCert is not
/// Ord/Hash), so the assertion is on the full concatenation in array order.
#[test]
fn providers_host_certs_unions_every_providers_certs_in_array_order() {
struct Certs(Vec<HostCert>);
impl KeyProvider for Certs {
fn host_certs(&self) -> Vec<HostCert> {
self.0.clone()
}
}
// Distinguish certs by their (non-secret) certificate body, so the
// assertion lands on WHICH certs came back, not merely how many.
let a = Certs(vec![cert(0xA1), cert(0xA2)]);
let b = Certs(vec![cert(0xB1)]);
let arr: &[&dyn KeyProvider] = &[&a, &b];
let got = Providers(arr).host_certs();
let bodies: Vec<Vec<u8>> = got.iter().map(|c| c.certificate.clone()).collect();
assert_eq!(
bodies,
vec![vec![0xA1u8; 92], vec![0xA2u8; 92], vec![0xB1u8; 92]],
"every provider's certs must survive the union, in array order"
);
// The private key travels with the cert — a union that returned default
// certs would still have the right count.
assert_eq!(got[0].private_key, [0xA1u8; 20]);
assert_eq!(got[2].private_key, [0xB1u8; 20]);
}
#[test] #[test]
fn providers_empty_array_yields_nothing() { fn providers_empty_array_yields_nothing() {
let arr: &[&dyn KeyProvider] = &[]; let arr: &[&dyn KeyProvider] = &[];
+16 -171
View File
@@ -379,18 +379,26 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
// One AES-D + magic check per candidate (cheap). mk_dv is hoisted // One AES-D + magic check per candidate (cheap). mk_dv is hoisted
// out of the loop so the MKB is not re-walked per candidate. // out of the loop so the MKB is not re-walked per candidate.
let mks = providers.media_keys(); let mks = providers.media_keys();
let chosen_mk = mkb_find_mk_dv(mkb).and_then(|mk_dv| { let mut mk_hits: Vec<[u8; 16]> = Vec::new();
unique_verifying_mk(&mks, |mk| { if let Some(mk_dv) = mkb_find_mk_dv(mkb) {
aes_ecb_decrypt(mk, &mk_dv)[..8] == MK_VERIFY_MAGIC for mk in &mks {
}) let verifies = aes_ecb_decrypt(mk, &mk_dv)[..8]
}); == [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
if let Some(mk) = chosen_mk { if verifies && !mk_hits.contains(mk) {
let vuk = derive_vuk(&mk, ctx.volume_id); mk_hits.push(*mk);
if mk_hits.len() > 1 {
break; // ambiguous — bail to avoid a wrong key
}
}
}
}
if mk_hits.len() == 1 {
let vuk = derive_vuk(&mk_hits[0], ctx.volume_id);
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_hit", mk_pool = mks.len(), "media key from keydb MK-pool brute (km_verifies)"); tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_hit", mk_pool = mks.len(), "media key from keydb MK-pool brute (km_verifies)");
// Same class as path 3 (KEYDB MK → derived VUK). // Same class as path 3 (KEYDB MK → derived VUK).
return Some(build(Some(vuk), derive_uks(&vuk), 3)); return Some(build(Some(vuk), derive_uks(&vuk), 3));
} }
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_miss", mk_pool = mks.len(), "MK-pool brute: no unique verifying MK"); tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_miss", mk_pool = mks.len(), mk_hits = mk_hits.len(), "MK-pool brute: no unique verifying MK");
} else { } else {
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_no_mkb", "no MKB; paths 1/2 skipped"); tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_no_mkb", "no MKB; paths 1/2 skipped");
} }
@@ -442,42 +450,6 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
None None
} }
/// First 8 bytes of the plaintext behind an MKB Verify Media Key record — the
/// AACS "this is the right Km" sentinel (`0123456789ABCDEF`). A candidate MK
/// verifies when AES-128-ECB-D(mk, mk_dv) starts with it.
const MK_VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
/// The MK-pool selection rule of path 2.5, split out of [`resolve_keys_v1`] so
/// the ambiguity guard has a reachable test.
///
/// `verifies` is the MKB check — in production
/// `AES-D(mk, mk_dv)[..8] == MK_VERIFY_MAGIC`. Returns a Media Key only when
/// EXACTLY ONE DISTINCT candidate passes. Duplicates of the same key are one
/// candidate (a pool aggregated across providers routinely repeats a key), but
/// two DIFFERENT keys that both verify mean the pool cannot say which is this
/// disc's Km: picking either derives a wrong VUK, and a wrong VUK decrypts to
/// plausible-looking garbage rather than failing loudly. Bail and let the
/// later hash/VID paths answer instead.
///
/// The predicate is a parameter rather than the inlined AES check because a
/// genuine two-key multi-hit cannot be synthesised: it needs one ciphertext
/// that decrypts under two distinct AES-128 keys to plaintexts sharing a
/// 64-bit prefix — a 2^64 search. Injecting the verifier is the only way the
/// ambiguity branch is reachable from a test at all.
fn unique_verifying_mk(mks: &[[u8; 16]], verifies: impl Fn(&[u8; 16]) -> bool) -> Option<[u8; 16]> {
let mut hits: Vec<[u8; 16]> = Vec::new();
for mk in mks {
if verifies(mk) && !hits.contains(mk) {
hits.push(*mk);
if hits.len() > 1 {
// Ambiguous — bail rather than pick a Media Key.
return None;
}
}
}
hits.first().copied()
}
/// For path 5: cross-reference the disc's `Unit_Key_RO.inf` CPS-unit /// For path 5: cross-reference the disc's `Unit_Key_RO.inf` CPS-unit
/// numbering against the KEYDB entry's pre-decrypted unit keys. Every /// numbering against the KEYDB entry's pre-decrypted unit keys. Every
/// CPS unit the disc declares must have a matching entry in KEYDB; /// CPS unit the disc declares must have a matching entry in KEYDB;
@@ -1328,62 +1300,6 @@ mod tests {
"VUK must derive from the verified Km + this disc's VID" "VUK must derive from the verified Km + this disc's VID"
); );
} }
/// Path 2.5's ambiguity guard: when MORE THAN ONE DISTINCT pooled Media Key
/// verifies against the MKB, the resolver must return no key at all rather
/// than pick one. A wrong Km derives a wrong VUK, and a wrong VUK does not
/// fail loudly — it decrypts the title to garbage that muxes and plays as a
/// corrupt rip.
///
/// The real MKB check cannot be forced into a multi-hit: two distinct
/// AES-128 keys decrypting one `mk_dv` to plaintexts that share the 64-bit
/// verify magic is a 2^64 search, not a fixture. So the rule is tested
/// through `unique_verifying_mk`, whose verifier is a parameter — the same
/// function `resolve_keys_v1` calls, with the same pool semantics.
#[test]
fn mk_pool_ambiguity_bails_rather_than_picking_a_media_key() {
let a = [0xAAu8; 16];
let b = [0xBBu8; 16];
let c = [0xCCu8; 16];
// One verifying candidate → that key.
assert_eq!(
unique_verifying_mk(&[a, b, c], |mk| *mk == b),
Some(b),
"a single verifying MK resolves"
);
// The SAME key repeated across providers is one candidate, not an
// ambiguity — the dedup (`!hits.contains`) must keep this resolvable.
assert_eq!(
unique_verifying_mk(&[b, b, b], |mk| *mk == b),
Some(b),
"duplicates of one key are not ambiguity"
);
// TWO DISTINCT verifying candidates → bail, no key.
assert_eq!(
unique_verifying_mk(&[a, b], |mk| *mk == a || *mk == b),
None,
"two distinct verifying MKs must yield NO key, not the first one"
);
// Ambiguity must still be detected when the second hit is last in the
// pool, i.e. the scan may not stop at the first hit.
assert_eq!(
unique_verifying_mk(&[a, c, [0u8; 16], b], |mk| *mk == a || *mk == b),
None,
"a late second hit is still ambiguous"
);
// Every candidate verifying is the degenerate ambiguous case.
assert_eq!(unique_verifying_mk(&[a, b, c], |_| true), None);
// No candidate verifies → no key (and no panic on an empty pool).
assert_eq!(unique_verifying_mk(&[a, b, c], |_| false), None);
assert_eq!(unique_verifying_mk(&[], |_| true), None);
}
#[test] #[test]
fn test_content_cert_parse() { fn test_content_cert_parse() {
// AACS 1.0 cert, bus encryption OFF. Content-cert layout: flag in // AACS 1.0 cert, bus encryption OFF. Content-cert layout: flag in
@@ -1906,77 +1822,6 @@ mod tests {
assert_eq!(r.vuk, Some(derive_vuk(&mk, &vid))); assert_eq!(r.vuk, Some(derive_vuk(&mk, &vid)));
} }
/// `resolve_keys_v21` gates paths 1 and 3 on `has_vid`, and an all-zero
/// Volume ID is the crate's "the VID was never read" sentinel — the SCSI
/// handshake leaves the buffer zeroed when it does not run or fails.
///
/// Both directions matter and both fail silently:
/// - treating the zero sentinel as a real VID runs path 3 and derives
/// `Kvu = AES-G(Km, 0…0)`, a perfectly well-formed but WRONG VUK. It
/// unwraps the title keys to garbage, and nothing downstream errors —
/// the rip just decodes to noise.
/// - treating a real VID as absent skips paths 1 and 3 entirely, so a
/// disc that could have been resolved from its Media Key reports no key.
///
/// Asserted through the final VUK, not through the flag.
#[test]
fn resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid() {
let uk_ro = minimal_unit_key_ro();
let vid = [0x42u8; 16];
let mk = [0x24u8; 16];
// A VID-keyed entry carrying an MK and nothing else: no VUK and no unit
// keys, so paths 4 and 5 cannot fire and ONLY the VID-gated path 3 can
// produce a result.
let entry = DiscEntry {
disc_hash: "not-this-disc".to_string(),
title: "sibling".to_string(),
media_key: Some(mk),
disc_id: Some(vid),
vuk: None,
unit_keys: Vec::new(),
};
let keydb = SuppliedKey {
device_keys: Vec::new(),
processing_keys: Vec::new(),
media_keys: Vec::new(),
disc_entry: Some(entry),
};
let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb];
// A real VID → path 3 fires and the VUK derives from Km + THIS VID.
let with_vid = ResolveContext {
unit_key_ro: &uk_ro,
content_cert: None,
volume_id: &vid,
providers,
mkb: None,
};
let r = resolve_keys_v21(&with_vid).expect("a real VID must reach path 3");
assert_eq!(r.key_source, 3);
assert_eq!(
r.vuk,
Some(derive_vuk(&mk, &vid)),
"VUK must derive from the Media Key and the disc's own VID"
);
// The all-zero sentinel → paths 1 and 3 are skipped entirely; with no
// VUK and no unit keys on the entry, nothing resolves.
let no_vid = ResolveContext {
unit_key_ro: &uk_ro,
content_cert: None,
volume_id: &[0u8; 16],
providers,
mkb: None,
};
let got = resolve_keys_v21(&no_vid);
assert!(
got.is_none(),
"a zero VID must not be used to derive a VUK; got key_source {:?} vuk {:?}",
got.as_ref().map(|r| r.key_source),
got.as_ref().map(|r| r.vuk.is_some())
);
}
#[test] #[test]
fn resolve_keys_returns_none_when_no_provider_has_anything() { fn resolve_keys_returns_none_when_no_provider_has_anything() {
// Empty provider array + VID present + no MKB → all paths miss → None. // Empty provider array + VID present + no MKB → all paths miss → None.
+3 -3
View File
@@ -25,7 +25,7 @@
//! u32 start_spn | u32 end_spn (source-packet numbers, inclusive) //! u32 start_spn | u32 end_spn (source-packet numbers, inclusive)
//! ``` //! ```
//! `index` is the 1..32 forensic index tag, NOT a sequential segment id: measured //! `index` is the 1..32 forensic index tag, NOT a sequential segment id: measured
//! on a retail 2.1 disc it cycles 1,2,…,32,1,2,… across records in //! on a retail 2.1 disc (Zombieland) it cycles 1,2,…,32,1,2,… across records in
//! file order — 24 full cycles of 32 plus a final partial cycle of 24 = 792 //! file order — 24 full cycles of 32 plus a final partial cycle of 24 = 792
//! records. Source-packet numbers are the 192-byte BDAV packet index: byte offset //! records. Source-packet numbers are the 192-byte BDAV packet index: byte offset
//! = `spn * 192`. Each segment is ~2560 packets (~480 KB) = 80 aligned units, //! = `spn * 192`. Each segment is ~2560 packets (~480 KB) = 80 aligned units,
@@ -334,7 +334,7 @@ mod tests {
#[test] #[test]
fn parses_real_disc_layout() { fn parses_real_disc_layout() {
// First three records observed on retail 2.1: the variant // First three records observed on retail 2.1 (Zombieland): the variant
// field counts 1,2,3,… (it wraps at 32 further into the table — see // field counts 1,2,3,… (it wraps at 32 further into the table — see
// `index_field_cycles_one_to_thirty_two`), segments are 2560 packets. // `index_field_cycles_one_to_thirty_two`), segments are 2560 packets.
let tbl = build_tbl(&[ let tbl = build_tbl(&[
@@ -396,7 +396,7 @@ mod tests {
#[test] #[test]
fn index_field_cycles_one_to_thirty_two() { fn index_field_cycles_one_to_thirty_two() {
// Reality on a retail 2.1 disc: field@4 is the index, cycling 1..=32 in file // Reality on Zombieland: field@4 is the index, cycling 1..=32 in file
// order (NOT a sequential segment id). Reproduce one-and-a-bit cycles. // order (NOT a sequential segment id). Reproduce one-and-a-bit cycles.
let mut recs = Vec::new(); let mut recs = Vec::new();
let mut spn = 1000u32; let mut spn = 1000u32;
-71
View File
@@ -195,77 +195,6 @@ impl std::fmt::Debug for DiscEntry {
} }
} }
#[cfg(test)]
mod unit_key_tests {
use super::*;
/// `is_default_index` is the public predicate that separates ordinary
/// (index-0) content keys from FMTS forensic index keys ([`UnitKey`] docs;
/// AACS 2.1 `IndividualSegment.tbl` tagging). A body answering `true` for
/// everything would present a forensic index key as an ordinary content
/// key — the caller would decrypt the bulk of the title with a key that
/// only opens 1/32nd of it; answering `false` for everything would hide
/// every ordinary key.
///
/// Pinned against the two NAMED constructors, which are the contract:
/// [`UnitKey::new`] builds the ordinary key, [`UnitKey::forensic`] builds
/// an index key for `1..=32`.
#[test]
fn is_default_index_separates_the_two_constructors() {
let ordinary = UnitKey::new(0, [0xAA; 16]);
assert!(
ordinary.is_default_index(),
"UnitKey::new builds the ordinary (index-0) key"
);
// Every forensic index the spec allows must be reported as NOT default.
for n in 1u8..=32 {
let k = UnitKey::forensic(0, [0xAA; 16], n);
assert!(
!k.is_default_index(),
"UnitKey::forensic({n}) is an index key, not the default key"
);
}
}
/// The predicate must agree with the one consumer of `index_number` in the
/// crate: [`crate::aacs::index_select::resolve_disc_index`] resolves the
/// disc's forensic index from exactly the keys that are NOT default. If
/// the two disagree, a disc resolves an index whose key the rest of the
/// pipeline treats as ordinary (or vice versa).
#[test]
fn is_default_index_agrees_with_the_forensic_index_resolver() {
use crate::aacs::index_select::resolve_disc_index;
let keys = [
UnitKey::new(0, [0x11; 16]),
UnitKey::forensic(1, [0x22; 16], 7),
];
assert_eq!(
resolve_disc_index(&keys),
Some(7),
"sanity: the resolver picks the forensic key's index"
);
let non_default: Vec<u8> = keys
.iter()
.filter(|k| !k.is_default_index())
.map(|k| k.index_number)
.collect();
assert_eq!(
non_default,
vec![7],
"exactly the key the resolver picked must be non-default"
);
// An all-ordinary key set resolves no index, and every key must report
// itself default.
let plain = [UnitKey::new(0, [0x11; 16]), UnitKey::new(1, [0x22; 16])];
assert_eq!(resolve_disc_index(&plain), None);
assert!(plain.iter().all(|k| k.is_default_index()));
}
}
#[cfg(test)] #[cfg(test)]
mod redaction_tests { mod redaction_tests {
use super::*; use super::*;
+8 -937
View File
@@ -91,8 +91,8 @@ pub fn is_variant_mkb(records: &[MkbRecord]) -> bool {
} }
/// Body of the `0x2d` record: the `VARIANTS` table followed by the trailing /// Body of the `0x2d` record: the `VARIANTS` table followed by the trailing
/// 16-byte `Kvn` Nonce. Measured `46_100*2 + 16 = 92_216` on one v70 disc and /// 16-byte `Kvn` Nonce. Measured `46_100*2 + 16 = 92_216` on Zombieland v70 and
/// `92_220` on another — in both, the leading `body.len() - 16` bytes are /// `92_220` on Stand By Me v70 — in both, the leading `body.len() - 16` bytes are
/// the big-endian `u16` `VARIANTS` table (one per subset-difference) and the last /// the big-endian `u16` `VARIANTS` table (one per subset-difference) and the last
/// 16 bytes are the Nonce, with NO leading header. This does NOT hold the C used /// 16 bytes are the Nonce, with NO leading header. This does NOT hold the C used
/// for `Kmp` — that is the per-slot block in `0x0c` /// for `Kmp` — that is the per-slot block in `0x0c`
@@ -361,7 +361,7 @@ impl std::error::Error for MediaKeyVariantError {}
/// Look up the per-slot `VARIANTS` value for the matched subset-difference slot, /// Look up the per-slot `VARIANTS` value for the matched subset-difference slot,
/// keyed by the same index that selected the cvalue ([`ProcessingKeyMatch::cvalue_index`]). /// keyed by the same index that selected the cvalue ([`ProcessingKeyMatch::cvalue_index`]).
/// ///
/// LAYOUT (fixed against a real 2.1 variant MKB — a v70 `MKB_RO.inf`): /// LAYOUT (fixed against a real 2.1 variant MKB — Zombieland v70, `MKB_RO.inf`):
/// the `0x2d` Encrypted-Media-Key-Variant-Data body is exactly /// the `0x2d` Encrypted-Media-Key-Variant-Data body is exactly
/// `46_100*2 + 16 = 92_216` bytes, i.e. one **big-endian u16 `VARIANTS` entry per /// `46_100*2 + 16 = 92_216` bytes, i.e. one **big-endian u16 `VARIANTS` entry per
/// subset-difference slot** (1:1 with the `0x0c` variant cvalues and the `0x04` /// subset-difference slot** (1:1 with the `0x0c` variant cvalues and the `0x04`
@@ -378,7 +378,7 @@ fn variants_for_uv(records: &[MkbRecord], sd_slot_index: usize) -> Option<u16> {
// The VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at // The VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at
// the TAIL (see [`variant_nonce`]). Bound the read to the table region so a // the TAIL (see [`variant_nonce`]). Bound the read to the table region so a
// near-end slot can never read Nonce bytes as a VARIANTS entry. NO leading // near-end slot can never read Nonce bytes as a VARIANTS entry. NO leading
// header (measured: a v70 `0x2d` body = 46_100*2 + 16 = 92_216). // header (measured: Zombieland v70 `0x2d` body = 46_100*2 + 16 = 92_216).
const NONCE: usize = 16; const NONCE: usize = 16;
let table_len = body.len().checked_sub(NONCE)?; let table_len = body.len().checked_sub(NONCE)?;
let off = sd_slot_index.checked_mul(2)?; let off = sd_slot_index.checked_mul(2)?;
@@ -938,15 +938,10 @@ mod tests {
} }
#[test] #[test]
fn walk_mkb_be24_middle_byte_is_honored() { fn walk_mkb_be24_high_byte_is_honored() {
// A record longer than 255 bytes needs the MIDDLE BE24 byte: total // A record longer than 255 bytes needs the high BE24 byte. Build a
// length 0x00_0110 (272) is `[0x00, 0x01, 0x10]`, so a parser reading // 0x10 record of total length 0x000110 (272) and confirm the body is
// only the low byte sees 0x10. The HIGH byte of this length is zero, so // 268 bytes (a parser that read only the low byte would see len 0x10).
// this test says nothing about the `<< 16` term — that is pinned
// separately by `mkb::tests::mkb_records_honors_the_high_byte_of_the_be24_length`,
// which uses a 0x01_0004 record. (Renamed from
// `walk_mkb_be24_high_byte_is_honored`, which claimed coverage this body
// does not deliver.)
let total = 0x0110usize; // 272 let total = 0x0110usize; // 272
let mut mkb = vec![0x10, 0x00, 0x01, 0x10]; let mut mkb = vec![0x10, 0x00, 0x01, 0x10];
mkb.resize(total, 0xAB); mkb.resize(total, 0xAB);
@@ -1210,928 +1205,4 @@ mod tests {
.expect_err("soft-correction bit → classified, not a key"); .expect_err("soft-correction bit → classified, not a key");
assert_eq!(err, MediaKeyVariantError::SoftCorrectionRequired); assert_eq!(err, MediaKeyVariantError::SoftCorrectionRequired);
} }
// ════════════════════════════════════════════════════════════════════
// A COMPLETE variant MKB — the AACS 2.1 happy path
//
// Every other test in this module asserts an ERROR classification, so
// until now no test ever drove `derive_media_key_variant` to a Media
// Key. That left the whole success path — the VARIANTS lookup, the VKD
// selection, the final `Km` unwrap and the verify gate — pinned by
// nothing: a body that answered a constant for any of those steps still
// produced the same errors these tests expect.
//
// No real key material is involved. Every AACS 2.1 relation in the chain
// is invertible, so the fixture below picks a Media Key and a Processing
// Key and computes the MKB records that connect them, exactly as
// `derive::position_recovery_tests::plant_mkb` does for the classical
// chain.
// ════════════════════════════════════════════════════════════════════
/// A planted variant MKB and the values it was built from.
struct PlantedVariant {
records: Vec<MkbRecord>,
/// The Processing Key that covers slot 0.
kp: [u8; 16],
/// The Media Key the chain must derive from `kp`.
km: [u8; 16],
/// The `0x86` Verify-Media-Key block.
mk_dv: [u8; 16],
/// The `VARIANTS[0]` entry planted in the `0x2d` table.
variants0: u16,
/// The `0x2d` tail Nonce.
nonce: [u8; 16],
/// The slot-0 `0x0c` C block the Kmp step consumes.
c_block: [u8; 16],
/// The subset-difference number of the single planted slot.
uv: u32,
}
/// An MKB record: 1-byte type + BE24 total length (header included) + body.
fn vrec(t: u8, body: &[u8]) -> Vec<u8> {
let total = 4 + body.len();
let mut r = vec![
t,
((total >> 16) & 0xFF) as u8,
((total >> 8) & 0xFF) as u8,
(total & 0xFF) as u8,
];
r.extend_from_slice(body);
r
}
/// Build a variant MKB by inverting the 2.1 chain for a CHOSEN `(Kp, Km)`.
///
/// One subset-difference slot (`uv = 2`, `u_mask_shift = 3`, slot index 0).
/// The VKD the chain must land on is planted at index **1** of the `0x2f`
/// table, behind a decoy at index 0, so `VARIANTS[0]` is load-bearing: it is
/// chosen as `Kvn XOR 1`, and any other value selects the decoy (wrong `Km`,
/// rejected by the verify gate) or indexes past the table.
fn plant_variant_mkb() -> PlantedVariant {
use crate::aacs::crypto::{aes_ecb_encrypt, aes_g};
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
const UV: u32 = 2;
const U_MASK_SHIFT: u8 = 3;
let kp: [u8; 16] = [
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
];
// `uv = 2` puts its only non-zero byte at index 15, so byte 15 is the ONE
// position where the `Km`/`Kmp` uv-XOR is observable. Its 0x02 bit is
// deliberately CLEAR: with the bit set, `km[15] ^= 2` and `km[15] |= 2`
// agree (the XOR would only be clearing a bit the OR re-sets) and an
// OR-for-XOR substitution in the final step would be invisible.
let km: [u8; 16] = [
0xC0, 0xC1, 0xC2, 0xC3, 0xC4, 0xC5, 0xC6, 0xC7, 0xC8, 0xC9, 0xCA, 0xCB, 0xCC, 0xCD,
0xCE, 0xCD,
];
assert_eq!(km[15] & 0x02, 0, "fixture check: see above");
let nonce: [u8; 16] = [
0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, 0x3D,
0x3E, 0x3F,
];
let uv_bytes = UV.to_be_bytes();
// ── Verify-Media-Key record (0x86): AES-D(Km, mk_dv) opens with the
// magic ([C] §3.2.5.1.4), so mk_dv = AES-E(Km, magic || padding).
let mut vd = [0x5Au8; 16];
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&km, &vd);
// ── C (0x0c): the chain computes Kmp = AES-D(Kp, C) XOR uv. Pick a Kmp
// with BOTH condition bits on byte 15 clear (0x02 soft-correction,
// 0x04 online challenge) so the default KCD path runs, then invert.
let mut kmp = [0x42u8; 16];
kmp[15] = 0x40; // neither 0x02 nor 0x04
let mut c_plain = kmp;
for i in 0..4 {
c_plain[12 + i] ^= uv_bytes[i];
}
let c_block = aes_ecb_encrypt(&kp, &c_plain);
// ── Kpnew = Kmp XOR KCD. Read through the production constant rather
// than assuming it is zero, so the fixture stays valid if a real
// per-licensee KCD is ever wired in (see `KEY_CORRECTION_DATA`).
let mut kpnew = [0u8; 16];
for i in 0..16 {
kpnew[i] = kmp[i] ^ KEY_CORRECTION_DATA[i];
}
// ── VKD: the chain computes Km = AES-D(Kpnew, VKD) XOR uv, so
// VKD = AES-E(Kpnew, Km with uv XORed back into its low 4 bytes).
let mut km_pre = km;
for i in 0..4 {
km_pre[12 + i] ^= uv_bytes[i];
}
let vkd = aes_ecb_encrypt(&kpnew, &km_pre);
// ── VARIANTS[0]: VKD_idx = Kvn XOR VARIANTS[uv], and we planted the
// real VKD at table index 1, so VARIANTS[0] = Kvn XOR 1.
// Kvn = low 16 bits (BE) of AES-G(Kp, Nonce).
let kvn_block = aes_g(&kp, &nonce);
let kvn = u16::from_be_bytes([kvn_block[14], kvn_block[15]]);
let variants0 = kvn ^ 1;
// ── Assemble.
let mut mkb = Vec::new();
mkb.extend_from_slice(&vrec(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
// 0x04 subset-difference: one slot.
let mut subdiff = vec![U_MASK_SHIFT];
subdiff.extend_from_slice(&uv_bytes);
mkb.extend_from_slice(&vrec(0x04, &subdiff));
// 0x0c per-slot C table: one 16-byte entry.
mkb.extend_from_slice(&vrec(0x0c, &c_block));
// 0x86 Verify-Media-Key.
mkb.extend_from_slice(&vrec(0x86, &mk_dv));
// 0x2d: VARIANTS table (one BE u16) then the 16-byte tail Nonce.
let mut vdata = Vec::new();
vdata.extend_from_slice(&variants0.to_be_bytes());
vdata.extend_from_slice(&nonce);
mkb.extend_from_slice(&vrec(0x2d, &vdata));
// 0x2f VKD table: a decoy at index 0, the real VKD at index 1.
let mut vkd_table = vec![0x9Au8; 16];
vkd_table.extend_from_slice(&vkd);
mkb.extend_from_slice(&vrec(0x2f, &vkd_table));
PlantedVariant {
records: walk_mkb(&mkb),
kp,
km,
mk_dv,
variants0,
nonce,
c_block,
uv: UV,
}
}
/// Sanity-check the fixture before anything is asserted through it: an MKB
/// the record finders cannot read would make every "returns an error" body
/// look correct.
#[test]
fn the_planted_variant_mkb_is_a_well_formed_variant_mkb() {
let p = plant_variant_mkb();
assert!(is_variant_mkb(&p.records), "0x2d/0x2f present");
assert_eq!(variant_nonce(&p.records), Some(p.nonce), "tail Nonce");
assert_eq!(
variant_key_data(&p.records).map(<[u8]>::len),
Some(32),
"two 16-byte VKD entries"
);
assert_eq!(
variant_uv_slots(&p.records),
Some(vec![(2u32, 0usize)]),
"one subset-difference slot at index 0 with uv=2"
);
}
/// THE happy path: a Processing Key covering slot 0 of a complete variant
/// MKB must derive the planted Media Key.
///
/// This is the assertion the whole 2.1 chain hangs from — `derive_media_key_variant`
/// is what `resolve` calls for a 2.1 disc, and its output becomes the VUK,
/// the title keys and every decrypted byte. The assertion lands on the FINAL
/// derived Media Key, so no intermediate step (VARIANTS lookup, VKD index,
/// Kpnew, the unwrap) can be replaced by a constant and still pass.
#[test]
fn variant_chain_derives_the_planted_media_key_for_a_covering_kp() {
let p = plant_variant_mkb();
assert_eq!(
derive_media_key_variant(&p.records, &p.kp),
Ok(p.km),
"a covering 2.1 Processing Key must derive the planted Media Key"
);
}
/// The other direction: a Processing Key one bit away must NOT yield a key.
/// The terminal Verify-Media-Key gate is what stands between a wrong Kp and
/// a wrong Media Key silently propagating into the VUK and title keys.
#[test]
fn variant_chain_yields_no_key_for_a_kp_one_bit_away() {
let p = plant_variant_mkb();
let mut stranger = p.kp;
stranger[0] ^= 0x01;
let got = derive_media_key_variant(&p.records, &stranger);
assert!(
got.is_err(),
"a non-covering Kp must never produce a Media Key, got {got:?}"
);
assert_ne!(got, Ok(p.km));
}
/// `mkb_find_mk_dv` supplies the block the terminal verify gate compares
/// against. A body answering a FIXED block would make the gate compare every
/// derived Media Key against a record no disc carries: on a real disc every
/// correct key is rejected (2.1 discs stop resolving entirely), and any key
/// that happened to open the fixed block would be accepted wholesale.
#[test]
fn mkb_find_mk_dv_returns_the_verify_records_actual_bytes() {
let p = plant_variant_mkb();
assert_eq!(
mkb_find_mk_dv(&p.records),
Some(p.mk_dv),
"mk_dv must be the bytes the 0x86 record carries"
);
assert_ne!(mkb_find_mk_dv(&p.records), Some([0u8; 16]));
assert_ne!(mkb_find_mk_dv(&p.records), Some([1u8; 16]));
// And it is the block the gate actually uses: swapping the 0x86 record
// for an unrelated one must break the derivation that just succeeded.
let mut recs = p.records.clone();
let v = recs
.iter_mut()
.find(|r| r.rec_type == 0x86)
.expect("verify record present");
v.body = vec![0x00; 16];
assert!(
derive_media_key_variant(&recs, &p.kp).is_err(),
"with a foreign verify block the same Kp must no longer verify"
);
}
/// `variants_for_uv` reads `VARIANTS[slot]` — the value XORed with `Kvn` to
/// index the VKD table. A body answering a constant picks the WRONG VKD
/// entry for every disc, so the derived Media Key fails the verify gate and
/// every 2.1 variant disc reports `ProcessingKeyUnavailable` with a
/// perfectly good Processing Key in hand.
///
/// Asserted two ways: the exact planted table entry, and — the load-bearing
/// one — that this entry is what carries the chain to the planted Media Key.
#[test]
fn variants_for_uv_reads_the_planted_table_entry_that_selects_the_vkd() {
let p = plant_variant_mkb();
assert_eq!(
variants_for_uv(&p.records, 0),
Some(p.variants0),
"slot 0 must read the planted VARIANTS entry"
);
// The planted entry is Kvn ^ 1 (the real VKD sits at table index 1), so
// it is neither 0 nor 1 — a constant body is a different value here.
assert_ne!(variants_for_uv(&p.records, 0), Some(0));
assert_ne!(variants_for_uv(&p.records, 0), Some(1));
// Perturbing ONLY the VARIANTS entry breaks the derivation: proof the
// value this function returns is the one that selects the VKD.
let mut recs = p.records.clone();
let d = recs
.iter_mut()
.find(|r| r.rec_type == 0x2d)
.expect("0x2d present");
d.body[0] ^= 0x80;
assert!(
derive_media_key_variant(&recs, &p.kp).is_err(),
"a different VARIANTS entry must select a different VKD and fail the gate"
);
}
/// The `0x2d` body is `VARIANTS` table then a 16-byte tail Nonce. A slot
/// index whose entry would fall inside the Nonce must be refused rather than
/// read Nonce bytes as a VARIANTS value.
#[test]
fn variants_for_uv_stops_before_the_tail_nonce() {
// Three-entry table with distinct values, then the Nonce.
let mut body = Vec::new();
body.extend_from_slice(&0x1234u16.to_be_bytes());
body.extend_from_slice(&0xABCDu16.to_be_bytes());
body.extend_from_slice(&0x00FFu16.to_be_bytes());
let nonce = [0x77u8; 16];
body.extend_from_slice(&nonce);
let recs = walk_mkb(&vrec(0x2d, &body));
assert_eq!(variants_for_uv(&recs, 0), Some(0x1234));
assert_eq!(variants_for_uv(&recs, 1), Some(0xABCD));
assert_eq!(variants_for_uv(&recs, 2), Some(0x00FF));
assert_eq!(
variants_for_uv(&recs, 3),
None,
"slot 3 starts inside the Nonce — must be refused, not read"
);
assert_eq!(variant_nonce(&recs), Some(nonce), "the Nonce is the tail");
}
/// `variant_uv_slots` enumerates the slots the chain will try a Processing
/// Key against, and it must drop the two shapes that are unusable — and
/// dangerous — rather than pass them on:
///
/// - `uv == 0`: no subset-difference. It would be XORed into `Kmp` and
/// `Km` as a no-op and the slot would be tried against every VKD entry.
/// - `u_mask_shift >= 32`: out of range for a `u32` shift. `0x20..=0x3F`
/// have the `0xC0` revoked-marker bits CLEAR, so they pass the table
/// terminator and reach the `wrapping_shl` in the walk, where shift 32
/// silently means shift 0 (`u_mask = 0xFFFF_FFFF`) and matches a slot
/// the device does not cover.
///
/// Both bytes are disc-supplied. Every existing fixture uses one in-range
/// non-zero slot, so neither rejection was executed.
#[test]
fn variant_uv_slots_drops_zero_uv_and_out_of_range_shift_slots() {
// Four slots: uv == 0, shift == 32 (the exact boundary), shift == 0x3F
// (the top of the marker-clear range), and one good slot last.
let mut body = Vec::new();
for (shift, uv) in [
(3u8, 0u32),
(32u8, 0x0000_0005u32),
(0x3Fu8, 0x0000_0006u32),
(12u8, 0x0000_0400u32),
] {
body.push(shift);
body.extend_from_slice(&uv.to_be_bytes());
}
// Fixture check: none of these bytes trips the 0xC0 table terminator, so
// the per-slot tests are the only thing rejecting them.
assert!(body.chunks(5).all(|c| c[0] & 0xC0 == 0));
let recs = walk_mkb(&vrec(REC_SUBSET_DIFFERENCE, &body));
assert_eq!(
variant_uv_slots(&recs),
Some(vec![(0x0000_0400u32, 3usize)]),
"only the in-range, non-zero slot is a usable subset-difference — \
and it keeps its own table index"
);
}
/// THE happy path for the EXPLICIT-INPUT entry point. `media_key_variant_from_kp`
/// is the harness twin of [`derive_media_key_variant`]: same chain, but the
/// caller supplies the `0x0c` C block, the slot's `uv` and its `VARIANTS[uv]`
/// instead of having them looked up on the MKB.
///
/// Before this test, the ONLY test that entered this function asserted the
/// `Kmp[15]` soft-correction bit — it returned before the Kpnew, Kvn, VKD,
/// Km and Kvu steps ever ran. Every arithmetic step past that early return
/// was executed by nothing, so a body that computed `Kpnew = Kmp | KCD`,
/// indexed the VKD table at `Kvn + VARIANTS` or dropped the `uv` XOR out of
/// `Km` produced exactly the same observable behaviour.
///
/// The assertion lands on the returned `(Km, Kvu)` — the two values that
/// become every title key and every decrypted byte on a 2.1 disc.
#[test]
fn media_key_variant_from_kp_derives_the_planted_media_key_and_volume_unique_key() {
let p = plant_variant_mkb();
let vid: [u8; 16] = [
0x1A, 0x2B, 0x3C, 0x4D, 0x5E, 0x6F, 0x70, 0x81, 0x92, 0xA3, 0xB4, 0xC5, 0xD6, 0xE7,
0xF8, 0x09,
];
let (km, kvu) =
media_key_variant_from_kp(&p.kp, &p.c_block, p.uv, p.variants0, &p.records, &vid)
.expect("the planted explicit inputs must complete the 2.1 variant chain");
assert_eq!(
km, p.km,
"the explicit-input entry must derive the same planted Media Key \
the MKB-driven entry does"
);
// Kvu = AES-G(Km, VID) ([C] §3.2.5.2). Computed from the PLANTED Km
// literal, so it does not move with any mutation of this module.
assert_eq!(
kvu,
aes_g(&p.km, &vid),
"Kvu must be AES-G of the derived Media Key with the Volume ID"
);
// ...and specifically NOT of the Processing Key: the two are one AES-D
// apart and a body that returned the wrong one would still be 16 bytes
// of key-shaped material that silently decrypts nothing.
assert_ne!(kvu, aes_g(&p.kp, &vid));
}
/// The terminal gate on the explicit-input entry. `media_key_variant_from_kp`
/// takes three caller-supplied values (`c_block`, `uv`, `variants_uv`); each
/// one wrong must yield `MediaKeyVerifyFailed`, never a key. Without this,
/// a harness feeding a mis-transcribed slot would be handed 16 bytes that
/// look exactly like a Media Key.
#[test]
fn media_key_variant_from_kp_refuses_every_single_wrong_explicit_input() {
let p = plant_variant_mkb();
let vid = [0x33u8; 16];
// Baseline: all three correct → a key.
assert!(
media_key_variant_from_kp(&p.kp, &p.c_block, p.uv, p.variants0, &p.records, &vid)
.is_ok()
);
// Wrong C block: EVERY one-bit neighbour must fail to produce a key.
// (Which classification it lands in depends on the two condition bits
// the perturbed Kmp happens to carry — the property being pinned is
// that none of the 128 reaches `Ok`.)
for byte in 0..16usize {
for bit in 0..8u32 {
let mut c_bad = p.c_block;
c_bad[byte] ^= 1u8 << bit;
let got =
media_key_variant_from_kp(&p.kp, &c_bad, p.uv, p.variants0, &p.records, &vid);
assert!(
got.is_err(),
"C block differing only in byte {byte} bit {bit} yielded a key"
);
}
}
// Wrong uv: it is XORed into BOTH Kmp and Km, so a wrong slot number
// must not reach a key.
for delta in 1..=8u32 {
let got = media_key_variant_from_kp(
&p.kp,
&p.c_block,
p.uv + delta,
p.variants0,
&p.records,
&vid,
);
assert!(got.is_err(), "uv + {delta} must not verify, got {got:?}");
}
// Wrong VARIANTS[uv]: selects a different VKD entry. The planted table
// has two entries, so `^ 1` lands on the decoy at index 0 (in range,
// wrong key) rather than out of range.
assert_eq!(
media_key_variant_from_kp(&p.kp, &p.c_block, p.uv, p.variants0 ^ 1, &p.records, &vid),
Err(MediaKeyVariantError::MediaKeyVerifyFailed),
"a VARIANTS entry selecting the decoy VKD must not verify"
);
// And a VARIANTS entry that indexes off the end of the table is
// classified as such, not read out of bounds.
assert_eq!(
media_key_variant_from_kp(
&p.kp,
&p.c_block,
p.uv,
p.variants0 ^ 0x8000,
&p.records,
&vid
),
Err(MediaKeyVariantError::VkdIndexOutOfRange),
"a VKD index past the table must be classified, not read"
);
}
/// The `Kmp[15]` online-challenge bit (`0x04`) on the explicit-input entry.
/// Its twin (`0x02`, soft correction) was already pinned; without this one a
/// body that classified both bits as soft correction — or ignored `0x04` and
/// ran the default-KCD chain to a wrong key — was unconstrained.
#[test]
fn media_key_variant_from_kp_classifies_online_challenge() {
use crate::aacs::crypto::aes_ecb_encrypt;
let p = plant_variant_mkb();
// Plant Kmp[15] = 0x04 (online challenge, soft-correction bit CLEAR) and
// invert the Kmp step for uv = 0 so Kmp == AES-D(kp, C).
let mut target_kmp = [0x00u8; 16];
target_kmp[15] = 0x04;
let c_block = aes_ecb_encrypt(&p.kp, &target_kmp);
assert_eq!(
media_key_variant_from_kp(&p.kp, &c_block, 0, 0, &p.records, &[0u8; 16]),
Err(MediaKeyVariantError::OnlineChallengeRequired),
);
}
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT variant MKB driven by a real DEVICE KEY
//
// `walk_processing_key` is the DK -> Kp step that feeds the whole 2.1
// chain. Every existing test of it either asserts `None` (out-of-range
// shift, uv == 0) or asserts only that SOME match came back — none pins
// WHICH Processing Key, cvalue or slot index it returns. And every one of
// them uses a SINGLE-slot MKB, where the slot index is 0: all the
// `uvs[1 + 5*idx]` / `cvalues[idx*16..]` stride arithmetic multiplies by
// zero and any stride at all gives the same answer.
//
// This fixture puts the covering slot at index 1, behind a decoy at
// index 0, so the strides are load-bearing.
// ════════════════════════════════════════════════════════════════════
/// A two-slot variant MKB whose SECOND slot is opened by a device key.
struct PlantedWalk {
records: Vec<MkbRecord>,
/// The device key that covers slot 1 with zero descent.
dk: DeviceKey,
/// The Media Key the full chain must reach from that Processing Key.
km: [u8; 16],
/// The `0x0c` C block of slot 1 — the cvalue the walk must select.
c_block1: [u8; 16],
}
/// Build a two-slot variant MKB keyed by a DEVICE key at slot **1**.
///
/// Positions follow the same reasoning as the classical
/// `derive::position_recovery_tests::plant_mkb`: `uv = 0x0400`
/// (`u_mask_shift = 12`) with a device node of `0x0C00` satisfies the
/// [C] §3.2.4 gate — equal under `u_mask = 0xFFFF_F000`, different under
/// `v_mask = 0xFFFF_F800`. The device key's own `uv` equals the slot's, so
/// `dev_key_v_mask == v_mask` and [`calc_pk_from_dk`] descends zero levels:
/// `Kp = AES-G3(dk, 1)`, written out explicitly below rather than taken from
/// the walk's own output.
///
/// Slot 0 is a decoy at `uv = 0x0800`, which the SAME device node fails the
/// `v_mask` half of the gate against (`0x0C00 & 0xFFFF_F000 == 0x0800 &
/// 0xFFFF_F000`), so the walk must skip it and land on slot 1.
fn plant_walk_variant_mkb() -> PlantedWalk {
use crate::aacs::crypto::{aes_ecb_encrypt, aes_g};
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
const UV_DECOY: u32 = 0x0000_0800;
const UV_REAL: u32 = 0x0000_0400;
const U_MASK_SHIFT: u8 = 12;
const NODE: u16 = 0x0C00;
let dkey: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2,
0xE1, 0xF0,
];
// Zero descent: the Processing Key is the AES-G3(.,1) of the device's own
// node ([C] §3.2.4). Written as the explicit primitive chain so it does
// NOT move with any mutation of the walk under test.
let kp = aesg3(&dkey, 1);
// As in `plant_variant_mkb`: `uv = 0x0400`'s only non-zero byte is at
// index 14, and its 0x04 bit must be CLEAR in `km` for the final
// `km[14] ^= 0x04` to be distinguishable from `|=`.
let km: [u8; 16] = [
0xB0, 0xB1, 0xB2, 0xB3, 0xB4, 0xB5, 0xB6, 0xB7, 0xB8, 0xB9, 0xBA, 0xBB, 0xBC, 0xBD,
0xBA, 0xBF,
];
assert_eq!(km[14] & 0x04, 0, "fixture check: see above");
let nonce: [u8; 16] = [
0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5A, 0x5B, 0x5C, 0x5D,
0x5E, 0x5F,
];
// ── 0x86 Verify-Media-Key ([C] §3.2.5.1.4).
let mut vd = [0x5Au8; 16];
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&km, &vd);
// ── C blocks. Both are built so `Kmp[15]` has the 0x02 / 0x04 condition
// bits CLEAR, so both slots run the default-KCD path to completion and
// the decoy is rejected by the terminal verify gate rather than
// short-circuiting into a correction-mode classification.
let c_for = |kmp: &[u8; 16], uv: u32| -> [u8; 16] {
let mut c_plain = *kmp;
for (b, u) in c_plain[12..16].iter_mut().zip(uv.to_be_bytes()) {
*b ^= u;
}
aes_ecb_encrypt(&kp, &c_plain)
};
let mut kmp1 = [0x42u8; 16];
kmp1[15] = 0x40;
let c_block1 = c_for(&kmp1, UV_REAL);
let mut kmp0 = [0x17u8; 16];
kmp0[15] = 0x40;
let c_block0 = c_for(&kmp0, UV_DECOY);
// ── VKD for slot 1: Km = AES-D(Kpnew, VKD) XOR uv.
let mut kpnew = [0u8; 16];
for i in 0..16 {
kpnew[i] = kmp1[i] ^ KEY_CORRECTION_DATA[i];
}
let mut km_pre = km;
for (b, u) in km_pre[12..16].iter_mut().zip(UV_REAL.to_be_bytes()) {
*b ^= u;
}
let vkd = aes_ecb_encrypt(&kpnew, &km_pre);
// ── VARIANTS: the real VKD is planted at table index 2, behind two
// decoys, so VARIANTS[1] = Kvn XOR 2 is load-bearing. VARIANTS[0] sends
// the decoy slot to entry 0 — in range, wrong key, rejected by the gate.
let kvn_block = aes_g(&kp, &nonce);
let kvn = u16::from_be_bytes([kvn_block[14], kvn_block[15]]);
let variants0 = kvn;
let variants1 = kvn ^ 2;
// ── Assemble.
let mut mkb = Vec::new();
mkb.extend_from_slice(&vrec(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
let mut subdiff = vec![U_MASK_SHIFT];
subdiff.extend_from_slice(&UV_DECOY.to_be_bytes());
subdiff.push(U_MASK_SHIFT);
subdiff.extend_from_slice(&UV_REAL.to_be_bytes());
mkb.extend_from_slice(&vrec(0x04, &subdiff));
let mut ctable = Vec::new();
ctable.extend_from_slice(&c_block0);
ctable.extend_from_slice(&c_block1);
mkb.extend_from_slice(&vrec(0x0c, &ctable));
mkb.extend_from_slice(&vrec(0x86, &mk_dv));
let mut vdata = Vec::new();
vdata.extend_from_slice(&variants0.to_be_bytes());
vdata.extend_from_slice(&variants1.to_be_bytes());
vdata.extend_from_slice(&nonce);
mkb.extend_from_slice(&vrec(0x2d, &vdata));
let mut vkd_table = vec![0x9Au8; 16];
vkd_table.extend_from_slice(&[0x6Bu8; 16]);
vkd_table.extend_from_slice(&vkd);
mkb.extend_from_slice(&vrec(0x2f, &vkd_table));
PlantedWalk {
records: walk_mkb(&mkb),
dk: DeviceKey {
key: dkey,
node: NODE,
uv: UV_REAL,
u_mask_shift: U_MASK_SHIFT,
},
km,
c_block1,
}
}
/// Sanity-check the two-slot fixture before anything is asserted through it.
#[test]
fn the_planted_walk_variant_mkb_has_two_slots_and_is_keyed_at_the_second() {
let p = plant_walk_variant_mkb();
assert!(is_variant_mkb(&p.records));
assert_eq!(
variant_uv_slots(&p.records),
Some(vec![(0x0800u32, 0usize), (0x0400u32, 1usize)]),
"two subset-difference slots, the covering one at index 1"
);
assert_eq!(
mkb_find_body(&p.records, REC_MEDIA_KEY_VARIANT_DATA).map(<[u8]>::len),
Some(32),
"two 16-byte C entries in the 0x0c table"
);
}
/// `walk_processing_key` must return the Processing Key, `uv`, cvalue AND
/// slot index of the covering slot — slot **1**, not slot 0.
///
/// This is the DK → Kp step the entire 2.1 chain starts from. Every prior
/// test of it asserted either `None` or merely `is_some()`, and all used a
/// one-slot MKB where every stride multiplies by zero. A body that read the
/// subset-difference at the wrong stride, sliced the wrong cvalue block, or
/// returned the slot-0 cvalue for a slot-1 match would have passed all of
/// them — and produced a Processing Key that opens nothing.
///
/// The expected `Kp` is written as the explicit `AES-G3(dk, 1)` zero-descent
/// relation from [C] §3.2.4, not taken from the walk's own output.
#[test]
fn walk_processing_key_returns_the_covering_slots_key_cvalue_and_index() {
let p = plant_walk_variant_mkb();
let m = walk_processing_key(&p.records, std::slice::from_ref(&p.dk))
.expect("the planted device key covers slot 1 of this MKB");
assert_eq!(m.uv, 0x0400, "the covering slot's uv, not the decoy's");
assert_eq!(m.cvalue_index, 1, "the covering slot sits at index 1");
assert_eq!(
m.kp,
aesg3(&p.dk.key, 1),
"zero descent: Kp is AES-G3(device key, 1)"
);
assert_eq!(
m.cvalue, p.c_block1,
"the cvalue must be slot 1's 16-byte C block, not slot 0's"
);
// The load-bearing consequence: that Processing Key drives the full
// variant chain to the planted Media Key.
assert_eq!(
derive_media_key_variant(&p.records, &m.kp),
Ok(p.km),
"the walked Processing Key must derive the planted Media Key"
);
}
/// The gate the walk applies is [C] §3.2.4's subset-difference test, and a
/// device key that fails it must get NO match. Pinned across all four
/// coordinates the gate reads — node, uv, u_mask_shift and the key bytes —
/// because a body that dropped any half of the gate would hand back a
/// Processing Key derived at the wrong tree position.
#[test]
fn walk_processing_key_refuses_a_device_key_that_fails_the_subset_difference_gate() {
let p = plant_walk_variant_mkb();
assert!(walk_processing_key(&p.records, std::slice::from_ref(&p.dk)).is_some());
// node equal to uv under v_mask (0xFFFF_F800): the "different under
// v_mask" half of the gate fails.
let mut d = p.dk.clone();
d.node = 0x0400;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"a node equal to uv under v_mask does not gate"
);
// node differing under u_mask (0xFFFF_F000): the "equal under u_mask"
// half fails.
let mut d = p.dk.clone();
d.node = 0x1C00;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"a node outside the slot's u_mask does not gate"
);
// A device key whose declared u_mask_shift is not the slot's.
let mut d = p.dk.clone();
d.u_mask_shift = 11;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"u_mask must equal dev_key_u_mask"
);
// A device key positioned in a different subtree.
let mut d = p.dk.clone();
d.uv = 0x0C00;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"the device key's uv must agree with the slot's under dev_key_v_mask"
);
}
/// A `0x04` subset-difference record whose byte count is not a multiple of 5
/// must have its trailing partial chunk REFUSED, not parsed as a slot.
///
/// The walk sizes the table with `take_while(|c| c.len() == 5 && ...)`. Drop
/// the length half of that conjunction and the partial chunk is counted, and
/// the very next line reads `p_uv[0..4]` off a slice with fewer than four
/// bytes left — an index-out-of-bounds PANIC on a disc-supplied record
/// length. This is untrusted input: a truncated or crafted MKB reaches this
/// with no other guard in between.
#[test]
fn a_trailing_partial_subset_difference_chunk_is_not_parsed_as_a_slot() {
let p = plant_walk_variant_mkb();
// Re-emit the 0x04 record with three trailing bytes — a partial chunk
// whose first byte has the 0xC0 revoked-marker bits CLEAR, so only the
// length test stands between it and a four-byte read off a one-byte tail.
let mut recs = p.records.clone();
let sd = recs
.iter_mut()
.find(|r| r.rec_type == REC_SUBSET_DIFFERENCE)
.expect("0x04 present");
assert_eq!(sd.body.len(), 10, "two whole slots before truncation");
sd.body.extend_from_slice(&[0x0C, 0xAB, 0xCD]);
// A device key that covers NOTHING, so the walk is forced to run past
// both whole slots and reach the partial chunk.
let mut stranger = p.dk.clone();
stranger.node = 0x1C00;
assert!(
walk_processing_key(&recs, std::slice::from_ref(&stranger)).is_none(),
"the partial chunk must terminate the table, not be walked"
);
// And the covering key still finds its slot with the junk appended.
assert!(walk_processing_key(&recs, std::slice::from_ref(&p.dk)).is_some());
}
/// A `0x0c` cvalue table SHORTER than the matching slot index must make the
/// walk skip the slot, not slice past the end of the record.
///
/// `cvalues[uvs_idx * 16..(uvs_idx + 1) * 16]` is an unchecked slice; the
/// only thing in front of it is `if uvs_idx >= cvalues.len() / 16`. The two
/// counts come from DIFFERENT disc-supplied records (`0x04` and `0x0c`),
/// so nothing but this guard keeps them in agreement — a real MKB with a
/// short cvalue table panics the rip thread without it.
#[test]
fn a_cvalue_table_shorter_than_the_matching_slot_is_not_sliced_past() {
let p = plant_walk_variant_mkb();
let mut recs = p.records.clone();
let cv = recs
.iter_mut()
.find(|r| r.rec_type == REC_MEDIA_KEY_VARIANT_DATA)
.expect("0x0c present");
// One entry only — the covering slot is index 1, so it is out of range.
cv.body.truncate(16);
assert!(
walk_processing_key(&recs, std::slice::from_ref(&p.dk)).is_none(),
"slot 1 with a one-entry cvalue table must be skipped, not read"
);
}
/// The classical-magic escape hatch. On a NON-variant MKB the walk must
/// return a match only when `AES-D(Kmp, mk_dv)` opens with the [C] §3.2.5.1.4
/// verify magic; on a variant MKB that relation does not hold (the walk
/// yields a Precursor) and the presence of `0x2d`/`0x2f` is what lets the
/// match through to the chain's own terminal gate.
///
/// Both halves of `classical_ok || variant_present` are pinned here: strip
/// the variant records from a fixture whose magic does NOT hold and the walk
/// must go quiet. Otherwise a body that dropped the guard entirely would
/// return an unauthenticated Processing Key on every classical MKB.
#[test]
fn walk_processing_key_needs_either_the_verify_magic_or_variant_records() {
let p = plant_walk_variant_mkb();
// As planted (variant records present, magic absent) → a match.
assert!(walk_processing_key(&p.records, std::slice::from_ref(&p.dk)).is_some());
// Same slots, same device key, variant records removed. Nothing now
// authenticates the Processing Key, so there must be no match.
let stripped: Vec<MkbRecord> = p
.records
.iter()
.filter(|r| r.rec_type != REC_VARIANT_DATA_AND_NONCE && r.rec_type != REC_VKD_TABLE)
.cloned()
.collect();
assert!(
!is_variant_mkb(&stripped),
"fixture check: the stripped MKB is no longer a variant MKB"
);
assert!(
walk_processing_key(&stripped, std::slice::from_ref(&p.dk)).is_none(),
"without variant records the verify magic must hold, and it does not \
for a Precursor the walk must not return an unauthenticated key"
);
}
/// The OTHER half of `classical_ok || variant_present`: a non-variant MKB
/// whose cvalue really does open the Verify-Media-Key magic must yield a
/// match, and the [C] §3.2.4 relation that produces the candidate — AES-D(Kp,
/// cvalue) with `uv` XORed into the LOW FOUR BYTES — must be computed
/// exactly.
///
/// This is the only path on which that XOR is observable. On a variant MKB
/// `variant_present` short-circuits the magic test, so the whole
/// `km_candidate` computation is dead weight there: a body that ORed `uv`
/// in, or XORed it at the wrong offset, changes nothing any variant fixture
/// can see. On a CLASSICAL MKB it is the entire authentication of the
/// Processing Key.
#[test]
fn walk_processing_key_authenticates_a_classical_match_through_the_verify_magic() {
use crate::aacs::crypto::aes_ecb_encrypt;
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
const UV: u32 = 0x0000_0400;
const U_MASK_SHIFT: u8 = 12;
let dkey: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2,
0xE1, 0xF0,
];
// Zero descent ([C] §3.2.4), written out as the primitive relation.
let kp = aesg3(&dkey, 1);
// `uv = 0x0400` puts its only non-zero byte at index 14, so byte 14 is
// the ONE position where the `uv` XOR is observable at all. Its 0x04 bit
// is deliberately CLEAR here: with the bit set, `km_candidate[14] |=
// 0x04` and `^= 0x04` agree (the XOR would only be clearing a bit the OR
// re-sets), and an OR-for-XOR substitution would be invisible.
let mk: [u8; 16] = [
0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7A, 0x7B, 0x7C, 0x7D,
0x7A, 0x7F,
];
assert_eq!(mk[14] & 0x04, 0, "fixture check: see above");
// Invert [C] §3.2.4: the walk computes AES-D(Kp, cvalue) then XORs `uv`
// into bytes 12..16 and expects the Media Key.
let mut mk_raw = mk;
for (b, u) in mk_raw[12..16].iter_mut().zip(UV.to_be_bytes()) {
*b ^= u;
}
let cv = aes_ecb_encrypt(&kp, &mk_raw);
// Invert [C] §3.2.5.1.4.
let mut vd = [0x5Au8; 16];
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&mk, &vd);
let mut subdiff = vec![U_MASK_SHIFT];
subdiff.extend_from_slice(&UV.to_be_bytes());
let mut mkb = Vec::new();
mkb.extend_from_slice(&vrec(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
mkb.extend_from_slice(&vrec(0x86, &mk_dv));
mkb.extend_from_slice(&vrec(0x04, &subdiff));
// cvalues in the classical `0x05` record; NO 0x2d / 0x2f.
mkb.extend_from_slice(&vrec(0x05, &cv));
let recs = walk_mkb(&mkb);
assert!(
!is_variant_mkb(&recs),
"fixture check: this must be a CLASSICAL MKB, so the magic is the \
only thing that can let a match through"
);
let dk = DeviceKey {
key: dkey,
node: 0x0C00,
uv: UV,
u_mask_shift: U_MASK_SHIFT,
};
let m = walk_processing_key(&recs, std::slice::from_ref(&dk))
.expect("the planted cvalue opens the verify magic for this key");
assert_eq!(m.kp, aesg3(&dkey, 1));
assert_eq!(m.uv, UV);
assert_eq!(m.cvalue, cv);
assert_eq!(m.cvalue_index, 0);
// And the magic is genuinely load-bearing: perturb the Verify-Media-Key
// record and the same key, slot and cvalue must stop matching.
let mut bad = recs.clone();
bad.iter_mut()
.find(|r| r.rec_type == 0x86)
.expect("0x86 present")
.body[0] ^= 0x01;
assert!(
walk_processing_key(&bad, std::slice::from_ref(&dk)).is_none(),
"a classical match must be authenticated by the verify magic"
);
// ...and so is the cvalue: one bit off and the candidate no longer opens
// the magic.
let mut bad = recs.clone();
bad.iter_mut()
.find(|r| r.rec_type == 0x05)
.expect("0x05 present")
.body[0] ^= 0x01;
assert!(walk_processing_key(&bad, std::slice::from_ref(&dk)).is_none());
}
} }
+840 -101
View File
File diff suppressed because it is too large Load Diff
+1 -9
View File
@@ -99,11 +99,7 @@ pub mod coding_type {
/// Secondary Dolby Digital Plus audio (BD-ROM convention). /// Secondary Dolby Digital Plus audio (BD-ROM convention).
pub const AC3_PLUS_SECONDARY: u8 = 0xA1; pub const AC3_PLUS_SECONDARY: u8 = 0xA1;
/// Secondary DTS-HD audio — DTS Express / DTS-HD LBR, a LOSSY low-bitrate /// Secondary DTS-HD audio (lossless MA, not lossy HR) (BD-ROM convention).
/// stream for picture-in-picture and BD-J mixing (BD-ROM Part 3
/// `stream_coding_type` table). The lossless primary is [`DTS_HD_MA`]
/// (0x86); this code is its lossy secondary counterpart, parallel to
/// [`AC3_PLUS_SECONDARY`] (0xA1) on the Dolby side.
pub const DTS_HD_SECONDARY: u8 = 0xA2; pub const DTS_HD_SECONDARY: u8 = 0xA2;
} }
@@ -114,10 +110,6 @@ pub mod coding_type {
pub mod pes_stream_id { pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`). /// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0; pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads. /// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD; pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux. /// padding_stream — stuffing bytes only, no payload to demux.
-43
View File
@@ -443,49 +443,6 @@ mod tests {
); );
} }
/// The length guard is a FLOOR, not a ceiling: `descramble_sector` is a
/// no-op below one sector, and processes the FIRST sector of anything at
/// least that long (the loop is `.take(2048)`). `css::descramble_sector` is
/// a public entry taking `&mut [u8]` of any length, so a caller handing it a
/// multi-sector buffer must get its first sector descrambled — a guard that
/// rejected over-long buffers would hand that caller its ciphertext back
/// unchanged, with the scramble flag cleared as if it had worked.
#[test]
fn descramble_processes_the_first_sector_of_an_over_long_buffer() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
// Two sectors' worth of buffer; only the first is a sector.
let mut buf = vec![0xAAu8; 4096];
buf[0x14] = 0x30;
buf[0x54..0x59].copy_from_slice(&seed);
let original = buf.clone();
descramble_sector(&title_key, &mut buf);
assert_ne!(
&buf[0x80..0x800],
&original[0x80..0x800],
"the first sector's body must be descrambled"
);
assert_eq!(buf[0x14] & 0x30, 0x00, "and its scramble flag cleared");
assert_eq!(
&buf[2048..4096],
&original[2048..4096],
"bytes past the first sector must be left untouched"
);
// The result must equal what a caller gets by passing exactly one
// sector — the same transform, not a length-dependent one.
let mut one = original[..2048].to_vec();
descramble_sector(&title_key, &mut one);
assert_eq!(
&buf[..2048],
&one[..],
"the first sector must descramble identically either way"
);
}
/// Descramble is keyed by `title_key XOR seed`: two different title keys /// Descramble is keyed by `title_key XOR seed`: two different title keys
/// produce two different bodies for the same scrambled input. A cipher that /// produce two different bodies for the same scrambled input. A cipher that
/// ignored the title key (or mixed it in wrongly) would yield identical /// ignored the title key (or mixed it in wrongly) would yield identical
+79 -687
View File
File diff suppressed because it is too large Load Diff
-566
View File
@@ -450,64 +450,6 @@ mod tests {
} }
} }
/// `descramble_matches` is the ONLY gate between the LFSR search and a key
/// handed back to the caller: both [`recover_title_key`] and the crib-driven
/// `crack_title_key_inner` return a candidate only if this says the key
/// really descrambles the sector to the known plaintext. A body that always
/// answered `true` would let the first spurious LFSR-seed match through as
/// the title key — the ripper would then descramble the whole title with a
/// key that opens nothing, producing garbage rather than a "no key" error.
///
/// Pinned both directions: the genuine key is accepted, and EVERY key one
/// bit away from it is rejected. The one-bit neighbours are the strongest
/// form of wrong key — a gate that only rejects wildly different keys would
/// still pass a near-miss out of the 2^16 seed search.
#[test]
fn descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
assert!(
descramble_matches(&sector, &title_key, &PES),
"the key the sector was scrambled with must be accepted"
);
for byte in 0..5usize {
for bit in 0..8u32 {
let mut wrong = title_key;
wrong[byte] ^= 1u8 << bit;
assert!(
!descramble_matches(&sector, &wrong, &PES),
"key differing only in byte {byte} bit {bit} must be rejected"
);
}
}
}
/// The gate is applied to a COPY: verifying a candidate must not modify the
/// caller's sector. `recover_title_key` runs the gate and then hands the
/// sector on to be descrambled for real — if verification descrambled in
/// place, that second descramble would run over already-transformed bytes
/// (and, worse, a rejected candidate would leave the sector corrupted).
#[test]
fn descramble_matches_does_not_disturb_the_caller_s_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
let before = sector.clone();
assert!(descramble_matches(&sector, &title_key, &PES));
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
assert_eq!(
sector, before,
"verification must leave the sector byte-for-byte unchanged"
);
}
/// MANDATORY (Task C.1): the crib-based entry point crack_title_key — /// MANDATORY (Task C.1): the crib-based entry point crack_title_key —
/// no plaintext supplied — recovers a round-tripping key when the /// no plaintext supplied — recovers a round-tripping key when the
/// cleartext ends in a periodic run that continues into 0x80. /// cleartext ends in a periodic run that continues into 0x80.
@@ -629,512 +571,4 @@ mod tests {
let _ = crack_title_key(&sector); let _ = crack_title_key(&sector);
} }
} }
// ── entry-point guards on caller- and disc-supplied lengths ────────────
/// A sector buffer that ENDS inside the encrypted region must be refused,
/// not sliced.
///
/// `recover_title_key` slices `sector[0x80..0x8A]` unconditionally after its
/// length guard. The existing short-sector test uses `SECTOR_BYTES - 1`,
/// which is still long enough for that slice to succeed — so the guard was
/// never the thing producing the `None`, and dropping it (or weakening the
/// `||` to `&&`, which a full-length crib satisfies) changed nothing
/// observable. On a real short read this is an out-of-bounds panic on the
/// rip thread.
#[test]
fn recover_rejects_a_sector_that_ends_inside_the_encrypted_region() {
for len in [0x81usize, 0x85, 0x89] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so no other guard fires first
assert!(
recover_title_key(&sector, &PES).is_none(),
"a {len}-byte buffer cannot supply ten ciphertext bytes at 0x80"
);
}
}
/// A buffer LONGER than one sector is still one sector: both entry points
/// read the first `SECTOR_BYTES` and must recover the key from it.
///
/// Callers read DVD data in multi-sector blocks, so an over-long slice is
/// the normal case, not an exotic one. A length guard that rejected it
/// (`len > SECTOR_BYTES` instead of `<`) would make every block-read caller
/// silently unable to crack anything.
#[test]
fn a_buffer_longer_than_one_sector_still_yields_its_key() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let mut padded = sector.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
recover_title_key(&padded, &PES),
Some(title_key),
"a two-and-a-bit-sector buffer must still recover the first sector's key"
);
let (periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
let mut padded = periodic.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
crack_title_key(&padded),
crack_title_key(&periodic),
"padding past the sector must not change the crack result"
);
assert!(crack_title_key(&padded).is_some());
}
/// `recover_title_key` accepts MORE than ten bytes of known plaintext, and
/// uses all of it: the extra bytes tighten the `descramble_matches` gate.
/// The ten-byte figure is a MINIMUM (the cipher is iterated ten times), not
/// an exact requirement — a guard reading it as an upper bound would reject
/// every caller that knows a longer crib.
#[test]
fn recover_accepts_more_than_ten_bytes_of_known_plaintext() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let long_plain: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(37).wrapping_add(5))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &long_plain);
assert_eq!(
recover_title_key(&sector, &long_plain),
Some(title_key),
"64 bytes of known plaintext must be accepted, not rejected as \
'more than ten'"
);
}
/// The scramble-flag gate on a sector whose BODY really is ciphertext.
///
/// Both entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
/// unscrambled sector has no title key to recover, and its bytes at 0x80
/// are already plaintext. Every prior test of this gate used an all-zero or
/// all-`0x11` sector, where the recovery would have found nothing anyway —
/// so widening the mask test (`&` to `|`, which makes it true for EVERY
/// flag byte) produced the same `None` and went unseen.
///
/// Here the sector is genuinely scrambled and its key IS recoverable; only
/// the cleared flag stands in the way. If the gate stops working, both
/// functions start returning keys for sectors the disc says are in the
/// clear.
#[test]
fn a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key(&sector, &PES),
Some(title_key),
"fixture check: with the flag set this sector's key IS recoverable"
);
sector[FLAG_BYTE] = 0x00;
assert_eq!(
recover_title_key(&sector, &PES),
None,
"scramble bits clear → no title key, even though one could be found"
);
let (mut periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
assert!(
crack_title_key(&periodic).is_some(),
"fixture check: with the flag set this sector cracks"
);
assert!(
attack_crib(&periodic).is_some(),
"fixture check: with the flag set this sector has a usable crib"
);
periodic[FLAG_BYTE] = 0x00;
assert_eq!(
crack_title_key(&periodic),
None,
"scramble bits clear → no crack, even though one would succeed"
);
// `attack_crib` carries its own copy of the same gate, and it is the one
// that actually stops the crack (`crack_title_key`'s is defensive
// duplication). The crib doubles as the decrypt path's cached-key
// oracle, so a widened mask there would hand that path a "predicted
// plaintext" for sectors that were never scrambled.
assert_eq!(
attack_crib(&periodic),
None,
"an unscrambled sector has no predicted plaintext to offer"
);
}
// ── descramble_matches: the verification gate's own mechanics ──────────
/// The gate must verify a candidate against the sector's CIPHERTEXT
/// regardless of what the sector's own flag byte says.
///
/// `descramble_matches` forces `0x10` on its copy precisely because
/// [`super::lfsr::descramble_sector`] is a no-op when the scramble bits are
/// clear — without that, verifying a scrambled-but-unflagged sector
/// compares raw ciphertext against the crib, and every candidate key is
/// rejected. Nothing exercised it: every fixture already had the flag set,
/// where forcing the bit is a no-op.
#[test]
fn descramble_matches_forces_the_scramble_flag_on_its_own_copy() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
sector[FLAG_BYTE] = 0x00;
assert!(
descramble_matches(&sector, &title_key, &PES),
"the body is ciphertext and the key is right — the gate must \
descramble it even though the flag byte says otherwise"
);
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
}
/// The gate compares the WHOLE supplied plaintext, clamped to the encrypted
/// region.
///
/// Two properties in one, because they are the two halves of
/// `plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
///
/// - it must compare beyond the first sixteen bytes, or a key that opens
/// only the head of the crib is accepted; and
/// - it must never compare past the end of the sector — a caller that
/// knows more plaintext than the 1920-byte encrypted region holds
/// otherwise indexes off the end of the buffer and panics.
#[test]
fn descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let body: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(29).wrapping_add(3))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &body);
assert!(descramble_matches(&sector, &title_key, &body));
// A crib agreeing for the first 16 bytes and diverging after must be
// rejected: the comparison window is the crib's length, not a fixed
// prefix.
let mut tail_wrong = body.clone();
tail_wrong[40] ^= 0xFF;
assert!(
!descramble_matches(&sector, &title_key, &tail_wrong),
"a crib that diverges at byte 40 must not match"
);
assert_eq!(
tail_wrong[..16],
body[..16],
"fixture check: the first 16 bytes are identical, so only a \
comparison that runs past them can tell these apart"
);
// A crib LONGER than the encrypted region: the comparison is clamped to
// the sector, not run off the end of it.
let plain_len = SECTOR_BYTES - ENCRYPTED_START;
let mut over_long = vec![0u8; plain_len + 10];
let (full_sector, full_body) = synth_sector(&title_key, &seed, &[0x00u8; 10]);
over_long[..plain_len].copy_from_slice(&full_body[ENCRYPTED_START..]);
assert!(
descramble_matches(&full_sector, &title_key, &over_long),
"a crib longer than the encrypted region must be clamped, not \
compared past the end of the sector"
);
}
// ── attack_crib: known-answer vectors ──────────────────────────────────
//
// `attack_crib` is BOTH the cracker's known plaintext and the decrypt
// path's "did the cached key descramble correctly?" oracle. Until now it
// was only ever exercised end-to-end through `crack_title_key`, on a
// fixture whose periodic run covered 39 bytes (0x59..0x80) — long enough
// that the run start, the cycle count and the `i % best_p` wrap were all
// slack. A crib that silently drifts costs a rip its title key.
/// Build a sector whose clear header ends in a `period`-length repeating
/// run of exactly `run_len` bytes immediately before 0x80.
///
/// The run is anchored to ABSOLUTE sector offset (`sec[x] = pat[x % period]`),
/// which is what makes "the run continues past 0x80" a statement independent
/// of the code under test: the byte at `0x80 + i` of the underlying
/// plaintext is `pat[(0x80 + i) % period]`.
///
/// Everything before the run is `0x00` (the pattern bytes are all >= 0xD0,
/// so the run cannot be extended backwards by accident), and the encrypted
/// region is filled with `0xFF` — so a crib that reads past 0x80 into
/// "ciphertext" is immediately visible.
fn sector_with_trailing_run(period: usize, run_len: usize) -> Vec<u8> {
assert!(
run_len < ENCRYPTED_START,
"the run lives in the clear header"
);
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for x in (ENCRYPTED_START - run_len)..ENCRYPTED_START {
sector[x] = pat[x % period];
}
sector
}
/// The crib the run PREDICTS: the periodic pattern continued past 0x80.
fn expected_crib(period: usize) -> [u8; 10] {
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
let mut out = [0u8; 10];
for (i, o) in out.iter_mut().enumerate() {
*o = pat[(ENCRYPTED_START + i) % period];
}
out
}
/// KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly
/// at 0x80, the crib is the run continued forward — the same ten bytes for
/// every run length, because the prediction depends only on the pattern and
/// the phase, never on how many cycles happened to be visible.
///
/// The short lengths are the load-bearing ones: at `run_len = 11` the crib
/// window starts at 0x76 and is only 10 bytes from the end of the header, so
/// any drift in `plain_start`, in `cycles * best_p`, or in the `i % best_p`
/// wrap reads the 0xFF "ciphertext" instead of the run.
#[test]
fn attack_crib_predicts_the_periodic_run_continuing_past_0x80() {
for &run_len in &[11usize, 12, 13, 14, 15, 16, 20, 31] {
let sector = sector_with_trailing_run(5, run_len);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(5)),
"period-5 run of {run_len} bytes must predict the run continuing"
);
}
}
/// The same known answer across several periods, including a period that
/// does NOT divide 0x80 (so the crib's phase is non-zero and a body that
/// restarted the pattern at index 0 gives a different answer).
#[test]
fn attack_crib_recovers_the_run_period_and_phase() {
// 0x80 % period: 3 for 5, 2 for 6, 2 for 7, 8 for 0x18 — all non-zero,
// so the predicted first byte is NOT pat[0] in any of these cases.
for &period in &[5usize, 6, 7, 0x18] {
let sector = sector_with_trailing_run(period, 3 * period + 1);
let crib =
attack_crib(&sector).unwrap_or_else(|| panic!("no crib for period {period}"));
assert_eq!(crib, expected_crib(period), "period {period}");
assert_ne!(
crib[0], 0xD0,
"period {period} does not divide 0x80, so the crib must not \
start at pattern index 0"
);
assert!(
crib.iter().all(|&b| b != 0xFF),
"period {period}: the crib must never contain a byte read from \
the encrypted region"
);
}
}
/// A run of exactly ONE cycle (plus the trivial tail the detector counts) is
/// not enough to predict forward: [`attack_crib`] requires at least two full
/// cycles. Weakening that guard would let a one-off byte sequence be
/// declared periodic and produce a confidently wrong crib — which the
/// decrypt path uses as its "is my cached key still right?" oracle.
#[test]
fn attack_crib_refuses_a_run_shorter_than_two_cycles() {
// period 8, run of 9 bytes: best_plen = 8, 8 / 8 == 1 cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 9)), None);
// period 0x18, run of 0x19 bytes: one cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(0x18, 0x19)), None);
// ...and one more byte of run does not conjure a second cycle either.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 10)), None);
}
/// A header with no repeating tail at all yields no crib. Asserted on a
/// header whose bytes are pairwise distinct right up to 0x80, so no cycle
/// length in 2..0x2F can match even one byte.
#[test]
fn attack_crib_refuses_a_header_with_no_periodic_tail() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
// 0x00..0x80 strictly increasing: sec[a] == sec[b] iff a == b, so the
// detector's `sec[0x7f - (j % i)] == sec[0x7f - j]` needs j % i == j,
// which the scan's starting `j = i + 1` already excludes.
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = x as u8;
}
assert_eq!(attack_crib(&sector), None);
// And the cracker built on it reports no key rather than guessing.
assert_eq!(crack_title_key(&sector), None);
}
/// `attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
/// own length guard is the only thing between a short buffer and an
/// out-of-bounds read. Nothing reached it: every caller-level test used a
/// full sector, and the entry points' guards fire first.
#[test]
fn attack_crib_refuses_a_buffer_shorter_than_a_sector() {
for len in [0x15usize, 0x40, 0x7F, SECTOR_BYTES - 1] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so the flag half cannot fire
assert_eq!(
attack_crib(&sector),
None,
"a {len}-byte buffer is not a sector"
);
}
}
/// A header that is periodic ALL THE WAY to offset 0 must not walk the
/// backward scan off the front of the sector.
///
/// The detector counts backwards from 0x7f while `j < 0x80`. On a fully
/// periodic header the run never breaks, so `j` reaches 0x7f and the bound
/// is the ONLY thing that stops it — one step further and `0x7f - j`
/// underflows a `usize` and panics. A constant or fully-patterned 128-byte
/// header is ordinary DVD data (padding, a run of zeros), not a crafted
/// input, and every existing fixture had a filler/run boundary well before
/// offset 0 that stopped the scan early.
#[test]
fn attack_crib_survives_a_header_that_is_periodic_to_offset_zero() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x30;
let period = 5usize;
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = pat[x % period];
}
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
// The FLAG byte sits inside the header at 0x14, so it interrupts the
// pattern there; re-lay it and accept that 0x14 breaks the run — the
// scan still reaches offset 0x15 - 1 = 0x14 going backwards, i.e.
// j = 0x7f - 0x14 = 0x6b, well short of the bound. Instead put the
// scramble flag bits into a byte value that IS the pattern's.
sector[FLAG_BYTE] = pat[FLAG_BYTE % period];
assert_ne!(
sector[FLAG_BYTE] & 0x30,
0,
"fixture check: the pattern byte at 0x14 must itself carry \
scramble bits, so the header stays unbroken"
);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(period)),
"a fully periodic header must predict its own continuation, and \
the backward scan must stop at offset 0"
);
}
/// The crib is read from the CLEAR header only. A run that reaches 0x80 must
/// predict from the header bytes, never from the encrypted region — the
/// previously-fixed bug this function's doc comment records. Pinned by
/// rewriting the encrypted region and requiring the crib not to move.
#[test]
fn attack_crib_is_independent_of_the_encrypted_region() {
let base = sector_with_trailing_run(5, 11);
let crib = attack_crib(&base).expect("crib");
for fill in [0x00u8, 0x5A, 0xD1, 0xFF] {
let mut s = base.clone();
for b in s[ENCRYPTED_START..].iter_mut() {
*b = fill;
}
assert_eq!(
attack_crib(&s),
Some(crib),
"the crib must not depend on the encrypted region (fill {fill:#04x})"
);
}
}
// ── recover_title_key_from_plain: input-length guard ───────────────────
/// `recover_title_key_from_plain` unconditionally builds a 10-byte keystream
/// buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard
/// is the only thing standing between a short slice and an
/// index-out-of-bounds PANIC.
///
/// Nothing reached that guard before: `recover_title_key` rejects
/// `plain.len() < 10` at its own door and always hands on exactly ten
/// ciphertext bytes, and `crack_title_key_inner` always passes a fixed
/// `[u8; 10]` crib. The guard is a live contract for any future caller and
/// was executed by no test at either boundary.
#[test]
fn recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input() {
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let full = [0xA5u8; 10];
for n in 0..10usize {
assert_eq!(
recover_title_key_from_plain(&full[..n], &full, &seed),
None,
"{n} ciphertext bytes is fewer than the ten the cipher iterates"
);
assert_eq!(
recover_title_key_from_plain(&full, &full[..n], &seed),
None,
"{n} plaintext bytes is fewer than the ten the cipher iterates"
);
}
// Exactly ten of each is ACCEPTED as far as the search — the boundary is
// `< 10`, not `<= 10`. (Whether this particular keystream has a seed is
// immaterial; what must not happen is an early `None` from the guard.)
// Proven through the round-trip fixture, whose inputs are exactly ten
// bytes and which does recover its key.
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key_from_plain(
&sector[ENCRYPTED_START..ENCRYPTED_START + 10],
&PES,
&seed
),
Some(title_key),
"exactly ten bytes of each input must run the search, not trip the guard"
);
}
/// The seed XOR-back ([`recover_title_key_from_plain`]'s last step) is what
/// turns the recovered LFSR key into the TITLE key: `key ^= sector_seed`.
/// Pinned as a known answer across seeds that differ only in one byte — the
/// same ciphertext/plaintext pair therefore must yield title keys differing
/// in exactly that byte.
///
/// Without this, a body that ORed the seed in (or dropped the step) still
/// round-trips on any fixture whose seed is zero, and on the non-zero ones
/// the failure looks like "no key found" rather than a wrong step.
#[test]
fn recover_title_key_from_plain_xors_the_sector_seed_back_out() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let crypted = &sector[ENCRYPTED_START..ENCRYPTED_START + 10];
// The cipher is seeded from `title_key XOR seed`, so re-running the SAME
// ciphertext/plaintext against a seed differing in one byte must return
// a title key differing in exactly that byte — the XOR is a bijection.
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &seed),
Some(title_key)
);
for byte in 0..5usize {
for bit in [0u32, 3, 7] {
let mut alt_seed = seed;
alt_seed[byte] ^= 1u8 << bit;
let mut expected = title_key;
expected[byte] ^= 1u8 << bit;
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &alt_seed),
Some(expected),
"seed byte {byte} bit {bit} must XOR straight through to the \
title key"
);
}
}
}
} }
+63 -661
View File
@@ -74,14 +74,9 @@ pub fn set_decrypt_threads(n: usize) {
DECRYPT_THREADS.store(clamped, Ordering::Relaxed); DECRYPT_THREADS.store(clamped, Ordering::Relaxed);
// Drop the existing pool. Next decrypt_pool() call rebuilds with // Drop the existing pool. Next decrypt_pool() call rebuilds with
// the new resolved thread count. // the new resolved thread count.
// if let Ok(mut guard) = DECRYPT_POOL.write() {
// Recover the guard on poisoning, exactly as `decrypt_pool` does. Skipping *guard = None;
// the swap on a poisoned lock silently kept the STALE pool alive while the }
// atomic above already reported the new thread count, so the setting appeared
// to take effect and never did. The pool Arc is immutable once stored, so a
// prior panic cannot have left it half-written.
let mut guard = DECRYPT_POOL.write().unwrap_or_else(|e| e.into_inner());
*guard = None;
} }
/// Get (or lazily build) the active rayon thread pool. Returns an /// Get (or lazily build) the active rayon thread pool. Returns an
@@ -189,34 +184,6 @@ pub enum Phase {
Odd, Odd,
} }
/// Does this unit belong to the phase we hold the key for?
///
/// An FMTS forensic segment interleaves two variants at the unit level: the
/// disc carries both, and we hold the key for exactly one parity. Decrypting
/// the alternate half with our key produces garbage; leaving it as ciphertext
/// is correct, because the muxer drops untouched ciphertext cleanly.
///
/// `Phase::All` means the whole range is ours (the non-forensic case).
///
/// This lived inline inside `apply_aacs_map`'s per-unit closure, where nothing
/// could reach it: a mutation run flipped the `-` to `+` and the `/` to `*` in
/// the index arithmetic and every test still passed. Getting either wrong
/// silently decrypts the wrong half of a forensic segment.
fn unit_is_our_phase(unit_lba: u32, range_start: u32, unit_sectors: u32, phase: Phase) -> bool {
let want_odd = match phase {
Phase::All => return true,
Phase::Even => false,
Phase::Odd => true,
};
// `saturating_sub` and `max(1)`: both inputs come from the key map, which is
// built from disc structure. A unit below its own range start, or a zero
// unit size, means the map is malformed — that must not panic (debug
// overflow / divide-by-zero) inside a library used by a long-running
// service. Unit 0 of the range is even, which is the safe default.
let unit_ix = unit_lba.saturating_sub(range_start) / unit_sectors.max(1);
(unit_ix % 2 == 1) == want_odd
}
/// Proactive AACS key-selection map: which held unit key decrypts each LBA of a /// Proactive AACS key-selection map: which held unit key decrypts each LBA of a
/// title's encrypted content, decided ONCE before mux from the disc's CPS-unit /// title's encrypted content, decided ONCE before mux from the disc's CPS-unit
/// (and, later, FMTS segment) structure — never by trial-decrypt-and-check per /// (and, later, FMTS segment) structure — never by trial-decrypt-and-check per
@@ -360,11 +327,11 @@ impl AacsKeyMap {
if sectors == 0 { if sectors == 0 {
return; return;
} }
if let Some(last) = plan.last_mut() if let Some(last) = plan.last_mut() {
&& last.start_lba.saturating_add(last.sector_count) == lba if last.start_lba.saturating_add(last.sector_count) == lba {
{ last.sector_count += sectors;
last.sector_count += sectors; return;
return; }
} }
plan.push(crate::disc::Extent { plan.push(crate::disc::Extent {
start_lba: lba, start_lba: lba,
@@ -414,30 +381,11 @@ impl AacsKeyMap {
/// decorator can dispatch uniformly. A map index outside the held pool is a /// decorator can dispatch uniformly. A map index outside the held pool is a
/// fail-loud [`Error::DecryptFailed`]: the resolver's job is to guarantee every /// fail-loud [`Error::DecryptFailed`]: the resolver's job is to guarantee every
/// selectable index is present, so a gap here is a resolver bug, not silent loss. /// selectable index is present, so a gap here is a resolver bug, not silent loss.
/// Decrypt `buf` with a resolved AACS key map. Thin wrapper over
/// [`decrypt_span`] — the map is the AACS scheme's input, not a second
/// orchestrator.
pub(crate) fn decrypt_sectors_mapped( pub(crate) fn decrypt_sectors_mapped(
buf: &mut [u8], buf: &mut [u8],
keys: &DecryptKeys, keys: &DecryptKeys,
base_lba: u32, base_lba: u32,
map: &AacsKeyMap, map: &AacsKeyMap,
) -> Result<(), crate::error::Error> {
let mut keys = keys.clone();
decrypt_span(buf, &mut keys, base_lba, Some(map), None).map(|_| ())
}
/// AACS scheme step: apply `map`'s per-unit keys to `buf`.
///
/// A SCHEME, not a policy. It reports what it could not open by returning
/// `Err(DecryptFailed)`; the decision that an unopenable unit must never be
/// emitted belongs to [`decrypt_span`], which is the one place that decides it
/// for every scheme.
fn apply_aacs_map(
buf: &mut [u8],
keys: &DecryptKeys,
base_lba: u32,
map: &AacsKeyMap,
) -> Result<(), crate::error::Error> { ) -> Result<(), crate::error::Error> {
let (unit_keys, rdk, format) = match keys { let (unit_keys, rdk, format) = match keys {
DecryptKeys::Aacs { DecryptKeys::Aacs {
@@ -488,31 +436,20 @@ fn apply_aacs_map(
return; return;
} }
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors); let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
// No range covers this LBA. That is expected for clear filesystem / nav // No range covers this LBA → the map keys no content here, so pass the
// on a whole-disc read — but "the map has no key here" and "there is // unit through untouched (clear filesystem / nav on a whole-disc read).
// nothing to decrypt here" are different statements, and only the
// second makes passing the unit through correct.
//
// An ENCRYPTED unit outside every range is content we cannot key: on a
// multi-CPS disc that is an orphan clip referenced by no playlist, so
// it sits in no title extent and therefore in no range. Emitting it
// verbatim ships ciphertext where plaintext is meant to be, and extract
// then counts those bytes as good and reports the file complete.
//
// The split-unit branch immediately above already draws exactly this
// distinction. This one did not, so it was reached before the
// `aacs_unit_encrypted` gate below ever ran.
let Some((key_idx, phase, range_start)) = map.entry_for(unit_lba) else { let Some((key_idx, phase, range_start)) = map.entry_for(unit_lba) else {
if aacs::content::aacs_unit_seed_encrypted(chunk, format) {
verify_failed.store(true, std::sync::atomic::Ordering::Relaxed);
}
return; return;
}; };
// PHASE GATE (FMTS forensic segment): the segment interleaves two variants // PHASE GATE (FMTS forensic segment): the segment interleaves two variants
// at the unit level. Decrypt ONLY our parity; leave the alternate half as // at the unit level. Decrypt ONLY our parity; leave the alternate half as
// ciphertext (the muxer drops untouched ciphertext cleanly — no garble). // ciphertext (the muxer drops untouched ciphertext cleanly — no garble).
if !unit_is_our_phase(unit_lba, range_start, unit_sectors, phase) { if matches!(phase, Phase::Even | Phase::Odd) {
return; // alternate half — leave as-is let unit_ix = (unit_lba - range_start) / unit_sectors;
let is_odd = unit_ix % 2 == 1;
if is_odd != matches!(phase, Phase::Odd) {
return; // alternate half — leave as-is
}
} }
// Gate on the authoritative encrypted flag ONLY (CPI bits in the clear // Gate on the authoritative encrypted flag ONLY (CPI bits in the clear
// seed): a clear unit is left untouched; an encrypted unit is decrypted // seed): a clear unit is left untouched; an encrypted unit is decrypted
@@ -578,8 +515,7 @@ pub fn decrypt_sectors(
keys: &mut DecryptKeys, keys: &mut DecryptKeys,
unit_key_idx: usize, unit_key_idx: usize,
) -> Result<usize, crate::error::Error> { ) -> Result<usize, crate::error::Error> {
let _ = unit_key_idx; decrypt_sectors_impl(buf, keys, unit_key_idx, None)
decrypt_span(buf, keys, 0, None, None)
} }
/// Legacy alias of [`decrypt_sectors`]. Under the keymap-only model AACS decrypts /// Legacy alias of [`decrypt_sectors`]. Under the keymap-only model AACS decrypts
@@ -596,50 +532,28 @@ pub fn decrypt_sectors_in_content(
base_lba: u32, base_lba: u32,
content_ranges: &[(u32, u32)], content_ranges: &[(u32, u32)],
) -> Result<usize, crate::error::Error> { ) -> Result<usize, crate::error::Error> {
let _ = unit_key_idx; decrypt_sectors_impl(buf, keys, unit_key_idx, Some((base_lba, content_ranges)))
decrypt_span(buf, keys, base_lba, None, Some((base_lba, content_ranges)))
} }
/// THE decrypt orchestrator. Every path into this crate's decryption goes fn decrypt_sectors_impl(
/// through here.
///
/// How a disc decrypts is one process — resolve a key for this span, apply it,
/// and refuse if no key can be proven. Only the resolve-and-apply step is
/// scheme-specific. This function owns the loop and the refusal; the schemes
/// below supply only what genuinely differs between AACS, CSS and clear media.
///
/// That split exists because its absence caused six separate defects in one
/// release. There used to be TWO top-level paths — this one for CSS and clear,
/// and a wholly separate `decrypt_sectors_mapped` for AACS whose arm here was a
/// bare `return Err` stub — so each scheme decided its own answer to "there is
/// no key for these bytes" and nothing held them to the same one. CSS drifted to
/// descrambling with a key it had just proven stale; the mapped path drifted to
/// passing an unkeyable encrypted unit through as ciphertext. Both looked like
/// success to the caller.
///
/// Adding a scheme means adding an arm here, which means answering the refusal
/// question. That is the point.
fn decrypt_span(
buf: &mut [u8], buf: &mut [u8],
keys: &mut DecryptKeys, keys: &mut DecryptKeys,
base_lba: u32, // Unused now that AACS decrypts via the key map only; the CSS arm self-gates on
map: Option<&AacsKeyMap>, // its per-sector scramble flag and `None` is a no-op. Kept so the wrapper
// signatures (decrypt_sectors / _in_content) stay stable for CSS/None callers.
_unit_key_idx: usize,
_content: Option<(u32, &[(u32, u32)])>, _content: Option<(u32, &[(u32, u32)])>,
) -> Result<usize, crate::error::Error> { ) -> Result<usize, crate::error::Error> {
let dropped: usize = match keys { let dropped: usize = match keys {
DecryptKeys::None => 0, DecryptKeys::None => 0,
DecryptKeys::Aacs { .. } => { DecryptKeys::Aacs { .. } => {
// AACS decrypts EXCLUSIVELY through a resolved key map: the map keys // AACS decrypts EXCLUSIVELY through the resolved key map
// every content unit up front and a missing key fails at RESOLVE // (`decrypt_sectors_mapped`): the map keys every content unit up front,
// time. No map here means an AACS reader was built without // and a missing key fails at RESOLVE time. The old trial-decrypt path
// installing one — the old trial-decrypt path (try each held key, // (try each held key, keep the first-tried plaintext on a miss) is gone
// keep the first-tried plaintext on a miss) is gone precisely // — reaching it means an AACS reader was built without installing its
// because it silently applied wrong keys. // key map, which would silently apply a wrong key. Fail loud instead.
let Some(map) = map else { return Err(crate::error::Error::DecryptFailed);
return Err(crate::error::Error::DecryptFailed);
};
apply_aacs_map(buf, keys, base_lba, map)?;
0
} }
DecryptKeys::Css { title_key } => { DecryptKeys::Css { title_key } => {
// CSS SELF-recovers: the title key changes per VOB region and is // CSS SELF-recovers: the title key changes per VOB region and is
@@ -648,7 +562,8 @@ fn decrypt_span(
// `css::descramble_region`), and CSS does not need the post-decrypt // `css::descramble_region`), and CSS does not need the post-decrypt
// recovery seam that AACS key-fetch / FMTS segment-skip use (those DO // recovery seam that AACS key-fetch / FMTS segment-skip use (those DO
// consume external inputs a `decrypt_sectors` caller cannot supply). // consume external inputs a `decrypt_sectors` caller cannot supply).
css::descramble_region(buf, title_key)? css::descramble_region(buf, title_key);
0
} }
}; };
Ok(dropped) Ok(dropped)
@@ -695,49 +610,6 @@ mod tests {
// ── `decrypt_sectors_in_content` (now a legacy alias of `decrypt_sectors`) ── // ── `decrypt_sectors_in_content` (now a legacy alias of `decrypt_sectors`) ──
/// `DecryptKeys::None` is a no-op even with a content map + scrambled bytes. /// `DecryptKeys::None` is a no-op even with a content map + scrambled bytes.
/// A forensic map's read plan is NOT the extents it was given — and that is
/// the precondition the mux's provenance guard keys on.
///
/// A clip's feed span is measured over the FULL extents at scan time, while
/// the mux reads this reduced plan, so the byte offsets stamped on frames
/// and the offsets recorded in the spans describe different streams. The
/// deficit accumulates, so every frame after the first segment resolves to
/// an earlier clip than it came from. The spans still tile each other, so
/// the tiling check cannot see it; the mux compares the plan against the
/// full extents instead and stops trusting provenance when they differ.
#[test]
fn a_forensic_read_plan_drops_units_the_full_extents_include() {
let full = vec![crate::disc::Extent {
start_lba: 1000,
sector_count: 60,
}];
// No forensic segment: the plan IS the extents, byte for byte, so
// provenance stays trustworthy on an ordinary disc.
let plain = AacsKeyMap::from_ranges_phased(vec![(1000, 1060, 5, Phase::All)]);
assert_eq!(
plain.read_plan(&full, 3),
full,
"a non-forensic map must return the extents unchanged"
);
// With alternate phases, units are omitted — fewer sectors are read
// than the spans describe.
let phased = AacsKeyMap::from_ranges_phased(vec![(1000, 1060, 5, Phase::Even)]);
let plan = phased.read_plan(&full, 3);
let planned: u32 = plan.iter().map(|e| e.sector_count).sum();
let whole: u32 = full.iter().map(|e| e.sector_count).sum();
assert!(
planned < whole,
"a forensic segment must drop units: planned {planned} of {whole}"
);
assert_ne!(
plan, full,
"the plan differs from the extents, which is exactly what the mux \
detects before deciding whether a byte offset means anything"
);
}
#[test] #[test]
fn content_gate_none_keys_is_noop() { fn content_gate_none_keys_is_noop() {
let mut keys = DecryptKeys::None; let mut keys = DecryptKeys::None;
@@ -763,92 +635,6 @@ mod tests {
); );
} }
/// `decrypt_sectors_in_content` is the entry point `DecryptingSectorSource`
/// dispatches to whenever a content map is installed (`sector/decrypting.rs`
/// line ~211), so it is on the live read path for every mapped rip. It must
/// actually DECRYPT. The two `_is_noop` tests above only assert its `usize`
/// return is `0` — which is what a body replaced by `Ok(0)` also returns, so
/// neither one constrains it at all.
///
/// Here a genuinely scrambled CSS sector goes in and the CONSTRUCTED
/// plaintext must come out. Anything that skips `css::descramble_region` —
/// including a body that just reports `Ok(0)` — leaves ciphertext in the
/// buffer and the caller muxes scrambled MPEG at exit 0.
///
/// Expected bytes come from the plaintext this test built BEFORE scrambling
/// (CSS scrambles only 0x80..2048; the header stays clear), not from
/// re-running any descramble routine.
#[test]
fn content_gate_css_actually_descrambles_the_buffer() {
const RUN_START: usize = 0x59;
const SEED_OFFSET: usize = 0x54;
const PERIOD: usize = 8;
let title_key = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let mut plaintext = vec![0u8; 2048];
plaintext[0x00..0x04].copy_from_slice(&css::PACK_START);
plaintext[0x14] = 0x10; // CSS scramble flag (DVD-Video sector header)
let pat: Vec<u8> = (0..PERIOD)
.map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A)
.collect();
for (i, b) in plaintext.iter_mut().enumerate().skip(RUN_START) {
*b = pat[i % PERIOD];
}
plaintext[SEED_OFFSET..SEED_OFFSET + 5].copy_from_slice(&[0x01, 0x02, 0x03, 0x04, 0x05]);
let mut buf = plaintext.clone();
css::lfsr::scramble_sector(&title_key, &mut buf);
let ciphertext = buf.clone();
assert_ne!(
&ciphertext[0x80..],
&plaintext[0x80..],
"fixture malformed — the sector was not actually scrambled"
);
let mut keys = DecryptKeys::Css { title_key };
decrypt_sectors_in_content(&mut buf, &mut keys, 0, 0, &[(0, 1)])
.expect("CSS descramble must not fail");
// Report the first differing offset rather than dumping 1.9 KB.
let mismatch = (0x80..2048).find(|&i| buf[i] != plaintext[i]);
assert!(
mismatch.is_none(),
"the scrambled body must come back as the plaintext it was built \
from; first mismatch at offset {mismatch:?} (buf={:#04x} \
expected={:#04x}) a wrapper that decrypts nothing leaves the \
ciphertext in place and the caller muxes scrambled MPEG",
buf[mismatch.unwrap_or(0x80)],
plaintext[mismatch.unwrap_or(0x80)],
);
}
/// The AACS arm of the same entry point must fail LOUD. Under the
/// keymap-only model AACS decrypts exclusively through
/// `decrypt_sectors_mapped`; reaching this wrapper with AACS keys means a
/// reader was built without installing its key map, and continuing would
/// hand the caller ciphertext under an `Ok`. `DecryptFailed` is the correct
/// verdict per the function's own contract — it must not be softened into a
/// success with a zero count.
#[test]
fn content_gate_aacs_keys_fail_loud_not_ok_zero() {
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![(1, [0xAB; 16])],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
let original = scrambled_region(aacs::content::ALIGNED_UNIT_LEN);
let mut buf = original.clone();
let r = decrypt_sectors_in_content(&mut buf, &mut keys, 0, 0, &[(0, 3)]);
assert!(
matches!(r, Err(crate::error::Error::DecryptFailed)),
"AACS without an installed key map must be DecryptFailed, got {r:?}"
);
assert_eq!(
buf, original,
"and it must not have half-decrypted the buffer on the way out"
);
}
/// Build a Stevenson-crackable scrambled CSS sector for `title_key` (mirrors /// Build a Stevenson-crackable scrambled CSS sector for `title_key` (mirrors
/// `crackable_sector` in the css::mod tests): a periodic run in the clear /// `crackable_sector` in the css::mod tests): a periodic run in the clear
/// header continues past 0x80 into the encrypted region, so /// header continues past 0x80 into the encrypted region, so
@@ -904,7 +690,7 @@ mod tests {
// descramble-and-rekey lives in `css::descramble_region` (the recovery // descramble-and-rekey lives in `css::descramble_region` (the recovery
// seam calls it); the region change must re-crack region B's key. // seam calls it); the region change must re-crack region B's key.
let mut ended = key_a; let mut ended = key_a;
css::descramble_region(&mut buf, &mut ended).expect("descramble"); css::descramble_region(&mut buf, &mut ended);
assert_eq!( assert_eq!(
&buf[0x80..2048], &buf[0x80..2048],
@@ -958,12 +744,6 @@ mod tests {
/// A CLEAR trailing partial (encrypted flag NOT set) is a legitimate content /// A CLEAR trailing partial (encrypted flag NOT set) is a legitimate content
/// tail and must pass through, never trip the guard above. /// tail and must pass through, never trip the guard above.
///
/// "Passes through" means byte-for-byte unchanged, not merely `Ok`. Asserting
/// only `is_ok()` let a mutant that corrupts the clear partial while still
/// returning `Ok` pass — which is the whole failure this test names.
/// Mutation: XOR any byte of the tail before returning -> the snapshot
/// comparison fails.
#[test] #[test]
fn aacs_clear_trailing_partial_passes_through() { fn aacs_clear_trailing_partial_passes_through() {
let keys = DecryptKeys::Aacs { let keys = DecryptKeys::Aacs {
@@ -975,14 +755,8 @@ mod tests {
let mut tail = clear_ts_region(4096); let mut tail = clear_ts_region(4096);
tail[0] &= 0x3F; // ensure the CPI bits are clear tail[0] &= 0x3F; // ensure the CPI bits are clear
buf.extend_from_slice(&tail); buf.extend_from_slice(&tail);
let snapshot = buf.clone();
let map = AacsKeyMap::from_ranges(vec![(0, u32::MAX, 0)]); let map = AacsKeyMap::from_ranges(vec![(0, u32::MAX, 0)]);
decrypt_sectors_mapped(&mut buf, &keys, 0, &map) assert!(decrypt_sectors_mapped(&mut buf, &keys, 0, &map).is_ok());
.expect("a clear trailing partial is legitimate content");
assert_eq!(
buf, snapshot,
"a clear trailing partial must pass through byte-for-byte, not just return Ok"
);
} }
// ── DecryptKeys::None and is_encrypted ───────────────────────────────── // ── DecryptKeys::None and is_encrypted ─────────────────────────────────
@@ -1030,12 +804,6 @@ mod tests {
/// scramble flag. /// scramble flag.
fn make_css_sector(title_key: &[u8; 5], seed: &[u8; 5], body_fill: u8) -> (Vec<u8>, Vec<u8>) { fn make_css_sector(title_key: &[u8; 5], seed: &[u8; 5], body_fill: u8) -> (Vec<u8>, Vec<u8>) {
let mut sector = vec![body_fill; 2048]; let mut sector = vec![body_fill; 2048];
// A real scrambled DVD sector is an MPEG-2 PS pack, so it begins with
// the pack start code. The descrambler requires it before trusting
// byte 0x14 — without it this fixture is a sector shape that cannot
// occur on a disc, and the test would pass while the production gate
// rejected every sector like it.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] = 0x30; // scramble flag (bits 4-5) sector[0x14] = 0x30; // scramble flag (bits 4-5)
sector[0x54..0x59].copy_from_slice(seed); sector[0x54..0x59].copy_from_slice(seed);
let plaintext = sector.clone(); let plaintext = sector.clone();
@@ -1059,7 +827,7 @@ mod tests {
let (mut sector, plaintext) = make_css_sector(&title_key, &seed, 0xA5); let (mut sector, plaintext) = make_css_sector(&title_key, &seed, 0xA5);
// CSS descramble lives in `css::descramble_region` (the recovery seam // CSS descramble lives in `css::descramble_region` (the recovery seam
// calls it); `decrypt_sectors` only flags CSS sectors for recovery. // calls it); `decrypt_sectors` only flags CSS sectors for recovery.
css::descramble_region(&mut sector, &mut title_key).expect("descramble"); css::descramble_region(&mut sector, &mut title_key);
assert_eq!( assert_eq!(
&sector[0x80..2048], &sector[0x80..2048],
&plaintext[0x80..2048], &plaintext[0x80..2048],
@@ -1089,7 +857,7 @@ mod tests {
let mut buf = s0; let mut buf = s0;
buf.extend_from_slice(&s1); buf.extend_from_slice(&s1);
let mut title_key = title_key; let mut title_key = title_key;
css::descramble_region(&mut buf, &mut title_key).expect("descramble"); css::descramble_region(&mut buf, &mut title_key);
assert_eq!( assert_eq!(
&buf[0x80..2048], &buf[0x80..2048],
&p0[0x80..2048], &p0[0x80..2048],
@@ -1113,9 +881,6 @@ mod tests {
period: usize, period: usize,
) -> (Vec<u8>, Vec<u8>) { ) -> (Vec<u8>, Vec<u8>) {
let mut plaintext = vec![0u8; 2048]; let mut plaintext = vec![0u8; 2048];
// Real scrambled DVD sectors are MPEG-2 PS packs; the scramble policy
// requires the pack start code as well as the flag bits.
plaintext[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plaintext[0x14] = 0x10; // scramble flag plaintext[0x14] = 0x10; // scramble flag
// Periodic run from 0x59 (just above the seed) through 0x80 and on into // Periodic run from 0x59 (just above the seed) through 0x80 and on into
// the encrypted region; phase anchored to offset 0 so it is continuous // the encrypted region; phase anchored to offset 0 so it is continuous
@@ -1171,7 +936,7 @@ mod tests {
// Cache primed to key_a only — exactly what the one-shot scan crack yields. // Cache primed to key_a only — exactly what the one-shot scan crack yields.
let mut title_key = key_a; let mut title_key = key_a;
css::descramble_region(&mut buf, &mut title_key).expect("descramble"); css::descramble_region(&mut buf, &mut title_key);
assert_eq!( assert_eq!(
&buf[0x80..2048], &buf[0x80..2048],
@@ -1283,15 +1048,34 @@ mod tests {
// ── Multi-CPS-unit key selection ────────────────────────────────────── // ── Multi-CPS-unit key selection ──────────────────────────────────────
/// Encrypt an aligned unit so `aacs::content::decrypt_unit` with the same key /// Encrypt an aligned unit with the AACS algorithm run in reverse so that
/// recovers the plaintext, flagging it encrypted first (bytes 0..16 are the key /// `aacs::content::decrypt_unit` with the same key recovers the plaintext. Mirrors
/// seed, so the flag must be set before the crypto runs). /// the `aacs_encrypt_unit` helper in `aacs::content::tests`.
fn aacs_encrypt_unit_for_test(unit: &mut [u8], unit_key: &[u8; 16]) { fn aacs_encrypt_unit_for_test(unit: &mut [u8], unit_key: &[u8; 16]) {
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
// CPI bits on byte 0 so the unit reads as encrypted; set before deriving
// the per-unit key so the recovered plaintext header matches.
unit[0] |= 0xC0; unit[0] |= 0xC0;
assert!( let header: [u8; 16] = unit[..16].try_into().unwrap();
aacs::content::encrypt_unit(unit, unit_key), let derived = crate::aacs::crypto::aes_ecb_encrypt(unit_key, &header);
"a full-length unit must encrypt" let mut k = [0u8; 16];
); for i in 0..16 {
k[i] = derived[i] ^ header[i];
}
let cipher = Aes128::new(GenericArray::from_slice(&k));
let mut prev = crate::aacs::crypto::AACS_IV;
let num_blocks = (aacs::content::ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..num_blocks {
let off = 16 + i * 16;
for j in 0..16 {
unit[off + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&unit[off..off + 16]);
cipher.encrypt_block(&mut block);
unit[off..off + 16].copy_from_slice(&block);
prev.copy_from_slice(&unit[off..off + 16]);
}
} }
/// Build a clear aligned unit with TS sync bytes placed at the BD-TS stride /// Build a clear aligned unit with TS sync bytes placed at the BD-TS stride
@@ -1415,256 +1199,6 @@ mod tests {
} }
} }
/// A forensic range does NOT start on an aligned-unit boundary. Its start LBA
/// comes from a source-packet number — `start_spn * 192` put through
/// `clip_byte_to_lba` (`mux/resolve.rs`) — and 192-byte packets have no
/// relationship to the 3-sector aligned unit, so `range_start % 3` is
/// whatever the disc says.
///
/// That makes `unit_ix = (lba - range_start) / us` load-bearing in both of
/// its operations, and the existing coverage used a range starting exactly
/// on the extent's first unit, where several wrong formulas agree with the
/// right one by arithmetic accident.
///
/// Getting the parity wrong is not a crash. It reads and decrypts the
/// ALTERNATE variant's half of a forensic segment: the units this disc's
/// key does not open decrypt to garbage, and the units it does open are
/// skipped. AACS 2.1 forensic marking is exactly the mechanism that makes
/// the two halves different, so a phase inversion is silent — it produces a
/// full-length rip carrying the wrong variant.
#[test]
fn read_plan_phase_parity_is_measured_from_an_unaligned_range_start() {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// Case A — range_start is itself unaligned (1001 % 3 == 2) and the extent
// begins on it, so unit offsets are 0, 3, 6, ... Under this shape the
// formula `(lba + range_start) / us` shifts every index by an ODD amount
// and inverts the kept half.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1001, 1013, 5, Phase::Even)]);
assert_eq!(
map.read_plan(&ext, us),
vec![
Extent {
start_lba: 1001,
sector_count: 3
}, // ix 0, even
Extent {
start_lba: 1007,
sector_count: 3
}, // ix 2, even
],
"unit index must be measured as (lba - range_start), so the kept \
units are the even-indexed ones counting from the range start"
);
// Case B — the extent begins one unit-remainder away from the range start
// (1001 - 1000 = 1), so offsets are 1, 4, 7, 10. Here `(lba - range_start)
// * us` inverts the halves instead: the division is what maps a byte
// offset onto a unit index, and multiplying happens to preserve parity
// only when the offset is already a multiple of the unit size.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1000, 1013, 5, Phase::Even)]);
assert_eq!(
map.read_plan(&ext, us),
vec![
Extent {
start_lba: 1001,
sector_count: 3
}, // (1001-1000)/3 = 0, even
Extent {
start_lba: 1007,
sector_count: 3
}, // (1007-1000)/3 = 2, even
],
"the offset must be DIVIDED by the unit size to become a unit index"
);
}
/// An extent whose last whole unit is an alternate-phase unit must still drop
/// it. The tail guard exists for a REMNANT shorter than a unit — bytes with
/// no following unit to desync — and an extent ending exactly on a unit
/// boundary has no remnant at all.
///
/// With the guard widened to `remaining <= us`, the final unit of every
/// extent bypasses the phase gate and is read unconditionally. On a forensic
/// segment that lands at an extent end, that is one alternate-variant unit
/// pulled into the rip and decrypted with a key that does not open it.
#[test]
fn read_plan_gates_the_last_whole_unit_of_an_extent_not_just_the_remnant() {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// Two whole units, no remnant. ix 0 is even (kept), ix 1 is odd (dropped)
// and it is the LAST thing in the extent.
let ext = vec![Extent {
start_lba: 1000,
sector_count: 6,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1000, 1006, 5, Phase::Even)]);
assert_eq!(
map.read_plan(&ext, us),
vec![Extent {
start_lba: 1000,
sector_count: 3
}],
"the trailing odd-phase unit is a whole unit and must be dropped; the \
short-tail guard is for a remnant SMALLER than a unit"
);
// And the remnant case the guard is actually for: 4 sectors = one whole
// unit plus a 1-sector tail. The tail is ordinary content and is kept
// even though the unit before it was dropped.
let ext = vec![Extent {
start_lba: 1000,
sector_count: 4,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1000, 1004, 5, Phase::Odd)]);
assert_eq!(
map.read_plan(&ext, us),
vec![Extent {
start_lba: 1003,
sector_count: 1
}],
"a sub-unit remnant is ordinary content and is always read"
);
}
/// Every scheme that CANNOT prove a key answers the same way.
///
/// This is the property `decrypt_span` exists to hold. There used to be two
/// top-level decrypt paths — one for CSS and clear media, one for AACS —
/// and each decided its own answer, so they drifted apart in opposite
/// directions within a single release: CSS descrambled with a key it had
/// just proven stale, and the AACS path passed an unkeyable encrypted unit
/// through as ciphertext. Both reported success.
///
/// Asserting one verdict across the schemes is what makes a future
/// divergence a test failure rather than a silent corruption. A per-scheme
/// test cannot do that: each would still pass while the two disagreed.
///
/// CSS is deliberately NOT in this list. Its title key is recovered from
/// the data, sector by sector, by a heuristic that false-positives — a crib
/// mismatch whose re-crack fails means the crib was wrong, not that the key
/// is stale, so the cached key is kept and used. Round 9 folded CSS in here
/// on the reasoning that "no key" should mean one thing everywhere; that
/// made real DVDs unrippable, and the real-media gate caught it. Uniform
/// policy is right for schemes that can PROVE a key wrong. CSS cannot.
#[test]
fn every_scheme_gives_the_same_verdict_when_no_key_can_be_proven() {
use crate::disc::ContentFormat;
let ul = aacs::content::ALIGNED_UNIT_LEN;
// AACS, encrypted, no map installed at all.
let mut aacs_keys = DecryptKeys::Aacs {
unit_keys: vec![(0, [0xAAu8; 16])],
read_data_key: None,
format: ContentFormat::BdTs,
};
let mut buf = vec![0u8; ul];
let aacs_no_map = decrypt_span(&mut buf, &mut aacs_keys, 0, None, None)
.expect_err("an AACS reader with no key map cannot prove any key");
// AACS, encrypted, mapped but the unit falls outside every range.
let mut orphan = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut orphan, &[0xCCu8; 16]);
let mut buf = orphan.to_vec();
let empty = AacsKeyMap::from_ranges(vec![]);
let aacs_unmapped = decrypt_span(&mut buf, &mut aacs_keys, 0, Some(&empty), None)
.expect_err("an encrypted unit no range covers cannot be keyed");
let want = crate::error::Error::DecryptFailed.code();
for (what, e) in [
("AACS, no map", aacs_no_map),
("AACS, unit outside every range", aacs_unmapped),
] {
assert_eq!(
e.code(),
want,
"{what}: every scheme must refuse identically, or one of them is \
quietly emitting data it could not decrypt"
);
}
// And clear media is NOT a refusal — the shared policy must not turn
// "nothing to decrypt" into an error.
let mut none_keys = DecryptKeys::None;
let mut buf = vec![0u8; 2048];
assert!(
decrypt_span(&mut buf, &mut none_keys, 0, None, None).is_ok(),
"clear media has no key to prove and must pass through"
);
}
/// An ENCRYPTED unit that falls outside every key-map range must fail, not
/// pass through as ciphertext.
///
/// "The map has no key here" and "there is nothing to decrypt here" are
/// different statements, and only the second makes passing the unit through
/// correct. On a multi-CPS disc an orphan clip — referenced by no playlist,
/// so in no title extent and therefore in no range — hits the first and was
/// treated as the second. `extract_tree` then counted those bytes as GOOD,
/// dropped the `.partial` suffix, and reported `complete: true`, exit 0:
/// a scrambled file on disk with a clean bill of health.
///
/// A CLEAR unit outside every range is the ordinary case (filesystem and
/// nav on a whole-disc read) and must still pass through untouched — so
/// this asserts both directions.
#[test]
fn an_encrypted_unit_outside_every_key_range_fails_instead_of_passing_through() {
use crate::disc::ContentFormat;
let key = [0xAAu8; 16];
let ul = aacs::content::ALIGNED_UNIT_LEN;
let usz = (ul / 2048) as u32;
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, key)],
read_data_key: None,
format: ContentFormat::BdTs,
};
// The map covers unit 0 only. Unit 1 is the orphan.
let map = AacsKeyMap::from_ranges(vec![(0, usz, 0)]);
// Clear orphan: untouched, no error. This is nav/filesystem.
let mut clear_buf = vec![0u8; 2 * ul];
let mut u0 = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u0, &key);
clear_buf[..ul].copy_from_slice(&u0);
clear_buf[ul..].copy_from_slice(&clear_ts_unit());
let orphan_before = clear_buf[ul..].to_vec();
decrypt_sectors_mapped(&mut clear_buf, &keys, 0, &map)
.expect("a CLEAR unit outside the map is ordinary nav and must pass");
assert_eq!(
&clear_buf[ul..],
&orphan_before[..],
"a clear out-of-range unit must be left byte-identical"
);
// Encrypted orphan: must fail loud.
let mut enc_buf = vec![0u8; 2 * ul];
let mut v0 = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut v0, &key);
enc_buf[..ul].copy_from_slice(&v0);
let mut orphan = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut orphan, &[0xCCu8; 16]);
enc_buf[ul..].copy_from_slice(&orphan);
let err = decrypt_sectors_mapped(&mut enc_buf, &keys, 0, &map)
.expect_err("an encrypted unit we hold no key for must not be emitted");
assert_eq!(
err.code(),
crate::error::Error::DecryptFailed.code(),
"same verdict CSS and the split-unit branch give for 'no provable key'"
);
}
/// Phase::Even → only even-index units in the range are decrypted; the odd /// Phase::Even → only even-index units in the range are decrypted; the odd
/// (alternate variant) half is left BYTE-FOR-BYTE as ciphertext for the muxer. /// (alternate variant) half is left BYTE-FOR-BYTE as ciphertext for the muxer.
#[test] #[test]
@@ -1708,62 +1242,6 @@ mod tests {
} }
} }
/// The mapped descramble indexes the committed key pool POSITIONALLY
/// (`unit_keys[key_idx].1`), so the ORDER of the `Vec<UnitKey>` a
/// `KeySource` returns is load-bearing — it is NOT "cosmetic, the decrypt path
/// strips it and tries every key", as `keysource::resolve_and_apply_traced`'s
/// doc used to claim. Trial-decrypt was deliberately deleted; nothing here
/// searches the pool. Reordering the same two keys therefore sends each range
/// to the WRONG key: the range that decrypted clean now fails the correct-phase
/// `is_clean` net loudly (or, off a forensic phase, would decrypt a whole span
/// under a neighbour's key). Pins the corrected doc.
#[test]
fn mapped_key_selection_is_positional_so_pool_order_matters() {
use crate::disc::ContentFormat;
let key_a = [0xAAu8; 16];
let key_b = [0xBBu8; 16];
let ul = aacs::content::ALIGNED_UNIT_LEN;
let usz = (ul / 2048) as u32;
// Unit 0 encrypted under key_a, unit 1 under key_b.
let build = || {
let mut buf = vec![0u8; 2 * ul];
for (i, k) in [key_a, key_b].iter().enumerate() {
let mut u = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u, k);
buf[i * ul..(i + 1) * ul].copy_from_slice(&u);
}
buf
};
// Map: unit 0 → pool position 0, unit 1 → pool position 1.
let map = AacsKeyMap::from_ranges_phased(vec![
(0, usz, 0, Phase::Even),
(usz, 2 * usz, 1, Phase::Even),
]);
// Pool in CPS-unit order: each range gets its own key, both come clean.
let mut buf = build();
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, key_a), (1, key_b)],
read_data_key: None,
format: ContentFormat::BdTs,
};
decrypt_sectors_mapped(&mut buf, &keys, 0, &map)
.expect("pool in CPS-unit order decrypts clean");
// SAME keys, SAME CPS-unit numbers, swapped POSITIONS. If the number were
// what mattered (or if the path searched the pool) this would be
// equivalent; positional indexing makes it decrypt both units wrong.
let mut buf = build();
let swapped = DecryptKeys::Aacs {
unit_keys: vec![(1, key_b), (0, key_a)],
read_data_key: None,
format: ContentFormat::BdTs,
};
assert!(
decrypt_sectors_mapped(&mut buf, &swapped, 0, &map).is_err(),
"a reordered pool must fail loud — key selection is positional, so the \
ORDER a KeySource returns its keys in is part of the contract"
);
}
/// The correct-phase safety `is_clean` fires loud: an even unit whose mapped /// The correct-phase safety `is_clean` fires loud: an even unit whose mapped
/// key is wrong does NOT come clean → `DecryptFailed` (not silent corruption). /// key is wrong does NOT come clean → `DecryptFailed` (not silent corruption).
#[test] #[test]
@@ -1843,80 +1321,4 @@ mod tests {
"decrypt thread count must not exceed MAX_THREADS ({MAX_THREADS}), got {n}" "decrypt thread count must not exceed MAX_THREADS ({MAX_THREADS}), got {n}"
); );
} }
/// The FMTS phase gate picks which half of an interleaved forensic segment
/// we decrypt. Both the `-` and the `/` in its index arithmetic survived a
/// mutation run, and getting either wrong silently decrypts the alternate
/// variant into garbage while reporting success.
#[test]
fn phase_gate_selects_only_our_parity_of_a_forensic_segment() {
use super::{Phase, unit_is_our_phase};
// A range starting at LBA 30, 3 sectors per aligned unit: units are at
// 30, 33, 36, 39, ... with indices 0, 1, 2, 3, ...
let ours = |lba, phase| unit_is_our_phase(lba, 30, 3, phase);
// Even phase takes indices 0, 2, 4 -> LBAs 30, 36, 42.
assert!(ours(30, Phase::Even));
assert!(!ours(33, Phase::Even));
assert!(ours(36, Phase::Even));
assert!(!ours(39, Phase::Even));
// Odd phase is the exact complement.
for lba in [30, 33, 36, 39, 42, 45] {
assert_ne!(
ours(lba, Phase::Even),
ours(lba, Phase::Odd),
"LBA {lba} must belong to exactly one parity"
);
}
// Non-forensic content: the whole range is ours.
for lba in [30, 33, 36, 39] {
assert!(ours(lba, Phase::All));
}
// The index must be RANGE-RELATIVE: `(lba - start)`, not `(lba + start)`.
// Most value pairs give the same parity either way, so pin one where
// they genuinely disagree: (5-1)/2 = 2 (even, ours) but
// (5+1)/2 = 3 (odd, not ours).
assert!(
unit_is_our_phase(5, 1, 2, Phase::Even),
"the index must be measured from the range start"
);
// The `/ unit_sectors` -> `* unit_sectors` mutant is EQUIVALENT here,
// and deliberately not chased. Proof: aligned offsets are exact
// multiples of the unit size, so offset = k*u for unit index k.
// Dividing gives k; multiplying gives k*u^2, whose parity is
// parity(k)*parity(u^2) = parity(k) whenever u is ODD. `unit_sectors`
// is `ALIGNED_UNIT_LEN / 2048` = 3, a compile-time constant, so u is
// always odd and the two agree on every reachable input. Only an even
// unit size would separate them, and none exists.
assert!(!unit_is_our_phase(33, 30, 3, Phase::Even));
}
/// A malformed key map must not take down a long-running service. A unit
/// below its own range start, or a zero unit size, are both map bugs — they
/// must return a defined answer rather than panicking on debug overflow or
/// dividing by zero.
///
/// Every case here asserts the DEFINED answer, not merely the absence of a
/// panic. The zero-unit-size case used to be written
/// `assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
/// accepts both answers and so pinned nothing at all: the guards could
/// invert and it would still pass. The answer is knowable —
/// `saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
/// is even — so pin it.
#[test]
fn phase_gate_does_not_panic_on_a_malformed_map() {
use super::{Phase, unit_is_our_phase};
// Unit below its own range start: saturating_sub clamps to 0, and unit
// 0 is even.
assert!(unit_is_our_phase(10, 100, 3, Phase::Even));
// Zero unit size: max(1) makes the divisor 1, so the index is the raw
// offset 70 — even.
assert!(unit_is_our_phase(100, 30, 0, Phase::Even));
// Both malformations at once: offset 0 over divisor 1 is unit 0, even.
assert!(unit_is_our_phase(5, 5, 0, Phase::Even));
}
} }
+45 -103
View File
@@ -22,7 +22,10 @@
//! `Disc`-level dump ([`dump_disc`]) covers everything that survives //! `Disc`-level dump ([`dump_disc`]) covers everything that survives
//! lowering: titles, streams, the picked main feature, and AACS state. //! lowering: titles, streams, the picked main feature, and AACS state.
use crate::disc::{ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, Stream}; use crate::disc::{
AudioChannels, ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, SampleRate,
Stream,
};
use crate::ifo::{CellCategory, DvdTitle}; use crate::ifo::{CellCategory, DvdTitle};
const DIAG: &str = "freemkv::diag"; const DIAG: &str = "freemkv::diag";
@@ -92,12 +95,36 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
} }
} }
// `channel_count` and `sample_rate_hz` lived here as a third copy of the /// Channel count from an [`AudioChannels`] layout (what lands in the MKV
// AudioChannels/SampleRate mappings. They were the only HONEST copy — returning /// `Channels` element).
// 0 for Unknown where the canonical accessors fabricated 6 channels at 48 kHz — pub fn channel_count(ch: AudioChannels) -> u8 {
// and their only caller was the trace line below, in this same file. The match ch {
// canonical accessors are honest now, so the duplicates are gone rather than AudioChannels::Mono => 1,
// left to drift a fourth time. AudioChannels::Stereo => 2,
AudioChannels::Stereo21 => 3,
AudioChannels::Quad => 4,
AudioChannels::Surround50 => 5,
AudioChannels::Surround51 => 6,
AudioChannels::Surround61 => 7,
AudioChannels::Surround71 => 8,
AudioChannels::Unknown => 0,
}
}
/// Sample-rate in Hz for a [`SampleRate`].
pub fn sample_rate_hz(s: SampleRate) -> u32 {
match s {
SampleRate::S44_1 => 44100,
SampleRate::S48 => 48000,
SampleRate::S88_2 => 88200,
SampleRate::S96 => 96000,
SampleRate::S176_4 => 176400,
SampleRate::S192 => 192000,
SampleRate::S48_96 => 96000,
SampleRate::S48_192 => 192000,
SampleRate::Unknown => 0,
}
}
// ── DVD cell-category dump (from the IFO scan, pre-lowering) ───────────────── // ── DVD cell-category dump (from the IFO scan, pre-lowering) ─────────────────
@@ -472,31 +499,15 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!( tracing::debug!(
target: DIAG, target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \ "tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason={}", size={}B clips={} reason=canonical_title_order(fits-disc, fewest-clips, longest, richest-audio)",
main.playlist, main.playlist,
main.duration_secs, main.duration_secs,
main.size_bytes, main.size_bytes,
main.clips.len(), main.clips.len(),
main_feature_reason(),
); );
} }
} }
/// The `reason=` token on the main-feature decision row.
///
/// DERIVED from [`Disc::CANONICAL_TITLE_ORDER_KEYS`], which lives beside the
/// comparator that actually implements them — never restated here. The previous
/// hand-written copy drifted (it advertised a `fewest-clips` key the comparator
/// had replaced with largest-physical-size), which made the self-diagnosing log
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
}
fn dump_aacs(disc: &Disc) { fn dump_aacs(disc: &Disc) {
let Some(a) = disc.aacs.as_ref() else { let Some(a) = disc.aacs.as_ref() else {
if disc.css.is_some() { if disc.css.is_some() {
@@ -599,8 +610,8 @@ fn dump_title(ti: usize, title: &DiscTitle) {
a.pid, a.pid,
a.codec, a.codec,
a.channels, a.channels,
a.channels.count(), channel_count(a.channels),
a.sample_rate.hz(), sample_rate_hz(a.sample_rate),
a.language, a.language,
a.secondary, a.secondary,
), ),
@@ -620,60 +631,6 @@ fn dump_title(ti: usize, title: &DiscTitle) {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
// Needed only by the tests: the production code in this file no longer names
// these types directly, since the local channel/sample-rate duplicates were
// deleted in favour of the canonical accessors.
use crate::disc::{AudioChannels, SampleRate};
/// The main-feature decision row must NAME `canonical_title_order`'s sort
/// keys, not restate them from memory. The restated copy had drifted: it
/// still advertised a "fewest-clips" key long after the comparator replaced
/// clip-count with largest-physical-size, so a bug report read at
/// `--log-level 3` explained the pick with a rule the code does not apply.
///
/// The behavioural half is asserted first — against the comparator itself,
/// with literals — so the key names are checked against what the code
/// actually does, not against the string that names them.
#[test]
fn main_feature_reason_names_the_comparators_real_keys() {
use crate::disc::{Clip, Disc, DiscTitle};
let sized = |size_bytes: u64, n_clips: usize| DiscTitle {
size_bytes,
clips: (0..n_clips)
.map(|i| Clip {
feed_span: None,
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 0,
duration_secs: 0.0,
source_packets: 0,
})
.collect(),
..DiscTitle::empty()
};
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's
// primary key among disc-fitting titles is LARGEST SIZE. "fewest clips"
// would predict the opposite, so the drifted string described a rule
// the comparator does not implement.
let many_clips_big = sized(8_000_000_000, 40);
let one_clip_small = sized(1_000_000_000, 1);
assert_eq!(
Disc::canonical_title_order(&many_clips_big, &one_clip_small, 25_000_000_000),
std::cmp::Ordering::Less,
"largest size wins regardless of clip count"
);
let reason = main_feature_reason();
assert!(
!reason.contains("clips"),
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
);
}
#[test] #[test]
fn res_str_keeps_interlace_marker() { fn res_str_keeps_interlace_marker() {
@@ -699,33 +656,18 @@ mod tests {
assert_eq!(hdr_str(HdrFormat::Sdr), "SDR"); assert_eq!(hdr_str(HdrFormat::Sdr), "SDR");
} }
/// Moved from the deleted local duplicates onto the canonical accessors,
/// with the Unknown case added — which is the whole point of the change.
#[test] #[test]
fn channel_count_matches_layout_and_is_zero_when_unknown() { fn channel_count_matches_layout() {
assert_eq!(AudioChannels::Mono.count(), 1); assert_eq!(channel_count(AudioChannels::Mono), 1);
assert_eq!(AudioChannels::Stereo.count(), 2); assert_eq!(channel_count(AudioChannels::Stereo), 2);
assert_eq!(AudioChannels::Surround51.count(), 6); assert_eq!(channel_count(AudioChannels::Surround51), 6);
assert_eq!(AudioChannels::Surround71.count(), 8); assert_eq!(channel_count(AudioChannels::Surround71), 8);
// The one that matters. This used to return 6, which is indistinguishable
// from a real 5.1 track and left every caller responsible for checking
// the variant first.
assert_eq!(
AudioChannels::Unknown.count(),
0,
"an unknown layout must not report a plausible channel count"
);
} }
#[test] #[test]
fn sample_rate_hz_values_and_zero_when_unknown() { fn sample_rate_hz_values() {
assert_eq!(SampleRate::S48.hz(), 48000.0); assert_eq!(sample_rate_hz(SampleRate::S48), 48000);
assert_eq!(SampleRate::S96.hz(), 96000.0); assert_eq!(sample_rate_hz(SampleRate::S96), 96000);
assert_eq!(
SampleRate::Unknown.hz(),
0.0,
"an unknown sample rate must not report a plausible 48 kHz"
);
} }
#[test] #[test]
-743
View File
@@ -1,743 +0,0 @@
//! ECMA-167 / UDF 1.02 descriptor encoder.
//!
//! Turns a [`Layout`](super::layout::Layout) — a directory tree with every
//! ICB, directory-data and file-data block already assigned — into the set of
//! metadata sectors a real UDF volume would carry. Nothing here touches the
//! filesystem: it is a pure function from layout to sectors, which is what
//! makes it testable against the production parser in `udf.rs`.
//!
//! What is emitted, in volume order:
//!
//! | sector | descriptor |
//! |---|---|
//! | 16, 17, 18 | Volume Recognition Sequence — `BEA01`, `NSR02`, `TEA01` (ECMA-167 2/9.1) |
//! | 32… | Main Volume Descriptor Sequence — PVD, IUVD, PD, LVD, USD, TD |
//! | 48… | Reserve VDS (byte-identical but for the tag locations) |
//! | 64, 65 | Logical Volume Integrity Sequence — LVID, TD |
//! | 256 | Anchor Volume Descriptor Pointer |
//! | `part_start` + 0, +1 | File Set Descriptor, TD |
//! | `part_start` + … | File Entries (ICBs) and directory data (FIDs) |
//! | last sector | Anchor Volume Descriptor Pointer (copy) |
//!
//! UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
//! the DVD-Video profile, it is the shape `read_filesystem` takes when
//! `num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
//! entirely. That also means a synthetic image never exercises the Metadata
//! Partition path in `udf.rs` (`:946-991`) — see the module docs on `dirimage`.
use super::layout::{DirNode, Layout};
use crate::error::{Error, Result};
use std::collections::BTreeMap;
/// Logical block / sector size. Fixed for every optical profile this crate
/// reads, and the same quantity as [`crate::consts::SECTOR_BYTES`] — aliased
/// rather than re-declared so the two cannot drift apart. The short name is
/// kept because it appears in ~25 extent and offset expressions across
/// `dirimage`, where the longer one would bury the arithmetic.
pub(super) use crate::consts::SECTOR_BYTES as SECTOR;
/// Descriptor version recorded in every tag. 2 = ECMA-167 2nd edition, which
/// is what UDF revisions up to and including 2.00 require.
const DESC_VERSION: u16 = 2;
/// UDF revision recorded in the domain EntityID suffix (1.02, BCD-ish u16).
const UDF_REVISION: u16 = 0x0102;
/// A fixed recording timestamp, so an image synthesized from the same folder
/// twice is byte-identical. Real mtimes would make every test golden-file
/// comparison and every `dir:// -> iso://` re-run differ for no benefit.
const FIXED_TIME: Timestamp = Timestamp {
year: 2000,
month: 1,
day: 1,
};
struct Timestamp {
year: i16,
month: u8,
day: u8,
}
/// The synthesized metadata: absolute LBA → sector contents. Data sectors are
/// NOT here; they are served from the backing files.
pub(super) type MetaSectors = BTreeMap<u32, Box<[u8; SECTOR]>>;
/// The descriptor-tag CRC of ECMA-167 7.2.4: polynomial 0x1021, initial value
/// ZERO, no reflection, no final XOR — the variant catalogued as CRC-16/XMODEM
/// (check value 0x31C3), NOT CCITT-FALSE, which seeds at 0xFFFF and would make
/// every descriptor this crate writes fail a conformant driver's validation.
fn crc16(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x1021
} else {
crc << 1
};
}
}
crc
}
/// Write an ECMA-167 3/7.2 descriptor tag over `buf[0..16]`.
///
/// `tag_loc` is the block number of the sector holding the descriptor —
/// ABSOLUTE for the volume-space descriptors (AVDP, VDS, LVID) and
/// PARTITION-RELATIVE for everything inside the partition (FSD, File Entries).
/// Getting that wrong is the classic reason a hand-built volume mounts nowhere:
/// a driver that validates the tag location rejects the descriptor outright.
///
/// `desc_len` is the descriptor's total length including the tag; the CRC
/// covers `buf[16..desc_len]`.
fn finish_tag(buf: &mut [u8], tag_id: u16, tag_loc: u32, desc_len: usize) {
buf[0..2].copy_from_slice(&tag_id.to_le_bytes());
buf[2..4].copy_from_slice(&DESC_VERSION.to_le_bytes());
buf[4] = 0; // checksum, filled below
buf[5] = 0; // reserved
buf[6..8].copy_from_slice(&0u16.to_le_bytes()); // tag serial number
let crc_len = desc_len - 16;
let crc = crc16(&buf[16..desc_len]);
buf[8..10].copy_from_slice(&crc.to_le_bytes());
buf[10..12].copy_from_slice(&(crc_len as u16).to_le_bytes());
buf[12..16].copy_from_slice(&tag_loc.to_le_bytes());
// ECMA-167 3/7.2.3: sum of bytes 0..16 EXCLUDING byte 4, modulo 256.
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
buf[4] = (sum % 256) as u8;
}
/// ECMA-167 1/7.2.1 charspec: type 0 (CS0) + "OSTA Compressed Unicode".
fn put_charspec(buf: &mut [u8]) {
buf[0] = 0;
let id = b"OSTA Compressed Unicode";
buf[1..1 + id.len()].copy_from_slice(id);
}
/// ECMA-167 1/7.4 EntityID: flags byte, 23 identifier bytes, 8 suffix bytes.
fn put_entity_id(buf: &mut [u8], id: &[u8], suffix: &[u8]) {
buf[0] = 0;
let n = id.len().min(23);
buf[1..1 + n].copy_from_slice(&id[..n]);
let m = suffix.len().min(8);
buf[24..24 + m].copy_from_slice(&suffix[..m]);
}
/// The `*OSTA UDF Compliant` domain EntityID suffix: UDF revision, domain
/// flags (0 = neither hard nor soft write-protected), reserved.
fn domain_suffix() -> [u8; 8] {
let mut s = [0u8; 8];
s[0..2].copy_from_slice(&UDF_REVISION.to_le_bytes());
s
}
/// This crate's implementation EntityID suffix: OS class / OS identifier
/// (0 = undefined, deliberately — the image is not OS-specific) + 6 free bytes.
fn impl_suffix() -> [u8; 8] {
[0u8; 8]
}
fn put_impl_id(buf: &mut [u8]) {
put_entity_id(buf, b"*freemkv", &impl_suffix());
}
fn put_domain_id(buf: &mut [u8]) {
put_entity_id(buf, b"*OSTA UDF Compliant", &domain_suffix());
}
/// OSTA CS0 d-string: a compression-ID byte, the characters, then the used
/// length in the FIELD'S LAST byte (ECMA-167 1/7.2.12 + UDF 2.1.3). An
/// all-zero field is the empty string.
fn put_dstring(buf: &mut [u8], s: &str) {
if s.is_empty() {
return;
}
let encoded = encode_cs0(s);
// Leave room for the trailing length byte.
let room = buf.len() - 1;
let n = encoded.len().min(room);
buf[..n].copy_from_slice(&encoded[..n]);
buf[buf.len() - 1] = n as u8;
}
/// OSTA CS0: compression ID 8 (one byte per character) when every character
/// is ASCII, otherwise compression ID 16 (UTF-16BE).
///
/// ASCII rather than Latin-1 for the 8-bit form on purpose: `parse_udf_name`
/// (`udf.rs:1467`) decodes a compression-8 name with `from_utf8_lossy`, so a
/// 0x80-0xFF byte — legal CS0 — would come back as U+FFFD. Every character
/// above 0x7F therefore takes the 16-bit form, which that parser decodes
/// correctly.
pub(super) fn encode_cs0(s: &str) -> Vec<u8> {
if s.is_ascii() {
let mut v = Vec::with_capacity(1 + s.len());
v.push(8u8);
v.extend_from_slice(s.as_bytes());
v
} else {
let mut v = vec![16u8];
for u in s.encode_utf16() {
v.extend_from_slice(&u.to_be_bytes());
}
v
}
}
/// ECMA-167 1/7.3 timestamp, 12 bytes. Type 1 (local time) with a zero
/// offset, i.e. UTC.
fn put_timestamp(buf: &mut [u8]) {
buf[0..2].copy_from_slice(&0x1000u16.to_le_bytes());
buf[2..4].copy_from_slice(&FIXED_TIME.year.to_le_bytes());
buf[4] = FIXED_TIME.month;
buf[5] = FIXED_TIME.day;
}
/// ECMA-167 3/7.1 extent_ad: length in BYTES, then location.
fn put_extent_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
/// ECMA-167 4/14.14.2 long_ad: length+type, then lb_addr (block, partition
/// reference), then 6 implementation-use bytes.
fn put_long_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
buf[8..10].copy_from_slice(&0u16.to_le_bytes()); // partition reference 0
}
/// ECMA-167 4/14.14.1 short_ad. The top two bits of the length word are the
/// extent TYPE (0 = recorded and allocated), which is exactly why `udf.rs`
/// masks with `0x3FFF_FFFF` when it reads one back — the mask is the field
/// boundary, not a truncation bug.
fn put_short_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
debug_assert!(len_bytes <= 0x3FFF_FFFF, "AD length must fit 30 bits");
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
fn blank() -> Box<[u8; SECTOR]> {
Box::new([0u8; SECTOR])
}
// ── Volume-space descriptors ────────────────────────────────────────────────
/// ECMA-167 2/9.1 Volume Structure Descriptor: the three-sector recognition
/// sequence an OS looks for before it will even consider the volume UDF.
fn volume_recognition(id: &[u8; 5]) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[0] = 0; // structure type
s[1..6].copy_from_slice(id);
s[6] = 1; // structure version
s
}
/// ECMA-167 3/10.1 Primary Volume Descriptor.
fn primary_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes()); // PVD number
put_dstring(&mut s[24..56], volume_id);
s[56..58].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[58..60].copy_from_slice(&1u16.to_le_bytes()); // max volume sequence number
s[60..62].copy_from_slice(&2u16.to_le_bytes()); // interchange level
s[62..64].copy_from_slice(&2u16.to_le_bytes()); // max interchange level
s[64..68].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[68..72].copy_from_slice(&1u32.to_le_bytes()); // max character set list
// UDF 2.2.2.5: the first 8 characters of the volume set identifier must be
// unique. A fixed hex prefix plus the volume id is sufficient here — the
// image is single-volume and never joins a real volume set.
put_dstring(&mut s[72..200], &format!("46524D4B{volume_id}"));
put_charspec(&mut s[200..264]); // descriptor character set
put_charspec(&mut s[264..328]); // explanatory character set
put_timestamp(&mut s[376..388]);
put_impl_id(&mut s[388..420]);
finish_tag(&mut s[..], 1, lba, 512);
s
}
/// ECMA-167 3/10.4 + UDF 2.2.7 Implementation Use Volume Descriptor
/// (`*UDF LV Info`). Not read by `udf.rs`, required by the spec.
fn impl_use_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_entity_id(&mut s[20..52], b"*UDF LV Info", &domain_suffix());
put_charspec(&mut s[52..116]); // LVI charset
put_dstring(&mut s[116..244], volume_id); // logical volume identifier
put_impl_id(&mut s[352..384]);
finish_tag(&mut s[..], 4, lba, 512);
s
}
/// ECMA-167 3/10.5 Partition Descriptor — the descriptor `read_filesystem`
/// takes `partition_start` from (offset 188).
fn partition(part_start: u32, part_sectors: u32, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..22].copy_from_slice(&1u16.to_le_bytes()); // partition flags: allocated
s[22..24].copy_from_slice(&0u16.to_le_bytes()); // partition number
put_entity_id(&mut s[24..56], b"+NSR02", &[]);
// s[56..184] partition contents use = Partition Header Descriptor. All
// zero: a read-only partition records no unallocated/freed space tables.
s[184..188].copy_from_slice(&1u32.to_le_bytes()); // access type: read only
s[188..192].copy_from_slice(&part_start.to_le_bytes());
s[192..196].copy_from_slice(&part_sectors.to_le_bytes());
put_impl_id(&mut s[196..228]);
finish_tag(&mut s[..], 5, lba, 512);
s
}
/// ECMA-167 3/10.6 Logical Volume Descriptor. Carries the FSD long_ad and the
/// partition map table; `read_filesystem` reads `num_partition_maps` at 268
/// and takes the single-partition path when it is 1.
fn logical_volume(
volume_id: &str,
fsd_lba: u32,
integrity_lba: u32,
integrity_sectors: u32,
lba: u32,
seq: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_charspec(&mut s[20..84]);
put_dstring(&mut s[84..212], volume_id);
s[212..216].copy_from_slice(&(SECTOR as u32).to_le_bytes()); // logical block size
put_domain_id(&mut s[216..248]);
// Logical volume contents use = long_ad of the File Set Descriptor,
// partition-relative. One sector.
put_long_ad(&mut s[248..264], SECTOR as u32, fsd_lba);
s[264..268].copy_from_slice(&6u32.to_le_bytes()); // map table length
s[268..272].copy_from_slice(&1u32.to_le_bytes()); // number of partition maps
put_impl_id(&mut s[272..304]);
put_extent_ad(
&mut s[432..440],
integrity_sectors * SECTOR as u32,
integrity_lba,
);
// ECMA-167 3/10.7.2 Type 1 partition map.
s[440] = 1; // map type
s[441] = 6; // map length
s[442..444].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[444..446].copy_from_slice(&0u16.to_le_bytes()); // partition number
finish_tag(&mut s[..], 6, lba, 446);
s
}
/// ECMA-167 3/10.8 Unallocated Space Descriptor with zero extents — the whole
/// volume is accounted for by the partition.
fn unallocated_space(lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes());
finish_tag(&mut s[..], 7, lba, 24);
s
}
/// ECMA-167 3/10.9 Terminating Descriptor.
fn terminating(lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
finish_tag(&mut s[..], 8, lba, 512);
s
}
/// ECMA-167 3/10.10 + UDF 2.2.6 Logical Volume Integrity Descriptor, closed.
fn integrity(
part_sectors: u32,
files: u32,
dirs: u32,
next_uid: u64,
lba: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..32].copy_from_slice(&1u32.to_le_bytes()); // integrity type: close
// s[32..40] next integrity extent: none.
s[40..48].copy_from_slice(&next_uid.to_le_bytes()); // logical volume contents use: next unique id
s[72..76].copy_from_slice(&1u32.to_le_bytes()); // number of partitions
s[76..80].copy_from_slice(&46u32.to_le_bytes()); // length of implementation use
s[80..84].copy_from_slice(&0u32.to_le_bytes()); // free space: none (read-only)
s[84..88].copy_from_slice(&part_sectors.to_le_bytes()); // size table
put_impl_id(&mut s[88..120]);
s[120..124].copy_from_slice(&files.to_le_bytes());
s[124..128].copy_from_slice(&dirs.to_le_bytes());
s[128..130].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min read revision
s[130..132].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min write revision
s[132..134].copy_from_slice(&UDF_REVISION.to_le_bytes()); // max write revision
finish_tag(&mut s[..], 9, lba, 134);
s
}
/// ECMA-167 3/10.2 Anchor Volume Descriptor Pointer. `read_filesystem` reads
/// the main VDS extent from offsets 16..24 and sweeps it.
fn anchor(main_lba: u32, reserve_lba: u32, vds_sectors: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_extent_ad(&mut s[16..24], vds_sectors * SECTOR as u32, main_lba);
put_extent_ad(&mut s[24..32], vds_sectors * SECTOR as u32, reserve_lba);
finish_tag(&mut s[..], 2, lba, 512);
s
}
/// ECMA-167 4/14.1 File Set Descriptor. `read_filesystem` requires tag 256 at
/// the first block of the (metadata =) partition and reads the root ICB block
/// from offset 404.
fn file_set(volume_id: &str, root_icb: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..30].copy_from_slice(&3u16.to_le_bytes()); // interchange level
s[30..32].copy_from_slice(&3u16.to_le_bytes()); // max interchange level
s[32..36].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[36..40].copy_from_slice(&1u32.to_le_bytes()); // max character set list
s[40..44].copy_from_slice(&0u32.to_le_bytes()); // file set number
s[44..48].copy_from_slice(&0u32.to_le_bytes()); // file set descriptor number
put_charspec(&mut s[48..112]);
put_dstring(&mut s[112..240], volume_id);
put_charspec(&mut s[240..304]);
put_dstring(&mut s[304..336], volume_id);
put_long_ad(&mut s[400..416], SECTOR as u32, root_icb);
put_domain_id(&mut s[416..448]);
finish_tag(&mut s[..], 256, lba, 512);
s
}
// ── Partition-space descriptors ─────────────────────────────────────────────
/// UDF permission word: read + execute for owner, group and other. No write
/// bit anywhere — the volume is read-only.
const PERM_R_X: u32 = 0x0000_1000 | 0x0000_0400 | 0x0000_0080 | 0x0000_0020 | 0x4 | 0x1;
/// ECMA-167 4/14.9 File Entry (tag 261).
///
/// Tag 261 rather than the Extended File Entry (266) real BD-ROMs use: an EFE
/// requires UDF 2.00+, and this image declares 1.02. `udf.rs` reads both — the
/// 261 field offsets it uses (l_ea 168, l_ad 172, ADs at 176 + l_ea) are the
/// ones written here.
///
/// `extents` are partition-relative (block, byte-length) pairs, already split
/// so no single one exceeds the 30-bit AD length field.
fn file_entry(
is_dir: bool,
info_len: u64,
extents: &[(u32, u32)],
link_count: u16,
unique_id: u64,
lba: u32,
) -> Result<Box<[u8; SECTOR]>> {
let mut s = blank();
// ICB tag (ECMA-167 4/14.6) at offset 16.
s[16..20].copy_from_slice(&0u32.to_le_bytes()); // prior recorded direct entries
s[20..22].copy_from_slice(&4u16.to_le_bytes()); // strategy type 4
s[24..26].copy_from_slice(&1u16.to_le_bytes()); // max number of entries
s[27] = if is_dir { 4 } else { 5 }; // file type: directory / byte sequence
// s[28..34] parent ICB location: not recorded (permitted).
// s[34..36] ICB flags: 0 => short allocation descriptors. `udf.rs:601`
// reads exactly this word to pick its AD stride.
s[34..36].copy_from_slice(&0u16.to_le_bytes());
// UDF's sentinel for "not specified" is 0xFFFFFFFF, not 0 — 0 is a real
// uid/gid (root). A synthesized image has no meaningful owner, and a driver
// that maps these through would otherwise report every file as root-owned.
s[36..40].copy_from_slice(&u32::MAX.to_le_bytes()); // uid: not specified
s[40..44].copy_from_slice(&u32::MAX.to_le_bytes()); // gid: not specified
s[44..48].copy_from_slice(&PERM_R_X.to_le_bytes());
s[48..50].copy_from_slice(&link_count.to_le_bytes());
s[56..64].copy_from_slice(&info_len.to_le_bytes());
let blocks: u64 = extents
.iter()
.map(|(_, len)| (*len as u64).div_ceil(SECTOR as u64))
.sum();
s[64..72].copy_from_slice(&blocks.to_le_bytes()); // logical blocks recorded
put_timestamp(&mut s[72..84]); // access
put_timestamp(&mut s[84..96]); // modification
put_timestamp(&mut s[96..108]); // attribute
s[108..112].copy_from_slice(&1u32.to_le_bytes()); // checkpoint
put_impl_id(&mut s[128..160]);
s[160..168].copy_from_slice(&unique_id.to_le_bytes());
s[168..172].copy_from_slice(&0u32.to_le_bytes()); // length of EAs
let l_ad = extents.len() * 8;
// A short AD is 8 bytes and the entry has 2048 - 176 = 1872 bytes for
// them, i.e. 234 extents — over 200 GiB at the per-AD ceiling. Beyond
// that an Allocation Extent Descriptor chain would be required; refuse
// rather than write a truncated list.
if 176 + l_ad > SECTOR {
return Err(Error::DirImageTooLarge);
}
s[172..176].copy_from_slice(&(l_ad as u32).to_le_bytes());
for (i, (elba, len)) in extents.iter().enumerate() {
let off = 176 + i * 8;
put_short_ad(&mut s[off..off + 8], *len, *elba);
}
finish_tag(&mut s[..], 261, lba, 176 + l_ad);
Ok(s)
}
/// ECMA-167 4/14.4 File Identifier Descriptor, appended to `buf`.
///
/// FIDs are packed with no inter-descriptor padding beyond the 4-byte
/// alignment the spec mandates, and they are allowed to span logical blocks —
/// which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
/// directory extent as one flat byte run and STOPS at the first non-257 tag,
/// so any block-alignment gap would truncate the directory.
fn push_fid(buf: &mut Vec<u8>, name: &str, icb_lba: u32, is_dir: bool, is_parent: bool) {
let start = buf.len();
let name_field: Vec<u8> = if is_parent {
Vec::new()
} else {
encode_cs0(name)
};
let l_fi = name_field.len();
let mut fid = vec![0u8; 38];
fid[16..18].copy_from_slice(&1u16.to_le_bytes()); // file version number
let mut chars = 0u8;
if is_dir {
chars |= 0x02;
}
if is_parent {
chars |= 0x08;
}
fid[18] = chars;
// The planner refuses any name whose encoding exceeds what this byte can
// hold (`layout::MAX_CS0_NAME_BYTES`), so this cannot wrap in practice. The
// assert states the invariant where it is relied on rather than trusting a
// check three files away; a wrap here would desynchronise the directory.
debug_assert!(
l_fi <= u8::MAX as usize,
"FID name length must fit one byte"
);
fid[19] = l_fi as u8;
put_long_ad(&mut fid[20..36], SECTOR as u32, icb_lba);
fid[36..38].copy_from_slice(&0u16.to_le_bytes()); // length of implementation use
buf.extend_from_slice(&fid);
buf.extend_from_slice(&name_field);
let unpadded = buf.len() - start;
let padded = unpadded.div_ceil(4) * 4;
buf.resize(start + padded, 0);
// The tag is written last: its CRC covers the descriptor body, which the
// padding is not part of (ECMA-167 4/14.4.9 counts padding outside the
// CRC'd length).
let tag_loc_placeholder = 0;
finish_tag(
&mut buf[start..start + unpadded],
257,
tag_loc_placeholder,
unpadded,
);
}
/// Serialize one directory's FID list (parent entry first, then children).
pub(super) fn dir_fids(dir: &DirNode) -> Vec<u8> {
let mut buf = Vec::new();
push_fid(&mut buf, "", dir.parent_icb_lba, true, true);
for sub in &dir.dirs {
push_fid(&mut buf, &sub.name, sub.icb_lba, true, false);
}
for f in &dir.files {
push_fid(&mut buf, &f.name, f.icb_lba, false, false);
}
buf
}
/// Patch every FID's tag location to the block it actually lands in. ECMA-167
/// 3/7.2.2 makes the tag location the block of the descriptor, and a FID that
/// spans two blocks records the block it STARTS in.
fn fix_fid_tag_locations(buf: &mut [u8], first_block: u32) {
let mut pos = 0usize;
while pos + 38 <= buf.len() {
let l_fi = buf[pos + 19] as usize;
let l_iu = u16::from_le_bytes([buf[pos + 36], buf[pos + 37]]) as usize;
let unpadded = 38 + l_iu + l_fi;
if pos + unpadded > buf.len() {
break;
}
let block = first_block + (pos / SECTOR) as u32;
finish_tag(&mut buf[pos..pos + unpadded], 257, block, unpadded);
pos += unpadded.div_ceil(4) * 4;
}
}
// ── Whole-image assembly ────────────────────────────────────────────────────
/// Volume-space block of the Volume Recognition Sequence.
const VRS_START: u32 = 16;
/// Volume-space block of the Main Volume Descriptor Sequence.
pub(super) const MAIN_VDS_START: u32 = 32;
/// Volume-space block of the Reserve Volume Descriptor Sequence.
pub(super) const RESERVE_VDS_START: u32 = 48;
/// Sectors reserved for each VDS. ECMA-167 3/10.2.1 requires an anchor to
/// record at least 16.
pub(super) const VDS_SECTORS: u32 = 16;
/// Volume-space block of the Logical Volume Integrity Sequence.
pub(super) const LVID_START: u32 = 64;
/// Sectors reserved for the integrity sequence (LVID + TD).
pub(super) const LVID_SECTORS: u32 = 2;
/// The mandatory anchor block (ECMA-167 3/10.2).
pub(super) const ANCHOR_LBA: u32 = 256;
/// First block a partition may start at. Everything above is volume space.
pub(super) const MIN_PART_START: u32 = 320;
/// Emit the six-descriptor Volume Descriptor Sequence at `start`.
fn write_vds(out: &mut MetaSectors, layout: &Layout, start: u32) {
let vid = &layout.volume_id;
out.insert(start, primary_volume(vid, start, 1));
out.insert(start + 1, impl_use_volume(vid, start + 1, 2));
out.insert(
start + 2,
partition(layout.part_start, layout.part_sectors, start + 2, 3),
);
out.insert(
start + 3,
logical_volume(vid, 0, LVID_START, LVID_SECTORS, start + 3, 4),
);
out.insert(start + 4, unallocated_space(start + 4, 5));
out.insert(start + 5, terminating(start + 5));
}
/// Recursively emit one directory's File Entry and FID list, then its
/// children's.
fn write_dir(out: &mut MetaSectors, layout: &Layout, dir: &DirNode) -> Result<()> {
let mut fids = dir_fids(dir);
fix_fid_tag_locations(&mut fids, dir.data_lba);
debug_assert_eq!(fids.len(), dir.data_bytes as usize);
// A directory's link count is 1 (its own FID in the parent) plus one for
// each child directory's parent FID pointing back at it.
// The planner caps subdirectory fan-out (`layout::MAX_SUBDIRS`) so this
// cannot overflow; saturating rather than wrapping keeps a future change to
// that cap from silently producing a wrong count.
let link_count = (dir.dirs.len() as u16).saturating_add(1);
let fe = file_entry(
true,
fids.len() as u64,
&[(dir.data_lba, fids.len() as u32)],
link_count,
dir.unique_id,
dir.icb_lba,
)?;
out.insert(layout.part_start + dir.icb_lba, fe);
for (i, chunk) in fids.chunks(SECTOR).enumerate() {
let mut s = blank();
s[..chunk.len()].copy_from_slice(chunk);
out.insert(layout.part_start + dir.data_lba + i as u32, s);
}
for f in &dir.files {
let extents: Vec<(u32, u32)> = f.extents.iter().map(|e| (e.lba, e.bytes)).collect();
let fe = file_entry(false, f.size, &extents, 1, f.unique_id, f.icb_lba)?;
out.insert(layout.part_start + f.icb_lba, fe);
}
for sub in &dir.dirs {
write_dir(out, layout, sub)?;
}
Ok(())
}
/// Build every metadata sector of the synthesized volume.
pub(super) fn encode(layout: &Layout) -> Result<MetaSectors> {
let mut out = MetaSectors::new();
out.insert(VRS_START, volume_recognition(b"BEA01"));
out.insert(VRS_START + 1, volume_recognition(b"NSR02"));
out.insert(VRS_START + 2, volume_recognition(b"TEA01"));
write_vds(&mut out, layout, MAIN_VDS_START);
write_vds(&mut out, layout, RESERVE_VDS_START);
out.insert(
LVID_START,
integrity(
layout.part_sectors,
layout.file_count,
layout.dir_count,
layout.next_unique_id,
LVID_START,
),
);
out.insert(LVID_START + 1, terminating(LVID_START + 1));
let avdp = anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, ANCHOR_LBA);
out.insert(ANCHOR_LBA, avdp);
let last = layout.total_sectors - 1;
out.insert(
last,
anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, last),
);
// Partition block 0 must hold the File Set Descriptor: `read_filesystem`
// reads exactly `metadata_start` (== partition start on a single-partition
// volume) and rejects the volume outright if the tag there is not 256.
out.insert(
layout.part_start,
file_set(&layout.volume_id, layout.root.icb_lba, 0),
);
out.insert(layout.part_start + 1, terminating(1));
write_dir(&mut out, layout, &layout.root)?;
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
/// The reference check value for CRC-16/XMODEM — poly 0x1021 seeded at 0,
/// which is what ECMA-167 7.2.4 specifies: "123456789" → 0x31C3. Seeding
/// at 0xFFFF instead (CCITT-FALSE) yields 0x29B1, and that mutant is
/// invisible to `udf.rs`, which never verifies a tag CRC — it would only
/// show up as a volume no operating system will mount.
#[test]
fn crc16_matches_the_ecma167_check_value() {
assert_eq!(crc16(b"123456789"), 0x31C3);
assert_ne!(crc16(b"123456789"), 0x29B1, "not the 0xFFFF-seeded variant");
}
/// ECMA-167 3/7.2.3: the checksum is the sum of the tag's first 16 bytes
/// EXCLUDING the checksum byte itself, modulo 256.
#[test]
fn tag_checksum_excludes_its_own_byte() {
let mut buf = [0u8; 512];
buf[16..24].copy_from_slice(&[1, 2, 3, 4, 5, 6, 7, 8]);
finish_tag(&mut buf, 261, 0x1234, 512);
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
assert_eq!(buf[4] as u32, sum % 256);
// And the recorded CRC covers the body, not the tag.
let crc = u16::from_le_bytes([buf[8], buf[9]]);
assert_eq!(crc, crc16(&buf[16..512]));
assert_eq!(u16::from_le_bytes([buf[10], buf[11]]), 496);
assert_eq!(
u32::from_le_bytes([buf[12], buf[13], buf[14], buf[15]]),
0x1234
);
}
/// ASCII takes compression ID 8; anything above takes 16 (UTF-16BE),
/// because `parse_udf_name` decodes compression-8 bytes as UTF-8.
#[test]
fn cs0_picks_the_encoding_the_parser_can_decode() {
assert_eq!(encode_cs0("AB"), vec![8, b'A', b'B']);
let e = encode_cs0("Ä");
assert_eq!(e[0], 16);
assert_eq!(&e[1..], &[0x00, 0xC4]);
assert_eq!(crate::udf::parse_udf_name(&e), "Ä");
}
/// A d-string records its used length in the field's LAST byte, and the
/// production parser must read the same string back.
#[test]
fn dstring_round_trips_through_the_production_parser() {
let mut field = [0u8; 32];
put_dstring(&mut field, "FREEMKV");
assert_eq!(field[31], 8, "compid byte + 7 characters");
assert_eq!(crate::udf::parse_dstring_for_test(&field), "FREEMKV");
}
}
File diff suppressed because it is too large Load Diff
-362
View File
@@ -1,362 +0,0 @@
//! `dir://` as an image-level SOURCE: a synthetic UDF volume over a folder.
//!
//! A user's extracted disc — a DVD `VIDEO_TS/` or a Blu-ray `BDMV/`, typically
//! a MakeMKV-style backup — has files but no sectors, and everything above the
//! sector layer in this crate wants sectors: `Disc::scan_image`, `UdfFs`,
//! `ifo.rs`, `mpls.rs`, `clpi.rs` and the mux all read through a
//! [`SectorSource`]. [`DirImage`] supplies one.
//!
//! The trick is that nothing is emulated. A real, minimal, valid UDF 1.02
//! volume is synthesized over the folder:
//!
//! * **Metadata sectors** (anchors, the volume descriptor sequences, the File
//! Set Descriptor, every File Entry, every directory's FID list) are encoded
//! into RAM by [`encode`] — a few MiB even for a large Blu-ray.
//! * **Data sectors** are not materialized at all. Each one maps to a byte
//! range of a real file, read on demand.
//!
//! So `udf::read_filesystem` parses this image by exactly the same code path it
//! parses a real disc with, and every consumer above it is unchanged. The cost
//! is that a single-partition synthetic volume never exercises the UDF 2.50
//! Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
//! this module's tests do not cover that block and must not be read as if they
//! did.
//!
//! What this module deliberately does NOT do:
//!
//! * **3D / SSIF** — rejected up front ([`Error::DirImageSsifUnsupported`]).
//! An SSIF aliases the same sectors as its base and dependent `.m2ts`; the
//! planner allocates disjoint extents, so a 3D folder would produce silently
//! wrong output.
//! * **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
//! * **Encrypted folders** — a folder whose content is still AACS-scrambled is
//! rejected by the caller-side probe, not decrypted here.
mod encode;
mod layout;
use crate::error::{Error, Result};
#[cfg(target_os = "linux")]
use crate::io::file_sector_source::linux::drop_window;
#[cfg(target_os = "macos")]
use crate::io::file_sector_source::macos::drop_window;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use crate::io::file_sector_source::other::drop_window;
#[cfg(target_os = "windows")]
use crate::io::file_sector_source::windows::drop_window;
use crate::sector::SectorSource;
use encode::{MetaSectors, SECTOR};
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::{Path, PathBuf};
/// How many host files may be held open at once.
///
/// A Blu-ray `BDMV/` can exceed a thousand files while macOS `RLIMIT_NOFILE`
/// defaults to 256, so "open every file up front" is not available. Reads are
/// overwhelmingly sequential through one large stream file at a time, so a
/// small LRU keeps the hit rate near 1 while bounding descriptors.
const HANDLE_CACHE: usize = 16;
/// One file's bytes at one place in the image.
#[derive(Debug, Clone)]
struct DataRange {
/// Absolute first block.
start_lba: u32,
/// Blocks covered (the last one may be partially used, and is zero-padded).
sectors: u32,
/// Index into [`DirImage::files`].
file: usize,
/// Byte offset within the file at which this range's bytes begin.
offset: u64,
/// Byte length of the range.
bytes: u64,
}
/// A file the image reads through.
#[derive(Debug)]
struct FileRef {
host: PathBuf,
disc_path: String,
size: u64,
/// Host mtime at plan time — see `layout::FileNode::mtime` for why size
/// alone is not enough.
mtime: Option<std::time::SystemTime>,
}
/// A synthesized UDF disc image over a host directory.
///
/// Owns everything it reads through (`PathBuf`s and its own file handles), so
/// it is `Send + 'static` and can be moved into `build_iso_pipeline`, which
/// hands it to `PrefetchedSectorSource`'s producer thread.
pub struct DirImage {
meta: MetaSectors,
/// Sorted by `start_lba`, non-overlapping.
ranges: Vec<DataRange>,
files: Vec<FileRef>,
open: Vec<(usize, File)>,
total_sectors: u32,
volume_id: String,
data_bytes: u64,
}
impl std::fmt::Debug for DirImage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DirImage")
.field("volume_id", &self.volume_id)
.field("total_sectors", &self.total_sectors)
.field("files", &self.files.len())
.field("meta_sectors", &self.meta.len())
.finish()
}
}
impl DirImage {
/// Plan and encode an image over `root`.
///
/// Every error is decided here, at plan time, where it can name the file
/// responsible — the read path is deliberately left with nothing to decide
/// except "this file changed underneath me".
pub fn open(root: &Path) -> Result<Self> {
let plan = layout::plan(root)?;
let meta = encode::encode(&plan)?;
let mut nodes = Vec::new();
layout::flatten(&plan.root, &mut nodes);
let mut files = Vec::with_capacity(nodes.len());
let mut ranges = Vec::new();
for (idx, node) in nodes.iter().enumerate() {
// Carry the plan-time mtime ONLY for files whose CONTENT the plan
// read — the DVD IFOs, whose bytes 0xC0/0xC4 decide where every VOB
// is placed (`layout::place_video_ts` -> `read_head`).
//
// For every other file the plan depends on the SIZE alone, and size
// is already checked. Comparing mtime on those buys nothing and
// costs real false positives: disc backups commonly live on
// exFAT/FAT32, which stores local time, so a long rip spanning a
// DST transition sees a whole-hour shift on a file nobody touched
// and would abort hours in, blaming a change that did not happen.
// The multi-gigabyte VOBs are exactly the files a long rip re-opens
// after the handle cache evicts them.
let content_sensitive = node
.disc_path
.rsplit('.')
.next()
.is_some_and(|e| e.eq_ignore_ascii_case("IFO"));
files.push(FileRef {
host: node.host.clone(),
disc_path: node.disc_path.clone(),
size: node.size,
mtime: content_sensitive.then_some(node.mtime).flatten(),
});
let mut offset = 0u64;
for e in &node.extents {
ranges.push(DataRange {
start_lba: plan.part_start + e.lba,
sectors: (e.bytes as u64).div_ceil(SECTOR as u64) as u32,
file: idx,
offset,
bytes: e.bytes as u64,
});
offset += e.bytes as u64;
}
}
ranges.sort_by_key(|r| r.start_lba);
debug_assert!(
ranges
.windows(2)
.all(|w| w[0].start_lba + w[0].sectors <= w[1].start_lba),
"planned data ranges must not overlap"
);
let data_bytes = layout::total_data_bytes(&plan.root);
tracing::info!(
target: "freemkv::dirimage",
volume_id = %plan.volume_id,
files = files.len(),
dirs = plan.dir_count,
meta_blocks = layout::metadata_block_count(&plan.root),
total_sectors = plan.total_sectors,
"synthesized UDF image over directory"
);
Ok(Self {
meta,
ranges,
files,
open: Vec::new(),
total_sectors: plan.total_sectors,
volume_id: plan.volume_id,
data_bytes,
})
}
/// UDF volume identifier the image declares (the folder's own name).
pub fn volume_id(&self) -> &str {
&self.volume_id
}
/// Total bytes of real file content the image carries — the folder's size,
/// not the image's (which also counts metadata and inter-file gaps).
pub fn data_bytes(&self) -> u64 {
self.data_bytes
}
/// The range covering `lba`, if any.
fn range_at(&self, lba: u32) -> Option<&DataRange> {
let i = self.ranges.partition_point(|r| r.start_lba <= lba);
let r = self.ranges.get(i.checked_sub(1)?)?;
(lba < r.start_lba + r.sectors).then_some(r)
}
/// Borrow an open handle for `file`, opening it (and evicting the
/// least-recently-used handle) if necessary.
///
/// Opening is also where the plan is revalidated. A folder is not a disc:
/// a file can be shortened or replaced between planning and reading, and
/// zero-filling the difference would turn "the user deleted something"
/// into corrupt output at exit 0. The size is re-checked here, and a
/// truncation that happens while the handle is already open is caught by
/// the short read in [`Self::fill`].
fn handle(&mut self, file: usize) -> Result<&mut File> {
if let Some(pos) = self.open.iter().position(|(i, _)| *i == file) {
// `open` is ordered most-recently-used first.
let entry = self.open.remove(pos);
self.open.insert(0, entry);
return Ok(&mut self.open[0].1);
}
let f = File::open(&self.files[file].host).map_err(Error::from)?;
let md = f.metadata().map_err(Error::from)?;
// Size AND mtime. Size alone is content-blind, and this plan depends on
// content: a DVD's VOB placement comes from bytes 0xC0/0xC4 of its IFO,
// and an IFO rewritten in place keeps its length because IFOs occupy a
// whole number of sectors. The size check would pass while every title
// extent pointed at the wrong sectors — corrupt video behind an intact
// structure, reported complete at exit 0.
//
// Only compared when both sides have a timestamp; a platform or
// filesystem that reports none simply falls back to the size check
// rather than failing every read.
let changed_size = md.len() != self.files[file].size;
let changed_mtime = match (self.files[file].mtime, md.modified().ok()) {
(Some(planned), Some(live)) => planned != live,
_ => false,
};
if changed_size || changed_mtime {
return Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
});
}
if self.open.len() >= HANDLE_CACHE {
self.open.pop();
}
self.open.insert(0, (file, f));
Ok(&mut self.open[0].1)
}
/// Fill `out` (a whole number of sectors) from one data range, starting at
/// `lba`. `out` is already zeroed, so a file's tail sector comes back
/// zero-padded — which is exactly what `file_extents`' `div_ceil(2048)`
/// (`udf.rs:816`) makes every consumer expect.
fn fill(&mut self, r: &DataRange, lba: u32, out: &mut [u8]) -> Result<()> {
let within = (lba - r.start_lba) as u64 * SECTOR as u64;
let want = (r.bytes.saturating_sub(within)).min(out.len() as u64) as usize;
if want == 0 {
return Ok(());
}
let at = r.offset + within;
let file = r.file;
let h = self.handle(file)?;
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Release the window just read, every time.
//
// The ISO source accumulates and drops in chunks because it reads
// one file linearly, so a running start offset always names the
// bytes it has consumed. Reads here jump between files, so there is
// no single cursor to accumulate against — an accumulated byte
// count paired with one read's offset names 1/Nth of what was
// actually consumed and leaves the rest pinned, which is how the
// first version of this got it wrong.
//
// Dropping per read costs one advisory syscall per batch (4-16 MiB),
// which is nothing against the read itself, and it is correct
// regardless of how reads interleave across files.
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
}
match res {
Ok(()) => Ok(()),
// The file shrank while the handle was open. Same verdict as the
// size check in `handle`, reached the other way.
Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
})
}
Err(e) => Err(Error::from(e)),
}
}
}
impl SectorSource for DirImage {
fn capacity_sectors(&self) -> u32 {
self.total_sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let need = count as usize * SECTOR;
if buf.len() < need {
return Err(Error::UdfBufferTooSmall);
}
buf[..need].fill(0);
// Walk the request in RUNS, not sector by sector. A mux batch is 8192
// sectors and almost always lands entirely inside one stream file's
// extent; per-sector seek+read would issue 8192 syscalls for what is
// one 16 MiB sequential read.
let mut i = 0u32;
while i < count as u32 {
// Checked: callers saturate their LBAs (`disc/dvd.rs` builds a cell
// start as `vob_start_sector.saturating_add(cell.first_sector)`, and
// the prefetcher adds an offset the same way), so a crafted IFO can
// present a request at the very top of the address space. Wrapping
// here would fold `at` back to a LOW sector and hand the muxer a
// different file's bytes with nothing reported.
let Some(at) = lba.checked_add(i) else {
break;
};
let off = i as usize * SECTOR;
if let Some(s) = self.meta.get(&at) {
buf[off..off + SECTOR].copy_from_slice(&s[..]);
i += 1;
continue;
}
// Metadata blocks all sit below the data floor, so a data range is
// never interrupted by one.
match self.range_at(at).cloned() {
Some(r) => {
let run = (r.start_lba + r.sectors - at).min(count as u32 - i);
let end = off + run as usize * SECTOR;
self.fill(&r, at, &mut buf[off..end])?;
i += run;
}
// A gap between planned extents. Reads as zeros, exactly as an
// unrecorded sector of a real image does.
None => i += 1,
}
}
Ok(need)
}
}
#[cfg(test)]
mod tests;
File diff suppressed because it is too large Load Diff
+121 -1377
View File
File diff suppressed because it is too large Load Diff
+22 -214
View File
@@ -7,32 +7,13 @@ use crate::udf;
impl Disc { impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO). /// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
pub(super) fn scan_dvd_titles( pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource, reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs, udf_fs: &udf::UdfFs,
halt: Option<&crate::halt::Halt>, ) -> Vec<DiscTitle> {
) -> Result<Vec<DiscTitle>> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) { let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info, Ok(info) => info,
Err(Error::Halted) => return Err(Error::Halted), Err(_) => return Vec::new(),
Err(_) => return Ok(Vec::new()),
}; };
let mut titles = Vec::new(); let mut titles = Vec::new();
@@ -198,10 +179,7 @@ impl Disc {
// the coded video frame the subpicture was authored against // the coded video frame the subpicture was authored against
// (720x480 NTSC / 720x576 PAL) so players place and scale the // (720x480 NTSC / 720x576 PAL) so players place and scale the
// bitmap correctly. // bitmap correctly.
// format_palette guards on (0, 0) and omits its `size:` line, let (vid_w, vid_h) = ts.video.resolution.pixels();
// so an unresolved resolution degrades to a palette-only .idx
// rather than one claiming a 0x0 frame.
let (vid_w, vid_h) = ts.video.resolution.pixels().unwrap_or((0, 0));
let codec_data = dvd_title let codec_data = dvd_title
.palette .palette
.as_ref() .as_ref()
@@ -262,14 +240,7 @@ impl Disc {
} }
} }
// Polled again AFTER the loop: a cancel raised during the IFO reads titles
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok(titles)
} }
} }
@@ -554,118 +525,6 @@ mod tests {
// Tests // Tests
// --------------------------------------------------------------- // ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|ts| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|ts| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed /// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics. /// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test] #[test]
@@ -673,11 +532,7 @@ mod tests {
let mut disc = MemDisc::new(); let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing. // VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]); let udf = build_video_ts_fs(&mut disc, &[]);
assert!( assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.is_empty()
);
} }
/// Single VTS, single title, one cell. Extent absolute LBA = /// Single VTS, single title, one cell. Extent absolute LBA =
@@ -713,7 +568,7 @@ mod tests {
}, },
], ],
); );
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan"); let titles = Disc::scan_dvd_titles(&mut disc, &udf);
assert_eq!(titles.len(), 1); assert_eq!(titles.len(), 1);
let t = &titles[0]; let t = &titles[0];
assert_eq!(t.extents.len(), 1); assert_eq!(t.extents.len(), 1);
@@ -769,7 +624,7 @@ mod tests {
}, },
], ],
); );
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan"); let titles = Disc::scan_dvd_titles(&mut disc, &udf);
assert_eq!(titles.len(), 1); assert_eq!(titles.len(), 1);
let t = &titles[0]; let t = &titles[0];
assert_eq!(t.extents.len(), 1); assert_eq!(t.extents.len(), 1);
@@ -827,7 +682,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
assert_eq!(t.extents.len(), 1); assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba; let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33). // The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -885,7 +740,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
assert_eq!(t.extents.len(), 2); assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0 assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100); assert_eq!(t.extents[0].sector_count, 100);
@@ -926,7 +781,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let v = t let v = t
.streams .streams
.iter() .iter()
@@ -981,7 +836,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let v = t let v = t
.streams .streams
.iter() .iter()
@@ -1042,7 +897,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let audios: Vec<_> = t let audios: Vec<_> = t
.streams .streams
.iter() .iter()
@@ -1053,7 +908,7 @@ mod tests {
.collect(); .collect();
assert_eq!(audios.len(), 2); assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3); assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "eng"); assert_eq!(audios[0].language, "en");
assert_eq!(audios[1].codec, Codec::Dts); assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the // Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch). // AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -1112,7 +967,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let audios: Vec<_> = t let audios: Vec<_> = t
.streams .streams
.iter() .iter()
@@ -1169,7 +1024,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let subs: Vec<_> = t let subs: Vec<_> = t
.streams .streams
.iter() .iter()
@@ -1182,7 +1037,7 @@ mod tests {
// Languages preserved in order. // Languages preserved in order.
assert_eq!( assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(), subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["eng", "fra", "deu"] vec!["en", "fr", "de"]
); );
// PIDs are 0x20 + ordinal, all distinct. // PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect(); let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1229,7 +1084,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let sub = t let sub = t
.streams .streams
.iter() .iter()
@@ -1239,7 +1094,7 @@ mod tests {
}) })
.expect("subtitle stream"); .expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub); assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "eng"); assert_eq!(sub.language, "en");
assert!( assert!(
sub.codec_data.is_some(), sub.codec_data.is_some(),
"non-zero palette must yield codec_data" "non-zero palette must yield codec_data"
@@ -1283,7 +1138,7 @@ mod tests {
}, },
], ],
); );
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan"); let titles = Disc::scan_dvd_titles(&mut disc, &udf);
assert_eq!(titles.len(), 2); assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets. // title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1); assert_eq!(titles[0].playlist_id, 1);
@@ -1320,7 +1175,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
// One program in the program map → one chapter time (0.0 for the // One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0). // first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1); assert_eq!(t.chapters.len(), 1);
@@ -1405,7 +1260,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3. // The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped"); assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0. // First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1457,7 +1312,7 @@ mod tests {
}, },
], ],
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
// Nothing dropped: both cells become extents, starting at the very head. // Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2); assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1465,51 +1320,4 @@ mod tests {
// Chapter 0 stays at 0.0 (no shift). // Chapter 0 stays at 0.0 (no shift).
assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01); assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01);
} }
/// Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio
/// stream has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio,
/// per `ifo::assign_audio_sub_stream_ids` — the PID falls back to
/// `0xBD00 + i` where `i` is the stream's positional index in the IFO
/// audio-attribute table. Two MPEG-audio (coding_mode 2) streams must
/// land on two DISTINCT, correctly-offset PIDs: 0xBD00 and 0xBD01. This
/// pins the `+` (not `-`/`*`) so the second stream doesn't collide with,
/// or wrap under, the first.
#[test]
fn scan_dvd_titles_mp2_audio_pid_fallback_is_additive() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// coding_mode bits are b0>>5 & 0x7; mode 2 = MPEG-1 Layer II (Mp2),
// which `assign_audio_sub_stream_ids` leaves at `sub_stream_id: None`.
// b0 = 0b010_00000 = 0x40. b1 = 0 (mono, sample rate 48k).
let audio = [(0x40u8, 0x00u8, [0u8, 0u8]), (0x40u8, 0x00u8, [0u8, 0u8])];
let vts = build_vts(1000, 0x00, &audio, &[], &[(10, 109)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts,
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
.iter()
.filter_map(|s| match s {
Stream::Audio(a) => Some(a.pid),
_ => None,
})
.collect();
assert_eq!(audio_pids, vec![0xBD00u16, 0xBD01u16]);
}
} }
+5 -173
View File
@@ -104,10 +104,10 @@ fn max_substream_channels(data: &[u8]) -> Option<u8> {
}; };
let start = pos + rel; let start = pos + rel;
let frame = &data[start..]; let frame = &data[start..];
if let Some(ch) = ac3::acmod_channels(frame) if let Some(ch) = ac3::acmod_channels(frame) {
&& ch > 0 if ch > 0 {
{ best = Some(best.map_or(ch, |b| b.max(ch)));
best = Some(best.map_or(ch, |b| b.max(ch))); }
} }
// Advance past this frame by its declared size when that is mappable; // Advance past this frame by its declared size when that is mappable;
// otherwise step 2 bytes past the sync and re-scan for the next one. // otherwise step 2 bytes past the sync and re-scan for the next one.
@@ -242,11 +242,7 @@ pub fn probe_and_remap<S: SectorSource + ?Sized>(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::disc::{ use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate};
AudioChannels, AudioStream, Codec, ContentFormat, DiscTitle, Extent, LabelPurpose,
SampleRate,
};
use crate::sector::SectorSource;
/// Build a single, correctly-SIZED AC-3 frame whose `acmod`/`lfeon` encode a /// Build a single, correctly-SIZED AC-3 frame whose `acmod`/`lfeon` encode a
/// known channel count. `byte4` is `fscod=0 | frmsizecod=0`, so /// known channel count. `byte4` is `fscod=0 | frmsizecod=0`, so
@@ -467,168 +463,4 @@ mod tests {
}; };
assert_eq!(a.pid, 0xBD80, "no probe data → keep ordinal"); assert_eq!(a.pid, 0xBD80, "no probe data → keep ordinal");
} }
/// `max_substream_channels` must locate the sync at its true ABSOLUTE
/// position (`pos + rel`) when it is preceded by non-sync bytes, not just
/// when the sync sits at offset 0. Regression guard for a hand-checked
/// mutation (`+` → `-` at the `pos + rel` offset computation): with `pos`
/// starting at 0 and the first sync found 3 bytes in, `pos - rel` would
/// underflow a `usize` and panic, or (if it somehow didn't) index the
/// wrong start entirely. `pos + rel` is the only computation that is
/// always in-bounds, since `rel` is itself bounded by the length of the
/// slice searched from `pos`.
#[test]
fn max_substream_channels_locates_sync_after_leading_non_sync_bytes() {
let mut data = vec![0xAA, 0xAA, 0xAA]; // no 0x0B77 pattern in here
data.extend(ac3_frame(2, false)); // real 2.0 frame, sync at absolute offset 3
assert_eq!(
max_substream_channels(&data),
Some(2),
"must find and decode the frame whose sync is NOT at offset 0"
);
}
/// When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
/// `fscod == 3`), `max_substream_channels` must fall back to stepping
/// `start + 2` bytes past the sync to re-lock onto the next genuine sync,
/// and must keep making forward progress doing so (never revisit the same
/// sync, which would loop forever, and never jump so far that it skips
/// the very next real frame). This lays a bogus-sized header at absolute
/// offset 4 (so `start == 4`, `start + 2 == 6`) immediately followed, at
/// offset 6, by a real, fully decodable 2.0 frame — the position the
/// `+ 2` fallback must land on exactly.
#[test]
fn max_substream_channels_unmappable_size_steps_forward_by_two() {
let mut real = ac3_frame(2, false);
// Overwrite the (unchecked) CRC bytes of the real frame — these double
// as byte4/byte5 of the bogus header 2 bytes earlier, at absolute
// offset 4: byte4 = 0xC0 (fscod=3 reserved -> ac3_frame_size == 0,
// unmappable), byte5 = 0xF8 (bsid=31 >= 11 -> acmod_channels == None,
// so the bogus header itself never contributes a spurious channel
// count).
real[2] = 0xC0;
real[3] = 0xF8;
let mut data = vec![0xAA, 0xAA, 0xAA, 0xAA]; // offsets 0..4, no sync
data.push(0x0B); // offset 4: bogus header sync byte 0
data.push(0x77); // offset 5: bogus header sync byte 1
data.extend(real); // offset 6..: the real frame (also serves as the
// bogus header's byte4/byte5 at offsets 8/9)
assert_eq!(
max_substream_channels(&data),
Some(2),
"must recover the real frame 2 bytes after the unmappable-size sync, not lose it"
);
}
/// Same fallback as above, but with the unmappable-size sync at absolute
/// offset 0 (`start == 0`) so that stepping backward instead of forward
/// (`start - 2`) would underflow rather than merely land on the wrong
/// byte. Also proves the real frame is still found 6 bytes further in,
/// confirming forward progress past the bogus header.
#[test]
fn max_substream_channels_unmappable_size_at_start_steps_forward_not_back() {
let mut data = vec![0x0B, 0x77, 0x00, 0x00, 0xC0, 0xF8]; // bogus header, offsets 0..6
data.extend(ac3_frame(2, false)); // real 2.0 frame at offset 6
assert_eq!(
max_substream_channels(&data),
Some(2),
"must step forward past the bogus header at offset 0 and find the real frame at offset 6"
);
}
/// `remap_audio_pids` must read a stream's CURRENT physical sub-stream id
/// from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
/// `0x00FF`, both of which force the low byte to `0xFF` regardless of the
/// real PID and so always miss the "already matches" shortcut. That
/// matters observably when TWO physical sub-streams share the same probed
/// channel count: with a correct read, a stream already sitting on a
/// matching sub-stream is left alone (conservative, per the module's
/// documented behaviour); with the low byte forced to `0xFF`,
/// `probed.get(&0xFF)` is always `None`, so the code falls through to the
/// "find any unclaimed match" path and picks the FIRST (lowest-keyed,
/// BTreeMap-ordered) matching physical sub-stream instead — which here is
/// a *different* sub-stream (0x80) than the one the PID already correctly
/// names (0x81), producing a spurious PID change.
#[test]
fn remap_reads_current_substream_via_and_not_or_or_xor() {
let mut probed = BTreeMap::new();
probed.insert(0x80u8, 6u8);
probed.insert(0x81u8, 6u8); // ambiguous: two physical 6ch sub-streams
let mut streams = vec![ac3_stream(0xBD81, AudioChannels::Surround51)];
let changed = remap_audio_pids(&mut streams, &probed);
assert_eq!(
changed, 0,
"already sitting on a matching physical sub-stream (0x81) must be left alone"
);
let Stream::Audio(a) = &streams[0] else {
panic!()
};
assert_eq!(
a.pid, 0xBD81,
"must not be bumped to the other matching sub-stream (0x80)"
);
}
/// A `SectorSource` stub that hands back fixed bytes regardless of the
/// requested LBA/count, for exercising `probe_and_remap`'s end-to-end
/// wiring (format/AC-3/extent/count guards -> read -> probe -> remap).
struct FixedSource {
data: Vec<u8>,
}
impl SectorSource for FixedSource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let n = self.data.len().min(buf.len());
buf[..n].copy_from_slice(&self.data[..n]);
Ok(n)
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
/// stream to 0xBD81. It also, by construction, proves each of the guards
/// along the way lets a real, positive case through: the content-format
/// check must NOT bail on `MpegPs` (only on non-`MpegPs`), the AC-3
/// presence check must NOT bail when AC-3 IS present, and the
/// sector-count check must NOT bail when the count is nonzero — any one
/// of those inverted would skip the probe entirely and leave the PID at
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
playlist: "00001.ifo".into(),
playlist_id: 1,
duration_secs: 60.0,
size_bytes: bytes.len() as u64,
clips: Vec::new(),
streams: vec![ac3_stream(0xBD80, AudioChannels::Surround51)],
chapters: Vec::new(),
extents: vec![Extent {
start_lba: 0,
sector_count: 2,
}],
content_format: ContentFormat::MpegPs,
codec_privates: vec![None],
};
let mut source = FixedSource { data: bytes };
probe_and_remap(&mut source, &mut title);
let Stream::Audio(a) = &title.streams[0] else {
panic!("audio")
};
assert_eq!(
a.pid, 0xBD81,
"declared 5.1 stream must be re-routed to the physical 5.1 sub-stream 0x81"
);
}
} }
+33 -64
View File
@@ -181,24 +181,6 @@ fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::trace::UnlockOutco
} }
} }
/// Did the cert handshake actually carry a Volume ID?
///
/// Extracted so it can be tested as a VALUE. It only ever reaches an operator
/// as the `has_volume_id` field of the `bus_key_unavailable` warn, and
/// asserting on a `tracing` field means installing a capturing subscriber —
/// which is thread-local, while `tracing`'s callsite-interest cache is global.
/// Those two facts race: the test failed roughly one run in ten under the full
/// parallel suite while passing every time in isolation, and serialising the
/// captures was not enough because the cache can be re-evaluated against the
/// process default rather than the thread-local dispatch.
///
/// A predicate this small does not need a subscriber to verify. The polarity is
/// the whole point: an `==` here would tell an operator a VID was absent on
/// exactly the discs where one was present.
fn handshake_has_volume_id(h: &HandshakeResult) -> bool {
h.volume_id != [0u8; 16]
}
impl Disc { impl Disc {
/// SCSI handshake — drives the VID-acquisition flow and returns /// SCSI handshake — drives the VID-acquisition flow and returns
/// a structured `HandshakeResult` for downstream key resolution. /// a structured `HandshakeResult` for downstream key resolution.
@@ -387,7 +369,7 @@ impl Disc {
// file/ISO, drive unlock, cert bus key). The gate enumerates nothing. // file/ISO, drive unlock, cert bus key). The gate enumerates nothing.
if !bus_encryption_removed(bus_encryption, handshake) { if !bus_encryption_removed(bus_encryption, handshake) {
let (rdk_err, has_vid) = handshake let (rdk_err, has_vid) = handshake
.map(|h| (h.read_data_key_err, handshake_has_volume_id(h))) .map(|h| (h.read_data_key_err, h.volume_id != [0u8; 16]))
.unwrap_or((None, false)); .unwrap_or((None, false));
tracing::warn!( tracing::warn!(
target: "freemkv::disc", target: "freemkv::disc",
@@ -967,54 +949,41 @@ mod tests {
/// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy /// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy
/// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`). /// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`).
/// This is the damaged-primary recovery path real discs rely on.
#[test] #[test]
fn handshake_has_volume_id_reports_presence_not_absence() { fn resolve_vid_only_falls_back_to_duplicate_unit_key_ro() {
let with_vid = HandshakeResult { let mut disc = MemDisc::new();
volume_id: [0x11u8; 16], // Build AACS dir with a DUPLICATE subdir holding Unit_Key_RO.inf.
read_data_key: None, let uk = vec![0x55u8; 48];
read_data_key_err: None, let mut dup_fids = Vec::new();
drive_unlocked: false, push_fid(&mut dup_fids, "", 70, true, true);
}; push_fid(&mut dup_fids, "Unit_Key_RO.inf", 72, false, false);
assert!( disc.put(PART_START + 72, build_file_icb(uk.len() as u32, 9000));
super::handshake_has_volume_id(&with_vid), disc.put_bytes(PART_START + 9000, &uk);
"a non-zero Volume ID must report as PRESENT" disc.put(PART_START + 70, build_file_icb(dup_fids.len() as u32, 71));
); disc.put_bytes(PART_START + 71, &dup_fids);
// AACS dir: only a DUPLICATE subdir (no primary Unit_Key_RO.inf).
let mut aacs_fids = Vec::new();
push_fid(&mut aacs_fids, "", 50, true, true);
push_fid(&mut aacs_fids, "DUPLICATE", 70, true, false);
disc.put(PART_START + 50, build_file_icb(aacs_fids.len() as u32, 51));
disc.put_bytes(PART_START + 51, &aacs_fids);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "AACS", 50, true, false);
disc.put(PART_START + 10, build_file_icb(root_fids.len() as u32, 11));
disc.put_bytes(PART_START + 11, &root_fids);
build_udf_skeleton(&mut disc, 10);
let udf = udf::read_filesystem(&mut disc).expect("fs");
let without = HandshakeResult { let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("DUPLICATE fallback");
volume_id: [0u8; 16], // disc_hash must be computed over the DUPLICATE bytes.
..with_vid assert_eq!(
}; st.disc_hash,
assert!( aacs::inf::disc_hash_hex(&aacs::inf::disc_hash(&uk)),
!super::handshake_has_volume_id(&without), "fallback must hash the DUPLICATE Unit_Key_RO.inf"
"an all-zero Volume ID is the absent case"
); );
assert_eq!(st.uk_ro, uk);
// One bit of difference is still a VID: the check is != all-zero, not a
// heuristic about how much of it looks populated.
let mut barely = [0u8; 16];
barely[15] = 1;
assert!(
super::handshake_has_volume_id(&HandshakeResult {
volume_id: barely,
..with_vid
}),
"any non-zero byte makes a Volume ID present"
);
}
/// The gate itself still hard-errors — the property the log line annotates.
#[test]
fn resolve_vid_only_bus_key_gate_hard_errors_without_a_read_data_key() {
let (mut disc, udf) = disc_with_cert(0x01, true);
let hs = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
let err = Disc::resolve_vid_only(&udf, &mut disc, Some(&hs))
.expect_err("bus-encrypted, no read_data_key must still hard-error");
assert!(matches!(err, Error::AacsBusKeyUnavailable));
} }
// --------------------------------------------------------------- // ---------------------------------------------------------------
+81 -963
View File
File diff suppressed because it is too large Load Diff
+61 -2303
View File
File diff suppressed because it is too large Load Diff
+116 -2770
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+8 -6
View File
@@ -23,12 +23,14 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
if !std::path::Path::new(&path).exists() { if !std::path::Path::new(&path).exists() {
continue; continue;
} }
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path)) if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path)) {
&& let Ok(id) = DriveId::from_drive(transport.as_mut()) if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
&& !id.raw_inquiry.is_empty() if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL && (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{ {
drives.push((path, id)); drives.push((path, id));
}
}
} }
} }
drives drives
+6 -35
View File
@@ -25,11 +25,12 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
let path = std::path::Path::new(&info.path); let path = std::path::Path::new(&info.path);
match crate::scsi::open(path) { match crate::scsi::open(path) {
Ok(mut transport) => { Ok(mut transport) => {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
&& !id.raw_inquiry.is_empty() if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL && (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{ {
drives.push((info.path.clone(), id)); drives.push((info.path.clone(), id));
}
} }
} }
Err(_) => { Err(_) => {
@@ -52,33 +53,3 @@ pub fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
} }
Ok((path.to_string(), DeviceResolution::Direct)) Ok((path.to_string(), DeviceResolution::Direct))
} }
#[cfg(test)]
mod resolve_device_tests {
use super::*;
/// An existing path resolves unchanged as `Direct` — macOS has no
/// `sr`->`sg` substitution, so the returned path must be byte-identical
/// to the input, not some canonicalised/mutated form.
#[test]
fn existing_path_resolves_direct_unchanged() {
// Use the test binary's own executable path: guaranteed to exist,
// no fixture file needed.
let exe = std::env::current_exe().unwrap();
let path = exe.to_str().unwrap();
let (resolved, kind) = resolve_device(path).expect("existing path must resolve");
assert_eq!(resolved, path, "path must be returned unchanged");
assert_eq!(kind, DeviceResolution::Direct);
}
/// A path that does not exist must error with `DeviceNotFound` carrying
/// the original path, never silently succeed.
#[test]
fn missing_path_is_device_not_found() {
let path = "/dev/freemkv-definitely-does-not-exist-0xdead";
match resolve_device(path) {
Err(Error::DeviceNotFound { path: p }) => assert_eq!(p, path),
other => panic!("expected DeviceNotFound, got {other:?}"),
}
}
}
+132 -1063
View File
File diff suppressed because it is too large Load Diff
+127 -792
View File
File diff suppressed because it is too large Load Diff
-267
View File
@@ -1,267 +0,0 @@
//! Seeded robustness harness for the untrusted-input parsers.
//!
//! Every parser reached from here takes bytes that came off a disc, and this
//! crate's primary boundary is that the disc is untrusted: a malformed, damaged
//! or hostile image must never crash the library. These tests assert exactly
//! that one property — **the parser returns `Ok` or `Err`, and never panics.**
//!
//! # Why this exists rather than `cargo-fuzz`
//!
//! `cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
//! for libFuzzer's coverage feedback) and this project pins stable. So the
//! generator lives here instead. It gives up coverage-guided mutation — the real
//! loss — and keeps everything else: millions of cases, structure-aware input,
//! and a crash corpus. It also gains determinism, which a fuzzer does not have:
//! the same seed replays the same cases on any machine.
//!
//! # Why no `proptest` or `arbitrary`
//!
//! This crate has exactly one dev-dependency. That is a deliberate posture, and
//! a randomness crate is not worth ten transitive dependencies when the parsers
//! take plain `&[u8]` and a good enough generator is forty lines.
//!
//! # Budget
//!
//! `FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
//! enough that the per-commit gate stays under a second). The overnight run sets
//! it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
//! fixed so a failure in CI reproduces locally verbatim.
//!
//! # On failure
//!
//! The panic message carries the seed, generator and case index. Re-run with
//! `FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
//! into `tests/corpus/` as a permanent regression fixture — discovery happens
//! here, defence happens there.
#![cfg(test)]
/// Marsaglia xorshift64. Not cryptographic and does not need to be: the job is
/// a reproducible spread of bytes, and a named algorithm beats an ad-hoc LCG
/// whose period nobody has checked.
struct Rng(u64);
impl Rng {
fn new(seed: u64) -> Self {
// A zero seed is a fixed point of xorshift — it would emit zeros forever
// and every generated case would be identical.
Self(if seed == 0 {
0x2545_F491_4F6C_DD1D
} else {
seed
})
}
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
fn byte(&mut self) -> u8 {
(self.next() >> 24) as u8
}
/// Uniform-ish in `0..n`. The modulo bias is irrelevant at these magnitudes.
fn below(&mut self, n: usize) -> usize {
if n == 0 {
0
} else {
(self.next() % n as u64) as usize
}
}
fn fill(&mut self, len: usize) -> Vec<u8> {
(0..len).map(|_| self.byte()).collect()
}
}
/// Budget per generator per target.
fn cases() -> usize {
std::env::var("FREEMKV_HARNESS_CASES")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(256)
}
fn seed() -> u64 {
std::env::var("FREEMKV_HARNESS_SEED")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(0x5EED_1234_ABCD_0001)
}
/// Largest generated input. Big enough to carry a plausible header plus a body,
/// small enough that millions of cases stay quick.
const MAX_LEN: usize = 4096;
/// Drive `f` over three generators and report which case broke it.
///
/// A panic inside `f` fails the test on its own — nothing is caught here,
/// because catching would risk reporting a pass on an input that aborted. The
/// wrapper exists to make the failing case *identifiable*: the harness prints
/// the seed, generator and index before each call, so the last line before a
/// panic names the exact case to reproduce.
fn sweep<F: FnMut(&[u8])>(target: &str, magic: &[u8], f: F) {
sweep_n(target, magic, cases(), f)
}
/// `sweep` with an explicit budget. The budget is a PARAMETER rather than read
/// from the environment inside the loop: the meta-tests below need a small,
/// fixed count, and `std::env::set_var` is unsound once the test harness runs
/// tests in parallel — two tests setting the same variable race, which is
/// exactly what happened on the first run of this file.
fn sweep_n<F: FnMut(&[u8])>(target: &str, magic: &[u8], n: usize, mut f: F) {
let s = seed();
// 1. Pure random bytes. Cheap, and almost always rejected at the magic
// number — it exercises the entry guards and little else. Kept because
// the entry guards are themselves worth exercising.
let mut rng = Rng::new(s);
for i in 0..n {
let len = rng.below(MAX_LEN);
let buf = rng.fill(len);
run(target, "random", s, i, &buf, &mut f);
}
// 2. Valid magic, random body. THE generator that matters: pure random
// input dies at the magic check and never reaches the parser body, so
// without this the sweep only ever tests the first few lines.
let mut rng = Rng::new(s ^ 0xA5A5_A5A5_A5A5_A5A5);
for i in 0..n {
let mut buf = magic.to_vec();
let tail = rng.below(MAX_LEN.saturating_sub(magic.len()));
buf.extend(rng.fill(tail));
run(target, "magic+noise", s, i, &buf, &mut f);
}
// 3. Structured mutation of a plausible record: a valid magic, then mostly
// zeroes, with a handful of bytes corrupted and a truncation. Length and
// offset fields live in those early bytes, so this is what reaches the
// arithmetic — the offsets, counts and sizes a hostile image would lie
// about.
let mut rng = Rng::new(s ^ 0x1234_5678_9ABC_DEF0);
for i in 0..n {
let mut buf = vec![0u8; 512];
buf[..magic.len().min(512)].copy_from_slice(&magic[..magic.len().min(512)]);
for _ in 0..rng.below(24) + 1 {
let at = rng.below(buf.len());
buf[at] = rng.byte();
}
buf.truncate(rng.below(buf.len()) + 1);
run(target, "mutate", s, i, &buf, &mut f);
}
}
fn run<F: FnMut(&[u8])>(target: &str, generator: &str, seed: u64, i: usize, buf: &[u8], f: &mut F) {
// Printed, not asserted: `cargo test` swallows stdout for passing tests and
// shows it for failing ones, so this line is invisible until it is the last
// thing before a panic — at which point it is exactly what is needed.
println!(
"harness {target}/{generator} seed={seed:#x} case={i} len={} :: \
FREEMKV_HARNESS_SEED={seed} to reproduce",
buf.len()
);
f(buf);
}
#[test]
fn mpls_parse_never_panics() {
sweep("mpls", b"MPLS", |b| {
let _ = crate::mpls::parse(b);
});
}
#[test]
fn clpi_parse_never_panics() {
sweep("clpi", b"HDMV", |b| {
let _ = crate::clpi::parse(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
// input to reject, so the "magic" is a byte the sweep will mutate anyway.
sweep("udf_name", &[8], |b| {
let _ = crate::udf::parse_udf_name(b);
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
// each case is fed to a FRESH demuxer and then a shared one. The shared pass
// is what exercises cross-feed state — a start code split over a boundary,
// a held PES completed by later bytes, the carry-over cap.
let mut shared = crate::mux::ps::PsDemuxer::new();
sweep("ps_demux", &[0x00, 0x00, 0x01, 0xBA], |b| {
let mut fresh = crate::mux::ps::PsDemuxer::new();
let _ = fresh.feed(b);
let _ = shared.feed(b);
});
}
#[test]
fn mkv_lacing_split_never_panics() {
// All four lacing modes, including the reserved bit pattern. A degenerate
// fixed lace was a real defect found by audit round 5.
sweep("mkv_lacing", &[0x00], |b| {
for lacing in 0u8..=3 {
let _ = crate::mux::mkvstream::split_lacing(lacing, b);
}
});
}
/// The generators must actually differ, or the sweep is one generator run three
/// times and the coverage claim is false.
#[test]
fn the_three_generators_produce_different_inputs() {
let mut seen: Vec<Vec<u8>> = Vec::new();
sweep_n("probe", b"MPLS", 1, |b| seen.push(b.to_vec()));
assert_eq!(seen.len(), 3, "one case per generator");
assert_ne!(seen[0], seen[1], "random and magic+noise must differ");
assert_ne!(seen[1], seen[2], "magic+noise and mutate must differ");
assert!(
seen[1].starts_with(b"MPLS"),
"the magic+noise generator must actually carry the magic, or it never \
reaches the parser body"
);
}
/// The same seed must replay the same bytes, or a reported failure cannot be
/// reproduced and the harness is worthless as a regression tool.
#[test]
fn a_seed_replays_identically() {
let mut a = Vec::new();
let mut b = Vec::new();
sweep_n("probe", b"MPLS", 4, |x| a.push(x.to_vec()));
sweep_n("probe", b"MPLS", 4, |x| b.push(x.to_vec()));
assert_eq!(a, b, "the same seed must produce the same cases");
}
/// The harness is worthless if its cases die at the entry guards, so this
/// MEASURES how deep they actually reach instead of assuming. A generator that
/// never gets past a length or magic check exercises the first ten lines and
/// nothing else — the fuzzing equivalent of a test that cannot fail.
#[test]
fn the_generators_actually_reach_the_parser_bodies() {
// mpls::parse rejects at: len < 40, bad magic, then playlist_start + 10 >
// len. Anything that returns Ok got all the way through the play-item loop.
let mut ok = 0usize;
let mut total = 0usize;
sweep_n("reach", b"MPLS", 20000, |b| {
total += 1;
if crate::mpls::parse(b).is_ok() {
ok += 1;
}
});
assert!(
ok > 0,
"not one of {total} generated cases parsed successfully — the generators \
are all being rejected at the entry guards, so this harness is testing \
the guards and nothing behind them"
);
println!("mpls reach: {ok}/{total} cases parsed to completion");
}
+1 -1
View File
@@ -20,7 +20,7 @@ pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
let bytes = body.as_bytes(); let bytes = body.as_bytes();
// Empty → empty Vec (a legitimately-empty variable-length field); odd length // Empty → empty Vec (a legitimately-empty variable-length field); odd length
// is malformed. (`parse_hex_fixed` enforces a concrete length separately.) // is malformed. (`parse_hex_fixed` enforces a concrete length separately.)
if !bytes.len().is_multiple_of(2) { if bytes.len() % 2 != 0 {
return None; return None;
} }
let mut out = Vec::with_capacity(bytes.len() / 2); let mut out = Vec::with_capacity(bytes.len() / 2);
+1 -193
View File
@@ -49,30 +49,13 @@ pub struct DriveId {
pub raw_gc_010c: Vec<u8>, pub raw_gc_010c: Vec<u8>,
} }
/// SPC-4 standard INQUIRY data: 36 bytes through `product_revision`. Anything
/// shorter cannot populate the identity fields this type promises.
const INQUIRY_STANDARD_LEN: usize = 36;
impl DriveId { impl DriveId {
/// Probe a real drive via SCSI and build its identity. /// Probe a real drive via SCSI and build its identity.
pub fn from_drive(transport: &mut dyn ScsiTransport) -> Result<Self> { pub fn from_drive(transport: &mut dyn ScsiTransport) -> Result<Self> {
// INQUIRY — SPC-4 §6.4 // INQUIRY — SPC-4 §6.4
let mut inquiry = vec![0u8; 96]; let mut inquiry = vec![0u8; 96];
let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00]; let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00];
let inq = transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?; transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
// `bytes_transferred` is device-reported and untrusted — the same rule
// the two GET CONFIGURATION calls below already apply. It was ignored
// here, and the buffer is pre-zeroed, so a drive answering GOOD with a
// short or empty data phase (a USB-SATA bridge mid-wedge does exactly
// this) decoded to blank identity strings and a byte 0 of 0x00. Every
// platform enumerator gates on `raw_inquiry[0] & 0x1F == OPTICAL`, so
// 0x00 reads as DIRECT ACCESS and the drive silently disappears from
// the device list instead of reporting a failed probe.
if inq.bytes_transferred < INQUIRY_STANDARD_LEN {
return Err(crate::error::Error::DriveInquiryShort);
}
// Never decode past what the drive actually sent.
inquiry.truncate(inq.bytes_transferred.min(inquiry.len()));
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6. // GET CONFIGURATION Feature 010Ch — MMC-6 §6.6.
// Best-effort: 010Ch (Firmware Information) is an optional feature. // Best-effort: 010Ch (Firmware Information) is an optional feature.
@@ -279,54 +262,6 @@ mod tests {
/// Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer /// Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer
/// (e.g. a device that reports fewer than 8 bytes) must not panic. /// (e.g. a device that reports fewer than 8 bytes) must not panic.
/// Mutation: removing the `data.len() > start` guard makes it panic on short inputs. /// Mutation: removing the `data.len() > start` guard makes it panic on short inputs.
/// A drive that answers INQUIRY with GOOD status but a short or empty
/// data phase must fail the probe, not present as a blank drive.
///
/// The buffer is pre-zeroed, so decoding it unconditionally yielded empty
/// vendor/product/revision strings and a byte 0 of 0x00. Every platform
/// enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
/// and 0x00 is DIRECT ACCESS — so the drive silently vanished from the
/// device list rather than reporting that its identity probe failed. A
/// USB-SATA bridge mid-wedge does exactly this.
///
/// The two GET CONFIGURATION calls in the same function already clamped on
/// `bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
/// lines above them, discarded it.
#[test]
fn inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive() {
/// GOOD status, no sense, and only `n` bytes written.
struct ShortInquiry(usize);
impl ScsiTransport for ShortInquiry {
fn execute(
&mut self,
_cdb: &[u8],
_dir: DataDirection,
_buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Ok(ScsiResult {
status: 0,
sense: [0u8; 32],
bytes_transferred: self.0,
})
}
}
// Empty data phase — the case that made a real drive disappear.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(0)),
Err(crate::error::Error::DriveInquiryShort)
));
// One byte short of the SPC-4 standard 36-byte header.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(35)),
Err(crate::error::Error::DriveInquiryShort)
));
// Exactly the standard length is acceptable: the optional
// vendor-specific tail past byte 36 is allowed to be absent.
assert!(DriveId::from_drive(&mut ShortInquiry(36)).is_ok());
}
#[test] #[test]
fn ascii_field_short_buffer_returns_empty() { fn ascii_field_short_buffer_returns_empty() {
// Buffer of length 5: start=8 is beyond the end → empty string. // Buffer of length 5: start=8 is beyond the end → empty string.
@@ -404,136 +339,9 @@ mod tests {
); );
} }
/// `ascii_field`'s guard is `data.len() > start` (strictly greater), not
/// `>=`: a buffer whose length is exactly `start` has NO byte at that
/// offset, so it must still yield empty, not attempt to slice.
/// Mutation: `>` -> `>=` would try to slice `data[start..]` when
/// `data.len() == start`, which panics (empty range at the very end is
/// fine, but the guard's job is the `< start` case below it — pinning the
/// exact boundary catches an off-by-one either direction).
#[test]
fn ascii_field_boundary_len_equals_start_is_empty() {
let buf = vec![0u8; 8];
assert_eq!(ascii_field(&buf, 8, 16), "");
}
/// One byte past the boundary: `data.len() == start + 1` must extract
/// that single byte (clamped to `end`), proving the guard is `>` and not
/// off by one in the other direction.
#[test]
fn ascii_field_boundary_len_one_past_start_extracts_one_byte() {
let mut buf = vec![0u8; 9];
buf[8] = b'X';
assert_eq!(ascii_field(&buf, 8, 16), "X");
}
/// `Display` renders the four trimmed identity fields space-separated —
/// the human-readable counterpart of `match_key`'s pipe-separated form.
/// Not exercised anywhere else in this test module.
/// Mutation: replacing the `fmt` body with `Ok(Default::default())`
/// writes nothing at all, so formatting any `DriveId` yields "".
#[test]
fn display_formats_trimmed_fields_space_separated() {
let mut inquiry = vec![0u8; 96];
inquiry[8..16].copy_from_slice(b"PIONEER ");
inquiry[16..32].copy_from_slice(b"BD-RW BDR-S09 ");
inquiry[32..36].copy_from_slice(b"1.34");
inquiry[36..43].copy_from_slice(b" 16/04/");
let id = DriveId::from_inquiry(&inquiry, "201604250000");
assert_eq!(id.to_string(), "PIONEER BD-RW BDR-S09 1.34 16/04/");
}
/// GET CONFIGURATION failure (transport error) must not abort the /// GET CONFIGURATION failure (transport error) must not abort the
/// identity probe — firmware_date is empty, raw_gc_010c is empty. /// identity probe — firmware_date is empty, raw_gc_010c is empty.
/// Mutation: propagating the GET_CONFIGURATION error with `?` aborts from_drive. /// Mutation: propagating the GET_CONFIGURATION error with `?` aborts from_drive.
/// Transport whose GET CONFIGURATION responses report an exact,
/// caller-chosen `bytes_transferred` for each of the two GC features
/// (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
/// boundary guards can be pinned precisely. INQUIRY always succeeds.
struct FixedGcCountTransport {
firmware_bytes: usize,
serial_bytes: usize,
}
impl ScsiTransport for FixedGcCountTransport {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
for b in buf.iter_mut() {
*b = b'Z';
}
let bytes_transferred = match cdb.first() {
Some(&0x12) => buf.len(),
Some(&0x46) if cdb[3] == 0x0C => self.firmware_bytes,
Some(&0x46) if cdb[3] == 0x08 => self.serial_bytes,
_ => buf.len(),
};
Ok(ScsiResult {
status: 0,
bytes_transferred,
sense: [0u8; 32],
})
}
}
/// `end > 12` in the firmware-date branch (`from_drive`) is a strict
/// inequality: `bytes_transferred == 12` reports the field absent
/// (offset 12 is the first byte of the 12-char date; a count of exactly
/// 12 covers bytes 0..12, none of which is the date), so `firmware_date`
/// must be empty, not the mutant's off-by-one read.
/// Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an
/// empty but non-panicking slice — silently reporting "present" data
/// that is actually all outside the transferred count.
#[test]
fn from_drive_firmware_date_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 12,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "");
}
/// One byte past the boundary (`bytes_transferred == 13`) must extract
/// exactly the one available date byte (offset 12), proving the guard
/// is `>` and the slice end is clamped to `end`, not always to 24.
#[test]
fn from_drive_firmware_date_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 13,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "Z");
}
/// Same `> 12` boundary for the serial-number branch: exactly 12
/// transferred bytes must yield an empty serial.
#[test]
fn from_drive_serial_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 12,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "");
}
/// One byte past the serial boundary extracts exactly that byte.
#[test]
fn from_drive_serial_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 13,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "Z");
}
#[test] #[test]
fn from_drive_gc_failure_yields_empty_firmware_date() { fn from_drive_gc_failure_yields_empty_firmware_date() {
struct GcFailTransport; struct GcFailTransport;
+109 -1170
View File
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -133,10 +133,10 @@ where
match rx.recv_timeout(slice) { match rx.recv_timeout(slice) {
Ok(v) => return Ok(v), Ok(v) => return Ok(v),
Err(RecvTimeoutError::Timeout) => { Err(RecvTimeoutError::Timeout) => {
if let Some(h) = halt if let Some(h) = halt {
&& h.is_cancelled() if h.is_cancelled() {
{ return Err(BoundedError::Halted);
return Err(BoundedError::Halted); }
} }
if Instant::now() >= deadline { if Instant::now() >= deadline {
return Err(BoundedError::Timeout); return Err(BoundedError::Timeout);
-21
View File
@@ -249,27 +249,6 @@ impl Drop for BytePrefetcher {
mod tests { mod tests {
use super::*; use super::*;
/// `RECYCLE_DEPTH` must be one MORE than `FORWARD_DEPTH` per its own
/// doc comment: the producer needs at least one buffer to fill while
/// the consumer holds the other `FORWARD_DEPTH`-worth in flight. A
/// `+` -> `*`/`-` mutation on `FORWARD_DEPTH + 1` would under-size the
/// recycle channel (e.g. `FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one
/// short), which starves the producer of a spare buffer.
#[test]
fn recycle_depth_is_forward_depth_plus_one() {
assert_eq!(RECYCLE_DEPTH, FORWARD_DEPTH + 1);
assert_eq!(RECYCLE_DEPTH, 3, "FORWARD_DEPTH is 2, so recycle must be 3");
}
/// `DEFAULT_CHUNK_BYTES` is documented as 16 MiB. Pins the literal so a
/// `*` -> `+`/`/` mutation on either factor (16 * 1024 * 1024) is
/// caught by a concrete, spec-derived expected value rather than by
/// recomputing the same expression.
#[test]
fn default_chunk_bytes_is_16_mib() {
assert_eq!(DEFAULT_CHUNK_BYTES, 16_777_216, "documented as 16 MiB");
}
/// Endless reader: every `read` fills the whole buffer and never /// Endless reader: every `read` fills the whole buffer and never
/// hits EOF, so the producer keeps trying to push batches forward /// hits EOF, so the producer keeps trying to push batches forward
/// until the forward channel disconnects. Exactly the shape that /// until the forward channel disconnects. Exactly the shape that
+3 -3
View File
@@ -23,7 +23,7 @@
use std::fs::File; use std::fs::File;
use std::os::unix::io::AsRawFd; use std::os::unix::io::AsRawFd;
pub(crate) fn hint_sequential(file: &File, _len_bytes: u64) { pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
// Best-effort: return value ignored. A fadvise failure has no // Best-effort: return value ignored. A fadvise failure has no
// user-observable consequence. // user-observable consequence.
unsafe { unsafe {
@@ -34,7 +34,7 @@ pub(crate) fn hint_sequential(file: &File, _len_bytes: u64) {
/// Drop pages in the half-open byte range `[start, start+len)` from /// Drop pages in the half-open byte range `[start, start+len)` from
/// the page cache. Called periodically by `read_sectors` to bound the /// the page cache. Called periodically by `read_sectors` to bound the
/// read-side page cache pressure. /// read-side page cache pressure.
pub(crate) fn drop_window(file: &File, start: u64, len: u64) { pub(super) fn drop_window(file: &File, start: u64, len: u64) {
unsafe { unsafe {
libc::posix_fadvise( libc::posix_fadvise(
file.as_raw_fd(), file.as_raw_fd(),
@@ -58,7 +58,7 @@ pub(crate) fn drop_window(file: &File, start: u64, len: u64) {
/// can only pre-stage a tiny slice of the next batch. An explicit /// can only pre-stage a tiny slice of the next batch. An explicit
/// `readahead()` of the same size as the current batch tells the /// `readahead()` of the same size as the current batch tells the
/// kernel to queue the full next-batch read now. /// kernel to queue the full next-batch read now.
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) { pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
unsafe { unsafe {
libc::readahead(file.as_raw_fd(), offset as i64, len as usize); libc::readahead(file.as_raw_fd(), offset as i64, len as usize);
} }
+3 -3
View File
@@ -15,7 +15,7 @@ use std::os::unix::io::AsRawFd;
/// pipeline depth. /// pipeline depth.
const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024; const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024;
pub(crate) fn hint_sequential(file: &File, len_bytes: u64) { pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES); let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory { let mut ra = libc::radvisory {
ra_offset: 0, ra_offset: 0,
@@ -33,14 +33,14 @@ pub(crate) fn hint_sequential(file: &File, len_bytes: u64) {
/// approximation: no-op. macOS's unified buffer cache is generally /// approximation: no-op. macOS's unified buffer cache is generally
/// less prone to the pin-everything pathology that triggers the /// less prone to the pin-everything pathology that triggers the
/// regression on Linux NFS clients. /// regression on Linux NFS clients.
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {} pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Async-prefetch the byte range `[offset, offset+len)`. macOS uses /// Async-prefetch the byte range `[offset, offset+len)`. macOS uses
/// the same `fcntl(F_RDADVISE, &radvisory)` primitive as the open- /// the same `fcntl(F_RDADVISE, &radvisory)` primitive as the open-
/// time sequential hint, just targeted at a moving window instead of /// time sequential hint, just targeted at a moving window instead of
/// the whole file. The kernel queues I/O for the requested range and /// the whole file. The kernel queues I/O for the requested range and
/// returns immediately. /// returns immediately.
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) { pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
let bytes = (len as i64).min(RDADVISE_MAX_BYTES); let bytes = (len as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory { let mut ra = libc::radvisory {
ra_offset: offset as libc::off_t, ra_offset: offset as libc::off_t,
+21 -131
View File
@@ -48,25 +48,22 @@
//! far smaller than our 16 MiB app-level batch. //! far smaller than our 16 MiB app-level batch.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub(crate) mod linux; mod linux;
#[cfg(target_os = "macos")] #[cfg(target_os = "macos")]
pub(crate) mod macos; mod macos;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
pub(crate) mod other; mod other;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub(crate) mod windows; mod windows;
// The page-cache hints are shared with any other file-backed sector source:
// `dirimage` reads host files the same way and needs the same eviction, or a
// large rip pins every byte it has read (see this module's DONTNEED note).
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub(crate) use linux as platform; use linux as platform;
#[cfg(target_os = "macos")] #[cfg(target_os = "macos")]
pub(crate) use macos as platform; use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
pub(crate) use other as platform; use other as platform;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub(crate) use windows as platform; use windows as platform;
use std::fs::File; use std::fs::File;
use std::io::{Read, Seek, SeekFrom}; use std::io::{Read, Seek, SeekFrom};
@@ -88,27 +85,11 @@ use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// pressure concurrent writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`. /// pressure concurrent writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
const READ_DROP_CHUNK_BYTES_DEFAULT: u64 = 32 * 1024 * 1024; const READ_DROP_CHUNK_BYTES_DEFAULT: u64 = 32 * 1024 * 1024;
/// Upper bound (in MiB) accepted from `FREEMKV_READ_DROP_CHUNK_MIB`. 64 GiB —
/// generous for any real medium, and small enough that `n * 1024 * 1024` cannot
/// wrap `u64`. Mirrors `WRITEBACK_CHUNK_MIB_MAX`, whose identical multiply is
/// bounded for exactly this reason: without the bound, a value above 2^44
/// overflows — a panic on the first ISO open in an overflow-checked build, and in
/// release a wrap to a near-zero window that fires `drop_window` on every read.
/// Out-of-range values fall back to the default.
const READ_DROP_CHUNK_MIB_MAX: u64 = 64 * 1024;
fn read_drop_chunk_bytes() -> u64 { fn read_drop_chunk_bytes() -> u64 {
resolve_read_drop_chunk( std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB") .ok()
.ok() .and_then(|v| v.parse::<u64>().ok())
.and_then(|v| v.parse::<u64>().ok()), .filter(|&n| n > 0)
)
}
/// The pure part of [`read_drop_chunk_bytes`], split out so the bound is
/// testable without mutating process environment.
fn resolve_read_drop_chunk(mib: Option<u64>) -> u64 {
mib.filter(|&n| n > 0 && n <= READ_DROP_CHUNK_MIB_MAX)
.map(|n| n * 1024 * 1024) .map(|n| n * 1024 * 1024)
.unwrap_or(READ_DROP_CHUNK_BYTES_DEFAULT) .unwrap_or(READ_DROP_CHUNK_BYTES_DEFAULT)
} }
@@ -190,20 +171,12 @@ impl SectorSource for FileSectorSource {
) -> Result<usize> { ) -> Result<usize> {
let count = count as u32; let count = count as u32;
let bytes = count as usize * SECTOR_BYTES; let bytes = count as usize * SECTOR_BYTES;
// A real check, not a debug_assert: this is a public `SectorSource` impl, debug_assert!(
// so an undersized `out` is caller input, and `out[..bytes]` below would out.len() >= bytes,
// panic with 'range end index out of range' in release where the assert is "FileSectorSource::read_sectors: out len {} < requested {}",
// compiled away. `Drive::read_fua` already carries exactly this guard, with out.len(),
// a comment recording the same panic being fixed there — this impl was bytes
// simply never given it, and `PrefetchedSectorSource` has a regression test );
// for the case that this one lacked.
if out.len() < bytes {
return Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
});
}
if count == 0 { if count == 0 {
return Ok(0); return Ok(0);
} }
@@ -246,40 +219,6 @@ mod tests {
use std::io::Write; use std::io::Write;
use tempfile::tempdir; use tempfile::tempdir;
/// An undersized output buffer must return an error, never panic. This is a
/// public `SectorSource` impl, so buffer length is caller input, and the guard
/// used to be a `debug_assert!` — compiled out in release, where the
/// `out[..bytes]` slice then panicked with 'range end index out of range'.
///
/// `Drive::read_fua` already carries this exact guard with a comment recording
/// the same panic being fixed there, and `PrefetchedSectorSource` has
/// `direct_read_too_small_buffer_errors` for the same case; this impl had
/// neither.
#[test]
fn read_sectors_with_an_undersized_buffer_errors_rather_than_panicking() {
let dir = tempdir().unwrap();
let iso = dir.path().join("t.iso");
make_iso(&iso, 8);
let mut src = FileSectorSource::open(&iso).expect("iso opens");
// Ask for four sectors but supply room for barely more than one.
let mut out = vec![0u8; SECTOR_BYTES + 1];
let err = src
.read_sectors(0, 4, &mut out, false)
.expect_err("an undersized buffer must be an error, not a panic");
assert!(
matches!(err, Error::DiscRead { .. }),
"expected DiscRead, got {err:?}"
);
// Exactly-sized still works, so the guard is not off by one.
let mut out = vec![0u8; 4 * SECTOR_BYTES];
assert_eq!(
src.read_sectors(0, 4, &mut out, false).unwrap(),
4 * SECTOR_BYTES
);
}
/// Build a deterministic ISO of `sectors` sectors where sector `n` /// Build a deterministic ISO of `sectors` sectors where sector `n`
/// is filled with the byte pattern `((n & 0xff) as u8)`. Lets us /// is filled with the byte pattern `((n & 0xff) as u8)`. Lets us
/// verify any sector by content alone. /// verify any sector by content alone.
@@ -437,36 +376,19 @@ mod tests {
// Additional coverage. // Additional coverage.
// --------------------------------------------------------------- // ---------------------------------------------------------------
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or reading, /// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or
/// even at an out-of-range LBA. Grounding: `if count == 0 { return Ok(0) }`. /// reading, even at an out-of-range LBA — the early-return guard
/// /// runs before any I/O. Grounding: `if count == 0 { return Ok(0) }`.
/// The `Ok(0)` return alone proves nothing: with the guard deleted, a seek
/// past EOF succeeds (POSIX permits seeking beyond the end of a file) and a
/// zero-length `read_exact` returns `Ok(())` immediately, so the call still
/// returns `Ok(0)`. The observable difference is the file's cursor — the
/// seek MOVES it to `lba * 2048`. Assert on that, so the guard is what the
/// test is actually measuring.
#[test] #[test]
fn zero_count_returns_zero_no_io() { fn zero_count_returns_zero_no_io() {
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
let path = dir.path().join("zc.iso"); let path = dir.path().join("zc.iso");
make_iso(&path, 4); make_iso(&path, 4);
let mut src = FileSectorSource::open(&path).unwrap(); let mut src = FileSectorSource::open(&path).unwrap();
let before = src.file.stream_position().expect("cursor readable");
assert_eq!(before, 0, "a freshly opened file starts at offset 0");
// LBA far past EOF — must not matter because count==0 returns early. // LBA far past EOF — must not matter because count==0 returns early.
let mut buf = [0u8; 1]; let mut buf = [0u8; 1];
let n = src.read_sectors(1_000_000, 0, &mut buf, false).unwrap(); let n = src.read_sectors(1_000_000, 0, &mut buf, false).unwrap();
assert_eq!(n, 0); assert_eq!(n, 0);
assert_eq!(
src.file.stream_position().expect("cursor readable"),
before,
"count == 0 must return before the seek — an unmoved cursor is the \
only observable proof that no I/O was issued"
);
// And the drop-window accounting must not have advanced either.
assert_eq!(src.bytes_read_since_drop, 0);
assert_eq!(src.drop_window_start, 0);
} }
/// Reading past EOF must ERROR (read_exact's UnexpectedEof), never /// Reading past EOF must ERROR (read_exact's UnexpectedEof), never
@@ -596,36 +518,4 @@ mod tests {
lba += batch as u32; lba += batch as u32;
} }
} }
/// `FREEMKV_READ_DROP_CHUNK_MIB` must be BOUNDED before the MiB→byte
/// multiply, exactly as its writeback twin bounds the identical multiply.
/// Unbounded, any value above 2^44 overflowed `n * 1024 * 1024`: a panic on
/// the first ISO open in an overflow-checked build, and in release a wrap to
/// a near-zero window that fires `drop_window` on essentially every read.
#[test]
fn read_drop_chunk_env_is_bounded_before_the_multiply() {
// Default when unset / zero / out of range.
assert_eq!(resolve_read_drop_chunk(None), READ_DROP_CHUNK_BYTES_DEFAULT);
assert_eq!(
resolve_read_drop_chunk(Some(0)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// The overflow value: `u64::MAX * 1024 * 1024` panicked here.
assert_eq!(
resolve_read_drop_chunk(Some(u64::MAX)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX + 1)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// In-range values convert MiB→bytes. Mutation: `* 1024` breaks this.
assert_eq!(resolve_read_drop_chunk(Some(1)), 1024 * 1024);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX)),
READ_DROP_CHUNK_MIB_MAX * 1024 * 1024
);
// And the bound itself keeps the multiply inside u64.
assert!((READ_DROP_CHUNK_MIB_MAX as u128) * 1024 * 1024 <= u64::MAX as u128);
}
} }
+3 -3
View File
@@ -4,8 +4,8 @@
use std::fs::File; use std::fs::File;
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {} pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {} pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {} pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+3 -3
View File
@@ -9,7 +9,7 @@ use std::fs::File;
/// No-op stub. `FILE_FLAG_SEQUENTIAL_SCAN` can only be set at /// No-op stub. `FILE_FLAG_SEQUENTIAL_SCAN` can only be set at
/// `CreateFile` open time, which the plain `File::open` path does not /// `CreateFile` open time, which the plain `File::open` path does not
/// do, so there is no post-open hint to issue here. /// do, so there is no post-open hint to issue here.
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) { pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
tracing::debug!( tracing::debug!(
target: "mux", target: "mux",
"FileSectorSource hint_sequential: windows no-op stub" "FileSectorSource hint_sequential: windows no-op stub"
@@ -19,10 +19,10 @@ pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {
/// Windows page-cache eviction is not exposed via a posix_fadvise /// Windows page-cache eviction is not exposed via a posix_fadvise
/// equivalent. The kernel does its own working-set management. No-op /// equivalent. The kernel does its own working-set management. No-op
/// for now. /// for now.
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {} pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Windows async-prefetch hint. With FILE_FLAG_SEQUENTIAL_SCAN at /// Windows async-prefetch hint. With FILE_FLAG_SEQUENTIAL_SCAN at
/// open the kernel already prefetches aggressively, so there's no /// open the kernel already prefetches aggressively, so there's no
/// per-range hint we'd add on top. No-op stub for parity with the /// per-range hint we'd add on top. No-op stub for parity with the
/// posix platforms. /// posix platforms.
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {} pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
-283
View File
@@ -1,283 +0,0 @@
//! `write_image` — write an image-level source out as a sector image.
//!
//! This is the plain image writer: sectors in from any [`SectorSource`], bytes
//! out to a file, in order, once. It is what an `iso://` DESTINATION means when
//! the source is not a physical drive.
//!
//! # Why this is not `freemkv_engine::copy`
//!
//! The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
//! sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
//! exists because an optical drive returns read errors on marginal media. A
//! file-backed or synthesized source has no marginal media: a read either
//! succeeds or the underlying file is broken, and retrying it is pointless.
//!
//! Routing a non-drive source through the recovery path is not merely wasteful,
//! it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
//! both of which are empty for an already-decrypted source, so identity passes
//! for ANY such source: a second run with a different input to the same output
//! path would resume over the previous image and produce wrong content at exit
//! zero. Keeping the two paths separate makes that unrepresentable.
//!
//! So: drive sources get `freemkv_engine::copy`. Everything else gets this.
use crate::consts::SECTOR_BYTES;
use crate::error::{Error, Result};
use crate::halt::Halt;
use crate::sector::SectorSource;
use std::fs::File;
use std::io::{BufWriter, Write};
use std::path::Path;
/// Sectors per read/write batch. 4 MiB — large enough that per-call overhead
/// disappears against a file-backed source, small enough that the buffer is not
/// a notable allocation and cancellation stays responsive.
const BATCH_SECTORS: u32 = 2048;
/// Write `total_sectors` sectors from `reader` to `dest`.
///
/// Reads sequentially from LBA 0 and writes in order, so the output is a faithful
/// image of whatever the source presents — decrypted if the caller wrapped the
/// source in a [`DecryptingSectorSource`](crate::sector::decrypting::DecryptingSectorSource),
/// ciphertext if it did not. This function performs no decryption itself and makes
/// no decryption decision; that belongs to the caller, which knows whether the run
/// is `--raw`.
///
/// `on_progress` is called after each batch with the cumulative byte count, for
/// front-end progress reporting. It must not block.
///
/// `halt` is checked once per batch. On cancellation the partial file is left in
/// place — the caller decides whether a partial image is worth keeping, and
/// deleting a multi-gigabyte file the user may want to inspect is not this
/// function's call to make.
///
/// Returns the number of bytes written.
///
/// # Errors
///
/// - [`Error::Halted`] if `halt` was cancelled.
/// - [`Error::IoError`] if the destination cannot be created or written.
/// - Whatever the source's `read_sectors` returns. A short read is an error, not
/// a zero-fill: silently padding a truncated source produces an image that
/// looks complete and is not.
pub fn write_image(
reader: &mut dyn SectorSource,
dest: &Path,
total_sectors: u32,
halt: &Halt,
mut on_progress: impl FnMut(u64),
) -> Result<u64> {
if total_sectors == 0 {
return Err(Error::EmptyImage);
}
let file = File::create(dest).map_err(|source| Error::IoError { source })?;
let mut out = BufWriter::with_capacity(BATCH_SECTORS as usize * SECTOR_BYTES, file);
let mut buf = vec![0u8; BATCH_SECTORS as usize * SECTOR_BYTES];
let mut written: u64 = 0;
let mut lba: u32 = 0;
while lba < total_sectors {
if halt.is_cancelled() {
return Err(Error::Halted);
}
let count = BATCH_SECTORS.min(total_sectors - lba);
let want = count as usize * SECTOR_BYTES;
// `recovery = false`: a file-backed source ignores the flag, and a
// retry loop over a local file would only re-read the same bytes.
let got = reader.read_sectors(lba, count as u16, &mut buf[..want], false)?;
if got != want {
return Err(Error::ShortImageRead {
lba,
expected: want as u32,
got: got as u32,
});
}
out.write_all(&buf[..want])
.map_err(|source| Error::IoError { source })?;
written += want as u64;
lba += count;
on_progress(written);
}
// flush() only pushes the BufWriter's bytes into the kernel via write(2).
// It makes no durability promise at all, so returning Ok here would report
// a finished image while up to several gigabytes of it still sit in the
// page cache. A crash, a power loss, or yanking the removable/network
// volume the image was written to then leaves a truncated or empty file
// that the caller was told was complete.
//
// For a 6-90 GB image that is exactly the failure this crate treats as
// worst: success reported over wrong output. `into_inner` is used rather
// than `flush` so a buffered-write error is surfaced instead of being
// dropped on the floor by BufWriter's Drop.
let file = out.into_inner().map_err(|e| Error::IoError {
source: e.into_error(),
})?;
file.sync_all()
.map_err(|source| Error::IoError { source })?;
Ok(written)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::error::Result as FmResult;
/// A source that yields a deterministic byte per sector, so the written
/// image can be checked positionally rather than just by length.
struct PatternSource {
sectors: u32,
/// Sectors after which `read_sectors` reports a short read.
short_after: Option<u32>,
}
impl SectorSource for PatternSource {
fn capacity_sectors(&self) -> u32 {
self.sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> FmResult<usize> {
let want = count as usize * SECTOR_BYTES;
if self.short_after.is_some_and(|after| lba >= after) {
return Ok(want - 1);
}
for s in 0..count as usize {
let byte = ((lba as usize + s) % 251) as u8;
buf[s * SECTOR_BYTES..(s + 1) * SECTOR_BYTES].fill(byte);
}
Ok(want)
}
}
fn tmp(name: &str) -> std::path::PathBuf {
let mut p = std::env::temp_dir();
p.push(format!("fmkv-image-writer-{name}-{}", std::process::id()));
p
}
/// The written image is byte-for-byte what the source presented, at the
/// right offsets — not merely the right length.
#[test]
fn writes_every_sector_in_order() {
let dest = tmp("order");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 5000 * SECTOR_BYTES as u64);
let data = std::fs::read(&dest).expect("read back");
assert_eq!(data.len(), 5000 * SECTOR_BYTES);
// Spot-check across batch boundaries (BATCH_SECTORS = 2048): the last
// sector of batch 0, the first of batch 1, and the final sector.
for lba in [0usize, 2047, 2048, 4095, 4096, 4999] {
let want = (lba % 251) as u8;
assert_eq!(
data[lba * SECTOR_BYTES],
want,
"sector {lba} head byte wrong — batching lost or duplicated a sector"
);
assert_eq!(
data[(lba + 1) * SECTOR_BYTES - 1],
want,
"sector {lba} tail"
);
}
let _ = std::fs::remove_file(&dest);
}
/// A tail shorter than a full batch must still be written whole — the
/// classic off-by-one when `total_sectors` is not a batch multiple.
#[test]
fn writes_a_partial_final_batch() {
let dest = tmp("tail");
let mut src = PatternSource {
sectors: 2049,
short_after: None,
};
let n = write_image(&mut src, &dest, 2049, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 2049 * SECTOR_BYTES as u64);
assert_eq!(
std::fs::metadata(&dest).expect("stat").len(),
2049 * SECTOR_BYTES as u64
);
let _ = std::fs::remove_file(&dest);
}
/// A short read is an error. Zero-filling would yield an image that looks
/// complete and is not — the single worst outcome for an archival copy.
#[test]
fn short_read_is_an_error_not_a_zero_fill() {
let dest = tmp("short");
let mut src = PatternSource {
sectors: 4096,
short_after: Some(2048),
};
let err = write_image(&mut src, &dest, 4096, &Halt::new(), |_| {}).expect_err("must fail");
assert!(
matches!(err, Error::ShortImageRead { lba: 2048, .. }),
"got {err:?}"
);
let _ = std::fs::remove_file(&dest);
}
/// Cancellation stops the run and reports it, rather than finishing quietly
/// or reporting success on a partial image.
#[test]
fn cancellation_halts_and_reports() {
let dest = tmp("halt");
let mut src = PatternSource {
sectors: 100_000,
short_after: None,
};
let halt = Halt::new();
halt.cancel();
let err = write_image(&mut src, &dest, 100_000, &halt, |_| {}).expect_err("must halt");
assert!(matches!(err, Error::Halted), "got {err:?}");
let _ = std::fs::remove_file(&dest);
}
/// Progress is cumulative and monotonic, and its final value equals the
/// returned byte count — a front-end that trusts the callback must not end
/// up disagreeing with the return value.
#[test]
fn progress_is_cumulative_and_ends_at_the_total() {
let dest = tmp("progress");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let mut seen: Vec<u64> = Vec::new();
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |b| seen.push(b)).expect("write");
assert!(
seen.windows(2).all(|w| w[1] > w[0]),
"not monotonic: {seen:?}"
);
assert_eq!(*seen.last().expect("at least one callback"), n);
let _ = std::fs::remove_file(&dest);
}
/// A zero-sector source is a caller error, not a zero-byte image: an empty
/// ISO is never what anyone wanted, and failing here names the problem.
#[test]
fn zero_sectors_is_an_error() {
let dest = tmp("empty");
let mut src = PatternSource {
sectors: 0,
short_after: None,
};
let err = write_image(&mut src, &dest, 0, &Halt::new(), |_| {}).expect_err("must fail");
assert!(matches!(err, Error::EmptyImage), "got {err:?}");
// The destination must not have been created — a failed run leaves no
// stub for a later run to mistake for output.
assert!(!dest.exists(), "empty run created a file");
}
}
-1
View File
@@ -31,7 +31,6 @@ pub(crate) mod bounded;
pub mod byte_prefetcher; pub mod byte_prefetcher;
pub mod file_sector_source; pub mod file_sector_source;
pub mod fsync; pub mod fsync;
pub mod image_writer;
pub mod sink; pub mod sink;
mod writeback; mod writeback;
mod writeback_file; mod writeback_file;
+59 -351
View File
@@ -33,7 +33,7 @@
//! consumer lag detection). This is critical for diagnosing stalls. //! consumer lag detection). This is critical for diagnosing stalls.
use std::sync::Arc; use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU8, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::thread::{self, JoinHandle}; use std::thread::{self, JoinHandle};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -117,32 +117,8 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
Error::PipelineConsumerPanicked Error::PipelineConsumerPanicked
} }
/// Consumer lifecycle state, shared between the caller and the consumer thread.
///
/// A plain `AtomicBool` could not make "the caller abandons" and "the consumer
/// commits to finalising" mutually exclusive: the consumer loaded the flag, the
/// caller stored it, and the consumer then finalised the container anyway — the
/// caller reporting the rip as interrupted while a fully finalised MKV (Cues
/// written, Segment size patched) landed on disk, indistinguishable from a
/// complete one. The two transitions are therefore a single compare-exchange each,
/// out of [`state::RUNNING`]: whoever wins decides, and the loser observes the
/// winner. (`ST_RUNNING` does not exist anywhere in the crate — the constants are
/// `state::RUNNING` / `state::ABANDONED` / `state::CLOSING` below, and both
/// compare-exchange sites that must stay in step with this argument name them.)
mod state {
/// Consumer is running; neither side has committed yet.
pub const RUNNING: u8 = 0;
/// The caller gave up on the consumer and will report failure — the consumer
/// must NOT finalise the output.
pub const ABANDONED: u8 = 1;
/// The consumer has committed to `close()` (finalising the output). The caller
/// can no longer abandon it; it must wait for the result it is about to
/// produce.
pub const CLOSING: u8 = 2;
}
/// After a halt or deadline fires, spin-poll `handle.is_finished()` for /// After a halt or deadline fires, spin-poll `handle.is_finished()` for
/// `grace` before accepting the thread leak. This converts /// [`FINISH_GRACE_SECS`] before accepting the thread leak. This converts
/// the common "nearly-done" consumer (whose own bounded_syscall just /// the common "nearly-done" consumer (whose own bounded_syscall just
/// returned and is about to drop its output file) into a clean join, /// returned and is about to drop its output file) into a clean join,
/// releasing the file handle without waiting the full grace period. /// releasing the file handle without waiting the full grace period.
@@ -156,12 +132,11 @@ mod state {
/// syscall itself; that still returns on its own (or at process exit). /// syscall itself; that still returns on its own (or at process exit).
fn finish_with_grace<R: Send + 'static>( fn finish_with_grace<R: Send + 'static>(
handle: thread::JoinHandle<Result<R, Error>>, handle: thread::JoinHandle<Result<R, Error>>,
state: &Arc<AtomicU8>, abandoned: &Arc<AtomicBool>,
grace: Duration,
leak_err: Error, leak_err: Error,
) -> Result<R, Error> { ) -> Result<R, Error> {
let deadline = Instant::now() + grace; let grace = Instant::now() + Duration::from_secs(FINISH_GRACE_SECS);
while Instant::now() < deadline { while Instant::now() < grace {
if handle.is_finished() { if handle.is_finished() {
return match handle.join() { return match handle.join() {
Ok(result) => result, Ok(result) => result,
@@ -170,50 +145,16 @@ fn finish_with_grace<R: Send + 'static>(
} }
thread::sleep(POLL_INTERVAL); thread::sleep(POLL_INTERVAL);
} }
// Grace expired. CLAIM abandonment, then log and leak. Claiming BEFORE // Grace expired. Signal abandonment, then log and leak. Setting the
// dropping the handle guarantees the leaked consumer observes it the moment // flag BEFORE dropping the handle guarantees the leaked consumer
// its wedged syscall returns: it then skips any further `apply` and skips // observes it the moment its wedged syscall returns: it then skips
// `close()`, rather than running on to finalise the abandoned output file. // any further `apply` and skips `close()`, rather than running on to
// // finalise the abandoned output file.
// A compare-exchange, not a store, because the consumer may have committed to // `Release` here pairs with the `Acquire` loads in the consumer loop so
// `close()` in the instant between our last `is_finished()` poll and now. It // the leaked consumer reliably observes the flag the moment its wedged
// then cannot be stopped — the finalise IS happening — so abandoning it would // syscall returns, even on weak memory models (ARM64/POWER) where
// report the rip as interrupted while a valid, fully finalised container // `Relaxed` gives no cross-thread visibility guarantee.
// lands on disk. Losing the race means waiting for the result the consumer is abandoned.store(true, Ordering::Release);
// already producing instead. `AcqRel` pairs with the consumer's own
// compare-exchange and with the `Acquire` loads in its drain loop, so the flag
// is reliably observed even on weak memory models (ARM64/POWER).
if state
.compare_exchange(
state::RUNNING,
state::ABANDONED,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_close_in_flight",
"pipeline consumer had already committed to finalising the output; \
waiting for it rather than reporting an unfinalised output"
);
let close_deadline = Instant::now() + grace;
while Instant::now() < close_deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
Err(payload) => Err(consumer_panicked(payload)),
};
}
thread::sleep(POLL_INTERVAL);
}
// Still finalising after a second grace window: leak and report the wedge.
// The output may end up finalised by the leaked thread — but that is now a
// wedged-`close()` case, not the check-then-finalise race.
drop(handle);
return Err(leak_err);
}
tracing::warn!( tracing::warn!(
target: "freemkv::pipeline", target: "freemkv::pipeline",
phase = "finish_with_halt_grace_expired", phase = "finish_with_halt_grace_expired",
@@ -243,8 +184,7 @@ pub const WRITE_PIPELINE_DEPTH: usize = 16;
/// Channel depth for write-through pipelines. Each `send` fully /// Channel depth for write-through pipelines. Each `send` fully
/// drains before the next can enqueue. Use this when the producer /// drains before the next can enqueue. Use this when the producer
/// must observe consumer side-effects (e.g. mapfile state) before /// must observe consumer side-effects (e.g. mapfile state) before
/// emitting the next item. Used by `freemkv_engine::recovery::patch` — the /// emitting the next item. Currently used by `disc::patch`.
/// recovery strategy moved to that crate in 1.6.0, so there is no `patch` here.
pub const WRITE_THROUGH_DEPTH: usize = 1; pub const WRITE_THROUGH_DEPTH: usize = 1;
/// Outcome of [`Sink::apply`]: either keep feeding items /// Outcome of [`Sink::apply`]: either keep feeding items
@@ -302,21 +242,7 @@ pub struct Pipeline<I: Send + 'static, R: Send + 'static> {
/// a syscall the consumer is currently wedged in, but it does bound /// a syscall the consumer is currently wedged in, but it does bound
/// the damage to "whatever write is already in flight" once that /// the damage to "whatever write is already in flight" once that
/// syscall returns, instead of running on to a clean finalise. /// syscall returns, instead of running on to a clean finalise.
/// abandoned: Arc<AtomicBool>,
/// One of [`state::RUNNING`] / [`state::ABANDONED`] / [`state::CLOSING`];
/// both transitions are compare-exchanges so abandoning and finalising are
/// mutually exclusive rather than racing.
state: Arc<AtomicU8>,
/// Set by the consumer the moment an `apply` returns `Err`. The consumer keeps
/// draining the channel after that (so the producer never blocks on a dead
/// receiver) — which means a producer watching only `send`'s return value
/// cannot tell the difference between "being consumed" and "being discarded
/// after a fatal write error", and would go on reading the whole remaining
/// disc before `finish()` finally surfaced the error. This flag is that
/// missing edge: [`Pipeline::send_with_halt`] fails fast on it, and
/// [`Pipeline::consumer_failed`] exposes it to producers that use plain
/// [`Pipeline::send`].
failed: Arc<AtomicBool>,
} }
impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> { impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
@@ -325,21 +251,17 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// ///
/// The thread is named `freemkv-pipeline-consumer` so it shows up /// The thread is named `freemkv-pipeline-consumer` so it shows up
/// distinctly in stack traces and `top -H`. Callers that want a /// distinctly in stack traces and `top -H`. Callers that want a
/// more specific name should use [`Pipeline::spawn_named`] instead. /// more specific name (e.g. `freemkv-sweep-consumer`) should use
/// Returns an `Error::IoError` if the OS refuses the thread spawn /// [`Pipeline::spawn_named`] instead. Returns an `Error::IoError`
/// (resource exhaustion); callers already operate in fallible context, so /// if the OS refuses the thread spawn (resource exhaustion);
/// this is propagated rather than panicked. /// callers already operate in fallible context, so this is
/// propagated rather than panicked.
/// ///
/// Inside this crate the only [`Pipeline::spawn_named`] caller is the mux /// Sweep uses [`Pipeline::spawn_named`] directly so the consumer
/// driver, which names its thread `freemkv-mux-consumer`. `Pipeline::spawn` /// thread shows up as `freemkv-sweep-consumer`; mux uses
/// (this function, with the default name) is used only by the unit tests in /// `freemkv-mux-consumer`. `Pipeline::spawn` (this function, with
/// this module. /// the default name) is used by `disc::patch` and by the unit
/// /// tests in this module.
/// This paragraph twice named a caller that had left the crate: first
/// `disc::patch`, then Sweep and its `freemkv-sweep-consumer` thread. Both
/// went to freemkv-engine with the recovery passes in 1.6.0, and each in
/// turn sent readers hunting a component that is not here. Name callers
/// that live in THIS crate, or none.
pub fn spawn<S: Sink<I, Output = R>>(depth: usize, sink: S) -> Result<Self, Error> { pub fn spawn<S: Sink<I, Output = R>>(depth: usize, sink: S) -> Result<Self, Error> {
Self::spawn_named("freemkv-pipeline-consumer", depth, sink) Self::spawn_named("freemkv-pipeline-consumer", depth, sink)
} }
@@ -347,17 +269,15 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Like [`Pipeline::spawn`] but lets the caller supply the /// Like [`Pipeline::spawn`] but lets the caller supply the
/// consumer thread's name. Useful when several pipelines run in /// consumer thread's name. Useful when several pipelines run in
/// the same process and stack traces / `top -H` need to tell them /// the same process and stack traces / `top -H` need to tell them
/// apart (e.g. `freemkv-mux-consumer`). /// apart (e.g. `freemkv-sweep-consumer`, `freemkv-mux-consumer`).
pub fn spawn_named<S: Sink<I, Output = R>>( pub fn spawn_named<S: Sink<I, Output = R>>(
name: &str, name: &str,
depth: usize, depth: usize,
sink: S, sink: S,
) -> Result<Self, Error> { ) -> Result<Self, Error> {
let (tx, rx) = bounded::<I>(depth); let (tx, rx) = bounded::<I>(depth);
let state = Arc::new(AtomicU8::new(state::RUNNING)); let abandoned = Arc::new(AtomicBool::new(false));
let state_consumer = state.clone(); let abandoned_consumer = abandoned.clone();
let failed = Arc::new(AtomicBool::new(false));
let failed_consumer = failed.clone();
let handle = thread::Builder::new() let handle = thread::Builder::new()
.name(name.into()) .name(name.into())
.spawn(move || -> Result<R, Error> { .spawn(move || -> Result<R, Error> {
@@ -388,7 +308,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// dead receiver, but we touch the output no further. The // dead receiver, but we touch the output no further. The
// final post-loop abandonment check returns the error // final post-loop abandonment check returns the error
// and skips `close()`. // and skips `close()`.
if state_consumer.load(Ordering::Acquire) == state::ABANDONED { if abandoned_consumer.load(Ordering::Acquire) {
continue; continue;
} }
@@ -417,12 +337,6 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e); tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
} }
first_err = Some(e); first_err = Some(e);
// Publish the failure so the producer can stop
// FEEDING a dead write side instead of only learning
// about it at `finish()` — by which time it has read
// the rest of the disc. `Release` pairs with the
// `Acquire` load in `send_with_halt`.
failed_consumer.store(true, Ordering::Release);
} }
} }
@@ -483,38 +397,13 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// MKV Cues + patching the segment header) on a file the // MKV Cues + patching the segment header) on a file the
// caller already reported as failed is exactly the // caller already reported as failed is exactly the
// write race we must not run. // write race we must not run.
if abandoned_consumer.load(Ordering::Acquire) {
return Err(Error::Halted);
}
match first_err { match first_err {
// No `close()` on this path, so there is nothing to claim — Some(e) => Err(e),
// just report, unless the caller has already given up on us. None => sink.close(),
Some(e) => {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
Err(Error::Halted)
} else {
Err(e)
}
}
// CLAIM the finalise. A plain load here left a window in which
// the caller stored `abandoned` AFTER we read it as clear, so
// `close()` ran anyway and finalised (Cues + Segment-size
// patch) an output the caller had already reported as
// interrupted — a truncated rip indistinguishable from a
// complete one. The compare-exchange closes that window: if the
// caller got there first we skip `close()`, and if we get there
// first the caller waits for us instead of abandoning.
None => {
if state_consumer
.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
return Err(Error::Halted);
}
sink.close()
}
} }
}) })
.map_err(|e| Error::IoError { source: e })?; .map_err(|e| Error::IoError { source: e })?;
@@ -522,24 +411,10 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Ok(Pipeline { Ok(Pipeline {
tx, tx,
handle, handle,
state, abandoned,
failed,
}) })
} }
/// Whether the consumer's `apply` has already failed fatally.
///
/// The consumer keeps draining the channel after an `apply` error (so the
/// producer never blocks on a dead receiver), which means `send` keeps
/// succeeding and a producer has no other way to tell that everything it feeds
/// is being discarded. A long-running producer — the mux frame pump reading a
/// 60 GB title off an optical drive — should check this and unwind instead of
/// reading the rest of the disc for a write that has already failed.
/// [`Pipeline::send_with_halt`] checks it automatically.
pub fn consumer_failed(&self) -> bool {
self.failed.load(Ordering::Acquire)
}
/// Push one item. Blocks if the channel is full — that's the /// Push one item. Blocks if the channel is full — that's the
/// back-pressure the whole primitive exists to provide. Returns /// back-pressure the whole primitive exists to provide. Returns
/// the item back if the consumer thread is gone (panicked or /// the item back if the consumer thread is gone (panicked or
@@ -632,34 +507,11 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// wedged inside an unkillable syscall, the producer can still /// wedged inside an unkillable syscall, the producer can still
/// observe `/api/stop` and unwind within /// observe `/api/stop` and unwind within
/// [`SEND_HALT_CHECK_INTERVAL`]. /// [`SEND_HALT_CHECK_INTERVAL`].
/// NOT a `foo_with_X` variant of [`Pipeline::send`], despite the name.
/// The two encode OPPOSITE policies on the same event, each with its own
/// test: after the consumer's `apply` has failed, `send` still succeeds
/// (the consumer keeps draining, so the channel accepts the item), while
/// this one hands the item straight back — so a producer does not read an
/// hour of disc for a write that died on the first frame. Collapsing them
/// into one Option-parameterised method deletes one of those behaviours;
/// it was tried and `apply_error_drains_then_propagates` caught it.
pub fn send_with_halt(&self, item: I, halt: &Halt, deadline: Duration) -> Result<(), I> { pub fn send_with_halt(&self, item: I, halt: &Halt, deadline: Duration) -> Result<(), I> {
use crossbeam_channel::SendTimeoutError; use crossbeam_channel::SendTimeoutError;
let end = Instant::now() + deadline; let end = Instant::now() + deadline;
let mut pending = item; let mut pending = item;
loop { loop {
// The consumer's `apply` has failed fatally: everything sent from here
// is drained and discarded, so hand the item back at once. Without this
// the producer saw every send succeed (the channel is always being
// drained) and went on reading the whole remaining title — an hour of
// drive time on a UHD — for a write that died on the first frame, only
// learning about it at `finish()`.
if self.consumer_failed() {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: consumer apply failed, returning item={}",
std::any::type_name::<I>()
);
}
return Err(pending);
}
// Pre-check the cheap exit conditions before parking. // Pre-check the cheap exit conditions before parking.
if halt.is_cancelled() { if halt.is_cancelled() {
if debug_enabled() { if debug_enabled() {
@@ -714,8 +566,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let Pipeline { let Pipeline {
tx, tx,
handle, handle,
state: _, abandoned: _,
failed: _,
} = self; } = self;
// Explicit drop, although the destructure already drops `tx` // Explicit drop, although the destructure already drops `tx`
// at end-of-scope. Being explicit keeps the intent obvious. // at end-of-scope. Being explicit keeps the intent obvious.
@@ -751,18 +602,11 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Plain [`Pipeline::finish`] is preserved for callers without a /// Plain [`Pipeline::finish`] is preserved for callers without a
/// halt-token plumbed through; that path still blocks indefinitely /// halt-token plumbed through; that path still blocks indefinitely
/// on `join()`, matching pre-0.20.8 behaviour. /// on `join()`, matching pre-0.20.8 behaviour.
/// Also not a `foo_with_X` variant: [`Pipeline::finish`] joins and waits
/// however long the consumer needs, while this one gives up after
/// `JOIN_TIMEOUT_SECS` and reports halted. Which is right depends on
/// whether the caller has a user waiting to cancel — the mux driver does
/// and uses this; the unit tests do not and use the plain join. Merging
/// them means picking one of those policies for both.
pub fn finish_with_halt(self, halt: Option<&Halt>) -> Result<R, Error> { pub fn finish_with_halt(self, halt: Option<&Halt>) -> Result<R, Error> {
let Pipeline { let Pipeline {
tx, tx,
handle, handle,
state, abandoned,
failed: _,
} = self; } = self;
drop(tx); drop(tx);
let deadline = Instant::now() + Duration::from_secs(JOIN_TIMEOUT_SECS); let deadline = Instant::now() + Duration::from_secs(JOIN_TIMEOUT_SECS);
@@ -773,23 +617,13 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Err(payload) => Err(consumer_panicked(payload)), Err(payload) => Err(consumer_panicked(payload)),
}; };
} }
if let Some(h) = halt if let Some(h) = halt {
&& h.is_cancelled() if h.is_cancelled() {
{ return finish_with_grace(handle, &abandoned, Error::Halted);
return finish_with_grace( }
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::Halted,
);
} }
if Instant::now() >= deadline { if Instant::now() >= deadline {
return finish_with_grace( return finish_with_grace(handle, &abandoned, Error::PipelineJoinTimeout);
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::PipelineJoinTimeout,
);
} }
thread::sleep(POLL_INTERVAL); thread::sleep(POLL_INTERVAL);
} }
@@ -1241,6 +1075,20 @@ mod tests {
/// A sink that records the exact order of items it receives, so we /// A sink that records the exact order of items it receives, so we
/// can prove the channel is FIFO (no reordering). `close` returns /// can prove the channel is FIFO (no reordering). `close` returns
/// the recorded vector. /// the recorded vector.
struct OrderSink {
seen: Vec<u64>,
}
impl Sink<u64> for OrderSink {
type Output = Vec<u64>;
fn apply(&mut self, item: u64) -> Result<Flow, Error> {
self.seen.push(item);
Ok(Flow::Continue)
}
fn close(self) -> Result<Vec<u64>, Error> {
Ok(self.seen)
}
}
/// Zero items sent: closing the pipeline immediately must still /// Zero items sent: closing the pipeline immediately must still
/// call `close()` exactly once and return its Output. The consumer /// call `close()` exactly once and return its Output. The consumer
/// loop's `while let Ok = rx.recv()` exits on the dropped tx with /// loop's `while let Ok = rx.recv()` exits on the dropped tx with
@@ -1745,144 +1593,4 @@ mod tests {
let res = pipe.finish_with_halt(None); let res = pipe.finish_with_halt(None);
assert!(matches!(res, Ok(190)), "expected Ok(190), got {res:?}"); assert!(matches!(res, Ok(190)), "expected Ok(190), got {res:?}");
} }
/// A fatal `apply` error must become visible to the PRODUCER, not only to
/// `finish()`. The consumer keeps draining after the error (so the producer
/// never blocks on a dead receiver), which meant every `send_with_halt`
/// returned `Ok` for the rest of the run: on a 60 GB mkv:// mux that hit
/// ENOSPC on the first frame, the mux driver read the entire remaining title —
/// an hour of optical-drive time — before learning the write had died.
#[test]
fn send_with_halt_fails_fast_once_apply_has_failed() {
struct FailFirst {
failed: Arc<AtomicUsize>,
}
impl Sink<u64> for FailFirst {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
self.failed.fetch_add(1, Ordering::SeqCst);
Err(Error::DecryptFailed)
}
fn close(self) -> Result<(), Error> {
Ok(())
}
}
let applied = Arc::new(AtomicUsize::new(0));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
FailFirst {
failed: applied.clone(),
},
)
.expect("spawn");
let halt = crate::halt::Halt::new();
let deadline = Duration::from_secs(5);
// Feed one item and wait until the consumer has actually applied (and
// failed on) it, so the check below is deterministic rather than racy.
pipe.send_with_halt(0u64, &halt, deadline)
.expect("the first send lands");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && applied.load(Ordering::SeqCst) == 0 {
std::thread::sleep(Duration::from_millis(5));
}
assert_eq!(applied.load(Ordering::SeqCst), 1, "apply ran and failed");
assert!(pipe.consumer_failed(), "the failure must be observable");
// The very next send must hand the item straight back — the producer's
// signal to stop reading the disc.
assert_eq!(
pipe.send_with_halt(1u64, &halt, deadline),
Err(1u64),
"send_with_halt must fail fast once the consumer's apply has failed"
);
// The halt was never fired, so this is not a cancellation: the real error
// still comes out of finish().
assert!(matches!(pipe.finish(), Err(Error::DecryptFailed)));
assert_eq!(
applied.load(Ordering::SeqCst),
1,
"no further item was applied"
);
}
/// The abandon/finalise race. A consumer that has ALREADY committed to
/// `close()` when the grace period expires cannot be stopped — the finalise is
/// happening — so the caller must wait for its result instead of reporting the
/// output as un-finalised. With a plain flag the consumer read it as clear, the
/// caller then stored it, and the caller returned `Err(Halted)`
/// (`completed = false`) while a fully finalised MKV (Cues written, Segment
/// size patched) landed on disk — a truncated rip indistinguishable from a
/// complete one.
#[test]
fn abandon_loses_to_a_close_already_committed() {
let state = Arc::new(AtomicU8::new(state::RUNNING));
let release = Arc::new(AtomicBool::new(false));
let in_close = Arc::new(AtomicBool::new(false));
let (st, rel, inc) = (state.clone(), release.clone(), in_close.clone());
let handle = thread::Builder::new()
.name("test-consumer".into())
.spawn(move || -> Result<u64, Error> {
// Exactly what the consumer does before finalising: claim the
// right to close.
assert!(
st.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire
)
.is_ok(),
"the consumer claims the finalise first"
);
inc.store(true, Ordering::SeqCst);
// Inside `close()`, finalising the container.
while !rel.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
Ok(42)
})
.expect("spawn");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && !in_close.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
assert!(in_close.load(Ordering::SeqCst), "consumer reached close()");
// Finish the close only AFTER the first grace window has expired, so the
// caller genuinely reaches the abandon decision with a close in flight.
let rel = release.clone();
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
//
// These intervals used to be 600 ms against a 300 ms grace, which
// left NO margin: two 300 ms windows end at 600 ms, and the 250 ms
// poll cadence can push the observation later still, so on a loaded
// runner the second window expired first and the caller abandoned —
// failing with Err(Halted) against a race, not a defect.
//
// Scaled up so the jitter is small relative to the intervals: the
// first window ends at ~1.0-1.25 s and the second at ~2.0-2.25 s,
// so releasing at 1.6 s sits well inside the second with roughly
// 350 ms of slack on either side. The ordering under test is
// unchanged; only the margin is.
thread::sleep(Duration::from_millis(1600));
rel.store(true, Ordering::SeqCst);
});
let grace = Duration::from_secs(1);
let res = finish_with_grace(handle, &state, grace, Error::Halted);
assert!(
matches!(res, Ok(42)),
"a finalise already in flight must be waited for, not abandoned: {res:?}"
);
assert_eq!(
state.load(Ordering::SeqCst),
state::CLOSING,
"the caller must not have overwritten the consumer's claim"
);
}
} }
+1 -1
View File
@@ -272,7 +272,7 @@ impl WritebackPipeline {
self.chunk_bytes, self.chunk_bytes,
self.skip_wait(), self.skip_wait(),
); );
if self.chunk_count.is_multiple_of(SIZE_LOG_INTERVAL) { if self.chunk_count % SIZE_LOG_INTERVAL == 0 {
tracing::debug!( tracing::debug!(
target: "mux", target: "mux",
"WritebackPipeline chunk_bytes={} after {} chunks is_nfs={} degraded={}", "WritebackPipeline chunk_bytes={} after {} chunks is_nfs={} degraded={}",
+29 -84
View File
@@ -30,15 +30,14 @@ pub(super) fn preallocate(file: &File, size_bytes: u64) {
); );
} }
/// Run `fsync` on `file` with a 60 s deadline. On timeout, halt or a lost /// Run `fsync` on `file` with a 60 s deadline. On timeout — and
/// worker we log and return `Err` — matching macOS. POSIX gives `fsync` /// likewise on halt or a lost worker we log and return `Ok(())`: the
/// exactly one way to say "the data is on stable storage" and that is a zero /// kernel will still flush on close, so the data is best-effort durable.
/// return; a call that never reached the device has not earned it, so `Ok(())` /// The alternative (trap the thread for the rest of the rip, or return
/// from here means the flush completed and nothing else. /// an error that aborts an otherwise-complete mux) is worse, so all
/// /// three fallbacks return `Ok(())`. `Ok(())` from these paths is NOT a
/// The kernel will still flush on close, so the data is usually durable /// durability barrier — the durable flush did not complete; only the
/// anyway — but that is a probability, not a barrier, and a caller that needs /// hang is bounded.
/// crash-consistency has to be able to tell the difference.
/// ///
/// ## fd-reuse safety /// ## fd-reuse safety
/// ///
@@ -79,7 +78,27 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
}, },
) { ) {
Ok(inner) => inner, Ok(inner) => inner,
Err(e) => bounded_failure_to_result(e), Err(crate::io::bounded::BoundedError::Timeout) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; kernel will flush on close (best-effort)"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::WorkerLost) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data not durably flushed, kernel will flush on close"
);
Ok(())
}
} }
} }
@@ -124,77 +143,3 @@ mod tests {
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile"); durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
} }
} }
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `fsync` onto the
/// `io::Error` `durable_sync` returns.
///
/// Every arm means the same thing: **no sync observably ran**. All three used to
/// return `Ok(())`, so `WritebackFile::sync_all` reported success for a
/// durability barrier that never happened. POSIX gives `fsync` one way to say
/// "the data is on stable storage" — a zero return — and a call that never
/// reached the device has not earned it.
///
/// This mirrors the macOS `F_FULLFSYNC` mapping exactly. The two were found
/// carrying the identical defect, and a platform disagreeing with its sibling
/// about whether a failed sync is an error is the "works on my platform" class
/// this crate has been bitten by before — most recently an over-length SCSI CDB
/// that macOS rejected and the other two silently truncated.
///
/// No message text (this crate ships no user-facing English): the kind, and
/// `EIO` for the worker-lost case, are the signal; `tracing` carries the detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
// EIO, matching the macOS sibling: a consumer distinguishing these
// three failures does so on the same value on every platform.
// ErrorKind::Other carries nothing a caller can branch on.
Err(crate::error::Error::SyncWorkerLost.into())
}
}
}
#[cfg(test)]
mod bounded_failure_tests {
use super::*;
use crate::io::bounded::BoundedError;
/// Every bounded-fsync failure must be an error. Asserted per variant rather
/// than as a loop so a new variant defaulting to Ok cannot slip through.
#[test]
fn no_bounded_fsync_failure_maps_to_ok() {
assert_eq!(
bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out fsync must be an error")
.kind(),
io::ErrorKind::TimedOut
);
assert_eq!(
bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted fsync must be an error")
.kind(),
io::ErrorKind::Interrupted
);
assert!(
bounded_failure_to_result(BoundedError::WorkerLost).is_err(),
"a lost fsync worker must be an error"
);
}
}
+5 -107
View File
@@ -105,46 +105,15 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
}, },
) { ) {
Ok(inner) => inner, Ok(inner) => inner,
Err(e) => bounded_failure_to_result(e), Err(crate::io::bounded::BoundedError::Timeout) => {
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `F_FULLFSYNC`
/// onto the `io::Error` `durable_sync` returns.
///
/// Every arm here means the same thing: **no sync observably ran**. All three
/// previously returned `Ok(())`, so `WritebackFile::sync_all` reported success
/// for a durability barrier that never happened — a total failure exiting 0,
/// with only a log line to distinguish it. POSIX gives `fsync` exactly one way
/// to say "the data is on stable storage" and that is a zero return; a call
/// that never reached the device has not earned it.
///
/// The errors carry no message text (this crate ships no user-facing English):
/// the kind, and `EIO` for the worker-lost case, are the whole signal, and the
/// `tracing` lines above/below carry the operator detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!( tracing::error!(
target: "mux", target: "mux",
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; data NOT durably flushed, kernel will flush on close" "WritebackFile::sync_all F_FULLFSYNC timed out after 60s; kernel will flush on close (best-effort)"
); );
Err(crate::error::Error::SyncTimeout.into()) Ok(())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncWorkerLost.into())
} }
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
} }
} }
@@ -187,75 +156,4 @@ mod tests {
// durable_sync must complete without error on the local tempfile. // durable_sync must complete without error on the local tempfile.
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile"); durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
} }
/// Every `BoundedError` arm of the bounded `F_FULLFSYNC` means no sync
/// observably ran. All three returned `Ok(())`, so `sync_all` reported a
/// durability barrier that never happened — the caller could not tell a
/// completed flush from a skipped one by any means except reading a log.
///
/// Asserted on the concrete `ErrorKind` / `errno` each arm must produce,
/// so a future arm that quietly reverts to `Ok(())` fails here.
#[test]
fn every_bounded_failure_is_reported_as_an_error() {
use crate::io::bounded::BoundedError;
let timeout = bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out F_FULLFSYNC must be an error");
assert_eq!(
timeout.kind(),
io::ErrorKind::TimedOut,
"a timed-out F_FULLFSYNC must not be reported as a completed sync"
);
let halted = bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted F_FULLFSYNC must be an error");
assert_eq!(
halted.kind(),
io::ErrorKind::Interrupted,
"a halted F_FULLFSYNC must not be reported as a completed sync"
);
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each
// carries its own numeric code through the "E<code>" prefix that
// `From<Error> for io::Error` mints — the only shape `error_code`
// recognises. A bare `ErrorKind` cannot be classified, which is how a
// user cancel here used to read as a hard I/O failure.
let lost = bounded_failure_to_result(BoundedError::WorkerLost)
.expect_err("a lost F_FULLFSYNC worker must be an error");
assert!(
lost.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_WORKER_LOST)),
"a lost worker must be identifiable, got {lost}"
);
assert!(
timeout
.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_TIMEOUT)),
"a timeout must be distinguishable from a lost worker, got {timeout}"
);
assert!(
crate::error::is_halt(&halted),
"a halt must satisfy the crate's own is_halt(), or the CLI reports a \
user cancel as a failure; got {halted}"
);
}
/// The failure path must be reachable through the public surface: a
/// `WritebackFile::sync_all` that hits any of these arms must surface an
/// `Err`, not a silent `Ok`. Pinned at the mapping boundary because the
/// timeout itself is not deterministically inducible in a unit test.
#[test]
fn bounded_failures_are_never_mapped_to_ok() {
use crate::io::bounded::BoundedError;
for e in [
BoundedError::Timeout,
BoundedError::Halted,
BoundedError::WorkerLost,
] {
assert!(
bounded_failure_to_result(e).is_err(),
"a bounded F_FULLFSYNC failure must never map to Ok"
);
}
}
} }
+9 -15
View File
@@ -178,19 +178,12 @@ impl WritebackFile {
/// is left to the kernel's normal flush-on-close path — best /// is left to the kernel's normal flush-on-close path — best
/// effort, but bounded. /// effort, but bounded.
/// ///
/// A bounded-fsync failure is returned as an `Err` on BOTH platforms, so /// IMPORTANT: on Linux/macOS a successful `Ok(())` does NOT
/// `Ok(())` means the flush completed and a caller needing /// guarantee the data is durable if the bounded fsync timed out or
/// crash-consistency can treat it as a durability barrier. /// was halted — only the hang is bounded, the fsync may not have
/// /// completed. Callers needing crash-consistency (e.g. mux-finish
/// The three causes are DISTINGUISHABLE by numeric code, because a caller /// then external commit/DB update) must not treat `Ok(())` as a
/// should not retry a lost worker the way it retries a timeout, and must /// durability barrier.
/// not report a user cancel as a failure:
///
/// * [`E_SYNC_TIMEOUT`](crate::error::E_SYNC_TIMEOUT) — deadline expired
/// * [`E_HALTED`](crate::error::E_HALTED) — cancelled;
/// [`is_halt`](crate::error::is_halt) recognises it
/// * [`E_SYNC_WORKER_LOST`](crate::error::E_SYNC_WORKER_LOST) — the worker
/// thread died before reporting
pub fn sync_all(&mut self) -> io::Result<()> { pub fn sync_all(&mut self) -> io::Result<()> {
if self.seek_count > 0 { if self.seek_count > 0 {
tracing::debug!( tracing::debug!(
@@ -263,8 +256,9 @@ impl super::sink::SequentialSink for WritebackFile {
/// the same work [`Self::sync_all`] does. Implemented explicitly (no /// the same work [`Self::sync_all`] does. Implemented explicitly (no
/// blanket impl) so a `dyn SequentialSink` / `dyn RandomAccessSink` /// blanket impl) so a `dyn SequentialSink` / `dyn RandomAccessSink`
/// `finish()` actually finalises + fsyncs instead of hitting a no-op /// `finish()` actually finalises + fsyncs instead of hitting a no-op
/// default. A bounded-fsync failure surfaces as an `Err` here, on every /// default. Note the bounded-fsync caveat from [`Self::sync_all`]
/// platform, exactly as it does from [`Self::sync_all`]. /// applies: `Ok(())` is not a durability barrier if the fsync timed
/// out or was halted.
fn finish(&mut self) -> io::Result<()> { fn finish(&mut self) -> io::Result<()> {
self.sync_all() self.sync_all()
} }
+10 -179
View File
@@ -53,30 +53,9 @@ pub const MIN_SAMPLE_UNITS: usize = 8;
/// units it yields); the *requested* count is a caller-side compile-time constant that /// units it yields); the *requested* count is a caller-side compile-time constant that
/// callers pin to `MIN_SAMPLE_UNITS` (see e.g. autorip's `SAMPLE_UNITS`). Together the /// callers pin to `MIN_SAMPLE_UNITS` (see e.g. autorip's `SAMPLE_UNITS`). Together the
/// two make under-sampling unrepresentable at the request boundary. /// two make under-sampling unrepresentable at the request boundary.
/// #[derive(Debug, Clone)]
/// The wrapped samples are on-disc AACS ciphertext — the same bytes the sibling
/// [`DiscInputs::samples`] redacts as key MATERIAL — so [`Debug`] is hand-written
/// and redacting; see the impl below.
#[derive(Clone)]
pub struct DecodeSampleSet(Vec<Vec<u8>>); pub struct DecodeSampleSet(Vec<Vec<u8>>);
impl std::fmt::Debug for DecodeSampleSet {
/// Prints the SHAPE only. A derived `Debug` dumped every wrapped sample
/// verbatim: a `DecodeSampleSet` carries at least [`MIN_SAMPLE_UNITS`]
/// 6144-byte aligned units (≥ 49 KiB, in practice multi-MB) of AACS
/// ciphertext plus each unit's clear 16-byte derivation seed, so one
/// `tracing::debug!("{set:?}")` on a failed `/decode` request — or an
/// `assert_eq!` whose panic message formats it — wrote all of it to the log
/// that gets attached to a bug report. Same policy and same shape as
/// [`DiscInputs`]'s impl below.
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DecodeSampleSet")
.field("units", &"<redacted>")
.field("units_len", &self.0.len())
.finish()
}
}
impl DecodeSampleSet { impl DecodeSampleSet {
/// Wrap `units` iff it carries at least [`MIN_SAMPLE_UNITS`] samples; `None` /// Wrap `units` iff it carries at least [`MIN_SAMPLE_UNITS`] samples; `None`
/// otherwise (the caller then skips the online source rather than sending an /// otherwise (the caller then skips the online source rather than sending an
@@ -103,12 +82,9 @@ impl DecodeSampleSet {
} }
/// The public AACS inputs a key source needs to look a disc up. Captured at /// The public AACS inputs a key source needs to look a disc up. Captured at
/// scan; carries no DERIVED secrets (no media key, VUK or plaintext unit key) — /// scan; contains no secrets — only the disc identity and the on-disc AACS
/// only the disc identity and the on-disc AACS structures a source or key server /// structures a source or key server may key on.
/// may key on. The on-disc structures are nonetheless key MATERIAL (the encrypted #[derive(Debug, Clone)]
/// title keys live in `unit_key_ro`), so [`Debug`] is hand-written and redacting;
/// see the impl below.
#[derive(Clone)]
pub struct DiscInputs { pub struct DiscInputs {
/// SHA-1 of `Unit_Key_RO.inf`, `0x`-prefixed hex. The value a keydb keys /// SHA-1 of `Unit_Key_RO.inf`, `0x`-prefixed hex. The value a keydb keys
/// its per-disc entries by, and a key server identifies the disc with. /// its per-disc entries by, and a key server identifies the disc with.
@@ -139,30 +115,6 @@ pub struct DiscInputs {
pub volume_label: Option<String>, pub volume_label: Option<String>,
} }
/// Redacting `Debug`, per the policy `aacs::types` documents (and which
/// `aacs::types::Vid` already applies to this very Volume ID). `DiscInputs` is
/// public and returned by [`crate::Disc::inputs`], so a consumer's
/// `tracing::debug!("{inputs:?}")` used to print the Volume ID, the whole
/// `Unit_Key_RO.inf` (the encrypted title keys), the entire MKB and every
/// ciphertext sample verbatim into a log that ends up attached to a bug report.
/// Only non-secret identity and shape (presence, lengths) is printed.
impl std::fmt::Debug for DiscInputs {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscInputs")
.field("disc_hash", &self.disc_hash)
.field("volume_id", &"<redacted>")
.field("version", &self.version)
.field("mkb", &"<redacted>")
.field("mkb_len", &self.mkb.len())
.field("unit_key_ro", &"<redacted>")
.field("unit_key_ro_len", &self.unit_key_ro.len())
.field("samples", &"<redacted>")
.field("samples_len", &self.samples.len())
.field("volume_label", &self.volume_label)
.finish()
}
}
/// A lazy view of a disc's AACS material, handed to [`KeySource::get_unit_keys`] so a /// A lazy view of a disc's AACS material, handed to [`KeySource::get_unit_keys`] so a
/// source can drive the derivation chain without holding the disc reader. /// source can drive the derivation chain without holding the disc reader.
/// ///
@@ -355,23 +307,8 @@ pub fn resolve_and_apply(
/// from [`crate::aacs::derive::decrypt_unit_key`]; the library's canonical CPS-unit number is /// from [`crate::aacs::derive::decrypt_unit_key`]; the library's canonical CPS-unit number is
/// `position + 1` (matching [`crate::aacs::inf::parse_unit_key_ro`]'s `(i + 1)`), so /// `position + 1` (matching [`crate::aacs::inf::parse_unit_key_ro`]'s `(i + 1)`), so
/// the committed `AacsState.unit_keys` is byte-identical to the library-resolved /// the committed `AacsState.unit_keys` is byte-identical to the library-resolved
/// path. /// path. The number is cosmetic for descramble (the decrypt path strips it and
/// /// tries every key) but is kept faithful to the resolver's convention.
/// The NUMBER itself is not what descramble indexes by — but the ORDER is
/// load-bearing, so a source must return its keys in CPS-unit order. Trial
/// decrypt-and-check was deliberately deleted (see
/// [`crate::decrypt::AacsKeyMap`]: decryption is driven by the disc's CPS-unit /
/// FMTS-segment structure, "never by trial-decrypt-and-check per unit"), and
/// `decrypt_sectors_mapped` indexes the committed pool POSITIONALLY —
/// `unit_keys[key_idx].1`, where `key_idx` is a POSITION in the Vec a source
/// returned, recorded by `resolve_mux_key_map_cached` / `resolve_fmts_key_map`.
/// Return the same keys in a different order and every `AacsKeyMap` points at the
/// wrong key: the whole title decrypts under a neighbour's key, or a forensic
/// range trips the `is_clean` net into `DecryptFailed`. (The doc used to say the
/// number "is cosmetic for descramble (the decrypt path strips it and tries every
/// key)", which is what the DELETED trial-decrypt path did; the only place that
/// still tries every key is `Disc::decrypt_with`'s sample VALIDATION, which does
/// not descramble content.)
pub fn resolve_and_apply_traced( pub fn resolve_and_apply_traced(
sources: &[Box<dyn KeySource>], sources: &[Box<dyn KeySource>],
inputs: &DiscInputs, inputs: &DiscInputs,
@@ -381,14 +318,6 @@ pub fn resolve_and_apply_traced(
let mut trace = crate::aacs::trace::ResolutionTrace::new(); let mut trace = crate::aacs::trace::ResolutionTrace::new();
// The FIRST source failure seen, if any. A source that returns `Err` did not
// answer "no key for this disc" — it could not answer at all — and that
// reason is stamped onto `disc.aacs_error` below so the decrypt gate reports
// THAT instead of the generic `NoDiscKey`. First-wins (not last) so the
// ordered sources' most-preferred failure is the one the operator is told
// about, matching the first-valid-wins rule for successes.
let mut source_failure: Option<crate::error::Error> = None;
// The ctx parses Unit_Key_RO.inf at the stride for `inputs.version` (the // The ctx parses Unit_Key_RO.inf at the stride for `inputs.version` (the
// disc's own AACS major), so the stride is the disc's single source of truth. // disc's own AACS major), so the stride is the disc's single source of truth.
let ctx = DiscInputsCtx::new(inputs); let ctx = DiscInputsCtx::new(inputs);
@@ -421,56 +350,17 @@ pub fn resolve_and_apply_traced(
outcome: KeyOutcome::NoKey, outcome: KeyOutcome::NoKey,
}); });
} }
// The source ANSWERED and holds nothing for this disc. This — and // Empty (no key here) or a source failure — both are "no key from
// only this — is `NoEntry`: the claim "I looked, it is not there". // this source"; move on to the next.
Ok(_) => { Ok(_) | Err(_) => {
trace.keys.push(KeyStep { trace.keys.push(KeyStep {
who, who,
path: vec![KeyNode::NoEntry], path: vec![KeyNode::NoEntry],
outcome: KeyOutcome::NoKey, outcome: KeyOutcome::NoKey,
}); });
} }
// The source could NOT answer — it was unreachable, it errored, or it
// refused. Nothing is known about whether a key exists, so the path
// is EMPTY: recording `NoEntry` here is exactly the conflation that
// made a seven-hour run of HTTP 502s render as
// `key: online > no entry > NO KEY` + `E7022 No key source has a
// decryption key for this disc`, and sent operators hunting for a VUK
// that was never missing.
//
// The reason itself rides out on `disc.aacs_error` (below), the
// channel `Disc::ensure_decryptable_keys` already reads for the
// E7017-vs-E7022 split — so the decrypt gate raises the SOURCE's code
// (`KeyServiceUnavailable` / `KeyServiceUnauthorized` /
// `KeyServiceRateLimited`) instead of the generic `NoDiscKey`.
//
// `KeyOutcome` deliberately gains no variant: it is matched
// exhaustively by every front-end's trace renderer (freemkv's
// `pipe::render_resolution_trace`, autorip's
// `keysource::render_resolution_trace`), and this fix must not turn
// into a breaking change across four repos to say something the error
// code already says precisely.
Err(e) => {
if source_failure.is_none() {
source_failure = Some(e);
}
trace.keys.push(KeyStep {
who,
path: Vec::new(),
outcome: KeyOutcome::NoKey,
});
}
} }
} }
// Nothing resolved. If a source FAILED rather than answered, stamp that
// reason onto the disc so the decrypt gate can report it — but never clobber
// a reason the scan already captured (e.g. `AacsVidUnavailable`), which is
// closer to the disc itself than a source outage is.
if let Some(e) = source_failure
&& disc.aacs_error.is_none()
{
disc.aacs_error = Some(e);
}
(false, trace) (false, trace)
} }
@@ -1323,7 +1213,7 @@ mod tests {
break; break;
} }
let abs = (lba - self.ext_start) / ALIGNED_UNIT_SECTORS + i as u32; let abs = (lba - self.ext_start) / ALIGNED_UNIT_SECTORS + i as u32;
if abs.is_multiple_of(2) { if abs % 2 == 0 {
chunk.fill(0x11); // CPI-clear (0x11 & 0xC0 == 0), no TS sync chunk.fill(0x11); // CPI-clear (0x11 & 0xC0 == 0), no TS sync
} else { } else {
chunk.fill(0xAB); // scrambled body (no TS sync) chunk.fill(0xAB); // scrambled body (no TS sync)
@@ -1421,63 +1311,4 @@ mod tests {
assert_eq!(k10[1], [0x10; 16], "V10 reads the 2nd key at +48"); assert_eq!(k10[1], [0x10; 16], "V10 reads the 2nd key at +48");
assert_ne!(k20[1], k10[1], "the parse stride follows inputs.version"); assert_ne!(k20[1], k10[1], "the parse stride follows inputs.version");
} }
/// `DiscInputs` is public and returned by `Disc::inputs`, so any consumer's
/// `tracing::debug!("{inputs:?}")` prints it. A derived `Debug` printed the
/// Volume ID (the value `aacs::types::Vid` deliberately renders as
/// `Vid(<redacted>)`), the whole `Unit_Key_RO.inf` (the encrypted title keys),
/// the entire MKB and every ciphertext sample verbatim. Sentinel byte
/// 0xD5 = decimal 213, matching `aacs::types::redaction_tests`. Mutation
/// guard: restoring `#[derive(Debug)]` fails this.
#[test]
fn disc_inputs_debug_is_redacted() {
let inputs = DiscInputs {
disc_hash: "0xAA".into(),
volume_id: [0xD5; 16],
version: 2,
mkb: vec![0xD5; 64],
unit_key_ro: vec![0xD5; 48],
samples: vec![vec![0xD5; 6144]],
volume_label: Some("TITLE_2024".into()),
};
let dbg = format!("{inputs:?}");
assert!(
!dbg.contains("213"),
"DiscInputs Debug leaked key material (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"DiscInputs Debug missing redaction marker: {dbg}"
);
// Non-secret identity and shape stay printable for diagnostics.
assert!(dbg.contains("0xAA"), "{dbg}");
assert!(dbg.contains("mkb_len: 64"), "{dbg}");
assert!(dbg.contains("unit_key_ro_len: 48"), "{dbg}");
assert!(dbg.contains("samples_len: 1"), "{dbg}");
assert!(dbg.contains("TITLE_2024"), "{dbg}");
}
/// `DecodeSampleSet` is public and wraps the SAME on-disc ciphertext the
/// sibling `DiscInputs` redacts, so a derived `Debug` dumped ≥ MIN_SAMPLE_UNITS
/// × 6144 bytes of verbatim AACS ciphertext (plus every unit's clear 16-byte
/// derivation seed) into any log that formatted it. Sentinel byte 0xD5 =
/// decimal 213, matching `aacs::types::redaction_tests` and the
/// `DiscInputs` test above. Mutation guard: restoring `#[derive(Debug)]`
/// fails this.
#[test]
fn decode_sample_set_debug_is_redacted() {
let set = DecodeSampleSet::new(vec![vec![0xD5; 6144]; MIN_SAMPLE_UNITS])
.expect("MIN_SAMPLE_UNITS units is a valid set");
let dbg = format!("{set:?}");
assert!(
!dbg.contains("213"),
"DecodeSampleSet Debug leaked ciphertext (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"DecodeSampleSet Debug missing redaction marker: {dbg}"
);
// Non-secret shape stays printable for diagnostics.
assert!(dbg.contains("units_len: 8"), "{dbg}");
}
} }
+16 -45
View File
@@ -100,10 +100,10 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<DiscMetadata>
// Disc-set position is disc-global; first one we successfully // Disc-set position is disc-global; first one we successfully
// read wins. (All bdmt_*.xml on a given disc carry the same // read wins. (All bdmt_*.xml on a given disc carry the same
// value in practice.) // value in practice.)
if out.disc_number.is_none() if out.disc_number.is_none() {
&& let Some(ds) = disc_set if let Some(ds) = disc_set {
{ out.disc_number = Some(ds);
out.disc_number = Some(ds); }
} }
} }
@@ -184,20 +184,20 @@ fn extract_title(xml_text: &str) -> Option<String> {
// xml::text already trims its result, so an empty string after // xml::text already trims its result, so an empty string after
// extraction means a genuinely empty element. // extraction means a genuinely empty element.
for tag in ["name", "title"] { for tag in ["name", "title"] {
if let Some(s) = xml::text(xml_text, tag) if let Some(s) = xml::text(xml_text, tag) {
&& !s.is_empty() if !s.is_empty() {
{ return Some(s);
return Some(s); }
} }
} }
// tableOfContents/titleName: search inside the toc block so we // tableOfContents/titleName: search inside the toc block so we
// don't accidentally pick a stray <titleName> from elsewhere. // don't accidentally pick a stray <titleName> from elsewhere.
if let Some((s, e)) = xml::find_element(xml_text, "tableOfContents", 0) { if let Some((s, e)) = xml::find_element(xml_text, "tableOfContents", 0) {
let block = &xml_text[s..e]; let block = &xml_text[s..e];
if let Some(t) = xml::text(block, "titleName") if let Some(t) = xml::text(block, "titleName") {
&& !t.is_empty() if !t.is_empty() {
{ return Some(t);
return Some(t); }
} }
} }
None None
@@ -370,10 +370,10 @@ mod tests {
if let Some(d) = desc { if let Some(d) = desc {
meta.descriptions.insert(lang.to_string(), d); meta.descriptions.insert(lang.to_string(), d);
} }
if meta.disc_number.is_none() if meta.disc_number.is_none() {
&& let Some(d) = ds if let Some(d) = ds {
{ meta.disc_number = Some(d);
meta.disc_number = Some(d); }
} }
} }
@@ -566,33 +566,4 @@ mod tests {
let (title, _, _) = parse_bdmt_xml(xml).unwrap(); let (title, _, _) = parse_bdmt_xml(xml).unwrap();
assert_eq!(title, "Real Title"); assert_eq!(title, "Real Title");
} }
/// `is_bdmt_filename` must recognize the `bdmt_<lang>.xml` convention
/// and reject everything else — it drives `detect`'s directory scan.
/// Mutation: stub the return to a constant `true`/`false` → every
/// directory listing (or none) would match regardless of filename.
#[test]
fn is_bdmt_filename_matches_convention_only() {
assert!(is_bdmt_filename("bdmt_eng.xml"));
assert!(is_bdmt_filename("BDMT_FRA.XML"));
assert!(!is_bdmt_filename("bdmt_engl.xml"));
assert!(!is_bdmt_filename("index.bdmv"));
assert!(!is_bdmt_filename("foo.xml"));
}
/// Spec: "Disc 1 of 1" (a single-disc release whose bdmt XML still
/// carries `<di:numSets>1</di:numSets>`) is a valid, non-nonsensical
/// pair — `total < 1` must reject only `total == 0`, not `total == 1`.
/// Mutation: `total < 1` -> `total == 1` or `total <= 1` would reject
/// this legitimate (1, 1) pair as if it were malformed.
#[test]
fn disc_set_allows_single_disc_release() {
let xml = r#"<discInfo xmlns:di="urn:BDA:bdmv;disclibmeta">
<di:name>Film</di:name>
<di:discNumber>1</di:discNumber>
<di:numSets>1</di:numSets>
</discInfo>"#;
let (_, _, set) = parse_bdmt_xml(xml).unwrap();
assert_eq!(set, Some((1, 1)));
}
} }
+1 -387
View File
@@ -989,18 +989,7 @@ impl<'a> Reader<'a> {
} }
fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> { fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> {
// `n` is attacker-supplied: it comes from a JVMS `u4` attribute_length if self.pos + n > self.data.len() {
// / code_length (§4.7, §4.7.3) or a `u2` Utf8 length (§4.4.7). Unlike
// the fixed-width readers above, whose `self.pos + k` cannot leave the
// buffer's own address range, `self.pos + n` can wrap — on a 32-bit
// target a `u4` length near 0xFFFF_FFFF plus a non-zero `pos` panics
// in debug and in release wraps to a SMALL end offset that passes the
// bounds check, after which the slice index itself panics. Checked, so
// an out-of-range length is the EOF error it always should have been.
let Some(end) = self.pos.checked_add(n) else {
return Err(Error::UnexpectedEof { needed });
};
if end > self.data.len() {
return Err(Error::UnexpectedEof { needed }); return Err(Error::UnexpectedEof { needed });
} }
let s = &self.data[self.pos..self.pos + n]; let s = &self.data[self.pos..self.pos + n];
@@ -1017,46 +1006,6 @@ impl<'a> Reader<'a> {
mod tests { mod tests {
use super::*; use super::*;
/// `Reader::slice` takes an attacker-supplied length: a JVMS `u4`
/// `attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8
/// length (§4.4.7). Adding it to `pos` without a wrap check panics on
/// overflow in debug and, in release, wraps to a small end offset that
/// slips past the bounds check and then panics inside the slice index.
/// Both are panics escaping a parser whose whole input is untrusted disc
/// bytes; the contract is an EOF error.
#[test]
fn slice_rejects_a_length_that_would_wrap_pos() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
r.u64("advance pos").expect("8 bytes available");
// pos is now 8; usize::MAX would wrap the end offset to 7.
match r.slice(usize::MAX, "wrapping length") {
Err(Error::UnexpectedEof { .. }) => {}
Err(other) => panic!("expected UnexpectedEof, got {other:?}"),
Ok(s) => panic!("expected UnexpectedEof, got a {}-byte slice", s.len()),
}
// The reader must not have consumed anything.
match r.slice(8, "remaining bytes") {
Ok(s) => assert_eq!(s.len(), 8, "pos moved on the rejected slice"),
Err(e) => panic!("the remaining 8 bytes must still be readable: {e:?}"),
}
}
/// The ordinary out-of-range case (no wrap) must keep returning EOF, and
/// an exactly-fitting length must still succeed — the check is `>`, not
/// `>=`.
#[test]
fn slice_boundary_is_inclusive_of_the_final_byte() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
assert_eq!(r.slice(16, "whole buffer").expect("exact fit").len(), 16);
let mut r = Reader::new(&data);
assert!(matches!(
r.slice(17, "one past"),
Err(Error::UnexpectedEof { .. })
));
}
#[test] #[test]
fn rejects_non_class_bytes() { fn rejects_non_class_bytes() {
match ClassFile::parse(b"\x00\x01\x02\x03DEAD") { match ClassFile::parse(b"\x00\x01\x02\x03DEAD") {
@@ -1388,339 +1337,4 @@ mod tests {
let _ = decode_modified_utf8(&buf); let _ = decode_modified_utf8(&buf);
} }
} }
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness
//
// These exercise plain data accessors on an already-parsed pool
// (built via the test-only `from_entries` constructor) — not the
// untrusted-bytes parsing path, just "does the right variant map to
// the right Option value."
// -----------------------------------------------------------------
fn sample_pool() -> ConstantPool {
// index: 0=Empty (reserved), 1=Utf8("Hello"), 2=Integer(42),
// 3=String{string_index:1}, 4=Class{name_index:1}, 5=Float(1.5),
// 6=Long(9), 7=Empty (2-slot tail), 8=Double(2.5), 9=Empty (tail).
ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("Hello".to_string()),
CpInfo::Integer(42),
CpInfo::String { string_index: 1 },
CpInfo::Class { name_index: 1 },
CpInfo::Float(1.5),
CpInfo::Long(9),
CpInfo::Empty,
CpInfo::Double(2.5),
CpInfo::Empty,
])
}
#[test]
fn constant_pool_string_resolves_through_string_index() {
let pool = sample_pool();
// index 3 is CpInfo::String{string_index: 1} -> utf8(1) = "Hello".
assert_eq!(pool.string(3), Some("Hello"));
// Wrong variant (Integer at index 2) must not resolve as a string.
assert_eq!(pool.string(2), None);
// Out of range index.
assert_eq!(pool.string(999), None);
}
#[test]
fn constant_pool_integer_resolves_only_integer_entries() {
let pool = sample_pool();
assert_eq!(pool.integer(2), Some(42));
// Wrong variant (Utf8 at index 1) must not resolve as an integer.
assert_eq!(pool.integer(1), None);
assert_eq!(pool.integer(999), None);
}
#[test]
fn constant_pool_load_constant_display_covers_ldc_operand_kinds() {
let pool = sample_pool();
assert_eq!(
pool.load_constant_display(1),
Some("utf8:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(2), Some("int:42".to_string()));
assert_eq!(
pool.load_constant_display(3),
Some("str:\"Hello\"".to_string())
);
assert_eq!(
pool.load_constant_display(4),
Some("class:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(5), Some("float:1.5".to_string()));
assert_eq!(pool.load_constant_display(6), Some("long:9".to_string()));
assert_eq!(
pool.load_constant_display(8),
Some("double:2.5".to_string())
);
// A variant with no display arm (e.g. reserved Empty slot) -> None.
assert_eq!(pool.load_constant_display(0), None);
assert_eq!(pool.load_constant_display(999), None);
}
#[test]
fn constant_pool_len_and_is_empty() {
let pool = sample_pool();
assert_eq!(pool.len(), 10);
assert!(!pool.is_empty());
let empty = ConstantPool::from_entries(vec![]);
assert_eq!(empty.len(), 0);
assert!(empty.is_empty());
}
#[test]
fn constant_pool_iter_yields_index_and_entry_pairs() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("A".to_string()),
CpInfo::Integer(7),
]);
let indices: Vec<u16> = pool.iter().map(|(i, _)| i).collect();
assert_eq!(indices, vec![0, 1, 2]);
// Confirm the entries themselves come through, not an empty iterator.
let utf8_at_1 = pool.iter().find(|(i, _)| *i == 1).map(|(_, e)| match e {
CpInfo::Utf8(s) => s.as_str(),
_ => "?",
});
assert_eq!(utf8_at_1, Some("A"));
}
fn class_file_with(this_class: u16, super_class: u16, pool: ConstantPool) -> ClassFile {
ClassFile {
minor_version: 0,
major_version: 0,
constant_pool: pool,
access_flags: 0,
this_class,
super_class,
interfaces: Vec::new(),
fields: Vec::new(),
methods: Vec::new(),
attributes: Vec::new(),
}
}
#[test]
fn this_class_name_and_super_class_name_resolve_distinct_indices() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("com/example/Foo".to_string()),
CpInfo::Utf8("com/example/Bar".to_string()),
CpInfo::Class { name_index: 1 },
CpInfo::Class { name_index: 2 },
]);
let cf = class_file_with(3, 4, pool);
assert_eq!(cf.this_class_name(), Some("com/example/Foo"));
assert_eq!(cf.super_class_name(), Some("com/example/Bar"));
// this_class index pointing at a non-Class entry must not resolve.
let pool2 = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("not a class ref".to_string()),
]);
let cf2 = class_file_with(1, 1, pool2);
assert_eq!(cf2.this_class_name(), None);
assert_eq!(cf2.super_class_name(), None);
}
#[test]
fn member_descriptor_resolves_the_descriptor_not_the_name() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("doStuff".to_string()), // index 1: name
CpInfo::Utf8("()V".to_string()), // index 2: descriptor
]);
let cf = class_file_with(0, 0, pool);
let m = Member {
access_flags: 0,
name_index: 1,
descriptor_index: 2,
attributes: Vec::new(),
};
assert_eq!(cf.member_descriptor(&m), Some("()V"));
assert_ne!(cf.member_descriptor(&m), Some("doStuff"));
}
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness
//
// Mirrors `slice_boundary_is_inclusive_of_the_final_byte`: an
// exact-fit read must succeed, one byte short must fail. Plus
// positive-value tests so a scrambled byte assembly (not just an
// out-of-bounds read) would be caught.
// -----------------------------------------------------------------
#[test]
fn u16_boundary_is_inclusive_of_the_final_byte() {
let data = [0xAB, 0xCD];
let mut r = Reader::new(&data);
assert_eq!(r.u16("exact fit").expect("2 bytes available"), 0xABCD);
let data = [0xAB];
let mut r = Reader::new(&data);
assert!(matches!(
r.u16("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u16_decodes_big_endian_value() {
let data = [0x01, 0x02];
let mut r = Reader::new(&data);
assert_eq!(r.u16("value").unwrap(), 0x0102);
}
#[test]
fn u32_boundary_is_inclusive_of_the_final_byte() {
let data = [0x00, 0x00, 0x00, 0x2A];
let mut r = Reader::new(&data);
assert_eq!(r.u32("exact fit").expect("4 bytes available"), 42);
let data = [0x00, 0x00, 0x00];
let mut r = Reader::new(&data);
assert!(matches!(
r.u32("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u32_decodes_big_endian_value() {
let data = [0x00, 0x00, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u32("value").unwrap(), 1337);
}
#[test]
fn u64_boundary_is_inclusive_of_the_final_byte() {
// pos == 0, buffer exactly 8 bytes: must succeed.
let data = [0, 0, 0, 0, 0, 0, 0, 0x7B]; // 123
let mut r = Reader::new(&data);
assert_eq!(r.u64("exact fit").expect("8 bytes available"), 123);
// pos == 0, buffer one byte short of 8: must fail cleanly, not
// panic on the internal self.data[self.pos + 7] index.
let data = [0u8; 7];
let mut r = Reader::new(&data);
assert!(matches!(
r.u64("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u64_decodes_big_endian_value() {
let data = [0, 0, 0, 0, 0, 0, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u64("value").unwrap(), 1337);
}
// -----------------------------------------------------------------
// decode_modified_utf8: 3-byte (BMP) decode path
// -----------------------------------------------------------------
#[test]
fn modified_utf8_three_byte_cjk() {
// U+3042 (hiragana あ) in modified UTF-8: 1110xxxx 10xxxxxx 10xxxxxx
// = 0xE3 0x81 0x82.
let s = decode_modified_utf8(&[0xE3, 0x81, 0x82]).unwrap();
assert_eq!(s, "\u{3042}");
}
#[test]
fn modified_utf8_three_byte_rejects_bad_first_continuation() {
// Valid lead byte (0xE3), but the first continuation byte is not
// 10xxxxxx (0x01 instead) — must be rejected, proving the first
// `& 0xC0 != 0x80` check is live.
assert!(decode_modified_utf8(&[0xE3, 0x01, 0x82]).is_err());
}
#[test]
fn modified_utf8_three_byte_rejects_bad_second_continuation() {
// Valid lead + first continuation, but the second continuation
// byte is not 10xxxxxx — proves the second check is independently
// live (not short-circuited by the first).
assert!(decode_modified_utf8(&[0xE3, 0x81, 0x01]).is_err());
}
// -----------------------------------------------------------------
// read_constant_pool: Long/Double two-slot skip, real byte parsing
// -----------------------------------------------------------------
#[test]
fn constant_pool_long_entry_occupies_two_slots_via_real_parse() {
// Real class-file bytes (not the `from_entries` synthetic ctor):
// magic + minor/major + cp_count=4 + tag=5 (Long, 8-byte payload
// at index 1, reserved slot at index 2) + tag=1 (Utf8 at index 3)
// + empty access_flags/this/super/interfaces/fields/methods/attrs.
let mut buf = vec![
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, // minor
0x00, 0x34, // major
0x00, 0x04, // cp_count = 4 (0=Empty,1=Long,2=Empty tail,3=Utf8)
5, // Long tag
];
buf.extend_from_slice(&0x1122_3344_5566_7788u64.to_be_bytes()); // 8-byte payload
buf.push(1); // Utf8 tag
let name = b"marker";
buf.extend_from_slice(&(name.len() as u16).to_be_bytes());
buf.extend_from_slice(name);
// access_flags, this_class, super_class, interfaces_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0, 0, 0]);
// fields_count, methods_count, attributes_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0]);
let cf = ClassFile::parse(&buf).expect("well-formed synthetic class file");
assert_eq!(cf.constant_pool.len(), 4);
// The Long occupies indices 1 AND 2 (its reserved tail slot).
// The Utf8 must resolve at index 3 = long_index(1) + 2, NOT +1.
assert_eq!(cf.constant_pool.utf8(3), Some("marker"));
// Index 2 is the reserved tail slot: not a Utf8, must not
// resolve as one (guards against the Utf8 landing one slot early).
assert_eq!(cf.constant_pool.utf8(2), None);
match cf.constant_pool.get(1) {
Some(CpInfo::Long(v)) => assert_eq!(*v, 0x1122_3344_5566_7788u64 as i64),
other => panic!("expected Long at index 1, got {:?}", other),
}
}
// -----------------------------------------------------------------
// instruction_size: tableswitch/lookupswitch with non-degenerate
// low/high/npairs (the existing tests only cover low==high==0 and
// npairs==0, which can't distinguish `-` from `+` in the entry-count
// arithmetic).
// -----------------------------------------------------------------
#[test]
fn instruction_size_tableswitch_non_degenerate_range() {
// low=1, high=4 -> 4 entries (high-low+1 = 4). A `-`->`+` mutation
// on that arithmetic would instead compute high+low+1 = 6.
let mut code = vec![TABLESWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default offset
code.extend_from_slice(&1i32.to_be_bytes()); // low = 1
code.extend_from_slice(&4i32.to_be_bytes()); // high = 4
code.extend_from_slice(&[0; 16]); // 4 jump entries * 4 bytes
// total = 1 (opcode) + 3 (pad) + 12 (default/low/high) + 16 (entries) = 32
assert_eq!(instruction_size(&code, 0), Some(32));
}
#[test]
fn instruction_size_lookupswitch_non_degenerate_npairs() {
// npairs = 3 -> 3 * 8 = 24 bytes of pairs.
let mut code = vec![LOOKUPSWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default
code.extend_from_slice(&3i32.to_be_bytes()); // npairs = 3
code.extend_from_slice(&[0; 24]); // 3 pairs
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
} }
+36 -208
View File
@@ -36,18 +36,17 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// Stream number mapping from playbackconfig.xml // Stream number mapping from playbackconfig.xml
let mut stream_map: HashMap<String, u16> = HashMap::new(); let mut stream_map: HashMap<String, u16> = HashMap::new();
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml") if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml") {
&& let Ok(pc_text) = std::str::from_utf8(&pc_data) if let Ok(pc_text) = std::str::from_utf8(&pc_data) {
{ parse_playback_config(pc_text, &mut stream_map);
parse_playback_config(pc_text, &mut stream_map); }
} }
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map)?; let stream_nums = assign_stream_numbers(&stream_infos, &stream_map);
let mut labels = Vec::new(); let mut labels = Vec::new();
for (info, &stream_num) in stream_infos.iter().zip(stream_nums.iter()) { for (info, &stream_num) in stream_infos.iter().zip(stream_nums.iter()) {
labels.push(StreamLabel { labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num, stream_number: stream_num,
stream_type: info.stream_type, stream_type: info.stream_type,
language: info.language.clone(), language: info.language.clone(),
@@ -77,29 +76,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// map-assigned one. (Both numbering domains are 1-based per type, and /// map-assigned one. (Both numbering domains are 1-based per type, and
/// `apply_labels` matches on `(type, stream_number)`, so a collision /// `apply_labels` matches on `(type, stream_number)`, so a collision
/// would mislabel tracks.) /// would mislabel tracks.)
/// fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>) -> Vec<u16> {
/// Returns `None` when the 1-based stream-number space is exhausted — every
/// number in `1..=u16::MAX` for that type is either already claimed by the map
/// or already synthesized. That is unreachable on real media: the BD STN_table
/// carries at most 32 primary audio and 32 PG streams per playlist, so the
/// 65535-wide space leaves >2000x headroom. It IS reachable from a crafted
/// `streamproperties.xml` listing >65535 stream entries, and the only correct
/// answers there are "fail the parse" or "emit colliding numbers"; we fail.
///
/// The skip search is bounded by the numbering space itself: a `u16`
/// `saturating_add` here parked the counter at `u16::MAX` forever whenever the
/// map also claimed `u16::MAX`, turning an overflow guard into a hang that
/// `apply()`'s `catch_unwind` cannot interrupt. The counters are therefore
/// widened to `u32` so the skip loop strictly increases toward a fixed ceiling
/// (guaranteeing termination) and exhaustion is reported rather than absorbed.
fn assign_stream_numbers(
infos: &[StreamInfo],
stream_map: &HashMap<String, u16>,
) -> Option<Vec<u16>> {
/// One past the last assignable stream number, as a `u32` so the
/// counters can step off the end of the `u16` domain without wrapping.
const NUMBER_SPACE_END: u32 = u16::MAX as u32 + 1;
// Numbers already claimed by the map, per type. A map value of 0 is NOT a // Numbers already claimed by the map, per type. A map value of 0 is NOT a
// claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable. // claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable.
// Treat 0 as "unmapped" here (defense in depth — parse_playback_config also // Treat 0 as "unmapped" here (defense in depth — parse_playback_config also
@@ -119,8 +96,8 @@ fn assign_stream_numbers(
} }
} }
let mut audio_idx: u32 = 1; let mut audio_idx: u16 = 1;
let mut sub_idx: u32 = 1; let mut sub_idx: u16 = 1;
let mut out = Vec::with_capacity(infos.len()); let mut out = Vec::with_capacity(infos.len());
for info in infos { for info in infos {
let n = match stream_map.get(&info.id).copied() { let n = match stream_map.get(&info.id).copied() {
@@ -130,31 +107,21 @@ fn assign_stream_numbers(
StreamLabelType::Audio => (&mut audio_idx, &taken_audio), StreamLabelType::Audio => (&mut audio_idx, &taken_audio),
StreamLabelType::Subtitle => (&mut sub_idx, &taken_sub), StreamLabelType::Subtitle => (&mut sub_idx, &taken_sub),
}; };
// Advance past any number already claimed via the map. The // Advance past any number already claimed via the map.
// counter strictly increases and NUMBER_SPACE_END is fixed, so // saturating: a crafted XML with >65k stream entries must
// this terminates in at most 65535 steps for any input. // not overflow (panic in debug, wrap-to-0 in release) on
while *idx < NUMBER_SPACE_END && taken.contains(&(*idx as u16)) { // untrusted disc bytes.
*idx += 1; while taken.contains(idx) {
*idx = idx.saturating_add(1);
} }
if *idx >= NUMBER_SPACE_END { let n = *idx;
// Numbering space exhausted. Emitting anything here would *idx = idx.saturating_add(1);
// either wrap to 0 (unmatchable) or duplicate a number
// already bound to a different stream, so the parse fails.
tracing::warn!(
streams = infos.len(),
"criterion: 1-based u16 stream-number space exhausted; \
refusing to synthesize a colliding stream number"
);
return None;
}
let n = *idx as u16;
*idx += 1;
n n
} }
}; };
out.push(n); out.push(n);
} }
Some(out) out
} }
struct StreamInfo { struct StreamInfo {
@@ -222,13 +189,14 @@ fn parse_playback_config(text: &str, map: &mut HashMap<String, u16>) {
if let (Some(stream_id_str), Some(info_id)) = ( if let (Some(stream_id_str), Some(info_id)) = (
xml::text(block, "StreamID"), xml::text(block, "StreamID"),
xml::text(block, "StreamInfo_ID"), xml::text(block, "StreamInfo_ID"),
) && let Ok(stream_num) = stream_id_str.parse::<u16>() ) {
{ if let Ok(stream_num) = stream_id_str.parse::<u16>() {
// Stream numbers are 1-based per the apply_labels // Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently // contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it. // drops the label. Skip it rather than store it.
if stream_num != 0 { if stream_num != 0 {
map.insert(info_id, stream_num); map.insert(info_id, stream_num);
}
} }
} }
from = end; from = end;
@@ -258,89 +226,11 @@ mod tests {
info("a1", StreamLabelType::Audio), info("a1", StreamLabelType::Audio),
info("s0", StreamLabelType::Subtitle), info("s0", StreamLabelType::Subtitle),
]; ];
let nums = let nums = assign_stream_numbers(&infos, &HashMap::new());
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Per-type 1-based: audio 1,2 ; subtitle 1. // Per-type 1-based: audio 1,2 ; subtitle 1.
assert_eq!(nums, vec![1, 2, 1]); assert_eq!(nums, vec![1, 2, 1]);
} }
/// Immunity pin. `parse_stream_infos` emits one `StreamInfo` per
/// `*StreamInfos` element unconditionally — no filter, no `continue` — so
/// an element whose fields are missing or unrecognized still occupies its
/// position, and `assign_stream_numbers` still spends a number on it.
///
/// That is the property that keeps this parser out of the failure mode
/// where a skipped entry pulls every later label one stream forward. It
/// is load-bearing for the fallback path specifically: with no
/// `playbackconfig.xml` the numbers come purely from position in this
/// list, so dropping an element there would shift the rest.
///
/// Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
/// real audio streams renumber to 1 and 2.
/// Immunity pin, section-boundary half. Each stream here is one closed XML
/// element, and every field is read out of `&text[start..end]` — the range
/// `xml::find_element` returned — so one element can never absorb the next
/// one's fields, however the document is malformed around it. Contrast the
/// flat-string walk in pixelogic, where a section whose end marker is
/// missing keeps consuming entries as STN slots.
///
/// The missing-boundary case fails closed. An element with no close tag of
/// its own ends at the NEXT close tag, so it absorbs the element behind it
/// — the list comes back SHORTER. It cannot come back longer: nothing
/// outside a returned range is ever read as a stream, and `find_element`
/// yields `None` rather than a range running to EOF when no close tag
/// exists at all. A malformed document can cost this parser a slot; it can
/// never invent one.
///
/// Mutation: read fields from the document rather than the element's
/// range, or let a close-less element run to EOF → the trailing elements
/// re-enter the list as extra streams.
#[test]
fn an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it() {
let sp = concat!(
"<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID></AudioStreamInfos>",
// No `</AudioStreamInfos>` for this one.
"<AudioStreamInfos><ID>a1</ID><LangInfoID>FRA</LangInfoID>",
"<AudioStreamInfos><ID>a2</ID><LangInfoID>DEU</LangInfoID></AudioStreamInfos>",
);
let infos = parse_stream_infos(sp);
assert_eq!(
infos.iter().map(|i| i.id.as_str()).collect::<Vec<_>>(),
vec!["a0", "a1"],
"the close-less element absorbs the one behind it — two slots, not \
three, and never four"
);
assert_eq!(infos[1].language, "fra", "and keeps its own leading fields");
// With no close tag anywhere behind it, the element is not returned at
// all and the walk ends — the tail of the document never becomes a
// stream list.
let no_close = "<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID>";
assert!(parse_stream_infos(no_close).is_empty());
}
#[test]
fn unusable_stream_element_still_occupies_its_position() {
let sp = r#"
<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG_US</LangInfoID></AudioStreamInfos>
<AudioStreamInfos></AudioStreamInfos>
<AudioStreamInfos><ID>a2</ID><LangInfoID>FRA</LangInfoID><Content>COMMENTARY</Content></AudioStreamInfos>
<SubtitleStreamInfos><ID>s0</ID><LangInfoID></LangInfoID><Qualifier>WAT</Qualifier></SubtitleStreamInfos>
<SubtitleStreamInfos><ID>s1</ID><LangInfoID>ENG</LangInfoID><Qualifier>SDH</Qualifier></SubtitleStreamInfos>
"#;
let infos = parse_stream_infos(sp);
assert_eq!(infos.len(), 5, "every element yields a StreamInfo");
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
assert_eq!(
nums,
vec![1, 2, 3, 1, 2],
"the blank element owns audio slot 2, so the commentary is slot 3"
);
assert_eq!(infos[2].purpose, LabelPurpose::Commentary);
assert_eq!(infos[4].qualifier, LabelQualifier::Sdh);
}
#[test] #[test]
fn fallback_does_not_collide_with_partial_map() { fn fallback_does_not_collide_with_partial_map() {
// Map claims audio "a1" -> 1. The unmapped audio "a0" must NOT // Map claims audio "a1" -> 1. The unmapped audio "a0" must NOT
@@ -352,7 +242,7 @@ mod tests {
info("a1", StreamLabelType::Audio), // mapped → 1 info("a1", StreamLabelType::Audio), // mapped → 1
info("a2", StreamLabelType::Audio), // unmapped → fallback info("a2", StreamLabelType::Audio), // unmapped → fallback
]; ];
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"); let nums = assign_stream_numbers(&infos, &map);
// a0 skips the taken 1 → 2; a1 keeps 1; a2 → 3. All distinct. // a0 skips the taken 1 → 2; a1 keeps 1; a2 → 3. All distinct.
assert_eq!(nums, vec![2, 1, 3]); assert_eq!(nums, vec![2, 1, 3]);
let mut sorted = nums.clone(); let mut sorted = nums.clone();
@@ -370,10 +260,7 @@ mod tests {
info("a0", StreamLabelType::Audio), info("a0", StreamLabelType::Audio),
info("a1", StreamLabelType::Audio), info("a1", StreamLabelType::Audio),
]; ];
assert_eq!( assert_eq!(assign_stream_numbers(&infos, &map), vec![5, 9]);
assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"),
vec![5, 9]
);
} }
// ── Additional hardening tests ───────────────────────────────────────── // ── Additional hardening tests ─────────────────────────────────────────
@@ -389,8 +276,7 @@ mod tests {
info("a1", StreamLabelType::Audio), info("a1", StreamLabelType::Audio),
info("s1", StreamLabelType::Subtitle), info("s1", StreamLabelType::Subtitle),
]; ];
let nums = let nums = assign_stream_numbers(&infos, &HashMap::new());
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Audio: 1, 2; Subtitle: 1, 2 — each counter resets at 1 per type. // Audio: 1, 2; Subtitle: 1, 2 — each counter resets at 1 per type.
assert_eq!(nums[0], 1); // audio 1 assert_eq!(nums[0], 1); // audio 1
assert_eq!(nums[1], 1); // subtitle 1 assert_eq!(nums[1], 1); // subtitle 1
@@ -407,7 +293,7 @@ mod tests {
let mut map = HashMap::new(); let mut map = HashMap::new();
map.insert("a0".to_string(), 0u16); // 0 must not be treated as a claim map.insert("a0".to_string(), 0u16); // 0 must not be treated as a claim
let infos = vec![info("a0", StreamLabelType::Audio)]; let infos = vec![info("a0", StreamLabelType::Audio)];
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"); let nums = assign_stream_numbers(&infos, &map);
// 0 is treated as unmapped → the fallback counter assigns 1. // 0 is treated as unmapped → the fallback counter assigns 1.
assert_eq!(nums[0], 1); assert_eq!(nums[0], 1);
} }
@@ -423,7 +309,7 @@ mod tests {
info("real", StreamLabelType::Audio), info("real", StreamLabelType::Audio),
info("bad", StreamLabelType::Audio), info("bad", StreamLabelType::Audio),
]; ];
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"); let nums = assign_stream_numbers(&infos, &map);
assert_eq!(nums[0], 1); // the genuinely-mapped stream keeps 1 assert_eq!(nums[0], 1); // the genuinely-mapped stream keeps 1
assert_eq!(nums[1], 2); // the 0-stream is synthesized to the next free slot assert_eq!(nums[1], 2); // the 0-stream is synthesized to the next free slot
} }
@@ -440,74 +326,16 @@ mod tests {
info("a0", StreamLabelType::Audio), // fallback info("a0", StreamLabelType::Audio), // fallback
info("s0", StreamLabelType::Subtitle), // mapped → 2 info("s0", StreamLabelType::Subtitle), // mapped → 2
]; ];
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"); let nums = assign_stream_numbers(&infos, &map);
// Audio fallback for a0 → 1 (subtitle's taken-2 doesn't block it). // Audio fallback for a0 → 1 (subtitle's taken-2 doesn't block it).
assert_eq!(nums[0], 1); assert_eq!(nums[0], 1);
assert_eq!(nums[1], 2); assert_eq!(nums[1], 2);
} }
/// A crafted `streamproperties.xml` can drive the fallback counter to the /// Spec: saturating_add prevents overflow when many streams are listed.
/// top of the 1-based u16 stream-number space and then present one more /// Mutation: use wrapping_add → counter wraps to 0 and collides.
/// unmapped stream whose successor number is also claimed by the map.
///
/// This must TERMINATE. The bound is the numbering space itself, so the
/// assertion is on the spec-derived exhaustion behaviour (`None`), not on
/// any tunable constant. Run on a worker thread with a deadline so a
/// non-terminating loop fails the test in 20 s instead of hanging CI.
#[test] #[test]
fn exhausted_numbering_terminates_instead_of_looping() { fn assign_stream_numbers_saturation_on_overflow() {
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
// One mapped audio stream claims the last number in the space.
let mut map = HashMap::new();
map.insert("claims_max".to_string(), u16::MAX);
let mut infos = vec![info("claims_max", StreamLabelType::Audio)];
// Enough unmapped audio streams to walk the counter to the top.
for i in 0..=(u16::MAX as u32) {
infos.push(info(&format!("u{i}"), StreamLabelType::Audio));
}
let _ = tx.send(assign_stream_numbers(&infos, &map));
});
match rx.recv_timeout(std::time::Duration::from_secs(20)) {
Ok(result) => {
worker.join().expect("worker panicked");
assert!(
result.is_none(),
"an exhausted 1-based u16 numbering space must fail the parse, \
not emit colliding or wrapped stream numbers"
);
}
Err(_) => panic!(
"assign_stream_numbers did not terminate within 20s — \
non-terminating skip loop on crafted stream_map"
),
}
}
/// The whole 1-based u16 space must remain usable: 65535 unmapped audio
/// streams get 65535 distinct numbers with no panic and no wrap. The
/// literals here are the JVMS-independent, spec-derived size of a u16
/// 1-based numbering domain, not a tunable cap.
#[test]
fn full_u16_numbering_space_is_usable_and_unique() {
let infos: Vec<StreamInfo> = (0..65_535u32)
.map(|i| info(&format!("a{i}"), StreamLabelType::Audio))
.collect();
let nums = assign_stream_numbers(&infos, &HashMap::new()).expect("space is not exhausted");
assert_eq!(nums.len(), 65_535);
assert_eq!(nums[0], 1);
assert_eq!(nums[65_534], 65_535);
let mut sorted = nums.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), 65_535, "stream numbers must all be distinct");
}
/// Spec: a partially-mapped playlist with many claimed numbers must still
/// synthesize past every claim without panicking or colliding.
/// Mutation: drop the skip loop → the fallback reuses a claimed number.
#[test]
fn fallback_skips_a_dense_block_of_claimed_numbers() {
// Force the counter past u16::MAX by pre-taking all values 1..=u16::MAX. // Force the counter past u16::MAX by pre-taking all values 1..=u16::MAX.
// Doing that for real would be slow; instead inject u16::MAX into taken. // Doing that for real would be slow; instead inject u16::MAX into taken.
let mut map = HashMap::new(); let mut map = HashMap::new();
@@ -534,7 +362,7 @@ mod tests {
qualifier: LabelQualifier::None, qualifier: LabelQualifier::None,
}); });
// This must not panic. // This must not panic.
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"); let nums = assign_stream_numbers(&infos, &map);
assert_eq!(nums.len(), 501); assert_eq!(nums.len(), 501);
// The last (unmapped) entry's number must be > 500 (skipped all taken). // The last (unmapped) entry's number must be > 500 (skipped all taken).
assert!(nums[500] > 500); assert!(nums[500] > 500);
+119 -276
View File
@@ -50,10 +50,10 @@ fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
if let Some(mb_match) = mb if let Some(mb_match) = mb
.iter() .iter()
.find(|m| m.stream_type == label.stream_type && m.stream_number == label.stream_number) .find(|m| m.stream_type == label.stream_type && m.stream_number == label.stream_number)
&& label.name.is_empty()
&& !mb_match.name.is_empty()
{ {
label.name = mb_match.name.clone(); if label.name.is_empty() && !mb_match.name.is_empty() {
label.name = mb_match.name.clone();
}
} }
} }
// Append any menu_base-only stream (present in mb but not in ls by // Append any menu_base-only stream (present in mb but not in ls by
@@ -87,21 +87,7 @@ fn prefix_is_commentary(prefix: &str) -> bool {
fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<Vec<StreamLabel>> { fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
let data = super::read_jar_file(reader, udf, "language_streams.txt")?; let data = super::read_jar_file(reader, udf, "language_streams.txt")?;
let text = std::str::from_utf8(&data).ok()?; let text = std::str::from_utf8(&data).ok()?;
let labels = parse_language_streams_text(text);
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels.
///
/// This is the shipping parser: [`parse_language_streams`] does the UDF read
/// and UTF-8 decode and then delegates here. It is split out — rather than
/// duplicated under `#[cfg(test)]`, which is what it used to be — so the unit
/// tests below exercise production code. A test that re-implements the
/// function it guards cannot fail when the real function breaks.
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new(); let mut labels = Vec::new();
for line in text.lines() { for line in text.lines() {
@@ -207,7 +193,122 @@ fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
} }
labels.push(StreamLabel { labels.push(StreamLabel {
stream_id: None, stream_number: stream_num,
stream_type,
language,
name: String::new(),
purpose: final_purpose,
qualifier,
codec_hint,
variant: variant_code,
});
}
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels. Split
/// out from [`parse_language_streams`] so unit tests exercise the real parsing
/// logic without needing a SectorSource / UdfFs.
#[cfg(test)]
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let parts: Vec<&str> = line.split(',').map(|s| s.trim()).collect();
if parts.len() < 4 {
continue;
}
let type_str = parts[1];
let stream_num: u16 = match parts[2].parse() {
Ok(n) if n > 0 => n,
_ => continue,
};
let language = parts[3].to_string();
let variant = if parts.len() > 4 {
parts[4].to_string()
} else {
String::new()
};
let (stream_type, purpose, qualifier) = match type_str {
"audio_production" => (
StreamLabelType::Audio,
LabelPurpose::Normal,
LabelQualifier::None,
),
"audio_commentary" => (
StreamLabelType::Audio,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"audio_ime" => (
StreamLabelType::Audio,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_production" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_commentary" => (
StreamLabelType::Subtitle,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"subtitle_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::Forced,
),
"subtitle_dual" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_bonus" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_ime" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_ime_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::Forced,
),
_ => continue,
};
let mut codec_hint = String::new();
let mut variant_code = String::new();
let mut final_purpose = purpose;
if !variant.is_empty() {
match variant.as_str() {
"eda" => final_purpose = LabelPurpose::Descriptive,
"csp" | "cs" | "lsp" | "ls" | "cf" | "pf" | "bp" | "pp" => {
variant_code = variant.clone();
}
_ => codec_hint = vocab::codec(&variant).to_string(),
}
}
labels.push(StreamLabel {
stream_number: stream_num, stream_number: stream_num,
stream_type, stream_type,
language, language,
@@ -325,68 +426,6 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::None); assert_eq!(labels[0].qualifier, LabelQualifier::None);
} }
/// Spec: `menu_base.prop` lines are skipped when `is_empty() ||
/// starts_with('#')` — either alone is sufficient. A commented-out
/// key=value line must never be parsed into an entry.
/// Mutation: `||` -> `&&` requires both, which a non-empty comment
/// line can't satisfy, so it falls through to `line.find('=')` and
/// gets parsed as a real property.
#[test]
fn menu_base_comment_line_with_equals_is_still_skipped() {
let labels = parse_props(
"#audio_1.class=AudioButton\n\
#audio_1.streamNumber=9\n\
#audio_1.name=Should Not Appear\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=1\n\
audio_2.name=Real Track\n",
);
assert_eq!(labels.len(), 1, "commented-out entry must not be parsed");
assert_eq!(labels[0].name, "Real Track");
}
/// Spec: `menu_base.prop` streamNumber (or audioStream/subtitleStream)
/// must be strictly positive — `0` means "no STN entry" and must be
/// skipped, matching the `n > 0` guard on the language_streams side.
/// Mutation: `n > 0` -> `n >= 0` (or the guard deleted) would let a
/// stream_num of 0 through, emitting a dead label apply_labels can
/// never match (its counter starts at 1).
#[test]
fn menu_base_zero_stream_number_skipped() {
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_1.name=Disabled Slot\n",
);
assert!(
labels.is_empty(),
"streamNumber=0 must be skipped, got {labels:?}"
);
}
/// Spec: `is_subtitle` is `class.contains("SubtitleButton") ||
/// prefix.starts_with("subtitle_")` — EITHER signal alone is
/// sufficient to classify (and keep) a subtitle entry whose prefix
/// doesn't follow the `subtitle_` naming convention.
/// Mutation: `||` -> `&&` would require BOTH signals; an entry whose
/// class says SubtitleButton but whose prefix is something else
/// (e.g. a vendor-specific button id) would then satisfy neither
/// `is_audio` nor `is_subtitle` and get dropped entirely.
#[test]
fn menu_base_subtitle_class_alone_is_sufficient() {
let labels = parse_props(
"menuBtn7.class=SubtitleButton\n\
menuBtn7.streamNumber=1\n\
menuBtn7.name=English SDH\n",
);
assert_eq!(
labels.len(),
1,
"class=SubtitleButton alone must classify as subtitle, not be dropped"
);
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
}
#[test] #[test]
fn prefix_commentary_segment_match_not_substring() { fn prefix_commentary_segment_match_not_substring() {
// Genuine commentary group segments match. // Genuine commentary group segments match.
@@ -402,7 +441,6 @@ mod tests {
fn lbl(t: StreamLabelType, n: u16, name: &str) -> StreamLabel { fn lbl(t: StreamLabelType, n: u16, name: &str) -> StreamLabel {
StreamLabel { StreamLabel {
stream_id: None,
stream_number: n, stream_number: n,
stream_type: t, stream_type: t,
language: String::new(), language: String::new(),
@@ -414,39 +452,6 @@ mod tests {
} }
} }
/// Spec: `merge`'s `mb.iter().find(...)` must match an mb entry by
/// (stream_type AND stream_number) TOGETHER — either alone is not a
/// unique key (there can be an audio #1 and a subtitle #1, or two
/// different audio streams).
/// Mutation: `&&` -> `||` inside the closure would match on type OR
/// number alone, so `.find` (which returns the FIRST match) can pick
/// an mb entry with the right type but the WRONG stream number.
#[test]
fn merge_matches_mb_entry_by_type_and_number_together() {
// ls wants audio #2 (empty name, so it will borrow from mb).
let ls = vec![lbl(StreamLabelType::Audio, 2, "")];
// mb's FIRST audio entry is #1 (wrong number); its #2 entry (the
// real match) comes second.
let mb = vec![
lbl(StreamLabelType::Audio, 1, "Wrong Number Match"),
lbl(StreamLabelType::Audio, 2, "Correct Match"),
];
let merged = merge(ls, mb);
assert_eq!(
merged.len(),
2,
"mb's own audio #1 must also survive as its own entry"
);
let a2 = merged
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio && l.stream_number == 2)
.unwrap();
assert_eq!(
a2.name, "Correct Match",
"must match mb by (type AND number), not type or number alone"
);
}
#[test] #[test]
fn merge_preserves_menu_base_only_streams() { fn merge_preserves_menu_base_only_streams() {
// language_streams covers audio 1; menu_base has audio 1 (name) // language_streams covers audio 1; menu_base has audio 1 (name)
@@ -516,47 +521,6 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::Forced); assert_eq!(labels[0].qualifier, LabelQualifier::Forced);
} }
/// Immunity pin against the defect measured in the `paramount` parser,
/// where a vendor `forced_sub` cell hung off a FULL dialogue track's own
/// slot to say "this track also contains forced signs", and reading that
/// cell as "this track is forced" flagged 30 MB dialogue tracks forced.
///
/// This format cannot express that. The forced signal is not a flag beside
/// a track's entry — it IS the entry's stream-kind token, drawn from a
/// closed vocabulary in which `subtitle_production` (the full dialogue
/// track) and `subtitle_narrative` (the forced-narrative track) are
/// mutually exclusive alternatives in the same position. A row is one or
/// the other; there is no cell a full track can carry to acquire the
/// qualifier, so the paramount failure mode has no encoding here.
///
/// Mutation: give `subtitle_production` a `Forced` qualifier, or add a
/// forced side-flag that both kinds may carry.
#[test]
fn a_full_subtitle_track_kind_can_never_carry_the_forced_qualifier() {
// Every subtitle kind in the vocabulary, one row each.
let text = "id1,subtitle_production,1,eng\n\
id2,subtitle_commentary,2,eng\n\
id3,subtitle_dual,3,eng\n\
id4,subtitle_bonus,4,eng\n\
id5,subtitle_ime,5,kor\n\
id6,subtitle_narrative,6,eng\n\
id7,subtitle_ime_narrative,7,kor\n";
let labels = parse_language_streams_text(text);
let forced: Vec<&str> = labels
.iter()
.filter(|l| l.qualifier == LabelQualifier::Forced)
.map(|l| l.language.as_str())
.collect();
assert_eq!(
forced.len(),
2,
"only the two narrative kinds are forced, got {forced:?}"
);
// The full dialogue kind specifically.
let production = parse_language_streams_text("id,subtitle_production,1,eng\n");
assert_eq!(production[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_commentary` → Subtitle / Commentary. /// Spec: `subtitle_commentary` → Subtitle / Commentary.
/// Mutation: treat as Normal → subtitle commentary not flagged. /// Mutation: treat as Normal → subtitle commentary not flagged.
#[test] #[test]
@@ -600,72 +564,6 @@ mod tests {
assert!(labels.is_empty()); assert!(labels.is_empty());
} }
/// Immunity pin. `language_streams.txt` states each stream's number in
/// field 3, so a row the parser cannot use is simply dropped — it can
/// never renumber the rows behind it. This is the property that keeps
/// this parser out of the STN-slot-shifting failure mode that bites
/// parsers which count positionally: there, a skipped entry silently
/// pulls every later label one stream forward.
///
/// Mutation: replace `parts[2]` with a running per-type counter → the
/// three unusable rows here collapse the survivors onto 1/2 and 1.
#[test]
fn ls_stream_numbers_come_from_the_row_not_a_counter() {
let labels = parse_language_streams_text(
"id,audio_production,4,eng\n\
id,audio_bonus_extended,5,eng\n\
id,audio_production,0,fra\n\
id,audio_production,7,fra\n\
id,subtitle_production\n\
id,subtitle_narrative,9,deu\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 4),
(StreamLabelType::Audio, 7),
(StreamLabelType::Subtitle, 9),
],
"an unusable row drops out without shifting the numbering"
);
assert_eq!(labels[2].qualifier, LabelQualifier::Forced);
}
/// Immunity pin, `menu_base.prop` side: the number comes from the
/// entry's own `streamNumber` property, so a skipped entry (commented
/// out, `streamNumber=0`, neither audio nor subtitle) leaves the
/// surviving entries on their authored slots.
///
/// Mutation: number by iteration order → the survivors collapse to 1/2.
#[test]
fn menu_base_stream_numbers_come_from_the_entry_not_a_counter() {
let labels = parse_props(
"#audio_0.class=AudioButton\n\
#audio_0.streamNumber=1\n\
audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=6\n\
other_1.class=SomeOtherButton\n\
other_1.streamNumber=2\n\
subtitle_1.class=SubtitleButton\n\
subtitle_1.streamNumber=11\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![(StreamLabelType::Audio, 6), (StreamLabelType::Subtitle, 11),],
"skipped entries must not renumber the ones that survive"
);
}
/// Spec: `eda` variant → `Descriptive` purpose. /// Spec: `eda` variant → `Descriptive` purpose.
/// Mutation: miss the `eda` branch → purpose stays Normal. /// Mutation: miss the `eda` branch → purpose stays Normal.
#[test] #[test]
@@ -710,60 +608,6 @@ mod tests {
assert_eq!(labels[0].language, "eng"); assert_eq!(labels[0].language, "eng");
} }
/// Spec: the skip test is `is_empty() || starts_with('#')` — EITHER
/// condition alone must skip the line. A commented-out line that
/// happens to look like valid CSV (a real authoring pattern for
/// disabling a stream entry) must never produce a label.
/// Mutation: `||` -> `&&` requires BOTH conditions, which a non-empty
/// comment line can never satisfy, so it would fall through to the
/// CSV parser and (since it has >= 4 comma fields) emit a spurious
/// label instead of being skipped.
#[test]
fn ls_comment_line_with_csv_shape_is_still_skipped() {
let labels =
parse_language_streams_text("#id,audio_production,1,eng\nid2,audio_production,2,fra\n");
assert_eq!(
labels.len(),
1,
"the commented-out CSV-shaped line must not parse"
);
assert_eq!(labels[0].language, "fra");
}
/// Spec: `subtitle_dual` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → falls to the
/// catch-all `_ => continue`, silently dropping the stream.
#[test]
fn ls_subtitle_dual_parsed() {
let labels = parse_language_streams_text("id,subtitle_dual,1,eng\n");
assert_eq!(labels.len(), 1, "subtitle_dual must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_bonus` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_bonus_parsed() {
let labels = parse_language_streams_text("id,subtitle_bonus,2,eng\n");
assert_eq!(labels.len(), 1, "subtitle_bonus must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
}
/// Spec: `subtitle_ime` maps to Subtitle/Ime (no Forced qualifier,
/// unlike `subtitle_ime_narrative`).
/// Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_ime_parsed() {
let labels = parse_language_streams_text("id,subtitle_ime,3,jpn\n");
assert_eq!(labels.len(), 1, "subtitle_ime must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Ime);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: multiple valid lines produce multiple labels. /// Spec: multiple valid lines produce multiple labels.
/// Mutation: stop after first label → only 1 label returned. /// Mutation: stop after first label → only 1 label returned.
#[test] #[test]
@@ -912,7 +756,6 @@ fn parse_menu_base_text(text: &str) -> Vec<StreamLabel> {
.unwrap_or_default(); .unwrap_or_default();
labels.push(StreamLabel { labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num, stream_number: stream_num,
stream_type, stream_type,
language, language,
+8 -265
View File
@@ -93,45 +93,6 @@ fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
out out
} }
/// Cap on the bytes retained for one stream label.
///
/// The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
/// whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
/// constant contributes up to 65535 bytes, and the `u16` stream-number
/// keyspace admits 65536 of them per type.
///
/// Headroom: real dbp menu labels are short display names — "English Dolby
/// Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
/// retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
/// bytes. 256 leaves >5x headroom over that, and any string past it is menu
/// geometry or padding, never a language name — `vocab::lang` would not
/// resolve it anyway.
const MAX_LABEL_BYTES: usize = 256;
/// Cap on retained stream slots per type.
///
/// The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
/// type are reachable; paired with [`MAX_LABEL_BYTES`] this bounds the whole
/// scan at 2 x 512 x 256 bytes.
///
/// Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
/// streams per playlist, and dbp emits one menu TextField per stream. 512
/// leaves 16x headroom over the spec maximum.
const MAX_LABELS_PER_TYPE: usize = 512;
/// Record `label` for stream `n`, honouring the retention caps. Existing
/// slots are still overwritten at the cap so the documented last-write-wins
/// behaviour is preserved; only NEW slots are refused.
fn retain_label(map: &mut BTreeMap<u16, String>, n: u16, label: &str) {
if label.len() > MAX_LABEL_BYTES {
return;
}
if map.len() >= MAX_LABELS_PER_TYPE && !map.contains_key(&n) {
return;
}
map.insert(n, label.to_string());
}
fn collect_textfield( fn collect_textfield(
s: &str, s: &str,
audios: &mut BTreeMap<u16, String>, audios: &mut BTreeMap<u16, String>,
@@ -151,15 +112,15 @@ fn collect_textfield(
} }
if let Some(rest) = kind_n.strip_prefix("Audio") { if let Some(rest) = kind_n.strip_prefix("Audio") {
if let Ok(n) = rest.parse::<u16>() { if let Ok(n) = rest.parse::<u16>() {
retain_label(audios, n, label); audios.insert(n, label.to_string());
} }
} else if let Some(rest) = kind_n.strip_prefix("Subtitle") } else if let Some(rest) = kind_n.strip_prefix("Subtitle") {
&& let Ok(n) = rest.parse::<u16>() if let Ok(n) = rest.parse::<u16>() {
{ // Subtitle0 is conventionally the "None / Off" disable
// Subtitle0 is conventionally the "None / Off" disable // button, not an actual subtitle stream.
// button, not an actual subtitle stream. if n > 0 {
if n > 0 { subs.insert(n, label.to_string());
retain_label(subs, n, label); }
} }
} }
} }
@@ -171,7 +132,6 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
let qualifier = vocab::qualifier(&label); let qualifier = vocab::qualifier(&label);
let purpose = vocab::purpose(&label); let purpose = vocab::purpose(&label);
StreamLabel { StreamLabel {
stream_id: None,
stream_number: num, stream_number: num,
stream_type, stream_type,
language, language,
@@ -187,223 +147,6 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
mod tests { mod tests {
use super::super::{LabelPurpose, LabelQualifier}; use super::super::{LabelPurpose, LabelQualifier};
use super::*; use super::*;
use std::io::{Cursor, Write as _};
/// Build a minimal, structurally valid `.class` file (JVMS §4.1) whose
/// constant pool holds exactly the given `Utf8` strings (indices 1..=N,
/// no long/double slot padding needed for plain strings). No fields,
/// methods, interfaces, or attributes — `scan_jar`'s only interest is
/// the constant pool.
fn build_class(utf8_entries: &[&str]) -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(&0xCAFEBABEu32.to_be_bytes()); // magic
out.extend_from_slice(&0u16.to_be_bytes()); // minor_version
out.extend_from_slice(&52u16.to_be_bytes()); // major_version (Java 8)
out.extend_from_slice(&((utf8_entries.len() + 1) as u16).to_be_bytes()); // cp_count
for s in utf8_entries {
out.push(1); // CONSTANT_Utf8 tag
out.extend_from_slice(&(s.len() as u16).to_be_bytes());
out.extend_from_slice(s.as_bytes());
}
out.extend_from_slice(&0u16.to_be_bytes()); // access_flags
out.extend_from_slice(&0u16.to_be_bytes()); // this_class
out.extend_from_slice(&0u16.to_be_bytes()); // super_class
out.extend_from_slice(&0u16.to_be_bytes()); // interfaces_count
out.extend_from_slice(&0u16.to_be_bytes()); // fields_count
out.extend_from_slice(&0u16.to_be_bytes()); // methods_count
out.extend_from_slice(&0u16.to_be_bytes()); // attributes_count
out
}
/// Zip `entries` (name -> bytes) into an in-memory, Stored (uncompressed)
/// `jar::Jar` via the `zip` crate's own writer — a real archive, not a
/// hand-rolled central directory.
fn build_jar(entries: &[(&str, Vec<u8>)]) -> jar::Jar {
let mut buf = Vec::new();
{
let mut writer = zip::ZipWriter::new(Cursor::new(&mut buf));
let opts = zip::write::SimpleFileOptions::default()
.compression_method(zip::CompressionMethod::Stored);
for (name, data) in entries {
writer.start_file(*name, opts).expect("start_file");
writer.write_all(&data[..]).expect("write class bytes");
}
writer.finish().expect("finish zip");
}
zip::ZipArchive::new(Cursor::new(buf)).expect("valid zip")
}
/// `scan_jar` wires together `for_each_class`, constant-pool iteration,
/// `collect_textfield`, and `make_label` into the actual per-jar scan
/// used by `parse`. The pure `collect_textfield`/`make_label` unit
/// tests above don't exercise this wiring at all.
///
/// Mutation: replace the whole function body with `vec![]` — every
/// dbp disc would silently lose all its stream labels regardless of
/// what's in the jar.
#[test]
fn scan_jar_extracts_labels_from_real_class_entries() {
let class_bytes = build_class(&[
"com/dbp/Whatever", // unrelated string — must be ignored
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
"HTextField,Subtitle1,English SDH,Fontstrip_Composite,1312,763",
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843", // disable button, skipped
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
assert_eq!(
labels.len(),
2,
"expected one audio + one real subtitle label"
);
let audio = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio)
.expect("audio label present");
assert_eq!(audio.stream_number, 1);
assert_eq!(audio.language, "eng");
let sub = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Subtitle)
.expect("subtitle label present");
assert_eq!(sub.stream_number, 1);
assert_eq!(sub.qualifier, LabelQualifier::Sdh);
}
/// Immunity pin. Every dbp label states its own slot in the `AudioN` /
/// `SubtitleN` token, so the numbering survives gaps and skipped entries
/// intact. Nothing here counts positionally, which is what keeps this
/// parser out of the failure mode where a skipped entry pulls every later
/// label one stream forward.
///
/// Mutation: number by iteration order → `Audio4` becomes 2 and
/// `Subtitle3` becomes 1, silently rebinding both to other streams.
#[test]
fn stream_numbers_come_from_the_token_not_iteration_order() {
let class_bytes = build_class(&[
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
// Slots 2 and 3 have no menu TextField authored.
"LTextField,Audio4,French 5.1 Dolby Digital,Fontstrip_Composite,296,803",
// Not a stream: the disable-subtitles button.
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843",
// Unparseable slot token — dropped, and must shift nothing.
"HTextField,SubtitleX,German,Fontstrip_Composite,1312,883",
"HTextField,Subtitle3,English SDH,Fontstrip_Composite,1312,763",
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 1),
(StreamLabelType::Audio, 4),
(StreamLabelType::Subtitle, 3),
],
"unlabelled and unusable slots leave the authored numbers alone"
);
}
/// A `CONSTANT_Utf8_info` carries a `u16` length (JVMS §4.4.7), so one
/// crafted constant contributes up to 65535 bytes and the `u16` stream
/// keyspace admits 65536 slots per type — ~4 GiB of retained `String` per
/// map from a jar that is orders of magnitude smaller.
///
/// Boundary literals, not the constant: a 256-byte label is kept, 257 and
/// the JVMS maximum 65535 are refused.
#[test]
fn oversized_labels_are_not_retained() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
collect_textfield(
&format!("XTextField,Audio1,{},rest", "A".repeat(256)),
&mut audios,
&mut subs,
);
assert_eq!(
audios.get(&1).map(String::len),
Some(256),
"a 256-byte label must still be retained"
);
collect_textfield(
&format!("XTextField,Audio2,{},rest", "A".repeat(257)),
&mut audios,
&mut subs,
);
assert!(!audios.contains_key(&2), "a 257-byte label must be refused");
collect_textfield(
&format!("XTextField,Subtitle1,{},rest", "B".repeat(65_535)),
&mut audios,
&mut subs,
);
assert!(
!subs.contains_key(&1),
"a JVMS-maximum 65535-byte Utf8 label must be refused"
);
}
/// The stream-slot keyspace is the full `u16` on both maps. Offer 600
/// distinct audio slots; exactly 512 are retained.
#[test]
fn retained_stream_slots_are_capped_per_type() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
assert_eq!(
audios.len(),
512,
"600 audio slots offered, {} retained — the slot count is unbounded",
audios.len()
);
}
/// Reaching the slot cap must not break the documented last-write-wins
/// behaviour for slots already held.
#[test]
fn existing_slot_is_still_overwritten_at_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
collect_textfield("XTextField,Audio1,Spanish,rest", &mut audios, &mut subs);
assert_eq!(audios.get(&1).map(String::as_str), Some("Spanish"));
}
/// Headroom: the longest plausible retail label must survive untouched.
#[test]
fn longest_realistic_label_survives_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
let real = "Portuguese (Brazilian) 5.1 Dolby Digital Plus";
assert_eq!(real.len(), 45, "fixture length changed");
collect_textfield(
&format!("XTextField,Audio1,{real},Fontstrip_Composite,296,763"),
&mut audios,
&mut subs,
);
assert_eq!(audios.get(&1).map(String::as_str), Some(real));
}
#[test] #[test]
fn collect_extracts_audio_and_subtitle_indices() { fn collect_extracts_audio_and_subtitle_indices() {
+39 -1632
View File
File diff suppressed because it is too large Load Diff
-21
View File
@@ -216,27 +216,6 @@ mod tests {
ZipArchive::new(Cursor::new(bytes)).expect("valid zip") ZipArchive::new(Cursor::new(bytes)).expect("valid zip")
} }
/// The doc comment states the cap is 64 MiB. Pin the exact numeric
/// value (not derived from the same `64 * 1024 * 1024` expression
/// under test — a hardcoded literal) so a mutation of the arithmetic
/// (e.g. `*` -> `+`) is caught even though no test builds an actual
/// 64 MiB buffer.
#[test]
fn max_class_bytes_is_64_mebibytes() {
assert_eq!(MAX_CLASS_BYTES, 67_108_864);
}
#[test]
fn has_path_prefix_matches_only_declared_prefix() {
let jar = open(build_stored_zip(
"com/dbp/Loader.class",
MINIMAL_CLASS,
MINIMAL_CLASS.len() as u32,
));
assert!(has_path_prefix(&jar, "com/dbp/"));
assert!(!has_path_prefix(&jar, "com/bydeluxe/"));
}
#[test] #[test]
fn try_each_class_reads_minimal_class() { fn try_each_class_reads_minimal_class() {
let mut jar = open(build_stored_zip( let mut jar = open(build_stored_zip(
+224 -1928
View File
File diff suppressed because it is too large Load Diff
+117 -243
View File
@@ -61,7 +61,22 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None; return None;
} }
let mut playlists: Vec<crate::mpls::Playlist> = Vec::new(); let mut labels: Vec<StreamLabel> = Vec::new();
// (stream_type_tag, language, codec_hint, pid) — PID is the
// canonical "same physical stream" key; type+lang+codec round
// out the rare case where two distinct logical streams happen
// to share a PID across playlists with different metadata.
let mut seen: Vec<(StreamLabelType, String, String, u16)> = Vec::new();
// Global 1-based counters keyed by StreamLabelType. Incremented
// only when an entry survives dedup, so stream_numbers are dense
// (1, 2, 3, ...) per type across the whole disc — not reset per
// playlist. A disc with 2 MPLS files that each list the same
// 8 audio streams ends up with audio_1..audio_8, not audio_1..
// audio_16 or audio_1..audio_8 with audio_1 duplicated.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for name in &mpls_names { for name in &mpls_names {
let path = format!("/BDMV/PLAYLIST/{}", name); let path = format!("/BDMV/PLAYLIST/{}", name);
let Ok(data) = udf.read_file(reader, &path) else { let Ok(data) = udf.read_file(reader, &path) else {
@@ -70,66 +85,28 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let Ok(playlist) = crate::mpls::parse(&data) else { let Ok(playlist) = crate::mpls::parse(&data) else {
continue; continue;
}; };
playlists.push(playlist);
}
let labels = build_labels(&playlists);
if labels.is_empty() {
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
Some(ParseResult::low(labels))
}
/// Convert every stream entry across `playlists` into one [`StreamLabel`] per
/// physical stream. Factored out of [`parse`] so unit tests can drive the
/// actual conversion logic (stream-type mapping, identity, slot numbering)
/// directly from already-parsed [`crate::mpls::Playlist`] values, without
/// needing a synthetic on-disc UDF image.
///
/// Identity is `(clip, PID)` — what the STN entry states — and it is both the
/// dedup key and the label's [`StreamId`]. A stream twenty playlists list is
/// one label; two clips that both open their first audio at 0x1100 are two.
/// This replaced a disc-global dense counter that numbered surviving entries
/// 1, 2, 3, … in playlist-directory order: that number was not an STN slot in
/// anything, but it was handed to a binder that reads `stream_number` as one.
fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
use std::collections::HashSet;
let mut labels: Vec<StreamLabel> = Vec::new();
let mut seen: HashSet<super::StreamId> = HashSet::new();
for playlist in playlists {
// `Playlist::streams` is the FIRST play item's STN table, so every
// entry here is a stream of that play item's clip — the same clip
// `disc::bluray` records as the title's `clips[0]`. That pairing is
// what makes the PID an identity rather than a 16-bit number.
//
// Streams cannot be non-empty without a play item to have read them
// from, so the empty case is unreachable on a real disc; entries we
// cannot identify are skipped rather than emitted as unbindable
// labels.
let Some(clip_id) = playlist.play_items.first().map(|pi| pi.clip_id.clone()) else {
continue;
};
// 1-based STN slot within THIS playlist's table, per type — the
// `stream_number` field's documented meaning, counted the same way
// `disc::bluray` counts the stream list it builds from these entries.
// Nothing binds through it (these labels bind by id); it is stated
// truthfully rather than invented so that a reader of the label list
// sees where on its own playlist each stream sits.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for entry in &playlist.streams { for entry in &playlist.streams {
let Some(label_type) = label_type_for(entry) else { let label_type = match entry.stream_type {
continue; 2 | 5 => StreamLabelType::Audio, // primary + secondary audio
3 => StreamLabelType::Subtitle, // PG subtitle
// 1 = primary video, 6 = secondary video, 7 = DV EL
// → no StreamLabelType variant for video, skip.
// 4 = IG (interactive graphics) — not a user-facing
// stream, skip.
_ => continue,
}; };
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
let key = (label_type, language.clone(), codec_hint.clone(), entry.pid);
if seen.contains(&key) {
continue;
}
seen.push(key);
let stream_number = match label_type { let stream_number = match label_type {
StreamLabelType::Audio => { StreamLabelType::Audio => {
audio_idx += 1; audio_idx += 1;
@@ -141,20 +118,7 @@ fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
} }
}; };
let stream_id = super::StreamId {
clip_id: clip_id.clone(),
pid: entry.pid,
};
if !seen.insert(stream_id.clone()) {
continue;
}
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
labels.push(StreamLabel { labels.push(StreamLabel {
stream_id: Some(stream_id),
stream_number, stream_number,
stream_type: label_type, stream_type: label_type,
language, language,
@@ -166,38 +130,17 @@ fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
}); });
} }
} }
labels
}
/// Which per-type numbering list an STN entry belongs to, or `None` when it if labels.is_empty() {
/// is not a labellable stream at all.
///
/// This MUST agree with the stream list `disc::bluray` builds from the same
/// entries, because that list is what `labels::apply_labels` counts against
/// when it binds `stream_number`. The two counters run over the same STN
/// entries in the same order, so any entry one side keeps and the other drops
/// — or files under a different type — shifts every later label of that type
/// onto the wrong stream. Three rules, all mirroring `disc::bluray`:
///
/// * `coding_type == 0` is the STN table's empty/padding slot. Not a
/// stream on either side.
/// * a PG coding_type in an audio STN slot is a subtitle, not audio.
/// `mpls::parse_stream_entry` has a dedicated arm for this layout, so it
/// is an authored shape rather than a corruption.
/// * video (1 / 6 / 7 = primary, secondary, Dolby Vision EL) and IG (4)
/// have no `StreamLabelType`; they are numbered in their own STN lists
/// and never interleave with the audio or PG lists.
fn label_type_for(entry: &crate::mpls::StreamEntry) -> Option<StreamLabelType> {
use crate::consts::coding_type as c;
if entry.coding_type == 0 {
return None; return None;
} }
match entry.stream_type {
2 | 5 if entry.coding_type == c::PG => Some(StreamLabelType::Subtitle), // MPLS gives language + codec but never editorial info (no
2 | 5 => Some(StreamLabelType::Audio), // commentary/SDH/director's cut). Low confidence means framework
3 => Some(StreamLabelType::Subtitle), // parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
_ => None, // win when they match. MPLS only gets chosen as the parser when
} // nothing else fired — exactly the universal-fallback role we want.
Some(ParseResult::low(labels))
} }
fn has_mpls_extension(name: &str) -> bool { fn has_mpls_extension(name: &str) -> bool {
@@ -384,37 +327,69 @@ mod tests {
} }
} }
/// A playlist over clip "00001". `Playlist::streams` is read out of the
/// first play item's STN table, so a playlist that has streams always has
/// a play item to have read them from — the fixture carries one so tests
/// exercise the shape production sees, and so each label gets the
/// `(clip, PID)` identity it is bound by.
fn playlist_with(streams: Vec<StreamEntry>) -> Playlist { fn playlist_with(streams: Vec<StreamEntry>) -> Playlist {
playlist_on("00001", streams)
}
fn playlist_on(clip_id: &str, streams: Vec<StreamEntry>) -> Playlist {
Playlist { Playlist {
version: "0200".to_string(), version: "0200".to_string(),
play_items: vec![crate::mpls::PlayItem { play_items: Vec::new(),
clip_id: clip_id.to_string(),
in_time: 0,
out_time: 0,
connection_condition: 1,
}],
streams, streams,
marks: Vec::new(), marks: Vec::new(),
} }
} }
/// Drive the actual production conversion logic (`build_labels`, the /// Drive the same conversion logic that `parse()` runs on real
/// function `parse()` calls) starting from already-parsed Playlists, /// disc data, but starting from already-parsed Playlists so we
/// so tests don't have to synthesize valid on-disc MPLS/UDF bytes. /// don't have to synthesize valid MPLS bytes.
/// This calls the *real* code under test rather than a hand-written
/// re-implementation, so mutations inside `build_labels` (stream-type
/// mapping, dedup key, counters) are actually caught here.
fn labels_from_playlists(playlists: &[Playlist]) -> Vec<StreamLabel> { fn labels_from_playlists(playlists: &[Playlist]) -> Vec<StreamLabel> {
build_labels(playlists) let mut labels: Vec<StreamLabel> = Vec::new();
let mut seen: Vec<(StreamLabelType, String, String, u16)> = Vec::new();
// Global counters hoisted OUT of the playlist loop to match
// production `parse()` (lines 77-78): stream_numbers are dense
// per type across the whole disc, not reset per playlist.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for playlist in playlists {
for entry in &playlist.streams {
let label_type = match entry.stream_type {
2 | 5 => StreamLabelType::Audio,
3 => StreamLabelType::Subtitle,
_ => continue,
};
// Dedup BEFORE consuming a counter value, matching prod
// parse() ordering so a deduped duplicate does not burn a
// stream number.
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
let key = (label_type, language.clone(), codec_hint.clone(), entry.pid);
if seen.contains(&key) {
continue;
}
seen.push(key);
let stream_number = match label_type {
StreamLabelType::Audio => {
audio_idx += 1;
audio_idx
}
StreamLabelType::Subtitle => {
sub_idx += 1;
sub_idx
}
};
labels.push(StreamLabel {
stream_number,
stream_type: label_type,
language,
name,
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::None,
codec_hint,
variant: String::new(),
});
}
}
labels
} }
#[test] #[test]
@@ -470,114 +445,31 @@ mod tests {
assert_eq!(labels[2].language, "fra"); assert_eq!(labels[2].language, "fra");
} }
/// `stream_number` is bound by `labels::apply_labels` against the title's
/// own stream list, which `disc::bluray` builds from these same STN
/// entries. That builder DROPS an entry whose `coding_type` is 0 — the
/// STN table's empty/padding slot — so it must not be counted here
/// either. Counting it advances the audio counter past a stream that
/// never materializes, and every label behind it binds one stream late.
#[test] #[test]
fn padding_stn_entry_does_not_consume_a_label_slot() { fn dedup_streams_across_playlists() {
let pl = playlist_with(vec![ // Two playlists, same English TrueHD 7.1 PID 0x1100 in both.
// Expect one Audio label, not two.
let pl1 = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), audio_entry(0x1100, 0x83, 12, 1, "eng"),
// coding_type 0: STN padding. Not a stream.
audio_entry(0x1101, 0x00, 0, 0, ""),
audio_entry(0x1102, 0x81, 6, 1, "fra"),
]);
let labels = labels_from_playlists(&[pl]);
assert_eq!(labels.len(), 2, "the padding slot yields no label");
assert_eq!(labels[0].language, "eng");
assert_eq!(labels[0].stream_number, 1);
assert_eq!(labels[1].language, "fra");
assert_eq!(
labels[1].stream_number, 2,
"padding is absent from the title's stream list, so `fra` is \
audio stream 2"
);
}
/// A PG coding_type sitting in an audio STN slot is a real, documented
/// shape — `mpls::parse_stream_entry` has an explicit arm for it, and
/// `disc::bluray` builds it as a Subtitle stream, not an Audio one. This
/// module must classify it the same way, or the audio counter runs one
/// ahead and the subtitle counter one behind for every later stream.
#[test]
fn pg_coding_type_in_an_audio_slot_counts_as_a_subtitle() {
let mut misplaced = audio_entry(0x1200, 0x90, 0, 0, "spa");
misplaced.stream_type = 2;
let pl = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
misplaced,
audio_entry(0x1101, 0x81, 6, 1, "fra"), audio_entry(0x1101, 0x81, 6, 1, "fra"),
pg_entry(0x1201, "deu"),
]); ]);
let labels = labels_from_playlists(&[pl]); let pl2 = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), // duplicate
let audio: Vec<_> = labels audio_entry(0x1102, 0x82, 6, 1, "deu"), // new
.iter() ]);
.filter(|l| l.stream_type == StreamLabelType::Audio)
.map(|l| (l.language.as_str(), l.stream_number))
.collect();
assert_eq!(
audio,
vec![("eng", 1), ("fra", 2)],
"the PG entry is not an audio stream and must not number one"
);
let sub: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.map(|l| (l.language.as_str(), l.stream_number))
.collect();
assert_eq!(
sub,
vec![("spa", 1), ("deu", 2)],
"it is subtitle stream 1, ahead of the PG-slot entry"
);
}
/// Two playlists over the SAME clip that both list PID 0x1100: one
/// physical stream, so one label. Identity is `(clip, PID)`, and each
/// label states the STN slot it holds in its own playlist.
#[test]
fn one_label_per_stream_across_playlists_on_the_same_clip() {
let pl1 = playlist_on(
"00001",
vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
audio_entry(0x1101, 0x81, 6, 1, "fra"),
],
);
let pl2 = playlist_on(
"00001",
vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), // same stream
audio_entry(0x1102, 0x82, 6, 1, "deu"), // new
],
);
let labels = labels_from_playlists(&[pl1, pl2]); let labels = labels_from_playlists(&[pl1, pl2]);
assert_eq!( // Expected: eng@0x1100, fra@0x1101, deu@0x1102 — three uniques.
labels.len(), assert_eq!(labels.len(), 3);
3, // PID isn't stored on StreamLabel, so assert on the surviving
"eng/fra/deu — the duplicate eng is one stream" // language set instead.
); let mut langs: Vec<String> = labels.iter().map(|l| l.language.clone()).collect();
langs.sort();
assert_eq!(langs, vec!["deu", "eng", "fra"]);
let id = |lang: &str| { // Stream numbers must be DENSE and GLOBAL across playlists, not
labels // reset per playlist. eng (pl1) = 1, fra (pl1) = 2, the duplicate
.iter() // eng in pl2 is deduped (no number consumed), and deu (pl2) = 3.
.find(|l| l.language == lang) // Regression guard for the per-playlist counter-reset divergence.
.and_then(|l| l.stream_id.clone())
.map(|i| (i.clip_id, i.pid))
};
assert_eq!(id("eng"), Some(("00001".into(), 0x1100)));
assert_eq!(id("fra"), Some(("00001".into(), 0x1101)));
assert_eq!(id("deu"), Some(("00001".into(), 0x1102)));
// `stream_number` is the entry's slot in ITS OWN playlist's STN table
// — deu is pl2's second audio, so 2, not "the third distinct stream
// seen while scanning the disc". It used to be the latter: a dense
// disc-global counter that named no table anyone could count against,
// handed to a binder that reads the field as an STN slot.
let num = |lang: &str| { let num = |lang: &str| {
labels labels
.iter() .iter()
@@ -586,25 +478,7 @@ mod tests {
}; };
assert_eq!(num("eng"), Some(1)); assert_eq!(num("eng"), Some(1));
assert_eq!(num("fra"), Some(2)); assert_eq!(num("fra"), Some(2));
assert_eq!(num("deu"), Some(2), "pl2's second audio slot"); assert_eq!(num("deu"), Some(3));
}
/// The same PID in two DIFFERENT clips is two different streams — a PID is
/// only unique within one clip. Deduping on the PID alone (as the old
/// key's `(type, language, codec_hint, pid)` did across clips) collapses
/// them into one label, and the second clip's stream is then described by
/// the first clip's.
#[test]
fn same_pid_in_two_clips_is_two_streams() {
let pl1 = playlist_on("00001", vec![audio_entry(0x1100, 0x83, 12, 1, "eng")]);
let pl2 = playlist_on("00002", vec![audio_entry(0x1100, 0x83, 12, 1, "eng")]);
let labels = labels_from_playlists(&[pl1, pl2]);
assert_eq!(labels.len(), 2, "different clips: two distinct streams");
let clips: Vec<String> = labels
.iter()
.filter_map(|l| l.stream_id.as_ref().map(|i| i.clip_id.clone()))
.collect();
assert_eq!(clips, vec!["00001", "00002"]);
} }
#[test] #[test]
+59 -563
View File
@@ -3,28 +3,18 @@
//! Richest structured format. Complete language lists with forced flags //! Richest structured format. Complete language lists with forced flags
//! and commentary indices per playlist, all in XML attributes. //! and commentary indices per playlist, all in XML attributes.
//! //!
//! NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this
//! file is one authoring house's internal metadata that happens to ship on
//! the pressing. There is nothing to look up: every field meaning here was
//! derived by measuring real discs and cross-checking against per-display-set
//! content. Treat an unfamiliar value as unknown rather than guessing — the
//! disc's own `forced_on_flag` is the only authoritative forced signal.
//!
//! ```xml //! ```xml
//! <playlist name="Feature" id="00222" //! <playlist name="Feature" id="00222"
//! aud="eng,deu,spa,spa,fra" //! aud="eng,deu,spa,spa,fra"
//! sub="eng,eng,zho,ces,dan" //! sub="eng,eng,zho,ces,dan"
//! forced_sub="0,0,0,1,3" //! forced_sub="0,0,0,1,0"
//! aud_com1_idx="10" //! aud_com1_idx="10"
//! sub_com1_idx="23,24,25" /> //! sub_com1_idx="23,24,25" />
//! ``` //! ```
//!
//! `forced_sub` is an ENUMERATION, not a boolean — see [`ForcedSub`].
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml}; use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorSource; use crate::sector::SectorSource;
use crate::udf::UdfFs; use crate::udf::UdfFs;
use std::collections::HashSet;
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool { pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "playlists.xml") super::jar_file_exists(udf, "playlists.xml")
@@ -42,138 +32,11 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
if labels.is_empty() { if labels.is_empty() {
return None; return None;
} }
// High confidence: this format is fully structured and we extract // High confidence: paramount's playlists.xml is fully structured
// every field whose meaning the corpus establishes. "Documented" would // and we extract every documented field.
// be the wrong word — see the module note; nothing about it is.
Some(ParseResult::high(labels)) Some(ParseResult::high(labels))
} }
/// One cell of the `forced_sub` CSV.
///
/// The attribute reads like a boolean and was parsed as one (`cell == "1"` →
/// forced). It is not. Every image in the corpus carrying this vendor's
/// `playlists.xml` — seven distinct discs — uses four values, and decoding
/// three of those discs' feature subtitle tracks and counting every PGS
/// display set separates them into two populations two orders of magnitude
/// apart:
///
/// * `0` — a subtitle track with no forced-narrative content. On the two
/// discs measured that use the flag at all, not one `0` track carried a
/// single `forced_on_flag` display set.
/// * `1` — a FULL DIALOGUE track that additionally contains some
/// forced-narrative signs. On one measured disc, all nine `1` cells are
/// full tracks of 949-1411 display sets, eight of them carrying 5-14
/// flagged sets and the ninth none; that disc has no dedicated forced
/// track at all. On another, all seven `1` cells are full tracks of
/// 1602-1651 display sets carrying 0-31 flagged sets. Reading `1` as
/// forced is what made one language present as two identical full
/// subtitle tracks with one of them flagged forced.
/// * `2` and `3` — a DEDICATED forced-narrative track. These take their own
/// trailing STN slots, one per localized language, duplicating a language
/// that already holds a full track earlier in the list. Measured: the two
/// `2` slots on one disc are 15 and 10 display sets, EVERY one flagged
/// forced, against ~1600 on that disc's full tracks; the four `3` slots on
/// another are 7, 14, 23 and 59 display sets against 1216-2655. What
/// distinguishes `2` from `3` the corpus does not reveal — both sit in the
/// same trailing position, both measure the same shape, and one disc uses
/// each for a different language — so both map alike.
///
/// So the old reading was wrong in BOTH directions: it flagged full dialogue
/// tracks forced, and it discarded the cells that name the real forced tracks.
///
/// The `1` case is deliberately NOT carried through as a weaker "contains
/// forced segments" hint. There is no qualifier for that, and the asymmetry
/// argues against inventing one here: a wrong forced flag on a 30 MB dialogue
/// track is the user-visible defect, while a missing hint costs nothing.
///
/// An unrecognised cell maps to [`ForcedSub::None`] — the conservative
/// direction, since asserting forced is the expensive mistake.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum ForcedSub {
/// No forced-narrative content, or an unrecognised cell.
None,
/// A full dialogue track that also carries forced-narrative segments.
ContainsForcedSegments,
/// A dedicated forced-narrative track.
ForcedNarrative,
}
/// The one number behind every cap in this parser: the highest CSV cell
/// position that can ever be addressed.
///
/// The labelling loops number cells 1-based into a `u16` and `break` at
/// `u16::try_from(i + 1)`, so cell `MAX_COM_INDICES` and everything past it is
/// never visited. Two different things are measured against that, and they are
/// not the same bound:
///
/// - **A VALUE at or beyond it cannot match any cell.** This is what caps the
/// set: values are filtered before insertion, so at most `MAX_COM_INDICES`
/// distinct entries can ever be stored, however long the attribute is. The
/// `HashSet` that replaced a linear scan fixed the LOOKUP cost; this is what
/// fixes the ALLOCATION, and a disc declaring half a billion indices no
/// longer costs half a billion entries.
/// - **A POSITION at or beyond it describes nothing new.** This caps the WORK,
/// not the memory — the value filter already made the set small, but without
/// it every one of those half a billion cells is still split and parsed. It
/// is an early exit at the first position whose contents provably cannot
/// matter, and it is why `forced_sub` — which holds no set at all, and so
/// gets no protection from the value rule — is bounded too.
///
/// Real authoring is nowhere near either limit: the BD STN table admits at
/// most 32 streams per playlist, so nothing legitimate is lost.
const MAX_COM_INDICES: usize = u16::MAX as usize;
/// Parse a `*_com1_idx` attribute into the set the labelling loops query.
///
/// Extracted so the BOUND is observable. Asserting it through
/// `labels_from_feature` is not possible: a `HashSet` collapses repeated
/// values, and an out-of-range index changes no label either way, so such a
/// test passes whether or not the cap exists — an assertion that cannot fail.
/// Returning the set lets a test hand in tens of thousands of DISTINCT
/// unaddressable indices and see them refused.
fn com_indices(attr: Option<String>) -> HashSet<usize> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.filter_map(|i| i.trim().parse().ok())
.filter(|&i| i < MAX_COM_INDICES)
.collect()
})
.unwrap_or_default()
}
/// Parse the `forced_sub` attribute into the cell list the subtitle loop
/// queries — the third attacker-controlled CSV in this file, and the last one
/// that was still unbounded.
///
/// Bounded by POSITION, and it has no other choice: a `*_com1_idx` list holds
/// values that can be filtered, and that filter is what caps its set, but a
/// `forced_sub` cell is a classification of the position it sits at, so there
/// is nothing to filter and nothing else would ever cap this. The Vec is read
/// only as `forced.get(i)` from a loop that stops at `MAX_COM_INDICES`, so
/// every cell past that is unreachable by construction.
///
/// Extracted, like [`com_indices`], so the bound is OBSERVABLE. Through
/// `labels_from_feature` it is not: the subtitle loop cannot reach those cells
/// either, so a label-level assertion passes whether or not the cap exists.
fn forced_subs(attr: Option<String>) -> Vec<ForcedSub> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.map(forced_sub_cell)
.collect()
})
.unwrap_or_default()
}
fn forced_sub_cell(cell: &str) -> ForcedSub {
match cell.trim() {
"1" => ForcedSub::ContainsForcedSegments,
"2" | "3" => ForcedSub::ForcedNarrative,
_ => ForcedSub::None,
}
}
/// Build the stream labels from a single `<playlist .../>` feature /// Build the stream labels from a single `<playlist .../>` feature
/// element. Split out from `parse` so the per-type numbering and /// element. Split out from `parse` so the per-type numbering and
/// commentary/forced-index logic is unit-testable without a /// commentary/forced-index logic is unit-testable without a
@@ -186,33 +49,17 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// aud_com1_idx is a trimmed, comma-separated list of CSV positions // aud_com1_idx is a trimmed, comma-separated list of CSV positions
// (some authoring tools emit whitespace, and multiple commentary // (some authoring tools emit whitespace, and multiple commentary
// tracks are possible) — symmetric with sub_com1_idx below. // tracks are possible) — symmetric with sub_com1_idx below.
// A HashSet, not a Vec: `com_indices` is parsed straight out of an let com_indices: Vec<usize> = xml::attr(feature, "aud_com1_idx")
// attacker-controlled attribute with no length bound and was scanned .map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
// linearly once per stream, so `aud="..."` and `aud_com1_idx="..."` .unwrap_or_default();
// both grown large make this quadratic in the size of one XML file.
// Membership is the only operation performed on it.
let com_indices = com_indices(xml::attr(feature, "aud_com1_idx"));
// The CSV *is* the STN list: one cell per stream, in stream order, // stream_number must match apply_labels' monotonic 1-based
// and `aud_com1_idx` is a 0-based index into those same cells. So // per-type counter, which increments once per *real* stream — so
// `stream_number` is the cell's own 1-based position — NOT a counter // it counts only non-empty slots, not the raw CSV index. The
// that only advances on cells carrying a language. // commentary index comparison stays on the raw CSV index `i`,
// // since aud_com1_idx is positional against the original CSV.
// A cell with an empty language still occupies its STN slot; it just let mut audio_num: u16 = 0;
// has nothing to label. Renumbering the surviving cells 1..N shifts
// every label behind an empty cell one slot forward, which is how a
// marker authored for one stream ends up written onto the stream in
// front of it (see the subtitle side, where the marker is `forced`).
//
// `u16::try_from` rather than `saturating_add`: past the 1-based u16
// numbering space every cell would collapse onto `u16::MAX`, binding
// several streams to one label. Stop emitting instead. Unreachable on
// real media — the BD STN_table admits at most 32 primary audio
// streams per playlist.
for (i, lang) in aud.split(',').enumerate() { for (i, lang) in aud.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
};
let lang = lang.trim(); let lang = lang.trim();
if lang.is_empty() { if lang.is_empty() {
continue; continue;
@@ -222,9 +69,9 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
} else { } else {
LabelPurpose::Normal LabelPurpose::Normal
}; };
audio_num = audio_num.saturating_add(1);
labels.push(StreamLabel { labels.push(StreamLabel {
stream_id: None, stream_number: audio_num,
stream_number,
stream_type: StreamLabelType::Audio, stream_type: StreamLabelType::Audio,
language: lang.to_string(), language: lang.to_string(),
name: String::new(), name: String::new(),
@@ -238,21 +85,18 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse subtitle streams // Parse subtitle streams
if let Some(sub) = xml::attr(feature, "sub") { if let Some(sub) = xml::attr(feature, "sub") {
let forced = forced_subs(xml::attr(feature, "forced_sub")); let forced: Vec<bool> = xml::attr(feature, "forced_sub")
.map(|s| s.split(',').map(|f| f.trim() == "1").collect())
.unwrap_or_default();
// HashSet for the same reason as the audio side above: unbounded let com_indices: Vec<usize> = xml::attr(feature, "sub_com1_idx")
// parsed input, membership-only use, linear scan once per stream. .map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
let com_indices = com_indices(xml::attr(feature, "sub_com1_idx")); .unwrap_or_default();
// As with audio: the cell position IS the STN slot. `forced_sub` and // As with audio: count only non-empty slots for stream_number,
// `sub_com1_idx` are indexed against those same cells, so an empty // but keep com/forced lookups on the raw CSV index `i`.
// cell must not renumber the cells behind it — a forced marker let mut sub_num: u16 = 0;
// authored for one PG slot would otherwise be written onto an
// earlier, full-dialogue subtitle track.
for (i, lang) in sub.split(',').enumerate() { for (i, lang) in sub.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
};
let lang = lang.trim(); let lang = lang.trim();
if lang.is_empty() { if lang.is_empty() {
continue; continue;
@@ -264,17 +108,15 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
LabelPurpose::Normal LabelPurpose::Normal
}; };
// Only a DEDICATED forced-narrative slot earns the forced flag. let qualifier = if forced.get(i).copied().unwrap_or(false) {
// A cell marking a full track as merely containing forced segments LabelQualifier::Forced
// is dropped, not weakened into a forced label (see [`ForcedSub`]). } else {
let qualifier = match forced.get(i).copied().unwrap_or(ForcedSub::None) { LabelQualifier::None
ForcedSub::ForcedNarrative => LabelQualifier::Forced,
ForcedSub::ContainsForcedSegments | ForcedSub::None => LabelQualifier::None,
}; };
sub_num = sub_num.saturating_add(1);
labels.push(StreamLabel { labels.push(StreamLabel {
stream_id: None, stream_number: sub_num,
stream_number,
stream_type: StreamLabelType::Subtitle, stream_type: StreamLabelType::Subtitle,
language: lang.to_string(), language: lang.to_string(),
name: String::new(), name: String::new(),
@@ -300,10 +142,10 @@ fn find_feature_playlist(text: &str) -> Option<String> {
let element = &text[start..end]; let element = &text[start..end];
// Prefer name="Feature" explicitly. // Prefer name="Feature" explicitly.
if let Some(name) = xml::attr(element, "name") if let Some(name) = xml::attr(element, "name") {
&& name.eq_ignore_ascii_case("Feature") if name.eq_ignore_ascii_case("Feature") {
{ return Some(element.to_string());
return Some(element.to_string()); }
} }
// Otherwise pick the one with the most audio streams. Count only // Otherwise pick the one with the most audio streams. Count only
@@ -326,206 +168,6 @@ fn find_feature_playlist(text: &str) -> Option<String> {
mod tests { mod tests {
use super::*; use super::*;
/// Immunity pin, section-boundary half. The pixelogic parser walks a flat
/// string sequence and recognises its feature section's END by marker
/// alone, so a section with no marker behind it runs off into whatever
/// follows and counts it as more STN slots. Nothing here can do that: the
/// stream list is one attribute of one XML element, so its length is the
/// CSV's own cell count and its scope is the element's byte range that
/// `xml::find_element` returns. Text after the element — including the
/// next playlist's own `aud` — is not reachable from it.
///
/// And when the boundary is MISSING the failure is closed, not open:
/// `xml::find_element` needs a matching close tag and yields `None`
/// without one, so an unterminated element ends the walk rather than
/// swallowing the rest of the document.
///
/// Mutation: hand `labels_from_feature` the document instead of the
/// element, or let an unterminated element run to EOF → the bonus
/// playlist's languages join the feature's stream list.
#[test]
fn a_playlists_stream_list_cannot_run_into_the_next_playlist() {
let doc = r#"
<playlist name="Feature" aud="eng,fra" sub="eng,spa" forced_sub="0,1"/>
<playlist name="Bonus" aud="deu,ita,jpn" sub="deu,ita,jpn"/>
"#;
let feature = find_feature_playlist(doc).expect("feature playlist found");
let labels = labels_from_feature(&feature);
let got: Vec<(StreamLabelType, u16, &str)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number, l.language.as_str()))
.collect();
assert_eq!(
got,
vec![
(StreamLabelType::Audio, 1, "eng"),
(StreamLabelType::Audio, 2, "fra"),
(StreamLabelType::Subtitle, 1, "eng"),
(StreamLabelType::Subtitle, 2, "spa"),
],
"the CSV's own cells are the whole stream list"
);
// Same document with the feature element left unterminated.
let unterminated = r#"
<playlist name="Feature" aud="eng,fra">
<playlist name="Bonus" aud="deu,ita,jpn"/>
"#;
assert!(
find_feature_playlist(unterminated).is_none(),
"a missing element boundary truncates the walk, never extends it"
);
}
/// `sub_com1_idx` is parsed straight out of the disc's `playlists.xml`,
/// which is attacker-controlled and has no length bound of its own.
///
/// This replaces a WALL-CLOCK test. That one built a 200 000 x 1 000 001
/// fixture and failed if it took over 10 s, to prove the membership test
/// was a set rather than a linear scan. Measured on the machine that
/// wrote this: 1.62 s alone, and OVER 10 s — a real failure — when the
/// suite's other 3 347 tests were running concurrently. A 6x margin
/// against a shared CPU is not a margin; it is a CI failure that looks
/// like a flake and gets re-run until it passes.
///
/// It also measured the wrong thing. Making the lookup O(1) bounded the
/// QUERY, not the PARSE: the set was still built from every entry the
/// disc declared, so a hostile playlist could still force an unbounded
/// allocation before any lookup happened. `MAX_COM_INDICES` bounds that.
///
/// What THIS test guards is that bounding did not change what a
/// legitimate playlist MEANS: it goes red if the bound is set too LOW
/// (verified at 2 — the real indices `0,2,4` stop resolving and the
/// purposes change). It does NOT go red if the bound is deleted
/// entirely, because the out-of-range filler is unobservable at the
/// label level and a `HashSet` collapses the repeats. Enforcement is
/// proven separately, by
/// `distinct_unaddressable_indices_are_refused_not_stored`, which reads
/// the set itself. Two tests, two properties; neither pretends to the
/// other's job.
#[test]
fn bounding_the_parse_does_not_change_a_legitimate_playlist() {
// Three real indices, then far more entries than can address a cell.
const OVERSIZED: usize = MAX_COM_INDICES + 10_000;
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(8));
feature.pop();
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(OVERSIZED));
feature.pop();
feature.push_str(r#"" />"#);
let labels = labels_from_feature(&feature);
// The fixture's real indices still decide the purposes: bounding the
// parse must not change what a legitimate playlist means.
assert_eq!(labels.len(), 8);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
/// The set REFUSES unaddressable indices, so a hostile playlist cannot
/// inflate it. DISTINCT values on purpose: a `HashSet` collapses repeats,
/// so a million copies of one index costs one entry and would prove
/// nothing. Fifty thousand distinct out-of-range indices cost fifty
/// thousand entries without the filter, and none with it — so this test
/// goes red if the bound is removed, which the label-level assertions
/// below cannot do.
#[test]
fn distinct_unaddressable_indices_are_refused_not_stored() {
let hostile: String = (MAX_COM_INDICES..MAX_COM_INDICES + 50_000)
.map(|i| i.to_string())
.collect::<Vec<_>>()
.join(",");
let set = com_indices(Some(hostile));
assert!(
set.is_empty(),
"kept {} unaddressable indices — the parse is still unbounded",
set.len()
);
// The addressable ones are still kept.
assert_eq!(com_indices(Some("0,2,4".to_string())).len(), 3);
}
/// `forced_sub` is bounded too — the third CSV in the same function, and
/// the one that had no value filter to hide behind.
///
/// Read through `forced_subs` rather than through the labels for the same
/// reason the two tests above read the set: the subtitle loop stops at
/// `MAX_COM_INDICES`, so a label-level assertion cannot tell a bounded
/// parse from an unbounded one.
#[test]
fn forced_sub_cells_past_the_last_addressable_one_are_not_parsed() {
let hostile = "0,".repeat(MAX_COM_INDICES + 50_000);
let cells = forced_subs(Some(hostile));
assert_eq!(
cells.len(),
MAX_COM_INDICES,
"parsed {} cells — the forced_sub parse is still unbounded",
cells.len()
);
}
/// Bounding it must not change what a legitimate playlist means: the
/// cells that CAN address a stream still classify exactly as before.
#[test]
fn bounding_forced_sub_leaves_the_addressable_cells_alone() {
let cells = forced_subs(Some("0,1,3,2".to_string()));
assert_eq!(
cells,
vec![
ForcedSub::None,
ForcedSub::ContainsForcedSegments,
ForcedSub::ForcedNarrative,
ForcedSub::ForcedNarrative,
]
);
}
/// An index that cannot address any cell is dropped rather than STORED.
///
/// Asserted through `com_indices`, not through the labels: the labelling
/// loop never queries a cell position that high, so at the label level
/// retaining the value is unobservable and the assertion could not fail.
/// Reading the set is what makes the claim checkable.
#[test]
fn an_index_that_cannot_address_any_cell_is_not_retained() {
let set = com_indices(Some(format!(
"1,{},{}",
MAX_COM_INDICES,
MAX_COM_INDICES + 1
)));
assert_eq!(
set.len(),
1,
"only the addressable index belongs in the set, got {set:?}"
);
assert!(set.contains(&1));
}
/// Headroom: the BD STN_table admits at most 32 PG streams per playlist,
/// and a real `sub_com1_idx` lists a handful of commentary tracks. The set
/// must behave identically to the old scan on real-shaped input.
#[test]
fn commentary_indices_still_match_on_real_shaped_input() {
let feature = r#"<playlist name="Feature" sub="eng,eng,zho,ces,dan" sub_com1_idx="1,3" />"#;
let labels = labels_from_feature(feature);
let purposes: Vec<LabelPurpose> = labels.iter().map(|l| l.purpose).collect();
assert_eq!(
purposes,
vec![
LabelPurpose::Normal,
LabelPurpose::Commentary,
LabelPurpose::Normal,
LabelPurpose::Commentary,
LabelPurpose::Normal,
]
);
}
fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> { fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels labels
.iter() .iter()
@@ -540,56 +182,11 @@ mod tests {
.collect() .collect()
} }
/// The `aud` / `sub` CSVs are the vendor's STN-ordered stream lists: one
/// slot per stream, and `aud_com1_idx` / `forced_sub` are indexed against
/// those same slot positions. A slot whose language cell is empty carries
/// nothing to label but still OCCUPIES its slot, so it must not renumber
/// the slots behind it.
///
/// Numbering only the slots that carry a language collapsed every later
/// label one position forward per empty cell, which is how a forced
/// marker authored for one STN slot lands on the full-subtitle track in
/// front of it.
#[test] #[test]
fn empty_csv_slot_still_occupies_its_stn_slot() { fn empty_middle_slot_does_not_inflate_stream_number() {
// Audio: slot 2 is empty; `fra` is STN slot 3 and is the commentary // aud="eng,,fra": the empty middle slot is skipped, and the
// the vendor pointed at with the 0-based CSV index 2. // second real stream (fra) must be numbered 2, matching
let feature = r#"<playlist name="Feature" aud="eng,,fra" aud_com1_idx="2" />"#; // apply_labels' monotonic counter — not 3 (its raw CSV index).
let labels = labels_from_feature(feature);
let a = audio(&labels);
assert_eq!(a.len(), 2, "the empty slot carries no label");
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(
a[1].stream_number, 3,
"an empty CSV cell occupies STN slot 2, so `fra` is slot 3"
);
assert_eq!(a[1].purpose, LabelPurpose::Commentary);
// Subtitles: same shape, and the consequence is a misplaced forced
// flag. `forced_sub` index 2 is the forced-narrative track; with the
// empty slot renumbered away it would be written onto STN slot 2.
let feature = r#"<playlist name="Feature" sub="eng,,fra" forced_sub="0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 2);
assert_eq!(s[0].language, "eng");
assert_eq!(s[0].stream_number, 1);
assert_eq!(s[0].qualifier, LabelQualifier::None);
assert_eq!(s[1].language, "fra");
assert_eq!(
s[1].stream_number, 3,
"the forced marker belongs to STN slot 3, not slot 2"
);
assert_eq!(s[1].qualifier, LabelQualifier::Forced);
}
#[test]
fn empty_middle_slot_carries_no_label_but_keeps_its_slot() {
// aud="eng,,fra": the empty middle cell yields no label — there is
// nothing to label — but it still owns STN slot 2, so `fra` is slot
// 3. (This test previously asserted 2, pinning the renumbering bug.)
let feature = r#"<playlist name="Feature" aud="eng,,fra" />"#; let feature = r#"<playlist name="Feature" aud="eng,,fra" />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let a = audio(&labels); let a = audio(&labels);
@@ -597,7 +194,7 @@ mod tests {
assert_eq!(a[0].language, "eng"); assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1); assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra"); assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 3); assert_eq!(a[1].stream_number, 2);
} }
#[test] #[test]
@@ -605,7 +202,7 @@ mod tests {
// Whitespace around the index, and a multi-value list, must both // Whitespace around the index, and a multi-value list, must both
// resolve. com index is positional against the raw CSV, so with // resolve. com index is positional against the raw CSV, so with
// an empty slot at position 1, " 2 " marks the 'fra' track // an empty slot at position 1, " 2 " marks the 'fra' track
// (CSV index 2, STN slot 3) as commentary. // (CSV index 2) as commentary.
let feature = r#"<playlist aud="eng,,fra" aud_com1_idx=" 2 " />"#; let feature = r#"<playlist aud="eng,,fra" aud_com1_idx=" 2 " />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let a = audio(&labels); let a = audio(&labels);
@@ -617,10 +214,10 @@ mod tests {
#[test] #[test]
fn forced_sub_aligns_with_raw_csv_index() { fn forced_sub_aligns_with_raw_csv_index() {
// sub="eng,eng,zho,ces" forced_sub="0,0,0,3": the forced marker is // sub="eng,eng,zho,ces" forced_sub="0,0,0,1": the forced flag is
// positional on the raw CSV, so 'ces' (index 3) is forced; its // positional on the raw CSV, so 'ces' (index 3) is forced; its
// stream_number is its 1-based cell position, 4. // stream_number is its non-empty position (4 here, no gaps).
let feature = r#"<playlist sub="eng,eng,zho,ces" forced_sub="0,0,0,3" />"#; let feature = r#"<playlist sub="eng,eng,zho,ces" forced_sub="0,0,0,1" />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let s = subs(&labels); let s = subs(&labels);
assert_eq!(s.len(), 4); assert_eq!(s.len(), 4);
@@ -665,12 +262,10 @@ mod tests {
assert!(feature.contains(r#"name="MainMovie""#)); assert!(feature.contains(r#"name="MainMovie""#));
} }
/// Spec: stream_number for audio is the cell's own 1-based CSV position, /// Spec: stream_number for audio is 1-based and increments only on non-empty slots.
/// because the CSV is the STN list and empty cells are slots too. /// Mutation: increment for empty slots too → stream numbers inflate.
/// Mutation: count only non-empty cells → every label behind an empty
/// cell shifts one slot forward.
#[test] #[test]
fn audio_stream_numbering_uses_raw_csv_slot_position() { fn audio_stream_numbering_skips_empty_slots() {
let feature = r#"<playlist name="Feature" aud="eng,,fra,,spa" />"#; let feature = r#"<playlist name="Feature" aud="eng,,fra,,spa" />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let a = audio(&labels); let a = audio(&labels);
@@ -678,9 +273,9 @@ mod tests {
assert_eq!(a[0].language, "eng"); assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1); assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra"); assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 3); assert_eq!(a[1].stream_number, 2);
assert_eq!(a[2].language, "spa"); assert_eq!(a[2].language, "spa");
assert_eq!(a[2].stream_number, 5); assert_eq!(a[2].stream_number, 3);
} }
/// Spec: forced subtitle at the last position with gaps in between. /// Spec: forced subtitle at the last position with gaps in between.
@@ -688,9 +283,9 @@ mod tests {
/// Mutation: use stream_number (dense) instead of raw index → wrong subtitle forced. /// Mutation: use stream_number (dense) instead of raw index → wrong subtitle forced.
#[test] #[test]
fn forced_sub_uses_raw_csv_index_with_gaps() { fn forced_sub_uses_raw_csv_index_with_gaps() {
// sub="eng,,fra,,spa" forced_sub="0,0,0,0,3" // sub="eng,,fra,,spa" forced_sub="0,0,0,0,1"
// raw CSV index 4 = "spa", i.e. STN slot 5. // raw CSV index 4 = "spa"; stream_number for spa = 3 (3rd non-empty).
let feature = r#"<playlist name="Feature" sub="eng,,fra,,spa" forced_sub="0,0,0,0,3" />"#; let feature = r#"<playlist name="Feature" sub="eng,,fra,,spa" forced_sub="0,0,0,0,1" />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let s = subs(&labels); let s = subs(&labels);
assert_eq!(s.len(), 3); assert_eq!(s.len(), 3);
@@ -700,7 +295,6 @@ mod tests {
assert_eq!(s[1].qualifier, LabelQualifier::None); assert_eq!(s[1].qualifier, LabelQualifier::None);
assert_eq!(s[2].language, "spa"); assert_eq!(s[2].language, "spa");
assert_eq!(s[2].qualifier, LabelQualifier::Forced); assert_eq!(s[2].qualifier, LabelQualifier::Forced);
assert_eq!(s[2].stream_number, 5);
} }
/// Spec: aud_com1_idx is positional against the raw CSV. /// Spec: aud_com1_idx is positional against the raw CSV.
@@ -709,14 +303,13 @@ mod tests {
/// Mutation: use stream_number instead of raw CSV index → wrong stream is commentary. /// Mutation: use stream_number instead of raw CSV index → wrong stream is commentary.
#[test] #[test]
fn audio_commentary_index_raw_csv_position() { fn audio_commentary_index_raw_csv_position() {
// aud="eng,,fra,spa" aud_com1_idx="2" → CSV index 2 = "fra", // aud="eng,,fra,spa" aud_com1_idx="2" → CSV index 2 = "fra".
// which is STN slot 3. // "fra" is stream_number 2 (second non-empty slot, skipping the empty).
let feature = r#"<playlist name="Feature" aud="eng,,fra,spa" aud_com1_idx="2" />"#; let feature = r#"<playlist name="Feature" aud="eng,,fra,spa" aud_com1_idx="2" />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let a = audio(&labels); let a = audio(&labels);
assert_eq!(a.len(), 3); assert_eq!(a.len(), 3);
assert_eq!(a[1].language, "fra"); assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 3);
assert_eq!(a[1].purpose, LabelPurpose::Commentary); assert_eq!(a[1].purpose, LabelPurpose::Commentary);
assert_eq!(a[0].purpose, LabelPurpose::Normal); assert_eq!(a[0].purpose, LabelPurpose::Normal);
assert_eq!(a[2].purpose, LabelPurpose::Normal); assert_eq!(a[2].purpose, LabelPurpose::Normal);
@@ -759,11 +352,10 @@ mod tests {
assert!(s.is_empty(), "no subtitle labels when sub is absent"); assert!(s.is_empty(), "no subtitle labels when sub is absent");
} }
/// Spec: audio stream_number is the cell's 1-based position and never /// Spec: audio stream_number uses saturating_add on overflow (per u16 cap).
/// wraps; past the u16 space the parser stops emitting. /// Mutation: use wrapping_add → stream numbers wrap to 0, skipping apply.
/// Mutation: cast `i + 1` to u16 → stream numbers wrap to 0, skipping apply.
#[test] #[test]
fn audio_stream_number_never_wraps() { fn audio_stream_number_saturates_not_wraps() {
// 65535 audio tracks is impossible on a real disc but the parser must // 65535 audio tracks is impossible on a real disc but the parser must
// not panic or produce 0. Build a comma-separated list of 65535 "eng"s. // not panic or produce 0. Build a comma-separated list of 65535 "eng"s.
// We only run the number-assignment logic via labels_from_feature. // We only run the number-assignment logic via labels_from_feature.
@@ -783,92 +375,15 @@ mod tests {
assert_eq!(last, 300); assert_eq!(last, 300);
} }
/// Spec: a `forced_sub` cell with surrounding whitespace still classifies. /// Spec: forced_sub with whitespace around "1" must still parse as true.
/// Mutation: drop the `trim()` → " 3 " falls through to the unrecognised /// Mutation: use `== "1"` instead of `trim() == "1"` → " 1 " fails.
/// arm and the disc's forced-narrative track loses its label.
#[test] #[test]
fn forced_sub_cells_are_trimmed_before_classification() { fn forced_sub_whitespace_around_one() {
let feature = r#"<playlist name="Feature" sub="eng,fra,spa" forced_sub="0, 3 , 1 " />"#; let feature = r#"<playlist name="Feature" sub="eng,fra" forced_sub="0, 1" />"#;
let labels = labels_from_feature(feature); let labels = labels_from_feature(feature);
let s = subs(&labels); let s = subs(&labels);
assert_eq!(s[0].qualifier, LabelQualifier::None); assert_eq!(s[0].qualifier, LabelQualifier::None);
assert_eq!(s[1].qualifier, LabelQualifier::Forced); assert_eq!(s[1].qualifier, LabelQualifier::Forced);
assert_eq!(s[2].qualifier, LabelQualifier::None);
}
/// `forced_sub` is an enumeration, and `1` is its "full dialogue track that
/// also carries forced signs" value — NOT "this track is forced".
///
/// Measured on a disc whose feature declares nine `1` cells among 32
/// subtitle slots: all nine are full dialogue tracks of 949-1411 display
/// sets, and the disc has no dedicated forced track at all. Reading `1` as
/// forced is what produced two identical full subtitle tracks for one
/// language with one of them flagged forced.
///
/// Nothing downstream can undo this on the discs that need it most:
/// `mux::codec::pgs::demotable` may only clear a vendor forced label where
/// some track on the disc demonstrably sets `forced_on_flag`, and measured
/// discs using this label format never set it.
///
/// Mutation: `"1" => ForcedNarrative` (the old reading) → red.
#[test]
fn a_contains_forced_segments_cell_is_not_a_forced_track() {
let feature = r#"<playlist name="Feature" sub="eng,ces,deu" forced_sub="0,1,1" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 3);
assert!(
s.iter().all(|l| l.qualifier == LabelQualifier::None),
"a `1` marks a full track containing forced signs, not a forced track"
);
}
/// `2` and `3` are the cells that DO name a dedicated forced-narrative
/// track, and the old boolean reading discarded both.
///
/// Measured: these cells occupy their own trailing STN slots, one per
/// localized language, duplicating a language that already holds a full
/// track earlier in the list. On one measured disc the four `3` slots carry
/// 7, 14, 23 and 59 display sets against 1216-2655 on the full tracks they
/// duplicate — and not one display set anywhere on that disc carries
/// `forced_on_flag`, so neither the scan probe nor the muxer can promote
/// them from content. The vendor cell is the only evidence there is.
///
/// Mutation: drop either arm of the `"2" | "3"` match → red.
#[test]
fn a_dedicated_forced_narrative_cell_is_a_forced_track() {
// The measured shape: full tracks first, their forced companions in
// trailing slots of the same languages.
let feature =
r#"<playlist name="Feature" sub="eng,cat,jpn,cat,jpn" forced_sub="0,0,0,2,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 5);
assert_eq!(s[1].qualifier, LabelQualifier::None, "the full cat track");
assert_eq!(s[2].qualifier, LabelQualifier::None, "the full jpn track");
assert_eq!(s[3].qualifier, LabelQualifier::Forced, "cat forced slot");
assert_eq!(s[3].stream_number, 4);
assert_eq!(s[4].qualifier, LabelQualifier::Forced, "jpn forced slot");
assert_eq!(s[4].stream_number, 5);
}
/// An unrecognised cell must fall to NOT forced. Asserting forced is the
/// expensive mistake (a full dialogue track a player then burns on screen),
/// so an unknown value from a future authoring revision must not be able to
/// make that claim.
///
/// Mutation: `_ => ForcedNarrative`, or treating "any non-zero" as forced.
#[test]
fn an_unrecognised_forced_sub_cell_is_not_forced() {
let feature = r#"<playlist name="Feature" sub="eng,fra,spa,ita" forced_sub="4,x,,-1" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 4);
assert!(s.iter().all(|l| l.qualifier == LabelQualifier::None));
// ...and so must a cell the CSV simply does not reach.
let feature = r#"<playlist name="Feature" sub="eng,fra" forced_sub="0" />"#;
let labels = labels_from_feature(feature);
assert_eq!(subs(&labels)[1].qualifier, LabelQualifier::None);
} }
/// Spec: `find_feature_playlist` returns None when XML has no `<playlist>` elements. /// Spec: `find_feature_playlist` returns None when XML has no `<playlist>` elements.
@@ -878,23 +393,4 @@ mod tests {
assert!(find_feature_playlist("").is_none()); assert!(find_feature_playlist("").is_none());
assert!(find_feature_playlist("<root />").is_none()); assert!(find_feature_playlist("<root />").is_none());
} }
/// Spec: on a tie in audio-slot count, the FIRST playlist encountered
/// wins (consistent with `select_result`'s first-wins tiebreak
/// elsewhere in the registry) — later playlists only displace the
/// current best on a STRICTLY greater count.
/// Mutation: `count > best_aud_count` -> `count >= best_aud_count`
/// would let a later tied playlist silently displace the first.
#[test]
fn find_feature_first_wins_on_audio_count_tie() {
let xml = r#"
<playlist name="A" aud="eng,fra" />
<playlist name="B" aud="deu,spa" />
"#;
let feature = find_feature_playlist(xml).expect("a feature is found");
assert!(
feature.contains(r#"name="A""#),
"first playlist must win a tie, got: {feature}"
);
}
} }
+62 -783
View File
File diff suppressed because it is too large Load Diff
+12 -19
View File
@@ -1,8 +1,8 @@
//! Menu-graphic filename language hints. //! Menu-graphic filename language hints.
//! //!
//! Some BD-J discs encode per-language menu artwork with the language in the //! Some BD-J discs encode per-language menu artwork with the language in the
//! filename, e.g. `Feature_UHD01_Eng_Composite1.png`, //! filename, e.g. `Dune_UHD01_Eng_Composite1.png`,
//! `AltFeature_UHD01_FRE_Composite2.png`. The `_UHD01_{LANG}_Composite` //! `VForVendetta_UHD01_FRE_Composite2.png`. The `_UHD01_{LANG}_Composite`
//! marker is authored deliberately, so the set of `{LANG}` tokens is the set //! marker is authored deliberately, so the set of `{LANG}` tokens is the set
//! of menu languages the disc ships. //! of menu languages the disc ships.
//! //!
@@ -41,16 +41,15 @@ pub fn parse(_reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
fn labels_from_filenames(names: &[String]) -> Vec<StreamLabel> { fn labels_from_filenames(names: &[String]) -> Vec<StreamLabel> {
let mut seen: Vec<&'static str> = Vec::new(); let mut seen: Vec<&'static str> = Vec::new();
for name in names { for name in names {
if let Some(code) = filename_lang(name) if let Some(code) = filename_lang(name) {
&& !seen.contains(&code) if !seen.contains(&code) {
{ seen.push(code);
seen.push(code); }
} }
} }
seen.into_iter() seen.into_iter()
.enumerate() .enumerate()
.map(|(i, code)| StreamLabel { .map(|(i, code)| StreamLabel {
stream_id: None,
stream_number: (i as u16).saturating_add(1), stream_number: (i as u16).saturating_add(1),
stream_type: StreamLabelType::Audio, stream_type: StreamLabelType::Audio,
language: code.to_string(), language: code.to_string(),
@@ -92,16 +91,10 @@ mod tests {
#[test] #[test]
fn extracts_confirmed_samples() { fn extracts_confirmed_samples() {
assert_eq!(filename_lang("Dune_UHD01_Eng_Composite1.png"), Some("eng"));
assert_eq!(filename_lang("Dune_UHD01_Ger_Composite2.png"), Some("deu"));
assert_eq!( assert_eq!(
filename_lang("Feature_UHD01_Eng_Composite1.png"), filename_lang("VForVendetta_UHD01_FRE_Composite2.png"),
Some("eng")
);
assert_eq!(
filename_lang("Feature_UHD01_Ger_Composite2.png"),
Some("deu")
);
assert_eq!(
filename_lang("AltFeature_UHD01_FRE_Composite2.png"),
Some("fra") Some("fra")
); );
} }
@@ -127,9 +120,9 @@ mod tests {
#[test] #[test]
fn dedups_and_numbers_distinct_languages() { fn dedups_and_numbers_distinct_languages() {
let names = vec![ let names = vec![
"Feature_UHD01_Eng_Composite1.png".to_string(), "Dune_UHD01_Eng_Composite1.png".to_string(),
"Feature_UHD01_Eng_Composite2.png".to_string(), "Dune_UHD01_Eng_Composite2.png".to_string(),
"Feature_UHD01_Ger_Composite1.png".to_string(), "Dune_UHD01_Ger_Composite1.png".to_string(),
"LoadingComposite1.png".to_string(), "LoadingComposite1.png".to_string(),
]; ];
let labels = labels_from_filenames(&names); let labels = labels_from_filenames(&names);
+1 -1
View File
@@ -169,7 +169,7 @@ mod tests {
/// Mutation: skip the final `if !current.is_empty()` emit → trailing run lost. /// Mutation: skip the final `if !current.is_empty()` emit → trailing run lost.
#[test] #[test]
fn large_buffer_trailing_run_emitted() { fn large_buffer_trailing_run_emitted() {
let buf: Vec<u8> = (0..1000u32).map(|i| 0x41u8 + (i % 26) as u8).collect(); let buf: Vec<u8> = (0..1000u32).map(|i| (0x41u8 + (i % 26) as u8)).collect();
let got = extract_ascii_strings(&buf, 1); let got = extract_ascii_strings(&buf, 1);
// All printable, so one big run at the end. // All printable, so one big run at the end.
assert!(!got.is_empty()); assert!(!got.is_empty());
+1 -431
View File
@@ -177,7 +177,7 @@ const BARE_LANGS: &[(&str, &str)] = &[
]; ];
/// Map a short menu-graphic language token (as embedded in authoring /// Map a short menu-graphic language token (as embedded in authoring
/// filenames like `Feature_UHD01_Eng_Composite1.png`) to an ISO-639-2/T code. /// filenames like `Dune_UHD01_Eng_Composite1.png`) to an ISO-639-2/T code.
/// ///
/// These filename tokens are compact 2/3-letter abbreviations, NOT the full /// These filename tokens are compact 2/3-letter abbreviations, NOT the full
/// language names [`lang`] handles, so they get their own certain table. /// language names [`lang`] handles, so they get their own certain table.
@@ -219,237 +219,6 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
Some(code) Some(code)
} }
// ── ISO 639-1 → ISO 639-2 ────────────────────────────────────────────────────
/// The complete ISO 639-1 set, paired with its ISO 639-2/**T** (terminological)
/// code. Every two-letter code ISO 639-1 defines appears exactly once.
///
/// /T is the variant the rest of this crate uses — [`lang`] and [`menu_lang`]
/// both normalize to it (`deu` not `ger`, `fra` not `fre`, `zho` not `chi`,
/// `ces`, `nld`, `ell`, `ron`, `slk`, `isl`, `eus`, `hrv`) — so the three
/// tables cannot disagree. `iso639_1_agrees_with_menu_lang` pins that.
///
/// For the 165 codes where 639-2/B and /T are identical this distinction does
/// not arise; it only matters for the 20-odd languages with a distinct
/// bibliographic code.
const ISO_639_1_TO_2: &[(&str, &str)] = &[
("aa", "aar"),
("ab", "abk"),
("ae", "ave"),
("af", "afr"),
("ak", "aka"),
("am", "amh"),
("an", "arg"),
("ar", "ara"),
("as", "asm"),
("av", "ava"),
("ay", "aym"),
("az", "aze"),
("ba", "bak"),
("be", "bel"),
("bg", "bul"),
("bh", "bih"),
("bi", "bis"),
("bm", "bam"),
("bn", "ben"),
("bo", "bod"),
("br", "bre"),
("bs", "bos"),
("ca", "cat"),
("ce", "che"),
("ch", "cha"),
("co", "cos"),
("cr", "cre"),
("cs", "ces"),
("cu", "chu"),
("cv", "chv"),
("cy", "cym"),
("da", "dan"),
("de", "deu"),
("dv", "div"),
("dz", "dzo"),
("ee", "ewe"),
("el", "ell"),
("en", "eng"),
("eo", "epo"),
("es", "spa"),
("et", "est"),
("eu", "eus"),
("fa", "fas"),
("ff", "ful"),
("fi", "fin"),
("fj", "fij"),
("fo", "fao"),
("fr", "fra"),
("fy", "fry"),
("ga", "gle"),
("gd", "gla"),
("gl", "glg"),
("gn", "grn"),
("gu", "guj"),
("gv", "glv"),
("ha", "hau"),
("he", "heb"),
("hi", "hin"),
("ho", "hmo"),
("hr", "hrv"),
("ht", "hat"),
("hu", "hun"),
("hy", "hye"),
("hz", "her"),
("ia", "ina"),
("id", "ind"),
("ie", "ile"),
("ig", "ibo"),
("ii", "iii"),
("ik", "ipk"),
("io", "ido"),
("is", "isl"),
("it", "ita"),
("iu", "iku"),
("ja", "jpn"),
("jv", "jav"),
("ka", "kat"),
("kg", "kon"),
("ki", "kik"),
("kj", "kua"),
("kk", "kaz"),
("kl", "kal"),
("km", "khm"),
("kn", "kan"),
("ko", "kor"),
("kr", "kau"),
("ks", "kas"),
("ku", "kur"),
("kv", "kom"),
("kw", "cor"),
("ky", "kir"),
("la", "lat"),
("lb", "ltz"),
("lg", "lug"),
("li", "lim"),
("ln", "lin"),
("lo", "lao"),
("lt", "lit"),
("lu", "lub"),
("lv", "lav"),
("mg", "mlg"),
("mh", "mah"),
("mi", "mri"),
("mk", "mkd"),
("ml", "mal"),
("mn", "mon"),
("mr", "mar"),
("ms", "msa"),
("mt", "mlt"),
("my", "mya"),
("na", "nau"),
("nb", "nob"),
("nd", "nde"),
("ne", "nep"),
("ng", "ndo"),
("nl", "nld"),
("nn", "nno"),
("no", "nor"),
("nr", "nbl"),
("nv", "nav"),
("ny", "nya"),
("oc", "oci"),
("oj", "oji"),
("om", "orm"),
("or", "ori"),
("os", "oss"),
("pa", "pan"),
("pi", "pli"),
("pl", "pol"),
("ps", "pus"),
("pt", "por"),
("qu", "que"),
("rm", "roh"),
("rn", "run"),
("ro", "ron"),
("ru", "rus"),
("rw", "kin"),
("sa", "san"),
("sc", "srd"),
("sd", "snd"),
("se", "sme"),
("sg", "sag"),
("si", "sin"),
("sk", "slk"),
("sl", "slv"),
("sm", "smo"),
("sn", "sna"),
("so", "som"),
("sq", "sqi"),
("sr", "srp"),
("ss", "ssw"),
("st", "sot"),
("su", "sun"),
("sv", "swe"),
("sw", "swa"),
("ta", "tam"),
("te", "tel"),
("tg", "tgk"),
("th", "tha"),
("ti", "tir"),
("tk", "tuk"),
("tl", "tgl"),
("tn", "tsn"),
("to", "ton"),
("tr", "tur"),
("ts", "tso"),
("tt", "tat"),
("tw", "twi"),
("ty", "tah"),
("ug", "uig"),
("uk", "ukr"),
("ur", "urd"),
("uz", "uzb"),
("ve", "ven"),
("vi", "vie"),
("vo", "vol"),
("wa", "wln"),
("wo", "wol"),
("xh", "xho"),
("yi", "yid"),
("yo", "yor"),
("za", "zha"),
("zh", "zho"),
("zu", "zul"),
];
/// The three two-letter codes ISO 639-1 has since withdrawn, mapped to their
/// replacements. DVD-Video froze its language list on the 1988 edition, so
/// discs authored to the spec carry these spellings and no other table sees
/// them: `iw` Hebrew (now `he`), `in` Indonesian (now `id`), `ji` Yiddish
/// (now `yi`).
const ISO_639_1_DEPRECATED: &[(&str, &str)] = &[("iw", "he"), ("in", "id"), ("ji", "yi")];
/// Map an ISO 639-1 two-letter language code to its ISO 639-2/T three-letter
/// code, accepting the withdrawn DVD-era spellings (`iw`, `in`, `ji`) as
/// aliases for their replacements.
///
/// Covers the WHOLE of ISO 639-1, unlike [`menu_lang`], whose table only spans
/// the languages that show up in Blu-ray menu-graphic filenames. Callers that
/// convert a spec field — a DVD IFO attribute block, say — need the whole set:
/// narrowing it to the menu vocabulary would fold every other language onto
/// one value and make a disc's tracks indistinguishable from each other.
///
/// Case-insensitive and trimmed. Returns `None` for anything that is not an
/// ISO 639-1 code, so callers decide the fallback rather than getting a guess.
pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
let c = code.trim().to_ascii_lowercase();
let c = ISO_639_1_DEPRECATED
.iter()
.find(|(old, _)| *old == c)
.map_or(c.as_str(), |(_, new)| new);
ISO_639_1_TO_2
.iter()
.find(|(two, _)| *two == c)
.map(|(_, three)| *three)
}
// ── Purpose ────────────────────────────────────────────────────────────────── // ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`]. /// Classify a free-form English label string into a [`LabelPurpose`].
@@ -916,203 +685,4 @@ mod tests {
fn codec_empty_passes_through() { fn codec_empty_passes_through() {
assert_eq!(codec(""), ""); assert_eq!(codec(""), "");
} }
/// `purpose()`'s multi-word-compound fast path ORs two independent
/// phrase checks ("audio description" / "descriptive service"). Each
/// phrase, when it appears as a *word*-bounded match, is independently
/// caught by the has_word fallback further down — so the OR only
/// matters when a phrase appears as a *substring inside a larger word*
/// (no boundary), which .contains() still catches but has_word() would
/// reject.
///
/// Mutation: replace `||` with `&&` at line 238 → since "audio
/// description" is absent here, the AND fails, the fast path doesn't
/// fire, and the fallback has_word("descriptive") also fails (no word
/// boundary before "descriptive" in "nondescriptive"), so purpose()
/// wrongly returns Normal instead of Descriptive.
#[test]
fn purpose_descriptive_service_substring_without_word_boundary() {
assert_eq!(
purpose("nondescriptive service track"),
LabelPurpose::Descriptive
);
}
/// `menu_lang()` maps every authoring-filename token in its table
/// (ISO-639-2/B and /T spellings, plus ISO-639-1) to the canonical
/// /T code used by the rest of the pipeline. Exhaustive per-arm check:
/// deleting any single match arm makes that arm's tokens return None
/// instead of the documented code.
#[test]
fn menu_lang_covers_every_table_entry() {
let cases: &[(&str, &str)] = &[
("eng", "eng"),
("en", "eng"),
("ger", "deu"),
("deu", "deu"),
("de", "deu"),
("fre", "fra"),
("fra", "fra"),
("fr", "fra"),
("spa", "spa"),
("es", "spa"),
("ita", "ita"),
("it", "ita"),
("por", "por"),
("pt", "por"),
("jpn", "jpn"),
("jap", "jpn"),
("ja", "jpn"),
("kor", "kor"),
("ko", "kor"),
("chi", "zho"),
("zho", "zho"),
("zh", "zho"),
("rus", "rus"),
("ru", "rus"),
("dut", "nld"),
("nld", "nld"),
("nl", "nld"),
("pol", "pol"),
("pl", "pol"),
("cze", "ces"),
("ces", "ces"),
("cs", "ces"),
("dan", "dan"),
("da", "dan"),
("fin", "fin"),
("fi", "fin"),
("nor", "nor"),
("no", "nor"),
("swe", "swe"),
("sv", "swe"),
("hun", "hun"),
("hu", "hun"),
("gre", "ell"),
("ell", "ell"),
("el", "ell"),
("tur", "tur"),
("tr", "tur"),
("ara", "ara"),
("ar", "ara"),
("hin", "hin"),
("hi", "hin"),
("tha", "tha"),
("th", "tha"),
("ukr", "ukr"),
("uk", "ukr"),
("cat", "cat"),
("ca", "cat"),
];
for (token, expected) in cases {
assert_eq!(
menu_lang(token),
Some(*expected),
"menu_lang({:?}) should map to {:?}",
token,
expected
);
}
// Case-insensitive and trimmed.
assert_eq!(menu_lang("ENG"), Some("eng"));
assert_eq!(menu_lang(" Eng "), Some("eng"));
// Unrecognized token -> None, never a guess.
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
/// Structural invariants of `ISO_639_1_TO_2`: it must hold the complete
/// ISO 639-1 set (184 codes), every key a distinct pair of lowercase
/// letters and every value three lowercase letters. A typo'd or duplicated
/// row fails here rather than silently mislabelling a track.
#[test]
fn iso639_1_table_is_complete_and_well_formed() {
assert_eq!(
ISO_639_1_TO_2.len(),
184,
"ISO 639-1 defines 184 two-letter codes; the table must hold all \
of them"
);
let mut keys: Vec<&str> = ISO_639_1_TO_2.iter().map(|(two, _)| *two).collect();
keys.sort_unstable();
let unique = keys.len();
keys.dedup();
assert_eq!(unique, keys.len(), "no ISO 639-1 code may appear twice");
for (two, three) in ISO_639_1_TO_2 {
assert!(
two.len() == 2 && two.bytes().all(|b| b.is_ascii_lowercase()),
"{two:?} is not a two-letter lowercase ISO 639-1 code"
);
assert!(
three.len() == 3 && three.bytes().all(|b| b.is_ascii_lowercase()),
"{three:?} is not a three-letter lowercase ISO 639-2 code"
);
}
// The withdrawn DVD-era spellings resolve, and are not themselves
// rows in the main table (they are aliases, not codes).
for (old, new) in ISO_639_1_DEPRECATED {
assert!(
!ISO_639_1_TO_2.iter().any(|(two, _)| two == old),
"withdrawn code {old:?} must not be a table row"
);
assert_eq!(
iso639_1_to_iso639_2(old),
iso639_1_to_iso639_2(new),
"withdrawn code {old:?} must resolve exactly as {new:?}"
);
}
}
/// The two tables must not disagree. Every two-letter token `menu_lang`
/// accepts has to yield the same ISO 639-2/T code through
/// `iso639_1_to_iso639_2`, so a DVD-sourced language and a Blu-ray
/// menu-label language for the same tongue never produce different
/// `Language` elements.
#[test]
fn iso639_1_agrees_with_menu_lang() {
for (two, three) in ISO_639_1_TO_2 {
if let Some(via_menu) = menu_lang(two) {
assert_eq!(
via_menu, *three,
"menu_lang({two:?}) = {via_menu:?} disagrees with the ISO \
639-1 table's {three:?}"
);
}
}
// Spot-check the /T choice itself, on the languages where /B differs.
for (two, t_code) in [
("de", "deu"),
("fr", "fra"),
("zh", "zho"),
("cs", "ces"),
("nl", "nld"),
("el", "ell"),
("ro", "ron"),
("sk", "slk"),
("is", "isl"),
("hy", "hye"),
("ka", "kat"),
("fa", "fas"),
] {
assert_eq!(
iso639_1_to_iso639_2(two),
Some(t_code),
"the crate standardises on ISO 639-2/T, so {two:?} is \
{t_code:?} and never the bibliographic form"
);
}
}
/// Trimming, case-insensitivity, and the no-guess contract.
#[test]
fn iso639_1_normalizes_input_and_never_guesses() {
assert_eq!(iso639_1_to_iso639_2("RO"), Some("ron"));
assert_eq!(iso639_1_to_iso639_2(" Ro "), Some("ron"));
assert_eq!(iso639_1_to_iso639_2("IW"), Some("heb"));
assert_eq!(iso639_1_to_iso639_2("zz"), None);
assert_eq!(iso639_1_to_iso639_2(""), None);
assert_eq!(iso639_1_to_iso639_2("e"), None);
// A three-letter code is not ISO 639-1 input — that is menu_lang's job.
assert_eq!(iso639_1_to_iso639_2("eng"), None);
}
} }
-132
View File
@@ -634,136 +634,4 @@ mod tests {
let (s, e) = find_element(xml, "name", 0).unwrap(); let (s, e) = find_element(xml, "name", 0).unwrap();
assert_eq!(&xml[s..e], "<di:name>Title</di:name>"); assert_eq!(&xml[s..e], "<di:name>Title</di:name>");
} }
// ── Malformed / truncated input (untrusted on-disc XML) ────────────────
//
// These scrapers run on XML lifted out of BD-J jar entries, which is
// attacker-controllable. Every scan in this module must terminate and
// stay in bounds on truncated or unbalanced input rather than panic.
// XML 1.0 §2.3 defines the Name production these boundary rules model.
/// A quoted attribute value that is never closed must terminate the
/// scan at EOF rather than reading past the end of the buffer.
#[test]
fn attr_unterminated_quoted_value_scan_stops_at_eof() {
// The scanner enters the `y="` value and runs off the end looking
// for the closing quote; `name` is never found.
assert_eq!(attr(r#"<x y="oops"#, "name"), None);
assert_eq!(attr("<x y='oops", "name"), None);
// The truncated attribute itself has no terminated value either.
assert_eq!(attr(r#"<x y="oops"#, "y"), None);
}
/// An attribute name at EOF followed only by whitespace (no `=`) must
/// return None, not read past the buffer while skipping that whitespace.
#[test]
fn attr_name_with_trailing_whitespace_and_no_equals_returns_none() {
assert_eq!(attr("<x name ", "name"), None);
}
/// `name=` followed only by whitespace to EOF has no value to return.
#[test]
fn attr_equals_with_trailing_whitespace_and_no_value_returns_none() {
assert_eq!(attr("<x name= ", "name"), None);
}
/// A quoted attribute value is opaque: a `name="..."` pair that appears
/// *inside* another attribute's value must never be reported, even when
/// it is preceded by whitespace so it would otherwise clear the
/// word-boundary check.
#[test]
fn attr_decoy_name_after_space_inside_quoted_value_is_skipped() {
assert_eq!(attr(r#"<x y=" name='decoy'" />"#, "name"), None);
// The real attribute after the decoy still resolves.
assert_eq!(
attr(r#"<x y=" name='decoy'" name="real" />"#, "name"),
Some("real".into())
);
}
/// XML 1.0 §2.3 NameChar includes `-`, `_` and `.`, so `q-a`, `q_a` and
/// `q.a` are each a single attribute name distinct from `a`. Searching
/// for `a` must not match the tail of any of them.
#[test]
fn attr_name_char_boundary_covers_hyphen_underscore_and_dot() {
assert_eq!(
attr(r#"<x q-a="decoy" a="real" />"#, "a"),
Some("real".into())
);
assert_eq!(
attr(r#"<x q_a="decoy" a="real" />"#, "a"),
Some("real".into())
);
assert_eq!(
attr(r#"<x q.a="decoy" a="real" />"#, "a"),
Some("real".into())
);
}
/// An open tag truncated mid-attribute never terminates, so no element
/// can be returned — and the attribute walk must not read past EOF.
#[test]
fn find_element_unterminated_open_tag_returns_none() {
assert_eq!(find_element("<x attr=", "x", 0), None);
}
/// A `/` as the final byte of the buffer is not a self-closing marker;
/// probing for the `>` that would follow it must stay in bounds.
#[test]
fn find_element_trailing_slash_at_eof_returns_none() {
assert_eq!(find_element("<a /", "a", 0), None);
}
/// `/>` inside a quoted attribute value does not close the element.
#[test]
fn find_element_quoted_self_close_marker_does_not_end_element() {
let xml = r#"<x a="/>"/>"#;
let (s, e) = find_element(xml, "x", 0).unwrap();
assert_eq!(&xml[s..e], r#"<x a="/>"/>"#);
}
/// An attribute value whose quote is never closed leaves the open tag
/// unterminated; the scan must end at EOF and report no element.
#[test]
fn find_element_unterminated_quoted_attr_returns_none() {
assert_eq!(find_element(r#"<x a="oops"#, "x", 0), None);
}
/// A `/` in the middle of an unquoted attribute value is not a
/// self-closing marker — only `/>` is.
#[test]
fn find_element_unquoted_slash_is_not_self_closing() {
let xml = "<a href=x/y>body</a>";
let (s, e) = find_element(xml, "a", 0).unwrap();
assert_eq!(&xml[s..e], "<a href=x/y>body</a>");
}
/// `text` must locate the real end of the open tag: a bare `/` inside
/// an unquoted attribute value must not be treated as `/>`, which would
/// shift the body start and leak tag bytes into the returned text.
#[test]
fn text_unquoted_slash_in_attr_does_not_truncate_body() {
assert_eq!(text("<x a=b/c>hello</x>", "x"), Some("hello".into()));
}
/// A `>` inside a quoted attribute value must not be mistaken for the
/// end of the open tag when `text` computes the body start.
#[test]
fn text_quoted_gt_in_attr_does_not_truncate_body() {
assert_eq!(text(r#"<x a="b>c">hello</x>"#, "x"), Some("hello".into()));
}
/// A close tag truncated mid-name (`</x` with no `>`) is not a close
/// tag; matching it must stay in bounds and report no text.
#[test]
fn text_truncated_close_tag_returns_none() {
assert_eq!(text("<x>body</x", "x"), None);
}
/// A `/` in element content is only a close tag when preceded by `<`.
/// Body text containing `a/x>` must not be mistaken for `</x>`.
#[test]
fn text_slash_in_body_is_not_a_close_tag() {
assert_eq!(text("<x>a/x> </x>", "x"), Some("a/x>".into()));
}
} }
+4 -19
View File
@@ -32,7 +32,7 @@
//! let opts = libfreemkv::InputOptions::default(); //! let opts = libfreemkv::InputOptions::default();
//! let mut input = libfreemkv::input("iso://disc.iso", &opts)?; //! let mut input = libfreemkv::input("iso://disc.iso", &opts)?;
//! let title = input.info().clone(); //! let title = input.info().clone();
//! let mut output = libfreemkv::output("mkv://Movie.mkv", &title, None)?; //! let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
//! // Propagate read errors instead of silently stopping on the first one. //! // Propagate read errors instead of silently stopping on the first one.
//! while let Some(frame) = input.read()? { //! while let Some(frame) = input.read()? {
//! output.write(&frame)?; //! output.write(&frame)?;
@@ -106,15 +106,12 @@ pub mod consts;
pub mod css; pub mod css;
pub mod decrypt; pub mod decrypt;
pub mod diag; pub mod diag;
pub mod dirimage;
pub mod disc; pub mod disc;
pub mod drive; pub mod drive;
pub mod dvdnav; pub mod dvdnav;
pub mod error; pub mod error;
pub mod event; pub mod event;
pub mod halt; pub mod halt;
#[cfg(test)]
mod harness;
pub mod hex; pub mod hex;
pub(crate) mod identity; pub(crate) mod identity;
pub(crate) mod ifo; pub(crate) mod ifo;
@@ -129,8 +126,6 @@ pub mod progress;
pub mod scsi; pub mod scsi;
pub mod sector; pub mod sector;
pub mod session; pub mod session;
#[cfg(test)]
pub(crate) mod testlog;
pub(crate) mod udf; pub(crate) mod udf;
pub(crate) mod unlock_bridge; pub(crate) mod unlock_bridge;
@@ -151,8 +146,7 @@ pub use drive::{Drive, DriveStatus, extract_scsi_context, find_drive};
// Owns the `Drive` by value; forwards consumer-built key material into // Owns the `Drive` by value; forwards consumer-built key material into
// `ScanOptions` (the library derives no certs — see `KeySpec`). // `ScanOptions` (the library derives no certs — see `KeySpec`).
pub use session::{ pub use session::{
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_dir, DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_iso,
scan_iso,
}; };
// ─── Errors ───────────────────────────────────────────────────────────────── // ─── Errors ─────────────────────────────────────────────────────────────────
@@ -160,9 +154,7 @@ pub use session::{
// All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a // All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a
// numeric `code()`; **no English text in the library** — applications map // numeric `code()`; **no English text in the library** — applications map
// codes to localized messages. See `error.rs` for the full taxonomy. // codes to localized messages. See `error.rs` for the full taxonomy.
pub use error::{ pub use error::{Error, Result, is_disc_level_no_key, is_halt, is_skippable_title_stub};
Error, Result, error_code, is_disc_level_no_key, is_halt, is_skippable_title_stub,
};
// ─── Cooperative cancellation ─────────────────────────────────────────────── // ─── Cooperative cancellation ───────────────────────────────────────────────
// //
@@ -195,11 +187,6 @@ pub use io::pipeline::{
// continuously instead of bursting. General I/O infra, not recovery policy; // continuously instead of bursting. General I/O infra, not recovery policy;
// promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too. // promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too.
pub use io::WritebackFile; pub use io::WritebackFile;
/// Write an image-level source out as a sector image — what an `iso://`
/// DESTINATION means for any source that is not a physical drive. Drive sources
/// go through `freemkv_engine::copy`, which is the recovery path; see
/// [`io::image_writer`] for why the two are deliberately separate.
pub use io::image_writer::write_image;
// ─── Drive events (low-level callbacks) ───────────────────────────────────── // ─── Drive events (low-level callbacks) ─────────────────────────────────────
pub use event::{BatchSizeReason, Event, EventKind}; pub use event::{BatchSizeReason, Event, EventKind};
@@ -232,7 +219,6 @@ pub use decrypt::{AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_threads, set
// — not the `pes::Stream` trait re-exported below as `PesStream`. Two // — not the `pes::Stream` trait re-exported below as `PesStream`. Two
// different concepts, the same short name; the trait gets the `Pes` // different concepts, the same short name; the trait gets the `Pes`
// prefix at the crate root to keep both addressable. // prefix at the crate root to keep both addressable.
pub use dirimage::DirImage;
pub use disc::{ pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc, AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult, DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult,
@@ -270,8 +256,7 @@ pub use mux::NullStream;
pub use mux::StdioStream; pub use mux::StdioStream;
pub use mux::WriteSeek; pub use mux::WriteSeek;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url}; pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
pub use mux::{Medium, SourceInfo}; pub use mux::{Mp4FitReport, Mp4SkipReason, mp4_fit_report};
pub use mux::{Mp4FitReport, Mp4Sink, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ─────────────────────────────────────────────────── // ─── Lower-level surfaces ───────────────────────────────────────────────────
// //
+3 -316
View File
@@ -39,18 +39,6 @@ pub(crate) struct PlaylistMark {
pub timestamp: u32, pub timestamp: u32,
} }
impl PlaylistMark {
/// Is this mark a chapter entry point?
///
/// Only `mark_type == 1` counts. Type 0 is reserved and type 2 is a link
/// point, and neither is a chapter. Every chapter filter in the crate goes
/// through here: two hand-rolled copies had already drifted, one testing
/// `<= 1` and silently counting reserved marks as chapters.
pub(crate) fn is_chapter_mark(&self) -> bool {
self.mark_type == 1
}
}
/// A play item — one clip reference with in/out times. /// A play item — one clip reference with in/out times.
#[derive(Debug)] #[derive(Debug)]
pub(crate) struct PlayItem { pub(crate) struct PlayItem {
@@ -444,13 +432,14 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
} }
} }
} }
STREAM_CATEGORY_PG_SUBTITLE STREAM_CATEGORY_PG_SUBTITLE => {
// PG: coding_type(1) + language(3). // PG: coding_type(1) + language(3).
// IG is parsed only to advance spos and is then discarded by the // IG is parsed only to advance spos and is then discarded by the
// caller, so it deliberately has no arm here. // caller, so it deliberately has no arm here.
if sa.len() >= 4 => { if sa.len() >= 4 {
language = String::from_utf8_lossy(&sa[1..4]).to_string(); language = String::from_utf8_lossy(&sa[1..4]).to_string();
} }
}
_ => {} _ => {}
} }
@@ -1428,306 +1417,4 @@ mod tests {
data[8..12].copy_from_slice(&40u32.to_be_bytes()); // playlist_start = 40 = len data[8..12].copy_from_slice(&40u32.to_be_bytes()); // playlist_start = 40 = len
assert!(parse(&data).is_err()); assert!(parse(&data).is_err());
} }
// ─────────────────────────────────────────────────────────────────────
// Added: STN-table block alignment and section-boundary hardening.
// ─────────────────────────────────────────────────────────────────────
/// Build an MPLS from raw PlayItem bodies, with no PlayListMark section
/// (mark_start = 0). Lets a test control item_length exactly.
fn build_mpls_raw_items(items: &[Vec<u8>]) -> Vec<u8> {
let playlist_start: u32 = 40;
let mut buf = Vec::new();
buf.extend_from_slice(b"MPLS0200");
buf.extend_from_slice(&playlist_start.to_be_bytes());
buf.extend_from_slice(&[0u8; 28]); // mark_start = 0, then padding
let pl_start = buf.len();
buf.extend_from_slice(&[0u8; 4]); // PlayList length placeholder
buf.extend_from_slice(&[0u8; 2]); // reserved
buf.extend_from_slice(&(items.len() as u16).to_be_bytes());
buf.extend_from_slice(&[0u8; 2]); // num_sub_paths
for it in items {
buf.extend_from_slice(&(it.len() as u16).to_be_bytes());
buf.extend_from_slice(it);
}
let pl_len = (buf.len() - pl_start - 4) as u32;
buf[pl_start..pl_start + 4].copy_from_slice(&pl_len.to_be_bytes());
buf
}
/// The 20 bytes a PlayItem needs for clip_id(5) + codec_id(4) +
/// connection_condition(1) + reserved(2) + IN_time(4) + OUT_time(4).
fn play_item_20(clip: &[u8; 5], cc: u8, in_t: u32, out_t: u32) -> Vec<u8> {
let mut it = Vec::new();
it.extend_from_slice(clip);
it.extend_from_slice(b"M2TS");
it.push(cc);
it.extend_from_slice(&[0u8; 2]);
it.extend_from_slice(&in_t.to_be_bytes());
it.extend_from_slice(&out_t.to_be_bytes());
assert_eq!(it.len(), 20);
it
}
/// A PlayItem body of exactly 20 bytes carries every field the parser
/// reads (the last is OUT_time at [16..20]), so it must be RECORDED,
/// not skipped — and it has no STN table, which starts at byte 32.
#[test]
fn play_item_of_exactly_20_bytes_is_recorded_without_stn() {
let data = build_mpls_raw_items(&[play_item_20(b"00007", 5, 90_000, 180_000)]);
let pl = parse(&data).expect("a 20-byte PlayItem must parse");
assert_eq!(pl.play_items.len(), 1);
assert_eq!(pl.play_items[0].clip_id, "00007");
assert_eq!(pl.play_items[0].in_time, 90_000);
assert_eq!(pl.play_items[0].out_time, 180_000);
assert_eq!(pl.play_items[0].connection_condition, 5);
assert!(pl.streams.is_empty(), "no STN table exists below byte 32");
}
/// A 40-byte MPLS whose PlayList section is exactly its 10-byte header
/// (length(4)+reserved(2)+num_play_items(2)+num_sub_paths(2)) ending at
/// EOF is structurally complete, not truncated: nothing the parser reads
/// lies past the buffer, so it must parse to an empty playlist.
#[test]
fn minimum_size_mpls_with_empty_playlist_header_parses() {
let mut data = vec![0u8; 40];
data[0..4].copy_from_slice(b"MPLS");
data[4..8].copy_from_slice(b"0200");
data[8..12].copy_from_slice(&30u32.to_be_bytes()); // playlist_start + 10 == 40
// mark_start (12..16) stays 0; num_play_items at data[36..38] is 0.
let pl = parse(&data).expect("40-byte MPLS with a complete PlayList header must parse");
assert!(pl.play_items.is_empty());
assert!(pl.streams.is_empty());
assert!(pl.marks.is_empty());
}
/// A mark_start of 0 means "no PlayListMark section". The file header
/// bytes at offset 0 must not be decoded as one — data[4..6] is the
/// version string "02", which as a big-endian num_marks would be 12338.
#[test]
fn mark_start_zero_does_not_parse_header_as_marks() {
let data = build_mpls_raw_items(&[play_item_20(b"00007", 1, 0, 90_000)]);
assert_eq!(
&data[12..16],
&[0, 0, 0, 0],
"fixture must have mark_start 0"
);
let pl = parse(&data).expect("should parse");
assert!(
pl.marks.is_empty(),
"mark_start == 0 must mean absent, got {} marks",
pl.marks.len()
);
}
/// Full STN table walk with every category populated and DISTINCT
/// counts, so no count byte can be read from a neighbour's offset
/// without changing the result.
///
/// Each secondary block is followed by its reference block(s), which
/// per the BD STN table are num_refs(1) + reserved(1) + one byte per
/// ref + one padding byte when the ref count is odd. Every ref count
/// here is 1 — the value that distinguishes `n % 2` (=1) from `n / 2`
/// (=0) — so a wrong skip length misaligns the cursor and every
/// following stream decodes from the wrong offset. IG entries are
/// consumed to keep the cursor aligned but never retained.
#[test]
fn full_stn_table_block_alignment() {
let mut entries: Vec<Vec<u8>> = vec![
build_stream_entry_video(0x1011, 0x1B, 6, 1, None),
build_stream_entry_audio(0x1100, 0x83, 6, 1, b"eng"),
build_stream_entry_audio(0x1101, 0x86, 3, 1, b"fra"),
build_stream_entry_pg(0x1200, 0x90, b"eng"),
build_stream_entry_pg(0x1201, 0x90, b"fra"),
build_stream_entry_pg(0x1202, 0x90, b"deu"),
];
for i in 0..4u16 {
entries.push(build_stream_entry_pg(0x1400 + i, 0x91, b"eng"));
}
// secondary audio + its secondary-audio ref block (1 ref → 1 pad)
let mut sec_audio = build_stream_entry_audio(0x1A00, 0x83, 3, 1, b"spa");
sec_audio.extend_from_slice(&[1, 0, 0x55, 0x00]);
entries.push(sec_audio);
// secondary video + audio-ref block + PiP-PG-ref block
let mut sec_video = build_stream_entry_video(0x1B00, 0x1B, 4, 1, None);
sec_video.extend_from_slice(&[1, 0, 0x55, 0x00]);
sec_video.extend_from_slice(&[1, 0, 0x66, 0x00]);
entries.push(sec_video);
// PiP PG + its ref block
let mut pip_pg = build_stream_entry_pg(0x1C00, 0x90, b"jpn");
pip_pg.extend_from_slice(&[1, 0, 0x77, 0x00]);
entries.push(pip_pg);
// Dolby Vision enhancement layer
entries.push(build_stream_entry_video(0x1015, 0x24, 8, 1, Some(0x12)));
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 2, 3, 4, 1, 1, 1, 1),
&entries,
);
let pl = parse(&data).expect("should parse");
let got: Vec<(u8, u16, bool)> = pl
.streams
.iter()
.map(|s| (s.stream_type, s.pid, s.secondary))
.collect();
assert_eq!(
got,
vec![
(1, 0x1011, false), // primary video
(2, 0x1100, false), // primary audio ×2
(2, 0x1101, false),
(3, 0x1200, false), // PG ×3
(3, 0x1201, false),
(3, 0x1202, false),
// the 4 IG entries are consumed and discarded
(5, 0x1A00, true), // secondary audio
(6, 0x1B00, true), // secondary video
(3, 0x1C00, true), // PiP PG
(7, 0x1015, true), // Dolby Vision EL
]
);
// Languages prove each entry was decoded at its own offset.
assert_eq!(pl.streams[1].language, "eng");
assert_eq!(pl.streams[2].language, "fra");
assert_eq!(pl.streams[6].language, "spa");
assert_eq!(pl.streams[8].language, "jpn");
}
/// A secondary block whose stream entry ends exactly at the end of the
/// PlayItem has no reference block at all; the count byte must not be
/// read from one-past-the-end. Covers all three secondary blocks that
/// carry reference data.
#[test]
fn secondary_ref_block_at_item_end_is_not_read() {
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
// Secondary audio is the last entry, with no ref bytes following.
let sec_audio = build_stream_entry_audio(0x1A00, 0x83, 3, 1, b"eng");
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 1, 0, 0, 0),
&[video.clone(), sec_audio],
);
let pl = parse(&data).expect("secondary audio at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1A00);
// Secondary video is the last entry, with no ref bytes following.
let sec_video = build_stream_entry_video(0x1B00, 0x1B, 4, 1, None);
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 1, 0, 0),
&[video.clone(), sec_video.clone()],
);
let pl = parse(&data).expect("secondary video at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1B00);
// Secondary video whose audio-ref block ends exactly at item end, so
// the PiP-PG ref count byte would sit one past it.
let mut sec_video_arefs = sec_video;
sec_video_arefs.extend_from_slice(&[0, 0]); // n_arefs = 0, reserved
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 1, 0, 0),
&[video.clone(), sec_video_arefs],
);
let pl = parse(&data).expect("secondary video aref block at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1B00);
// PiP PG is the last entry, with no ref bytes following.
let pip_pg = build_stream_entry_pg(0x1C00, 0x90, b"jpn");
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 0, 1, 0),
&[video, pip_pg],
);
let pl = parse(&data).expect("PiP PG at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1C00);
}
// ─────────────────────────────────────────────────────────────────────
// parse_stream_entry bounds, exercised directly.
// ─────────────────────────────────────────────────────────────────────
/// Fewer than 2 bytes remain for the stream_entry header
/// (length(1) + stream_entry_type(1)) → None, without reading either.
#[test]
fn stream_entry_header_past_end_is_none() {
let item = [0u8; 8];
for pos in 7..12usize {
assert!(
parse_stream_entry(&item, pos, STREAM_CATEGORY_VIDEO).is_none(),
"pos={pos}"
);
}
}
/// The stream_attributes header (length(1) + coding_type(1)) lies past
/// the end of the PlayItem → None, without reading the length byte.
#[test]
fn stream_attributes_header_past_end_is_none() {
// se_len = 3 → se_end = 4 == item.len(); the sa length byte would be
// at item[4] and the coding type at item[5].
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11];
assert!(parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).is_none());
}
/// A declared stream_attributes length of 0 has no coding_type byte and
/// must be rejected — even when the (empty) attribute region is itself
/// in bounds.
#[test]
fn zero_length_attributes_in_bounds_is_none() {
// se_len = 3 → se_end = 4; sa_len = item[4] = 0 → sa_end = 5 ≤ 6.
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11, 0, 0];
assert!(parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).is_none());
}
/// stream_attributes of exactly 1 byte carries only the coding_type.
/// That is the minimum the parser accepts, so the entry is returned
/// with its PID and coding_type and no format-specific fields — for a
/// PG stream the 3-byte language must NOT be read past the attributes.
#[test]
fn one_byte_stream_attributes_yields_bare_entry() {
// se_len = 3 → se_end = 4; sa_len = 1 → sa_end = 6 == item.len().
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11, 1, 0x1B];
let (entry, next) =
parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).expect("1-byte attrs are valid");
assert_eq!(entry.pid, 0x1011);
assert_eq!(entry.coding_type, 0x1B);
assert_eq!(entry.video_format, 0);
assert_eq!(entry.video_rate, 0);
assert_eq!(next, 6);
let pg = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x12, 0x00, 1, 0x90];
let (entry, _) = parse_stream_entry(&pg, 0, STREAM_CATEGORY_PG_SUBTITLE)
.expect("1-byte PG attrs are valid");
assert_eq!(entry.pid, 0x1200);
assert_eq!(entry.coding_type, 0x90);
assert_eq!(entry.language, "");
}
/// Type 0 is reserved and type 2 is a link point; neither is a chapter.
/// `labels::collect_chapter_summary` used to filter on `mark_type <= 1`,
/// which counted reserved marks and inflated the public `chapter_count`
/// (and let a playlist whose only marks are reserved pass the
/// `chapter_count == 0` skip). Both call sites now share this predicate.
#[test]
fn only_entry_marks_count_as_chapters() {
let mk = |mark_type| PlaylistMark {
mark_type,
play_item_ref: 0,
timestamp: 0,
};
assert!(
!mk(0).is_chapter_mark(),
"type 0 is reserved, not a chapter"
);
assert!(mk(1).is_chapter_mark());
assert!(!mk(2).is_chapter_mark(), "type 2 is a link point");
}
} }
+4 -566
View File
@@ -109,12 +109,6 @@ pub(crate) struct AuAssembler {
marks: VecDeque<Mark>, marks: VecDeque<Mark>,
/// Absolute offsets of fragments flagged with an upstream discontinuity. /// Absolute offsets of fragments flagged with an upstream discontinuity.
disc_marks: VecDeque<u64>, disc_marks: VecDeque<u64>,
/// A `MAX_AU_BUFFER` backstop discard happened and no AU has been emitted
/// since. Sticky rather than an offset mark, because the bytes it refers to
/// no longer exist: the discard is followed by a pre-sync trim that would
/// retire any mark placed at the new base, and the gap must outlive that.
/// Consumed by the next AU to emit. See `discard_gap_before`.
pending_gap: bool,
/// Incremental boundary-scan cursor: the offset into `buf` up to which the /// Incremental boundary-scan cursor: the offset into `buf` up to which the
/// current AU has already been searched for its end without finding one. Each /// current AU has already been searched for its end without finding one. Each
/// `push` resumes the boundary search from here instead of rescanning the /// `push` resumes the boundary search from here instead of rescanning the
@@ -132,12 +126,6 @@ pub(crate) struct AuAssembler {
/// so a long run of junk with no start code (hostile/corrupt input) costs /// so a long run of junk with no start code (hostile/corrupt input) costs
/// O(bytes) total, not O(buffer) per push. Reset when `buf[0]` moves. /// O(bytes) total, not O(buffer) per push. Reset when `buf[0]` moves.
opener_pos: usize, opener_pos: usize,
/// Test-only: how many times `take_front` fell back to the COPY path. The
/// handover is the whole point of `take_front`, so "did it actually fire" is a
/// property to MEASURE, not to reason about. See
/// `handover_survives_a_large_au_instead_of_copying_every_later_one`.
#[cfg(test)]
copy_path_hits: usize,
} }
impl AuAssembler { impl AuAssembler {
@@ -164,12 +152,9 @@ impl AuAssembler {
base: 0, base: 0,
marks: VecDeque::new(), marks: VecDeque::new(),
disc_marks: VecDeque::new(), disc_marks: VecDeque::new(),
pending_gap: false,
scan_pos: 0, scan_pos: 0,
seen_unit: false, seen_unit: false,
opener_pos: 0, opener_pos: 0,
#[cfg(test)]
copy_path_hits: 0,
} }
} }
@@ -184,12 +169,9 @@ impl AuAssembler {
base: 0, base: 0,
marks: VecDeque::new(), marks: VecDeque::new(),
disc_marks: VecDeque::new(), disc_marks: VecDeque::new(),
pending_gap: false,
scan_pos: 0, scan_pos: 0,
seen_unit: false, seen_unit: false,
opener_pos: 0, opener_pos: 0,
#[cfg(test)]
copy_path_hits: 0,
} }
} }
@@ -289,7 +271,7 @@ impl AuAssembler {
self.buf.drain(..drop); self.buf.drain(..drop);
self.base += drop as u64; self.base += drop as u64;
self.reset_scan(); self.reset_scan();
self.discard_gap_before(self.base); self.drop_marks_before(self.base);
} }
break; break;
}; };
@@ -331,10 +313,7 @@ impl AuAssembler {
dts = dts.or(m.dts); dts = dts.or(m.dts);
source = source.or(m.source); source = source.or(m.source);
} }
// A backstop discard is a gap in its own right, independent of any let mut discontinuity = false;
// upstream signal: bytes were thrown away, so this AU does not
// continue the last one emitted.
let mut discontinuity = std::mem::take(&mut self.pending_gap);
if self.disc_marks.front().is_some_and(|&o| o < end_abs) { if self.disc_marks.front().is_some_and(|&o| o < end_abs) {
discontinuity = true; discontinuity = true;
} }
@@ -342,7 +321,8 @@ impl AuAssembler {
self.disc_marks.pop_front(); self.disc_marks.pop_front();
} }
let data = self.take_front(end); let data = self.buf[..end].to_vec();
self.buf.drain(..end);
self.base += end as u64; self.base += end as u64;
self.reset_scan(); self.reset_scan();
out.push(AssembledAu { out.push(AssembledAu {
@@ -356,65 +336,6 @@ impl AuAssembler {
out out
} }
/// Detach `buf[..end]` as the emitted AU's own `Vec` and leave `buf` holding
/// the tail.
///
/// The AU's bytes are HANDED OVER — `buf`'s allocation becomes the returned
/// `Vec` and a fresh buffer (pre-sized to the same capacity, so the next AU
/// accumulates without re-growing) takes its place holding only the short
/// tail. `buf[..end].to_vec()` + `drain(..end)` instead copied every AU out
/// in full: on a UHD HEVC title that is a whole-frame memcpy (hundreds of KB)
/// per coded picture, ~200k times, for bytes that are about to be discarded
/// from `buf` anyway.
///
/// The allocation COUNT is unchanged (one per AU either way — the frame `Vec`
/// before, the replacement buffer now), so the only difference is the copy
/// that no longer happens. Nothing depends on `buf` keeping its identity: the
/// only state tied to `buf[0]`'s position is `base`/`scan_pos`/`opener_pos`,
/// which the caller updates immediately after.
///
/// Falls back to a copy when the buffer's capacity is far larger than the AU
/// (a small AU after a multi-MB one): handing over would otherwise attach an
/// oversized idle allocation to a small frame for as long as the frame queues
/// downstream, trading a copy for resident memory.
///
/// That fallback must not become permanent. `buf`'s capacity used to be a
/// one-way high-water mark — the replacement buffer was created with
/// `cap.max(tail_len)`, and the copy path's `drain` also preserves `cap` — so
/// once ONE large AU had been assembled, every later smaller AU satisfied
/// `cap > 2*end` and took the copy path forever. On a UHD HEVC title the first
/// IDR grows `buf` to ~4-8 MB, after which each ~200-400 KB P/B AU paid a
/// whole-AU allocation plus a whole-AU memcpy plus a tail memmove for ~99% of
/// the ~200,000 coded pictures — tens of GB of exactly the memcpy this handover
/// exists to remove. So the copy path now also RELEASES the high-water
/// capacity, which re-arms the handover for the next AU: one copy after a size
/// step down, not one per frame forever.
fn take_front(&mut self, end: usize) -> Vec<u8> {
let cap = self.buf.capacity();
let tail_len = self.buf.len() - end;
if cap > end.saturating_mul(2) {
#[cfg(test)]
{
self.copy_path_hits += 1;
}
let data = self.buf[..end].to_vec();
self.buf.drain(..end);
// Shrink toward what this AU actually needed (the tail plus room for
// another AU of about this size). Only the short tail is copied, and it
// brings `cap` back under the `2*end` threshold so the next AU of this
// size hands over instead of copying.
self.buf.shrink_to(end.max(tail_len));
return data;
}
// Replacement buffer: enough for the tail plus room to accumulate the next
// AU of about this size. NOT `cap`, which would re-pin the high-water mark.
let mut tail = Vec::with_capacity(end.max(tail_len));
tail.extend_from_slice(&self.buf[end..]);
let mut data = std::mem::replace(&mut self.buf, tail);
data.truncate(end);
data
}
/// Reset the incremental boundary-scan cursor. Called whenever `buf[0]` moves /// Reset the incremental boundary-scan cursor. Called whenever `buf[0]` moves
/// (an AU drained, or leading bytes discarded) so the next scan starts fresh /// (an AU drained, or leading bytes discarded) so the next scan starts fresh
/// from the new AU opener. /// from the new AU opener.
@@ -497,14 +418,6 @@ impl AuAssembler {
None None
} }
/// Retire every mark that falls before `off`, timing and discontinuity
/// alike.
///
/// This is the STREAM-START case: bytes ahead of the first AU boundary are
/// the tail of an access unit that began before we had sync, and there is
/// no prior AU for them to be discontinuous *from*. Carrying a mark forward
/// here would arm the resync gate at the head of every title and drop its
/// first GOP.
fn drop_marks_before(&mut self, off: u64) { fn drop_marks_before(&mut self, off: u64) {
while self.marks.front().is_some_and(|m| m.off < off) { while self.marks.front().is_some_and(|m| m.off < off) {
self.marks.pop_front(); self.marks.pop_front();
@@ -513,31 +426,6 @@ impl AuAssembler {
self.disc_marks.pop_front(); self.disc_marks.pop_front();
} }
} }
/// Retire stale timing marks before `off` and record that a GAP occurred
/// there.
///
/// This is the BACKSTOP case: `MAX_AU_BUFFER` bytes accumulated with no AU
/// start code in them, so the run is unusable and gets thrown away. Unlike
/// the stream-start trim above, there IS a prior AU here, and whatever
/// follows definitively does not continue it — a decoder handed the next
/// picture would resolve its references against frames separated from it by
/// megabytes of discarded data.
///
/// So the discard is itself a discontinuity, whether or not the source
/// signalled one. It is recorded as a sticky flag rather than an offset
/// mark because a mark placed at the new base would be retired moments
/// later by the pre-sync trim that follows resync — the gap has to outlive
/// the bytes that caused it. It arms the resync gate, which drops to the
/// next keyframe instead of emitting a picture with dangling references.
///
/// Timing marks before `off` are still retired — they describe bytes that
/// no longer exist, and the AU that eventually emits takes its PTS from the
/// fragment that actually opened it.
fn discard_gap_before(&mut self, off: u64) {
self.drop_marks_before(off);
self.pending_gap = true;
}
} }
/// Offset of the start code that opens the next AU in `buf` (at or after 0), or /// Offset of the start code that opens the next AU in `buf` (at or after 0), or
@@ -943,131 +831,6 @@ mod tests {
); );
} }
/// The 8 MiB backstop throws away a start-code-free run as unusable. The
/// AU that eventually emits after that discard MUST be marked
/// discontinuous, whether or not the source ever signalled a
/// discontinuity: megabytes of the stream are simply gone, so the next
/// picture cannot resolve its references against the last one that was
/// emitted.
///
/// `discontinuity` is what arms the resync gate downstream
/// (`resync.rs`, driven from `mux/disc.rs`), which drops to the next
/// keyframe rather than emitting a picture with dangling references. If the
/// flag is retired with the discarded bytes, the gate never arms and the
/// broken picture goes out — a silent corruption, which is the one class of
/// loss this crate refuses to have.
#[test]
fn a_backstop_discard_marks_the_next_au_discontinuous() {
let mut a = AuAssembler::for_codec(Codec::H264);
// A clean AU first, so there IS a prior AU to be discontinuous from.
let first = au(0x11, 64);
let mut stream = first.clone();
stream.extend_from_slice(AUD);
let out = a.push(&stream, Some(1000), None, None, false);
assert_eq!(out.len(), 1, "the first AU emits normally");
assert!(
!out[0].discontinuity,
"an ordinary AU at the head of a clean run is continuous"
);
// Now start-code-free junk past the cap. The FIRST over-cap run still
// has the next AU's delimiter at buf[0], so it is force-flushed as an
// (over-long) access unit — nothing is discarded and nothing is lost.
// Only once the buffer holds no opener at all does the backstop throw
// bytes away, which is the case this test is about.
let junk = vec![0xAB; MAX_AU_BUFFER + 4096];
a.push(&junk, Some(2000), None, None, false);
a.push(&junk, Some(2100), None, None, false);
// Resync: a fresh AU, followed by the delimiter that closes it.
let mut resumed = au(0x22, 64);
resumed.extend_from_slice(AUD);
let out = a.push(&resumed, Some(3000), None, None, false);
let au2 = out
.iter()
.find(|x| x.data.contains(&0x22))
.expect("the post-gap AU must emit");
assert!(
au2.discontinuity,
"the AU following an 8 MiB backstop discard follows a gap and must \
say so; without the flag the resync gate never arms and a picture \
with dangling references is emitted as if it were sound"
);
}
/// The opposite case, and the reason the two call sites are separate.
///
/// Bytes ahead of the FIRST access-unit delimiter are the tail of an AU
/// that began before we had sync. There is no prior AU for them to be
/// discontinuous from, so retiring the marks there is right — and
/// necessary: marking the first AU of every title discontinuous would arm
/// the resync gate at the head of each one and drop its opening GOP.
#[test]
fn a_stream_start_trim_does_not_mark_the_first_au_discontinuous() {
let mut a = AuAssembler::for_codec(Codec::H264);
// Junk BEFORE the first delimiter — a partial AU from before sync.
// Small enough that the backstop never fires; this is the a0 > 0 path.
let mut stream = vec![0xCD; 512];
stream.extend_from_slice(&au(0x33, 64));
stream.extend_from_slice(AUD);
let out = a.push(&stream, Some(1000), None, None, false);
let first = out.first().expect("the first synced AU must emit");
assert!(
!first.discontinuity,
"trimming pre-sync bytes at stream start is not a gap in the \
stream; flagging it would drop the opening GOP of every title"
);
}
/// A source-signalled discontinuity reaches the AU it opens.
///
/// This is the `disc_marks` path — the ORIGINAL mechanism, distinct from
/// the sticky `pending_gap` the backstop sets. Nothing else pins it: the
/// two tests above drive `pending_gap`, and a mark placed on a fragment
/// that is later discarded is retired by design.
///
/// Deliberately NOT combined with the backstop. A previous version of this
/// test signalled the discontinuity on the first over-cap push and asserted
/// the flag on the AU after the discard — but that first run still has the
/// next AU's delimiter at `buf[0]`, so it force-flushes as an over-long AU,
/// and THAT AU consumes the mark. The assertion was then satisfied entirely
/// by `pending_gap`, making the test a duplicate of the one above it under
/// a name promising something else. The two mechanisms cannot be isolated
/// in one fixture, so they get one test each.
#[test]
fn a_source_signalled_discontinuity_reaches_the_au_it_opens() {
let mut a = AuAssembler::for_codec(Codec::H264);
// A clean AU first, so there is a prior AU and the gate has somewhere
// to be discontinuous FROM.
let mut first = au(0x11, 64);
first.extend_from_slice(AUD);
let out = a.push(&first, Some(1000), None, None, false);
assert_eq!(out.len(), 1);
assert!(!out[0].discontinuity, "a clean run is continuous");
// The source flags this fragment as following a gap. It carries the
// body of the next AU and its closing delimiter, so it is emitted
// rather than discarded — the mark must ride through to it.
let mut second = au(0x22, 64);
second.extend_from_slice(AUD);
let out = a.push(&second, Some(2000), None, None, true);
let au2 = out
.iter()
.find(|x| x.data.contains(&0x22))
.expect("the flagged AU must emit");
assert!(
au2.discontinuity,
"a discontinuity the SOURCE signalled must reach the AU whose bytes \
carried it; this is the disc_marks path and no other test drives it"
);
}
#[test] #[test]
fn over_cap_without_boundary_force_flushes() { fn over_cap_without_boundary_force_flushes() {
let mut a = AuAssembler::for_codec(Codec::H264); let mut a = AuAssembler::for_codec(Codec::H264);
@@ -1078,329 +841,4 @@ mod tests {
"over-cap AU is force-flushed, not buffered forever" "over-cap AU is force-flushed, not buffered forever"
); );
} }
/// MEASURED: a drained AU must be HANDED the accumulation buffer's
/// allocation, not copied out of it. The emitted `Vec`'s data pointer is the
/// buffer's own pointer — which is only true if no full-frame copy happened.
/// (`buf[..end].to_vec()` allocates fresh, so the pointers differ.) One
/// whole-AU memcpy per coded picture is ~200k memcpys of a few hundred KB
/// each on a UHD feature.
#[test]
fn drained_au_takes_over_the_buffer_allocation_without_copying() {
let mut a = AuAssembler::for_codec(Codec::H264);
// An AU large enough that the buffer's capacity is not >2x its size (the
// small-AU copy path exists so a small frame cannot carry an oversized
// idle allocation downstream).
let au1 = au(0x11, 400 * 1024);
let au2 = au(0x22, 400 * 1024);
let mut stream = au1.clone();
stream.extend_from_slice(&au2);
// Push everything except the final byte of AU2's delimiter, so no AU has
// been emitted yet but the buffer holds the whole of AU1.
a.push(&stream[..au1.len() + 3], Some(1), None, None, false);
let before = a.buf.as_ptr();
let cap_before = a.buf.capacity();
let out = a.push(
&stream[au1.len() + 3..au1.len() + 4],
None,
None,
None,
false,
);
assert_eq!(out.len(), 1);
assert_eq!(
out[0].data, au1,
"handover must preserve the AU bytes exactly"
);
assert_eq!(
out[0].data.as_ptr(),
before,
"the emitted AU must own the buffer's allocation (no whole-frame copy)"
);
// The replacement buffer keeps room for another AU of about this size, so
// the next AU does not re-grow — but it is NOT pinned to the OLD capacity,
// which would make `buf` a permanent high-water mark and send every later
// smaller AU down the copy path (see
// `handover_survives_a_large_au_instead_of_copying_every_later_one`).
assert!(
a.buf.capacity() >= au1.len(),
"replacement buffer must fit another AU of this size: {} < {}",
a.buf.capacity(),
au1.len()
);
assert!(
a.buf.capacity() <= cap_before,
"replacement buffer must never EXCEED the old capacity"
);
assert_eq!(a.buf.len(), 4, "the buffer holds only AU2's delimiter tail");
}
/// MEASURED: `take_front`'s copy fallback must not become permanent.
///
/// `buf`'s capacity used to be a one-way high-water mark, and the copy path's
/// `drain` preserves it, so after ONE large AU every later smaller AU satisfied
/// `cap > 2*end` and copied forever. On a UHD HEVC title the first IDR grows
/// `buf` to multiple MB, after which ~99% of the ~200,000 coded pictures each
/// paid a whole-AU allocation + whole-AU memcpy + tail memmove — tens of GB of
/// exactly the copy the handover exists to remove. Counted at the copy path
/// itself: one copy is expected right after the size step down; a per-frame
/// copy is the bug.
#[test]
fn handover_survives_a_large_au_instead_of_copying_every_later_one() {
let mut a = AuAssembler::for_codec(Codec::H264);
// Production shape: BD-TS aligns one access unit per PES, so each `push`
// carries about one AU and the buffer holds ~one AU at a time. One large AU
// (the IDR) followed by a run of much smaller ones (P/B frames). Each AU is
// pushed with the NEXT AU's opener so the previous one closes.
const SMALL: usize = 64 * 1024;
let mut pending = au(0x11, 2 * 1024 * 1024);
for i in 0..20u8 {
let next = au(0x30 + i, SMALL);
// Append the next AU's 4-byte opener to close `pending`, push, and
// carry the rest of `next` forward.
pending.extend_from_slice(&next[..4]);
a.push(&pending, Some(1), None, None, false);
pending = next[4..].to_vec();
}
let hits = a.copy_path_hits;
assert!(
hits <= 2,
"the copy fallback must re-arm the handover, not fire for every AU \
after a large one: {hits} copies over 20 access units"
);
}
// ── AU-opener detection: the per-mode start-code rule ─────────────────
//
// `au_opener_from` is the SECOND implementation of a rule each codec parser
// also encodes (h264 `NAL_AUD`, hevc `NAL_AUD`, vc1 `SC_*`, mpeg2
// `PICTURE_CODE`/`SEQ_HEADER_CODE`/`GOP_CODE`). Two independent copies of one
// rule drift; these cases pin this copy to the normative byte values and to
// the codes that are explicitly NOT openers, so a drift shows up here.
/// The opener offset must be the position of the real start code, never a
/// fixed 0. A constant `Some(0)` makes every pre-sync run of junk bytes look
/// like the head of an access unit, so the first AU of every stream that does
/// not begin exactly on a start code is emitted with junk glued to its front.
#[test]
fn au_opener_from_locates_the_real_start_code_per_codec() {
// Junk that contains a start-code PREFIX but no opener suffix, so a
// scanner that stopped at `00 00 01` alone would answer wrongly.
let junk: &[u8] = &[0xFF, 0x00, 0x00, 0x01, 0x67, 0xAA];
let cases: &[(Mode, u8, &str)] = &[
// ISO/IEC 14496-10 §7.4.1: nal_unit_type 9 = access unit delimiter,
// and nal_ref_idc shall be 0 for it, so the header byte is 0x09.
(Mode::StartCode(0x09), 0x09, "H.264 AUD"),
// ITU-T H.265 §7.4.2.2: nal_unit_type 35 = AUD_NUT. The first NAL
// header byte is forbidden_zero_bit(1) | nal_unit_type(6) |
// nuh_layer_id MSB(1) = (35 << 1) = 0x46 on the base layer.
(Mode::StartCode(0x46), 0x46, "HEVC AUD"),
// SMPTE 421M Annex E BDU types.
(Mode::Vc1, VC1_SEQ, "VC-1 sequence header"),
(Mode::Vc1, VC1_ENTRY, "VC-1 entry point"),
(Mode::Vc1, VC1_FRAME, "VC-1 frame"),
// ISO/IEC 13818-2 §6.2.1 Table 6-1 start code values.
(Mode::Mpeg2, MP2_PICTURE, "MPEG-2 picture"),
(Mode::Mpeg2, MP2_SEQ, "MPEG-2 sequence header"),
(Mode::Mpeg2, MP2_GOP, "MPEG-2 GOP header"),
];
for &(mode, code, what) in cases {
let mut buf = junk.to_vec();
buf.extend_from_slice(&[0x00, 0x00, 0x01, code, 0x5A]);
assert_eq!(
au_opener_from(mode, &buf, 0),
Some(junk.len()),
"{what}: opener must be found at the start code, not at 0"
);
// `from` must actually skip: searching past the only opener finds none.
assert_eq!(
au_opener_from(mode, &buf, junk.len() + 1),
None,
"{what}: the resume cursor must be honoured"
);
}
}
/// Start codes that are NOT access-unit openers must not be reported as one.
/// Treating a slice or an extension header as an AU start splits one coded
/// picture into several frames, each missing its picture header.
#[test]
fn non_opening_start_codes_are_not_au_openers() {
// ISO/IEC 13818-2 Table 6-1: slice (0x01..=0xAF), user data (0xB2),
// extension (0xB5), sequence end (0xB7) all appear INSIDE an access unit.
for code in [0x01u8, 0xAF, 0xB2, 0xB5, 0xB7] {
let buf = [0x00, 0x00, 0x01, code, 0x11, 0x22];
assert_eq!(
au_opener_from(Mode::Mpeg2, &buf, 0),
None,
"MPEG-2 start code {code:#04x} must not open an access unit"
);
}
// SMPTE 421M: slice (0x0B) and field (0x0C) BDUs belong to the frame
// already in progress; end-of-sequence (0x0A) opens nothing.
for code in [0x0Au8, 0x0B, 0x0C] {
let buf = [0x00, 0x00, 0x01, code, 0x11, 0x22];
assert_eq!(
au_opener_from(Mode::Vc1, &buf, 0),
None,
"VC-1 BDU {code:#04x} must not open an access unit"
);
}
// H.264: an SPS (7) / PPS (8) / IDR slice (5) is not the AU DELIMITER the
// StartCode mode splits on.
for code in [0x05u8, 0x67, 0x68] {
let buf = [0x00, 0x00, 0x01, code, 0x11, 0x22];
assert_eq!(au_opener_from(Mode::StartCode(0x09), &buf, 0), None);
}
// Passthrough never frames — the codec self-frames.
assert_eq!(
au_opener_from(Mode::Passthrough, &[0, 0, 1, 0x09, 0xAA], 0),
None
);
}
/// `au_opener_resumable` must return the true offset AND advance
/// `opener_pos` only over bytes that cannot hide a straddling start code.
/// A constant `Some(0)` short-circuits both.
#[test]
fn au_opener_resumable_reports_the_real_offset_and_resumes_safely() {
let mut a = AuAssembler::for_codec(Codec::H264);
// A junk run with no opener: None, and the cursor parks 3 bytes back so a
// start code split across the append boundary is still found.
a.buf.extend_from_slice(&[0xFFu8; 32]);
assert_eq!(a.au_opener_resumable(), None, "no opener in a junk run");
assert_eq!(
a.opener_pos, 29,
"resume 3 bytes back for a straddling code"
);
// Now append a start code that STRADDLES the previous end: the first three
// bytes of `00 00 01 09` land at offsets 29..32.
a.buf.truncate(29);
a.buf.extend_from_slice(&[0x00, 0x00, 0x01, 0x09, 0x77]);
assert_eq!(
a.au_opener_resumable(),
Some(29),
"a start code straddling the previous scan end must still be found"
);
}
/// After the pre-sync bytes are discarded, the emitted AU must take the
/// timing of the fragment that ACTUALLY opened it. `drop_marks_before` is
/// what retires the discarded fragment's marks; a no-op there stamps the
/// first real access unit with the PTS and source of bytes that were thrown
/// away — a whole-title A/V sync offset, since every later frame is timed
/// relative to it.
#[test]
fn discarded_pre_sync_marks_do_not_time_the_first_access_unit() {
let src = |b: u64| SourcePos {
byte: b,
..Default::default()
};
let mut a = AuAssembler::for_codec(Codec::H264);
// Fragment 1: pre-sync junk, no start code. Carries its own PTS/source.
assert!(
a.push(&[0xFFu8; 24], Some(1_000), Some(900), Some(src(11)), false)
.is_empty()
);
// Fragment 2: the first real AU opener, with the timing that belongs to it.
assert!(
a.push(
&au(0x33, 40),
Some(2_000),
Some(1_900),
Some(src(22)),
false
)
.is_empty()
);
// Fragment 3: a second AU, closing the first.
let out = a.push(
&au(0x44, 40),
Some(3_000),
Some(2_900),
Some(src(33)),
false,
);
assert_eq!(out.len(), 1, "the first AU closes on the second opener");
assert_eq!(out[0].data, au(0x33, 40), "junk discarded, AU intact");
assert_eq!(
out[0].pts,
Some(2_000),
"the AU must take the opening fragment's PTS, not the discarded junk's"
);
assert_eq!(out[0].dts, Some(1_900), "same for DTS");
assert_eq!(
out[0].source.map(|s| s.byte),
Some(22),
"same for the source position used by the recovery map"
);
let tail = a.flush();
assert_eq!(tail.len(), 1);
assert_eq!(tail[0].pts, Some(3_000), "the second AU keeps its own PTS");
}
/// `for_codec` is the dispatch that decides whether a stream is REASSEMBLED
/// across PES fragments or passed straight through. Getting it wrong is
/// silent: an H.264/HEVC/VC-1 stream routed to `Passthrough` on a program
/// source emits one "frame" per PES fragment — a few hundred bytes of a
/// coded picture, framed as a whole access unit — and the output plays as
/// corruption, not as an error.
///
/// Each mode is identified BEHAVIOURALLY (feed a two-AU stream in two halves
/// and see whether it reassembles), so the case cannot pass by matching a
/// constant.
#[test]
fn for_codec_routes_each_video_codec_to_its_reassembly_mode() {
// Buffering codecs: a stream split mid-AU must NOT emit until the second
// AU's opener arrives, and must then emit the FIRST AU whole.
let buffering: &[(Codec, u8)] = &[
(Codec::H264, 0x09), // ISO/IEC 14496-10 §7.4.1 AUD
(Codec::Hevc, 0x46), // ITU-T H.265 §7.4.2.2 AUD_NUT, (35 << 1)
];
for &(codec, marker) in buffering {
let mut a = AuAssembler::for_codec(codec);
let mut unit = vec![0x00, 0x00, 0x01, marker];
unit.extend(std::iter::repeat_n(0x5Au8, 30));
// First half of AU 1: nothing complete yet.
assert!(
a.push(&unit[..20], Some(1), None, None, false).is_empty(),
"{codec:?} must buffer a partial access unit, not emit it"
);
assert!(
a.push(&unit[20..], None, None, None, false).is_empty(),
"{codec:?} must hold AU 1 until the next opener"
);
// AU 2's opener closes AU 1.
let out = a.push(&unit, Some(2), None, None, false);
assert_eq!(out.len(), 1, "{codec:?} emits exactly one AU");
assert_eq!(out[0].data, unit, "{codec:?} reassembles AU 1 whole");
assert_eq!(out[0].pts, Some(1), "{codec:?} carries the AU-start PTS");
}
// VC-1 buffers too, on its own boundary rule (no single AU delimiter).
let mut a = AuAssembler::for_codec(Codec::Vc1);
let frame = bdu(VC1_FRAME, 0x77, 30);
assert!(a.push(&frame, Some(1), None, None, false).is_empty());
assert_eq!(
a.push(&frame, Some(2), None, None, false).len(),
1,
"VC-1 emits AU 1 when the next frame BDU opens AU 2"
);
// Self-framing codecs pass each fragment through immediately — the same
// half-AU input that the buffering modes held back comes straight out.
for codec in [Codec::Mpeg2, Codec::Ac3, Codec::TrueHd, Codec::Pgs] {
let mut a = AuAssembler::for_codec(codec);
let out = a.push(&[0x00, 0x00, 0x01, 0x09, 0xAA], Some(7), None, None, false);
assert_eq!(out.len(), 1, "{codec:?} must pass through, not buffer");
assert_eq!(out[0].pts, Some(7));
assert!(a.flush().is_empty(), "{codec:?} buffers nothing at EOF");
}
}
} }
+169 -1235
View File
File diff suppressed because it is too large Load Diff
+3 -98
View File
@@ -50,15 +50,7 @@ fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5]. // aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length = let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5); ((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
// The floor is the header the frame SAYS it carries, not a constant. if frame_length < 7 {
// protection_absent (byte1 bit0) clear means a 16-bit crc_check follows the
// 7-byte fixed+variable header, so the frame cannot be shorter than 9 —
// aac_frame_length counts the header and the CRC, not just the payload.
// Comparing against a flat 7 let a CRC-present frame declaring 7 or 8
// through as structurally Valid, and the muxer then carried a frame whose
// own header says it is impossible.
let header_bytes = if data[1] & 0x01 == 0 { 9 } else { 7 };
if frame_length < header_bytes {
return AdtsVerdict::Invalid; return AdtsVerdict::Invalid;
} }
AdtsVerdict::Valid AdtsVerdict::Valid
@@ -120,14 +112,10 @@ impl CodecParser for AdtsParser {
} }
self.tally.record_kept(); self.tally.record_kept();
// One PES is one unit here, so the unit's first byte is in THIS packet
// and its facts are this packet's -- the same rule the buffering
// parsers apply through `PesBuf::front`, with nothing carried over.
let facts = super::pesbuf::PesFacts::of(pes);
vec![Frame { vec![Frame {
discontinuity: facts.discontinuity, discontinuity: pes.discontinuity,
coding: None, coding: None,
source: facts.source, source: None,
pts_ns, pts_ns,
keyframe: true, keyframe: true,
data: pes.data.clone(), data: pes.data.clone(),
@@ -160,51 +148,6 @@ mod tests {
} }
} }
/// A header that CLAIMS a CRC (protection_absent = 0) but declares a
/// frame length too short to contain one.
///
/// `aac_frame_length` counts the header and the CRC, not just the payload,
/// so with a CRC present the smallest structurally possible frame is 9
/// bytes: the 7-byte fixed+variable header plus the 16-bit crc_check.
/// The gate compared against a flat 7 and never read protection_absent at
/// all, so a frame whose own header says it is impossible was classified
/// Valid and forwarded to the muxer.
#[test]
fn a_crc_present_header_shorter_than_its_own_crc_is_invalid() {
for declared in [7u32, 8] {
let mut f = adts_frame(16);
f[1] = 0xF0; // sync + MPEG-4, protection_absent = 0 => CRC present
f[3] = (f[3] & 0xFC) | ((declared >> 11) & 0x03) as u8;
f[4] = ((declared >> 3) & 0xFF) as u8;
f[5] = (f[5] & 0x1F) | ((declared & 0x07) << 5) as u8;
assert!(
matches!(adts_verdict(&f), AdtsVerdict::Invalid),
"protection_absent=0 declaring {declared} bytes cannot hold its \
own 7-byte header plus a 2-byte CRC"
);
}
// 9 is the smallest length that CAN hold header + CRC, so it must pass
// the structural gate — the floor moved, it did not become stricter
// than the spec.
let mut ok = adts_frame(16);
ok[1] = 0xF0;
let nine = 9u32;
ok[3] = (ok[3] & 0xFC) | ((nine >> 11) & 0x03) as u8;
ok[4] = ((nine >> 3) & 0xFF) as u8;
ok[5] = (ok[5] & 0x1F) | ((nine & 0x07) << 5) as u8;
assert!(matches!(adts_verdict(&ok), AdtsVerdict::Valid));
// And with NO CRC the floor is still 7, unchanged.
let mut no_crc = adts_frame(16);
no_crc[1] = 0xF1; // protection_absent = 1
let seven = 7u32;
no_crc[3] = (no_crc[3] & 0xFC) | ((seven >> 11) & 0x03) as u8;
no_crc[4] = ((seven >> 3) & 0xFF) as u8;
no_crc[5] = (no_crc[5] & 0x1F) | ((seven & 0x07) << 5) as u8;
assert!(matches!(adts_verdict(&no_crc), AdtsVerdict::Valid));
}
/// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with /// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with
/// aac_frame_length set to the total size. /// aac_frame_length set to the total size.
fn adts_frame(payload: usize) -> Vec<u8> { fn adts_frame(payload: usize) -> Vec<u8> {
@@ -247,33 +190,6 @@ mod tests {
); );
} }
/// A dropped ADTS frame is dropped BECAUSE its header failed validation, so
/// the very fields a duration would come from (sampling_frequency_index, and
/// the 1024-samples-per-AAC-frame constant applied to it) are the ones known
/// to be untrustworthy. This gate therefore reports the drop's duration as
/// zero rather than deriving a number from a header it has just rejected —
/// the honest answer, and the one the count alongside it must be read with.
/// A nonzero constant here would report silence that was never measured.
#[test]
fn dropped_frames_are_counted_but_their_duration_is_not_invented() {
let mut parser = AdtsParser::new();
// Three frames whose sampling_frequency_index is a reserved value (13),
// so `adts_verdict` rejects each one.
let mut bad = adts_frame(32);
bad[2] = (bad[2] & 0b1100_0011) | (13 << 2);
for i in 0..3 {
let out = parser.parse(&make_pes(bad.clone(), Some(i * 90_000)));
assert!(out.is_empty(), "an invalid ADTS frame is not emitted");
}
assert_eq!(parser.dropped_frames(), 3, "every drop is counted");
assert_eq!(
parser.dropped_duration_ns(),
0,
"the duration comes from the header that just failed validation, so \
it is reported as unmeasured rather than guessed"
);
}
#[test] #[test]
fn reserved_sample_rate_index_is_dropped() { fn reserved_sample_rate_index_is_dropped() {
// sr_index = 13 (reserved). byte2 bits5..2 = 1101 → 0x34. // sr_index = 13 (reserved). byte2 bits5..2 = 1101 → 0x34.
@@ -329,15 +245,4 @@ mod tests {
let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0))); let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0)));
assert_eq!(f.len(), 1, "too short to validate → kept"); assert_eq!(f.len(), 1, "too short to validate → kept");
} }
/// One PES is one unit here, so the frame carries that packet's offset.
#[test]
fn a_frame_carries_its_packets_source() {
let mut parser = AdtsParser::new();
let mut p = make_pes(adts_frame(64), Some(90_000));
p.source = Some(crate::pes::SourcePos::at_byte(4_242));
let frames = parser.parse(&p);
assert!(!frames.is_empty(), "a valid ADTS frame is emitted");
assert_eq!(frames[0].source.map(|s| s.byte), Some(4_242));
}
} }
+6 -18
View File
@@ -29,17 +29,10 @@ pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
} }
/// CRC-16 with polynomial 0x002D, init 0, MSB-first, used by the MLP / Dolby /// CRC-16 with polynomial 0x002D, init 0, MSB-first, used by the MLP / Dolby
/// TrueHD major-sync header checksum. /// TrueHD major-sync header checksum. NOTE: MLP's checksum is the "reversed"
/// /// scheme — the stored trailer word is the little-endian-read CRC, so this
/// NOTE: MLP's checksum is the "reversed" scheme. This function emits its two /// standard CRC must be compared against the stored bytes read big-endian.
/// bytes in the OPPOSITE order to a standard little-endian CRC readout, so the /// The caller handles that comparison (see `truehd::mlp_major_sync_ok`).
/// caller swaps them back and compares against the stored trailer word read
/// LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
///
/// Comparing big-endian instead is precisely the bug that function was fixed
/// for: it could never validate a real extended major sync, so whole TrueHD
/// tracks were dropped silently. This comment used to prescribe exactly that,
/// and to point at a `truehd::mlp_major_sync_ok` that does not exist.
/// Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs). /// Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 { pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0; let mut crc: u16 = 0;
@@ -110,13 +103,8 @@ mod tests {
#[test] #[test]
fn crc16_mlp_residue_property_holds() { fn crc16_mlp_residue_property_holds() {
// Appending the big-endian CRC zeroes the residue over message+crc. // Appending the big-endian CRC zeroes the residue over message+crc — the
// This is a property of the CRC itself, pinned here so a change to the // scheme `truehd::mlp_major_sync_ok` relies on.
// polynomial or the bit order is caught. It is NOT how the TrueHD
// caller validates a major sync: `truehd::mlp_major_sync_crc_ok` does a
// swap-and-XOR compare against the little-endian trailer word. (This
// comment used to claim the caller relied on the residue, and named a
// `truehd::mlp_major_sync_ok` that does not exist.)
let msg = [0xF8u8, 0x72, 0x6F, 0xBA]; let msg = [0xF8u8, 0x72, 0x6F, 0xBA];
let c = crc16_mlp(&msg); let c = crc16_mlp(&msg);
let mut framed = msg.to_vec(); let mut framed = msg.to_vec();
-25
View File
@@ -200,31 +200,6 @@ mod tests {
assert!(!t.is_poisoned()); assert!(!t.is_poisoned());
} }
/// The poison verdict is a RATIO — verified drops against every AU seen — so
/// the kept count is half of it. `does_not_poison_a_mostly_good_track` above
/// records its keeps AFTER the single drop, and `maybe_poison` only runs
/// inside `record_drop`, so the keeps are never in the denominator when the
/// verdict is actually computed: that test passes even with the kept count
/// never incremented. Interleaving them puts the kept count on the critical
/// path, where losing it turns the ratio into "verified drops vs verified
/// drops" — always >50% — and silently discards a healthy track.
#[test]
fn interleaved_keeps_are_in_the_poison_denominator() {
let mut t = DropTally::new("test");
// 2 kept per 1 dropped, well past the minimum-AU gate: a third of the
// track is undecodable, which is bad but nowhere near the >50% threshold.
for _ in 0..(TRACK_VERDICT_MIN_AUS * 3) {
t.record_kept();
t.record_kept();
t.record_drop(0, 1000, 512, "bad");
assert!(
!t.is_poisoned(),
"33% dropped must never poison, at any point in the run"
);
}
assert_eq!(t.dropped_frames(), TRACK_VERDICT_MIN_AUS * 3);
}
#[test] #[test]
fn collateral_drops_never_poison_the_track() { fn collateral_drops_never_poison_the_track() {
// A TrueHD resync-forward run collaterally drops a long burst of AUs, but // A TrueHD resync-forward run collaterally drops a long burst of AUs, but

Some files were not shown because too many files have changed in this diff Show More