Compare commits

...
24 Commits
Author SHA1 Message Date
Matthew Jackson 497b7726dc v1.6.6: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 20:03:44 -07:00
Matthew Jackson 94b549a9d8 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson fc2d35a19c v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew Jackson 3c51190c32 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson dfe0390445 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 8678a2bf94 drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson d953b46023 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 443ac6bcb1 Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 794393f24e CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson c6e2614c4c Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson b02c402dae CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 128e3bb120 Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson dc100f1c80 Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 286b5d4527 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson d9f3f82c65 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson e30209091e restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
Matthew Jackson ac46e2382f v1.6.4: bump version (freemkv-unlock git-pinned for the tag) 2026-08-15 09:13:26 -07:00
Matthew Jackson 862c1460c3 Trim single-clip titles to their playlist marks (1.6.4) 2026-08-15 08:14:31 -07:00
Matthew Jackson fea64d07e2 Stop an uncrackable VTS borrowing another VTS's title key
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.

`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.

Also, three things nothing could catch:

- The demux output filename took the stream's language raw while the
  base beside it was sanitised. A language code is three raw STN bytes
  through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
  encoding on real discs — a NUL in a path fails `File::create` with
  InvalidInput, taking the whole export down before one track file
  opened. `sanitize` now maps control characters too; it did not.

- `css::crack_key_scan`'s short-read handling was dead code under test:
  every source in the module returned the full request, so reverting
  `advance` to the requested count, or dropping the `.max(1)`, left the
  suite green. The `.max(1)` is load-bearing — without it a source
  returning `Ok(0)` never moves the cursor and never increments the
  budget, so the scan spins forever. That mutation now HANGS the test
  rather than failing it, which is the honest demonstration.

- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
  `#[cfg(test)] mod tests`, so it compiled in no configuration and could
  never fire — the dead gate the test above it was written to replace.
  Moved to module scope, and verified it now rejects a wrong constant at
  compile time.
2026-08-11 17:47:56 -07:00
Matthew Jackson 6e8da15c34 Bound the third hostile CSV, and stop two docs overclaiming
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.

The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.

`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
2026-08-11 16:59:33 -07:00
Matthew Jackson 8898351cb1 Round 2: stop two of round 1's tests claiming more than they prove
Both are mine, and both pass with the bound they "cover" deleted.

`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.

`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.

Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
2026-08-11 11:54:28 -07:00
Matthew Jackson 7a03be5496 Bound the commentary-index parse, and stop proving it with a clock
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.

It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.

`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.

The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.

Applies to both `sub_com1_idx` and `aud_com1_idx`.

NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
2026-08-11 11:30:53 -07:00
Matthew Jackson de94e12864 Cover the dir:// input door, and correct the CSS scramble-gate docs
Three round-1 audit findings on v1.6.0..HEAD.

Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.

Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.

Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
2026-08-11 10:23:37 -07:00
Matthew Jackson 21ec1d3f80 restore freemkv-unlock path dep for local dev (post-v1.6.3) 2026-08-10 09:04:40 -07:00
38 changed files with 8253 additions and 808 deletions
+15
View File
@@ -46,3 +46,18 @@ updates:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+162
View File
@@ -1,5 +1,167 @@
# Changelog
## [1.6.6] — 2026-08-20
### Changed
- Version aligned to 1.6.6 for the unified release. No functional changes
to this crate; the release was driven by autorip (webhooks may now target
private/LAN addresses — see the autorip 1.6.6 notes).
## [1.6.5] — 2026-08-20
### Fixed
- **On some Blu-ray/UHD titles the picture stopped short of the declared
end.** When a title is stitched across a clip seam, freemkv drops video
after the join until it sees a fresh keyframe it can restart decoding
from. It only recognised one kind of keyframe (an IDR frame), but a
Blu-ray's final segment can open on a different kind of self-contained
frame instead — so on those discs the restart point was never
recognised and every remaining frame to the end of the film was
dropped, losing up to a group-of-pictures-plus tail of picture. That
kind of frame is now recognised as a valid restart point, matching how
the HEVC path already worked, so the tail is kept. HEVC and DVD titles
were never affected.
- **A split HD-DVD feature could be exported as half the film while still
claiming the full running time.** A feature stored across parts (for
example `FEATURE_1.EVO` + `FEATURE_2.EVO`) is composed back into one
title. If a part resolved with no usable data — a zero-length file, or
an extent map that yielded nothing — that part was neither used nor
flagged, so the feature was quietly composed from the surviving parts
alone, still advertising the whole runtime, and exited success with no
log. The missing part is now marked unusable and logged under its own
code (E6019), so half a movie can no longer be presented as a whole one.
- **A Blu-ray title with an unreadable clip could be exported short while
still reporting the full running time.** A scratched or malformed
clip-info (`.clpi`) sector, or a clip whose extents could not be
resolved (a bad sector, a broken allocation chain, an embedded-data
file), was silently skipped — but the title's duration had already been
counted from the playlist, so it shipped short of the runtime it claimed
at exit success, with nothing logged. Previously only one narrow failure
kind counted; now every unresolvable clip drops the title and warns with
the read's own error code. A truly absent optional file (such as the 3D
`.ssif` on a 2D disc) is still treated as benign.
- **Pressing Stop during a disc scan could leave the disc looking like it
holds fewer titles than it does — or none at all.** Once cancelled,
every remaining drive command fails, but the Blu-ray and DVD title
enumerators treated those failures as ordinary skips: the Blu-ray scan
returned a truncated title list and the DVD scan returned zero titles,
both at exit success, so a cancelled scan was indistinguishable from a
disc that simply held that many titles. A cancel is now propagated as an
error out of every enumerator and at every read site, so it can never be
cached, displayed, or ripped from as if it were the real disc. HD-DVD
already behaved this way and is the model the others now follow.
- **A decrypted HD-DVD could be refused with a DVD copy-protection error
(E7023).** HD-DVD and DVD share the same MPEG program-stream container,
and the scramble detector mistook an HD-DVD navigation packet for a
CSS-scrambled sector — so a good, already-decrypted HD-DVD was reported
as carrying an unrecoverable CSS key and hard-failed, sending anyone
triaging it hunting for a missing DVD key on a disc that never had one.
The detector now excludes the structural packet types that CSS never
scrambles, identified from a field that is readable even on ciphertext,
so a decrypted HD-DVD scans clean. The DVD CSS crack is unchanged and a
genuinely scrambled, uncrackable DVD still hard-fails as before, so
ciphertext can never be muxed as plaintext.
- **A file with an allocated-but-never-written region could splice
undefined sectors into the rip.** The UDF reader treated an
allocated-but-unrecorded extent as ordinary content and read whatever
happened to be on those sectors into the output, and it decoded
embedded-data files (which store their content inline, not as a sector
map) as if their bytes were an extent list, pointing the reader at
unrelated sectors. Such extents are now refused when they actually
occupy space, and embedded files are handled as their own case; a
legitimately zero-length file still reads as empty rather than dropping
its title.
- **Audio and subtitle tracks in many languages were all labelled
"undefined."** The language mapping recognised only a handful of codes
and collapsed fifteen others to `und`. Every ISO 639-1 language code is
now mapped, so those tracks carry their real language. The DVD subtitle
colour palette, which was written in the wrong order, is also corrected.
- **If the disc read-ahead thread died mid-rip, a whole title could be
fabricated and reported complete.** A prefetch producer that terminated
returned an end-of-data signal that the reader legitimately read as a
short read and zero-filled, so a failed read could be papered over with
zeros and the pass still reported as successful. It now reports a
distinct source-terminated error instead. Dead-bus drive faults are
likewise classified rather than flattened, so the wedged-drive recovery
path can see them, and the drive now responds to Stop during spin-up and
spin-down instead of staying deaf for up to ~30 seconds.
- **A holed extraction could climb to a clean 100% on the live progress
channel.** The progress feed hardcoded its unreadable-byte count to zero
and counted every zero-filled hole as good data, so a progress-only
consumer saw a damaged extraction finish spotless — even though the
authoritative result was already truthful. The real good/unreadable
split is now threaded through the live channel. Separately, an
unreadable HD-DVD authored clip order is now logged with its error code
instead of being silently discarded before falling back to the
per-clip heuristic.
### Security
- **Bounded several unbounded amplification axes in the HD-DVD and Blu-ray
scanners.** A crafted disc could drive the playlist nesting depth, the
title count, and the clips and chapters per title without limit, and
could force a repeated clip-name fallback probe — each of which alone
left the worst-case scan unbounded. All now carry positional caps (512
titles and clips, roughly ten times any retail disc) and the fallback
probe is memoized. No effect on a well-formed disc.
### Changed
- **How freemkv picks the per-platform drive code was consolidated, with
no change to reading a disc.** The drive layer chose its
operating-system-specific module separately inside each entry point; it
now selects that module once, so a new entry point cannot silently
forget a platform. Purely internal.
## [1.6.4] — 2026-08-15
### Fixed
- **On a few Blu-ray/UHD titles the sound ran on for half a minute after the
picture had ended.** A disc stores each part of a film as a clip, and the
playlist marks exactly where that clip's content begins and ends. Where a
title is a single clip, freemkv trimmed the picture to those marks but not the
sound — and some discs leave extra audio in the file past the end mark (a
quiet fade authored after the last frame of picture). That trailing audio was
copied through, so the file claimed one running time while carrying up to ~36
seconds more sound than picture. Measured on `The Bourne Supremacy`: the
picture ends at 1:48:26 as declared, but every sound track ran to 1:49:02.
Single-clip titles are now trimmed to their playlist marks the same way
multi-clip titles already were, so sound and picture end together at the
declared duration. A title that had no extra material past its marks is
byte-for-byte unchanged. Multi-clip titles were never affected.
- **A multi-title-set CSS DVD could descramble one title set under another
set's key.** When a DVD's second title set resisted the keyless title-key
recovery, the decrypt fell back to the disc-wide key instead of failing —
writing a corrupt title behind an intact header and reporting success at exit
0. A failed recovery is now a hard error, the same as every other path in the
crate already does; ordering makes the recovery more likely to succeed but
cannot make a failed one safe.
- **A disc whose stream language field was all-zero could abort the whole track
export.** An all-zero language code (the ordinary "undefined" value on real
discs) put a NUL byte into a demux output filename and failed file creation
before a single track opened. Control bytes in that field are now sanitised
the same way the rest of the name already was.
### Security
- **Bounded the last unbounded attacker-controlled list in the DVD label
parser.** A crafted IFO could grow the forced-subtitle index list without
limit; it now carries the same positional cap as the neighbouring command
lists. No effect on a well-formed disc.
## [1.6.3] — 2026-08-10
### Changed
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "1.6.3"
version = "1.6.6"
edition = "2024"
rust-version = "1.97"
license = "MIT"
@@ -26,7 +26,7 @@ aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.3" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.5" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
+4
View File
@@ -114,6 +114,10 @@ pub mod coding_type {
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
+329 -59
View File
@@ -75,7 +75,7 @@ pub fn crack_key(
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
crack_key_scan(reader, extents, batch_sectors, None).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -85,7 +85,7 @@ pub fn crack_key(
///
/// - [`CrackOutcome::Cracked`] — a scrambled sector yielded a title key.
/// - [`CrackOutcome::Unencrypted`] — NO scrambled sector was seen across the
/// scanned extents (`is_scrambled` never true): the content is genuinely
/// scanned extents (`is_scrambled_pack` never true): the content is genuinely
/// plaintext, so proceeding without a key is correct.
/// - [`CrackOutcome::ScrambledUncracked`] — scrambled sectors WERE seen but no
/// key could be recovered (the Stevenson attack found no crackable crib, or
@@ -136,7 +136,7 @@ pub fn crack_key_outcome(
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
) -> CrackOutcome {
crack_key_scan(reader, extents, batch_sectors, halt, true)
crack_key_scan(reader, extents, batch_sectors, halt)
}
/// Resolve a DVD title's CSS descramble key from the reader when the caller
@@ -212,10 +212,6 @@ fn crack_key_scan(
extents: &[Extent],
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
// True only on the INITIAL scan: a fully CSS-locked (`05/6F/03`) result is a
// hard `ScrambledUncracked`. False on the per-VTS re-crack so a lapsed-AGID
// locked read returns None instead of killing a genuinely crackable title.
fail_on_locked: bool,
) -> CrackOutcome {
// Batch the reads: a live optical drive at 1 sector/read is glacial, and the
// crack only needs to FIND one scrambled sector whose 0x80 plaintext matches
@@ -301,6 +297,18 @@ fn crack_key_scan(
let usable = (got / 2048).min(n as usize);
// At least one, so a source returning Ok(0) cannot spin here.
advance = (usable as u32).max(1);
if usable == 0 {
// Nothing was inspected, so the per-sector `tried`
// charge below never runs — but the cursor still moves
// one sector (the `.max(1)` above). Charge that sector
// to the budget, or `tried` stays frozen and the loop
// is bounded only by the disc-declared
// `ext.sector_count`: the anti-grind budget stops
// applying to exactly the misbehaving source it exists
// for. Mirrors the `Err` arm's `tried += n`, which
// likewise charges an uninspected advance.
tried += 1;
}
for s in 0..usable {
tried += 1;
let sect = &buf[s * 2048..(s + 1) * 2048];
@@ -344,14 +352,29 @@ fn crack_key_scan(
}
// Budget exhausted / extents walked / early-bailed with no key recovered.
// The disc is ENCRYPTED-but-uncracked (a hard failure on the initial scan)
// when EITHER a scrambled sector was actually seen, OR — on the initial scan
// only (`fail_on_locked`) — every read was CSS-locked (`05/6F/03`), itself
// proof of scrambling. A re-crack (`fail_on_locked` false) stays soft: a
// lapsed-AGID locked read yields None, not a hard fail, so a crackable title
// in another VTS isn't killed. Only a scan that saw neither a scrambled
// sector nor a CSS-lock is genuinely unencrypted.
if saw_scrambled || (saw_locked && fail_on_locked) {
// The disc is ENCRYPTED-but-uncracked (a hard failure) when EITHER a
// scrambled sector was actually seen, OR every read was CSS-locked
// (`05/6F/03`), itself proof of scrambling. Only a scan that saw neither a
// scrambled sector nor a CSS-lock is genuinely unencrypted.
//
// A prior revision made this conditional on a caller-supplied
// `fail_on_locked: bool`, documented as "false on the per-VTS re-crack, so
// a lapsed-AGID locked read returns None instead of killing a genuinely
// crackable title." That parameter never had an observable effect: its
// ONLY non-test caller with `false` was [`crack_key`], whose `Option`
// return collapses `ScrambledUncracked` and `Unencrypted` alike to `None`
// via [`CrackOutcome::into_state`] — so the branch this comment describes
// was unreachable from the moment it was introduced (see the crate's audit
// notes for the git-archaeology). The one production caller that DOES
// observe the `Cracked` / `Unencrypted` / `ScrambledUncracked` split for a
// per-VTS re-crack (`Disc::decrypt_keys_for_title`) has always gone
// through [`crack_key_outcome`], which hardcoded this to always-hard-fail.
// Reconnecting the soft variant there would mean a locked read on a live
// drive gets reported as `Unencrypted` (`title_is_clear = true`) and the
// title is muxed with NO key — precisely the silent-garbage failure mode
// `CrackOutcome` exists to prevent. So the parameter is removed rather
// than revived: the scan is unconditionally hard-fail-on-locked.
if saw_scrambled || saw_locked {
CrackOutcome::ScrambledUncracked
} else {
CrackOutcome::Unencrypted
@@ -402,19 +425,16 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
///
/// # Errors
///
/// [`Error::DecryptFailed`] when a sector's own crib proves the cached key stale
/// and the re-crack from that same sector also fails. CSS has no external key
/// source — the title key comes only from cracking the data — so on a readable
/// sector this is not a missing input, it is recovery failing on data we can
/// see. Emitting the sector anyway means one of two bad outcomes: descrambled
/// with the key its crib just rejected, which yields garbage behind an intact
/// clear header (valid pack start, passes every structural check the PS demuxer
/// applies, corruption confined to the PES payload where nothing looks); or
/// passed through still scrambled, which is ciphertext delivered where plaintext
/// is meant to be. Both are bad data reported as success.
/// Never returns `Err` — the signature is `Result` only to match the decrypt
/// seam it is dispatched from, and the `usize` is that seam's legacy
/// always-zero loss count (see [`crate::decrypt::decrypt_sectors`]).
///
/// This matches the AACS sibling, which returns [`Error::DecryptFailed`] rather
/// than apply a neighbouring CPS unit's key.
/// This section used to document an [`Error::DecryptFailed`] for the case where
/// a sector's crib rejects the cached key and the re-crack from that sector also
/// fails. That behaviour was tried and REVERTED, for the reason set out at the
/// `None =>` arm below: crib mismatch plus crack failure is the signature of a
/// crib FALSE POSITIVE, not of a stale key, and failing there made real discs
/// unrippable. The arm descrambles with the cached key and returns `Ok`.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
// `is_scrambled_pack`, NOT the looser `is_scrambled`. The raw flag test
@@ -515,7 +535,7 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// Check if a sector is a CSS-scrambled DVD **video pack** — the HARDENED test
/// the crack scan uses to set its `saw_scrambled` evidence flag (Fix 3).
///
/// [`is_scrambled`] keys solely on bits 4-5 of byte 0x14. That single byte is
/// [`has_scramble_flag_bits`] keys solely on bits 4-5 of byte 0x14. That byte is
/// only meaningful inside a real DVD sector — an MPEG-2 Program Stream pack,
/// which ALWAYS begins with the 32-bit pack-start code `00 00 01 BA` at offset
/// 0x00. A tiny clear / nav-only stub (a 0.5 s menu loop, an FBI-warning title)
@@ -527,14 +547,53 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// structurally a DVD video pack can be counted as scramble evidence. This does
/// NOT weaken the genuine "encrypted but uncrackable" hard-fail: a real
/// scrambled feature is made of valid PS packs, so its scrambled sectors still
/// pass this check and still drive `ScrambledUncracked` when no key cracks. (The
/// descramble loop keeps the looser [`is_scrambled`]: by the time it runs we
/// already know the title is CSS, and it only needs to skip interleaved clear
/// NAV packs — a wrongly-skipped or wrongly-included sector there is recoverable
/// per-sector, whereas a false scramble verdict in the scan poisons the whole
/// title's outcome.)
/// pass this check and still drive `ScrambledUncracked` when no key cracks.
///
/// The pack-start check alone is not enough, and the sector's PES `stream_id`
/// (offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
/// prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
/// streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
/// (`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
/// stream_id even on a scrambled sector. The MPEG-PS structural packets that
/// are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
/// private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
/// because on those the byte at 0x14 is NOT a PES scrambling-control field but
/// raw payload/structure whose bits 4-5 land set by chance.
///
/// This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
/// MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
/// private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
/// set. With no `0x11` gate those nav packs flipped the crack scan's
/// `saw_scrambled` flag on a disc that carries no CSS at all; the crack then
/// found no key (there is none) and the scan returned `ScrambledUncracked`,
/// hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
/// `0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
/// can act on (the Stevenson attack only recovers a key from a scrambled ES
/// pack), so a decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
///
/// This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
/// real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
/// private_stream_1 (`0xBD`) packs, none of which are excluded, so its
/// scrambled sectors still set `saw_scrambled` and still drive
/// `ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
/// so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
///
/// The DESCRAMBLE path gates on this same function — [`descramble_sector`] and
/// [`descramble_region`] both call it, not the raw flag test — because the raw
/// test does not merely mis-skip a sector there: it descrambles one that was
/// never scrambled and destroys it. The measured case is written up on
/// [`descramble_region`]: a `VIDEO_TS.IFO` sector holding 0x15 at offset 0x14
/// lost 1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38
/// titles became 10 — silently, at exit 0. One gate, both paths.
pub fn is_scrambled_pack(sector: &[u8]) -> bool {
sector.len() >= 2048 && sector[0x00..0x04] == PACK_START && (sector[0x14] >> 4) & 0x03 != 0
use crate::consts::pes_stream_id::{PADDING_STREAM, PRIVATE_STREAM_2, SYSTEM_HEADER};
sector.len() >= 2048
&& sector[0x00..0x04] == PACK_START
&& !matches!(
sector[0x11],
SYSTEM_HEADER | PADDING_STREAM | PRIVATE_STREAM_2
)
&& (sector[0x14] >> 4) & 0x03 != 0
}
#[cfg(test)]
@@ -590,7 +649,22 @@ mod tests {
let out = descramble_region(&mut sector, &mut key)
.expect("a crib false positive must NOT fail the rip");
assert_eq!(out, 0, "CSS reports no loss term of its own");
// The "no loss term" contract belongs to the SEAM, not to this
// function: `decrypt_sectors`' `usize` is a legacy always-zero count
// that the CSS arm feeds from here. Asserting `out == 0` on
// `descramble_region` alone only restates its single `Ok(0)` return —
// a body replaced by `Ok(0)` satisfies it just as well. Assert it one
// level up, where the value is actually assembled and returned, so the
// arm dispatch and the plumbing are exercised too.
assert_eq!(out, 0);
let mut seam_sector = sector;
let mut seam_keys = crate::decrypt::DecryptKeys::Css { title_key: key };
assert_eq!(
crate::decrypt::decrypt_sectors(&mut seam_sector, &mut seam_keys, 0)
.expect("a crib false positive must NOT fail the rip at the seam either"),
0,
"CSS reports no loss term of its own through decrypt_sectors"
);
assert_eq!(
key, key_before,
"a failed re-crack must leave the cached key in place — it is still \
@@ -618,9 +692,9 @@ mod tests {
);
}
// ── is_scrambled ───────────────────────────────────────────────────────
// ── has_scramble_flag_bits ─────────────────────────────────────────────
/// is_scrambled returns false for any buffer shorter than one sector,
/// has_scramble_flag_bits returns false for any buffer shorter than one sector,
/// WITHOUT indexing byte 0x14 (which would panic on a tiny buffer). The
/// length guard is short-circuited before the flag read.
///
@@ -630,13 +704,13 @@ mod tests {
/// (`(sector[0x14]...) && sector.len() >= 2048`) -> panics indexing a
/// 20-byte slice; this test catches it.
#[test]
fn is_scrambled_short_buffer_is_false_no_panic() {
fn has_scramble_flag_bits_short_buffer_is_false_no_panic() {
assert!(!has_scramble_flag_bits(&[]));
assert!(!has_scramble_flag_bits(&[0u8; 20])); // shorter than 0x14+1 even
assert!(!has_scramble_flag_bits(&[0xFFu8; 2047])); // one byte short of a sector
}
/// is_scrambled keys on bits 4-5 of byte 0x14 (the CSS scramble field).
/// has_scramble_flag_bits keys on bits 4-5 of byte 0x14 (the CSS scramble field).
/// A full sector flagged 0x10/0x20/0x30 is scrambled; 0x00 and the
/// high-bit-only values 0x40/0x80 are clear.
///
@@ -644,7 +718,7 @@ mod tests {
/// Mutation: widen mask to `& 0x0F` -> 0x40 reports scrambled, the 0x40
/// assert fails.
#[test]
fn is_scrambled_uses_bits_4_5_only() {
fn has_scramble_flag_bits_uses_bits_4_5_only() {
let mut s = vec![0u8; 2048];
for (flag, expected) in [
(0x00u8, false),
@@ -665,14 +739,14 @@ mod tests {
}
}
/// is_scrambled accepts exactly 2048 bytes as the minimum (boundary at the
/// has_scramble_flag_bits accepts exactly 2048 bytes as the minimum (boundary at the
/// inclusive value 2048).
///
/// Grounding: `sector.len() >= 2048`.
/// Mutation: change `>= 2048` to `> 2048` -> an exact 2048-byte scrambled
/// sector reports false; this fails.
#[test]
fn is_scrambled_exact_sector_length_accepted() {
fn has_scramble_flag_bits_exact_sector_length_accepted() {
let mut s = vec![0u8; 2048];
s[0x14] = 0x30;
assert!(
@@ -686,8 +760,8 @@ mod tests {
/// bits. A clear / nav-only stub whose bytes happen to set bits 4-5 of 0x14
/// but lacks the pack-start is NOT counted as scramble evidence — without
/// this the scan flips `saw_scrambled` and a genuinely unencrypted title
/// reports `ScrambledUncracked` (the false E7023). The looser `is_scrambled`
/// (descramble gate) still reads the same sector as flagged.
/// reports `ScrambledUncracked` (the false E7023). The raw flag test
/// `has_scramble_flag_bits` still reads the same sector as flagged.
///
/// Grounding: `sector[0x00..0x04] == 00 00 01 BA && (sector[0x14] >> 4)...`.
/// Mutation: drop the pack-start clause -> the 0x14-only sector counts as a
@@ -719,6 +793,52 @@ mod tests {
);
}
/// Detection fix: `is_scrambled_pack` must EXCLUDE the MPEG-PS structural
/// packets that CSS never scrambles — system_header (0xBB), padding (0xBE)
/// and private_stream_2 (0xBF, DVD PCI/DSI and HD-DVD `.evo` RDI nav) — by
/// their `stream_id` at offset 0x11. On those packets byte 0x14 is raw
/// payload/structure, not a PES scrambling-control field, so its bits 4-5
/// land set by chance.
///
/// This is the exact decrypted-HD-DVD defect: an `.evo` RDI pack is
/// private_stream_2 (0xBF) with the pack-start code and bits set at 0x14, so
/// the old gate flipped `saw_scrambled` on a CSS-free disc → the crack found
/// no key → `ScrambledUncracked` → E7023 on a good HD-DVD.
///
/// Grounding: `!matches!(sector[0x11], SYSTEM_HEADER | PADDING_STREAM |
/// PRIVATE_STREAM_2)`. Mutation: drop the 0x11 exclusion → the 0xBF nav pack
/// counts as scrambled evidence and the first assert fails. The video and
/// private_stream_1 packs prove the gate does NOT reject a real scrambled
/// DVD sector (the catastrophic direction).
#[test]
fn is_scrambled_pack_excludes_nav_and_structural_stream_ids() {
use crate::consts::pes_stream_id::{
PADDING_STREAM, PRIVATE_STREAM_1, PRIVATE_STREAM_2, SYSTEM_HEADER, VIDEO, VIDEO_MAX,
};
// A pack-start pack with 0x14 scramble bits set, varying only 0x11.
let mut s = vec![0u8; 2048];
s[0x00..0x04].copy_from_slice(&PACK_START);
s[0x14] = 0x30;
for excluded in [SYSTEM_HEADER, PADDING_STREAM, PRIVATE_STREAM_2] {
s[0x11] = excluded;
assert!(
!is_scrambled_pack(&s),
"stream_id {excluded:#04x} is a structural/nav pack CSS never scrambles — \
it must NOT count as scramble evidence (else a decrypted HD-DVD RDI pack → E7023)"
);
}
// A genuinely scramblable elementary-stream pack must STILL register —
// proving the exclusion did not weaken real-DVD CSS detection.
for scramblable in [VIDEO, VIDEO_MAX, PRIVATE_STREAM_1, 0xE2] {
s[0x11] = scramblable;
assert!(
is_scrambled_pack(&s),
"stream_id {scramblable:#04x} is a scramblable ES pack — a real CSS DVD's \
scrambled sector must still be counted, never passed through as plaintext"
);
}
}
// ── crack_key scanning over a mock SectorSource ────────────────────────
/// Records every (lba, count) read; returns a caller-supplied flag byte at
@@ -735,6 +855,16 @@ mod tests {
/// instead of the uniform `flag_byte` fill. Lets the scan actually
/// reach `CrackOutcome::Cracked` from a synthetic ISO.
crackable: Option<(u32, Vec<u8>)>,
/// Sectors actually filled per batch, however many were asked for —
/// the SHORT READ a `recovery: true` source is allowed to return over
/// a damaged region. `Some(0)` is the degenerate case that must not
/// spin the scan.
short_read: Option<usize>,
/// PES `stream_id` written at offset 0x11 of each uniform-fill sector.
/// `0x00` (the default) is a scramblable-looking pack; set to
/// private_stream_2 (`0xBF`) to model an HD-DVD `.evo` RDI nav pack,
/// which carries the pack-start code and 0x14 bits but no CSS.
stream_id: u8,
}
impl MockSource {
@@ -745,6 +875,8 @@ mod tests {
fail_all: false,
lock_all: false,
crackable: None,
short_read: None,
stream_id: 0x00,
}
}
}
@@ -795,14 +927,19 @@ mod tests {
if self.fail_all {
return Err(Error::DecryptFailed);
}
let n = count as usize * 2048;
// A short read fills, and reports, fewer sectors than asked.
let filled = match self.short_read {
Some(k) => (k as u16).min(count),
None => count,
};
let n = filled as usize * 2048;
let end = n.min(buf.len());
for b in buf[..end].iter_mut() {
*b = 0;
}
// Fill each sector in the batch with the uniform flag byte, EXCEPT a
// designated crackable LBA which gets the full synthetic sector.
for s in 0..count as u32 {
for s in 0..filled as u32 {
let sect_lba = lba + s;
let base = s as usize * 2048;
if base + 2048 > end {
@@ -814,11 +951,14 @@ mod tests {
}
_ => {
// Real DVD video sectors always open with the MPEG-PS
// pack-start code; `is_scrambled` (Fix 3) requires it
// before trusting the 0x14 scramble bits, so the fixture
// must include it for a `flag_byte` of 0x30 to register
// as scrambled.
// pack-start code; `is_scrambled_pack` requires it before
// trusting the 0x14 scramble bits, so the fixture must
// include it for a `flag_byte` of 0x30 to register as
// scrambled. `stream_id` (byte 0x11) defaults to 0x00 (a
// scramblable pack); an HD-DVD RDI nav pack sets it to
// private_stream_2 (0xBF), which the 0x11 exclusion drops.
buf[base..base + 4].copy_from_slice(&PACK_START);
buf[base + 0x11] = self.stream_id;
buf[base + 0x14] = self.flag_byte;
}
}
@@ -827,6 +967,93 @@ mod tests {
}
}
// ── Short reads: the branch nothing exercised ─────────────────────────
//
// `crack_key_scan` passes `recovery = true`, which is precisely the mode
// where a `SectorSource` may return Ok with fewer bytes than asked. Every
// source in this module returned the full request, so `usable`, `advance`
// and the `.max(1)` anti-spin guard were dead code under test: reverting
// `advance` to `n`, or dropping the `.max(1)`, left the whole suite green.
/// A short batch is RE-READ from where it stopped, not skipped. Skipping
/// would quietly shrink the crack's coverage on exactly the damaged media
/// where a key is hardest to find.
#[test]
fn a_short_read_resumes_from_where_it_stopped() {
let mut src = MockSource::new(0x00);
src.short_read = Some(1);
let ext = [crate::disc::Extent {
start_lba: 100,
sector_count: 4,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().clone();
assert_eq!(
reads,
vec![100, 101, 102, 103],
"a source that filled one sector per batch must be asked for the \
next one, not advanced a whole batch past it"
);
}
/// A source that reads NOTHING must terminate. Without the `.max(1)` the
/// cursor never moves and `tried` never increments — the budget cannot end
/// the loop, so the scan spins forever inside a library whose whole job is
/// surviving hostile input.
#[test]
fn a_source_that_returns_zero_sectors_terminates() {
let mut src = MockSource::new(0x00);
src.short_read = Some(0);
let ext = [crate::disc::Extent {
start_lba: 0,
sector_count: 8,
}];
let outcome = crack_key_scan(&mut src, &ext, 4, None);
assert!(
matches!(outcome, CrackOutcome::Unencrypted),
"nothing was read, so nothing scrambled was seen"
);
assert!(
src.reads.borrow().len() <= 8,
"the cursor must advance even on an empty read; got {} reads over \
an 8-sector extent",
src.reads.borrow().len()
);
}
/// The 50_000-sector budget must hold whatever the source returns, not
/// only when the source delivers sectors.
///
/// `tried` is incremented ONLY per inspected sector, inside
/// `for s in 0..usable`. An `Ok(0)` inspects nothing, so that loop never
/// runs — yet `advance` is forced to 1 to stop the scan spinning, so the
/// cursor keeps walking. The budget is then never consulted and the scan
/// runs for the extent's full, disc-declared `sector_count`: a misbehaving
/// or adversarial source (an emulated drive, a bridge answering short)
/// converts the anti-grind bound into no bound at all.
///
/// Mutation: delete the `tried` charge in the `usable == 0` arm and this
/// goes red at 60_000 reads.
#[test]
fn a_source_that_returns_zero_sectors_still_obeys_the_scan_budget() {
const MAX_TRIES: usize = 50_000;
let mut src = MockSource::new(0x00);
src.short_read = Some(0);
// Deliberately LARGER than the budget: if the budget is what stops the
// scan, the extent's own length is never reached.
let ext = [crate::disc::Extent {
start_lba: 0,
sector_count: 60_000,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().len();
assert!(
reads <= MAX_TRIES,
"an Ok(0)-returning source must be stopped by the {MAX_TRIES}-sector \
budget, not by the disc-declared extent length; got {reads} reads"
);
}
/// crack_key caps total scanned sectors at 50_000 even when extents are
/// far larger, and counts EVERY scanned sector (clear ones included)
/// toward the budget. With one 200_000-sector extent of clear sectors, it
@@ -1032,12 +1259,17 @@ mod tests {
);
}
/// MISSING #1 guard: the re-crack path (the `Option`-returning `crack_key`,
/// `fail_on_locked == false`) must NOT hard-fail on a CSS-locked read — it
/// returns `None`. A lapsed-AGID re-crack of another VTS stays soft so a
/// genuinely crackable title isn't killed by a transient locked read.
/// `crack_key` (the `Option`-returning convenience wrapper) collapses
/// `ScrambledUncracked` and `Unencrypted` alike to `None` via
/// [`CrackOutcome::into_state`], so an all-locked scan still reads `None`
/// here even though the scan itself now treats every CSS-lock as a hard
/// `ScrambledUncracked` (see `crack_key_scan`'s removal of the dead
/// `fail_on_locked` parameter). Callers that need to tell "locked/
/// uncrackable" apart from "genuinely clear" must use `crack_key_outcome`,
/// which the `all_locked_synthetic_iso_yields_css_key_missing_signal` test
/// pins directly.
#[test]
fn crack_key_recrack_locked_is_none_not_hard_fail() {
fn crack_key_all_locked_collapses_to_none() {
let mut src = MockSource::new(0x30);
src.lock_all = true;
let extents = [Extent {
@@ -1123,7 +1355,7 @@ mod tests {
/// budget). With a small failing extent, every sector is attempted and the
/// function returns None.
///
/// Grounding: `if reader.read_sectors(...).is_ok() && is_scrambled(...)` —
/// Grounding: `if reader.read_sectors(...).is_ok() && is_scrambled_pack(...)` —
/// an Err simply falls through to `i += 1`.
/// Mutation: change the read-error handling to `reader.read_sectors(...)?`
/// (propagate) -> crack_key would stop after the first error and read only
@@ -1362,6 +1594,44 @@ mod tests {
);
}
/// The decrypted-HD-DVD regression, end to end through `resolve_dvd_title_key`.
/// An `.evo` is MPEG-PS (`ContentFormat::MpegPs`) exactly like a DVD `.vob`
/// and carries `None` keys once decrypted, so it reaches the CSS crack. Its
/// RDI navigation packs are private_stream_2 (0xBF) with the pack-start code
/// and bits set at offset 0x14 — but HD-DVD carries no CSS at all. The 0x11
/// exclusion in `is_scrambled_pack` keeps those nav packs from flipping the
/// scan's `saw_scrambled` gate, so the scan returns `Unencrypted` and the
/// title muxes cleanly instead of hard-failing.
///
/// Catches the mutation of dropping the 0x11 exclusion: without it every 0xBF
/// RDI pack counts as scramble evidence, the crack finds no key (there is
/// none), and the scan returns `ScrambledUncracked` → `CssKeyMissing` (E7023)
/// on a perfectly good HD-DVD — the exact defect a real CI run produced.
#[test]
fn resolve_dvd_title_key_decrypted_hddvd_rdi_packs_scan_clean_no_e7023() {
let mut src = MockSource::new(0x30); // 0x14 bits set…
src.stream_id = crate::consts::pes_stream_id::PRIVATE_STREAM_2; // …but a 0xBF nav pack
let extents = [Extent {
start_lba: 0,
sector_count: 64,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
8,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("a decrypted HD-DVD's RDI nav packs are not CSS — the scan must not hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"no CSS key exists on an HD-DVD; keys must stay None and the title mux clean"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
@@ -1527,8 +1797,8 @@ mod tests {
/// PER-VTS RE-CRACK SUCCESS (audit gap "success path missing"): the prior
/// re-crack test only covered the locked→None path. Here a re-crack
/// (`crack_key`, `fail_on_locked == false`) over a DIFFERENT VTS's extents
/// finds that VTS's own crackable sector and returns a `CssState` whose
/// (`crack_key`) over a DIFFERENT VTS's extents finds that VTS's own
/// crackable sector and returns a `CssState` whose
/// `crack_span` matches the new extents — proving a key cracked for one VTS
/// is genuinely re-derived (not reused) for another.
#[test]
+14 -1
View File
@@ -1899,11 +1899,24 @@ mod tests {
/// below its own range start, or a zero unit size, are both map bugs — they
/// must return a defined answer rather than panicking on debug overflow or
/// dividing by zero.
///
/// Every case here asserts the DEFINED answer, not merely the absence of a
/// panic. The zero-unit-size case used to be written
/// `assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
/// accepts both answers and so pinned nothing at all: the guards could
/// invert and it would still pass. The answer is knowable —
/// `saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
/// is even — so pin it.
#[test]
fn phase_gate_does_not_panic_on_a_malformed_map() {
use super::{Phase, unit_is_our_phase};
// Unit below its own range start: saturating_sub clamps to 0, and unit
// 0 is even.
assert!(unit_is_our_phase(10, 100, 3, Phase::Even));
assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true);
// Zero unit size: max(1) makes the divisor 1, so the index is the raw
// offset 70 — even.
assert!(unit_is_our_phase(100, 30, 0, Phase::Even));
// Both malformations at once: offset 0 over divisor 1 is unit 0, even.
assert!(unit_is_our_phase(5, 5, 0, Phase::Even));
}
}
+6 -2
View File
@@ -29,8 +29,12 @@ use super::layout::{DirNode, Layout};
use crate::error::{Error, Result};
use std::collections::BTreeMap;
/// Logical block / sector size. Fixed for every optical profile this crate reads.
pub(super) const SECTOR: usize = 2048;
/// Logical block / sector size. Fixed for every optical profile this crate
/// reads, and the same quantity as [`crate::consts::SECTOR_BYTES`] — aliased
/// rather than re-declared so the two cannot drift apart. The short name is
/// kept because it appears in ~25 extent and offset expressions across
/// `dirimage`, where the longer one would bury the arithmetic.
pub(super) use crate::consts::SECTOR_BYTES as SECTOR;
/// Descriptor version recorded in every tag. 2 = ECMA-167 2nd edition, which
/// is what UDF revisions up to and including 2.00 require.
+9 -5
View File
@@ -67,6 +67,15 @@ const MAX_SUBDIRS: usize = (u16::MAX - 1) as usize;
#[cfg(test)]
const MAX_SUBDIRS: usize = 4;
/// One entry per child plus the parent's own must still fit the 16-bit field.
///
/// At MODULE scope, and it has to be. This assertion previously sat inside
/// `mod tests`, which is `#[cfg(test)]`, while carrying `#[cfg(not(test))]`
/// itself — so it was compiled in NO configuration and could never fire, which
/// made it exactly the dead gate the test above it was written to replace.
#[cfg(not(test))]
const _: () = assert!(MAX_SUBDIRS + 1 == u16::MAX as usize);
/// Largest image this planner will synthesize, in sectors (128 GiB).
///
/// A DVD title set records where its VOBS begins as an offset in its own IFO,
@@ -998,11 +1007,6 @@ mod tests {
"expected DirImageFanout, got {err:?}"
);
// And the real production value is what ships: one per child plus the
// parent's own entry must still fit in the 16-bit field.
#[cfg(not(test))]
const _: () = assert!(MAX_SUBDIRS + 1 == u16::MAX as usize);
let _ = std::fs::remove_dir_all(&dir);
}
}
+121 -9
View File
@@ -17,18 +17,31 @@ struct Scratch(PathBuf);
impl Scratch {
fn new(tag: &str) -> Self {
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{:?}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
let p = Self::unique_path(tag);
std::fs::create_dir_all(&p).unwrap();
Self(p)
}
/// The path a scratch dir takes — kept separate from directory creation so
/// its uniqueness is testable without a syscall between draws. A monotonic
/// counter, NOT a timestamp, is what guarantees it: `SystemTime::now()`
/// resolves to only a MICROSECOND on macOS, so two of the seven parallel
/// tests that share the "bdmv" tag routinely read the same value within one
/// microsecond, collide on the same directory, and the first to finish
/// `remove_dir_all`s it out from under the others' reads — an intermittent
/// `read_sectors` ENOENT. The counter is unique regardless of clock
/// granularity; pairing it with the pid keeps it unique across test-binary
/// processes.
fn unique_path(tag: &str) -> PathBuf {
static SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let uniq = SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{uniq}",
std::process::id()
));
p
}
fn path(&self) -> &Path {
&self.0
}
@@ -69,6 +82,28 @@ fn bdmv_scratch() -> (Scratch, Vec<u8>, Vec<u8>) {
(s, index, clip)
}
/// Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
/// tag and run in parallel; if two land the same path, the first to drop
/// `remove_dir_all`s it out from under the other's reads, which surfaced as an
/// intermittent `read_sectors` failure. Regression: with the old
/// `SystemTime::now()` name this is RED, because macOS's clock advances only
/// per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
/// between them to nudge the clock, mirroring the real cross-thread collision —
/// hands back the same value many times over. The counter-based name is unique
/// regardless of clock granularity.
#[test]
fn scratch_paths_are_unique_even_at_clock_resolution() {
let paths: Vec<PathBuf> = (0..1000).map(|_| Scratch::unique_path("uniq")).collect();
let distinct: std::collections::HashSet<&PathBuf> = paths.iter().collect();
assert_eq!(
distinct.len(),
paths.len(),
"every Scratch must own a distinct path; a timestamp-based name collides \
under macOS's microsecond clock and lets one parallel test delete \
another's files mid-read"
);
}
// ── The de-risking spike ────────────────────────────────────────────────────
/// THE load-bearing assertion of the whole design: metadata synthesized here
@@ -650,6 +685,83 @@ fn an_aacs_folder_with_scrambled_content_is_rejected() {
assert_eq!(err.code(), crate::error::E_DIR_IMAGE_ENCRYPTED);
}
// ── The OTHER door: `dir://` through the PES input path ─────────────────────
//
// `session::scan_dir` (covered above) and `mux::resolve::input("dir://…")` are
// two doors into the same folder, and they once disagreed: a folder that ripped
// fine through one failed through the other, because only `scan_dir` re-judged
// the tree-shape encryption verdict from CONTENT. The fix was to share
// `session::apply_folder_encryption_verdict` between them — see its doc
// comment, which names this exact failure. Nothing tested the second door, so
// dropping the `is_folder` argument at `mux::resolve`'s call site restored the
// bug silently.
/// The two doors must AGREE. Same folder, same verdict, same extents.
///
/// Stated as a differential rather than a bare `is_ok()` so it cannot go
/// vacuous: if the fixture ever stops producing an AACS state, a one-sided
/// `Ok` assertion would still pass while guarding nothing, whereas "both doors
/// see the same title" is the invariant the shared function actually exists to
/// hold.
#[test]
fn both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory() {
// `s` MUST outlive `stream`: the pipeline's producer thread is still
// reading the folder until the stream is dropped, and `Scratch::drop`
// removes the directory out from under it.
let s = playable_bdmv("dirdoor", false);
s.file("AACS/Unit_Key_RO.inf", &[0u8; 64]);
s.file("AACS/MKB_RO.inf", &[0u8; 64]);
// Door 1 — the scan path.
let (disc, _reader) =
crate::session::scan_dir(s.path(), crate::disc::ScanOptions::default()).unwrap();
assert!(
!disc.encrypted,
"door 1 must judge the clear folder decrypted"
);
let scanned_extents = disc.titles[0].extents.clone();
assert!(
!scanned_extents.is_empty(),
"fixture must have real extents"
);
// Door 2 — the PES input path the CLI actually rips through.
let url = format!("dir://{}", s.path().display());
let stream = crate::input(&url, &crate::InputOptions::default())
.expect("dir:// input must open a clear folder, exactly as scan_dir does");
assert_eq!(
stream.info().extents,
scanned_extents,
"the two doors selected different titles from the same folder"
);
drop(stream);
}
/// The other verdict, through the same door: a folder whose content units are
/// really scrambled is refused with the TYPED code, not muxed into garbage.
///
/// This is the load-bearing half. `E_DIR_IMAGE_ENCRYPTED` is produced at
/// exactly one site in the crate (`session::apply_folder_encryption_verdict`),
/// reachable from here only through the `is_folder` argument — so this test
/// cannot pass if that argument is dropped, whatever else changes.
#[test]
fn a_scrambled_folder_is_refused_through_the_dir_url_door_too() {
let s = playable_bdmv("dirdoorenc", true);
s.file("AACS/Unit_Key_RO.inf", &[0u8; 64]);
s.file("AACS/MKB_RO.inf", &[0u8; 64]);
let url = format!("dir://{}", s.path().display());
let err = match crate::input(&url, &crate::InputOptions::default()) {
Ok(_) => panic!("a scrambled folder must not open as a PES source"),
Err(e) => e,
};
assert_eq!(
crate::error::error_code(&err),
Some(crate::error::E_DIR_IMAGE_ENCRYPTED),
"expected the typed dir-source-encrypted code, got: {err}"
);
}
// ── External oracle ─────────────────────────────────────────────────────────
/// Write a synthesized image to a real file and ask the OS to mount it.
+905 -118
View File
File diff suppressed because it is too large Load Diff
+164 -22
View File
@@ -7,13 +7,32 @@ use crate::udf;
impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Vec<DiscTitle> {
halt: Option<&crate::halt::Halt>,
) -> Result<Vec<DiscTitle>> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(_) => return Vec::new(),
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok(Vec::new()),
};
let mut titles = Vec::new();
@@ -243,7 +262,14 @@ impl Disc {
}
}
titles
// Polled again AFTER the loop: a cancel raised during the IFO reads
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok(titles)
}
}
@@ -528,6 +554,118 @@ mod tests {
// Tests
// ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|ts| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|ts| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test]
@@ -535,7 +673,11 @@ mod tests {
let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]);
assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.is_empty()
);
}
/// Single VTS, single title, one cell. Extent absolute LBA =
@@ -571,7 +713,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -627,7 +769,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -685,7 +827,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -743,7 +885,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -784,7 +926,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -839,7 +981,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -900,7 +1042,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -911,7 +1053,7 @@ mod tests {
.collect();
assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "en");
assert_eq!(audios[0].language, "eng");
assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -970,7 +1112,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1027,7 +1169,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1040,7 +1182,7 @@ mod tests {
// Languages preserved in order.
assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["en", "fr", "de"]
vec!["eng", "fra", "deu"]
);
// PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1087,7 +1229,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let sub = t
.streams
.iter()
@@ -1097,7 +1239,7 @@ mod tests {
})
.expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "en");
assert_eq!(sub.language, "eng");
assert!(
sub.codec_data.is_some(),
"non-zero palette must yield codec_data"
@@ -1141,7 +1283,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1178,7 +1320,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1263,7 +1405,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1315,7 +1457,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1358,7 +1500,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
+362 -29
View File
@@ -107,8 +107,10 @@ struct PlannedFile {
size: u64,
/// Inline (ICB-embedded) data, if any. When `Some`, `extents` is empty.
inline: Option<Vec<u8>>,
/// Absolute disc extents `(abs_lba, byte_len)`.
extents: Vec<(u32, u32)>,
/// Absolute disc extents, each carrying whether it was ever RECORDED (an
/// ECMA-167 4/14.14.1.1 type-1 extent is allocated but not recorded: it
/// occupies the file's byte space and its contents are zeros).
extents: Vec<crate::udf::AbsExtent>,
}
impl Disc {
@@ -144,7 +146,7 @@ impl Disc {
let fs = udf::read_filesystem(reader)?;
let mut planned: Vec<PlannedFile> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
let mut seen_hosts: std::collections::HashMap<PathBuf, String> =
let mut seen_hosts: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
plan_tree(
reader,
@@ -235,6 +237,9 @@ impl Disc {
let mut result = ExtractResult::default();
let total_bytes = required;
let mut done_bytes: u64 = 0;
// Cumulative zero-filled-unreadable bytes, so the live progress channel
// can report the good/unreadable split instead of pinning unreadable at 0.
let mut done_unreadable: u64 = 0;
// CSS per-VTS key cache; only consulted for CSS discs.
let is_css = matches!(base_keys, DecryptKeys::Css { .. });
@@ -254,7 +259,7 @@ impl Disc {
let key = match vts_keys.get(&vts) {
Some(k) => k.clone(),
None => {
let k = self.resolve_vts_key(&vts, &planned, &mut dec, &base_keys);
let k = self.resolve_vts_key(&vts, &planned, &mut dec, &base_keys)?;
vts_keys.insert(vts.clone(), k.clone());
k
}
@@ -269,8 +274,15 @@ impl Disc {
// non-tolerate), so extract_one_file already zero-filled it and
// counted it in bytes_unreadable — one 'lost' bucket covers both
// media damage and decrypt failure.
let (fr, halted) =
extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?;
let (fr, halted) = extract_one_file(
&mut dec,
dest,
pf,
total_bytes,
&mut done_bytes,
&mut done_unreadable,
opts,
)?;
result.bytes_good = result.bytes_good.saturating_add(fr.bytes_good);
result.bytes_unreadable = result.bytes_unreadable.saturating_add(fr.bytes_unreadable);
@@ -291,13 +303,24 @@ impl Disc {
/// VOB extents (keyless Stevenson attack). The disc-wide key is reused when
/// it already covers this VTS (single-VTS discs, or this VTS's span). The
/// reader is borrowed from the decrypting decorator (its inner source).
///
/// FALLIBLE, because the three outcomes of a crack are not two.
/// [`crate::css::CrackOutcome`] exists precisely to separate "no scrambled
/// sector was seen, so there is nothing to decrypt" from "scrambled sectors
/// were seen and no key came out", and its doc says callers MUST surface
/// the second as a hard error. This function used the `Option`-returning
/// `crack_key`, which collapses both into `None`, and then fell back to the
/// disc-wide key — descrambling this VTS with ANOTHER VTS's title key. The
/// hazard is described in the ordering note below, which was written about
/// the same fallback; ordering makes the crack far more likely to succeed,
/// but it cannot make a failed crack safe.
fn resolve_vts_key<S: SectorSource>(
&self,
vts: &str,
planned: &[PlannedFile],
dec: &mut DecryptingSectorSource<S>,
base_keys: &DecryptKeys,
) -> DecryptKeys {
) -> Result<DecryptKeys> {
// Gather the title VOB extents for this VTS (VTS_xx_1.VOB .. _9.VOB;
// VTS_xx_0.VOB is the menu and is clear, so excluded from the crack).
// Gather this VTS's title VOBs BY NAME, ascending.
@@ -328,15 +351,19 @@ impl Disc {
let mut extents: Vec<crate::disc::Extent> = Vec::new();
for pf in files {
for &(abs_lba, byte_len) in &pf.extents {
// Unrecorded extents hold no bytes the VOB ever wrote, so they can
// carry no scrambled sector for the crack to work from — feeding
// them in spends the shared sector budget on media that is not part
// of the title.
for ext in pf.extents.iter().filter(|e| e.recorded) {
extents.push(crate::disc::Extent {
start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
start_lba: ext.lba,
sector_count: (ext.len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
});
}
}
if extents.is_empty() {
return base_keys.clone();
return Ok(base_keys.clone());
}
// PLAYBACK ORDER — do NOT sort. This is the 1.5.1 garbage bug, and it
// grew back here in a new code path: the comment this replaced said it
@@ -358,11 +385,33 @@ impl Disc {
// in file order, which together is playback order for a DVD title set.
// Crack against the raw (still-scrambled) inner reader, NOT the
// decrypting view — `crack_key` runs the descrambler itself.
match crate::css::crack_key(dec.inner_mut(), &extents, 64) {
Some(state) => DecryptKeys::Css {
match crate::css::crack_key_outcome(dec.inner_mut(), &extents, 64, None) {
crate::css::CrackOutcome::Cracked(state) => Ok(DecryptKeys::Css {
title_key: state.title_key,
},
None => base_keys.clone(),
}),
// No scrambled sector anywhere in this VTS: the content is clear,
// and any key descrambles it as a no-op. The disc-wide key is the
// right answer, and this is the ONLY case that ever was.
crate::css::CrackOutcome::Unencrypted => Ok(base_keys.clone()),
// Scrambled sectors WERE seen and no key came out. Reusing the
// disc-wide key here writes corrupt PES behind an intact header and
// reports a complete extract at exit 0.
//
// `CssKeyMissing` is the per-TITLE code rather than the disc-level
// `CssNoDiscKey` because the two are treated differently elsewhere:
// it is what `error::is_skippable_title_stub` matches, and the MUX
// path really does skip a title on it and carry on with the rest of
// the disc.
//
// EXTRACT does not skip. This error is `?`-propagated by the only
// caller, out of the loop over every planned file, so one
// uncrackable VTS aborts the whole extract — including files from
// sibling VTS groups that would have cracked. That is deliberate:
// aborting loudly is the safe failure here, and pinned by
// `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`.
// Do not read the code choice as a promise that extraction
// continues past this point.
crate::css::CrackOutcome::ScrambledUncracked => Err(Error::CssKeyMissing),
}
}
}
@@ -409,7 +458,7 @@ fn plan_tree(
is_root: bool,
files: &mut Vec<PlannedFile>,
dirs: &mut Vec<PathBuf>,
seen_hosts: &mut std::collections::HashMap<PathBuf, String>,
seen_hosts: &mut std::collections::HashMap<String, String>,
) -> Result<()> {
for entry in &dir.entries {
if entry.name.is_empty() {
@@ -427,13 +476,48 @@ fn plan_tree(
let safe = sanitize_component(&entry.name)?;
let child_rel = host_rel.join(&safe);
let child_disc = format!("{disc_path}/{}", entry.name);
// Collision: two distinct disc paths → same host path.
if let Some(prev) = seen_hosts.insert(child_rel.clone(), child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: child_rel.to_string_lossy().into_owned(),
});
// Collision: two distinct disc paths → same host FILE. The key must
// model the HOST's namespace, not the disc's, and the two differ twice
// over:
//
// * CASE. macOS APFS and Windows NTFS are case-insensitive by default,
// so `Movie` and `movie` are one file there. Keyed by the
// case-preserving path they were two entries, nothing collided, and
// the second extraction overwrote the first with both files still
// reported `complete: true`.
// * The `.partial` SUFFIX. `extract_one_file` streams through
// `<final>.partial` before renaming, so a file's temp path lives in
// the same namespace as every other file's final path: a disc
// holding both `X` and `X.partial` planned two distinct final names,
// and extracting `X` then truncated the real `X.partial`.
//
// Only files get the `.partial` alias — directories are created
// directly and never stream through a temp name — but it is checked
// against every entry's primary key, so a directory `X.partial` beside
// a file `X` is caught too. The alias stores the OWNING file's disc
// path, so the `prev != child_disc` test keeps its meaning: a repeated
// identical disc path is not a collision, two different ones are.
//
// The fold is `to_lowercase`, which closes the ASCII and simple-Unicode
// case classes. It is NOT full case folding, and it does NOT normalize:
// APFS also unifies NFC/NFD, so `é` recorded as U+00E9 and as
// `e`+U+0301 remain two keys here and one file there. That residual is
// a false NEGATIVE (a missed collision, never a spurious one), so this
// is a strict improvement rather than a complete model of the host.
let mut register = |key: PathBuf| -> Result<()> {
let folded = key.to_string_lossy().to_lowercase();
if let Some(prev) = seen_hosts.insert(folded, child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: key.to_string_lossy().into_owned(),
});
}
Ok(())
};
register(child_rel.clone())?;
if !entry.is_dir {
register(with_partial_suffix(&child_rel))?;
}
if entry.is_dir {
dirs.push(child_rel.clone());
@@ -479,6 +563,7 @@ fn extract_one_file<S: SectorSource>(
pf: &PlannedFile,
total_bytes: u64,
done_bytes: &mut u64,
done_unreadable: &mut u64,
opts: &ExtractOptions,
) -> Result<(FileResult, bool)> {
let final_path = dest.join(&pf.host_rel);
@@ -508,16 +593,53 @@ fn extract_one_file<S: SectorSource>(
finalize_file(writer, &partial_path, pf.size, &final_path)?;
fr.complete = true;
*done_bytes = done_bytes.saturating_add(pf.size);
report(opts, *done_bytes, total_bytes);
report(opts, *done_bytes, *done_unreadable, total_bytes);
return Ok((fr, false));
}
let mut written: u64 = 0;
let mut buf = vec![0u8; READ_BATCH_SECTORS as usize * SECTOR_BYTES];
'extents: for &(abs_lba, byte_len) in &pf.extents {
'extents: for &crate::udf::AbsExtent {
lba: abs_lba,
len: byte_len,
recorded,
} in &pf.extents
{
if written >= pf.size {
break;
}
// ECMA-167 4/14.14.1.1 type 1: allocated but NOT recorded. The extent
// is part of the file's byte space and its contents are defined to be
// zeros, so write the zeros WITHOUT reading the media — those sectors
// hold nothing this file ever wrote (on an AACS disc, ciphertext that
// decrypts to noise). This mirrors `UdfFs::read_file_limited`, which
// takes the same decision from the same flag. Skipping the extent
// entirely instead would slide every later extent's bytes down by the
// hole's length.
if !recorded {
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64);
let hole_bytes = (sectors * SECTOR_BYTES_U64).min(pf.size.saturating_sub(written));
let mut left = hole_bytes;
for b in buf.iter_mut() {
*b = 0;
}
while left > 0 {
let n = left.min(buf.len() as u64) as usize;
write_all(&mut writer, &buf[..n], &partial_path)?;
written = written.saturating_add(n as u64);
*done_bytes = done_bytes.saturating_add(n as u64);
left -= n as u64;
}
fr.bytes_good = fr.bytes_good.saturating_add(hole_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
if opts.cancelled(cont) {
return Ok((fr, true));
}
if written >= pf.size {
break 'extents;
}
continue;
}
// Anchor AACS unit alignment at THIS extent's start (clip-anchored
// gate, not absolute LBA 0 and NOT the file's first extent). A file
// may span multiple extents (fragmented / Long-AD / continuation ICB
@@ -557,10 +679,11 @@ fn extract_one_file<S: SectorSource>(
}
write_all(&mut writer, &buf[..usable], &partial_path)?;
fr.bytes_unreadable = fr.bytes_unreadable.saturating_add(usable as u64);
*done_unreadable = done_unreadable.saturating_add(usable as u64);
}
written = written.saturating_add(usable as u64);
*done_bytes = done_bytes.saturating_add(usable as u64);
let cont = report(opts, *done_bytes, total_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
sector_off += batch;
if opts.cancelled(cont) {
// Leave the `.partial`; do NOT rename. The aggregate run
@@ -673,15 +796,20 @@ fn finalize_file(
/// Emit a progress report. Returns `true` to continue, `false` if the sink
/// requested an early stop (or there is no sink — always continue).
fn report(opts: &ExtractOptions, done: u64, total: u64) -> bool {
fn report(opts: &ExtractOptions, done: u64, unreadable: u64, total: u64) -> bool {
match opts.progress {
Some(p) => {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Mux,
work_done: done,
work_total: total,
bytes_good_total: done,
bytes_unreadable_total: 0,
// `done` counts good AND zero-filled-unreadable bytes together;
// split them so a consumer driven only by the live progress
// channel sees a holed extraction as holed rather than as a clean
// climb to 100%. The final ExtractResult already carries the true
// split — this used to pin unreadable at 0 and call every byte good.
bytes_good_total: done.saturating_sub(unreadable),
bytes_unreadable_total: unreadable,
bytes_pending_total: 0,
bytes_retryable_total: 0,
bytes_total_disc: total,
@@ -1165,6 +1293,17 @@ mod tests {
s
}
/// Build a file ICB whose FIRST short AD is an ECMA-167 4/14.14.1.1 type-1
/// (allocated, NOT recorded) extent and whose second is ordinary recorded
/// data. Both are `sectors_each` sectors long.
fn build_hole_then_data_icb(sectors_each: u32, hole_lba: u32, data_lba: u32) -> [u8; 2048] {
let mut s = build_two_extent_icb(sectors_each, hole_lba, data_lba);
let len = sectors_each * SECTOR_BYTES as u32;
// Re-stamp AD #0 with extent type 1 in bits 30..31 of the length field.
s[216..220].copy_from_slice(&(0x4000_0000u32 | (len & 0x3FFF_FFFF)).to_le_bytes());
s
}
/// Encrypt the clear unit from `clear_aacs_unit(tag)` under `unit_key` so
/// `aacs::content::decrypt_unit` recovers it cleanly (zero decrypt loss).
/// `tag` distinguishes two units' payloads.
@@ -1578,6 +1717,62 @@ mod tests {
assert!(matches!(err, Error::DirNameCollision { .. }));
}
/// Two disc names differing only by CASE are one host file on macOS APFS
/// and Windows NTFS, both case-insensitive by default. Keyed by the
/// case-preserving path, the collision map sees two entries, raises
/// nothing, and the second file extracted overwrites the first — while both
/// `PlannedFile`s report `complete: true`. Silent data loss reported as a
/// clean extract is the one outcome this crate must never produce, so the
/// host-equivalence key has to model the host's namespace, not the disc's.
#[test]
fn names_differing_only_by_case_are_a_collision() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("Movie", 30, 31, b"a".to_vec(), false),
file("movie", 32, 33, b"b".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("case_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err("two names that fold to one host file must collide");
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A file's in-flight `.partial` path shares the host namespace with every
/// other planned file's FINAL path. A disc carrying both `X` and
/// `X.partial` plans two distinct final names, so nothing collides — but
/// extracting `X` writes through `X.partial`, the same host path the other
/// file owns. Whichever lands second truncates the other, and both entries
/// are still reported complete.
#[test]
fn a_files_partial_path_colliding_with_another_files_final_name_is_an_error() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("X", 30, 31, b"aaaa".to_vec(), false),
file("X.partial", 32, 33, b"bbbb".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("partial_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err(
"X's temp path IS X.partial's final path — one host file for two \
disc files, so it must be refused up front",
);
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A non-empty target dir is refused without `--force`, and accepted with.
#[test]
fn non_empty_target_requires_force() {
@@ -1707,6 +1902,63 @@ mod tests {
);
}
/// An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED:
/// the space belongs to the file and occupies its byte range, but nothing
/// was ever written there, and the standard defines its contents as zeros.
/// `read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
/// `read_file_limited` honours it by emitting zeros WITHOUT touching the
/// media. The tree extractor reads the same ICBs and must agree: reading
/// those sectors returns whatever the media happens to hold there — on an
/// AACS disc, ciphertext that decrypts to noise — and writes it into the
/// extracted file as if the disc had recorded it.
///
/// The hole here is filled with a recognisable non-zero pattern, so
/// "emitted zeros" and "read the media" are told apart by CONTENT.
#[test]
fn extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it() {
const SECTORS_EACH: u32 = 1;
const HOLE: u32 = 5000;
const DATA: u32 = 5004;
let hole_bytes = vec![0xEEu8; SECTOR_BYTES];
let data_bytes = vec![0x5Au8; SECTOR_BYTES];
// The file's byte space: the hole's zeros FIRST, then the real data.
let mut expect = vec![0u8; SECTOR_BYTES];
expect.extend_from_slice(&data_bytes);
let mut disc = MemDisc::new();
build_udf_skeleton(&mut disc, 10);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "INDEX.BDMV", 42, false, false);
disc.put(
PART_START + 42,
build_hole_then_data_icb(SECTORS_EACH, HOLE, DATA),
);
disc.put_bytes(PART_START + HOLE, &hole_bytes);
disc.put_bytes(PART_START + DATA, &data_bytes);
disc.put(PART_START + 10, build_dir_icb(11, root_fids.len() as u32));
disc.put_bytes(PART_START + 11, &root_fids);
let out = TmpDir::new("unrecorded_extent");
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract");
let got = read_out(out.path(), "INDEX.BDMV").expect("file written");
assert_eq!(
got, expect,
"an unrecorded extent contributes zeros to the file, never the \
bytes that happen to sit on those sectors"
);
assert_eq!(
res.bytes_unreadable, 0,
"a hole is not a read failure — nothing was attempted"
);
assert!(res.complete);
}
/// Focused alignment-computation check underpinning the per-extent fix:
/// when each extent anchors its OWN start as the unit base, the extent's
/// own batch starts are always unit-aligned; anchoring a later extent
@@ -1918,6 +2170,87 @@ mod tests {
assert!(res.complete);
}
/// A VTS that IS scrambled but whose key could not be recovered must
/// FAIL, not borrow another VTS's key.
///
/// `crack_key` returns `Option`, which collapses "no scrambled sector was
/// seen" with "scrambled sectors were seen and no key came out"; the
/// fallback then descrambled this VTS under the disc-wide key and
/// `extract_tree` reported `complete = true` at exit 0, because no read had
/// failed. `CrackOutcome` exists to keep those two apart, and every sibling
/// path in the crate already honours it.
///
/// The fixture: VTS_01 is crackable, VTS_02 is scrambled with NO periodic
/// crib, so its crack genuinely fails.
#[test]
fn a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key() {
let key_1 = [0x10u8, 0x20, 0x30, 0x40, 0x50];
let (_plain_1, scrambled_1) = {
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x01];
let mut plain = vec![0u8; 2048];
plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plain[0x14] = 0x10;
let pat: Vec<u8> = (0..8)
.map(|k| (0xA0u8.wrapping_add(k as u8) ^ 0x01) ^ 0x5A)
.collect();
for (i, b) in plain.iter_mut().enumerate().skip(0x59) {
*b = pat[i % 8];
}
plain[0x54..0x59].copy_from_slice(&seed);
let mut scrambled = plain.clone();
lfsr::scramble_sector(&key_1, &mut scrambled);
(plain, scrambled)
};
// Scrambled — the pack header and the 0x14 flag are set, so the scan
// SEES ciphertext — but the cleartext region carries no periodic run,
// so the Stevenson crib never forms and no key is recoverable.
let uncrackable = {
let mut sect = vec![0u8; 2048];
sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sect[0x14] = 0x10;
for (i, b) in sect.iter_mut().enumerate().skip(0x59) {
// Non-repeating, so no run of any period survives to 0x80.
*b = (i as u8).wrapping_mul(37).wrapping_add(11);
}
sect
};
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "VIDEO_TS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTS_01_1.VOB", 30, 5000, scrambled_1, false),
file("VTS_02_1.VOB", 32, 6000, uncrackable, false),
],
subdirs: vec![],
}],
};
let mut disc = build_disc(root);
let out = TmpDir::new("css_uncrackable_vts");
let mut d = clear_disc();
d.content_format = crate::disc::ContentFormat::MpegPs;
d.css = Some(crate::css::CssState {
title_key: [0xFFu8; 5],
crack_span: None,
});
let err = d
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err(
"a VTS whose key could not be recovered must be a hard error, \
not a silent extract under another VTS's key",
);
assert!(
matches!(err, Error::CssKeyMissing),
"expected CssKeyMissing, got {err:?}"
);
}
/// `Borrowed` is a thin forwarding wrapper the decrypting decorator uses
/// to avoid taking ownership of the caller's reader -- every
/// `SectorSource` method must forward to the wrapped `&mut dyn
+1774 -52
View File
File diff suppressed because it is too large Load Diff
+300 -8
View File
@@ -2056,17 +2056,17 @@ impl Disc {
// derived below from the AACS MKB generation, not the tree.
let (mut titles, content_format) = if udf_fs.find_dir("/BDMV").is_some() {
(
Self::scan_bluray_titles(reader, &udf_fs),
Self::scan_bluray_titles(reader, &udf_fs, opts.halt.as_ref())?,
ContentFormat::BdTs,
)
} else if udf_fs.find_dir("/HVDVD_TS").is_some() {
(
Self::scan_hddvd_titles(reader, &udf_fs),
Self::scan_hddvd_titles(reader, &udf_fs, opts.halt.as_ref())?,
ContentFormat::MpegPs,
)
} else if udf_fs.find_dir("/VIDEO_TS").is_some() {
(
Self::scan_dvd_titles(reader, &udf_fs),
Self::scan_dvd_titles(reader, &udf_fs, opts.halt.as_ref())?,
ContentFormat::MpegPs,
)
} else {
@@ -2187,7 +2187,9 @@ impl Disc {
/// **Sort priority (titles[0] = most likely main feature):**
/// 1. Real titles (`size_bytes ≤ capacity_bytes`) before virtual
/// composites. The capacity check is a hard "physically
/// possible data on this disc" gate.
/// possible data on this disc" gate. `capacity_bytes == 0` means
/// the capacity is UNKNOWN (READ CAPACITY failed), so the gate is
/// skipped entirely rather than demoting every real title.
/// 2. Among real titles, LARGEST physical size first — the main
/// feature is the biggest real title on the disc. (This replaced
/// the old clip-count ordering, which mis-ranked chapter-per-clip
@@ -2249,8 +2251,18 @@ impl Disc {
// A title bigger than the whole disc is a "play-all" composite artifact
// (its declared size double-counts clips shared with other playlists) —
// demote it below any real single title.
let a_oversize = a.size_bytes > capacity_bytes;
let b_oversize = b.size_bytes > capacity_bytes;
//
// `capacity_bytes == 0` means the capacity is UNKNOWN, not that the
// disc holds nothing: `read_udf` substitutes 0 when READ CAPACITY
// fails and scans on regardless. Applied literally the gate would
// INVERT there — every real title (`size_bytes > 0`) would be
// "oversize" and demoted, while a CLPI-less `size_bytes == 0` title
// would not, landing at `titles[0]` ahead of the feature. With no
// capacity to compare against, the gate is inert and the size /
// duration / audio keys decide the order on their own.
let capacity_known = capacity_bytes > 0;
let a_oversize = capacity_known && a.size_bytes > capacity_bytes;
let b_oversize = capacity_known && b.size_bytes > capacity_bytes;
a_oversize
.cmp(&b_oversize)
// PRIMARY: largest physical size = the main feature. Robust where
@@ -3575,8 +3587,8 @@ mod tests {
) -> (crate::udf::fixture::MemDisc, udf::UdfFs) {
use crate::udf::fixture::*;
let files = vec![
file("MAIN.EVO", 100, 5_000, main_bytes, true),
file("OTHER.EVO", 101, 50_000, other_bytes, true),
file("MAIN.EVO", 100, 5_000, main_bytes as u64, true),
file("OTHER.EVO", 101, 50_000, other_bytes as u64, true),
];
let root = DirSpec {
name: String::new(),
@@ -3598,6 +3610,237 @@ mod tests {
(disc, udf)
}
/// A cancelled [`crate::halt::Halt`] must stop the HD-DVD title scan.
///
/// The scan is bounded but big — up to `MAX_HDDVD_CLIPS` clips, each
/// costing an ICB resolve plus a 16 MiB `EVO_PROBE_SECTORS` stream probe —
/// so on a live drive an operator Stop that only takes effect after the
/// whole enumerator returns is no Stop at all. `ScanOptions::halt` is
/// already honoured by the CSS crack and the forced-subtitle probe; the
/// title enumerator must honour it too.
///
/// It must also not report a HALF-ENUMERATED disc as a successful scan:
/// a truncated title list is indistinguishable from a disc that genuinely
/// holds fewer titles.
#[test]
fn scan_with_cancelled_halt_stops_the_hddvd_title_scan() {
use crate::udf::fixture::PART_START;
/// Counts reads that land on CLIP DATA (at or past the first clip's
/// data extent) — i.e. the per-clip stream probing, the expensive part
/// of the scan. Everything below that is filesystem metadata.
struct CountingReader<'a> {
inner: &'a mut crate::udf::fixture::MemDisc,
clip_reads: usize,
}
impl SectorSource for CountingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
if lba >= PART_START + 5_000 {
self.clip_reads += 1;
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
let (mut disc, udf) = hddvd_two_clip_disc(3_000_000, 5_000_000);
let halt = crate::halt::Halt::new();
halt.cancel();
let opts = ScanOptions {
halt: Some(halt),
..Default::default()
};
let mut reader = CountingReader {
inner: &mut disc,
clip_reads: 0,
};
let res = Disc::scan_with(&mut reader, 3_997_952, None, None, &opts, udf);
let clip_reads = reader.clip_reads;
assert!(
matches!(res, Err(Error::Halted)),
"a cancelled scan must say so, not return a partial title list as \
a completed scan; got {:?}",
res.map(|d| d.titles.len())
);
assert_eq!(
clip_reads, 0,
"cancellation must be observed before the per-clip stream probes, \
not after all of them"
);
}
/// `scan_with` must hand `ScanOptions::halt` to the BLU-RAY enumerator.
///
/// Every BD and DVD cancellation test in this crate calls
/// `scan_bluray_titles` / `scan_dvd_titles` DIRECTLY, so the three-line
/// wiring in `scan_with` that connects the operator's Stop to them was
/// covered by nothing at all: replacing `opts.halt.as_ref()` with `None`
/// on either branch left the whole suite green while a Stop during a BD
/// or DVD scan silently did nothing. Only the HD-DVD branch was pinned.
///
/// The fixture is deliberately the smallest disc that takes the BD branch
/// — a bare `/BDMV` with no PLAYLIST — because the point under test is the
/// ARGUMENT, not the enumerator's own (separately tested) halt polling.
/// Nothing between `scan_with`'s entry and the enumerator reads the flag
/// (the disc is unencrypted, so the AACS path is skipped), so a green here
/// can only mean the flag arrived.
///
/// Mutation: `Self::scan_bluray_titles(reader, &udf_fs, None)?` fails here.
#[test]
fn scan_with_passes_the_halt_flag_to_the_bluray_enumerator() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "BDMV".into(),
icb_lba: 12,
dir_data_lba: 13,
files: Vec::new(),
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
// Sanity: the same disc scans clean when nothing is cancelled, so a
// pass below cannot be some unrelated failure wearing Halted.
assert!(
Disc::scan_with(&mut disc, 500_000, None, None, &ScanOptions::default(), udf).is_ok(),
"fixture must scan successfully when not cancelled"
);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
let halt = crate::halt::Halt::new();
halt.cancel();
let opts = ScanOptions {
halt: Some(halt),
..Default::default()
};
let res = Disc::scan_with(&mut disc, 500_000, None, None, &opts, udf);
assert!(
matches!(res, Err(Error::Halted)),
"a cancelled BD scan must say so; returning a title list built \
after Stop reports a truncated enumeration as a completed one. \
Got {:?}",
res.map(|d| d.titles.len())
);
}
/// The DVD half of the same wiring, and the same reasoning.
///
/// Mutation: `Self::scan_dvd_titles(reader, &udf_fs, None)?` fails here.
#[test]
fn scan_with_passes_the_halt_flag_to_the_dvd_enumerator() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "VIDEO_TS".into(),
icb_lba: 50,
dir_data_lba: 51,
files: vec![
file_with("VIDEO_TS.IFO", 60, 5000, dvd_vmg_bytes(), true),
file_with("VTS_01_0.IFO", 62, 6000, dvd_vts_bytes(1000, 10, 10), true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
Disc::scan_with(&mut disc, 500_000, None, None, &ScanOptions::default(), udf).is_ok(),
"fixture must scan successfully when not cancelled"
);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
let halt = crate::halt::Halt::new();
halt.cancel();
let opts = ScanOptions {
halt: Some(halt),
..Default::default()
};
let res = Disc::scan_with(&mut disc, 500_000, None, None, &opts, udf);
assert!(
matches!(res, Err(Error::Halted)),
"a cancelled DVD scan must say so, not hand back whatever it had \
enumerated so far as a finished scan. Got {:?}",
res.map(|d| d.titles.len())
);
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The HD-DVD enumerator must not
/// swallow that into a successful scan.
///
/// Measured before this was fixed: the scan returned `Ok` with both
/// titles present and ZERO streams on each — a cancelled scan wearing the
/// shape of a disc whose clips carry no video or audio. Downstream that is
/// a title list to cache, display and rip from.
#[test]
fn halted_reads_do_not_report_the_hddvd_scan_as_successful() {
use crate::udf::fixture::PART_START;
/// Fails clip-data reads the way a live drive does once Stop is
/// pressed; filesystem metadata below the first clip still resolves,
/// so the scan gets far enough to enumerate titles.
struct HaltingReader<'a> {
inner: &'a mut crate::udf::fixture::MemDisc,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
if lba >= PART_START + 5_000 {
return Err(Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
let (mut disc, udf) = hddvd_two_clip_disc(3_000_000, 5_000_000);
let mut reader = HaltingReader { inner: &mut disc };
let res = Disc::scan_with(
&mut reader,
3_997_952,
None,
None,
&ScanOptions::default(),
udf,
);
assert!(
matches!(res, Err(Error::Halted)),
"reads cancelled by the drive's own halt flag must surface as a \
cancelled scan, not as titles that merely look stream-less; got \
{:?}",
res.map(|d| d
.titles
.iter()
.map(|t| (t.playlist.clone(), t.streams.len()))
.collect::<Vec<_>>())
);
}
/// `scan_with`'s `capacity_bytes = capacity as u64 * 2048` feeds
/// `canonical_title_order`'s "bigger than the whole disc = play-all
/// composite" threshold. Chosen so `capacity * 2048` clears both titles
@@ -6817,6 +7060,55 @@ mod tests {
);
}
/// `capacity_bytes == 0` means the disc capacity is UNKNOWN — `read_udf`
/// substitutes 0 when READ CAPACITY fails (transient spin-up, SCSI error)
/// and proceeds with the scan. It does NOT mean "the disc holds nothing".
///
/// With a literal reading of the gate, 0 inverts it: EVERY real title is
/// `size_bytes > 0` and therefore "oversize", while a title with no CLPI
/// (`size_bytes == 0`) is not — so the empty title sorts to `titles[0]`
/// ahead of the feature and `freemkv -t 1` rips garbage. The gate must be
/// INERT when the capacity is unknown.
#[test]
fn canonical_order_unknown_capacity_does_not_demote_every_real_title() {
const UNKNOWN: u64 = 0; // READ CAPACITY failed
let feature = title_with("00800.mpls", 7_320.0, 57_200_000_000, 1);
// A playlist whose CLPI files are missing/unparseable: no declared size.
let sizeless = title_with("00001.mpls", 120.0, 0, 1);
let mut titles = [sizeless, feature];
titles.sort_by(|a, b| Disc::canonical_title_order(a, b, UNKNOWN));
assert_eq!(
titles[0].playlist, "00800.mpls",
"with an UNKNOWN capacity the real feature must still sort first; \
a size-0 title must not be promoted ahead of it"
);
assert_eq!(titles[1].playlist, "00001.mpls");
}
/// Control for [`canonical_order_unknown_capacity_does_not_demote_every_real_title`]:
/// making the gate inert on an UNKNOWN capacity must not make it dead. With
/// a KNOWN capacity a genuinely oversize play-all composite is still demoted
/// below a smaller real title — even though "largest size first" would
/// otherwise rank it first. Asserted on the comparator in both argument
/// orders so an inconsistent comparator cannot pass.
#[test]
fn canonical_order_known_capacity_still_demotes_a_genuinely_oversize_title() {
use std::cmp::Ordering;
const CAP: u64 = 58_500_000_000;
let composite = title_with("00020.mpls", 15_180.0, 92_400_000_000, 253);
let real = title_with("00800.mpls", 7_320.0, 57_200_000_000, 1);
assert_eq!(
Disc::canonical_title_order(&real, &composite, CAP),
Ordering::Less,
"a known capacity must still demote the oversize composite"
);
assert_eq!(
Disc::canonical_title_order(&composite, &real, CAP),
Ordering::Greater,
"…in either argument order"
);
}
// ── audio_richness: the same-size / same-duration tiebreak ─────────────
/// A title carrying the given audio tracks, with size and duration fixed so
+421 -57
View File
@@ -8,6 +8,17 @@ pub fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
// A failed `ioctl(SG_IO)` and a vanished device never produced a SCSI
// reply at all — they are dead-bus faults, not recoverable bad sectors.
// `Error::is_scsi_transport_failure` (error.rs) already declares both
// variants transport failures so sweep / patch / fill_extents abort the
// pass instead of zero-filling against a wedged device; callers that
// flatten an error through here (Drive::read_one, DiscStream::fill_extents,
// freemkv-engine's recovery sweep/patch) would otherwise collapse them to
// status 0 and silently destroy that classification.
Error::IoError { .. } | Error::DeviceNotFound { .. } => {
(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None)
}
_ => (0, None),
}
}
@@ -24,6 +35,15 @@ pub(crate) mod macos;
#[cfg(windows)]
pub(crate) mod windows;
// Pick the platform module ONCE, here, so the cross-platform entry points below
// dispatch through `platform::…` with no per-function `#[cfg]` in their bodies.
#[cfg(target_os = "linux")]
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
pub(crate) use macos as platform;
#[cfg(windows)]
pub(crate) use windows as platform;
use crate::error::{Error, Result};
use crate::event::Event;
use crate::identity::DriveId;
@@ -279,22 +299,33 @@ impl Drive {
for attempt in 0..60u64 {
hb.tick(attempt, 60);
let mut buf = [0u8; 0];
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
// `checked_exec`, not a bare `execute`. This poll was the ONE
// drive path that talked to the transport directly, and it is the
// longest-running one: 60 x 500 ms = ~30 s. It never read
// `self.halt`, so an operator Stop pressed while a cold drive spun
// up was ignored for up to half a minute while every other path in
// this file returned `Halted` at its next command boundary. A Stop
// that does nothing for 30 s is indistinguishable from a hung app.
//
// A TUR that FAILS is the ordinary not-ready-yet answer and must
// keep the loop going — only `Halted` aborts it.
match self.checked_exec(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
}
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => {}
}
std::thread::sleep(std::time::Duration::from_millis(500));
// Halt-aware backoff: the flag can also flip DURING the 500 ms
// gap, which is where most of the 30 s is actually spent.
sleep_until_halted(&self.halt, std::time::Duration::from_millis(500))?;
}
tracing::warn!(
target: "freemkv::drive",
@@ -926,11 +957,34 @@ impl Drive {
// range NonTrimmed and retries (a loud miss, never a silent commit).
// The sector/file path enforces the same invariant in
// sector/prefetched.rs; this is the live-drive counterpart.
Ok(_) => Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
}),
//
// SAY SO. The refusal was correct and completely silent: the
// sibling `Err(e)` arm below warns with lba/count/status, this one
// logged nothing at all, so a drive that residual-underruns on
// GOOD status was indistinguishable in the logs from a scratched
// disc — two different populations (replace the drive vs. clean
// the disc) collapsed into one. Absence of a log is itself the
// defect here; the read still fails either way.
//
// `scsi_status`/`sense` are deliberately absent from the line
// rather than faked: the command SUCCEEDED, so there is no sense
// data to report. `transferred` vs `expected` is the whole signal.
Ok(result) => {
tracing::warn!(
target: "freemkv::drive",
lba,
count,
transferred = result.bytes_transferred,
expected = count as usize * 2048,
code = crate::error::E_DISC_READ,
"READ(10) returned GOOD status with a residual underrun; refusing the short transfer"
);
Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
})
}
Err(Error::Halted) => Err(Error::Halted),
Err(e) => {
let (status, sense) = extract_scsi_context(&e);
@@ -1086,14 +1140,25 @@ impl Drive {
let stop = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x00, 0]; // START=0, LOEJ=0 → spin down
let start = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x01, 0]; // START=1, LOEJ=0 → spin up
let mut buf = [0u8; 0];
self.scsi
.as_mut()
.execute(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS));
self.scsi
.as_mut()
.execute(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS));
// `checked_exec` + `sleep_until_halted`, not `execute` + a blind
// `thread::sleep`. This routine is ~15 s of deliberate waiting
// (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) issued from the
// recovery path, i.e. exactly when a run is going badly and the
// operator is most likely to press Stop. Both commands bypassed the
// halt flag and both sleeps were unconditional, so the whole cycle was
// deaf: Stop appeared to hang. Nothing here is un-interruptible — a
// half-finished spin cycle leaves the drive spun down, which the next
// command spins back up.
self.checked_exec(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS),
)?;
self.checked_exec(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS),
)?;
Ok(())
}
@@ -1299,11 +1364,18 @@ pub(crate) fn decode_read_capacity(buf: &[u8; 8], bytes_transferred: usize) -> R
last_lba.checked_add(1).ok_or(Error::DiscCapacityOverflow)
}
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
/// to true. Kept for the unit tests that cover the slicing behaviour;
/// production code paths no longer sleep on the recovery hot path
/// (recovery loop removed in 0.13.6).
#[cfg(test)]
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping to
/// true, returning [`Error::Halted`].
///
/// This was `#[cfg(test)]` for two releases, kept alive only by the four unit
/// tests below, while the two production paths that actually sleep — the
/// `wait_ready` poll backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle
/// pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
/// `std::thread::sleep` and so were DEAF to the operator's Stop for ~30 s and
/// ~15 s respectively. Every other drive path returns `Halted` at the next
/// `checked_exec` boundary; a Stop pressed during spin-up simply did nothing
/// visible until the poll ran out. The primitive existed; the sleeping code
/// just did not call it. It is production code again.
fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<()> {
const SLICE: std::time::Duration = std::time::Duration::from_millis(100);
let deadline = std::time::Instant::now() + total;
@@ -1322,18 +1394,7 @@ fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<(
/// Internal: discover drive paths + IDs without opening full Drive objects.
fn discover_drives() -> Vec<(String, DriveId)> {
#[cfg(target_os = "linux")]
{
linux::find_drives()
}
#[cfg(target_os = "macos")]
{
macos::find_drives()
}
#[cfg(windows)]
{
windows::find_drives()
}
platform::find_drives()
}
/// Structured outcome of [`resolve_device`] — a machine-readable signal
@@ -1353,20 +1414,14 @@ pub enum DeviceResolution {
/// Resolve a device path to its raw SCSI device. Returns the resolved
/// path plus a structured [`DeviceResolution`] signal describing whether
/// any substitution happened; the application layer maps that to UX text.
///
/// Staged, not yet wired: the cross-platform dispatch is kept ready for the
/// caller that will consume it, so the per-platform implementations below it
/// (and their tests) stay live. `allow(dead_code)` marks that deliberately —
/// this is not an accidental orphan.
#[allow(dead_code)]
pub(crate) fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
#[cfg(target_os = "linux")]
{
linux::resolve_device(path)
}
#[cfg(target_os = "macos")]
{
macos::resolve_device(path)
}
#[cfg(windows)]
{
windows::resolve_device(path)
}
platform::resolve_device(path)
}
#[cfg(test)]
@@ -1936,6 +1991,90 @@ mod command_tests {
assert!(err.scsi_sense().is_none());
}
/// `extract_scsi_context` must map the two non-SCSI dead-bus faults
/// (a failed `ioctl(SG_IO)` → `Error::IoError`, a vanished fd →
/// `Error::DeviceNotFound`) to the 0xFF TRANSPORT_FAILURE sentinel, not
/// to 0x00. Everything else keeps the (0, None) catch-all.
/// Spec: `Error::is_scsi_transport_failure` (error.rs) declares both
/// variants transport failures so sweep/patch/fill_extents abort
/// the pass instead of zero-filling against a wedged device.
/// Mutation: returning (0, None) here flattens IoError into
/// `DiscRead { status: Some(0) }`, which is_scsi_transport_failure
/// rejects — a wedged USB bridge zero-fills the whole title.
#[test]
fn extract_scsi_context_maps_dead_bus_faults_to_transport_failure() {
let (status, sense) = extract_scsi_context(&Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a failed ioctl(SG_IO) is a transport-layer fault"
);
assert!(sense.is_none(), "no SCSI reply means no sense data");
let (status, sense) = extract_scsi_context(&Error::DeviceNotFound {
path: "/dev/sg9".into(),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a vanished device is a transport-layer fault"
);
assert!(sense.is_none());
// Control: the catch-all still yields (0, None) for unrelated errors,
// so the two asserts above are about these variants specifically.
assert_eq!(extract_scsi_context(&Error::Halted), (0, None));
// Control: real SCSI replies still pass their own status through.
let s = crate::scsi::ScsiSense {
sense_key: 3,
asc: 0x11,
ascq: 0x00,
};
assert_eq!(
extract_scsi_context(&Error::ScsiError {
opcode: 0x28,
status: 0x02,
sense: Some(s),
}),
(0x02, Some(s))
);
}
/// End-to-end: an `Error::IoError` raised by the transport must still be
/// classified as a transport failure after `Drive::read` flattens it into
/// `Error::DiscRead`. Without the `extract_scsi_context` mapping the
/// variant is destroyed (status becomes `Some(0)`) and the caller treats a
/// dead bus as a recoverable bad sector.
#[test]
fn read_io_error_surfaces_as_transport_failure_not_a_bad_sector() {
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
},
}));
let mut buf = vec![0u8; 2048];
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a wedged bus must abort the pass, not zero-fill: got {err:?}"
);
// The same for a device that vanished mid-read.
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::DeviceNotFound {
path: "/dev/sg9".into(),
},
}));
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a vanished device must abort the pass: got {err:?}"
);
}
#[test]
fn read_returns_halted_before_dispatch_without_touching_transport() {
// When the halt flag is set, checked_exec returns Halted BEFORE
@@ -2589,6 +2728,231 @@ mod command_tests {
);
}
/// A Stop pressed before the poll starts must be answered at once.
///
/// `wait_ready` was the ONE drive path that called
/// `self.scsi.as_mut().execute(..)` instead of `checked_exec`, and its
/// 60 x 500 ms loop never read `self.halt`. A cancel during spin-up was
/// therefore ignored for ~30 s — the test above measures exactly how long
/// — while every other drive path returns `Halted` at its next command
/// boundary. Thirty seconds of a dead Stop button is indistinguishable
/// from a hung application.
///
/// Mutation: restoring the bare `execute` makes this run the full poll and
/// return `DeviceNotReady`, failing both assertions.
#[test]
fn wait_ready_returns_halted_when_stopped_before_the_poll() {
struct NeverReady;
impl ScsiTransport for NeverReady {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(NeverReady));
d.halt();
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop is the operator, not a drive that failed to spin up: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"the poll must abandon immediately, not run its ~30 s course"
);
}
/// ...and a Stop pressed PART WAY THROUGH the poll must be answered at the
/// next command boundary, not at the end of the 30 s.
///
/// The flag is flipped by the transport itself on its third TEST UNIT
/// READY, which is deterministic (no wall-clock race): `checked_exec`
/// re-reads the flag after the command completes, so the loop must exit on
/// attempt three of sixty.
#[test]
fn wait_ready_returns_halted_when_stopped_during_the_poll() {
struct StopsOnThirdPoll {
halt: Arc<AtomicBool>,
seen: usize,
}
impl ScsiTransport for StopsOnThirdPoll {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
self.seen += 1;
if self.seen == 3 {
self.halt.store(true, Ordering::Relaxed);
}
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(StopsOnThirdPoll {
halt: Arc::new(AtomicBool::new(false)),
seen: 0,
}));
// Hand the transport the drive's OWN flag, so setting it is exactly
// what an operator's Stop does.
let flag = d.halt_flag();
d.scsi = Box::new(StopsOnThirdPoll {
halt: flag,
seen: 0,
});
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a mid-poll Stop must surface as Halted, not as DeviceNotReady \
after the full 30 s: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"must exit on the third poll, not the sixtieth"
);
}
/// `spin_cycle` must not be deaf to Stop for its ~15 s of deliberate
/// waiting.
///
/// Both START STOP UNIT commands went out through a bare `execute` and
/// both pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) were plain
/// `std::thread::sleep`, so the whole routine ignored the halt flag. It
/// runs from the RECOVERY path — precisely when a run is going badly and
/// the operator is most likely to press Stop.
///
/// The flag is set BEFORE the call, so this is deterministic: the very
/// first `checked_exec` must refuse.
///
/// Mutation: restoring either bare `execute` lets the first command
/// through and the test then waits out a real 5 s sleep, failing the
/// elapsed bound.
#[test]
fn spin_cycle_returns_halted_when_stopped_before_it_starts() {
let RecordingHarness {
drive: mut d,
cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
d.halt();
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop must abort the spin cycle: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"no sleep may run after the flag is set"
);
assert!(
cdb.lock().unwrap().is_empty(),
"not one START STOP UNIT may be issued after Stop"
);
}
/// ...and a Stop that lands DURING the spin-down pause must wake it.
///
/// This is the half a `checked_exec` alone cannot fix: the flag flips
/// while the thread is parked inside the 5 s `SPIN_DOWN_IDLE_SECS` sleep,
/// so only a halt-aware sleep can observe it. `sleep_until_halted` — which
/// already lived in this file with four tests, marked `#[cfg(test)]` and
/// called from nowhere — wakes within ~100 ms.
///
/// The margin is deliberately enormous (≈0.3 s against 5 s) so the bound
/// is not a wall-clock race: a plain `thread::sleep` cannot come in under
/// two seconds, and the halt-aware one cannot take that long.
#[test]
fn spin_cycle_wakes_from_its_spin_down_pause_when_stopped() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
let flag = d.halt_flag();
let stopper = std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(200));
flag.store(true, Ordering::Relaxed);
});
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
stopper.join().expect("stopper thread");
assert!(
matches!(r, Err(Error::Halted)),
"a Stop during the spin-down pause must surface as Halted: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"the pause must be halt-aware, not a blind {SPIN_DOWN_IDLE_SECS}s \
thread::sleep; took {:?}",
t0.elapsed()
);
}
/// A READ(10) that completes with GOOD status but a residual underrun is
/// correctly refused — and must SAY SO.
///
/// The refusal itself already existed and is right: the tail of the buffer
/// still holds stale bytes, so committing them would be silent
/// corruption. What was missing is any trace of it. The sibling `Err(e)`
/// arm warns with lba/count/status; this arm logged NOTHING, so a drive
/// that residual-underruns on GOOD status produced exactly the same
/// journal as a scratched disc — two populations with opposite remedies
/// (replace the drive vs. clean the disc) collapsed into one, with the
/// operator sent after the wrong one.
///
/// Mutation: deleting the `tracing::warn!` leaves no event; logging a
/// different code, or dropping the transferred/expected pair that is the
/// entire signal, fails the field assertions.
#[test]
fn read_logs_a_good_status_short_transfer() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(1024)); // half of one sector
let mut buf = vec![0u8; 2048];
let (r, events) = crate::testlog::capture(|| d.read(7, 1, &mut buf, false));
assert!(
matches!(r, Err(Error::DiscRead { sector: 7, .. })),
"a short transfer stays a failed read: {r:?}"
);
let line = events
.iter()
.find(|e| e.target == "freemkv::drive" && e.field("transferred").is_some())
.unwrap_or_else(|| {
panic!("a silently refused short transfer is the defect; got {events:?}")
});
assert_eq!(line.level, tracing::Level::WARN);
assert_eq!(line.field("lba"), Some("7"));
assert_eq!(line.field("transferred"), Some("1024"));
assert_eq!(
line.field("expected"),
Some("2048"),
"the underrun is only legible as transferred-vs-expected"
);
assert_eq!(
line.field("code"),
Some(crate::error::E_DISC_READ.to_string().as_str())
);
}
// ── report_key / mode_sense / read_buffer empty-vs-some ─────────
#[test]
+238 -126
View File
@@ -45,6 +45,7 @@ pub const E_INVALID_CDB_LENGTH: u16 = 4001;
// I/O (5xxx)
pub const E_IO_ERROR: u16 = 5000;
pub const E_SOURCE_TERMINATED: u16 = 5001;
// Disc format (6xxx)
pub const E_DISC_READ: u16 = 6000;
@@ -63,6 +64,22 @@ pub const E_SELECTION_PID_UNKNOWN: u16 = 6014;
pub const E_UDF_BUFFER_TOO_SMALL: u16 = 6012;
pub const E_UDF_NOT_FILESYSTEM: u16 = 6013;
pub const E_IMAGE_TRUNCATED: u16 = 6015;
pub const E_UDF_AD_CHAIN_TOO_LONG: u16 = 6016;
pub const E_UDF_UNRECORDED_EXTENT: u16 = 6017;
pub const E_UDF_EMBEDDED_DATA: u16 = 6018;
/// A file that EXISTS and whose allocation descriptors resolved without error,
/// yet yields not one usable extent: an empty AD list, or a list every entry of
/// which is zero-length or points at LBA 0. Reported by the HD-DVD clip
/// resolver (`disc::hddvd`).
///
/// It has no [`Error`] variant on purpose. `UdfFs::file_extents` returns
/// `Ok(vec![])` here rather than failing — the emptiness is only a defect in
/// the eye of a caller that needs bytes — so the condition is DETECTED by the
/// caller, not returned to it. The code exists so that detection can be
/// accounted in the log with something other than a neighbouring error's code:
/// logging it as E6017 would file a zero-length AD list as an authoring hole
/// and send whoever triages it at the wrong population.
pub const E_UDF_NO_USABLE_EXTENT: u16 = 6019;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -393,6 +410,18 @@ pub enum Error {
UdfNotFound {
path: String,
},
/// The file's ICB allocation list contains an unrecorded (ECMA-167
/// 4/14.14.1.1 type-1/type-2) extent: space allocated to the file at that
/// location but never written, so its true content there is zeros while
/// the media holds something else.
///
/// Raised by [`crate::udf::UdfFs::file_extents`] because a
/// `(lba, sector_count)` read plan cannot express a hole — reading it
/// splices undefined sectors into the rip as content, and dropping it
/// slides every later extent's byte space.
UdfUnrecordedExtent {
path: String,
},
/// The reader was addressable but the bytes are structurally NOT a UDF
/// filesystem — a deterministic tag/format mismatch (e.g. no Anchor Volume
/// Descriptor Pointer at sector 256, no partition descriptor, no File Set
@@ -405,6 +434,32 @@ pub enum Error {
/// 2048-byte sector. A contract violation on the public reader API —
/// returned instead of panicking on the slice.
UdfBufferTooSmall,
/// A file's allocation-descriptor continuation chain did not end within the
/// hop budget the UDF reader allows.
///
/// The budget exists so a crafted or corrupt disc cannot loop the reader
/// forever. Hitting it is NOT the end of the chain: the extents beyond that
/// point are unknown, so the extent list in hand describes only part of the
/// file. Returning that list would let a caller zero-pad the remainder to
/// the declared size and report a mostly-empty file as a complete
/// extraction, so the read fails instead.
UdfAdChainTooLong,
/// A file's ICB declares its data EMBEDDED inline (ECMA-167 4/14.6.8
/// allocation-descriptor type 3), so it has no out-of-line extents at all.
///
/// Returned only when a caller asked for a read plan over such a file.
/// The bytes in the allocation-descriptor field are then the file's own
/// CONTENT, not descriptors, so decoding them as (length, LBA) pairs
/// manufactures extents out of arbitrary data and points the reader at
/// unrelated sectors — a rip that completes at rc=0 carrying whatever
/// happened to be there. `read_directory` already refuses the same shape
/// for directories; this is the file half of that decision.
///
/// A file that legitimately stores its data this way is tiny (an ICB caps
/// it at well under 2 KiB — the AACS `*.inf` key files are the usual
/// case), and the callers that expect one read it via `read_inline_data`
/// long before extents are ever requested. A stream file cannot be one.
UdfEmbeddedData,
DiscTitleRange {
index: usize,
count: usize,
@@ -733,6 +788,19 @@ pub enum Error {
/// it silently leaves encrypted. The producer surfaces this rather
/// than emit still-encrypted bytes.
ExtentNotUnitAligned,
/// A [`crate::sector::SectorSource`] that feeds its reads from a
/// producer thread has terminated for good — the thread exited after
/// an error or before delivering the extents it was given — so it can
/// never return another byte.
///
/// It exists because the alternative answer is a lie: a dead source
/// that reports `Ok(0)` is indistinguishable from end-of-stream, and
/// `DiscStream::fill_extents` legitimately reads a short count as a
/// skippable hole — zero-filling and advancing over every remaining
/// sector of the title and still returning success. Unlike a bad
/// sector, this condition cannot be retried at a smaller size or
/// skipped past, so every consumer must abort the pass on it.
SourceTerminated,
/// An MPEG-TS packet under construction violated the 188-byte fixed
/// size (over-long adaptation field, overflowing payload, or a
/// short/mis-assembled packet). Indicates a muxer invariant break,
@@ -850,13 +918,17 @@ impl Error {
Error::ScsiError { .. } => E_SCSI_ERROR,
Error::InvalidCdbLength { .. } => E_INVALID_CDB_LENGTH,
Error::IoError { .. } => E_IO_ERROR,
Error::SourceTerminated => E_SOURCE_TERMINATED,
Error::DiscRead { .. } => E_DISC_READ,
Error::Halted => E_HALTED,
Error::MplsParse => E_MPLS_PARSE,
Error::ClpiParse => E_CLPI_PARSE,
Error::UdfNotFound { .. } => E_UDF_NOT_FOUND,
Error::UdfUnrecordedExtent { .. } => E_UDF_UNRECORDED_EXTENT,
Error::UdfNotFilesystem => E_UDF_NOT_FILESYSTEM,
Error::UdfBufferTooSmall => E_UDF_BUFFER_TOO_SMALL,
Error::UdfAdChainTooLong => E_UDF_AD_CHAIN_TOO_LONG,
Error::UdfEmbeddedData => E_UDF_EMBEDDED_DATA,
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::ShortImageRead { .. } => E_SHORT_IMAGE_READ,
Error::EmptyImage => E_EMPTY_IMAGE,
@@ -1063,6 +1135,7 @@ impl std::fmt::Display for Error {
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::UdfUnrecordedExtent { path } => write!(f, "E{}: {}", self.code(), path),
Error::SeamPlanDroppedMost { dropped, written } => {
write!(f, "E{} {dropped}/{written}", self.code())
}
@@ -1128,7 +1201,21 @@ impl std::error::Error for Error {
impl From<std::io::Error> for Error {
fn from(e: std::io::Error) -> Self {
Error::IoError { source: e }
// If this `io::Error` is one WE produced (`From<Error> for io::Error`
// carries the typed value in its boxed payload), give the original
// back instead of burying it in `Error::IoError`. That wrapper is not
// neutral: `is_scsi_transport_failure` treats `IoError` as a
// transport-layer fault (dead bus / wedged bridge), so re-wrapping a
// round-tripped `DiscRead` MEDIUM ERROR turned a skippable bad sector
// into a pass-aborting bridge wedge — the exact inverse of what that
// arm exists for. Any error that crosses a thread boundary as an
// `io::Error` (the prefetch channel's `Batch`) keeps its
// classification, its SCSI status, and its sense data.
match e.downcast::<Error>() {
Ok(typed) => typed,
// A genuine OS/`std` error — the `IoError` wrapper is correct here.
Err(io) => Error::IoError { source: io },
}
}
}
@@ -1143,7 +1230,6 @@ impl From<Error> for std::io::Error {
return source;
}
let code = e.code();
let msg = e.to_string();
// Map our error categories to io::ErrorKind
let kind = match code {
// Device access-denied semantics map to PermissionDenied;
@@ -1244,7 +1330,13 @@ impl From<Error> for std::io::Error {
E_DIR_IMAGE_FILE_CHANGED => std::io::ErrorKind::InvalidData,
_ => std::io::ErrorKind::Other,
};
std::io::Error::new(kind, msg)
// Carry the typed value itself as the payload rather than its
// stringification. `Display` is unchanged (`io::Error` delegates to the
// boxed error, whose `Display` is the same `E<code>[: …]` string), so
// `error_code` and every consumer built on it are unaffected — but the
// typed error now SURVIVES the conversion and `From<io::Error> for
// Error` can hand it back intact.
std::io::Error::new(kind, e)
}
}
@@ -1263,13 +1355,14 @@ pub type Result<T> = std::result::Result<T, Error>;
/// removing comes back.
///
/// [`From<Error> for io::Error`] is the ONLY path from a typed [`Error`] to an
/// `io::Error` in this crate, and it stringifies (`io::Error::new(kind, msg)`
/// where `msg` is the `Error`'s `E<code>[: …]` [`Display`](std::fmt::Display)
/// string) rather than boxing the typed value — no code path constructs an
/// `io::Error` that still holds a `crate::error::Error` via `get_ref`. So the
/// only recognised shape is the round-tripped `E<code>` message prefix.
/// `io::Error` in this crate. It boxes the typed value as the payload
/// (`io::Error::new(kind, e)`), whose [`Display`](std::fmt::Display) is the
/// same `E<code>[: …]` string the stringifying version produced — so this
/// parse is unaffected, and `From<io::Error> for Error` can additionally
/// `downcast` the payload back to the exact typed error. Errors that did NOT
/// come from this crate carry no `E<code>` prefix and yield `None`.
pub fn error_code(e: &std::io::Error) -> Option<u16> {
// Round-tripped: `From<Error> for io::Error` stringifies as "E<code>[: …]".
// Round-tripped: `From<Error> for io::Error` renders as "E<code>[: …]".
let s = e.to_string();
let digits = s.strip_prefix('E')?;
let end = digits
@@ -1398,6 +1491,22 @@ impl Error {
)
}
/// True if the read SOURCE itself is gone, as opposed to one range of
/// media being unreadable. Kept separate from
/// [`is_scsi_transport_failure`](Self::is_scsi_transport_failure) —
/// which is about the bus/bridge and drives "power-cycle the drive"
/// advice — because a terminated producer thread is neither a wedged
/// bridge nor a bad sector, and reporting it as SCSI status 0xFF would
/// be a fabricated status byte.
///
/// What it shares with a transport failure is the only thing the read
/// loops need to know: retrying smaller or skipping ahead cannot
/// recover anything, so the pass must abort rather than fabricate
/// zeros for the rest of the title.
pub fn is_source_terminated(&self) -> bool {
matches!(self, Error::SourceTerminated)
}
/// True if this error indicates bridge degradation — the SCSI status
/// is neither GOOD (0x00), CHECK CONDITION (0x02), nor transport failure
/// (0xFF). Observed on the Initio INIC-1618L USB bridge preceding a full
@@ -1804,126 +1913,129 @@ mod tests {
// ── New comprehensive tests ────────────────────────────────────────────────
/// Every `pub const E_*` code declared in this file, as
/// `(name, value)`, PARSED OUT OF THE SOURCE at compile time.
///
/// WHY THIS IS PARSED AND NOT LISTED. The uniqueness test below used to
/// carry a hand-maintained `vec![]` of the constants, with a doc comment
/// claiming it "pins all code assignments". It did not. At the time this
/// was written the file declared **127** `pub const E_*` and the vector
/// named **109** of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
/// `E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`,
/// `E_DIR_NAME_COLLISION`, `E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`,
/// `E_DIR_RAW_REJECTED`, `E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`,
/// `E_DRIVE_INQUIRY_SHORT`, `E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`,
/// `E_SEAM_PLAN_DROPPED_MOST`, `E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`,
/// `E_SOURCE_TERMINATED`, `E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were
/// outside the guarantee entirely, so a new variant colliding with any of
/// them passed green. Worse, an earlier audit READ that doc comment and
/// trusted it while assigning new codes.
///
/// The defect is not the eighteen omissions, it is that the list is
/// hand-maintained at all: adding a constant and forgetting the vector is
/// a silent no-op, which is the definition of a guarantee that decays.
/// Deriving the list from the declarations makes forgetting impossible —
/// the only way to escape the check is to stop declaring the constant.
///
/// `include_str!` of this very file is the cheapest seam that does that:
/// no `build.rs`, no proc macro, no dependency, and — being `#[cfg(test)]`
/// — not one byte of the source embedded in a release build.
fn declared_error_codes() -> Vec<(&'static str, u16)> {
const SRC: &str = include_str!("error.rs");
SRC.lines()
.filter_map(|line| {
// Only real declarations at column 0. A retired code is
// recorded as `// 2001: burned/retired`, which carries no `pub
// const` and is therefore correctly invisible here.
let decl = line.strip_prefix("pub const ")?;
let (name, value) = decl.split_once(": u16 = ")?;
if !name.starts_with("E_") {
return None;
}
let value = value.strip_suffix(';')?;
Some((
name,
value
.parse::<u16>()
.unwrap_or_else(|_| panic!("non-literal error code for `{name}`")),
))
})
.collect()
}
/// The parser above must actually FIND the declarations, and read their
/// values correctly. Without this, a `declared_error_codes` that returned
/// an empty vector would make the uniqueness test below pass vacuously —
/// exactly the failure mode (a guarantee that is really a no-op) this
/// whole change exists to remove.
///
/// Mutation: a `strip_prefix` typo, an off-by-one in the name slice, or a
/// parser that drops the last line fails here.
#[test]
fn declared_error_codes_parses_the_declarations_it_claims_to() {
let declared = declared_error_codes();
// Independent count of the declarations, computed a different way
// from the parser under test.
let expected = include_str!("error.rs")
.lines()
.filter(|l| l.starts_with("pub const E_"))
.count();
assert_eq!(
declared.len(),
expected,
"the parser must see every `pub const E_*` line"
);
assert!(
expected >= 120,
"sanity floor: this file declares well over a hundred codes, got {expected}"
);
// Names and values must match the compiled constants, so a parser that
// mis-slices the name or mis-reads the digits cannot pass.
for (name, value) in [
("E_DEVICE_NOT_FOUND", E_DEVICE_NOT_FOUND),
("E_UDF_UNRECORDED_EXTENT", E_UDF_UNRECORDED_EXTENT),
("E_KEYDB_PARSE", E_KEYDB_PARSE),
] {
assert!(
declared.contains(&(name, value)),
"`{name}` = {value} not parsed out of the source; got {:?}",
declared
.iter()
.filter(|(n, _)| *n == name)
.collect::<Vec<_>>()
);
}
// A comment-only retired code must NOT be picked up as a constant.
assert!(
!declared.iter().any(|(n, _)| n.is_empty()),
"no empty names"
);
}
/// Every published error code constant must be unique.
/// This pins all code assignments: a new variant that accidentally reuses
/// an existing code will make this test fail.
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating E_KEYDB_CONNECT) fails here.
///
/// The set is derived from the declarations by
/// [`declared_error_codes`], so — unlike the hand-kept vector this
/// replaced — a newly added constant is covered the moment it is
/// written, with no second edit to remember.
///
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating
/// E_KEYDB_CONNECT) fails here, and so now does the same collision on any
/// of the eighteen codes the old hand-kept list had never heard of.
#[test]
fn all_error_code_constants_are_unique() {
let mut codes = vec![
E_DEVICE_NOT_FOUND,
E_DEVICE_PERMISSION,
E_DEVICE_NOT_READY,
E_DEVICE_RESET_FAILED,
E_SCSI_INTERFACE_UNAVAILABLE,
E_DEVICE_LOCKED,
E_IOKIT_PLUGIN_FAILED,
E_UNSUPPORTED_DRIVE,
E_PROFILE_PARSE,
E_UNSUPPORTED_PLATFORM,
E_PLATFORM_NOT_IMPLEMENTED,
E_UNLOCK_FAILED,
E_SIGNATURE_MISMATCH,
E_SCSI_ERROR,
E_INVALID_CDB_LENGTH,
E_IO_ERROR,
E_DISC_READ,
E_MPLS_PARSE,
E_CLPI_PARSE,
E_UDF_NOT_FOUND,
E_DISC_TITLE_RANGE,
E_IFO_PARSE,
E_MKV_INVALID,
E_NO_STREAMS,
E_HALTED,
E_MAPFILE_INVALID,
E_IMAGE_TRUNCATED,
E_UDF_BUFFER_TOO_SMALL,
E_UDF_NOT_FILESYSTEM,
E_AACS_NO_KEYS,
E_AACS_CERT_SHORT,
E_AACS_AGID_ALLOC,
E_AACS_CERT_REJECTED,
E_AACS_CERT_READ,
E_AACS_CERT_VERIFY,
E_AACS_KEY_READ,
E_AACS_KEY_REJECTED,
E_AACS_KEY_VERIFY,
E_AACS_VID_READ,
E_AACS_VID_MAC,
E_AACS_DATA_KEY,
E_DECRYPT_FAILED,
E_CSS_AUTH_FAILED,
E_AACS_HOST_CERT_REJECTED,
E_AACS_RAW_READ_UNSUPPORTED,
E_AACS_VID_UNAVAILABLE,
E_AACS_MK_UNAVAILABLE,
E_AACS_VUK_NOT_IN_KEYDB,
E_DRIVE_PROFILE_MISSING,
E_VID_CDB_UNAVAILABLE,
E_NO_DISC_KEY,
E_CSS_KEY_MISSING,
E_CSS_NO_DISC_KEY,
E_KEY_SERVICE_UNAVAILABLE,
E_KEY_SERVICE_UNAUTHORIZED,
E_KEY_SERVICE_RATE_LIMITED,
E_AACS_NO_HOST_CERT,
E_AACS_BUS_KEY_UNAVAILABLE,
E_FMTS_KEY_MISSING,
E_KEYDB_CONNECT,
E_KEYDB_HTTP,
E_KEYDB_INVALID,
E_KEYDB_WRITE,
E_KEYDB_PARSE,
E_KEYDB_LOAD,
E_KEYDB_UNSUPPORTED_SCHEME,
E_KEYDB_TOO_MANY_REDIRECTS,
E_STREAM_READ_ONLY,
E_STREAM_WRITE_ONLY,
E_STREAM_URL_INVALID,
E_STREAM_URL_MISSING_PATH,
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MUX_HEADER_BUFFER_EXCEEDED,
E_MKV_LACING_INVALID,
E_MKV_SOURCE_INVALID,
E_MKV_UNENCODABLE,
E_MP4_NO_VIDEO_TRACK,
E_MP4_INVALID,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
E_ISO_TOO_LARGE,
E_NO_METADATA,
E_DISC_URL_NOT_DIRECT,
E_HEVC_PARAM_PARSE,
E_MUX_TRACK_RANGE,
E_FMP4_UNIMPLEMENTED,
E_DEMUX_THREAD_PANICKED,
E_PIPELINE_JOIN_TIMEOUT,
E_PIPELINE_CONSUMER_PANICKED,
E_SWEEP_CONSUMER_GONE,
E_PIPELINE_CONSUMER_GONE,
E_DISC_CAPACITY_OVERFLOW,
E_M2TS_PACKET_MALFORMED,
E_EXTENT_NOT_UNIT_ALIGNED,
E_DISC_CAPACITY_MALFORMED,
E_DIR_IMAGE_SSIF_UNSUPPORTED,
E_DIR_IMAGE_PLACEMENT,
E_DIR_IMAGE_ENCRYPTED,
E_DIR_IMAGE_UNSUPPORTED_TREE,
E_DIR_IMAGE_FILE_CHANGED,
E_DIR_IMAGE_TOO_LARGE,
];
let original_len = codes.len();
codes.sort();
codes.dedup();
assert_eq!(
codes.len(),
original_len,
"duplicate error code constants detected — check error.rs"
let mut by_code: std::collections::BTreeMap<u16, Vec<&str>> =
std::collections::BTreeMap::new();
for (name, value) in declared_error_codes() {
by_code.entry(value).or_default().push(name);
}
let dupes: Vec<_> = by_code
.iter()
.filter(|(_, names)| names.len() > 1)
.collect();
assert!(
dupes.is_empty(),
"duplicate error code constants detected — check error.rs: {dupes:?}"
);
}
+117 -44
View File
@@ -49,7 +49,7 @@ pub struct DvdTitle {
pub cells: Vec<DvdCell>,
/// Chapter start times in seconds (derived from program map + cell times)
pub chapter_times: Vec<f64>,
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cb, Cr].
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cr, Cb].
pub palette: Option<Vec<[u8; 4]>>,
}
@@ -425,6 +425,13 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
for (&vts_number, titles_info) in &title_set_map {
match parse_vts(reader, udf, vts_number, titles_info) {
Ok(ts) => title_sets.push(ts),
// The operator's Stop is not a placeholder title set. Once the
// drive's halt flag is set EVERY command fails with `Halted`, so
// swallowing it here would skip every REMAINING title set in turn
// and return `Ok` with a truncated list — a cancelled scan that
// is indistinguishable from a disc genuinely holding fewer
// titles. `scan_dvd_titles` propagates it to the caller.
Err(Error::Halted) => return Err(Error::Halted),
Err(e) => {
// Some discs carry placeholder TT_SRPT entries for title sets
// that are not really there, so one failure is not fatal. But
@@ -691,7 +698,16 @@ fn parse_video_attr(data: &[u8]) -> Result<DvdVideoAttr> {
}
/// Parse one audio stream attribute block (8 bytes at `offset`).
fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
/// `pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
/// `parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
/// calls it directly so its language-mapping tests run the real parser over real
/// on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
/// could agree with the muxer while both disagree with the disc. Narrowing this
/// would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
/// that can drift apart, or moving those tests away from the code they exist to
/// pin — both worse than the widened crate-internal visibility, which reaches no
/// public API.
pub(crate) fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let b0 = byte_at(data, offset)?;
let b1 = byte_at(data, offset + 1)?;
@@ -718,25 +734,9 @@ fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let channels = (b1 & 0x07) + 1; // (channels - 1) in low 3 bits of byte 1
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
{
String::from_utf8_lossy(lang_bytes).to_string()
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
// Try to interpret as printable ASCII
let s: String = lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdAudioAttr {
codec,
@@ -785,9 +785,27 @@ fn assign_audio_sub_stream_ids(streams: &mut [DvdAudioAttr]) {
/// Parse one subtitle stream attribute block (6 bytes at `offset`).
fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdSubtitleAttr { language })
}
/// Decode the raw 2-byte on-disc language code shared by the audio and
/// subtitle attribute blocks. A pair of lowercase a-z bytes is taken
/// verbatim (the ISO 639-1 code the DVD-Video spec puts there); an all-zero
/// pair means unspecified (empty string); anything else falls through to an
/// ASCII-alphanumeric salvage — letters (either case) and digits are kept,
/// everything else (control bytes, punctuation, high bytes from a corrupt or
/// hostile disc) is dropped.
///
/// The salvage is deliberately not narrowed to a-z: whatever survives is only
/// ever a lookup key for [`dvd_lang_to_iso639_2`], which degrades anything it
/// does not recognize to `und`, so a stray `X` or `5` costs nothing and cannot
/// reach an output stream as a language code.
fn parse_raw_dvd_lang_bytes(lang_bytes: &[u8]) -> String {
if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
@@ -796,15 +814,40 @@ fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
let s: String = lang_bytes
lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
.collect()
}
}
Ok(DvdSubtitleAttr { language })
/// Convert a DVD IFO audio/subtitle attribute's language code — ISO 639-1
/// (2 lowercase letters) per the DVD-Video spec, or empty when unspecified —
/// to the ISO 639-2 form every downstream consumer (`disc::AudioStream` /
/// `disc::SubtitleStream::language`, and in turn Matroska's `Language`
/// element per RFC 9559 §12 and the MP4 sink's `mdhd` language) requires.
///
/// Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
/// 639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
/// frozen-1988 language list still puts on disc). The narrower
/// `vocab::menu_lang` table is deliberately NOT used here: it exists for
/// Blu-ray menu-graphic filename tokens and knows only 25 languages, so a
/// Region-2 disc's Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian,
/// Hebrew, Estonian, Latvian, Lithuanian and Icelandic tracks would all fold
/// onto `und` together. DVD streams carry an empty `label`, so the language
/// is the only thing distinguishing one subtitle track from the next — a
/// valid code that is identical for six tracks is worse for the user than the
/// invalid one it replaced. Both tables normalize to ISO 639-2/T, so they
/// agree wherever they overlap.
///
/// An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's
/// own "undetermined" value) — a valid element value — rather than passing
/// through an invalid 2-letter code or an empty string. Never guesses.
fn dvd_lang_to_iso639_2(raw: &str) -> String {
crate::labels::vocab::iso639_1_to_iso639_2(raw)
.unwrap_or("und")
.to_string()
}
// ── PGC parser ──────────────────────────────────────────────────────────────
@@ -1016,7 +1059,7 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
times
};
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cb, Cr]
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cr, Cb]
let palette = if pgc_offset + 0xA4 + 64 <= data.len() {
let mut colors = Vec::with_capacity(16);
for i in 0..16 {
@@ -1441,7 +1484,7 @@ mod tests {
data[0] = 0x00;
// b1: bits 2-0=101 (channels-1=5) => 0x05
data[1] = 0x05;
// language "en"
// on-disc language "en" (ISO 639-1) -> parsed as ISO 639-2 "eng"
data[2] = b'e';
data[3] = b'n';
@@ -1449,7 +1492,7 @@ mod tests {
assert_eq!(attr.codec, Codec::Ac3);
assert_eq!(attr.sample_rate, 48000);
assert_eq!(attr.channels, 6);
assert_eq!(attr.language, "en");
assert_eq!(attr.language, "eng");
}
#[test]
@@ -1579,7 +1622,7 @@ mod tests {
assert_eq!(attr.codec, Codec::Dts);
assert_eq!(attr.sample_rate, 96000);
assert_eq!(attr.channels, 2);
assert_eq!(attr.language, "fr");
assert_eq!(attr.language, "fra");
}
// ─────────────────────────────────────────────────────────────────────
@@ -1693,15 +1736,17 @@ mod tests {
}
/// Audio language bytes [offset+2..+4]: when both bytes are 0x00 the
/// language is the empty string (unspecified), per source.
/// on-disc code is unspecified, and `dvd_lang_to_iso639_2` maps that to
/// the valid ISO 639-2 "und" (undetermined) rather than an empty string,
/// which is not a legal Matroska `Language` element value.
#[test]
fn audio_attr_zero_language_is_empty() {
fn audio_attr_zero_language_becomes_und() {
let mut data = vec![0u8; 8];
data[0] = 0x00;
data[2] = 0x00;
data[3] = 0x00;
let attr = parse_audio_attr(&data, 0).unwrap();
assert_eq!(attr.language, "");
assert_eq!(attr.language, "und");
}
/// Audio sample_rate flag (b0>>3 & 0x03): 0=48kHz, 1=96kHz, else 48kHz.
@@ -1722,11 +1767,11 @@ mod tests {
data[2] = b'd';
data[3] = b'e';
let attr = parse_subtitle_attr(&data, 0).unwrap();
assert_eq!(attr.language, "de");
assert_eq!(attr.language, "deu");
let zero = vec![0u8; 6];
let attr2 = parse_subtitle_attr(&zero, 0).unwrap();
assert_eq!(attr2.language, "");
assert_eq!(attr2.language, "und");
}
/// assign_audio_sub_stream_ids: MP1/MP2 and other non-private-stream-1
@@ -1812,8 +1857,8 @@ mod tests {
assert_eq!(title.cells[1].first_sector, 20);
}
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cb, Cr].
/// A palette with at least one non-zero Y/Cb/Cr is returned as Some;
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cr, Cb].
/// A palette with at least one non-zero Y/Cr/Cb is returned as Some;
/// an all-zero palette returns None (source filters empty palettes).
#[test]
fn pgc_palette_present_and_empty() {
@@ -1833,14 +1878,14 @@ mod tests {
assert!(title2.palette.is_none());
}
/// parse_pgc palette layout: each color is [padding, Y, Cb, Cr] and the
/// parse_pgc palette layout: each color is [padding, Y, Cr, Cb] and the
/// "non-empty" test ignores the padding byte (index 0). A palette whose
/// ONLY non-zero bytes are padding must still be treated as empty (None).
#[test]
fn pgc_palette_padding_only_is_empty() {
let mut pgc = vec![0u8; 0xEA];
pgc[0x03] = 0;
// Set padding byte (index 0) of color 0 non-zero, but Y/Cb/Cr zero.
// Set padding byte (index 0) of color 0 non-zero, but Y/Cr/Cb zero.
pgc[0xA4] = 0xFF;
let title = parse_pgc(&pgc, 0, 1).unwrap();
assert!(
@@ -2278,10 +2323,13 @@ mod tests {
/// two bytes at +2. Only a pair of lowercase a-z bytes is taken verbatim;
/// anything else falls through to the ASCII-alphanumeric salvage, which
/// keeps only the usable characters. A byte outside a-z must never end up
/// in the language string.
/// in the raw salvaged string. This exercises `parse_raw_dvd_lang_bytes`
/// directly — the byte-level salvage step — separately from the ISO
/// 639-1 -> 639-2 mapping `parse_audio_attr`/`parse_subtitle_attr` apply
/// on top (see `dvd_two_letter_and_malformed_language_becomes_iso639_2`).
#[test]
fn language_code_rejects_non_lowercase_bytes() {
// (byte0, byte1, expected language)
// (byte0, byte1, expected raw salvage)
let cases: [(u8, u8, &str); 8] = [
(b'e', b'n', "en"), // both in range → verbatim
(0x21, b'n', "n"), // '!' is below 'a'
@@ -2292,6 +2340,31 @@ mod tests {
(0x00, b'E', "E"), // only the first byte is zero
(0x00, 0x00, ""), // both zero → unset
];
for (b0, b1, want) in cases {
assert_eq!(
parse_raw_dvd_lang_bytes(&[b0, b1]),
want,
"raw language salvage for ({b0:#04x}, {b1:#04x})"
);
}
}
/// The full pipeline `parse_audio_attr`/`parse_subtitle_attr` apply on
/// top of the raw salvage: a valid ISO 639-1 code maps to its ISO 639-2
/// equivalent, and anything the raw salvage does NOT produce a mapped
/// code for (empty, or a single leftover letter from a malformed byte
/// pair) degrades to "und" — never an invalid 2-letter/1-letter code and
/// never an empty string, both of which are illegal Matroska `Language`
/// element values (RFC 9559 §12).
#[test]
fn dvd_two_letter_and_malformed_language_becomes_iso639_2() {
// (byte0, byte1, expected final language)
let cases: [(u8, u8, &str); 4] = [
(b'e', b'n', "eng"), // valid ISO 639-1 → mapped
(0x21, b'n', "und"), // malformed → salvage "n", unmapped → und
(b'E', 0x00, "und"), // malformed → salvage "E", unmapped → und
(0x00, 0x00, "und"), // unspecified → und
];
for (b0, b1, want) in cases {
let mut audio = vec![0u8; 8];
audio[2] = b0;
@@ -2496,7 +2569,7 @@ mod tests {
}
/// The subtitle palette is 16 entries of 4 bytes at PGC+0xA4, each
/// `[padding, Y, Cb, Cr]`. Every byte of every entry is distinct here, so
/// `[padding, Y, Cr, Cb]`. Every byte of every entry is distinct here, so
/// a wrong stride, a wrong base or a shifted component shows up.
#[test]
// The loop variable is the DOMAIN VALUE being checked (a palette entry number), not a
@@ -2524,7 +2597,7 @@ mod tests {
}
}
/// A palette is "present" when ANY of Y, Cb or Cr is non-zero in ANY
/// A palette is "present" when ANY of Y, Cr or Cb is non-zero in ANY
/// entry — a single non-zero chroma component is enough. Only the
/// padding byte [0] is ignored.
#[test]
+1 -3
View File
@@ -21,6 +21,7 @@
//!
//! So: drive sources get `freemkv_engine::copy`. Everything else gets this.
use crate::consts::SECTOR_BYTES;
use crate::error::{Error, Result};
use crate::halt::Halt;
use crate::sector::SectorSource;
@@ -33,9 +34,6 @@ use std::path::Path;
/// a notable allocation and cancellation stays responsive.
const BATCH_SECTORS: u32 = 2048;
/// Bytes per sector. Fixed for every medium this crate reads.
const SECTOR_BYTES: usize = 2048;
/// Write `total_sectors` sectors from `reader` to `dest`.
///
/// Reads sequentially from LBA 0 and writes in order, so the output is a faithful
+8 -8
View File
@@ -85,16 +85,16 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let enums = identify_master_enums(archive);
if enums.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: com/bydeluxe/ present but no master enum fingerprint matched"
);
return None;
}
for (label, m) in &enums {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
enum = %label,
class = %m.class_name,
class = ?m.class_name,
count = m.values.len(),
"deluxe master enum identified",
);
@@ -110,15 +110,15 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let binding_classes = find_binding_classes(archive, &master_table.class_name_set());
if binding_classes.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: no binding class found (no class has enough getstatic refs to master enums)"
);
return None;
}
for (name, count) in &binding_classes {
tracing::info!(
jar = %entry_name,
binding_class = %name,
jar = ?entry_name,
binding_class = ?name,
getstatic_count = count,
"deluxe binding class candidate",
);
@@ -138,7 +138,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
if streams.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: binding classes found but produced 0 decoded streams"
);
return None;
@@ -149,7 +149,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
audio = labels.iter().filter(|l| l.stream_type == StreamLabelType::Audio).count(),
subtitle = labels.iter().filter(|l| l.stream_type == StreamLabelType::Subtitle).count(),
"deluxe emitted labels",
+169 -6
View File
@@ -450,13 +450,22 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
}) else {
continue;
};
for clip in &title.clips {
// ONLY the anchor's first clip. `disc::bluray` builds a title's
// stream list from `play_items[0]`'s STN table, so that is the only
// clip in which these PIDs were ever observed — the same clip tier 3
// keys its derived ids on (`clip0`, below). Recording the fact
// against every clip the anchor plays claims knowledge of stream
// tables never read: a sibling playlist over a LATER clip then binds
// the anchor's editorial label onto whichever stream of that clip
// reuses the PID, which is a different physical stream (PIDs are
// unique only within a clip).
if let Some(clip) = title.clips.first() {
pid_map.insert((clip.clip_id.as_str(), *pid), pos);
}
}
tracing::info!(
stream_type = ?stream_type,
playlist = %titles[anchor].playlist,
playlist = ?titles[anchor].playlist,
slots = slots_of(&titles[anchor], stream_type).len(),
"label list anchored to a title by its stream-language sequence",
);
@@ -1418,11 +1427,20 @@ mod registry_tests {
/// straight from the disc's UDF directory records, with attacker-chosen
/// name lengths to inflate each comparison.
///
/// Proof is by deadline. With the linear scan this fixture measures well
/// past the deadline; with a set it is milliseconds. Bounded so a
/// regression fails fast instead of hanging CI.
/// This is a HANG GUARD, and the name says so: a return to the linear scan
/// makes this fixture run for minutes (120 000² / 2 comparisons over a
/// 180-byte shared prefix), which without the deadline would wedge CI
/// rather than fail it. It is not a complexity proof — no assertion here
/// can distinguish `BTreeSet` from any other sub-quadratic dedup, and the
/// clock-free half of the claim (dedup, sort, directory exclusion) belongs
/// to `jar_inventory_dedups_sorts_and_skips_dirs` below.
///
/// The deadline is a real margin, unlike the 6x one that made
/// `paramount.rs`'s wall-clock test flake under a loaded CI box: measured
/// at 0.14 s debug / 0.07 s release against 10 s, so ~70x. A shared CPU
/// does not close that; a quadratic dedup does not survive it.
#[test]
fn jar_inventory_dedup_is_not_quadratic() {
fn jar_inventory_dedup_does_not_hang_on_a_hostile_directory() {
const FILES: usize = 120_000;
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
@@ -1898,6 +1916,78 @@ mod apply_tests {
}
}
/// Sentinel embedded in the crafted playlist name below. The capture keeps
/// ONLY fields whose rendered form contains it, so the assertion cannot be
/// fooled by another test's log output.
const LOG_INJECTION_SENTINEL: &str = "FMKV-LOG-INJECTION-PROBE";
/// A playlist name is a raw UDF directory entry — disc-controlled bytes,
/// validated no further than a lossy UTF-8 decode. Logging it through
/// tracing's `%` (Display) sigil writes those bytes VERBATIM, so a crafted
/// `.mpls` filename carrying ANSI escapes or control characters forges
/// terminal output and log structure in any consumer rendering the event
/// (CWE-117). `?` (Debug) escapes them, and `str`'s Debug is exactly the
/// escaping this needs.
///
/// `info!` is not covered by the debug/trace-logging exemption: this fires
/// on an ordinary rip of an ordinary disc.
///
/// Mutation: put `%` back on `playlist` in `apply_labels` and this goes red.
#[test]
fn a_disc_derived_playlist_name_is_escaped_in_the_log_not_written_verbatim() {
// A name whose bytes would clear the line and repaint it.
let evil = format!("\u{1b}[2K\u{1b}[31m{LOG_INJECTION_SENTINEL}\u{7}\u{1b}[0m.mpls");
let labels = vec![
sub_label(1, "eng", LabelQualifier::None),
sub_label(2, "spa", LabelQualifier::None),
sub_label(3, "fra", LabelQualifier::None),
];
let mut titles = vec![title_on_clip(
&evil,
"00294",
vec![
subtitle(0x12A0, "eng"),
subtitle(0x12A1, "spa"),
subtitle(0x12A2, "fra"),
],
)];
// Capture through the crate's ONE global `tracing` subscriber:
// `testlog::capture` installs it exactly once and routes each event to a
// thread-local sink. A process-wide `set_global_default` HERE instead
// would poison every other test's callsite interest cache for the rest of
// the binary — `tracing` caches interest GLOBALLY — and only the first
// `set_global_default` in a process takes effect anyway. The shared
// subscriber answers interest for every callsite and isolates concurrent
// captures per thread, which is the invariant these log-accounting
// assertions rely on.
let ((), events) = crate::testlog::capture(|| {
apply_labels(&labels, &mut titles);
});
let playlist: Vec<&str> = events
.iter()
.filter(|e| e.target.starts_with("libfreemkv::labels"))
.filter_map(|e| e.field("playlist"))
.filter(|v| v.contains(LOG_INJECTION_SENTINEL))
.collect();
assert!(
!playlist.is_empty(),
"the anchoring event must actually have fired, or this test proves \
nothing; captured: {events:?}"
);
for rendered in playlist {
assert!(
!rendered.contains('\u{1b}') && !rendered.contains('\u{7}'),
"a disc-controlled playlist name reached the log with its raw \
control bytes intact: {rendered:?}"
);
assert!(
rendered.contains(LOG_INJECTION_SENTINEL),
"the name must still be legible once escaped: {rendered:?}"
);
}
}
fn sub_label(num: u16, lang: &str, qualifier: LabelQualifier) -> StreamLabel {
StreamLabel {
stream_id: None,
@@ -3094,6 +3184,79 @@ mod apply_tests {
);
}
/// A title that plays SEVERAL clips in order — the shape the anchor's
/// PID facts are harvested from.
fn title_on_clips(playlist: &str, clip_ids: &[&str], streams: Vec<Stream>) -> DiscTitle {
DiscTitle {
playlist: playlist.into(),
clips: clip_ids
.iter()
.map(|id| crate::disc::Clip {
feed_span: None,
clip_id: (*id).into(),
in_time: 0,
out_time: 0,
duration_secs: 3600.0,
source_packets: 0,
})
.collect(),
..title_with(streams)
}
}
/// Spec: the anchor proves a `(clip, PID)` fact only for the clip its
/// stream table was READ FROM — the first play item — never for every clip
/// the anchor happens to play.
///
/// `disc::bluray` builds a title's stream list from `play_items[0]`'s STN
/// table, and tier 3 fifty lines below says exactly that by keying its
/// derived ids on `clip0`. Tier 2's harvest contradicted it: it recorded
/// the anchor's slot PIDs against EVERY clip the anchor plays, so a sibling
/// playlist that plays a LATER clip of the anchor bound the anchor's
/// editorial label onto whatever stream in that clip happens to reuse the
/// PID — a different physical stream, in a different clip, whose own
/// language says so.
///
/// Mutation: harvest over `&title.clips` instead of its first clip — the
/// featurette wears the feature's SDH again.
#[test]
fn an_anchor_proves_pids_only_for_the_clip_its_table_came_from() {
let labels = vec![
sub_label(1, "eng", LabelQualifier::Sdh),
sub_label(2, "fra", LabelQualifier::None),
];
let mut titles = vec![
// The anchor: its stream table is clip 00082's (the first play
// item); it merely CONTINUES into 00090.
title_on_clips(
"00800.mpls",
&["00082", "00090"],
vec![subtitle(0x1200, "eng"), subtitle(0x1201, "fra")],
),
// A sibling playlist over the anchor's SECOND clip. Its subtitle
// reuses PID 0x1200 — PIDs are only unique within a clip — and it
// is Spanish, so nothing about the anchor's English SDH slot
// describes it.
title_on_clips("00451.mpls", &["00090"], vec![subtitle(0x1200, "spa")]),
];
apply_labels(&labels, &mut titles);
assert_eq!(
sub_state(&titles[0]),
vec![
(0x1200, false, LabelQualifier::Sdh),
(0x1201, false, LabelQualifier::None)
],
"the anchor itself keeps the qualifiers the list states for it"
);
assert_eq!(
sub_state(&titles[1]),
vec![(0x1200, false, LabelQualifier::None)],
"a PID in a clip the anchor's table never described is not that \
table's stream 1"
);
}
/// Spec: a vendor codec/variant claim does not follow the ordinal onto a
/// bonus clip that carries a different codec.
///
+194 -60
View File
@@ -98,6 +98,74 @@ enum ForcedSub {
ForcedNarrative,
}
/// The one number behind every cap in this parser: the highest CSV cell
/// position that can ever be addressed.
///
/// The labelling loops number cells 1-based into a `u16` and `break` at
/// `u16::try_from(i + 1)`, so cell `MAX_COM_INDICES` and everything past it is
/// never visited. Two different things are measured against that, and they are
/// not the same bound:
///
/// - **A VALUE at or beyond it cannot match any cell.** This is what caps the
/// set: values are filtered before insertion, so at most `MAX_COM_INDICES`
/// distinct entries can ever be stored, however long the attribute is. The
/// `HashSet` that replaced a linear scan fixed the LOOKUP cost; this is what
/// fixes the ALLOCATION, and a disc declaring half a billion indices no
/// longer costs half a billion entries.
/// - **A POSITION at or beyond it describes nothing new.** This caps the WORK,
/// not the memory — the value filter already made the set small, but without
/// it every one of those half a billion cells is still split and parsed. It
/// is an early exit at the first position whose contents provably cannot
/// matter, and it is why `forced_sub` — which holds no set at all, and so
/// gets no protection from the value rule — is bounded too.
///
/// Real authoring is nowhere near either limit: the BD STN table admits at
/// most 32 streams per playlist, so nothing legitimate is lost.
const MAX_COM_INDICES: usize = u16::MAX as usize;
/// Parse a `*_com1_idx` attribute into the set the labelling loops query.
///
/// Extracted so the BOUND is observable. Asserting it through
/// `labels_from_feature` is not possible: a `HashSet` collapses repeated
/// values, and an out-of-range index changes no label either way, so such a
/// test passes whether or not the cap exists — an assertion that cannot fail.
/// Returning the set lets a test hand in tens of thousands of DISTINCT
/// unaddressable indices and see them refused.
fn com_indices(attr: Option<String>) -> HashSet<usize> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.filter_map(|i| i.trim().parse().ok())
.filter(|&i| i < MAX_COM_INDICES)
.collect()
})
.unwrap_or_default()
}
/// Parse the `forced_sub` attribute into the cell list the subtitle loop
/// queries — the third attacker-controlled CSV in this file, and the last one
/// that was still unbounded.
///
/// Bounded by POSITION, and it has no other choice: a `*_com1_idx` list holds
/// values that can be filtered, and that filter is what caps its set, but a
/// `forced_sub` cell is a classification of the position it sits at, so there
/// is nothing to filter and nothing else would ever cap this. The Vec is read
/// only as `forced.get(i)` from a loop that stops at `MAX_COM_INDICES`, so
/// every cell past that is unreachable by construction.
///
/// Extracted, like [`com_indices`], so the bound is OBSERVABLE. Through
/// `labels_from_feature` it is not: the subtitle loop cannot reach those cells
/// either, so a label-level assertion passes whether or not the cap exists.
fn forced_subs(attr: Option<String>) -> Vec<ForcedSub> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.map(forced_sub_cell)
.collect()
})
.unwrap_or_default()
}
fn forced_sub_cell(cell: &str) -> ForcedSub {
match cell.trim() {
"1" => ForcedSub::ContainsForcedSegments,
@@ -123,9 +191,7 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// linearly once per stream, so `aud="..."` and `aud_com1_idx="..."`
// both grown large make this quadratic in the size of one XML file.
// Membership is the only operation performed on it.
let com_indices: HashSet<usize> = xml::attr(feature, "aud_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
let com_indices = com_indices(xml::attr(feature, "aud_com1_idx"));
// The CSV *is* the STN list: one cell per stream, in stream order,
// and `aud_com1_idx` is a 0-based index into those same cells. So
@@ -172,15 +238,11 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse subtitle streams
if let Some(sub) = xml::attr(feature, "sub") {
let forced: Vec<ForcedSub> = xml::attr(feature, "forced_sub")
.map(|s| s.split(',').map(forced_sub_cell).collect())
.unwrap_or_default();
let forced = forced_subs(xml::attr(feature, "forced_sub"));
// HashSet for the same reason as the audio side above: unbounded
// parsed input, membership-only use, linear scan once per stream.
let com_indices: HashSet<usize> = xml::attr(feature, "sub_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
let com_indices = com_indices(xml::attr(feature, "sub_com1_idx"));
// As with audio: the cell position IS the STN slot. `forced_sub` and
// `sub_com1_idx` are indexed against those same cells, so an empty
@@ -315,61 +377,133 @@ mod tests {
);
}
/// `sub_com1_idx` is an unbounded index list parsed straight out of the
/// disc's `playlists.xml` and was membership-tested with a linear
/// `Vec::contains` once per subtitle stream — quadratic in the size of a
/// single attacker-supplied file.
/// `sub_com1_idx` is parsed straight out of the disc's `playlists.xml`,
/// which is attacker-controlled and has no length bound of its own.
///
/// Proof is by deadline rather than micro-benchmark. With the linear scan
/// the original fixture (200 000 streams x 1 000 001 indices) measured
/// 31 s in a release build and far longer in debug; with a set it measured
/// 0.03 s release / 0.56 s debug. A 10 s deadline sits ~18x above the
/// slowest passing measurement and ~3x below the fastest failing one, and
/// makes a regression fail fast instead of hanging CI.
/// This replaces a WALL-CLOCK test. That one built a 200 000 x 1 000 001
/// fixture and failed if it took over 10 s, to prove the membership test
/// was a set rather than a linear scan. Measured on the machine that
/// wrote this: 1.62 s alone, and OVER 10 s — a real failure — when the
/// suite's other 3 347 tests were running concurrently. A 6x margin
/// against a shared CPU is not a margin; it is a CI failure that looks
/// like a flake and gets re-run until it passes.
///
/// The CSV now stops at the end of the 1-based `u16` stream-numbering
/// space, so only the first 65 535 cells are scanned. `INDICES` is raised
/// to keep the linear-scan work product (`cells x indices`) at or above
/// the original fixture's, preserving that deadline margin.
/// It also measured the wrong thing. Making the lookup O(1) bounded the
/// QUERY, not the PARSE: the set was still built from every entry the
/// disc declared, so a hostile playlist could still force an unbounded
/// allocation before any lookup happened. `MAX_COM_INDICES` bounds that.
///
/// Correctness is pinned on fixture-derived literals: indices 0, 2 and 4
/// are the commentary tracks, 1 and 3 are not.
/// What THIS test guards is that bounding did not change what a
/// legitimate playlist MEANS: it goes red if the bound is set too LOW
/// (verified at 2 — the real indices `0,2,4` stop resolving and the
/// purposes change). It does NOT go red if the bound is deleted
/// entirely, because the out-of-range filler is unobservable at the
/// label level and a `HashSet` collapses the repeats. Enforcement is
/// proven separately, by
/// `distinct_unaddressable_indices_are_refused_not_stored`, which reads
/// the set itself. Two tests, two properties; neither pretends to the
/// other's job.
#[test]
fn commentary_index_lookup_is_not_quadratic() {
/// Cells offered. Everything past `u16::MAX` is unnumberable and the
/// parser stops there, so the scanned prefix is 65 535 cells.
const STREAMS: usize = 200_000;
const SCANNED: usize = u16::MAX as usize;
const INDICES: usize = 3_100_000;
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(STREAMS));
feature.pop();
// Three real commentary indices, then a long run of one
// out-of-range value: nothing here is bounded by the stream count.
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(INDICES));
feature.pop();
feature.push_str(r#"" />"#);
let _ = tx.send(labels_from_feature(&feature));
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(labels) => {
worker.join().expect("worker panicked");
assert_eq!(labels.len(), SCANNED);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
Err(_) => panic!(
"labels_from_feature did not finish {STREAMS} streams x \
{INDICES} commentary indices within 10s — the membership \
test is still a linear scan"
),
}
fn bounding_the_parse_does_not_change_a_legitimate_playlist() {
// Three real indices, then far more entries than can address a cell.
const OVERSIZED: usize = MAX_COM_INDICES + 10_000;
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(8));
feature.pop();
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(OVERSIZED));
feature.pop();
feature.push_str(r#"" />"#);
let labels = labels_from_feature(&feature);
// The fixture's real indices still decide the purposes: bounding the
// parse must not change what a legitimate playlist means.
assert_eq!(labels.len(), 8);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
/// The set REFUSES unaddressable indices, so a hostile playlist cannot
/// inflate it. DISTINCT values on purpose: a `HashSet` collapses repeats,
/// so a million copies of one index costs one entry and would prove
/// nothing. Fifty thousand distinct out-of-range indices cost fifty
/// thousand entries without the filter, and none with it — so this test
/// goes red if the bound is removed, which the label-level assertions
/// below cannot do.
#[test]
fn distinct_unaddressable_indices_are_refused_not_stored() {
let hostile: String = (MAX_COM_INDICES..MAX_COM_INDICES + 50_000)
.map(|i| i.to_string())
.collect::<Vec<_>>()
.join(",");
let set = com_indices(Some(hostile));
assert!(
set.is_empty(),
"kept {} unaddressable indices — the parse is still unbounded",
set.len()
);
// The addressable ones are still kept.
assert_eq!(com_indices(Some("0,2,4".to_string())).len(), 3);
}
/// `forced_sub` is bounded too — the third CSV in the same function, and
/// the one that had no value filter to hide behind.
///
/// Read through `forced_subs` rather than through the labels for the same
/// reason the two tests above read the set: the subtitle loop stops at
/// `MAX_COM_INDICES`, so a label-level assertion cannot tell a bounded
/// parse from an unbounded one.
#[test]
fn forced_sub_cells_past_the_last_addressable_one_are_not_parsed() {
let hostile = "0,".repeat(MAX_COM_INDICES + 50_000);
let cells = forced_subs(Some(hostile));
assert_eq!(
cells.len(),
MAX_COM_INDICES,
"parsed {} cells — the forced_sub parse is still unbounded",
cells.len()
);
}
/// Bounding it must not change what a legitimate playlist means: the
/// cells that CAN address a stream still classify exactly as before.
#[test]
fn bounding_forced_sub_leaves_the_addressable_cells_alone() {
let cells = forced_subs(Some("0,1,3,2".to_string()));
assert_eq!(
cells,
vec![
ForcedSub::None,
ForcedSub::ContainsForcedSegments,
ForcedSub::ForcedNarrative,
ForcedSub::ForcedNarrative,
]
);
}
/// An index that cannot address any cell is dropped rather than STORED.
///
/// Asserted through `com_indices`, not through the labels: the labelling
/// loop never queries a cell position that high, so at the label level
/// retaining the value is unobservable and the assertion could not fail.
/// Reading the set is what makes the claim checkable.
#[test]
fn an_index_that_cannot_address_any_cell_is_not_retained() {
let set = com_indices(Some(format!(
"1,{},{}",
MAX_COM_INDICES,
MAX_COM_INDICES + 1
)));
assert_eq!(
set.len(),
1,
"only the addressable index belongs in the set, got {set:?}"
);
assert!(set.contains(&1));
}
/// Headroom: the BD STN_table admits at most 32 PG streams per playlist,
+326
View File
@@ -219,6 +219,237 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
Some(code)
}
// ── ISO 639-1 → ISO 639-2 ────────────────────────────────────────────────────
/// The complete ISO 639-1 set, paired with its ISO 639-2/**T** (terminological)
/// code. Every two-letter code ISO 639-1 defines appears exactly once.
///
/// /T is the variant the rest of this crate uses — [`lang`] and [`menu_lang`]
/// both normalize to it (`deu` not `ger`, `fra` not `fre`, `zho` not `chi`,
/// `ces`, `nld`, `ell`, `ron`, `slk`, `isl`, `eus`, `hrv`) — so the three
/// tables cannot disagree. `iso639_1_agrees_with_menu_lang` pins that.
///
/// For the 165 codes where 639-2/B and /T are identical this distinction does
/// not arise; it only matters for the 20-odd languages with a distinct
/// bibliographic code.
const ISO_639_1_TO_2: &[(&str, &str)] = &[
("aa", "aar"),
("ab", "abk"),
("ae", "ave"),
("af", "afr"),
("ak", "aka"),
("am", "amh"),
("an", "arg"),
("ar", "ara"),
("as", "asm"),
("av", "ava"),
("ay", "aym"),
("az", "aze"),
("ba", "bak"),
("be", "bel"),
("bg", "bul"),
("bh", "bih"),
("bi", "bis"),
("bm", "bam"),
("bn", "ben"),
("bo", "bod"),
("br", "bre"),
("bs", "bos"),
("ca", "cat"),
("ce", "che"),
("ch", "cha"),
("co", "cos"),
("cr", "cre"),
("cs", "ces"),
("cu", "chu"),
("cv", "chv"),
("cy", "cym"),
("da", "dan"),
("de", "deu"),
("dv", "div"),
("dz", "dzo"),
("ee", "ewe"),
("el", "ell"),
("en", "eng"),
("eo", "epo"),
("es", "spa"),
("et", "est"),
("eu", "eus"),
("fa", "fas"),
("ff", "ful"),
("fi", "fin"),
("fj", "fij"),
("fo", "fao"),
("fr", "fra"),
("fy", "fry"),
("ga", "gle"),
("gd", "gla"),
("gl", "glg"),
("gn", "grn"),
("gu", "guj"),
("gv", "glv"),
("ha", "hau"),
("he", "heb"),
("hi", "hin"),
("ho", "hmo"),
("hr", "hrv"),
("ht", "hat"),
("hu", "hun"),
("hy", "hye"),
("hz", "her"),
("ia", "ina"),
("id", "ind"),
("ie", "ile"),
("ig", "ibo"),
("ii", "iii"),
("ik", "ipk"),
("io", "ido"),
("is", "isl"),
("it", "ita"),
("iu", "iku"),
("ja", "jpn"),
("jv", "jav"),
("ka", "kat"),
("kg", "kon"),
("ki", "kik"),
("kj", "kua"),
("kk", "kaz"),
("kl", "kal"),
("km", "khm"),
("kn", "kan"),
("ko", "kor"),
("kr", "kau"),
("ks", "kas"),
("ku", "kur"),
("kv", "kom"),
("kw", "cor"),
("ky", "kir"),
("la", "lat"),
("lb", "ltz"),
("lg", "lug"),
("li", "lim"),
("ln", "lin"),
("lo", "lao"),
("lt", "lit"),
("lu", "lub"),
("lv", "lav"),
("mg", "mlg"),
("mh", "mah"),
("mi", "mri"),
("mk", "mkd"),
("ml", "mal"),
("mn", "mon"),
("mr", "mar"),
("ms", "msa"),
("mt", "mlt"),
("my", "mya"),
("na", "nau"),
("nb", "nob"),
("nd", "nde"),
("ne", "nep"),
("ng", "ndo"),
("nl", "nld"),
("nn", "nno"),
("no", "nor"),
("nr", "nbl"),
("nv", "nav"),
("ny", "nya"),
("oc", "oci"),
("oj", "oji"),
("om", "orm"),
("or", "ori"),
("os", "oss"),
("pa", "pan"),
("pi", "pli"),
("pl", "pol"),
("ps", "pus"),
("pt", "por"),
("qu", "que"),
("rm", "roh"),
("rn", "run"),
("ro", "ron"),
("ru", "rus"),
("rw", "kin"),
("sa", "san"),
("sc", "srd"),
("sd", "snd"),
("se", "sme"),
("sg", "sag"),
("si", "sin"),
("sk", "slk"),
("sl", "slv"),
("sm", "smo"),
("sn", "sna"),
("so", "som"),
("sq", "sqi"),
("sr", "srp"),
("ss", "ssw"),
("st", "sot"),
("su", "sun"),
("sv", "swe"),
("sw", "swa"),
("ta", "tam"),
("te", "tel"),
("tg", "tgk"),
("th", "tha"),
("ti", "tir"),
("tk", "tuk"),
("tl", "tgl"),
("tn", "tsn"),
("to", "ton"),
("tr", "tur"),
("ts", "tso"),
("tt", "tat"),
("tw", "twi"),
("ty", "tah"),
("ug", "uig"),
("uk", "ukr"),
("ur", "urd"),
("uz", "uzb"),
("ve", "ven"),
("vi", "vie"),
("vo", "vol"),
("wa", "wln"),
("wo", "wol"),
("xh", "xho"),
("yi", "yid"),
("yo", "yor"),
("za", "zha"),
("zh", "zho"),
("zu", "zul"),
];
/// The three two-letter codes ISO 639-1 has since withdrawn, mapped to their
/// replacements. DVD-Video froze its language list on the 1988 edition, so
/// discs authored to the spec carry these spellings and no other table sees
/// them: `iw` Hebrew (now `he`), `in` Indonesian (now `id`), `ji` Yiddish
/// (now `yi`).
const ISO_639_1_DEPRECATED: &[(&str, &str)] = &[("iw", "he"), ("in", "id"), ("ji", "yi")];
/// Map an ISO 639-1 two-letter language code to its ISO 639-2/T three-letter
/// code, accepting the withdrawn DVD-era spellings (`iw`, `in`, `ji`) as
/// aliases for their replacements.
///
/// Covers the WHOLE of ISO 639-1, unlike [`menu_lang`], whose table only spans
/// the languages that show up in Blu-ray menu-graphic filenames. Callers that
/// convert a spec field — a DVD IFO attribute block, say — need the whole set:
/// narrowing it to the menu vocabulary would fold every other language onto
/// one value and make a disc's tracks indistinguishable from each other.
///
/// Case-insensitive and trimmed. Returns `None` for anything that is not an
/// ISO 639-1 code, so callers decide the fallback rather than getting a guess.
pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
let c = code.trim().to_ascii_lowercase();
let c = ISO_639_1_DEPRECATED
.iter()
.find(|(old, _)| *old == c)
.map_or(c.as_str(), |(_, new)| new);
ISO_639_1_TO_2
.iter()
.find(|(two, _)| *two == c)
.map(|(_, three)| *three)
}
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
@@ -789,4 +1020,99 @@ mod tests {
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
/// Structural invariants of `ISO_639_1_TO_2`: it must hold the complete
/// ISO 639-1 set (184 codes), every key a distinct pair of lowercase
/// letters and every value three lowercase letters. A typo'd or duplicated
/// row fails here rather than silently mislabelling a track.
#[test]
fn iso639_1_table_is_complete_and_well_formed() {
assert_eq!(
ISO_639_1_TO_2.len(),
184,
"ISO 639-1 defines 184 two-letter codes; the table must hold all \
of them"
);
let mut keys: Vec<&str> = ISO_639_1_TO_2.iter().map(|(two, _)| *two).collect();
keys.sort_unstable();
let unique = keys.len();
keys.dedup();
assert_eq!(unique, keys.len(), "no ISO 639-1 code may appear twice");
for (two, three) in ISO_639_1_TO_2 {
assert!(
two.len() == 2 && two.bytes().all(|b| b.is_ascii_lowercase()),
"{two:?} is not a two-letter lowercase ISO 639-1 code"
);
assert!(
three.len() == 3 && three.bytes().all(|b| b.is_ascii_lowercase()),
"{three:?} is not a three-letter lowercase ISO 639-2 code"
);
}
// The withdrawn DVD-era spellings resolve, and are not themselves
// rows in the main table (they are aliases, not codes).
for (old, new) in ISO_639_1_DEPRECATED {
assert!(
!ISO_639_1_TO_2.iter().any(|(two, _)| two == old),
"withdrawn code {old:?} must not be a table row"
);
assert_eq!(
iso639_1_to_iso639_2(old),
iso639_1_to_iso639_2(new),
"withdrawn code {old:?} must resolve exactly as {new:?}"
);
}
}
/// The two tables must not disagree. Every two-letter token `menu_lang`
/// accepts has to yield the same ISO 639-2/T code through
/// `iso639_1_to_iso639_2`, so a DVD-sourced language and a Blu-ray
/// menu-label language for the same tongue never produce different
/// `Language` elements.
#[test]
fn iso639_1_agrees_with_menu_lang() {
for (two, three) in ISO_639_1_TO_2 {
if let Some(via_menu) = menu_lang(two) {
assert_eq!(
via_menu, *three,
"menu_lang({two:?}) = {via_menu:?} disagrees with the ISO \
639-1 table's {three:?}"
);
}
}
// Spot-check the /T choice itself, on the languages where /B differs.
for (two, t_code) in [
("de", "deu"),
("fr", "fra"),
("zh", "zho"),
("cs", "ces"),
("nl", "nld"),
("el", "ell"),
("ro", "ron"),
("sk", "slk"),
("is", "isl"),
("hy", "hye"),
("ka", "kat"),
("fa", "fas"),
] {
assert_eq!(
iso639_1_to_iso639_2(two),
Some(t_code),
"the crate standardises on ISO 639-2/T, so {two:?} is \
{t_code:?} and never the bibliographic form"
);
}
}
/// Trimming, case-insensitivity, and the no-guess contract.
#[test]
fn iso639_1_normalizes_input_and_never_guesses() {
assert_eq!(iso639_1_to_iso639_2("RO"), Some("ron"));
assert_eq!(iso639_1_to_iso639_2(" Ro "), Some("ron"));
assert_eq!(iso639_1_to_iso639_2("IW"), Some("heb"));
assert_eq!(iso639_1_to_iso639_2("zz"), None);
assert_eq!(iso639_1_to_iso639_2(""), None);
assert_eq!(iso639_1_to_iso639_2("e"), None);
// A three-letter code is not ISO 639-1 input — that is menu_lang's job.
assert_eq!(iso639_1_to_iso639_2("eng"), None);
}
}
+2
View File
@@ -129,6 +129,8 @@ pub mod progress;
pub mod scsi;
pub mod sector;
pub mod session;
#[cfg(test)]
pub(crate) mod testlog;
pub(crate) mod udf;
pub(crate) mod unlock_bridge;
+325 -7
View File
@@ -91,6 +91,18 @@ pub struct Ac3Parser {
/// across the PES boundary because it may be the core of an AC-3-core +
/// E-AC-3-dependent frame set whose remaining substreams are in the next PES.
saw_extension: bool,
/// The access unit held open across the last PES boundary, ALREADY
/// scanned. The carry-over begins at its first byte, so without this the
/// next call re-scans and re-CRCs every syncframe of it from byte 0 — and
/// an access unit that keeps gaining substreams grows to [`MAX_AC3_BUF`]
/// (1 MiB) before the resync guard drops it, which on a ~2 KiB DVD PES is
/// three orders of magnitude of repeated work per packet.
held: Option<HeldAu>,
/// Test-only: syncframes examined (sized + CRC-gated) by
/// `scan_access_units`. Pins the resume above — the property it exists for
/// is a WORK bound, which no frame-level assertion can observe.
#[cfg(test)]
frames_scanned: u64,
}
impl Default for Ac3Parser {
@@ -107,6 +119,9 @@ impl Ac3Parser {
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
saw_extension: false,
held: None,
#[cfg(test)]
frames_scanned: 0,
}
}
@@ -149,20 +164,56 @@ impl Ac3Parser {
anchor: Option<PtsAnchor>,
at_eos: bool,
marks: &[(usize, super::pesbuf::PesFacts)],
) -> (Vec<Frame>, usize, i64) {
held: Option<HeldAu>,
) -> ScanOut {
let mut frames = Vec::new();
let mut pos = 0usize;
// Running PTS for the next access unit to emit in this call.
let mut frame_pts_ns = base_pts_ns;
let mut anchor = anchor;
let mut pending: Option<PendingAu> = None;
// How far this call has proved there is no further syncframe to
// process; carried over so the held access unit's own bytes (and the
// junk after them) are not searched again next call.
let mut scanned_to = 0usize;
// Resume a held access unit instead of re-deriving it. `keep_from` was
// its first byte, so it starts at 0 of this buffer, and every frame in
// it was sized and CRC-gated on the call that built it.
if let Some(h) = held {
let mut drop_reason = h.drop_reason;
// The one verdict that can have changed since: the track may have
// become poisoned while this access unit was held, and a re-scan
// would have picked that up.
if drop_reason.is_none() && self.tally.is_poisoned() {
drop_reason = Some("track-poisoned");
}
pending = Some(PendingAu {
start: 0,
end: h.end,
pts_ns: base_pts_ns,
duration_ns: h.duration_ns,
drop_reason,
bsid: h.bsid,
});
frame_pts_ns = base_pts_ns + h.duration_ns as i64;
pos = h.scanned_to;
scanned_to = h.scanned_to;
}
while pos < data.len() {
let sync = find_ac3_sync(&data[pos..]);
let start = match sync {
Some(offset) => pos + offset,
None => break,
None => {
// No syncword in `data[pos..]` at all: every byte but the
// last is proved sync-free (a syncword is two bytes and the
// second may still arrive).
scanned_to = data.len().saturating_sub(1).max(pos);
break;
}
};
scanned_to = start;
let remaining = &data[start..];
@@ -182,6 +233,7 @@ impl Ac3Parser {
// Invalid/sub-header frame size (e.g. an E-AC-3 frmsiz of 0/1
// sizing to a 2/4-byte fragment) — skip this sync word.
pos = start + 2;
scanned_to = pos;
continue;
}
@@ -191,6 +243,10 @@ impl Ac3Parser {
}
let frame = &data[start..start + frame_size];
#[cfg(test)]
{
self.frames_scanned += 1;
}
// Decodability gate: a syncframe with an out-of-range bsid (> 16) or
// a failed native CRC (payload corruption) poisons the access unit it
// belongs to — a dependent substream is useless without its parent and
@@ -258,6 +314,7 @@ impl Ac3Parser {
}
pos = start + frame_size;
scanned_to = pos;
}
// Close or HOLD the trailing access unit. The rest of its frame set — its
@@ -271,10 +328,22 @@ impl Ac3Parser {
// substream that extends an access unit — a plain AC-3 track keeps
// emitting every frame in-call.
let mut hold_from = None;
let mut held_out = None;
if let Some(au) = pending {
if !at_eos && (au.bsid >= 11 || self.saw_extension) {
frame_pts_ns = au.pts_ns;
hold_from = Some(au.start);
// Everything below `scanned_to` has been searched already, and
// the access unit's own frames have been sized and CRC-gated;
// record both, rebased onto the carry-over (which starts at
// `au.start`), so the next call resumes instead of redoing it.
held_out = Some(HeldAu {
end: au.end - au.start,
scanned_to: scanned_to.max(au.end) - au.start,
duration_ns: au.duration_ns,
drop_reason: au.drop_reason,
bsid: au.bsid,
});
} else {
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
@@ -306,7 +375,12 @@ impl Ac3Parser {
None => data.len(),
};
(frames, keep_from, frame_pts_ns)
ScanOut {
frames,
keep_from,
frame_pts_ns,
held: held_out,
}
}
}
@@ -319,6 +393,36 @@ struct PtsAnchor {
pts_ns: i64,
}
/// What one `scan_access_units` pass produced: the access units it emitted,
/// the offset in the scanned buffer from which bytes must be carried over to
/// the next call, the PTS to stamp on the access unit that begins that
/// carry-over, and — when the trailing access unit was HELD — the state that
/// lets the next call resume rather than re-derive it.
struct ScanOut {
frames: Vec<Frame>,
keep_from: usize,
frame_pts_ns: i64,
held: Option<HeldAu>,
}
/// A trailing access unit held across the PES boundary, already scanned.
/// Offsets are relative to the carry-over, which begins at the access unit's
/// first byte — so the access unit occupies `0..end`.
#[derive(Clone, Copy)]
struct HeldAu {
/// End of the access unit's bytes.
end: usize,
/// How far the scan that built it had searched (`>= end`). Bytes below it
/// hold no further syncframe to process.
scanned_to: usize,
/// Duration contributed by the access unit's `substreamid`-0 substream.
duration_ns: u64,
/// Decodability verdict reached for it so far.
drop_reason: Option<&'static str>,
/// bsid of the substream that opened it.
bsid: u8,
}
/// An access unit (frame set) under construction: `data[start..end]` is the
/// `substreamid`-0 independent substream frame plus every substream appended to it
/// so far — its dependents, and any additional independent substreams 1..7 with
@@ -477,6 +581,8 @@ impl CodecParser for Ac3Parser {
// non-empty PES today; this is defensive for any future caller).
if pes.discontinuity {
self.acc.clear();
// The held access unit's bytes went with it.
self.held = None;
}
if pes.data.is_empty() {
return Vec::new();
@@ -516,8 +622,13 @@ impl CodecParser for Ac3Parser {
buf.extend_from_slice(self.acc.as_slice());
let marks = self.acc.marks_snapshot();
let data = &buf;
let (frames, keep_from, frame_pts_ns) =
self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks);
let held = self.held.take();
let ScanOut {
frames,
keep_from,
frame_pts_ns,
held: still_held,
} = self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks, held);
if keep_from < data.len() {
let tail = &data[keep_from..];
@@ -531,6 +642,7 @@ impl CodecParser for Ac3Parser {
MAX_AC3_BUF
);
self.acc.clear();
self.held = None;
// Advance the cadence, as both sibling branches below do, so the
// three paths out of this block cannot disagree. Defensive: no
// input reaching this parser was found that both parses frames and
@@ -539,6 +651,7 @@ impl CodecParser for Ac3Parser {
self.flush_pts_ns = frame_pts_ns;
} else {
self.acc.drain(keep_from);
self.held = still_held;
// The carried bytes, when later completed and emitted (next call
// or by flush() at EOS), are timed at the PTS the scanner reached
// here: the PTS of the next access unit in presentation order, or
@@ -548,6 +661,7 @@ impl CodecParser for Ac3Parser {
}
} else {
self.acc.clear();
self.held = None;
// Nothing carried, but keep the cadence so a following PES with no
// PTS (no anchor) continues the timeline instead of reusing a stale
// value.
@@ -569,9 +683,10 @@ impl CodecParser for Ac3Parser {
let buf = self.acc.as_slice().to_vec();
let marks = self.acc.marks_snapshot();
self.acc.clear();
let held = self.held.take();
let out = self
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks)
.0;
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks, held)
.frames;
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
@@ -2199,6 +2314,125 @@ mod tests {
);
}
/// A 256-byte E-AC-3 syncframe with a valid CRC. `strmtyp`/`substreamid`
/// go into byte 2 (A/52 Annex E), which is what `substream_role` reads:
/// (0, 0) OPENS an access unit, (1, 0) is a dependent substream that
/// EXTENDS the open one.
fn eac3_substream_frame(strmtyp: u8, substreamid: u8) -> Vec<u8> {
const SIZE: usize = 256;
let frmsiz = SIZE / 2 - 1; // (frmsiz + 1) * 2 == SIZE
let mut f = vec![0u8; SIZE];
f[0] = 0x0B;
f[1] = 0x77;
f[2] = (strmtyp << 6) | (substreamid << 3) | ((frmsiz >> 8) as u8 & 0x07);
f[3] = (frmsiz & 0xFF) as u8;
f[5] = 16 << 3; // bsid 16 → E-AC-3
finalize_ac3_crc(&mut f);
f
}
/// An access unit closes only at the next `substreamid`-0 independent
/// substream, so one that keeps gaining dependent substreams stays OPEN
/// across PES boundaries and its bytes stay in the carry-over. The
/// carry-over must not be re-scanned — and re-CRCed — from the access
/// unit's first byte on every packet: the buffer only stops growing at
/// MAX_AC3_BUF (1 MiB), and a PES on a DVD is about 2 KiB, so re-deriving
/// the held access unit costs work quadratic in the packets fed.
///
/// Measured directly, because a work bound has no frame-level shadow:
/// `frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
/// Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
///
/// Mutation: pass `None` for `held` in `parse` (or drop the `if let
/// Some(h) = held` resume) — the count returns to the quadratic figure.
#[test]
fn a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet() {
const DEPENDENTS: usize = 200;
let mut parser = Ac3Parser::new();
// Opens the access unit.
let emitted = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
emitted.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..DEPENDENTS {
let f = parser.parse(&make_eac3_pes(eac3_substream_frame(1, 0)));
assert!(f.is_empty(), "a dependent substream extends the open unit");
}
let fed = (DEPENDENTS + 1) as u64;
assert!(
parser.frames_scanned <= 2 * fed,
"the scanner examined {} syncframes for {fed} fed — a held access \
unit must be resumed, not re-derived",
parser.frames_scanned
);
// ...and the resume must not have cost correctness: the whole frame
// set is still one access unit, emitted intact at EOS.
let out = parser.flush();
assert_eq!(out.len(), 1, "the frame set is a single access unit");
assert_eq!(
out[0].data.len(),
256 * (DEPENDENTS + 1),
"every substream of the frame set belongs to it"
);
}
/// A concealed gap must drop the HELD access unit, not just the byte
/// buffer.
///
/// `parse` clears `self.acc` on a discontinuity because the buffered bytes
/// are a truncated frame. The held access unit is described by OFFSETS into
/// exactly those bytes, so it has to go with them. Without
/// `self.held = None`, the next packet resumes a HeldAu whose `start`/`end`
/// were computed against the pre-gap buffer but are applied to the
/// unrelated post-gap bytes — splicing audio across the gap at best, and
/// indexing past the end of the new, shorter buffer at worst.
///
/// The two existing discontinuity tests use plain AC-3 (bsid < 11), which
/// never holds an access unit open, so neither of them reaches this reset.
#[test]
fn a_discontinuity_drops_the_held_access_unit_with_its_bytes() {
let mut parser = Ac3Parser::new();
// Open an access unit and extend it, so a HeldAu exists describing
// offsets into a large buffer.
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(0, 0)))
.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..8 {
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(1, 0)))
.is_empty(),
"a dependent substream extends the open unit"
);
}
// The gap. Its post-gap payload is deliberately far SHORTER than the
// held unit's bytes, so a stale HeldAu indexes past its end.
let mut gap = make_eac3_pes(eac3_substream_frame(0, 0));
gap.discontinuity = true;
let _ = parser.parse(&gap);
// Whatever comes out, nothing may carry pre-gap bytes: the truncated
// unit was dropped, so the only access unit that can be emitted is the
// one opened after the gap.
let out = parser.flush();
let total: usize = out.iter().map(|f| f.data.len()).sum();
assert!(
total <= 256,
"a post-gap access unit must not be spliced onto the 9 frames held \
before the gap; got {total} bytes across {} frame(s)",
out.len()
);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
@@ -2248,4 +2482,88 @@ mod tests {
"the unit belongs to the packet its FIRST byte came from"
);
}
/// A track that becomes POISONED while an access unit is held open across a
/// PES boundary must not emit that access unit when it resumes.
///
/// Mutation this catches: deleting (or inverting) the resume-path re-check
/// `if drop_reason.is_none() && self.tally.is_poisoned()` at the top of
/// `scan_access_units`. The verdict on a held access unit is frozen at the
/// moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
/// previous access unit is closed — so the very drop that crosses the
/// poison threshold lands after the next unit's verdict was already taken
/// as `None`. Without the re-check that unit is emitted as an ordinary
/// frame after the track has been judged too damaged to mux: corrupt audio
/// passed through as success, and worse, it is the ONE frame that escapes
/// a whole-track fallback whose entire point is that nothing after the
/// verdict ships.
///
/// Neither existing held-AU test reaches this: both keep a pristine tally.
///
/// The fixture drives the exact interleaving above. `DropTally` poisons
/// once `verified_dropped * 2 > kept + dropped` past the 200-AU gate, so
/// one PES carries 200 CRC-failing `substreamid`-0 syncframes followed by a
/// clean one. Closing corrupt unit #200 (which happens only when the clean
/// frame is reached) latches the poison — after that clean frame's own
/// verdict was computed. Being an E-AC-3 unit that can still gain
/// substreams, it is then HELD across the boundary with `drop_reason:
/// None`, which is precisely the state the re-check exists for.
#[test]
fn a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes() {
// One more than the verdict gate: the Nth close is what poisons.
const CORRUPT_AUS: usize = 200;
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..CORRUPT_AUS {
let mut f = eac3_substream_frame(0, 0);
// Corrupt a payload byte AFTER the CRC was finalized: the header
// (sizing, strmtyp/substreamid, bsid) stays intact so the frame is
// still parsed as a whole access unit and fails only the CRC —
// a VERIFIED drop, the only kind that feeds the poison verdict.
f[100] ^= 0xFF;
assert!(!frame_crc_ok(&f), "the fixture frame must fail its CRC");
data.extend_from_slice(&f);
}
// The clean access unit. Last in the PES, so it is held open.
let clean = eac3_substream_frame(0, 0);
assert!(
frame_crc_ok(&clean),
"the held unit is individually decodable"
);
data.extend_from_slice(&clean);
let emitted = parser.parse(&make_eac3_pes(data));
assert!(
emitted.is_empty(),
"every corrupt unit is dropped and the clean one is held; got {} frame(s)",
emitted.len()
);
// The state the re-check depends on: the track IS poisoned, and the
// held unit was opened before that verdict existed.
assert!(
parser.tally.is_poisoned(),
"fixture must actually cross the whole-track poison threshold"
);
// Resume. The next PES opens a new unit, which closes the held one.
let out = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
out.is_empty(),
"an access unit held across the boundary must not be emitted once \
the track is poisoned; got {} frame(s) totalling {} bytes",
out.len(),
out.iter().map(|f| f.data.len()).sum::<usize>()
);
// ...and nothing may leak at end of stream either.
let tail = parser.flush();
assert!(
tail.is_empty(),
"a poisoned track emits nothing at EOS; got {} frame(s)",
tail.len()
);
assert!(
parser.dropped_frames() > CORRUPT_AUS as u64,
"the held unit must be ACCOUNTED as a drop, not silently discarded"
);
}
}
+18 -6
View File
@@ -29,10 +29,17 @@ pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
}
/// CRC-16 with polynomial 0x002D, init 0, MSB-first, used by the MLP / Dolby
/// TrueHD major-sync header checksum. NOTE: MLP's checksum is the "reversed"
/// scheme — the stored trailer word is the little-endian-read CRC, so this
/// standard CRC must be compared against the stored bytes read big-endian.
/// The caller handles that comparison (see `truehd::mlp_major_sync_ok`).
/// TrueHD major-sync header checksum.
///
/// NOTE: MLP's checksum is the "reversed" scheme. This function emits its two
/// bytes in the OPPOSITE order to a standard little-endian CRC readout, so the
/// caller swaps them back and compares against the stored trailer word read
/// LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
///
/// Comparing big-endian instead is precisely the bug that function was fixed
/// for: it could never validate a real extended major sync, so whole TrueHD
/// tracks were dropped silently. This comment used to prescribe exactly that,
/// and to point at a `truehd::mlp_major_sync_ok` that does not exist.
/// Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
@@ -103,8 +110,13 @@ mod tests {
#[test]
fn crc16_mlp_residue_property_holds() {
// Appending the big-endian CRC zeroes the residue over message+crc — the
// scheme `truehd::mlp_major_sync_ok` relies on.
// Appending the big-endian CRC zeroes the residue over message+crc.
// This is a property of the CRC itself, pinned here so a change to the
// polynomial or the bit order is caught. It is NOT how the TrueHD
// caller validates a major sync: `truehd::mlp_major_sync_crc_ok` does a
// swap-and-XOR compare against the little-endian trailer word. (This
// comment used to claim the caller relied on the residue, and named a
// `truehd::mlp_major_sync_ok` that does not exist.)
let msg = [0xF8u8, 0x72, 0x6F, 0xBA];
let c = crc16_mlp(&msg);
let mut framed = msg.to_vec();
+60 -11
View File
@@ -159,7 +159,13 @@ impl CodecParser for DvdSubParser {
/// Convert a single YCbCr color to RGB, clamping to [0, 255].
///
/// Input: `[padding, Y, Cb, Cr]` (as stored in DVD IFO PGC data).
/// Input: `[padding, Y, Cr, Cb]` (as stored in DVD IFO PGC data). Note the
/// chroma order: the on-disc PGC CLUT is **Cr before Cb** — byte 2 is Cr and
/// byte 3 is Cb. Reading byte 2 as Cb swaps red and blue on every chromatic
/// entry, and is invisible on the achromatic (white/black/grey, Cb = Cr = 128)
/// entries that dominate real palettes, which is how it survives casual
/// inspection. The order is fixed by the DVD-Video PGC format, not by us.
///
/// Returns `[R, G, B]`.
///
/// Range convention (deliberate): this uses the **full-range (JFIF) BT.601**
@@ -175,8 +181,8 @@ impl CodecParser for DvdSubParser {
/// side in lockstep.
pub fn ycbcr_to_rgb(color: &[u8; 4]) -> [u8; 3] {
let y = color[1] as f64;
let cb = color[2] as f64;
let cr = color[3] as f64;
let cr = color[2] as f64;
let cb = color[3] as f64;
let r = y + 1.402 * (cr - 128.0);
let g = y - 0.344 * (cb - 128.0) - 0.714 * (cr - 128.0);
@@ -198,7 +204,7 @@ fn clamp_u8(v: f64) -> u8 {
/// Format a 16-color YCbCr palette as a VobSub `.idx` header for S_VOBSUB
/// CodecPrivate.
///
/// Each entry is `[padding, Y, Cb, Cr]`. Output is a UTF-8 text block carrying
/// Each entry is `[padding, Y, Cr, Cb]`. Output is a UTF-8 text block carrying
/// the two `.idx` header lines mkvmerge / libvobsub expect:
///
/// ```text
@@ -432,24 +438,26 @@ mod tests {
#[test]
fn ycbcr_to_rgb_clamps_overflow() {
// Y=255, Cr=255 → R would be 255 + 1.402*127 = ~433, should clamp to 255
let color = [0x00, 255, 128, 255];
// Y=255, Cr=255 → R would be 255 + 1.402*127 = ~433, should clamp to 255.
// Cr is byte 2 in the on-disc [pad, Y, Cr, Cb] layout.
let color = [0x00, 255, 255, 128];
let [r, _g, _b] = ycbcr_to_rgb(&color);
assert_eq!(r, 255);
}
#[test]
fn ycbcr_to_rgb_clamps_underflow() {
// Y=0, Cr=0 → R = 0 + 1.402*(0-128) = -179, should clamp to 0
let color = [0x00, 0, 128, 0];
// Y=0, Cr=0 → R = 0 + 1.402*(0-128) = -179, should clamp to 0.
// Cr is byte 2 in the on-disc [pad, Y, Cr, Cb] layout.
let color = [0x00, 0, 0, 128];
let [r, _g, _b] = ycbcr_to_rgb(&color);
assert_eq!(r, 0);
}
#[test]
fn ycbcr_to_rgb_red() {
// Approximate red: Y=82, Cb=90, Cr=240
let color = [0x00, 82, 90, 240];
// Approximate red: Y=82, Cr=240, Cb=90 — on disc as [pad, Y, Cr, Cb].
let color = [0x00, 82, 240, 90];
let [r, g, b] = ycbcr_to_rgb(&color);
// R = 82 + 1.402*(240-128) = 82 + 156.9 ≈ 239
// G = 82 - 0.344*(90-128) - 0.714*(240-128) = 82 + 13.1 - 79.97 ≈ 15
@@ -459,6 +467,46 @@ mod tests {
assert!(b < 30, "B should be low for red, got {}", b);
}
/// On-disc DVD PGC CLUT byte order is `[0, Y, Cr, Cb]` — byte 2 is **Cr**
/// and byte 3 is **Cb**, per the DVD-Video PGC format.
///
/// This fixture uses a real on-disc red entry, so it fails if the two
/// chroma bytes are ever exchanged again. It is deliberately NOT built
/// from this crate's own doc comments: those described the order wrongly
/// for a long time, and the previous version of this test inherited the
/// error from them and therefore could not detect it.
///
/// A saturated RED entry therefore appears on disc as Y=76, Cr=255, Cb=85
/// (full-range BT.601 encoding of RGB #FF0000), i.e. bytes
/// `[0x00, 76, 255, 85]`. Reading byte 2 as Cb and byte 3 as Cr instead
/// turns this entry BLUE, which is the exact user-visible symptom.
///
/// The pre-existing `_white` / `_black` tests cannot catch this: they use
/// Cb = Cr = 128, so exchanging two equal bytes is a literal no-op.
#[test]
fn ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb() {
// On-disc [pad, Y, Cr, Cb] for saturated red.
let on_disc_red = [0x00u8, 76, 255, 85];
let [r, g, b] = ycbcr_to_rgb(&on_disc_red);
assert!(
r > 200 && b < 60,
"on-disc red [0,Y=76,Cr=255,Cb=85] must render red-dominant, \
got R={r} G={g} B={b} (R and B swapped => byte 2/3 are transposed)"
);
assert_eq!([r, g, b], [254, 0, 0], "exact full-range BT.601 red");
// And the converse: a saturated BLUE on-disc entry (Y=29, Cr=107, Cb=255)
// must not come out red.
let on_disc_blue = [0x00u8, 29, 107, 255];
let [r2, g2, b2] = ycbcr_to_rgb(&on_disc_blue);
assert!(
b2 > 200 && r2 < 60,
"on-disc blue [0,Y=29,Cr=107,Cb=255] must render blue-dominant, \
got R={r2} G={g2} B={b2}"
);
}
// ── Palette formatting tests ──────────────────────────────────────────
#[test]
@@ -699,7 +747,8 @@ mod tests {
#[test]
fn ycbcr_blue_channel_clamps_high() {
// B = Y + 1.772*(Cb-128). Y=128, Cb=255 → 128 + 1.772*127 ≈ 353 → clamp 255.
let [_r, _g, b] = ycbcr_to_rgb(&[0x00, 128, 255, 128]);
// Cb is byte 3 in the on-disc [pad, Y, Cr, Cb] layout.
let [_r, _g, b] = ycbcr_to_rgb(&[0x00, 128, 128, 255]);
assert_eq!(b, 255, "blue clamps at 255");
}
+73 -1
View File
@@ -1,7 +1,8 @@
//! H.264 (AVC) elementary stream parser.
//!
//! Extracts SPS and PPS NAL units for MKV codecPrivate.
//! Detects keyframes (IDR slices).
//! Detects keyframes: IDR slices, and open-GOP intra-coded access units (the
//! non-IDR recovery points BD titles use — see the intra promotion in `parse`).
//! Each PES packet = one access unit = one frame.
use super::coding::{CodingType, PictureInfo};
@@ -351,6 +352,26 @@ impl CodecParser for H264Parser {
return Vec::new();
}
// Open-GOP resync anchor: an intra-coded access unit is a keyframe even
// when it is NOT an IDR. BD H.264 titles use open GOPs whose only
// random-access point is a non-IDR I-frame (the recovery point, which an
// encoder MAY additionally tag with a recovery_point SEI). Keying off the
// already-measured picture coding type covers every signalling — a
// recovery_point SEI, or a disc that marks the recovery point only by GOP
// structure with no SEI at all — where SEI parsing alone would miss the
// latter. Without this, the B1 resync gate (`mux/resync.rs`), once armed
// by a concealed-gap / clip-seam discontinuity, never sees a keyframe on
// such a tail and drops every frame to EOF, truncating the video. HEVC
// does not hit this: its open-GOP IRAP (CRA) is a distinct NAL type
// already flagged a keyframe (`mux/codec/hevc.rs`); H.264 has no such NAL
// tag, so the intra coding type is the equivalent signal. Consistent with
// `videomap.rs`'s model that a keyframe IS the intra decode-restart point.
// IDR is still flagged by NAL type in the loop above, so a slice whose
// header failed to parse (`coding_type == None`) is unaffected here.
if coding_type == Some(CodingType::I) {
keyframe = true;
}
// Every keyframe is self-contained: re-assert the active SPS/PPS in-band
// ahead of the slices (even when unchanged vs codecPrivate) so a decoder
// that dropped the set at a reset recovers, and a stale avcC re-apply
@@ -694,6 +715,57 @@ mod tests {
}
}
/// Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
/// keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD open-
/// GOP tail instead of dropping every frame to EOF and truncating the video.
/// BD H.264 titles reach random access at a non-IDR I-frame (the recovery
/// point); HEVC gets this free from the CRA NAL type, H.264 does not, so the
/// measured intra coding type is the signal. Red before the intra promotion
/// in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
#[test]
fn open_gop_intra_access_unit_is_a_keyframe() {
// Annex B AU: one non-IDR coded slice (NAL type 1, nal_ref_idc 3 → 0x61)
// whose header decodes first_mb_in_slice = 0 (ue = "1") and slice_type = 7
// (ue = "0001000"), i.e. an all-I slice. The two ue(v) fields pack into
// one byte: "1" + "0001000" = 0b1000_1000 = 0x88.
let au = vec![0x00, 0x00, 0x01, 0x61, 0x88, 0x00];
let mut parser = H264Parser::new();
let frames = parser.parse(&make_pes(au, Some(0)));
assert_eq!(frames.len(), 1, "one PES access unit → one frame");
assert_eq!(
frames[0].coding.map(|c| c.coding_type()),
Some(CodingType::I),
"the slice header measures an I picture",
);
assert!(
frames[0].keyframe,
"an open-GOP intra access unit is a resync-safe keyframe even without an IDR",
);
}
/// Guard the promotion's precision: a non-intra (P) access unit must NOT be
/// promoted to a keyframe, or the resync gate would resume on a frame that
/// carries inter-references to dropped data and the MKV cue index would gain
/// bogus random-access points.
#[test]
fn inter_coded_access_unit_is_not_a_keyframe() {
// Same NAL type 1, but slice_type = 0 (P): ue "1", so with first_mb = 0
// the byte is "1" + "1" padded = 0b1100_0000 = 0xC0.
let au = vec![0x00, 0x00, 0x01, 0x61, 0xC0, 0x00];
let mut parser = H264Parser::new();
let frames = parser.parse(&make_pes(au, Some(0)));
assert_eq!(frames.len(), 1);
assert_eq!(
frames[0].coding.map(|c| c.coding_type()),
Some(CodingType::P),
"the slice header measures a P picture",
);
assert!(
!frames[0].keyframe,
"an inter-coded access unit must never be a keyframe",
);
}
/// The slice-header parse must remove emulation-prevention bytes first
/// (ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is
/// EBSP, and a 0x03 following 0x00 0x00 is not part of the RBSP.
+58 -1
View File
@@ -637,10 +637,18 @@ fn xml_escape(s: &str) -> String {
}
/// Replace path-hostile characters in a filename component.
///
/// Control characters included, NUL above all. Every string this touches is
/// disc bytes, and a language code is three raw STN bytes run through
/// `from_utf8_lossy` with no validation — `00 00 00` is the ordinary
/// "undefined" encoding on real Blu-rays. A NUL in a path aborts `File::create`
/// with `InvalidInput`, which took the whole demux export down before a single
/// track file was opened.
fn sanitize(s: &str) -> String {
s.chars()
.map(|c| match c {
'/' | '\\' | ':' | '*' | '?' | '"' | '<' | '>' | '|' => '_',
c if c.is_control() => '_',
_ => c,
})
.collect()
@@ -787,8 +795,11 @@ impl DemuxSink {
Naming::Pid => format!("{} {:04x}", sanitize(&opts.base), pid),
Naming::Friendly => {
let mut parts = vec![sanitize(&opts.base), format!("t{idx:02}")];
// The language is disc bytes too, and got none of the
// treatment `opts.base` two lines up already had.
let lang = sanitize(lang);
if !lang.is_empty() {
parts.push(lang.to_string());
parts.push(lang);
}
parts.push(codec_label(codec).to_string());
parts.join(" ")
@@ -987,6 +998,52 @@ mod tests {
Resolution, SampleRate, VideoStream,
};
// ── The language component is disc bytes ──────────────────────────────
//
// `opts.base` was sanitised and the language beside it was not, though a
// language code is three raw STN bytes run through `from_utf8_lossy` with
// no validation. `00 00 00` is the ordinary "undefined" encoding on real
// Blu-rays, and a NUL in a path fails `File::create` with `InvalidInput` —
// which aborted the whole export before a single track file was opened.
/// The stem stays one usable filename component whatever the disc says.
#[test]
fn a_hostile_language_code_cannot_break_the_output_filename() {
let opts = DemuxOptions {
base: "Movie".to_string(),
..Default::default()
};
assert!(matches!(opts.naming, Naming::Friendly), "default naming");
for lang in ["\u{0}\u{0}\u{0}", "a/b", "..", "a\nb", "e:s"] {
let stem = DemuxSink::stem_for(&opts, 1, 0x1100, lang, Codec::Ac3);
assert!(
!stem.chars().any(|c| c.is_control()),
"control character survived into the filename for {lang:?}: {stem:?}"
);
assert!(
!stem.contains('/') && !stem.contains('\\') && !stem.contains(':'),
"a path separator survived for {lang:?}: {stem:?}"
);
assert!(
std::path::Path::new(&stem).components().count() == 1,
"the stem must stay ONE component for {lang:?}: {stem:?}"
);
}
}
/// A legitimate language is still carried through untouched — the
/// sanitiser must not be so lossy that it stops naming the track.
#[test]
fn an_ordinary_language_code_survives_sanitising() {
let opts = DemuxOptions {
base: "Movie".to_string(),
..Default::default()
};
let stem = DemuxSink::stem_for(&opts, 1, 0x1100, "eng", Codec::Ac3);
assert!(stem.contains("eng"), "got {stem:?}");
}
fn video_stream(codec: Codec) -> DiscStream {
DiscStream::Video(VideoStream {
pid: 0x1011,
+148
View File
@@ -566,6 +566,20 @@ impl DiscStream {
.into());
}
// The read SOURCE is gone (a prefetch producer thread that
// terminated), not one range of media. Shrinking and retrying at
// the same LBA asks a dead source for data it can never produce,
// and the `skip_errors` branch below would then zero-fill and
// advance over every remaining sector of the title and still
// return success. Abort with the terminal error itself — a
// fabricated SCSI status would be a lie, so this is deliberately
// NOT folded into the transport-failure arm above.
if let Some(e) = res.as_ref().err()
&& e.is_source_terminated()
{
return Err(crate::error::Error::SourceTerminated.into());
}
if (sectors as u32) <= align {
// Bottomed out at one unit (AACS) / one sector (CSS) / the
// extent tail. This is single-pass disc→MKV, which has NO Pass N
@@ -621,6 +635,16 @@ impl DiscStream {
.into());
}
// Same rule as after the first-attempt read: the 60s recovery
// read goes through the same source, so it can be the call
// that discovers the source is dead. Skipping the unit would
// zero-fill the rest of the title as fabricated content.
if let Some(e) = rec.as_ref().err()
&& e.is_source_terminated()
{
return Err(crate::error::Error::SourceTerminated.into());
}
// Recovery read also failed. Skip the WHOLE failed unit or bail.
// Zero-filling and advancing by the full unit keeps
// current_offset unit-aligned, so the next read still begins on a
@@ -1859,6 +1883,130 @@ mod tests {
);
}
/// REGRESSION (round-4 audit): an ordinary MEDIUM ERROR bad sector must
/// keep its identity when it crosses the prefetch producer channel — the
/// same `DiscRead` with its SCSI status, NOT a transport failure.
///
/// `PrefetchedSectorSource::read_sectors` re-wrapped every error that
/// crossed the channel as `Error::IoError`, and `is_scsi_transport_failure`
/// matches `IoError` (the wedged-USB-bridge arm). So a bad sector reached
/// `fill_extents` looking like a dead bus and aborted the pass with a
/// fabricated status 0xFF — the exact inverse of what that short-circuit
/// exists for, and it told the user to power-cycle a healthy drive.
///
/// Asserted on the source, not on a `fill_extents` skip: the producer
/// thread exits for good after sending an error, so nothing downstream of
/// it can genuinely recover the rest of the title (see
/// `dead_prefetch_producer_does_not_silently_zero_fill_the_title`). An
/// assertion that the pass continues could only ever have been satisfied
/// by fabricated zeros.
#[test]
fn bad_sector_keeps_its_identity_across_the_prefetch_channel() {
const COUNT: u32 = 9;
let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let reader = RecordingReader {
capacity: COUNT,
bad_sector: 4,
log: log.clone(),
};
let mut prefetched = crate::sector::PrefetchedSectorSource::new_with_events(
reader,
vec![crate::disc::Extent {
start_lba: 0,
sector_count: COUNT,
}],
8,
1,
None,
None,
)
.expect("spawn producer");
let mut buf = vec![0u8; 8 * 2048];
let err = crate::sector::SectorSource::read_sectors(&mut prefetched, 0, 8, &mut buf, false)
.expect_err("the batch covering the bad sector must fail");
assert!(
!err.is_scsi_transport_failure(),
"a MEDIUM ERROR bad sector is not a dead bus; got {err:?}"
);
assert!(
matches!(
err,
crate::error::Error::DiscRead {
sector: 4,
status: Some(0x02),
..
}
),
"the producer's typed error must survive the channel intact; got {err:?}"
);
}
/// The prefetch producer thread terminates PERMANENTLY on its first read
/// error, so once one bad sector has crossed the channel the source can
/// never deliver another byte. Driving `fill_extents` to exhaustion after
/// that must NOT look like a completed pass: every remaining sector would
/// be fabricated zeros, and DATA LOSS MUST NEVER LOOK LIKE SUCCESS.
///
/// The expectation is the product rule, not the code: a source that is
/// permanently out of data must report that, not answer `Ok(0)` forever —
/// which `commit_read` legitimately reads as an ordinary short read and
/// zero-fills.
#[test]
fn dead_prefetch_producer_does_not_silently_zero_fill_the_title() {
const COUNT: u32 = 30;
let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let reader = RecordingReader {
capacity: COUNT,
bad_sector: 4,
log: log.clone(),
};
let prefetched = crate::sector::PrefetchedSectorSource::new_with_events(
reader,
vec![crate::disc::Extent {
start_lba: 0,
sector_count: COUNT,
}],
8,
1,
None,
None,
)
.expect("spawn producer");
let mut stream = DiscStream::new(
Box::new(prefetched),
synthetic_title(COUNT),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
stream.skip_errors = true;
// Drive the whole title. Bounded so a regression cannot hang the suite.
let mut completed_clean = false;
for _ in 0..(COUNT as usize * 4) {
match stream.fill_extents() {
Ok(true) => continue,
Ok(false) => {
completed_clean = true;
break;
}
Err(_) => break,
}
}
assert!(
!completed_clean,
"the producer died at sector 4, so sectors 4..{COUNT} were never \
read — reporting the pass as complete zero-fills {} of {} bytes \
and calls it success",
stream.lost_bytes,
COUNT as u64 * 2048
);
}
/// AACS unit-alignment skip (the #1 coverage gap). With `unit_align=3`
/// (DecryptKeys::Aacs) and `skip_errors=true`, a single bad mid-extent
/// sector must NOT desync the rest of the title: every `read_sectors`
+300 -3
View File
@@ -359,6 +359,22 @@ fn write_hdr10<W: Write + Seek>(w: &mut W, h: &crate::mux::codec::Hdr10Metadata)
Ok(())
}
/// The Matroska `Language` element the muxer writes for a stream whose source
/// reported `lang`. RFC 9559 §12 defines the element as an ISO 639-2 code and
/// gives no meaning to an empty one; the code for "no language stated" is
/// `und`, and that is what a source with no language table (the HD-DVD EVO
/// stream probe, a Blu-ray STN slot with no language bytes) has to emit. The
/// element is written unconditionally by `MkvMuxer::new`, so this is the one
/// place that decides it — a source-side default would have to be repeated in
/// every scanner and would still leave the muxer able to ship an invalid file.
fn language_or_und(lang: &str) -> String {
if lang.is_empty() {
"und".to_string()
} else {
lang.to_string()
}
}
impl MkvTrack {
/// Build a video track from a [`VideoStream`]. Language defaults to `"und"`;
/// colour metadata is derived from the stream's colour space and HDR format
@@ -530,7 +546,7 @@ impl MkvTrack {
Self {
track_type: ebml::TRACK_TYPE_AUDIO,
codec_id,
language: a.language.clone(),
language: language_or_und(&a.language),
name,
codec_private: None,
is_default: !a.secondary,
@@ -587,7 +603,7 @@ impl MkvTrack {
Self {
track_type: ebml::TRACK_TYPE_SUBTITLE,
codec_id,
language: s.language.clone(),
language: language_or_und(&s.language),
name: String::new(),
codec_private: s.codec_data.clone(),
is_default: false,
@@ -1448,7 +1464,9 @@ impl<W: Write + Seek> MkvMuxer<W> {
// all tracks, A/V-sync-preserving) keeps the boundary from becoming a
// band of non-monotonic block timestamps. Only the PRIMARY video track
// drives the boundary decision; every other track (audio, subtitle, DV
// EL) rides the current offset. No-op for single-clip titles.
// EL) rides the current offset. A single-clip BD title still trims to
// its one clip's [in, out] marks (dropping trailing audio the m2ts
// carries past OUT); only a source with no marks at all is a true no-op.
// `None` means the playlist does not include this frame — material
// outside every clip's IN/OUT marks, which only a seam-plan-driven
// title can report. Dropping it is the point: emitting it is what put
@@ -3305,6 +3323,285 @@ mod tests {
);
}
/// Read back the value of the FIRST `Language` element in `data` as a
/// UTF-8 string. `LANGUAGE` (0x22B59C) is a 3-byte EBML ID; its size is
/// always a 1-byte VINT for the short strings this writer emits.
fn first_language_value(data: &[u8]) -> &str {
let pos = find_id(data, ebml::LANGUAGE).expect("Language element must be present");
let len = (data[pos + 3] & 0x7F) as usize;
std::str::from_utf8(&data[pos + 4..pos + 4 + len]).unwrap()
}
/// RFC 9559 §12 / the Matroska `Language` element spec restrict the
/// legacy `Language` element to the Matroska language form (ISO 639-2,
/// three lowercase letters), never ISO 639-1 (two letters). The DVD
/// IFO audio-attribute block itself carries a raw ISO 639-1 code (e.g.
/// "en") on disc — `ifo::parse_audio_attr` converts it to ISO 639-2
/// before returning, via `ifo::dvd_lang_to_iso639_2`. This mimics the
/// real DVD pipeline (`disc/dvd.rs`'s `Stream::Audio` construction) end
/// to end: real on-disc IFO bytes -> `ifo::parse_audio_attr` ->
/// `disc::AudioStream` -> `MkvTrack::audio` -> the muxer -> the emitted
/// `Language` element.
#[test]
fn dvd_two_letter_language_becomes_iso_639_2_in_language_element() {
// AC-3 (coding_mode=0), 48 kHz, 6 channels, on-disc language "en" —
// the exact byte layout `ifo::audio_attr_parsing` pins.
let mut attr_bytes = vec![0u8; 8];
attr_bytes[0] = 0x00;
attr_bytes[1] = 0x05;
attr_bytes[2] = b'e';
attr_bytes[3] = b'n';
let attr = crate::ifo::parse_audio_attr(&attr_bytes, 0).unwrap();
assert_eq!(
attr.language, "eng",
"parse_audio_attr must already return the ISO 639-2 form"
);
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"eng",
"a DVD-sourced ISO 639-1 code must be written as its ISO 639-2 \
equivalent in the Matroska Language element, per RFC 9559 §12"
);
}
/// A source that knows no language at all leaves `language` EMPTY, and the
/// muxer writes the `Language` element unconditionally — so an empty
/// string becomes a zero-length `Language` in the shipped file, which is
/// not a Matroska language form (RFC 9559 §12 wants three ISO 639-2
/// letters) and is not the ISO 639-2 code for "unknown" either.
///
/// This is the state every HD-DVD rip is in: `disc::hddvd`'s EVO stream
/// probe has no language table to read and sets `language: String::new()`
/// on every audio stream it finds. The DVD path normalises to "und" in
/// `ifo::parse_audio_attr`; the guard has to exist at the muxer too, which
/// is the one place every source funnels through.
#[test]
fn a_source_with_no_language_emits_und_not_an_empty_language_element() {
// Exactly what `disc::hddvd::probe_evo_streams` builds.
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: Codec::Ac3Plus,
channels: crate::disc::AudioChannels::Surround51,
language: String::new(),
sample_rate: crate::disc::SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let subtitle_stream = crate::disc::SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: String::new(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
};
for track in [
MkvTrack::audio(&audio_stream),
MkvTrack::subtitle(&subtitle_stream),
] {
let track_type = track.track_type;
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"und",
"track type {track_type}: a stream with no known language must \
emit the ISO 639-2 'undetermined' code, never a zero-length \
Language element"
);
}
}
/// An unmapped or absent DVD language code (bytes 0x00 0x00 in the IFO
/// attribute block) must degrade to the valid Matroska "undetermined"
/// code `und`, never to an empty string or a raw 2-letter code — both of
/// which violate the Matroska language form.
#[test]
fn dvd_unmapped_or_empty_language_becomes_und_in_language_element() {
// Empty IFO language bytes (0x00 0x00).
let mut empty_bytes = vec![0u8; 8];
empty_bytes[0] = 0x00; // AC-3, 48k
empty_bytes[1] = 0x05; // 6ch
let empty_attr = crate::ifo::parse_audio_attr(&empty_bytes, 0).unwrap();
assert_eq!(
empty_attr.language, "und",
"IFO zero bytes (unspecified) must resolve to 'und', not empty"
);
// An IFO code with no known ISO 639-1 -> 639-2 mapping (e.g. a
// fictitious "zz").
let mut unmapped_bytes = vec![0u8; 8];
unmapped_bytes[0] = 0x00;
unmapped_bytes[1] = 0x05;
unmapped_bytes[2] = b'z';
unmapped_bytes[3] = b'z';
let unmapped_attr = crate::ifo::parse_audio_attr(&unmapped_bytes, 0).unwrap();
assert_eq!(
unmapped_attr.language, "und",
"an on-disc code with no known ISO 639-1 -> 639-2 mapping must \
degrade to 'und', never pass through raw"
);
for attr in [empty_attr, unmapped_attr] {
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"und",
"an empty or unmapped DVD language code must degrade to 'und', \
never an empty string or an invalid ISO 639-1 code"
);
}
}
/// Build an 8-byte DVD IFO audio-attribute block (AC-3, 48 kHz, 6ch)
/// carrying `code` in the language bytes, run it through the real DVD
/// pipeline (`ifo::parse_audio_attr` -> `disc::AudioStream` ->
/// `MkvTrack::audio` -> the muxer) and return the value that actually
/// lands in the emitted Matroska `Language` element.
fn emitted_language_for_dvd_code(code: &[u8; 2]) -> String {
let mut attr_bytes = vec![0u8; 8];
attr_bytes[0] = 0x00; // AC-3, 48 kHz
attr_bytes[1] = 0x05; // 6 channels
attr_bytes[2] = code[0];
attr_bytes[3] = code[1];
let attr = crate::ifo::parse_audio_attr(&attr_bytes, 0).unwrap();
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
first_language_value(&data).to_string()
}
/// The ISO 639-1 -> ISO 639-2 conversion must cover the WHOLE of ISO
/// 639-1, not just the handful of languages that happen to appear in
/// Blu-ray menu-graphic filenames. A Region-2 disc routinely carries
/// Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian, Hebrew,
/// Estonian, Latvian, Lithuanian, Icelandic and so on; if those all
/// collapse to `und`, every one of a disc's subtitle tracks emits the
/// same `Language` value and nothing else tells them apart (DVD streams
/// carry an empty `label`). A valid-but-identical code is worse for the
/// user than the invalid one it replaced, so each of these must reach the
/// emitted `Language` element as its own correct three-letter code.
#[test]
fn dvd_language_outside_the_menu_vocabulary_is_still_mapped() {
assert_eq!(
emitted_language_for_dvd_code(b"ro"),
"ron",
"Romanian ('ro'), common on Region-2 discs, must reach the \
Matroska Language element as 'ron' — not 'und'"
);
// The rest of the set the menu-label table never knew, one per
// language so a single missing table row fails loudly.
for (code, expected) in [
(b"bg", "bul"),
(b"hr", "hrv"),
(b"sr", "srp"),
(b"sk", "slk"),
(b"sl", "slv"),
(b"he", "heb"),
(b"et", "est"),
(b"lv", "lav"),
(b"lt", "lit"),
(b"is", "isl"),
(b"id", "ind"),
(b"vi", "vie"),
(b"fa", "fas"),
] {
assert_eq!(
emitted_language_for_dvd_code(code),
expected,
"DVD language {:?} must map to {expected:?} in the emitted \
Language element",
std::str::from_utf8(code).unwrap()
);
}
}
/// DVD-Video froze its language list on the 1988 edition of ISO 639-1,
/// which spelled Hebrew `iw`, Indonesian `in` and Yiddish `ji`. Real
/// discs authored to that list carry those bytes, so they must map to the
/// same ISO 639-2 codes as the modern `he` / `id` / `yi` spellings rather
/// than degrading to `und`.
#[test]
fn dvd_era_language_aliases_map_to_the_modern_code() {
assert_eq!(
emitted_language_for_dvd_code(b"iw"),
"heb",
"the DVD-era spelling of Hebrew ('iw') must emit 'heb'"
);
assert_eq!(emitted_language_for_dvd_code(b"in"), "ind");
assert_eq!(emitted_language_for_dvd_code(b"ji"), "yid");
// ...and agree with the modern spellings.
assert_eq!(emitted_language_for_dvd_code(b"he"), "heb");
assert_eq!(emitted_language_for_dvd_code(b"id"), "ind");
assert_eq!(emitted_language_for_dvd_code(b"yi"), "yid");
}
/// Widening the table must not weaken the degradation guarantee: a code
/// that is not ISO 639-1 at all still has to yield exactly `und`, a valid
/// Matroska language value, and never a passed-through two-letter code,
/// an empty string, or a guess.
#[test]
fn unknown_dvd_language_still_yields_exactly_und() {
for code in [b"zz", b"qq", b"xx"] {
assert_eq!(
emitted_language_for_dvd_code(code),
"und",
"a code outside ISO 639-1 must degrade to exactly 'und'"
);
}
// Empty language bytes (0x00 0x00) likewise.
assert_eq!(emitted_language_for_dvd_code(&[0x00, 0x00]), "und");
}
#[test]
fn mkv_forced_flag_on_forced_subtitle() {
use crate::disc::SubtitleStream;
+84 -1
View File
@@ -12,8 +12,19 @@ use super::{WriteSeek, ebml};
type MkvHeaderResult = io::Result<(crate::disc::DiscTitle, Vec<(u16, Vec<u8>)>, i64, TrackTable)>;
/// Skip `n` bytes on a forward-only reader (no Seek required).
///
/// A skip that runs out of input before `n` bytes is a TRUNCATED element, and is
/// reported the same way `ebml::read_binary_val` reports a truncated body: as
/// `MkvSourceInvalid`. Discarding `io::copy`'s byte count instead made a skip
/// that hit EOF look like a success, so one corrupt size field mid-Clusters
/// drained the rest of the file, the next element header raised
/// `UnexpectedEof`, and `Stream::read` mapped that to `Ok(None)` — half the
/// title missing, `errors = 0`, `completed = true`.
fn skip_bytes(r: &mut impl Read, n: u64) -> io::Result<()> {
io::copy(&mut r.take(n), &mut io::sink())?;
let skipped = io::copy(&mut r.take(n), &mut io::sink())?;
if skipped != n {
return Err(crate::error::Error::MkvSourceInvalid.into());
}
Ok(())
}
@@ -3193,6 +3204,78 @@ mod tests {
assert!(stream.read().unwrap().is_none(), "clean EOF → None");
}
/// A skipped element whose declared size runs PAST the end of the file is a
/// truncated element, exactly like a truncated `read_binary_val` body — and
/// must be reported the same way, as `MkvSourceInvalid`.
///
/// `skip_bytes` used to discard `io::copy`'s returned count, so the skip
/// "succeeded" having drained the rest of the file. The next element header
/// then hit `UnexpectedEof`, which `read()` maps to `Ok(None)` — a clean end
/// of stream. One corrupt size field mid-Clusters therefore threw away every
/// remaining frame of the title and reported `errors = 0`, `complete = true`.
#[test]
fn a_skip_past_eof_is_an_error_not_a_clean_end_of_stream() {
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
// Frame 1 — read normally.
let block = [0x81u8, 0x00, 0x00, 0x80, 0xAA];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block.len() as u64).unwrap();
cluster.extend_from_slice(&block);
// A VOID whose size field is corrupt: it claims 1 MiB, and the file
// holds only the handful of bytes below. This is the "corrupt size
// field mid-Clusters" case.
ebml::write_id(&mut cluster, ebml::VOID).unwrap();
ebml::write_size(&mut cluster, 1024 * 1024).unwrap();
// Frame 2 — the rest of the title, swallowed by the bad skip.
let block2 = [0x81u8, 0x00, 0x01, 0x80, 0xBB];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block2.len() as u64).unwrap();
cluster.extend_from_slice(&block2);
let bytes = mkv_with_track_and_cluster(1, 1, &cluster);
let mut stream = MkvStream::open(Cursor::new(bytes)).unwrap();
assert!(stream.read().unwrap().is_some(), "first frame reads");
let e = match stream.read() {
Err(e) => e,
Ok(None) => panic!(
"a skip that hit EOF was reported as a CLEAN END OF STREAM: the \
rest of the title is gone and the caller sees errors = 0, \
complete = true"
),
Ok(Some(_)) => panic!("the truncated skip must not yield a frame"),
};
assert!(is_mkv_source_invalid(&e), "{e:?}");
}
/// The honest path this fix must not break: a skipped element whose declared
/// size is exactly satisfied by the bytes present is still skipped cleanly,
/// and the genuine EOF that follows is still `Ok(None)`.
#[test]
fn a_fully_satisfied_skip_still_ends_at_a_clean_eof() {
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
// A VOID that is fully present.
ebml::write_id(&mut cluster, ebml::VOID).unwrap();
ebml::write_size(&mut cluster, 8).unwrap();
cluster.extend_from_slice(&[0u8; 8]);
let block = [0x81u8, 0x00, 0x00, 0x80, 0xAA];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block.len() as u64).unwrap();
cluster.extend_from_slice(&block);
let bytes = mkv_with_track_and_cluster(1, 1, &cluster);
let mut stream = MkvStream::open(Cursor::new(bytes)).unwrap();
let f = stream.read().unwrap().expect("the frame after the VOID");
assert_eq!(f.data, vec![0xAA]);
assert!(
stream.read().unwrap().is_none(),
"a genuine EOF at a record boundary is still a clean end"
);
}
// ============================================================
// Block LACING (RFC 9559 §10.3) and TrackNumber→stream routing
// (RFC 9559 §5.1.4.1.1).
+22 -2
View File
@@ -627,16 +627,36 @@ mod tests {
/// accept_from() must reject a connection whose first bytes are NOT the
/// FMKV magic — there is no metadata to drive muxing, so it surfaces
/// NoMetadata rather than proceeding with an empty/garbage title.
///
/// FLAKE FIXED, not the behaviour under test: this used to
/// `shutdown(Shutdown::Both)` the instant the bytes were written. Closing
/// the READ half while the server had not yet read makes the kernel answer
/// the server's in-flight data with an RST, so `accept_from` came back
/// `ConnectionReset` instead of the `InvalidInput` this asserts — rarely
/// when run alone, reproducibly under the loaded concurrent suite, where
/// the server thread is descheduled long enough for the race to open. A
/// logging/protocol assertion that fails at random teaches the next person
/// to re-run until green, which is how a real regression gets waved
/// through.
///
/// Half-closing (`Shutdown::Write`) delivers the same EOF the test needs
/// while leaving the read half open, and blocking on a read until the
/// server drops its end keeps the socket alive for as long as the server
/// is looking at it. The port was already ephemeral (`:0`), so it was
/// never a port collision.
#[test]
fn accept_from_rejects_stream_without_fmkv_header() {
use std::io::Read as _;
use std::io::Write as _;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
let handle = std::thread::spawn(move || {
// Raw non-FMKV bytes (not starting with 'F') then close.
// Raw non-FMKV bytes (not starting with 'F') then EOF.
let mut s = TcpStream::connect(addr).unwrap();
s.write_all(&[0x47u8; 64]).unwrap(); // TS sync bytes, no FMKV magic
s.shutdown(std::net::Shutdown::Both).unwrap();
s.shutdown(std::net::Shutdown::Write).unwrap();
// Park until the server closes, so no RST can overtake the data.
let _ = s.read(&mut [0u8; 1]);
});
let err = match NetworkStream::accept_from(listener) {
Ok(_) => panic!("missing FMKV header must error, not silently accept"),
+256 -10
View File
@@ -17,7 +17,7 @@ use super::codec::startcode::find_start_code;
const PACK_HEADER_ID: u8 = 0xBA;
/// System header start code suffix.
const SYSTEM_HEADER_ID: u8 = 0xBB;
const SYSTEM_HEADER_ID: u8 = crate::consts::pes_stream_id::SYSTEM_HEADER;
/// Program end start code suffix.
const PROGRAM_END_ID: u8 = 0xB9;
@@ -169,6 +169,24 @@ pub struct PsDemuxer {
/// stay byte-identical.
buffer_base: u64,
has_base: bool,
/// Boundary-scan cursor for an unbounded (length-0) PES still waiting for
/// its terminating PS-layer unit: `(buffer offset of the PES start code,
/// buffer offset up to which the search has already proved there is no
/// boundary)`. Both are buffer-relative and are rebased when the buffer
/// drains.
///
/// Without it, every `feed` re-searches the WHOLE accumulated payload from
/// the PES header: the buffer only stops growing at [`MAX_PS_BUFFER`], so a
/// stream that declares an unbounded PES and then never emits a PS-layer
/// start code (a corrupt or crafted VOB) makes the demuxer scan up to 4 MiB
/// per call, quadratic in the bytes fed. Cleared whenever the PES is
/// emitted, so it can never outlive the packet it describes.
pending_scan: Option<(usize, usize)>,
/// Test-only: total bytes examined by `find_ps_boundary`. Pins the cursor
/// above — the property it exists for is a WORK bound, which no
/// packet-level assertion can observe.
#[cfg(test)]
boundary_bytes_scanned: u64,
}
impl Default for PsDemuxer {
@@ -184,6 +202,9 @@ impl PsDemuxer {
buffer: Vec::with_capacity(64 * 1024),
buffer_base: 0,
has_base: false,
pending_scan: None,
#[cfg(test)]
boundary_bytes_scanned: 0,
}
}
@@ -220,6 +241,8 @@ impl PsDemuxer {
// discarded.
let packets = self.extract_packets(true);
self.buffer.clear();
// The buffer the cursor indexes into is gone.
self.pending_scan = None;
packets
}
@@ -288,11 +311,29 @@ impl PsDemuxer {
// start code — the video ES payload is itself full of
// 00 00 01 xx codes that would otherwise cut the PES short.
let end = if pes_packet_len == 0 {
match find_ps_boundary(&self.buffer, sc + 4) {
Some(next) => next,
// Resume where the last call stopped searching for
// THIS PES's terminating unit; anything before that is
// already proved boundary-free.
let from = match self.pending_scan {
Some((pes_at, searched_to)) if pes_at == sc => searched_to,
_ => sc + 4,
};
let (found, searched_to) = find_ps_boundary(&self.buffer, from);
#[cfg(test)]
{
self.boundary_bytes_scanned += searched_to.saturating_sub(from) as u64;
}
match found {
Some(next) => {
self.pending_scan = None;
next
}
// At EOF the rest of the buffer is this PES's
// payload — emit it.
None if flushing => self.buffer.len(),
None if flushing => {
self.pending_scan = None;
self.buffer.len()
}
None => {
// No boundary buffered yet. Normally wait for
// more data, but a corrupt stream could declare
@@ -301,8 +342,10 @@ impl PsDemuxer {
// stops untrusted input forcing unbounded
// allocation. Past the cap, flush what we have.
if self.buffer.len() - sc > MAX_PS_BUFFER {
self.pending_scan = None;
self.buffer.len()
} else {
self.pending_scan = Some((sc, searched_to));
break; // wait for more data
}
}
@@ -338,6 +381,13 @@ impl PsDemuxer {
if self.has_base {
self.buffer_base += pos as u64;
}
// The cursor is a BUFFER offset, so it moves with the drain. A
// pending PES always starts at or after `pos` (the loop broke on
// it, having already consumed everything before it), so neither
// component can underflow.
self.pending_scan = self
.pending_scan
.map(|(pes_at, searched_to)| (pes_at - pos, searched_to - pos));
}
// Trim a start-code-free tail. Every other exit from the loop above
@@ -359,6 +409,10 @@ impl PsDemuxer {
if self.has_base {
self.buffer_base += drop as u64;
}
// A pending PES implies a start code IS in the buffer, so this
// branch cannot run while one is open; drop the cursor anyway
// rather than leave a stale offset behind this drain.
self.pending_scan = None;
}
packets
@@ -380,11 +434,19 @@ const START_CODE_PREFIX_KEEP: usize = 2;
/// PES inside its own payload and re-scan the discarded video bytes as bogus PS
/// units. Restricting the search to PS-layer IDs (>= 0xB9, excluding the video
/// ES codes below it) frames the unbounded PES at the right boundary.
fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
/// Returns `(boundary, searched_to)`. `searched_to` is the offset up to which
/// every byte has been PROVED not to begin a PS-layer boundary start code, so
/// a later call over the same buffer (grown at the tail) may resume there
/// instead of re-scanning the payload from the PES header. When the scan runs
/// off the end, the last two bytes are NOT proved: a `00 00 01` prefix can
/// straddle the next feed's boundary by up to two bytes.
fn find_ps_boundary(data: &[u8], from: usize) -> (Option<usize>, usize) {
let mut pos = from;
while let Some(sc) = find_start_code(data, pos) {
if sc + 3 >= data.len() {
return None;
// A start code whose ID byte has not arrived yet: undecided, so
// the next scan must look at it again.
return (None, sc);
}
let id = data[sc + 3];
if id == PACK_HEADER_ID
@@ -392,11 +454,11 @@ fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
|| id == PROGRAM_END_ID
|| is_pes_stream_id(id)
{
return Some(sc);
return (Some(sc), sc);
}
pos = sc + 4;
}
None
(None, data.len().saturating_sub(2).max(from))
}
/// Check whether a start code byte is a valid PES stream ID that carries payload.
@@ -1752,7 +1814,191 @@ mod tests {
/// after it — exactly the tail a real feed can end on.
#[test]
fn find_ps_boundary_handles_a_bare_start_code_at_the_buffer_head() {
assert_eq!(find_ps_boundary(&[0x00, 0x00, 0x01], 0), None);
assert_eq!(
find_ps_boundary(&[0x00, 0x00, 0x01], 0),
(None, 0),
"an undecided trailing start code is not proved boundary-free"
);
}
/// An unbounded (length-0) PES is terminated by the next PS-LAYER unit, and
/// until one arrives the payload accumulates in the buffer. The search for
/// that unit must not restart at the PES header on every feed: the buffer
/// only stops growing at `MAX_PS_BUFFER` (4 MiB), and a feed is one read
/// batch (at most 510 sectors ≈ 1 MiB, 60 sectors ≈ 120 KiB on an optical
/// drive), so re-scanning from byte 0 costs work quadratic in the bytes
/// fed — up to 4 MiB of scanning per call, for as long as a corrupt or
/// crafted VOB withholds the boundary. A conformant DVD ends every pack
/// within 2048 bytes and never reaches this state.
///
/// Measured directly, because a work bound has no packet-level shadow:
/// `boundary_bytes_scanned` counts the bytes `find_ps_boundary` examines.
/// 256 chunks x 4 KiB of boundary-free payload is 1 MiB of input;
/// re-scanning from the header on every call examines
/// 4 KiB * 256*257/2 = ~128 MiB.
///
/// Mutation: drop the `Some((pes_at, searched_to)) if pes_at == sc` arm so
/// `from` is always `sc + 4`.
#[test]
fn an_unterminated_pes_is_not_rescanned_from_its_header_every_feed() {
const CHUNKS: usize = 256;
const CHUNK: usize = 4096;
let mut demuxer = PsDemuxer::new();
// Unbounded PES header (length 0), then payload that carries no start
// code at all, so no PS-layer boundary is ever found.
demuxer.feed(&[0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00]);
for _ in 0..CHUNKS {
assert!(
demuxer.feed(&[0xFFu8; CHUNK]).is_empty(),
"no boundary yet, so no PES can be emitted"
);
}
let fed = (CHUNKS * CHUNK) as u64;
assert!(
demuxer.boundary_bytes_scanned <= 2 * fed,
"boundary search examined {} bytes over {fed} bytes of payload — \
the scan must advance with the buffer, not restart at the PES header",
demuxer.boundary_bytes_scanned
);
// ...and the cursor must not have cost correctness: the PES still ends
// at the pack header that finally arrives, with its whole payload.
let pack = [
0x00,
0x00,
0x01,
PACK_HEADER_ID,
0x44,
0x00,
0x04,
0x00,
0x04,
0x01,
0x00,
0x00,
0x03,
0xF8,
];
let packets = demuxer.feed(&pack);
assert_eq!(packets.len(), 1, "the pack header terminates the PES");
assert_eq!(
packets[0].data.len(),
CHUNKS * CHUNK,
"the whole accumulated payload belongs to the PES"
);
}
/// The resume cursor is a BUFFER offset, so it must be rebased when the
/// buffer drains — and this is the only test in which a drain actually
/// happens while a cursor is live.
///
/// Mutations this catches, both halves of
/// `self.pending_scan.map(|(pes_at, searched_to)| (pes_at - pos, searched_to - pos))`
/// in `extract_packets`:
/// * `searched_to - pos` -> `searched_to`: the next call resumes `pos`
/// bytes PAST where the previous scan actually stopped, so that window
/// is never examined. Here the terminating pack header lands inside it
/// and is missed outright — the unbounded PES runs on past its real
/// end, swallowing the following unit. That is a CORRECTNESS failure,
/// not a slow path, and the assertion on the emitted packet catches it.
/// * `pes_at - pos` -> `pes_at`: the stale offset no longer equals the
/// PES's post-drain `sc`, the resume arm stops matching and the search
/// restarts at the PES header. Caught by `boundary_bytes_scanned`,
/// which is why the fixture puts 64 KiB of payload in the SAME chunk
/// that opens the PES: that is exactly the span a restart re-examines,
/// so the mutant roughly doubles the bytes scanned.
///
/// `an_unterminated_pes_is_not_rescanned_from_its_header_every_feed` cannot
/// reach either: it opens the unbounded PES as the very FIRST bytes of the
/// very first feed, so nothing ever drains ahead of it, `pos` stays 0 and
/// the subtraction is a no-op. The comment above it asserts neither
/// component can underflow; nothing exercised the arithmetic at all.
///
/// So this fixture puts COMPLETE PS units — a pack header and a
/// length-bounded PES — ahead of the unbounded video PES *in the same
/// chunk*. The loop consumes them, breaks on the unbounded PES, and drains
/// `pos` bytes with `pending_scan` live: exactly the real DVD shape, where
/// a video PES opens partway through a read batch.
#[test]
fn a_resume_cursor_survives_the_drain_of_units_ahead_of_the_unbounded_pes() {
// Payload fed in the SAME chunk that opens the PES. Large enough that
// re-scanning it is unmistakable in `boundary_bytes_scanned`, and it is
// the exact span the un-rebased `pes_at` mutant re-examines.
const PAYLOAD: usize = 64 * 1024;
// A 14-byte MPEG-2 pack header with pack_stuffing_length 0.
const PACK: [u8; 14] = [
0x00,
0x00,
0x01,
PACK_HEADER_ID,
0x44,
0x00,
0x04,
0x00,
0x04,
0x01,
0x00,
0x00,
0x03,
0xF8,
];
// A length-BOUNDED PES — a complete unit, so the loop consumes it and
// `pos` advances past it before breaking on the unbounded PES.
const BOUNDED_PES: [u8; 11] = [
0x00, 0x00, 0x01, 0xE0, 0x00, 0x05, 0x80, 0x00, 0x00, 0xAA, 0xBB,
];
// The unbounded (length-0) video PES whose scan must be resumed.
const OPEN_PES: [u8; 9] = [0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00];
// Bytes drained ahead of the unbounded PES on the first feed — the
// `pos` the cursor must be rebased by.
const DRAINED: usize = PACK.len() + BOUNDED_PES.len();
let mut demuxer = PsDemuxer::new();
let mut first = PACK.to_vec();
first.extend_from_slice(&BOUNDED_PES);
first.extend_from_slice(&OPEN_PES);
first.extend_from_slice(&[0xFFu8; PAYLOAD]);
let head = demuxer.feed(&first);
assert_eq!(
head.len(),
1,
"the bounded PES ahead of the open one is emitted immediately, \
which is what makes the buffer drain with a cursor live"
);
// The terminating pack arrives at the head of the next feed — i.e.
// within `DRAINED` bytes of where the previous scan stopped, which is
// precisely the window an un-rebased `searched_to` skips over.
assert!(
PACK.len() <= DRAINED,
"the terminating pack must fit inside the window a stale \
`searched_to` would skip, or the mutant survives"
);
let packets = demuxer.feed(&PACK);
assert_eq!(
packets.len(),
1,
"the pack header terminates the open PES; a scan resumed past it \
never sees it and the PES runs on"
);
assert_eq!(
packets[0].data.len(),
PAYLOAD,
"exactly the payload fed belongs to the PES"
);
// Work bound: the payload is proved boundary-free ONCE. Re-scanning it
// after the drain roughly doubles this.
assert!(
demuxer.boundary_bytes_scanned <= (PAYLOAD + 1024) as u64,
"boundary search examined {} bytes over {PAYLOAD} bytes of payload — \
a cursor left un-rebased across the drain never matches the PES's \
new offset, so the scan restarts at the header",
demuxer.boundary_bytes_scanned
);
}
/// The boundary-ID check is a 4-way `||`; a mutant that turns the FIRST
@@ -1763,7 +2009,7 @@ mod tests {
let data = [0x00, 0x00, 0x01, PACK_HEADER_ID, 0xAA];
assert_eq!(
find_ps_boundary(&data, 0),
Some(0),
(Some(0), 0),
"a pack header start code alone must register as a PS-layer boundary"
);
}
+5 -2
View File
@@ -1521,8 +1521,11 @@ fn forensic_clip_extents(
tracing::warn!(target: "freemkv::keysource", "fmts: more than one forensic clip on the disc — segment byte space is ambiguous");
return Ok(None);
}
// Addressing variant: these extents are a byte-space map for the forensic
// segment table (`clip_byte_to_lba`), not a read plan — an unrecorded
// extent must stay in place here or every later segment offset shifts.
let exts: Vec<crate::disc::Extent> = udf
.file_extents(reader, &format!("/BDMV/STREAM/{name}"))
.file_extents_addressing(reader, &format!("/BDMV/STREAM/{name}"))
.map_err(io::Error::from)?
.into_iter()
.filter(|&(lba, sectors)| lba > 0 && sectors > 0)
@@ -4440,7 +4443,7 @@ mod tests {
"00001.fmts",
20,
FMTS_CONTENT_LBA - PART_START,
FMTS_CONTENT_SECTORS * 2048,
u64::from(FMTS_CONTENT_SECTORS) * 2048,
true,
)],
subdirs: Vec::new(),
+65 -11
View File
@@ -151,14 +151,25 @@ struct TrackPos {
impl SeamPlan {
/// Build a plan from a title's clips, or `None` when there is nothing to
/// place: no clips (DVD, HD-DVD, `mkv://`/`m2ts://` sources — none of which
/// carry PlayItem marks), a single clip (nothing to join), or marks that are
/// not usable (a zero/inverted span means the playlist is not telling us
/// anything we can act on, and guessing is what this type exists to avoid).
/// carry PlayItem marks), or marks that are not usable (a zero/inverted span
/// means the playlist is not telling us anything we can act on, and guessing
/// is what this type exists to avoid).
///
/// A **single** clip still gets a plan. Joining is not the marks' only job —
/// trimming to `[in, out]` is — and it matters whenever a clip's physical
/// extent runs past its OUT mark. Real discs author trailing audio (a fade
/// after the last video frame) beyond OUT in the m2ts; without a plan the
/// inference path keeps it, leaving audio seconds past the declared duration.
/// One clip needs no cross-clip placement, so the loop below reduces to the
/// `[in, out]` drop filter plus the standard `offset = −in_ns` rebase — the
/// same one clip 0 of a multi-clip title already gets, which the MKV muxer
/// then re-anchors, so every KEPT frame is byte-identical to the no-plan
/// path. The only change is that out-of-mark frames are now dropped.
///
/// Returning `None` leaves [`TimelineContinuity`] on its PTS-jump inference,
/// which is what every non-BD path has always used.
pub(crate) fn from_clips(clips: &[crate::disc::Clip]) -> Option<Self> {
if clips.len() < 2 {
if clips.is_empty() {
return None;
}
// Trust the spans only if they tile the feed contiguously from 0.
@@ -730,9 +741,10 @@ impl TimelineContinuity {
/// Corrector driven by a title's PlayItem marks where they exist.
///
/// Falls back to [`Self::new`]'s inference when the title has fewer than two
/// clips or its marks are unusable — so DVD, HD-DVD, `mkv://` and `m2ts://`
/// sources behave exactly as before.
/// Falls back to [`Self::new`]'s inference when the title has no clips or
/// its marks are unusable — so DVD, HD-DVD, `mkv://` and `m2ts://` sources
/// behave exactly as before. A single BD clip DOES get a plan: its marks
/// still trim trailing/pre-roll material outside `[in, out]`.
pub(crate) fn with_clips(
clips: &[crate::disc::Clip],
content_format: crate::disc::ContentFormat,
@@ -2226,10 +2238,6 @@ mod tests {
#[test]
fn no_seam_plan_without_usable_marks() {
assert!(SeamPlan::from_clips(&[]).is_none(), "no clips");
assert!(
SeamPlan::from_clips(&seamless_branching_clips()[..1]).is_none(),
"a single clip has nothing to join"
);
let mut bad = seamless_branching_clips();
bad[3].out_time = bad[3].in_time; // zero-length span
assert!(
@@ -2238,6 +2246,52 @@ mod tests {
);
}
/// A SINGLE-clip title still gets a plan: joining is not the marks' only job,
/// trimming to `[in, out]` is. Real discs author trailing audio (a fade after
/// the last video frame) PAST the OUT mark in the m2ts; the no-plan inference
/// path kept it, leaving audio seconds past the declared duration — the
/// `The Bourne Supremacy` defect (audio +35.6 s past a single-clip title's
/// end). The plan drops it, and every KEPT frame is placed exactly as the
/// no-plan path would (raw rebased by `−in_ns`, which the MKV muxer already
/// does), so a disc with no out-of-mark content is byte-identical.
#[test]
fn single_clip_trims_content_outside_its_marks() {
let clips = seamless_branching_clips(); // clip 0: in 4199.0s, out 6033.04s
let mut plan = SeamPlan::from_clips(&clips[..1]).expect("a single clip is planned");
let in_ns = mpls_ticks_to_ns(clips[0].in_time);
let out_ns = mpls_ticks_to_ns(clips[0].out_time);
// In-mark frames are kept and rebased to 0 at IN — and drop nothing.
assert_eq!(plan.place(in_ns, 0, true, None), Some(0), "IN rebases to 0");
assert_eq!(
plan.place(in_ns + 10 * S, 1, false, None),
Some(10 * S),
"a mid-clip audio frame keeps its offset from IN"
);
assert_eq!(
plan.place(out_ns, 2, false, None),
Some(out_ns - in_ns),
"a frame AT the OUT mark is inside and kept"
);
assert_eq!(
plan.dropped_total(),
0,
"no in-mark frame may be dropped (output is unchanged for a clean disc)"
);
// Trailing audio past OUT is dropped AND counted, so the finish() gates
// in the sinks can see the volume.
assert!(
plan.place(out_ns + 30 * S, 1, false, None).is_none(),
"trailing audio 30s past OUT must be dropped, not emitted past the end"
);
assert!(
plan.dropped_for(1) >= 1,
"the dropped tail frame must be counted"
);
}
/// Without a plan, `map` is exactly the old `adjust` and never drops.
#[test]
fn map_without_a_plan_is_the_old_behaviour() {
+200 -9
View File
@@ -85,6 +85,12 @@ pub struct PrefetchedSectorSource {
///
/// [`capacity_sectors`]: SectorSource::capacity_sectors
total_sectors: u32,
/// Latched the moment a terminal error crosses the channel. The
/// producer NEVER resumes after sending one (every error arm
/// `return`s), so the closed channel that follows is a dead source,
/// not end-of-stream — and `read_sectors` must keep saying so instead
/// of answering `Ok(0)` for the rest of the title.
producer_failed: bool,
}
impl PrefetchedSectorSource {
@@ -300,6 +306,22 @@ impl PrefetchedSectorSource {
return;
}
let sectors_read = (n / 2048) as u32;
// A genuine zero-byte read with no error
// would otherwise spin this loop forever.
// It is not end-of-stream either: the
// extent list still has `remaining`
// sectors to serve, so the inner source
// has quit early. Send a terminal
// sentinel — dropping `tx` here instead
// would reach the consumer as a clean EOF
// and finalize a TRUNCATED title as
// success, exactly as the panic sentinel
// below exists to prevent.
if sectors_read == 0 {
let _ =
tx.send(Err(crate::error::Error::SourceTerminated.into()));
return;
}
buf.truncate(n);
bytes_read_total = bytes_read_total.saturating_add(n as u64);
if let Some(ref f) = event_fn {
@@ -313,12 +335,6 @@ impl PrefetchedSectorSource {
if tx.send(Ok(buf)).is_err() {
return; // consumer dropped
}
// A genuine zero-byte read with no error would
// otherwise spin this loop forever; treat it
// as end-of-source.
if sectors_read == 0 {
return;
}
offset = offset.saturating_add(sectors_read);
}
Err(e) => {
@@ -344,6 +360,7 @@ impl PrefetchedSectorSource {
recycle_tx,
producer: Some(producer),
total_sectors,
producer_failed: false,
})
}
@@ -486,8 +503,29 @@ impl SectorSource for PrefetchedSectorSource {
let _ = self.recycle_tx.send(filled);
Ok(n)
}
Ok(Err(e)) => Err(crate::error::Error::IoError { source: e }),
// Channel closed (producer finished or panicked).
// Recover the producer's TYPED error rather than blanket-wrapping
// it as `Error::IoError`. That wrapper is a classification, not a
// container: `is_scsi_transport_failure` matches `IoError`, so a
// wrapped MEDIUM ERROR bad sector reached `fill_extents` looking
// like a wedged bridge and aborted the pass instead of being
// skipped under `skip_errors`. `From<io::Error> for Error`
// downcasts the boxed payload back to the exact variant the
// producer sent (status + sense intact); a genuine OS-level
// `io::Error` — the real dead-bus case — still becomes `IoError`.
Ok(Err(e)) => {
// The producer `return`s after every error it sends, so
// this is also the moment the source dies. Latch it: the
// closed channel that follows must not read as EOF.
self.producer_failed = true;
Err(crate::error::Error::from(e))
}
// Channel closed. Clean EOF only if the producer never
// signalled a failure — otherwise it exited without
// delivering the rest of the extents, and answering `Ok(0)`
// would let `fill_extents` mistake a dead source for a short
// read, zero-fill every remaining sector of the title and
// still report the pass as complete.
Err(_) if self.producer_failed => Err(crate::error::Error::SourceTerminated),
Err(_) => Ok(0),
}
}
@@ -1214,7 +1252,12 @@ mod tests {
/// an error (not Ok(0)/EOF), and its ErrorKind must survive the
/// round-trip through the channel. Grounding: the producer's
/// `Err(e) => tx.send(Err(e.into()))` arm, and `read_sectors`'
/// `Ok(Err(e)) => Err(IoError{source:e})`.
/// `Ok(Err(e)) => { self.producer_failed = true; Err(Error::from(e)) }`.
///
/// That arm recovers the producer's TYPED error by downcast rather than
/// blanket-wrapping it as `Error::IoError`, so the kind survives; the
/// `producer_failed` latch it also sets is what turns the channel close
/// that follows into `SourceTerminated` instead of a clean EOF.
#[test]
fn reader_error_propagates_with_kind() {
with_watchdog(Duration::from_secs(10), || {
@@ -1235,6 +1278,63 @@ mod tests {
});
}
/// An inner source that answers a mid-extent read with `Ok(0)` has quit
/// early: the extent list still has sectors to serve. The producer must
/// say so, not simply drop `tx` — a closed channel reads as clean
/// end-of-stream, and `DiscStream::fill_extents` then fabricates zeros
/// for every remaining sector of the title and reports the pass
/// complete. Same rule as the panic sentinel: a truncated title must
/// never be finalized as success.
struct QuitsEarlySource;
impl SectorSource for QuitsEarlySource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Ok(0)
}
fn capacity_sectors(&self) -> u32 {
9
}
}
#[test]
fn inner_source_quitting_early_is_not_reported_as_end_of_stream() {
with_watchdog(Duration::from_secs(10), || {
let extents = vec![Extent {
start_lba: 0,
sector_count: 9,
}];
let mut pf =
PrefetchedSectorSource::new(QuitsEarlySource, extents, 3, None).expect("spawn");
let mut buf = vec![0u8; 3 * 2048];
let mut last = pf.read_sectors(0, 3, &mut buf, false);
// Whatever the first answer, no call may ever settle on a clean
// `Ok(0)`: 9 sectors were promised and none were delivered.
for _ in 0..4 {
if last.is_err() {
break;
}
assert_eq!(
*last.as_ref().unwrap(),
0,
"the source delivered no bytes, so nothing can be Ok(n>0)"
);
last = pf.read_sectors(0, 3, &mut buf, false);
}
let err =
last.expect_err("an undelivered extent list must surface as an error, not as EOF");
assert!(
err.is_source_terminated(),
"the source is gone for good — retrying or skipping cannot \
recover anything; got {err:?}"
);
});
}
/// A read returning a byte count that is not a whole number of
/// sectors (n % 2048 != 0) must be rejected — never truncated and
/// advanced, which would split a sector and hand decrypt a partial
@@ -1472,4 +1572,95 @@ mod tests {
assert_eq!(got.len(), 30 * 2048, "all 10 extents must be drained");
});
}
/// Source whose every read fails with a `DiscRead` carrying the given
/// SCSI status (and optional sense) — an ordinary MEDIUM ERROR bad
/// sector (0x02 + 03/11/00) or the transport-failure sentinel (0xFF).
struct FailingSource {
status: u8,
sense: Option<crate::scsi::ScsiSense>,
}
impl SectorSource for FailingSource {
fn capacity_sectors(&self) -> u32 {
9999
}
fn read_sectors(
&mut self,
lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(self.status),
sense: self.sense,
})
}
}
fn read_one_err(status: u8, sense: Option<crate::scsi::ScsiSense>) -> crate::error::Error {
let extents = vec![Extent {
start_lba: 100,
sector_count: 9,
}];
let mut pf = PrefetchedSectorSource::new(FailingSource { status, sense }, extents, 3, None)
.expect("spawn");
let mut buf = vec![0u8; 3 * 2048];
pf.read_sectors(100, 3, &mut buf, false)
.expect_err("the producer's read failure must surface")
}
/// REGRESSION: an ordinary MEDIUM ERROR bad sector that crosses the
/// prefetch channel must NOT be classified as a SCSI transport failure.
///
/// `read_sectors` used to re-wrap EVERY channel error as
/// `Error::IoError { source }`, and `is_scsi_transport_failure` matches
/// `IoError` (the wedged-USB-bridge arm). So a skippable bad sector
/// arrived at `DiscStream::fill_extents` looking like a dead bus and
/// aborted the whole pass instead of honouring `skip_errors`.
#[test]
fn bad_sector_across_channel_is_not_a_transport_failure() {
with_watchdog(Duration::from_secs(10), || {
let sense = crate::scsi::ScsiSense {
sense_key: 0x03,
asc: 0x11,
ascq: 0x00,
};
let err = read_one_err(crate::scsi::SCSI_STATUS_CHECK_CONDITION, Some(sense));
assert!(
!err.is_scsi_transport_failure(),
"a MEDIUM ERROR bad sector must stay a bad sector across the \
prefetch channel, got {err:?}"
);
// The classification survives because the typed variant does.
assert!(
matches!(err, crate::error::Error::DiscRead { status: Some(s), .. } if s == 0x02),
"expected the producer's DiscRead to survive the channel, got {err:?}"
);
assert_eq!(
err.scsi_sense().map(|s| (s.sense_key, s.asc, s.ascq)),
Some((0x03, 0x11, 0x00)),
"the drive's sense triple must survive the channel"
);
});
}
/// OPPOSITE-DIRECTION CONTROL: a genuine transport failure (status 0xFF,
/// wedged USB bridge) crossing the same channel MUST still classify as a
/// transport failure, so `fill_extents` / sweep keep aborting the pass
/// instead of zero-filling every read against a dead bus.
#[test]
fn transport_failure_across_channel_still_classifies() {
with_watchdog(Duration::from_secs(10), || {
let err = read_one_err(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None);
assert!(
err.is_scsi_transport_failure(),
"a 0xFF transport failure must remain one across the prefetch \
channel, got {err:?}"
);
});
}
}
+226
View File
@@ -0,0 +1,226 @@
//! Test-only capture of `tracing` events, so the crate's logging contract is
//! ENFORCED rather than merely commented.
//!
//! # Why this exists
//!
//! The error contract is "Account / Log / Classify", and the round-2 audit
//! found the third leg unverifiable: three separate sites carry long comments
//! insisting they log **the error's OWN code, not a fixed one** — because
//! flattening a scratched sector (E6000) or an over-long allocation-descriptor
//! chain (E6016) into E6017 sends whoever triages them after authoring holes
//! and hides the population that actually exists. Nothing tested that. Putting
//! a literal back at `bluray.rs`'s or `hddvd.rs`'s warn sites broke no test, so
//! the guarantee was a convention one careless edit away from being false. Two
//! of those very sites were changed in round 1, and a third still carried a
//! hardcoded `code = 6017`.
//!
//! Likewise "absence of a log is itself a bug": a refusal that returns the
//! right error but says nothing produces the wrong population downstream (a
//! residual-underrunning drive is indistinguishable from a scratched disc).
//! That is only checkable by looking at what was emitted.
//!
//! # Why a hand-rolled subscriber and not `tracing-subscriber`
//!
//! Same posture as [`crate::harness`]: this crate has exactly one
//! dev-dependency on purpose. `tracing-subscriber` would pull a tree of them to
//! do what forty lines of [`tracing::Subscriber`] does here. The capture is
//! installed with [`tracing::subscriber::with_default`], which is
//! THREAD-LOCAL — so it composes with `cargo test`'s parallel harness and two
//! capturing tests cannot see each other's events.
//!
//! Field values are stringified through [`std::fmt::Debug`]/`Display` because
//! that is all the visitor API offers without a typed schema; tests compare
//! against the string form of the expected constant, which is exactly the
//! comparison that catches a hardcoded code.
#![cfg(test)]
use std::sync::{Arc, Mutex};
/// One captured `tracing` event: its target, level, message and fields.
#[derive(Debug, Clone)]
pub(crate) struct CapturedEvent {
pub target: String,
pub level: tracing::Level,
/// Every field, in emission order, stringified. The implicit `message`
/// field (the format string) is included under the name `message`.
pub fields: Vec<(String, String)>,
}
impl CapturedEvent {
/// The stringified value of `name`, or `None` if the event has no such
/// field.
pub fn field(&self, name: &str) -> Option<&str> {
self.fields
.iter()
.find(|(k, _)| k == name)
.map(|(_, v)| v.as_str())
}
/// The event's message (the `tracing` format string), or `""`.
pub fn message(&self) -> &str {
self.field("message").unwrap_or("")
}
}
#[derive(Default)]
struct Visitor(Vec<(String, String)>);
impl tracing::field::Visit for Visitor {
fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) {
self.0
.push((field.name().to_string(), format!("{value:?}")));
}
fn record_str(&mut self, field: &tracing::field::Field, value: &str) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_u64(&mut self, field: &tracing::field::Field, value: u64) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_i64(&mut self, field: &tracing::field::Field, value: i64) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_bool(&mut self, field: &tracing::field::Field, value: bool) {
self.0.push((field.name().to_string(), value.to_string()));
}
}
type Sink = Arc<Mutex<Vec<CapturedEvent>>>;
thread_local! {
/// The sink for a capture ACTIVE ON THIS THREAD, if any. Thread-local so
/// concurrent captures across `cargo test`'s parallel harness never see
/// each other's events, and so a non-capturing thread simply has `None`.
static SINK: std::cell::RefCell<Option<Sink>> = const { std::cell::RefCell::new(None) };
}
/// The ONE process-wide subscriber. Installed once and left installed; it is
/// offered every event and records into whichever thread's sink is active,
/// dropping the event when none is.
struct Capture;
impl tracing::Subscriber for Capture {
// `sometimes`, deliberately NOT the default `always`/`never`. A cacheable
// interest lets `tracing`'s GLOBAL per-callsite cache short-circuit a
// callsite to "off", and under `cargo test`'s parallel harness a cache
// rebuild triggered by ANY other thread can leave it there while a capture
// is live — the capture then observes NOTHING. That is the race that made
// `parse_playlist_unreadable_clip_icb_yields_no_title` flake ~1 run in 15.
// `sometimes` forces `enabled` to be consulted on the emitting thread for
// every event, so a capture always sees its own.
fn register_callsite(
&self,
_meta: &'static tracing::Metadata<'static>,
) -> tracing::subscriber::Interest {
tracing::subscriber::Interest::sometimes()
}
fn enabled(&self, _metadata: &tracing::Metadata<'_>) -> bool {
SINK.with(|s| s.borrow().is_some())
}
// Spans are irrelevant here — nothing in this crate asserts on span
// structure, only on events — so they get a constant id and no storage.
fn new_span(&self, _span: &tracing::span::Attributes<'_>) -> tracing::span::Id {
tracing::span::Id::from_u64(1)
}
fn record(&self, _span: &tracing::span::Id, _values: &tracing::span::Record<'_>) {}
fn record_follows_from(&self, _span: &tracing::span::Id, _follows: &tracing::span::Id) {}
fn event(&self, event: &tracing::Event<'_>) {
SINK.with(|s| {
if let Some(sink) = s.borrow().as_ref() {
let mut v = Visitor::default();
event.record(&mut v);
let meta = event.metadata();
sink.lock().expect("capture mutex").push(CapturedEvent {
target: meta.target().to_string(),
level: *meta.level(),
fields: v.0,
});
}
});
}
fn enter(&self, _span: &tracing::span::Id) {}
fn exit(&self, _span: &tracing::span::Id) {}
}
/// Install the single global capturing subscriber, exactly once.
fn install() {
static INSTALLED: std::sync::OnceLock<()> = std::sync::OnceLock::new();
INSTALLED.get_or_init(|| {
// This is the crate's ONLY `set_global_default`; a second call cannot
// happen. Tolerate it via `.ok()` rather than panic if that ever
// changes — capture then no-ops, which the non-empty assertions catch.
let _ = tracing::subscriber::set_global_default(Capture);
});
}
/// Run `f` with every `tracing` event it emits ON THIS THREAD captured.
///
/// Returns `f`'s value alongside the events, in emission order.
///
/// # Why one global subscriber, not scoped `with_default`
///
/// `tracing`'s per-callsite INTEREST CACHE is GLOBAL, but `with_default` is
/// thread-local. Under `cargo test`'s parallel harness a rebuild of that cache
/// — triggered by ANY other thread registering any callsite for the first time
/// — re-evaluates the target callsite against the global dispatcher, which a
/// scoped subscriber is NOT part of, and can leave it cached "off" while a
/// capture is live, so the capture observes NOTHING. Serialising captures
/// against each other does not help: the poisoning thread is not itself
/// capturing. `parse_playlist_unreadable_clip_icb_yields_no_title` flaked ~1
/// run in 15 on exactly this — an empty event list.
///
/// The robust shape is a single subscriber installed globally for the whole
/// run, whose `register_callsite` returns `sometimes` (so no callsite is ever
/// hard-cached) and which routes each event to the emitting thread's own sink.
/// No scoped-dispatcher transitions, no cross-thread cache race, and concurrent
/// captures on different threads stay isolated by the thread-local sink.
pub(crate) fn capture<T>(f: impl FnOnce() -> T) -> (T, Vec<CapturedEvent>) {
install();
let sink: Sink = Arc::default();
// Save/restore any outer sink so a nested capture on one thread still works.
let prev = SINK.with(|s| s.borrow_mut().replace(sink.clone()));
let out = f();
SINK.with(|s| *s.borrow_mut() = prev);
let events = std::mem::take(&mut *sink.lock().expect("capture mutex"));
(out, events)
}
#[cfg(test)]
mod tests {
use super::*;
/// The capture must actually see events and their field VALUES — if it
/// silently recorded nothing, every logging assertion built on it would
/// pass vacuously, which is worse than having no harness at all.
///
/// Mutation: an `enabled()` returning `false`, or an `event()` that drops
/// the visitor's fields, fails here.
#[test]
fn capture_records_target_level_and_fields() {
let ((), events) = capture(|| {
tracing::warn!(target: "freemkv::testlog", code = 6017u16, clip = ?"A.EVO", "E6017");
});
assert_eq!(events.len(), 1, "exactly one event: {events:?}");
assert_eq!(events[0].target, "freemkv::testlog");
assert_eq!(events[0].level, tracing::Level::WARN);
assert_eq!(events[0].field("code"), Some("6017"));
assert_eq!(events[0].field("clip"), Some("\"A.EVO\""));
assert_eq!(events[0].message(), "E6017");
}
/// A field that is absent must read as `None`, not as an empty string — an
/// assertion of the shape `field("code") == Some(..)` has to be able to
/// fail when the site stops logging the code at all.
#[test]
fn missing_field_is_none_and_capture_is_scoped() {
let ((), events) = capture(|| tracing::warn!(target: "freemkv::testlog", "no fields"));
assert_eq!(events[0].field("code"), None);
// Emitted outside `capture`, so it must not appear in a later capture.
tracing::warn!(target: "freemkv::testlog", code = 1u16, "outside");
let ((), later) = capture(|| {});
assert!(
later.is_empty(),
"capture is scoped to its closure: {later:?}"
);
}
}
+770 -133
View File
File diff suppressed because it is too large Load Diff