Compare commits

...
19 Commits
Author SHA1 Message Date
Matthew Jackson 231cbf3d76 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 21c7b12e2b docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 799039df21 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
Matthew Jackson 497b7726dc v1.6.6: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 20:03:44 -07:00
Matthew Jackson 94b549a9d8 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson fc2d35a19c v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew Jackson 3c51190c32 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson dfe0390445 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 8678a2bf94 drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson d953b46023 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 443ac6bcb1 Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 794393f24e CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson c6e2614c4c Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson b02c402dae CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 128e3bb120 Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson dc100f1c80 Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 286b5d4527 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson d9f3f82c65 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson e30209091e restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
32 changed files with 7606 additions and 699 deletions
+15
View File
@@ -46,3 +46,18 @@ updates:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+132
View File
@@ -1,5 +1,137 @@
# Changelog
## [1.6.7] — 2026-08-21
### Changed
- Version aligned to 1.6.7 for the unified release. No functional changes to
this crate; the release was driven by autorip (per-webhook event selection,
a progress bar per moved artifact, and move-queue / webhook-error fixes —
see the autorip 1.6.7 notes).
## [1.6.6] — 2026-08-20
### Changed
- Version aligned to 1.6.6 for the unified release. No functional changes
to this crate; the release was driven by autorip (webhooks may now target
private/LAN addresses — see the autorip 1.6.6 notes).
## [1.6.5] — 2026-08-20
### Fixed
- **On some Blu-ray/UHD titles the picture stopped short of the declared
end.** When a title is stitched across a clip seam, freemkv drops video
after the join until it sees a fresh keyframe it can restart decoding
from. It only recognised one kind of keyframe (an IDR frame), but a
Blu-ray's final segment can open on a different kind of self-contained
frame instead — so on those discs the restart point was never
recognised and every remaining frame to the end of the film was
dropped, losing up to a group-of-pictures-plus tail of picture. That
kind of frame is now recognised as a valid restart point, matching how
the HEVC path already worked, so the tail is kept. HEVC and DVD titles
were never affected.
- **A split HD-DVD feature could be exported as half the film while still
claiming the full running time.** A feature stored across parts (for
example `FEATURE_1.EVO` + `FEATURE_2.EVO`) is composed back into one
title. If a part resolved with no usable data — a zero-length file, or
an extent map that yielded nothing — that part was neither used nor
flagged, so the feature was quietly composed from the surviving parts
alone, still advertising the whole runtime, and exited success with no
log. The missing part is now marked unusable and logged under its own
code (E6019), so half a movie can no longer be presented as a whole one.
- **A Blu-ray title with an unreadable clip could be exported short while
still reporting the full running time.** A scratched or malformed
clip-info (`.clpi`) sector, or a clip whose extents could not be
resolved (a bad sector, a broken allocation chain, an embedded-data
file), was silently skipped — but the title's duration had already been
counted from the playlist, so it shipped short of the runtime it claimed
at exit success, with nothing logged. Previously only one narrow failure
kind counted; now every unresolvable clip drops the title and warns with
the read's own error code. A truly absent optional file (such as the 3D
`.ssif` on a 2D disc) is still treated as benign.
- **Pressing Stop during a disc scan could leave the disc looking like it
holds fewer titles than it does — or none at all.** Once cancelled,
every remaining drive command fails, but the Blu-ray and DVD title
enumerators treated those failures as ordinary skips: the Blu-ray scan
returned a truncated title list and the DVD scan returned zero titles,
both at exit success, so a cancelled scan was indistinguishable from a
disc that simply held that many titles. A cancel is now propagated as an
error out of every enumerator and at every read site, so it can never be
cached, displayed, or ripped from as if it were the real disc. HD-DVD
already behaved this way and is the model the others now follow.
- **A decrypted HD-DVD could be refused with a DVD copy-protection error
(E7023).** HD-DVD and DVD share the same MPEG program-stream container,
and the scramble detector mistook an HD-DVD navigation packet for a
CSS-scrambled sector — so a good, already-decrypted HD-DVD was reported
as carrying an unrecoverable CSS key and hard-failed, sending anyone
triaging it hunting for a missing DVD key on a disc that never had one.
The detector now excludes the structural packet types that CSS never
scrambles, identified from a field that is readable even on ciphertext,
so a decrypted HD-DVD scans clean. The DVD CSS crack is unchanged and a
genuinely scrambled, uncrackable DVD still hard-fails as before, so
ciphertext can never be muxed as plaintext.
- **A file with an allocated-but-never-written region could splice
undefined sectors into the rip.** The UDF reader treated an
allocated-but-unrecorded extent as ordinary content and read whatever
happened to be on those sectors into the output, and it decoded
embedded-data files (which store their content inline, not as a sector
map) as if their bytes were an extent list, pointing the reader at
unrelated sectors. Such extents are now refused when they actually
occupy space, and embedded files are handled as their own case; a
legitimately zero-length file still reads as empty rather than dropping
its title.
- **Audio and subtitle tracks in many languages were all labelled
"undefined."** The language mapping recognised only a handful of codes
and collapsed fifteen others to `und`. Every ISO 639-1 language code is
now mapped, so those tracks carry their real language. The DVD subtitle
colour palette, which was written in the wrong order, is also corrected.
- **If the disc read-ahead thread died mid-rip, a whole title could be
fabricated and reported complete.** A prefetch producer that terminated
returned an end-of-data signal that the reader legitimately read as a
short read and zero-filled, so a failed read could be papered over with
zeros and the pass still reported as successful. It now reports a
distinct source-terminated error instead. Dead-bus drive faults are
likewise classified rather than flattened, so the wedged-drive recovery
path can see them, and the drive now responds to Stop during spin-up and
spin-down instead of staying deaf for up to ~30 seconds.
- **A holed extraction could climb to a clean 100% on the live progress
channel.** The progress feed hardcoded its unreadable-byte count to zero
and counted every zero-filled hole as good data, so a progress-only
consumer saw a damaged extraction finish spotless — even though the
authoritative result was already truthful. The real good/unreadable
split is now threaded through the live channel. Separately, an
unreadable HD-DVD authored clip order is now logged with its error code
instead of being silently discarded before falling back to the
per-clip heuristic.
### Security
- **Bounded several unbounded amplification axes in the HD-DVD and Blu-ray
scanners.** A crafted disc could drive the playlist nesting depth, the
title count, and the clips and chapters per title without limit, and
could force a repeated clip-name fallback probe — each of which alone
left the worst-case scan unbounded. All now carry positional caps (512
titles and clips, roughly ten times any retail disc) and the fallback
probe is memoized. No effect on a well-formed disc.
### Changed
- **How freemkv picks the per-platform drive code was consolidated, with
no change to reading a disc.** The drive layer chose its
operating-system-specific module separately inside each entry point; it
now selects that module once, so a new entry point cannot silently
forget a platform. Purely internal.
## [1.6.4] — 2026-08-15
### Fixed
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "1.6.4"
version = "1.6.7"
edition = "2024"
rust-version = "1.97"
license = "MIT"
@@ -26,7 +26,7 @@ aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.4" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.7" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
+4
View File
@@ -114,6 +114,10 @@ pub mod coding_type {
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
+243 -41
View File
@@ -75,7 +75,7 @@ pub fn crack_key(
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
crack_key_scan(reader, extents, batch_sectors, None).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -136,7 +136,7 @@ pub fn crack_key_outcome(
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
) -> CrackOutcome {
crack_key_scan(reader, extents, batch_sectors, halt, true)
crack_key_scan(reader, extents, batch_sectors, halt)
}
/// Resolve a DVD title's CSS descramble key from the reader when the caller
@@ -212,10 +212,6 @@ fn crack_key_scan(
extents: &[Extent],
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
// True only on the INITIAL scan: a fully CSS-locked (`05/6F/03`) result is a
// hard `ScrambledUncracked`. False on the per-VTS re-crack so a lapsed-AGID
// locked read returns None instead of killing a genuinely crackable title.
fail_on_locked: bool,
) -> CrackOutcome {
// Batch the reads: a live optical drive at 1 sector/read is glacial, and the
// crack only needs to FIND one scrambled sector whose 0x80 plaintext matches
@@ -301,6 +297,18 @@ fn crack_key_scan(
let usable = (got / 2048).min(n as usize);
// At least one, so a source returning Ok(0) cannot spin here.
advance = (usable as u32).max(1);
if usable == 0 {
// Nothing was inspected, so the per-sector `tried`
// charge below never runs — but the cursor still moves
// one sector (the `.max(1)` above). Charge that sector
// to the budget, or `tried` stays frozen and the loop
// is bounded only by the disc-declared
// `ext.sector_count`: the anti-grind budget stops
// applying to exactly the misbehaving source it exists
// for. Mirrors the `Err` arm's `tried += n`, which
// likewise charges an uninspected advance.
tried += 1;
}
for s in 0..usable {
tried += 1;
let sect = &buf[s * 2048..(s + 1) * 2048];
@@ -344,14 +352,29 @@ fn crack_key_scan(
}
// Budget exhausted / extents walked / early-bailed with no key recovered.
// The disc is ENCRYPTED-but-uncracked (a hard failure on the initial scan)
// when EITHER a scrambled sector was actually seen, OR — on the initial scan
// only (`fail_on_locked`) — every read was CSS-locked (`05/6F/03`), itself
// proof of scrambling. A re-crack (`fail_on_locked` false) stays soft: a
// lapsed-AGID locked read yields None, not a hard fail, so a crackable title
// in another VTS isn't killed. Only a scan that saw neither a scrambled
// sector nor a CSS-lock is genuinely unencrypted.
if saw_scrambled || (saw_locked && fail_on_locked) {
// The disc is ENCRYPTED-but-uncracked (a hard failure) when EITHER a
// scrambled sector was actually seen, OR every read was CSS-locked
// (`05/6F/03`), itself proof of scrambling. Only a scan that saw neither a
// scrambled sector nor a CSS-lock is genuinely unencrypted.
//
// A prior revision made this conditional on a caller-supplied
// `fail_on_locked: bool`, documented as "false on the per-VTS re-crack, so
// a lapsed-AGID locked read returns None instead of killing a genuinely
// crackable title." That parameter never had an observable effect: its
// ONLY non-test caller with `false` was [`crack_key`], whose `Option`
// return collapses `ScrambledUncracked` and `Unencrypted` alike to `None`
// via [`CrackOutcome::into_state`] — so the branch this comment describes
// was unreachable from the moment it was introduced (see the crate's audit
// notes for the git-archaeology). The one production caller that DOES
// observe the `Cracked` / `Unencrypted` / `ScrambledUncracked` split for a
// per-VTS re-crack (`Disc::decrypt_keys_for_title`) has always gone
// through [`crack_key_outcome`], which hardcoded this to always-hard-fail.
// Reconnecting the soft variant there would mean a locked read on a live
// drive gets reported as `Unencrypted` (`title_is_clear = true`) and the
// title is muxed with NO key — precisely the silent-garbage failure mode
// `CrackOutcome` exists to prevent. So the parameter is removed rather
// than revived: the scan is unconditionally hard-fail-on-locked.
if saw_scrambled || saw_locked {
CrackOutcome::ScrambledUncracked
} else {
CrackOutcome::Unencrypted
@@ -402,19 +425,16 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
///
/// # Errors
///
/// [`Error::DecryptFailed`] when a sector's own crib proves the cached key stale
/// and the re-crack from that same sector also fails. CSS has no external key
/// source — the title key comes only from cracking the data — so on a readable
/// sector this is not a missing input, it is recovery failing on data we can
/// see. Emitting the sector anyway means one of two bad outcomes: descrambled
/// with the key its crib just rejected, which yields garbage behind an intact
/// clear header (valid pack start, passes every structural check the PS demuxer
/// applies, corruption confined to the PES payload where nothing looks); or
/// passed through still scrambled, which is ciphertext delivered where plaintext
/// is meant to be. Both are bad data reported as success.
/// Never returns `Err` — the signature is `Result` only to match the decrypt
/// seam it is dispatched from, and the `usize` is that seam's legacy
/// always-zero loss count (see [`crate::decrypt::decrypt_sectors`]).
///
/// This matches the AACS sibling, which returns [`Error::DecryptFailed`] rather
/// than apply a neighbouring CPS unit's key.
/// This section used to document an [`Error::DecryptFailed`] for the case where
/// a sector's crib rejects the cached key and the re-crack from that sector also
/// fails. That behaviour was tried and REVERTED, for the reason set out at the
/// `None =>` arm below: crib mismatch plus crack failure is the signature of a
/// crib FALSE POSITIVE, not of a stale key, and failing there made real discs
/// unrippable. The arm descrambles with the cached key and returns `Ok`.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
// `is_scrambled_pack`, NOT the looser `is_scrambled`. The raw flag test
@@ -529,6 +549,35 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// scrambled feature is made of valid PS packs, so its scrambled sectors still
/// pass this check and still drive `ScrambledUncracked` when no key cracks.
///
/// The pack-start check alone is not enough, and the sector's PES `stream_id`
/// (offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
/// prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
/// streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
/// (`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
/// stream_id even on a scrambled sector. The MPEG-PS structural packets that
/// are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
/// private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
/// because on those the byte at 0x14 is NOT a PES scrambling-control field but
/// raw payload/structure whose bits 4-5 land set by chance.
///
/// This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
/// MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
/// private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
/// set. With no `0x11` gate those nav packs flipped the crack scan's
/// `saw_scrambled` flag on a disc that carries no CSS at all; the crack then
/// found no key (there is none) and the scan returned `ScrambledUncracked`,
/// hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
/// `0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
/// can act on (the Stevenson attack only recovers a key from a scrambled ES
/// pack), so a decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
///
/// This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
/// real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
/// private_stream_1 (`0xBD`) packs, none of which are excluded, so its
/// scrambled sectors still set `saw_scrambled` and still drive
/// `ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
/// so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
///
/// The DESCRAMBLE path gates on this same function — [`descramble_sector`] and
/// [`descramble_region`] both call it, not the raw flag test — because the raw
/// test does not merely mis-skip a sector there: it descrambles one that was
@@ -537,7 +586,14 @@ pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA];
/// lost 1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38
/// titles became 10 — silently, at exit 0. One gate, both paths.
pub fn is_scrambled_pack(sector: &[u8]) -> bool {
sector.len() >= 2048 && sector[0x00..0x04] == PACK_START && (sector[0x14] >> 4) & 0x03 != 0
use crate::consts::pes_stream_id::{PADDING_STREAM, PRIVATE_STREAM_2, SYSTEM_HEADER};
sector.len() >= 2048
&& sector[0x00..0x04] == PACK_START
&& !matches!(
sector[0x11],
SYSTEM_HEADER | PADDING_STREAM | PRIVATE_STREAM_2
)
&& (sector[0x14] >> 4) & 0x03 != 0
}
#[cfg(test)]
@@ -593,7 +649,22 @@ mod tests {
let out = descramble_region(&mut sector, &mut key)
.expect("a crib false positive must NOT fail the rip");
assert_eq!(out, 0, "CSS reports no loss term of its own");
// The "no loss term" contract belongs to the SEAM, not to this
// function: `decrypt_sectors`' `usize` is a legacy always-zero count
// that the CSS arm feeds from here. Asserting `out == 0` on
// `descramble_region` alone only restates its single `Ok(0)` return —
// a body replaced by `Ok(0)` satisfies it just as well. Assert it one
// level up, where the value is actually assembled and returned, so the
// arm dispatch and the plumbing are exercised too.
assert_eq!(out, 0);
let mut seam_sector = sector;
let mut seam_keys = crate::decrypt::DecryptKeys::Css { title_key: key };
assert_eq!(
crate::decrypt::decrypt_sectors(&mut seam_sector, &mut seam_keys, 0)
.expect("a crib false positive must NOT fail the rip at the seam either"),
0,
"CSS reports no loss term of its own through decrypt_sectors"
);
assert_eq!(
key, key_before,
"a failed re-crack must leave the cached key in place — it is still \
@@ -722,6 +793,52 @@ mod tests {
);
}
/// Detection fix: `is_scrambled_pack` must EXCLUDE the MPEG-PS structural
/// packets that CSS never scrambles — system_header (0xBB), padding (0xBE)
/// and private_stream_2 (0xBF, DVD PCI/DSI and HD-DVD `.evo` RDI nav) — by
/// their `stream_id` at offset 0x11. On those packets byte 0x14 is raw
/// payload/structure, not a PES scrambling-control field, so its bits 4-5
/// land set by chance.
///
/// This is the exact decrypted-HD-DVD defect: an `.evo` RDI pack is
/// private_stream_2 (0xBF) with the pack-start code and bits set at 0x14, so
/// the old gate flipped `saw_scrambled` on a CSS-free disc → the crack found
/// no key → `ScrambledUncracked` → E7023 on a good HD-DVD.
///
/// Grounding: `!matches!(sector[0x11], SYSTEM_HEADER | PADDING_STREAM |
/// PRIVATE_STREAM_2)`. Mutation: drop the 0x11 exclusion → the 0xBF nav pack
/// counts as scrambled evidence and the first assert fails. The video and
/// private_stream_1 packs prove the gate does NOT reject a real scrambled
/// DVD sector (the catastrophic direction).
#[test]
fn is_scrambled_pack_excludes_nav_and_structural_stream_ids() {
use crate::consts::pes_stream_id::{
PADDING_STREAM, PRIVATE_STREAM_1, PRIVATE_STREAM_2, SYSTEM_HEADER, VIDEO, VIDEO_MAX,
};
// A pack-start pack with 0x14 scramble bits set, varying only 0x11.
let mut s = vec![0u8; 2048];
s[0x00..0x04].copy_from_slice(&PACK_START);
s[0x14] = 0x30;
for excluded in [SYSTEM_HEADER, PADDING_STREAM, PRIVATE_STREAM_2] {
s[0x11] = excluded;
assert!(
!is_scrambled_pack(&s),
"stream_id {excluded:#04x} is a structural/nav pack CSS never scrambles — \
it must NOT count as scramble evidence (else a decrypted HD-DVD RDI pack → E7023)"
);
}
// A genuinely scramblable elementary-stream pack must STILL register —
// proving the exclusion did not weaken real-DVD CSS detection.
for scramblable in [VIDEO, VIDEO_MAX, PRIVATE_STREAM_1, 0xE2] {
s[0x11] = scramblable;
assert!(
is_scrambled_pack(&s),
"stream_id {scramblable:#04x} is a scramblable ES pack — a real CSS DVD's \
scrambled sector must still be counted, never passed through as plaintext"
);
}
}
// ── crack_key scanning over a mock SectorSource ────────────────────────
/// Records every (lba, count) read; returns a caller-supplied flag byte at
@@ -743,6 +860,11 @@ mod tests {
/// a damaged region. `Some(0)` is the degenerate case that must not
/// spin the scan.
short_read: Option<usize>,
/// PES `stream_id` written at offset 0x11 of each uniform-fill sector.
/// `0x00` (the default) is a scramblable-looking pack; set to
/// private_stream_2 (`0xBF`) to model an HD-DVD `.evo` RDI nav pack,
/// which carries the pack-start code and 0x14 bits but no CSS.
stream_id: u8,
}
impl MockSource {
@@ -754,6 +876,7 @@ mod tests {
lock_all: false,
crackable: None,
short_read: None,
stream_id: 0x00,
}
}
}
@@ -828,11 +951,14 @@ mod tests {
}
_ => {
// Real DVD video sectors always open with the MPEG-PS
// pack-start code; `is_scrambled_pack` (Fix 3) requires it
// before trusting the 0x14 scramble bits, so the fixture
// must include it for a `flag_byte` of 0x30 to register
// as scrambled.
// pack-start code; `is_scrambled_pack` requires it before
// trusting the 0x14 scramble bits, so the fixture must
// include it for a `flag_byte` of 0x30 to register as
// scrambled. `stream_id` (byte 0x11) defaults to 0x00 (a
// scramblable pack); an HD-DVD RDI nav pack sets it to
// private_stream_2 (0xBF), which the 0x11 exclusion drops.
buf[base..base + 4].copy_from_slice(&PACK_START);
buf[base + 0x11] = self.stream_id;
buf[base + 0x14] = self.flag_byte;
}
}
@@ -860,7 +986,7 @@ mod tests {
start_lba: 100,
sector_count: 4,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None, false);
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().clone();
assert_eq!(
reads,
@@ -882,7 +1008,7 @@ mod tests {
start_lba: 0,
sector_count: 8,
}];
let outcome = crack_key_scan(&mut src, &ext, 4, None, false);
let outcome = crack_key_scan(&mut src, &ext, 4, None);
assert!(
matches!(outcome, CrackOutcome::Unencrypted),
"nothing was read, so nothing scrambled was seen"
@@ -895,6 +1021,39 @@ mod tests {
);
}
/// The 50_000-sector budget must hold whatever the source returns, not
/// only when the source delivers sectors.
///
/// `tried` is incremented ONLY per inspected sector, inside
/// `for s in 0..usable`. An `Ok(0)` inspects nothing, so that loop never
/// runs — yet `advance` is forced to 1 to stop the scan spinning, so the
/// cursor keeps walking. The budget is then never consulted and the scan
/// runs for the extent's full, disc-declared `sector_count`: a misbehaving
/// or adversarial source (an emulated drive, a bridge answering short)
/// converts the anti-grind bound into no bound at all.
///
/// Mutation: delete the `tried` charge in the `usable == 0` arm and this
/// goes red at 60_000 reads.
#[test]
fn a_source_that_returns_zero_sectors_still_obeys_the_scan_budget() {
const MAX_TRIES: usize = 50_000;
let mut src = MockSource::new(0x00);
src.short_read = Some(0);
// Deliberately LARGER than the budget: if the budget is what stops the
// scan, the extent's own length is never reached.
let ext = [crate::disc::Extent {
start_lba: 0,
sector_count: 60_000,
}];
let _ = crack_key_scan(&mut src, &ext, 4, None);
let reads = src.reads.borrow().len();
assert!(
reads <= MAX_TRIES,
"an Ok(0)-returning source must be stopped by the {MAX_TRIES}-sector \
budget, not by the disc-declared extent length; got {reads} reads"
);
}
/// crack_key caps total scanned sectors at 50_000 even when extents are
/// far larger, and counts EVERY scanned sector (clear ones included)
/// toward the budget. With one 200_000-sector extent of clear sectors, it
@@ -1100,12 +1259,17 @@ mod tests {
);
}
/// MISSING #1 guard: the re-crack path (the `Option`-returning `crack_key`,
/// `fail_on_locked == false`) must NOT hard-fail on a CSS-locked read — it
/// returns `None`. A lapsed-AGID re-crack of another VTS stays soft so a
/// genuinely crackable title isn't killed by a transient locked read.
/// `crack_key` (the `Option`-returning convenience wrapper) collapses
/// `ScrambledUncracked` and `Unencrypted` alike to `None` via
/// [`CrackOutcome::into_state`], so an all-locked scan still reads `None`
/// here even though the scan itself now treats every CSS-lock as a hard
/// `ScrambledUncracked` (see `crack_key_scan`'s removal of the dead
/// `fail_on_locked` parameter). Callers that need to tell "locked/
/// uncrackable" apart from "genuinely clear" must use `crack_key_outcome`,
/// which the `all_locked_synthetic_iso_yields_css_key_missing_signal` test
/// pins directly.
#[test]
fn crack_key_recrack_locked_is_none_not_hard_fail() {
fn crack_key_all_locked_collapses_to_none() {
let mut src = MockSource::new(0x30);
src.lock_all = true;
let extents = [Extent {
@@ -1430,6 +1594,44 @@ mod tests {
);
}
/// The decrypted-HD-DVD regression, end to end through `resolve_dvd_title_key`.
/// An `.evo` is MPEG-PS (`ContentFormat::MpegPs`) exactly like a DVD `.vob`
/// and carries `None` keys once decrypted, so it reaches the CSS crack. Its
/// RDI navigation packs are private_stream_2 (0xBF) with the pack-start code
/// and bits set at offset 0x14 — but HD-DVD carries no CSS at all. The 0x11
/// exclusion in `is_scrambled_pack` keeps those nav packs from flipping the
/// scan's `saw_scrambled` gate, so the scan returns `Unencrypted` and the
/// title muxes cleanly instead of hard-failing.
///
/// Catches the mutation of dropping the 0x11 exclusion: without it every 0xBF
/// RDI pack counts as scramble evidence, the crack finds no key (there is
/// none), and the scan returns `ScrambledUncracked` → `CssKeyMissing` (E7023)
/// on a perfectly good HD-DVD — the exact defect a real CI run produced.
#[test]
fn resolve_dvd_title_key_decrypted_hddvd_rdi_packs_scan_clean_no_e7023() {
let mut src = MockSource::new(0x30); // 0x14 bits set…
src.stream_id = crate::consts::pes_stream_id::PRIVATE_STREAM_2; // …but a 0xBF nav pack
let extents = [Extent {
start_lba: 0,
sector_count: 64,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
8,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("a decrypted HD-DVD's RDI nav packs are not CSS — the scan must not hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"no CSS key exists on an HD-DVD; keys must stay None and the title mux clean"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
@@ -1595,8 +1797,8 @@ mod tests {
/// PER-VTS RE-CRACK SUCCESS (audit gap "success path missing"): the prior
/// re-crack test only covered the locked→None path. Here a re-crack
/// (`crack_key`, `fail_on_locked == false`) over a DIFFERENT VTS's extents
/// finds that VTS's own crackable sector and returns a `CssState` whose
/// (`crack_key`) over a DIFFERENT VTS's extents finds that VTS's own
/// crackable sector and returns a `CssState` whose
/// `crack_span` matches the new extents — proving a key cracked for one VTS
/// is genuinely re-derived (not reused) for another.
#[test]
+14 -1
View File
@@ -1899,11 +1899,24 @@ mod tests {
/// below its own range start, or a zero unit size, are both map bugs — they
/// must return a defined answer rather than panicking on debug overflow or
/// dividing by zero.
///
/// Every case here asserts the DEFINED answer, not merely the absence of a
/// panic. The zero-unit-size case used to be written
/// `assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
/// accepts both answers and so pinned nothing at all: the guards could
/// invert and it would still pass. The answer is knowable —
/// `saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
/// is even — so pin it.
#[test]
fn phase_gate_does_not_panic_on_a_malformed_map() {
use super::{Phase, unit_is_our_phase};
// Unit below its own range start: saturating_sub clamps to 0, and unit
// 0 is even.
assert!(unit_is_our_phase(10, 100, 3, Phase::Even));
assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true);
// Zero unit size: max(1) makes the divisor 1, so the index is the raw
// offset 70 — even.
assert!(unit_is_our_phase(100, 30, 0, Phase::Even));
// Both malformations at once: offset 0 over divisor 1 is unit 0, even.
assert!(unit_is_our_phase(5, 5, 0, Phase::Even));
}
}
+44 -9
View File
@@ -17,18 +17,31 @@ struct Scratch(PathBuf);
impl Scratch {
fn new(tag: &str) -> Self {
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{:?}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
let p = Self::unique_path(tag);
std::fs::create_dir_all(&p).unwrap();
Self(p)
}
/// The path a scratch dir takes — kept separate from directory creation so
/// its uniqueness is testable without a syscall between draws. A monotonic
/// counter, NOT a timestamp, is what guarantees it: `SystemTime::now()`
/// resolves to only a MICROSECOND on macOS, so two of the seven parallel
/// tests that share the "bdmv" tag routinely read the same value within one
/// microsecond, collide on the same directory, and the first to finish
/// `remove_dir_all`s it out from under the others' reads — an intermittent
/// `read_sectors` ENOENT. The counter is unique regardless of clock
/// granularity; pairing it with the pid keeps it unique across test-binary
/// processes.
fn unique_path(tag: &str) -> PathBuf {
static SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let uniq = SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let mut p = std::env::temp_dir();
p.push(format!(
"freemkv-dirimage-{tag}-{}-{uniq}",
std::process::id()
));
p
}
fn path(&self) -> &Path {
&self.0
}
@@ -69,6 +82,28 @@ fn bdmv_scratch() -> (Scratch, Vec<u8>, Vec<u8>) {
(s, index, clip)
}
/// Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
/// tag and run in parallel; if two land the same path, the first to drop
/// `remove_dir_all`s it out from under the other's reads, which surfaced as an
/// intermittent `read_sectors` failure. Regression: with the old
/// `SystemTime::now()` name this is RED, because macOS's clock advances only
/// per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
/// between them to nudge the clock, mirroring the real cross-thread collision —
/// hands back the same value many times over. The counter-based name is unique
/// regardless of clock granularity.
#[test]
fn scratch_paths_are_unique_even_at_clock_resolution() {
let paths: Vec<PathBuf> = (0..1000).map(|_| Scratch::unique_path("uniq")).collect();
let distinct: std::collections::HashSet<&PathBuf> = paths.iter().collect();
assert_eq!(
distinct.len(),
paths.len(),
"every Scratch must own a distinct path; a timestamp-based name collides \
under macOS's microsecond clock and lets one parallel test delete \
another's files mid-read"
);
}
// ── The de-risking spike ────────────────────────────────────────────────────
/// THE load-bearing assertion of the whole design: metadata synthesized here
+905 -118
View File
File diff suppressed because it is too large Load Diff
+164 -22
View File
@@ -7,13 +7,32 @@ use crate::udf;
impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Vec<DiscTitle> {
halt: Option<&crate::halt::Halt>,
) -> Result<Vec<DiscTitle>> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(_) => return Vec::new(),
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok(Vec::new()),
};
let mut titles = Vec::new();
@@ -243,7 +262,14 @@ impl Disc {
}
}
titles
// Polled again AFTER the loop: a cancel raised during the IFO reads
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok(titles)
}
}
@@ -528,6 +554,118 @@ mod tests {
// Tests
// ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|ts| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|ts| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test]
@@ -535,7 +673,11 @@ mod tests {
let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]);
assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.is_empty()
);
}
/// Single VTS, single title, one cell. Extent absolute LBA =
@@ -571,7 +713,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -627,7 +769,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -685,7 +827,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -743,7 +885,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -784,7 +926,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -839,7 +981,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -900,7 +1042,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -911,7 +1053,7 @@ mod tests {
.collect();
assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "en");
assert_eq!(audios[0].language, "eng");
assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -970,7 +1112,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1027,7 +1169,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1040,7 +1182,7 @@ mod tests {
// Languages preserved in order.
assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["en", "fr", "de"]
vec!["eng", "fra", "deu"]
);
// PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1087,7 +1229,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let sub = t
.streams
.iter()
@@ -1097,7 +1239,7 @@ mod tests {
})
.expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "en");
assert_eq!(sub.language, "eng");
assert!(
sub.codec_data.is_some(),
"non-zero palette must yield codec_data"
@@ -1141,7 +1283,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1178,7 +1320,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1263,7 +1405,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1315,7 +1457,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1358,7 +1500,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
+257 -25
View File
@@ -107,8 +107,10 @@ struct PlannedFile {
size: u64,
/// Inline (ICB-embedded) data, if any. When `Some`, `extents` is empty.
inline: Option<Vec<u8>>,
/// Absolute disc extents `(abs_lba, byte_len)`.
extents: Vec<(u32, u32)>,
/// Absolute disc extents, each carrying whether it was ever RECORDED (an
/// ECMA-167 4/14.14.1.1 type-1 extent is allocated but not recorded: it
/// occupies the file's byte space and its contents are zeros).
extents: Vec<crate::udf::AbsExtent>,
}
impl Disc {
@@ -144,7 +146,7 @@ impl Disc {
let fs = udf::read_filesystem(reader)?;
let mut planned: Vec<PlannedFile> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
let mut seen_hosts: std::collections::HashMap<PathBuf, String> =
let mut seen_hosts: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
plan_tree(
reader,
@@ -235,6 +237,9 @@ impl Disc {
let mut result = ExtractResult::default();
let total_bytes = required;
let mut done_bytes: u64 = 0;
// Cumulative zero-filled-unreadable bytes, so the live progress channel
// can report the good/unreadable split instead of pinning unreadable at 0.
let mut done_unreadable: u64 = 0;
// CSS per-VTS key cache; only consulted for CSS discs.
let is_css = matches!(base_keys, DecryptKeys::Css { .. });
@@ -269,8 +274,15 @@ impl Disc {
// non-tolerate), so extract_one_file already zero-filled it and
// counted it in bytes_unreadable — one 'lost' bucket covers both
// media damage and decrypt failure.
let (fr, halted) =
extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?;
let (fr, halted) = extract_one_file(
&mut dec,
dest,
pf,
total_bytes,
&mut done_bytes,
&mut done_unreadable,
opts,
)?;
result.bytes_good = result.bytes_good.saturating_add(fr.bytes_good);
result.bytes_unreadable = result.bytes_unreadable.saturating_add(fr.bytes_unreadable);
@@ -339,10 +351,14 @@ impl Disc {
let mut extents: Vec<crate::disc::Extent> = Vec::new();
for pf in files {
for &(abs_lba, byte_len) in &pf.extents {
// Unrecorded extents hold no bytes the VOB ever wrote, so they can
// carry no scrambled sector for the crack to work from — feeding
// them in spends the shared sector budget on media that is not part
// of the title.
for ext in pf.extents.iter().filter(|e| e.recorded) {
extents.push(crate::disc::Extent {
start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
start_lba: ext.lba,
sector_count: (ext.len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
});
}
}
@@ -379,9 +395,22 @@ impl Disc {
crate::css::CrackOutcome::Unencrypted => Ok(base_keys.clone()),
// Scrambled sectors WERE seen and no key came out. Reusing the
// disc-wide key here writes corrupt PES behind an intact header and
// reports a complete extract at exit 0. Skippable per title, which
// is why this is the per-title code and not the disc-level one — a
// sibling VTS may still crack.
// reports a complete extract at exit 0.
//
// `CssKeyMissing` is the per-TITLE code rather than the disc-level
// `CssNoDiscKey` because the two are treated differently elsewhere:
// it is what `error::is_skippable_title_stub` matches, and the MUX
// path really does skip a title on it and carry on with the rest of
// the disc.
//
// EXTRACT does not skip. This error is `?`-propagated by the only
// caller, out of the loop over every planned file, so one
// uncrackable VTS aborts the whole extract — including files from
// sibling VTS groups that would have cracked. That is deliberate:
// aborting loudly is the safe failure here, and pinned by
// `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`.
// Do not read the code choice as a promise that extraction
// continues past this point.
crate::css::CrackOutcome::ScrambledUncracked => Err(Error::CssKeyMissing),
}
}
@@ -429,7 +458,7 @@ fn plan_tree(
is_root: bool,
files: &mut Vec<PlannedFile>,
dirs: &mut Vec<PathBuf>,
seen_hosts: &mut std::collections::HashMap<PathBuf, String>,
seen_hosts: &mut std::collections::HashMap<String, String>,
) -> Result<()> {
for entry in &dir.entries {
if entry.name.is_empty() {
@@ -447,13 +476,48 @@ fn plan_tree(
let safe = sanitize_component(&entry.name)?;
let child_rel = host_rel.join(&safe);
let child_disc = format!("{disc_path}/{}", entry.name);
// Collision: two distinct disc paths → same host path.
if let Some(prev) = seen_hosts.insert(child_rel.clone(), child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: child_rel.to_string_lossy().into_owned(),
});
// Collision: two distinct disc paths → same host FILE. The key must
// model the HOST's namespace, not the disc's, and the two differ twice
// over:
//
// * CASE. macOS APFS and Windows NTFS are case-insensitive by default,
// so `Movie` and `movie` are one file there. Keyed by the
// case-preserving path they were two entries, nothing collided, and
// the second extraction overwrote the first with both files still
// reported `complete: true`.
// * The `.partial` SUFFIX. `extract_one_file` streams through
// `<final>.partial` before renaming, so a file's temp path lives in
// the same namespace as every other file's final path: a disc
// holding both `X` and `X.partial` planned two distinct final names,
// and extracting `X` then truncated the real `X.partial`.
//
// Only files get the `.partial` alias — directories are created
// directly and never stream through a temp name — but it is checked
// against every entry's primary key, so a directory `X.partial` beside
// a file `X` is caught too. The alias stores the OWNING file's disc
// path, so the `prev != child_disc` test keeps its meaning: a repeated
// identical disc path is not a collision, two different ones are.
//
// The fold is `to_lowercase`, which closes the ASCII and simple-Unicode
// case classes. It is NOT full case folding, and it does NOT normalize:
// APFS also unifies NFC/NFD, so `é` recorded as U+00E9 and as
// `e`+U+0301 remain two keys here and one file there. That residual is
// a false NEGATIVE (a missed collision, never a spurious one), so this
// is a strict improvement rather than a complete model of the host.
let mut register = |key: PathBuf| -> Result<()> {
let folded = key.to_string_lossy().to_lowercase();
if let Some(prev) = seen_hosts.insert(folded, child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: key.to_string_lossy().into_owned(),
});
}
Ok(())
};
register(child_rel.clone())?;
if !entry.is_dir {
register(with_partial_suffix(&child_rel))?;
}
if entry.is_dir {
dirs.push(child_rel.clone());
@@ -499,6 +563,7 @@ fn extract_one_file<S: SectorSource>(
pf: &PlannedFile,
total_bytes: u64,
done_bytes: &mut u64,
done_unreadable: &mut u64,
opts: &ExtractOptions,
) -> Result<(FileResult, bool)> {
let final_path = dest.join(&pf.host_rel);
@@ -528,16 +593,53 @@ fn extract_one_file<S: SectorSource>(
finalize_file(writer, &partial_path, pf.size, &final_path)?;
fr.complete = true;
*done_bytes = done_bytes.saturating_add(pf.size);
report(opts, *done_bytes, total_bytes);
report(opts, *done_bytes, *done_unreadable, total_bytes);
return Ok((fr, false));
}
let mut written: u64 = 0;
let mut buf = vec![0u8; READ_BATCH_SECTORS as usize * SECTOR_BYTES];
'extents: for &(abs_lba, byte_len) in &pf.extents {
'extents: for &crate::udf::AbsExtent {
lba: abs_lba,
len: byte_len,
recorded,
} in &pf.extents
{
if written >= pf.size {
break;
}
// ECMA-167 4/14.14.1.1 type 1: allocated but NOT recorded. The extent
// is part of the file's byte space and its contents are defined to be
// zeros, so write the zeros WITHOUT reading the media — those sectors
// hold nothing this file ever wrote (on an AACS disc, ciphertext that
// decrypts to noise). This mirrors `UdfFs::read_file_limited`, which
// takes the same decision from the same flag. Skipping the extent
// entirely instead would slide every later extent's bytes down by the
// hole's length.
if !recorded {
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64);
let hole_bytes = (sectors * SECTOR_BYTES_U64).min(pf.size.saturating_sub(written));
let mut left = hole_bytes;
for b in buf.iter_mut() {
*b = 0;
}
while left > 0 {
let n = left.min(buf.len() as u64) as usize;
write_all(&mut writer, &buf[..n], &partial_path)?;
written = written.saturating_add(n as u64);
*done_bytes = done_bytes.saturating_add(n as u64);
left -= n as u64;
}
fr.bytes_good = fr.bytes_good.saturating_add(hole_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
if opts.cancelled(cont) {
return Ok((fr, true));
}
if written >= pf.size {
break 'extents;
}
continue;
}
// Anchor AACS unit alignment at THIS extent's start (clip-anchored
// gate, not absolute LBA 0 and NOT the file's first extent). A file
// may span multiple extents (fragmented / Long-AD / continuation ICB
@@ -577,10 +679,11 @@ fn extract_one_file<S: SectorSource>(
}
write_all(&mut writer, &buf[..usable], &partial_path)?;
fr.bytes_unreadable = fr.bytes_unreadable.saturating_add(usable as u64);
*done_unreadable = done_unreadable.saturating_add(usable as u64);
}
written = written.saturating_add(usable as u64);
*done_bytes = done_bytes.saturating_add(usable as u64);
let cont = report(opts, *done_bytes, total_bytes);
let cont = report(opts, *done_bytes, *done_unreadable, total_bytes);
sector_off += batch;
if opts.cancelled(cont) {
// Leave the `.partial`; do NOT rename. The aggregate run
@@ -693,15 +796,20 @@ fn finalize_file(
/// Emit a progress report. Returns `true` to continue, `false` if the sink
/// requested an early stop (or there is no sink — always continue).
fn report(opts: &ExtractOptions, done: u64, total: u64) -> bool {
fn report(opts: &ExtractOptions, done: u64, unreadable: u64, total: u64) -> bool {
match opts.progress {
Some(p) => {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Mux,
work_done: done,
work_total: total,
bytes_good_total: done,
bytes_unreadable_total: 0,
// `done` counts good AND zero-filled-unreadable bytes together;
// split them so a consumer driven only by the live progress
// channel sees a holed extraction as holed rather than as a clean
// climb to 100%. The final ExtractResult already carries the true
// split — this used to pin unreadable at 0 and call every byte good.
bytes_good_total: done.saturating_sub(unreadable),
bytes_unreadable_total: unreadable,
bytes_pending_total: 0,
bytes_retryable_total: 0,
bytes_total_disc: total,
@@ -1185,6 +1293,17 @@ mod tests {
s
}
/// Build a file ICB whose FIRST short AD is an ECMA-167 4/14.14.1.1 type-1
/// (allocated, NOT recorded) extent and whose second is ordinary recorded
/// data. Both are `sectors_each` sectors long.
fn build_hole_then_data_icb(sectors_each: u32, hole_lba: u32, data_lba: u32) -> [u8; 2048] {
let mut s = build_two_extent_icb(sectors_each, hole_lba, data_lba);
let len = sectors_each * SECTOR_BYTES as u32;
// Re-stamp AD #0 with extent type 1 in bits 30..31 of the length field.
s[216..220].copy_from_slice(&(0x4000_0000u32 | (len & 0x3FFF_FFFF)).to_le_bytes());
s
}
/// Encrypt the clear unit from `clear_aacs_unit(tag)` under `unit_key` so
/// `aacs::content::decrypt_unit` recovers it cleanly (zero decrypt loss).
/// `tag` distinguishes two units' payloads.
@@ -1598,6 +1717,62 @@ mod tests {
assert!(matches!(err, Error::DirNameCollision { .. }));
}
/// Two disc names differing only by CASE are one host file on macOS APFS
/// and Windows NTFS, both case-insensitive by default. Keyed by the
/// case-preserving path, the collision map sees two entries, raises
/// nothing, and the second file extracted overwrites the first — while both
/// `PlannedFile`s report `complete: true`. Silent data loss reported as a
/// clean extract is the one outcome this crate must never produce, so the
/// host-equivalence key has to model the host's namespace, not the disc's.
#[test]
fn names_differing_only_by_case_are_a_collision() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("Movie", 30, 31, b"a".to_vec(), false),
file("movie", 32, 33, b"b".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("case_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err("two names that fold to one host file must collide");
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A file's in-flight `.partial` path shares the host namespace with every
/// other planned file's FINAL path. A disc carrying both `X` and
/// `X.partial` plans two distinct final names, so nothing collides — but
/// extracting `X` writes through `X.partial`, the same host path the other
/// file owns. Whichever lands second truncates the other, and both entries
/// are still reported complete.
#[test]
fn a_files_partial_path_colliding_with_another_files_final_name_is_an_error() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![
file("X", 30, 31, b"aaaa".to_vec(), false),
file("X.partial", 32, 33, b"bbbb".to_vec(), false),
],
subdirs: vec![],
};
let mut disc = build_disc(root);
let out = TmpDir::new("partial_collision");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err(
"X's temp path IS X.partial's final path — one host file for two \
disc files, so it must be refused up front",
);
assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}");
}
/// A non-empty target dir is refused without `--force`, and accepted with.
#[test]
fn non_empty_target_requires_force() {
@@ -1727,6 +1902,63 @@ mod tests {
);
}
/// An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED:
/// the space belongs to the file and occupies its byte range, but nothing
/// was ever written there, and the standard defines its contents as zeros.
/// `read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
/// `read_file_limited` honours it by emitting zeros WITHOUT touching the
/// media. The tree extractor reads the same ICBs and must agree: reading
/// those sectors returns whatever the media happens to hold there — on an
/// AACS disc, ciphertext that decrypts to noise — and writes it into the
/// extracted file as if the disc had recorded it.
///
/// The hole here is filled with a recognisable non-zero pattern, so
/// "emitted zeros" and "read the media" are told apart by CONTENT.
#[test]
fn extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it() {
const SECTORS_EACH: u32 = 1;
const HOLE: u32 = 5000;
const DATA: u32 = 5004;
let hole_bytes = vec![0xEEu8; SECTOR_BYTES];
let data_bytes = vec![0x5Au8; SECTOR_BYTES];
// The file's byte space: the hole's zeros FIRST, then the real data.
let mut expect = vec![0u8; SECTOR_BYTES];
expect.extend_from_slice(&data_bytes);
let mut disc = MemDisc::new();
build_udf_skeleton(&mut disc, 10);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "INDEX.BDMV", 42, false, false);
disc.put(
PART_START + 42,
build_hole_then_data_icb(SECTORS_EACH, HOLE, DATA),
);
disc.put_bytes(PART_START + HOLE, &hole_bytes);
disc.put_bytes(PART_START + DATA, &data_bytes);
disc.put(PART_START + 10, build_dir_icb(11, root_fids.len() as u32));
disc.put_bytes(PART_START + 11, &root_fids);
let out = TmpDir::new("unrecorded_extent");
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract");
let got = read_out(out.path(), "INDEX.BDMV").expect("file written");
assert_eq!(
got, expect,
"an unrecorded extent contributes zeros to the file, never the \
bytes that happen to sit on those sectors"
);
assert_eq!(
res.bytes_unreadable, 0,
"a hole is not a read failure — nothing was attempted"
);
assert!(res.complete);
}
/// Focused alignment-computation check underpinning the per-extent fix:
/// when each extent anchors its OWN start as the unit base, the extent's
/// own batch starts are always unit-aligned; anchoring a later extent
+1774 -52
View File
File diff suppressed because it is too large Load Diff
+300 -8
View File
@@ -2056,17 +2056,17 @@ impl Disc {
// derived below from the AACS MKB generation, not the tree.
let (mut titles, content_format) = if udf_fs.find_dir("/BDMV").is_some() {
(
Self::scan_bluray_titles(reader, &udf_fs),
Self::scan_bluray_titles(reader, &udf_fs, opts.halt.as_ref())?,
ContentFormat::BdTs,
)
} else if udf_fs.find_dir("/HVDVD_TS").is_some() {
(
Self::scan_hddvd_titles(reader, &udf_fs),
Self::scan_hddvd_titles(reader, &udf_fs, opts.halt.as_ref())?,
ContentFormat::MpegPs,
)
} else if udf_fs.find_dir("/VIDEO_TS").is_some() {
(
Self::scan_dvd_titles(reader, &udf_fs),
Self::scan_dvd_titles(reader, &udf_fs, opts.halt.as_ref())?,
ContentFormat::MpegPs,
)
} else {
@@ -2187,7 +2187,9 @@ impl Disc {
/// **Sort priority (titles[0] = most likely main feature):**
/// 1. Real titles (`size_bytes ≤ capacity_bytes`) before virtual
/// composites. The capacity check is a hard "physically
/// possible data on this disc" gate.
/// possible data on this disc" gate. `capacity_bytes == 0` means
/// the capacity is UNKNOWN (READ CAPACITY failed), so the gate is
/// skipped entirely rather than demoting every real title.
/// 2. Among real titles, LARGEST physical size first — the main
/// feature is the biggest real title on the disc. (This replaced
/// the old clip-count ordering, which mis-ranked chapter-per-clip
@@ -2249,8 +2251,18 @@ impl Disc {
// A title bigger than the whole disc is a "play-all" composite artifact
// (its declared size double-counts clips shared with other playlists) —
// demote it below any real single title.
let a_oversize = a.size_bytes > capacity_bytes;
let b_oversize = b.size_bytes > capacity_bytes;
//
// `capacity_bytes == 0` means the capacity is UNKNOWN, not that the
// disc holds nothing: `read_udf` substitutes 0 when READ CAPACITY
// fails and scans on regardless. Applied literally the gate would
// INVERT there — every real title (`size_bytes > 0`) would be
// "oversize" and demoted, while a CLPI-less `size_bytes == 0` title
// would not, landing at `titles[0]` ahead of the feature. With no
// capacity to compare against, the gate is inert and the size /
// duration / audio keys decide the order on their own.
let capacity_known = capacity_bytes > 0;
let a_oversize = capacity_known && a.size_bytes > capacity_bytes;
let b_oversize = capacity_known && b.size_bytes > capacity_bytes;
a_oversize
.cmp(&b_oversize)
// PRIMARY: largest physical size = the main feature. Robust where
@@ -3575,8 +3587,8 @@ mod tests {
) -> (crate::udf::fixture::MemDisc, udf::UdfFs) {
use crate::udf::fixture::*;
let files = vec![
file("MAIN.EVO", 100, 5_000, main_bytes, true),
file("OTHER.EVO", 101, 50_000, other_bytes, true),
file("MAIN.EVO", 100, 5_000, main_bytes as u64, true),
file("OTHER.EVO", 101, 50_000, other_bytes as u64, true),
];
let root = DirSpec {
name: String::new(),
@@ -3598,6 +3610,237 @@ mod tests {
(disc, udf)
}
/// A cancelled [`crate::halt::Halt`] must stop the HD-DVD title scan.
///
/// The scan is bounded but big — up to `MAX_HDDVD_CLIPS` clips, each
/// costing an ICB resolve plus a 16 MiB `EVO_PROBE_SECTORS` stream probe —
/// so on a live drive an operator Stop that only takes effect after the
/// whole enumerator returns is no Stop at all. `ScanOptions::halt` is
/// already honoured by the CSS crack and the forced-subtitle probe; the
/// title enumerator must honour it too.
///
/// It must also not report a HALF-ENUMERATED disc as a successful scan:
/// a truncated title list is indistinguishable from a disc that genuinely
/// holds fewer titles.
#[test]
fn scan_with_cancelled_halt_stops_the_hddvd_title_scan() {
use crate::udf::fixture::PART_START;
/// Counts reads that land on CLIP DATA (at or past the first clip's
/// data extent) — i.e. the per-clip stream probing, the expensive part
/// of the scan. Everything below that is filesystem metadata.
struct CountingReader<'a> {
inner: &'a mut crate::udf::fixture::MemDisc,
clip_reads: usize,
}
impl SectorSource for CountingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
if lba >= PART_START + 5_000 {
self.clip_reads += 1;
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
let (mut disc, udf) = hddvd_two_clip_disc(3_000_000, 5_000_000);
let halt = crate::halt::Halt::new();
halt.cancel();
let opts = ScanOptions {
halt: Some(halt),
..Default::default()
};
let mut reader = CountingReader {
inner: &mut disc,
clip_reads: 0,
};
let res = Disc::scan_with(&mut reader, 3_997_952, None, None, &opts, udf);
let clip_reads = reader.clip_reads;
assert!(
matches!(res, Err(Error::Halted)),
"a cancelled scan must say so, not return a partial title list as \
a completed scan; got {:?}",
res.map(|d| d.titles.len())
);
assert_eq!(
clip_reads, 0,
"cancellation must be observed before the per-clip stream probes, \
not after all of them"
);
}
/// `scan_with` must hand `ScanOptions::halt` to the BLU-RAY enumerator.
///
/// Every BD and DVD cancellation test in this crate calls
/// `scan_bluray_titles` / `scan_dvd_titles` DIRECTLY, so the three-line
/// wiring in `scan_with` that connects the operator's Stop to them was
/// covered by nothing at all: replacing `opts.halt.as_ref()` with `None`
/// on either branch left the whole suite green while a Stop during a BD
/// or DVD scan silently did nothing. Only the HD-DVD branch was pinned.
///
/// The fixture is deliberately the smallest disc that takes the BD branch
/// — a bare `/BDMV` with no PLAYLIST — because the point under test is the
/// ARGUMENT, not the enumerator's own (separately tested) halt polling.
/// Nothing between `scan_with`'s entry and the enumerator reads the flag
/// (the disc is unencrypted, so the AACS path is skipped), so a green here
/// can only mean the flag arrived.
///
/// Mutation: `Self::scan_bluray_titles(reader, &udf_fs, None)?` fails here.
#[test]
fn scan_with_passes_the_halt_flag_to_the_bluray_enumerator() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "BDMV".into(),
icb_lba: 12,
dir_data_lba: 13,
files: Vec::new(),
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
// Sanity: the same disc scans clean when nothing is cancelled, so a
// pass below cannot be some unrelated failure wearing Halted.
assert!(
Disc::scan_with(&mut disc, 500_000, None, None, &ScanOptions::default(), udf).is_ok(),
"fixture must scan successfully when not cancelled"
);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
let halt = crate::halt::Halt::new();
halt.cancel();
let opts = ScanOptions {
halt: Some(halt),
..Default::default()
};
let res = Disc::scan_with(&mut disc, 500_000, None, None, &opts, udf);
assert!(
matches!(res, Err(Error::Halted)),
"a cancelled BD scan must say so; returning a title list built \
after Stop reports a truncated enumeration as a completed one. \
Got {:?}",
res.map(|d| d.titles.len())
);
}
/// The DVD half of the same wiring, and the same reasoning.
///
/// Mutation: `Self::scan_dvd_titles(reader, &udf_fs, None)?` fails here.
#[test]
fn scan_with_passes_the_halt_flag_to_the_dvd_enumerator() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "VIDEO_TS".into(),
icb_lba: 50,
dir_data_lba: 51,
files: vec![
file_with("VIDEO_TS.IFO", 60, 5000, dvd_vmg_bytes(), true),
file_with("VTS_01_0.IFO", 62, 6000, dvd_vts_bytes(1000, 10, 10), true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
Disc::scan_with(&mut disc, 500_000, None, None, &ScanOptions::default(), udf).is_ok(),
"fixture must scan successfully when not cancelled"
);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
let halt = crate::halt::Halt::new();
halt.cancel();
let opts = ScanOptions {
halt: Some(halt),
..Default::default()
};
let res = Disc::scan_with(&mut disc, 500_000, None, None, &opts, udf);
assert!(
matches!(res, Err(Error::Halted)),
"a cancelled DVD scan must say so, not hand back whatever it had \
enumerated so far as a finished scan. Got {:?}",
res.map(|d| d.titles.len())
);
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The HD-DVD enumerator must not
/// swallow that into a successful scan.
///
/// Measured before this was fixed: the scan returned `Ok` with both
/// titles present and ZERO streams on each — a cancelled scan wearing the
/// shape of a disc whose clips carry no video or audio. Downstream that is
/// a title list to cache, display and rip from.
#[test]
fn halted_reads_do_not_report_the_hddvd_scan_as_successful() {
use crate::udf::fixture::PART_START;
/// Fails clip-data reads the way a live drive does once Stop is
/// pressed; filesystem metadata below the first clip still resolves,
/// so the scan gets far enough to enumerate titles.
struct HaltingReader<'a> {
inner: &'a mut crate::udf::fixture::MemDisc,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
if lba >= PART_START + 5_000 {
return Err(Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
let (mut disc, udf) = hddvd_two_clip_disc(3_000_000, 5_000_000);
let mut reader = HaltingReader { inner: &mut disc };
let res = Disc::scan_with(
&mut reader,
3_997_952,
None,
None,
&ScanOptions::default(),
udf,
);
assert!(
matches!(res, Err(Error::Halted)),
"reads cancelled by the drive's own halt flag must surface as a \
cancelled scan, not as titles that merely look stream-less; got \
{:?}",
res.map(|d| d
.titles
.iter()
.map(|t| (t.playlist.clone(), t.streams.len()))
.collect::<Vec<_>>())
);
}
/// `scan_with`'s `capacity_bytes = capacity as u64 * 2048` feeds
/// `canonical_title_order`'s "bigger than the whole disc = play-all
/// composite" threshold. Chosen so `capacity * 2048` clears both titles
@@ -6817,6 +7060,55 @@ mod tests {
);
}
/// `capacity_bytes == 0` means the disc capacity is UNKNOWN — `read_udf`
/// substitutes 0 when READ CAPACITY fails (transient spin-up, SCSI error)
/// and proceeds with the scan. It does NOT mean "the disc holds nothing".
///
/// With a literal reading of the gate, 0 inverts it: EVERY real title is
/// `size_bytes > 0` and therefore "oversize", while a title with no CLPI
/// (`size_bytes == 0`) is not — so the empty title sorts to `titles[0]`
/// ahead of the feature and `freemkv -t 1` rips garbage. The gate must be
/// INERT when the capacity is unknown.
#[test]
fn canonical_order_unknown_capacity_does_not_demote_every_real_title() {
const UNKNOWN: u64 = 0; // READ CAPACITY failed
let feature = title_with("00800.mpls", 7_320.0, 57_200_000_000, 1);
// A playlist whose CLPI files are missing/unparseable: no declared size.
let sizeless = title_with("00001.mpls", 120.0, 0, 1);
let mut titles = [sizeless, feature];
titles.sort_by(|a, b| Disc::canonical_title_order(a, b, UNKNOWN));
assert_eq!(
titles[0].playlist, "00800.mpls",
"with an UNKNOWN capacity the real feature must still sort first; \
a size-0 title must not be promoted ahead of it"
);
assert_eq!(titles[1].playlist, "00001.mpls");
}
/// Control for [`canonical_order_unknown_capacity_does_not_demote_every_real_title`]:
/// making the gate inert on an UNKNOWN capacity must not make it dead. With
/// a KNOWN capacity a genuinely oversize play-all composite is still demoted
/// below a smaller real title — even though "largest size first" would
/// otherwise rank it first. Asserted on the comparator in both argument
/// orders so an inconsistent comparator cannot pass.
#[test]
fn canonical_order_known_capacity_still_demotes_a_genuinely_oversize_title() {
use std::cmp::Ordering;
const CAP: u64 = 58_500_000_000;
let composite = title_with("00020.mpls", 15_180.0, 92_400_000_000, 253);
let real = title_with("00800.mpls", 7_320.0, 57_200_000_000, 1);
assert_eq!(
Disc::canonical_title_order(&real, &composite, CAP),
Ordering::Less,
"a known capacity must still demote the oversize composite"
);
assert_eq!(
Disc::canonical_title_order(&composite, &real, CAP),
Ordering::Greater,
"…in either argument order"
);
}
// ── audio_richness: the same-size / same-duration tiebreak ─────────────
/// A title carrying the given audio tracks, with size and duration fixed so
+421 -57
View File
@@ -8,6 +8,17 @@ pub fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
// A failed `ioctl(SG_IO)` and a vanished device never produced a SCSI
// reply at all — they are dead-bus faults, not recoverable bad sectors.
// `Error::is_scsi_transport_failure` (error.rs) already declares both
// variants transport failures so sweep / patch / fill_extents abort the
// pass instead of zero-filling against a wedged device; callers that
// flatten an error through here (Drive::read_one, DiscStream::fill_extents,
// freemkv-engine's recovery sweep/patch) would otherwise collapse them to
// status 0 and silently destroy that classification.
Error::IoError { .. } | Error::DeviceNotFound { .. } => {
(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None)
}
_ => (0, None),
}
}
@@ -24,6 +35,15 @@ pub(crate) mod macos;
#[cfg(windows)]
pub(crate) mod windows;
// Pick the platform module ONCE, here, so the cross-platform entry points below
// dispatch through `platform::…` with no per-function `#[cfg]` in their bodies.
#[cfg(target_os = "linux")]
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
pub(crate) use macos as platform;
#[cfg(windows)]
pub(crate) use windows as platform;
use crate::error::{Error, Result};
use crate::event::Event;
use crate::identity::DriveId;
@@ -279,22 +299,33 @@ impl Drive {
for attempt in 0..60u64 {
hb.tick(attempt, 60);
let mut buf = [0u8; 0];
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
// `checked_exec`, not a bare `execute`. This poll was the ONE
// drive path that talked to the transport directly, and it is the
// longest-running one: 60 x 500 ms = ~30 s. It never read
// `self.halt`, so an operator Stop pressed while a cold drive spun
// up was ignored for up to half a minute while every other path in
// this file returned `Halted` at its next command boundary. A Stop
// that does nothing for 30 s is indistinguishable from a hung app.
//
// A TUR that FAILS is the ordinary not-ready-yet answer and must
// keep the loop going — only `Halted` aborts it.
match self.checked_exec(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(
target: "freemkv::drive",
phase = "wait_ready",
attempts = attempt + 1,
elapsed_ms = t0.elapsed().as_millis() as u64,
"end"
);
return Ok(());
}
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => {}
}
std::thread::sleep(std::time::Duration::from_millis(500));
// Halt-aware backoff: the flag can also flip DURING the 500 ms
// gap, which is where most of the 30 s is actually spent.
sleep_until_halted(&self.halt, std::time::Duration::from_millis(500))?;
}
tracing::warn!(
target: "freemkv::drive",
@@ -926,11 +957,34 @@ impl Drive {
// range NonTrimmed and retries (a loud miss, never a silent commit).
// The sector/file path enforces the same invariant in
// sector/prefetched.rs; this is the live-drive counterpart.
Ok(_) => Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
}),
//
// SAY SO. The refusal was correct and completely silent: the
// sibling `Err(e)` arm below warns with lba/count/status, this one
// logged nothing at all, so a drive that residual-underruns on
// GOOD status was indistinguishable in the logs from a scratched
// disc — two different populations (replace the drive vs. clean
// the disc) collapsed into one. Absence of a log is itself the
// defect here; the read still fails either way.
//
// `scsi_status`/`sense` are deliberately absent from the line
// rather than faked: the command SUCCEEDED, so there is no sense
// data to report. `transferred` vs `expected` is the whole signal.
Ok(result) => {
tracing::warn!(
target: "freemkv::drive",
lba,
count,
transferred = result.bytes_transferred,
expected = count as usize * 2048,
code = crate::error::E_DISC_READ,
"READ(10) returned GOOD status with a residual underrun; refusing the short transfer"
);
Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
})
}
Err(Error::Halted) => Err(Error::Halted),
Err(e) => {
let (status, sense) = extract_scsi_context(&e);
@@ -1086,14 +1140,25 @@ impl Drive {
let stop = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x00, 0]; // START=0, LOEJ=0 → spin down
let start = [SCSI_START_STOP_UNIT, 0, 0, 0, 0x01, 0]; // START=1, LOEJ=0 → spin up
let mut buf = [0u8; 0];
self.scsi
.as_mut()
.execute(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS));
self.scsi
.as_mut()
.execute(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
std::thread::sleep(std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS));
// `checked_exec` + `sleep_until_halted`, not `execute` + a blind
// `thread::sleep`. This routine is ~15 s of deliberate waiting
// (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) issued from the
// recovery path, i.e. exactly when a run is going badly and the
// operator is most likely to press Stop. Both commands bypassed the
// halt flag and both sleeps were unconditional, so the whole cycle was
// deaf: Stop appeared to hang. Nothing here is un-interruptible — a
// half-finished spin cycle leaves the drive spun down, which the next
// command spins back up.
self.checked_exec(&stop, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_DOWN_IDLE_SECS),
)?;
self.checked_exec(&start, crate::scsi::DataDirection::None, &mut buf, 30_000)?;
sleep_until_halted(
&self.halt,
std::time::Duration::from_secs(SPIN_UP_SETTLE_SECS),
)?;
Ok(())
}
@@ -1299,11 +1364,18 @@ pub(crate) fn decode_read_capacity(buf: &[u8; 8], bytes_transferred: usize) -> R
last_lba.checked_add(1).ok_or(Error::DiscCapacityOverflow)
}
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
/// to true. Kept for the unit tests that cover the slicing behaviour;
/// production code paths no longer sleep on the recovery hot path
/// (recovery loop removed in 0.13.6).
#[cfg(test)]
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping to
/// true, returning [`Error::Halted`].
///
/// This was `#[cfg(test)]` for two releases, kept alive only by the four unit
/// tests below, while the two production paths that actually sleep — the
/// `wait_ready` poll backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle
/// pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
/// `std::thread::sleep` and so were DEAF to the operator's Stop for ~30 s and
/// ~15 s respectively. Every other drive path returns `Halted` at the next
/// `checked_exec` boundary; a Stop pressed during spin-up simply did nothing
/// visible until the poll ran out. The primitive existed; the sleeping code
/// just did not call it. It is production code again.
fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<()> {
const SLICE: std::time::Duration = std::time::Duration::from_millis(100);
let deadline = std::time::Instant::now() + total;
@@ -1322,18 +1394,7 @@ fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<(
/// Internal: discover drive paths + IDs without opening full Drive objects.
fn discover_drives() -> Vec<(String, DriveId)> {
#[cfg(target_os = "linux")]
{
linux::find_drives()
}
#[cfg(target_os = "macos")]
{
macos::find_drives()
}
#[cfg(windows)]
{
windows::find_drives()
}
platform::find_drives()
}
/// Structured outcome of [`resolve_device`] — a machine-readable signal
@@ -1353,20 +1414,14 @@ pub enum DeviceResolution {
/// Resolve a device path to its raw SCSI device. Returns the resolved
/// path plus a structured [`DeviceResolution`] signal describing whether
/// any substitution happened; the application layer maps that to UX text.
///
/// Staged, not yet wired: the cross-platform dispatch is kept ready for the
/// caller that will consume it, so the per-platform implementations below it
/// (and their tests) stay live. `allow(dead_code)` marks that deliberately —
/// this is not an accidental orphan.
#[allow(dead_code)]
pub(crate) fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
#[cfg(target_os = "linux")]
{
linux::resolve_device(path)
}
#[cfg(target_os = "macos")]
{
macos::resolve_device(path)
}
#[cfg(windows)]
{
windows::resolve_device(path)
}
platform::resolve_device(path)
}
#[cfg(test)]
@@ -1936,6 +1991,90 @@ mod command_tests {
assert!(err.scsi_sense().is_none());
}
/// `extract_scsi_context` must map the two non-SCSI dead-bus faults
/// (a failed `ioctl(SG_IO)` → `Error::IoError`, a vanished fd →
/// `Error::DeviceNotFound`) to the 0xFF TRANSPORT_FAILURE sentinel, not
/// to 0x00. Everything else keeps the (0, None) catch-all.
/// Spec: `Error::is_scsi_transport_failure` (error.rs) declares both
/// variants transport failures so sweep/patch/fill_extents abort
/// the pass instead of zero-filling against a wedged device.
/// Mutation: returning (0, None) here flattens IoError into
/// `DiscRead { status: Some(0) }`, which is_scsi_transport_failure
/// rejects — a wedged USB bridge zero-fills the whole title.
#[test]
fn extract_scsi_context_maps_dead_bus_faults_to_transport_failure() {
let (status, sense) = extract_scsi_context(&Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a failed ioctl(SG_IO) is a transport-layer fault"
);
assert!(sense.is_none(), "no SCSI reply means no sense data");
let (status, sense) = extract_scsi_context(&Error::DeviceNotFound {
path: "/dev/sg9".into(),
});
assert_eq!(
status,
crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
"a vanished device is a transport-layer fault"
);
assert!(sense.is_none());
// Control: the catch-all still yields (0, None) for unrelated errors,
// so the two asserts above are about these variants specifically.
assert_eq!(extract_scsi_context(&Error::Halted), (0, None));
// Control: real SCSI replies still pass their own status through.
let s = crate::scsi::ScsiSense {
sense_key: 3,
asc: 0x11,
ascq: 0x00,
};
assert_eq!(
extract_scsi_context(&Error::ScsiError {
opcode: 0x28,
status: 0x02,
sense: Some(s),
}),
(0x02, Some(s))
);
}
/// End-to-end: an `Error::IoError` raised by the transport must still be
/// classified as a transport failure after `Drive::read` flattens it into
/// `Error::DiscRead`. Without the `extract_scsi_context` mapping the
/// variant is destroyed (status becomes `Some(0)`) and the caller treats a
/// dead bus as a recoverable bad sector.
#[test]
fn read_io_error_surfaces_as_transport_failure_not_a_bad_sector() {
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotConnected),
},
}));
let mut buf = vec![0u8; 2048];
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a wedged bus must abort the pass, not zero-fill: got {err:?}"
);
// The same for a device that vanished mid-read.
let mut d = Drive::from_transport_for_test(Box::new(AlwaysErr {
err: || Error::DeviceNotFound {
path: "/dev/sg9".into(),
},
}));
let err = d.read(42, 1, &mut buf, false).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"a vanished device must abort the pass: got {err:?}"
);
}
#[test]
fn read_returns_halted_before_dispatch_without_touching_transport() {
// When the halt flag is set, checked_exec returns Halted BEFORE
@@ -2589,6 +2728,231 @@ mod command_tests {
);
}
/// A Stop pressed before the poll starts must be answered at once.
///
/// `wait_ready` was the ONE drive path that called
/// `self.scsi.as_mut().execute(..)` instead of `checked_exec`, and its
/// 60 x 500 ms loop never read `self.halt`. A cancel during spin-up was
/// therefore ignored for ~30 s — the test above measures exactly how long
/// — while every other drive path returns `Halted` at its next command
/// boundary. Thirty seconds of a dead Stop button is indistinguishable
/// from a hung application.
///
/// Mutation: restoring the bare `execute` makes this run the full poll and
/// return `DeviceNotReady`, failing both assertions.
#[test]
fn wait_ready_returns_halted_when_stopped_before_the_poll() {
struct NeverReady;
impl ScsiTransport for NeverReady {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(NeverReady));
d.halt();
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop is the operator, not a drive that failed to spin up: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"the poll must abandon immediately, not run its ~30 s course"
);
}
/// ...and a Stop pressed PART WAY THROUGH the poll must be answered at the
/// next command boundary, not at the end of the 30 s.
///
/// The flag is flipped by the transport itself on its third TEST UNIT
/// READY, which is deterministic (no wall-clock race): `checked_exec`
/// re-reads the flag after the command completes, so the loop must exit on
/// attempt three of sixty.
#[test]
fn wait_ready_returns_halted_when_stopped_during_the_poll() {
struct StopsOnThirdPoll {
halt: Arc<AtomicBool>,
seen: usize,
}
impl ScsiTransport for StopsOnThirdPoll {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
self.seen += 1;
if self.seen == 3 {
self.halt.store(true, Ordering::Relaxed);
}
Err(Error::ScsiError {
opcode: cdb[0],
status: 2,
sense: None,
})
}
}
let mut d = Drive::from_transport_for_test(Box::new(StopsOnThirdPoll {
halt: Arc::new(AtomicBool::new(false)),
seen: 0,
}));
// Hand the transport the drive's OWN flag, so setting it is exactly
// what an operator's Stop does.
let flag = d.halt_flag();
d.scsi = Box::new(StopsOnThirdPoll {
halt: flag,
seen: 0,
});
let t0 = std::time::Instant::now();
let r = d.wait_ready();
assert!(
matches!(r, Err(Error::Halted)),
"a mid-poll Stop must surface as Halted, not as DeviceNotReady \
after the full 30 s: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(5),
"must exit on the third poll, not the sixtieth"
);
}
/// `spin_cycle` must not be deaf to Stop for its ~15 s of deliberate
/// waiting.
///
/// Both START STOP UNIT commands went out through a bare `execute` and
/// both pauses (`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) were plain
/// `std::thread::sleep`, so the whole routine ignored the halt flag. It
/// runs from the RECOVERY path — precisely when a run is going badly and
/// the operator is most likely to press Stop.
///
/// The flag is set BEFORE the call, so this is deterministic: the very
/// first `checked_exec` must refuse.
///
/// Mutation: restoring either bare `execute` lets the first command
/// through and the test then waits out a real 5 s sleep, failing the
/// elapsed bound.
#[test]
fn spin_cycle_returns_halted_when_stopped_before_it_starts() {
let RecordingHarness {
drive: mut d,
cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
d.halt();
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
assert!(
matches!(r, Err(Error::Halted)),
"a Stop must abort the spin cycle: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"no sleep may run after the flag is set"
);
assert!(
cdb.lock().unwrap().is_empty(),
"not one START STOP UNIT may be issued after Stop"
);
}
/// ...and a Stop that lands DURING the spin-down pause must wake it.
///
/// This is the half a `checked_exec` alone cannot fix: the flag flips
/// while the thread is parked inside the 5 s `SPIN_DOWN_IDLE_SECS` sleep,
/// so only a halt-aware sleep can observe it. `sleep_until_halted` — which
/// already lived in this file with four tests, marked `#[cfg(test)]` and
/// called from nowhere — wakes within ~100 ms.
///
/// The margin is deliberately enormous (≈0.3 s against 5 s) so the bound
/// is not a wall-clock race: a plain `thread::sleep` cannot come in under
/// two seconds, and the halt-aware one cannot take that long.
#[test]
fn spin_cycle_wakes_from_its_spin_down_pause_when_stopped() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(0));
let flag = d.halt_flag();
let stopper = std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(200));
flag.store(true, Ordering::Relaxed);
});
let t0 = std::time::Instant::now();
let r = d.spin_cycle();
stopper.join().expect("stopper thread");
assert!(
matches!(r, Err(Error::Halted)),
"a Stop during the spin-down pause must surface as Halted: {r:?}"
);
assert!(
t0.elapsed() < std::time::Duration::from_secs(2),
"the pause must be halt-aware, not a blind {SPIN_DOWN_IDLE_SECS}s \
thread::sleep; took {:?}",
t0.elapsed()
);
}
/// A READ(10) that completes with GOOD status but a residual underrun is
/// correctly refused — and must SAY SO.
///
/// The refusal itself already existed and is right: the tail of the buffer
/// still holds stale bytes, so committing them would be silent
/// corruption. What was missing is any trace of it. The sibling `Err(e)`
/// arm warns with lba/count/status; this arm logged NOTHING, so a drive
/// that residual-underruns on GOOD status produced exactly the same
/// journal as a scratched disc — two populations with opposite remedies
/// (replace the drive vs. clean the disc) collapsed into one, with the
/// operator sent after the wrong one.
///
/// Mutation: deleting the `tracing::warn!` leaves no event; logging a
/// different code, or dropping the transferred/expected pair that is the
/// entire signal, fails the field assertions.
#[test]
fn read_logs_a_good_status_short_transfer() {
let RecordingHarness {
drive: mut d,
cdb: _cdb,
timeouts: _to,
} = recording(TransportOutcome::Ok(1024)); // half of one sector
let mut buf = vec![0u8; 2048];
let (r, events) = crate::testlog::capture(|| d.read(7, 1, &mut buf, false));
assert!(
matches!(r, Err(Error::DiscRead { sector: 7, .. })),
"a short transfer stays a failed read: {r:?}"
);
let line = events
.iter()
.find(|e| e.target == "freemkv::drive" && e.field("transferred").is_some())
.unwrap_or_else(|| {
panic!("a silently refused short transfer is the defect; got {events:?}")
});
assert_eq!(line.level, tracing::Level::WARN);
assert_eq!(line.field("lba"), Some("7"));
assert_eq!(line.field("transferred"), Some("1024"));
assert_eq!(
line.field("expected"),
Some("2048"),
"the underrun is only legible as transferred-vs-expected"
);
assert_eq!(
line.field("code"),
Some(crate::error::E_DISC_READ.to_string().as_str())
);
}
// ── report_key / mode_sense / read_buffer empty-vs-some ─────────
#[test]
+238 -126
View File
@@ -45,6 +45,7 @@ pub const E_INVALID_CDB_LENGTH: u16 = 4001;
// I/O (5xxx)
pub const E_IO_ERROR: u16 = 5000;
pub const E_SOURCE_TERMINATED: u16 = 5001;
// Disc format (6xxx)
pub const E_DISC_READ: u16 = 6000;
@@ -63,6 +64,22 @@ pub const E_SELECTION_PID_UNKNOWN: u16 = 6014;
pub const E_UDF_BUFFER_TOO_SMALL: u16 = 6012;
pub const E_UDF_NOT_FILESYSTEM: u16 = 6013;
pub const E_IMAGE_TRUNCATED: u16 = 6015;
pub const E_UDF_AD_CHAIN_TOO_LONG: u16 = 6016;
pub const E_UDF_UNRECORDED_EXTENT: u16 = 6017;
pub const E_UDF_EMBEDDED_DATA: u16 = 6018;
/// A file that EXISTS and whose allocation descriptors resolved without error,
/// yet yields not one usable extent: an empty AD list, or a list every entry of
/// which is zero-length or points at LBA 0. Reported by the HD-DVD clip
/// resolver (`disc::hddvd`).
///
/// It has no [`Error`] variant on purpose. `UdfFs::file_extents` returns
/// `Ok(vec![])` here rather than failing — the emptiness is only a defect in
/// the eye of a caller that needs bytes — so the condition is DETECTED by the
/// caller, not returned to it. The code exists so that detection can be
/// accounted in the log with something other than a neighbouring error's code:
/// logging it as E6017 would file a zero-length AD list as an authoring hole
/// and send whoever triages it at the wrong population.
pub const E_UDF_NO_USABLE_EXTENT: u16 = 6019;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -393,6 +410,18 @@ pub enum Error {
UdfNotFound {
path: String,
},
/// The file's ICB allocation list contains an unrecorded (ECMA-167
/// 4/14.14.1.1 type-1/type-2) extent: space allocated to the file at that
/// location but never written, so its true content there is zeros while
/// the media holds something else.
///
/// Raised by [`crate::udf::UdfFs::file_extents`] because a
/// `(lba, sector_count)` read plan cannot express a hole — reading it
/// splices undefined sectors into the rip as content, and dropping it
/// slides every later extent's byte space.
UdfUnrecordedExtent {
path: String,
},
/// The reader was addressable but the bytes are structurally NOT a UDF
/// filesystem — a deterministic tag/format mismatch (e.g. no Anchor Volume
/// Descriptor Pointer at sector 256, no partition descriptor, no File Set
@@ -405,6 +434,32 @@ pub enum Error {
/// 2048-byte sector. A contract violation on the public reader API —
/// returned instead of panicking on the slice.
UdfBufferTooSmall,
/// A file's allocation-descriptor continuation chain did not end within the
/// hop budget the UDF reader allows.
///
/// The budget exists so a crafted or corrupt disc cannot loop the reader
/// forever. Hitting it is NOT the end of the chain: the extents beyond that
/// point are unknown, so the extent list in hand describes only part of the
/// file. Returning that list would let a caller zero-pad the remainder to
/// the declared size and report a mostly-empty file as a complete
/// extraction, so the read fails instead.
UdfAdChainTooLong,
/// A file's ICB declares its data EMBEDDED inline (ECMA-167 4/14.6.8
/// allocation-descriptor type 3), so it has no out-of-line extents at all.
///
/// Returned only when a caller asked for a read plan over such a file.
/// The bytes in the allocation-descriptor field are then the file's own
/// CONTENT, not descriptors, so decoding them as (length, LBA) pairs
/// manufactures extents out of arbitrary data and points the reader at
/// unrelated sectors — a rip that completes at rc=0 carrying whatever
/// happened to be there. `read_directory` already refuses the same shape
/// for directories; this is the file half of that decision.
///
/// A file that legitimately stores its data this way is tiny (an ICB caps
/// it at well under 2 KiB — the AACS `*.inf` key files are the usual
/// case), and the callers that expect one read it via `read_inline_data`
/// long before extents are ever requested. A stream file cannot be one.
UdfEmbeddedData,
DiscTitleRange {
index: usize,
count: usize,
@@ -733,6 +788,19 @@ pub enum Error {
/// it silently leaves encrypted. The producer surfaces this rather
/// than emit still-encrypted bytes.
ExtentNotUnitAligned,
/// A [`crate::sector::SectorSource`] that feeds its reads from a
/// producer thread has terminated for good — the thread exited after
/// an error or before delivering the extents it was given — so it can
/// never return another byte.
///
/// It exists because the alternative answer is a lie: a dead source
/// that reports `Ok(0)` is indistinguishable from end-of-stream, and
/// `DiscStream::fill_extents` legitimately reads a short count as a
/// skippable hole — zero-filling and advancing over every remaining
/// sector of the title and still returning success. Unlike a bad
/// sector, this condition cannot be retried at a smaller size or
/// skipped past, so every consumer must abort the pass on it.
SourceTerminated,
/// An MPEG-TS packet under construction violated the 188-byte fixed
/// size (over-long adaptation field, overflowing payload, or a
/// short/mis-assembled packet). Indicates a muxer invariant break,
@@ -850,13 +918,17 @@ impl Error {
Error::ScsiError { .. } => E_SCSI_ERROR,
Error::InvalidCdbLength { .. } => E_INVALID_CDB_LENGTH,
Error::IoError { .. } => E_IO_ERROR,
Error::SourceTerminated => E_SOURCE_TERMINATED,
Error::DiscRead { .. } => E_DISC_READ,
Error::Halted => E_HALTED,
Error::MplsParse => E_MPLS_PARSE,
Error::ClpiParse => E_CLPI_PARSE,
Error::UdfNotFound { .. } => E_UDF_NOT_FOUND,
Error::UdfUnrecordedExtent { .. } => E_UDF_UNRECORDED_EXTENT,
Error::UdfNotFilesystem => E_UDF_NOT_FILESYSTEM,
Error::UdfBufferTooSmall => E_UDF_BUFFER_TOO_SMALL,
Error::UdfAdChainTooLong => E_UDF_AD_CHAIN_TOO_LONG,
Error::UdfEmbeddedData => E_UDF_EMBEDDED_DATA,
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::ShortImageRead { .. } => E_SHORT_IMAGE_READ,
Error::EmptyImage => E_EMPTY_IMAGE,
@@ -1063,6 +1135,7 @@ impl std::fmt::Display for Error {
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::UdfUnrecordedExtent { path } => write!(f, "E{}: {}", self.code(), path),
Error::SeamPlanDroppedMost { dropped, written } => {
write!(f, "E{} {dropped}/{written}", self.code())
}
@@ -1128,7 +1201,21 @@ impl std::error::Error for Error {
impl From<std::io::Error> for Error {
fn from(e: std::io::Error) -> Self {
Error::IoError { source: e }
// If this `io::Error` is one WE produced (`From<Error> for io::Error`
// carries the typed value in its boxed payload), give the original
// back instead of burying it in `Error::IoError`. That wrapper is not
// neutral: `is_scsi_transport_failure` treats `IoError` as a
// transport-layer fault (dead bus / wedged bridge), so re-wrapping a
// round-tripped `DiscRead` MEDIUM ERROR turned a skippable bad sector
// into a pass-aborting bridge wedge — the exact inverse of what that
// arm exists for. Any error that crosses a thread boundary as an
// `io::Error` (the prefetch channel's `Batch`) keeps its
// classification, its SCSI status, and its sense data.
match e.downcast::<Error>() {
Ok(typed) => typed,
// A genuine OS/`std` error — the `IoError` wrapper is correct here.
Err(io) => Error::IoError { source: io },
}
}
}
@@ -1143,7 +1230,6 @@ impl From<Error> for std::io::Error {
return source;
}
let code = e.code();
let msg = e.to_string();
// Map our error categories to io::ErrorKind
let kind = match code {
// Device access-denied semantics map to PermissionDenied;
@@ -1244,7 +1330,13 @@ impl From<Error> for std::io::Error {
E_DIR_IMAGE_FILE_CHANGED => std::io::ErrorKind::InvalidData,
_ => std::io::ErrorKind::Other,
};
std::io::Error::new(kind, msg)
// Carry the typed value itself as the payload rather than its
// stringification. `Display` is unchanged (`io::Error` delegates to the
// boxed error, whose `Display` is the same `E<code>[: …]` string), so
// `error_code` and every consumer built on it are unaffected — but the
// typed error now SURVIVES the conversion and `From<io::Error> for
// Error` can hand it back intact.
std::io::Error::new(kind, e)
}
}
@@ -1263,13 +1355,14 @@ pub type Result<T> = std::result::Result<T, Error>;
/// removing comes back.
///
/// [`From<Error> for io::Error`] is the ONLY path from a typed [`Error`] to an
/// `io::Error` in this crate, and it stringifies (`io::Error::new(kind, msg)`
/// where `msg` is the `Error`'s `E<code>[: …]` [`Display`](std::fmt::Display)
/// string) rather than boxing the typed value — no code path constructs an
/// `io::Error` that still holds a `crate::error::Error` via `get_ref`. So the
/// only recognised shape is the round-tripped `E<code>` message prefix.
/// `io::Error` in this crate. It boxes the typed value as the payload
/// (`io::Error::new(kind, e)`), whose [`Display`](std::fmt::Display) is the
/// same `E<code>[: …]` string the stringifying version produced — so this
/// parse is unaffected, and `From<io::Error> for Error` can additionally
/// `downcast` the payload back to the exact typed error. Errors that did NOT
/// come from this crate carry no `E<code>` prefix and yield `None`.
pub fn error_code(e: &std::io::Error) -> Option<u16> {
// Round-tripped: `From<Error> for io::Error` stringifies as "E<code>[: …]".
// Round-tripped: `From<Error> for io::Error` renders as "E<code>[: …]".
let s = e.to_string();
let digits = s.strip_prefix('E')?;
let end = digits
@@ -1398,6 +1491,22 @@ impl Error {
)
}
/// True if the read SOURCE itself is gone, as opposed to one range of
/// media being unreadable. Kept separate from
/// [`is_scsi_transport_failure`](Self::is_scsi_transport_failure) —
/// which is about the bus/bridge and drives "power-cycle the drive"
/// advice — because a terminated producer thread is neither a wedged
/// bridge nor a bad sector, and reporting it as SCSI status 0xFF would
/// be a fabricated status byte.
///
/// What it shares with a transport failure is the only thing the read
/// loops need to know: retrying smaller or skipping ahead cannot
/// recover anything, so the pass must abort rather than fabricate
/// zeros for the rest of the title.
pub fn is_source_terminated(&self) -> bool {
matches!(self, Error::SourceTerminated)
}
/// True if this error indicates bridge degradation — the SCSI status
/// is neither GOOD (0x00), CHECK CONDITION (0x02), nor transport failure
/// (0xFF). Observed on the Initio INIC-1618L USB bridge preceding a full
@@ -1804,126 +1913,129 @@ mod tests {
// ── New comprehensive tests ────────────────────────────────────────────────
/// Every `pub const E_*` code declared in this file, as
/// `(name, value)`, PARSED OUT OF THE SOURCE at compile time.
///
/// WHY THIS IS PARSED AND NOT LISTED. The uniqueness test below used to
/// carry a hand-maintained `vec![]` of the constants, with a doc comment
/// claiming it "pins all code assignments". It did not. At the time this
/// was written the file declared **127** `pub const E_*` and the vector
/// named **109** of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
/// `E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`,
/// `E_DIR_NAME_COLLISION`, `E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`,
/// `E_DIR_RAW_REJECTED`, `E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`,
/// `E_DRIVE_INQUIRY_SHORT`, `E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`,
/// `E_SEAM_PLAN_DROPPED_MOST`, `E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`,
/// `E_SOURCE_TERMINATED`, `E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were
/// outside the guarantee entirely, so a new variant colliding with any of
/// them passed green. Worse, an earlier audit READ that doc comment and
/// trusted it while assigning new codes.
///
/// The defect is not the eighteen omissions, it is that the list is
/// hand-maintained at all: adding a constant and forgetting the vector is
/// a silent no-op, which is the definition of a guarantee that decays.
/// Deriving the list from the declarations makes forgetting impossible —
/// the only way to escape the check is to stop declaring the constant.
///
/// `include_str!` of this very file is the cheapest seam that does that:
/// no `build.rs`, no proc macro, no dependency, and — being `#[cfg(test)]`
/// — not one byte of the source embedded in a release build.
fn declared_error_codes() -> Vec<(&'static str, u16)> {
const SRC: &str = include_str!("error.rs");
SRC.lines()
.filter_map(|line| {
// Only real declarations at column 0. A retired code is
// recorded as `// 2001: burned/retired`, which carries no `pub
// const` and is therefore correctly invisible here.
let decl = line.strip_prefix("pub const ")?;
let (name, value) = decl.split_once(": u16 = ")?;
if !name.starts_with("E_") {
return None;
}
let value = value.strip_suffix(';')?;
Some((
name,
value
.parse::<u16>()
.unwrap_or_else(|_| panic!("non-literal error code for `{name}`")),
))
})
.collect()
}
/// The parser above must actually FIND the declarations, and read their
/// values correctly. Without this, a `declared_error_codes` that returned
/// an empty vector would make the uniqueness test below pass vacuously —
/// exactly the failure mode (a guarantee that is really a no-op) this
/// whole change exists to remove.
///
/// Mutation: a `strip_prefix` typo, an off-by-one in the name slice, or a
/// parser that drops the last line fails here.
#[test]
fn declared_error_codes_parses_the_declarations_it_claims_to() {
let declared = declared_error_codes();
// Independent count of the declarations, computed a different way
// from the parser under test.
let expected = include_str!("error.rs")
.lines()
.filter(|l| l.starts_with("pub const E_"))
.count();
assert_eq!(
declared.len(),
expected,
"the parser must see every `pub const E_*` line"
);
assert!(
expected >= 120,
"sanity floor: this file declares well over a hundred codes, got {expected}"
);
// Names and values must match the compiled constants, so a parser that
// mis-slices the name or mis-reads the digits cannot pass.
for (name, value) in [
("E_DEVICE_NOT_FOUND", E_DEVICE_NOT_FOUND),
("E_UDF_UNRECORDED_EXTENT", E_UDF_UNRECORDED_EXTENT),
("E_KEYDB_PARSE", E_KEYDB_PARSE),
] {
assert!(
declared.contains(&(name, value)),
"`{name}` = {value} not parsed out of the source; got {:?}",
declared
.iter()
.filter(|(n, _)| *n == name)
.collect::<Vec<_>>()
);
}
// A comment-only retired code must NOT be picked up as a constant.
assert!(
!declared.iter().any(|(n, _)| n.is_empty()),
"no empty names"
);
}
/// Every published error code constant must be unique.
/// This pins all code assignments: a new variant that accidentally reuses
/// an existing code will make this test fail.
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating E_KEYDB_CONNECT) fails here.
///
/// The set is derived from the declarations by
/// [`declared_error_codes`], so — unlike the hand-kept vector this
/// replaced — a newly added constant is covered the moment it is
/// written, with no second edit to remember.
///
/// Mutation: changing E_KEYDB_PARSE from 8004 to 8000 (duplicating
/// E_KEYDB_CONNECT) fails here, and so now does the same collision on any
/// of the eighteen codes the old hand-kept list had never heard of.
#[test]
fn all_error_code_constants_are_unique() {
let mut codes = vec![
E_DEVICE_NOT_FOUND,
E_DEVICE_PERMISSION,
E_DEVICE_NOT_READY,
E_DEVICE_RESET_FAILED,
E_SCSI_INTERFACE_UNAVAILABLE,
E_DEVICE_LOCKED,
E_IOKIT_PLUGIN_FAILED,
E_UNSUPPORTED_DRIVE,
E_PROFILE_PARSE,
E_UNSUPPORTED_PLATFORM,
E_PLATFORM_NOT_IMPLEMENTED,
E_UNLOCK_FAILED,
E_SIGNATURE_MISMATCH,
E_SCSI_ERROR,
E_INVALID_CDB_LENGTH,
E_IO_ERROR,
E_DISC_READ,
E_MPLS_PARSE,
E_CLPI_PARSE,
E_UDF_NOT_FOUND,
E_DISC_TITLE_RANGE,
E_IFO_PARSE,
E_MKV_INVALID,
E_NO_STREAMS,
E_HALTED,
E_MAPFILE_INVALID,
E_IMAGE_TRUNCATED,
E_UDF_BUFFER_TOO_SMALL,
E_UDF_NOT_FILESYSTEM,
E_AACS_NO_KEYS,
E_AACS_CERT_SHORT,
E_AACS_AGID_ALLOC,
E_AACS_CERT_REJECTED,
E_AACS_CERT_READ,
E_AACS_CERT_VERIFY,
E_AACS_KEY_READ,
E_AACS_KEY_REJECTED,
E_AACS_KEY_VERIFY,
E_AACS_VID_READ,
E_AACS_VID_MAC,
E_AACS_DATA_KEY,
E_DECRYPT_FAILED,
E_CSS_AUTH_FAILED,
E_AACS_HOST_CERT_REJECTED,
E_AACS_RAW_READ_UNSUPPORTED,
E_AACS_VID_UNAVAILABLE,
E_AACS_MK_UNAVAILABLE,
E_AACS_VUK_NOT_IN_KEYDB,
E_DRIVE_PROFILE_MISSING,
E_VID_CDB_UNAVAILABLE,
E_NO_DISC_KEY,
E_CSS_KEY_MISSING,
E_CSS_NO_DISC_KEY,
E_KEY_SERVICE_UNAVAILABLE,
E_KEY_SERVICE_UNAUTHORIZED,
E_KEY_SERVICE_RATE_LIMITED,
E_AACS_NO_HOST_CERT,
E_AACS_BUS_KEY_UNAVAILABLE,
E_FMTS_KEY_MISSING,
E_KEYDB_CONNECT,
E_KEYDB_HTTP,
E_KEYDB_INVALID,
E_KEYDB_WRITE,
E_KEYDB_PARSE,
E_KEYDB_LOAD,
E_KEYDB_UNSUPPORTED_SCHEME,
E_KEYDB_TOO_MANY_REDIRECTS,
E_STREAM_READ_ONLY,
E_STREAM_WRITE_ONLY,
E_STREAM_URL_INVALID,
E_STREAM_URL_MISSING_PATH,
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MUX_HEADER_BUFFER_EXCEEDED,
E_MKV_LACING_INVALID,
E_MKV_SOURCE_INVALID,
E_MKV_UNENCODABLE,
E_MP4_NO_VIDEO_TRACK,
E_MP4_INVALID,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
E_ISO_TOO_LARGE,
E_NO_METADATA,
E_DISC_URL_NOT_DIRECT,
E_HEVC_PARAM_PARSE,
E_MUX_TRACK_RANGE,
E_FMP4_UNIMPLEMENTED,
E_DEMUX_THREAD_PANICKED,
E_PIPELINE_JOIN_TIMEOUT,
E_PIPELINE_CONSUMER_PANICKED,
E_SWEEP_CONSUMER_GONE,
E_PIPELINE_CONSUMER_GONE,
E_DISC_CAPACITY_OVERFLOW,
E_M2TS_PACKET_MALFORMED,
E_EXTENT_NOT_UNIT_ALIGNED,
E_DISC_CAPACITY_MALFORMED,
E_DIR_IMAGE_SSIF_UNSUPPORTED,
E_DIR_IMAGE_PLACEMENT,
E_DIR_IMAGE_ENCRYPTED,
E_DIR_IMAGE_UNSUPPORTED_TREE,
E_DIR_IMAGE_FILE_CHANGED,
E_DIR_IMAGE_TOO_LARGE,
];
let original_len = codes.len();
codes.sort();
codes.dedup();
assert_eq!(
codes.len(),
original_len,
"duplicate error code constants detected — check error.rs"
let mut by_code: std::collections::BTreeMap<u16, Vec<&str>> =
std::collections::BTreeMap::new();
for (name, value) in declared_error_codes() {
by_code.entry(value).or_default().push(name);
}
let dupes: Vec<_> = by_code
.iter()
.filter(|(_, names)| names.len() > 1)
.collect();
assert!(
dupes.is_empty(),
"duplicate error code constants detected — check error.rs: {dupes:?}"
);
}
+117 -44
View File
@@ -49,7 +49,7 @@ pub struct DvdTitle {
pub cells: Vec<DvdCell>,
/// Chapter start times in seconds (derived from program map + cell times)
pub chapter_times: Vec<f64>,
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cb, Cr].
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cr, Cb].
pub palette: Option<Vec<[u8; 4]>>,
}
@@ -425,6 +425,13 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
for (&vts_number, titles_info) in &title_set_map {
match parse_vts(reader, udf, vts_number, titles_info) {
Ok(ts) => title_sets.push(ts),
// The operator's Stop is not a placeholder title set. Once the
// drive's halt flag is set EVERY command fails with `Halted`, so
// swallowing it here would skip every REMAINING title set in turn
// and return `Ok` with a truncated list — a cancelled scan that
// is indistinguishable from a disc genuinely holding fewer
// titles. `scan_dvd_titles` propagates it to the caller.
Err(Error::Halted) => return Err(Error::Halted),
Err(e) => {
// Some discs carry placeholder TT_SRPT entries for title sets
// that are not really there, so one failure is not fatal. But
@@ -691,7 +698,16 @@ fn parse_video_attr(data: &[u8]) -> Result<DvdVideoAttr> {
}
/// Parse one audio stream attribute block (8 bytes at `offset`).
fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
/// `pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
/// `parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
/// calls it directly so its language-mapping tests run the real parser over real
/// on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
/// could agree with the muxer while both disagree with the disc. Narrowing this
/// would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
/// that can drift apart, or moving those tests away from the code they exist to
/// pin — both worse than the widened crate-internal visibility, which reaches no
/// public API.
pub(crate) fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let b0 = byte_at(data, offset)?;
let b1 = byte_at(data, offset + 1)?;
@@ -718,25 +734,9 @@ fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let channels = (b1 & 0x07) + 1; // (channels - 1) in low 3 bits of byte 1
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
{
String::from_utf8_lossy(lang_bytes).to_string()
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
// Try to interpret as printable ASCII
let s: String = lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdAudioAttr {
codec,
@@ -785,9 +785,27 @@ fn assign_audio_sub_stream_ids(streams: &mut [DvdAudioAttr]) {
/// Parse one subtitle stream attribute block (6 bytes at `offset`).
fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
// Language code: bytes 2-3 as ISO 639
// Language code: bytes 2-3 as ISO 639-1 (the DVD-Video spec's form).
let lang_bytes = sub_slice(data, offset + 2, 2)?;
let language = if lang_bytes[0] >= b'a'
let language = dvd_lang_to_iso639_2(&parse_raw_dvd_lang_bytes(lang_bytes));
Ok(DvdSubtitleAttr { language })
}
/// Decode the raw 2-byte on-disc language code shared by the audio and
/// subtitle attribute blocks. A pair of lowercase a-z bytes is taken
/// verbatim (the ISO 639-1 code the DVD-Video spec puts there); an all-zero
/// pair means unspecified (empty string); anything else falls through to an
/// ASCII-alphanumeric salvage — letters (either case) and digits are kept,
/// everything else (control bytes, punctuation, high bytes from a corrupt or
/// hostile disc) is dropped.
///
/// The salvage is deliberately not narrowed to a-z: whatever survives is only
/// ever a lookup key for [`dvd_lang_to_iso639_2`], which degrades anything it
/// does not recognize to `und`, so a stray `X` or `5` costs nothing and cannot
/// reach an output stream as a language code.
fn parse_raw_dvd_lang_bytes(lang_bytes: &[u8]) -> String {
if lang_bytes[0] >= b'a'
&& lang_bytes[0] <= b'z'
&& lang_bytes[1] >= b'a'
&& lang_bytes[1] <= b'z'
@@ -796,15 +814,40 @@ fn parse_subtitle_attr(data: &[u8], offset: usize) -> Result<DvdSubtitleAttr> {
} else if lang_bytes[0] == 0 && lang_bytes[1] == 0 {
String::new()
} else {
let s: String = lang_bytes
lang_bytes
.iter()
.filter(|&&b| b.is_ascii_alphanumeric())
.map(|&b| b as char)
.collect();
s
};
.collect()
}
}
Ok(DvdSubtitleAttr { language })
/// Convert a DVD IFO audio/subtitle attribute's language code — ISO 639-1
/// (2 lowercase letters) per the DVD-Video spec, or empty when unspecified —
/// to the ISO 639-2 form every downstream consumer (`disc::AudioStream` /
/// `disc::SubtitleStream::language`, and in turn Matroska's `Language`
/// element per RFC 9559 §12 and the MP4 sink's `mdhd` language) requires.
///
/// Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
/// 639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
/// frozen-1988 language list still puts on disc). The narrower
/// `vocab::menu_lang` table is deliberately NOT used here: it exists for
/// Blu-ray menu-graphic filename tokens and knows only 25 languages, so a
/// Region-2 disc's Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian,
/// Hebrew, Estonian, Latvian, Lithuanian and Icelandic tracks would all fold
/// onto `und` together. DVD streams carry an empty `label`, so the language
/// is the only thing distinguishing one subtitle track from the next — a
/// valid code that is identical for six tracks is worse for the user than the
/// invalid one it replaced. Both tables normalize to ISO 639-2/T, so they
/// agree wherever they overlap.
///
/// An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's
/// own "undetermined" value) — a valid element value — rather than passing
/// through an invalid 2-letter code or an empty string. Never guesses.
fn dvd_lang_to_iso639_2(raw: &str) -> String {
crate::labels::vocab::iso639_1_to_iso639_2(raw)
.unwrap_or("und")
.to_string()
}
// ── PGC parser ──────────────────────────────────────────────────────────────
@@ -1016,7 +1059,7 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
times
};
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cb, Cr]
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cr, Cb]
let palette = if pgc_offset + 0xA4 + 64 <= data.len() {
let mut colors = Vec::with_capacity(16);
for i in 0..16 {
@@ -1441,7 +1484,7 @@ mod tests {
data[0] = 0x00;
// b1: bits 2-0=101 (channels-1=5) => 0x05
data[1] = 0x05;
// language "en"
// on-disc language "en" (ISO 639-1) -> parsed as ISO 639-2 "eng"
data[2] = b'e';
data[3] = b'n';
@@ -1449,7 +1492,7 @@ mod tests {
assert_eq!(attr.codec, Codec::Ac3);
assert_eq!(attr.sample_rate, 48000);
assert_eq!(attr.channels, 6);
assert_eq!(attr.language, "en");
assert_eq!(attr.language, "eng");
}
#[test]
@@ -1579,7 +1622,7 @@ mod tests {
assert_eq!(attr.codec, Codec::Dts);
assert_eq!(attr.sample_rate, 96000);
assert_eq!(attr.channels, 2);
assert_eq!(attr.language, "fr");
assert_eq!(attr.language, "fra");
}
// ─────────────────────────────────────────────────────────────────────
@@ -1693,15 +1736,17 @@ mod tests {
}
/// Audio language bytes [offset+2..+4]: when both bytes are 0x00 the
/// language is the empty string (unspecified), per source.
/// on-disc code is unspecified, and `dvd_lang_to_iso639_2` maps that to
/// the valid ISO 639-2 "und" (undetermined) rather than an empty string,
/// which is not a legal Matroska `Language` element value.
#[test]
fn audio_attr_zero_language_is_empty() {
fn audio_attr_zero_language_becomes_und() {
let mut data = vec![0u8; 8];
data[0] = 0x00;
data[2] = 0x00;
data[3] = 0x00;
let attr = parse_audio_attr(&data, 0).unwrap();
assert_eq!(attr.language, "");
assert_eq!(attr.language, "und");
}
/// Audio sample_rate flag (b0>>3 & 0x03): 0=48kHz, 1=96kHz, else 48kHz.
@@ -1722,11 +1767,11 @@ mod tests {
data[2] = b'd';
data[3] = b'e';
let attr = parse_subtitle_attr(&data, 0).unwrap();
assert_eq!(attr.language, "de");
assert_eq!(attr.language, "deu");
let zero = vec![0u8; 6];
let attr2 = parse_subtitle_attr(&zero, 0).unwrap();
assert_eq!(attr2.language, "");
assert_eq!(attr2.language, "und");
}
/// assign_audio_sub_stream_ids: MP1/MP2 and other non-private-stream-1
@@ -1812,8 +1857,8 @@ mod tests {
assert_eq!(title.cells[1].first_sector, 20);
}
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cb, Cr].
/// A palette with at least one non-zero Y/Cb/Cr is returned as Some;
/// parse_pgc palette: at PGC+0xA4, 16 colors × 4 bytes [pad, Y, Cr, Cb].
/// A palette with at least one non-zero Y/Cr/Cb is returned as Some;
/// an all-zero palette returns None (source filters empty palettes).
#[test]
fn pgc_palette_present_and_empty() {
@@ -1833,14 +1878,14 @@ mod tests {
assert!(title2.palette.is_none());
}
/// parse_pgc palette layout: each color is [padding, Y, Cb, Cr] and the
/// parse_pgc palette layout: each color is [padding, Y, Cr, Cb] and the
/// "non-empty" test ignores the padding byte (index 0). A palette whose
/// ONLY non-zero bytes are padding must still be treated as empty (None).
#[test]
fn pgc_palette_padding_only_is_empty() {
let mut pgc = vec![0u8; 0xEA];
pgc[0x03] = 0;
// Set padding byte (index 0) of color 0 non-zero, but Y/Cb/Cr zero.
// Set padding byte (index 0) of color 0 non-zero, but Y/Cr/Cb zero.
pgc[0xA4] = 0xFF;
let title = parse_pgc(&pgc, 0, 1).unwrap();
assert!(
@@ -2278,10 +2323,13 @@ mod tests {
/// two bytes at +2. Only a pair of lowercase a-z bytes is taken verbatim;
/// anything else falls through to the ASCII-alphanumeric salvage, which
/// keeps only the usable characters. A byte outside a-z must never end up
/// in the language string.
/// in the raw salvaged string. This exercises `parse_raw_dvd_lang_bytes`
/// directly — the byte-level salvage step — separately from the ISO
/// 639-1 -> 639-2 mapping `parse_audio_attr`/`parse_subtitle_attr` apply
/// on top (see `dvd_two_letter_and_malformed_language_becomes_iso639_2`).
#[test]
fn language_code_rejects_non_lowercase_bytes() {
// (byte0, byte1, expected language)
// (byte0, byte1, expected raw salvage)
let cases: [(u8, u8, &str); 8] = [
(b'e', b'n', "en"), // both in range → verbatim
(0x21, b'n', "n"), // '!' is below 'a'
@@ -2292,6 +2340,31 @@ mod tests {
(0x00, b'E', "E"), // only the first byte is zero
(0x00, 0x00, ""), // both zero → unset
];
for (b0, b1, want) in cases {
assert_eq!(
parse_raw_dvd_lang_bytes(&[b0, b1]),
want,
"raw language salvage for ({b0:#04x}, {b1:#04x})"
);
}
}
/// The full pipeline `parse_audio_attr`/`parse_subtitle_attr` apply on
/// top of the raw salvage: a valid ISO 639-1 code maps to its ISO 639-2
/// equivalent, and anything the raw salvage does NOT produce a mapped
/// code for (empty, or a single leftover letter from a malformed byte
/// pair) degrades to "und" — never an invalid 2-letter/1-letter code and
/// never an empty string, both of which are illegal Matroska `Language`
/// element values (RFC 9559 §12).
#[test]
fn dvd_two_letter_and_malformed_language_becomes_iso639_2() {
// (byte0, byte1, expected final language)
let cases: [(u8, u8, &str); 4] = [
(b'e', b'n', "eng"), // valid ISO 639-1 → mapped
(0x21, b'n', "und"), // malformed → salvage "n", unmapped → und
(b'E', 0x00, "und"), // malformed → salvage "E", unmapped → und
(0x00, 0x00, "und"), // unspecified → und
];
for (b0, b1, want) in cases {
let mut audio = vec![0u8; 8];
audio[2] = b0;
@@ -2496,7 +2569,7 @@ mod tests {
}
/// The subtitle palette is 16 entries of 4 bytes at PGC+0xA4, each
/// `[padding, Y, Cb, Cr]`. Every byte of every entry is distinct here, so
/// `[padding, Y, Cr, Cb]`. Every byte of every entry is distinct here, so
/// a wrong stride, a wrong base or a shifted component shows up.
#[test]
// The loop variable is the DOMAIN VALUE being checked (a palette entry number), not a
@@ -2524,7 +2597,7 @@ mod tests {
}
}
/// A palette is "present" when ANY of Y, Cb or Cr is non-zero in ANY
/// A palette is "present" when ANY of Y, Cr or Cb is non-zero in ANY
/// entry — a single non-zero chroma component is enough. Only the
/// padding byte [0] is ignored.
#[test]
+8 -8
View File
@@ -85,16 +85,16 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let enums = identify_master_enums(archive);
if enums.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: com/bydeluxe/ present but no master enum fingerprint matched"
);
return None;
}
for (label, m) in &enums {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
enum = %label,
class = %m.class_name,
class = ?m.class_name,
count = m.values.len(),
"deluxe master enum identified",
);
@@ -110,15 +110,15 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let binding_classes = find_binding_classes(archive, &master_table.class_name_set());
if binding_classes.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: no binding class found (no class has enough getstatic refs to master enums)"
);
return None;
}
for (name, count) in &binding_classes {
tracing::info!(
jar = %entry_name,
binding_class = %name,
jar = ?entry_name,
binding_class = ?name,
getstatic_count = count,
"deluxe binding class candidate",
);
@@ -138,7 +138,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
if streams.is_empty() {
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
"deluxe: binding classes found but produced 0 decoded streams"
);
return None;
@@ -149,7 +149,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
tracing::info!(
jar = %entry_name,
jar = ?entry_name,
audio = labels.iter().filter(|l| l.stream_type == StreamLabelType::Audio).count(),
subtitle = labels.iter().filter(|l| l.stream_type == StreamLabelType::Subtitle).count(),
"deluxe emitted labels",
+156 -2
View File
@@ -450,13 +450,22 @@ pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
}) else {
continue;
};
for clip in &title.clips {
// ONLY the anchor's first clip. `disc::bluray` builds a title's
// stream list from `play_items[0]`'s STN table, so that is the only
// clip in which these PIDs were ever observed — the same clip tier 3
// keys its derived ids on (`clip0`, below). Recording the fact
// against every clip the anchor plays claims knowledge of stream
// tables never read: a sibling playlist over a LATER clip then binds
// the anchor's editorial label onto whichever stream of that clip
// reuses the PID, which is a different physical stream (PIDs are
// unique only within a clip).
if let Some(clip) = title.clips.first() {
pid_map.insert((clip.clip_id.as_str(), *pid), pos);
}
}
tracing::info!(
stream_type = ?stream_type,
playlist = %titles[anchor].playlist,
playlist = ?titles[anchor].playlist,
slots = slots_of(&titles[anchor], stream_type).len(),
"label list anchored to a title by its stream-language sequence",
);
@@ -1907,6 +1916,78 @@ mod apply_tests {
}
}
/// Sentinel embedded in the crafted playlist name below. The capture keeps
/// ONLY fields whose rendered form contains it, so the assertion cannot be
/// fooled by another test's log output.
const LOG_INJECTION_SENTINEL: &str = "FMKV-LOG-INJECTION-PROBE";
/// A playlist name is a raw UDF directory entry — disc-controlled bytes,
/// validated no further than a lossy UTF-8 decode. Logging it through
/// tracing's `%` (Display) sigil writes those bytes VERBATIM, so a crafted
/// `.mpls` filename carrying ANSI escapes or control characters forges
/// terminal output and log structure in any consumer rendering the event
/// (CWE-117). `?` (Debug) escapes them, and `str`'s Debug is exactly the
/// escaping this needs.
///
/// `info!` is not covered by the debug/trace-logging exemption: this fires
/// on an ordinary rip of an ordinary disc.
///
/// Mutation: put `%` back on `playlist` in `apply_labels` and this goes red.
#[test]
fn a_disc_derived_playlist_name_is_escaped_in_the_log_not_written_verbatim() {
// A name whose bytes would clear the line and repaint it.
let evil = format!("\u{1b}[2K\u{1b}[31m{LOG_INJECTION_SENTINEL}\u{7}\u{1b}[0m.mpls");
let labels = vec![
sub_label(1, "eng", LabelQualifier::None),
sub_label(2, "spa", LabelQualifier::None),
sub_label(3, "fra", LabelQualifier::None),
];
let mut titles = vec![title_on_clip(
&evil,
"00294",
vec![
subtitle(0x12A0, "eng"),
subtitle(0x12A1, "spa"),
subtitle(0x12A2, "fra"),
],
)];
// Capture through the crate's ONE global `tracing` subscriber:
// `testlog::capture` installs it exactly once and routes each event to a
// thread-local sink. A process-wide `set_global_default` HERE instead
// would poison every other test's callsite interest cache for the rest of
// the binary — `tracing` caches interest GLOBALLY — and only the first
// `set_global_default` in a process takes effect anyway. The shared
// subscriber answers interest for every callsite and isolates concurrent
// captures per thread, which is the invariant these log-accounting
// assertions rely on.
let ((), events) = crate::testlog::capture(|| {
apply_labels(&labels, &mut titles);
});
let playlist: Vec<&str> = events
.iter()
.filter(|e| e.target.starts_with("libfreemkv::labels"))
.filter_map(|e| e.field("playlist"))
.filter(|v| v.contains(LOG_INJECTION_SENTINEL))
.collect();
assert!(
!playlist.is_empty(),
"the anchoring event must actually have fired, or this test proves \
nothing; captured: {events:?}"
);
for rendered in playlist {
assert!(
!rendered.contains('\u{1b}') && !rendered.contains('\u{7}'),
"a disc-controlled playlist name reached the log with its raw \
control bytes intact: {rendered:?}"
);
assert!(
rendered.contains(LOG_INJECTION_SENTINEL),
"the name must still be legible once escaped: {rendered:?}"
);
}
}
fn sub_label(num: u16, lang: &str, qualifier: LabelQualifier) -> StreamLabel {
StreamLabel {
stream_id: None,
@@ -3103,6 +3184,79 @@ mod apply_tests {
);
}
/// A title that plays SEVERAL clips in order — the shape the anchor's
/// PID facts are harvested from.
fn title_on_clips(playlist: &str, clip_ids: &[&str], streams: Vec<Stream>) -> DiscTitle {
DiscTitle {
playlist: playlist.into(),
clips: clip_ids
.iter()
.map(|id| crate::disc::Clip {
feed_span: None,
clip_id: (*id).into(),
in_time: 0,
out_time: 0,
duration_secs: 3600.0,
source_packets: 0,
})
.collect(),
..title_with(streams)
}
}
/// Spec: the anchor proves a `(clip, PID)` fact only for the clip its
/// stream table was READ FROM — the first play item — never for every clip
/// the anchor happens to play.
///
/// `disc::bluray` builds a title's stream list from `play_items[0]`'s STN
/// table, and tier 3 fifty lines below says exactly that by keying its
/// derived ids on `clip0`. Tier 2's harvest contradicted it: it recorded
/// the anchor's slot PIDs against EVERY clip the anchor plays, so a sibling
/// playlist that plays a LATER clip of the anchor bound the anchor's
/// editorial label onto whatever stream in that clip happens to reuse the
/// PID — a different physical stream, in a different clip, whose own
/// language says so.
///
/// Mutation: harvest over `&title.clips` instead of its first clip — the
/// featurette wears the feature's SDH again.
#[test]
fn an_anchor_proves_pids_only_for_the_clip_its_table_came_from() {
let labels = vec![
sub_label(1, "eng", LabelQualifier::Sdh),
sub_label(2, "fra", LabelQualifier::None),
];
let mut titles = vec![
// The anchor: its stream table is clip 00082's (the first play
// item); it merely CONTINUES into 00090.
title_on_clips(
"00800.mpls",
&["00082", "00090"],
vec![subtitle(0x1200, "eng"), subtitle(0x1201, "fra")],
),
// A sibling playlist over the anchor's SECOND clip. Its subtitle
// reuses PID 0x1200 — PIDs are only unique within a clip — and it
// is Spanish, so nothing about the anchor's English SDH slot
// describes it.
title_on_clips("00451.mpls", &["00090"], vec![subtitle(0x1200, "spa")]),
];
apply_labels(&labels, &mut titles);
assert_eq!(
sub_state(&titles[0]),
vec![
(0x1200, false, LabelQualifier::Sdh),
(0x1201, false, LabelQualifier::None)
],
"the anchor itself keeps the qualifiers the list states for it"
);
assert_eq!(
sub_state(&titles[1]),
vec![(0x1200, false, LabelQualifier::None)],
"a PID in a clip the anchor's table never described is not that \
table's stream 1"
);
}
/// Spec: a vendor codec/variant claim does not follow the ordinal onto a
/// bonus clip that carries a different codec.
///
+326
View File
@@ -219,6 +219,237 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
Some(code)
}
// ── ISO 639-1 → ISO 639-2 ────────────────────────────────────────────────────
/// The complete ISO 639-1 set, paired with its ISO 639-2/**T** (terminological)
/// code. Every two-letter code ISO 639-1 defines appears exactly once.
///
/// /T is the variant the rest of this crate uses — [`lang`] and [`menu_lang`]
/// both normalize to it (`deu` not `ger`, `fra` not `fre`, `zho` not `chi`,
/// `ces`, `nld`, `ell`, `ron`, `slk`, `isl`, `eus`, `hrv`) — so the three
/// tables cannot disagree. `iso639_1_agrees_with_menu_lang` pins that.
///
/// For the 165 codes where 639-2/B and /T are identical this distinction does
/// not arise; it only matters for the 20-odd languages with a distinct
/// bibliographic code.
const ISO_639_1_TO_2: &[(&str, &str)] = &[
("aa", "aar"),
("ab", "abk"),
("ae", "ave"),
("af", "afr"),
("ak", "aka"),
("am", "amh"),
("an", "arg"),
("ar", "ara"),
("as", "asm"),
("av", "ava"),
("ay", "aym"),
("az", "aze"),
("ba", "bak"),
("be", "bel"),
("bg", "bul"),
("bh", "bih"),
("bi", "bis"),
("bm", "bam"),
("bn", "ben"),
("bo", "bod"),
("br", "bre"),
("bs", "bos"),
("ca", "cat"),
("ce", "che"),
("ch", "cha"),
("co", "cos"),
("cr", "cre"),
("cs", "ces"),
("cu", "chu"),
("cv", "chv"),
("cy", "cym"),
("da", "dan"),
("de", "deu"),
("dv", "div"),
("dz", "dzo"),
("ee", "ewe"),
("el", "ell"),
("en", "eng"),
("eo", "epo"),
("es", "spa"),
("et", "est"),
("eu", "eus"),
("fa", "fas"),
("ff", "ful"),
("fi", "fin"),
("fj", "fij"),
("fo", "fao"),
("fr", "fra"),
("fy", "fry"),
("ga", "gle"),
("gd", "gla"),
("gl", "glg"),
("gn", "grn"),
("gu", "guj"),
("gv", "glv"),
("ha", "hau"),
("he", "heb"),
("hi", "hin"),
("ho", "hmo"),
("hr", "hrv"),
("ht", "hat"),
("hu", "hun"),
("hy", "hye"),
("hz", "her"),
("ia", "ina"),
("id", "ind"),
("ie", "ile"),
("ig", "ibo"),
("ii", "iii"),
("ik", "ipk"),
("io", "ido"),
("is", "isl"),
("it", "ita"),
("iu", "iku"),
("ja", "jpn"),
("jv", "jav"),
("ka", "kat"),
("kg", "kon"),
("ki", "kik"),
("kj", "kua"),
("kk", "kaz"),
("kl", "kal"),
("km", "khm"),
("kn", "kan"),
("ko", "kor"),
("kr", "kau"),
("ks", "kas"),
("ku", "kur"),
("kv", "kom"),
("kw", "cor"),
("ky", "kir"),
("la", "lat"),
("lb", "ltz"),
("lg", "lug"),
("li", "lim"),
("ln", "lin"),
("lo", "lao"),
("lt", "lit"),
("lu", "lub"),
("lv", "lav"),
("mg", "mlg"),
("mh", "mah"),
("mi", "mri"),
("mk", "mkd"),
("ml", "mal"),
("mn", "mon"),
("mr", "mar"),
("ms", "msa"),
("mt", "mlt"),
("my", "mya"),
("na", "nau"),
("nb", "nob"),
("nd", "nde"),
("ne", "nep"),
("ng", "ndo"),
("nl", "nld"),
("nn", "nno"),
("no", "nor"),
("nr", "nbl"),
("nv", "nav"),
("ny", "nya"),
("oc", "oci"),
("oj", "oji"),
("om", "orm"),
("or", "ori"),
("os", "oss"),
("pa", "pan"),
("pi", "pli"),
("pl", "pol"),
("ps", "pus"),
("pt", "por"),
("qu", "que"),
("rm", "roh"),
("rn", "run"),
("ro", "ron"),
("ru", "rus"),
("rw", "kin"),
("sa", "san"),
("sc", "srd"),
("sd", "snd"),
("se", "sme"),
("sg", "sag"),
("si", "sin"),
("sk", "slk"),
("sl", "slv"),
("sm", "smo"),
("sn", "sna"),
("so", "som"),
("sq", "sqi"),
("sr", "srp"),
("ss", "ssw"),
("st", "sot"),
("su", "sun"),
("sv", "swe"),
("sw", "swa"),
("ta", "tam"),
("te", "tel"),
("tg", "tgk"),
("th", "tha"),
("ti", "tir"),
("tk", "tuk"),
("tl", "tgl"),
("tn", "tsn"),
("to", "ton"),
("tr", "tur"),
("ts", "tso"),
("tt", "tat"),
("tw", "twi"),
("ty", "tah"),
("ug", "uig"),
("uk", "ukr"),
("ur", "urd"),
("uz", "uzb"),
("ve", "ven"),
("vi", "vie"),
("vo", "vol"),
("wa", "wln"),
("wo", "wol"),
("xh", "xho"),
("yi", "yid"),
("yo", "yor"),
("za", "zha"),
("zh", "zho"),
("zu", "zul"),
];
/// The three two-letter codes ISO 639-1 has since withdrawn, mapped to their
/// replacements. DVD-Video froze its language list on the 1988 edition, so
/// discs authored to the spec carry these spellings and no other table sees
/// them: `iw` Hebrew (now `he`), `in` Indonesian (now `id`), `ji` Yiddish
/// (now `yi`).
const ISO_639_1_DEPRECATED: &[(&str, &str)] = &[("iw", "he"), ("in", "id"), ("ji", "yi")];
/// Map an ISO 639-1 two-letter language code to its ISO 639-2/T three-letter
/// code, accepting the withdrawn DVD-era spellings (`iw`, `in`, `ji`) as
/// aliases for their replacements.
///
/// Covers the WHOLE of ISO 639-1, unlike [`menu_lang`], whose table only spans
/// the languages that show up in Blu-ray menu-graphic filenames. Callers that
/// convert a spec field — a DVD IFO attribute block, say — need the whole set:
/// narrowing it to the menu vocabulary would fold every other language onto
/// one value and make a disc's tracks indistinguishable from each other.
///
/// Case-insensitive and trimmed. Returns `None` for anything that is not an
/// ISO 639-1 code, so callers decide the fallback rather than getting a guess.
pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
let c = code.trim().to_ascii_lowercase();
let c = ISO_639_1_DEPRECATED
.iter()
.find(|(old, _)| *old == c)
.map_or(c.as_str(), |(_, new)| new);
ISO_639_1_TO_2
.iter()
.find(|(two, _)| *two == c)
.map(|(_, three)| *three)
}
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
@@ -789,4 +1020,99 @@ mod tests {
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
/// Structural invariants of `ISO_639_1_TO_2`: it must hold the complete
/// ISO 639-1 set (184 codes), every key a distinct pair of lowercase
/// letters and every value three lowercase letters. A typo'd or duplicated
/// row fails here rather than silently mislabelling a track.
#[test]
fn iso639_1_table_is_complete_and_well_formed() {
assert_eq!(
ISO_639_1_TO_2.len(),
184,
"ISO 639-1 defines 184 two-letter codes; the table must hold all \
of them"
);
let mut keys: Vec<&str> = ISO_639_1_TO_2.iter().map(|(two, _)| *two).collect();
keys.sort_unstable();
let unique = keys.len();
keys.dedup();
assert_eq!(unique, keys.len(), "no ISO 639-1 code may appear twice");
for (two, three) in ISO_639_1_TO_2 {
assert!(
two.len() == 2 && two.bytes().all(|b| b.is_ascii_lowercase()),
"{two:?} is not a two-letter lowercase ISO 639-1 code"
);
assert!(
three.len() == 3 && three.bytes().all(|b| b.is_ascii_lowercase()),
"{three:?} is not a three-letter lowercase ISO 639-2 code"
);
}
// The withdrawn DVD-era spellings resolve, and are not themselves
// rows in the main table (they are aliases, not codes).
for (old, new) in ISO_639_1_DEPRECATED {
assert!(
!ISO_639_1_TO_2.iter().any(|(two, _)| two == old),
"withdrawn code {old:?} must not be a table row"
);
assert_eq!(
iso639_1_to_iso639_2(old),
iso639_1_to_iso639_2(new),
"withdrawn code {old:?} must resolve exactly as {new:?}"
);
}
}
/// The two tables must not disagree. Every two-letter token `menu_lang`
/// accepts has to yield the same ISO 639-2/T code through
/// `iso639_1_to_iso639_2`, so a DVD-sourced language and a Blu-ray
/// menu-label language for the same tongue never produce different
/// `Language` elements.
#[test]
fn iso639_1_agrees_with_menu_lang() {
for (two, three) in ISO_639_1_TO_2 {
if let Some(via_menu) = menu_lang(two) {
assert_eq!(
via_menu, *three,
"menu_lang({two:?}) = {via_menu:?} disagrees with the ISO \
639-1 table's {three:?}"
);
}
}
// Spot-check the /T choice itself, on the languages where /B differs.
for (two, t_code) in [
("de", "deu"),
("fr", "fra"),
("zh", "zho"),
("cs", "ces"),
("nl", "nld"),
("el", "ell"),
("ro", "ron"),
("sk", "slk"),
("is", "isl"),
("hy", "hye"),
("ka", "kat"),
("fa", "fas"),
] {
assert_eq!(
iso639_1_to_iso639_2(two),
Some(t_code),
"the crate standardises on ISO 639-2/T, so {two:?} is \
{t_code:?} and never the bibliographic form"
);
}
}
/// Trimming, case-insensitivity, and the no-guess contract.
#[test]
fn iso639_1_normalizes_input_and_never_guesses() {
assert_eq!(iso639_1_to_iso639_2("RO"), Some("ron"));
assert_eq!(iso639_1_to_iso639_2(" Ro "), Some("ron"));
assert_eq!(iso639_1_to_iso639_2("IW"), Some("heb"));
assert_eq!(iso639_1_to_iso639_2("zz"), None);
assert_eq!(iso639_1_to_iso639_2(""), None);
assert_eq!(iso639_1_to_iso639_2("e"), None);
// A three-letter code is not ISO 639-1 input — that is menu_lang's job.
assert_eq!(iso639_1_to_iso639_2("eng"), None);
}
}
+2
View File
@@ -129,6 +129,8 @@ pub mod progress;
pub mod scsi;
pub mod sector;
pub mod session;
#[cfg(test)]
pub(crate) mod testlog;
pub(crate) mod udf;
pub(crate) mod unlock_bridge;
+325 -7
View File
@@ -91,6 +91,18 @@ pub struct Ac3Parser {
/// across the PES boundary because it may be the core of an AC-3-core +
/// E-AC-3-dependent frame set whose remaining substreams are in the next PES.
saw_extension: bool,
/// The access unit held open across the last PES boundary, ALREADY
/// scanned. The carry-over begins at its first byte, so without this the
/// next call re-scans and re-CRCs every syncframe of it from byte 0 — and
/// an access unit that keeps gaining substreams grows to [`MAX_AC3_BUF`]
/// (1 MiB) before the resync guard drops it, which on a ~2 KiB DVD PES is
/// three orders of magnitude of repeated work per packet.
held: Option<HeldAu>,
/// Test-only: syncframes examined (sized + CRC-gated) by
/// `scan_access_units`. Pins the resume above — the property it exists for
/// is a WORK bound, which no frame-level assertion can observe.
#[cfg(test)]
frames_scanned: u64,
}
impl Default for Ac3Parser {
@@ -107,6 +119,9 @@ impl Ac3Parser {
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
saw_extension: false,
held: None,
#[cfg(test)]
frames_scanned: 0,
}
}
@@ -149,20 +164,56 @@ impl Ac3Parser {
anchor: Option<PtsAnchor>,
at_eos: bool,
marks: &[(usize, super::pesbuf::PesFacts)],
) -> (Vec<Frame>, usize, i64) {
held: Option<HeldAu>,
) -> ScanOut {
let mut frames = Vec::new();
let mut pos = 0usize;
// Running PTS for the next access unit to emit in this call.
let mut frame_pts_ns = base_pts_ns;
let mut anchor = anchor;
let mut pending: Option<PendingAu> = None;
// How far this call has proved there is no further syncframe to
// process; carried over so the held access unit's own bytes (and the
// junk after them) are not searched again next call.
let mut scanned_to = 0usize;
// Resume a held access unit instead of re-deriving it. `keep_from` was
// its first byte, so it starts at 0 of this buffer, and every frame in
// it was sized and CRC-gated on the call that built it.
if let Some(h) = held {
let mut drop_reason = h.drop_reason;
// The one verdict that can have changed since: the track may have
// become poisoned while this access unit was held, and a re-scan
// would have picked that up.
if drop_reason.is_none() && self.tally.is_poisoned() {
drop_reason = Some("track-poisoned");
}
pending = Some(PendingAu {
start: 0,
end: h.end,
pts_ns: base_pts_ns,
duration_ns: h.duration_ns,
drop_reason,
bsid: h.bsid,
});
frame_pts_ns = base_pts_ns + h.duration_ns as i64;
pos = h.scanned_to;
scanned_to = h.scanned_to;
}
while pos < data.len() {
let sync = find_ac3_sync(&data[pos..]);
let start = match sync {
Some(offset) => pos + offset,
None => break,
None => {
// No syncword in `data[pos..]` at all: every byte but the
// last is proved sync-free (a syncword is two bytes and the
// second may still arrive).
scanned_to = data.len().saturating_sub(1).max(pos);
break;
}
};
scanned_to = start;
let remaining = &data[start..];
@@ -182,6 +233,7 @@ impl Ac3Parser {
// Invalid/sub-header frame size (e.g. an E-AC-3 frmsiz of 0/1
// sizing to a 2/4-byte fragment) — skip this sync word.
pos = start + 2;
scanned_to = pos;
continue;
}
@@ -191,6 +243,10 @@ impl Ac3Parser {
}
let frame = &data[start..start + frame_size];
#[cfg(test)]
{
self.frames_scanned += 1;
}
// Decodability gate: a syncframe with an out-of-range bsid (> 16) or
// a failed native CRC (payload corruption) poisons the access unit it
// belongs to — a dependent substream is useless without its parent and
@@ -258,6 +314,7 @@ impl Ac3Parser {
}
pos = start + frame_size;
scanned_to = pos;
}
// Close or HOLD the trailing access unit. The rest of its frame set — its
@@ -271,10 +328,22 @@ impl Ac3Parser {
// substream that extends an access unit — a plain AC-3 track keeps
// emitting every frame in-call.
let mut hold_from = None;
let mut held_out = None;
if let Some(au) = pending {
if !at_eos && (au.bsid >= 11 || self.saw_extension) {
frame_pts_ns = au.pts_ns;
hold_from = Some(au.start);
// Everything below `scanned_to` has been searched already, and
// the access unit's own frames have been sized and CRC-gated;
// record both, rebased onto the carry-over (which starts at
// `au.start`), so the next call resumes instead of redoing it.
held_out = Some(HeldAu {
end: au.end - au.start,
scanned_to: scanned_to.max(au.end) - au.start,
duration_ns: au.duration_ns,
drop_reason: au.drop_reason,
bsid: au.bsid,
});
} else {
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
@@ -306,7 +375,12 @@ impl Ac3Parser {
None => data.len(),
};
(frames, keep_from, frame_pts_ns)
ScanOut {
frames,
keep_from,
frame_pts_ns,
held: held_out,
}
}
}
@@ -319,6 +393,36 @@ struct PtsAnchor {
pts_ns: i64,
}
/// What one `scan_access_units` pass produced: the access units it emitted,
/// the offset in the scanned buffer from which bytes must be carried over to
/// the next call, the PTS to stamp on the access unit that begins that
/// carry-over, and — when the trailing access unit was HELD — the state that
/// lets the next call resume rather than re-derive it.
struct ScanOut {
frames: Vec<Frame>,
keep_from: usize,
frame_pts_ns: i64,
held: Option<HeldAu>,
}
/// A trailing access unit held across the PES boundary, already scanned.
/// Offsets are relative to the carry-over, which begins at the access unit's
/// first byte — so the access unit occupies `0..end`.
#[derive(Clone, Copy)]
struct HeldAu {
/// End of the access unit's bytes.
end: usize,
/// How far the scan that built it had searched (`>= end`). Bytes below it
/// hold no further syncframe to process.
scanned_to: usize,
/// Duration contributed by the access unit's `substreamid`-0 substream.
duration_ns: u64,
/// Decodability verdict reached for it so far.
drop_reason: Option<&'static str>,
/// bsid of the substream that opened it.
bsid: u8,
}
/// An access unit (frame set) under construction: `data[start..end]` is the
/// `substreamid`-0 independent substream frame plus every substream appended to it
/// so far — its dependents, and any additional independent substreams 1..7 with
@@ -477,6 +581,8 @@ impl CodecParser for Ac3Parser {
// non-empty PES today; this is defensive for any future caller).
if pes.discontinuity {
self.acc.clear();
// The held access unit's bytes went with it.
self.held = None;
}
if pes.data.is_empty() {
return Vec::new();
@@ -516,8 +622,13 @@ impl CodecParser for Ac3Parser {
buf.extend_from_slice(self.acc.as_slice());
let marks = self.acc.marks_snapshot();
let data = &buf;
let (frames, keep_from, frame_pts_ns) =
self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks);
let held = self.held.take();
let ScanOut {
frames,
keep_from,
frame_pts_ns,
held: still_held,
} = self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks, held);
if keep_from < data.len() {
let tail = &data[keep_from..];
@@ -531,6 +642,7 @@ impl CodecParser for Ac3Parser {
MAX_AC3_BUF
);
self.acc.clear();
self.held = None;
// Advance the cadence, as both sibling branches below do, so the
// three paths out of this block cannot disagree. Defensive: no
// input reaching this parser was found that both parses frames and
@@ -539,6 +651,7 @@ impl CodecParser for Ac3Parser {
self.flush_pts_ns = frame_pts_ns;
} else {
self.acc.drain(keep_from);
self.held = still_held;
// The carried bytes, when later completed and emitted (next call
// or by flush() at EOS), are timed at the PTS the scanner reached
// here: the PTS of the next access unit in presentation order, or
@@ -548,6 +661,7 @@ impl CodecParser for Ac3Parser {
}
} else {
self.acc.clear();
self.held = None;
// Nothing carried, but keep the cadence so a following PES with no
// PTS (no anchor) continues the timeline instead of reusing a stale
// value.
@@ -569,9 +683,10 @@ impl CodecParser for Ac3Parser {
let buf = self.acc.as_slice().to_vec();
let marks = self.acc.marks_snapshot();
self.acc.clear();
let held = self.held.take();
let out = self
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks)
.0;
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks, held)
.frames;
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
@@ -2199,6 +2314,125 @@ mod tests {
);
}
/// A 256-byte E-AC-3 syncframe with a valid CRC. `strmtyp`/`substreamid`
/// go into byte 2 (A/52 Annex E), which is what `substream_role` reads:
/// (0, 0) OPENS an access unit, (1, 0) is a dependent substream that
/// EXTENDS the open one.
fn eac3_substream_frame(strmtyp: u8, substreamid: u8) -> Vec<u8> {
const SIZE: usize = 256;
let frmsiz = SIZE / 2 - 1; // (frmsiz + 1) * 2 == SIZE
let mut f = vec![0u8; SIZE];
f[0] = 0x0B;
f[1] = 0x77;
f[2] = (strmtyp << 6) | (substreamid << 3) | ((frmsiz >> 8) as u8 & 0x07);
f[3] = (frmsiz & 0xFF) as u8;
f[5] = 16 << 3; // bsid 16 → E-AC-3
finalize_ac3_crc(&mut f);
f
}
/// An access unit closes only at the next `substreamid`-0 independent
/// substream, so one that keeps gaining dependent substreams stays OPEN
/// across PES boundaries and its bytes stay in the carry-over. The
/// carry-over must not be re-scanned — and re-CRCed — from the access
/// unit's first byte on every packet: the buffer only stops growing at
/// MAX_AC3_BUF (1 MiB), and a PES on a DVD is about 2 KiB, so re-deriving
/// the held access unit costs work quadratic in the packets fed.
///
/// Measured directly, because a work bound has no frame-level shadow:
/// `frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
/// Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
///
/// Mutation: pass `None` for `held` in `parse` (or drop the `if let
/// Some(h) = held` resume) — the count returns to the quadratic figure.
#[test]
fn a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet() {
const DEPENDENTS: usize = 200;
let mut parser = Ac3Parser::new();
// Opens the access unit.
let emitted = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
emitted.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..DEPENDENTS {
let f = parser.parse(&make_eac3_pes(eac3_substream_frame(1, 0)));
assert!(f.is_empty(), "a dependent substream extends the open unit");
}
let fed = (DEPENDENTS + 1) as u64;
assert!(
parser.frames_scanned <= 2 * fed,
"the scanner examined {} syncframes for {fed} fed — a held access \
unit must be resumed, not re-derived",
parser.frames_scanned
);
// ...and the resume must not have cost correctness: the whole frame
// set is still one access unit, emitted intact at EOS.
let out = parser.flush();
assert_eq!(out.len(), 1, "the frame set is a single access unit");
assert_eq!(
out[0].data.len(),
256 * (DEPENDENTS + 1),
"every substream of the frame set belongs to it"
);
}
/// A concealed gap must drop the HELD access unit, not just the byte
/// buffer.
///
/// `parse` clears `self.acc` on a discontinuity because the buffered bytes
/// are a truncated frame. The held access unit is described by OFFSETS into
/// exactly those bytes, so it has to go with them. Without
/// `self.held = None`, the next packet resumes a HeldAu whose `start`/`end`
/// were computed against the pre-gap buffer but are applied to the
/// unrelated post-gap bytes — splicing audio across the gap at best, and
/// indexing past the end of the new, shorter buffer at worst.
///
/// The two existing discontinuity tests use plain AC-3 (bsid < 11), which
/// never holds an access unit open, so neither of them reaches this reset.
#[test]
fn a_discontinuity_drops_the_held_access_unit_with_its_bytes() {
let mut parser = Ac3Parser::new();
// Open an access unit and extend it, so a HeldAu exists describing
// offsets into a large buffer.
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(0, 0)))
.is_empty(),
"the access unit is held open, not emitted"
);
for _ in 0..8 {
assert!(
parser
.parse(&make_eac3_pes(eac3_substream_frame(1, 0)))
.is_empty(),
"a dependent substream extends the open unit"
);
}
// The gap. Its post-gap payload is deliberately far SHORTER than the
// held unit's bytes, so a stale HeldAu indexes past its end.
let mut gap = make_eac3_pes(eac3_substream_frame(0, 0));
gap.discontinuity = true;
let _ = parser.parse(&gap);
// Whatever comes out, nothing may carry pre-gap bytes: the truncated
// unit was dropped, so the only access unit that can be emitted is the
// one opened after the gap.
let out = parser.flush();
let total: usize = out.iter().map(|f| f.data.len()).sum();
assert!(
total <= 256,
"a post-gap access unit must not be spliced onto the 9 frames held \
before the gap; got {total} bytes across {} frame(s)",
out.len()
);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
@@ -2248,4 +2482,88 @@ mod tests {
"the unit belongs to the packet its FIRST byte came from"
);
}
/// A track that becomes POISONED while an access unit is held open across a
/// PES boundary must not emit that access unit when it resumes.
///
/// Mutation this catches: deleting (or inverting) the resume-path re-check
/// `if drop_reason.is_none() && self.tally.is_poisoned()` at the top of
/// `scan_access_units`. The verdict on a held access unit is frozen at the
/// moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
/// previous access unit is closed — so the very drop that crosses the
/// poison threshold lands after the next unit's verdict was already taken
/// as `None`. Without the re-check that unit is emitted as an ordinary
/// frame after the track has been judged too damaged to mux: corrupt audio
/// passed through as success, and worse, it is the ONE frame that escapes
/// a whole-track fallback whose entire point is that nothing after the
/// verdict ships.
///
/// Neither existing held-AU test reaches this: both keep a pristine tally.
///
/// The fixture drives the exact interleaving above. `DropTally` poisons
/// once `verified_dropped * 2 > kept + dropped` past the 200-AU gate, so
/// one PES carries 200 CRC-failing `substreamid`-0 syncframes followed by a
/// clean one. Closing corrupt unit #200 (which happens only when the clean
/// frame is reached) latches the poison — after that clean frame's own
/// verdict was computed. Being an E-AC-3 unit that can still gain
/// substreams, it is then HELD across the boundary with `drop_reason:
/// None`, which is precisely the state the re-check exists for.
#[test]
fn a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes() {
// One more than the verdict gate: the Nth close is what poisons.
const CORRUPT_AUS: usize = 200;
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..CORRUPT_AUS {
let mut f = eac3_substream_frame(0, 0);
// Corrupt a payload byte AFTER the CRC was finalized: the header
// (sizing, strmtyp/substreamid, bsid) stays intact so the frame is
// still parsed as a whole access unit and fails only the CRC —
// a VERIFIED drop, the only kind that feeds the poison verdict.
f[100] ^= 0xFF;
assert!(!frame_crc_ok(&f), "the fixture frame must fail its CRC");
data.extend_from_slice(&f);
}
// The clean access unit. Last in the PES, so it is held open.
let clean = eac3_substream_frame(0, 0);
assert!(
frame_crc_ok(&clean),
"the held unit is individually decodable"
);
data.extend_from_slice(&clean);
let emitted = parser.parse(&make_eac3_pes(data));
assert!(
emitted.is_empty(),
"every corrupt unit is dropped and the clean one is held; got {} frame(s)",
emitted.len()
);
// The state the re-check depends on: the track IS poisoned, and the
// held unit was opened before that verdict existed.
assert!(
parser.tally.is_poisoned(),
"fixture must actually cross the whole-track poison threshold"
);
// Resume. The next PES opens a new unit, which closes the held one.
let out = parser.parse(&make_eac3_pes(eac3_substream_frame(0, 0)));
assert!(
out.is_empty(),
"an access unit held across the boundary must not be emitted once \
the track is poisoned; got {} frame(s) totalling {} bytes",
out.len(),
out.iter().map(|f| f.data.len()).sum::<usize>()
);
// ...and nothing may leak at end of stream either.
let tail = parser.flush();
assert!(
tail.is_empty(),
"a poisoned track emits nothing at EOS; got {} frame(s)",
tail.len()
);
assert!(
parser.dropped_frames() > CORRUPT_AUS as u64,
"the held unit must be ACCOUNTED as a drop, not silently discarded"
);
}
}
+18 -6
View File
@@ -29,10 +29,17 @@ pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
}
/// CRC-16 with polynomial 0x002D, init 0, MSB-first, used by the MLP / Dolby
/// TrueHD major-sync header checksum. NOTE: MLP's checksum is the "reversed"
/// scheme — the stored trailer word is the little-endian-read CRC, so this
/// standard CRC must be compared against the stored bytes read big-endian.
/// The caller handles that comparison (see `truehd::mlp_major_sync_ok`).
/// TrueHD major-sync header checksum.
///
/// NOTE: MLP's checksum is the "reversed" scheme. This function emits its two
/// bytes in the OPPOSITE order to a standard little-endian CRC readout, so the
/// caller swaps them back and compares against the stored trailer word read
/// LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
///
/// Comparing big-endian instead is precisely the bug that function was fixed
/// for: it could never validate a real extended major sync, so whole TrueHD
/// tracks were dropped silently. This comment used to prescribe exactly that,
/// and to point at a `truehd::mlp_major_sync_ok` that does not exist.
/// Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
@@ -103,8 +110,13 @@ mod tests {
#[test]
fn crc16_mlp_residue_property_holds() {
// Appending the big-endian CRC zeroes the residue over message+crc — the
// scheme `truehd::mlp_major_sync_ok` relies on.
// Appending the big-endian CRC zeroes the residue over message+crc.
// This is a property of the CRC itself, pinned here so a change to the
// polynomial or the bit order is caught. It is NOT how the TrueHD
// caller validates a major sync: `truehd::mlp_major_sync_crc_ok` does a
// swap-and-XOR compare against the little-endian trailer word. (This
// comment used to claim the caller relied on the residue, and named a
// `truehd::mlp_major_sync_ok` that does not exist.)
let msg = [0xF8u8, 0x72, 0x6F, 0xBA];
let c = crc16_mlp(&msg);
let mut framed = msg.to_vec();
+60 -11
View File
@@ -159,7 +159,13 @@ impl CodecParser for DvdSubParser {
/// Convert a single YCbCr color to RGB, clamping to [0, 255].
///
/// Input: `[padding, Y, Cb, Cr]` (as stored in DVD IFO PGC data).
/// Input: `[padding, Y, Cr, Cb]` (as stored in DVD IFO PGC data). Note the
/// chroma order: the on-disc PGC CLUT is **Cr before Cb** — byte 2 is Cr and
/// byte 3 is Cb. Reading byte 2 as Cb swaps red and blue on every chromatic
/// entry, and is invisible on the achromatic (white/black/grey, Cb = Cr = 128)
/// entries that dominate real palettes, which is how it survives casual
/// inspection. The order is fixed by the DVD-Video PGC format, not by us.
///
/// Returns `[R, G, B]`.
///
/// Range convention (deliberate): this uses the **full-range (JFIF) BT.601**
@@ -175,8 +181,8 @@ impl CodecParser for DvdSubParser {
/// side in lockstep.
pub fn ycbcr_to_rgb(color: &[u8; 4]) -> [u8; 3] {
let y = color[1] as f64;
let cb = color[2] as f64;
let cr = color[3] as f64;
let cr = color[2] as f64;
let cb = color[3] as f64;
let r = y + 1.402 * (cr - 128.0);
let g = y - 0.344 * (cb - 128.0) - 0.714 * (cr - 128.0);
@@ -198,7 +204,7 @@ fn clamp_u8(v: f64) -> u8 {
/// Format a 16-color YCbCr palette as a VobSub `.idx` header for S_VOBSUB
/// CodecPrivate.
///
/// Each entry is `[padding, Y, Cb, Cr]`. Output is a UTF-8 text block carrying
/// Each entry is `[padding, Y, Cr, Cb]`. Output is a UTF-8 text block carrying
/// the two `.idx` header lines mkvmerge / libvobsub expect:
///
/// ```text
@@ -432,24 +438,26 @@ mod tests {
#[test]
fn ycbcr_to_rgb_clamps_overflow() {
// Y=255, Cr=255 → R would be 255 + 1.402*127 = ~433, should clamp to 255
let color = [0x00, 255, 128, 255];
// Y=255, Cr=255 → R would be 255 + 1.402*127 = ~433, should clamp to 255.
// Cr is byte 2 in the on-disc [pad, Y, Cr, Cb] layout.
let color = [0x00, 255, 255, 128];
let [r, _g, _b] = ycbcr_to_rgb(&color);
assert_eq!(r, 255);
}
#[test]
fn ycbcr_to_rgb_clamps_underflow() {
// Y=0, Cr=0 → R = 0 + 1.402*(0-128) = -179, should clamp to 0
let color = [0x00, 0, 128, 0];
// Y=0, Cr=0 → R = 0 + 1.402*(0-128) = -179, should clamp to 0.
// Cr is byte 2 in the on-disc [pad, Y, Cr, Cb] layout.
let color = [0x00, 0, 0, 128];
let [r, _g, _b] = ycbcr_to_rgb(&color);
assert_eq!(r, 0);
}
#[test]
fn ycbcr_to_rgb_red() {
// Approximate red: Y=82, Cb=90, Cr=240
let color = [0x00, 82, 90, 240];
// Approximate red: Y=82, Cr=240, Cb=90 — on disc as [pad, Y, Cr, Cb].
let color = [0x00, 82, 240, 90];
let [r, g, b] = ycbcr_to_rgb(&color);
// R = 82 + 1.402*(240-128) = 82 + 156.9 ≈ 239
// G = 82 - 0.344*(90-128) - 0.714*(240-128) = 82 + 13.1 - 79.97 ≈ 15
@@ -459,6 +467,46 @@ mod tests {
assert!(b < 30, "B should be low for red, got {}", b);
}
/// On-disc DVD PGC CLUT byte order is `[0, Y, Cr, Cb]` — byte 2 is **Cr**
/// and byte 3 is **Cb**, per the DVD-Video PGC format.
///
/// This fixture uses a real on-disc red entry, so it fails if the two
/// chroma bytes are ever exchanged again. It is deliberately NOT built
/// from this crate's own doc comments: those described the order wrongly
/// for a long time, and the previous version of this test inherited the
/// error from them and therefore could not detect it.
///
/// A saturated RED entry therefore appears on disc as Y=76, Cr=255, Cb=85
/// (full-range BT.601 encoding of RGB #FF0000), i.e. bytes
/// `[0x00, 76, 255, 85]`. Reading byte 2 as Cb and byte 3 as Cr instead
/// turns this entry BLUE, which is the exact user-visible symptom.
///
/// The pre-existing `_white` / `_black` tests cannot catch this: they use
/// Cb = Cr = 128, so exchanging two equal bytes is a literal no-op.
#[test]
fn ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb() {
// On-disc [pad, Y, Cr, Cb] for saturated red.
let on_disc_red = [0x00u8, 76, 255, 85];
let [r, g, b] = ycbcr_to_rgb(&on_disc_red);
assert!(
r > 200 && b < 60,
"on-disc red [0,Y=76,Cr=255,Cb=85] must render red-dominant, \
got R={r} G={g} B={b} (R and B swapped => byte 2/3 are transposed)"
);
assert_eq!([r, g, b], [254, 0, 0], "exact full-range BT.601 red");
// And the converse: a saturated BLUE on-disc entry (Y=29, Cr=107, Cb=255)
// must not come out red.
let on_disc_blue = [0x00u8, 29, 107, 255];
let [r2, g2, b2] = ycbcr_to_rgb(&on_disc_blue);
assert!(
b2 > 200 && r2 < 60,
"on-disc blue [0,Y=29,Cr=107,Cb=255] must render blue-dominant, \
got R={r2} G={g2} B={b2}"
);
}
// ── Palette formatting tests ──────────────────────────────────────────
#[test]
@@ -699,7 +747,8 @@ mod tests {
#[test]
fn ycbcr_blue_channel_clamps_high() {
// B = Y + 1.772*(Cb-128). Y=128, Cb=255 → 128 + 1.772*127 ≈ 353 → clamp 255.
let [_r, _g, b] = ycbcr_to_rgb(&[0x00, 128, 255, 128]);
// Cb is byte 3 in the on-disc [pad, Y, Cr, Cb] layout.
let [_r, _g, b] = ycbcr_to_rgb(&[0x00, 128, 128, 255]);
assert_eq!(b, 255, "blue clamps at 255");
}
+73 -1
View File
@@ -1,7 +1,8 @@
//! H.264 (AVC) elementary stream parser.
//!
//! Extracts SPS and PPS NAL units for MKV codecPrivate.
//! Detects keyframes (IDR slices).
//! Detects keyframes: IDR slices, and open-GOP intra-coded access units (the
//! non-IDR recovery points BD titles use — see the intra promotion in `parse`).
//! Each PES packet = one access unit = one frame.
use super::coding::{CodingType, PictureInfo};
@@ -351,6 +352,26 @@ impl CodecParser for H264Parser {
return Vec::new();
}
// Open-GOP resync anchor: an intra-coded access unit is a keyframe even
// when it is NOT an IDR. BD H.264 titles use open GOPs whose only
// random-access point is a non-IDR I-frame (the recovery point, which an
// encoder MAY additionally tag with a recovery_point SEI). Keying off the
// already-measured picture coding type covers every signalling — a
// recovery_point SEI, or a disc that marks the recovery point only by GOP
// structure with no SEI at all — where SEI parsing alone would miss the
// latter. Without this, the B1 resync gate (`mux/resync.rs`), once armed
// by a concealed-gap / clip-seam discontinuity, never sees a keyframe on
// such a tail and drops every frame to EOF, truncating the video. HEVC
// does not hit this: its open-GOP IRAP (CRA) is a distinct NAL type
// already flagged a keyframe (`mux/codec/hevc.rs`); H.264 has no such NAL
// tag, so the intra coding type is the equivalent signal. Consistent with
// `videomap.rs`'s model that a keyframe IS the intra decode-restart point.
// IDR is still flagged by NAL type in the loop above, so a slice whose
// header failed to parse (`coding_type == None`) is unaffected here.
if coding_type == Some(CodingType::I) {
keyframe = true;
}
// Every keyframe is self-contained: re-assert the active SPS/PPS in-band
// ahead of the slices (even when unchanged vs codecPrivate) so a decoder
// that dropped the set at a reset recovers, and a stale avcC re-apply
@@ -694,6 +715,57 @@ mod tests {
}
}
/// Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
/// keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD open-
/// GOP tail instead of dropping every frame to EOF and truncating the video.
/// BD H.264 titles reach random access at a non-IDR I-frame (the recovery
/// point); HEVC gets this free from the CRA NAL type, H.264 does not, so the
/// measured intra coding type is the signal. Red before the intra promotion
/// in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
#[test]
fn open_gop_intra_access_unit_is_a_keyframe() {
// Annex B AU: one non-IDR coded slice (NAL type 1, nal_ref_idc 3 → 0x61)
// whose header decodes first_mb_in_slice = 0 (ue = "1") and slice_type = 7
// (ue = "0001000"), i.e. an all-I slice. The two ue(v) fields pack into
// one byte: "1" + "0001000" = 0b1000_1000 = 0x88.
let au = vec![0x00, 0x00, 0x01, 0x61, 0x88, 0x00];
let mut parser = H264Parser::new();
let frames = parser.parse(&make_pes(au, Some(0)));
assert_eq!(frames.len(), 1, "one PES access unit → one frame");
assert_eq!(
frames[0].coding.map(|c| c.coding_type()),
Some(CodingType::I),
"the slice header measures an I picture",
);
assert!(
frames[0].keyframe,
"an open-GOP intra access unit is a resync-safe keyframe even without an IDR",
);
}
/// Guard the promotion's precision: a non-intra (P) access unit must NOT be
/// promoted to a keyframe, or the resync gate would resume on a frame that
/// carries inter-references to dropped data and the MKV cue index would gain
/// bogus random-access points.
#[test]
fn inter_coded_access_unit_is_not_a_keyframe() {
// Same NAL type 1, but slice_type = 0 (P): ue "1", so with first_mb = 0
// the byte is "1" + "1" padded = 0b1100_0000 = 0xC0.
let au = vec![0x00, 0x00, 0x01, 0x61, 0xC0, 0x00];
let mut parser = H264Parser::new();
let frames = parser.parse(&make_pes(au, Some(0)));
assert_eq!(frames.len(), 1);
assert_eq!(
frames[0].coding.map(|c| c.coding_type()),
Some(CodingType::P),
"the slice header measures a P picture",
);
assert!(
!frames[0].keyframe,
"an inter-coded access unit must never be a keyframe",
);
}
/// The slice-header parse must remove emulation-prevention bytes first
/// (ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is
/// EBSP, and a 0x03 following 0x00 0x00 is not part of the RBSP.
+148
View File
@@ -566,6 +566,20 @@ impl DiscStream {
.into());
}
// The read SOURCE is gone (a prefetch producer thread that
// terminated), not one range of media. Shrinking and retrying at
// the same LBA asks a dead source for data it can never produce,
// and the `skip_errors` branch below would then zero-fill and
// advance over every remaining sector of the title and still
// return success. Abort with the terminal error itself — a
// fabricated SCSI status would be a lie, so this is deliberately
// NOT folded into the transport-failure arm above.
if let Some(e) = res.as_ref().err()
&& e.is_source_terminated()
{
return Err(crate::error::Error::SourceTerminated.into());
}
if (sectors as u32) <= align {
// Bottomed out at one unit (AACS) / one sector (CSS) / the
// extent tail. This is single-pass disc→MKV, which has NO Pass N
@@ -621,6 +635,16 @@ impl DiscStream {
.into());
}
// Same rule as after the first-attempt read: the 60s recovery
// read goes through the same source, so it can be the call
// that discovers the source is dead. Skipping the unit would
// zero-fill the rest of the title as fabricated content.
if let Some(e) = rec.as_ref().err()
&& e.is_source_terminated()
{
return Err(crate::error::Error::SourceTerminated.into());
}
// Recovery read also failed. Skip the WHOLE failed unit or bail.
// Zero-filling and advancing by the full unit keeps
// current_offset unit-aligned, so the next read still begins on a
@@ -1859,6 +1883,130 @@ mod tests {
);
}
/// REGRESSION (round-4 audit): an ordinary MEDIUM ERROR bad sector must
/// keep its identity when it crosses the prefetch producer channel — the
/// same `DiscRead` with its SCSI status, NOT a transport failure.
///
/// `PrefetchedSectorSource::read_sectors` re-wrapped every error that
/// crossed the channel as `Error::IoError`, and `is_scsi_transport_failure`
/// matches `IoError` (the wedged-USB-bridge arm). So a bad sector reached
/// `fill_extents` looking like a dead bus and aborted the pass with a
/// fabricated status 0xFF — the exact inverse of what that short-circuit
/// exists for, and it told the user to power-cycle a healthy drive.
///
/// Asserted on the source, not on a `fill_extents` skip: the producer
/// thread exits for good after sending an error, so nothing downstream of
/// it can genuinely recover the rest of the title (see
/// `dead_prefetch_producer_does_not_silently_zero_fill_the_title`). An
/// assertion that the pass continues could only ever have been satisfied
/// by fabricated zeros.
#[test]
fn bad_sector_keeps_its_identity_across_the_prefetch_channel() {
const COUNT: u32 = 9;
let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let reader = RecordingReader {
capacity: COUNT,
bad_sector: 4,
log: log.clone(),
};
let mut prefetched = crate::sector::PrefetchedSectorSource::new_with_events(
reader,
vec![crate::disc::Extent {
start_lba: 0,
sector_count: COUNT,
}],
8,
1,
None,
None,
)
.expect("spawn producer");
let mut buf = vec![0u8; 8 * 2048];
let err = crate::sector::SectorSource::read_sectors(&mut prefetched, 0, 8, &mut buf, false)
.expect_err("the batch covering the bad sector must fail");
assert!(
!err.is_scsi_transport_failure(),
"a MEDIUM ERROR bad sector is not a dead bus; got {err:?}"
);
assert!(
matches!(
err,
crate::error::Error::DiscRead {
sector: 4,
status: Some(0x02),
..
}
),
"the producer's typed error must survive the channel intact; got {err:?}"
);
}
/// The prefetch producer thread terminates PERMANENTLY on its first read
/// error, so once one bad sector has crossed the channel the source can
/// never deliver another byte. Driving `fill_extents` to exhaustion after
/// that must NOT look like a completed pass: every remaining sector would
/// be fabricated zeros, and DATA LOSS MUST NEVER LOOK LIKE SUCCESS.
///
/// The expectation is the product rule, not the code: a source that is
/// permanently out of data must report that, not answer `Ok(0)` forever —
/// which `commit_read` legitimately reads as an ordinary short read and
/// zero-fills.
#[test]
fn dead_prefetch_producer_does_not_silently_zero_fill_the_title() {
const COUNT: u32 = 30;
let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let reader = RecordingReader {
capacity: COUNT,
bad_sector: 4,
log: log.clone(),
};
let prefetched = crate::sector::PrefetchedSectorSource::new_with_events(
reader,
vec![crate::disc::Extent {
start_lba: 0,
sector_count: COUNT,
}],
8,
1,
None,
None,
)
.expect("spawn producer");
let mut stream = DiscStream::new(
Box::new(prefetched),
synthetic_title(COUNT),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
stream.skip_errors = true;
// Drive the whole title. Bounded so a regression cannot hang the suite.
let mut completed_clean = false;
for _ in 0..(COUNT as usize * 4) {
match stream.fill_extents() {
Ok(true) => continue,
Ok(false) => {
completed_clean = true;
break;
}
Err(_) => break,
}
}
assert!(
!completed_clean,
"the producer died at sector 4, so sectors 4..{COUNT} were never \
read — reporting the pass as complete zero-fills {} of {} bytes \
and calls it success",
stream.lost_bytes,
COUNT as u64 * 2048
);
}
/// AACS unit-alignment skip (the #1 coverage gap). With `unit_align=3`
/// (DecryptKeys::Aacs) and `skip_errors=true`, a single bad mid-extent
/// sector must NOT desync the rest of the title: every `read_sectors`
+297 -2
View File
@@ -359,6 +359,22 @@ fn write_hdr10<W: Write + Seek>(w: &mut W, h: &crate::mux::codec::Hdr10Metadata)
Ok(())
}
/// The Matroska `Language` element the muxer writes for a stream whose source
/// reported `lang`. RFC 9559 §12 defines the element as an ISO 639-2 code and
/// gives no meaning to an empty one; the code for "no language stated" is
/// `und`, and that is what a source with no language table (the HD-DVD EVO
/// stream probe, a Blu-ray STN slot with no language bytes) has to emit. The
/// element is written unconditionally by `MkvMuxer::new`, so this is the one
/// place that decides it — a source-side default would have to be repeated in
/// every scanner and would still leave the muxer able to ship an invalid file.
fn language_or_und(lang: &str) -> String {
if lang.is_empty() {
"und".to_string()
} else {
lang.to_string()
}
}
impl MkvTrack {
/// Build a video track from a [`VideoStream`]. Language defaults to `"und"`;
/// colour metadata is derived from the stream's colour space and HDR format
@@ -530,7 +546,7 @@ impl MkvTrack {
Self {
track_type: ebml::TRACK_TYPE_AUDIO,
codec_id,
language: a.language.clone(),
language: language_or_und(&a.language),
name,
codec_private: None,
is_default: !a.secondary,
@@ -587,7 +603,7 @@ impl MkvTrack {
Self {
track_type: ebml::TRACK_TYPE_SUBTITLE,
codec_id,
language: s.language.clone(),
language: language_or_und(&s.language),
name: String::new(),
codec_private: s.codec_data.clone(),
is_default: false,
@@ -3307,6 +3323,285 @@ mod tests {
);
}
/// Read back the value of the FIRST `Language` element in `data` as a
/// UTF-8 string. `LANGUAGE` (0x22B59C) is a 3-byte EBML ID; its size is
/// always a 1-byte VINT for the short strings this writer emits.
fn first_language_value(data: &[u8]) -> &str {
let pos = find_id(data, ebml::LANGUAGE).expect("Language element must be present");
let len = (data[pos + 3] & 0x7F) as usize;
std::str::from_utf8(&data[pos + 4..pos + 4 + len]).unwrap()
}
/// RFC 9559 §12 / the Matroska `Language` element spec restrict the
/// legacy `Language` element to the Matroska language form (ISO 639-2,
/// three lowercase letters), never ISO 639-1 (two letters). The DVD
/// IFO audio-attribute block itself carries a raw ISO 639-1 code (e.g.
/// "en") on disc — `ifo::parse_audio_attr` converts it to ISO 639-2
/// before returning, via `ifo::dvd_lang_to_iso639_2`. This mimics the
/// real DVD pipeline (`disc/dvd.rs`'s `Stream::Audio` construction) end
/// to end: real on-disc IFO bytes -> `ifo::parse_audio_attr` ->
/// `disc::AudioStream` -> `MkvTrack::audio` -> the muxer -> the emitted
/// `Language` element.
#[test]
fn dvd_two_letter_language_becomes_iso_639_2_in_language_element() {
// AC-3 (coding_mode=0), 48 kHz, 6 channels, on-disc language "en" —
// the exact byte layout `ifo::audio_attr_parsing` pins.
let mut attr_bytes = vec![0u8; 8];
attr_bytes[0] = 0x00;
attr_bytes[1] = 0x05;
attr_bytes[2] = b'e';
attr_bytes[3] = b'n';
let attr = crate::ifo::parse_audio_attr(&attr_bytes, 0).unwrap();
assert_eq!(
attr.language, "eng",
"parse_audio_attr must already return the ISO 639-2 form"
);
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"eng",
"a DVD-sourced ISO 639-1 code must be written as its ISO 639-2 \
equivalent in the Matroska Language element, per RFC 9559 §12"
);
}
/// A source that knows no language at all leaves `language` EMPTY, and the
/// muxer writes the `Language` element unconditionally — so an empty
/// string becomes a zero-length `Language` in the shipped file, which is
/// not a Matroska language form (RFC 9559 §12 wants three ISO 639-2
/// letters) and is not the ISO 639-2 code for "unknown" either.
///
/// This is the state every HD-DVD rip is in: `disc::hddvd`'s EVO stream
/// probe has no language table to read and sets `language: String::new()`
/// on every audio stream it finds. The DVD path normalises to "und" in
/// `ifo::parse_audio_attr`; the guard has to exist at the muxer too, which
/// is the one place every source funnels through.
#[test]
fn a_source_with_no_language_emits_und_not_an_empty_language_element() {
// Exactly what `disc::hddvd::probe_evo_streams` builds.
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: Codec::Ac3Plus,
channels: crate::disc::AudioChannels::Surround51,
language: String::new(),
sample_rate: crate::disc::SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let subtitle_stream = crate::disc::SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: String::new(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
};
for track in [
MkvTrack::audio(&audio_stream),
MkvTrack::subtitle(&subtitle_stream),
] {
let track_type = track.track_type;
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"und",
"track type {track_type}: a stream with no known language must \
emit the ISO 639-2 'undetermined' code, never a zero-length \
Language element"
);
}
}
/// An unmapped or absent DVD language code (bytes 0x00 0x00 in the IFO
/// attribute block) must degrade to the valid Matroska "undetermined"
/// code `und`, never to an empty string or a raw 2-letter code — both of
/// which violate the Matroska language form.
#[test]
fn dvd_unmapped_or_empty_language_becomes_und_in_language_element() {
// Empty IFO language bytes (0x00 0x00).
let mut empty_bytes = vec![0u8; 8];
empty_bytes[0] = 0x00; // AC-3, 48k
empty_bytes[1] = 0x05; // 6ch
let empty_attr = crate::ifo::parse_audio_attr(&empty_bytes, 0).unwrap();
assert_eq!(
empty_attr.language, "und",
"IFO zero bytes (unspecified) must resolve to 'und', not empty"
);
// An IFO code with no known ISO 639-1 -> 639-2 mapping (e.g. a
// fictitious "zz").
let mut unmapped_bytes = vec![0u8; 8];
unmapped_bytes[0] = 0x00;
unmapped_bytes[1] = 0x05;
unmapped_bytes[2] = b'z';
unmapped_bytes[3] = b'z';
let unmapped_attr = crate::ifo::parse_audio_attr(&unmapped_bytes, 0).unwrap();
assert_eq!(
unmapped_attr.language, "und",
"an on-disc code with no known ISO 639-1 -> 639-2 mapping must \
degrade to 'und', never pass through raw"
);
for attr in [empty_attr, unmapped_attr] {
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert_eq!(
first_language_value(&data),
"und",
"an empty or unmapped DVD language code must degrade to 'und', \
never an empty string or an invalid ISO 639-1 code"
);
}
}
/// Build an 8-byte DVD IFO audio-attribute block (AC-3, 48 kHz, 6ch)
/// carrying `code` in the language bytes, run it through the real DVD
/// pipeline (`ifo::parse_audio_attr` -> `disc::AudioStream` ->
/// `MkvTrack::audio` -> the muxer) and return the value that actually
/// lands in the emitted Matroska `Language` element.
fn emitted_language_for_dvd_code(code: &[u8; 2]) -> String {
let mut attr_bytes = vec![0u8; 8];
attr_bytes[0] = 0x00; // AC-3, 48 kHz
attr_bytes[1] = 0x05; // 6 channels
attr_bytes[2] = code[0];
attr_bytes[3] = code[1];
let attr = crate::ifo::parse_audio_attr(&attr_bytes, 0).unwrap();
let audio_stream = crate::disc::AudioStream {
pid: 0xBD80,
codec: attr.codec,
channels: crate::disc::AudioChannels::from_count(attr.channels),
language: attr.language,
sample_rate: crate::disc::SampleRate::from_hz(attr.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let track = MkvTrack::audio(&audio_stream);
let buf = Cursor::new(Vec::new());
let tracks = [track];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
first_language_value(&data).to_string()
}
/// The ISO 639-1 -> ISO 639-2 conversion must cover the WHOLE of ISO
/// 639-1, not just the handful of languages that happen to appear in
/// Blu-ray menu-graphic filenames. A Region-2 disc routinely carries
/// Romanian, Bulgarian, Croatian, Serbian, Slovak, Slovenian, Hebrew,
/// Estonian, Latvian, Lithuanian, Icelandic and so on; if those all
/// collapse to `und`, every one of a disc's subtitle tracks emits the
/// same `Language` value and nothing else tells them apart (DVD streams
/// carry an empty `label`). A valid-but-identical code is worse for the
/// user than the invalid one it replaced, so each of these must reach the
/// emitted `Language` element as its own correct three-letter code.
#[test]
fn dvd_language_outside_the_menu_vocabulary_is_still_mapped() {
assert_eq!(
emitted_language_for_dvd_code(b"ro"),
"ron",
"Romanian ('ro'), common on Region-2 discs, must reach the \
Matroska Language element as 'ron' — not 'und'"
);
// The rest of the set the menu-label table never knew, one per
// language so a single missing table row fails loudly.
for (code, expected) in [
(b"bg", "bul"),
(b"hr", "hrv"),
(b"sr", "srp"),
(b"sk", "slk"),
(b"sl", "slv"),
(b"he", "heb"),
(b"et", "est"),
(b"lv", "lav"),
(b"lt", "lit"),
(b"is", "isl"),
(b"id", "ind"),
(b"vi", "vie"),
(b"fa", "fas"),
] {
assert_eq!(
emitted_language_for_dvd_code(code),
expected,
"DVD language {:?} must map to {expected:?} in the emitted \
Language element",
std::str::from_utf8(code).unwrap()
);
}
}
/// DVD-Video froze its language list on the 1988 edition of ISO 639-1,
/// which spelled Hebrew `iw`, Indonesian `in` and Yiddish `ji`. Real
/// discs authored to that list carry those bytes, so they must map to the
/// same ISO 639-2 codes as the modern `he` / `id` / `yi` spellings rather
/// than degrading to `und`.
#[test]
fn dvd_era_language_aliases_map_to_the_modern_code() {
assert_eq!(
emitted_language_for_dvd_code(b"iw"),
"heb",
"the DVD-era spelling of Hebrew ('iw') must emit 'heb'"
);
assert_eq!(emitted_language_for_dvd_code(b"in"), "ind");
assert_eq!(emitted_language_for_dvd_code(b"ji"), "yid");
// ...and agree with the modern spellings.
assert_eq!(emitted_language_for_dvd_code(b"he"), "heb");
assert_eq!(emitted_language_for_dvd_code(b"id"), "ind");
assert_eq!(emitted_language_for_dvd_code(b"yi"), "yid");
}
/// Widening the table must not weaken the degradation guarantee: a code
/// that is not ISO 639-1 at all still has to yield exactly `und`, a valid
/// Matroska language value, and never a passed-through two-letter code,
/// an empty string, or a guess.
#[test]
fn unknown_dvd_language_still_yields_exactly_und() {
for code in [b"zz", b"qq", b"xx"] {
assert_eq!(
emitted_language_for_dvd_code(code),
"und",
"a code outside ISO 639-1 must degrade to exactly 'und'"
);
}
// Empty language bytes (0x00 0x00) likewise.
assert_eq!(emitted_language_for_dvd_code(&[0x00, 0x00]), "und");
}
#[test]
fn mkv_forced_flag_on_forced_subtitle() {
use crate::disc::SubtitleStream;
+84 -1
View File
@@ -12,8 +12,19 @@ use super::{WriteSeek, ebml};
type MkvHeaderResult = io::Result<(crate::disc::DiscTitle, Vec<(u16, Vec<u8>)>, i64, TrackTable)>;
/// Skip `n` bytes on a forward-only reader (no Seek required).
///
/// A skip that runs out of input before `n` bytes is a TRUNCATED element, and is
/// reported the same way `ebml::read_binary_val` reports a truncated body: as
/// `MkvSourceInvalid`. Discarding `io::copy`'s byte count instead made a skip
/// that hit EOF look like a success, so one corrupt size field mid-Clusters
/// drained the rest of the file, the next element header raised
/// `UnexpectedEof`, and `Stream::read` mapped that to `Ok(None)` — half the
/// title missing, `errors = 0`, `completed = true`.
fn skip_bytes(r: &mut impl Read, n: u64) -> io::Result<()> {
io::copy(&mut r.take(n), &mut io::sink())?;
let skipped = io::copy(&mut r.take(n), &mut io::sink())?;
if skipped != n {
return Err(crate::error::Error::MkvSourceInvalid.into());
}
Ok(())
}
@@ -3193,6 +3204,78 @@ mod tests {
assert!(stream.read().unwrap().is_none(), "clean EOF → None");
}
/// A skipped element whose declared size runs PAST the end of the file is a
/// truncated element, exactly like a truncated `read_binary_val` body — and
/// must be reported the same way, as `MkvSourceInvalid`.
///
/// `skip_bytes` used to discard `io::copy`'s returned count, so the skip
/// "succeeded" having drained the rest of the file. The next element header
/// then hit `UnexpectedEof`, which `read()` maps to `Ok(None)` — a clean end
/// of stream. One corrupt size field mid-Clusters therefore threw away every
/// remaining frame of the title and reported `errors = 0`, `complete = true`.
#[test]
fn a_skip_past_eof_is_an_error_not_a_clean_end_of_stream() {
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
// Frame 1 — read normally.
let block = [0x81u8, 0x00, 0x00, 0x80, 0xAA];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block.len() as u64).unwrap();
cluster.extend_from_slice(&block);
// A VOID whose size field is corrupt: it claims 1 MiB, and the file
// holds only the handful of bytes below. This is the "corrupt size
// field mid-Clusters" case.
ebml::write_id(&mut cluster, ebml::VOID).unwrap();
ebml::write_size(&mut cluster, 1024 * 1024).unwrap();
// Frame 2 — the rest of the title, swallowed by the bad skip.
let block2 = [0x81u8, 0x00, 0x01, 0x80, 0xBB];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block2.len() as u64).unwrap();
cluster.extend_from_slice(&block2);
let bytes = mkv_with_track_and_cluster(1, 1, &cluster);
let mut stream = MkvStream::open(Cursor::new(bytes)).unwrap();
assert!(stream.read().unwrap().is_some(), "first frame reads");
let e = match stream.read() {
Err(e) => e,
Ok(None) => panic!(
"a skip that hit EOF was reported as a CLEAN END OF STREAM: the \
rest of the title is gone and the caller sees errors = 0, \
complete = true"
),
Ok(Some(_)) => panic!("the truncated skip must not yield a frame"),
};
assert!(is_mkv_source_invalid(&e), "{e:?}");
}
/// The honest path this fix must not break: a skipped element whose declared
/// size is exactly satisfied by the bytes present is still skipped cleanly,
/// and the genuine EOF that follows is still `Ok(None)`.
#[test]
fn a_fully_satisfied_skip_still_ends_at_a_clean_eof() {
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
// A VOID that is fully present.
ebml::write_id(&mut cluster, ebml::VOID).unwrap();
ebml::write_size(&mut cluster, 8).unwrap();
cluster.extend_from_slice(&[0u8; 8]);
let block = [0x81u8, 0x00, 0x00, 0x80, 0xAA];
ebml::write_id(&mut cluster, ebml::SIMPLE_BLOCK).unwrap();
ebml::write_size(&mut cluster, block.len() as u64).unwrap();
cluster.extend_from_slice(&block);
let bytes = mkv_with_track_and_cluster(1, 1, &cluster);
let mut stream = MkvStream::open(Cursor::new(bytes)).unwrap();
let f = stream.read().unwrap().expect("the frame after the VOID");
assert_eq!(f.data, vec![0xAA]);
assert!(
stream.read().unwrap().is_none(),
"a genuine EOF at a record boundary is still a clean end"
);
}
// ============================================================
// Block LACING (RFC 9559 §10.3) and TrackNumber→stream routing
// (RFC 9559 §5.1.4.1.1).
+22 -2
View File
@@ -627,16 +627,36 @@ mod tests {
/// accept_from() must reject a connection whose first bytes are NOT the
/// FMKV magic — there is no metadata to drive muxing, so it surfaces
/// NoMetadata rather than proceeding with an empty/garbage title.
///
/// FLAKE FIXED, not the behaviour under test: this used to
/// `shutdown(Shutdown::Both)` the instant the bytes were written. Closing
/// the READ half while the server had not yet read makes the kernel answer
/// the server's in-flight data with an RST, so `accept_from` came back
/// `ConnectionReset` instead of the `InvalidInput` this asserts — rarely
/// when run alone, reproducibly under the loaded concurrent suite, where
/// the server thread is descheduled long enough for the race to open. A
/// logging/protocol assertion that fails at random teaches the next person
/// to re-run until green, which is how a real regression gets waved
/// through.
///
/// Half-closing (`Shutdown::Write`) delivers the same EOF the test needs
/// while leaving the read half open, and blocking on a read until the
/// server drops its end keeps the socket alive for as long as the server
/// is looking at it. The port was already ephemeral (`:0`), so it was
/// never a port collision.
#[test]
fn accept_from_rejects_stream_without_fmkv_header() {
use std::io::Read as _;
use std::io::Write as _;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
let handle = std::thread::spawn(move || {
// Raw non-FMKV bytes (not starting with 'F') then close.
// Raw non-FMKV bytes (not starting with 'F') then EOF.
let mut s = TcpStream::connect(addr).unwrap();
s.write_all(&[0x47u8; 64]).unwrap(); // TS sync bytes, no FMKV magic
s.shutdown(std::net::Shutdown::Both).unwrap();
s.shutdown(std::net::Shutdown::Write).unwrap();
// Park until the server closes, so no RST can overtake the data.
let _ = s.read(&mut [0u8; 1]);
});
let err = match NetworkStream::accept_from(listener) {
Ok(_) => panic!("missing FMKV header must error, not silently accept"),
+256 -10
View File
@@ -17,7 +17,7 @@ use super::codec::startcode::find_start_code;
const PACK_HEADER_ID: u8 = 0xBA;
/// System header start code suffix.
const SYSTEM_HEADER_ID: u8 = 0xBB;
const SYSTEM_HEADER_ID: u8 = crate::consts::pes_stream_id::SYSTEM_HEADER;
/// Program end start code suffix.
const PROGRAM_END_ID: u8 = 0xB9;
@@ -169,6 +169,24 @@ pub struct PsDemuxer {
/// stay byte-identical.
buffer_base: u64,
has_base: bool,
/// Boundary-scan cursor for an unbounded (length-0) PES still waiting for
/// its terminating PS-layer unit: `(buffer offset of the PES start code,
/// buffer offset up to which the search has already proved there is no
/// boundary)`. Both are buffer-relative and are rebased when the buffer
/// drains.
///
/// Without it, every `feed` re-searches the WHOLE accumulated payload from
/// the PES header: the buffer only stops growing at [`MAX_PS_BUFFER`], so a
/// stream that declares an unbounded PES and then never emits a PS-layer
/// start code (a corrupt or crafted VOB) makes the demuxer scan up to 4 MiB
/// per call, quadratic in the bytes fed. Cleared whenever the PES is
/// emitted, so it can never outlive the packet it describes.
pending_scan: Option<(usize, usize)>,
/// Test-only: total bytes examined by `find_ps_boundary`. Pins the cursor
/// above — the property it exists for is a WORK bound, which no
/// packet-level assertion can observe.
#[cfg(test)]
boundary_bytes_scanned: u64,
}
impl Default for PsDemuxer {
@@ -184,6 +202,9 @@ impl PsDemuxer {
buffer: Vec::with_capacity(64 * 1024),
buffer_base: 0,
has_base: false,
pending_scan: None,
#[cfg(test)]
boundary_bytes_scanned: 0,
}
}
@@ -220,6 +241,8 @@ impl PsDemuxer {
// discarded.
let packets = self.extract_packets(true);
self.buffer.clear();
// The buffer the cursor indexes into is gone.
self.pending_scan = None;
packets
}
@@ -288,11 +311,29 @@ impl PsDemuxer {
// start code — the video ES payload is itself full of
// 00 00 01 xx codes that would otherwise cut the PES short.
let end = if pes_packet_len == 0 {
match find_ps_boundary(&self.buffer, sc + 4) {
Some(next) => next,
// Resume where the last call stopped searching for
// THIS PES's terminating unit; anything before that is
// already proved boundary-free.
let from = match self.pending_scan {
Some((pes_at, searched_to)) if pes_at == sc => searched_to,
_ => sc + 4,
};
let (found, searched_to) = find_ps_boundary(&self.buffer, from);
#[cfg(test)]
{
self.boundary_bytes_scanned += searched_to.saturating_sub(from) as u64;
}
match found {
Some(next) => {
self.pending_scan = None;
next
}
// At EOF the rest of the buffer is this PES's
// payload — emit it.
None if flushing => self.buffer.len(),
None if flushing => {
self.pending_scan = None;
self.buffer.len()
}
None => {
// No boundary buffered yet. Normally wait for
// more data, but a corrupt stream could declare
@@ -301,8 +342,10 @@ impl PsDemuxer {
// stops untrusted input forcing unbounded
// allocation. Past the cap, flush what we have.
if self.buffer.len() - sc > MAX_PS_BUFFER {
self.pending_scan = None;
self.buffer.len()
} else {
self.pending_scan = Some((sc, searched_to));
break; // wait for more data
}
}
@@ -338,6 +381,13 @@ impl PsDemuxer {
if self.has_base {
self.buffer_base += pos as u64;
}
// The cursor is a BUFFER offset, so it moves with the drain. A
// pending PES always starts at or after `pos` (the loop broke on
// it, having already consumed everything before it), so neither
// component can underflow.
self.pending_scan = self
.pending_scan
.map(|(pes_at, searched_to)| (pes_at - pos, searched_to - pos));
}
// Trim a start-code-free tail. Every other exit from the loop above
@@ -359,6 +409,10 @@ impl PsDemuxer {
if self.has_base {
self.buffer_base += drop as u64;
}
// A pending PES implies a start code IS in the buffer, so this
// branch cannot run while one is open; drop the cursor anyway
// rather than leave a stale offset behind this drain.
self.pending_scan = None;
}
packets
@@ -380,11 +434,19 @@ const START_CODE_PREFIX_KEEP: usize = 2;
/// PES inside its own payload and re-scan the discarded video bytes as bogus PS
/// units. Restricting the search to PS-layer IDs (>= 0xB9, excluding the video
/// ES codes below it) frames the unbounded PES at the right boundary.
fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
/// Returns `(boundary, searched_to)`. `searched_to` is the offset up to which
/// every byte has been PROVED not to begin a PS-layer boundary start code, so
/// a later call over the same buffer (grown at the tail) may resume there
/// instead of re-scanning the payload from the PES header. When the scan runs
/// off the end, the last two bytes are NOT proved: a `00 00 01` prefix can
/// straddle the next feed's boundary by up to two bytes.
fn find_ps_boundary(data: &[u8], from: usize) -> (Option<usize>, usize) {
let mut pos = from;
while let Some(sc) = find_start_code(data, pos) {
if sc + 3 >= data.len() {
return None;
// A start code whose ID byte has not arrived yet: undecided, so
// the next scan must look at it again.
return (None, sc);
}
let id = data[sc + 3];
if id == PACK_HEADER_ID
@@ -392,11 +454,11 @@ fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
|| id == PROGRAM_END_ID
|| is_pes_stream_id(id)
{
return Some(sc);
return (Some(sc), sc);
}
pos = sc + 4;
}
None
(None, data.len().saturating_sub(2).max(from))
}
/// Check whether a start code byte is a valid PES stream ID that carries payload.
@@ -1752,7 +1814,191 @@ mod tests {
/// after it — exactly the tail a real feed can end on.
#[test]
fn find_ps_boundary_handles_a_bare_start_code_at_the_buffer_head() {
assert_eq!(find_ps_boundary(&[0x00, 0x00, 0x01], 0), None);
assert_eq!(
find_ps_boundary(&[0x00, 0x00, 0x01], 0),
(None, 0),
"an undecided trailing start code is not proved boundary-free"
);
}
/// An unbounded (length-0) PES is terminated by the next PS-LAYER unit, and
/// until one arrives the payload accumulates in the buffer. The search for
/// that unit must not restart at the PES header on every feed: the buffer
/// only stops growing at `MAX_PS_BUFFER` (4 MiB), and a feed is one read
/// batch (at most 510 sectors ≈ 1 MiB, 60 sectors ≈ 120 KiB on an optical
/// drive), so re-scanning from byte 0 costs work quadratic in the bytes
/// fed — up to 4 MiB of scanning per call, for as long as a corrupt or
/// crafted VOB withholds the boundary. A conformant DVD ends every pack
/// within 2048 bytes and never reaches this state.
///
/// Measured directly, because a work bound has no packet-level shadow:
/// `boundary_bytes_scanned` counts the bytes `find_ps_boundary` examines.
/// 256 chunks x 4 KiB of boundary-free payload is 1 MiB of input;
/// re-scanning from the header on every call examines
/// 4 KiB * 256*257/2 = ~128 MiB.
///
/// Mutation: drop the `Some((pes_at, searched_to)) if pes_at == sc` arm so
/// `from` is always `sc + 4`.
#[test]
fn an_unterminated_pes_is_not_rescanned_from_its_header_every_feed() {
const CHUNKS: usize = 256;
const CHUNK: usize = 4096;
let mut demuxer = PsDemuxer::new();
// Unbounded PES header (length 0), then payload that carries no start
// code at all, so no PS-layer boundary is ever found.
demuxer.feed(&[0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00]);
for _ in 0..CHUNKS {
assert!(
demuxer.feed(&[0xFFu8; CHUNK]).is_empty(),
"no boundary yet, so no PES can be emitted"
);
}
let fed = (CHUNKS * CHUNK) as u64;
assert!(
demuxer.boundary_bytes_scanned <= 2 * fed,
"boundary search examined {} bytes over {fed} bytes of payload — \
the scan must advance with the buffer, not restart at the PES header",
demuxer.boundary_bytes_scanned
);
// ...and the cursor must not have cost correctness: the PES still ends
// at the pack header that finally arrives, with its whole payload.
let pack = [
0x00,
0x00,
0x01,
PACK_HEADER_ID,
0x44,
0x00,
0x04,
0x00,
0x04,
0x01,
0x00,
0x00,
0x03,
0xF8,
];
let packets = demuxer.feed(&pack);
assert_eq!(packets.len(), 1, "the pack header terminates the PES");
assert_eq!(
packets[0].data.len(),
CHUNKS * CHUNK,
"the whole accumulated payload belongs to the PES"
);
}
/// The resume cursor is a BUFFER offset, so it must be rebased when the
/// buffer drains — and this is the only test in which a drain actually
/// happens while a cursor is live.
///
/// Mutations this catches, both halves of
/// `self.pending_scan.map(|(pes_at, searched_to)| (pes_at - pos, searched_to - pos))`
/// in `extract_packets`:
/// * `searched_to - pos` -> `searched_to`: the next call resumes `pos`
/// bytes PAST where the previous scan actually stopped, so that window
/// is never examined. Here the terminating pack header lands inside it
/// and is missed outright — the unbounded PES runs on past its real
/// end, swallowing the following unit. That is a CORRECTNESS failure,
/// not a slow path, and the assertion on the emitted packet catches it.
/// * `pes_at - pos` -> `pes_at`: the stale offset no longer equals the
/// PES's post-drain `sc`, the resume arm stops matching and the search
/// restarts at the PES header. Caught by `boundary_bytes_scanned`,
/// which is why the fixture puts 64 KiB of payload in the SAME chunk
/// that opens the PES: that is exactly the span a restart re-examines,
/// so the mutant roughly doubles the bytes scanned.
///
/// `an_unterminated_pes_is_not_rescanned_from_its_header_every_feed` cannot
/// reach either: it opens the unbounded PES as the very FIRST bytes of the
/// very first feed, so nothing ever drains ahead of it, `pos` stays 0 and
/// the subtraction is a no-op. The comment above it asserts neither
/// component can underflow; nothing exercised the arithmetic at all.
///
/// So this fixture puts COMPLETE PS units — a pack header and a
/// length-bounded PES — ahead of the unbounded video PES *in the same
/// chunk*. The loop consumes them, breaks on the unbounded PES, and drains
/// `pos` bytes with `pending_scan` live: exactly the real DVD shape, where
/// a video PES opens partway through a read batch.
#[test]
fn a_resume_cursor_survives_the_drain_of_units_ahead_of_the_unbounded_pes() {
// Payload fed in the SAME chunk that opens the PES. Large enough that
// re-scanning it is unmistakable in `boundary_bytes_scanned`, and it is
// the exact span the un-rebased `pes_at` mutant re-examines.
const PAYLOAD: usize = 64 * 1024;
// A 14-byte MPEG-2 pack header with pack_stuffing_length 0.
const PACK: [u8; 14] = [
0x00,
0x00,
0x01,
PACK_HEADER_ID,
0x44,
0x00,
0x04,
0x00,
0x04,
0x01,
0x00,
0x00,
0x03,
0xF8,
];
// A length-BOUNDED PES — a complete unit, so the loop consumes it and
// `pos` advances past it before breaking on the unbounded PES.
const BOUNDED_PES: [u8; 11] = [
0x00, 0x00, 0x01, 0xE0, 0x00, 0x05, 0x80, 0x00, 0x00, 0xAA, 0xBB,
];
// The unbounded (length-0) video PES whose scan must be resumed.
const OPEN_PES: [u8; 9] = [0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00];
// Bytes drained ahead of the unbounded PES on the first feed — the
// `pos` the cursor must be rebased by.
const DRAINED: usize = PACK.len() + BOUNDED_PES.len();
let mut demuxer = PsDemuxer::new();
let mut first = PACK.to_vec();
first.extend_from_slice(&BOUNDED_PES);
first.extend_from_slice(&OPEN_PES);
first.extend_from_slice(&[0xFFu8; PAYLOAD]);
let head = demuxer.feed(&first);
assert_eq!(
head.len(),
1,
"the bounded PES ahead of the open one is emitted immediately, \
which is what makes the buffer drain with a cursor live"
);
// The terminating pack arrives at the head of the next feed — i.e.
// within `DRAINED` bytes of where the previous scan stopped, which is
// precisely the window an un-rebased `searched_to` skips over.
assert!(
PACK.len() <= DRAINED,
"the terminating pack must fit inside the window a stale \
`searched_to` would skip, or the mutant survives"
);
let packets = demuxer.feed(&PACK);
assert_eq!(
packets.len(),
1,
"the pack header terminates the open PES; a scan resumed past it \
never sees it and the PES runs on"
);
assert_eq!(
packets[0].data.len(),
PAYLOAD,
"exactly the payload fed belongs to the PES"
);
// Work bound: the payload is proved boundary-free ONCE. Re-scanning it
// after the drain roughly doubles this.
assert!(
demuxer.boundary_bytes_scanned <= (PAYLOAD + 1024) as u64,
"boundary search examined {} bytes over {PAYLOAD} bytes of payload — \
a cursor left un-rebased across the drain never matches the PES's \
new offset, so the scan restarts at the header",
demuxer.boundary_bytes_scanned
);
}
/// The boundary-ID check is a 4-way `||`; a mutant that turns the FIRST
@@ -1763,7 +2009,7 @@ mod tests {
let data = [0x00, 0x00, 0x01, PACK_HEADER_ID, 0xAA];
assert_eq!(
find_ps_boundary(&data, 0),
Some(0),
(Some(0), 0),
"a pack header start code alone must register as a PS-layer boundary"
);
}
+5 -2
View File
@@ -1521,8 +1521,11 @@ fn forensic_clip_extents(
tracing::warn!(target: "freemkv::keysource", "fmts: more than one forensic clip on the disc — segment byte space is ambiguous");
return Ok(None);
}
// Addressing variant: these extents are a byte-space map for the forensic
// segment table (`clip_byte_to_lba`), not a read plan — an unrecorded
// extent must stay in place here or every later segment offset shifts.
let exts: Vec<crate::disc::Extent> = udf
.file_extents(reader, &format!("/BDMV/STREAM/{name}"))
.file_extents_addressing(reader, &format!("/BDMV/STREAM/{name}"))
.map_err(io::Error::from)?
.into_iter()
.filter(|&(lba, sectors)| lba > 0 && sectors > 0)
@@ -4440,7 +4443,7 @@ mod tests {
"00001.fmts",
20,
FMTS_CONTENT_LBA - PART_START,
FMTS_CONTENT_SECTORS * 2048,
u64::from(FMTS_CONTENT_SECTORS) * 2048,
true,
)],
subdirs: Vec::new(),
+200 -9
View File
@@ -85,6 +85,12 @@ pub struct PrefetchedSectorSource {
///
/// [`capacity_sectors`]: SectorSource::capacity_sectors
total_sectors: u32,
/// Latched the moment a terminal error crosses the channel. The
/// producer NEVER resumes after sending one (every error arm
/// `return`s), so the closed channel that follows is a dead source,
/// not end-of-stream — and `read_sectors` must keep saying so instead
/// of answering `Ok(0)` for the rest of the title.
producer_failed: bool,
}
impl PrefetchedSectorSource {
@@ -300,6 +306,22 @@ impl PrefetchedSectorSource {
return;
}
let sectors_read = (n / 2048) as u32;
// A genuine zero-byte read with no error
// would otherwise spin this loop forever.
// It is not end-of-stream either: the
// extent list still has `remaining`
// sectors to serve, so the inner source
// has quit early. Send a terminal
// sentinel — dropping `tx` here instead
// would reach the consumer as a clean EOF
// and finalize a TRUNCATED title as
// success, exactly as the panic sentinel
// below exists to prevent.
if sectors_read == 0 {
let _ =
tx.send(Err(crate::error::Error::SourceTerminated.into()));
return;
}
buf.truncate(n);
bytes_read_total = bytes_read_total.saturating_add(n as u64);
if let Some(ref f) = event_fn {
@@ -313,12 +335,6 @@ impl PrefetchedSectorSource {
if tx.send(Ok(buf)).is_err() {
return; // consumer dropped
}
// A genuine zero-byte read with no error would
// otherwise spin this loop forever; treat it
// as end-of-source.
if sectors_read == 0 {
return;
}
offset = offset.saturating_add(sectors_read);
}
Err(e) => {
@@ -344,6 +360,7 @@ impl PrefetchedSectorSource {
recycle_tx,
producer: Some(producer),
total_sectors,
producer_failed: false,
})
}
@@ -486,8 +503,29 @@ impl SectorSource for PrefetchedSectorSource {
let _ = self.recycle_tx.send(filled);
Ok(n)
}
Ok(Err(e)) => Err(crate::error::Error::IoError { source: e }),
// Channel closed (producer finished or panicked).
// Recover the producer's TYPED error rather than blanket-wrapping
// it as `Error::IoError`. That wrapper is a classification, not a
// container: `is_scsi_transport_failure` matches `IoError`, so a
// wrapped MEDIUM ERROR bad sector reached `fill_extents` looking
// like a wedged bridge and aborted the pass instead of being
// skipped under `skip_errors`. `From<io::Error> for Error`
// downcasts the boxed payload back to the exact variant the
// producer sent (status + sense intact); a genuine OS-level
// `io::Error` — the real dead-bus case — still becomes `IoError`.
Ok(Err(e)) => {
// The producer `return`s after every error it sends, so
// this is also the moment the source dies. Latch it: the
// closed channel that follows must not read as EOF.
self.producer_failed = true;
Err(crate::error::Error::from(e))
}
// Channel closed. Clean EOF only if the producer never
// signalled a failure — otherwise it exited without
// delivering the rest of the extents, and answering `Ok(0)`
// would let `fill_extents` mistake a dead source for a short
// read, zero-fill every remaining sector of the title and
// still report the pass as complete.
Err(_) if self.producer_failed => Err(crate::error::Error::SourceTerminated),
Err(_) => Ok(0),
}
}
@@ -1214,7 +1252,12 @@ mod tests {
/// an error (not Ok(0)/EOF), and its ErrorKind must survive the
/// round-trip through the channel. Grounding: the producer's
/// `Err(e) => tx.send(Err(e.into()))` arm, and `read_sectors`'
/// `Ok(Err(e)) => Err(IoError{source:e})`.
/// `Ok(Err(e)) => { self.producer_failed = true; Err(Error::from(e)) }`.
///
/// That arm recovers the producer's TYPED error by downcast rather than
/// blanket-wrapping it as `Error::IoError`, so the kind survives; the
/// `producer_failed` latch it also sets is what turns the channel close
/// that follows into `SourceTerminated` instead of a clean EOF.
#[test]
fn reader_error_propagates_with_kind() {
with_watchdog(Duration::from_secs(10), || {
@@ -1235,6 +1278,63 @@ mod tests {
});
}
/// An inner source that answers a mid-extent read with `Ok(0)` has quit
/// early: the extent list still has sectors to serve. The producer must
/// say so, not simply drop `tx` — a closed channel reads as clean
/// end-of-stream, and `DiscStream::fill_extents` then fabricates zeros
/// for every remaining sector of the title and reports the pass
/// complete. Same rule as the panic sentinel: a truncated title must
/// never be finalized as success.
struct QuitsEarlySource;
impl SectorSource for QuitsEarlySource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Ok(0)
}
fn capacity_sectors(&self) -> u32 {
9
}
}
#[test]
fn inner_source_quitting_early_is_not_reported_as_end_of_stream() {
with_watchdog(Duration::from_secs(10), || {
let extents = vec![Extent {
start_lba: 0,
sector_count: 9,
}];
let mut pf =
PrefetchedSectorSource::new(QuitsEarlySource, extents, 3, None).expect("spawn");
let mut buf = vec![0u8; 3 * 2048];
let mut last = pf.read_sectors(0, 3, &mut buf, false);
// Whatever the first answer, no call may ever settle on a clean
// `Ok(0)`: 9 sectors were promised and none were delivered.
for _ in 0..4 {
if last.is_err() {
break;
}
assert_eq!(
*last.as_ref().unwrap(),
0,
"the source delivered no bytes, so nothing can be Ok(n>0)"
);
last = pf.read_sectors(0, 3, &mut buf, false);
}
let err =
last.expect_err("an undelivered extent list must surface as an error, not as EOF");
assert!(
err.is_source_terminated(),
"the source is gone for good — retrying or skipping cannot \
recover anything; got {err:?}"
);
});
}
/// A read returning a byte count that is not a whole number of
/// sectors (n % 2048 != 0) must be rejected — never truncated and
/// advanced, which would split a sector and hand decrypt a partial
@@ -1472,4 +1572,95 @@ mod tests {
assert_eq!(got.len(), 30 * 2048, "all 10 extents must be drained");
});
}
/// Source whose every read fails with a `DiscRead` carrying the given
/// SCSI status (and optional sense) — an ordinary MEDIUM ERROR bad
/// sector (0x02 + 03/11/00) or the transport-failure sentinel (0xFF).
struct FailingSource {
status: u8,
sense: Option<crate::scsi::ScsiSense>,
}
impl SectorSource for FailingSource {
fn capacity_sectors(&self) -> u32 {
9999
}
fn read_sectors(
&mut self,
lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(self.status),
sense: self.sense,
})
}
}
fn read_one_err(status: u8, sense: Option<crate::scsi::ScsiSense>) -> crate::error::Error {
let extents = vec![Extent {
start_lba: 100,
sector_count: 9,
}];
let mut pf = PrefetchedSectorSource::new(FailingSource { status, sense }, extents, 3, None)
.expect("spawn");
let mut buf = vec![0u8; 3 * 2048];
pf.read_sectors(100, 3, &mut buf, false)
.expect_err("the producer's read failure must surface")
}
/// REGRESSION: an ordinary MEDIUM ERROR bad sector that crosses the
/// prefetch channel must NOT be classified as a SCSI transport failure.
///
/// `read_sectors` used to re-wrap EVERY channel error as
/// `Error::IoError { source }`, and `is_scsi_transport_failure` matches
/// `IoError` (the wedged-USB-bridge arm). So a skippable bad sector
/// arrived at `DiscStream::fill_extents` looking like a dead bus and
/// aborted the whole pass instead of honouring `skip_errors`.
#[test]
fn bad_sector_across_channel_is_not_a_transport_failure() {
with_watchdog(Duration::from_secs(10), || {
let sense = crate::scsi::ScsiSense {
sense_key: 0x03,
asc: 0x11,
ascq: 0x00,
};
let err = read_one_err(crate::scsi::SCSI_STATUS_CHECK_CONDITION, Some(sense));
assert!(
!err.is_scsi_transport_failure(),
"a MEDIUM ERROR bad sector must stay a bad sector across the \
prefetch channel, got {err:?}"
);
// The classification survives because the typed variant does.
assert!(
matches!(err, crate::error::Error::DiscRead { status: Some(s), .. } if s == 0x02),
"expected the producer's DiscRead to survive the channel, got {err:?}"
);
assert_eq!(
err.scsi_sense().map(|s| (s.sense_key, s.asc, s.ascq)),
Some((0x03, 0x11, 0x00)),
"the drive's sense triple must survive the channel"
);
});
}
/// OPPOSITE-DIRECTION CONTROL: a genuine transport failure (status 0xFF,
/// wedged USB bridge) crossing the same channel MUST still classify as a
/// transport failure, so `fill_extents` / sweep keep aborting the pass
/// instead of zero-filling every read against a dead bus.
#[test]
fn transport_failure_across_channel_still_classifies() {
with_watchdog(Duration::from_secs(10), || {
let err = read_one_err(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE, None);
assert!(
err.is_scsi_transport_failure(),
"a 0xFF transport failure must remain one across the prefetch \
channel, got {err:?}"
);
});
}
}
+226
View File
@@ -0,0 +1,226 @@
//! Test-only capture of `tracing` events, so the crate's logging contract is
//! ENFORCED rather than merely commented.
//!
//! # Why this exists
//!
//! The error contract is "Account / Log / Classify", and the round-2 audit
//! found the third leg unverifiable: three separate sites carry long comments
//! insisting they log **the error's OWN code, not a fixed one** — because
//! flattening a scratched sector (E6000) or an over-long allocation-descriptor
//! chain (E6016) into E6017 sends whoever triages them after authoring holes
//! and hides the population that actually exists. Nothing tested that. Putting
//! a literal back at `bluray.rs`'s or `hddvd.rs`'s warn sites broke no test, so
//! the guarantee was a convention one careless edit away from being false. Two
//! of those very sites were changed in round 1, and a third still carried a
//! hardcoded `code = 6017`.
//!
//! Likewise "absence of a log is itself a bug": a refusal that returns the
//! right error but says nothing produces the wrong population downstream (a
//! residual-underrunning drive is indistinguishable from a scratched disc).
//! That is only checkable by looking at what was emitted.
//!
//! # Why a hand-rolled subscriber and not `tracing-subscriber`
//!
//! Same posture as [`crate::harness`]: this crate has exactly one
//! dev-dependency on purpose. `tracing-subscriber` would pull a tree of them to
//! do what forty lines of [`tracing::Subscriber`] does here. The capture is
//! installed with [`tracing::subscriber::with_default`], which is
//! THREAD-LOCAL — so it composes with `cargo test`'s parallel harness and two
//! capturing tests cannot see each other's events.
//!
//! Field values are stringified through [`std::fmt::Debug`]/`Display` because
//! that is all the visitor API offers without a typed schema; tests compare
//! against the string form of the expected constant, which is exactly the
//! comparison that catches a hardcoded code.
#![cfg(test)]
use std::sync::{Arc, Mutex};
/// One captured `tracing` event: its target, level, message and fields.
#[derive(Debug, Clone)]
pub(crate) struct CapturedEvent {
pub target: String,
pub level: tracing::Level,
/// Every field, in emission order, stringified. The implicit `message`
/// field (the format string) is included under the name `message`.
pub fields: Vec<(String, String)>,
}
impl CapturedEvent {
/// The stringified value of `name`, or `None` if the event has no such
/// field.
pub fn field(&self, name: &str) -> Option<&str> {
self.fields
.iter()
.find(|(k, _)| k == name)
.map(|(_, v)| v.as_str())
}
/// The event's message (the `tracing` format string), or `""`.
pub fn message(&self) -> &str {
self.field("message").unwrap_or("")
}
}
#[derive(Default)]
struct Visitor(Vec<(String, String)>);
impl tracing::field::Visit for Visitor {
fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) {
self.0
.push((field.name().to_string(), format!("{value:?}")));
}
fn record_str(&mut self, field: &tracing::field::Field, value: &str) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_u64(&mut self, field: &tracing::field::Field, value: u64) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_i64(&mut self, field: &tracing::field::Field, value: i64) {
self.0.push((field.name().to_string(), value.to_string()));
}
fn record_bool(&mut self, field: &tracing::field::Field, value: bool) {
self.0.push((field.name().to_string(), value.to_string()));
}
}
type Sink = Arc<Mutex<Vec<CapturedEvent>>>;
thread_local! {
/// The sink for a capture ACTIVE ON THIS THREAD, if any. Thread-local so
/// concurrent captures across `cargo test`'s parallel harness never see
/// each other's events, and so a non-capturing thread simply has `None`.
static SINK: std::cell::RefCell<Option<Sink>> = const { std::cell::RefCell::new(None) };
}
/// The ONE process-wide subscriber. Installed once and left installed; it is
/// offered every event and records into whichever thread's sink is active,
/// dropping the event when none is.
struct Capture;
impl tracing::Subscriber for Capture {
// `sometimes`, deliberately NOT the default `always`/`never`. A cacheable
// interest lets `tracing`'s GLOBAL per-callsite cache short-circuit a
// callsite to "off", and under `cargo test`'s parallel harness a cache
// rebuild triggered by ANY other thread can leave it there while a capture
// is live — the capture then observes NOTHING. That is the race that made
// `parse_playlist_unreadable_clip_icb_yields_no_title` flake ~1 run in 15.
// `sometimes` forces `enabled` to be consulted on the emitting thread for
// every event, so a capture always sees its own.
fn register_callsite(
&self,
_meta: &'static tracing::Metadata<'static>,
) -> tracing::subscriber::Interest {
tracing::subscriber::Interest::sometimes()
}
fn enabled(&self, _metadata: &tracing::Metadata<'_>) -> bool {
SINK.with(|s| s.borrow().is_some())
}
// Spans are irrelevant here — nothing in this crate asserts on span
// structure, only on events — so they get a constant id and no storage.
fn new_span(&self, _span: &tracing::span::Attributes<'_>) -> tracing::span::Id {
tracing::span::Id::from_u64(1)
}
fn record(&self, _span: &tracing::span::Id, _values: &tracing::span::Record<'_>) {}
fn record_follows_from(&self, _span: &tracing::span::Id, _follows: &tracing::span::Id) {}
fn event(&self, event: &tracing::Event<'_>) {
SINK.with(|s| {
if let Some(sink) = s.borrow().as_ref() {
let mut v = Visitor::default();
event.record(&mut v);
let meta = event.metadata();
sink.lock().expect("capture mutex").push(CapturedEvent {
target: meta.target().to_string(),
level: *meta.level(),
fields: v.0,
});
}
});
}
fn enter(&self, _span: &tracing::span::Id) {}
fn exit(&self, _span: &tracing::span::Id) {}
}
/// Install the single global capturing subscriber, exactly once.
fn install() {
static INSTALLED: std::sync::OnceLock<()> = std::sync::OnceLock::new();
INSTALLED.get_or_init(|| {
// This is the crate's ONLY `set_global_default`; a second call cannot
// happen. Tolerate it via `.ok()` rather than panic if that ever
// changes — capture then no-ops, which the non-empty assertions catch.
let _ = tracing::subscriber::set_global_default(Capture);
});
}
/// Run `f` with every `tracing` event it emits ON THIS THREAD captured.
///
/// Returns `f`'s value alongside the events, in emission order.
///
/// # Why one global subscriber, not scoped `with_default`
///
/// `tracing`'s per-callsite INTEREST CACHE is GLOBAL, but `with_default` is
/// thread-local. Under `cargo test`'s parallel harness a rebuild of that cache
/// — triggered by ANY other thread registering any callsite for the first time
/// — re-evaluates the target callsite against the global dispatcher, which a
/// scoped subscriber is NOT part of, and can leave it cached "off" while a
/// capture is live, so the capture observes NOTHING. Serialising captures
/// against each other does not help: the poisoning thread is not itself
/// capturing. `parse_playlist_unreadable_clip_icb_yields_no_title` flaked ~1
/// run in 15 on exactly this — an empty event list.
///
/// The robust shape is a single subscriber installed globally for the whole
/// run, whose `register_callsite` returns `sometimes` (so no callsite is ever
/// hard-cached) and which routes each event to the emitting thread's own sink.
/// No scoped-dispatcher transitions, no cross-thread cache race, and concurrent
/// captures on different threads stay isolated by the thread-local sink.
pub(crate) fn capture<T>(f: impl FnOnce() -> T) -> (T, Vec<CapturedEvent>) {
install();
let sink: Sink = Arc::default();
// Save/restore any outer sink so a nested capture on one thread still works.
let prev = SINK.with(|s| s.borrow_mut().replace(sink.clone()));
let out = f();
SINK.with(|s| *s.borrow_mut() = prev);
let events = std::mem::take(&mut *sink.lock().expect("capture mutex"));
(out, events)
}
#[cfg(test)]
mod tests {
use super::*;
/// The capture must actually see events and their field VALUES — if it
/// silently recorded nothing, every logging assertion built on it would
/// pass vacuously, which is worse than having no harness at all.
///
/// Mutation: an `enabled()` returning `false`, or an `event()` that drops
/// the visitor's fields, fails here.
#[test]
fn capture_records_target_level_and_fields() {
let ((), events) = capture(|| {
tracing::warn!(target: "freemkv::testlog", code = 6017u16, clip = ?"A.EVO", "E6017");
});
assert_eq!(events.len(), 1, "exactly one event: {events:?}");
assert_eq!(events[0].target, "freemkv::testlog");
assert_eq!(events[0].level, tracing::Level::WARN);
assert_eq!(events[0].field("code"), Some("6017"));
assert_eq!(events[0].field("clip"), Some("\"A.EVO\""));
assert_eq!(events[0].message(), "E6017");
}
/// A field that is absent must read as `None`, not as an empty string — an
/// assertion of the shape `field("code") == Some(..)` has to be able to
/// fail when the site stops logging the code at all.
#[test]
fn missing_field_is_none_and_capture_is_scoped() {
let ((), events) = capture(|| tracing::warn!(target: "freemkv::testlog", "no fields"));
assert_eq!(events[0].field("code"), None);
// Emitted outside `capture`, so it must not appear in a later capture.
tracing::warn!(target: "freemkv::testlog", code = 1u16, "outside");
let ((), later) = capture(|| {});
assert!(
later.is_empty(),
"capture is scoped to its closure: {later:?}"
);
}
}
+770 -133
View File
File diff suppressed because it is too large Load Diff