281 Commits
Author SHA1 Message Date
Matthew Jackson 245fe89f94 restore freemkv-unlock path dep for local dev (post-v1.6.7)
CI / check-macos (push) Waiting to run
CI / check-windows (push) Waiting to run
CI / test (push) Failing after 47s
CI / lint (push) Failing after 50s
CI / consumers (push) Failing after 56s
leak-guard / leak-guard (push) Successful in 22s
2026-08-21 09:31:40 -07:00
Matthew Jackson e8ab33d6f2 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 3b5e25c3f2 docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson bd122dd3a4 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
Matthew Jackson 7a63bac709 v1.6.6: bump version (freemkv-unlock git-pinned for the tag)
Release / verify (push) Successful in 5s
leak-guard / leak-guard (push) Successful in 35s
Release / release (push) Failing after 10s
Release / test (push) Successful in 2m15s
2026-08-20 20:03:44 -07:00
Matthew Jackson 14eafa0692 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
freemkv release 9560248c49 v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew 14408b75a0 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson e7b6f727d9 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 040bc8b14d drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson 7abcfaab72 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 341e079e1e Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 25893f1be4 CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit 4cd9b7b ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-4cd9b7b
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson efb69e3ba5 Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson 4cd9b7baa1 CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 02e7bc605d Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson 0563b58f2e Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 313460c97f ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson 68a1a55958 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson 0955730045 restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
Matthew Jackson d3103453f1 v1.6.4: bump version (freemkv-unlock git-pinned for the tag) 2026-08-15 09:13:26 -07:00
Matthew Jackson b64a96042e Trim single-clip titles to their playlist marks (1.6.4) 2026-08-15 08:14:31 -07:00
Matthew Jackson 074b1ee829 Stop an uncrackable VTS borrowing another VTS's title key
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.

`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.

Also, three things nothing could catch:

- The demux output filename took the stream's language raw while the
  base beside it was sanitised. A language code is three raw STN bytes
  through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
  encoding on real discs — a NUL in a path fails `File::create` with
  InvalidInput, taking the whole export down before one track file
  opened. `sanitize` now maps control characters too; it did not.

- `css::crack_key_scan`'s short-read handling was dead code under test:
  every source in the module returned the full request, so reverting
  `advance` to the requested count, or dropping the `.max(1)`, left the
  suite green. The `.max(1)` is load-bearing — without it a source
  returning `Ok(0)` never moves the cursor and never increments the
  budget, so the scan spins forever. That mutation now HANGS the test
  rather than failing it, which is the honest demonstration.

- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
  `#[cfg(test)] mod tests`, so it compiled in no configuration and could
  never fire — the dead gate the test above it was written to replace.
  Moved to module scope, and verified it now rejects a wrong constant at
  compile time.
2026-08-11 17:47:56 -07:00
Matthew Jackson 8f9bde9b9a Bound the third hostile CSV, and stop two docs overclaiming
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.

The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.

`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
2026-08-11 16:59:33 -07:00
Matthew Jackson 2d1563c63a Round 2: stop two of round 1's tests claiming more than they prove
Both are mine, and both pass with the bound they "cover" deleted.

`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.

`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.

Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
2026-08-11 11:54:28 -07:00
Matthew Jackson cbb127a175 Bound the commentary-index parse, and stop proving it with a clock
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.

It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.

`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.

The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.

Applies to both `sub_com1_idx` and `aud_com1_idx`.

NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
2026-08-11 11:30:53 -07:00
Matthew Jackson 9c6b7baf83 Cover the dir:// input door, and correct the CSS scramble-gate docs
Three round-1 audit findings on v1.6.0..HEAD.

Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.

Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.

Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
2026-08-11 10:23:37 -07:00
Matthew Jackson 3d738af58f restore freemkv-unlock path dep for local dev (post-v1.6.3) 2026-08-10 09:04:40 -07:00
Matthew Jackson 3e400edd4f v1.6.3: bump version (freemkv-unlock git-pinned for the tag) 2026-08-10 09:04:35 -07:00
Matthew Jackson 9c9de0e095 Carry the same licence and community files as the other repos
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.

SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
2026-08-09 22:08:22 -07:00
Matthew Jackson 539dd0131b Keep the cross-platform jobs for the release candidate
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.

Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.

They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
2026-08-09 20:24:07 -07:00
Matthew Jackson 5677f42c69 Release 1.6.3
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
2026-08-09 20:01:33 -07:00
Matthew Jackson f434b9cf2c Drop six unused crates, and align the rest with the workspace
Two problems, both invisible until the whole graph is looked at together.

DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are
declared here and referenced nowhere -- not in src, tests or benches.
They were being compiled, audited and offered version bumps forever for
no reason. Removing beats bumping.

cbc nearly survived the sweep: a substring search for "cbc" matches 44
occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked
used. Only a word-boundary search exposed it.

SKEW: this crate was the outlier on every shared dependency -- aes 0.8,
rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8
elsewhere. Cargo cannot unify across a major version, so it compiled
BOTH: 32 duplicated crates in the freemkv binary's graph, including two
complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two
digest stacks and two getrandom. Two crypto stacks in one product is
worth removing on its own.

The aes bump is an API rename -- BlockCipher-prefixed traits, Array for
GenericArray -- and the obvious translation uses Array::from_slice,
which the new version deprecates and clippy's -D warnings would reject.
These use the From<[T; N]> conversion the crate points at instead.

3441 tests pass in debug and release. The AACS crypto here is covered by
known-answer tests, so a byte-order or sizing mistake in that rename
could not have passed.
2026-08-09 18:37:24 -07:00
dependabot[bot]andGitHub b17761a24e Bump softprops/action-gh-release from 2 to 3 (#4)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 17:14:30 -07:00
dependabot[bot]andGitHub f2dd1d2e34 Bump actions/checkout from 5 to 7 (#5)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:58:27 -07:00
Matthew Jackson 671c3c7c8c Have Dependabot watch the dependencies, and land its PRs on dev
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".

Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.

Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.

The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
2026-08-09 16:48:31 -07:00
Matthew Jackson c6be942d1a Retire the README version bot
It fired on `release: published` and committed to main, the branch the
release points at its tag, so it could only ever leave main ahead of the
tag. release.sh now makes the same edit inside the release commit.

In this repo it was a no-op regardless: the pattern it rewrote does not
appear in this README, so it has been running and changing nothing.
2026-08-09 07:20:44 -07:00
Matthew Jackson 90d20304cd restore freemkv-unlock path dep for local dev (post-v1.6.2) 2026-08-08 22:18:33 -07:00
Matthew Jackson 50dfe877b9 v1.6.2: bump version (freemkv-unlock git-pinned for the tag) 2026-08-08 22:18:28 -07:00
Matthew Jackson 17622a1b59 Stop trusting a byte offset when the feed is not what was measured
A clip's span is measured over the title's full extents when the disc is
scanned, and a frame is placed by the offset it was read from. Those only
agree while the mux reads every byte the scan counted.

On a disc carrying a forensic segment it does not: the read plan omits the
units belonging to another device group, so fewer bytes are fed than the
spans describe, and the shortfall grows through the title. Every frame past
the first segment then looks earlier than it is — placed in a clip it did
not come from, or dropped at a join for failing marks it was never inside.
The spans still tile one another perfectly, so the check that asks whether
they can be trusted cannot see any of it.

The plan is now compared against the extents it was built from. When they
differ the spans are dropped and placement falls back to timestamps, which
is what that path is for and what the surrounding comment already promised
would happen when an offset stops meaning anything. Ordinary discs are
untouched: with no forensic segment the plan IS the extents.
2026-08-08 19:42:53 -07:00
Matthew Jackson 32824fba5b Make four guards testable, and one guard see both spellings
The provenance guard knew one way to lose a frame's source offset —
writing it as absent. Omitting the field entirely does the same thing,
because the frame type fills it in by default, and the guard read straight
past that. A parser rewritten into the second spelling would have kept its
green light while its track silently went back to being placed by guesswork.
It now reports which spelling it found and where.

The title-count clamp was asserted over a fixture too small to hold more
titles than the cap allows, so the walk stopped when the buffer ran out and
the clamp was never what bounded it — the assertion held with the clamp
deleted. The fixture now carries more entries than the cap.

The subdirectory limit was checked by restating the constant's own
definition; the guard itself had never run, and deleting it changed
nothing. The limit is lowered under test so a real folder can exceed it,
and the test now walks one and requires the refusal.

One test also carried two unrelated grounding notes while the test they
described had none, so reading the note above a test told you about a
different one.
2026-08-08 19:06:37 -07:00
Matthew Jackson 109afcdcf7 Keep both halves of a clip file that two play items share
A playlist may point at the same .m2ts from several play items — a
seamless split, a looped segment, multiple angles. The file has one set of
bytes, so it has one span, and the byte a frame was read from therefore
identifies the FILE, not which play item's range it falls in. Placement
took the first of them and judged every frame against its marks, so
everything past that range's end was treated as material the playlist
excludes and dropped: half the clip missing from the rip, with the
timeline still charged for its full duration.

The offset narrows a frame to the run; only its timestamp can finish the
job, and each play item carries its own marks. A frame is now matched
against the marks of the entry it actually falls in, falling back to the
first when it falls in none, which leaves genuinely-excluded material
dropped as before.

The test for this construction asserted only that such a playlist is
trusted, and never placed a frame from the second range — so the loss it
described in prose was invisible to it. It now places one from every range
and requires them all to survive.

The deferred-mux replay of buffered frames wrote them without their source
offset, quietly sending the head of every such title down the timestamp
heuristic the rest of this work exists to retire. It passes the offset
through now, and the frame writer that omits it is compiled out of the
library entirely: no production path can discard provenance any more.

Also: the AC-3 buffer-reuse test could not see the regression it named.
Feeding equal-sized packets, a fresh allocation per call yields the same
capacity as a reused one. It now feeds a large packet then a small one,
where only a reused buffer keeps the larger capacity.
2026-08-08 17:18:35 -07:00
Matthew Jackson dd749132d5 Audit round: an ADTS frame that cannot hold its own CRC, and three drifts
An ADTS header that declares a CRC follows must be at least nine bytes —
seven of header plus the two the CRC occupies — because the declared frame
length counts them. The structural gate compared against a flat seven and
never read the bit that says whether a CRC is there at all, so a frame
whose own header describes something impossible was accepted and handed to
the muxer as decodable.

The pipeline's spawn doc named Sweep, and the thread Sweep would have
created, as callers to look for. Neither has been in this crate since the
recovery passes moved out. The same paragraph already records fixing this
once, for a different departed caller — it simply drifted again a sentence
later, so it now says to name callers that live here or name none.

Whether a disc is structurally AACS-encrypted was spelled out by hand in
both the fast identify and the full scan. They agreed today; nothing made
them agree tomorrow, and disagreeing would mean the same disc reported
encrypted by one path and clear by the other. There is one definition now,
and the comment that pointed at it by line number points at its name.

The AC-3 parser built a fresh buffer on every packet — of the order of a
hundred thousand times per title — to work around a borrow it cannot
avoid. The copy stays; the allocation does not. The buffer is now lent out
and handed back, and a test pins that, because reverting it would be
invisible in behaviour.

Left alone deliberately: send/send_with_halt and finish/finish_with_halt
look like one action under two names, and are not. After the consumer
fails, one must still accept items and the other must refuse them; that
difference is what stops a producer reading an entire disc for a write
that died on its first frame. Collapsing them was tried here and the
existing test caught it. Both now say so where the choice is made.
2026-08-08 16:55:55 -07:00
Matthew Jackson 3ff2abbff5 Add an error for an image shorter than its recovery data
A recovery resumed against an image that has been truncated since the
previous pass — a full disk, an interrupted transfer, a remount — cannot
repair it: the pass only revisits the ranges the recovery data calls bad,
so everything past the cut stays a hole while the counts still describe a
whole disc. There was no way to say that. Reusing the recovery-data error
would have been wrong, because the recovery data is intact; the image is
not.

Registered in the uniqueness list, the code table and the range check, so
a future variant cannot silently reuse 6015 or map to the wrong code.
2026-08-08 13:35:44 -07:00
Matthew Jackson 03c6d20447 changelog: 1.6.2 2026-08-08 08:53:13 -07:00
Matthew Jackson 237794a6ea ci: run the dependents' test suites, not just a type-check
The `consumers` job checked out all five dependents against this commit and
ran `cargo check --all-targets`. That proves they still COMPILE, which
catches a changed signature and nothing else. The failures worth catching
here keep every signature intact and change behaviour: the library still
builds, the dependent still builds, and the dependent's tests are what go
red. Those never ran.

Run their suites instead. Same checkouts, same patched paths — only the
verb changes.
2026-08-08 08:46:17 -07:00
Matthew Jackson 835e97ce71 Give a frame that outruns its epoch's video a provisional offset
The demuxer interleaves, so at a boundary the streams do not reset on the
same frame: audio for the next segment can reach the muxer before the video
frame that opens the epoch it belongs to. Only the primary video track may
open one, so those frames rode the just-ended offset and landed a whole
segment in the past. Downstream the strictly-monotonic block nudge then
crushed the entire run onto one instant a tick apart, which is audible.

Such a frame is recognised on its OWN raw PTS, which within an epoch only
advances, so a large backward step is unambiguous — a different signal from
the shared frontier, which is what the old false-positive ratchet keyed on.
It then rides a provisional offset private to its track, computed the same
way the video path computes a real one, and drops it the moment the video
retires an epoch, so the run rejoins with no seam.

The offset is deliberately private: it never writes offset_ns, never
advances the frontier and never retires an epoch, so it cannot move the
video timeline. Letting a passive track open a REAL epoch was tried first
and inflated a 476.776 s title to 656.216 s, because every track meets a
boundary at its own pace and the video path rebased again on top.

Measured on a real DVD title with 8 cell boundaries: audio frames stamped
inside one cadence 120 -> 0, outsized gaps 7 -> 0, subtitle 1 -> 0, and the
video span byte-identical at 476.484. On the HD-DVD title the remaining
boundary gap on its first audio track falls from 0.999 s to 0.275 s.
Acceptance: 76 pass, 0 fail, 0 skip.
2026-08-08 00:48:10 -07:00
Matthew Jackson 70e1807e61 Place a straggler by its own epoch's end, not the current frontier
A non-video frame can reach the muxer after the video that opened the next
epoch has already been processed. It carries an old-epoch PTS, so adding
the new offset flings it forward by a whole clip. The remap that exists to
catch this compared the frame against the CURRENT frontier and only
accepted it within one backstep, which is the wrong yardstick: by the time
a straggler is seen the frontier has moved on into a new epoch, and how far
below it the frame lands says nothing about where it belongs.

Retiring an epoch now records the frontier it closed at alongside its
offset, and a straggler is judged against the end of the epoch its PTS came
from. Keeping the whole history rather than one previous offset also fixes
the case a single `prev_offset_ns` cannot express at all: a frame from two
or more epochs back.

Measured on a real HD-DVD title whose second audio track put its final
packet at 12834.587 s in a 6434.100 s file. That frame sits 23 s below the
current frontier — outside the old window, so it was refused — but 0.15 s
below the end of the clip it actually came from. It now lands at 6417.216,
one 32 ms cadence step after its neighbour, and the track spans 6416.160
with no outsized gaps. The other two streams are unchanged.

A title that never rebases retires no epoch, so the lookup finds nothing
and the mapping is bit-identical to before.
2026-08-08 00:02:57 -07:00
Matthew Jackson 418abfe79e test: cover the 1.6.1 provenance work where it was assumed, not asserted
Four parsers (dvdsub, flac, lpcm, mpegaudio) stamp a source offset on
every frame but had no test that read one back, so a regression to
`source: None` would have been caught only by the brace-balanced audit
in codec/mod.rs — a lint, not a behavioural check. Each now asserts the
emitted frame carries the offset of the packet that supplied its first
byte.

The Blu-ray feed spans had no direct test at all. Add one that walks a
multi-item playlist and requires the spans to tile the feed with no gap
or overlap; it catches a one-sector-per-clip drift, which is exactly the
error class that would misattribute frames near a seam.

`no_provenance_still_places_by_marks` asserted only that placement
returned something, which passes for a frame placed in the wrong clip.
Its probe timestamp lands in an overlap between two clips in the real
mark table, so pinning one clip would assert a coin-flip; instead
require the offset to be one that a clip actually containing that
timestamp would produce.
2026-08-07 22:16:02 -07:00
Matthew Jackson 42c62d46b6 Test the DVD title table's sector-offset arithmetic
The TT_SRPT pointer at 0xC4 is a SECTOR offset from the start of
VIDEO_TS.IFO. Reading it as a byte offset lands 1/2048th of the way in and
does not error — it silently returns a different title list, which is how
a disc enumerating 38 titles and an image enumerating 10 looked like a
scan difference rather than a failure.

The table already had tests for its entry layout, dedup and the 99-title
clamp. What it did not have was anything pinning the offset unit itself,
or the grouping of titles by title set, or a truncated table keeping the
rows it can read rather than failing a whole disc.
2026-08-07 22:01:37 -07:00
Matthew Jackson 06d0f9ef8b restore freemkv-unlock path dep for local dev (post-v1.6.1) 2026-08-07 19:08:46 -07:00
Matthew Jackson b3887be90f v1.6.1: bump version (freemkv-unlock git-pinned for the tag) 2026-08-07 19:08:42 -07:00
Matthew Jackson 3ecebf5b40 changelog: mark 1.6.1 UNRELEASED so release.sh can date it
release.sh dates the heading when it cuts the tag, and refuses a bare
'## [1.6.1]' because it cannot tell an undated heading from one it has
already stamped. Shipping a tag whose public changelog says UNRELEASED is
permanent and cannot be corrected inside that tag, so the check is right
to stop rather than guess.
2026-08-07 17:57:29 -07:00
Matthew Jackson 282651186c Guard the invariant that provenance is universal
Every emitted frame must carry the source byte offset of the packet it
came from. That held only for video for as long as it existed, and
nothing asserted it, so nothing caught it: ten parsers built frames with
`source: None` and a multi-clip title could not place audio or subtitles
by byte at all.

Finding them took a brace-balanced scan of the tree by hand — a regex
cannot do it, because a Frame literal contains nested braces and a
non-greedy match stops at the first `}`, which is how five sites survived
the first pass. This is that scan, as a test.

A second test fails if a codec module is added and not listed, since an
unchecked parser is exactly how the gap persists. Modules that emit no
Frame are named explicitly rather than skipped silently.

The guard tripped on itself twice while being written — first on its own
doc comment, then on its own string literals — so it strips line comments
and excludes its own module. A check that matches prose about the defect
rather than the defect is the same mistake in a different place.
2026-08-07 13:37:27 -07:00
Matthew Jackson c3c37380ae changelog: tighten 1.6.1, and record two entries it was missing
The entries had grown into narrative paragraphs; each is now the outcome
first and the mechanism in a sentence or two. The website changelog page
mirrors libfreemkv's, so it inherits this directly.

Two things that shipped in this cycle were not in it at all:

dir:// as a source. It was pulled from 1.6.1 on 2026-08-05 because the
folder reader produced a wrong title list on a real DVD. That defect was
the CSS descramble bug below, fixed since — the acceptance suite now
shows a folder reporting the same 38 titles as its ISO, with matching
streams, languages and runtime, and the CLI has been wired for it all
along. It ships, so it is listed.

The CSS title-table corruption itself: a disc enumerating 38 titles
produced a decrypted image enumerating 10, silently, at exit 0. That is
a data-integrity fix a user needs to know about.
2026-08-07 13:15:36 -07:00
Matthew Jackson 8ec71834dd Stamp the last frames that were still leaving without provenance
A real rip caught these. The warning added for a track placed from
timestamps under a seam plan fired once, on one subtitle track of a
23-clip title, eighteen minutes in — a display set that arrives as a lone
non-PCS segment is emitted straight through rather than accumulated, and
that path still built its frame with source: None.

Auditing the rest the same way found four more: flac, lpcm, mpegaudio and
the passthrough parser. All are one-PES-one-frame, so the packet's own
facts are the unit's facts.

Every Frame construction in every codec parser now carries a source,
checked by walking balanced braces rather than by eye — the earlier
count was taken with a regex that cannot see a block containing nested
braces, which is how these survived the first pass.
2026-08-07 10:01:39 -07:00
Matthew Jackson 991977f297 Place a non-advancing mark table by provenance
Marks that do not advance across a title are normal, not a defect: each
clip file carries its own STC, so one clip's IN has no ordering
relationship to the previous clip's. Refusing those tables dropped
exactly the branched titles this type exists for onto the inference path
that cannot read them, and that is the overrun.

Lifting it was tried once before every track carried a source offset, and
nine audio and subtitle tracks with nothing to place them by pinned
themselves to clip 0 and dropped most of the title. Every parser stamps
provenance now, so the clip comes from the byte offset and the marks keep
one job: whether a frame lies inside its own clip's [in, out].

Still gated on spans_trusted. Without usable spans there is no offset to
place by and inference is all that is left, so a table inference cannot
read is still refused rather than silently truncating a title.

A frame reaching the timestamp path under a plan now says so once per
track. It is not expected any more, and it is how a mostly-dropped track
reached a user without a single line in the log.
2026-08-07 07:49:53 -07:00
Matthew Jackson d4a32d3a26 Every parser now carries provenance, on the one shared buffer
dts, ac3 and truehd assembled access units across PES packets with three
private implementations of the same bookkeeping, and none of them carried
the source byte offset. They now hold a PesBuf, so a unit takes the
timestamp AND the source of the packet covering its first byte, from the
same mark, and no codec can answer that question its own way again.

dts is the reference case: its pts_marks already implemented the rule
correctly, and all 61 of its existing tests -- including the PTS
attribution ones -- pass unchanged on the shared type. That is the
evidence the type preserves the behaviour dts had right.

ac3 kept a single carry-over timestamp and one anchor offset, so it could
only attribute the first unit in a call; it now resolves each unit at its
own offset. truehd had no attribution at all beyond a running clock.

New tests cover the case that motivated this: a unit whose first bytes
arrive in one packet and whose remainder arrives in the next keeps the
FIRST packet's offset. At a clip seam those two packets belong to
different clips, and taking the later one places the audio in the wrong
one.

Clippy on the pinned toolchain caught an empty `if` block left where dts
used to clear stale marks -- restored as an explicit clear, with why it
is still needed once drain keeps the covering mark.
2026-08-07 07:45:32 -07:00
Matthew Jackson 813edd0965 One rule for which packet a unit's facts come from
A parser that assembles an access unit across PES packets has to answer
one question for every unit it emits: which packet carried this unit's
FIRST byte? Its timestamp comes from that packet, and so does the source
byte offset that says which clip of a multi-clip title it belongs to.
The packets that complete the unit carry later values that must not
override it.

That question was being answered three ways. DTS kept a deque of
(offset, pts) markers and took the one covering offset 0. AC-3 kept a
single carry-over timestamp. TrueHD kept its own. PGS and DVD subtitles
each held a pending unit with just a start time. None of them carried
the source offset at all, which is why provenance existed only for video
and why nine audio and subtitle tracks on a branched title had nothing
to place them by.

So it lives in one place now. PesBuf owns the bytes AND the marks;
PesFacts returns a packet's timestamp, source and discontinuity
together, so a parser cannot take one from one packet and another from
the next, because it does not assemble them itself.

The type answers WHICH packet. How a codec reads a timestamp out of that
packet stayed the codec's business at first, and that turned out to be
the same drift one level down: dvdsub read pts alone and returned 0 for
a packet carrying only dts, while everything else took pts.or(dts). Now
there is one derivation. It is not a choice between two fields — for
audio and subtitles there is no reordering, so dts IS the presentation
time and reading it is reading the same value from whichever field the
packet used. Reordering video never calls it; that path reconstructs
display order instead.

Migrated: adts, pgs, dvdsub. dts, ac3 and truehd follow.
2026-08-07 07:19:58 -07:00
Matthew Jackson b16eacd6b4 Restore the refusal: only the video parsers carry provenance
Accepting a table whose marks do not advance was wrong, and a real
22-clip title showed exactly how. The rip failed the drop-volume gate at
84 percent with more frames dropped than kept.

The cause is a fact the code never stated: provenance is VIDEO-ONLY.
Every audio and subtitle parser -- dts, ac3, adts, truehd, pgs -- builds
its frames with `source: None`, while hevc, h264 and vc1 propagate it.
So on that title the video track placed correctly by byte offset and
nine audio and subtitle tracks arrived with nothing to place them by.

The branch added to keep an unprovenanced frame from stranding did the
stranding itself. It held such a frame on its track's current clip,
reasoning that the cursor only ever advances under provenance and so
could not be wrong -- which is true only for a track that eventually
receives some. A track that never receives any stays pinned to clip 0
for the whole title, and every frame past clip 0's OUT mark is dropped.
That is what the disc showed: nine tracks pinned to clip 0, the first
drop nine milliseconds past its OUT.

So the refusal goes back, now with the real reason recorded, and the
test asserts it rather than asserting the behaviour that failed.

Lifting it needs the audio and subtitle parsers to carry provenance
first, stamped from the PES that STARTED each access unit -- the same
rule au_assembly already applies to video. The diagnostics that found
this are kept.
2026-08-07 03:21:21 -07:00
Matthew Jackson 776a4fd6eb Say which frame and which marks caused a provenance drop
The heuristic path has always logged its first drop per track with the
frame's timestamp and the clip marks it was judged against. The
provenance path did not log at all, so a title that dropped MOST of its
frames there failed the volume gate without a single line saying which
frame, which clip, or which marks — a full-length rip produced zero drop
events and an error, which is not a diagnosable failure.

Also pins the all-identical-spans hazard as a test. Every PlayItem
referencing one clip file gives every clip the same span, which passes
the tiling check and is therefore "trusted" while carrying no
information at all about which PlayItem a byte belongs to. That is not
what the discs on hand do — theirs have one distinct span per clip — but
the check conflates "the spans tile" with "the spans distinguish", and
only the second justifies placing a frame by its byte offset.
2026-08-07 03:05:32 -07:00
Matthew Jackson a0f76a4f18 Say how many DISTINCT feed spans a title's clips share
A seamlessly branched title re-references one clip file from several
PlayItems with different mark ranges, and those references share a single
feed span because the bytes are read once. A byte offset alone cannot
then tell them apart.

Whether that is happening on a given title is the fact that decides
whether provenance can identify a clip on its own, and it was not
observable. distinct_spans < clips says it directly.
2026-08-07 02:45:57 -07:00
Matthew Jackson afa0a213d6 Say which placement strategy a title got
Whether a title was placed by the seam plan or fell back to inference is
the single most useful fact about a branched rip, and it was invisible.
The two looked identical in the output, so telling them apart meant
rebuilding and re-ripping — which is exactly what it cost to find that
most branched discs were silently on the inference path.

Log both flags once at plan construction, and log each refusal with the
reason that caused it.
2026-08-07 02:01:47 -07:00
Matthew Jackson e5f92e591a Place a restarting-clock playlist by provenance instead of refusing it
A playlist whose clips each restart their own STC has marks that all
cover the same low values, so they are not points on one title-wide
clock. from_clips refused those tables outright and fell back to PTS-jump
inference.

That refusal predates provenance, and it is only about inference. With
the feed spans tiling the title the clip comes from the frame's byte
offset and the marks are never read across clips: each clip's offset_ns
maps its own private clock onto the output timeline, which is exactly
the right operation for a restarting STC.

Refusing unconditionally therefore turned the seam plan off on most of
the branched discs on hand — the very titles it was written for. They
ran on inference with a timeline minutes past the title's real length:
one 2h29m title came out with its audio and video both spanning 3h15m.

So the refusal now applies only when the spans cannot be trusted, where
it is still load-bearing: without a usable clock or a usable byte offset
there is nothing to place with.

Accepting these tables opens one hole, and it is the same stranding the
refusal guarded against. A frame arriving with no byte offset would fall
through to the mark heuristics, which on this kind of table are
meaningless and can strand a track on a clip it has already left,
dropping the rest of the title. Such a frame now stays on its track's
current clip; the cursor only ever advances under provenance, which is
never wrong about which clip a byte came from.
2026-08-07 01:17:04 -07:00
Matthew Jackson 79dbb2cf84 Resolve a frame clip by binary search, not a linear scan
The provenance lookup runs once per frame per track. A linear scan over the
clip list is fine for the 11-PlayItem fixture and is not fine for the real
hoard: two discs there carry 900 PlayItems in one title, and several carry
250-450, so a scan would be hundreds of comparisons on every one of millions of
frames.

spans_trusted already guarantees the spans tile the feed contiguously and in
order, which makes this a partition point. A repeated clip reuses its first
reference span, so the search walks back to the FIRST entry sharing it — the
bytes are read once, so the material is emitted once, at that entry offset, and
the answer is stable regardless of which duplicate the partition lands on.

Also adds a test at the real 900-clip scale: every clip places, output stays
monotonic across all 900, an arbitrary byte resolves to the right clip out of
ascending order, and a byte past the end belongs to no clip.
2026-08-06 23:22:38 -07:00
Matthew Jackson b9ca75f471 Identify a frame clip by provenance, not by inferring it from timestamps
Four audit rounds each fixed one rule in SeamPlan::place and broke another,
because the question the rules were answering has no answer. Inside a
seamless-branching overlap clip k OUT comes AFTER clip k+1 IN — 57.8s of
overlap on the real fixture table — so a single timestamp is legitimately
inside two clips, and a clip file is not trimmed to its marks, so it also
carries material from before its own IN. No rule over timestamps can say which
clip a frame came from, and each attempt was right for one disc layout and
silently wrong for another: 65s of rewind, 17 minutes stranded, 28 minutes
dropped, 55s refused.

Frames already carry the byte offset they were read from (PesFrame::source,
stamped by the TS demuxer). Clip now carries the byte span its stream occupies
in the title feed, recorded while the extents are gathered. So the clip is a
LOOKUP: the offset falls in exactly one span. There is no decision to get wrong.

Every track of a clip lives in the same stream file and therefore shares one
span, so video, audio and subtitles agree by construction. Divergence between
them — each track guessing separately under its own tolerance — is how audio
and video ended up on different clips and drifted apart in the first place.

spans_trusted gates the whole path: unless the spans tile the feed contiguously
from zero, an offset means nothing and provenance is ignored in favour of the
mark heuristics, which is the 1.6.0 behaviour. A broken map degrades instead of
confidently selecting a wrong clip for every frame. A clip referenced twice
reuses its first span (the bytes are read once) and is still trusted.

Sources that stamp no provenance — a mkv:// remux, the deserialize hop — take
the heuristics, which is what they have always used and where they have always
been right, because they have no overlapping clips to be ambiguous about.

36 timeline tests, six of them new and covering: the overlap case marks cannot
see, all tracks agreeing, out-of-marks material dropped AND counted, a holed
span map, a discontiguous one, a repeated clip, and no provenance at all.
2026-08-06 23:08:13 -07:00
Matthew Jackson adc8ee37be Say when an entry is omitted for not being a plain file
A symlink to a directory — a normal way to keep tens of gigabytes of streams
off the system disk — was dropped from the plan silently. `entry.file_type()`
reports the LINK, so such a subtree never enters `dirs`, and `metadata()` then
follows it and reports a directory, so it fails the is_file() test and was
skipped with nothing said.

The result is the worst class this release exists to close: PLAYLIST and
CLIPINF still synthesize, so the folder scans and enumerates titles, every clip
resolves to no extents, the mux takes its clean-EOF path, and a near-empty MKV
is written at exit 0.

It is still skipped rather than followed — following link targets invites
cycles and escapes from the folder — but at the same volume as the
unrepresentable-name skip eight lines above.
2026-08-06 22:48:27 -07:00
Matthew Jackson c3e82f28b5 Enforce the no-rewind invariant instead of guessing at clip crossings
Round 7 enumerated the state space of SeamPlan::place and found three more
holes in the mark heuristics — the third consecutive round to find a defect
here, each time in the guard the previous round added:

- a clip file opening ON its own IN mark, after a tail frame past the previous
  clip OUT had already advanced the cursor, read as another crossing and jumped
  a clip too far: 28 minutes of clip 6 dropped;
- the last clip was unguarded entirely (`clip + 1 < len` skipped the check), so
  the 9->10 seam rewound 75.6s with dropped_total() at 0 — the exact round-5
  signature, reachable without any corrupt input;
- a glitch mid-clip satisfies "inside the current clip", so the round-6
  stranding survived for the rest of that clip.

Three patches to the same heuristic, three wrong in a new way. So stop patching
it. Everything above the placement is still heuristics over marks, but the
property they exist to serve is now checked DIRECTLY: a track output must not
run backwards. A placement that would rewind by more than
DISCONTINUITY_BACKSTEP_NS is refused; the scan then looks for a LATER clip
containing the frame that does not rewind, which is exactly what a genuine
overlap crossing looks like; and if none exists the frame is dropped and
counted, because emitting it anywhere moves the track backwards.

The tolerance is the backstep constant because B-frame reorder legitimately
emits out of order by a fraction of a second, while a rewind that matters is
orders of magnitude larger.

The cursor moves ONLY when a later clip actually accepts the frame. My first
version of this advanced on a failed search, which strands a track exactly as
before — caught immediately by the round-6 regression test.

All 29 previous timeline tests still pass, including
continuity_preserves_forward_gap. The three round-7 scenarios now have a test
each, with the real 00801.mpls numbers.
2026-08-06 12:31:23 -07:00
Matthew Jackson 6b67c52249 Round 6: a large backstep only crosses when it would be misplaced
The round-5 fix advanced the clip cursor on ANY backward step over 3s. That
also fires for a corrupt PTS, and for a legitimate STC discontinuity inside one
clip — and nothing moves the cursor back, because a forward step matches
neither past_out nor stepped_back. Every later frame then sits below the new
clip IN and is dropped: on the fixture table that is ~17 minutes of one track
gone, and the only volume gate compares total drops against ALL tracks frames,
so it exits 0.

The branch now also requires the frame to be INSIDE the current clip marks,
which is the only case that would otherwise be silently placed at the old
offset — the rewind. A frame outside them needs no help: the containment check
drops and counts it, the cursor stays put, and the next good frame is placed
normally. Both behaviours have a test, each confirmed to fail without the guard.

Separately, the dir:// PES input path scanned the folder and never applied the
encryption verdict scan_dir exists to produce, so the same folder ripped
through one door and failed through the other asking for a key it does not
need. That logic now lives in one function, session::apply_folder_encryption_verdict,
called by both.
2026-08-06 11:02:48 -07:00
Matthew Jackson dd9e92ed52 Retry a short read instead of skipping past it
The previous commit bounded the crack scan inspection by the bytes actually
read, which stopped it examining stale buffer bytes from an earlier batch. But
the cursor still advanced by the REQUESTED count, so those sectors were skipped
outright — trading "scans the wrong data" for "silently scans less than it
thinks", on exactly the damaged media where a title key is hardest to find.

The cursor now advances by what was read, so the next iteration resumes where
the read stopped. Floored at one sector so a source returning Ok(0) cannot spin.

Found by reading the fix again rather than by the next audit round.
2026-08-06 10:15:53 -07:00
Matthew Jackson 65ccbcbd92 Scan only the sectors the crack actually read
read_sectors returns the number of bytes written and a source may return Ok
with fewer than asked — a recovery read over a damaged region does exactly
that. The crack loop discarded the count and inspected all n sectors of a
buffer that is REUSED across batches, so the tail still held the previous
batch sectors.

Cracking a key from those means cracking from data belonging to a different
extent, possibly a different VTS, while crack_span records the CURRENT one. A
key that opens nothing in this region is then installed, and its wrong
descrambles are only partly caught by the per-sector crib.
2026-08-06 09:31:37 -07:00
Matthew Jackson aa149099ec Make the public per-sector descramble refuse a non-pack sector
descramble_region was fixed to require the MPEG-2 pack start code, but
descramble_sector — the PUBLIC per-sector entry point, and the one the
module-level example tells callers to use — still keyed on the byte 0x14 flag
bits alone. The crate own documented guidance therefore led straight back into
the defect this release exists to fix: a VIDEO_TS.IFO sector holding 0x15 at
0x14 while starting 00 26 00 00 loses 1912 of its 2048 bytes, and because that
sector carries TT_SRPT the disc enumerates 38 titles while an image decrypted
from it enumerates 10, at exit 0.

It has no callers inside the crate, which is exactly why it survived three
rounds: nothing exercised it. Putting the guard inside the function rather than
in each caller is what keeps the safe path the easy one.

The integration test that covers it built its sector from the flag byte alone,
which no real scrambled sector looks like, so it stopped representing the path
it names — the same fixture-realism gap already fixed in four other places this
release.
2026-08-06 09:26:15 -07:00
Matthew Jackson 7deacf1761 A clip join must never rewind the output timeline
A hole in the seam fix itself, found in audit round 5 and confirmed by
measurement against the real 00801.mpls marks already in the test file.

Nothing upstream trims a clip stream to its marks, so a clip file can open with
material from BEFORE its IN. With the cursor on clip 5 (7708.99..7910.79) a
frame at 7845.00 — clip 6 pre-mark lead-in, 8s below clip 6 IN of 7853.00 —
sits further from that mark than the 250ms tolerance, so neither crossing rule
fires. The cursor stays on clip 5, and 7845.00 IS inside clip 5 range, so the
frame is PLACED with clip 5 offset.

Measured: output went from 3714.51s to 3649.51s. Backwards 65 seconds, with
dropped_total() still 0 — no counter, no gate, nothing noticed — while the
whole 57.8s overlap band was emitted a second time over clip 5 written tail.
That is the collide-and-flatten symptom this type exists to remove.

A track own PTS only ever runs forward inside a clip, so a backward step larger
than DISCONTINUITY_BACKSTEP_NS is always a clip change. Such a frame now
advances to the first clip that could contain it — which terminates on its own,
rather than running to the end of the list, which is what the next_in bound
exists to prevent — and a frame still below that clip IN is material the
playlist excludes, so the containment check drops and COUNTS it.

All 27 pre-existing timeline tests still pass, including
continuity_preserves_forward_gap, which is the one that catches an over-eager
heuristic here. The new test asserts the output never moves backwards at a
join and was confirmed to FAIL before the fix.
2026-08-06 09:20:39 -07:00
Matthew Jackson 47789f71bc Test the demux seam gates too
The same gap as the MKV muxer: DemuxSink builds its timeline with
TimelineContinuity::with_clips, but no test ever handed it a title carrying
usable PlayItem marks, so frames_mapped and dropped_total() were always zero
and both gates in finish() were unreachable.

Deleting either left the whole suite green while a demux:// export of a
seamless-branching title wrote a directory of zero-byte track files beside a
populated chapters document and reported success at exit 0.

The test gives the title two clips with marks that exclude every frame, and was
confirmed to FAIL with the gate removed.
2026-08-06 09:05:30 -07:00
Matthew Jackson c31d9fc88e Test the seam gate that stops a drift-broken title being skipped
Audit finding: both seam gates in MkvMuxer::finish were dead code under test.
set_clips was never called anywhere in the suite, so continuity.dropped_total()
was always zero and neither branch could be reached.

The ordering of the two zero-frame checks is load-bearing and the comment says
so: a title the seam plan emptied also has a zero frame count, and MkvInvalid
is classified by is_skippable_title_stub as an empty nav/menu stub — so
reporting it that way makes an all-titles rip drop a real feature and finish the
rest at exit 0. Swapping the checks left the entire suite green.

The new test builds a two-clip plan (from_clips needs at least two, strictly
increasing) whose marks exclude every frame, and asserts both that the error is
SinkWroteNothing and that it is NOT classified as skippable. Confirmed to FAIL
with the two checks swapped.
2026-08-06 08:45:58 -07:00
Matthew Jackson 0e8c31a9c3 Round 3: fix three defects introduced by the round-2 fixes
Auditing my own fixes found all three. None were in the original code.

The VTS crack sort was byte-wise case-SENSITIVE while the filters that select
those files (vts_group_of / is_title_vob) are case-insensitive. On a
case-sensitive volume a set holding vts_01_1.vob beside VTS_01_2.VOB sorted
part 2 first, because V (0x56) precedes v (0x76) — reintroducing exactly the
budget-exhaustion the ordering exists to prevent. Now sorted on the same
uppercase normalisation the filters apply.

Refusing a name that round-trips to empty aborted the WHOLE plan. A sidecar
folder named with a single emoji made a backup un-rippable that 1.6.0 handled
fine, and reported it as a collision with a file that does not exist. It is now
skipped with a warning: the entry is unaddressable either way, but one
irrelevant file should not cost the user their rip.

The mtime check now applies only to files whose CONTENT the plan read — the
IFOs, whose bytes 0xC0/0xC4 place every VOB. Everything else is planned from
size alone, which is already checked, so comparing mtime there bought nothing
and risked a real false positive: disc backups commonly live on exFAT/FAT32,
which stores local time, so a long rip spanning a DST transition would see a
whole-hour shift on an untouched multi-gigabyte VOB and abort hours in.
2026-08-06 08:25:54 -07:00
Matthew Jackson 03d088abfc Catch names the reader cannot tell apart, and in-place content changes
Two dirimage findings, both silent-wrong-output.

The per-directory uniqueness check compared raw host names, but the reader does
not see raw host names: parse_udf_name trims leading and trailing whitespace
and drops any code unit char::from_u32 rejects — which is every half of the
surrogate pairs the encoder emits for non-BMP characters. So " 00000.m2ts" and
"00000.m2ts", or "A<astral>.m2ts" and "A.m2ts", were two entries at plan time
and ONE name at read time. find/read_file take the first match, so a title
resolved to the wrong file extents and muxed the wrong bytes at exit 0 — the
exact shadowing DirNameCollision exists to prevent.

The key is now derived by round-tripping the name through the very encoder and
parser that will be used, so it cannot drift from them. A name that survives
that round trip as empty is refused outright: it would exist in the image and
be addressable by nothing. The new test builds a real folder, calls plan, and
was confirmed to FAIL with the fix reverted.

Separately, the plan-vs-read revalidation compared file LENGTH only, while the
plan depends on CONTENT: a DVD VOB placement comes from bytes 0xC0/0xC4 of its
IFO, and IFOs occupy a whole number of sectors so an in-place rewrite keeps the
length. A re-authoring tool touching the folder mid-rip would pass the size
check while every title extent pointed at stale sectors. mtime is now compared
alongside size, and only when both sides report one, so a filesystem without
timestamps falls back to the old behaviour rather than failing every read.
2026-08-06 08:00:23 -07:00
Matthew Jackson 84e0ba9fa8 Anchor the folder encryption probe on a unit boundary
probe_folder_encryption sampled AACS units starting from the largest extent
anywhere in the disc. AACS units are 3 sectors and a unit boundary is only
guaranteed at the START of a clip, so that anchor is only correct when the
largest extent happens to be a clip first extent.

For any clip over ~2 GiB it is not. The planner caps an allocation descriptor
at MAX_AD_BYTES = 524287 sectors, so every full piece of a split file ties on
sector_count and max_by_key returns the LAST tie — an extent beginning
(k-1)*524287 sectors into the file. 524287 % 3 == 1, so that start misses the
unit boundary for two file sizes in three.

The 6144-byte sample windows then begin mid-source-packet and the byte read as
the CPI flag is content. Both verdicts fail in a costly direction: a decrypted
folder is rejected as encrypted (DirImageEncrypted on something perfectly
rippable — the exact case scan_dir was added to rescue), or real ciphertext
reads as clear and the mux writes it out as video at exit 0. is_unit_aligned
cannot catch it: it measures against the same wrong base.

Now the largest TITLE first extent, which is unit-aligned by construction and
is also the more meaningful sample — the main feature rather than whichever
fragment happened to be biggest.

The existing tests could not reach this: their fixture m2ts is 786,432 bytes,
a single extent, which is always its own first.
2026-08-06 07:37:59 -07:00
Matthew Jackson bc5e3453b4 Order a VTS crack by filename, not by directory order
Round 1 removed a largest-first sort from resolve_vts_key because
largest-first is the 1.5.1 garbage bug. Auditing that fix showed it was only
half right: `planned` comes from walking the UDF directory, which yields File
Identifier Descriptors in on-disc authoring order with nothing sorting them. So
deleting the sort did not restore playback order, it left the order undefined —
whatever the disc happened to list first.

DVD-Video numbers a title sets VOBs in playback order by spec (VTS_xx_1.VOB ..
VTS_xx_9.VOB, single digit), so ascending filename IS playback order and is
deterministic regardless of how the directory is laid out.

Order decides correctness here: crack_key shares one sector budget across the
whole extent list, a CSS DVDs biggest cell opens with a long clear run, and CSS
recovers the title key from scrambled data itself. Starting in the wrong place
can exhaust the budget without ever meeting scrambled data, whereupon the
caller falls back to the disc-wide key and the whole VTS is descrambled wrongly
— corrupt PES behind an intact header, written out at exit 0.
2026-08-06 07:31:35 -07:00
Matthew Jackson 6d9791affc Audit round 1: playback order, silent title drops, image durability
Four fixes from the first audit round. Every finding was verified against a
pinned tree and read directly before being accepted.

resolve_vts_key sorted a VTS title-VOB extents largest-first. That is the 1.5.1
garbage bug, and it grew back in a new code path: the comment claimed it
"matched the scan heuristic", but that heuristic WAS the bug and had already
been fixed in decrypt_keys_for_title, which documents the rule (PLAYBACK ORDER,
never largest-cell-first) and pins it with a regression test. A CSS DVDs biggest
cell opens with a long clear run and crack_key shares one sector budget across
the extent list, so starting there can exhaust it without ever MEETING
scrambled data — and CSS recovers the key from scrambled data itself. The crack
then returns None, the caller falls back to the disc-wide key, and every VOB in
that VTS is descrambled wrongly: corrupt PES behind an intact header, written
out as a complete extract at exit 0.

parse_pgcit dropped titles silently in THREE places — an unparseable PGC, an
out-of-range PGC index, and a truncated entry table. The finder caught one; the
other two turned up on reading the function. parse_vmg already counts and warns
per skipped title SET for exactly this reason, and this was the last place a
disc could quietly report fewer titles than it has.

write_image called flush() and returned Ok. flush() only pushes bytes into the
page cache and promises nothing about durability, so a 6-90 GB image could be
reported complete while still unwritten — a crash or an unmounted volume then
leaves a truncated file the caller was told was finished. Now into_inner (so a
buffered-write error surfaces instead of being dropped by BufWriter::drop)
followed by sync_all.

timeline used abs() on a saturating_sub result. Every other comparison in that
module is saturating because the timestamps come off a disc and are not
trusted; abs() panics on i64::MIN, which saturating_sub can produce.
2026-08-06 07:11:52 -07:00
Matthew Jackson d1551eb588 Rename is_scrambled to has_scramble_flag_bits
The 1.6.1 DVD fix removed the USE of the loose predicate but left the
predicate sitting there with the better name. Anyone asking "is this sector
scrambled?" finds is_scrambled before is_scrambled_pack, and reintroduces the
defect that destroyed 1912 bytes of a real VIDEO_TS.IFO — the sector carrying
TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
enumerated 10, silently, at exit 0.

Byte 0x14 only means "scrambling control" inside an MPEG-2 pack. In an IFO,
UDF or ISO 9660 sector it is whatever that format stores there. The new name
says what the function actually tests and nothing more, so the honest question
has the obvious name and the dangerous one has to be asked for deliberately.

Its doc comment also claimed a caller, decrypt::decrypt_sectors, that does not
exist — so the name was an invitation and the documentation was an argument
for accepting it. It has no production callers at all; it stays public because
an integration test asserts the flag extraction directly.
2026-08-05 22:27:04 -07:00
Matthew Jackson c875df49e3 Scope the qa matrix to the platforms each crate actually supports
The first cut ran release-profile tests on all three platforms everywhere,
which invented coverage no crate had ever claimed. bdemu is Linux-only — its
ci.yml has no macOS or Windows job at all, and its tests call
ExitStatus::signal(), which does not exist on Windows, so they do not compile
there let alone run. The rest follow the policy ci.yml already set: tests
execute on Linux and macOS, Windows compiles them so the cfg(windows) halves
still get codegen before release time.

That policy is now explicit in qa.yml as a windows-build job rather than
implied by which jobs happen to exist.

Recorded, not papered over: running the suites on Windows DOES find real
defects — autorip has five Windows path-handling failures in mover.rs and
freemkv-engine has three tests hardcoding /dev/null. freemkv ships a Windows
GUI, so they matter. They are pre-existing rather than 1.6.1 regressions, and
fixing them is its own piece of work.

Also build the binary before cli-parity.sh. cli-integration.sh builds its own;
cli-parity.sh only checks for one and exits 2 with "build it first", so both
legs died before running a single case.
2026-08-05 21:22:30 -07:00
Matthew Jackson 93ad1f4854 Give Windows a free-space gate, and stop two tests timing the scheduler
Three release-profile failures on macOS and Windows, all found by the qa gate
on its first run. Release-profile tests on those platforms had never run
before it existed, so none of these were regressions — they had simply never
been visible.

available_space returned None on every non-unix target. That did not merely
skip a test, it skipped the GATE: a Windows user extracting a disc to a full
volume got a confusing failure part-way through instead of a clear refusal up
front, and Windows is where the GUI ships. GetDiskFreeSpaceExW is declared
directly against kernel32, matching how scsi::windows already reaches Win32
rather than pulling in a binding crate for one call. It asks for
FreeBytesAvailableToCaller, which accounts for per-user quotas — the same
question f_bavail answers on unix.

The other two asserted on wall-clock timing with no margin:

- sleep_until_halted_wakes_mid_sleep bounded the wait at 350 ms and measured
  377 ms on a loaded runner. That bound measures the scheduler, not the wake.
  What the test is for is distinguishing "woke because the flag flipped" from
  "woke because the 10 s timeout expired", and 2 s does that just as well.

- abandon_loses_to_a_close_already_committed released at 600 ms against two
  300 ms grace windows plus a 250 ms poll cadence, so the windows could expire
  first and the caller abandoned — a race, not a defect. The intervals are
  scaled up so jitter is small relative to them; the ordering under test is
  unchanged, only the margin.
2026-08-05 21:20:29 -07:00
Matthew Jackson d0393fb629 Stamp every qa push as a release candidate
Every push to qa now tags v<version>-rc<N>, N incrementing, before the gates
run. That answers "which build is on qa, and is it the one I tested?" without
anyone having to remember it.

The tag lands whether the run goes green or red, deliberately. A red candidate
needs a name more than a green one does: "rc3 failed release-tests on windows"
is a sentence you can act on, "qa is red" is not. Red on qa is the gate doing
its job — the branch saying this is not production worth yet.

release.yml now excludes v*-rc*. Its trigger was v*, which matches the
candidate tags, so without this every push to qa would have built and PUBLISHED
a GitHub release — including for the candidates that failed.
2026-08-05 21:03:01 -07:00
Matthew Jackson c64bc311ff Add the qa gate: dev for speed, qa for proof
dev -> qa -> main, across every public repo.

dev is where work lands and is meant to be pushed to often, so ci.yml stays
the fast answer: fmt, clippy, unit tests, leak-guard. qa is the release
candidate, and qa.yml is the claim that a commit is production worth —
release-profile tests on all three platforms, and the Linux cross-target
clippy that the local precommit gate has always run but CI never did.

Release profile matters as its own gate: overflow checks are off, debug_assert
is compiled out, and inlining changes what the optimiser can prove. A test
that only passes in debug never guarded the binary anyone ships.

qa.yml checks siblings out at qa rather than dev. This is not a monorepo, and
a qa run resolving its dependencies from unreleased dev tips would go green on
a combination that is not the one shipping — the exact mismatch the branch
model exists to prevent. ci.yml now tracks whichever branch triggered it for
the same reason, since it also fires on qa.

The consequence is that repos move to qa together, in dependency order. qa is
backfilled from main so an unchanged crate still presents working code to the
crates built on it.
2026-08-05 20:59:34 -07:00
Matthew Jackson a018e1adc4 Require a pack start code before descrambling a sector
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a
sound test once a caller has committed to a title's VOB data, where every
sector is an MPEG-2 PS pack and byte 0x14 always means what it says.
descramble_region is not such a caller: it is handed arbitrary regions of a
disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where
byte 0x14 is whatever that format happens to store there.

Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at
offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test
read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes.
That sector carries TT_SRPT, so the title table went with it — the disc
enumerated 38 titles and an image decrypted from it enumerated 10, silently,
at exit 0.

is_scrambled_pack already existed with the right predicate. Use it here. It
costs nothing: a genuinely scrambled VOB sector always carries the pack start
code, and no IFO sector does.

Verified end to end — the decrypted image's `info` output is now identical to
the source disc's, 38 titles both, differing only in the CSS: Encrypted line.

The fixtures moved with it. Four of them built a sector by setting byte 0x14
alone, which no real scrambled sector looks like; they now build packs.
2026-08-05 20:59:14 -07:00
Matthew Jackson 1d35dcf7c6 Report the resolved extent alongside each IFO read
The diagnostic hashed IFO contents but not where they came from, so a
content difference could not be told apart from a path resolving to a
different place. It now prints file_start_lba and file_extents next to the
hash.

That measurement is what inverted this investigation: both a CSS disc and
its decrypted copy resolve VIDEO_TS.IFO to the same extent, and only one
sector of the fourteen differs — by exactly a descrambled payload with the
scrambling-control bits cleared. So the decrypted copy holds the correct
bytes, and an iso:// scan of a CSS disc is parsing a still-scrambled IFO
sector, because scan_iso opens a plain FileSectorSource.
2026-08-05 20:24:20 -07:00
Matthew Jackson 77ad147563 Diagnostics that localise the decrypted-DVD title loss
Two opt-in dumps, driven by FMKV_IMAGE, that narrow where a decrypted DVD
image loses titles:

dump_title_sets_for_an_image reports what survives parse_vmg. On one disc
the CSS image yields 13 title sets and 38 titles; its decrypted copy
yields 8 and 10. Sets 8, 9, 10, 12 and 13 are dropped outright, set 11
parses but returns no titles at all, and set 7 returns 3 of 5.

dump_vts_ifo_reads_for_an_image reads every VTS IFO and hashes the
CONTENT. All thirteen are byte-identical across the two images, so
parse_vts is handed the same bytes and the same TT_SRPT info and still
fails on one of them — the divergence is in what it reads from the READER
afterwards, which is file_start_lba and the PGCIT.

An earlier version of the second dump compared only length and magic and
so wrongly reported the images as identical; it hashes the bytes now.
2026-08-05 20:14:46 -07:00
Matthew Jackson 5c64662213 Say when a title set is dropped from a DVD scan
parse_vmg skipped any title set whose parse failed, with no log and no
counter. A real disc enumerated 38 titles from one image and 10 from
another, and the 28 discarded failures were invisible — the symptom read
as a scan difference rather than as dropped reads, which is most of why
it took so long to localise.

Behaviour is unchanged: a disc may legitimately carry placeholder TT_SRPT
entries, so one failure is still not fatal. It now warns per skip with the
title set and the error, and once at the end with kept-versus-declared.

Also adds an opt-in diagnostic that reads every VTS IFO from an image, to
separate a read failure from a parse failure. It reports all 13 sets
reading identically from both a CSS image and its decrypted copy, which is
what proves the 38-to-10 loss is downstream of these reads.
2026-08-05 19:41:43 -07:00
Matthew Jackson 1f70398774 Two opt-in scan diagnostics for the DVD decrypted-image defect
dvd_placement_invariant_on_a_real_folder checks, per title set, the sum
ifo.rs relies on: file_start_lba(VTS_nn_0.IFO) + vtstt_vobs must land on
VTS_nn_1.VOB. It reports all 13 sets correct on a real DVD folder, which
is what excluded placement as the cause.

dump_titles_for_an_image prints every title a scan produces with the
numbers canonical_title_order sorts on. It is what showed the real shape:
the same disc scans to 38 titles as a CSS image and 10 once decrypted,
with identical capacity and a byte-complete image.

Both are #[ignore]d and read their target from the environment, so they
cost the gate nothing and are there for whoever picks the defect up.
2026-08-05 19:33:58 -07:00
Matthew Jackson 35c5eedc20 Round 5: fix the gates added in round 4, and two placement holes
The zero-frame check ran before the seam gate, and its error is
classified as a skippable nav stub — so a title the plan dropped
ENTIRELY was reported as an empty stub and an all-titles rip would omit
a real feature and finish the rest at exit 0. The seam case is decided
first now, with a code that is not skippable.

The demux sink read a frame's track kind out of the FILTERED slot, which
is empty for a class the export drops. On an audio:// or sub:// export
the video track was therefore called non-video and handed the permissive
crossing rule — the same defect round 4 fixed for a Dolby Vision layer,
reintroduced one file over. Video tracks are now recorded before the kind
filter, beside the primary-video reference that exists for this reason.

Its frame counter counted frames PLACED, not written, while its name and
doc claimed otherwise. Renamed and documented for what it is, including
that it cannot see a single lost track among many.

Placement: files in a subdirectory of VIDEO_TS were never given data.
They were declared at full size with no extents, so they appeared in the
tree and read as nothing. The same folder under BDMV was always placed
correctly. And the duplicate title-set guard keyed on the constraint maps,
so an IFO declaring no offsets inserted nothing and a colliding second IFO
went undetected — it keys on the groups seen now.

Display for SeamPlanDroppedMost and ShortImageRead discarded their
payloads, and four new variants were missing from the code-uniqueness
test.
2026-08-05 18:40:14 -07:00
Matthew Jackson f4b95b3dea Test the image-size cap against a real folder
The cap was added without a test. Its companion — a modest oversize is
honoured as a gap — already existed, so this pins the other side: an
offset past the ceiling is refused rather than grown into, which is what
keeps a rewritten IFO from planning a multi-terabyte image.
2026-08-05 18:30:23 -07:00
Matthew Jackson 0f61be00b7 Make the drop count gate the verdict, not just the log
Round 3 counted frames the clip marks excluded and reported them at
finish. Counting is not bounding: the only other gate was a global
zero-frame check, and its error is additionally classified as a skippable
nav stub, so a title whose marks do not line up with its PES clock could
discard almost all of itself and still exit 0 — a two-hour feature
emitting seconds, which is the defect this change set already shipped
once. Dropping more than was kept is never a real join, so it now fails.

The demux sink had no zero-output guard at all, so a fully-dropped title
finished cleanly: a directory of zero-byte track files beside a populated
chapters document. It now refuses, keyed on frames having been OFFERED —
a chapters-only export, or a track class the title does not carry,
legitimately writes none, and two existing tests correctly said so.

A title set's placement group is the parsed number, so VTS_01_0.IFO and
VTS_1_0.IFO land on one key and the second silently overwrote the first's
constraint, placing a VOB where the IFO the reader uses does not point.
Refused rather than resolved by arrival order.
2026-08-05 18:25:47 -07:00
Matthew Jackson f4fb5c65e0 Key the seam-crossing rule on reorder, not on driving epochs
A Dolby Vision enhancement layer is a second video track: it does not
drive epochs, but it does carry B-frame reorder. The crossing rule was
keyed on driving epochs, so the EL took the branch whose premise is that
the track has no reorder — and its ordinary reorder dip near the end of a
clip, which during an overlap also lands inside the next clip's range,
was read as a join. The EL was then placed on the next clip's offset, out
of step with the base-layer frame it must be co-timed with by the width
of the overlap: the same desync the per-track cursor was added to remove,
reintroduced for one track.

The property the rule actually depends on is whether a backward step can
be reorder, so it is now keyed on that. Both sinks derive it from the
track kind rather than from the epoch driver.
2026-08-05 17:52:13 -07:00
Matthew Jackson c8fafec393 Keep the seam plan to Blu-ray, and stop a missed crossing truncating a title
Two findings from the same escalation, both silent-wrong-output.

The plan was built for every multi-clip title. Only a Blu-ray PlayItem's
IN/OUT are positions in the clock the PES PTS runs on. HD-DVD fills the
same fields from the XPL's title-relative times and a DVD's come from
cell tables, so a plan built from them is an identity map with a drop
filter: it suppresses the layer-break rebase inference performs, and
drops whatever falls outside marks the PTS was never measured against. An
earlier reading of this called HD-DVD safe because its marks are
contiguous and every computed offset was zero — true, and irrelevant,
because they were zero in the wrong clock. Gated on the content format,
with a test using an HD-DVD-shaped table that the clock check alone
accepts.

The crossing test was also one-shot. A table whose clips restart their
own bases could miss it, and a missed crossing STRANDS the track: every
later frame falls outside the stranded clip's marks and is dropped for
the rest of the title. Counting drops, which is all the previous round
added, does not bound them. A table that is not one advancing clock is
now refused outright and falls back to inference, which is the documented
safe path for those titles.

Also from the same round: read_sectors added an unchecked lba + i, where
callers deliberately saturate their LBAs — a wrap folds the read back to
a low sector and hands the muxer another file's bytes. classify added 1
to two numbers parsed verbatim out of a filename. read_head used a single
read() where a short read on a network mount silently records no
placement constraint at all. And the page-cache eviction added last round
released only the read that crossed its threshold rather than everything
accumulated, so seven eighths of what was read stayed pinned.
2026-08-05 17:31:43 -07:00
Matthew Jackson 764535bb7d Make the name-cap tests exercise the planner, not the constants
The round-1 tests asserted arithmetic about MAX_CS0_NAME_BYTES and never
called plan(), so both would have passed with the guard deleted — which
is the failure mode this audit exists to catch, committed by the audit's
own fix. They now build a real folder containing a 255-byte name and
require the planner to refuse it, plus a companion proving a name at the
cap is still accepted so the guard is not merely refusing everything.

The subdirectory cap keeps its arithmetic-only test — creating 65,535
directories is not reasonable in a unit test — but now says so instead of
implying coverage it does not have.
2026-08-05 17:21:43 -07:00
Matthew Jackson 60d9cc1bac Audit round 2 fixes: an unsafe default, four omissions, and two swallowed errors
The folder encryption probe returned "not encrypted" when it had sampled
nothing at all — a title shorter than one aligned unit skipped the loop
entirely. That verdict CLEARS the structural one an AACS directory
raised, so a genuinely encrypted folder would have been ripped as clear
and written ciphertext as video at exit 0. With no evidence it now keeps
the structural verdict, and its bounds arithmetic no longer trusts
disc-derived values not to wrap.

Reading an IFO header swallowed every I/O error and returned an empty
buffer, which sent each placement offset through unwrap_or(0) and
recorded no constraint at all — a permission error on one file produced a
silently misplaced VOB. The directory walk swallowed the same class while
claiming to skip only vanished files. Both now propagate; only NotFound
is skipped.

Four things the round-1 changes left inconsistent: two new error codes had
no doc comments, were absent from the io::Error mapping, printed no path
in Display, and were missing from the test that proves codes are distinct.
The demux sink dropped frames silently while the MKV muxer reported them.
And set_clips had been inserted INTO write_frame's doc comment, leaving
write_frame undocumented and its paragraphs describing the wrong function.

uid/gid used 0 as "not specified"; UDF's sentinel is 0xFFFFFFFF, and 0 is
root.
2026-08-05 17:16:03 -07:00
Matthew Jackson cbb3517afe Bound the synthesized image, and stop it pinning the page cache
A DVD title set records where its VOBS begins as an offset inside its own
IFO, and the planner honours that offset because honouring it is what
makes a real backup readable. Nothing bounded it: a regenerated .BUP or a
hand-assembled folder naming an offset far past the content grew the
image to wherever it pointed — a u32 sector count reaches ~8.8 TB, and
writing that to an iso:// destination fills a disk with zeros before
anything notices. Capped at 128 GiB, which clears BD-100 with room.

Metadata is materialized up front and held for the life of the image, at
a 2 KiB File Entry per node, so the 100,000-entry cap alone permitted
~205 MB of it for content of no size at all — and the mux holds two
images at once while probing. The module claimed a budget of a few MiB;
that budget is now enforced rather than asserted.

Host reads had no page-cache eviction. The ISO source documents what that
costs, measured: an 85 GB read pins the whole file, starves the writer,
and collapses the mux to 2.7 MB/s against 70 MB/s isolated. A folder
source reads host files the same way, so it now uses the same eviction —
the hints move from private-to-that-module to crate-internal rather than
being reimplemented.
2026-08-05 17:10:19 -07:00
Matthew Jackson 3980aa8976 The VMG's 0xC4 is TT_SRPT, not VMGM_C_ADT
The placement code skips that field for the Video Manager, which is
right, but said it was skipping it because the field is the menu cell
address table. It is the title search pointer table, and it is an offset
inside the IFO rather than a pointer to another file — which is the
actual reason it constrains nothing. ifo.rs reads the same offset under
the correct name, so the two would have drifted.
2026-08-05 17:03:19 -07:00
Matthew Jackson ffbc1d8399 Audit round 1 fixes: sparse-track joins, silent drops, and two encoder wraps
A sparse passive track — a subtitle with no event near a clip's mark —
was held to the dense-video crossing window, so it stayed on the previous
clip's offset until its PTS passed that clip's OUT and every event in
between was mistimed by the overlap. Video keeps the tight window,
because its backward steps are also B-frame reorder; passive tracks have
no reorder, so any backward step into the next clip's range is a join.

Frames the marks exclude were dropped without a trace. Dropping is right
at a join, but this codebase has shipped complete-looking wrong output
before, so the count is kept per track and reported when the mux
finishes, alongside the pre-cluster counter that exists for the same
reason.

A File Identifier Descriptor records its name length in one byte, and the
length was narrowed with a cast: a 255-byte name — POSIX NAME_MAX,
entirely ordinary — encodes to 256 and wrote zero, which would read every
later entry in that directory from the wrong offset. A directory's link
count is 16 bits and was computed as 1 + subdirectory count, which the
global entry cap alone permits overflowing. Both are refused while
planning, where the tree can still be rejected cleanly.

The module and struct docs described inference as the whole algorithm;
they now say which path decides what.
2026-08-05 16:58:49 -07:00
Matthew Jackson 9247e7da2f changelog: dir:// is not in 1.6.1
The folder reader produces a wrong title list on a real DVD — the title
table is read at a sector offset relative to VIDEO_TS.IFO's start, and a
synthesized layout does not put that file where the offset resolves the
same way. Output is silently wrong at exit 0, so the feature is not
wired to the CLI in this release. The library module stays, unreachable,
and ships once both disc families are verified against real content.
2026-08-05 16:44:23 -07:00
Matthew Jackson 6c92370013 changelog: dir:// as a source, and the multi-clip Blu-ray timing fix 2026-08-05 15:47:47 -07:00
Matthew Jackson 3090314717 Cross a clip join per track, not on the video's frame
The first cut placed every track with the cursor the video had moved. At
an overlap join the previous clip's audio is still arriving after video
has crossed, and those tail frames sit inside both clips' mark ranges —
so they took the new clip's offset, jumped forward by the overlap, and
collided with the new clip's own audio, which the muxer's monotonic nudge
then flattened. A remux confirmed it: the timeline length was already
correct and the original symptom was still there, 169 audio packets on
the tick floor.

A track's PTS only runs forward inside a clip, so its own backward step
to the next clip's IN is its crossing. That is per track, so the cursor
is too.
2026-08-05 15:46:40 -07:00
Matthew Jackson ce246a1c87 rustfmt the seam-plan tests
Line-wrapping only, no behaviour change.
2026-08-05 15:28:40 -07:00
Matthew Jackson 617532e6ff Merge branch 'fix/mpls-seam-timeline' into dev 2026-08-05 15:24:37 -07:00
Matthew Jackson dfd2f023d0 Place Blu-ray clips by the playlist's marks, not by guessing at PTS jumps
A seamless-branching title's PlayItems do not chain contiguously: one
clip's OUT can sit after the next clip's IN, where the disc stores the
join twice, or before it, where the playlist skips material. The mux
never saw those marks — its own header said so — and inferred seams from
PTS jumps instead.

Inference cannot recover this. A forward jump is ambiguous: it means the
playlist skipped, or it means frames were lost to damaged media, and
compressing the latter would falsify timing on exactly the rips that most
need it faithful. An overlap smaller than the B-frame reorder threshold
is invisible to inference entirely, and its duplicate content then
collided in the muxer, where the monotonic nudge flattened a run of audio
onto the tick floor and put sound ahead of picture for the rest of the
film.

Measured on one 11-PlayItem title: the file declared 7893.385 s, which is
what the playlist says the title is, and carried packets to 8029.298 s.
Both numbers came from the same program on the same disc. Four skips
totalling 135.9 s became dead timeline, and a 1.79 s overlap put audio
1.8 s ahead at the half-hour mark. Five of forty-seven titles were
affected; every single-clip title was exact.

So the marks are read. Each clip contributes exactly out - in, laid end to
end, so the output runs as long as the playlist says and a join never
rewinds. Titles without usable marks — DVD, HD-DVD, file sources — keep
the inference path unchanged, and clips that already chain contiguously
produce a constant offset, which is pinned by a test.
2026-08-05 15:24:31 -07:00
Matthew Jackson bd2ba08bb7 Read a disc folder as an input: dir:// becomes a source
Users keep discs as extracted folders — a DVD VIDEO_TS or a Blu-ray BDMV,
usually a backup that is already decrypted. dir:// could only ever be a
destination, so those folders could be produced and never read back.

Everything above the sector layer wants a UdfFs over a SectorSource, and
every UdfFs read re-reads the ICB off that source at call time, so a
folder has to present itself as sectors. It does: dirimage plans a block
layout over the real files, encodes a UDF 1.02 filesystem for the
metadata, and serves data straight from disk. read_filesystem then parses
it exactly as it parses a disc, so nothing above changes — and the
iso:// arm of input() is now shared rather than duplicated, so dir://
inherits its decrypt gates, title selection and stream pruning.

The encoder is validated by more than its own reader: macOS mounts the
synthesized image and the mounted files compare byte-identical to the
originals. A round-trip through our own parser could not have shown that
— the tag CRC seeds at zero, and a wrong seed would satisfy us and no
real driver.

DVD placement is not free packing: a VTS IFO records where its title
VOBS begins relative to itself, so the VOB has to land exactly there.
Unsatisfiable marks fail loudly rather than misplace the file. 3D folders
are refused for now: the scanner detects SSIF and the planner cannot
alias its extents yet, so accepting them would produce quiet nonsense.

Left for later: metadata capture, HD-DVD, FMTS, encrypted folders.
2026-08-05 15:24:18 -07:00
Matthew Jackson dc7c3a7db5 Make iso:// a destination for any image source, not just a drive
Writing an iso:// meant a raw sector copy off a drive, so an existing
image could only ever be a source. Decrypting one you already had meant
finding the disc again.

The engine's copy is the recovery path — mapfile, multipass, damage-jump,
auto-resume — and all of it exists because optical media returns read
errors. A file does not, so a non-drive source gets write_image instead:
sectors in, bytes out, once, no recovery machinery. Keeping them apart is
not just tidiness. The mapfile identity check compares AACS unit keys and
the VID, both empty for an already-decrypted source, so identity passes
for any such source — a second run with different input to the same
output path would resume over the previous image and report success.

A short read is an error rather than a zero-fill: padding a truncated
source yields an image that looks complete and is not, which is the worst
outcome for a copy someone means to keep.
2026-08-05 14:45:01 -07:00
AnimeFNandMatthew Jackson cfce270186 Fix NTSC chapter/duration drift: dvd_time_t is a timecode, not seconds
`dvd_time_t` stores H:M:S:F as non-drop-frame timecode with a rate flag,
not elapsed wall-clock time. On NTSC the seconds field advances every 30
frames, but a frame lasts 1001/30000 s, so 30 frames occupy 1.001 s of
real time. Reading H:M:S as literal seconds under-reports real time by
exactly 0.1% — 3.6 s per hour, growing with elapsed time, which is what
made chapter marks drift ~4 s by the 67-minute mark.

Convert the whole timecode through an integer frame count and apply the
exact 1001/30000 fraction once, instead of reading H:M:S literally and
dividing only the frame remainder by a decimal 29.97. Chapter marks sum
per-cell frame counts as integers and convert once per chapter, so every
mark lands on an exact frame boundary instead of accumulating f64
rounding across a long title.

PAL is arithmetically unaffected: 25 frames = 1.000 s exactly, so the
old and new paths agree to within one ULP. Sweeping all 900k H:M:S:F
combinations under 10 hours, 12 differ, by at most 8.9e-16 s — the new
path divides one exact integer instead of adding a rounded fraction to a
large one, so where they differ it is the more accurate of the two.

Five existing NTSC fixtures asserted the old literal-seconds values and
were updated to real seconds (each is exactly 1.001x its old figure);
`bcd_secs` now documents that its argument is timecode, not real time.

Fixes #1
Reported-by: AnimeFN <admin@animefn.com>
2026-08-05 13:48:11 -07:00
Matthew Jackson 4ab8303a65 changelog: write it for users, not for the people who fixed it
Entries had grown into investigative narratives — root causes, disc
counts, PIDs, percentages, function names — which is the right level for
a post-mortem and the wrong one for a public changelog. Each entry is now
one to three sentences leading with what a user observes.

Halves the file (1538 -> 735 lines) without dropping a version. API
signature changes stay explicit, since a consumer migrates against those.
2026-08-03 08:31:04 -07:00
Matthew Jackson 80564e0470 restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-03 05:57:31 -07:00
Matthew Jackson bab566da40 v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-03 05:57:28 -07:00
Matthew Jackson 4fe05ba4c0 restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-02 22:35:49 -07:00
Matthew Jackson 7199ee497a v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-02 22:35:47 -07:00
Matthew Jackson 9ccb3c8444 Merge branch 'fix/label-pid-binding' into dev 2026-08-02 18:56:10 -07:00
Matthew Jackson 4959b48386 Bind borrowed stream labels by the stream they name, not by a slot
A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.

Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check 94377c7
added; 301 are applied. Those labels carry no editorial payload, so the
direct damage is confined to codec text — but the polluted list is also
what the anchor gate reads, and on 11 disc/stream-type pairs it is what
decides the anchor, which is how it reaches the vendor's forced and SDH
flags. 53 anchor facts are harvested off a merged slot. The clip-info
orphans had the same shape, numbered from `max + 1` of a list they share
no coordinate system with.

A label now either NAMES its stream — `StreamId { clip, pid }`, read out
of the very table `disc::bluray` builds the stream from — or it does
not, and only the ones that do not are ever reached by counting:

  * `label_at` returns vendor labels only, so the two numberings can no
    longer be confused by construction.
  * Playlist and clip-info labels bind by id. Exact, no anchor, no
    sequence, no ordinal.
  * A named stream outranks a guessed one, so an editorial flag reaches
    a stream only where the disc's own numbering puts it there.
  * The presence of an id is the provenance the anchor gate was missing.
    It reads the vendor's slots alone now, so a slot the vendor never
    named no longer breaks the sequence — under-yield is the normal
    shape of these blobs — and a title with fewer streams than the list
    has slots is no longer eligible to hold it. Ranking prefers the
    title that positively confirms most of the list.
  * An orphan is in no playlist, hence in no title, so it binds to
    nothing rather than to whatever counted its way.

The MPLS floor is one label per physical stream keyed by `(clip, PID)`,
not by `(type, language, codec, pid)`: the same PID in two clips is two
streams, and the old key collapsed them. Its `stream_number` is now the
entry's real slot in its own playlist's table.

41 of 44 images are byte-identical; all three that move lose a label
they should not have had, and no feature title changes on any image. A
dozen featurette playlists stop reporting a feature subtitle's SDH
marking on their own unrelated subtitle; eleven menu and bonus titles
stop advertising the feature's object-audio format on plain stereo; and
on a disc whose every title carries a single audio stream — too short a
table to anchor anything — a regional-variant tag asserted on all
seventeen titles is now asserted on none, in exchange for every title
stating the codec it actually carries, which none of them did.

Six tests had been asserting the invented numbering, including one
pinning the disc-global counter as a deliberate property.

Also fixes a defect in the same family that the corpus work surfaced:
`pgs_forced_probe::apply_verdicts` set `forced` but left `qualifier`,
so a demoted track shipped with a metadata sidecar calling it forced
next to a Matroska header saying it is not. Only a forced claim is
cleared; an SDH marking is not the probe's to touch.
2026-08-02 18:52:40 -07:00
Matthew Jackson 37e056f070 Say plainly that the vendor playlist format is not a specification 2026-08-02 18:27:01 -07:00
Matthew Jackson 1df36c78b2 Merge branch 'fix/paramount-forced-sub-semantics' into dev 2026-08-02 18:22:25 -07:00
Matthew Jackson 6d2ff4d1fc Read the vendor forced-subtitle field as the enumeration it is
One vendor's playlists.xml carries a per-subtitle-slot cell that looks
like a boolean, and it was parsed as one: value 1 meant forced, anything
else meant not forced. Across every image in the corpus that uses this
format the cell takes four values, and 1 is not the forced one.

Decoding three of those discs and counting every PGS display set:

  * 1 marks a FULL dialogue track that additionally contains some
    forced-narrative signs. All nine cells bearing it on one disc are
    full tracks of 949-1411 display sets; all seven on another are full
    tracks of 1602-1651. Neither disc has a small track among them.
  * 2 and 3 mark a DEDICATED forced-narrative track, in its own trailing
    stream slot, duplicating a language that already holds a full track.
    The two 2 slots measured are 15 and 10 display sets with every one
    flagged forced; the four 3 slots are 7, 14, 23 and 59 against
    1216-2655 on the tracks they duplicate.

So the old reading was wrong in both directions — it flagged full
dialogue tracks forced, which is how one language came to present as two
identical full subtitle tracks with one of them marked forced, and it
threw away the cells naming the real forced tracks.

Content could not have corrected this afterwards. Clearing a wrong
forced label needs a disc whose authoring sets forced_on_flag, and on
the measured disc carrying four genuine forced tracks not one display
set anywhere sets it — there, the vendor cell is the only evidence there
is. The classification is now explicit: only a dedicated forced slot
earns the flag, an unrecognised value never does, and the
contains-forced-signs value is dropped rather than weakened into a
forced label, since a wrong forced flag on a full dialogue track is the
user-visible defect while a missing hint costs nothing.

Four of the crate's own tests had been asserting the boolean reading;
their subject was positional alignment, so they keep it and now use a
real forced value. The other two parsers that emit a forced qualifier
from vendor metadata were audited and are structurally immune — in both,
the forced marker names a slot of its own rather than hanging off a full
track's entry, so the failure has no encoding there — and each is now
pinned by a test saying so.
2026-08-02 18:21:32 -07:00
Matthew Jackson 94377c75fd Bind vendor stream labels by PID, not by per-title ordinal
A vendor label list describes ONE playlist's stream table, but
apply_labels re-numbered it from 1 inside every title. Where sibling
playlists cover the identical feature clip and enumerate different
subtitle sets, identical ordinals resolve to different PIDs, and the
same physical stream came out flagged forced in one title and not in
the other. On the title such a disc offers as its rip target that put
`forced` on an 873 MB full-dialogue English subtitle track — the
reported "I see English and English (Forced), they are identical".

The blobs are not wrong; the binding was. The list carries no playlist
id, but it carries a language per slot, and that sequence is a
fingerprint: on the corpus exactly one title's per-type language
sequence reproduces the list position for position, and content
confirms that title's binding is the correct one. So binding is now
two-tier:

  * The title whose whole per-type language sequence sits on the list
    (>= 2 streams, longest wins) is the ANCHOR — the table the list is
    describing. Each of its slots yields a `(clip, PID) -> label` fact,
    and a PID is the same elementary stream in every playlist that
    plays that clip, so sibling playlists bind through the map. A slot
    the anchor never showed us is not bound at all.

  * Streams no anchor fact reaches still bind by the STN ordinal, but a
    label whose language contradicts the stream it would land on is
    dropped. Subtitle labels carry nothing but the qualifier, so an
    unverifiable one is all risk and no gain: off the authoritative
    path they additionally require both sides to STATE a language and
    state the same one. Unlabelled beats mislabelled — the muxer's
    demotable() guard can only clear a wrong `forced` on discs whose
    authoring uses forced_on_flag, and half the measured discs never
    set it.

Measured over 44 disc images, 9 change and every cross-title label
conflict goes away: 171 forced flags that contradicted a sibling
playlist are cleared, 61 correct ones are recovered on playlists that
had been missing them, 5 subtitle qualifiers and 1 audio purpose bound
against a contradicting language are dropped. No title gains a label it
did not have.

Known residual: on one disc the featurette playlists keep two forced
flags (down from nine) where a shifted list happens to coincide on
language. Ruling those out needs the list's provenance — which slots
are the vendor's and which were merged in by the MPLS gap-fill — and a
whole-sequence gate without it costs correct flags on discs whose
vendor slots are interleaved with gap-filled ones.
2026-08-02 17:50:20 -07:00
Matthew Jackson 0d4aab99df Stop naming specific commercial discs in the AACS and codec comments
The same scrub as the previous commit, over the files it did not reach:
the variant-MKB layout notes, the 2.1 segment index observations, the
PPS-revert regressions and the playlist-twin tiebreak.

Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16"
is the useful part, and the title it came from never was.
2026-08-02 17:10:43 -07:00
Matthew Jackson b93d10082d Merge branch 'fix/pgs-forced-probe-sampling' into dev 2026-08-02 17:07:47 -07:00
Matthew Jackson 17bb13b077 Do not call a track forced off one display set of a sample
A forced verdict is an absence claim, and a sampled run sees a fraction
of a track. Measured on real discs: tracks exist that flag about a
quarter of their display sets and leave the rest unflagged, so a sample
that catches one flagged set and nothing else promotes a full dialogue
track to forced -- which players then force on screen. A sampled run now
needs two display sets; a run that read every extent end to end has no
unread gap and may still promote off one.
2026-08-02 17:05:28 -07:00
Matthew Jackson b68765fe84 Stop naming specific commercial discs in comments and tests
Fifteen references across six files named the discs a defect was first
seen on. The parser leak found earlier was not an isolated slip — the
same habit runs through the mux comments, the changelog and the AACS
content verdict, where a title name was standing in for the shape of
the problem.

Every one is replaced with the property that actually mattered: a
multi-clip title, a UHD Dolby Vision profile 7 dual-layer stream, a
disc carrying an authored-bad TS packet. The comments are more useful
for it — the reader needs to recognise the shape on a disc they have,
not the one we happened to have.

`SEG_MainFeature` stays: the parser matches on that literal, so it is
a format token rather than a title.
2026-08-02 17:00:10 -07:00
Matthew Jackson abffa4235a Halve the seeks: eight 32 MiB windows, not sixteen 16 MiB ones
Measured on a real title: the same 256 MiB budget cut into sixteen
windows per extent took 41.8s against the head-first read's 6.6s, because
every jump collapsed the source's read batching into three-sector calls.
Expected observations depend on total bytes read, not on how finely they
are cut, and measured subtitle density (a display set every ~30-50 MB of
clip) makes a 32 MiB window about even money on its own.
2026-08-02 16:55:55 -07:00
Matthew Jackson c94e9f4fb7 End a label section where the next one's stream list begins
The pixelogic walk finds the feature playlist's section by name and ends
it at the next `SEG_`/`SF_`/`FPL_` marker. Those markers are section
NAMES, and a project's trailing sections — the per-language notice,
disclaimer and dub-credit cards — carry none. On 8 of the 11
affected-format discs in the corpus the feature playlist is the last
NAMED section in the blob, so the terminator never fires and the walk
consumes the whole tail of the file as more of the feature's stream
list.

The card names are `{lang3}_{card}`, which passes `is_stream_token`, so
each one advances an STN counter, and a card whose name collides with a
catalogued component emits a label outright. Measured on the worst disc:
95 entries past the end of a 9-audio/21-PG list, five phantom audio
labels at STN 10-14 from `*_AC` notice cards (`AC` reads as the AC-3
codec), and 94 uncatalogued-component occurrences — which also took the
parse from High to Medium confidence and fired the vocabulary-gap
warning on four components that are deliberately not catalogued. A
second disc fabricated one subtitle label from a token in a following
playlist section named `FP_SingAlong`, which `FPL_` does not match.

What every section has, named or not, is a stream list that opens with
its video slots. So a `Video Stream N` entry repeating one this section
already listed is the first entry of the NEXT section, and ends this
one. Distinct video entries are kept, since a section may legitimately
list a secondary video stream; the memo of them is bounded at the BD STN
table's ceiling so disc bytes cannot grow it.

Replaying all 11 blobs through `assign_labels` before and after: the two
discs above lose exactly their phantom labels (11→6 and 5→4), the other
nine are byte-identical.

One residue is pinned rather than papered over: a card's name precedes
its own section's video slot, so a forward-only walk can still count the
FIRST card after the last real slot. It sits at the tail of a list
nothing follows in, so it can renumber nothing — at worst it costs a
parse its High confidence.

No other parser in src/labels/ walks a flat entry sequence with a
terminator set; the rest scope each stream to a structural range or read
its number off the entry itself. paramount and criterion gain immunity
pins for the boundary property specifically: a stream list cannot run
into the next element's, and a missing element boundary shortens the
list rather than extending it.
2026-08-02 16:55:08 -07:00
Matthew Jackson 28d5897b86 Apply the shape test to a mixed track too
A track that flags some of its own display sets and not others proves the
authoring house makes the distinction, so it needs no sibling to
corroborate the flag being in use -- but it still has to look like a full
dialogue track before a forced label is cleared. A small track with a
couple of flagged signs is a forced track, and demoting it is the mistake
the shape test exists to prevent.
2026-08-02 16:44:10 -07:00
Matthew Jackson 3ed8630535 Take a lone sample window from the middle of its extent
A title cut into 50-odd clips gives each extent one window's worth of
budget. At the extent's head, every clip is sampled at the same relative
position and the first clip's window lands on the opening of the feature
-- the one stretch with no subtitles in it.
2026-08-02 16:31:39 -07:00
Matthew Jackson d0d8e2c9bf Record the label numbering and vocabulary fixes in the changelog 2026-08-02 16:28:02 -07:00
Matthew Jackson 01d4a1ba00 Catalogue the DUB forced-narrative marker and report vocabulary gaps once
Two halves of the same subtitle-labelling bug class. The numbering half
landed already: an entry the parser could not parse still occupies an STN
slot, so skipping it shifted every later label onto the wrong stream. This
is the other half — an entry the parser counts correctly but cannot
INTERPRET.

`DUB` names the forced-narrative subtitle track authored to accompany a
language's dubbed audio presentation: the signs and on-screen-text pass a
viewer still needs once the dialogue itself is dubbed. It is the same
editorial class as `*_TXT_FOR_`, spelled differently by some authoring
runs. Uncatalogued, it matched no component arm, so the token signalled
neither audio nor subtitle and the whole stream record was dropped at the
domain guard — a genuine forced track left with no forced qualifier even
after the numbering was right.

Evidence, from two independent discs in the corpus: the token appears only
inside the PG list, embedded in an otherwise contiguous run of
`{lang}[_{region}]_TXT_FOR_` siblings — one forced-narrative slot per
localized language — and takes exactly the slot where that language's
forced entry belongs. Both discs also carry that language's FULL subtitle
track as a separate, separately-labelled slot, so the DUB entry is not it.
The stream it lands on is a sparse PG track, the signature of a forced
pass rather than full dialogue. Deliberately not added to vocab::qualifier:
that maps free-form English label text, where a bare "dub" means dubbed
AUDIO. The forced-subtitle reading is specific to this token grammar.

The corpus sweep that found it also produced four components that are
deliberately NOT catalogued. They are per-language notice and disclaimer
clip names that merely collide with the `{lang3}_{component}` token shape;
each occurrence sits in a one-video-stream section next to the disclaimer
entry it names. They carry no editorial meaning, and mapping them would
attach a qualifier to a stream on the strength of a filename.

Which is also why an unmapped component now reports once per parse rather
than once per occurrence. It was a debug line nobody reads, and that is how
this gap survived to a user complaint; but a per-occurrence warn would bury
the signal under dozens of routine collisions on an ordinary disc. One
bounded, deduplicated line names the distinct components and says plainly
that any forced/SDH/commentary meaning they carry went unapplied. The
backing set is capped and truncates by chars, not bytes — the components
come from untrusted disc bytes, and a byte-offset slice can split a
multi-byte sequence and panic.

Two existing tests encoded the wrong behaviour and are corrected: the STN
numbering test expected the forced run to skip the DUB slot, and the
unclassifiable-slot test used DUB as its example of a token with no
meaning. The latter now uses a genuinely uncatalogued component.
2026-08-02 16:25:04 -07:00
Matthew Jackson 5c8b4dc7c5 Note the bounded display-count over-count on a stalled read retry 2026-08-02 16:18:19 -07:00
Matthew Jackson 841aa1a1c6 Drain a sampled run's tail and merge repeated extent memos
A window's last PES stays open in the demuxer until the next PUSI, which
under sampling is in a different window or nowhere — so the last display
set of every window was discarded, worst where the sample is thinnest.
Flush the demuxer at the end of each window.

A playlist that lists the same clip twice now merges the second read into
that extent's memo (facts OR'd, display count MAXed, coverage the larger
of the two) instead of overwriting it.
2026-08-02 16:16:20 -07:00
Matthew Jackson 14049bb477 Document the forced-probe redesign in the changelog 2026-08-02 16:09:27 -07:00
Matthew Jackson 8ffce6b621 Sample PGS across the title instead of reading its head
The content-based forced-subtitle probe spent its whole 256 MiB budget
on the first sectors of a title. A feature's subtitles begin minutes in,
so the probe read the opening logos, hit the budget, observed no display
set at all and contributed nothing to any verdict — the vendor label was
always the only input.

The forced predicate is asymmetric: one non-forced display set disproves
forced permanently, while proving forced needs the whole track, and
genuine forced tracks are tiny where full tracks are huge. So the same
budget is now SPREAD over each extent in ~16 MiB windows placed on the
AACS unit grid, sized in proportion to the extent, ending at the extent's
end. Cost is unchanged; placement is not.

Also:

  * Per-track early exit. A track that is disproven (and whose label
    needs no correcting) stops asking for budget; an extent that owes
    evidence only for such tracks is skipped outright, and evidence
    already in the cache is never demuxed a second time.

  * Content may now DEMOTE a wrong vendor forced flag, in the probe and
    in the muxer, behind one shared guard: absence of forced_on_flag
    only means something if some other track demonstrably uses it, and
    the track must have the shape of a full dialogue track rather than
    of a forced-narrative one. On a disc where no track sets the flag,
    nothing is demotable.

  * A sampled or budget-cut extent's evidence is memoised with the
    COVERAGE behind it. It used to be filed under the extent's full key
    and replayed to playlists that would have read far more of the clip,
    turning a prefix into an absence claim about the whole extent.
2026-08-02 16:07:02 -07:00
Matthew Jackson 9fda690d00 Number vendor label streams by STN slot, not by parsed entry
Sweep of every parser in src/labels/ for the numbering bug fixed in
pixelogic: a blob lists one entry per STN slot, but the parser advances
its per-kind counter only for entries it can use, so every entry it
skips shifts all later labels onto the wrong stream.

Three parsers were affected; the rest key each label off a number the
blob states outright and are immune.

  * paramount — `aud` / `sub` are the STN-ordered stream lists, and
    `forced_sub` / `*_com1_idx` index those same cells. A cell with an
    empty language was skipped without consuming its slot, so every
    label behind it bound one stream early while the vendor's own
    positional indices still pointed at the raw cell. `stream_number`
    is now the cell's 1-based position. The forced flag is the payload
    here, so the shift lands `forced` on a full-dialogue track.

  * mpls_universal — its counters must agree with the stream list
    `disc::bluray` builds from the same STN entries, since that list is
    what `apply_labels` counts against. They disagreed twice: a
    `coding_type == 0` padding entry was counted here and dropped
    there, and a PG coding_type in an audio STN slot (a layout
    `mpls::parse_stream_entry` has a dedicated arm for) was counted as
    audio here and built as a subtitle there. Both rules now live in
    one `label_type_for`.

  * deluxe — a binding construction whose Language `getstatic` did not
    resolve was skipped outright. It is still an STN slot; it just has
    nothing to label. It now advances the counter, with the list it
    belongs to taken from its CodingType argument or, failing that,
    from what its binding type's resolved siblings showed.

Two paramount tests asserted the renumbering as if it were the spec
(`empty_middle_slot_does_not_inflate_stream_number`,
`audio_stream_numbering_skips_empty_slots`) and are rewritten. Immunity
pins added for ctrm, dbp and criterion so the property cannot rot.

Also scrubs two commercial disc titles from the menu-graphic filename
examples in png_filenames and vocab.
2026-08-02 15:51:09 -07:00
Matthew Jackson f53abfe0d4 Stop naming a specific commercial disc in the label parser
Two comments identified the disc the STN-numbering bug was found on by its
vendor project name. The reproduction does not need it: what matters is the
SHAPE of the token list — placeholder slots, region-only tokens, an
uncatalogued component — not which release happened to exhibit it. Both now
describe the shape.

The remaining `SEG_MainFeature` references stay. That is a vendor section name
the parser matches on at pixelogic.rs:88, not a disc identifier — it is the
format's vocabulary, like `FPL_` or the `eng_MLP_` stream tokens beside it, and
removing it would break the parser.

Also drops the last prohibited citation from the changelog: an mp4:// bullet
said "no ffmpeg". The website changelog page is REGENERATED from this file at
release time, so a scrub of the site alone would have been reverted by the next
release.
2026-08-02 15:30:56 -07:00
Matthew Jackson 80be34bff5 Number pixelogic subtitle labels by STN slot, not by parsed token
A pixelogic feature section lists one entry per STN slot. Only the
entries that parse were advancing the counters, so every surviving
label was renumbered 1..N and applied to the wrong stream.

Three kinds of entry were being skipped:

  * `PG Stream N` placeholders — the subtitle twin of `Audio Stream N`,
    which was already counted. The old comment claimed the corpus showed
    subtitle tokens align without counting them; on a disc that has both
    placeholders and later editorial tokens they do not.
  * region-only tokens (`fra_CF_`, `spa_LS_`) — REGIONS sets `variant`
    but neither `is_audio` nor `is_subtitle`, so the token is dropped.
  * tokens whose distinguishing component is uncatalogued (`jpn_DUB_`).

On UHD_Crime101_WW_150728 the PG list is 18 slots: five placeholders,
four region-only tokens, and `jpn_DUB_`, with the seven `*_TXT_FOR_`
forced-narrative tokens at STN 11-16 and 18. Counting only the ten
that parse put `eng_SDH_` on STN 1 and the seven forced markers on STN
2-8 — the disc's FULL subtitle tracks. `labels::apply_labels` turns a
Forced qualifier into `SubtitleStream::forced`, which the muxer writes
as Matroska `FlagForced`, so the output offered an "English (forced)"
track carrying 1731 display sets of complete English dialogue while the
real 19-set forced track went unflagged.

Unclassifiable entries carry no stream type, so they advance the list
currently being enumerated; sections run video -> audio -> PG and
`domain` follows the last entry whose type was known.
2026-08-02 15:27:28 -07:00
Matthew Jackson 4447bd60ce Record the 1.6.0 fixes in the changelog
The fvi provenance fix, the key-service outage codes, the settings BOM
data-loss fix and the per-track-kind picker rows all landed on dev without a
changelog line. A release that ships undocumented fixes is one nobody can
audit later.
2026-08-02 15:08:27 -07:00
Matthew Jackson c054893540 Merge branch 'fix/key-service-outage-not-missing-key' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson 6758370f8d Merge branch 'fix/fvi-source-provenance' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson b2a274782a Report a key source that could not answer as its own failure, not as "no key"
The online key service returned HTTP 502 for about seven hours. Every rip in
that window ended with

    key: online > no entry > NO KEY
    Error: E7022 No key source has a decryption key for this disc (id: 422EB...)

which reads as "this disc is not in the key database". Operators went hunting
for a VUK that was never missing; the correct action was to wait.

E7022 is a claim about the WORLD: every source answered, and none holds a key
for this disc. A source that could not be reached made no such claim -- nothing
at all was learned. `resolve_and_apply_traced` collapsed the two anyway, with
its own comment naming the incident and pointing at the fix.

Three codes for the three different operator actions, each a variant with a
number and no English (the library ships none):

    E7028 KeyServiceUnavailable   unreachable / DNS / timeout / 5xx -> wait
    E7029 KeyServiceUnauthorized  401 or 403                        -> fix token
    E7030 KeyServiceRateLimited   429                               -> back off

None carries a payload: the key-service URL and its resolved address are
operator-confidential and must not ride out in a Display an operator pastes
into a bug report.

`resolve_and_apply_traced` now keeps `Err` and `Ok(empty)` apart. A source that
answered and holds nothing still records `KeyNode::NoEntry` -- that is the one
true "I looked, it is not there". A source that FAILED records an empty path,
and its reason is stamped onto `Disc::aacs_error`, the channel
`ensure_decryptable_keys` already reads for the E7017-vs-E7022 split. The gate
gained three arms alongside `AacsVidUnavailable` and raises the source's own
code.

`KeyOutcome` deliberately gains no variant. It is matched exhaustively by every
front-end's trace renderer (freemkv's `pipe::render_resolution_trace`,
autorip's `keysource::render_resolution_trace`), and this fix must not become a
breaking change across four repos to say something the error code already says
precisely. Dropping the false `NoEntry` node is enough for the trace line.

The three codes join `is_disc_level_no_key`: a service that is down, refusing
the token or throttling is down for every title, so a rip loop must stop rather
than issue N doomed requests -- and on 429, dig the hole deeper.

`FetchOutcome::errored`, documented as unreachable-in-production, now fires for
real: the negative-result cache stops memoising a transient outage.

Red before green: `key_source_failure_is_not_reported_as_a_missing_disc_key`
drives KeySource -> resolve -> aacs_error -> ensure_decryptable twice over the
same disc and asserts the verdicts differ. With the old conflation restored it
fails on the trace node, and with that assertion removed it fails
`left: 7022, right: 7028` at the gate.
2026-08-02 11:53:09 -07:00
Matthew Jackson cf7ee69fd5 Record the source, not the destination, in the FVI header
The `fvi://` arm of `output()` passed the destination `.fvi` path as
`FviSink::create`'s `source_path`, so every index named itself as its
own source. `SourceInfo::default()` supplied the rest, making
`source.medium` always "file" and `source.title` always 0 — three
header members wrong, where FVI_FORMAT.md §6.2 defines `source` as
describing the input.

Beyond the wrong data, it made the output unreproducible: two machines
indexing identical bytes emitted different files purely from where
they wrote them, and a local filesystem path leaked into a shareable
file.

`output()` cannot see the source, so thread the provenance down from
the driver, which can: `mux_stream` derives a `SourceInfo` per
`MuxInput` arm and passes it through `drive_mux` to `output()`. Per the
one-method-per-action rule this is a signature change, not an
`output_with_source()` variant; the parameter is `Option<&SourceInfo>`
so a caller with no provenance declares none rather than back-filling
the destination. `SourceInfo`/`Medium` become public API.

What each arm can honestly reach:

- Session: everything — device path, the caller's title index, the
  title's playlist, the scanned volume id.
- Url: the source URL, its scheme's medium, `title_index`, and the
  playlist off the opened stream's scanned title.
- Iso: the image path and playlist. The title index is not in
  `MuxInput::Iso` (it carries a scanned `DiscTitle`, which has no
  index), so it stays 0.
- Live: medium and playlist. The reader is an opaque
  `Box<dyn SectorSource>` with no path, and again no title index.

Unreachable members are left empty rather than guessed — the sink
already omits the empty ones.
2026-08-02 11:19:54 -07:00
Matthew Jackson e008e71a17 Add the missing direct coverage for parse_stss
Every other sample-table parser (stco, stsc, stts, ctts) had a "count lie"
test proving the declared count is bounded by what the box actually holds,
and stco/stsc had their own arithmetic pinned. parse_stss had neither - no
test in this file ever called it with real entries, only with a too-short
buffer. Added the same two: two distinct entries read from their own
offsets (catching the o = 8 + i*4 arithmetic and the per-entry bounds
check), and a declared count of 3 backed by only 2 real entries (catching
the same "trust the box, not the count" contract the other four parsers
already had).
2026-08-01 16:37:10 -07:00
Matthew Jackson c59e1e3342 Pin the sample-table parsers' short-buffer safety and byte arithmetic
parse_stco, parse_stsc, parse_stts, parse_ctts and parse_stss all open with
the same "if b.len() < 8" guard before reading count = be32(b, 4), but no
fixture anywhere in this file ever called any of them with a buffer shorter
than 8 bytes - every test builds a complete box. A <-to-== mutant of that
guard only rejects a buffer of EXACTLY 8 bytes and lets everything shorter
fall through to an out-of-bounds be32 read, and nothing was exercising that
fall-through to notice. One test now drives all five through every length
from 0 to 8.

Also: co64's 8-byte offsets were never actually built by any co64 fixture
in this file (only the 32-bit stco path was exercised), so its manual
byte-by-byte u64 assembly was unconstrained the same way parse_elst's was.
And sample_offsets's sidx only advanced correctly by coincidence in every
existing fixture, because none of them placed three or more samples in a
single chunk back to back - the only shape where reusing the wrong sample's
size becomes observable.

Documented the <=8 boundary as equivalent across all five parsers (and the
matching start < end in sample_offsets): the per-entry guard immediately
below always breaks on the first entry at that exact boundary, so both
branches converge on the same empty result. Confirmed by re-running each
mutation against the full suite.
2026-08-01 16:33:17 -07:00
Matthew Jackson 39d9714ae7 Pin parse_esds_asc's two independent boundary checks
asc_len == 0 and end > b.len() reject for different reasons - a useless
zero-length ASC, and a truncated one - and nothing distinguished the || from
an && that would only reject when both are true simultaneously, or the >
from a < that would reject the common case of an esds with bytes after the
ASC (more child boxes, padding) instead of only a genuine truncation.

Documented the | in read_descriptor_len's accumulator as the same
shift-then-mask equivalent already recorded in audio.rs's BitReader::read:
the shift always vacates exactly the bits the mask fills, so | and ^ can't
disagree.
2026-08-01 16:23:10 -07:00
Matthew Jackson dd940583c7 Pin mdhd_language's length boundary and three unasserted stsd codec paths
mdhd_language's guard was written as "b.len() < off + 2" (reject too short)
but nothing distinguished that from "reject anything not exactly off + 2" -
a buffer one byte longer than the minimum has to keep working, and a buffer
missing the field entirely has to return None rather than read past the end.

parse_stsd's mp4a path had no test at all: the AAC codec_private extraction
depends on both codec == Aac and body.len() >= 28 being true together, and
with no mp4a fixture anywhere in this file neither half of that condition,
nor the boundary itself, was constrained. Also added the same header-length
boundary check parse_elst already had (< 8 vs <= 8 - proved equivalent this
time, since the very next guard on the empty slice catches the <= 8 case
too), and one test walking every recognised audio fourcc (ac-3/ec-3/mp4a/the
four dtsX variants) so a deleted match arm for any of them fails loudly
instead of silently dropping that track.
2026-08-01 16:18:20 -07:00
Matthew Jackson 4b7e4ddbb3 Pin find_boxes_capped's cap boundary and its size-field byte offsets
Nothing asserted the scan actually STOPS at cap rather than one match past
it, or that the declared box size is decoded from its own four bytes rather
than an adjacent one - every existing fixture used sizes small enough that
all but the last size byte are zero, so an index slip reading the wrong byte
would read the same zero and go unnoticed.
2026-08-01 16:08:11 -07:00
Matthew Jackson 5222458411 Pin Stream::read's MAX_ALLOC_BYTES boundary and that write always rejects
Stream::read has its own s.size > MAX_ALLOC_BYTES cap, a separate call site
from read_moov's over the same policy - checked they agree (both reject
strictly greater than the cap, exact cap allowed) and they do, so this is not
one of tonight's one-policy-two-copies bugs. But the boundary itself and the
one-byte-over case were unasserted, and Mp4Reader::write returning an error
(mp4:// is read-only) had no test either.

Building Mp4Reader directly in the test (its fields are private but visible
within this module) over the existing FakeBigReader avoids a real 256 MiB
backing file for the boundary case.
2026-08-01 16:04:03 -07:00
Matthew Jackson dd5118ee9c Pin per-track handler routing, PID arithmetic and the shared sample budget
Nothing asserted that a hdlr other than vide/soun gets dropped rather than
folded into the audio branch, that the per-track PID formulas
(0x1011/0x1100 + track_idx) use the right operator and the right operand,
that a sample-less track still advances track_idx for the next one, or that
the cross-track sample_budget is actually decremented (as opposed to grown or
divided) by each track's real count. All four were reachable with a single
track_idx == 0, which made every existing fixture blind to +/-/* confusion on
these sites - track_idx never moved past 0 in any of them.

Also let audio_trak_missing omit stsz, needed to build a sample-less track for
the track_idx test.
2026-08-01 15:59:53 -07:00
Matthew Jackson a8db2435cd Pin elst byte-offset decoding and a zero-timescale boundary in elst_offset_ticks
parse_elst's existing tests used segment_duration/media_time values that were
almost all zero or 0xFF bytes, so an index slip in the version-1 byte
extraction (reading a neighbouring byte, or one outside the entry entirely)
could return the same value by coincidence and the test wouldn't notice. Added
a fixture with every byte distinct and nonzero so any wrong offset is caught.

elst_offset_ticks's `empty_movie_ticks > 0` guard on the Some(mts) arm looked
like a pure optimisation, but shifting its boundary lets an
empty_movie_ticks == 0 call fall into the division instead of skipping it -
and a zero movie timescale (unreachable through from_reader, which filters it,
but not through this function's own contract) makes that division panic.
Pinned the boundary directly so the function stays safe on its own terms.

Documented nine further mutants as equivalent rather than chasing them:
media_edits/odd_rate and the None-arm's empty_movie_ticks check only gate a
tracing::warn!, never the returned offset, and parse_elst's b.len() < 8 vs
<= 8 boundary computes the same empty Vec either way once available = 0 is
worked through. Confirmed by re-running each mutation against the full test
suite.
2026-08-01 15:54:24 -07:00
Matthew Jackson ff18d4c3c8 Close the MEDIUM mutation gaps across transport, labels and codecs
The remaining triage items after tonight's HIGH fixes: 1,290 lines, almost
all tests. Covers disc/mod.rs's DVD scan path (with real minimal VMG/VTS IFO
fixtures rather than mocks), drive/mod.rs, labels/class_reader.rs and
labels/mod.rs — the two biggest untriaged survivor clusters in the crate —
plus hevc.rs and ps.rs.

One production change, and it is an extraction rather than a behaviour
change: MacScsiTransport::open mapped the shim's negative failure sentinels
to typed errors inline, where nothing could reach it without a real IOKit
FFI call. It is now map_shim_open_error, so the mapping can be pinned. It
matters because collapsing -5 into the DeviceNotFound catch-all turns
"another process holds the drive" into "no such drive", and an operator
chasing the wrong problem is worse than a blunt error.

Gate green on the pinned toolchain including the secrets scanner.
2026-08-01 15:00:01 -07:00
Matthew Jackson f8ed0b99f4 Pin read_moov's box-size boundaries and MAX_ALLOC_BYTES exactly
read_moov's forward-progress guard (box_size < header_len, OR'd with the
EOF check) and the MAX_ALLOC_BYTES cap were only exercised on inputs well
away from their boundaries, so a mutation testing pass found the exact
edges unasserted: a size-8 (header-only) moov, a box that overruns the
file by exactly the amount the OR/AND distinction can see, and a payload
of precisely MAX_ALLOC_BYTES. Added a shared FakeBigReader (lifted out of
an existing test's local struct so a new test can reuse it) to exercise
the MAX_ALLOC_BYTES boundary without a multi-hundred-MiB backing file.
2026-08-01 14:39:39 -07:00
Matthew Jackson 8189da1b0c Document why audio.rs's bit-packing | mutants are equivalent
Every mutation-testing survivor in this file is a | with ^ flip inside a
bitstream packer: BitReader::read's accumulate step, the push closures in
dac3_box/dec3_box/ddts_box, and the multi-field extractions in parse_eac3
and parse_dts. All nine are the same shape: shift an accumulator left by
exactly the width of the next field, then OR it in, so the two operands
never share a set bit and | and ^ agree on every input. Confirmed by
running cargo-mutants against just these nine mutations after the existing
test suite (which already exercises each function's field values) - all
nine still survive, as expected for a genuinely equivalent mutant. Recorded
the reasoning once at BitReader::read so nobody spends time chasing it
site by site.
2026-08-01 14:24:47 -07:00
Matthew Jackson 698ba36ae4 Document the pack_language and detect_rate equivalent mutants
Three pack_language mutants and two detect_rate boundary mutants survive
mutation testing with no test able to close them, and it's not for lack
of trying: they're equivalent by construction. Recording the proofs next
to the code so nobody re-chases them:

- (b[0] - 0x60) as u16, shifted << 10 then truncated to u16, is congruent
  mod 65536 to (b[0] + 0x60) as u16 shifted the same way, because
  0x60 * 2 * 1024 is an exact multiple of 65536. The same swap on the
  second letter (shifted only << 5) is NOT equivalent, which is why only
  the first letter's mutant survives.
- The two | with ^ mutations that OR the three packed fields together
  are equivalent because the fields (a lowercase letter minus 0x60, so
  1..=26) always fit in 5 bits and never share a set bit once shifted
  into their 0/5/10 positions.
- detect_rate's tolerance and tie-break comparisons only diverge from
  their <= mutants on an exact 0.5 fps distance or an exact tie, and a
  brute-force search over every achievable integer-nanosecond median
  found no case that lands on either boundary bit-exactly.
2026-08-01 14:20:13 -07:00
Matthew Jackson 1eb8bdc9c7 Merge branch 'mux-mp4' into dev 2026-08-01 14:06:24 -07:00
Matthew Jackson 90a7fe2ff1 Assert the MP4 timing arithmetic, and name the faststart slack rule
MP4 track timing was numerically unasserted. Every operator in the
PTS-to-ticks, duration, tkhd_dur and ctts chain could be flipped and the
whole suite stayed green, because no test decoded an output file and checked
a concrete number — the existing tests assert box presence and gross
container shape only. That is the crate's worst failure mode: a title muxes
"successfully" with silently wrong A/V sync or total duration, and nothing
above can tell.

The new tests build tracks with known PTS deltas and compare the emitted
stts, ctts and tkhd.duration against computed values.

Also lifted the faststart slack rule out of the match guard into
faststart_fits(). A leftover hole of 1-7 bytes cannot be expressed as any
ISO-BMFF box, since a box header is 8 bytes, so finish() must fall back to
moov-at-end rather than write a free box that lies about its own size. The
condition now has a name and a test instead of being an unexplained
`g == 0 || g >= 8` inside a pattern guard.
2026-08-01 14:06:16 -07:00
Matthew Jackson 4e70d9a5c5 Assert the undersized-buffer guard on the chunked read path
Drive::read and read_fua split any request larger than the transport's
transfer limit into chunks and slice the caller's buffer by count * 2048.
The up-front length check is the only thing between an undersized buffer and
a "range end index out of range" panic out of a public API, and the comment
above it records that this was once a live panic.

Every existing read test stays on the single-chunk path, where an undersized
buffer is already tolerated and returns Err(DiscRead), so the guard itself
had no coverage at all and a mutation run flipped its arithmetic freely.

The test drives a mock transport with a small transfer limit and asserts the
two paths agree: an undersized buffer is an error either way, and behaviour
on a caller mistake does not depend on the drive's transfer limit. Confirmed
by hand that both the reported * -> + mutation and a < -> > flip now fail.
2026-08-01 13:56:58 -07:00
Matthew Jackson 1b95d346bb Cite the H.264 spec directly, not a reference implementation
The escape-stripper comments named a third-party decoder as the authority
for the cumulative-zero rule. This repo is public and does not cite other
implementations; the rule is specified in ITU-T H.264 §7.3.1, which is the
citation that belongs here anyway.

No behaviour change — comments only. The scan-secrets gate caught it.
2026-08-01 13:49:47 -07:00
Matthew Jackson 48663c6a2f Merge branch 'mux-codec' into dev 2026-08-01 13:47:15 -07:00
Matthew Jackson a05f1d4498 Merge branch 'mux-mkv' into dev 2026-08-01 13:45:04 -07:00
Matthew Jackson 3ecb2510e8 Assert what the MKV mux and demux actually produce
A mutation run over mux/mkv.rs and mux/mkvstream.rs left 148 survivors.
Reading them turned up no wrong code, but a lot of code whose output
nothing ever looked at. Most of that is on the read side: parse_track
had a dedicated arm for Language, TrackName, FlagForced, Video and
Channels and not one of them was checked, so a re-mux could have lost
the audio language, the subtitle forced flag, every track label, the
resolution and the channel layout with the suite still green. Ten of
the eleven CodecID comparisons were unasserted too — only HEVC was
pinned — so any of them could have been mis-wired and the stream would
have gone to the wrong parser. The round-trip test now writes a real
three-track title through the muxer and reads it back through the
reader, and a separate test walks every registered CodecID.

The BPS statistics tag was the worst of the write side. Its test
asserted `file_bytes.contains("800")`, which a wrong bitrate passes
trivially — 80000 contains "800". Both the tag and the back-patched
Segment duration are now decoded and compared to a computed number, on
a title that declares no duration so the whole max_block_ticks →
seconds → bits chain is exercised. Cue points get the same treatment:
their CueTrack and CueClusterPosition were never read back, on either
the keyframe path or the i16-forced-split path, which are two hand-
written copies of the same three fields.

The rest closes arithmetic that only a bad disc reaches: a zero frame
rate or zero display-aspect denominator (both divisions), a
TimestampScale that does not fit an i64, a cluster timestamp of exactly
i64::MAX, a TrackNumber of 65537 that truncates onto the valid track 1,
and the shortest legal Block at both VINT widths. Two tests separate a
clean end of stream from a device failure: swallowing the second one
truncates the output at a bad sector and reports the rip complete.

Also pinned: only the first video and first audio track may be default
(the de-duplication lives in MkvStream::create and had no test at all),
the activation trigger is the first VIDEO track rather than track 0,
the measured field order reaches the file rather than just the helper
that computes it, and a Blu-ray 3D base/dependent pair builds the merge
instead of shipping two unrelated H.264 tracks.

96 of the 148 mutants verified killed by hand. Of the remainder, most
are equivalent — disjoint-bit `|` that `^` cannot change, delete-arm
mutants whose fallback is the same constant, guards on tracing calls —
and the write_frame branch at 1452 is unreachable: a cluster is always
open by the time it can be entered.
2026-08-01 13:44:08 -07:00
Matthew Jackson 048f125879 Kill codec mutation survivors and unify H.264's duplicated escape stripper
The mux/codec parsers (startcode, h264, hevc, dts) had 300 surviving
mutants between them, and it turned out to be for the reason you'd
fear: the exp-Golomb readers and the AU-boundary bitstream scanners had
essentially no direct unit coverage, only indirect exercise through
full-frame parse() calls that never touched the actual edge cases.

Direct fixes to test gaps:

- The shared BitReader's read_ue truncation guard (`leading_zeros >
  31`) and skip_start_code's 4-byte-vs-3-byte boundary check had no
  test at their exact boundary. Added tests that hit the boundary
  precisely; a `>=`/`==`/`<=` typo either rejects a legal 31-leading-
  zero code or reads one byte past the buffer.
- H.264's private SpsReader duplicates the same read_bits/read_ue
  shapes with no tests of its own at all (only reached through
  multi-field SPS parsing, several fields deep). Added direct tests.
- HEVC's per-AU trailing-zero strip after the last NAL (no start code
  following) walks `end` down to trim padding; a wrong-direction typo
  there walks off the end of the buffer instead of terminating -
  exactly the "loop must make positive progress on malformed input"
  class. Added a test with a zero-padded trailing NAL.
- HEVC's SEI match guards (`sei_mastering.is_none()` /
  `sei_content_light.is_none()`) implement "first HDR10 value in the
  title wins" - untested, and a naive test using both-messages-per-AU
  can't even exercise the guards because the whole-scan early return
  above them already handles that case. Split into single-message-
  per-AU tests that actually reach the arms.
- parse_mastering_display/parse_content_light_level's length guards
  were `< N` with no boundary test; one-byte-short input now confirmed
  to return None instead of indexing out of bounds.
- DTS's drain_front collapses duplicate offset-0 PTS markers after
  rebasing; untested, and the visible effect (front_pts()) can't tell
  a working collapse from a broken one since it already returns the
  right marker either way - the actual defect is unbounded growth of
  pts_marks over a long recording, so the new test asserts the bound
  directly across repeated drains.
- DTS's dts_core_samples/dts_core_sample_rate header-length guard and
  next_core_boundary's syncword-length guard got exact-boundary tests
  the same way; also caught a nblks `<<`/`>>` direction bug candidate
  in the mutant (confirmed the real code is correct, just untested).

Real bug found and fixed, not just a test gap:

H.264's parse_sps_high_profile_ext re-implemented emulation-prevention
byte stripping inline (a window scan: match `00 00 03` at position i,
advance 3, else advance 1) instead of calling the existing
unescape_ebsp_prefix used by slice-header parsing. On a run of 3+ real
zero bytes ahead of an 0x03 - non-conformant, but this is disc bytes,
not a spec-clean encoder - the two disagreed: unescape_ebsp_prefix's
cumulative zero counter (matching the H.264 reference decode process
and libavcodec's RBSP extractor) treats it as an escape and drops the
0x03; the window scan treats it as real payload and keeps it,
corrupting the SPS bits read after it. Extracted the shared rule into
`unescape_ebsp` (parameterized on output length so both the 16-byte
slice-header prefix and the unbounded SPS case can share it) and
pointed both call sites at the one implementation. Added a regression
test pinning the shared function's behaviour on the input that used to
separate them.

All new tests hand-verified against the actual mutation (operator
flipped or guard replaced by hand, confirmed red, then restored) per
the mutation-testing brief, not just written and trusted.
2026-08-01 13:35:43 -07:00
Matthew Jackson 65dbcb1ca6 Close mutation-testing gaps in the TS/PS mux (ts.rs, ps.rs, tsmux.rs)
A 12,330-mutant run left 159 survivors across these three files, all from
missing assertions rather than wrong code — every gap here is a test, no
production logic changed.

Two shapes accounted for most of them:

- Buffer-cap constants (MAX_PES_BUFFER_TOTAL, MAX_PS_BUFFER,
  MAX_BD_PES_PAYLOAD, PES_BUFFER_INIT_CAP) were only ever read by tests
  through their own symbol, so a mutated `*`/`-` in the constant's
  definition changes what the symbol itself evaluates to and every
  self-referential assertion still passes. Pinned each against a literal
  computed independently in the test.

- Several `>`/`==` boundary checks on framing lengths (MPEG-2 pack header,
  system header, BD-TS adaptation field) were only ever exercised with
  slack in the buffer, never at the exact byte the check exists for.
  Added exact-fit cases for the pack header, system header, and
  psi_payload_base's AF-consumes-everything boundary.

Real, higher-value gaps closed along the way:

- ts.rs's per-PID discontinuity_flag and the NULL-TS concealment marker
  both require adaptation_field_length > 0 before trusting the AF flags
  byte; neither branch had a test proving af_len == 0 (no flags byte at
  all, ordinary payload underneath) is left alone.
- header_remaining (PES header spillover across TS packets) only had
  single-continuation-packet coverage, which can't distinguish `-=` from
  `+=`/`*=` because the corrupted value never gets read again. Added a
  case spanning two continuations.
- ps.rs's parse_stream_id_extension (used for HD-DVD 0xFD routing) walks
  nine optional PES-header/extension fields with a `pos +=` each; only
  the PTS/DTS pair had ever been exercised. One test now arms every
  field and checks the walk lands on the right byte.
- find_ps_boundary's `sc + 3 >= len` guard had no test at sc == 0 with a
  bare 3-byte start code, the case a `+` -> `-` mutation turns into a
  debug-mode subtract-overflow panic on ordinary tail-of-buffer input.
- tsmux.rs: an oversized video access unit must go out as a single
  unbounded-length PES; the `is_video || small-enough` guard that
  enforces this had no test with a video frame actually over the
  bounded-PES threshold, so a `||` -> `&&` mutant survived (it would
  silently split a keyframe across several look-alike-independent PES
  units). Also pinned the PES-length and PTS big-endian encodes at
  values above 255 / with bit 29+ set, where a `>>`/`<<` swap first
  becomes observable.

Every test above was verified by hand: applied the exact mutation,
confirmed the test fails (or the specific panic fires), then reverted.

Left unclosed, all confirmed equivalent by hand-tracing rather than
just left alone:
- Every `<<8 | byte` PID/length bit-combine (ts.rs pid/PAT/PMT parsing,
  ps.rs dvd_audio_pid/hddvd_extended_pid/parse_pts): the two halves
  never share a bit, so `|` and `^` produce identical output for every
  input - no test can tell them apart.
- ts.rs's `af_len > 183` check in process_packet: fully subsumed by the
  `payload_start >= TS_PACKET_BYTES` check three lines later for every
  af_len that could trip it.
- ts.rs's out-of-range `pid_index` sentinel (-1 vs 1): unreachable, since
  a TS PID is masked to 13 bits (max 8191) and the table is always sized
  to at least 8192.
- A cluster of "push an empty slice on an exact boundary" mutants in
  tsmux.rs's write_pes_chain (offset < hdr_len, af_bytes stuffing
  guards): the guarded write becomes a length-0 write_all, a no-op
  either way.

Not reached this pass, for lack of a clean seam within the time
available - ps.rs's extract_packets bounded-PES-length exact-fit
checks (lines 278/282/303, the `sc+6>len` / `sc+6+pes_len>len` /
force-flush cap arithmetic). The first two need a scenario where
"proceed vs. wait one more byte" is observable in the packet list, and
the third only shows up at a start-code offset (sc) that survives to
the moment the cap check runs - in this code path sc is always 0 once
an unbounded PES buffer starts accumulating, since nothing before it
ever drains. Didn't find a construction in the time available; flagged
rather than papered over with a self-referential assert.
2026-08-01 13:34:48 -07:00
Matthew Jackson b002da4221 Fail the identity probe when INQUIRY returns a short data phase
DriveId::from_drive issues three data-in commands. The two GET CONFIGURATION
calls both clamp on bytes_transferred, with a comment noting it is
device-reported and untrusted. INQUIRY, three lines above them, discarded it
and decoded bytes 8..43 unconditionally.

The buffer is pre-zeroed, so a drive answering GOOD status with a short or
empty data phase — a USB-SATA bridge mid-wedge does exactly this — produced
blank vendor, product and revision strings and a byte 0 of 0x00. Every
platform enumerator gates on raw_inquiry[0] & 0x1F == the optical peripheral
type, and 0x00 is DIRECT ACCESS, so the drive silently disappeared from the
device list instead of reporting that its identity probe had failed. The
operator sees no drive at all rather than an error.

Anything shorter than the SPC-4 standard 36-byte header is now
E9058 DriveInquiryShort, and the buffer is truncated to what actually
arrived so nothing decodes past it. Exactly 36 bytes is still accepted: the
vendor-specific tail is optional.

This is the same defect as the READ CAPACITY short-transfer bug fixed
earlier today, in the same crate, found the same way.
2026-08-01 12:21:15 -07:00
Matthew Jackson 51d2b14d03 Give extract and analyze one label-parser tie-break, not two
mod.rs had two independent implementations of "pick the winning label
parser". select_result owns the rule — highest confidence wins, and on a tie
the earlier entry in PARSERS wins — and carries a regression test for a past
bug where analyze() picked the LAST equal-confidence parser instead of the
first. extract(), the path that actually ships, re-derived the same rule with
its own inline scan and had no test of its own.

The tie-break is load-bearing rather than incidental: array order encodes a
trust ordering, with the hand-vetted parsers registered ahead of the ones
that detect on any BD-J disc. A later parser winning a tie means the disc
gets labels from a less trusted source, silently.

extract now collects its candidates and calls select_result. Confirmed by
flipping the tie-break to prefer the later entry: the shared test fails,
where before the fix that mutation was invisible to the whole suite.

This is the seventh instance tonight of one policy implemented twice with
only one copy hardened, and the second in this file after the chapter
mark_type filter.
2026-08-01 12:15:48 -07:00
Matthew Jackson c4ad4184ec Make the FMTS phase gate reachable by tests
An FMTS forensic segment interleaves two variants at the unit level: the
disc carries both halves and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving it as
ciphertext is correct, because the muxer drops untouched ciphertext cleanly.

The decision lived inline in apply_aacs_map's per-unit closure, where no test
could reach it. A mutation run flipped the subtraction to an addition and the
parity comparison, and every test still passed — so the gate that decides
which half of a forensic segment we decrypt was entirely unasserted.

It is now unit_is_our_phase(). The index arithmetic also saturates and
clamps: both inputs come from the key map, so a unit below its own range
start or a zero unit size means a malformed map, and neither may panic on
debug overflow or a divide by zero inside a library a long-running service
depends on.

Four of the five surviving mutants now fail. The fifth, dividing by the unit
size versus multiplying, is equivalent rather than uncovered: aligned offsets
are exact multiples of the unit size, so the two differ only by a factor of
unit_sectors squared, and unit_sectors is the constant 3 — odd, so parity is
preserved on every reachable input. The proof is recorded in the test instead
of a test that pretends to cover it.
2026-08-01 11:48:39 -07:00
Matthew Jackson 528a6b7345 Walk past empty extents iteratively instead of recursing
fill_extents skipped an exhausted or zero-sector extent by calling itself,
which costs a stack frame per skipped extent. Nothing filters
sector_count == 0 out of a UDF or MPLS extent list, so a malformed disc
declaring a long run of empty extents recursed once per extent before
reading a single sector. Rust does not guarantee tail-call elimination, so
that overflows the stack — which aborts the process rather than returning
an io::Error, taking a long-running service down with it.

The skip is now a loop. The regression test runs on a 256 KiB stack, where
the recursive version dies and the loop finishes immediately.
2026-08-01 11:05:12 -07:00
Matthew Jackson fb321f51eb Reject a short READ CAPACITY reply, and count only entry marks as chapters
Two cases of the same shape: one policy implemented twice, with only one
copy hardened.

Disc::read_capacity decoded buf[0..4] from READ CAPACITY (10) without
checking that the transport actually delivered four bytes, even though its
comment claims to mirror decode_read_capacity — which has exactly that
check, and documents why. A drive answering GOOD with an empty data phase
leaves the buffer zeroed, so last_lba decodes to 0 and the probe reports a
one-sector disc instead of an error. It now calls the shared decoder rather
than re-deriving it.

collect_chapter_summary filtered chapters on mark_type <= 1, counting the
reserved type 0. PlaylistMark's own doc says filters must test == 1, and
disc/bluray.rs did; the labels path did not, inflating the public
chapter_count and letting a playlist whose only marks are reserved pass the
chapter_count == 0 skip. Both sites now share PlaylistMark::is_chapter_mark
so the copies cannot drift again.

Both fixes were confirmed red before green.
2026-08-01 11:00:49 -07:00
Matthew Jackson e0ff0cfeb4 ci: make libfreemkv prove its five dependents still compile
Every job above proves libfreemkv builds; none proved anything built on
it does. That gap bit today one level up — an engine signature change
broke autorip and went unnoticed, because consumer CI only fires on a
push to that consumer, and nobody pushed one.

libfreemkv sits underneath all five dependents, so a break here costs
more than a break anywhere else in the project. It is now the place the
question gets asked, since it is the place the change happened.

cargo check --all-targets only: each dependent has its own suite for its
own behaviour. This answers the narrower question that went unanswered.
2026-07-31 18:40:24 -07:00
Matthew Jackson 71686f1407 Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv,
the crate the other seven build against and the one held up as the
reference workflow, was the last one still linting the library only — so
its ~3,000 tests, by far the largest body of test code in the project,
had never been linted at all. Turning the flag on surfaced 74 findings.

Most were mechanical and applied with clippy --fix. The rest, by hand:

- Four discarded Results in decrypt.rs. css::descramble_region returns a
  Result and four CSS tests threw it away, so a descramble that FAILED
  would have surfaced as a confusing buffer-comparison mismatch instead
  of the actual error. They expect() now.
- A dead `kp` field on the PlantedWalk fixture. The test deliberately
  asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4
  rather than against a stored value — its doc comment says so — which
  makes the field not just unused but a trap: the obvious "fix" of
  asserting against it would quietly weaken the test to comparing the
  fixture with itself. Removed.
- Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut,
  three vec!s that only ever needed arrays, a filter_map whose every arm
  was Some, and a Vec::new()+push chain.
- Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering
  in the generated docs.
- A five-[u8; 16]-tuple return type named FourLevelParts.

Three lints are allowed at the specific sites, with reasons, because
they are wrong for this domain: the underscores in the bitstream-header
literals mark BITFIELD boundaries, not digit groups, so regrouping them
uniformly would satisfy the lint by destroying the only thing they
encode; and in three table-validation loops the loop variable is the
domain value under test (a DTS SFREQ code, an AMODE value, a palette
entry number), which is what the assertion messages name.
2026-07-31 15:08:37 -07:00
Matthew Jackson d50a7173ad Expose error_code so consumers can read a code instead of parsing one
io_error_code was private, so the predicates built on it (is_halt,
is_skippable_title_stub, is_disc_level_no_key) were the only way to ask
anything about an io::Error's origin. A consumer that needs the code
itself — to report WHY a title failed rather than to branch on one of
three known cases — had no route to it: mux_stream returns an io::Error,
the typed Error is gone by then, and only the E<code> string prefix
survives.

That left every front-end to re-implement the prefix parse by hand,
which is precisely the string-matching 1.5.x spent its time removing.
One parser, exported.

No behaviour change: the function is unchanged and the three predicates
still call it.
2026-07-31 14:46:39 -07:00
Matthew Jackson e9811a1e01 ci: make the branch tip actually buildable
CI checked out one repo, but libfreemkv path-deps ../freemkv-unlock on
the branch tip — release.sh swaps that to a git tag only inside the
TAGGED commit, then restores the path dep on the branch. So the branch
tip has never been buildable in CI by construction, and every green run
we have ever had was a tag build. Windows and Linux were first compiled
at release time, which is the worst moment to discover a build error.

Both repos now check out into subdirectories (actions/checkout refuses a
`path:` outside $GITHUB_WORKSPACE, and ../freemkv-unlock is outside), so
the path dep resolves exactly as it does on a developer's machine. Every
cargo step runs with working-directory: libfreemkv, and rust-cache is
pointed at the same workspace.

The sibling is taken from `dev`. On main/tag builds the Cargo.toml in
that commit carries the git-tag dep instead, so the extra checkout is
simply unused there.

This is what makes the new branch policy's "dev must be green"
achievable rather than aspirational.
2026-07-31 13:27:05 -07:00
Matthew Jackson f3841c8aca ci: build dev as well as main
Work now lands on dev and CI must be green there; main only moves at
release time, to the tagged commit, so a push to main is the release
validation run rather than day-to-day feedback.

leak-guard already runs on every branch (on: [push, pull_request]) and
release.yml stays tag-triggered, so neither needed a change.
2026-07-31 13:16:56 -07:00
Matthew Jackson 7d48d820e5 fix(css): revert the hard-fail — it made real DVDs unrippable
I broke DVD ripping earlier today and the real-media acceptance gate
caught it on its first full run. Greenland.iso failed with E7013
"Decryption failed"; reverting only this change made it rip clean in 8
seconds. That is a regression I introduced, not a pre-existing defect.

WHAT I GOT WRONG.

Round 9's crypto lens reported that descramble_region "descrambles with
a key it just proved wrong" when the crib check rejects the cached key
and the re-crack also fails. I agreed, and made it Error::DecryptFailed
to match the AACS path, on the reasoning that CSS has no external key
source so a failed crack on a readable sector should never happen.

The premise was wrong. `attack_crib` is a HEURISTIC, not a proof: it
finds a periodic run in the unscrambled header and predicts the run
continues past 0x80. When that prediction does not hold, the crib
reports a mismatch even for a CORRECT key — and the re-crack then fails
BECAUSE the crib was never valid. So crib mismatch plus crack failure is
the signature of a crib false positive, not of a stale key. The cached
key is not proven wrong; it remains the best available evidence, and on
a real DVD it is very probably right. Real discs hit this constantly.

The deeper error was treating "no key" as one thing across schemes. An
AACS unit key either opens a unit or it does not — the Verify-Media-Key
relation decides it, and a wrong key is provable. A CSS title key is
recovered from the data itself by an attack whose success varies sector
by sector, so "the crack failed here" says something about THIS SECTOR's
plaintext, not about the key. Unifying the policy was right for the
schemes that can prove a key wrong. CSS cannot, and I folded it in
anyway.

decrypt_span keeps its shape and the cross-scheme test keeps its two
AACS arms, with CSS now explicitly excluded and the reason stated.

Three tests asserted the wrong behaviour and are corrected, including
one I rewrote earlier today to pin exactly this. Every one of them
passed the whole time the code was broken — because none of them had
ever seen a real disc.

The lesson is the one I kept stating and then did not act on: 3,013 unit
tests, ~400 mutants killed and nine audit rounds did not catch this, and
one acceptance run did. Synthetic media cannot reproduce what a real
disc does.
2026-07-30 22:07:32 -07:00
Matthew Jackson 42591c77fc test: constrain the AC-3/E-AC-3/DTS header decode and the boxes it emits
All 59 measured survivors in mp4/audio.rs: 50 killed, 9 proven
equivalent, none left unaddressed. No production change — every
extraction reads correct against ETSI TS 102 366 (5.3.2, 5.4.2, Annex
E.1.3, F.4, F.6.1) and TS 102 114 5.3.1.

It was a fixture gap, not a code defect, and a specific one: the
existing fixtures gave several fields the SAME value (fscod=0, bsmod=0,
lfeon=1, acmod=7) and asserted only derived channel counts. Nothing
asserted the emitted dac3/dec3/ddts payload BYTES at all, so the
packer's shifts and masks were entirely unconstrained. A wrong mask
there does not crash — it writes a box declaring the wrong channel
configuration, and a player believes it.

Several kills needed fixtures designed to discriminate rather than
merely exercise:

  the reduced-rate branch needed fscod == 3, which no test in the file
  reached — and `== -> !=` survived on an fscod=0 fixture only because
  the reduced table happens to return 48000 there too

  the DTS LFF mask needed a value where XOR and AND differ in MEANING:
  on the 5.1 fixtures LFF flips 1 -> 2 and BOTH codes mean "LFE present"

  the acmod mix-level skips needed three different acmods, because `^`
  is a no-op unless acmod == 4 exactly

  the ddts bitrate needed 44100/512, which does not divide evenly — at
  48000 the rounding is invisible and `/ -> %` on den/2 survives

The 9 equivalents are one pattern: OR-ing a shifted high part with a
masked low part on disjoint bit lanes, where the mask is on the same
line as the OR. Since cargo-mutants applies one mutation at a time, no
single mutant can break both. Each was applied and observed green.

FILED, not fixed: reserved sample-rate codes are silently guessed as
48 kHz (audio.rs:133, :20/:153) while a reserved DTS AMODE is refused
with a comment explaining why. Same silent-wrong-metadata class,
opposite answer. Refusing would make such a stream unmuxable, which is a
product call.
2026-07-30 21:51:26 -07:00
Matthew Jackson 2efe1425d6 refactor(decrypt): one orchestrator owns the no-key decision
There were TWO top-level decrypt paths: decrypt_sectors_impl for CSS and
clear media, whose AACS arm was a bare `return Err` stub, and a wholly
separate decrypt_sectors_mapped for AACS. Each scheme therefore decided
its own answer to "there is no key for these bytes", and nothing held
them to the same one.

They drifted, in opposite directions, within a single release:

  css::descramble_region descrambled with a key the sector's own crib had
  just proven stale — garbage behind an intact clear header, reported Ok.

  the mapped path returned early for any LBA outside every range, before
  ever asking whether those bytes were ciphertext, so an unkeyable
  encrypted unit passed through and extract counted it as good.

Both were fixed individually earlier today. This removes the shape that
allowed them.

decrypt_span is now the single orchestrator: it owns the loop, the
refusal, and the loss count, and each scheme supplies only what
genuinely differs. apply_aacs_map is a scheme step that reports what it
could not open; it no longer decides what that means. The public
wrappers (decrypt_sectors, _in_content, _mapped) are unchanged in
signature and all funnel through it.

Adding a scheme now means adding an arm here, which means answering the
refusal question. That is the point.

The new test asserts ONE verdict across all three schemes — AACS with no
map, AACS with an encrypted unit outside every range, and CSS whose
re-crack failed — plus that clear media is NOT a refusal. A per-scheme
test cannot hold this: each would keep passing while the two disagreed.
Flipping the AACS arm back to pass-through reds it.

Also removes the last of the tracing-capture scaffolding. Serialising
those captures crate-wide did not fix the 1-in-10 flake, and asserting
the predicates directly made the helper, both capture subscribers and
an unrelated dead OrderSink unused. Deleted rather than left behind.
2026-07-30 21:35:33 -07:00
Matthew Jackson b86f7aef17 fix(session): delete two dead accessors, make into_drive fallible
drive() and drive_mut() had ZERO callers — not in libfreemkv, freemkv,
autorip, bdemu, keysources or kdb. Deleted rather than converted: dead
public API that panics is not an API worth preserving the shape of.

into_drive() had two callers and now returns Result. The empty-slot
state is reachable through ordinary public use — stage_drive_as_reader
moves the drive into the reader slot, and calling into_drive twice moves
it out — so the panic was not guarding a caller error. identify() was
converted for exactly this reason in this same release; the fix went to
one of four public sinks and the other three were left.

I deferred this on the assumption the blast radius was large. It was
three call sites. Checking beats assuming.

Also fixes a REAL FLAKE in the gate, which is worth more than the above.
resolve_vid_only_bus_key_gate_reports_true_has_volume_id... failed about
one full-suite run in ten while passing every time in isolation. It
installed a capturing tracing subscriber to read back the has_volume_id
field of a warn.

That cannot be made reliable: dispatcher::set_default is THREAD-LOCAL
while tracing's callsite-interest cache is GLOBAL. The original author
knew, and called rebuild_interest_cache() — necessary but not
sufficient. I first serialised every capture in the crate behind one
lock (harness::with_captured_tracing, which also removed the same
hand-rolled dance from three other sites). Still 1-in-10, because the
cache can be re-evaluated against the process-default dispatch rather
than the thread-local one.

So the predicate is now a named function, handshake_has_volume_id, and
the test asserts the VALUE. A boolean does not need a subscriber to
check. The gate's hard-error behaviour keeps its own test.

Measured: 14 consecutive full-suite runs, 2994 passed, 0 failed.

A flaky gate is worse than a missing one — every green after it means
less, and this one had been eroding trust in the whole suite.
2026-07-30 21:18:13 -07:00
Matthew Jackson 5559987325 test(mux): replace a false-green discontinuity test with the property it named
a_signalled_discontinuity_survives_a_backstop_discard asserted that a
SOURCE-signalled discontinuity on discarded bytes still reaches the AU
that follows. It did not test that. Deleting the disc_marks push, or the
mark-retirement loop inside discard_gap_before, left it passing.

The mechanism: the `discontinuity = true` rode the FIRST over-cap push,
which still has the next AU's delimiter at buf[0] — so it force-flushes
as an over-long AU rather than discarding, and THAT AU consumes the
mark. The assertion's `.find(|x| x.data.contains(&0x22))` then filters
it out, and the flag it reads comes entirely from `pending_gap`, set by
the second push's backstop. Behaviourally identical to the test 40 lines
above it, under a name promising something else.

I wrote it this morning, in the same commit that fixed a different test
for having a fixture that never reached the code it named, while
cataloguing that exact shape. Third instance today of writing the bug I
was hunting.

The two mechanisms cannot be isolated in one fixture — a fragment that
trips the backstop sets pending_gap regardless — so they now get one
test each. The replacement drives disc_marks end to end with no backstop
involved: a flagged fragment that carries a complete AU and is emitted,
not discarded. Nothing else pinned that path. Removing the disc_marks
push reds it.

Found by the round-9 opus escalation over test quality, dispatched
because the sonnet pass over the same 17,000 lines of new test code
returned zero findings.
2026-07-30 20:07:11 -07:00
Matthew Jackson 72bcc371fb fix(io): a halted fsync is recognisable as a halt, not a hard failure
The three bounded-fsync failures returned bare io::ErrorKind values —
TimedOut, Interrupted, Other/EIO. `is_halt()` matches on
`io_error_code(e) == Some(E_HALTED)`, i.e. the "E<code>" prefix that
`From<Error> for io::Error` mints, and that is documented as the ONLY
recognised shape. A bare ErrorKind carries no prefix.

So cancelling a rip while sync_all / finish was inside the bounded fsync
made `is_halt()` return false, and the CLI reported a clean user cancel
as a hard I/O failure at the end of an otherwise complete mux.

All three were also mutually unclassifiable, which is the same
information-loss the numeric-code scheme exists to prevent: a caller
could not tell "cancelled" from "NFS wedged" from "worker died", and
should not retry the third the way it retries the first. And their
Display text is std English — "timed out", "operation interrupted" —
reaching a user from library code, which this crate does not do.

Now Error::Halted, Error::SyncTimeout (E_SYNC_TIMEOUT 9056) and
Error::SyncWorkerLost (E_SYNC_WORKER_LOST 9057), on both platforms.

E_HALTED also now maps to ErrorKind::Interrupted rather than falling into
the 6000..=6999 InvalidData bucket. A stop is an interruption, not
invalid data. Nothing branched on the old kind — every consumer uses
is_halt() — so this is safe as well as more accurate.

Two things worth recording. I first placed the E_HALTED arm AFTER the
6000..=6999 range arm and wrote a comment claiming it preceded it; match
arms are ordered, so the range won and the comment was simply false. The
test caught it. And the macOS test asserted only that each arm was
non-Ok with a particular ErrorKind — it passed throughout the period the
three were indistinguishable. It now asserts they can be TOLD APART,
which is the property that actually matters.

Found by the round-9 opus escalation over the API contract.
2026-07-30 20:04:08 -07:00
Matthew Jackson 54d038e478 fix(udf): file_start_lba must skip a leading unrecorded extent
Regression from the round-8 change that started RETAINING ECMA-167
4/14.14.1.1 type-1 (allocated, not recorded) descriptors. Retaining them
is correct — dropping one slides every later extent's data down by the
hole's length, corrupting the file silently. But read_icb_extent still
took extents.first(), so the value it returns can now be a hole.

A type-1 extent's lba is where SPACE is allocated, not where bytes live.
IcbExtent's own doc says exactly that. file_start_lba hands the value
out as "the absolute starting LBA of a file's first data extent", and
ifo.rs uses it as the base for every VTS VOB extent:

    file_start_lba(IFO) + vtstt_vobs + cell.first_sector

So a DVD whose IFO's first descriptor is type-1 reads its entire video
title set from the wrong place on disc. No error anywhere — the reads
succeed, they just land on unrelated sectors. Verified: the test reports
2900 instead of 2040 with the old code.

Same shape as file_extents/extents_abs_at dropping the recorded flag,
and the same root cause: one change taught read_icb_extents about a new
extent type and did not visit the accessors that consume its output.
Three of them; two are still open (task filed).

Found by the round-9 opus escalation over the API contract, dispatched
because the sonnet pass over the same scope returned zero findings.
Seven of its eight items were absences rather than wrong lines — the
class a wrong-line scan structurally cannot see.
2026-07-30 20:00:15 -07:00
Matthew Jackson 6868b93b7e fix(udf): retry the customary VDS location when the recorded one holds nothing
The Main Volume Descriptor Sequence was selected from the anchor's
declared extent whenever that extent's SHAPE was usable — length >= 16
sectors (ECMA-167 3/10.2.1), non-zero location, no address wrap — and
the customary location was tried only when the shape failed.

Shape is a property of the field, not of what is there. An anchor can
pass all three checks and point at nothing: a mastering tool that wrote
the reserve location, a stale anchor on a rewritten volume, or
deliberate corruption on an untrusted disc. The sweep then finds no
Partition Descriptor, partition_start stays 0, and the volume is
rejected as UdfNotFilesystem — while the real sequence sits unread at
the location the old fixed sweep would have found.

So the branch a DAMAGED disc actually takes had no recovery path, which
is backwards: that is the branch recovery exists for.

Now both are candidates and the fallback is retried on OUTCOME. This is
the same principle the Metadata File Location chain in this same
function already uses — treat the recorded value as a candidate, fall
back when it does not pan out — applied one level up. The sibling was
added in d5a9e70 and the asymmetry has been sitting there since.

A read fault inside a sweep no longer aborts before the next candidate
is tried, but it does not vanish either: it is held and returned if no
candidate yields a partition. "Could not read" stays distinct from "read
fine and the bytes say no", because mux::resolve caches the second for
the whole disc.

Not addressed, and pre-existing: the RESERVE sequence at avdp[24..32]
exists in ECMA-167 3/8.4.2 for exactly a damaged main sequence and is
still not consulted. Worth doing; a bigger change than this one.
2026-07-30 19:49:03 -07:00
Matthew Jackson 8d39ccc613 fix(decrypt): an encrypted unit outside every key range fails, not passes
decrypt_sectors_mapped returned early for any LBA no map range covers,
on the reasoning that unmapped means clear filesystem or nav. "The map
has no key here" and "there is nothing to decrypt here" are different
statements, and only the second makes passing the unit through correct.

On a multi-CPS disc an orphan clip — referenced by no playlist, so in no
title extent and therefore in no range — hits the first and was treated
as the second. The early return fired BEFORE the aacs_unit_encrypted
gate below it, so nothing ever asked whether those bytes were
ciphertext. extract_tree then counted them as bytes_good, dropped the
.partial suffix, set complete = true and exited 0: a scrambled file on
disk with a clean bill of health.

FileResult's own doc already stated the intended contract — "unreadable
sectors AND undecryptable units both land here (extract fails a bad
decrypt loud)". The code did not implement it on this path.

The fix is one line of symmetry. The split-unit branch immediately above
already makes exactly this distinction, checking aacs_unit_seed_encrypted
before refusing. This branch did not, so the same question got two
answers eight lines apart — the duplication shape this release keeps
finding.

Deliberately NOT changed: the decrypt decision on an orphan is still to
refuse rather than guess. Blind trial-decrypt is what the keymap-only
model exists to remove, and extract has no CPS/forensic fetch source.
The defect was never that we declined to key it; it was that declining
looked like success.

The test asserts BOTH directions — a clear out-of-range unit must still
pass through byte-identical, because that is the ordinary whole-disc
read and breaking it would trade one silent defect for a loud one.
2026-07-30 19:44:37 -07:00
Matthew Jackson 8c0de5711e fix(mux): resync-gate drops reach errors(), including after the gap resolves
ResyncGate::dropped is zeroed the moment a keyframe disarms the gate,
and the only EOF warning fires for gates STILL armed. So a mid-title gap
that resolves left no trace anywhere — and most gaps do resolve. A rip
with several concealed gaps reported 0 errors and 0 lost bytes while
whole GOPs had been discarded, which disc.rs's own test comment calls
the ONLY channel through which loss is reported.

This is the other half of e99b634. Arming the gate after an 8 MiB
backstop discard is right — a picture with dangling references must not
ship — but until the drop is counted that trades silent corruption for
silent loss.

The gate now carries dropped_total alongside dropped: per-run answers
"how expensive was this gap", cumulative answers "what did the caller
lose". errors() sums the gates.

Summed in ONE place rather than counted at the three admit call sites.
Three copies of the same increment is how the mux-flush path ends up
counting and the main path not, or the reverse — the duplication shape
this release has been removing. The gate already knows its own total;
the accessor just has to ask.

Found independently by two round-9 lenses, which is what raised it from
plausible to worth acting on.

Both halves are pinned: removing the dropped_total increment reds the
resync test, and removing the sum from errors() reds the DiscStream one.
The second test asserts the ACCESSOR rather than the gate's counter,
because a test on the counter would have passed throughout the entire
period the defect existed.
2026-07-30 19:40:48 -07:00
Matthew Jackson 9f25a4c454 fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote
it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry
(12.1.3). Both fields are MANDATORY there, so unlike Matroska — which
omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to
leave out. The result was a structurally complete file that passes every
container check, declares a 0x0 video track, cannot be rendered, and is
written with no error anywhere.

WHY IT WAS POSSIBLE, which is the part worth keeping:

pixels() previously fabricated 1920x1080 for Unknown. That was wrong but
playable, so this sink never needed a guard and the absence of one was
invisible. Changing the sentinel to (0, 0) moved the defect instead of
removing it — a zero PAIR still reads as a usable value, so the sink
stored it and serialised it.

The accessor's doc comment then ENUMERATED the callers it believed were
safe: "the Matroska sink omits the optional elements, the VobSub writer
omits its size: line, and no caller divides by either dimension." Two of
those three are true. MP4 was not on the list because MP4 has no guard
at all, and a prose list cannot enforce itself. mkv.rs's own comment
even states the principle — "the check belongs in the one accessor
rather than in each caller that remembered to write it" — and
labels/mod.rs still carried its own duplicate Unknown test long after
the accessor took that job over.

So: pixels() now returns Option. Not because Option is tidier, but
because every caller genuinely needs a DIFFERENT answer and the compiler
is the only thing that reliably makes them choose one. Matroska and the
metadata sinks take unwrap_or((0, 0)) with the reason stated at each
site; the VobSub path degrades to a palette-only .idx; MP4 fails with
E_MP4_UNKNOWN_RESOLUTION (9055).

Six call sites, not the five my first grep showed — I piped it through
`head` and acted on a truncated list. The compiler caught the sixth.
That is the same mistake as trusting a lens that reported silence.
2026-07-30 19:34:54 -07:00
Matthew Jackson 30bea12392 fix(css): no provable key is a hard failure, matching AACS
descramble_region descrambled with the key a sector's own crib had just
proven stale, whenever the re-crack from that sector also failed. The
clear header is not scrambled, so it survives intact: the sector still
opens with a valid pack start and passes every structural check the PS
demuxer applies. Only the payload is corrupted — exactly where nothing
looks. Ok(0) dropped, exit 0.

CSS has no external key source. The title key comes only from cracking
the data, so on a READABLE sector "no key" is not a missing input, it is
recovery failing on bytes we can see. That should never happen, and when
it does the answer is not to emit something.

Now Error::DecryptFailed — the same verdict the AACS path already gives
for a unit no held key opens. Both alternatives to failing are bad data
reported as success: descrambled with a rejected key it is garbage
behind a valid header, and passed through untouched it is ciphertext
where plaintext is meant to be.

WHY IT WAS POSSIBLE, which matters more than the fix:

There is no single place that owns "what do we do when there is no key".
decrypt_sectors_impl looks like the central dispatch, but its AACS arm
is a `return Err` stub — AACS decrypts entirely through
decrypt_sectors_mapped, a separate top-level path. So CSS decided its
own policy inside css/, AACS decided in decrypt.rs and mux/resolve.rs,
and nothing held them to the same answer. The asymmetry was not an
oversight; it was structurally permitted.

How a disc decrypts is one process — resolve a key for this data, apply
it, refuse if it cannot be proven. Only the resolve-and-apply step is
scheme-specific. Filed as a task: the policy belongs in one orchestrator
with the schemes supplying only what genuinely differs.

Two tests changed rather than added, both of which pinned the old
behaviour: the unit test asserted the sector was descrambled, and the
integration test asserted the scramble flag was cleared, which is what
descrambling-with-any-key does. Neither established that the result was
CORRECT — the fourth bad-test shape.
2026-07-30 19:10:32 -07:00
Matthew Jackson b2b611fa3b fix(udf): a read fault locating the Metadata File is not a non-UDF disc
Regression I introduced in d5a9e70 earlier today. The Metadata File
Location fix replaced

    read_sector(reader, meta_file_lba, &mut meta_icb)?;

with a candidate loop guarded by `.is_ok()`, which discards the error.
A transient read fault — marginal sector, drive re-read, an ECC recovery
that reports failure once — then falls through to the block-0 fallback,
the File Set Descriptor read there finds the wrong tag, and the volume
comes back as Error::UdfNotFilesystem: the deterministic verdict "this
is not a UDF disc" for a retryable I/O event.

mux::resolve MEMOISES that negative for the whole disc, so one flaky
read silently demotes every remaining title to the base-Unit-Key-only
path — the AACS 2.1 forensic units garble and the demux drops them. The
mux completes with less content and no error, which resolve.rs's own
comment says must never happen.

This file already draws the same distinction twice in prose, at the AVDP
check and the FSD check: read fine but the bytes say no is structural;
could not read is transient. The loop erased it.

Two things I got wrong on the way, both worth recording:

The first guard was `meta_tag == 0` — nothing read at all. That misses
the damaging case, which is the RECORDED candidate faulting while block 0
reads fine and holds some other descriptor. meta_tag is then non-zero and
not 266, and the fault is still laundered into a structural negative. The
guard is now `meta_tag != 266`: any unread candidate leaves the verdict
unproven, so the fault wins.

The first test denied BOTH candidate LBAs, and passed WITHOUT the fix —
the block-0 read fails too, so a read error propagates from further down
either way. A test whose fixture never reaches the changed line. That is
the eleventh bad-test shape this audit catalogued, and I walked straight
into it while fixing a defect found by looking for it. The fixture now
denies only the recorded location, and removing the guard reds it.

Found by the round-9 opus escalation over correctness, dispatched
because the sonnet pass over the same scope returned zero findings.
Silence from a cheap model is not evidence.
2026-07-30 18:57:13 -07:00
Matthew Jackson 4f4b1ed222 fix(labels): make deluxe master-enum selection deterministic
identify_master_enums picks, for each fingerprint, the best candidate
class out of CandidatePool. Its tie-break only prefers an exact ldc
count over an inexact one, so two candidates that are BOTH inexact but
both within LDC_COUNT_TOLERANCE are decided purely by iteration order —
and the pool was a HashMap.

Rust seeds HashMap per instance, so this is not merely unstable across
runs: the new test resolves the SAME jar to both Alpha and Beta within a
single process, across 16 iterations. The same disc could emit different
commentary/SDH/descriptive labels on consecutive rips of unchanged
input, with nothing in the output saying the choice was arbitrary.

BTreeMap fixes it by construction rather than by a sort someone can
forget to keep. The pool is capped at MAX_CANDIDATE_CLASSES, so the
ordering cost is irrelevant.

The test runs the whole identification sixteen times and asserts one
distinct winner. A single run cannot distinguish deterministic from
lucky, and the seed does not change within a process — so repetition is
what makes this a test rather than a hope.

Found by the round-9 labels pass, which was dispatched specifically
because every one of the ten lenses had reported leaving deluxe.rs
unread. 1,159 new lines that nobody had opened.
2026-07-30 18:49:59 -07:00
Matthew Jackson 944e6a8b09 fix: align the Linux fsync error with macOS, and clear three stale docs
Round 9 findings, triaged and verified against the pinned tree.

writeback_file: a bounded-fsync WorkerLost returned bare ErrorKind::Other
on Linux where macOS returns EIO. Round 8 fixed the Linux arm to return
Err at all — the right fix — but stopped short of matching the value, so
a consumer distinguishing timeout / halt / lost-worker had nothing to
branch on for the third case on one platform. Now EIO on both.

Three doc comments described the pre-fix behaviour, one of them for
longer than the bug existed:

  linux.rs durable_sync still said "all three fallbacks return Ok(())"
  mod.rs sync_all still said Linux silently swallows fsync failures and
    callers must not treat Ok(()) as a durability barrier
  mod.rs SequentialSink::finish repeated the same caveat

All three now say what the code does: a bounded-fsync failure is an Err
on every platform, so Ok(()) IS a durability barrier. A doc that
describes a fixed bug is worse than no doc — it tells a caller to write
a workaround for something that no longer exists.

au_assembly: discard_gap_before duplicated drop_marks_before's
mark-retirement body verbatim and added one statement. Mine, from
earlier today. It now calls it. Two copies of the same retirement loop
is exactly how the two call sites would drift back together.

clpi: ClpiStream's audio_format / audio_rate / video_format / video_rate
are decoded from untrusted on-disc bytes on every parse and read by
nothing. The identically-named fields consumed in disc/bluray.rs belong
to mpls::StreamEntry, not to this struct — checked, because an earlier
round wrongly called a live function dead. Deleted, along with the seven
test assertions that pinned them; the tests that pin pid, coding_type
and language remain. Also removed a section-header comment orphaned by
the get_extents deletion, describing a fixture that no longer exists.
2026-07-30 18:39:47 -07:00
Matthew Jackson 5360f8d309 test: salvage the orphaned labels/disc triage, and extract build_labels
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to freemkv-private first, because a
worktree is not a backup and this one had already nearly been lost.

One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.

Two collisions resolved by hand:

A second mod pass_progress_tests, written independently against the
same survivors as the one committed in c610285. Kept mine — it covers
the distinct-counters case and the Progress blanket impl, which theirs
does not — but theirs had three clamp tests mine lacked: good_pct,
bad_pct and pending_pct also clamp an overshoot, and I had only tested
that for work_pct. Merged those in as one test and proved each of the
three clamps load-bearing by removing them individually.

An unused_parens warning in a new fixture.

Method note, recorded because it cost real time: git apply --3way
STAGES its result, so `git diff` reads empty and the tree looks
untouched. I nearly concluded the patch had silently failed. Worse, the
first attempt piped through `head -20`, so `echo exit=$?` reported
head's status rather than git's — the same mistake this audit has
already documented once. Check the real exit status, and check
--cached, not just the working tree.
2026-07-30 16:36:13 -07:00
Matthew Jackson 8b8bcff106 test: pin five untrusted-input guards in the AACS 2.1 and CSS paths
Second pass over src/aacs and src/css. No production change; the only
non-test edits are two fixture bytes and one test rename.

Five latent panics on untrusted data, every guard correct and none
tested — so each was free to be deleted:

  variant.rs:224  a 0x04 record not a multiple of 5 indexes p_uv[0..4]
                  off a one-byte tail
  variant.rs:269  a 0x0c record shorter than the 0x04 slot count
                  slices past the cvalue table
  stevenson.rs:177  short sector read -> index 138 into a 129-byte slice
  stevenson.rs:208  a crib longer than the 1920-byte encrypted region
                    -> index 2058 into 2048
  stevenson.rs:272  a header periodic all the way to offset 0 ->
                    subtract with overflow

That last one is reachable from ORDINARY DVD data — constant or padding
bytes are periodic. Verified on HEAD: widening the guard to <= 0x80
passes all 64 css tests unmutated.

media_key_variant_from_kp had only a soft-correction test, so every
step past that early return was unexecuted. The new two-slot fixture
puts the covering slot at index 1, so the uvs[1 + 5*idx] and
cvalues[idx*16] strides stop multiplying by zero.

derive.rs:319 + -> - confirmed killable, as the first pass predicted:
p == 0 makes (p-1)..32 underflow. Every prior fixture used a uv whose
lowest set bit was 4, 10 or 11, so trailing_zeros() was never 0.

One fixture bug caught and fixed rather than papered over: a |= mutant
first SURVIVED because mk[14]'s 0x04 bit happened to be set, making OR
and XOR agree. The byte is now clear and an assert_eq! pins it, so the
fixture cannot drift back into agreeing with the mutation it exists to
catch.

walk_mkb_be24_high_byte_is_honored renamed to
walk_mkb_be24_middle_byte_is_honored. Its 0x00_0110 length exercises
the << 8 term only, which is why << 16 -> >> 16 survived it. The name
was the lie; both framings are worth having, and the comment now points
at the genuine high-byte test at 0x01_0004.

derive.rs 146:32 and 154:30 stay untested, now with a proof rather than
a judgement: bit_pos == -1 requires current_v_mask == 0xFFFF_FFFF, and
calc_v_mask can never return that — its loop condition holds at
!v_mask == 0, so it always shifts at least once. Both branches are
reachable only after the walk has gone non-convergent and is heading
for the bounded exit, where the return value is undefined. Termination
is already pinned.

Equivalents proven by observing green, including six more OR/XOR pairs
on provably disjoint bit fields, and the two KEY_CORRECTION_DATA sites
where the constant is the documented all-zero placeholder so x ^ 0 ==
x | 0. Those become killable only if a real per-licensee KCD is wired
in.

A partial confirmation sweep (138 of 415 mutants before the box
saturated) found 135 caught, one timeout that is itself a detection,
and exactly one survivor — the KEY_CORRECTION_DATA equivalent above.
2026-07-30 16:27:03 -07:00
Matthew Jackson d5a9e70700 fix(udf): read the Metadata File Location from the partition map
read_filesystem hardcoded the Metadata File's File Entry at block 0 of
the physical partition — 'the metadata file ICB is at physical
partition lba 0'. UDF 2.50 2.2.10 records where it actually lives, as a
partition-relative Uint32 at offset 40 of the Metadata Partition Map.
That field is the only thing on the volume that says where the entry
is; block 0 is merely where authoring tools usually put it.

On a conformant volume that recorded it elsewhere, block 0 holds
something that is not a File Entry, metadata_start falls back to
partition_start, the File Set Descriptor read there carries the wrong
tag, and the volume is rejected as UdfNotFilesystem. Worse, a volume
with a decoy file set at block 0 — as a rewritten or dual-structure
volume can have — does not error at all: it mounts a different
filesystem and reports success.

Verified on HEAD: reverting the lookup reds four tests, e.g. the
metadata partition beginning at 2000 where the map records 33754069.

The recorded location is trusted only when the map's partition type
identifier reads '*UDF Metadata Partition'. A Virtual (2.2.8) or
Sparable (2.2.9) map is ALSO ECMA-167 3/10.7.3 Type 2 and records
unrelated fields at offset 40, so its bytes must never be read as a
location. Deleting that guard reds its own test.

Block 0 stays in the candidate chain, so a volume whose map is absent
or wrong but whose Metadata File does sit there keeps mounting exactly
as before. This is additive, not a behaviour swap.

Also 30 tests and ~55 more mutants across read_icb_extents,
read_file_limited, read_inline_data, the prefetch stubs, parse_dstring
and parse_udf_name. The metadata-partition branch — the branch EVERY
real BD-ROM takes — had no test at all; nothing in the crate built a
two-partition-map volume.

Closes the max_bytes gap flagged earlier: 259 > -> == and > -> < now
die on both the declared-size and the inline-ICB paths.

Equivalents proven by application, notably two guards that read as
protective but are unreachable: pm1_len is a single byte so
440 + pm1_len < 2048 always holds, and ad_offset + l_ad <= 2048 is
enforced upstream so off + ad_size never exceeds the block.

Bit 0 (Existence) stays unread, deliberately. ECMA-167 4/14.4.4 makes
it a display hint, not a statement that the file is absent, and UDF
2.50 2.3.4.2 carries it through as the DOS hidden attribute. For a
ripper the consequences are asymmetric: honouring it can silently drop
a real .m2ts from the title list, ignoring it costs an extra name in a
listing.

Known structural limit, not fixed: read_filesystem takes only the FIRST
extent of the Metadata File, so a fragmented metadata partition would
map every sector past that extent to the wrong place. metadata_start
being a single base LBA is what forbids the fix.
2026-07-30 16:09:35 -07:00
Matthew Jackson 0bc8d7af9c test: constrain the AACS key-map gap fill, the PSI walk, and MP4 field offsets
Third pass over src/mux/. 40 survivors killed, no production change.

resolve.rs — the deleted-statement cluster is now fully constrained.
All 14 deletable statements probed; 9 were already caught, 5 survived:

  c.sort_unstable() in fill_base_key_gaps. Every existing case handed
  it cuts already in LBA order, but IndividualSegment.tbl is a record
  list. Verified on HEAD: deleting the sort passes all 54 resolve
  tests. The mutant lays a base-key fill straight over a forensic
  segment.

  last_idx = idx (FMTS gap fill) and last_idx = hit (multi-CPS cache
  hit). An extent with nothing to sample must inherit its neighbour's
  CPS unit; the mutants fall back to the first unit's key. Exactly the
  shape this file's own comments name — wrong key, no error,
  lost_bytes == 0.

  Both check_halt()? polls in probe_fmts_index_keys. These cannot be
  killed by outcome, since a later poll returns Halted too. The tests
  count reads instead, which is what the don't-hammer-a-struggling-
  drive rule actually says: after a Stop the drive is asked for zero
  content sectors.

The four unresolved += 1 arms each got a test, and deleting each fails
exactly one — one-to-one, so no fixture passes for the wrong reason. A
control test pins that the baseline table resolves, so an expect_err
cannot succeed for an unrelated reason.

ts.rs::scan_streams was never entered. Six killed, two of which return
wrong answers that look right: reading the PAT/PMT CRC as a table entry
invents a stream on PID 546 out of CRC bytes, and dropping the
ES_info_length skip decodes a descriptor as an entry and loses the one
after it. Every existing PMT fixture declares ES_info_length = 0; a
real BD PMT carries a registration descriptor on essentially every
entry. Also ISO/IEC 13818-1 2.4.4.3 program_number == 0 is the network
PID, not a program.

mp4/read.rs — 13. Height read as the width beside it; channelcount;
the 4-byte base-128 descriptor varint (every existing esds fixture uses
a single byte); all three optional ES_Descriptor fields, whose loss is
silent (an AAC track just loses its CodecPrivate); first-vs-last media
edit, which is A/V desync of the difference; and the version-1 mvhd
timescale offset, emitted by any writer whose duration exceeds 32 bits.

dts.rs — 7, from a real cargo-mutants run over the file rather than
guesswork. Including a buffer that IS the syncword, which is the state
a sync split across PES packets lands in the moment its last byte
arrives.

Equivalents proven by application, not argued: the sample_encrypted_units
guard pair is mutually redundant by construction (total*p/9 < total for
p <= 8), so either alone is equivalent and both together are not; the
PMT section_len guard is dead code where its PAT twin panics; three of
the seven EXSS_HEADER_MIN_BYTES arithmetic mutants still sum to 10.
2026-07-30 16:05:37 -07:00
Matthew Jackson e99b634635 fix(mux): a backstop discard is a discontinuity; a stream-start trim is not
drop_marks_before retired discontinuity marks alongside timing marks at
both of its call sites. At stream start that is right. At the
MAX_AU_BUFFER backstop it is not, and the two are now separate.

The backstop fires when 8 MiB accumulate with no AU start code in them
— corrupt or hostile input — and throws the run away. There IS a prior
AU in that case, and whatever emits next definitively does not continue
it: a decoder handed that picture resolves its references against
frames separated from it by megabytes of discarded data. Retiring the
flag meant the resync gate (resync.rs, driven from mux/disc.rs) never
armed, so the broken picture went out looking sound. Silent corruption
is the one class of loss this crate refuses to have.

At stream start the opposite holds. Bytes ahead of the first
access-unit delimiter are the tail of an AU that began before sync, and
there is no prior AU to be discontinuous from. Marking it would arm the
gate at the head of every title and drop its opening GOP. That risk is
why this was a decision rather than a fix, and splitting the call sites
is what avoids paying it.

Recorded as a sticky flag, not an offset mark. A mark placed at the new
base is retired moments later by the pre-sync trim that follows resync
— the gap has to outlive the bytes that caused it. I found that by
writing the test first and watching it fail with the mark approach.

The discard is a discontinuity whether or not the source signalled one,
and a signalled one on discarded bytes still reaches the AU that
follows; both directions are tested.

Note the first over-cap run is NOT a discard: the next AU's delimiter
is still at buf[0], so it force-flushes as an over-long access unit and
loses nothing. Only a run with no opener at all reaches the backstop.
The tests push twice for that reason — the single-push version passes
without the fix.

Swapping either call site for the other fails: reverting the backstop
reds the two gap tests, and arming the gate at stream start reds the
third.
2026-07-30 14:52:39 -07:00
Matthew Jackson f9d081ed45 test: drive the AACS 2.1 variant chain to a Media Key, and pin AES-G3
163 of 322 surviving mutants across src/aacs and src/css. No production
line changed — every function read correct; the finding was always an
absent test.

Two structural holes, both verified against HEAD before landing.

variant.rs had no test that ever produced a Media Key. Every terminal
assertion in the module was an Err classification — NotVariantMkb,
SoftCorrectionRequired, OnlineChallengeRequired. So the entire 2.1
success path (VARIANTS lookup, VKD selection, Kpnew, the final unwrap,
the verify gate) was pinned by nothing, and that path produces the
Media Key that becomes the VUK that decrypts every byte of a 2.1 disc.
Built the first complete planted variant MKB: the VARIANTS entry is
chosen as Kvn ^ 1 so the real VKD sits behind a decoy at table index 1,
making the lookup load-bearing rather than incidentally correct. That
one fixture kills 23 operator mutants across three functions.

aesg3 — the subset-difference tree node function — was in the survivor
list as replaceable by [0; 16], meaning every device key in the crate
would derive the same Processing Key. It is caught today only as a side
effect of a negative test added after the mutation run; nothing asserted
the relation itself. Pinned now via the spec relation ([C] 3.2.2) using
the FORWARD primitive, with s0 transcribed independently rather than
read back from AESG3_SEED, so the test cannot agree with a mutated
constant.

Same shape in derive.rs: plant_mkb was one slot with zero descent, so
slot indexing was the identity permutation and the ancestor-descent
branch never ran — which is why 39 of recover_dk_position's mutants
survived. Added a 3-slot fixture keyed at index 2 and a four-level
descent fixture whose expected Processing Key is written out as an
explicit aesg3 chain rather than computed by calc_pk_from_dk; a fixture
built by the function under test moves with its own mutations.

Two latent panics on untrusted input now have tests: a 0x05 cvalue
table shorter than the 0x04 slot index, and a drive declaring more
payload than the 32772-byte response buffer holds.

23 equivalents claimed with reasoning, and confirmed empirically where
possible — all eight css/lfsr mutants were run and exactly the seven
disjoint-bit-lane ones survived.

Explicitly NOT claimed equivalent: derive.rs 146:32 and 154:30 are
reachable, but only on the non-convergent bounded-exit path where the
function's sole contract is termination. A test there would pin
defined-but-meaningless output.

Noted for the next pass: the pre-existing walk_mkb_be24_high_byte_is_honored
used total length 0x0110, whose high byte is zero — it exercised the
middle byte only, which is why << 16 -> >> 16 survived it. Left in
place; a real one was added at 0x01_0004.
2026-07-30 14:44:35 -07:00
Matthew Jackson 3e13a155fa refactor(clpi): delete the unused EP-map to sector-extent path
get_extents had no caller anywhere in the ecosystem, and neither did
anything feeding it. Removed: get_extents, resolved_ep_map, full_pts,
full_spn, parse_cpi, EpCoarse, EpFine, the ep_coarse/ep_fine fields,
the unused version field, and the 35 tests that exercised them.

This reverses the fix in b2e1982, deliberately. That commit corrected a
real truncation — out_time past the last EP entry resolved to
last_ep_spn + 1, dropping everything from the final I-frame to EOF —
and the fix stands in history as the record of what was wrong. But the
defect existed for eight audit rounds precisely because the code had no
caller: nothing exercised it, so nothing noticed. Keeping speculative
infrastructure alive on the strength of a comment saying it is reserved
for a path that does not exist is how that happens again.

ClipInfo keeps source_packet_count (read by disc/bluray.rs) and streams
(the CLPI/MPLS cross-validation in labels/clpi_audit.rs). 1,139 lines
out, gate green in debug and release.
2026-07-30 14:39:22 -07:00
Matthew Jackson b2e1982051 fix(clpi): resolve out_time past the last EP entry to the end of the clip
ClipInfo::get_extents fell back to `last EP SPN + 1` whenever out_time
lay past the last entry-point. EP entries mark I-frames (BD-ROM Part 3,
CPI / EP map) and a clip's final GOP lies after the last one, so a
PlayItem covering a whole clip — whose OUT_time is the presentation end
— always lands in that arm. The extent then stopped one source packet
after the last I-frame.

Measured on a fixture with 200,000 source packets and the last EP at
SPN 131,072: sector_count came back 12,289 where covering the clip
needs 18,750. Everything from the last entry point to EOF is outside
the returned extent.

Scope, stated plainly: get_extents has NO callers anywhere in the
ecosystem today — it is #[allow(dead_code)] and documented as reserved
for the timestamp-range read path. Nothing ships this loss. It is
fixed now because a latent truncation in extent arithmetic is far
cheaper to correct before it has callers than after.

The SPN at-or-after an out-of-range out_time is the end of the clip,
source_packet_count, with .max(last + 1) so a disc that under-declares
its own packet count against its own EP map still yields a sane bound.

Also 174 mutants killed across clpi, mpls, ifo and ebml — the first
time any of these four files has been examined. And ebml's 8-byte VINT
back-patch was duplicated verbatim in end_master and end_master_buf
with its top four payload octets unreachable through either (they need
a 16 MiB..256 TiB buffer); extracted to fixed_width_vint8 and tested
across the full 56-bit payload, no behaviour change.

38 of ebml's 46 survivors are one equivalence cluster: every | in
write_size / read_id / read_size / read_uint_val ORs into disjoint bit
lanes, where ^ is the identical operation. Applied all 38 at once —
green — then spot-checked four individually.
2026-07-30 14:24:57 -07:00
Matthew Jackson 84a77f6a0e test: pin FMTS read_plan unit indexing to an unaligned range start
Five surviving mutants in AacsKeyMap::read_plan, all in the arithmetic
that decides which half of a forensic segment this disc's key opens.

The existing coverage used a forensic range starting exactly on the
extent's first unit. Under that shape several wrong formulas agree with
the right one by arithmetic accident: (lba + range_start) / us and
(lba - range_start) * us both produce the correct kept set.

Real ranges are not shaped like that. A range start comes from a source
packet number — start_spn * 192 through clip_byte_to_lba in
mux/resolve.rs — and 192-byte packets bear no relation to the 3-sector
aligned unit, so range_start % 3 is whatever the disc says.

Getting the parity wrong does not crash. It reads and decrypts the
ALTERNATE variant's half: the units this key does not open decrypt to
garbage, the units it does open are skipped. AACS 2.1 forensic marking
is precisely what makes the two halves differ, so the failure is silent
— a full-length rip carrying the wrong variant.

Two fixtures are needed because no single one kills both: an unaligned
range start with the extent beginning on it inverts the halves under
the + form, and an extent offset one unit-remainder from the range
start inverts them under the * form.

Also pinned the short-tail guard.  is for a remnant
SMALLER than a unit — bytes with no following unit to desync. Widened
to <=, the last whole unit of every extent bypasses the phase gate, so
a forensic segment ending at an extent boundary contributes one
alternate-variant unit to the rip.
2026-07-30 14:18:39 -07:00
Matthew Jackson 9de88969ca test: constrain the DiscStream loss surface and the empty-title guards
Second mutation pass over src/mux/. 26 survivors killed, no production
change. Verified on HEAD before landing: each mutation below passes all
1,237 mux tests unmutated-suite.

The priority item was the honest-loss-reporting surface. Both
DiscStream::errors and DiscStream::lost_bytes could return a constant
with nothing failing — a rip that lost sectors would report zero loss
to the caller. This project has already shipped one defect of that
shape (a total decryption failure reported as an empty title, exit 0).
Driven now through two short-read fills so both land on values that are
neither 0 nor 1 and differ from each other; no constant and no field
swap survives.

MkvStream::finish -> Ok(()) also survived. MkvMuxer::finish has the
zero-frame MkvInvalid guard and two tests cover it, but the Stream
wrapper above it could return Ok unconditionally and bypass the guard
entirely — the empty-title defence was one layer thinner than it looked.

au_assembly: pinned au_opener_from behaviourally to the normative byte
values for all four modes, with negative cases for codes that are
explicitly not openers (MPEG-2 slice 0x01..0xAF, user data 0xB2,
extension 0xB5, sequence end 0xB7 per 13818-2 Table 6-1; VC-1
0x0A/0x0B/0x0C; H.264 SPS/PPS/IDR-slice). au_assembly and codec/ hold
independent copies of these constants; they agree today, and comparing
constants would not catch logic drifting apart, so both sides are now
pinned to the spec instead of to each other.

demux_sink::sanitize: every filename component demux:// writes comes
from disc-controlled text, so the path-separator arm is a traversal
guard. Deleting it now fails, including an end-to-end case where
base = "../evil/Title" must produce exactly one file inside the
chosen directory.

stts_and_ctts_expand renamed to stts_expands_runs_to_per_sample_deltas_in_order
and given runs with distinct deltas AND distinct lengths. Its old name
claimed ctts coverage it never had, which is why the composition-time
chain went unconstrained for eight rounds; the doc comment now points
at the tests that do cover ctts.

Correction to the previous pass: codec/truehd.rs flush -> vec![] IS
equivalent. Applied it, full mux suite green. TrueHD buffers across PES
but parse emits every complete unit immediately, so a residual buffer
at EOF is a truncated access unit and is correctly discarded. The
vec![Default::default()] variants are genuinely different and are
killed.

Deliberately not constrained: mkv::set_opening_capture (diagnostics
behind a process-global tracing check, flaky under the parallel
runner), and the three stdio.rs header paths (StdioStream holds
concrete io::Stdin/Stdout and cannot be driven without a production
refactor to injectable Read/Write).
2026-07-30 14:13:33 -07:00
Matthew Jackson 170fd0c064 test: constrain MP4 composition timing, MLP substream directory, and codec-private absence
Mutation testing over src/mux/. No production change — 49 survivors
killed, all proven red before green.

The MP4 composition-time chain was entirely unconstrained: VideoTiming::ctts,
build_ctts and parse_ctts could each return a constant and the suite
stayed green. Confirmed on HEAD: build_ctts -> vec![] passes all 1,220
mux tests. A demuxed B-frame title presenting in decode order would
have shipped.

The cause is a test whose name asserts coverage its body does not
deliver — stts_and_ctts_expand builds an stts box and never touches
ctts, and write_then_read_round_trip asserts sample sizes and keyframe
flags but not one PTS. Same shape as the set_speed forwarding finding,
different disguise.

mlp_num_substreams / mlp_substr_header_size: every TrueHD fixture in
the crate uses one substream and no extraword, so both could return a
constant and agree with all of them. These position mlp_parity_ok's
window over the AU header, so a constant mis-windows the parity check
on exactly the multi-substream AUs that carry 7.1 and Atmos.

CodecPrivate absent vs empty: mkv.rs writes Some(bytes) verbatim and
omits the element on None (RFC 9559 5.1.4.1.24), so a zero-length Some
emits a track header asserting the config IS empty. Four parsers could
return Some(vec![]) before any frame.

Also: mandatory ISO/IEC 14496-12 boxes (tkhd, vmhd, smhd, dinf, mdhd)
could each build empty; HEVC num_extra_slice_header_bits (H.265 7.3.2.3)
was never non-zero in any fixture, so the slice-type offset skip was
unexercised; chapter names from the disc go straight into
<ChapterString> and the & escape must run first; a stray 0x47 in a
payload must not latch a TS resync.

Documented as equivalent rather than killed: CodecParser::flush and the
three parser flush bodies that differ from the mutant only by a tracing
call, and DropTally::log_summary.
2026-07-30 13:39:02 -07:00
Matthew Jackson 55b97ac576 fix(udf): skip deleted File Identifier Descriptors
read_directory decoded file-characteristics bit 1 (Directory) and bit 3
(Parent) but never bit 2 (Deleted) — ECMA-167 4/14.4.4. It followed the
ICB of a descriptor naming a file that no longer exists.

4/14.4.3 permits a deleted FID's ICB field to specify an extent of
length zero, so it need not point at a File Entry at all. Following it
reads whatever descriptor occupies that metadata LBA:

- deleted DIRECTORY FID: recursion lands on the File Set Descriptor
  (tag 256), hits the not-a-File-Entry arm and returns Err. One stale
  descriptor in one directory fails enumeration of the entire volume.
- deleted FILE FID: read_file_size returns Ok(0) for a non-File-Entry
  tag, so a deleted name is reported as a real zero-byte file.

Both directions of the same shape at once: a recoverable condition
becoming a hard failure, and a non-existent entry becoming a plausible
success.

Bit 0 (Existence) is still unread. Skipping hidden files could hide
real content, so it is left alone deliberately rather than folded in.

Found by mutation testing: 48 of the 51 surviving mutants in
read_directory are killed by the 17 tests added here, covering the
short_ad decode byte by byte (ECMA-167 4/14.14.1), the extent-type
mask, the AD bounds guard at the exact sector end, the tag-261 File
Entry directory arm that no test reached at all, the multi-sector
read offset, the FID stride including L_IU, and the nesting cap.

Three survivors are genuine equivalents and are documented as such:
l_fi > 0 vs >= 0 (the emptiness guard below reaches the same state),
and the << 32 / | in the visited-set key (meta_start is constant
across a walk, and the two halves are disjoint).
2026-07-30 13:32:06 -07:00
Matthew Jackson c610285910 test: constrain SectorSource speed forwarding and PassProgress percentages
Mutation testing left both unconstrained.

sector/mod.rs — set_speed on the Box<dyn> and &mut dyn forwarding impls
could be replaced with an empty body and nothing failed. This one hides
better than the read methods because the trait's own default body is
already a no-op, so a forwarder that swallowed the call is
indistinguishable from a source with no speed control. Consequence is a
silently absent value, not a wrong one: the recovery path lowers read
speed through a damaged region, and a swallowed call leaves the drive
at full speed while the caller believes it slowed down. Routed through
a generic S: SectorSource bound, since a direct call on a &mut dyn
receiver auto-derefs to the vtable and never enters the forwarding body.

progress.rs — 42 survivors. All four percentage accessors could return
a constant, read the wrong byte counter, or have their divide-by-zero
guard inverted. Added exact-value tests (25%, not 'some percentage'),
both sides of each guard, the overshoot clamp, and one test setting all
three disc counters to distinct values at once — without it, a swapped
field still passes every single-counter test.

The Progress blanket impl for closures could return a constant true.
That return value is the cancellation signal, so a constant-true body
makes every closure-based consumer uncancellable.

Each mutation applied, observed red, reverted.
2026-07-30 13:26:31 -07:00
Matthew Jackson e4b1e5b19e docs: correct info invocation and read timeouts
TROUBLESHOOTING step 2 said `freemkv info`, which needs a source URL;
the drive route is `freemkv info disc://`.

architecture.md quoted 1.5 s / 30 s for the read timeouts. The
constants are READ_TIMEOUT_MS = 10_000 and READ_RECOVERY_TIMEOUT_MS =
60_000 (src/scsi/mod.rs:72,94).
2026-07-30 13:21:04 -07:00
Matthew Jackson 8d4a6d54a4 Constrain five behaviours that mutation testing showed nothing constrained
Fifteen surviving mutants killed, from the highest-risk class: functions a
mutant could replace wholesale with a constant while all 2,555 tests
passed. None of the code was wrong. In every case a test was absent, which
is why eight rounds of reading never found any of them.

The one that generalises is in sector/mod.rs. Its existing test READS as
covering `read_sectors` on the `&mut dyn SectorSource` forwarding impl —
it takes a `&mut dyn`, calls the method, checks the spy. But the receiver
auto-derefs and dispatches through the vtable straight to the spy, so the
forwarding body is never entered. An earlier round hit this exact trap on
`set_unit_base` and fixed it with a generic helper; the read path kept the
test that looked right. Verified by stubbing the forwarding impl to Ok(0):
the new test fails, the old one passes. That makes a tenth distinct shape
of bad test in this audit, and the mutation list is how to find the rest —
any forwarding-impl method in it has the same problem.

decrypt.rs's two existing gate tests assert only `dropped == 0`, which is
precisely what the `Ok(0)` mutant returns; one asserts nothing else at all.
A wrapper that decrypts nothing therefore looked correct while the caller
muxed scrambled MPEG. Now pinned by descrambling a real CSS sector and
comparing against the plaintext it was built from — not against a
re-derived descramble, which would only assert the code agrees with
itself.

css/mod.rs's `is_scrambled_uncracked` turns out to have no production
callers at all; the enum is matched directly. Its three tests all assert
only the true direction, which is exactly why the `-> true` mutant
survived. It is public API, so a consumer routing on it would, under that
mutant, refuse to rip every clear DVD.

aacs/inf.rs's MKB drive read had no test whatsoever. Now pinned
byte-for-byte across multi-pack concatenation, the single-pack case, a
genuinely empty response, and error propagation — an unreadable MKB must
surface as an error, not as an empty one.

aacs/derive.rs's nine mutants are killed with planted MKBs built by
inverting the AACS relations, so no real key material is involved. The
assertions land on the derived Media Key rather than the intermediate
positions: a recovered position that does not actually walk to the planted
key is no better than None. A fixture-guard test asserts the planted MKB
parses, since an unparseable one would make every `-> None` body look
right.

2570 lib tests, debug and release.
2026-07-30 12:41:28 -07:00
Matthew Jackson 93e1436fc0 Test that the Media Key verifier actually rejects a wrong key
`km_verifies` is the gate deciding whether a candidate Media Key belongs
to the disc. The MK-pool brute force in resolve.rs runs every candidate
through it, so a version that said yes to everything would accept
whichever candidate it tried first and the rip would continue with a wrong
Media Key — wrong VUK, wrong title keys, garbage plaintext, and no error
raised anywhere.

Nothing tested it. Whole-crate mutation testing reported
`replace km_verifies -> bool with true` as SURVIVING: the body could be
replaced with `true` and all 2,556 tests still passed. A verification
routine whose verification was itself unverified, in the most
safety-critical function in the crate.

The implementation is correct — it matches the AACS relation
`AES-D(km, mk_dv)[0..8] == 01 23 45 67 89 AB CD EF`. Only the defence was
missing.

No real key material is required to test it. That relation means a valid
record for any chosen km is just `AES-E(km, <the constant> || anything)`,
so the fixture is self-contained. The test asserts three things: the key
the record was built for verifies; a key differing by ONE BIT does not,
which is the assertion that kills the mutant and is a near-miss rather
than a random key; and an MKB carrying no verify record does not default
to yes, because unverifiable and verified are different answers.

Confirmed by reintroducing the exact reported mutation and watching this
test fail.

This is the first defect found by mutation testing rather than by reading.
Eight audit rounds and a security lens that read this file in full all
missed it, because it is not a wrong line — it is an absent test, and only
an instrument that asks "would anything notice if this were broken?" can
see that.
2026-07-30 12:14:14 -07:00
Matthew Jackson 18f8b285c4 Bound the BD-J label parsers, and stop a crafted disc hanging the scan
Ten defects in code no previous round had ever scoped. `src/labels/`
identifies a disc's studio by parsing jar archives and JVM class files off
untrusted media, so every byte here is attacker-controllable — and 813 of
its lines were executed by no test at all.

The worst is a non-terminating loop. A fallback stream-number scan
advanced with `saturating_add`, and the comment says why: a crafted XML
"must not overflow (panic in debug, wrap-to-0 in release)". Once the
counter pins at u16::MAX and that number is taken, the loop cannot exit.
So a fix for an overflow panic produced an unbounded hang, which is
strictly worse — a panic is observable and catchable, and catch_unwind
cannot interrupt a live loop. Reachable from about 8 MB of XML.

Where the same overflow appears in the deluxe decoder the fix is
checked_add and stop, NOT saturation — twice wrong there, because
saturating would peg every stream past the ceiling at one number and
apply_labels binds on (type, number), silently mislabelling tracks. A
correctness bug wearing the costume of success.

Round 7 capped the ldc-string retention per class; nothing capped the
aggregate, so a 64 MiB jar held that budget for every class at once. Same
defect one level up, which is the shape that keeps recurring in this
directory. Four other amplifications are bounded the same way, each with a
stated headroom and a paired test proving real media passes untouched —
the tightest is 5x on a label length, the loosest 2000x on the stream
numbering space, against BD's 32-per-type STN_table limit.

Two are not caps at all: a quadratic membership scan became a set, and an
attacker-derived length added to a cursor without saturation now cannot
wrap. Nothing is excluded by either.

A `#[cfg(test)]` hand-copy of a shipping parser was the ninth bad test
this audit has found, and the first proven by mutation rather than
inspection: deleting the guard from the REAL function left all 26 tests
green, including the one named for that guard. Pointed at the real
function, the same mutation fails.

Separately, all three failure arms of the bounded fsync returned Ok(()) on
both macOS and Linux, so sync_all reported success for a durability
barrier that never ran. Only macOS was in scope; the Linux twin is fixed
here too, because a platform disagreeing with its sibling about whether a
failed sync is an error is the class that already produced an over-length
SCSI CDB macOS rejected and the other two truncated. Note the behaviour
change: a mux whose final sync times out on a wedged mount now fails
rather than exiting 0.

Three of the caps are proven by wall-clock deadline rather than an
operation count, with 18-80x margin on the passing side. On a heavily
oversubscribed machine those could flake.
2026-07-30 11:30:24 -07:00
Matthew Jackson c63dafcf1a Key each FMTS extent from its own CPS unit, and stop a bad ICB tag reading
as an empty directory

On an AACS 2.1 disc the non-forensic gap fill hardcoded pool slot 0 as
"the" base Unit Key, so every content LBA outside a forensic segment was
keyed with CPS unit 1's key even on a disc carrying several CPS units. It
does not fail loudly — it produces garbage plaintext. The gap fill now
resolves each extent's own base key from its ciphertext, sharing the
sampling and slot-picking the multi-CPS path already had rather than
adding a second copy, and memoised in the existing per-disc cache. A disc
with one base key still short-circuits with zero extra reads, which the
existing probe-cost test pins. Forcing the slot back to a constant fails
four tests, so the choice is load-bearing rather than incidental.

A directory ICB whose descriptor tag is neither File Entry nor Extended
File Entry (ECMA-167 4/14.9, 4/14.17) was turned into a successfully-read
EMPTY directory, indistinguishable from a genuinely empty one, while the
same tag on a file ICB was already a hard error. Fifth instance in this
audit of a failure converted into a plausible success value, and the
second in this very function — round 5 fixed a read error becoming a file
size of zero here.

An unrecorded extent (ECMA-167 4/14.14.1.1: allocated but not recorded,
logically zeros that still occupy file space) was dropped entirely rather
than contributing its length, so every later extent landed at the wrong
file offset. Silent corruption, not an error. Extents now carry a recorded
flag and the hole emits zeros without touching the media.

The Volume Descriptor Sequence was swept at hardcoded sectors 32..64 while
the anchor's own Main VDS Extent pointer was parsed into a comment and
ignored; ECMA-167 3/10.2.1 defines that extent by the field, not by
position, so a conformant volume placing it elsewhere failed to mount.

drive_status decoded byte 5 as Media Status without checking the event
header's NEA bit or notification class (MMC-6 §6.7), so a reply carrying
no media event descriptor decoded as "no disc". The drive is untrusted
input here, and this is the works-on-my-drive class.

Two more tests were found asserting the defects they sit next to — one
requiring unrecorded extents to be dropped, one that four drive fixtures
built non-conformant replies the corrected decoder rightly rejects. Both
rewritten. Combined with the DTS one in the previous commit that makes
three tests this round that locked a bug in as intended behaviour, which
is a different and worse failure than the tautological tests found so far:
a tautology fails to catch a regression, these actively defend the defect.

Not fixed, adjacent: extract_one_file streams extents sequentially and
will now READ an unrecorded extent's sectors rather than writing
guaranteed zeros. Offsets are right, and pressed media reads as zeros
there, but it is not zero-guaranteed the way read_file now is; that needs
a recorded flag through PlannedFile.
2026-07-30 11:15:52 -07:00
Matthew Jackson 46eb88c51f Feed the CSS crack the canonical extent order, and stop Resolution faking 1080p
Seven defects in the code the test suite executes least — 913 lines of
disc/mod.rs alone are run by no test at all, which is why this round scoped
from coverage rather than from what previous rounds said they had read.

Disc::scan_image kept its own copy of the crack's extent ordering and fed
crack_key_outcome largest-cell-first. That is the fifth instance in this
audit of a local reimplementation drifting from the canonical one, and the
cost here is a key that does not descramble the feature: picking by sector
count bypasses the capacity gate and can select a different VTS entirely.
The copy is gone — which title comes from the canonical order the scan
already applied, and the extents are handed over in playback order,
exactly as decrypt_keys_for_title does. Its doc records why the duplicate
existed so it cannot grow back.

Resolution::pixels returned 1920x1080 for Unknown. That is the FOURTH
instance of one trap and the other three were in this same file, two of
them fixed hours earlier — without sweeping for siblings, which is the
whole reason this one survived. It now returns (0, 0), and the sweep was
done properly this time: every remaining Unknown arm across the crate is
honest, and the two ColorSpace sites that look like fabrication are
emitting H.273 code point 2, which is the spec's own "unspecified". Two
callers carried local Unknown-to-zero workarounds — precisely the cost of
making callers responsible for a lie — and one is now redundant.

BD-ROM Part 3 code 0xA2 is the lossy secondary DTS stream, not lossless
Master Audio. A test asserted the wrong mapping as intended behaviour, so
correcting the code failed it; the test is deleted with a note pointing at
its replacement. That is a NEW failure mode for this audit: not a test
that cannot fail, but one that locks the defect in. There is no
DtsExpress variant to map to, so it takes the lossy DTS-HD member and the
approximation is documented.

Also: DiscSession::identify could panic through drive_mut once the public
API allows an absent drive — two siblings were converted in an earlier
round and this one was missed; an extent end that added without saturating
where the rest of the crate saturates; a diag reason string restating the
comparator's sort keys and drifting from them, now derived from them; and
a short read that advanced the offset by the full request, silently
skipping the gap. That last one existed twice, in two reads with the same
shape, now merged so they cannot drift apart.

The short-read policy is a judgement call I could not derive from a spec:
no skip_errors is a hard error, with skip_errors zero-fills and charges
the loss. It deliberately does not retry mid-unit, because resuming inside
an AACS aligned unit would trade a silent gap for a silent decrypt
desync — the worse of the two.
2026-07-30 11:12:55 -07:00
Matthew Jackson dea968f32b Stop AudioChannels and SampleRate fabricating a value for Unknown
Three copies of the same two mappings existed. The canonical accessors
returned 6 channels and 48000 Hz for Unknown; a third copy in diag.rs
returned 0. The honest one was the copy.

A plausible wrong answer is worse than an obvious one. Six channels at
48 kHz is indistinguishable from a real 5.1 track, so every caller became
responsible for remembering to check the variant first — and this crate
walked into exactly that: the json:// sink reported a confident 5.1 for
audio whose neighbouring fields said "unknown". That was fixed at the call
site earlier in this audit; this fixes it at the source.

The accessors now return 0, which is what both in-crate call sites already
coerced Unknown to by hand, so their guards are gone and the behaviour is
unchanged. Zero is also obviously wrong if it ever reaches output, where
six is not.

The diag.rs duplicates are deleted rather than corrected — a fourth copy
would have drifted too. Their only caller was a trace line in the same
file, now on the canonical accessors. Their tests moved across and gained
the Unknown case, which is the point: restoring either fabricated value
fails both.

Found by the round-7 correctness agent while fixing the json:// sink; it
flagged the third copy as out of its scope rather than touching it.
2026-07-30 10:09:01 -07:00
Matthew Jackson 079c9b1327 Add a seeded robustness harness for the untrusted-input parsers
Five parsers that take bytes straight off a disc are now swept with
generated input asserting one property: they return Ok or Err and never
panic. That is this crate's own hard rule, and the class seven rounds of
reading is worst at.

Written in-crate rather than with cargo-fuzz, which needs a nightly
toolchain this project does not use, and without proptest or arbitrary,
because one dev-dependency is a deliberate posture and the parsers take
plain byte slices. What is given up is coverage-guided mutation, which is
the real loss. What is gained is determinism: the same seed replays the
same cases anywhere, so a CI failure reproduces locally verbatim.

Three generators, and the second is the one that matters. Pure random
bytes die at the magic check and exercise the entry guards only; prefixing
valid magic is what reaches the parser body; mutating a mostly-zero record
is what reaches the offset and count arithmetic a hostile image would lie
about.

That claim is MEASURED, not asserted. A harness whose cases all bounce off
the entry guards is the fuzzing equivalent of a test that cannot fail, so
one test counts how many generated cases parse to completion: 15,606 of
60,000, about 26%. If a future change to a guard drops that to zero, the
test fails rather than continuing to report a meaningless pass. Two further
tests pin that the three generators produce different bytes and that a seed
replays identically.

1.2M cases across all five targets found nothing. On this evidence that is
a real negative rather than an empty one.

The first version of this file was itself broken in the way this audit
keeps finding: its two meta-tests set FREEMKV_HARNESS_CASES and raced,
because the test harness runs them in parallel and env mutation is unsound
there. The budget is a parameter now, and the environment is read once at
the call site.

Two crate-internal parsers widened from private to pub(crate) so the
harness can reach them. No public API change.
2026-07-30 09:45:56 -07:00
Matthew Jackson 327087c70e Make five tests capable of failing, and stop the presence probe unmounting the disc
The worst of the five was a regression suite that never touched the code
it guarded: nine batch-count tests called `safe_batch_count` and
`buggy_batch_count`, both defined in the test file itself. The u16
truncation they exist to prevent could be reintroduced in
sector/prefetched.rs with every one of them green. They now drive the real
producer through the public API, and reinstating the truncation fails five
of the nine. Worth recording that the symptom has changed since the
original fix: the unit-alignment clamp below floors a zero batch at three
sectors, so the bug is now a twenty-fold throughput cliff rather than the
stall it once was.

The MP4 reserve test's only numeric case was dominated by the floor and
the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now
has a case where the per-sample term dominates. The zero-count guard in
FileSectorSource was likewise unfalsifiable — seek-past-EOF and a
zero-length read both succeed — so the test now observes the file cursor.
The AACS media-key ambiguity guard had no test at all; the pool scan is
extracted so the verifier can be injected, because a genuine two-key
collision needs one ciphertext decrypting under two AES-128 keys to
plaintexts sharing a 64-bit magic, which is a 2^64 search and not a
fixture.

macOS implemented the documented cheap, side-effect-free presence probe by
building a full exclusive transport — which force-unmounts the disc. Linux
and Windows issue one TEST UNIT READY with no unmount; macOS was the
outlier. It now walks the IOKit registry for the media object instead.

The C shim's registry reads assumed CoreFoundation types the registry does
not guarantee, so a driver publishing a CFNumber where a CFString was
expected aborted the process from inside public API. Types are checked and
a wrong type treated as absent. The unbounded waitpid on the unmount child
is now a polled deadline, and the last-resort match gained the NULL check
its two siblings already had.

The empty-CDB guard existed only on Linux while a shared helper's comment
claimed all three backends had it. Moved into the helper, so the comment
is now true and macOS and Windows are covered.

One finding was REJECTED with evidence rather than fixed. The TrueHD
buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be
unreachable by any input: the parser only retains data when the buffer is
shorter than the declared AU, and that declaration is twelve bits, so the
worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over
all 65536 AU headers confirmed it. The fixture now sits at the reachable
ceiling and asserts that instead. The cap itself is left in place as
defence, unreachable by construction, matching how the AC-3 resync guard
was handled earlier in this audit.

Two behaviour changes worth naming: Linux's empty-CDB error becomes
InvalidCdbLength rather than a transport failure, and an unknown device
now reports absent media rather than a not-found error, because the
registry cannot tell an empty drive from a missing one. The latter is a
conflation of the kind this audit has fixed three times; it is recorded
for the next round rather than left silent.
2026-07-30 09:26:56 -07:00
Matthew Jackson b8fa5e74dc Stop the live rip path muxing Blu-ray 3D differently from the ISO path
Five defects, four of them the same shape: a local reimplementation of
logic the crate already had, which had drifted from it. Each is now fixed
by calling the canonical version rather than by patching the copy.

DiscStream::new — the live disc:// path — built every parser through the
plain codec lookup and never asked whether a video stream was an MVC
dependent view, though resolve::build_demux_state does. The same 3D disc
therefore muxed correctly from an ISO and incorrectly ripped live. The
open-coded loop is gone; both paths now call build_demux_state.

collect_psi_section reimplemented the continuity-counter gap test and
disagreed with process_packet in the same file: it tolerated neither a
duplicate packet nor an adaptation-field-only packet, which per ISO/IEC
13818-1 §2.4.3.3 does not increment the counter. A spec-legal PMT
continuation was read as desync and the title's stream list came back
empty. Both callers now share one `cc_is_gap`, and a duplicate packet's
payload is no longer appended twice — doing so would have corrupted the
section the check exists to protect.

The json:// sink called the channel-count and sample-rate accessors
unconditionally, and both fabricate a concrete value for Unknown, so it
reported a confident 5.1 at 48 kHz for audio whose format was unknown
while its own neighbouring string fields said "unknown". The keys are now
omitted, matching mkv.rs. This matters more than it did: a sample-rate
ladder fixed earlier in this audit means Unknown now reaches consumers
that used to receive a wrong-but-concrete value.

For an audio:// or sub:// sink the reference video track's output is
filtered out, so its first PTS was never recorded and every delay was
computed against zero — baking a wrong DELAY into the filename. The
reference is now recorded whenever a frame is on the reference track,
independent of whether that track has an output, so a normal title gets a
correct delay; where no reference is ever observed the tag is omitted
rather than guessed.

A third copy of the channel/sample-rate mapping exists in src/diag.rs and
was left alone as outside the confirmed set. It is the same drift shape
and is recorded for the next round.
2026-07-30 09:18:33 -07:00
Matthew Jackson 3f7d7af472 Bound three allocations an untrusted disc can drive without limit
The Program Stream demuxer appended every fed byte and enforced its 4 MiB
cap only inside a branch reached once a start code had been found. Input
containing no start code anywhere therefore hit no cap at all, and since a
whole title is fed through this demuxer, a zero-filled or ciphertext VOB
extent buffered the entire title — up to ~90 GB. When the buffer holds no
start code, only a two-byte `00 00` prefix can begin a PS unit on the next
feed, so that is kept and the rest dropped. The bound is exact rather than
a heuristic: a start code can straddle a feed boundary by at most its
first two bytes, so no real byte is discarded, and a test feeding
`FF FF 00 00` then `01 E0 ...` pins that.

The existing test named for this case fed a real start code first, so the
cap it exercised was the in-PES one. Renamed to say what it covers.

The BD-J label path had a different shape to anything found so far: the
cap is on the COMPRESSED size of a disc file while the allocation scales
with the decompressed size. A `.class` gated only by a path prefix
inflates to the 64 MiB ceiling, yielding ~33M retained strings from `ldc`
operands or ~67M pushes onto a symbolic stack whose depth was unbounded
despite the Code attribute's own `max_stack` being parsed and then
ignored. Bounded both, the stack by `max_stack` itself (JVMS §4.7.3).

The VMG TT_SRPT title count is an untrusted u16 with no de-duplication,
so ~800 KB of crafted IFO re-parsed one PGC 65535 times. Capped at 99, the
DVD-Video maximum, so no conformant disc is clipped.

Every cap carries stated headroom against real media, and each has a test
locking that real media still passes.

I rewrote three of the new assertions before landing them. They compared
the result against the very constant under test — `total <= MAX_TT_SRPT_TITLES`
— which passes vacuously the moment someone raises the constant, the most
likely future regression and the seventh instance of this tautology shape
in this audit. They now assert literals derived from the spec.

The TT_SRPT fixture also had to change: with 65535 identical entries the
de-duplication collapsed them on its own and the cap was never what
bounded the result, so the test passed with the cap removed entirely.
Distinct entries defeat dedup and leave the cap as the only guard;
de-duplication now has its own fixture. Verified by raising each of the
three constants and confirming all three tests fail.
2026-07-30 09:17:16 -07:00
Matthew Jackson fdd473d7e9 Remove emulation-prevention bytes before reading the H.264 slice header
The bytes after a NAL header are EBSP, not RBSP: ISO/IEC 14496-10 §7.4.1
has the encoder insert 0x03 after any 0x00 0x00, and §7.3.1 removes it
before parsing. The measured-picture-type parse read the raw NAL instead,
on the stated reasoning that slice_type is too early for an escape to
intervene.

That holds only up to a point. first_mb_in_slice is ue(v), so a value of
65535 or more needs sixteen leading zero bits and opens the payload with
0x00 0x00, which an encoder must then escape. A UHD frame is ~32,400
macroblocks, so a conforming Blu-ray never reaches it — but 8K does, and
the disc is untrusted input. Such a stream decoded slice_type against a
byte the encoder had inserted and reported the wrong picture type: a wrong
result rather than an error, which is the class this lens exists for.

The prefix is un-escaped into a 16-octet buffer rather than the whole NAL:
the two ue(v) fields are at most 32 bits each, so nothing longer can be
needed, and it keeps a per-frame allocation proportional to the frame off
the path.

The test pins both directions. It asserts the un-escaped prefix decodes to
first_mb_in_slice = 65535 and slice_type = 2, AND that the raw EBSP does
NOT — without that second assertion the test would pass whether or not the
fix were present, which is the failure mode this audit has now found four
times. The bit string was derived independently rather than by hand: my
first attempt at the fixture was wrong by one nibble and the test caught
it.

Also covers the cases that must NOT be unescaped: a 0x03 not preceded by
00 00 is ordinary payload, and 00 00 03 03 keeps its second 0x03 because
the escape resets the zero run.
2026-07-30 08:49:51 -07:00
Matthew Jackson 05fed1b0e0 Log the OS error when a SCSI command fails on Windows and macOS
Both backends discarded the platform's own error code on the execute hot
path — the one every READ(10) of a rip goes through — and collapsed every
cause to the same status-0xFF transport failure.

On Windows, open() and reset() in the same file both capture the Win32
error; execute() did not. That left ERROR_INVALID_PARAMETER (a struct
layout regression, the exact class this file's SDK-layout tests exist to
catch), ERROR_ACCESS_DENIED and ERROR_GEN_FAILURE (a genuinely wedged
drive) indistinguishable, with nothing in the log to tell a code bug from
a hardware one.

On macOS the same, and worse: the file had no tracing calls at all, where
the Linux and Windows backends both log their execute failures. Its open()
carefully decodes the shim's sentinel into typed variants instead of
flattening them, but execute() threw the IOKit return away — so another
process taking exclusive access mid-rip and a real hardware wedge produced
identical, empty diagnostics.

Logged rather than added to the error type: the typed variant is public
API, and the recovery classification is deliberately the same for all of
these. What was missing is the breadcrumb, not the distinction.

Two further findings from the same sweep were rejected. Windows reset()
always returning Ok(()) and macOS ignoring timeout_ms are both already
documented in the code as deliberate, and the reporter flagged them for
completeness rather than as defects.

Neither fix has a test: reaching either branch needs a failing ioctl or a
failing IOKit call, and both files are compiled only on their own platform.
2026-07-30 08:42:02 -07:00
Matthew Jackson d444afbdfc Turn a release-only slice panic into an error, and stop calling 32 kHz 48 kHz
Four round-6 findings.

FileSectorSource::read_sectors guarded its output buffer with a
debug_assert, which is compiled out in release — so an undersized buffer
panicked with 'range end index out of range' instead of returning an
error, out of a public SectorSource impl where the length is caller input.
Drive::read_fua already carries this exact guard, with a comment recording
the same panic being fixed there, and PrefetchedSectorSource has a
regression test for the same case; this impl had been given neither. The
new test is red in release for precisely the predicted reason: 'range end
index 8192 out of range for slice of length 2049'.

parse_track's sample-rate ladder ended in an unconditional S48, so any
SamplingFrequency below 44100 was recorded as 48 kHz. A 32000 Hz AC-3 or
DTS track is legal and common in broadcast-sourced content, and the wrong
rate then propagated into the reconstructed AudioStream. Anything below
the lowest mapped rate is now Unknown, which is what the crate's canonical
SampleRate::from_hz already returned — the ladder disagreed with it. The
ladder itself stays, because the MKV element is a float and wants
tolerance rather than exact equality.

shim_open_exclusive used the mach port from IOMainPort without checking
the return; on failure the port is left untouched and every IOKit call
below ran against an uninitialised value. shim_list_drives in the same
file does check it.

build.rs treated cc and ar as successful if the process merely SPAWNED, so
a genuine compile error in the macOS C shim produced no object file and
surfaced later as an unexplained link failure against a missing symbol.
The shim is macOS-only and is neither linted nor compiled on the other two
platforms, so a mistake in it has exactly one chance to be noticed.

The last two have no test: one needs IOMainPort to fail, the other needs a
deliberately broken C shim, and neither is reachable from the test
harness. Both mirror a correct sibling in the same file, which is the
evidence available.
2026-07-29 22:56:33 -07:00
Matthew Jackson 921404d135 Fix five clippy errors that only appear on the target CI lints
CI's clippy job runs on ubuntu-latest, so cfg(target_os = "linux") code is
what the gate actually compiles — and none of it is built by clippy on a
Mac. Five `-D warnings` errors were sitting in drive/linux.rs,
scsi/linux.rs, io/writeback/linux.rs and the Linux arm of drive/mod.rs:
four collapsible let-chains and one manual `% n == 0`. CI was red on the
lint job while the local gate reported all green.

Collapsed into let-chains, which the declared toolchain supports, and
folded drive/mod.rs's length precondition into its chain so the body no
longer needs a nested block.

Found because an agent working on the SCSI backends reported the lints in
passing while checking that a Windows-only file compiled. Worth noting how
it stayed hidden: every one of these files is cfg-gated to a platform this
machine is not, so no amount of local gating would have surfaced them. The
companion change to the precommit script closes that hole.
2026-07-29 22:37:47 -07:00
Matthew Jackson 399c3d2769 Reject an over-length CDB on every transport, splice H.264 param sets in place
Two fixes from round 5.

The Linux and Windows backends truncated a CDB longer than 16 bytes
(`cdb.len().min(16)`) where macOS returned InvalidCdbLength. Under SPC-4 a
command's length is fixed by its opcode group code, so a shortened CDB is
not a shorter form of the same command — it is a DIFFERENT command, and
the drive executes it and answers GOOD with data for a request nobody
made. A silently wrong result on the layer everything else sits on.

Rather than mirror the guard a third time it now lives in scsi::mod as
checked_cdb_len, with all three backends routed through it, so it cannot
drift per platform again. That also makes it testable everywhere: each
platform module is cfg-gated to its own host, so a guard inlined into
linux.rs and windows.rs would have had no test coverage on any single
machine. The shared helper is the only place the behaviour can be
asserted on every platform's CI.

The two existing macOS tests were tautological — they replicated the
guard's logic inline instead of calling it, so they would have passed with
the guard deleted. They now call the real helper.

Separately, the H.264 keyframe parameter-set re-assert grew a
few-hundred-byte prefix buffer to the full access-unit size, copied the
whole frame into it, and dropped the presized buffer: one extra
whole-frame allocation and copy per keyframe. A UHD title is ~200,000
frames of 150-400 KB with a keyframe every second or two, so that is
thousands of avoidable multi-hundred-KB copies per title, each large
enough to go through mmap. It now splices into the reserved headroom in
place. This mirrors the identical fix already made in hevc.rs, which the
H.264 path had drifted from.

Byte-for-byte equivalence is pinned by a test whose expected literals
were captured from the pre-change implementation, and which I confirmed
still passes when the old build-and-copy code is restored. The
no-reallocation claim is measured rather than argued: a counter over 30
bare keyframes, which reports 30 of 30 against the old path and 0 with
the splice.

The reallocation test initially passed even with PARAM_REASSERT_HEADROOM
set to zero, because a small parameter set fits in the presize's
incidental slack — it proved the fixture did not reallocate, not that the
headroom prevented it. Its SPS is now large enough that the constant is
load-bearing, so zeroing it fails the test.

Not verified: no runtime behaviour on Linux or Windows: no drive, no
ioctl. Both files were confirmed to compile for their own targets.
2026-07-29 22:34:22 -07:00
Matthew Jackson dc5b67ed46 Stop reporting an uncrackable CSS disc as N empty titles
Same shape as the mkv:// conflation fixed earlier in this round, found by
looking for it deliberately. E7023 carried two conditions with opposite
correct responses: one title on a multi-VTS DVD failing its own re-crack,
where skipping it and finishing the rest is right, and the main feature's
crack failing outright, which is disc-wide and dooms every title
identically. Because both raised the same code and that code is in
is_skippable_title_stub, an uncrackable disc walked all N titles printing
"title skipped, it was empty" and exited 0.

The disc-wide condition gets E7027 CssNoDiscKey, mirroring the AACS-side
E7022 NoDiscKey it is the analogue of, and joins is_disc_level_no_key.
The per-title raise keeps E7023 and stays skippable. Because the engine's
classifier already tests is_disc_level_no_key before the skippable
branch, this reaches the right outcome downstream with no change there:
such a disc now stops on the first title and reports no-key instead of
returning success with nothing written.

Disc::css_error deliberately still stores CssKeyMissing — autorip matches
that variant on the field to pick the CSS rather than AACS message, and
what consumers classify on is the gate's returned verdict, which is the
only thing that changed.

Two neighbouring CSS raises were examined and deliberately left alone:
the no-key branch in the same function is genuinely unreachable via
ensure_decryptable and documented as defensive, and resolve_dvd_title_key
is per-title on both of its call paths.

Verified by removing the new code from is_disc_level_no_key, which fails
both new tests; each pins both directions so neither can silently flip.
Not proven end to end against a real uncrackable disc — none available.
2026-07-29 22:29:51 -07:00
Matthew Jackson 5c6a6d0785 Round 5: reject a degenerate fixed lace, bound the pending buffer by bytes
Five fixes. Three are real defects with regression tests; two are bounds
that were expressible but not expressed.

A fixed-size lace (RFC 9559 §10.3.4) whose body is empty declared n
frames and carried none. The divisibility check passed, because 0 % n is
0, and `chunks` yields nothing on an empty slice whatever width it is
given — so the clamp that existed to avoid chunks(0) returned zero frames
where the Lacing Head said n. The whole lace vanished with no error
raised and the caller saw a clean short block. A zero-size frame cannot
be a valid frame, so it is now malformed.

A disc read failure while fetching a directory entry's ICB became a file
size of zero rather than an error. Zero is indistinguishable from a
genuinely empty file, so an unreadable ICB on a damaged disc silently
changed which titles a caller saw as present — read_directory already
fails hard on its entry-budget guard, so propagating is also what the
surrounding code does. read_file_size still returns Ok(0) for an ICB
whose tag is neither File Entry nor Extended File Entry, which is a real
zero and not a failure.

The pending-frame buffer was capped at 4096 frames, which does not bound
memory: frames are arbitrarily large and a UHD video frame runs to a few
hundred KB, so the existing cap permitted over a gigabyte. Now bounded by
bytes as well, at 64 MiB.

round_up_grain overflowed for inputs within one grain of u64::MAX —
div_ceil then multiply — and the wrapped product is small, turning the
largest possible estimate into a negligible reserve. It saturates, and
the reserve is clamped to what a `free` box's 32-bit size field can
actually hold, since writing a larger one truncated the size and left
mdat beyond a box claiming to be far shorter. No real title comes close;
a 90 GB UHD title estimates a few MiB.

The AC-3 resync guard now advances the PTS cadence like both of its
sibling branches, so the three paths out of that block cannot disagree.
This one is defensive and has NO test: reaching it needs input that both
parses frames and leaves a megabyte of residue, and the parser's own
carry rules drop pre-sync junk and cap a partial frame at 8192 bytes, so
no such input was found. Stated here rather than covered by a test that
would pass either way.

Two findings from this round were rejected on inspection. A reported
panic in the .mpls suffix check does not exist: the `.get(..)` on the
line above returns None off a char boundary and `filter` never runs its
closure, so the byte index is unreachable. A test written for it passed
against the unfixed code, which is what surfaced the error.
2026-07-29 22:28:43 -07:00
Matthew Jackson f5e169efb3 Stop reporting a corrupt mkv:// source as an empty title
E6008 meant two unrelated things: "this title produced no muxable
frames", which is a benign stub worth skipping, and "the source file is
malformed", which is not. Because a single code carried both,
is_skippable_title_stub answered yes to the second one — so feeding a
truncated or corrupt mkv:// input made the engine classify it
SkippableStub, print a notice saying the title was empty, and exit 0.
Silent data loss reported as success.

Split into E9053 MkvSourceInvalid for the read path (25 raise sites
across mkvstream.rs and ebml.rs's read primitives) and E9054
MkvUnencodable for the four write-side sites, which are the encoder
refusing to emit a body at or above the 56-bit VINT limit — an output
limit with no input involved, so calling it a corrupt source would be
wrong in the other direction. E6008 keeps only the zero-frame guard it
was documented to mean.

Kept one code for the whole read path rather than one per raise site:
nothing a consumer does differs between a bad VINT, a non-UTF-8 string
element, a truncated body and a child overrunning its parent. E9052 is
the model for when a carve-out earns its keep — laced blocks name one
specific RFC 9559 §10.3 feature with its own diagnosis.

Also fixed meta_sink.rs raising MkvInvalid for a serde_json encode
failure in the json:// sink, where no MKV is involved at all; it now
matches the identical guard in mux/meta.rs.

Reverting the split at the single code() arm reproduces the old
classification: 22 tests fail, including both new assertions. The
opposite direction is pinned too — dropping E6008 from the predicate
fails the genuine-stub test, which drives the real muxer end to end.
2026-07-29 22:11:53 -07:00
Matthew Jackson 0bbceed985 Round 4: fix 26 defects across crypto, resource use and codec paths
Twenty-six confirmed findings from the fourth audit round, landed as one
cluster because they were found by agents working over disjoint file sets.

The one worth calling out is a pair of AACS tests that could not fail.
Both asserted CBC behaviour against a hand-rolled expectation that
happened to be IV-independent, so replacing AACS_IV with sixteen zero
bytes left them passing — they were pinning the code's own arithmetic,
not the published constant. Replaced with a literal witness of the
published IV plus the NIST SP 800-38A F.2.2 CBC-AES128 vector, and
verified the other way round: zeroing AACS_IV now fails three tests.

The rest are allocation and correctness work on hot paths: the Annex-B
writer in demux_sink allocated and freed a whole-frame Vec per frame,
which for a UHD title is ~200,000 allocations over the mmap threshold
plus the page faults to first-touch each one; it now reuses a buffer on
the writer, and still takes the NAL prefix width from the configuration
record rather than assuming four.

Six findings whose real fix lives in a consumer crate are recorded for
re-filing rather than patched here.
2026-07-29 22:09:52 -07:00
Matthew Jackson 4fcd28b487 Parse MKV lacing, route by real TrackNumber, honour NAL length size and edit lists
Four conformance defects in the read paths, two of them silent corruption.

**Lacing was ignored entirely.** RFC 9559 §10.2 defines Xiph, EBML and
fixed-size lacing, where one Block carries several frames; the reader took the
Block payload verbatim, so a laced Block became a single "frame" consisting of a
lacing header followed by concatenated frames — garbage to the codec parser, no
error. Audio tracks from other muxers commonly use lacing, so an ordinary
foreign MKV was silently mangled.

All three modes are now parsed: Xiph 255-run sizes including the trailing-zero
rule for exact multiples of 255, EBML unsigned first size plus SIGNED VINT deltas
with the 2^((7*n)-1)-1 bias of §10.3.3, and fixed-size even division, with the
last frame's size deduced from the remainder. Laced timestamps follow §10.3.5:
the first frame takes the Block timestamp and the rest are spaced by the track's
DefaultDuration, else BlockDuration/count, else shared with a warn.

Parsing was chosen over refusing because refusal would leave freemkv unable to
remux common foreign audio at all, and each mode is about fifteen lines.

A malformed lacing header now raises a NEW code, E_MKV_LACING_INVALID = 9052,
deliberately NOT MkvInvalid — because is_skippable_title_stub classifies
MkvInvalid as a skippable nav stub, so reusing it would have recreated the exact
conflation that is still open as a separate finding. A test asserts the new code
is not skippable.

**TrackNumber was assumed to be 1..N in TrackEntry order.** RFC 9559 §5.1.4.1.1
only requires it to be non-zero and unique, so sparse or unordered numbers are
legal. Block routing and codec_private both computed track + 1. A real
TrackNumber map is now built, recorded only for TrackEntries that yield a stream
so dropped track types no longer shift the mapping.

Verified red here independently, and the failure mode is worse than mis-routing:
with track + 1 restored, a buttons track's payload was attributed to the AUDIO
stream — wrong payload into the wrong codec parser.

**The NAL length prefix was hardcoded to 4 bytes.** lengthSizeMinusOne lives in
avcC byte 4 and hvcC byte 21 (ISO/IEC 14496-15 §5.3.3.1.2, §8.3.3.1.2) and was
never read, so a source declaring 1- or 2-byte prefixes had its raw prefixed
bytes emitted verbatim with no start codes. All four conversion sites now derive
the width from the track's own configuration record.

**Edit lists were ignored.** No edts/elst was parsed, so the presentation
timeline an edit list defines (ISO/IEC 14496-12 §8.6.5/§8.6.6) was dropped —
which is how encoder delay is normally expressed. Leading empty edits and the
first media edit's media_time are now applied to both dts and pts, with the movie
vs media timescale distinction respected. A list needing more than a constant
shift applies the leading edit and warns rather than presenting the result as
faithful.

17 tests. I reproduced the lacing mutant independently: returning the body whole
kills five of them, including the exact-payload and malformed-header cases.

Still open and deliberately untouched: the MkvInvalid / is_skippable_title_stub
conflation across ~20 reader raise sites. It is a cross-cutting error.rs change
and E_MKV_LACING_INVALID is the template for it.
2026-07-29 21:53:19 -07:00
Matthew Jackson 9527bc1e13 Anchor forensic segments to the forensic clip, and stop CPS branching on resolve order
Two correctness defects in AACS 2.1 / FMTS key-map resolution, both order- or
anchor-dependent, and both able to abort a whole disc or silently garble it.

**The single-CPS short-circuit depended on which title resolved first.**
`pool_len` counted the WHOLE unit-key pool, and resolve_fmts_key_map appends the
disc's forensic index keys to that same caller-owned pool. The count was captured
before THIS call's FMTS branch but not before earlier titles', so once any forensic
title resolved, every later title saw a pool larger than one and fell into
multi-CPS sampling — 8 random reads per extent, and a whole-disc DecryptFailed if
no pooled key opened a menu extent's samples. A disc that ripped fine when a
non-forensic playlist sorted first failed when a forensic one did.

Forensic keys are now tagged FMTS_POOL_TAG_BASE = 1 << 24 and the short-circuit
asks single_base_key_slot(), which excludes them. The old 1000 tag was NOT kept,
and the reasoning is worth recording: base CPS ids are Unit_Key_RO.inf position + 1
and that count is a BE16, so 1000 sits inside a genuinely reachable id space. 1<<24
cannot collide. The id field is cosmetic — decrypt.rs indexes the pool by slot and
reads only the key — so widening the tag is safe.

**Forensic segment SPNs were anchored to the wrong clip.** They live in the
forensic feature clip's byte space, but were mapped through
clip_byte_to_lba(&title.extents, ..), which treats byte 0 as the start of the
title's FIRST extent. Any playlist not beginning with the forensic clip mapped
every segment to the wrong LBA: either the anchor probe sampled the wrong clip and
the whole-disc resolve aborted with FmtsKeyMissing, or — worse — a forensic index
key was applied to non-forensic sectors while the real forensic units kept the base
key, giving silently garbled output with no error at all.

The correct anchor turns out to be a DISC fact, not title data: an AACS 2.1 disc
names its forensic feature BDMV/STREAM/<clip>.fmts, and carries one
IndividualSegment.tbl, so the SPNs are in that one clip's byte space. A new
forensic_clip_extents() finds the unique .fmts in the already-walked UDF tree, and
those extents now drive the segment arithmetic, the addressability filter, and the
index probe — whose title parameter is gone, since its reads were mis-anchored too.
"Does this title carry forensic content" is now "does it read the forensic clip's
sectors" rather than "do the segment bytes land somewhere in the concatenation".

Where the clip is NOT identifiable — no .fmts, or several, making the SPN space
ambiguous — on a disc that does carry a non-empty table, the resolve now fails loud
with FmtsKeyMissing rather than guessing an anchor. That is a deliberate behaviour
change: a hypothetical disc with two .fmts clips hard-fails where it previously
produced a possibly-wrong map. Failing loud beats silently garbled output, and
inventing an anchor was not acceptable.

This site had been flagged independently three times — by the agent that added the
FMTS per-disc memo, by the round-4 correctness lens with a concrete scenario, and
by the round-4 conformance pass.

Verified red here independently: reverting single_base_key_slot to count the whole
pool fails both new tests. The agent's own evidence was probe_reads 48 vs 40 (the
8 extra sampling reads) and an E7013 DecryptFailed whole-disc abort, and for the
anchor an E7026 FmtsKeyMissing on a [trailer, forensic] extent list.

All six pre-existing FMTS tests pass unchanged through the new anchor, including
the exact-cost assertions (40 probe reads, one key-service call per disc, one UDF
walk for 60 titles), so the round-3 memoisation wins are intact.
2026-07-29 21:37:50 -07:00
Matthew Jackson 58bdb42f8e Group whole E-AC-3 frame sets, and keep short reads unit-aligned
Two defects in fixes landed the same day, both found by round 4 auditing round 3's
work rather than trusting it.

**The E-AC-3 grouping ignored substreamid, so the timeline still doubled.**
Confirmed independently by the correctness and conformance lenses and verified by
hand: substreamid appeared only in test helpers, never in the production path. Per
ETSI TS 102 366 (A/52) Annex E a frame set is independent substream 0 — mandatory,
always first — with its dependents, then the OPTIONAL additional independent
substreams 1..7 with theirs, all covering the same time period. Treating an
additional independent substream as a new access unit advanced the clock a second
time for the same 32 ms, which is exactly the doubling the grouping fix existed to
prevent. No fixture caught it because every fixture used substreamid 0.

is_dependent_substream becomes substream_role -> Starts | Extends: strmtyp 1
extends; strmtyp 0/2 with substreamid != 0 now extends (this was the bug); strmtyp
0/2 with substreamid == 0, legacy AC-3, and reserved strmtyp 3 start. Reserved 3
starts regardless of its id bits, because its BSI layout is undefined so those bits
cannot be trusted — an unknown frame is neither merged into an unrelated programme
nor silently discarded.

The frame set stays ONE sample rather than being split into a separate track for
the associated service, and the reasoning is in the module doc: a substream
numbered 1..7 with no substream 0 is not conforming, so extracting one would mean
renumbering ids and rebuilding frame sets — a transcode, not a remux. Programme
selection is the player's job.

A stream joined mid-frame-set (first sync is substreamid 3) is skipped with a debug
and resyncs at the next id-0, mirroring the orphan-dependent rule: its mandatory
id-0 substream was never seen, so it is neither decodable alone nor timeable.

MAX_AC3_BUF 128 KiB -> 1 MiB, because an AU is now a whole frame set: worst case 8
independent x 9 substreams x 8192 B = 576 KiB, which the old cap could have dropped
mid-hold.

**The forced probe's two round-3 fixes cancelled each other.** CHUNK_SECTORS = 1023
exists (with a const assert) so every read starts on a 3-sector AACS aligned-unit
boundary; the short-read fix advanced by actual bytes, making the advance a
non-multiple of 3. Every later read was then misaligned, DecryptingSectorSource
refused it before reading, the stop became ReadFailed, and no verdict was asserted
— so content-based forced detection silently fell back to the vendor label on
exactly the encrypted discs the 1023 change was written for.

A partially-satisfied read now advances only by whole aligned units and re-reads
the <=2 residue sectors from the next boundary, feeding only the aligned prefix so
nothing is double-fed and no partial unit reaches the parsers. A read that fully
satisfies its request still advances by all of it. When less than one aligned unit
comes back the bytes are fed and the same LBA is retried twice before stopping, so
a starved source cannot spin — verified by raising the retry limit and watching the
test hang.

Verified red independently here: reverting substream_role to strmtyp-only fails
eac3_additional_independent_substream_stays_in_the_frame_set (6 access units where
3 are correct — the doubling, literally) and the mid-frame-set resync test.

Reported, not fixed: dec3_box still hardcodes num_ind_sub - 1 = 0 and
num_dep_sub = 0, so it under-declares any stream carrying additional independent
or dependent substreams now that frame sets arrive whole. DolbyConfig has no
fields for either; a real fix needs the parser to surface observed substream
counts. That file is another lens's this round.

Unverified: no real multi-programme DD+ stream exists here, so defect 1 rests on
synthetic Annex-E fixtures. The retail DD+ check (No Time to Die, all substreamid
0) confirms single-programme discs are unaffected.
2026-07-29 21:33:18 -07:00
Matthew Jackson 62450e19bd Make two public-API panics return errors, and drop two shipped citations
**The session panics are reachable, and my earlier triage of them was wrong.**
`DiscSession::scan` and `resolve_keys` both did
`self.drive.as_mut().expect(..)`. I previously downgraded these to LOW on the
grounds that no shipped consumer calls them after the drive has been staged into
the reader slot. That is the wrong test: `stage_drive_as_reader` is a PUBLIC
method that empties the drive slot, so the public surface permits the sequence,
and a library must not panic from public API regardless of what current callers
happen to do. Both now return Error::DeviceNotReady.

**A shipped doc comment cited a third-party source FILE** as the authority for
the CLPI ProgramInfo layout ("Layout per the BD CLPI spec clpi_parse.c"). Now
cites the Blu-ray Disc Read-Only Format Part 3 CLIPINF specification.

**The CHANGELOG justified a muxer decision by naming a commercial competitor**
("MakeMKV's rip of the same disc omits it", "matching MakeMKV"). Reworded to
stand on its own terms: the element is optional in RFC 9559, nothing requires it
for interlaced SD, and the 40 ms DefaultDuration is the frame rate the source
actually carries.

The leak gate is extended for both new classes — third-party `*_parse.c` /
`*_dec.c` / `*_demux.c` style filenames, and a competitor named as authority.

Narrowing that rule took two attempts, which is worth recording. A bare
`\.(c|cpp|cc)` pattern produced eight false positives: this repo has its own C
shim (`macos_shim.c`) that build.rs and the docs legitimately reference, and the
pattern also matched the Rust field access `p.cc`. It now matches only the
suffixes typical of third-party media-library sources. This is the second
false-positive round on this rule — the first flagged ffmpeg INVOCATIONS in the
test harness — so the lesson is that a hygiene pattern needs testing in both
directions before it lands, exactly like any other code.
2026-07-29 21:21:29 -07:00
Matthew Jackson 013881ac06 Restore the MP4 conformance fixes I clobbered while landing another agent's work
detect_rate's nearest-match fix, the colr HLG/BT.470 fix and their four tests
were silently reverted. Cause: the r3fix-silent worktree was cut BEFORE the
conformance commit landed, and I landed its work by copying whole files into the
main tree. mp4/mod.rs was in both agents' file sets, so silent's copy — built on
the older base — overwrote the conformance changes wholesale. The gate stayed
green throughout, because reverting a fix and its tests together is perfectly
consistent.

Re-applied the conformance commit's diff for that file with a three-way merge;
both agents' changes to mp4/mod.rs now coexist (final_report, UndescribableAudio
and the max(track_id) id fix are all still present alongside RATE_TOLERANCE_FPS
and the colr resolver delegation).

Only mp4/mod.rs was affected. mp4/audio.rs and mkv.rs were in no other agent's
file set and were intact.

Process lesson, recorded because I would otherwise repeat it: NEVER land a
parallel agent's work by copying whole files, when its worktree was cut at an
older base than HEAD. Apply its DIFF (git apply -3), or rebase its worktree
first. Copying files silently discards anything committed to those files in the
interim, and no test can catch it because the tests disappear with the code.
2026-07-29 21:03:26 -07:00
Matthew Jackson 5f8dc392c0 Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.

The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.

Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.

Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.

The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
2026-07-29 21:00:55 -07:00
Matthew Jackson a32373ff40 Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files
another agent held this round, 0 rejected.

**A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and
1024 % 3 == 1 — verified — so every chunk after the first was misaligned against
the 6144-byte AACS aligned unit and would be REJECTED by
DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle
probe I added last round would have read almost nothing past its first chunk.
Now 1023 sectors (341 aligned units) with a const assertion that fails the build
if it stops dividing, plus set_unit_base per extent so the source's gate is
anchored where the extent actually starts.

**The same probe skipped sectors on a short read**, advancing by the REQUESTED
count rather than the bytes actually returned, so a partial read silently left a
gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n
to the buffer.

**Its cache key omitted the PGS PID set**, so a playlist declaring an extra
subtitle PID got another playlist's verdict for a track that had never been
probed. And the key was the whole extent list, so partial clip sharing missed
entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent
keying was shown SOUND rather than assumed: ForcedTracker is two monotone
booleans, so per-extent evidence composes by field-wise OR, order- and
grouping-independently. Making that honest required per-extent demux state, so an
extent's evidence comes only from its own bytes, and memoising only extents whose
read reached a designed stop.

**A reachable panic in the timeline.** mkvstream::parse_block accepts a
TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the
next passive frame panicked adding the backstep. In release it wrapped negative
instead, firing the straggler clamp for essentially every passive frame — audio
and subtitles rewritten onto the wrong point of the output timeline. All four
sites saturate.

**A public constructor divided by zero**: PrefetchedSectorSource::new_with_events
with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling.

**Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro,
samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as
prevention, because the next tracing::debug! someone adds is the leak. A doc claim
that DiscInputs "contains no secrets" was false and is corrected.

**An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB
like its writeback sibling, with the parse split out so the bound is testable
without touching process env.

**The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification.
Now file_len/16, since only vide/soun tracks are indexed and the shortest legal
AC-3 frame is 128 bytes.

**Two pipeline concurrency defects**: a consumer apply() error was invisible to the
producer, and abandon/finalise had a TOCTOU where a caller could report an
unfinalised output. Both fixed with compare-exchange state rather than a bool.

**Two per-frame copies removed**, both MEASURED rather than reasoned: the AU
assembler now hands its allocation to the frame (same pointer, unchanged capacity,
proven by asserting the pointer) and tsmux reuses one Annex-B buffer across
frames. Both keep capacity deliberately — a naive split_off would have cost more
than it saved.

**A comment pointed at the wrong file** for a mirrored constant; the mirror is now
compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift
fails the build.

Deferred to another agent's files, all confirmed: detect_rate's fractional-twin
snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic
base-key gap fill, and MkvStream's frame cap counting frames rather than bytes.

Every fix verified red by reverting it. Also noted for later:
DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext.
2026-07-29 20:47:31 -07:00
Matthew Jackson 3efa6211f3 Make six silent mux failures observable
All six confirmed against the code. The governing rule this cluster serves: a
lossy or degraded outcome is never silent, because a corrupt rip the user does
not know about is the worst failure available.

**A 3D MKV re-mux silently lost one eye.** The BlockGroup read path had arms for
BLOCK / BLOCK_DURATION / REFERENCE_BLOCK only, so BLOCK_ADDITIONS fell into the
skip arm — while the writer does emit BlockAdditions > BlockMore > BlockAdditional
for the MVC dependent view. Reconstruction was judged out of scope and the
reasoning is recorded: PesFrame has no side-payload field and the header parser
never reads BlockAdditionMapping, so there is no dependent-view track to route the
AU to. Instead the loss is now LOUD — counted in bytes and events, warned once,
and surfaced through MkvStream's errors()/lost_bytes(), which the driver already
samples into MuxOutcome. One detail in the finding was wrong and is corrected: the
re-mux does NOT still advertise the mvcC mapping, because the header parser
ignores that element, so the output is a plain 2D H.264 track.

**An all-titles rip silently skipped real titles.** The header-buffer-cap
overflow returned Error::MkvInvalid, and is_skippable_title_stub matches exactly
E_MKV_INVALID | E_CSS_KEY_MISSING — verified here — so a 512 MiB-of-frames title
was classified as an empty nav/menu PGC stub and dropped. It now has its own
E9051 / MuxHeaderBufferExceeded { bytes }, outside the skippable set.

**The public pre-mux report contradicted the file.** Mp4Sink::finish() drops an
audio track it cannot describe, which I chose last round over failing an export
whose video is fine — but mp4_fit_report still listed that stream as included, so
the application's plan and the actual output disagreed. Fixed at both levels:
Mp4SkipReason is now non_exhaustive with NoSamples and UndescribableAudio,
Mp4Sink::final_report() describes the FILE rather than the plan, and for the
boxed dyn Stream path a defaulted Stream::undelivered_streams() carries the
information out to MuxOutcome::undelivered_streams with a driver-side warn.

**MP4 track ids could collide.** ids were assigned before the retain that drops
sample-less tracks, while next_id came from the post-retain count, so [1,3]
yielded next_id 3. Now max(track_id) + 1, saturating.

**Stream selection silently skipped its codec_privates prune** when the lists were
not the same length — but codec_privates is consumed POSITIONALLY and trailing
extras are documented as benign, so the length-equality guard was itself the bug.
The prune now runs unconditionally by index.

**The m2ts_mux scaffolding armed params_written on both the absent and the
unparseable codec_private arms** — the same defect already fixed in tsmux.rs.
Split into params_attempted (a latch, since retrying identical bytes cannot help)
and params_emitted, with a warn on each failure arm and an accessor so the
eventual wiring and its test can observe it.

Each fix verified red by mutating back to the prior behaviour: errors() 0 vs 1,
E6008 vs E9051, final_report [0,1] vs [0], next_track_id 3 vs [1,3], and the
selection prune resolving index 1 to the wrong track's record.

API surface deliberately widened: MuxOutcome gains a public field and Mp4Sink
becomes public. Nothing in-repo breaks. Note a behaviour change on the mkv://
input path — a 3D re-mux now reports non-zero loss, so a consumer treating
errors > 0 as disc damage will trip on it. That is intended: the outcome IS
degraded.
2026-07-29 20:46:10 -07:00
Matthew Jackson 9ad68dd092 Fix four MP4 conformance defects against the standards
**dec3 declared a 0 kbit/s AC-3 substream.** parse_dolby routes bsid < 11 to
parse_ac3, which leaves data_rate_kbps = 0 and keeps the AC-3 bsid, yet
dolby_sample_entry wrapped that config in ec-3/dec3 for any Codec::Ac3Plus track.
ETSI TS 102 366 Annex F.4 assigns ac-3/dac3 to an AC-3 bitstream and F.6 assigns
ec-3/dec3 to an Enhanced AC-3 one, so the entry now follows the SYNCFRAME that was
actually parsed, not the playlist's codec label. Computing an AC-3 data rate and
keeping ec-3 was rejected: it fixes one field while bit_stream_identification and
num_dep_sub keep misdescribing the stream. The bsid threshold is hoisted into one
constant so parser and entry-chooser cannot drift. Adjacent defect fixed in the
same box: data_rate is 13 bits from a u16 source, so push's mask WRAPPED anything
above 8191 (9000 became 808); it now saturates.

**colr tagged HLG as PQ, and PAL as BT.601.** video_colr carried a second,
drifted copy of the ColorSpace-to-CICP map: transfer 16 (PQ) for every BT.2020
stream with no HdrFormat override, and 6 (BT.601) for Bt470bg. Per ITU-T H.273
Table 3, HLG is 18 and BT.470-6 System B/G is 5 — and mkv::cicp_for_video already
got both right. video_colr now delegates to that shared resolver, so the
duplicated table is gone and cannot drift again. It keeps only its own decision
about WHETHER to emit the box, since an absent colr and an all-unspecified colr
mean the same thing per ISO/IEC 14496-12.

**Exact 24.000 / 30.000 / 60.000 fps was declared 23.976 / 29.97 / 59.94.**
detect_rate took the FIRST STD_RATES entry within 0.5 fps, and every 1000/1001
entry precedes its integer twin 0.024 fps away — a 0.1% error across the whole
track's mdhd and stts. Fixed as nearest-wins rather than by reordering the table:
reordering fixes today's table and re-breaks the moment someone appends a rate,
while nearest-wins is order-independent. Verified red here independently by
reverting to first-match, which fails exactly the two timing tests.

**ddts MultiAssetFlag was set from has_extension.** In the DTSSpecificBox
(ETSI TS 102 114) that flag signals more than one audio ASSET. A DTS-HD MA/HRA
track is one asset whose extension substream carries the XLL/XBR component, so
setting it from "an EXSS sync follows the core" sent a parser looking for a second
asset descriptor while StreamConstruction simultaneously said there was no
extension — the box contradicting itself. This module parses the core header only
and never reads the EXSS asset table, so 0 is the only honest declaration. A
StreamConstruction index for core+EXSS was deliberately NOT invented: that field
is a table lookup that could not be confirmed against the standard, and a wrong
index is worse than an under-declaration.

DTS_AMODE_LAYOUT's masks were independently re-derived against ETSI TS 102 114
§5.3.1 and all 16 are CORRECT — only three adjacent comments were wrong (the
AMODE 2/3/4 annotations were rotated by one, and AMODE 9's said "5.1 with LFE"
when 0x0007 is the 5.0 mask and LFE is OR'd in separately). Comments corrected.

Every one of the eight new tests decodes the field back OUT of the emitted bytes —
data_rate from the dec3 body's leading 13 bits, MultiAssetFlag from bit 48 of the
ddts tail, colr from the nclx payload inside a real stsd, and the frame rate from
mdhd.timescale plus stts.sample_delta of a fully muxed MP4 — rather than
restating arithmetic. This audit has already caught one of my own tests doing the
latter.

NOT fixed, root-caused and recorded at the site instead: an A_PCM/INT/BIG track
ships with no BitDepth, which the Matroska Codec Specifications make a MUST. The
width exists on disc (BD LPCM signals it in the ES header byte 3, DVD in the IFO
audio attribute byte 1) but neither source reaches MkvTrack::audio, and the fix
needs a new AudioStream member plus a deferred setter in files another agent held
this round. Guessing 16 was rejected — it would confidently misdecode every
24-bit disc — as was refusing the track, which would regress the 16-bit majority
that currently plays by accident.
2026-07-29 20:29:17 -07:00
Matthew Jackson 13897e14f0 Resolve the forensic key map once per disc, not once per playlist
resolve_content_key_map loops every title into resolve_mux_key_map, which called
resolve_fmts_key_map FIRST — before the CpsUnitCache — and on an FMTS disc
returned immediately. So every playlist re-derived facts that belong to the DISC:
a full UDF walk plus /AACS/IndividualSegment.tbl, and on an FMTS disc the anchor
probe, the 32-index phase probe, and a fetch.fmts_indexes round trip.

On a 60-playlist disc, measured on a synthetic fixture: 840 -> 14 metadata reads,
2,400 -> 40 probe reads, and 60 -> 1 key-service calls. Worst case before was up
to 256 probe reads and 32 key-service calls per title. The 60 redundant
key-service round trips are a strong candidate for the keyserver storm seen in
the field.

Two memos behind a pub(crate) DiscKeyCache. The table memo (UDF walk + tbl parse)
is disc-invariant outright — nothing in that path mentions the title — and runs on
EVERY disc, so a plain BD benefits too. Only the deterministic negatives are
memoised as "not FMTS"; a DiscRead fault propagates uncached so a later title
retries.

A blind once-per-disc hoist of the PROBES was rejected as unsafe, and this is the
load-bearing reasoning: the title enters through clip_byte_to_lba, which decides
which segments are addressable and which LBA every probed clip byte reads from, so
two titles with different extent lists probe different physical bytes. A hoist
would serve title B an answer derived from title A's media and could silently turn
a per-title FmtsKeyMissing into a success. The extent list is the ONLY per-title
input, so keying on it is exactly sufficient — matching the ForcedProbeCache
precedent.

Result-identity was proved, not assumed: NEITHER probe reads the key pool.
Verified here independently — probe_fmts_index_keys takes no keys parameter at
all; index keys come from `fetch`, and the anchor's reply feeds the phase probe.
So the pool's growth across titles, the one thing that does change between calls,
cannot move a memoised value, and the result is order-independent. A test resolves
three titles through a shared memo and through fresh memos and asserts both the
per-title ranges and the final key pool (keys, slots, order) are identical.

Not memoised, deliberately: fail-loud FmtsKeyMissing, and any run where an index
hit a read fault — that is a property of a transient drive fault, not of the
extents, and caching it would spread one bad read across 59 playlists.

A fully-memoised title now does zero I/O, which made the old in-loop halt polls
unreachable for it, so a check_halt on entry was added with a test that cancels
after warming the memos.

Also corrects my own overstatement from last round: the CpsUnitCache doc now says
plainly that on an FMTS disc it removes NO reads, because this function returns
before the extent loop ever runs.

Five mutations, all verified red. Pre-existing bug flagged but not fixed:
filter_addressable_segments only checks that a segment's START byte maps to some
LBA in the title, so a play-all playlist can pass the filter while mapping segment
bytes into the wrong clip, whose anchor then returns empty and aborts the sweep.
2026-07-29 20:27:50 -07:00
Matthew Jackson b4bf0daa82 Group E-AC-3 dependent substreams into one access unit
The AC-3 parser's own module doc stated the assumption: "AC3 frames are
self-contained and always start with syncword 0x0B77". True for legacy AC-3,
false for E-AC-3 above 5.1. Per ETSI TS 102 366 (A/52) Annex E, byte 2 of an
E-AC-3 syncframe is strmtyp(2) | substreamid(3) | frmsiz[10:8], and an access
unit is one INDEPENDENT substream plus every DEPENDENT substream that follows it
until the next independent one. The parser emitted one PES frame per syncframe,
so a decoder saw each dependent substream as a standalone frame with no parent —
including the AC-3-core + E-AC-3-dependent form Blu-ray uses for Dolby Digital
Plus. The extra channels were lost and the timeline ran at 2x.

The bit position is cross-checked against code already in the tree: the existing
frmsiz parse takes byte2 & 0x07 as its high bits, which is only consistent with
strmtyp occupying byte2's top two bits. Legacy AC-3 is excluded by bsid < 11,
where byte 2 is crc1 and reading strmtyp there would be nonsense. Reserved
strmtyp 3 is treated as INDEPENDENT so an unknown type starts a fresh AU rather
than merging into an unrelated one.

The AU carries the INDEPENDENT substream's PTS, and only the independent
substream advances the clock — dependents cover the same time period and add zero
duration. That is what removes the doubled timeline.

A trailing AU that can still grow is HELD across the PES boundary, because the
boundary is unknowable until the next independent sync; the whole AU is re-scanned
next call, so there is no shift and no double-count in the loss tally. Plain AC-3
is never held, which keeps DVD/AC-3 latency and behaviour unchanged.

A latent pre-existing bug surfaced while testing this: a new PES's PTS was
re-stamping an AU that began in an earlier PES, a constant one-frame shift. Fixed
with a PtsAnchor so a PES timestamp applies to the first AU that STARTS in that
PES's own bytes, while a genuine PTS jump is still adopted.

Nine tests. Verified red against five mutations, each killing a specific set:
reverting to the pre-fix behaviour kills 8 while
plain_ac3_frames_are_not_grouped_or_delayed SURVIVES as the no-regression guard —
reproduced independently here. Stamping the dependent's PTS kills 6; not holding
across PES kills 4; holding plain AC-3 too kills 15; neutering the PTS anchor
kills exactly the 2 split-across-PES timing tests.

Three sibling defects found and deliberately NOT fixed, all in mp4/audio.rs:
dec3 hardcodes num_dep_sub = 0 (and a nonzero value changes the box LAYOUT, not
just a field, per Annex F/G); parse_eac3 ignores strmtyp/substreamid entirely;
and a 7.1 DD+ track is still labelled 5.1 because the channel count comes from
the independent substream while the extra channels are described by the
dependent's chanmap, which nothing parses.

Not verified: no real E-AC-3-with-dependents sample exists here, so all evidence
is synthetic frames plus the spec layout. The multi-independent-substream case
(num_ind_sub > 1, main + associated audio in one PID) is deliberately treated as
one AU per independent substream and is untested.
2026-07-29 20:26:51 -07:00
Matthew Jackson ef36b452ad Fix three defects in last round's own fixes
Round 3 audited the round-1/2 fix commits rather than trusting them, and found
three defects in that new code. This is why the pin moves each round.

1. LICENCE REGRESSION, and it was mine. Reverting the "distinguish a failed key
   source" commit also restored a verbatim reference-decoder table citation in
   src/mux/codec/dts.rs, because both changes were in that one commit. The MIT
   licence cleanup was silently undone at HEAD and nothing caught it.

   The citation is replaced with ETSI TS 102 114 §5.3.1 again, and — more
   importantly — the rule now lives in the leak gate instead of in my memory.
   scan-secrets.sh gains LICENCE_RE, which flags ff_dca*, dcadec, l-smash,
   libav*, and bare ffmpeg/FFmpeg as REF-IMPL-CITATION. `no ffmpeg` is
   explicitly allowed via negative lookbehind: stating what this project does
   NOT depend on carries no risk and is a genuine selling point. Verified by
   re-introducing the citation (gate fails) and removing it (gate clean).

2. TsMuxer armed params_written even when the avcC/hvcC parser returned None, so
   a track whose codec_private exists but will not parse was muxed to BD-TS with
   no VPS/SPS/PPS ever emitted — undecodable video, reported as success, with no
   log line. Last round's fix corrected WHICH parser is used and left this half
   untouched. Arming the flag is still right (retrying identical bytes cannot
   succeed) but it is no longer silent: it now warns with the track, codec and
   codec_private length.

3. test_aes_cbc_roundtrip defined a LOCAL fn aes_cbc_encrypt that SHADOWED the
   production primitive, so it round-tripped a copy of the algorithm against
   itself and never touched crypto::aes_cbc_encrypt — the function this cycle
   added. Any mutation to the shipped code passed it. The shadow is deleted and
   the test now calls the real primitive; verified by mutating
   crypto::aes_cbc_encrypt, which now fails it and previously would not have.
2026-07-29 20:08:09 -07:00
Matthew Jackson f338552969 Pin the toolchain to Rust 1.87
The Windows UI needs winsafe, whose current release requires rustc 1.87. The
alternative was pinning winsafe back to an older release, which would bake a
stale API surface into a brand-new UI permanently to dodge one minor version.

The pin's purpose is to sit BELOW the Mac default so clippy drift is caught
locally before CI, not to stay on 1.86 specifically, so 1.87 preserves the
discipline exactly.

Verified before moving anything, not after: `cargo +1.87 clippy -- -D warnings`
and `cargo +1.87 fmt --check` are clean across all eight repos, and the full
precommit gate (fmt + clippy + tests) passes on libfreemkv, autorip, bdemu,
freemkv-engine and freemkv-keysources. Zero new lints, zero formatting drift.
2026-07-29 20:02:02 -07:00
Matthew Jackson 38aa895038 Memoise multi-CPS key-map sampling per extent, not per title
resolve_content_key_map calls resolve_mux_key_map once per title, and on a
multi-CPS disc that path issues 8 random single-unit reads per extent. A disc's
playlists overwhelmingly reference the same few clips — main feature, play-all,
per-chapter and seamless-branch variants — so the same physical extents were
re-sampled from the drive once per playlist. On a 60-playlist / 15-clip disc
that is ~2,400 non-sequential 6144-byte reads, roughly 8 minutes of pure seeking
at 200 ms per seek, before the mux starts. Now ~600 reads.

Keyed per EXTENT — (format, start_lba, sector_count) — rather than per title's
whole extent list, which is finer-grained than the forced-subtitle probe's cache
and strictly better here: a play-all playlist sharing 4 of 5 extents with the
main feature still hits on those 4.

Why a cached pool index is provably identical to a recomputed one, verified
rather than assumed:

  * `pick` iterates the pool IN ORDER and returns the FIRST index whose key
    decrypts a sample to clean.
  * The pool is APPEND-ONLY. Checked across the whole crate: only `push`, with no
    insert/remove/clear/retain/sort/dedup/truncate/drain/swap/reverse anywhere.
    So appended keys can only land AFTER a matched index, and the first match for
    the same samples cannot shift.
  * The samples are a pure function of the three values in the key, read from
    read-only optical media.

Two outcomes are deliberately NOT cached, which is what makes this safe rather
than merely faster:

  * the inherited index (`None if samples.is_empty() => last_idx`) is per-TITLE
    state, not a property of the extent — caching it would let one title's
    carry-in index leak into another title's clear extent, i.e. a WRONG key;
  * the fail-loud DecryptFailed verdict, so a retry after a key source banks the
    missing key re-samples instead of inheriting a stale answer.

Halt is still polled before the cache lookup, so cancellation is unchanged.
resolve_mux_key_map keeps its exact signature and delegates with a fresh cache,
so there is no public API change. ContentFormat gains Eq + Hash (additive).

Four tests, and the two mutants that matter both verified red: disabling the
cache short-circuit fails the hit and recompute-equivalence tests, and wrongly
caching the inherited index fails multi_cps_inherited_index_is_not_cached.
2026-07-29 19:49:55 -07:00
Matthew Jackson e3676e7cdf Build BlockGroups in memory so the hot path never seeks
Every BlockGroup frame back-patched its element size via ebml::end_master, which
does two stream_position() calls and two real seeks. BufWriter does not override
Seek::stream_position, so each position query is seek(Current(0)) = flush_buf +
lseek — and each of those flushed the 4 MiB BufWriter while every position-moving
seek reset WritebackPipeline::last_flush_pos. The buffer never got to do its job.

An MPEG-2 title takes this path for EVERY frame (the parser stamps a per-frame
duration, so I, P and B all become BlockGroups) — roughly 350,000 per feature.

A BlockGroup's size is knowable before writing, so there is no need to back-patch
at all. New seek-free twins start_master_buf / end_master_buf patch a placeholder
by buffer INDEX instead of file offset, and build_block_group assembles the whole
element into a persistent buffer that write_block_group and its MVC sibling take,
fill, write once, and hand back — including on the error path — so the allocation
is made once rather than per frame.

Measured with a counting writer that, like BufWriter, does not override
stream_position, over 200 frames:

              seek calls   position-moving
  plain   before 912              451
  plain   after  112               51
  MVC     before 2516            1253
  MVC     after  116               53

Per-frame cost is now zero; the residual is the header, the per-cluster
back-patch and Cues. For 350k BlockGroups that is 1.4M seek calls removed on the
plain path, 4.2M on an MVC title.

end_master is deliberately NOT changed for its other callers. The Cluster master
genuinely streams — frames are appended to an open cluster over time, so its body
cannot be buffered without holding a whole cluster in memory — and the rest
(EBML header, Tracks, Info, Chapters, Cues) run once, not per frame.

Byte-identity is the safety property, and it is structural: end_master always
patches a FIXED-width 8-byte VINT, and the buffered pair writes and patches
exactly that same placeholder, so the encodings cannot differ. Verified by
capture-then-compare over 200 frames (17 keyframes / 183 non-keyframes, multiple
clusters, BlockDuration present and absent, both reference branches) — output
byte-for-byte identical.

Three tests now pin it permanently: buffered vs seeking output byte-for-byte for
empty/tiny/multi-byte bodies, the same for NESTED masters (the MVC path nests
BlockAdditions > BlockMore, where an index-arithmetic slip would surface), and
end_master_buf erroring rather than panicking on a position outside the buffer.
All three verified red against a deliberately divergent placeholder width.

A note on verifying this kind of change: comparing emitted MKV across two
worktrees at different commits shows a spurious 14-byte difference, because
MuxingApp/WritingApp embed the build's git SHA twice. Compare at the same base.
2026-07-29 19:33:31 -07:00
Matthew Jackson 807eb053ca Only assert a forced-subtitle verdict the read actually supports
probe_and_set_forced broke out of its read loop on a read error but still
applied whatever partial observation it had accumulated as an authoritative
verdict, overwriting the vendor-label-derived forced flag. A disc that faults
early could have a correct flag replaced by a guess from a fraction of the data.

The file already got the zero-observation case right — it deliberately leaves
the vendor flag alone rather than "assert not-forced from having seen nothing".
The defect was that a PARTIAL observation cut short by a fault was treated as
complete.

The fix rests on the two verdicts not being symmetric evidence.
settled_not_forced() is POSITIVE evidence — a non-forced display set was
actually seen, and no unread data can retract it. is_forced() is an ABSENCE
claim — display sets were seen and none was non-forced — which is only sound if
the read got far enough for the absence to mean something.

So every loop exit now yields a named StopReason, and absence claims are
asserted only for a designed stop:

  * Exhausted / Budget → conclusive. The budget is deliberately conclusive: the
    natural exit is "every track settled not-forced", which a genuinely forced
    track never satisfies, so the budget is the ONLY path by which a real forced
    verdict is ever reached. Treating it as inconclusive would disable forced
    detection entirely.
  * Halted / ReadFailed → inconclusive. A cancelled probe's cut-off point is as
    arbitrary as a faulted one, so its absence claim is worth no more.

Evaluated per track, matching the existing observed() gate: a track that already
saw a non-forced set keeps its sound verdict even on a truncated run, while a
forced-so-far sibling keeps the vendor flag.

An inconclusive run is also NOT memoised. The cache key is the extent list and a
disc's playlists share clips, so caching a truncated run would replay one read
fault onto every playlist referencing those extents and deny any later title the
chance to re-read them.

Four tests; three of them verified red by forcing absence_is_conclusive() back
to always-true (the old semantics), while the budget test correctly stays green
either way — confirming the guard was not over-corrected.

Also moves the function doc comment back onto probe_and_set_forced; the earlier
probe commit left it attached to the ForcedProbeCache type alias.
2026-07-29 19:26:58 -07:00
Matthew Jackson 7322f4dd8a Record that the failed-vs-absent key source arm is unreachable
The `Ok(_) | Err(_)` arm in resolve_and_apply_traced conflates "this source
had no entry for the disc" with "this source failed", and reports both as
KeyNode::NoEntry. An operator whose key server is returning 502s is therefore
told their disc is not in the database.

The conflation is real but LATENT, and fixing it here would change nothing an
operator can see, because no shipped KeySource ever returns Err:
KeydbSource::get_unit_keys maps a load/parse failure to Ok(Vec::new()),
OnlineSource::get_unit_keys is Ok(self.query(ctx)) where query returns empty on
transport error, HTTP status, oversize body and bad JSON alike, and MultiSource
discards inner Errs. Only test doubles return Err. FetchOutcome::errored in
drive_unit_keys / drive_fmts_indexes is dead for the same reason — the right
contract, honoured by no source.

autorip already works around the missing signal by re-probing the service over
HTTP (probe_online_reachability / key_service_transient_status), and its own
comment names the incident: "the online keysource swallows every failure
(transport error, 502, timeout)".

So the fix belongs at the source boundary in freemkv-keysources, with
Disc::aacs_error as the channel the operator actually reads — not in this
trace. Documented here so the next reader does not assume the arm works, and
does not "fix" a dead path as I nearly did twice.
2026-07-29 19:24:58 -07:00
Matthew Jackson 4ed245868e Revert "Distinguish a failed key source from one with no entry"
This reverts commit 22a3e3fd01.
2026-07-29 19:15:21 -07:00
Matthew Jackson 50f37462db Remove reference-decoder citations from a public MIT-licensed repo
This crate is MIT licensed. Comments citing another decoder's internal symbols
and reproducing its tables verbatim create licence risk that no engineering
benefit justifies, so every such citation is replaced with the primary source:
ETSI TS 102 114 §5.3.1.

Eleven sites across src/mux/codec/dts.rs and src/mux/mp4/audio.rs. The technical
substance is unchanged in every case — the deficit-sample-count semantics, the
reserved-field skips, the invalid LFF value, and the 16 legal AMODE codes are all
spec facts and are now attributed as such. Four CHANGELOG entries that named a
validator are reworded; the "no a reference decoder" dependency claim stays, since stating
what this project does NOT depend on carries no risk.

I initially argued this was a false positive on the grounds that the project's
hygiene rules name internal infrastructure and reverse-engineering material, not
open-source citations, and that a channel-count table from a standard is fact
rather than expression. That reasoning missed the point: the exposure is MIT
distributing text derived from GPL/LGPL sources, and that is the maintainer's
risk to weigh, not mine. Reversed in full.
2026-07-29 19:09:10 -07:00
Matthew Jackson 22a3e3fd01 Distinguish a failed key source from one with no entry
resolve_and_apply_traced collapsed `Ok(_) | Err(_)` into a single
KeyNode::NoEntry step, so a key source that FAILED — server unreachable, keydb
unreadable, malformed entry — was recorded identically to one that simply had no
entry for this disc. The front-end renders that trace, so it told the operator
their disc is not in the database when the real cause was a fixable
infrastructure problem. drive_unit_keys and drive_fmts_indexes were refactored
this cycle to preserve exactly this distinction; this path had not been.

KeyNode gains a SourceFailed variant and the two arms are split. freemkv's
trace renderer matches KeyNode exhaustively with no catch-all, so its arm is
added in the same change — otherwise the consumer would not build.

Also made ETSI TS 102 114 the primary authority for DTS_AMODE_COUNT's comment
rather than a reference decoder internal symbol, and pointed it at this
crate's own cross-checked DTS_AMODE_LAYOUT / DTS_AMODE_CH tables.

A round-2 finding asked for every a reference decoder and a reference decoder citation in the DTS parser
to be stripped as a public-repo hygiene violation. Rejected: the project's rules
(scan-secrets.sh, CLAUDE.md) prohibit internal infrastructure references and
reverse-engineering material, and a reference-decoder citation is neither. The
AMODE channel-count table is a factual table from the standard, not expression
copied from an implementation. Citing the spec plus a corroborating
implementation is how a decodability gate should be justified.
2026-07-29 19:07:15 -07:00
Matthew Jackson 99c5fd3500 Reference keyframes per track, size the DTS reserve, correct two claims
The ReferenceBlock offset was computed for ANY video track, but the keyframe tick
it measures against was recorded in a single global slot gated to the PRIMARY
video track. On a title with two video tracks — an MVC base plus secondary view,
or a multi-angle disc — a secondary track's non-keyframe therefore referenced a
keyframe on a different track, or 0 (a self-reference) when the primary had not
produced one yet. The tick is now recorded per track, so a non-keyframe can only
reference a keyframe on its own track.

The faststart moov-hole estimate modelled every audio track as (E-)AC-3 at 1536
samples per frame. 1.6.0 added DTS to the writer's carried set, and a DTS core AU
is commonly 512 samples — a third of that — so a DTS track's sample table was
under-reserved threefold and the mux fell back to moov-at-end, losing faststart
on exactly the files 1.6.0 newly supports.

mvc_frame_emits_blockgroup_additional_and_reference asserted only that the
non-keyframe's ReferenceBlock was Some(_). Its non-MVC sibling, added in the same
commit, pins the exact offset; this one now does too, so a mutant emitting a
constant or wrong-signed offset no longer passes.

The comment above the mp4 sample budget claimed file_len stops a crafted file
inflating allocations "past the file's own size". Each indexed sample costs ~52
bytes, so the real ceiling is ~52x file_len (still capped by MAX_SAMPLE_COUNT).
The bound is real; the comment overstated how tight it is.
2026-07-29 19:03:59 -07:00
Matthew Jackson d4c913e0d3 Validate stream-selection PIDs per class, not across both
StreamSelection::apply validated a listed PID by scanning ALL streams, so a PID
named in the wrong class's filter passed validation — an audio filter listing a
subtitle PID, say. `keeps` then matched it against the audio streams only, so
the requested track was silently absent from the output. That is precisely the
outcome this validation documents itself as preventing: "fail loud rather than
silently emit an MKV missing a requested track".

Each filter is now checked against its own stream class. The `listed_pids`
helper existed only for the cross-class scan and is removed rather than left
behind as dead code.

Test covers both directions plus the sanity case, and asserts a rejected
selection leaves the title unpruned.
2026-07-29 19:00:53 -07:00
Matthew Jackson 0e23a6b291 Halve the per-frame allocation and copy on the m2ts NAL video path
The NAL path called length_prefixed_to_annex_b, which allocates a whole-frame
Vec of its own, then copied the result into a second whole-frame Vec — two
full-frame allocations and two full-frame copies per video frame. The crate
already has append_length_prefixed_as_annex_b, which writes the conversion
straight into a destination buffer; it is the same code path with the
intermediate removed.

The destination is also sized once up front instead of starting from Vec::new(),
which re-grew from zero capacity inside every conversion.

On a UHD HEVC title muxed to m2ts:// — ~200k video frames averaging ~310 KB of
ES at 60 Mb/s — that removes roughly 62 GB of allocation and 62 GB of memcpy.

Behaviour is unchanged: the existing tsmux conversion tests, including the
non-NAL passthrough and Annex-B default pair added last round, all still pass.

A first attempt reused a persistent scratch buffer across frames, which does not
work: the buffer is handed out as Cow::Owned and so can never be returned. A
right-sized single allocation gets most of the win without restructuring the
function around the borrow.
2026-07-29 18:58:39 -07:00
Matthew Jackson bcf47cc4ca Fix ddts numeric truncation and the decrypt pool's poison asymmetry
ddts CoreSize wrapped to zero on a maximum-size core. core_size is FSIZE + 1 and
FSIZE is itself 14 bits, so the maximum is 16384 — one past what the 14-bit
CoreSize field holds — and push()'s mask turned that into 0, declaring an empty
core frame. Clamped to 16383 instead: one byte short beats telling a decoder
there is no core. Proven red first (the field read back as 0).

ddts avg/max bitrate under-declared every non-integral frame rate. It computed
sample_rate / frame_samples first, so a 512-sample core at 48 kHz truncated
93.75 frames/s to 93. Multiplying before dividing, with round-to-nearest, keeps
the precision.

set_decrypt_threads skipped the pool swap on a poisoned lock while
DECRYPT_THREADS had already been updated, so the new thread count was reported
as taking effect while the stale pool kept serving. decrypt_pool() deliberately
recovers from poisoning for exactly this reason; the setter now does the same.
The pool Arc is immutable once stored, so a prior panic cannot have left it
half-written.

The CoreSize test decodes the value back out of the emitted box rather than
restating the clamp — a first draft asserted the clamp arithmetic against
itself, which would have passed against the unfixed writer.
2026-07-29 18:56:48 -07:00
Matthew Jackson a9dc3d7244 Make encrypt_unit report a refused slice, and expand its key once
Two defects in the encrypt_unit promoted to public API last round, both found by
round 2 auditing that new code.

It returned silently without encrypting when the slice was shorter than
ALIGNED_UNIT_LEN. Its own contract requires the caller to set the container's
encrypted flag BEFORE calling — the header is the key seed — so a silent no-op
leaves a unit advertised as encrypted while still carrying plaintext, with
nothing for an authoring caller to check. It now returns bool and is
#[must_use], so ignoring the refusal is a compile-time warning; every call site
was updated to assert on it.

bool rather than Result deliberately: a wrong buffer length is a programming
error at a library boundary, not a disc condition, and a new Error variant would
mean a new numeric code plus its rendering in another repo.

It also drove CBC from the single-block aes_ecb_encrypt, rebuilding the AES key
schedule for each of the 383 blocks in a unit — an order of magnitude slower
than its inverse, which expands the key once via aes_cbc_decrypt. The missing
counterpart aes_cbc_encrypt now exists alongside it, and encrypt_unit calls it,
so the two directions are symmetric in structure as well as in result. For an
authoring caller encrypting a 90 GB image that removes ~5.6 billion redundant
key expansions.

New test pins the boundary: ALIGNED_UNIT_LEN - 1 returns false and leaves the
buffer byte-identical, ALIGNED_UNIT_LEN succeeds. The existing round-trip and
padding-asymmetry tests still pass, so the CBC rewrite is provably the same
transform.
2026-07-29 18:54:19 -07:00
Matthew Jackson 94a876664b Correct six stale comments and doc claims
All six describe code that does something different from what they say, which
is the class of defect that gets a maintainer to write a bug on purpose.

docs/clpi.md presented the CLPI stream-PID entry as byte-aligned 2/2/2/4/4-byte
fields with a 32-bit fine-entry count. It is one 80-bit packed block —
reserved(10) + EP_stream_type(4) + num_EP_coarse(16) + num_EP_fine(18) +
EP_map_start_address(32) — and num_EP_fine is 18 bits. Anyone parsing to the
doc's offsets would read garbage. Replaced with the real bit layout.

docs/udf.md said read_directory()'s recursion cap is 3; MAX_DIR_DEPTH is 8.

TROUBLESHOOTING.md called Pass 1 `recovery::copy`. The engine's `sweep` is
documented as "Pass 1 of a multipass rip"; `copy` is the dispatch verb that
chooses between sweep and patch. This inconsistency was mine, introduced in the
1.6.0 doc rewrite. docs/drive-access.md already said `sweep` and was right — a
round-2 finding claimed the opposite on the grounds that `recovery::sweep`
appears nowhere else in this crate, which it cannot, being in another crate.

io/pipeline.rs cited `disc::patch` as WRITE_THROUGH_DEPTH's caller; that moved
to freemkv-engine in 1.6.0 and no `patch` exists here.

truehd.rs's doc on mlp_major_sync_crc_ok said the trailer is compared
big-endian while the body compares u16::from_le_bytes — and a big-endian
compare was the bug the function was fixed for, so the comment described the
defect rather than the code.

sector/decrypting.rs claimed the decorator owns "the only mutable state (its
call-count cap and spent flag)". DecryptingSectorSource has no such fields and
no KeyFetch field at all in this revision.
2026-07-29 18:50:53 -07:00
Matthew Jackson 7030de4ec9 Apply stream selection on the live path, and treat a halt as a clean stop
Three defects around MuxOptions in the mux driver.

MuxInput::Live never applied MuxOptions.selection, so a caller's audio/subtitle
selection was silently ignored on the live-drive path while the field's own
documentation said it was applied before the demux pipeline is built. The Iso
and Session arms both apply it; Live now does the same, in the same place —
before resolve_inline_base_map, which is keyed on extents and so unaffected by
pruning the stream list.

The header gate returned Error::MkvInvalid whenever headers had not resolved.
On the prefetch-highway path a halt landing while the pump is blocked in a read
can end the stream as Ok(None) rather than Err(Halted), so the loop breaks with
headers unresolved through no fault of the data. Reporting that as MkvInvalid
tells the consumer its disc is malformed and skips the stop-preserves-staging
path that a clean completed=false triggers. The gate now re-checks halt first.

MuxOptions.selection's doc claimed it was applied without naming the one input
it is not applied to. That exception lived only in an internal comment at the
Url match arm, where a caller reading the public field docs would never see it.
It is now on the field, pointing at InputOptions::selection instead.
2026-07-29 18:48:51 -07:00
Matthew Jackson c0434e87de Fix the DVD MPEG-audio codec mapping and the fabricated AACS docs
parse_audio_attr mapped DVD audio_coding_mode 2 to Codec::Mpeg1 — the MPEG-1
VIDEO variant. Codec::kind() reports Video for it, so a DVD MPEG-audio stream
was classified and handled as video everywhere downstream. Modes 2 and 3 are
both MPEG audio Layer II (3 adds the MPEG-2 multichannel extension), so both
map to Codec::Mp2. A test now walks every coding mode and asserts each result's
kind() is Audio, so no mode can map to a non-audio codec again.

docs/aacs.md documented an entire keydb-resolving API that does not exist:
ScanOptions::with_keydb, Disc::open_title, reader.read_unit(). None of those
symbols appear anywhere in the crate, and ScanOptions has no keydb field — its
own doc comment says "libfreemkv is lookup-free — it resolves no keys". A
reader following that page would conclude the library reads keydb.cfg, which
inverts the actual design: the caller resolves keys out-of-band through a
KeySource and applies them with Disc::decrypt_with.

The section is rewritten against the real API, and the AacsState table's
`key_source` type corrected from KeySource to KeyOrigin.

Worth recording: the first replacement example I wrote was itself wrong. It
used `input("disc://...")`, which resolve.rs explicitly rejects with
Error::DiscUrlNotDirect — live disc must go through Drive::open + Disc::scan +
DiscStream::new. Every symbol and signature in the committed example was
checked against the source rather than assumed.
2026-07-29 18:47:06 -07:00
Matthew Jackson ec5cd31ae1 Stop Mp4Sink losing audio frames and writing an empty sample entry
Mp4Sink::write returned Ok(()) without recording the sample whenever an audio
track's frame would not parse into a sample entry. Two consequences, both
silent: leading audio frames were lost until one frame parsed, and a track
whose frames never parsed disappeared from the output entirely — finish()'s
retain() removed the sample-less trak and the run reported success. That
contradicts this crate's stated policy that a skipped track is never silently
dropped.

The drop was never necessary. audio_entry is read in exactly one place,
build_trak, reached only from build_moov inside finish() — nothing on the write
path consumes it. So write() now records every sample and derives the entry
opportunistically from whichever frame parses first.

That makes build_trak's `audio_entry.unwrap_or_default()` reachable, which
would emit an stsd declaring entry_count=1 around an EMPTY sample entry: a
structurally invalid mp4 returned as success. finish() therefore drops any
audio track it cannot describe, with a tracing::warn! naming the codec and
sample count.

Dropping rather than erroring is deliberate. It matches finish()'s existing
treatment of sample-less tracks, keeps an export whose video is fine from
failing outright, and needs no new error code — a new code would mean a new
i18n key across 29 locale files in another repo, which is not this change's
scope. The track's bytes stay unreferenced in mdat: wasted space in a valid
file, which is the cheaper failure.

Test pins both halves — moov describes only the video track, and the
unparseable audio bytes still reach mdat rather than being discarded at write
time.
2026-07-29 18:44:49 -07:00
Matthew Jackson a1304f9e78 Stop the mp4 demuxer dropping tracks silently or inventing sample offsets
Three defects in the mp4:// read path, all of the same family: a damaged
source was remuxed minus a track, or with fabricated data, and the run
reported success.

Silent drops. Eight paths dropped a whole track on malformed input with no
report of any kind, so an mp4:// source missing its audio looked like a clean
run. Each now emits a tracing::warn! naming the track and the missing or
inconsistent table (tracing English is permitted in this crate; the numeric
error codes are unchanged). The non-A/V handler case is debug!, since skipping
a timecode or hint track is normal.

Fabricated offsets. sample_offsets ended with a `while offsets.len() <
sizes.len()` loop that packed unplaced samples after the last known offset.
Those samples have no known location, so the invented offsets made the reader
pull frame data from arbitrary file bytes — the exact "emit garbage" outcome
the stco/stsc presence guards refuse. It now returns the short list and the
caller drops the track.

Short stts. `durations.get(i).unwrap_or(0)` gave every sample past the end of a
short stts a duration of 0, collapsing the whole tail onto one timestamp. That
is the same degenerate timing the `durations.is_empty()` guard was written to
refuse, so the guard now refuses both cases.

Two shared test fixtures were internally inconsistent and only passed because
the reader was lenient: stsz declared 3 samples while stsc placed 1, and the
hostile-stsz fixture's stsc/stts covered a single sample. Both are now
consistent. The hostile fixture keeps its lying stsz count — that lie is what
it tests — but its stsc and stts now cover whatever count survives the
file_len bound, so it exercises the allocation bound rather than the
inconsistency guards.

Two new tests pin the new refusals by mutating the consistent fixture: an stsc
that places 1 of 3 samples, and an stts that covers 1 of 3.
2026-07-29 18:42:38 -07:00
Matthew Jackson a39045adf1 Bound the forced-subtitle probe, make it cancellable, and stop re-reading clips
The probe's only natural exit was "every PGS track has shown a non-forced
display set". A genuinely FORCED track never satisfies that, so on the common
authoring — a forced-narrative track for foreign dialogue — the loop read the
title's entire extent set at 2 MiB per call with no byte cap, no time cap and
no halt check. It was also invoked once per title rather than once per distinct
clip, and a disc's playlists overwhelmingly reference the same few clips (main
feature, play-all, seamless-branch variants), so the same physical extents were
re-read 30-150 times. The two defects multiplied: tens of GB, times the
playlist count, off an optical drive.

Reached via ScanOptions::probe_forced_subtitles, whose only consumer is
`freemkv info -v` (freemkv/src/disc_info.rs). The rip path leaves it off. So
the symptom is `info -v` never returning on an ordinary UHD, not a corrupt rip.

Three changes:

  * PROBE_BUDGET_SECTORS caps a probe at 256 MiB. A forced track's display sets
    appear throughout the title, so a bounded prefix classifies it; the budget
    only decides how long we keep looking for a non-forced set before accepting
    the forced verdict.
  * ScanOptions::halt is now plumbed in and checked per chunk, so `info -v` is
    cancellable. The probe previously took no halt at all.
  * A ForcedProbeCache memoises verdicts against the title's exact extent list.
    Keying on byte-identical input means a hit cannot change any result — it
    only removes the re-read.

Verdict application is factored into apply_verdicts so the cached and freshly
probed paths cannot diverge.

Three tests pin the behaviour, each against a reader that counts sectors and
never ends: the budget stops at exactly PROBE_BUDGET_SECTORS, a cancelled halt
reads zero sectors, and a second title with identical extents costs no further
reads while a different extent list still misses the cache.

Also worth recording: the CLI acceptance harness never exercised `info -v`
against an optical drive — it reads ISOs from local SSD, where a full-extent
read is fast enough to hide both defects.
2026-07-29 18:38:25 -07:00
Matthew Jackson ea72e6df5f Give every audio and video codec its own registered Matroska CodecID
MkvTrack::audio's catch-all was `_ => CODEC_AC3`, and ebml.rs defined no
A_AAC, A_MPEG/L2, A_MPEG/L3, A_FLAC or A_OPUS constant at all. A CodecID
names the payload, so any of those codecs was written into the MKV declaring
AC-3 while carrying something else — a player either refuses the track or
decodes noise.

Reachable through two ordinary paths, both verified: ifo.rs:577 maps DVD
audio_coding_mode 3 to Codec::Mp2, so a DVD with MPEG audio muxed to mkv://
produced a track declaring A_AC3 over MP2 bytes; and mp4/read.rs:479 maps the
`mp4a` sample entry to Codec::Aac for an mp4:// source. codec/mod.rs already
has working parsers for MP2, MP3, AAC, FLAC and Opus, so the pipeline carried
these codecs end to end and only the container label was wrong.

MkvTrack::video had the same shape: `_ => CODEC_MPEG2` announced Codec::Mpeg1
and Codec::Av1 as MPEG-2 video. V_MPEG1 and V_AV1 added.

Both catch-alls stay, because MkvTrack::audio/video return Self and have no
error channel, but they are now reachable only by a non-audio/non-video or
Unknown codec routed there in error. Two tests enumerate every real codec of
each kind and cross-check each against Codec::kind(), so a codec added to the
enum later cannot silently inherit another codec's ID.

Both proven red first: Aac declared A_AC3 before the fix.
2026-07-29 18:34:53 -07:00
Matthew Jackson b2c7490b5b Parse H.264 parameter sets with the avcC parser, not the hvcC one
TsMuxer::write_frame handed every NAL video track's codec_private to
hvcc_to_annex_b. An H.264 track carries an avcC record, whose box layout is
different, so the parser returned None and no parameter sets were emitted —
and params_written was set unconditionally, so it never retried. H.264
muxed to m2ts:// reached the player with no SPS/PPS and was undecodable,
silently: frame_count still advanced and the mux reported success. AVC is
the dominant Blu-ray video codec, so this was not an edge case.

The correct dispatch already existed and was already used by
demux_sink::annexb_param_sets. tsmux simply never got the codec: it knew
only PIDs and a NAL-or-not bool, so it could not tell hvcC from avcC.

Rather than add a second setter, set_nal_video(track, bool) becomes
set_video_codec(track, Codec). One fact decides both the ES framing and the
parameter-set parser, so the two can no longer disagree — and that
disagreement is precisely this defect. The default stays Codec::Hevc, which
is the behaviour the bool's `true` default encoded, so a caller that never
calls it is unaffected.

Proven red first, end-to-end through M2tsStream::create with a real avcC
record: before the fix neither the SPS nor the PPS reached the transport
stream.
2026-07-29 18:32:21 -07:00
Matthew Jackson 3db4106253 Stop documenting the recovery API that 1.6.0 deleted
Disc::sweep, Disc::patch, Disc::copy, SweepOptions and PatchOptions have
zero occurrences in src/ — recovery moved to freemkv-engine — but they were
still documented in 30 places across README.md, TROUBLESHOOTING.md, six
files under docs/, seven src/ doc comments and a Cargo.toml comment.
README.md is the crate's GitHub front page and carried a full multi-pass
code example that cannot compile.

Two of the src/ references were intra-doc LINKS to deleted items
([`disc::Disc::copy`], [`disc::Disc::patch`] in scsi/mod.rs). They produced
no warning on a normal `cargo doc` only because they sit on pub(crate)
items; `--document-private-items` reports both, and they are gone now.

The README example is deleted rather than rewritten against the engine's
API: libfreemkv documenting a downstream crate's API on its own front page
is the drift that produced this, and it cannot even depend on it. The src/
references become plain code spans naming freemkv_engine::recovery::* —
deliberately not links, for the same reason.

docs/rip-recovery.md was 202 lines about relocated code. It now documents
only what this crate owns — Drive::read, SenseFamily, DiscStream's adaptive
batch halving — plus the read-path design constraints, which belong with the
code that enforces them, and points at freemkv-engine/src/recovery/ for the
strategy. api-design.md's module tree is regenerated from the real src/disc/
and src/drive/ layouts instead of hand-patched; it had listed sweep.rs,
patch.rs, mapfile.rs and read_error.rs, none of which exist.

Three stale facts surfaced while rewriting and are corrected: the read
timeouts are 10 s / 60 s, not the documented 1.5 s / 30 s; Drive::reset and
SgIoTransport::reset no longer exist at all, so "no SCSI reset from any read
path" is now stated as the stronger fact it has become; and verify_title,
listed as a progress-emitting operation, was removed entirely.

CHANGELOG.md keeps its references — those are the historical record of the
releases that shipped the API.
2026-07-29 18:04:51 -07:00
Matthew Jackson d34979ac57 Assert ReferenceBlock per block instead of scanning for the byte 0xFB
Two tests checked only find_id(&data, ebml::REFERENCE_BLOCK).is_some().
ReferenceBlock's EBML ID is the single byte 0xFB, so that asks whether one
exists ANYWHERE in the output — not which blocks carry one. Inside a
BlockGroup, keyframe-ness is signalled by the ABSENCE of ReferenceBlock, so
"which" is the entire question.

Measured rather than assumed. A mutant emitting a ReferenceBlock on every
BlockGroup, keyframes included — which reintroduces the exact 1.6.0 defect
these tests were added for, every frame reading as a non-keyframe — passed
mvc_frame_emits_blockgroup_additional_and_reference untouched. The
duration-bearing roundtrip test did catch it, via its MkvStream read-back
rather than via the presence check. A mutant corrupting the offset VALUE
was invisible to both.

all_block_groups() parses every BlockGroup in emission order with its
Block flags, relative timestamp, duration and decoded signed ReferenceBlock.
first_block_group() now delegates to it, so there is one walker rather than
two. Both tests assert per block: the keyframe carries NO ReferenceBlock,
each non-keyframe carries one, the offset equals the distance back to the
keyframe, and the SimpleBlock-only 0x80 flag is clear throughout.

Both tests previously took muxer.writer.into_inner() WITHOUT calling
finish(), leaving cluster sizes unwritten — an unparseable file, which is
why they could only byte-scan in the first place. They now write through
SharedWriter and finish(), so the assertions run against a well-formed
Matroska file.

Verified: with the fixes in place, the every-block mutant and the
off-by-one-offset mutant both fail.
2026-07-29 18:00:06 -07:00
Matthew Jackson bf2d15f39c Cover the non-NAL video path, including the wiring that selects it
set_nal_video had exactly one production caller and zero test callers. The
branch it gates decides whether a video track's ES goes through
length_prefixed_to_annex_b, and MPEG-2 and VC-1 must not: they are already
start-code ES. Getting it wrong is silent — frame_count still increments,
so the mux reports success while emitting a video-less file.

Five tests, each verified against a real mutant:

  * non-NAL ES passes through byte-for-byte, and the default path still
    converts. Both use a deliberately length-prefix-SHAPED payload so a
    wrongly-applied conversion rewrites the leading four bytes into a start
    code — a payload the converter happened to leave alone would let a
    mutant pass.
  * a non-NAL keyframe arms params_written, so following non-keyframes are
    not dropped by the pre-keyframe guard.
  * set_nal_video rejects an out-of-range track instead of panicking.
  * M2tsStream::create wires a VC-1 track to the non-NAL path.

That last one matters more than it looks. The four TsMuxer-level tests set
the flag themselves, so deleting the set_nal_video loop from
M2tsStream::create left all 2366 tests passing — the exact mutant the
finding named went undetected until a test drove the real wiring. It now
also catches the subtler mutant of widening the matches! arm to include
Vc1.
2026-07-29 17:54:27 -07:00
Matthew Jackson d09ed76e07 Promote AACS unit encryption to real API, and assert decrypt byte-exactly
encrypt_unit becomes public library API rather than a #[cfg(test)] helper.
Authoring an encrypted disc image is a legitimate use of this crate, and
the capability was already written four times over: a pub(crate) test-only
copy in aacs/content.rs plus three hand-rolled duplicates in decrypt.rs,
sector/decrypting.rs and disc/extract.rs. All four now call one function,
removing ~110 lines of duplicated cipher code that could drift from
decrypt_unit independently.

It mirrors decrypt_unit's purity contract: crypto only, no encrypted-flag
handling, because where that flag lives is container-specific (CPI bits in
byte 0 for BD-TS, elsewhere for HD-DVD-PS). Callers set the flag BEFORE
encrypting — bytes 0..16 are the key seed left in plaintext, so touching a
header byte afterwards changes the key a decryptor derives. That footgun is
documented at the function and at every call site.

Two tests pin it: an exact round trip through both directions, and the one
place the pair is deliberately asymmetric — decrypt_unit restores
all-zero-on-disc packets to zero, and the test proves an all-zero plaintext
packet enciphers to non-zero bytes so it is never mistaken for padding.
That asymmetry was previously only prose.

Two decrypt tests were also weaker than their own names:

  * aacs_clear_trailing_partial_passes_through asserted only is_ok(), so a
    mutant corrupting the clear partial while returning Ok passed. It now
    snapshots the buffer and asserts byte equality, matching the
    none_keys_is_noop pattern already in the file.
  * aacs_decorator_decrypts_encrypted_unit_via_map checked only that 0x47
    reappeared at the 192-byte stride, leaving corruption in the other 6112
    bytes undetected. The plaintext is fully known, so it now asserts
    byte-exact recovery against it.

Both were verified red first by mutating the production path.
2026-07-29 17:50:24 -07:00
Matthew Jackson f76688a0dc Make the ddts speaker mask agree with its own channel count
The ddts box declares both a channel count and a 16-bit speaker mask, and
a decoder may trust either. dts_channel_layout ended in a `_ => 0x0007`
catch-all describing five speakers (C + L/R + Ls/Rs), so for AMODE 6, 7,
and 10 through 15 the box contradicted the count DTS_AMODE_CH declared
alongside it — provoking a downmix or an outright decode error.

AMODE 6 (L + R + S) is the reachable case: its S is a single
centre-surround, not the Ls/Rs pair, so it is three channels and 0x0012,
not four and 0x0006. AMODE 13/14/15 do not occur on retail media, which
ships a 5.1 AMODE-9 core plus an extension substream.

Replace the catch-all with the full 16-entry ETSI TS 102 114 mask table.
Every entry is cross-checked against DTS_AMODE_CH by counting the
speakers its bits imply: sixteen independent constraints, all satisfied,
and that table is itself pinned to the per-AMODE channel counts in ETSI TS 102 114.

AMODE is a 6-bit field, so the reserved 16..=63 are reachable from a
malformed stream. The old `unwrap_or(6)` invented a channel count for
them that no mask could match; parse_dts now refuses those frames rather
than guessing a layout it cannot name.

One existing fixture placed the ext-sync pattern at f[4..8], which
incidentally set f[7]=0x25 → AMODE 20. That test is about where the
pattern sits, not about AMODE, so the frame is now spec-legal (AMODE 9,
48 kHz) with the pattern moved into the payload proper.
2026-07-29 17:43:38 -07:00
Matthew Jackson 840cb9aef0 Drop the comment that promised tests this crate no longer has
The bisect ReadAction regression tests moved to freemkv-engine with the
recovery strategy, but their explanatory block stayed behind — seventeen lines
describing a `let _ = handle_read_error(..)` bug and asserting "the tests below
prove the required ReadAction values are produced". There are no tests below it;
handle_read_error is not even resolvable here any more. Anyone auditing whether
that bug is still guarded would read this and conclude yes. The block moved to
the engine alongside the tests it describes.
2026-07-29 16:36:22 -07:00
Matthew Jackson f3e80c8499 Route conduct reports through GitHub instead of a private address
The enforcement contact was an address on a domain used for internal
infrastructure, which does not belong in a public repository. GitHub needs no
mailbox to exist and is reachable for anyone who can read this file.
2026-07-29 16:07:14 -07:00
Matthew Jackson c812a32f3d mux: fix keyframe signalling for BlockGroup frames
Inside a Matroska BlockGroup the SimpleBlock 0x80 keyframe bit is
reserved and is always written as 0; keyframe-ness is carried only by
the presence or absence of a ReferenceBlock child. Both halves of the
round-trip got this wrong:

  - the reader skipped past ReferenceBlock and read the reserved bit,
    so every BlockGroup frame came back as a non-keyframe;
  - write_block_group discarded its `keyframe` argument and never
    emitted a ReferenceBlock, on the assumption that only intra frames
    (PGS subtitles) reached that path.

The MPEG-2 parser stamps a per-frame duration on I, P and B pictures
alike, so all MPEG-2 video is written as a BlockGroup. That makes the
assumption false and left no video frame looking like a keyframe on
read-back. Downstream, mkv:// -> m2ts:// dropped every video frame (the
TS muxer discards non-key video until the first keyframe) while still
reporting success, and mkv:// -> mkv:// and the stdio round-trip failed
E6008, because the MKV muxer opens a cluster only on a track-0 video
keyframe and so wrote nothing at all. HEVC was unaffected: it carries no
per-frame duration, so it takes the SimpleBlock path where the flag bit
is authoritative.

Verified on a real CSS DVD: 841 keyframes out of 11440 video packets
survive a re-mux, matching the I-picture count in the source bitstream.

Also stop running non-NAL video through the Annex-B converter. MPEG-2
and VC-1 elementary streams are already start-code framed, so
length-prefix conversion corrupts them. TsMuxer takes a per-track
nal_video flag, defaulting to the previous behaviour, which M2tsStream
sets from each video stream's codec.
2026-07-29 15:29:48 -07:00
Matthew Jackson eedd27e352 mux: document that the Url arm selects via InputOptions.selection
The Url mux path prunes streams inside input() via InputOptions.selection;
MuxOptions.selection only applies on the File/Session arms. A Url-source caller
must set InputOptions.selection — noting it so a future caller does not put the
selection on MuxOptions and silently keep every track (the bug the GUI hit).
2026-07-28 18:40:47 -07:00
Matthew Jackson d8e5b97c86 1.6.0: remove recovery strategy (moved to freemkv-engine) + trim dead surface
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).

- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
  Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
  classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
  three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
  READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
  consumer), and the DriveSpeed enum (its one live use — set max drive
  speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
  decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
2026-07-28 15:35:19 -07:00
MattJackson 0151e199ef changelog: 1.6.0 section (layering API, mux fixes, engine relocation, stream selection); date 1.5.2 2026-07-28 13:54:42 -07:00
MattJackson ea99c82e32 mux: end-to-end test that stream selection drops excluded-PID frames
A title declaring two audio PIDs, pruned to one via StreamSelection::apply
before build_iso_pipeline, must never surface a frame from the excluded PID.
Proves the declaration-driven seam end-to-end through the full highway (read ->
demux -> codec parse): the demuxer is built from the pruned title.streams, so
the excluded PID is untracked and skipped, and both track headers and frames
follow the pruned list. Builds on the existing synthetic-TS harness.
2026-07-28 13:36:36 -07:00
MattJackson 8822c29905 disc: add DiscTitle audio_streams/subtitle_streams/video_streams accessors
Typed iterators over the audio/subtitle/video streams, cleaner than matching on
the Stream enum for the common iterate-the-tracks case (stream selection, the
desktop UI info panel, disc-info). Additive; all lib tests pass on 1.86.
2026-07-28 13:35:12 -07:00
MattJackson 0ee8341aea mux: StreamSelection primitive + apply sites for per-title stream selection
The demux pipeline is declaration-driven off DiscTitle.streams (build_demux_state,
DiscStream::new, and the MKV writer all key off that list), so 'which streams to
keep' is already a pipeline capability with no public knob. This adds the knob:

- mux/select.rs: StreamSelection { audio, subtitle: PidFilter::All | Only(Vec<u16>) }
  + apply(&mut DiscTitle): keep Video always, keep Audio/Subtitle whose PID the
  filter lists, prune the rest (and the parallel codec_privates in lockstep);
  error SelectionPidUnknown on a listed PID absent from the title (fail loud, not
  a silently-missing track). Pure; 6 unit tests. Re-exported at crate root.
- Error::SelectionPidUnknown (E6014).
- MuxOptions gains  (+ derives Default now) applied in mux_stream's
  Iso/Session arms before the highway/DiscStream builds demux state (and before
  probe_and_remap's DVD AC-3 PID rewrite). InputOptions gains  applied
  in input()'s iso arm right after the title-index bounds check.

PIDs not languages -- language->PID is caller/engine policy. Default All/All is a
no-op (apply gated on !is_all()), so the no-selection path is byte-identical:
nothing below the title-finalization line changes (ts/ps/demux_thread/
pipelined_stream/mkv/disc untouched). All 2488 lib tests pass on 1.86.
2026-07-28 13:14:40 -07:00
MattJackson d22d09c898 error: add is_disc_level_no_key classifier (re-exported)
Distinguishes a WHOLE-DISC key failure (E_NO_DISC_KEY / E_KEYDB_LOAD /
E_AACS_NO_KEYS -- every title fails identically) from a per-title skippable
stub. The engine's multi-title loop uses it to fail-fast on the first no-key
title instead of iterating all N. Additive.
2026-07-28 12:45:58 -07:00
MattJackson 43564d5752 error: re-export is_halt + is_skippable_title_stub at crate root
The engine's multi-title rip loop classifies per-title mux failures (halt vs
skippable stub vs hard) using these typed classifiers instead of E-code string
matching. Additive; no behavior change.
2026-07-28 12:31:40 -07:00
MattJackson 508a2c3376 disc: promote locate_ranges to pub (engine multipass reads it)
Small pub promotion + fmt one-lining. The relocated multipass progress
reporting in freemkv-engine needs locate_ranges externally. No behavior change.
2026-07-28 12:09:38 -07:00
MattJackson 45a16991ff drive: promote extract_scsi_context to pub
Small pure error->(status,sense) introspection helper the relocated
sweep/patch will need externally. Same category as the prior WritebackFile/
resolve_content_key_map promotions -- infra, not policy. No behavior change.
2026-07-28 11:41:49 -07:00
MattJackson ca0daaea07 io: promote WritebackFile to pub
Bounded-cache buffered File replacement used across mux/extract/sweep/patch
-- general I/O infrastructure, not recovery policy. freemkv-engine's
relocated sweep/patch need to construct it directly once those methods
leave this crate. No behavior change.
2026-07-28 11:36:43 -07:00
MattJackson a02bbbdd24 scsi: promote SenseFamily to a lib-level SCSI-fact primitive
Moved SenseFamily::from_sense_key + is_wedge_family from disc/read_error.rs
into scsi/mod.rs (with its own tests) and re-exported at the crate root.
This is pure SCSI sense-code classification -- objective hardware fact, zero
recovery-policy opinion -- so it belongs in the library primitives, unlike
the retry-DECISION state machine (ReadCtx/PassSummary/ReadAction/
handle_read_error) built on top of it, which is freemkv's specific recovery
strategy and is moving to freemkv-engine next.

disc/read_error.rs and disc/section_recover.rs now import SenseFamily from
crate::scsi instead of defining/re-exporting their own copy. No behavior
change. Precommit green on Rust 1.86 (fmt+clippy+test).
2026-07-28 11:32:13 -07:00
MattJackson ba8114f29b disc: promote resolve_content_key_map + encrypted_content_ranges to pub
The upcoming freemkv-engine crate needs both to build the multipass
sweep/patch recovery strategy externally over Disc's public API. Everything
else sweep/patch touch on Disc was already pub; these were the only two
gaps. No behavior change -- visibility only.
2026-07-28 11:26:37 -07:00
Matthew Jackson 8421c227cd mux: fix DTS core-header false-drops + close TrueHD/mux gate coverage
DTS core decodability gate (core_header_drop_reason) — full ETSI TS 102 114
spec-conformance sweep against a reference decoder the spec core-header rules and
a reference decoder parse_frame_header:

- deficit_samples: only require ==32 for NORMAL frames (FTYPE==1). A
  TERMINATION frame (FTYPE==0, the last frame of a stream) legitimately
  carries fewer and is fully decodable; the old unconditional check dropped
  it on every stream that ends on one — a guaranteed per-track silence gap.
  Matches a reference decoder (normal_frame && deficit != DCA_PCMBLOCK_SAMPLES) and
  a reference decoder (branches on normal_frame).
- reserved bit (after RATE): both reference decoders SKIP it (a reference decoder
  skip_bits1, a reference decoder bits_skip1 "Reserved field") and never reject on it.
  Rejecting was a false-drop that silenced any real stream whose encoder
  set the bit. Relaxed to read-and-discard; DropReason::ReservedBit removed.

Swept and confirmed spec-correct as-is (no change): npcmblocks multiple-of-8,
frame_size>=96, audio_mode>=16 (a reference decoder-permissive), sample-rate validity
table (matches avpriv_dca_sample_rates incl 96k/192k at 14/15), LFE flag==3
invalid, PCMR bits table (matches a reference decoder sample_res {16,16,20,20,0,24,24,0}).
Bit-read order verified field-by-field against a reference decoder. bit_rate is left
unvalidated (lenient, never-false-drop direction) as before.

Tests: termination frame with small deficit is kept; normal frame with bad
deficit is dropped; reserved-bit-set frame is kept. make_bad_dts_core now
uses an invalid LFE flag (duration-neutral) instead of the relaxed reserved
bit.

TrueHD: add coverage for the EXTENDED major-sync header CRC path (ms[25]&1,
mshdr=28+2+2n) — previously zero-tested, the exact path a shipped endianness
bug once used to silently drop whole 7.1/Atmos tracks. Trailer is an
independently-computed oracle (separate CRC-16/0x2D, anchored to the 0x4FF7
catalogue value, NOT crc16_mlp), stored little-endian; test asserts accept,
body-corruption reject, and big-endian-trailer reject.

mux driver: extract the finish completion mapping into pure mux_run_completed
so the finalize_failed -> completed=false branch (reachable only via real
write-thread wedge timing) is unit-tested; add an out-of-range
MuxInput::Session title_index test asserting a clean Error::MuxTrackRange
(E9011) instead of a panic.
2026-07-24 10:28:21 -07:00
Matthew Jackson b79ff71b43 Fix read-fault misclassification, DTS AMODE channel table, and untestable guards
- resolve_fmts_key_map: distinguish a genuinely-not-FMTS disc from a
  transient live-drive read fault. read_filesystem now returns the new
  Error::UdfNotFilesystem for a deterministic tag/format mismatch (no AVDP,
  no partition descriptor, no FSD); resolve maps only UdfNotFilesystem (fs)
  and UdfNotFound (.tbl absent) to Ok(None), and PROPAGATES DiscRead / other
  I/O faults so a marginal AACS 2.1 disc fails loud instead of silently
  dropping forensic content under a base-Unit-Key-only map.

- DTS_AMODE_CH (mp4/audio.rs): extend 10→16 entries
  {1,2,2,2,2,3,3,4,4,5,6,6,6,7,8,8} (the spec per-AMODE channel table / ETSI TS 102 114) so
  the spec-legal high AMODEs that now pass the decodability gate declare
  their true channelcount (AMODE 13→7, 14/15→8) instead of a truncated 6.

- session.rs resolve_keys "called before scan" guard is now testable:
  from_parts_for_test takes Option<Disc>; added a test that a disc-less
  session returns a clean DeviceNotReady Err rather than panicking.

- mp4/read.rs: a track with samples but a missing/malformed stts (mandatory
  per ISO/IEC 14496-12) is dropped rather than emitting all-zero timestamps,
  matching the existing stco/stsc guards; all-tracks-dropped → Mp4Invalid.

- Remove the inert MuxInput::Iso.key_map field (the Iso path re-derives its
  map inside build_iso_pipeline); the live path keeps Live.key_map.

All four fixes are mutation-verified.
2026-07-24 09:59:33 -07:00
Matthew Jackson 9b3e281f4d mux: distinguish FMTS phase-probe read fault from wrong key; cover multi-CPS
Fix 1 (correctness): resolve_fmts_key_map's per-index phase probe read a
single representative segment via read_unit (whose read_sectors(...).ok()?
swallows read errors into None) with no fault fallback. A transient live-drive
read fault (e.g. NOT READY 2/04/3E on the BU40N) made every probe read return
None, giving even==0 && odd==0 — indistinguishable from a genuine wrong key —
so resolve_tie_phase returned FmtsKeyMissing and aborted the entire multi-title
rip even though the forensic index keys were valid and in hand.

Extract the probe into probe_index_phase, which returns Phase / WrongKey /
ReadFault. It mirrors the anchor loop's tolerance: it tries multiple same-index
segments and only concludes WrongKey once a read actually succeeded and decrypted
to no clean parity. If every read of every same-index segment faults it returns
ReadFault; the caller then leaves the phase unresolved so the range-builder
defaults to Phase::All (decrypt both parities, demux drops the garbled alternate
half) instead of aborting. A wrong key can never be masked as a read fault:
ReadFault requires that not a single read succeeded, so zero decrypt evidence.

Fix 2 (test coverage): resolve_mux_key_map's multi-CPS branch was only exercised
with an all-zero source, so pick(), the KeyFetch cold path, and the fail-loud
DecryptFailed guard never ran. Add tests over real AACS ciphertext (via
aacs_encrypt_unit_for_test) covering pick() selecting the correct pool index,
the DecryptFailed guard firing when a clean sample matches no held/fetched key,
and the fetch cold path recovering a missing unit key. Mutation-verified.

Fix 3 (doc): move the reader_event_fn EventKind->MuxEvents mapping doc off
session_mux_keys onto reader_event_fn.
2026-07-24 09:28:09 -07:00
Matthew Jackson e0456c72da mux: thread halt into live AACS key-map resolution; cover Session arm
Round-2 follow-ups to 6d6e60f (inline base-map resolve on the live
single-pass Session/Live mux arms).

Fix 1 (halt threading) — the inline resolve chain sampled ciphertext off
the LIVE drive with no cancel token, so an operator /api/stop during key
resolution was not honored (the FMTS probe can issue hundreds of reads,
each able to stall to the 60s SCSI recovery timeout — violating the
"don't hammer a struggling live drive" rule). Add an optional
`halt: Option<&Halt>` to `resolve_mux_key_map`, `resolve_fmts_key_map`,
`resolve_inline_base_map`, and `Disc::resolve_content_key_map`, and poll
it at each loop boundary (FMTS anchor + per-index probe loops, multi-CPS
extent loop) — returning Err(Halted) promptly. Live/Session arms pass the
driver's halt; sweep/patch pass their own token (via Halt::from_arc);
file-backed probe/ISO callers pass None. Tested with a pre-cancelled halt
(Err Halted, no extent sampling) and a None-halt no-abort case;
mutation-verified (dropping the extent-loop check → Ok, not Err).

Fix 2 (Session-arm coverage) — the MuxInput::Session arm ran the same
resolve→install→decrypt sequence as Live but had NO end-to-end test
(DiscSession only exposed open(), which needs live hardware). Add a
#[cfg(test)] DiscSession::from_parts_for_test (injected reader + scanned
disc, no Drive), an end-to-end AACS decrypt test through the Session arm
(mutation-verified: dropping with_key_map → mux aborts), and a
missing-reader clean-error (not panic) test.

Fix 3 (cleanups) — io_error_code: remove the unreachable typed-Error
downcast branch (From<Error> for io::Error stringifies; no path builds an
io::Error holding a typed Error), keeping the stringify parse is_halt /
is_skippable_title_stub rely on. Add a resolve_keys_for test covering the
largest-title sampling branch. Document the patch wedge-exit coverage gap
(TODO) in passn_handler_ab.rs.
2026-07-24 09:04:19 -07:00
Matthew Jackson 1c7ccd5a85 mux: resolve+install AACS key map on live single-pass (Session/Live)
Under the map-only decrypt model an AACS DecryptingSectorSource decrypts
nothing until a key map is installed; with no map the AACS arm fails loud
with DecryptFailed on the first content unit. The two inline live-mux arms
in mux_stream did not install one:

  - MuxInput::Session (freemkv `rip disc://…mkv`) installed NO map at all.
  - MuxInput::Live (autorip non-FMTS single-pass) installed only a
    caller-supplied forensic FMTS map, which is None for a plain AACS disc.

So EVERY plain AACS Blu-ray/UHD ripped via the live single-pass path failed
DecryptFailed on the first content read. This predates the mux_stream
refactor: the bug was introduced with the map-only decrypt model, and the
pre-refactor CLI likewise built DiscStream::new without with_key_map.

Fix: add resolve_inline_base_map, the inline counterpart to what
build_iso_pipeline does for the file highway. Both arms now resolve the
AACS map off the reader (borrow to sample, then move into DiscStream) and
install it via with_key_map before any read. DVD/CSS keeps DecryptKeys::None
(DiscStream's per-title CSS crack owns it); clear/raw resolve to no map.
Session passes session.key_fetch() so a multi-CPS/orphan unit can still be
recovered; a caller-supplied FMTS map (autorip) is used verbatim, never
re-resolved.

Tests: an end-to-end MuxInput::Live mux over a genuinely-AACS-encrypted
synthetic unit now decrypts and finalises (mutation-verified: dropping the
resolve/install makes the mux abort). Adds a pub(crate) test-only AACS
encrypt helper so the mux test can build a real encrypted fixture, and a
gating test for resolve_inline_base_map (AACS→map, CSS/clear/raw→none).
2026-07-24 08:34:51 -07:00
Matthew Jackson 3bd2fd23b0 Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just
  0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements;
  only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are
  decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal
  multichannel core frames as undecodable, silencing recoverable audio. Add a
  regression test (literal 0..16 range) that fails if the bound reverts to 10.

- keysource: only memoize a NEGATIVE (empty) key-fetch result when every source
  genuinely ran and none held the key — never when a source Err'd (network down,
  unreachable). A transient outage was being cached as a permanent "no key" for
  the fingerprint, permanently dropping a unit that could be recovered once the
  source came back. Thread an `errored` flag out of the drivers and gate the
  cache insert on it. Tests cover both the recover-after-outage case and that a
  genuine absence is still memoized.

- pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS
  display sets through the full demux -> parse -> observe -> apply path, both a
  forced verdict landing and a non-forced verdict clearing a vendor flag.

- mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD).

- scan_iso test: add independent fixture expectations (volume id) so the parity
  test is no longer purely tautological against a re-run of the same composition.
2026-07-24 08:32:37 -07:00
173 changed files with 65095 additions and 18987 deletions
+63
View File
@@ -0,0 +1,63 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+188 -10
View File
@@ -2,50 +2,228 @@ name: CI
on:
push:
branches: [main]
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
pull_request:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
- run: cargo clippy -- -D warnings
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
working-directory: libfreemkv
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo check
working-directory: libfreemkv
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
# Build the tests (not just `cargo check`): catches errors in test
# code and forces full codegen of the Windows-only SPTI transport
# (src/scsi/windows.rs), which never compiles on the Linux/macOS dev
# hosts. We don't `cargo test` here — the suite needs no drive but the
# extra build is the value; running tests is covered by the Linux job.
- run: cargo build --tests
working-directory: libfreemkv
# ── Did this change break anything downstream? ──────────────────────────────
#
# Every job above proves libfreemkv builds. None proved its DEPENDENTS do,
# and that gap is real: an engine signature change broke autorip today and
# went unnoticed because consumer CI only fires on a push to that consumer.
# libfreemkv sits below all five of them, so a break here is worth strictly
# more than a break anywhere else in the project.
#
# `cargo check --all-targets` only — each dependent owns its own behaviour
# and has its own suite. The question here is just "does everything built on
# me still compile against this commit".
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo test --tests
working-directory: freemkv
- run: cargo test --tests
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
+133
View File
@@ -0,0 +1,133 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
on:
push:
branches: [qa]
workflow_dispatch:
jobs:
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
working-directory: libfreemkv
+11 -6
View File
@@ -4,6 +4,11 @@ on:
push:
tags:
- 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions:
contents: write
@@ -12,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -24,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.86, same commit). Binary consumers
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -32,8 +37,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
@@ -51,8 +56,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
-36
View File
@@ -1,36 +0,0 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push
+9
View File
@@ -14,3 +14,12 @@ scratch/
# internal agent context — never publish (path AND dir; leak-guard blocks both)
CLAUDE.md
.claude/
# Nightly harness output. Written into the repo it audits, and it embeds
# absolute paths from the machine that ran it — which must never reach a public
# repo. Ignored rather than relocated so a run from any working copy is safe.
.nightly/
# cargo-mutants working output: large, machine-specific, never committed
mutants.out/
mutants.out.old/
+757 -895
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -36,7 +36,7 @@ This Code of Conduct applies within all community spaces, and also applies when
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at matthew@pq.io. All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+8 -13
View File
@@ -1,8 +1,8 @@
[package]
name = "libfreemkv"
version = "1.5.2"
version = "1.6.7"
edition = "2024"
rust-version = "1.86"
rust-version = "1.97"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@@ -22,27 +22,22 @@ codegen-units = 1
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { path = "../freemkv-unlock" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
+9 -42
View File
@@ -55,48 +55,15 @@ output.finish()?;
### Multi-pass recovery rip
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md).
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the
ddrescue mapfile, damage classification and the multipass loop now live in the
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`.
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
libfreemkv keeps the layers underneath: the raw single-shot read
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's
strategy is built on. The dependency runs engine → libfreemkv, so this crate
cannot call into it; front-ends get recovery from the engine directly. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
## What It Does
@@ -114,7 +81,7 @@ loop {
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
+22
View File
@@ -0,0 +1,22 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
+3 -3
View File
@@ -141,8 +141,8 @@ If your machine has a free SATA port, use it.
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
@@ -302,7 +302,7 @@ Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not adver
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
2. **Drive model:** from the drive label, or from `freemkv info disc://`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
+15 -8
View File
@@ -22,7 +22,7 @@ fn main() {
&target_arch // x86_64 → x86_64
};
std::process::Command::new("cc")
let cc_status = std::process::Command::new("cc")
.args([
"-arch",
clang_arch,
@@ -38,12 +38,19 @@ fn main() {
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
.expect("failed to spawn cc for macos_shim.c");
// `.status()` succeeding only means the process RAN. A real compile error
// exits non-zero, and ignoring that left no object file, which surfaced
// much later as an unexplained link failure against a missing symbol. The
// shim is macOS-only and is neither linted nor compiled on the other two
// platforms, so a mistake in it has exactly one chance to be noticed.
assert!(cc_status.success(), "cc failed to compile macos_shim.c");
std::process::Command::new("ar")
let ar_status = std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
.expect("failed to spawn ar");
assert!(ar_status.success(), "ar failed to create the static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
@@ -77,10 +84,10 @@ fn emit_git_suffix() {
// Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD") {
if let Some(ref_path) = head.strip_prefix("ref: ") {
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
if let Ok(head) = std::fs::read_to_string(".git/HEAD")
&& let Some(ref_path) = head.strip_prefix("ref: ")
{
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
+27 -16
View File
@@ -65,27 +65,38 @@ if disc.encrypted {
}
}
// Read content -- decryption is automatic
let mut reader = disc.open_title(&mut session, 0).unwrap();
while let Some(unit) = reader.read_unit().unwrap() {
// decrypted content
// Read content -- decryption is applied on read by the DiscStream decorator.
// Live disc does NOT go through the URL resolver: `input("disc://...")` returns
// Error::DiscUrlNotDirect by design.
let keys = disc.decrypt_keys();
let mut stream = DiscStream::new(
Box::new(drive),
disc.titles[0].clone(),
keys,
batch_sectors,
disc.titles[0].content_format,
false, // raw: false → decrypt on read
None, // halt
)?;
while let Ok(Some(frame)) = stream.read() {
// decrypted PES frames
}
```
The application never touches keys, never calls decryption functions, and never
manages handshakes. All of that is internal to `Disc::scan()` and the content
reader.
The application never calls decryption functions and never manages the
drive-level handshake. It DOES own key resolution — see below.
### KEYDB Location
### Key resolution is the caller's job
`ScanOptions` controls where the keydb is loaded from. If no explicit path is
set, the library checks the standard config locations. To specify an explicit
path:
`libfreemkv` is **lookup-free: it resolves no keys and reads no keydb.** There is
no `ScanOptions::with_keydb`, and `ScanOptions` has no keydb field — its only
scan input is the optional drive credentials for the live-drive authenticated
handshake.
```rust
let opts = ScanOptions::with_keydb("/path/to/keydb.cfg");
let disc = Disc::scan(&mut session, &opts).unwrap();
```
The caller resolves a key out-of-band through a key source and applies it with
[`Disc::decrypt_with`]. `freemkv-keysources` is the crate that implements the
keydb and key-server sources; `ScanOptions::key_sources` takes them as
`Box<dyn KeySource>`.
### AacsState
@@ -97,7 +108,7 @@ After a successful scan, `disc.aacs` contains an `AacsState`:
| `bus_encryption` | `bool` | Whether bus encryption is active |
| `mkb_version` | `Option<u32>` | MKB version from disc |
| `disc_hash` | `String` | Identifier for the disc's key-input files |
| `key_source` | `KeySource` | How the disc's key was resolved |
| `key_source` | `KeyOrigin` | How the disc's key was resolved |
## keydb.cfg
+10 -7
View File
@@ -170,17 +170,20 @@ libfreemkv/src/
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── mod.rs Drive struct, init, read (single-shot), eject
│ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
│ ├── mod.rs Disc struct, scan, titles, formats; Disc::copy + Disc::sweep (Pass 1)
│ ├── sweep.rs Pass 1 internal helpers (pub(super))
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── mapfile.rs ddrescue-format mapfile
── read_error.rs ReadCtx / ReadAction state machine
├── disc/ Disc (scan, titles, AACS setup, per-format parsing)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── bluray.rs Blu-ray / UHD scanning (MPLS/CLPI-driven)
│ ├── dvd.rs DVD-Video scanning (IFO-driven)
│ ├── hddvd.rs HD-DVD scanning
── extract.rs Per-extent content extraction
│ ├── encrypt.rs Encrypted-range mapping for content reads
│ ├── dvd_audio_probe.rs DVD audio-stream probing
│ └── pgs_forced_probe.rs PGS forced-subtitle probing
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── unlock.rs Unlocker trait + registry (pluggable unlock seam)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
+3 -2
View File
@@ -96,12 +96,13 @@ After open:
- `init()` -- routes to the matching registered unlocker (if any); otherwise
a no-op and the cert handshake carries the disc
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (`READ_TIMEOUT_MS` 10 s vs. `READ_RECOVERY_TIMEOUT_MS` 60 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
(`freemkv_engine::recovery::patch`, in the engine crate) handles bad-range
retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
+15 -5
View File
@@ -69,13 +69,23 @@ Each stream PID entry header (14 bytes):
```
Offset Size Field
------ ---- -----
0 2 Stream PID
2 2 Reserved + EP stream type
4 2 Number of coarse entries
6 4 Number of fine entries (note: 32-bit, can be large)
10 4 EP map start offset (relative to EP map start)
2 2 stream_PID (byte-aligned)
4 10 Bit-packed block, 80 bits total (see below)
```
The stream PID entry is **not** byte-aligned past `stream_PID`. Bytes 4..14 are one
80-bit packed field, read as a `u64` plus a trailing `u16`:
Bits Width Field
---- ----- -----
0-9 10 reserved
10-13 4 EP_stream_type
14-29 16 num_EP_coarse
30-47 18 num_EP_fine
48-79 32 EP_map_start_address (relative to the EP map start)
Note `num_EP_fine` is **18 bits**, not 32. See `parse_cpi` in `src/clpi.rs`.
libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking.
### Two-Level Index
+2 -2
View File
@@ -79,7 +79,7 @@ Insert disc
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — Disc::patch re-runs against the mapfile.
│ 1 above this — freemkv_engine::recovery::patch re-runs against the mapfile.
PES frames → output stream (MKV, M2TS, network, etc.)
@@ -123,7 +123,7 @@ output.finish()?;
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan (sweep/patch/mapfile moved to freemkv-engine in 1.6.0) |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
+4 -3
View File
@@ -58,15 +58,16 @@ selects the per-CDB timeout:
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
| `false` | 10 s | `freemkv_engine::recovery::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 60 s | `freemkv_engine::recovery::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
- **Layer 1 — `freemkv_engine::recovery::patch`** (in the engine crate, not
here) loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
+64 -167
View File
@@ -1,135 +1,38 @@
# Rip recovery — three-layer architecture
# Rip recovery — what libfreemkv owns
`libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
discs: a fast forward sweep that tolerates read failures, in-loop request-size
adaptation that survives transient drive trouble without bailing, and targeted
retry passes against a persistent bad-range map. The stream pipeline
(`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
the mux stage never touches the drive.
**Recovery strategy moved OUT of this crate in 1.6.0.** The forward sweep, the
targeted retry pass, the ddrescue mapfile, damage classification and the
multipass loop now live in the **`freemkv-engine`** crate as
`freemkv_engine::recovery::{copy, sweep, patch}`. The dependency runs
engine → libfreemkv, so this crate cannot call into the engine; front-ends
(`freemkv` CLI, autorip) get recovery from the engine directly.
Recovery is layered cleanly. Each layer has one responsibility and does not
reach into the others.
What stayed here are the two layers underneath the strategy: the single-shot
read primitive, and the in-stream request-size adaptation that sits in front of
it. This document covers those, plus the design constraints they encode — the
constraints are the reason the strategy above them looks the way it does, so
they belong with the code that enforces them.
For the strategy itself — damage-jump thresholds, pass ordering, mapfile status
state machine, wedge detection — read `freemkv-engine/src/recovery/`.
| Layer | Where it lives | What it does |
|-------|---------------|--------------|
| 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 1 — Bad-range retry | **`freemkv-engine`** (`recovery::patch`) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` in `src/mux/disc.rs` | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
The library exposes flat verbs; the caller drives the multipass loop. Autorip
runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
clean or the configured retry budget is exhausted, then hands the ISO off to
the mux pipeline. The `freemkv` CLI does the same shape with a
terminal-output progress sink. Layer 3 runs inside any consumer of
`DiscStream` (direct PES pipeline, ISO playback, etc.) without caller
involvement.
Layer 2 also translates drive facts: [`SenseFamily`](../src/scsi/mod.rs)
classifies SCSI sense data into the categories the engine's strategy routes on
(marginal vs. hardware vs. not-ready). Getting that classification wrong
silently misroutes recovery, which is why it lives next to the transport rather
than in the strategy.
Three primitives compose the disc-side flow:
Layer 3 runs inside any consumer of `DiscStream` — direct PES pipeline, ISO
playback — without caller involvement, and applies whether or not the engine's
recovery is in play.
| Primitive | What it does |
|---------------------------|-----------------------------------------------------------------------|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
## Data model
### Mapfile
Format: [ddrescue](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)-compatible
plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
so a crashed rip loses at most one block.
```
# Rescue Logfile. Created by libfreemkv v0.13.6
# Current pos / status / pass / pass_time
0x000000000 ? 1 0
# pos size status
0x000000000 0x12a35d000 +
0x12a35d000 0x000003000 -
0x12a360000 0x009c4a000 +
0x12d00a000 0x000064000 *
```
Status characters match ddrescue:
| Char | Meaning |
|------|----------------------------------------------------|
| `?` | Not yet attempted |
| `*` | Fast-pass failed; needs edge-trim |
| `/` | Trimmed; interior needs sector scrape |
| `-` | Unreadable this session |
| `+` | Finished (good) |
Position and size are hex byte offsets into the ISO.
### `SweepOptions` and `PatchOptions`
The library no longer dispatches between sweep and patch internally — the
caller picks the verb explicitly per pass. The two option structs are flat
and have no overlap:
```rust
SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true, // damage-jump + zero-fill on read failure
progress: Some(&reporter),
halt: Some(flag),
}
PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true, // walk bad ranges high → low LBA
wedged_threshold: 50,
progress: Some(&reporter),
halt: Some(flag),
}
```
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
- No mapfile → `sweep` (fresh Pass 1).
- Mapfile with `?` ranges → `sweep` with `resume: true`.
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
- Mapfile clean → done; no further pass needed.
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
lines of `Mapfile::stats()` checks.
## Algorithm
### Pass 1 — fast sweep (`Disc::sweep`)
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
**damage-jump**: skip ahead by `1024×batch×multiplier` sectors (64 MB base for
UHD). Double the multiplier on each jump (64→128→256→512 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
## In-stream — adaptive batch halving (`DiscStream::fill_extents`)
When a consumer reads a `DiscStream` directly (no ISO intermediate),
`fill_extents` runs an adaptive sizer in front of `Drive::read`:
@@ -143,59 +46,53 @@ When a consumer reads a `DiscStream` directly (no ISO intermediate),
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
`Err(DiscRead)`.
This is layer 3. It exists so a transient single-sector glitch in a 32-sector
batch can be isolated and read individually without the caller needing to
implement retry logic.
This exists so a transient single-sector glitch inside a 32-sector batch can be
isolated and read individually without the caller implementing retry logic. See
[`src/event.rs`](../src/event.rs) for the emitted events.
## Design choices
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
handles ranges, layer 3 handles request size, neither touches the
wedge-prone reset path.
These are constraints on the read path, enforced here and relied on by the
engine's strategy.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue
nor any consumer ripper does this. Drive firmware has access to raw analog signal, laser
power control, and drive-specific ECC tuning that userspace can't replicate —
disabling it throws away recovery headroom on marginal sectors. We fail fast
via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
in pass 2 / patch.
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset, no
eject cycle. The `recovery` flag controls only the per-CDB timeout (10 s vs.
60 s); on any failure it returns `Err(DiscRead)` immediately. Inline recovery
(5× gentle retry → close + SCSI reset + reopen → 5× more) was removed in
0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale: the inline
reset on the LG BU40N (Initio USB-SATA bridge) wedged drive firmware below the
bridge without ever recovering a sector, and the gentle-retry phase produced
long stretches of 0 KB/s with nothing to show for it. Recovery responsibility
is layered instead: layer 1 handles ranges, layer 3 handles request size,
neither touches the wedge-prone reset path.
**No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
`SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
The macOS reset (which had been a no-op) was removed entirely. The top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
also removed (no callers). The remaining `Drive::reset()` is only invoked
explicitly by callers that need an eject-cycle escape hatch — it is never
reached from a read path.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue nor any
consumer ripper does this. Drive firmware has access to raw analog signal,
laser power control and drive-specific ECC tuning that userspace cannot
replicate — disabling it throws away recovery headroom on marginal sectors. The
fast pass fails quickly via short SG_IO timeouts and lets the firmware work the
long timeout during retry.
**ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
mux stage reads the ISO via `FileSectorSource`. For single-pass (no retries),
this adds ~2-3 min (local disk mux) but gains resumability across crashes,
**No SCSI reset from any read path.** There is no reset escape hatch on
`Drive` at all: the `SG_SCSI_RESET` ioctl and STOP/START UNIT escalation went in
0.13.6, the macOS reset (always a no-op) was removed entirely, and the
top-level `scsi::reset()` wrappers went with their last callers. The only
remaining reset is a Windows-specific device-level helper in
[`src/scsi/windows.rs`](../src/scsi/windows.rs), never reached from a read.
**ISO intermediate, even for single-pass.** The engine's Pass 1 always writes
an ISO, and the mux stage reads it back via `FileSectorSource`. For a
no-retry rip this costs a few minutes but buys resumability across crashes,
re-muxability without re-ripping, and a persistent forensic artifact. Callers
who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
still applies there.
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
coalesce on adjacent same-status ranges so files stay small.
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
ddrescue's but `patch` collapses the distinction — it just tries every
non-finished range with the long timeout. Future work can specialize (trim vs.
scrape vs. retry with direction reversal) if there's measured benefit.
who need pure speed can bypass it with `DiscStream::new(Box::new(drive), …)`
nothing forbids it, and layer 3 still applies there.
## References
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
- Recovery strategy and mapfile: `freemkv-engine/src/recovery/`
- In this crate: [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`),
[`src/scsi/mod.rs`](../src/scsi/mod.rs) (`SenseFamily`),
[`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`),
[`src/event.rs`](../src/event.rs) (progress events).
+1 -1
View File
@@ -114,7 +114,7 @@ The `read_filesystem()` function in `src/udf.rs` follows the pointer chain above
2. Scans sectors 32-63 for the Partition Descriptor and Logical Volume Descriptor.
3. If two partition maps exist and the second is Type 2, reads the metadata file ICB at partition_start to find metadata_start.
4. Reads the FSD at metadata_start, extracts the root directory ICB LBA.
5. Calls `read_directory()` recursively (max depth 3) to build the full file tree.
5. Calls `read_directory()` recursively (max depth `MAX_DIR_DEPTH` = 8) to build the full file tree.
Each directory read involves two sector reads: one for the ICB, then one or more for the directory data. File sizes are read from info_length in each file's ICB.
-83
View File
@@ -1,83 +0,0 @@
// Minimal ISO dumper — find exact stall point
use libfreemkv::Drive;
use std::io::{BufWriter, Write};
use std::path::Path;
use std::time::Instant;
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("Usage: iso_dump <device> <output>");
std::process::exit(1);
}
let mut drive = Drive::open(Path::new(&args[1])).unwrap();
drive.wait_ready().unwrap();
let _ = drive.init();
let _ = drive.probe_disc();
// AACS handshake — required to read past the protected area
eprint!("Scanning disc... ");
let _ = libfreemkv::Disc::scan(&mut drive, &libfreemkv::ScanOptions::default());
eprintln!("OK");
let cap = drive.read_capacity().unwrap();
let batch = libfreemkv::disc::detect_max_batch_sectors(drive.device_path());
eprintln!("Device: {} | {} sectors | batch {}", args[1], cap, batch);
let file = std::fs::File::create(&args[2]).unwrap();
let mut w = BufWriter::with_capacity(4 * 1024 * 1024, file);
let mut buf = vec![0u8; batch as usize * 2048];
let mut lba: u32 = 0;
let start = Instant::now();
let mut last = Instant::now();
let mut bytes: u64 = 0;
let mut last_bytes: u64 = 0;
while lba < cap {
let count = ((cap - lba) as u16).min(batch);
let n = count as usize * 2048;
// Tiny yield between reads — test if pacing prevents firmware throttle
std::thread::yield_now();
let t0 = Instant::now();
let ok = drive.read(lba, count, &mut buf[..n], true).is_ok();
let read_ms = t0.elapsed().as_millis();
// Flag slow reads
if read_ms > 2000 {
eprintln!("\n SLOW READ: LBA {} took {}ms (ok={})", lba, read_ms, ok);
}
if !ok {
buf[..n].fill(0);
}
w.write_all(&buf[..n]).unwrap();
lba += count as u32;
bytes += n as u64;
if last.elapsed().as_millis() >= 1000 {
let delta = bytes - last_bytes;
let speed = delta as f64 / last.elapsed().as_secs_f64() / 1_048_576.0;
let avg = bytes as f64 / start.elapsed().as_secs_f64() / 1_048_576.0;
let pct = bytes as f64 / (cap as f64 * 2048.0) * 100.0;
eprint!(
"\r {:.1}% LBA {} | {:.0} MB/s (avg {:.0}) | {:.1} GB ",
pct,
lba,
speed,
avg,
bytes as f64 / 1e9
);
last_bytes = bytes;
last = Instant::now();
}
}
w.flush().unwrap();
eprintln!(
"\nDone: {:.1} GB in {:.0}s",
bytes as f64 / 1e9,
start.elapsed().as_secs_f64()
);
}
+384 -66
View File
@@ -4,11 +4,11 @@
#[cfg(test)]
use aes::Aes128;
#[cfg(test)]
use aes::cipher::{KeyInit, generic_array::GenericArray};
use aes::cipher::{Array, KeyInit};
use super::crypto::{aes_cbc_decrypt, aes_ecb_encrypt};
// Available at module scope for this module's test fixtures (they reference
// `super::AACS_IV` when building CBC ciphertext directly); test-only.
use super::crypto::{aes_cbc_decrypt, aes_cbc_encrypt, aes_ecb_encrypt};
// Only this module's test fixtures build CBC ciphertext by hand now — the
// production paths get the IV from `aes_cbc_encrypt` / `aes_cbc_decrypt`.
#[cfg(test)]
use super::crypto::AACS_IV;
@@ -41,7 +41,8 @@ pub const ALIGNED_UNIT_SECTORS: u32 = (ALIGNED_UNIT_LEN / SECTOR_BYTES) as u32;
/// underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`, mis-reports the
/// alignment (e.g. `lba == unit_base - 1` would falsely read as aligned).
pub fn is_unit_aligned(lba: u32, unit_base: u32) -> bool {
lba.saturating_sub(unit_base) % ALIGNED_UNIT_SECTORS == 0
lba.saturating_sub(unit_base)
.is_multiple_of(ALIGNED_UNIT_SECTORS)
}
use crate::consts::SECTOR_BYTES;
@@ -146,8 +147,14 @@ pub fn aacs_unit_needs_decrypt(unit: &[u8], format: crate::disc::ContentFormat)
}
/// Minimum synced content packets that PROVE a key opened a unit. Four `0x47`
/// syncs 32 bits of MPEG-TS structure ≈ 1-in-4-billion that a wrong key (uniform
/// AES noise, `0x47` at 1/256 per packet) fakes it. It is an ABSOLUTE proof floor,
/// syncs are 32 bits of MPEG-TS structure, but the per-UNIT false-pass risk is
/// NOT 2^-32: `is_clean_ts` accepts ANY four of the ~31 encrypted packets in a
/// 6144-byte aligned unit, so for a wrong key (uniform AES noise, `0x47` at 1/256
/// per packet) it is ≈ C(31,4)·256^-4 ≈ 7e-6, i.e. ~1e-5 — the figure
/// [`is_clean_ts`]'s own doc below states. 1-in-4-billion is the probability for
/// four SPECIFIC packets and overstates the margin by ~4000x; at
/// `KEY_PROOF_PACKETS = 3` the per-unit rate is ≈ C(31,3)·256^-3 ≈ 2.6e-4, so do
/// NOT lower it on the strength of slack that is not there. It is an ABSOLUTE proof floor,
/// NOT a proportion — a unit the key opened but whose content is bad-encoded
/// (many non-conforming packets) is proven by ANY four good packets, not rejected
/// for the bad ones.
@@ -171,8 +178,16 @@ pub fn is_clean(unit: &[u8], format: crate::disc::ContentFormat) -> bool {
/// Structural "does this unit carry enough valid MPEG-TS to prove a key opened
/// it?" — the Transport-Stream arm of [`is_clean`]. It is
/// NOT a decryption verdict: [`decrypt_unit`] applies a key (that is
/// "decrypt"); whether the plaintext is clean TS is this SEPARATE question. The
/// mux never calls this — TS validity is a muxer concern, never a decrypt result.
/// "decrypt"); whether the plaintext is clean TS is this SEPARATE question.
///
/// The mux is its PRINCIPAL consumer for `BdTs` discs, reaching it through
/// [`is_clean`]: `mux::resolve`'s multi-CPS `pick` closure selects a unit key by
/// it, `probe_index_phase` reports each FMTS index's interleave parity by it, and
/// `decrypt::decrypt_sectors_mapped` uses it as the forensic-range verify net.
/// (The doc used to say "the mux never calls this", which invited a maintainer to
/// tighten or loosen the proof rule below believing only whole-disc read
/// verification was affected — while it in fact changes which unit key a
/// multi-CPS disc muxes with and which phase an FMTS index is muxed at.)
///
/// Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the
/// ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed, so
@@ -326,16 +341,74 @@ pub fn decrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) {
}
}
/// Encrypt one AACS aligned unit (6144 bytes) IN PLACE — the exact inverse of
/// [`decrypt_unit`], for authoring an encrypted disc image (and for building
/// genuinely-encrypted read-path fixtures).
///
/// PURE, on the same terms as `decrypt_unit`: it applies the key and nothing else.
/// It does NOT set the encrypted flag, because where that flag lives is
/// container-specific (CPI bits in byte 0 for BD-TS, elsewhere for HD-DVD-PS) and
/// keeping it out is what lets this stay container-agnostic. The only guard is the
/// length check, since the crypto is defined only over a whole 6144-byte unit.
///
/// **Set the encrypted flag BEFORE calling, never after.** Bytes 0..16 are the key
/// seed and are left in plaintext, so the Block Key derives from them: mutating any
/// header byte after encrypting changes the seed a decryptor will derive from and
/// silently yields garbage. The caller's order must be flag, then encrypt.
///
/// Block Key = AES-128E(Kcu, seed) ⊕ seed, then AES-128-CBC **encrypt** bytes
/// 16..6144 under the AACS IV — the forward direction of the same construction
/// `decrypt_unit` documents, sharing its module-scope primitives so the two cannot
/// drift apart.
///
/// Note one deliberate asymmetry: `decrypt_unit` restores all-zero-on-disc source
/// padding packets to zero. This does not, and need not — an all-zero plaintext
/// packet encrypts to ciphertext that is not all-zero, so it is not mistaken for
/// padding on the way back and the round trip is still exact. Authoring that wants
/// true source-zero padding leaves those packets unencrypted instead.
///
/// Returns `false` — encrypting nothing — when `unit` is shorter than
/// [`ALIGNED_UNIT_LEN`]. That case MUST be checked: the caller has already set the
/// container's encrypted flag by then (this function's contract requires it), so
/// ignoring the result leaves a unit marked encrypted while still carrying
/// plaintext, which is the worst possible outcome for an authoring tool.
#[must_use = "returns false when the slice is too short to encrypt, leaving \
plaintext behind a flag that already says 'encrypted'"]
pub fn encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) -> bool {
if unit.len() < ALIGNED_UNIT_LEN {
return false;
}
let mut header = [0u8; 16];
header.copy_from_slice(&unit[..16]);
let derived = aes_ecb_encrypt(unit_key, &header);
let mut k = [0u8; 16];
for i in 0..16 {
k[i] = derived[i] ^ header[i];
}
// CBC-encrypt bytes 16.. under the fixed AACS IV — the exact forward of the
// `aes_cbc_decrypt` call in `decrypt_unit`, and one key expansion for the whole
// unit rather than one per 16-byte block.
aes_cbc_encrypt(&k, &mut unit[16..ALIGNED_UNIT_LEN]);
true
}
/// Remove bus encryption from an aligned unit (AACS 2.0 / UHD).
/// Bus encryption uses read_data_key, decrypting bytes 16..2048 of each 2048-byte sector.
pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
// Expand the key schedule ONCE for the whole unit. `read_data_key` is
// loop-invariant here (and constant for the entire disc), but calling
// `aes_cbc_decrypt` per sector rebuilt the AES-128 schedule per sector — three
// expansions per 6144-byte aligned unit, i.e. ~29 million redundant expansions
// over a 90 GB read on a stock drive, on the per-unit decrypt hot path.
// Measured by `decrypt_bus_expands_the_read_data_key_once_per_unit`.
let cipher = crate::aacs::crypto::new_cipher_for(read_data_key);
for sector_start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
if sector_start + SECTOR_BYTES > unit.len() {
break;
}
// First 16 bytes of each sector are plaintext
aes_cbc_decrypt(
read_data_key,
crate::aacs::crypto::cbc_decrypt_blocks(
&cipher,
&mut unit[sector_start + 16..sector_start + SECTOR_BYTES],
);
}
@@ -345,7 +418,105 @@ pub(crate) fn decrypt_bus(unit: &mut [u8], read_data_key: &[u8; 16]) {
mod tests {
use super::super::crypto::aes_ecb_decrypt;
use super::*;
use aes::cipher::BlockEncrypt; // test fixtures build ciphertext directly
use aes::cipher::BlockCipherEncrypt; // test fixtures build ciphertext directly
/// [`encrypt_unit`] is the exact inverse of [`decrypt_unit`]: whatever an
/// authoring caller encrypts, the read path must recover byte-for-byte.
///
/// Mutation: drop the trailing `⊕ header` from either function's Block Key
/// derivation, or swap `AACS_IV` for zeroes in one of them, and the two stop
/// agreeing -> this fails.
#[test]
fn encrypt_unit_is_the_exact_inverse_of_decrypt_unit() {
let key = [0x3Cu8; 16];
// Content with no all-zero packets: every byte position exercised.
let mut clear: Vec<u8> = (0..ALIGNED_UNIT_LEN)
.map(|i| (i * 7 % 251 + 1) as u8)
.collect();
// The encrypted flag belongs to the caller and must be set BEFORE the
// crypto, since bytes 0..16 are the key seed.
clear[0] |= 0xC0;
let mut unit = clear.clone();
assert!(
encrypt_unit(&mut unit, &key),
"a full-length unit must encrypt"
);
assert_ne!(
unit[16..],
clear[16..],
"the payload must actually be enciphered"
);
assert_eq!(
unit[..16],
clear[..16],
"the 16-byte seed stays plaintext on disc"
);
decrypt_unit(&mut unit, &key);
assert_eq!(unit, clear, "round trip must be byte-exact");
}
/// The documented padding asymmetry actually holds: `decrypt_unit` restores
/// all-zero-ON-DISC packets to zero, but an all-zero PLAINTEXT packet enciphers
/// to non-zero bytes, so it is not mistaken for padding and still round-trips.
/// This is the one place the two functions are deliberately not symmetric, so
/// the claim is worth pinning rather than asserting in prose alone.
/// A slice too short to encrypt must SAY so. The caller has already set the
/// container's encrypted flag by the time it calls this (the contract requires
/// flag-before-crypto, since the header is the key seed), so a silent no-op
/// leaves a unit advertised as encrypted while still carrying plaintext.
#[test]
fn encrypt_unit_reports_a_slice_too_short_to_encrypt() {
let key = [0x11u8; 16];
let mut short = vec![0u8; ALIGNED_UNIT_LEN - 1];
short[0] |= 0xC0; // the caller already flagged it encrypted
let before = short.clone();
assert!(
!encrypt_unit(&mut short, &key),
"a short slice must report false, not silently succeed"
);
assert_eq!(
short, before,
"a refused encrypt must leave the buffer untouched"
);
// Exactly ALIGNED_UNIT_LEN is the boundary and must succeed.
let mut exact = vec![0u8; ALIGNED_UNIT_LEN];
exact[0] |= 0xC0;
assert!(encrypt_unit(&mut exact, &key), "a full unit must encrypt");
}
#[test]
fn encrypt_unit_round_trips_all_zero_plaintext_packets() {
let key = [0xA5u8; 16];
let mut clear = vec![0u8; ALIGNED_UNIT_LEN];
clear[0] |= 0xC0; // flag before crypto
// Give packet 0 some content; leave every later packet entirely zero.
for (i, b) in clear[16..192].iter_mut().enumerate() {
*b = (i % 255 + 1) as u8;
}
let mut unit = clear.clone();
assert!(
encrypt_unit(&mut unit, &key),
"a full-length unit must encrypt"
);
// No later packet may encipher to all-zero, or decrypt would treat it as
// source padding and the asymmetry would bite.
for p in 1..ALIGNED_UNIT_LEN / BD_SOURCE_PACKET_BYTES {
let off = p * BD_SOURCE_PACKET_BYTES;
assert!(
unit[off..off + BD_SOURCE_PACKET_BYTES]
.iter()
.any(|&b| b != 0),
"packet {p} enciphered to all-zero, which decrypt reads as padding"
);
}
decrypt_unit(&mut unit, &key);
assert_eq!(unit, clear, "zero-payload packets must round trip exactly");
}
#[test]
fn test_aes_ecb_roundtrip() {
@@ -515,23 +686,11 @@ mod tests {
let original = vec![0x42u8; 128]; // 8 blocks
let mut data = original.clone();
// Encrypt with CBC manually (forward direction)
fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut prev = super::AACS_IV;
let num_blocks = data.len() / 16;
for i in 0..num_blocks {
let offset = i * 16;
for j in 0..16 {
data[offset + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&data[offset..offset + 16]);
cipher.encrypt_block(&mut block);
data[offset..offset + 16].copy_from_slice(&block);
prev.copy_from_slice(&data[offset..offset + 16]);
}
}
// Encrypt with the REAL production primitive. This test previously
// defined a local `fn aes_cbc_encrypt` that SHADOWED it, so it round-
// tripped a copy of the algorithm against itself and never exercised
// `crypto::aes_cbc_encrypt` at all — a mutation to the shipped function
// could not fail it.
aes_cbc_encrypt(&key, &mut data);
assert_ne!(data, original); // should be different after encrypt
@@ -566,7 +725,7 @@ mod tests {
}
// CBC encrypt bytes 16..6143
let cipher = Aes128::new(GenericArray::from_slice(&encrypt_key));
let cipher = Aes128::new(&encrypt_key.into());
let mut prev = AACS_IV;
let num_blocks = (ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..num_blocks {
@@ -574,7 +733,9 @@ mod tests {
for j in 0..16 {
plain[off + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&plain[off..off + 16]);
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&plain[off..off + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.encrypt_block(&mut block);
plain[off..off + 16].copy_from_slice(&block);
prev.copy_from_slice(&plain[off..off + 16]);
@@ -615,29 +776,13 @@ mod tests {
/// header) XOR header`, then CBC-encrypt bytes 16..6144 under the
/// fixed AACS IV.
fn aacs_encrypt_unit(unit: &mut [u8], unit_key: &[u8; 16]) {
// Set the CPI bits (top 2 of byte 0) so the unit reads as encrypted under
// `aacs_unit_encrypted` — done BEFORE key derivation so the plaintext
// header the real decrypt recovers matches what we encrypt under.
// Delegate to the module-scope `pub(crate)` helper (the single encrypt
// implementation, shared with the mux `driver.rs` decrypt test).
unit[0] |= 0xC0;
let header: [u8; 16] = unit[..16].try_into().unwrap();
let derived = aes_ecb_encrypt(unit_key, &header);
let mut k = [0u8; 16];
for i in 0..16 {
k[i] = derived[i] ^ header[i];
}
let cipher = Aes128::new(GenericArray::from_slice(&k));
let mut prev = AACS_IV;
let num_blocks = (ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..num_blocks {
let off = 16 + i * 16;
for j in 0..16 {
unit[off + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&unit[off..off + 16]);
cipher.encrypt_block(&mut block);
unit[off..off + 16].copy_from_slice(&block);
prev.copy_from_slice(&unit[off..off + 16]);
}
assert!(
super::encrypt_unit(unit, unit_key),
"a full-length unit must encrypt"
);
}
/// Build a clear aligned unit with TS sync bytes at offset 4 + k*192.
@@ -732,7 +877,7 @@ mod tests {
// ── Defect-tolerant "did a key OPEN this unit?" verdict ─────────────────
//
// The Bourne-UHD bug: a commercial disc carries the odd authored-bad TS
// The authored-bad-packet bug: a commercial disc carries the odd bad TS
// packet (a pressing/encoding defect, or an AACS 2.1 forensic-variant frame)
// — one non-conforming packet inside an otherwise perfectly-decrypted 6144
// unit. The OLD strict per-packet acceptance rejected the WHOLE unit over
@@ -1060,21 +1205,100 @@ mod tests {
assert_eq!(aes_ecb_decrypt(&key, &expected), pt);
}
// ── decrypt_bus: one key schedule per unit, not one per sector ─────────
/// MEASURED, not reasoned: `decrypt_bus` called `aes_cbc_decrypt` once per
/// 2048-byte sector, and each call built its own AES-128 key schedule, so a
/// 6144-byte aligned unit performed THREE key expansions under the same
/// loop-invariant `read_data_key`. On a 90 GB UHD read on a stock (non-
/// LibreDrive) drive — ~14.6 million aligned units — that is ~29 million
/// redundant expansions on the per-unit decrypt hot path, for a key that is
/// constant for the whole disc. The counter is incremented inside
/// `crypto::new_cipher`, the single construction site.
#[test]
fn decrypt_bus_expands_the_read_data_key_once_per_unit() {
use crate::aacs::crypto::KEY_EXPANSIONS;
let mut unit = clear_unit();
let rdk = [0x4Eu8; 16];
KEY_EXPANSIONS.with(|c| c.set(0));
decrypt_bus(&mut unit, &rdk);
let n = KEY_EXPANSIONS.with(|c| c.get());
assert_eq!(
n, 1,
"one aligned unit under one read_data_key must expand the schedule \
exactly once, not once per 2048-byte sector"
);
}
/// The single-expansion refactor must be byte-identical: bus encryption
/// ([C] §4.2) covers bytes 16..2048 of every 2048-byte sector, so a
/// three-sector aligned unit round-trips through the forward direction
/// sector by sector and `decrypt_bus` must recover it exactly.
#[test]
fn decrypt_bus_roundtrips_every_sector_region() {
let rdk = [0x91u8; 16];
let original = clear_unit();
let mut unit = original.clone();
// Forward direction, region by region — the inverse of decrypt_bus.
for start in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
crate::aacs::crypto::aes_cbc_encrypt(&rdk, &mut unit[start + 16..start + SECTOR_BYTES]);
}
assert_ne!(
&unit[16..64],
&original[16..64],
"the forward direction must have changed the bytes"
);
decrypt_bus(&mut unit, &rdk);
assert_eq!(
unit.as_slice(),
original.as_slice(),
"decrypt_bus must invert the per-sector bus encryption exactly"
);
}
// ── CBC decrypt: first-block uses fixed AACS IV ────────────────────────
/// The published `iv0` bytes, INDEPENDENT of the production constant.
///
/// [C] §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation.
/// Both IV tests below used to compute their expected value from
/// `crypto::AACS_IV` itself, so the constant was asserted against itself and
/// NOTHING in the suite pinned its bytes: swapping `AACS_IV` for `[0u8; 16]`
/// left both tests passing (one builds its ciphertext with the same value and
/// the other cancels the change in a triple XOR) while every real AACS disc
/// decrypted to noise — block 0 of every 6128-byte aligned unit and of every
/// bus-encrypted sector XORed with the wrong IV. This literal is the
/// independent witness the tests assert against.
const IV0_PUBLISHED: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F,
0x78,
];
/// Pins the fixed AACS CBC IV ([C] §2.1.2 `iv0`) against a literal, so a
/// change to `crypto::AACS_IV` fails HERE rather than silently shipping.
#[test]
fn aacs_iv_matches_published_iv0() {
assert_eq!(
AACS_IV, IV0_PUBLISHED,
"the fixed AACS CBC IV must be the published iv0"
);
}
#[test]
fn cbc_decrypt_first_block_xors_aacs_iv() {
// CBC: P[0] = AES-D(K, C[0]) XOR IV, and the IV is the fixed AACS
// constant (not zero). Encrypt a single block forward with IV, then
// confirm aes_cbc_decrypt recovers it — proving the IV used on block
// 0 is exactly AACS_IV. A mutation that swaps AACS_IV for [0u8;16]
// makes the recovered block wrong.
// constant (not zero). Encrypt a single block forward with the PUBLISHED
// iv0 literal, then confirm aes_cbc_decrypt recovers it — proving the IV
// the production code uses on block 0 is exactly that value. Building the
// fixture from `IV0_PUBLISHED` rather than from `AACS_IV` is what makes
// the claim real: a mutation that swaps AACS_IV for [0u8;16] now makes the
// recovered block wrong.
let key = [0x24u8; 16];
let plain = [0x5Au8; 16];
// Forward CBC for one block: C = AES-E(K, P XOR IV).
let mut x = plain;
for j in 0..16 {
x[j] ^= AACS_IV[j];
x[j] ^= IV0_PUBLISHED[j];
}
let ct = aes_ecb_encrypt(&key, &x);
let mut buf = ct;
@@ -1103,10 +1327,13 @@ mod tests {
// * Blocks 1..=3 are independent of the IV — they MUST equal the NIST
// plaintext byte-for-byte (P[i] = AES-D(K, C[i]) XOR C[i-1]). This
// pins the real reverse-order CBC chaining against a published KAT.
// * Block 0 = AES-D(K, C[0]) XOR AACS_IV = NIST_PT[0] XOR NIST_IV
// XOR AACS_IV — the documented IV substitution. Asserting this exact
// relation pins both the AES decrypt of C[0] AND that block 0 uses
// AACS_IV (a swap to [0u8;16] or a chaining bug fails it).
// * Block 0 = AES-D(K, C[0]) XOR iv0 = NIST_PT[0] XOR NIST_IV XOR iv0 —
// the documented IV substitution. Asserting this exact relation pins
// both the AES decrypt of C[0] AND that block 0 uses iv0. The expected
// value is built from the `IV0_PUBLISHED` literal, NOT from
// `crypto::AACS_IV`: computing it from the production constant made
// the change cancel out of the triple XOR, so a swap to [0u8;16] still
// passed. It now fails.
let key = [
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
@@ -1146,7 +1373,7 @@ mod tests {
// Block 0: NIST_PT[0] XOR NIST_IV XOR AACS_IV (the fixed-IV substitution).
let mut expected_block0 = [0u8; 16];
for i in 0..16 {
expected_block0[i] = nist_plaintext[i] ^ nist_iv[i] ^ AACS_IV[i];
expected_block0[i] = nist_plaintext[i] ^ nist_iv[i] ^ IV0_PUBLISHED[i];
}
assert_eq!(
&buf[0..16],
@@ -1271,7 +1498,7 @@ mod tests {
let plain = unit.clone();
// Forward: CBC-encrypt unit[s+16 .. s+2048] per sector under AACS IV.
let cipher = Aes128::new(GenericArray::from_slice(&rdk));
let cipher = Aes128::new(&rdk.into());
for s in (0..ALIGNED_UNIT_LEN).step_by(SECTOR_BYTES) {
let mut prev = AACS_IV;
let body = s + 16;
@@ -1282,7 +1509,9 @@ mod tests {
for j in 0..16 {
unit[off + j] ^= prev[j];
}
let mut blk = GenericArray::clone_from_slice(&unit[off..off + 16]);
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&unit[off..off + 16]);
let mut blk: Array<u8, _> = chunk.into();
cipher.encrypt_block(&mut blk);
unit[off..off + 16].copy_from_slice(&blk);
prev.copy_from_slice(&unit[off..off + 16]);
@@ -1339,6 +1568,95 @@ mod tests {
assert_eq!(ts_sync_count(&unit), 1);
}
// ── the encrypted-flag readers ────────────────────────────────────────
/// `aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the
/// guard that stops a truncated encrypted fragment from being emitted as
/// clear content. It reads ONLY the two Copy Permission Indicator bits
/// ([BD] §3.10.2, byte 0 bits 6-7); the remaining six bits are
/// `TP_extra_header` arrival-timestamp bits and carry no encryption
/// meaning.
///
/// Both failure directions are damaging and silent: a reader that answers
/// "encrypted" for a clear fragment discards good content, and one that
/// answers "clear" for an encrypted fragment writes ciphertext into the
/// output as if it were video.
#[test]
fn aacs_unit_seed_encrypted_reads_only_the_two_cpi_bits() {
use crate::disc::ContentFormat::BdTs;
// CPI bits clear → NOT encrypted, whatever the ATS bits say.
for ats in 0u8..=0x3F {
assert!(
!aacs_unit_seed_encrypted(&[ats], BdTs),
"byte0={ats:#04x} has both CPI bits clear → not encrypted"
);
}
// Either CPI bit set → encrypted, whatever the ATS bits say.
for &cpi in &[0x40u8, 0x80, 0xC0] {
assert!(
aacs_unit_seed_encrypted(&[cpi], BdTs),
"byte0={cpi:#04x} has a CPI bit set → encrypted"
);
assert!(
aacs_unit_seed_encrypted(&[cpi | 0x3F], BdTs),
"ATS bits must not change the answer"
);
}
// Too short to hold the flag → false rather than a panic.
assert!(!aacs_unit_seed_encrypted(&[], BdTs));
}
/// The MpegPs (HD-DVD `.evo`) side reads `PES_scrambling_control` at its own
/// fixed offset, and a fragment shorter than that offset must be reported
/// clear rather than panic.
#[test]
fn aacs_unit_seed_encrypted_reads_the_ps_scramble_flag_or_says_clear() {
use crate::disc::ContentFormat::MpegPs;
let mut frag = vec![0u8; PS_SCRAMBLE_OFF + 1];
assert!(!aacs_unit_seed_encrypted(&frag, MpegPs), "flag byte zero");
frag[PS_SCRAMBLE_OFF] = PS_SCRAMBLE_MASK;
assert!(aacs_unit_seed_encrypted(&frag, MpegPs), "flag byte set");
// Bits outside the mask are not the scrambling control.
frag[PS_SCRAMBLE_OFF] = !PS_SCRAMBLE_MASK;
assert!(!aacs_unit_seed_encrypted(&frag, MpegPs), "outside the mask");
// A fragment that stops short of the flag byte is not classifiable.
assert!(!aacs_unit_seed_encrypted(&frag[..PS_SCRAMBLE_OFF], MpegPs));
}
/// `aacs_unit_encrypted` is the AUTHORITATIVE gate and requires a WHOLE
/// 6144-byte aligned unit: on anything shorter the flag byte is not
/// guaranteed to be the unit's, so it must answer `false` and leave the
/// partial-unit case to `aacs_unit_seed_encrypted`. A reversed length guard
/// would both classify fragments off arbitrary mid-stream bytes and, on an
/// empty slice, index out of bounds.
#[test]
fn aacs_unit_encrypted_requires_a_whole_aligned_unit() {
use crate::disc::ContentFormat::BdTs;
// A short buffer whose byte 0 has the CPI bits set is still NOT a unit.
let mut short = vec![0u8; ALIGNED_UNIT_LEN - 1];
short[0] = 0xC0;
assert!(
!aacs_unit_encrypted(&short, BdTs),
"a sub-unit buffer must not be classified"
);
assert!(!aacs_unit_encrypted(&[], BdTs), "empty must not index");
// Exactly one aligned unit IS classified.
let mut unit = vec![0u8; ALIGNED_UNIT_LEN];
unit[0] = 0xC0;
assert!(
aacs_unit_encrypted(&unit, BdTs),
"a full unit with CPI set is encrypted"
);
unit[0] = 0x00;
assert!(!aacs_unit_encrypted(&unit, BdTs), "CPI clear is not");
}
#[test]
fn ts_packet_total_for_various_lengths() {
// total = len / 192 (BD-TS packet size). Pin a few lengths.
+175 -9
View File
@@ -8,17 +8,44 @@
//! content / keys / variant modules.
use aes::Aes128;
use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit, generic_array::GenericArray};
use aes::cipher::{Array, BlockCipherDecrypt, BlockCipherEncrypt, KeyInit};
/// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`).
pub(crate) const AACS_IV: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78,
];
// Per-thread count of AES-128 key schedules built through `new_cipher`.
// Test-only instrumentation: an AES-128 key expansion is 10 round-key
// derivations, and the CBC helpers here run on the per-aligned-unit decrypt hot
// path of a whole disc read, so "how many times was the schedule built for one
// loop-invariant key" is a property worth asserting rather than reasoning about.
// THREAD-LOCAL, not a global atomic: `cargo test` runs tests concurrently, so a
// shared counter would see every other test's expansions. See
// `content::tests::decrypt_bus_expands_the_read_data_key_once_per_unit`.
#[cfg(test)]
thread_local! {
pub(crate) static KEY_EXPANSIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Build an AES-128 key schedule for a caller that will drive
/// [`cbc_decrypt_blocks`] over several regions under one key.
pub(crate) fn new_cipher_for(key: &[u8; 16]) -> Aes128 {
new_cipher(key)
}
/// Build an AES-128 key schedule. The single construction site for the CBC
/// helpers, so [`KEY_EXPANSIONS`] can count them under test.
fn new_cipher(key: &[u8; 16]) -> Aes128 {
#[cfg(test)]
KEY_EXPANSIONS.with(|c| c.set(c.get() + 1));
Aes128::new(&(*key).into())
}
/// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`).
pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
cipher.encrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
@@ -27,25 +54,81 @@ pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
/// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`).
pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
cipher.decrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
out
}
/// AES-128-CBC decrypt in-place with the fixed AACS IV. [C] §2.1.2 (`AES-128CBCD`).
/// AES-128-CBC ENCRYPT in place under the fixed [`AACS_IV`] — the forward
/// direction of [`aes_cbc_decrypt`], and its exact inverse. [C] §2.1.2
/// (`AES-128CBCE`).
///
/// Precondition: `data.len()` is a multiple of 16; the assert
/// documents/enforces that contract.
///
/// Constructs the cipher ONCE for the whole slice. Driving this from the
/// single-block [`aes_ecb_encrypt`] instead rebuilds the AES key schedule per
/// 16-byte block, which for a 6144-byte aligned unit is 383 redundant key
/// expansions.
pub(crate) fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len().is_multiple_of(16),
"aes_cbc_encrypt requires a block-aligned slice"
);
let cipher = new_cipher(key);
let num_blocks = data.len() / 16;
let mut prev = AACS_IV;
// Forward order: each block is XORed with the PRECEDING ciphertext block.
for i in 0..num_blocks {
let offset = i * 16;
let mut block = [0u8; 16];
for j in 0..16 {
block[j] = data[offset + j] ^ prev[j];
}
let mut ga: Array<u8, _> = block.into();
cipher.encrypt_block(&mut ga);
data[offset..offset + 16].copy_from_slice(&ga);
prev.copy_from_slice(&ga);
}
}
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. [C] §2.1.2
/// (`AES-128CBCD`).
///
/// Precondition: `data.len()` is a multiple of 16. Any trailing partial
/// block is silently ignored; all callers pass aligned regions (6128 and
/// 2032 bytes), and the assert documents/enforces that contract.
///
/// (This doc block was orphaned onto `aes_cbc_encrypt` above when that function
/// was inserted directly after it with no separating blank line, so rustdoc
/// rendered the crate's only forward-direction AACS primitive as "decrypt" and
/// cited the spec's DECRYPT clause for it, while this function had no doc at
/// all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in `content.rs` pins
/// the directions behaviourally so a maintainer 'fixing' the contradiction by
/// swapping the two bodies fails the suite instead of shipping a second
/// decryptor behind an already-set encrypted flag.)
pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len() % 16 == 0,
data.len().is_multiple_of(16),
"aes_cbc_decrypt requires a block-aligned slice"
);
let cipher = Aes128::new(GenericArray::from_slice(key));
cbc_decrypt_blocks(&new_cipher(key), data);
}
/// AES-128-CBC decrypt in place under the fixed [`AACS_IV`] with an ALREADY
/// EXPANDED key schedule.
///
/// Split out of [`aes_cbc_decrypt`] so a caller that decrypts several regions
/// under one loop-invariant key expands the schedule once. `decrypt_bus`
/// ([`super::content::decrypt_bus`]) is that caller: bus encryption
/// ([C] §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// 2048-byte sector, so a 6144-byte aligned unit is three regions under one
/// `read_data_key` — three key schedules where one suffices, on the per-unit
/// decrypt hot path of a whole 90 GB read.
pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
let num_blocks = data.len() / 16;
// Process blocks in reverse to avoid clobbering ciphertext needed for XOR
for i in (0..num_blocks).rev() {
@@ -57,7 +140,9 @@ pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
p.copy_from_slice(&data[(i - 1) * 16..i * 16]);
p
};
let mut block = GenericArray::clone_from_slice(&data[offset..offset + 16]);
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&data[offset..offset + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.decrypt_block(&mut block);
for j in 0..16 {
data[offset + j] = block[j] ^ prev[j];
@@ -100,3 +185,84 @@ pub(crate) fn aesg3(key: &[u8; 16], inc: u8) -> [u8; 16] {
}
out
}
#[cfg(test)]
mod tests {
use super::*;
/// The AACS-G3 seed `s0`, transcribed independently from [C] §3.2.2 rather
/// than read from [`AESG3_SEED`] — a test that sourced the seed from the
/// production constant would assert that constant against itself and would
/// still pass if it were edited.
const S0: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B,
0xD9,
];
/// An arbitrary non-degenerate key. Nothing about it is secret or special;
/// the AES-G3 relation holds for every key, and a constant-returning body
/// cannot satisfy it for any.
const K: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2, 0xE1,
0xF0,
];
/// `aesg3` is the node function of the AACS subset-difference tree: every
/// Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every
/// descent step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that
/// returned a fixed block would make every device key in the crate derive
/// the SAME Processing Key, and a `^` that became `|` or `&` would derive a
/// wrong-but-plausible one — in both cases the MKB walk simply stops
/// finding Media Keys, with no error to say why.
///
/// Pinned through the spec relation rather than a re-implementation:
/// [C] §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `s = s0 + inc` (added into the last seed byte), so applying the
/// FORWARD primitive [`aes_ecb_encrypt`] — a different function from the
/// one under test — to `aesg3(k, inc) XOR s` must reproduce `s` exactly.
#[test]
fn aesg3_inverts_to_the_spec_seed_under_aes_encrypt() {
for inc in 0u8..=2 {
let mut seed = S0;
seed[15] = seed[15].wrapping_add(inc);
let out = aesg3(&K, inc);
// out == AES-128D(K, seed) XOR seed, so out XOR seed is the raw
// decryption and re-encrypting it must land back on the seed.
let mut pre = [0u8; 16];
for i in 0..16 {
pre[i] = out[i] ^ seed[i];
}
assert_eq!(
aes_ecb_encrypt(&K, &pre),
seed,
"AES-G3 inc={inc} must satisfy out = AES-128D(k, s0+inc) XOR (s0+inc)"
);
}
}
/// The Triple Generator's three outputs ([C] §3.2.2: left = inc 0, the
/// Processing Key = inc 1, right = inc 2) are the two child node keys and
/// the Processing Key of ONE tree node. They must be three different keys —
/// if `inc` were ignored, a descent would revisit its own parent and the
/// walk would derive the same key at every level of the tree.
#[test]
fn aesg3_yields_three_distinct_subkeys_for_the_three_increments() {
let left = aesg3(&K, 0);
let pk = aesg3(&K, 1);
let right = aesg3(&K, 2);
assert_ne!(left, pk, "left child and Processing Key must differ");
assert_ne!(pk, right, "Processing Key and right child must differ");
assert_ne!(left, right, "left and right children must differ");
}
/// Distinct parent keys must yield distinct subkeys — the tree would
/// collapse otherwise.
#[test]
fn aesg3_separates_distinct_parent_keys() {
let mut other = K;
other[0] ^= 0x01;
assert_ne!(aesg3(&K, 1), aesg3(&other, 1));
}
}
+1044
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -189,7 +189,7 @@ mod tests {
let unit_packets = (ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32; // 32
// Start so the unit covers [80, 80+31] = [80, 111]: overlaps at 100.
let off = 80u64 * SOURCE_PACKET_LEN;
assert!(80 + unit_packets - 1 >= 100, "sanity: unit tails into seg");
assert!(80 + unit_packets > 100, "sanity: unit tails into seg");
assert_eq!(
unit_disposition(off, &segs, Some(5)),
UnitDisposition::Index(5)
+382 -1
View File
@@ -5,7 +5,6 @@
use super::mkb::*;
/// Parsed Unit_Key_RO.inf file.
#[derive(Debug)]
pub struct UnitKeyFile {
/// Disc hash (SHA1 of the entire file) — used as KEYDB lookup key
pub disc_hash: [u8; 20],
@@ -23,6 +22,29 @@ pub struct UnitKeyFile {
pub title_cps_unit: Vec<u16>,
}
/// Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
/// the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
/// — plus the disc hash they are looked up by. A derived `Debug` printed every key
/// byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
/// message, a future `tracing::debug!` in this module) leaked them. Only
/// non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
impl std::fmt::Debug for UnitKeyFile {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKeyFile")
// The disc hash is the public keydb lookup key, printed as hex the
// same way `DiscEntry` prints its own — never as raw bytes.
.field("disc_hash", &disc_hash_hex(&self.disc_hash))
.field("app_type", &self.app_type)
.field("num_bdmv_dir", &self.num_bdmv_dir)
.field("use_skb_mkb", &self.use_skb_mkb)
.field("version", &self.version)
.field("encrypted_keys", &"<redacted>")
.field("encrypted_keys_len", &self.encrypted_keys.len())
.field("title_cps_unit", &self.title_cps_unit)
.finish()
}
}
/// Compute disc hash (SHA1 of Unit_Key_RO.inf content).
pub fn disc_hash(data: &[u8]) -> [u8; 20] {
use sha1::{Digest, Sha1};
@@ -497,4 +519,363 @@ mod vtkf_tests {
// Same as applying the shared unwrap directly to the stored enc key.
assert_eq!(derived, super::super::derive::decrypt_unit_key(&vuk, &enc));
}
/// `UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
/// printed every byte; the hand-written impl must not. Sentinel key byte
/// 0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
/// same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
/// `#[derive(Debug)]` back fails this.
#[test]
fn unit_key_file_debug_is_redacted() {
let f = UnitKeyFile {
disc_hash: [0xD5; 20],
app_type: 1,
num_bdmv_dir: 1,
use_skb_mkb: false,
version: AacsVersion::V20,
encrypted_keys: vec![(0, [0xD5; 16]), (1, [0xD5; 16])],
title_cps_unit: vec![0, 1],
};
let dbg = format!("{f:?}");
assert!(
!dbg.contains("213"),
"UnitKeyFile Debug leaked key bytes (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"UnitKeyFile Debug missing redaction marker: {dbg}"
);
// Non-secret shape is still useful for diagnostics.
assert!(dbg.contains("encrypted_keys_len: 2"), "{dbg}");
}
}
#[cfg(test)]
mod read_mkb_tests {
use super::*;
use crate::scsi::{DataDirection, SCSI_READ_DISC_STRUCTURE, ScsiResult, ScsiTransport};
/// A drive that answers READ DISC STRUCTURE format 0x83 from a scripted set
/// of packs and records every CDB it was handed.
struct MkbDrive {
/// One entry per pack: the pack's MKB payload bytes.
packs: Vec<Vec<u8>>,
cdbs: Vec<Vec<u8>>,
}
impl ScsiTransport for MkbDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
self.cdbs.push(cdb.to_vec());
// Pack number is carried in the CDB address field (bytes 2..6),
// MMC-6 READ DISC STRUCTURE.
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]) as usize;
let body = self.packs.get(pack).cloned().unwrap_or_default();
// Header: BE16 data length (counts the 2 header bytes that follow
// it plus the payload), reserved byte, pack count, then payload.
let data_len = body.len() + 2;
data[0..2].copy_from_slice(&(data_len as u16).to_be_bytes());
data[2] = 0x00;
data[3] = self.packs.len() as u8;
data[4..4 + body.len()].copy_from_slice(&body);
Ok(ScsiResult {
status: 0,
bytes_transferred: 4 + body.len(),
sense: [0u8; 32],
})
}
}
/// `read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
/// downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
/// Media Key ladder) consumes exactly what it returns. An empty return is
/// not a benign "no MKB" — it is a total read failure reported as success,
/// and every derivation then fails with a key-not-found code that points
/// the operator at their keydb rather than at the drive.
///
/// This pins the CONTENT: the concatenated payload of all packs, in pack
/// order, byte for byte.
#[test]
fn read_mkb_from_drive_returns_the_concatenated_pack_payload() {
let pack0: Vec<u8> = (0..600u32).map(|i| (i % 251) as u8).collect();
let pack1: Vec<u8> = (0..300u32).map(|i| (i % 253) as u8 ^ 0xA5).collect();
let mut drive = MkbDrive {
packs: vec![pack0.clone(), pack1.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
let mut expected = pack0.clone();
expected.extend_from_slice(&pack1);
assert_eq!(
mkb.len(),
expected.len(),
"every pack's payload must be concatenated, none dropped"
);
assert!(
mkb == expected,
"MKB bytes must be the drive's payload in pack order; first \
mismatch at {:?}",
(0..expected.len()).find(|&i| mkb[i] != expected[i])
);
// MMC-6 READ DISC STRUCTURE with the AACS MKB format code, one command
// per pack, pack number in the address field.
assert_eq!(drive.cdbs.len(), 2, "one command per declared pack");
for (i, cdb) in drive.cdbs.iter().enumerate() {
assert_eq!(cdb[0], SCSI_READ_DISC_STRUCTURE, "opcode");
assert_eq!(cdb[7], 0x83, "AACS MKB disc-structure format code");
assert_eq!(
u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]),
i as u32,
"pack {i} must be requested by number"
);
}
}
/// The CDB is what the drive actually acts on, and every byte of it is
/// load-bearing: a wrong format code returns a different disc structure
/// entirely, and a wrong allocation length truncates the pack. The existing
/// test above pins the opcode, the format code and the pack number; this
/// pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
///
/// Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
/// `[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
/// (BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
/// BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
#[test]
fn read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack() {
let mut drive = MkbDrive {
packs: vec![vec![0x11u8; 64], vec![0x22u8; 64], vec![0x33u8; 64]],
cdbs: Vec::new(),
};
read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(drive.cdbs.len(), 3, "one command per declared pack");
for (pack, cdb) in drive.cdbs.iter().enumerate() {
let p = pack as u32;
let expected: [u8; 12] = [
SCSI_READ_DISC_STRUCTURE,
0x01,
(p >> 24) as u8,
(p >> 16) as u8,
(p >> 8) as u8,
p as u8,
0x00,
0x83, // AACS MKB disc-structure format
0x80, // allocation length 32772 = 0x8004, high byte
0x04, // …low byte
0x00,
0x00,
];
assert_eq!(
cdb.as_slice(),
&expected[..],
"CDB for pack {pack} must match the MMC-6 READ DISC STRUCTURE layout"
);
}
}
/// A pack payload filling the FULL 32768-byte window must come back whole.
/// The `len > 0 && len <= 32768` bound is what stands between a maximal
/// pack and a silently dropped one, and the small payloads used elsewhere
/// in this module never reach it.
#[test]
fn read_mkb_from_drive_accepts_a_full_size_pack() {
let full: Vec<u8> = (0..32768u32).map(|i| (i % 251) as u8).collect();
let other: Vec<u8> = (0..32768u32).map(|i| (i % 241) as u8 ^ 0x5A).collect();
// TWO maximal packs: the first-pack read and the per-pack loop carry
// separate bounds, so both must accept a full-window payload.
let mut drive = MkbDrive {
packs: vec![full.clone(), other.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
65536,
"neither maximal pack may be dropped at the size bound"
);
let mut expected = full.clone();
expected.extend_from_slice(&other);
assert!(mkb == expected, "both maximal packs' bytes must be intact");
}
/// A pack that declares only the 2-byte header and NO payload contributes
/// nothing, and must not push a phantom byte into the MKB — an off-by-one
/// at the zero-length boundary corrupts every following pack's alignment.
#[test]
fn read_mkb_from_drive_zero_length_pack_contributes_nothing() {
let mut drive = MkbDrive {
packs: vec![Vec::new(), vec![0xABu8; 32]],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
32,
"an empty pack adds no bytes; only pack 1's payload is present"
);
assert!(mkb == vec![0xABu8; 32], "and the bytes are pack 1's");
}
/// A drive that DECLARES more payload than it returned must not be
/// believed. The BE16 length in the response header is drive-supplied data:
/// a firmware bug, a short transfer, or a hostile device can put a value in
/// it that runs past the 32772-byte buffer. Copying `len` bytes on that word
/// alone panics the rip thread mid-scan.
///
/// Both the first-pack read and the per-pack loop carry the same bound, so
/// both are exercised here: the over-declared pack contributes nothing and
/// the honest pack still comes through.
#[test]
fn read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds() {
/// Pack 0 is honest; pack 1 declares a 60000-byte payload it never sent.
struct LyingDrive {
honest: Vec<u8>,
}
impl ScsiTransport for LyingDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]);
data[3] = 2; // two packs declared
if pack == 0 {
let dl = self.honest.len() + 2;
data[0..2].copy_from_slice(&(dl as u16).to_be_bytes());
data[4..4 + self.honest.len()].copy_from_slice(&self.honest);
} else {
// A length far beyond the 32772-byte response buffer.
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
}
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let honest = vec![0xC7u8; 256];
let mut drive = LyingDrive {
honest: honest.clone(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("an over-declared pack is not an error");
assert_eq!(
mkb.len(),
honest.len(),
"only the honest pack's bytes may be taken; the over-declared pack \
contributes nothing and must not be read past the buffer"
);
assert!(mkb == honest, "and those bytes are pack 0's");
}
/// The same over-declaration on the FIRST pack, which uses a separate bound
/// from the loop's.
#[test]
fn read_mkb_from_drive_ignores_a_first_pack_declaring_more_than_the_buffer() {
struct LyingFirst;
impl ScsiTransport for LyingFirst {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
data[3] = 1;
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut LyingFirst).expect("not an error");
assert!(
mkb.is_empty(),
"a first pack declaring more than the buffer holds yields no bytes"
);
}
/// A single-pack disc still yields that pack's bytes — the common case, and
/// the one where a body returning an empty vector looks most plausible.
#[test]
fn read_mkb_from_drive_returns_a_single_packs_payload() {
let pack: Vec<u8> = (0..1024u32).map(|i| (i * 7 % 256) as u8).collect();
let mut drive = MkbDrive {
packs: vec![pack.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(mkb.len(), pack.len(), "single pack payload length");
assert!(mkb == pack, "single pack payload bytes");
}
/// A drive that reports a header-only response (`data_len < 2`) has no MKB
/// to give. That must be an EMPTY vec, not a partial one — the distinction
/// matters because the AACS paths treat a non-empty MKB as parseable.
#[test]
fn read_mkb_from_drive_empty_response_is_empty() {
struct NoMkb;
impl ScsiTransport for NoMkb {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&0u16.to_be_bytes());
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut NoMkb).expect("no-MKB drive still returns Ok");
assert!(
mkb.is_empty(),
"a header-only response carries no MKB bytes"
);
}
/// A transport failure on the FIRST pack must propagate as an error — the
/// MKB is the root of the whole AACS ladder, so an unreadable one cannot be
/// downgraded to "an MKB with no records".
#[test]
fn read_mkb_from_drive_propagates_the_first_pack_failure() {
struct DeadDrive;
impl ScsiTransport for DeadDrive {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
Err(crate::error::Error::ScsiError {
opcode: SCSI_READ_DISC_STRUCTURE,
status: 0x02,
sense: None,
})
}
}
assert!(
read_mkb_from_drive(&mut DeadDrive).is_err(),
"an unreadable MKB must surface as an error, not an empty MKB"
);
}
}
+105
View File
@@ -432,4 +432,109 @@ mod tests {
"trim keeps the framed records, dropping the end marker and padding"
);
}
// ── BE24 length field: all THREE bytes ────────────────────────────────
/// The record length is a big-endian **24-bit** field, so the high byte
/// carries lengths of 64 KiB and up. The MKB records that matter most are
/// exactly that size — a real UHD cvalue table is `46_101 * 16` bytes and a
/// `0x2d` variant record is ~92 KiB — so a walker that dropped the high
/// byte would mis-frame every record of a real MKB from the first big one
/// onward, and every downstream key lookup would read the wrong bytes.
///
/// (The pre-existing high-byte test used total length `0x0110`, whose high
/// byte is ZERO — it exercised the middle byte only. This one puts a
/// non-zero value in the high byte.)
#[test]
fn mkb_records_honors_the_high_byte_of_the_be24_length() {
const TOTAL: usize = 0x0001_0004; // 65_540 — high byte 0x01
let mut mkb = vec![REC_VKD_TABLE, 0x01, 0x00, 0x04];
mkb.resize(TOTAL, 0xAB);
// A second record follows, so a walker that mis-read the length would
// frame a different number of records rather than merely a short one.
mkb.extend(rec(REC_TYPE_AND_VERSION, &[0x11; 8]));
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the big record must be framed as ONE record");
assert_eq!(
recs[0].rec_len, TOTAL,
"rec_len must include the high BE24 byte"
);
assert_eq!(recs[0].body.len(), TOTAL - 4);
assert_eq!(
recs[1].rec_type, REC_TYPE_AND_VERSION,
"the following record must start where the big one ends"
);
}
// ── Header-only records and the exact end marker ──────────────────────
/// `rec_len == 4` is a well-formed HEADER-ONLY record (the minimum the
/// walker accepts), including one sitting at the very end of the buffer
/// with no bytes after it. Rejecting either — the `pos + 4` bound or the
/// `rec_len < 4` floor being off by one — silently drops the MKB's last
/// record, and "the record isn't there" is indistinguishable from "the disc
/// doesn't carry it".
#[test]
fn mkb_records_yields_a_header_only_record_at_the_buffer_end() {
let mut mkb = rec(REC_TYPE_AND_VERSION, &[0xAA, 0xBB]);
mkb.extend([REC_VKD_TABLE, 0x00, 0x00, 0x04]); // 4-byte, empty body, at EOF
assert_eq!(
mkb.len(),
10,
"sanity: the last record ends at the buffer end"
);
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the trailing header-only record is a record");
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].rec_len, 4);
assert!(recs[1].body.is_empty());
}
/// ONLY the exact `00 00 00 00` marker ends the walk. A record whose TYPE
/// happens to be `0x00` but which declares a real length is a record, not
/// the end of the MKB — stopping there would truncate everything after it,
/// including the cvalue and verify records the key derivation needs.
#[test]
fn mkb_records_stops_only_on_the_all_zero_end_marker() {
// A type-0 record of length 8, then a normal record, then the marker.
let mut mkb = vec![0x00, 0x00, 0x00, 0x08, 1, 2, 3, 4];
mkb.extend(rec(REC_VKD_TABLE, &[0x55; 16]));
mkb.extend([0x00, 0x00, 0x00, 0x00]); // the real end marker
mkb.extend(rec(0x99, &[0xFF; 4])); // past the marker: not walked
let recs = walk_mkb(&mkb);
assert_eq!(
recs.len(),
2,
"a type-0 record with a non-zero length is a record, not the end"
);
assert_eq!(recs[0].rec_type, 0x00);
assert_eq!(recs[0].rec_len, 8);
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].body, vec![0x55; 16]);
}
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim ([C] §3.2.5.1.1
/// Table 3-2), including a value this build does not recognise — the caller
/// uses it to tell "unknown MKB generation" from "no Type record at all".
/// All four bytes must come from the record body; reading any of them from
/// the wrong offset yields a type that silently classifies as a different
/// AACS generation.
///
/// The recognised constants all share bytes with the `0x10` record-type
/// header byte (e.g. `MKB_21_CATEGORY_C` is `48 15 10 03`), so this uses a
/// value with four distinct bytes, none of them `0x10`.
#[test]
fn mkb_type_raw_reads_all_four_body_bytes() {
const RAW: u32 = 0xDEAD_BEEF;
let mkb = type_and_version(RAW, 7);
assert_eq!(
mkb_type_raw(&mkb),
Some(RAW),
"every byte of the MKBType field must come from the record body"
);
assert_eq!(mkb_version(&mkb), Some(7));
}
}
+54
View File
@@ -313,6 +313,60 @@ mod tests {
);
}
/// The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required —
/// the discovery is a conjunction, not a disjunction.
///
/// The existing fixtures only ever present a directory that satisfies both
/// (`AAC!` with `MKBROM.AACS`) alongside one that satisfies neither
/// (`AAC!_BAK` — which contains `MKBROM.AACS` but is ALSO reached only after
/// the real dir), so either half of the conjunction could be dropped and the
/// same directory would still be found. Here a directory satisfies the name
/// half and NOT the contents half: it must not be picked.
///
/// If it were, the HD DVD path would resolve `MKBROM.AACS`,
/// `CONTENT_CERT.AACS` and the title-key file under a directory that holds
/// none of them — the disc reports "no AACS key files" and never rips.
#[test]
fn a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
// Ends in '!' — but carries no MKBROM.AACS, so it is not the
// HD DVD AACS directory.
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTKF090.AACS", 102, 5200, 2048, true),
file("CONTENT_CERT.AACS", 103, 5300, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
super::find_hddvd_aacs_dir(&udf).is_none(),
"a '!' directory without MKBROM.AACS is not the AACS directory"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
],
"no HD DVD candidates may be appended from a directory that was \
never identified as the AACS directory"
);
}
#[test]
fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() {
use crate::udf::fixture::*;
+47
View File
@@ -197,6 +197,17 @@ mod tests {
}
}
/// A host cert whose (non-secret) certificate body and private key are both
/// filled with `byte`, so a cert is identifiable in an aggregated list.
fn cert(byte: u8) -> HostCert {
HostCert {
private_key: [byte; 20],
certificate: vec![byte; 92],
private_key_v2: None,
certificate_v2: None,
}
}
fn dk(byte: u8, node: u16) -> DeviceKey {
DeviceKey {
key: [byte; 16],
@@ -357,6 +368,42 @@ mod tests {
assert_eq!(got.disc_hash, "vid-a");
}
/// `Providers::host_certs` is the union across the provider array. It is not
/// wired into the handshake today (see the module docs), so nothing else in
/// the crate would notice a body that dropped every cert on the floor — and
/// the day it IS wired in, a silently-empty cert list means the drive AACS
/// authentication finds no host certificate to present and every disc fails
/// to open, with no indication that the caller's certs were discarded.
///
/// Unlike the bulk key unions this one does NOT dedup (HostCert is not
/// Ord/Hash), so the assertion is on the full concatenation in array order.
#[test]
fn providers_host_certs_unions_every_providers_certs_in_array_order() {
struct Certs(Vec<HostCert>);
impl KeyProvider for Certs {
fn host_certs(&self) -> Vec<HostCert> {
self.0.clone()
}
}
// Distinguish certs by their (non-secret) certificate body, so the
// assertion lands on WHICH certs came back, not merely how many.
let a = Certs(vec![cert(0xA1), cert(0xA2)]);
let b = Certs(vec![cert(0xB1)]);
let arr: &[&dyn KeyProvider] = &[&a, &b];
let got = Providers(arr).host_certs();
let bodies: Vec<Vec<u8>> = got.iter().map(|c| c.certificate.clone()).collect();
assert_eq!(
bodies,
vec![vec![0xA1u8; 92], vec![0xA2u8; 92], vec![0xB1u8; 92]],
"every provider's certs must survive the union, in array order"
);
// The private key travels with the cert — a union that returned default
// certs would still have the right count.
assert_eq!(got[0].private_key, [0xA1u8; 20]);
assert_eq!(got[2].private_key, [0xB1u8; 20]);
}
#[test]
fn providers_empty_array_yields_nothing() {
let arr: &[&dyn KeyProvider] = &[];
+171 -16
View File
@@ -379,26 +379,18 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
// One AES-D + magic check per candidate (cheap). mk_dv is hoisted
// out of the loop so the MKB is not re-walked per candidate.
let mks = providers.media_keys();
let mut mk_hits: Vec<[u8; 16]> = Vec::new();
if let Some(mk_dv) = mkb_find_mk_dv(mkb) {
for mk in &mks {
let verifies = aes_ecb_decrypt(mk, &mk_dv)[..8]
== [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
if verifies && !mk_hits.contains(mk) {
mk_hits.push(*mk);
if mk_hits.len() > 1 {
break; // ambiguous — bail to avoid a wrong key
}
}
}
}
if mk_hits.len() == 1 {
let vuk = derive_vuk(&mk_hits[0], ctx.volume_id);
let chosen_mk = mkb_find_mk_dv(mkb).and_then(|mk_dv| {
unique_verifying_mk(&mks, |mk| {
aes_ecb_decrypt(mk, &mk_dv)[..8] == MK_VERIFY_MAGIC
})
});
if let Some(mk) = chosen_mk {
let vuk = derive_vuk(&mk, ctx.volume_id);
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_hit", mk_pool = mks.len(), "media key from keydb MK-pool brute (km_verifies)");
// Same class as path 3 (KEYDB MK → derived VUK).
return Some(build(Some(vuk), derive_uks(&vuk), 3));
}
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_miss", mk_pool = mks.len(), mk_hits = mk_hits.len(), "MK-pool brute: no unique verifying MK");
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_miss", mk_pool = mks.len(), "MK-pool brute: no unique verifying MK");
} else {
tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_no_mkb", "no MKB; paths 1/2 skipped");
}
@@ -450,6 +442,42 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
None
}
/// First 8 bytes of the plaintext behind an MKB Verify Media Key record — the
/// AACS "this is the right Km" sentinel (`0123456789ABCDEF`). A candidate MK
/// verifies when AES-128-ECB-D(mk, mk_dv) starts with it.
const MK_VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
/// The MK-pool selection rule of path 2.5, split out of [`resolve_keys_v1`] so
/// the ambiguity guard has a reachable test.
///
/// `verifies` is the MKB check — in production
/// `AES-D(mk, mk_dv)[..8] == MK_VERIFY_MAGIC`. Returns a Media Key only when
/// EXACTLY ONE DISTINCT candidate passes. Duplicates of the same key are one
/// candidate (a pool aggregated across providers routinely repeats a key), but
/// two DIFFERENT keys that both verify mean the pool cannot say which is this
/// disc's Km: picking either derives a wrong VUK, and a wrong VUK decrypts to
/// plausible-looking garbage rather than failing loudly. Bail and let the
/// later hash/VID paths answer instead.
///
/// The predicate is a parameter rather than the inlined AES check because a
/// genuine two-key multi-hit cannot be synthesised: it needs one ciphertext
/// that decrypts under two distinct AES-128 keys to plaintexts sharing a
/// 64-bit prefix — a 2^64 search. Injecting the verifier is the only way the
/// ambiguity branch is reachable from a test at all.
fn unique_verifying_mk(mks: &[[u8; 16]], verifies: impl Fn(&[u8; 16]) -> bool) -> Option<[u8; 16]> {
let mut hits: Vec<[u8; 16]> = Vec::new();
for mk in mks {
if verifies(mk) && !hits.contains(mk) {
hits.push(*mk);
if hits.len() > 1 {
// Ambiguous — bail rather than pick a Media Key.
return None;
}
}
}
hits.first().copied()
}
/// For path 5: cross-reference the disc's `Unit_Key_RO.inf` CPS-unit
/// numbering against the KEYDB entry's pre-decrypted unit keys. Every
/// CPS unit the disc declares must have a matching entry in KEYDB;
@@ -1300,6 +1328,62 @@ mod tests {
"VUK must derive from the verified Km + this disc's VID"
);
}
/// Path 2.5's ambiguity guard: when MORE THAN ONE DISTINCT pooled Media Key
/// verifies against the MKB, the resolver must return no key at all rather
/// than pick one. A wrong Km derives a wrong VUK, and a wrong VUK does not
/// fail loudly — it decrypts the title to garbage that muxes and plays as a
/// corrupt rip.
///
/// The real MKB check cannot be forced into a multi-hit: two distinct
/// AES-128 keys decrypting one `mk_dv` to plaintexts that share the 64-bit
/// verify magic is a 2^64 search, not a fixture. So the rule is tested
/// through `unique_verifying_mk`, whose verifier is a parameter — the same
/// function `resolve_keys_v1` calls, with the same pool semantics.
#[test]
fn mk_pool_ambiguity_bails_rather_than_picking_a_media_key() {
let a = [0xAAu8; 16];
let b = [0xBBu8; 16];
let c = [0xCCu8; 16];
// One verifying candidate → that key.
assert_eq!(
unique_verifying_mk(&[a, b, c], |mk| *mk == b),
Some(b),
"a single verifying MK resolves"
);
// The SAME key repeated across providers is one candidate, not an
// ambiguity — the dedup (`!hits.contains`) must keep this resolvable.
assert_eq!(
unique_verifying_mk(&[b, b, b], |mk| *mk == b),
Some(b),
"duplicates of one key are not ambiguity"
);
// TWO DISTINCT verifying candidates → bail, no key.
assert_eq!(
unique_verifying_mk(&[a, b], |mk| *mk == a || *mk == b),
None,
"two distinct verifying MKs must yield NO key, not the first one"
);
// Ambiguity must still be detected when the second hit is last in the
// pool, i.e. the scan may not stop at the first hit.
assert_eq!(
unique_verifying_mk(&[a, c, [0u8; 16], b], |mk| *mk == a || *mk == b),
None,
"a late second hit is still ambiguous"
);
// Every candidate verifying is the degenerate ambiguous case.
assert_eq!(unique_verifying_mk(&[a, b, c], |_| true), None);
// No candidate verifies → no key (and no panic on an empty pool).
assert_eq!(unique_verifying_mk(&[a, b, c], |_| false), None);
assert_eq!(unique_verifying_mk(&[], |_| true), None);
}
#[test]
fn test_content_cert_parse() {
// AACS 1.0 cert, bus encryption OFF. Content-cert layout: flag in
@@ -1822,6 +1906,77 @@ mod tests {
assert_eq!(r.vuk, Some(derive_vuk(&mk, &vid)));
}
/// `resolve_keys_v21` gates paths 1 and 3 on `has_vid`, and an all-zero
/// Volume ID is the crate's "the VID was never read" sentinel — the SCSI
/// handshake leaves the buffer zeroed when it does not run or fails.
///
/// Both directions matter and both fail silently:
/// - treating the zero sentinel as a real VID runs path 3 and derives
/// `Kvu = AES-G(Km, 0…0)`, a perfectly well-formed but WRONG VUK. It
/// unwraps the title keys to garbage, and nothing downstream errors —
/// the rip just decodes to noise.
/// - treating a real VID as absent skips paths 1 and 3 entirely, so a
/// disc that could have been resolved from its Media Key reports no key.
///
/// Asserted through the final VUK, not through the flag.
#[test]
fn resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid() {
let uk_ro = minimal_unit_key_ro();
let vid = [0x42u8; 16];
let mk = [0x24u8; 16];
// A VID-keyed entry carrying an MK and nothing else: no VUK and no unit
// keys, so paths 4 and 5 cannot fire and ONLY the VID-gated path 3 can
// produce a result.
let entry = DiscEntry {
disc_hash: "not-this-disc".to_string(),
title: "sibling".to_string(),
media_key: Some(mk),
disc_id: Some(vid),
vuk: None,
unit_keys: Vec::new(),
};
let keydb = SuppliedKey {
device_keys: Vec::new(),
processing_keys: Vec::new(),
media_keys: Vec::new(),
disc_entry: Some(entry),
};
let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb];
// A real VID → path 3 fires and the VUK derives from Km + THIS VID.
let with_vid = ResolveContext {
unit_key_ro: &uk_ro,
content_cert: None,
volume_id: &vid,
providers,
mkb: None,
};
let r = resolve_keys_v21(&with_vid).expect("a real VID must reach path 3");
assert_eq!(r.key_source, 3);
assert_eq!(
r.vuk,
Some(derive_vuk(&mk, &vid)),
"VUK must derive from the Media Key and the disc's own VID"
);
// The all-zero sentinel → paths 1 and 3 are skipped entirely; with no
// VUK and no unit keys on the entry, nothing resolves.
let no_vid = ResolveContext {
unit_key_ro: &uk_ro,
content_cert: None,
volume_id: &[0u8; 16],
providers,
mkb: None,
};
let got = resolve_keys_v21(&no_vid);
assert!(
got.is_none(),
"a zero VID must not be used to derive a VUK; got key_source {:?} vuk {:?}",
got.as_ref().map(|r| r.key_source),
got.as_ref().map(|r| r.vuk.is_some())
);
}
#[test]
fn resolve_keys_returns_none_when_no_provider_has_anything() {
// Empty provider array + VID present + no MKB → all paths miss → None.
+3 -3
View File
@@ -25,7 +25,7 @@
//! u32 start_spn | u32 end_spn (source-packet numbers, inclusive)
//! ```
//! `index` is the 1..32 forensic index tag, NOT a sequential segment id: measured
//! on a retail 2.1 disc (Zombieland) it cycles 1,2,…,32,1,2,… across records in
//! on a retail 2.1 disc it cycles 1,2,…,32,1,2,… across records in
//! file order — 24 full cycles of 32 plus a final partial cycle of 24 = 792
//! records. Source-packet numbers are the 192-byte BDAV packet index: byte offset
//! = `spn * 192`. Each segment is ~2560 packets (~480 KB) = 80 aligned units,
@@ -334,7 +334,7 @@ mod tests {
#[test]
fn parses_real_disc_layout() {
// First three records observed on retail 2.1 (Zombieland): the variant
// First three records observed on retail 2.1: the variant
// field counts 1,2,3,… (it wraps at 32 further into the table — see
// `index_field_cycles_one_to_thirty_two`), segments are 2560 packets.
let tbl = build_tbl(&[
@@ -396,7 +396,7 @@ mod tests {
#[test]
fn index_field_cycles_one_to_thirty_two() {
// Reality on Zombieland: field@4 is the index, cycling 1..=32 in file
// Reality on a retail 2.1 disc: field@4 is the index, cycling 1..=32 in file
// order (NOT a sequential segment id). Reproduce one-and-a-bit cycles.
let mut recs = Vec::new();
let mut spn = 1000u32;
+71
View File
@@ -195,6 +195,77 @@ impl std::fmt::Debug for DiscEntry {
}
}
#[cfg(test)]
mod unit_key_tests {
use super::*;
/// `is_default_index` is the public predicate that separates ordinary
/// (index-0) content keys from FMTS forensic index keys ([`UnitKey`] docs;
/// AACS 2.1 `IndividualSegment.tbl` tagging). A body answering `true` for
/// everything would present a forensic index key as an ordinary content
/// key — the caller would decrypt the bulk of the title with a key that
/// only opens 1/32nd of it; answering `false` for everything would hide
/// every ordinary key.
///
/// Pinned against the two NAMED constructors, which are the contract:
/// [`UnitKey::new`] builds the ordinary key, [`UnitKey::forensic`] builds
/// an index key for `1..=32`.
#[test]
fn is_default_index_separates_the_two_constructors() {
let ordinary = UnitKey::new(0, [0xAA; 16]);
assert!(
ordinary.is_default_index(),
"UnitKey::new builds the ordinary (index-0) key"
);
// Every forensic index the spec allows must be reported as NOT default.
for n in 1u8..=32 {
let k = UnitKey::forensic(0, [0xAA; 16], n);
assert!(
!k.is_default_index(),
"UnitKey::forensic({n}) is an index key, not the default key"
);
}
}
/// The predicate must agree with the one consumer of `index_number` in the
/// crate: [`crate::aacs::index_select::resolve_disc_index`] resolves the
/// disc's forensic index from exactly the keys that are NOT default. If
/// the two disagree, a disc resolves an index whose key the rest of the
/// pipeline treats as ordinary (or vice versa).
#[test]
fn is_default_index_agrees_with_the_forensic_index_resolver() {
use crate::aacs::index_select::resolve_disc_index;
let keys = [
UnitKey::new(0, [0x11; 16]),
UnitKey::forensic(1, [0x22; 16], 7),
];
assert_eq!(
resolve_disc_index(&keys),
Some(7),
"sanity: the resolver picks the forensic key's index"
);
let non_default: Vec<u8> = keys
.iter()
.filter(|k| !k.is_default_index())
.map(|k| k.index_number)
.collect();
assert_eq!(
non_default,
vec![7],
"exactly the key the resolver picked must be non-default"
);
// An all-ordinary key set resolves no index, and every key must report
// itself default.
let plain = [UnitKey::new(0, [0x11; 16]), UnitKey::new(1, [0x22; 16])];
assert_eq!(resolve_disc_index(&plain), None);
assert!(plain.iter().all(|k| k.is_default_index()));
}
}
#[cfg(test)]
mod redaction_tests {
use super::*;
+937 -8
View File
@@ -91,8 +91,8 @@ pub fn is_variant_mkb(records: &[MkbRecord]) -> bool {
}
/// Body of the `0x2d` record: the `VARIANTS` table followed by the trailing
/// 16-byte `Kvn` Nonce. Measured `46_100*2 + 16 = 92_216` on Zombieland v70 and
/// `92_220` on Stand By Me v70 — in both, the leading `body.len() - 16` bytes are
/// 16-byte `Kvn` Nonce. Measured `46_100*2 + 16 = 92_216` on one v70 disc and
/// `92_220` on another — in both, the leading `body.len() - 16` bytes are
/// the big-endian `u16` `VARIANTS` table (one per subset-difference) and the last
/// 16 bytes are the Nonce, with NO leading header. This does NOT hold the C used
/// for `Kmp` — that is the per-slot block in `0x0c`
@@ -361,7 +361,7 @@ impl std::error::Error for MediaKeyVariantError {}
/// Look up the per-slot `VARIANTS` value for the matched subset-difference slot,
/// keyed by the same index that selected the cvalue ([`ProcessingKeyMatch::cvalue_index`]).
///
/// LAYOUT (fixed against a real 2.1 variant MKB — Zombieland v70, `MKB_RO.inf`):
/// LAYOUT (fixed against a real 2.1 variant MKB — a v70 `MKB_RO.inf`):
/// the `0x2d` Encrypted-Media-Key-Variant-Data body is exactly
/// `46_100*2 + 16 = 92_216` bytes, i.e. one **big-endian u16 `VARIANTS` entry per
/// subset-difference slot** (1:1 with the `0x0c` variant cvalues and the `0x04`
@@ -378,7 +378,7 @@ fn variants_for_uv(records: &[MkbRecord], sd_slot_index: usize) -> Option<u16> {
// The VARIANTS table is the leading bytes; the 16-byte Kvn Nonce is packed at
// the TAIL (see [`variant_nonce`]). Bound the read to the table region so a
// near-end slot can never read Nonce bytes as a VARIANTS entry. NO leading
// header (measured: Zombieland v70 `0x2d` body = 46_100*2 + 16 = 92_216).
// header (measured: a v70 `0x2d` body = 46_100*2 + 16 = 92_216).
const NONCE: usize = 16;
let table_len = body.len().checked_sub(NONCE)?;
let off = sd_slot_index.checked_mul(2)?;
@@ -938,10 +938,15 @@ mod tests {
}
#[test]
fn walk_mkb_be24_high_byte_is_honored() {
// A record longer than 255 bytes needs the high BE24 byte. Build a
// 0x10 record of total length 0x000110 (272) and confirm the body is
// 268 bytes (a parser that read only the low byte would see len 0x10).
fn walk_mkb_be24_middle_byte_is_honored() {
// A record longer than 255 bytes needs the MIDDLE BE24 byte: total
// length 0x00_0110 (272) is `[0x00, 0x01, 0x10]`, so a parser reading
// only the low byte sees 0x10. The HIGH byte of this length is zero, so
// this test says nothing about the `<< 16` term — that is pinned
// separately by `mkb::tests::mkb_records_honors_the_high_byte_of_the_be24_length`,
// which uses a 0x01_0004 record. (Renamed from
// `walk_mkb_be24_high_byte_is_honored`, which claimed coverage this body
// does not deliver.)
let total = 0x0110usize; // 272
let mut mkb = vec![0x10, 0x00, 0x01, 0x10];
mkb.resize(total, 0xAB);
@@ -1205,4 +1210,928 @@ mod tests {
.expect_err("soft-correction bit → classified, not a key");
assert_eq!(err, MediaKeyVariantError::SoftCorrectionRequired);
}
// ════════════════════════════════════════════════════════════════════
// A COMPLETE variant MKB — the AACS 2.1 happy path
//
// Every other test in this module asserts an ERROR classification, so
// until now no test ever drove `derive_media_key_variant` to a Media
// Key. That left the whole success path — the VARIANTS lookup, the VKD
// selection, the final `Km` unwrap and the verify gate — pinned by
// nothing: a body that answered a constant for any of those steps still
// produced the same errors these tests expect.
//
// No real key material is involved. Every AACS 2.1 relation in the chain
// is invertible, so the fixture below picks a Media Key and a Processing
// Key and computes the MKB records that connect them, exactly as
// `derive::position_recovery_tests::plant_mkb` does for the classical
// chain.
// ════════════════════════════════════════════════════════════════════
/// A planted variant MKB and the values it was built from.
struct PlantedVariant {
records: Vec<MkbRecord>,
/// The Processing Key that covers slot 0.
kp: [u8; 16],
/// The Media Key the chain must derive from `kp`.
km: [u8; 16],
/// The `0x86` Verify-Media-Key block.
mk_dv: [u8; 16],
/// The `VARIANTS[0]` entry planted in the `0x2d` table.
variants0: u16,
/// The `0x2d` tail Nonce.
nonce: [u8; 16],
/// The slot-0 `0x0c` C block the Kmp step consumes.
c_block: [u8; 16],
/// The subset-difference number of the single planted slot.
uv: u32,
}
/// An MKB record: 1-byte type + BE24 total length (header included) + body.
fn vrec(t: u8, body: &[u8]) -> Vec<u8> {
let total = 4 + body.len();
let mut r = vec![
t,
((total >> 16) & 0xFF) as u8,
((total >> 8) & 0xFF) as u8,
(total & 0xFF) as u8,
];
r.extend_from_slice(body);
r
}
/// Build a variant MKB by inverting the 2.1 chain for a CHOSEN `(Kp, Km)`.
///
/// One subset-difference slot (`uv = 2`, `u_mask_shift = 3`, slot index 0).
/// The VKD the chain must land on is planted at index **1** of the `0x2f`
/// table, behind a decoy at index 0, so `VARIANTS[0]` is load-bearing: it is
/// chosen as `Kvn XOR 1`, and any other value selects the decoy (wrong `Km`,
/// rejected by the verify gate) or indexes past the table.
fn plant_variant_mkb() -> PlantedVariant {
use crate::aacs::crypto::{aes_ecb_encrypt, aes_g};
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
const UV: u32 = 2;
const U_MASK_SHIFT: u8 = 3;
let kp: [u8; 16] = [
0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF,
0x4F, 0x3C,
];
// `uv = 2` puts its only non-zero byte at index 15, so byte 15 is the ONE
// position where the `Km`/`Kmp` uv-XOR is observable. Its 0x02 bit is
// deliberately CLEAR: with the bit set, `km[15] ^= 2` and `km[15] |= 2`
// agree (the XOR would only be clearing a bit the OR re-sets) and an
// OR-for-XOR substitution in the final step would be invisible.
let km: [u8; 16] = [
0xC0, 0xC1, 0xC2, 0xC3, 0xC4, 0xC5, 0xC6, 0xC7, 0xC8, 0xC9, 0xCA, 0xCB, 0xCC, 0xCD,
0xCE, 0xCD,
];
assert_eq!(km[15] & 0x02, 0, "fixture check: see above");
let nonce: [u8; 16] = [
0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, 0x3D,
0x3E, 0x3F,
];
let uv_bytes = UV.to_be_bytes();
// ── Verify-Media-Key record (0x86): AES-D(Km, mk_dv) opens with the
// magic ([C] §3.2.5.1.4), so mk_dv = AES-E(Km, magic || padding).
let mut vd = [0x5Au8; 16];
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&km, &vd);
// ── C (0x0c): the chain computes Kmp = AES-D(Kp, C) XOR uv. Pick a Kmp
// with BOTH condition bits on byte 15 clear (0x02 soft-correction,
// 0x04 online challenge) so the default KCD path runs, then invert.
let mut kmp = [0x42u8; 16];
kmp[15] = 0x40; // neither 0x02 nor 0x04
let mut c_plain = kmp;
for i in 0..4 {
c_plain[12 + i] ^= uv_bytes[i];
}
let c_block = aes_ecb_encrypt(&kp, &c_plain);
// ── Kpnew = Kmp XOR KCD. Read through the production constant rather
// than assuming it is zero, so the fixture stays valid if a real
// per-licensee KCD is ever wired in (see `KEY_CORRECTION_DATA`).
let mut kpnew = [0u8; 16];
for i in 0..16 {
kpnew[i] = kmp[i] ^ KEY_CORRECTION_DATA[i];
}
// ── VKD: the chain computes Km = AES-D(Kpnew, VKD) XOR uv, so
// VKD = AES-E(Kpnew, Km with uv XORed back into its low 4 bytes).
let mut km_pre = km;
for i in 0..4 {
km_pre[12 + i] ^= uv_bytes[i];
}
let vkd = aes_ecb_encrypt(&kpnew, &km_pre);
// ── VARIANTS[0]: VKD_idx = Kvn XOR VARIANTS[uv], and we planted the
// real VKD at table index 1, so VARIANTS[0] = Kvn XOR 1.
// Kvn = low 16 bits (BE) of AES-G(Kp, Nonce).
let kvn_block = aes_g(&kp, &nonce);
let kvn = u16::from_be_bytes([kvn_block[14], kvn_block[15]]);
let variants0 = kvn ^ 1;
// ── Assemble.
let mut mkb = Vec::new();
mkb.extend_from_slice(&vrec(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
// 0x04 subset-difference: one slot.
let mut subdiff = vec![U_MASK_SHIFT];
subdiff.extend_from_slice(&uv_bytes);
mkb.extend_from_slice(&vrec(0x04, &subdiff));
// 0x0c per-slot C table: one 16-byte entry.
mkb.extend_from_slice(&vrec(0x0c, &c_block));
// 0x86 Verify-Media-Key.
mkb.extend_from_slice(&vrec(0x86, &mk_dv));
// 0x2d: VARIANTS table (one BE u16) then the 16-byte tail Nonce.
let mut vdata = Vec::new();
vdata.extend_from_slice(&variants0.to_be_bytes());
vdata.extend_from_slice(&nonce);
mkb.extend_from_slice(&vrec(0x2d, &vdata));
// 0x2f VKD table: a decoy at index 0, the real VKD at index 1.
let mut vkd_table = vec![0x9Au8; 16];
vkd_table.extend_from_slice(&vkd);
mkb.extend_from_slice(&vrec(0x2f, &vkd_table));
PlantedVariant {
records: walk_mkb(&mkb),
kp,
km,
mk_dv,
variants0,
nonce,
c_block,
uv: UV,
}
}
/// Sanity-check the fixture before anything is asserted through it: an MKB
/// the record finders cannot read would make every "returns an error" body
/// look correct.
#[test]
fn the_planted_variant_mkb_is_a_well_formed_variant_mkb() {
let p = plant_variant_mkb();
assert!(is_variant_mkb(&p.records), "0x2d/0x2f present");
assert_eq!(variant_nonce(&p.records), Some(p.nonce), "tail Nonce");
assert_eq!(
variant_key_data(&p.records).map(<[u8]>::len),
Some(32),
"two 16-byte VKD entries"
);
assert_eq!(
variant_uv_slots(&p.records),
Some(vec![(2u32, 0usize)]),
"one subset-difference slot at index 0 with uv=2"
);
}
/// THE happy path: a Processing Key covering slot 0 of a complete variant
/// MKB must derive the planted Media Key.
///
/// This is the assertion the whole 2.1 chain hangs from — `derive_media_key_variant`
/// is what `resolve` calls for a 2.1 disc, and its output becomes the VUK,
/// the title keys and every decrypted byte. The assertion lands on the FINAL
/// derived Media Key, so no intermediate step (VARIANTS lookup, VKD index,
/// Kpnew, the unwrap) can be replaced by a constant and still pass.
#[test]
fn variant_chain_derives_the_planted_media_key_for_a_covering_kp() {
let p = plant_variant_mkb();
assert_eq!(
derive_media_key_variant(&p.records, &p.kp),
Ok(p.km),
"a covering 2.1 Processing Key must derive the planted Media Key"
);
}
/// The other direction: a Processing Key one bit away must NOT yield a key.
/// The terminal Verify-Media-Key gate is what stands between a wrong Kp and
/// a wrong Media Key silently propagating into the VUK and title keys.
#[test]
fn variant_chain_yields_no_key_for_a_kp_one_bit_away() {
let p = plant_variant_mkb();
let mut stranger = p.kp;
stranger[0] ^= 0x01;
let got = derive_media_key_variant(&p.records, &stranger);
assert!(
got.is_err(),
"a non-covering Kp must never produce a Media Key, got {got:?}"
);
assert_ne!(got, Ok(p.km));
}
/// `mkb_find_mk_dv` supplies the block the terminal verify gate compares
/// against. A body answering a FIXED block would make the gate compare every
/// derived Media Key against a record no disc carries: on a real disc every
/// correct key is rejected (2.1 discs stop resolving entirely), and any key
/// that happened to open the fixed block would be accepted wholesale.
#[test]
fn mkb_find_mk_dv_returns_the_verify_records_actual_bytes() {
let p = plant_variant_mkb();
assert_eq!(
mkb_find_mk_dv(&p.records),
Some(p.mk_dv),
"mk_dv must be the bytes the 0x86 record carries"
);
assert_ne!(mkb_find_mk_dv(&p.records), Some([0u8; 16]));
assert_ne!(mkb_find_mk_dv(&p.records), Some([1u8; 16]));
// And it is the block the gate actually uses: swapping the 0x86 record
// for an unrelated one must break the derivation that just succeeded.
let mut recs = p.records.clone();
let v = recs
.iter_mut()
.find(|r| r.rec_type == 0x86)
.expect("verify record present");
v.body = vec![0x00; 16];
assert!(
derive_media_key_variant(&recs, &p.kp).is_err(),
"with a foreign verify block the same Kp must no longer verify"
);
}
/// `variants_for_uv` reads `VARIANTS[slot]` — the value XORed with `Kvn` to
/// index the VKD table. A body answering a constant picks the WRONG VKD
/// entry for every disc, so the derived Media Key fails the verify gate and
/// every 2.1 variant disc reports `ProcessingKeyUnavailable` with a
/// perfectly good Processing Key in hand.
///
/// Asserted two ways: the exact planted table entry, and — the load-bearing
/// one — that this entry is what carries the chain to the planted Media Key.
#[test]
fn variants_for_uv_reads_the_planted_table_entry_that_selects_the_vkd() {
let p = plant_variant_mkb();
assert_eq!(
variants_for_uv(&p.records, 0),
Some(p.variants0),
"slot 0 must read the planted VARIANTS entry"
);
// The planted entry is Kvn ^ 1 (the real VKD sits at table index 1), so
// it is neither 0 nor 1 — a constant body is a different value here.
assert_ne!(variants_for_uv(&p.records, 0), Some(0));
assert_ne!(variants_for_uv(&p.records, 0), Some(1));
// Perturbing ONLY the VARIANTS entry breaks the derivation: proof the
// value this function returns is the one that selects the VKD.
let mut recs = p.records.clone();
let d = recs
.iter_mut()
.find(|r| r.rec_type == 0x2d)
.expect("0x2d present");
d.body[0] ^= 0x80;
assert!(
derive_media_key_variant(&recs, &p.kp).is_err(),
"a different VARIANTS entry must select a different VKD and fail the gate"
);
}
/// The `0x2d` body is `VARIANTS` table then a 16-byte tail Nonce. A slot
/// index whose entry would fall inside the Nonce must be refused rather than
/// read Nonce bytes as a VARIANTS value.
#[test]
fn variants_for_uv_stops_before_the_tail_nonce() {
// Three-entry table with distinct values, then the Nonce.
let mut body = Vec::new();
body.extend_from_slice(&0x1234u16.to_be_bytes());
body.extend_from_slice(&0xABCDu16.to_be_bytes());
body.extend_from_slice(&0x00FFu16.to_be_bytes());
let nonce = [0x77u8; 16];
body.extend_from_slice(&nonce);
let recs = walk_mkb(&vrec(0x2d, &body));
assert_eq!(variants_for_uv(&recs, 0), Some(0x1234));
assert_eq!(variants_for_uv(&recs, 1), Some(0xABCD));
assert_eq!(variants_for_uv(&recs, 2), Some(0x00FF));
assert_eq!(
variants_for_uv(&recs, 3),
None,
"slot 3 starts inside the Nonce — must be refused, not read"
);
assert_eq!(variant_nonce(&recs), Some(nonce), "the Nonce is the tail");
}
/// `variant_uv_slots` enumerates the slots the chain will try a Processing
/// Key against, and it must drop the two shapes that are unusable — and
/// dangerous — rather than pass them on:
///
/// - `uv == 0`: no subset-difference. It would be XORed into `Kmp` and
/// `Km` as a no-op and the slot would be tried against every VKD entry.
/// - `u_mask_shift >= 32`: out of range for a `u32` shift. `0x20..=0x3F`
/// have the `0xC0` revoked-marker bits CLEAR, so they pass the table
/// terminator and reach the `wrapping_shl` in the walk, where shift 32
/// silently means shift 0 (`u_mask = 0xFFFF_FFFF`) and matches a slot
/// the device does not cover.
///
/// Both bytes are disc-supplied. Every existing fixture uses one in-range
/// non-zero slot, so neither rejection was executed.
#[test]
fn variant_uv_slots_drops_zero_uv_and_out_of_range_shift_slots() {
// Four slots: uv == 0, shift == 32 (the exact boundary), shift == 0x3F
// (the top of the marker-clear range), and one good slot last.
let mut body = Vec::new();
for (shift, uv) in [
(3u8, 0u32),
(32u8, 0x0000_0005u32),
(0x3Fu8, 0x0000_0006u32),
(12u8, 0x0000_0400u32),
] {
body.push(shift);
body.extend_from_slice(&uv.to_be_bytes());
}
// Fixture check: none of these bytes trips the 0xC0 table terminator, so
// the per-slot tests are the only thing rejecting them.
assert!(body.chunks(5).all(|c| c[0] & 0xC0 == 0));
let recs = walk_mkb(&vrec(REC_SUBSET_DIFFERENCE, &body));
assert_eq!(
variant_uv_slots(&recs),
Some(vec![(0x0000_0400u32, 3usize)]),
"only the in-range, non-zero slot is a usable subset-difference — \
and it keeps its own table index"
);
}
/// THE happy path for the EXPLICIT-INPUT entry point. `media_key_variant_from_kp`
/// is the harness twin of [`derive_media_key_variant`]: same chain, but the
/// caller supplies the `0x0c` C block, the slot's `uv` and its `VARIANTS[uv]`
/// instead of having them looked up on the MKB.
///
/// Before this test, the ONLY test that entered this function asserted the
/// `Kmp[15]` soft-correction bit — it returned before the Kpnew, Kvn, VKD,
/// Km and Kvu steps ever ran. Every arithmetic step past that early return
/// was executed by nothing, so a body that computed `Kpnew = Kmp | KCD`,
/// indexed the VKD table at `Kvn + VARIANTS` or dropped the `uv` XOR out of
/// `Km` produced exactly the same observable behaviour.
///
/// The assertion lands on the returned `(Km, Kvu)` — the two values that
/// become every title key and every decrypted byte on a 2.1 disc.
#[test]
fn media_key_variant_from_kp_derives_the_planted_media_key_and_volume_unique_key() {
let p = plant_variant_mkb();
let vid: [u8; 16] = [
0x1A, 0x2B, 0x3C, 0x4D, 0x5E, 0x6F, 0x70, 0x81, 0x92, 0xA3, 0xB4, 0xC5, 0xD6, 0xE7,
0xF8, 0x09,
];
let (km, kvu) =
media_key_variant_from_kp(&p.kp, &p.c_block, p.uv, p.variants0, &p.records, &vid)
.expect("the planted explicit inputs must complete the 2.1 variant chain");
assert_eq!(
km, p.km,
"the explicit-input entry must derive the same planted Media Key \
the MKB-driven entry does"
);
// Kvu = AES-G(Km, VID) ([C] §3.2.5.2). Computed from the PLANTED Km
// literal, so it does not move with any mutation of this module.
assert_eq!(
kvu,
aes_g(&p.km, &vid),
"Kvu must be AES-G of the derived Media Key with the Volume ID"
);
// ...and specifically NOT of the Processing Key: the two are one AES-D
// apart and a body that returned the wrong one would still be 16 bytes
// of key-shaped material that silently decrypts nothing.
assert_ne!(kvu, aes_g(&p.kp, &vid));
}
/// The terminal gate on the explicit-input entry. `media_key_variant_from_kp`
/// takes three caller-supplied values (`c_block`, `uv`, `variants_uv`); each
/// one wrong must yield `MediaKeyVerifyFailed`, never a key. Without this,
/// a harness feeding a mis-transcribed slot would be handed 16 bytes that
/// look exactly like a Media Key.
#[test]
fn media_key_variant_from_kp_refuses_every_single_wrong_explicit_input() {
let p = plant_variant_mkb();
let vid = [0x33u8; 16];
// Baseline: all three correct → a key.
assert!(
media_key_variant_from_kp(&p.kp, &p.c_block, p.uv, p.variants0, &p.records, &vid)
.is_ok()
);
// Wrong C block: EVERY one-bit neighbour must fail to produce a key.
// (Which classification it lands in depends on the two condition bits
// the perturbed Kmp happens to carry — the property being pinned is
// that none of the 128 reaches `Ok`.)
for byte in 0..16usize {
for bit in 0..8u32 {
let mut c_bad = p.c_block;
c_bad[byte] ^= 1u8 << bit;
let got =
media_key_variant_from_kp(&p.kp, &c_bad, p.uv, p.variants0, &p.records, &vid);
assert!(
got.is_err(),
"C block differing only in byte {byte} bit {bit} yielded a key"
);
}
}
// Wrong uv: it is XORed into BOTH Kmp and Km, so a wrong slot number
// must not reach a key.
for delta in 1..=8u32 {
let got = media_key_variant_from_kp(
&p.kp,
&p.c_block,
p.uv + delta,
p.variants0,
&p.records,
&vid,
);
assert!(got.is_err(), "uv + {delta} must not verify, got {got:?}");
}
// Wrong VARIANTS[uv]: selects a different VKD entry. The planted table
// has two entries, so `^ 1` lands on the decoy at index 0 (in range,
// wrong key) rather than out of range.
assert_eq!(
media_key_variant_from_kp(&p.kp, &p.c_block, p.uv, p.variants0 ^ 1, &p.records, &vid),
Err(MediaKeyVariantError::MediaKeyVerifyFailed),
"a VARIANTS entry selecting the decoy VKD must not verify"
);
// And a VARIANTS entry that indexes off the end of the table is
// classified as such, not read out of bounds.
assert_eq!(
media_key_variant_from_kp(
&p.kp,
&p.c_block,
p.uv,
p.variants0 ^ 0x8000,
&p.records,
&vid
),
Err(MediaKeyVariantError::VkdIndexOutOfRange),
"a VKD index past the table must be classified, not read"
);
}
/// The `Kmp[15]` online-challenge bit (`0x04`) on the explicit-input entry.
/// Its twin (`0x02`, soft correction) was already pinned; without this one a
/// body that classified both bits as soft correction — or ignored `0x04` and
/// ran the default-KCD chain to a wrong key — was unconstrained.
#[test]
fn media_key_variant_from_kp_classifies_online_challenge() {
use crate::aacs::crypto::aes_ecb_encrypt;
let p = plant_variant_mkb();
// Plant Kmp[15] = 0x04 (online challenge, soft-correction bit CLEAR) and
// invert the Kmp step for uv = 0 so Kmp == AES-D(kp, C).
let mut target_kmp = [0x00u8; 16];
target_kmp[15] = 0x04;
let c_block = aes_ecb_encrypt(&p.kp, &target_kmp);
assert_eq!(
media_key_variant_from_kp(&p.kp, &c_block, 0, 0, &p.records, &[0u8; 16]),
Err(MediaKeyVariantError::OnlineChallengeRequired),
);
}
// ════════════════════════════════════════════════════════════════════
// A MULTI-SLOT variant MKB driven by a real DEVICE KEY
//
// `walk_processing_key` is the DK -> Kp step that feeds the whole 2.1
// chain. Every existing test of it either asserts `None` (out-of-range
// shift, uv == 0) or asserts only that SOME match came back — none pins
// WHICH Processing Key, cvalue or slot index it returns. And every one of
// them uses a SINGLE-slot MKB, where the slot index is 0: all the
// `uvs[1 + 5*idx]` / `cvalues[idx*16..]` stride arithmetic multiplies by
// zero and any stride at all gives the same answer.
//
// This fixture puts the covering slot at index 1, behind a decoy at
// index 0, so the strides are load-bearing.
// ════════════════════════════════════════════════════════════════════
/// A two-slot variant MKB whose SECOND slot is opened by a device key.
struct PlantedWalk {
records: Vec<MkbRecord>,
/// The device key that covers slot 1 with zero descent.
dk: DeviceKey,
/// The Media Key the full chain must reach from that Processing Key.
km: [u8; 16],
/// The `0x0c` C block of slot 1 — the cvalue the walk must select.
c_block1: [u8; 16],
}
/// Build a two-slot variant MKB keyed by a DEVICE key at slot **1**.
///
/// Positions follow the same reasoning as the classical
/// `derive::position_recovery_tests::plant_mkb`: `uv = 0x0400`
/// (`u_mask_shift = 12`) with a device node of `0x0C00` satisfies the
/// [C] §3.2.4 gate — equal under `u_mask = 0xFFFF_F000`, different under
/// `v_mask = 0xFFFF_F800`. The device key's own `uv` equals the slot's, so
/// `dev_key_v_mask == v_mask` and [`calc_pk_from_dk`] descends zero levels:
/// `Kp = AES-G3(dk, 1)`, written out explicitly below rather than taken from
/// the walk's own output.
///
/// Slot 0 is a decoy at `uv = 0x0800`, which the SAME device node fails the
/// `v_mask` half of the gate against (`0x0C00 & 0xFFFF_F000 == 0x0800 &
/// 0xFFFF_F000`), so the walk must skip it and land on slot 1.
fn plant_walk_variant_mkb() -> PlantedWalk {
use crate::aacs::crypto::{aes_ecb_encrypt, aes_g};
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
const UV_DECOY: u32 = 0x0000_0800;
const UV_REAL: u32 = 0x0000_0400;
const U_MASK_SHIFT: u8 = 12;
const NODE: u16 = 0x0C00;
let dkey: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2,
0xE1, 0xF0,
];
// Zero descent: the Processing Key is the AES-G3(.,1) of the device's own
// node ([C] §3.2.4). Written as the explicit primitive chain so it does
// NOT move with any mutation of the walk under test.
let kp = aesg3(&dkey, 1);
// As in `plant_variant_mkb`: `uv = 0x0400`'s only non-zero byte is at
// index 14, and its 0x04 bit must be CLEAR in `km` for the final
// `km[14] ^= 0x04` to be distinguishable from `|=`.
let km: [u8; 16] = [
0xB0, 0xB1, 0xB2, 0xB3, 0xB4, 0xB5, 0xB6, 0xB7, 0xB8, 0xB9, 0xBA, 0xBB, 0xBC, 0xBD,
0xBA, 0xBF,
];
assert_eq!(km[14] & 0x04, 0, "fixture check: see above");
let nonce: [u8; 16] = [
0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5A, 0x5B, 0x5C, 0x5D,
0x5E, 0x5F,
];
// ── 0x86 Verify-Media-Key ([C] §3.2.5.1.4).
let mut vd = [0x5Au8; 16];
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&km, &vd);
// ── C blocks. Both are built so `Kmp[15]` has the 0x02 / 0x04 condition
// bits CLEAR, so both slots run the default-KCD path to completion and
// the decoy is rejected by the terminal verify gate rather than
// short-circuiting into a correction-mode classification.
let c_for = |kmp: &[u8; 16], uv: u32| -> [u8; 16] {
let mut c_plain = *kmp;
for (b, u) in c_plain[12..16].iter_mut().zip(uv.to_be_bytes()) {
*b ^= u;
}
aes_ecb_encrypt(&kp, &c_plain)
};
let mut kmp1 = [0x42u8; 16];
kmp1[15] = 0x40;
let c_block1 = c_for(&kmp1, UV_REAL);
let mut kmp0 = [0x17u8; 16];
kmp0[15] = 0x40;
let c_block0 = c_for(&kmp0, UV_DECOY);
// ── VKD for slot 1: Km = AES-D(Kpnew, VKD) XOR uv.
let mut kpnew = [0u8; 16];
for i in 0..16 {
kpnew[i] = kmp1[i] ^ KEY_CORRECTION_DATA[i];
}
let mut km_pre = km;
for (b, u) in km_pre[12..16].iter_mut().zip(UV_REAL.to_be_bytes()) {
*b ^= u;
}
let vkd = aes_ecb_encrypt(&kpnew, &km_pre);
// ── VARIANTS: the real VKD is planted at table index 2, behind two
// decoys, so VARIANTS[1] = Kvn XOR 2 is load-bearing. VARIANTS[0] sends
// the decoy slot to entry 0 — in range, wrong key, rejected by the gate.
let kvn_block = aes_g(&kp, &nonce);
let kvn = u16::from_be_bytes([kvn_block[14], kvn_block[15]]);
let variants0 = kvn;
let variants1 = kvn ^ 2;
// ── Assemble.
let mut mkb = Vec::new();
mkb.extend_from_slice(&vrec(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
let mut subdiff = vec![U_MASK_SHIFT];
subdiff.extend_from_slice(&UV_DECOY.to_be_bytes());
subdiff.push(U_MASK_SHIFT);
subdiff.extend_from_slice(&UV_REAL.to_be_bytes());
mkb.extend_from_slice(&vrec(0x04, &subdiff));
let mut ctable = Vec::new();
ctable.extend_from_slice(&c_block0);
ctable.extend_from_slice(&c_block1);
mkb.extend_from_slice(&vrec(0x0c, &ctable));
mkb.extend_from_slice(&vrec(0x86, &mk_dv));
let mut vdata = Vec::new();
vdata.extend_from_slice(&variants0.to_be_bytes());
vdata.extend_from_slice(&variants1.to_be_bytes());
vdata.extend_from_slice(&nonce);
mkb.extend_from_slice(&vrec(0x2d, &vdata));
let mut vkd_table = vec![0x9Au8; 16];
vkd_table.extend_from_slice(&[0x6Bu8; 16]);
vkd_table.extend_from_slice(&vkd);
mkb.extend_from_slice(&vrec(0x2f, &vkd_table));
PlantedWalk {
records: walk_mkb(&mkb),
dk: DeviceKey {
key: dkey,
node: NODE,
uv: UV_REAL,
u_mask_shift: U_MASK_SHIFT,
},
km,
c_block1,
}
}
/// Sanity-check the two-slot fixture before anything is asserted through it.
#[test]
fn the_planted_walk_variant_mkb_has_two_slots_and_is_keyed_at_the_second() {
let p = plant_walk_variant_mkb();
assert!(is_variant_mkb(&p.records));
assert_eq!(
variant_uv_slots(&p.records),
Some(vec![(0x0800u32, 0usize), (0x0400u32, 1usize)]),
"two subset-difference slots, the covering one at index 1"
);
assert_eq!(
mkb_find_body(&p.records, REC_MEDIA_KEY_VARIANT_DATA).map(<[u8]>::len),
Some(32),
"two 16-byte C entries in the 0x0c table"
);
}
/// `walk_processing_key` must return the Processing Key, `uv`, cvalue AND
/// slot index of the covering slot — slot **1**, not slot 0.
///
/// This is the DK → Kp step the entire 2.1 chain starts from. Every prior
/// test of it asserted either `None` or merely `is_some()`, and all used a
/// one-slot MKB where every stride multiplies by zero. A body that read the
/// subset-difference at the wrong stride, sliced the wrong cvalue block, or
/// returned the slot-0 cvalue for a slot-1 match would have passed all of
/// them — and produced a Processing Key that opens nothing.
///
/// The expected `Kp` is written as the explicit `AES-G3(dk, 1)` zero-descent
/// relation from [C] §3.2.4, not taken from the walk's own output.
#[test]
fn walk_processing_key_returns_the_covering_slots_key_cvalue_and_index() {
let p = plant_walk_variant_mkb();
let m = walk_processing_key(&p.records, std::slice::from_ref(&p.dk))
.expect("the planted device key covers slot 1 of this MKB");
assert_eq!(m.uv, 0x0400, "the covering slot's uv, not the decoy's");
assert_eq!(m.cvalue_index, 1, "the covering slot sits at index 1");
assert_eq!(
m.kp,
aesg3(&p.dk.key, 1),
"zero descent: Kp is AES-G3(device key, 1)"
);
assert_eq!(
m.cvalue, p.c_block1,
"the cvalue must be slot 1's 16-byte C block, not slot 0's"
);
// The load-bearing consequence: that Processing Key drives the full
// variant chain to the planted Media Key.
assert_eq!(
derive_media_key_variant(&p.records, &m.kp),
Ok(p.km),
"the walked Processing Key must derive the planted Media Key"
);
}
/// The gate the walk applies is [C] §3.2.4's subset-difference test, and a
/// device key that fails it must get NO match. Pinned across all four
/// coordinates the gate reads — node, uv, u_mask_shift and the key bytes —
/// because a body that dropped any half of the gate would hand back a
/// Processing Key derived at the wrong tree position.
#[test]
fn walk_processing_key_refuses_a_device_key_that_fails_the_subset_difference_gate() {
let p = plant_walk_variant_mkb();
assert!(walk_processing_key(&p.records, std::slice::from_ref(&p.dk)).is_some());
// node equal to uv under v_mask (0xFFFF_F800): the "different under
// v_mask" half of the gate fails.
let mut d = p.dk.clone();
d.node = 0x0400;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"a node equal to uv under v_mask does not gate"
);
// node differing under u_mask (0xFFFF_F000): the "equal under u_mask"
// half fails.
let mut d = p.dk.clone();
d.node = 0x1C00;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"a node outside the slot's u_mask does not gate"
);
// A device key whose declared u_mask_shift is not the slot's.
let mut d = p.dk.clone();
d.u_mask_shift = 11;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"u_mask must equal dev_key_u_mask"
);
// A device key positioned in a different subtree.
let mut d = p.dk.clone();
d.uv = 0x0C00;
assert!(
walk_processing_key(&p.records, std::slice::from_ref(&d)).is_none(),
"the device key's uv must agree with the slot's under dev_key_v_mask"
);
}
/// A `0x04` subset-difference record whose byte count is not a multiple of 5
/// must have its trailing partial chunk REFUSED, not parsed as a slot.
///
/// The walk sizes the table with `take_while(|c| c.len() == 5 && ...)`. Drop
/// the length half of that conjunction and the partial chunk is counted, and
/// the very next line reads `p_uv[0..4]` off a slice with fewer than four
/// bytes left — an index-out-of-bounds PANIC on a disc-supplied record
/// length. This is untrusted input: a truncated or crafted MKB reaches this
/// with no other guard in between.
#[test]
fn a_trailing_partial_subset_difference_chunk_is_not_parsed_as_a_slot() {
let p = plant_walk_variant_mkb();
// Re-emit the 0x04 record with three trailing bytes — a partial chunk
// whose first byte has the 0xC0 revoked-marker bits CLEAR, so only the
// length test stands between it and a four-byte read off a one-byte tail.
let mut recs = p.records.clone();
let sd = recs
.iter_mut()
.find(|r| r.rec_type == REC_SUBSET_DIFFERENCE)
.expect("0x04 present");
assert_eq!(sd.body.len(), 10, "two whole slots before truncation");
sd.body.extend_from_slice(&[0x0C, 0xAB, 0xCD]);
// A device key that covers NOTHING, so the walk is forced to run past
// both whole slots and reach the partial chunk.
let mut stranger = p.dk.clone();
stranger.node = 0x1C00;
assert!(
walk_processing_key(&recs, std::slice::from_ref(&stranger)).is_none(),
"the partial chunk must terminate the table, not be walked"
);
// And the covering key still finds its slot with the junk appended.
assert!(walk_processing_key(&recs, std::slice::from_ref(&p.dk)).is_some());
}
/// A `0x0c` cvalue table SHORTER than the matching slot index must make the
/// walk skip the slot, not slice past the end of the record.
///
/// `cvalues[uvs_idx * 16..(uvs_idx + 1) * 16]` is an unchecked slice; the
/// only thing in front of it is `if uvs_idx >= cvalues.len() / 16`. The two
/// counts come from DIFFERENT disc-supplied records (`0x04` and `0x0c`),
/// so nothing but this guard keeps them in agreement — a real MKB with a
/// short cvalue table panics the rip thread without it.
#[test]
fn a_cvalue_table_shorter_than_the_matching_slot_is_not_sliced_past() {
let p = plant_walk_variant_mkb();
let mut recs = p.records.clone();
let cv = recs
.iter_mut()
.find(|r| r.rec_type == REC_MEDIA_KEY_VARIANT_DATA)
.expect("0x0c present");
// One entry only — the covering slot is index 1, so it is out of range.
cv.body.truncate(16);
assert!(
walk_processing_key(&recs, std::slice::from_ref(&p.dk)).is_none(),
"slot 1 with a one-entry cvalue table must be skipped, not read"
);
}
/// The classical-magic escape hatch. On a NON-variant MKB the walk must
/// return a match only when `AES-D(Kmp, mk_dv)` opens with the [C] §3.2.5.1.4
/// verify magic; on a variant MKB that relation does not hold (the walk
/// yields a Precursor) and the presence of `0x2d`/`0x2f` is what lets the
/// match through to the chain's own terminal gate.
///
/// Both halves of `classical_ok || variant_present` are pinned here: strip
/// the variant records from a fixture whose magic does NOT hold and the walk
/// must go quiet. Otherwise a body that dropped the guard entirely would
/// return an unauthenticated Processing Key on every classical MKB.
#[test]
fn walk_processing_key_needs_either_the_verify_magic_or_variant_records() {
let p = plant_walk_variant_mkb();
// As planted (variant records present, magic absent) → a match.
assert!(walk_processing_key(&p.records, std::slice::from_ref(&p.dk)).is_some());
// Same slots, same device key, variant records removed. Nothing now
// authenticates the Processing Key, so there must be no match.
let stripped: Vec<MkbRecord> = p
.records
.iter()
.filter(|r| r.rec_type != REC_VARIANT_DATA_AND_NONCE && r.rec_type != REC_VKD_TABLE)
.cloned()
.collect();
assert!(
!is_variant_mkb(&stripped),
"fixture check: the stripped MKB is no longer a variant MKB"
);
assert!(
walk_processing_key(&stripped, std::slice::from_ref(&p.dk)).is_none(),
"without variant records the verify magic must hold, and it does not \
for a Precursor the walk must not return an unauthenticated key"
);
}
/// The OTHER half of `classical_ok || variant_present`: a non-variant MKB
/// whose cvalue really does open the Verify-Media-Key magic must yield a
/// match, and the [C] §3.2.4 relation that produces the candidate — AES-D(Kp,
/// cvalue) with `uv` XORed into the LOW FOUR BYTES — must be computed
/// exactly.
///
/// This is the only path on which that XOR is observable. On a variant MKB
/// `variant_present` short-circuits the magic test, so the whole
/// `km_candidate` computation is dead weight there: a body that ORed `uv`
/// in, or XORed it at the wrong offset, changes nothing any variant fixture
/// can see. On a CLASSICAL MKB it is the entire authentication of the
/// Processing Key.
#[test]
fn walk_processing_key_authenticates_a_classical_match_through_the_verify_magic() {
use crate::aacs::crypto::aes_ecb_encrypt;
const VERIFY_MAGIC: [u8; 8] = [0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF];
const UV: u32 = 0x0000_0400;
const U_MASK_SHIFT: u8 = 12;
let dkey: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2,
0xE1, 0xF0,
];
// Zero descent ([C] §3.2.4), written out as the primitive relation.
let kp = aesg3(&dkey, 1);
// `uv = 0x0400` puts its only non-zero byte at index 14, so byte 14 is
// the ONE position where the `uv` XOR is observable at all. Its 0x04 bit
// is deliberately CLEAR here: with the bit set, `km_candidate[14] |=
// 0x04` and `^= 0x04` agree (the XOR would only be clearing a bit the OR
// re-sets), and an OR-for-XOR substitution would be invisible.
let mk: [u8; 16] = [
0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7A, 0x7B, 0x7C, 0x7D,
0x7A, 0x7F,
];
assert_eq!(mk[14] & 0x04, 0, "fixture check: see above");
// Invert [C] §3.2.4: the walk computes AES-D(Kp, cvalue) then XORs `uv`
// into bytes 12..16 and expects the Media Key.
let mut mk_raw = mk;
for (b, u) in mk_raw[12..16].iter_mut().zip(UV.to_be_bytes()) {
*b ^= u;
}
let cv = aes_ecb_encrypt(&kp, &mk_raw);
// Invert [C] §3.2.5.1.4.
let mut vd = [0x5Au8; 16];
vd[..8].copy_from_slice(&VERIFY_MAGIC);
let mk_dv = aes_ecb_encrypt(&mk, &vd);
let mut subdiff = vec![U_MASK_SHIFT];
subdiff.extend_from_slice(&UV.to_be_bytes());
let mut mkb = Vec::new();
mkb.extend_from_slice(&vrec(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
mkb.extend_from_slice(&vrec(0x86, &mk_dv));
mkb.extend_from_slice(&vrec(0x04, &subdiff));
// cvalues in the classical `0x05` record; NO 0x2d / 0x2f.
mkb.extend_from_slice(&vrec(0x05, &cv));
let recs = walk_mkb(&mkb);
assert!(
!is_variant_mkb(&recs),
"fixture check: this must be a CLASSICAL MKB, so the magic is the \
only thing that can let a match through"
);
let dk = DeviceKey {
key: dkey,
node: 0x0C00,
uv: UV,
u_mask_shift: U_MASK_SHIFT,
};
let m = walk_processing_key(&recs, std::slice::from_ref(&dk))
.expect("the planted cvalue opens the verify magic for this key");
assert_eq!(m.kp, aesg3(&dkey, 1));
assert_eq!(m.uv, UV);
assert_eq!(m.cvalue, cv);
assert_eq!(m.cvalue_index, 0);
// And the magic is genuinely load-bearing: perturb the Verify-Media-Key
// record and the same key, slot and cvalue must stop matching.
let mut bad = recs.clone();
bad.iter_mut()
.find(|r| r.rec_type == 0x86)
.expect("0x86 present")
.body[0] ^= 0x01;
assert!(
walk_processing_key(&bad, std::slice::from_ref(&dk)).is_none(),
"a classical match must be authenticated by the verify magic"
);
// ...and so is the cvalue: one bit off and the candidate no longer opens
// the magic.
let mut bad = recs.clone();
bad.iter_mut()
.find(|r| r.rec_type == 0x05)
.expect("0x05 present")
.body[0] ^= 0x01;
assert!(walk_processing_key(&bad, std::slice::from_ref(&dk)).is_none());
}
}
+101 -840
View File
File diff suppressed because it is too large Load Diff
+9 -1
View File
@@ -99,7 +99,11 @@ pub mod coding_type {
/// Secondary Dolby Digital Plus audio (BD-ROM convention).
pub const AC3_PLUS_SECONDARY: u8 = 0xA1;
/// Secondary DTS-HD audio (lossless MA, not lossy HR) (BD-ROM convention).
/// Secondary DTS-HD audio — DTS Express / DTS-HD LBR, a LOSSY low-bitrate
/// stream for picture-in-picture and BD-J mixing (BD-ROM Part 3
/// `stream_coding_type` table). The lossless primary is [`DTS_HD_MA`]
/// (0x86); this code is its lossy secondary counterpart, parallel to
/// [`AC3_PLUS_SECONDARY`] (0xA1) on the Dolby side.
pub const DTS_HD_SECONDARY: u8 = 0xA2;
}
@@ -110,6 +114,10 @@ pub mod coding_type {
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
+43
View File
@@ -443,6 +443,49 @@ mod tests {
);
}
/// The length guard is a FLOOR, not a ceiling: `descramble_sector` is a
/// no-op below one sector, and processes the FIRST sector of anything at
/// least that long (the loop is `.take(2048)`). `css::descramble_sector` is
/// a public entry taking `&mut [u8]` of any length, so a caller handing it a
/// multi-sector buffer must get its first sector descrambled — a guard that
/// rejected over-long buffers would hand that caller its ciphertext back
/// unchanged, with the scramble flag cleared as if it had worked.
#[test]
fn descramble_processes_the_first_sector_of_an_over_long_buffer() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
// Two sectors' worth of buffer; only the first is a sector.
let mut buf = vec![0xAAu8; 4096];
buf[0x14] = 0x30;
buf[0x54..0x59].copy_from_slice(&seed);
let original = buf.clone();
descramble_sector(&title_key, &mut buf);
assert_ne!(
&buf[0x80..0x800],
&original[0x80..0x800],
"the first sector's body must be descrambled"
);
assert_eq!(buf[0x14] & 0x30, 0x00, "and its scramble flag cleared");
assert_eq!(
&buf[2048..4096],
&original[2048..4096],
"bytes past the first sector must be left untouched"
);
// The result must equal what a caller gets by passing exactly one
// sector — the same transform, not a length-dependent one.
let mut one = original[..2048].to_vec();
descramble_sector(&title_key, &mut one);
assert_eq!(
&buf[..2048],
&one[..],
"the first sector must descramble identically either way"
);
}
/// Descramble is keyed by `title_key XOR seed`: two different title keys
/// produce two different bodies for the same scrambled input. A cipher that
/// ignored the title key (or mixed it in wrongly) would yield identical
+687 -79
View File
File diff suppressed because it is too large Load Diff
+566
View File
@@ -450,6 +450,64 @@ mod tests {
}
}
/// `descramble_matches` is the ONLY gate between the LFSR search and a key
/// handed back to the caller: both [`recover_title_key`] and the crib-driven
/// `crack_title_key_inner` return a candidate only if this says the key
/// really descrambles the sector to the known plaintext. A body that always
/// answered `true` would let the first spurious LFSR-seed match through as
/// the title key — the ripper would then descramble the whole title with a
/// key that opens nothing, producing garbage rather than a "no key" error.
///
/// Pinned both directions: the genuine key is accepted, and EVERY key one
/// bit away from it is rejected. The one-bit neighbours are the strongest
/// form of wrong key — a gate that only rejects wildly different keys would
/// still pass a near-miss out of the 2^16 seed search.
#[test]
fn descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
assert!(
descramble_matches(&sector, &title_key, &PES),
"the key the sector was scrambled with must be accepted"
);
for byte in 0..5usize {
for bit in 0..8u32 {
let mut wrong = title_key;
wrong[byte] ^= 1u8 << bit;
assert!(
!descramble_matches(&sector, &wrong, &PES),
"key differing only in byte {byte} bit {bit} must be rejected"
);
}
}
}
/// The gate is applied to a COPY: verifying a candidate must not modify the
/// caller's sector. `recover_title_key` runs the gate and then hands the
/// sector on to be descrambled for real — if verification descrambled in
/// place, that second descramble would run over already-transformed bytes
/// (and, worse, a rejected candidate would leave the sector corrupted).
#[test]
fn descramble_matches_does_not_disturb_the_caller_s_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
let before = sector.clone();
assert!(descramble_matches(&sector, &title_key, &PES));
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
assert_eq!(
sector, before,
"verification must leave the sector byte-for-byte unchanged"
);
}
/// MANDATORY (Task C.1): the crib-based entry point crack_title_key —
/// no plaintext supplied — recovers a round-tripping key when the
/// cleartext ends in a periodic run that continues into 0x80.
@@ -571,4 +629,512 @@ mod tests {
let _ = crack_title_key(&sector);
}
}
// ── entry-point guards on caller- and disc-supplied lengths ────────────
/// A sector buffer that ENDS inside the encrypted region must be refused,
/// not sliced.
///
/// `recover_title_key` slices `sector[0x80..0x8A]` unconditionally after its
/// length guard. The existing short-sector test uses `SECTOR_BYTES - 1`,
/// which is still long enough for that slice to succeed — so the guard was
/// never the thing producing the `None`, and dropping it (or weakening the
/// `||` to `&&`, which a full-length crib satisfies) changed nothing
/// observable. On a real short read this is an out-of-bounds panic on the
/// rip thread.
#[test]
fn recover_rejects_a_sector_that_ends_inside_the_encrypted_region() {
for len in [0x81usize, 0x85, 0x89] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so no other guard fires first
assert!(
recover_title_key(&sector, &PES).is_none(),
"a {len}-byte buffer cannot supply ten ciphertext bytes at 0x80"
);
}
}
/// A buffer LONGER than one sector is still one sector: both entry points
/// read the first `SECTOR_BYTES` and must recover the key from it.
///
/// Callers read DVD data in multi-sector blocks, so an over-long slice is
/// the normal case, not an exotic one. A length guard that rejected it
/// (`len > SECTOR_BYTES` instead of `<`) would make every block-read caller
/// silently unable to crack anything.
#[test]
fn a_buffer_longer_than_one_sector_still_yields_its_key() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let mut padded = sector.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
recover_title_key(&padded, &PES),
Some(title_key),
"a two-and-a-bit-sector buffer must still recover the first sector's key"
);
let (periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
let mut padded = periodic.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
crack_title_key(&padded),
crack_title_key(&periodic),
"padding past the sector must not change the crack result"
);
assert!(crack_title_key(&padded).is_some());
}
/// `recover_title_key` accepts MORE than ten bytes of known plaintext, and
/// uses all of it: the extra bytes tighten the `descramble_matches` gate.
/// The ten-byte figure is a MINIMUM (the cipher is iterated ten times), not
/// an exact requirement — a guard reading it as an upper bound would reject
/// every caller that knows a longer crib.
#[test]
fn recover_accepts_more_than_ten_bytes_of_known_plaintext() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let long_plain: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(37).wrapping_add(5))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &long_plain);
assert_eq!(
recover_title_key(&sector, &long_plain),
Some(title_key),
"64 bytes of known plaintext must be accepted, not rejected as \
'more than ten'"
);
}
/// The scramble-flag gate on a sector whose BODY really is ciphertext.
///
/// Both entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
/// unscrambled sector has no title key to recover, and its bytes at 0x80
/// are already plaintext. Every prior test of this gate used an all-zero or
/// all-`0x11` sector, where the recovery would have found nothing anyway —
/// so widening the mask test (`&` to `|`, which makes it true for EVERY
/// flag byte) produced the same `None` and went unseen.
///
/// Here the sector is genuinely scrambled and its key IS recoverable; only
/// the cleared flag stands in the way. If the gate stops working, both
/// functions start returning keys for sectors the disc says are in the
/// clear.
#[test]
fn a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key(&sector, &PES),
Some(title_key),
"fixture check: with the flag set this sector's key IS recoverable"
);
sector[FLAG_BYTE] = 0x00;
assert_eq!(
recover_title_key(&sector, &PES),
None,
"scramble bits clear → no title key, even though one could be found"
);
let (mut periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
assert!(
crack_title_key(&periodic).is_some(),
"fixture check: with the flag set this sector cracks"
);
assert!(
attack_crib(&periodic).is_some(),
"fixture check: with the flag set this sector has a usable crib"
);
periodic[FLAG_BYTE] = 0x00;
assert_eq!(
crack_title_key(&periodic),
None,
"scramble bits clear → no crack, even though one would succeed"
);
// `attack_crib` carries its own copy of the same gate, and it is the one
// that actually stops the crack (`crack_title_key`'s is defensive
// duplication). The crib doubles as the decrypt path's cached-key
// oracle, so a widened mask there would hand that path a "predicted
// plaintext" for sectors that were never scrambled.
assert_eq!(
attack_crib(&periodic),
None,
"an unscrambled sector has no predicted plaintext to offer"
);
}
// ── descramble_matches: the verification gate's own mechanics ──────────
/// The gate must verify a candidate against the sector's CIPHERTEXT
/// regardless of what the sector's own flag byte says.
///
/// `descramble_matches` forces `0x10` on its copy precisely because
/// [`super::lfsr::descramble_sector`] is a no-op when the scramble bits are
/// clear — without that, verifying a scrambled-but-unflagged sector
/// compares raw ciphertext against the crib, and every candidate key is
/// rejected. Nothing exercised it: every fixture already had the flag set,
/// where forcing the bit is a no-op.
#[test]
fn descramble_matches_forces_the_scramble_flag_on_its_own_copy() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
sector[FLAG_BYTE] = 0x00;
assert!(
descramble_matches(&sector, &title_key, &PES),
"the body is ciphertext and the key is right — the gate must \
descramble it even though the flag byte says otherwise"
);
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
}
/// The gate compares the WHOLE supplied plaintext, clamped to the encrypted
/// region.
///
/// Two properties in one, because they are the two halves of
/// `plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
///
/// - it must compare beyond the first sixteen bytes, or a key that opens
/// only the head of the crib is accepted; and
/// - it must never compare past the end of the sector — a caller that
/// knows more plaintext than the 1920-byte encrypted region holds
/// otherwise indexes off the end of the buffer and panics.
#[test]
fn descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let body: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(29).wrapping_add(3))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &body);
assert!(descramble_matches(&sector, &title_key, &body));
// A crib agreeing for the first 16 bytes and diverging after must be
// rejected: the comparison window is the crib's length, not a fixed
// prefix.
let mut tail_wrong = body.clone();
tail_wrong[40] ^= 0xFF;
assert!(
!descramble_matches(&sector, &title_key, &tail_wrong),
"a crib that diverges at byte 40 must not match"
);
assert_eq!(
tail_wrong[..16],
body[..16],
"fixture check: the first 16 bytes are identical, so only a \
comparison that runs past them can tell these apart"
);
// A crib LONGER than the encrypted region: the comparison is clamped to
// the sector, not run off the end of it.
let plain_len = SECTOR_BYTES - ENCRYPTED_START;
let mut over_long = vec![0u8; plain_len + 10];
let (full_sector, full_body) = synth_sector(&title_key, &seed, &[0x00u8; 10]);
over_long[..plain_len].copy_from_slice(&full_body[ENCRYPTED_START..]);
assert!(
descramble_matches(&full_sector, &title_key, &over_long),
"a crib longer than the encrypted region must be clamped, not \
compared past the end of the sector"
);
}
// ── attack_crib: known-answer vectors ──────────────────────────────────
//
// `attack_crib` is BOTH the cracker's known plaintext and the decrypt
// path's "did the cached key descramble correctly?" oracle. Until now it
// was only ever exercised end-to-end through `crack_title_key`, on a
// fixture whose periodic run covered 39 bytes (0x59..0x80) — long enough
// that the run start, the cycle count and the `i % best_p` wrap were all
// slack. A crib that silently drifts costs a rip its title key.
/// Build a sector whose clear header ends in a `period`-length repeating
/// run of exactly `run_len` bytes immediately before 0x80.
///
/// The run is anchored to ABSOLUTE sector offset (`sec[x] = pat[x % period]`),
/// which is what makes "the run continues past 0x80" a statement independent
/// of the code under test: the byte at `0x80 + i` of the underlying
/// plaintext is `pat[(0x80 + i) % period]`.
///
/// Everything before the run is `0x00` (the pattern bytes are all >= 0xD0,
/// so the run cannot be extended backwards by accident), and the encrypted
/// region is filled with `0xFF` — so a crib that reads past 0x80 into
/// "ciphertext" is immediately visible.
fn sector_with_trailing_run(period: usize, run_len: usize) -> Vec<u8> {
assert!(
run_len < ENCRYPTED_START,
"the run lives in the clear header"
);
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for x in (ENCRYPTED_START - run_len)..ENCRYPTED_START {
sector[x] = pat[x % period];
}
sector
}
/// The crib the run PREDICTS: the periodic pattern continued past 0x80.
fn expected_crib(period: usize) -> [u8; 10] {
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
let mut out = [0u8; 10];
for (i, o) in out.iter_mut().enumerate() {
*o = pat[(ENCRYPTED_START + i) % period];
}
out
}
/// KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly
/// at 0x80, the crib is the run continued forward — the same ten bytes for
/// every run length, because the prediction depends only on the pattern and
/// the phase, never on how many cycles happened to be visible.
///
/// The short lengths are the load-bearing ones: at `run_len = 11` the crib
/// window starts at 0x76 and is only 10 bytes from the end of the header, so
/// any drift in `plain_start`, in `cycles * best_p`, or in the `i % best_p`
/// wrap reads the 0xFF "ciphertext" instead of the run.
#[test]
fn attack_crib_predicts_the_periodic_run_continuing_past_0x80() {
for &run_len in &[11usize, 12, 13, 14, 15, 16, 20, 31] {
let sector = sector_with_trailing_run(5, run_len);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(5)),
"period-5 run of {run_len} bytes must predict the run continuing"
);
}
}
/// The same known answer across several periods, including a period that
/// does NOT divide 0x80 (so the crib's phase is non-zero and a body that
/// restarted the pattern at index 0 gives a different answer).
#[test]
fn attack_crib_recovers_the_run_period_and_phase() {
// 0x80 % period: 3 for 5, 2 for 6, 2 for 7, 8 for 0x18 — all non-zero,
// so the predicted first byte is NOT pat[0] in any of these cases.
for &period in &[5usize, 6, 7, 0x18] {
let sector = sector_with_trailing_run(period, 3 * period + 1);
let crib =
attack_crib(&sector).unwrap_or_else(|| panic!("no crib for period {period}"));
assert_eq!(crib, expected_crib(period), "period {period}");
assert_ne!(
crib[0], 0xD0,
"period {period} does not divide 0x80, so the crib must not \
start at pattern index 0"
);
assert!(
crib.iter().all(|&b| b != 0xFF),
"period {period}: the crib must never contain a byte read from \
the encrypted region"
);
}
}
/// A run of exactly ONE cycle (plus the trivial tail the detector counts) is
/// not enough to predict forward: [`attack_crib`] requires at least two full
/// cycles. Weakening that guard would let a one-off byte sequence be
/// declared periodic and produce a confidently wrong crib — which the
/// decrypt path uses as its "is my cached key still right?" oracle.
#[test]
fn attack_crib_refuses_a_run_shorter_than_two_cycles() {
// period 8, run of 9 bytes: best_plen = 8, 8 / 8 == 1 cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 9)), None);
// period 0x18, run of 0x19 bytes: one cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(0x18, 0x19)), None);
// ...and one more byte of run does not conjure a second cycle either.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 10)), None);
}
/// A header with no repeating tail at all yields no crib. Asserted on a
/// header whose bytes are pairwise distinct right up to 0x80, so no cycle
/// length in 2..0x2F can match even one byte.
#[test]
fn attack_crib_refuses_a_header_with_no_periodic_tail() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
// 0x00..0x80 strictly increasing: sec[a] == sec[b] iff a == b, so the
// detector's `sec[0x7f - (j % i)] == sec[0x7f - j]` needs j % i == j,
// which the scan's starting `j = i + 1` already excludes.
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = x as u8;
}
assert_eq!(attack_crib(&sector), None);
// And the cracker built on it reports no key rather than guessing.
assert_eq!(crack_title_key(&sector), None);
}
/// `attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
/// own length guard is the only thing between a short buffer and an
/// out-of-bounds read. Nothing reached it: every caller-level test used a
/// full sector, and the entry points' guards fire first.
#[test]
fn attack_crib_refuses_a_buffer_shorter_than_a_sector() {
for len in [0x15usize, 0x40, 0x7F, SECTOR_BYTES - 1] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so the flag half cannot fire
assert_eq!(
attack_crib(&sector),
None,
"a {len}-byte buffer is not a sector"
);
}
}
/// A header that is periodic ALL THE WAY to offset 0 must not walk the
/// backward scan off the front of the sector.
///
/// The detector counts backwards from 0x7f while `j < 0x80`. On a fully
/// periodic header the run never breaks, so `j` reaches 0x7f and the bound
/// is the ONLY thing that stops it — one step further and `0x7f - j`
/// underflows a `usize` and panics. A constant or fully-patterned 128-byte
/// header is ordinary DVD data (padding, a run of zeros), not a crafted
/// input, and every existing fixture had a filler/run boundary well before
/// offset 0 that stopped the scan early.
#[test]
fn attack_crib_survives_a_header_that_is_periodic_to_offset_zero() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x30;
let period = 5usize;
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = pat[x % period];
}
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
// The FLAG byte sits inside the header at 0x14, so it interrupts the
// pattern there; re-lay it and accept that 0x14 breaks the run — the
// scan still reaches offset 0x15 - 1 = 0x14 going backwards, i.e.
// j = 0x7f - 0x14 = 0x6b, well short of the bound. Instead put the
// scramble flag bits into a byte value that IS the pattern's.
sector[FLAG_BYTE] = pat[FLAG_BYTE % period];
assert_ne!(
sector[FLAG_BYTE] & 0x30,
0,
"fixture check: the pattern byte at 0x14 must itself carry \
scramble bits, so the header stays unbroken"
);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(period)),
"a fully periodic header must predict its own continuation, and \
the backward scan must stop at offset 0"
);
}
/// The crib is read from the CLEAR header only. A run that reaches 0x80 must
/// predict from the header bytes, never from the encrypted region — the
/// previously-fixed bug this function's doc comment records. Pinned by
/// rewriting the encrypted region and requiring the crib not to move.
#[test]
fn attack_crib_is_independent_of_the_encrypted_region() {
let base = sector_with_trailing_run(5, 11);
let crib = attack_crib(&base).expect("crib");
for fill in [0x00u8, 0x5A, 0xD1, 0xFF] {
let mut s = base.clone();
for b in s[ENCRYPTED_START..].iter_mut() {
*b = fill;
}
assert_eq!(
attack_crib(&s),
Some(crib),
"the crib must not depend on the encrypted region (fill {fill:#04x})"
);
}
}
// ── recover_title_key_from_plain: input-length guard ───────────────────
/// `recover_title_key_from_plain` unconditionally builds a 10-byte keystream
/// buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard
/// is the only thing standing between a short slice and an
/// index-out-of-bounds PANIC.
///
/// Nothing reached that guard before: `recover_title_key` rejects
/// `plain.len() < 10` at its own door and always hands on exactly ten
/// ciphertext bytes, and `crack_title_key_inner` always passes a fixed
/// `[u8; 10]` crib. The guard is a live contract for any future caller and
/// was executed by no test at either boundary.
#[test]
fn recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input() {
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let full = [0xA5u8; 10];
for n in 0..10usize {
assert_eq!(
recover_title_key_from_plain(&full[..n], &full, &seed),
None,
"{n} ciphertext bytes is fewer than the ten the cipher iterates"
);
assert_eq!(
recover_title_key_from_plain(&full, &full[..n], &seed),
None,
"{n} plaintext bytes is fewer than the ten the cipher iterates"
);
}
// Exactly ten of each is ACCEPTED as far as the search — the boundary is
// `< 10`, not `<= 10`. (Whether this particular keystream has a seed is
// immaterial; what must not happen is an early `None` from the guard.)
// Proven through the round-trip fixture, whose inputs are exactly ten
// bytes and which does recover its key.
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key_from_plain(
&sector[ENCRYPTED_START..ENCRYPTED_START + 10],
&PES,
&seed
),
Some(title_key),
"exactly ten bytes of each input must run the search, not trip the guard"
);
}
/// The seed XOR-back ([`recover_title_key_from_plain`]'s last step) is what
/// turns the recovered LFSR key into the TITLE key: `key ^= sector_seed`.
/// Pinned as a known answer across seeds that differ only in one byte — the
/// same ciphertext/plaintext pair therefore must yield title keys differing
/// in exactly that byte.
///
/// Without this, a body that ORed the seed in (or dropped the step) still
/// round-trips on any fixture whose seed is zero, and on the non-zero ones
/// the failure looks like "no key found" rather than a wrong step.
#[test]
fn recover_title_key_from_plain_xors_the_sector_seed_back_out() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let crypted = &sector[ENCRYPTED_START..ENCRYPTED_START + 10];
// The cipher is seeded from `title_key XOR seed`, so re-running the SAME
// ciphertext/plaintext against a seed differing in one byte must return
// a title key differing in exactly that byte — the XOR is a bijection.
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &seed),
Some(title_key)
);
for byte in 0..5usize {
for bit in [0u32, 3, 7] {
let mut alt_seed = seed;
alt_seed[byte] ^= 1u8 << bit;
let mut expected = title_key;
expected[byte] ^= 1u8 << bit;
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &alt_seed),
Some(expected),
"seed byte {byte} bit {bit} must XOR straight through to the \
title key"
);
}
}
}
}
+662 -64
View File
@@ -74,9 +74,14 @@ pub fn set_decrypt_threads(n: usize) {
DECRYPT_THREADS.store(clamped, Ordering::Relaxed);
// Drop the existing pool. Next decrypt_pool() call rebuilds with
// the new resolved thread count.
if let Ok(mut guard) = DECRYPT_POOL.write() {
*guard = None;
}
//
// Recover the guard on poisoning, exactly as `decrypt_pool` does. Skipping
// the swap on a poisoned lock silently kept the STALE pool alive while the
// atomic above already reported the new thread count, so the setting appeared
// to take effect and never did. The pool Arc is immutable once stored, so a
// prior panic cannot have left it half-written.
let mut guard = DECRYPT_POOL.write().unwrap_or_else(|e| e.into_inner());
*guard = None;
}
/// Get (or lazily build) the active rayon thread pool. Returns an
@@ -184,6 +189,34 @@ pub enum Phase {
Odd,
}
/// Does this unit belong to the phase we hold the key for?
///
/// An FMTS forensic segment interleaves two variants at the unit level: the
/// disc carries both, and we hold the key for exactly one parity. Decrypting
/// the alternate half with our key produces garbage; leaving it as ciphertext
/// is correct, because the muxer drops untouched ciphertext cleanly.
///
/// `Phase::All` means the whole range is ours (the non-forensic case).
///
/// This lived inline inside `apply_aacs_map`'s per-unit closure, where nothing
/// could reach it: a mutation run flipped the `-` to `+` and the `/` to `*` in
/// the index arithmetic and every test still passed. Getting either wrong
/// silently decrypts the wrong half of a forensic segment.
fn unit_is_our_phase(unit_lba: u32, range_start: u32, unit_sectors: u32, phase: Phase) -> bool {
let want_odd = match phase {
Phase::All => return true,
Phase::Even => false,
Phase::Odd => true,
};
// `saturating_sub` and `max(1)`: both inputs come from the key map, which is
// built from disc structure. A unit below its own range start, or a zero
// unit size, means the map is malformed — that must not panic (debug
// overflow / divide-by-zero) inside a library used by a long-running
// service. Unit 0 of the range is even, which is the safe default.
let unit_ix = unit_lba.saturating_sub(range_start) / unit_sectors.max(1);
(unit_ix % 2 == 1) == want_odd
}
/// Proactive AACS key-selection map: which held unit key decrypts each LBA of a
/// title's encrypted content, decided ONCE before mux from the disc's CPS-unit
/// (and, later, FMTS segment) structure — never by trial-decrypt-and-check per
@@ -327,11 +360,11 @@ impl AacsKeyMap {
if sectors == 0 {
return;
}
if let Some(last) = plan.last_mut() {
if last.start_lba.saturating_add(last.sector_count) == lba {
last.sector_count += sectors;
return;
}
if let Some(last) = plan.last_mut()
&& last.start_lba.saturating_add(last.sector_count) == lba
{
last.sector_count += sectors;
return;
}
plan.push(crate::disc::Extent {
start_lba: lba,
@@ -381,7 +414,26 @@ impl AacsKeyMap {
/// decorator can dispatch uniformly. A map index outside the held pool is a
/// fail-loud [`Error::DecryptFailed`]: the resolver's job is to guarantee every
/// selectable index is present, so a gap here is a resolver bug, not silent loss.
pub fn decrypt_sectors_mapped(
/// Decrypt `buf` with a resolved AACS key map. Thin wrapper over
/// [`decrypt_span`] — the map is the AACS scheme's input, not a second
/// orchestrator.
pub(crate) fn decrypt_sectors_mapped(
buf: &mut [u8],
keys: &DecryptKeys,
base_lba: u32,
map: &AacsKeyMap,
) -> Result<(), crate::error::Error> {
let mut keys = keys.clone();
decrypt_span(buf, &mut keys, base_lba, Some(map), None).map(|_| ())
}
/// AACS scheme step: apply `map`'s per-unit keys to `buf`.
///
/// A SCHEME, not a policy. It reports what it could not open by returning
/// `Err(DecryptFailed)`; the decision that an unopenable unit must never be
/// emitted belongs to [`decrypt_span`], which is the one place that decides it
/// for every scheme.
fn apply_aacs_map(
buf: &mut [u8],
keys: &DecryptKeys,
base_lba: u32,
@@ -436,20 +488,31 @@ pub fn decrypt_sectors_mapped(
return;
}
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
// No range covers this LBA → the map keys no content here, so pass the
// unit through untouched (clear filesystem / nav on a whole-disc read).
// No range covers this LBA. That is expected for clear filesystem / nav
// on a whole-disc read — but "the map has no key here" and "there is
// nothing to decrypt here" are different statements, and only the
// second makes passing the unit through correct.
//
// An ENCRYPTED unit outside every range is content we cannot key: on a
// multi-CPS disc that is an orphan clip referenced by no playlist, so
// it sits in no title extent and therefore in no range. Emitting it
// verbatim ships ciphertext where plaintext is meant to be, and extract
// then counts those bytes as good and reports the file complete.
//
// The split-unit branch immediately above already draws exactly this
// distinction. This one did not, so it was reached before the
// `aacs_unit_encrypted` gate below ever ran.
let Some((key_idx, phase, range_start)) = map.entry_for(unit_lba) else {
if aacs::content::aacs_unit_seed_encrypted(chunk, format) {
verify_failed.store(true, std::sync::atomic::Ordering::Relaxed);
}
return;
};
// PHASE GATE (FMTS forensic segment): the segment interleaves two variants
// at the unit level. Decrypt ONLY our parity; leave the alternate half as
// ciphertext (the muxer drops untouched ciphertext cleanly — no garble).
if matches!(phase, Phase::Even | Phase::Odd) {
let unit_ix = (unit_lba - range_start) / unit_sectors;
let is_odd = unit_ix % 2 == 1;
if is_odd != matches!(phase, Phase::Odd) {
return; // alternate half — leave as-is
}
if !unit_is_our_phase(unit_lba, range_start, unit_sectors, phase) {
return; // alternate half — leave as-is
}
// Gate on the authoritative encrypted flag ONLY (CPI bits in the clear
// seed): a clear unit is left untouched; an encrypted unit is decrypted
@@ -515,7 +578,8 @@ pub fn decrypt_sectors(
keys: &mut DecryptKeys,
unit_key_idx: usize,
) -> Result<usize, crate::error::Error> {
decrypt_sectors_impl(buf, keys, unit_key_idx, None)
let _ = unit_key_idx;
decrypt_span(buf, keys, 0, None, None)
}
/// Legacy alias of [`decrypt_sectors`]. Under the keymap-only model AACS decrypts
@@ -532,28 +596,50 @@ pub fn decrypt_sectors_in_content(
base_lba: u32,
content_ranges: &[(u32, u32)],
) -> Result<usize, crate::error::Error> {
decrypt_sectors_impl(buf, keys, unit_key_idx, Some((base_lba, content_ranges)))
let _ = unit_key_idx;
decrypt_span(buf, keys, base_lba, None, Some((base_lba, content_ranges)))
}
fn decrypt_sectors_impl(
/// THE decrypt orchestrator. Every path into this crate's decryption goes
/// through here.
///
/// How a disc decrypts is one process — resolve a key for this span, apply it,
/// and refuse if no key can be proven. Only the resolve-and-apply step is
/// scheme-specific. This function owns the loop and the refusal; the schemes
/// below supply only what genuinely differs between AACS, CSS and clear media.
///
/// That split exists because its absence caused six separate defects in one
/// release. There used to be TWO top-level paths — this one for CSS and clear,
/// and a wholly separate `decrypt_sectors_mapped` for AACS whose arm here was a
/// bare `return Err` stub — so each scheme decided its own answer to "there is
/// no key for these bytes" and nothing held them to the same one. CSS drifted to
/// descrambling with a key it had just proven stale; the mapped path drifted to
/// passing an unkeyable encrypted unit through as ciphertext. Both looked like
/// success to the caller.
///
/// Adding a scheme means adding an arm here, which means answering the refusal
/// question. That is the point.
fn decrypt_span(
buf: &mut [u8],
keys: &mut DecryptKeys,
// Unused now that AACS decrypts via the key map only; the CSS arm self-gates on
// its per-sector scramble flag and `None` is a no-op. Kept so the wrapper
// signatures (decrypt_sectors / _in_content) stay stable for CSS/None callers.
_unit_key_idx: usize,
base_lba: u32,
map: Option<&AacsKeyMap>,
_content: Option<(u32, &[(u32, u32)])>,
) -> Result<usize, crate::error::Error> {
let dropped: usize = match keys {
DecryptKeys::None => 0,
DecryptKeys::Aacs { .. } => {
// AACS decrypts EXCLUSIVELY through the resolved key map
// (`decrypt_sectors_mapped`): the map keys every content unit up front,
// and a missing key fails at RESOLVE time. The old trial-decrypt path
// (try each held key, keep the first-tried plaintext on a miss) is gone
// — reaching it means an AACS reader was built without installing its
// key map, which would silently apply a wrong key. Fail loud instead.
return Err(crate::error::Error::DecryptFailed);
// AACS decrypts EXCLUSIVELY through a resolved key map: the map keys
// every content unit up front and a missing key fails at RESOLVE
// time. No map here means an AACS reader was built without
// installing one — the old trial-decrypt path (try each held key,
// keep the first-tried plaintext on a miss) is gone precisely
// because it silently applied wrong keys.
let Some(map) = map else {
return Err(crate::error::Error::DecryptFailed);
};
apply_aacs_map(buf, keys, base_lba, map)?;
0
}
DecryptKeys::Css { title_key } => {
// CSS SELF-recovers: the title key changes per VOB region and is
@@ -562,8 +648,7 @@ fn decrypt_sectors_impl(
// `css::descramble_region`), and CSS does not need the post-decrypt
// recovery seam that AACS key-fetch / FMTS segment-skip use (those DO
// consume external inputs a `decrypt_sectors` caller cannot supply).
css::descramble_region(buf, title_key);
0
css::descramble_region(buf, title_key)?
}
};
Ok(dropped)
@@ -610,6 +695,49 @@ mod tests {
// ── `decrypt_sectors_in_content` (now a legacy alias of `decrypt_sectors`) ──
/// `DecryptKeys::None` is a no-op even with a content map + scrambled bytes.
/// A forensic map's read plan is NOT the extents it was given — and that is
/// the precondition the mux's provenance guard keys on.
///
/// A clip's feed span is measured over the FULL extents at scan time, while
/// the mux reads this reduced plan, so the byte offsets stamped on frames
/// and the offsets recorded in the spans describe different streams. The
/// deficit accumulates, so every frame after the first segment resolves to
/// an earlier clip than it came from. The spans still tile each other, so
/// the tiling check cannot see it; the mux compares the plan against the
/// full extents instead and stops trusting provenance when they differ.
#[test]
fn a_forensic_read_plan_drops_units_the_full_extents_include() {
let full = vec![crate::disc::Extent {
start_lba: 1000,
sector_count: 60,
}];
// No forensic segment: the plan IS the extents, byte for byte, so
// provenance stays trustworthy on an ordinary disc.
let plain = AacsKeyMap::from_ranges_phased(vec![(1000, 1060, 5, Phase::All)]);
assert_eq!(
plain.read_plan(&full, 3),
full,
"a non-forensic map must return the extents unchanged"
);
// With alternate phases, units are omitted — fewer sectors are read
// than the spans describe.
let phased = AacsKeyMap::from_ranges_phased(vec![(1000, 1060, 5, Phase::Even)]);
let plan = phased.read_plan(&full, 3);
let planned: u32 = plan.iter().map(|e| e.sector_count).sum();
let whole: u32 = full.iter().map(|e| e.sector_count).sum();
assert!(
planned < whole,
"a forensic segment must drop units: planned {planned} of {whole}"
);
assert_ne!(
plan, full,
"the plan differs from the extents, which is exactly what the mux \
detects before deciding whether a byte offset means anything"
);
}
#[test]
fn content_gate_none_keys_is_noop() {
let mut keys = DecryptKeys::None;
@@ -635,6 +763,92 @@ mod tests {
);
}
/// `decrypt_sectors_in_content` is the entry point `DecryptingSectorSource`
/// dispatches to whenever a content map is installed (`sector/decrypting.rs`
/// line ~211), so it is on the live read path for every mapped rip. It must
/// actually DECRYPT. The two `_is_noop` tests above only assert its `usize`
/// return is `0` — which is what a body replaced by `Ok(0)` also returns, so
/// neither one constrains it at all.
///
/// Here a genuinely scrambled CSS sector goes in and the CONSTRUCTED
/// plaintext must come out. Anything that skips `css::descramble_region` —
/// including a body that just reports `Ok(0)` — leaves ciphertext in the
/// buffer and the caller muxes scrambled MPEG at exit 0.
///
/// Expected bytes come from the plaintext this test built BEFORE scrambling
/// (CSS scrambles only 0x80..2048; the header stays clear), not from
/// re-running any descramble routine.
#[test]
fn content_gate_css_actually_descrambles_the_buffer() {
const RUN_START: usize = 0x59;
const SEED_OFFSET: usize = 0x54;
const PERIOD: usize = 8;
let title_key = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let mut plaintext = vec![0u8; 2048];
plaintext[0x00..0x04].copy_from_slice(&css::PACK_START);
plaintext[0x14] = 0x10; // CSS scramble flag (DVD-Video sector header)
let pat: Vec<u8> = (0..PERIOD)
.map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A)
.collect();
for (i, b) in plaintext.iter_mut().enumerate().skip(RUN_START) {
*b = pat[i % PERIOD];
}
plaintext[SEED_OFFSET..SEED_OFFSET + 5].copy_from_slice(&[0x01, 0x02, 0x03, 0x04, 0x05]);
let mut buf = plaintext.clone();
css::lfsr::scramble_sector(&title_key, &mut buf);
let ciphertext = buf.clone();
assert_ne!(
&ciphertext[0x80..],
&plaintext[0x80..],
"fixture malformed — the sector was not actually scrambled"
);
let mut keys = DecryptKeys::Css { title_key };
decrypt_sectors_in_content(&mut buf, &mut keys, 0, 0, &[(0, 1)])
.expect("CSS descramble must not fail");
// Report the first differing offset rather than dumping 1.9 KB.
let mismatch = (0x80..2048).find(|&i| buf[i] != plaintext[i]);
assert!(
mismatch.is_none(),
"the scrambled body must come back as the plaintext it was built \
from; first mismatch at offset {mismatch:?} (buf={:#04x} \
expected={:#04x}) a wrapper that decrypts nothing leaves the \
ciphertext in place and the caller muxes scrambled MPEG",
buf[mismatch.unwrap_or(0x80)],
plaintext[mismatch.unwrap_or(0x80)],
);
}
/// The AACS arm of the same entry point must fail LOUD. Under the
/// keymap-only model AACS decrypts exclusively through
/// `decrypt_sectors_mapped`; reaching this wrapper with AACS keys means a
/// reader was built without installing its key map, and continuing would
/// hand the caller ciphertext under an `Ok`. `DecryptFailed` is the correct
/// verdict per the function's own contract — it must not be softened into a
/// success with a zero count.
#[test]
fn content_gate_aacs_keys_fail_loud_not_ok_zero() {
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![(1, [0xAB; 16])],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
let original = scrambled_region(aacs::content::ALIGNED_UNIT_LEN);
let mut buf = original.clone();
let r = decrypt_sectors_in_content(&mut buf, &mut keys, 0, 0, &[(0, 3)]);
assert!(
matches!(r, Err(crate::error::Error::DecryptFailed)),
"AACS without an installed key map must be DecryptFailed, got {r:?}"
);
assert_eq!(
buf, original,
"and it must not have half-decrypted the buffer on the way out"
);
}
/// Build a Stevenson-crackable scrambled CSS sector for `title_key` (mirrors
/// `crackable_sector` in the css::mod tests): a periodic run in the clear
/// header continues past 0x80 into the encrypted region, so
@@ -690,7 +904,7 @@ mod tests {
// descramble-and-rekey lives in `css::descramble_region` (the recovery
// seam calls it); the region change must re-crack region B's key.
let mut ended = key_a;
css::descramble_region(&mut buf, &mut ended);
css::descramble_region(&mut buf, &mut ended).expect("descramble");
assert_eq!(
&buf[0x80..2048],
@@ -744,6 +958,12 @@ mod tests {
/// A CLEAR trailing partial (encrypted flag NOT set) is a legitimate content
/// tail and must pass through, never trip the guard above.
///
/// "Passes through" means byte-for-byte unchanged, not merely `Ok`. Asserting
/// only `is_ok()` let a mutant that corrupts the clear partial while still
/// returning `Ok` pass — which is the whole failure this test names.
/// Mutation: XOR any byte of the tail before returning -> the snapshot
/// comparison fails.
#[test]
fn aacs_clear_trailing_partial_passes_through() {
let keys = DecryptKeys::Aacs {
@@ -755,8 +975,14 @@ mod tests {
let mut tail = clear_ts_region(4096);
tail[0] &= 0x3F; // ensure the CPI bits are clear
buf.extend_from_slice(&tail);
let snapshot = buf.clone();
let map = AacsKeyMap::from_ranges(vec![(0, u32::MAX, 0)]);
assert!(decrypt_sectors_mapped(&mut buf, &keys, 0, &map).is_ok());
decrypt_sectors_mapped(&mut buf, &keys, 0, &map)
.expect("a clear trailing partial is legitimate content");
assert_eq!(
buf, snapshot,
"a clear trailing partial must pass through byte-for-byte, not just return Ok"
);
}
// ── DecryptKeys::None and is_encrypted ─────────────────────────────────
@@ -804,6 +1030,12 @@ mod tests {
/// scramble flag.
fn make_css_sector(title_key: &[u8; 5], seed: &[u8; 5], body_fill: u8) -> (Vec<u8>, Vec<u8>) {
let mut sector = vec![body_fill; 2048];
// A real scrambled DVD sector is an MPEG-2 PS pack, so it begins with
// the pack start code. The descrambler requires it before trusting
// byte 0x14 — without it this fixture is a sector shape that cannot
// occur on a disc, and the test would pass while the production gate
// rejected every sector like it.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] = 0x30; // scramble flag (bits 4-5)
sector[0x54..0x59].copy_from_slice(seed);
let plaintext = sector.clone();
@@ -827,7 +1059,7 @@ mod tests {
let (mut sector, plaintext) = make_css_sector(&title_key, &seed, 0xA5);
// CSS descramble lives in `css::descramble_region` (the recovery seam
// calls it); `decrypt_sectors` only flags CSS sectors for recovery.
css::descramble_region(&mut sector, &mut title_key);
css::descramble_region(&mut sector, &mut title_key).expect("descramble");
assert_eq!(
&sector[0x80..2048],
&plaintext[0x80..2048],
@@ -857,7 +1089,7 @@ mod tests {
let mut buf = s0;
buf.extend_from_slice(&s1);
let mut title_key = title_key;
css::descramble_region(&mut buf, &mut title_key);
css::descramble_region(&mut buf, &mut title_key).expect("descramble");
assert_eq!(
&buf[0x80..2048],
&p0[0x80..2048],
@@ -881,6 +1113,9 @@ mod tests {
period: usize,
) -> (Vec<u8>, Vec<u8>) {
let mut plaintext = vec![0u8; 2048];
// Real scrambled DVD sectors are MPEG-2 PS packs; the scramble policy
// requires the pack start code as well as the flag bits.
plaintext[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plaintext[0x14] = 0x10; // scramble flag
// Periodic run from 0x59 (just above the seed) through 0x80 and on into
// the encrypted region; phase anchored to offset 0 so it is continuous
@@ -936,7 +1171,7 @@ mod tests {
// Cache primed to key_a only — exactly what the one-shot scan crack yields.
let mut title_key = key_a;
css::descramble_region(&mut buf, &mut title_key);
css::descramble_region(&mut buf, &mut title_key).expect("descramble");
assert_eq!(
&buf[0x80..2048],
@@ -1048,34 +1283,15 @@ mod tests {
// ── Multi-CPS-unit key selection ──────────────────────────────────────
/// Encrypt an aligned unit with the AACS algorithm run in reverse so that
/// `aacs::content::decrypt_unit` with the same key recovers the plaintext. Mirrors
/// the `aacs_encrypt_unit` helper in `aacs::content::tests`.
/// Encrypt an aligned unit so `aacs::content::decrypt_unit` with the same key
/// recovers the plaintext, flagging it encrypted first (bytes 0..16 are the key
/// seed, so the flag must be set before the crypto runs).
fn aacs_encrypt_unit_for_test(unit: &mut [u8], unit_key: &[u8; 16]) {
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
// CPI bits on byte 0 so the unit reads as encrypted; set before deriving
// the per-unit key so the recovered plaintext header matches.
unit[0] |= 0xC0;
let header: [u8; 16] = unit[..16].try_into().unwrap();
let derived = crate::aacs::crypto::aes_ecb_encrypt(unit_key, &header);
let mut k = [0u8; 16];
for i in 0..16 {
k[i] = derived[i] ^ header[i];
}
let cipher = Aes128::new(GenericArray::from_slice(&k));
let mut prev = crate::aacs::crypto::AACS_IV;
let num_blocks = (aacs::content::ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..num_blocks {
let off = 16 + i * 16;
for j in 0..16 {
unit[off + j] ^= prev[j];
}
let mut block = GenericArray::clone_from_slice(&unit[off..off + 16]);
cipher.encrypt_block(&mut block);
unit[off..off + 16].copy_from_slice(&block);
prev.copy_from_slice(&unit[off..off + 16]);
}
assert!(
aacs::content::encrypt_unit(unit, unit_key),
"a full-length unit must encrypt"
);
}
/// Build a clear aligned unit with TS sync bytes placed at the BD-TS stride
@@ -1199,6 +1415,256 @@ mod tests {
}
}
/// A forensic range does NOT start on an aligned-unit boundary. Its start LBA
/// comes from a source-packet number — `start_spn * 192` put through
/// `clip_byte_to_lba` (`mux/resolve.rs`) — and 192-byte packets have no
/// relationship to the 3-sector aligned unit, so `range_start % 3` is
/// whatever the disc says.
///
/// That makes `unit_ix = (lba - range_start) / us` load-bearing in both of
/// its operations, and the existing coverage used a range starting exactly
/// on the extent's first unit, where several wrong formulas agree with the
/// right one by arithmetic accident.
///
/// Getting the parity wrong is not a crash. It reads and decrypts the
/// ALTERNATE variant's half of a forensic segment: the units this disc's
/// key does not open decrypt to garbage, and the units it does open are
/// skipped. AACS 2.1 forensic marking is exactly the mechanism that makes
/// the two halves different, so a phase inversion is silent — it produces a
/// full-length rip carrying the wrong variant.
#[test]
fn read_plan_phase_parity_is_measured_from_an_unaligned_range_start() {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// Case A — range_start is itself unaligned (1001 % 3 == 2) and the extent
// begins on it, so unit offsets are 0, 3, 6, ... Under this shape the
// formula `(lba + range_start) / us` shifts every index by an ODD amount
// and inverts the kept half.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1001, 1013, 5, Phase::Even)]);
assert_eq!(
map.read_plan(&ext, us),
vec![
Extent {
start_lba: 1001,
sector_count: 3
}, // ix 0, even
Extent {
start_lba: 1007,
sector_count: 3
}, // ix 2, even
],
"unit index must be measured as (lba - range_start), so the kept \
units are the even-indexed ones counting from the range start"
);
// Case B — the extent begins one unit-remainder away from the range start
// (1001 - 1000 = 1), so offsets are 1, 4, 7, 10. Here `(lba - range_start)
// * us` inverts the halves instead: the division is what maps a byte
// offset onto a unit index, and multiplying happens to preserve parity
// only when the offset is already a multiple of the unit size.
let ext = vec![Extent {
start_lba: 1001,
sector_count: 12,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1000, 1013, 5, Phase::Even)]);
assert_eq!(
map.read_plan(&ext, us),
vec![
Extent {
start_lba: 1001,
sector_count: 3
}, // (1001-1000)/3 = 0, even
Extent {
start_lba: 1007,
sector_count: 3
}, // (1007-1000)/3 = 2, even
],
"the offset must be DIVIDED by the unit size to become a unit index"
);
}
/// An extent whose last whole unit is an alternate-phase unit must still drop
/// it. The tail guard exists for a REMNANT shorter than a unit — bytes with
/// no following unit to desync — and an extent ending exactly on a unit
/// boundary has no remnant at all.
///
/// With the guard widened to `remaining <= us`, the final unit of every
/// extent bypasses the phase gate and is read unconditionally. On a forensic
/// segment that lands at an extent end, that is one alternate-variant unit
/// pulled into the rip and decrypted with a key that does not open it.
#[test]
fn read_plan_gates_the_last_whole_unit_of_an_extent_not_just_the_remnant() {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// Two whole units, no remnant. ix 0 is even (kept), ix 1 is odd (dropped)
// and it is the LAST thing in the extent.
let ext = vec![Extent {
start_lba: 1000,
sector_count: 6,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1000, 1006, 5, Phase::Even)]);
assert_eq!(
map.read_plan(&ext, us),
vec![Extent {
start_lba: 1000,
sector_count: 3
}],
"the trailing odd-phase unit is a whole unit and must be dropped; the \
short-tail guard is for a remnant SMALLER than a unit"
);
// And the remnant case the guard is actually for: 4 sectors = one whole
// unit plus a 1-sector tail. The tail is ordinary content and is kept
// even though the unit before it was dropped.
let ext = vec![Extent {
start_lba: 1000,
sector_count: 4,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1000, 1004, 5, Phase::Odd)]);
assert_eq!(
map.read_plan(&ext, us),
vec![Extent {
start_lba: 1003,
sector_count: 1
}],
"a sub-unit remnant is ordinary content and is always read"
);
}
/// Every scheme that CANNOT prove a key answers the same way.
///
/// This is the property `decrypt_span` exists to hold. There used to be two
/// top-level decrypt paths — one for CSS and clear media, one for AACS —
/// and each decided its own answer, so they drifted apart in opposite
/// directions within a single release: CSS descrambled with a key it had
/// just proven stale, and the AACS path passed an unkeyable encrypted unit
/// through as ciphertext. Both reported success.
///
/// Asserting one verdict across the schemes is what makes a future
/// divergence a test failure rather than a silent corruption. A per-scheme
/// test cannot do that: each would still pass while the two disagreed.
///
/// CSS is deliberately NOT in this list. Its title key is recovered from
/// the data, sector by sector, by a heuristic that false-positives — a crib
/// mismatch whose re-crack fails means the crib was wrong, not that the key
/// is stale, so the cached key is kept and used. Round 9 folded CSS in here
/// on the reasoning that "no key" should mean one thing everywhere; that
/// made real DVDs unrippable, and the real-media gate caught it. Uniform
/// policy is right for schemes that can PROVE a key wrong. CSS cannot.
#[test]
fn every_scheme_gives_the_same_verdict_when_no_key_can_be_proven() {
use crate::disc::ContentFormat;
let ul = aacs::content::ALIGNED_UNIT_LEN;
// AACS, encrypted, no map installed at all.
let mut aacs_keys = DecryptKeys::Aacs {
unit_keys: vec![(0, [0xAAu8; 16])],
read_data_key: None,
format: ContentFormat::BdTs,
};
let mut buf = vec![0u8; ul];
let aacs_no_map = decrypt_span(&mut buf, &mut aacs_keys, 0, None, None)
.expect_err("an AACS reader with no key map cannot prove any key");
// AACS, encrypted, mapped but the unit falls outside every range.
let mut orphan = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut orphan, &[0xCCu8; 16]);
let mut buf = orphan.to_vec();
let empty = AacsKeyMap::from_ranges(vec![]);
let aacs_unmapped = decrypt_span(&mut buf, &mut aacs_keys, 0, Some(&empty), None)
.expect_err("an encrypted unit no range covers cannot be keyed");
let want = crate::error::Error::DecryptFailed.code();
for (what, e) in [
("AACS, no map", aacs_no_map),
("AACS, unit outside every range", aacs_unmapped),
] {
assert_eq!(
e.code(),
want,
"{what}: every scheme must refuse identically, or one of them is \
quietly emitting data it could not decrypt"
);
}
// And clear media is NOT a refusal — the shared policy must not turn
// "nothing to decrypt" into an error.
let mut none_keys = DecryptKeys::None;
let mut buf = vec![0u8; 2048];
assert!(
decrypt_span(&mut buf, &mut none_keys, 0, None, None).is_ok(),
"clear media has no key to prove and must pass through"
);
}
/// An ENCRYPTED unit that falls outside every key-map range must fail, not
/// pass through as ciphertext.
///
/// "The map has no key here" and "there is nothing to decrypt here" are
/// different statements, and only the second makes passing the unit through
/// correct. On a multi-CPS disc an orphan clip — referenced by no playlist,
/// so in no title extent and therefore in no range — hits the first and was
/// treated as the second. `extract_tree` then counted those bytes as GOOD,
/// dropped the `.partial` suffix, and reported `complete: true`, exit 0:
/// a scrambled file on disk with a clean bill of health.
///
/// A CLEAR unit outside every range is the ordinary case (filesystem and
/// nav on a whole-disc read) and must still pass through untouched — so
/// this asserts both directions.
#[test]
fn an_encrypted_unit_outside_every_key_range_fails_instead_of_passing_through() {
use crate::disc::ContentFormat;
let key = [0xAAu8; 16];
let ul = aacs::content::ALIGNED_UNIT_LEN;
let usz = (ul / 2048) as u32;
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, key)],
read_data_key: None,
format: ContentFormat::BdTs,
};
// The map covers unit 0 only. Unit 1 is the orphan.
let map = AacsKeyMap::from_ranges(vec![(0, usz, 0)]);
// Clear orphan: untouched, no error. This is nav/filesystem.
let mut clear_buf = vec![0u8; 2 * ul];
let mut u0 = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u0, &key);
clear_buf[..ul].copy_from_slice(&u0);
clear_buf[ul..].copy_from_slice(&clear_ts_unit());
let orphan_before = clear_buf[ul..].to_vec();
decrypt_sectors_mapped(&mut clear_buf, &keys, 0, &map)
.expect("a CLEAR unit outside the map is ordinary nav and must pass");
assert_eq!(
&clear_buf[ul..],
&orphan_before[..],
"a clear out-of-range unit must be left byte-identical"
);
// Encrypted orphan: must fail loud.
let mut enc_buf = vec![0u8; 2 * ul];
let mut v0 = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut v0, &key);
enc_buf[..ul].copy_from_slice(&v0);
let mut orphan = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut orphan, &[0xCCu8; 16]);
enc_buf[ul..].copy_from_slice(&orphan);
let err = decrypt_sectors_mapped(&mut enc_buf, &keys, 0, &map)
.expect_err("an encrypted unit we hold no key for must not be emitted");
assert_eq!(
err.code(),
crate::error::Error::DecryptFailed.code(),
"same verdict CSS and the split-unit branch give for 'no provable key'"
);
}
/// Phase::Even → only even-index units in the range are decrypted; the odd
/// (alternate variant) half is left BYTE-FOR-BYTE as ciphertext for the muxer.
#[test]
@@ -1242,6 +1708,62 @@ mod tests {
}
}
/// The mapped descramble indexes the committed key pool POSITIONALLY
/// (`unit_keys[key_idx].1`), so the ORDER of the `Vec<UnitKey>` a
/// `KeySource` returns is load-bearing — it is NOT "cosmetic, the decrypt path
/// strips it and tries every key", as `keysource::resolve_and_apply_traced`'s
/// doc used to claim. Trial-decrypt was deliberately deleted; nothing here
/// searches the pool. Reordering the same two keys therefore sends each range
/// to the WRONG key: the range that decrypted clean now fails the correct-phase
/// `is_clean` net loudly (or, off a forensic phase, would decrypt a whole span
/// under a neighbour's key). Pins the corrected doc.
#[test]
fn mapped_key_selection_is_positional_so_pool_order_matters() {
use crate::disc::ContentFormat;
let key_a = [0xAAu8; 16];
let key_b = [0xBBu8; 16];
let ul = aacs::content::ALIGNED_UNIT_LEN;
let usz = (ul / 2048) as u32;
// Unit 0 encrypted under key_a, unit 1 under key_b.
let build = || {
let mut buf = vec![0u8; 2 * ul];
for (i, k) in [key_a, key_b].iter().enumerate() {
let mut u = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u, k);
buf[i * ul..(i + 1) * ul].copy_from_slice(&u);
}
buf
};
// Map: unit 0 → pool position 0, unit 1 → pool position 1.
let map = AacsKeyMap::from_ranges_phased(vec![
(0, usz, 0, Phase::Even),
(usz, 2 * usz, 1, Phase::Even),
]);
// Pool in CPS-unit order: each range gets its own key, both come clean.
let mut buf = build();
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, key_a), (1, key_b)],
read_data_key: None,
format: ContentFormat::BdTs,
};
decrypt_sectors_mapped(&mut buf, &keys, 0, &map)
.expect("pool in CPS-unit order decrypts clean");
// SAME keys, SAME CPS-unit numbers, swapped POSITIONS. If the number were
// what mattered (or if the path searched the pool) this would be
// equivalent; positional indexing makes it decrypt both units wrong.
let mut buf = build();
let swapped = DecryptKeys::Aacs {
unit_keys: vec![(1, key_b), (0, key_a)],
read_data_key: None,
format: ContentFormat::BdTs,
};
assert!(
decrypt_sectors_mapped(&mut buf, &swapped, 0, &map).is_err(),
"a reordered pool must fail loud — key selection is positional, so the \
ORDER a KeySource returns its keys in is part of the contract"
);
}
/// The correct-phase safety `is_clean` fires loud: an even unit whose mapped
/// key is wrong does NOT come clean → `DecryptFailed` (not silent corruption).
#[test]
@@ -1321,4 +1843,80 @@ mod tests {
"decrypt thread count must not exceed MAX_THREADS ({MAX_THREADS}), got {n}"
);
}
/// The FMTS phase gate picks which half of an interleaved forensic segment
/// we decrypt. Both the `-` and the `/` in its index arithmetic survived a
/// mutation run, and getting either wrong silently decrypts the alternate
/// variant into garbage while reporting success.
#[test]
fn phase_gate_selects_only_our_parity_of_a_forensic_segment() {
use super::{Phase, unit_is_our_phase};
// A range starting at LBA 30, 3 sectors per aligned unit: units are at
// 30, 33, 36, 39, ... with indices 0, 1, 2, 3, ...
let ours = |lba, phase| unit_is_our_phase(lba, 30, 3, phase);
// Even phase takes indices 0, 2, 4 -> LBAs 30, 36, 42.
assert!(ours(30, Phase::Even));
assert!(!ours(33, Phase::Even));
assert!(ours(36, Phase::Even));
assert!(!ours(39, Phase::Even));
// Odd phase is the exact complement.
for lba in [30, 33, 36, 39, 42, 45] {
assert_ne!(
ours(lba, Phase::Even),
ours(lba, Phase::Odd),
"LBA {lba} must belong to exactly one parity"
);
}
// Non-forensic content: the whole range is ours.
for lba in [30, 33, 36, 39] {
assert!(ours(lba, Phase::All));
}
// The index must be RANGE-RELATIVE: `(lba - start)`, not `(lba + start)`.
// Most value pairs give the same parity either way, so pin one where
// they genuinely disagree: (5-1)/2 = 2 (even, ours) but
// (5+1)/2 = 3 (odd, not ours).
assert!(
unit_is_our_phase(5, 1, 2, Phase::Even),
"the index must be measured from the range start"
);
// The `/ unit_sectors` -> `* unit_sectors` mutant is EQUIVALENT here,
// and deliberately not chased. Proof: aligned offsets are exact
// multiples of the unit size, so offset = k*u for unit index k.
// Dividing gives k; multiplying gives k*u^2, whose parity is
// parity(k)*parity(u^2) = parity(k) whenever u is ODD. `unit_sectors`
// is `ALIGNED_UNIT_LEN / 2048` = 3, a compile-time constant, so u is
// always odd and the two agree on every reachable input. Only an even
// unit size would separate them, and none exists.
assert!(!unit_is_our_phase(33, 30, 3, Phase::Even));
}
/// A malformed key map must not take down a long-running service. A unit
/// below its own range start, or a zero unit size, are both map bugs — they
/// must return a defined answer rather than panicking on debug overflow or
/// dividing by zero.
///
/// Every case here asserts the DEFINED answer, not merely the absence of a
/// panic. The zero-unit-size case used to be written
/// `assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
/// accepts both answers and so pinned nothing at all: the guards could
/// invert and it would still pass. The answer is knowable —
/// `saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
/// is even — so pin it.
#[test]
fn phase_gate_does_not_panic_on_a_malformed_map() {
use super::{Phase, unit_is_our_phase};
// Unit below its own range start: saturating_sub clamps to 0, and unit
// 0 is even.
assert!(unit_is_our_phase(10, 100, 3, Phase::Even));
// Zero unit size: max(1) makes the divisor 1, so the index is the raw
// offset 70 — even.
assert!(unit_is_our_phase(100, 30, 0, Phase::Even));
// Both malformations at once: offset 0 over divisor 1 is unit 0, even.
assert!(unit_is_our_phase(5, 5, 0, Phase::Even));
}
}
+103 -45
View File
@@ -22,10 +22,7 @@
//! `Disc`-level dump ([`dump_disc`]) covers everything that survives
//! lowering: titles, streams, the picked main feature, and AACS state.
use crate::disc::{
AudioChannels, ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, SampleRate,
Stream,
};
use crate::disc::{ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, Stream};
use crate::ifo::{CellCategory, DvdTitle};
const DIAG: &str = "freemkv::diag";
@@ -95,36 +92,12 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
}
}
/// Channel count from an [`AudioChannels`] layout (what lands in the MKV
/// `Channels` element).
pub fn channel_count(ch: AudioChannels) -> u8 {
match ch {
AudioChannels::Mono => 1,
AudioChannels::Stereo => 2,
AudioChannels::Stereo21 => 3,
AudioChannels::Quad => 4,
AudioChannels::Surround50 => 5,
AudioChannels::Surround51 => 6,
AudioChannels::Surround61 => 7,
AudioChannels::Surround71 => 8,
AudioChannels::Unknown => 0,
}
}
/// Sample-rate in Hz for a [`SampleRate`].
pub fn sample_rate_hz(s: SampleRate) -> u32 {
match s {
SampleRate::S44_1 => 44100,
SampleRate::S48 => 48000,
SampleRate::S88_2 => 88200,
SampleRate::S96 => 96000,
SampleRate::S176_4 => 176400,
SampleRate::S192 => 192000,
SampleRate::S48_96 => 96000,
SampleRate::S48_192 => 192000,
SampleRate::Unknown => 0,
}
}
// `channel_count` and `sample_rate_hz` lived here as a third copy of the
// AudioChannels/SampleRate mappings. They were the only HONEST copy — returning
// 0 for Unknown where the canonical accessors fabricated 6 channels at 48 kHz —
// and their only caller was the trace line below, in this same file. The
// canonical accessors are honest now, so the duplicates are gone rather than
// left to drift a fourth time.
// ── DVD cell-category dump (from the IFO scan, pre-lowering) ─────────────────
@@ -499,15 +472,31 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!(
target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason=canonical_title_order(fits-disc, fewest-clips, longest, richest-audio)",
size={}B clips={} reason={}",
main.playlist,
main.duration_secs,
main.size_bytes,
main.clips.len(),
main_feature_reason(),
);
}
}
/// The `reason=` token on the main-feature decision row.
///
/// DERIVED from [`Disc::CANONICAL_TITLE_ORDER_KEYS`], which lives beside the
/// comparator that actually implements them — never restated here. The previous
/// hand-written copy drifted (it advertised a `fewest-clips` key the comparator
/// had replaced with largest-physical-size), which made the self-diagnosing log
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
}
fn dump_aacs(disc: &Disc) {
let Some(a) = disc.aacs.as_ref() else {
if disc.css.is_some() {
@@ -610,8 +599,8 @@ fn dump_title(ti: usize, title: &DiscTitle) {
a.pid,
a.codec,
a.channels,
channel_count(a.channels),
sample_rate_hz(a.sample_rate),
a.channels.count(),
a.sample_rate.hz(),
a.language,
a.secondary,
),
@@ -631,6 +620,60 @@ fn dump_title(ti: usize, title: &DiscTitle) {
#[cfg(test)]
mod tests {
use super::*;
// Needed only by the tests: the production code in this file no longer names
// these types directly, since the local channel/sample-rate duplicates were
// deleted in favour of the canonical accessors.
use crate::disc::{AudioChannels, SampleRate};
/// The main-feature decision row must NAME `canonical_title_order`'s sort
/// keys, not restate them from memory. The restated copy had drifted: it
/// still advertised a "fewest-clips" key long after the comparator replaced
/// clip-count with largest-physical-size, so a bug report read at
/// `--log-level 3` explained the pick with a rule the code does not apply.
///
/// The behavioural half is asserted first — against the comparator itself,
/// with literals — so the key names are checked against what the code
/// actually does, not against the string that names them.
#[test]
fn main_feature_reason_names_the_comparators_real_keys() {
use crate::disc::{Clip, Disc, DiscTitle};
let sized = |size_bytes: u64, n_clips: usize| DiscTitle {
size_bytes,
clips: (0..n_clips)
.map(|i| Clip {
feed_span: None,
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 0,
duration_secs: 0.0,
source_packets: 0,
})
.collect(),
..DiscTitle::empty()
};
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's
// primary key among disc-fitting titles is LARGEST SIZE. "fewest clips"
// would predict the opposite, so the drifted string described a rule
// the comparator does not implement.
let many_clips_big = sized(8_000_000_000, 40);
let one_clip_small = sized(1_000_000_000, 1);
assert_eq!(
Disc::canonical_title_order(&many_clips_big, &one_clip_small, 25_000_000_000),
std::cmp::Ordering::Less,
"largest size wins regardless of clip count"
);
let reason = main_feature_reason();
assert!(
!reason.contains("clips"),
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
);
}
#[test]
fn res_str_keeps_interlace_marker() {
@@ -656,18 +699,33 @@ mod tests {
assert_eq!(hdr_str(HdrFormat::Sdr), "SDR");
}
/// Moved from the deleted local duplicates onto the canonical accessors,
/// with the Unknown case added — which is the whole point of the change.
#[test]
fn channel_count_matches_layout() {
assert_eq!(channel_count(AudioChannels::Mono), 1);
assert_eq!(channel_count(AudioChannels::Stereo), 2);
assert_eq!(channel_count(AudioChannels::Surround51), 6);
assert_eq!(channel_count(AudioChannels::Surround71), 8);
fn channel_count_matches_layout_and_is_zero_when_unknown() {
assert_eq!(AudioChannels::Mono.count(), 1);
assert_eq!(AudioChannels::Stereo.count(), 2);
assert_eq!(AudioChannels::Surround51.count(), 6);
assert_eq!(AudioChannels::Surround71.count(), 8);
// The one that matters. This used to return 6, which is indistinguishable
// from a real 5.1 track and left every caller responsible for checking
// the variant first.
assert_eq!(
AudioChannels::Unknown.count(),
0,
"an unknown layout must not report a plausible channel count"
);
}
#[test]
fn sample_rate_hz_values() {
assert_eq!(sample_rate_hz(SampleRate::S48), 48000);
assert_eq!(sample_rate_hz(SampleRate::S96), 96000);
fn sample_rate_hz_values_and_zero_when_unknown() {
assert_eq!(SampleRate::S48.hz(), 48000.0);
assert_eq!(SampleRate::S96.hz(), 96000.0);
assert_eq!(
SampleRate::Unknown.hz(),
0.0,
"an unknown sample rate must not report a plausible 48 kHz"
);
}
#[test]
+743
View File
@@ -0,0 +1,743 @@
//! ECMA-167 / UDF 1.02 descriptor encoder.
//!
//! Turns a [`Layout`](super::layout::Layout) — a directory tree with every
//! ICB, directory-data and file-data block already assigned — into the set of
//! metadata sectors a real UDF volume would carry. Nothing here touches the
//! filesystem: it is a pure function from layout to sectors, which is what
//! makes it testable against the production parser in `udf.rs`.
//!
//! What is emitted, in volume order:
//!
//! | sector | descriptor |
//! |---|---|
//! | 16, 17, 18 | Volume Recognition Sequence — `BEA01`, `NSR02`, `TEA01` (ECMA-167 2/9.1) |
//! | 32… | Main Volume Descriptor Sequence — PVD, IUVD, PD, LVD, USD, TD |
//! | 48… | Reserve VDS (byte-identical but for the tag locations) |
//! | 64, 65 | Logical Volume Integrity Sequence — LVID, TD |
//! | 256 | Anchor Volume Descriptor Pointer |
//! | `part_start` + 0, +1 | File Set Descriptor, TD |
//! | `part_start` + … | File Entries (ICBs) and directory data (FIDs) |
//! | last sector | Anchor Volume Descriptor Pointer (copy) |
//!
//! UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
//! the DVD-Video profile, it is the shape `read_filesystem` takes when
//! `num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
//! entirely. That also means a synthetic image never exercises the Metadata
//! Partition path in `udf.rs` (`:946-991`) — see the module docs on `dirimage`.
use super::layout::{DirNode, Layout};
use crate::error::{Error, Result};
use std::collections::BTreeMap;
/// Logical block / sector size. Fixed for every optical profile this crate
/// reads, and the same quantity as [`crate::consts::SECTOR_BYTES`] — aliased
/// rather than re-declared so the two cannot drift apart. The short name is
/// kept because it appears in ~25 extent and offset expressions across
/// `dirimage`, where the longer one would bury the arithmetic.
pub(super) use crate::consts::SECTOR_BYTES as SECTOR;
/// Descriptor version recorded in every tag. 2 = ECMA-167 2nd edition, which
/// is what UDF revisions up to and including 2.00 require.
const DESC_VERSION: u16 = 2;
/// UDF revision recorded in the domain EntityID suffix (1.02, BCD-ish u16).
const UDF_REVISION: u16 = 0x0102;
/// A fixed recording timestamp, so an image synthesized from the same folder
/// twice is byte-identical. Real mtimes would make every test golden-file
/// comparison and every `dir:// -> iso://` re-run differ for no benefit.
const FIXED_TIME: Timestamp = Timestamp {
year: 2000,
month: 1,
day: 1,
};
struct Timestamp {
year: i16,
month: u8,
day: u8,
}
/// The synthesized metadata: absolute LBA → sector contents. Data sectors are
/// NOT here; they are served from the backing files.
pub(super) type MetaSectors = BTreeMap<u32, Box<[u8; SECTOR]>>;
/// The descriptor-tag CRC of ECMA-167 7.2.4: polynomial 0x1021, initial value
/// ZERO, no reflection, no final XOR — the variant catalogued as CRC-16/XMODEM
/// (check value 0x31C3), NOT CCITT-FALSE, which seeds at 0xFFFF and would make
/// every descriptor this crate writes fail a conformant driver's validation.
fn crc16(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x1021
} else {
crc << 1
};
}
}
crc
}
/// Write an ECMA-167 3/7.2 descriptor tag over `buf[0..16]`.
///
/// `tag_loc` is the block number of the sector holding the descriptor —
/// ABSOLUTE for the volume-space descriptors (AVDP, VDS, LVID) and
/// PARTITION-RELATIVE for everything inside the partition (FSD, File Entries).
/// Getting that wrong is the classic reason a hand-built volume mounts nowhere:
/// a driver that validates the tag location rejects the descriptor outright.
///
/// `desc_len` is the descriptor's total length including the tag; the CRC
/// covers `buf[16..desc_len]`.
fn finish_tag(buf: &mut [u8], tag_id: u16, tag_loc: u32, desc_len: usize) {
buf[0..2].copy_from_slice(&tag_id.to_le_bytes());
buf[2..4].copy_from_slice(&DESC_VERSION.to_le_bytes());
buf[4] = 0; // checksum, filled below
buf[5] = 0; // reserved
buf[6..8].copy_from_slice(&0u16.to_le_bytes()); // tag serial number
let crc_len = desc_len - 16;
let crc = crc16(&buf[16..desc_len]);
buf[8..10].copy_from_slice(&crc.to_le_bytes());
buf[10..12].copy_from_slice(&(crc_len as u16).to_le_bytes());
buf[12..16].copy_from_slice(&tag_loc.to_le_bytes());
// ECMA-167 3/7.2.3: sum of bytes 0..16 EXCLUDING byte 4, modulo 256.
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
buf[4] = (sum % 256) as u8;
}
/// ECMA-167 1/7.2.1 charspec: type 0 (CS0) + "OSTA Compressed Unicode".
fn put_charspec(buf: &mut [u8]) {
buf[0] = 0;
let id = b"OSTA Compressed Unicode";
buf[1..1 + id.len()].copy_from_slice(id);
}
/// ECMA-167 1/7.4 EntityID: flags byte, 23 identifier bytes, 8 suffix bytes.
fn put_entity_id(buf: &mut [u8], id: &[u8], suffix: &[u8]) {
buf[0] = 0;
let n = id.len().min(23);
buf[1..1 + n].copy_from_slice(&id[..n]);
let m = suffix.len().min(8);
buf[24..24 + m].copy_from_slice(&suffix[..m]);
}
/// The `*OSTA UDF Compliant` domain EntityID suffix: UDF revision, domain
/// flags (0 = neither hard nor soft write-protected), reserved.
fn domain_suffix() -> [u8; 8] {
let mut s = [0u8; 8];
s[0..2].copy_from_slice(&UDF_REVISION.to_le_bytes());
s
}
/// This crate's implementation EntityID suffix: OS class / OS identifier
/// (0 = undefined, deliberately — the image is not OS-specific) + 6 free bytes.
fn impl_suffix() -> [u8; 8] {
[0u8; 8]
}
fn put_impl_id(buf: &mut [u8]) {
put_entity_id(buf, b"*freemkv", &impl_suffix());
}
fn put_domain_id(buf: &mut [u8]) {
put_entity_id(buf, b"*OSTA UDF Compliant", &domain_suffix());
}
/// OSTA CS0 d-string: a compression-ID byte, the characters, then the used
/// length in the FIELD'S LAST byte (ECMA-167 1/7.2.12 + UDF 2.1.3). An
/// all-zero field is the empty string.
fn put_dstring(buf: &mut [u8], s: &str) {
if s.is_empty() {
return;
}
let encoded = encode_cs0(s);
// Leave room for the trailing length byte.
let room = buf.len() - 1;
let n = encoded.len().min(room);
buf[..n].copy_from_slice(&encoded[..n]);
buf[buf.len() - 1] = n as u8;
}
/// OSTA CS0: compression ID 8 (one byte per character) when every character
/// is ASCII, otherwise compression ID 16 (UTF-16BE).
///
/// ASCII rather than Latin-1 for the 8-bit form on purpose: `parse_udf_name`
/// (`udf.rs:1467`) decodes a compression-8 name with `from_utf8_lossy`, so a
/// 0x80-0xFF byte — legal CS0 — would come back as U+FFFD. Every character
/// above 0x7F therefore takes the 16-bit form, which that parser decodes
/// correctly.
pub(super) fn encode_cs0(s: &str) -> Vec<u8> {
if s.is_ascii() {
let mut v = Vec::with_capacity(1 + s.len());
v.push(8u8);
v.extend_from_slice(s.as_bytes());
v
} else {
let mut v = vec![16u8];
for u in s.encode_utf16() {
v.extend_from_slice(&u.to_be_bytes());
}
v
}
}
/// ECMA-167 1/7.3 timestamp, 12 bytes. Type 1 (local time) with a zero
/// offset, i.e. UTC.
fn put_timestamp(buf: &mut [u8]) {
buf[0..2].copy_from_slice(&0x1000u16.to_le_bytes());
buf[2..4].copy_from_slice(&FIXED_TIME.year.to_le_bytes());
buf[4] = FIXED_TIME.month;
buf[5] = FIXED_TIME.day;
}
/// ECMA-167 3/7.1 extent_ad: length in BYTES, then location.
fn put_extent_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
/// ECMA-167 4/14.14.2 long_ad: length+type, then lb_addr (block, partition
/// reference), then 6 implementation-use bytes.
fn put_long_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
buf[8..10].copy_from_slice(&0u16.to_le_bytes()); // partition reference 0
}
/// ECMA-167 4/14.14.1 short_ad. The top two bits of the length word are the
/// extent TYPE (0 = recorded and allocated), which is exactly why `udf.rs`
/// masks with `0x3FFF_FFFF` when it reads one back — the mask is the field
/// boundary, not a truncation bug.
fn put_short_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
debug_assert!(len_bytes <= 0x3FFF_FFFF, "AD length must fit 30 bits");
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
fn blank() -> Box<[u8; SECTOR]> {
Box::new([0u8; SECTOR])
}
// ── Volume-space descriptors ────────────────────────────────────────────────
/// ECMA-167 2/9.1 Volume Structure Descriptor: the three-sector recognition
/// sequence an OS looks for before it will even consider the volume UDF.
fn volume_recognition(id: &[u8; 5]) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[0] = 0; // structure type
s[1..6].copy_from_slice(id);
s[6] = 1; // structure version
s
}
/// ECMA-167 3/10.1 Primary Volume Descriptor.
fn primary_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes()); // PVD number
put_dstring(&mut s[24..56], volume_id);
s[56..58].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[58..60].copy_from_slice(&1u16.to_le_bytes()); // max volume sequence number
s[60..62].copy_from_slice(&2u16.to_le_bytes()); // interchange level
s[62..64].copy_from_slice(&2u16.to_le_bytes()); // max interchange level
s[64..68].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[68..72].copy_from_slice(&1u32.to_le_bytes()); // max character set list
// UDF 2.2.2.5: the first 8 characters of the volume set identifier must be
// unique. A fixed hex prefix plus the volume id is sufficient here — the
// image is single-volume and never joins a real volume set.
put_dstring(&mut s[72..200], &format!("46524D4B{volume_id}"));
put_charspec(&mut s[200..264]); // descriptor character set
put_charspec(&mut s[264..328]); // explanatory character set
put_timestamp(&mut s[376..388]);
put_impl_id(&mut s[388..420]);
finish_tag(&mut s[..], 1, lba, 512);
s
}
/// ECMA-167 3/10.4 + UDF 2.2.7 Implementation Use Volume Descriptor
/// (`*UDF LV Info`). Not read by `udf.rs`, required by the spec.
fn impl_use_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_entity_id(&mut s[20..52], b"*UDF LV Info", &domain_suffix());
put_charspec(&mut s[52..116]); // LVI charset
put_dstring(&mut s[116..244], volume_id); // logical volume identifier
put_impl_id(&mut s[352..384]);
finish_tag(&mut s[..], 4, lba, 512);
s
}
/// ECMA-167 3/10.5 Partition Descriptor — the descriptor `read_filesystem`
/// takes `partition_start` from (offset 188).
fn partition(part_start: u32, part_sectors: u32, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..22].copy_from_slice(&1u16.to_le_bytes()); // partition flags: allocated
s[22..24].copy_from_slice(&0u16.to_le_bytes()); // partition number
put_entity_id(&mut s[24..56], b"+NSR02", &[]);
// s[56..184] partition contents use = Partition Header Descriptor. All
// zero: a read-only partition records no unallocated/freed space tables.
s[184..188].copy_from_slice(&1u32.to_le_bytes()); // access type: read only
s[188..192].copy_from_slice(&part_start.to_le_bytes());
s[192..196].copy_from_slice(&part_sectors.to_le_bytes());
put_impl_id(&mut s[196..228]);
finish_tag(&mut s[..], 5, lba, 512);
s
}
/// ECMA-167 3/10.6 Logical Volume Descriptor. Carries the FSD long_ad and the
/// partition map table; `read_filesystem` reads `num_partition_maps` at 268
/// and takes the single-partition path when it is 1.
fn logical_volume(
volume_id: &str,
fsd_lba: u32,
integrity_lba: u32,
integrity_sectors: u32,
lba: u32,
seq: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_charspec(&mut s[20..84]);
put_dstring(&mut s[84..212], volume_id);
s[212..216].copy_from_slice(&(SECTOR as u32).to_le_bytes()); // logical block size
put_domain_id(&mut s[216..248]);
// Logical volume contents use = long_ad of the File Set Descriptor,
// partition-relative. One sector.
put_long_ad(&mut s[248..264], SECTOR as u32, fsd_lba);
s[264..268].copy_from_slice(&6u32.to_le_bytes()); // map table length
s[268..272].copy_from_slice(&1u32.to_le_bytes()); // number of partition maps
put_impl_id(&mut s[272..304]);
put_extent_ad(
&mut s[432..440],
integrity_sectors * SECTOR as u32,
integrity_lba,
);
// ECMA-167 3/10.7.2 Type 1 partition map.
s[440] = 1; // map type
s[441] = 6; // map length
s[442..444].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[444..446].copy_from_slice(&0u16.to_le_bytes()); // partition number
finish_tag(&mut s[..], 6, lba, 446);
s
}
/// ECMA-167 3/10.8 Unallocated Space Descriptor with zero extents — the whole
/// volume is accounted for by the partition.
fn unallocated_space(lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes());
finish_tag(&mut s[..], 7, lba, 24);
s
}
/// ECMA-167 3/10.9 Terminating Descriptor.
fn terminating(lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
finish_tag(&mut s[..], 8, lba, 512);
s
}
/// ECMA-167 3/10.10 + UDF 2.2.6 Logical Volume Integrity Descriptor, closed.
fn integrity(
part_sectors: u32,
files: u32,
dirs: u32,
next_uid: u64,
lba: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..32].copy_from_slice(&1u32.to_le_bytes()); // integrity type: close
// s[32..40] next integrity extent: none.
s[40..48].copy_from_slice(&next_uid.to_le_bytes()); // logical volume contents use: next unique id
s[72..76].copy_from_slice(&1u32.to_le_bytes()); // number of partitions
s[76..80].copy_from_slice(&46u32.to_le_bytes()); // length of implementation use
s[80..84].copy_from_slice(&0u32.to_le_bytes()); // free space: none (read-only)
s[84..88].copy_from_slice(&part_sectors.to_le_bytes()); // size table
put_impl_id(&mut s[88..120]);
s[120..124].copy_from_slice(&files.to_le_bytes());
s[124..128].copy_from_slice(&dirs.to_le_bytes());
s[128..130].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min read revision
s[130..132].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min write revision
s[132..134].copy_from_slice(&UDF_REVISION.to_le_bytes()); // max write revision
finish_tag(&mut s[..], 9, lba, 134);
s
}
/// ECMA-167 3/10.2 Anchor Volume Descriptor Pointer. `read_filesystem` reads
/// the main VDS extent from offsets 16..24 and sweeps it.
fn anchor(main_lba: u32, reserve_lba: u32, vds_sectors: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_extent_ad(&mut s[16..24], vds_sectors * SECTOR as u32, main_lba);
put_extent_ad(&mut s[24..32], vds_sectors * SECTOR as u32, reserve_lba);
finish_tag(&mut s[..], 2, lba, 512);
s
}
/// ECMA-167 4/14.1 File Set Descriptor. `read_filesystem` requires tag 256 at
/// the first block of the (metadata =) partition and reads the root ICB block
/// from offset 404.
fn file_set(volume_id: &str, root_icb: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..30].copy_from_slice(&3u16.to_le_bytes()); // interchange level
s[30..32].copy_from_slice(&3u16.to_le_bytes()); // max interchange level
s[32..36].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[36..40].copy_from_slice(&1u32.to_le_bytes()); // max character set list
s[40..44].copy_from_slice(&0u32.to_le_bytes()); // file set number
s[44..48].copy_from_slice(&0u32.to_le_bytes()); // file set descriptor number
put_charspec(&mut s[48..112]);
put_dstring(&mut s[112..240], volume_id);
put_charspec(&mut s[240..304]);
put_dstring(&mut s[304..336], volume_id);
put_long_ad(&mut s[400..416], SECTOR as u32, root_icb);
put_domain_id(&mut s[416..448]);
finish_tag(&mut s[..], 256, lba, 512);
s
}
// ── Partition-space descriptors ─────────────────────────────────────────────
/// UDF permission word: read + execute for owner, group and other. No write
/// bit anywhere — the volume is read-only.
const PERM_R_X: u32 = 0x0000_1000 | 0x0000_0400 | 0x0000_0080 | 0x0000_0020 | 0x4 | 0x1;
/// ECMA-167 4/14.9 File Entry (tag 261).
///
/// Tag 261 rather than the Extended File Entry (266) real BD-ROMs use: an EFE
/// requires UDF 2.00+, and this image declares 1.02. `udf.rs` reads both — the
/// 261 field offsets it uses (l_ea 168, l_ad 172, ADs at 176 + l_ea) are the
/// ones written here.
///
/// `extents` are partition-relative (block, byte-length) pairs, already split
/// so no single one exceeds the 30-bit AD length field.
fn file_entry(
is_dir: bool,
info_len: u64,
extents: &[(u32, u32)],
link_count: u16,
unique_id: u64,
lba: u32,
) -> Result<Box<[u8; SECTOR]>> {
let mut s = blank();
// ICB tag (ECMA-167 4/14.6) at offset 16.
s[16..20].copy_from_slice(&0u32.to_le_bytes()); // prior recorded direct entries
s[20..22].copy_from_slice(&4u16.to_le_bytes()); // strategy type 4
s[24..26].copy_from_slice(&1u16.to_le_bytes()); // max number of entries
s[27] = if is_dir { 4 } else { 5 }; // file type: directory / byte sequence
// s[28..34] parent ICB location: not recorded (permitted).
// s[34..36] ICB flags: 0 => short allocation descriptors. `udf.rs:601`
// reads exactly this word to pick its AD stride.
s[34..36].copy_from_slice(&0u16.to_le_bytes());
// UDF's sentinel for "not specified" is 0xFFFFFFFF, not 0 — 0 is a real
// uid/gid (root). A synthesized image has no meaningful owner, and a driver
// that maps these through would otherwise report every file as root-owned.
s[36..40].copy_from_slice(&u32::MAX.to_le_bytes()); // uid: not specified
s[40..44].copy_from_slice(&u32::MAX.to_le_bytes()); // gid: not specified
s[44..48].copy_from_slice(&PERM_R_X.to_le_bytes());
s[48..50].copy_from_slice(&link_count.to_le_bytes());
s[56..64].copy_from_slice(&info_len.to_le_bytes());
let blocks: u64 = extents
.iter()
.map(|(_, len)| (*len as u64).div_ceil(SECTOR as u64))
.sum();
s[64..72].copy_from_slice(&blocks.to_le_bytes()); // logical blocks recorded
put_timestamp(&mut s[72..84]); // access
put_timestamp(&mut s[84..96]); // modification
put_timestamp(&mut s[96..108]); // attribute
s[108..112].copy_from_slice(&1u32.to_le_bytes()); // checkpoint
put_impl_id(&mut s[128..160]);
s[160..168].copy_from_slice(&unique_id.to_le_bytes());
s[168..172].copy_from_slice(&0u32.to_le_bytes()); // length of EAs
let l_ad = extents.len() * 8;
// A short AD is 8 bytes and the entry has 2048 - 176 = 1872 bytes for
// them, i.e. 234 extents — over 200 GiB at the per-AD ceiling. Beyond
// that an Allocation Extent Descriptor chain would be required; refuse
// rather than write a truncated list.
if 176 + l_ad > SECTOR {
return Err(Error::DirImageTooLarge);
}
s[172..176].copy_from_slice(&(l_ad as u32).to_le_bytes());
for (i, (elba, len)) in extents.iter().enumerate() {
let off = 176 + i * 8;
put_short_ad(&mut s[off..off + 8], *len, *elba);
}
finish_tag(&mut s[..], 261, lba, 176 + l_ad);
Ok(s)
}
/// ECMA-167 4/14.4 File Identifier Descriptor, appended to `buf`.
///
/// FIDs are packed with no inter-descriptor padding beyond the 4-byte
/// alignment the spec mandates, and they are allowed to span logical blocks —
/// which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
/// directory extent as one flat byte run and STOPS at the first non-257 tag,
/// so any block-alignment gap would truncate the directory.
fn push_fid(buf: &mut Vec<u8>, name: &str, icb_lba: u32, is_dir: bool, is_parent: bool) {
let start = buf.len();
let name_field: Vec<u8> = if is_parent {
Vec::new()
} else {
encode_cs0(name)
};
let l_fi = name_field.len();
let mut fid = vec![0u8; 38];
fid[16..18].copy_from_slice(&1u16.to_le_bytes()); // file version number
let mut chars = 0u8;
if is_dir {
chars |= 0x02;
}
if is_parent {
chars |= 0x08;
}
fid[18] = chars;
// The planner refuses any name whose encoding exceeds what this byte can
// hold (`layout::MAX_CS0_NAME_BYTES`), so this cannot wrap in practice. The
// assert states the invariant where it is relied on rather than trusting a
// check three files away; a wrap here would desynchronise the directory.
debug_assert!(
l_fi <= u8::MAX as usize,
"FID name length must fit one byte"
);
fid[19] = l_fi as u8;
put_long_ad(&mut fid[20..36], SECTOR as u32, icb_lba);
fid[36..38].copy_from_slice(&0u16.to_le_bytes()); // length of implementation use
buf.extend_from_slice(&fid);
buf.extend_from_slice(&name_field);
let unpadded = buf.len() - start;
let padded = unpadded.div_ceil(4) * 4;
buf.resize(start + padded, 0);
// The tag is written last: its CRC covers the descriptor body, which the
// padding is not part of (ECMA-167 4/14.4.9 counts padding outside the
// CRC'd length).
let tag_loc_placeholder = 0;
finish_tag(
&mut buf[start..start + unpadded],
257,
tag_loc_placeholder,
unpadded,
);
}
/// Serialize one directory's FID list (parent entry first, then children).
pub(super) fn dir_fids(dir: &DirNode) -> Vec<u8> {
let mut buf = Vec::new();
push_fid(&mut buf, "", dir.parent_icb_lba, true, true);
for sub in &dir.dirs {
push_fid(&mut buf, &sub.name, sub.icb_lba, true, false);
}
for f in &dir.files {
push_fid(&mut buf, &f.name, f.icb_lba, false, false);
}
buf
}
/// Patch every FID's tag location to the block it actually lands in. ECMA-167
/// 3/7.2.2 makes the tag location the block of the descriptor, and a FID that
/// spans two blocks records the block it STARTS in.
fn fix_fid_tag_locations(buf: &mut [u8], first_block: u32) {
let mut pos = 0usize;
while pos + 38 <= buf.len() {
let l_fi = buf[pos + 19] as usize;
let l_iu = u16::from_le_bytes([buf[pos + 36], buf[pos + 37]]) as usize;
let unpadded = 38 + l_iu + l_fi;
if pos + unpadded > buf.len() {
break;
}
let block = first_block + (pos / SECTOR) as u32;
finish_tag(&mut buf[pos..pos + unpadded], 257, block, unpadded);
pos += unpadded.div_ceil(4) * 4;
}
}
// ── Whole-image assembly ────────────────────────────────────────────────────
/// Volume-space block of the Volume Recognition Sequence.
const VRS_START: u32 = 16;
/// Volume-space block of the Main Volume Descriptor Sequence.
pub(super) const MAIN_VDS_START: u32 = 32;
/// Volume-space block of the Reserve Volume Descriptor Sequence.
pub(super) const RESERVE_VDS_START: u32 = 48;
/// Sectors reserved for each VDS. ECMA-167 3/10.2.1 requires an anchor to
/// record at least 16.
pub(super) const VDS_SECTORS: u32 = 16;
/// Volume-space block of the Logical Volume Integrity Sequence.
pub(super) const LVID_START: u32 = 64;
/// Sectors reserved for the integrity sequence (LVID + TD).
pub(super) const LVID_SECTORS: u32 = 2;
/// The mandatory anchor block (ECMA-167 3/10.2).
pub(super) const ANCHOR_LBA: u32 = 256;
/// First block a partition may start at. Everything above is volume space.
pub(super) const MIN_PART_START: u32 = 320;
/// Emit the six-descriptor Volume Descriptor Sequence at `start`.
fn write_vds(out: &mut MetaSectors, layout: &Layout, start: u32) {
let vid = &layout.volume_id;
out.insert(start, primary_volume(vid, start, 1));
out.insert(start + 1, impl_use_volume(vid, start + 1, 2));
out.insert(
start + 2,
partition(layout.part_start, layout.part_sectors, start + 2, 3),
);
out.insert(
start + 3,
logical_volume(vid, 0, LVID_START, LVID_SECTORS, start + 3, 4),
);
out.insert(start + 4, unallocated_space(start + 4, 5));
out.insert(start + 5, terminating(start + 5));
}
/// Recursively emit one directory's File Entry and FID list, then its
/// children's.
fn write_dir(out: &mut MetaSectors, layout: &Layout, dir: &DirNode) -> Result<()> {
let mut fids = dir_fids(dir);
fix_fid_tag_locations(&mut fids, dir.data_lba);
debug_assert_eq!(fids.len(), dir.data_bytes as usize);
// A directory's link count is 1 (its own FID in the parent) plus one for
// each child directory's parent FID pointing back at it.
// The planner caps subdirectory fan-out (`layout::MAX_SUBDIRS`) so this
// cannot overflow; saturating rather than wrapping keeps a future change to
// that cap from silently producing a wrong count.
let link_count = (dir.dirs.len() as u16).saturating_add(1);
let fe = file_entry(
true,
fids.len() as u64,
&[(dir.data_lba, fids.len() as u32)],
link_count,
dir.unique_id,
dir.icb_lba,
)?;
out.insert(layout.part_start + dir.icb_lba, fe);
for (i, chunk) in fids.chunks(SECTOR).enumerate() {
let mut s = blank();
s[..chunk.len()].copy_from_slice(chunk);
out.insert(layout.part_start + dir.data_lba + i as u32, s);
}
for f in &dir.files {
let extents: Vec<(u32, u32)> = f.extents.iter().map(|e| (e.lba, e.bytes)).collect();
let fe = file_entry(false, f.size, &extents, 1, f.unique_id, f.icb_lba)?;
out.insert(layout.part_start + f.icb_lba, fe);
}
for sub in &dir.dirs {
write_dir(out, layout, sub)?;
}
Ok(())
}
/// Build every metadata sector of the synthesized volume.
pub(super) fn encode(layout: &Layout) -> Result<MetaSectors> {
let mut out = MetaSectors::new();
out.insert(VRS_START, volume_recognition(b"BEA01"));
out.insert(VRS_START + 1, volume_recognition(b"NSR02"));
out.insert(VRS_START + 2, volume_recognition(b"TEA01"));
write_vds(&mut out, layout, MAIN_VDS_START);
write_vds(&mut out, layout, RESERVE_VDS_START);
out.insert(
LVID_START,
integrity(
layout.part_sectors,
layout.file_count,
layout.dir_count,
layout.next_unique_id,
LVID_START,
),
);
out.insert(LVID_START + 1, terminating(LVID_START + 1));
let avdp = anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, ANCHOR_LBA);
out.insert(ANCHOR_LBA, avdp);
let last = layout.total_sectors - 1;
out.insert(
last,
anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, last),
);
// Partition block 0 must hold the File Set Descriptor: `read_filesystem`
// reads exactly `metadata_start` (== partition start on a single-partition
// volume) and rejects the volume outright if the tag there is not 256.
out.insert(
layout.part_start,
file_set(&layout.volume_id, layout.root.icb_lba, 0),
);
out.insert(layout.part_start + 1, terminating(1));
write_dir(&mut out, layout, &layout.root)?;
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
/// The reference check value for CRC-16/XMODEM — poly 0x1021 seeded at 0,
/// which is what ECMA-167 7.2.4 specifies: "123456789" → 0x31C3. Seeding
/// at 0xFFFF instead (CCITT-FALSE) yields 0x29B1, and that mutant is
/// invisible to `udf.rs`, which never verifies a tag CRC — it would only
/// show up as a volume no operating system will mount.
#[test]
fn crc16_matches_the_ecma167_check_value() {
assert_eq!(crc16(b"123456789"), 0x31C3);
assert_ne!(crc16(b"123456789"), 0x29B1, "not the 0xFFFF-seeded variant");
}
/// ECMA-167 3/7.2.3: the checksum is the sum of the tag's first 16 bytes
/// EXCLUDING the checksum byte itself, modulo 256.
#[test]
fn tag_checksum_excludes_its_own_byte() {
let mut buf = [0u8; 512];
buf[16..24].copy_from_slice(&[1, 2, 3, 4, 5, 6, 7, 8]);
finish_tag(&mut buf, 261, 0x1234, 512);
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
assert_eq!(buf[4] as u32, sum % 256);
// And the recorded CRC covers the body, not the tag.
let crc = u16::from_le_bytes([buf[8], buf[9]]);
assert_eq!(crc, crc16(&buf[16..512]));
assert_eq!(u16::from_le_bytes([buf[10], buf[11]]), 496);
assert_eq!(
u32::from_le_bytes([buf[12], buf[13], buf[14], buf[15]]),
0x1234
);
}
/// ASCII takes compression ID 8; anything above takes 16 (UTF-16BE),
/// because `parse_udf_name` decodes compression-8 bytes as UTF-8.
#[test]
fn cs0_picks_the_encoding_the_parser_can_decode() {
assert_eq!(encode_cs0("AB"), vec![8, b'A', b'B']);
let e = encode_cs0("Ä");
assert_eq!(e[0], 16);
assert_eq!(&e[1..], &[0x00, 0xC4]);
assert_eq!(crate::udf::parse_udf_name(&e), "Ä");
}
/// A d-string records its used length in the field's LAST byte, and the
/// production parser must read the same string back.
#[test]
fn dstring_round_trips_through_the_production_parser() {
let mut field = [0u8; 32];
put_dstring(&mut field, "FREEMKV");
assert_eq!(field[31], 8, "compid byte + 7 characters");
assert_eq!(crate::udf::parse_dstring_for_test(&field), "FREEMKV");
}
}
File diff suppressed because it is too large Load Diff
+362
View File
@@ -0,0 +1,362 @@
//! `dir://` as an image-level SOURCE: a synthetic UDF volume over a folder.
//!
//! A user's extracted disc — a DVD `VIDEO_TS/` or a Blu-ray `BDMV/`, typically
//! a MakeMKV-style backup — has files but no sectors, and everything above the
//! sector layer in this crate wants sectors: `Disc::scan_image`, `UdfFs`,
//! `ifo.rs`, `mpls.rs`, `clpi.rs` and the mux all read through a
//! [`SectorSource`]. [`DirImage`] supplies one.
//!
//! The trick is that nothing is emulated. A real, minimal, valid UDF 1.02
//! volume is synthesized over the folder:
//!
//! * **Metadata sectors** (anchors, the volume descriptor sequences, the File
//! Set Descriptor, every File Entry, every directory's FID list) are encoded
//! into RAM by [`encode`] — a few MiB even for a large Blu-ray.
//! * **Data sectors** are not materialized at all. Each one maps to a byte
//! range of a real file, read on demand.
//!
//! So `udf::read_filesystem` parses this image by exactly the same code path it
//! parses a real disc with, and every consumer above it is unchanged. The cost
//! is that a single-partition synthetic volume never exercises the UDF 2.50
//! Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
//! this module's tests do not cover that block and must not be read as if they
//! did.
//!
//! What this module deliberately does NOT do:
//!
//! * **3D / SSIF** — rejected up front ([`Error::DirImageSsifUnsupported`]).
//! An SSIF aliases the same sectors as its base and dependent `.m2ts`; the
//! planner allocates disjoint extents, so a 3D folder would produce silently
//! wrong output.
//! * **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
//! * **Encrypted folders** — a folder whose content is still AACS-scrambled is
//! rejected by the caller-side probe, not decrypted here.
mod encode;
mod layout;
use crate::error::{Error, Result};
#[cfg(target_os = "linux")]
use crate::io::file_sector_source::linux::drop_window;
#[cfg(target_os = "macos")]
use crate::io::file_sector_source::macos::drop_window;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use crate::io::file_sector_source::other::drop_window;
#[cfg(target_os = "windows")]
use crate::io::file_sector_source::windows::drop_window;
use crate::sector::SectorSource;
use encode::{MetaSectors, SECTOR};
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::{Path, PathBuf};
/// How many host files may be held open at once.
///
/// A Blu-ray `BDMV/` can exceed a thousand files while macOS `RLIMIT_NOFILE`
/// defaults to 256, so "open every file up front" is not available. Reads are
/// overwhelmingly sequential through one large stream file at a time, so a
/// small LRU keeps the hit rate near 1 while bounding descriptors.
const HANDLE_CACHE: usize = 16;
/// One file's bytes at one place in the image.
#[derive(Debug, Clone)]
struct DataRange {
/// Absolute first block.
start_lba: u32,
/// Blocks covered (the last one may be partially used, and is zero-padded).
sectors: u32,
/// Index into [`DirImage::files`].
file: usize,
/// Byte offset within the file at which this range's bytes begin.
offset: u64,
/// Byte length of the range.
bytes: u64,
}
/// A file the image reads through.
#[derive(Debug)]
struct FileRef {
host: PathBuf,
disc_path: String,
size: u64,
/// Host mtime at plan time — see `layout::FileNode::mtime` for why size
/// alone is not enough.
mtime: Option<std::time::SystemTime>,
}
/// A synthesized UDF disc image over a host directory.
///
/// Owns everything it reads through (`PathBuf`s and its own file handles), so
/// it is `Send + 'static` and can be moved into `build_iso_pipeline`, which
/// hands it to `PrefetchedSectorSource`'s producer thread.
pub struct DirImage {
meta: MetaSectors,
/// Sorted by `start_lba`, non-overlapping.
ranges: Vec<DataRange>,
files: Vec<FileRef>,
open: Vec<(usize, File)>,
total_sectors: u32,
volume_id: String,
data_bytes: u64,
}
impl std::fmt::Debug for DirImage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DirImage")
.field("volume_id", &self.volume_id)
.field("total_sectors", &self.total_sectors)
.field("files", &self.files.len())
.field("meta_sectors", &self.meta.len())
.finish()
}
}
impl DirImage {
/// Plan and encode an image over `root`.
///
/// Every error is decided here, at plan time, where it can name the file
/// responsible — the read path is deliberately left with nothing to decide
/// except "this file changed underneath me".
pub fn open(root: &Path) -> Result<Self> {
let plan = layout::plan(root)?;
let meta = encode::encode(&plan)?;
let mut nodes = Vec::new();
layout::flatten(&plan.root, &mut nodes);
let mut files = Vec::with_capacity(nodes.len());
let mut ranges = Vec::new();
for (idx, node) in nodes.iter().enumerate() {
// Carry the plan-time mtime ONLY for files whose CONTENT the plan
// read — the DVD IFOs, whose bytes 0xC0/0xC4 decide where every VOB
// is placed (`layout::place_video_ts` -> `read_head`).
//
// For every other file the plan depends on the SIZE alone, and size
// is already checked. Comparing mtime on those buys nothing and
// costs real false positives: disc backups commonly live on
// exFAT/FAT32, which stores local time, so a long rip spanning a
// DST transition sees a whole-hour shift on a file nobody touched
// and would abort hours in, blaming a change that did not happen.
// The multi-gigabyte VOBs are exactly the files a long rip re-opens
// after the handle cache evicts them.
let content_sensitive = node
.disc_path
.rsplit('.')
.next()
.is_some_and(|e| e.eq_ignore_ascii_case("IFO"));
files.push(FileRef {
host: node.host.clone(),
disc_path: node.disc_path.clone(),
size: node.size,
mtime: content_sensitive.then_some(node.mtime).flatten(),
});
let mut offset = 0u64;
for e in &node.extents {
ranges.push(DataRange {
start_lba: plan.part_start + e.lba,
sectors: (e.bytes as u64).div_ceil(SECTOR as u64) as u32,
file: idx,
offset,
bytes: e.bytes as u64,
});
offset += e.bytes as u64;
}
}
ranges.sort_by_key(|r| r.start_lba);
debug_assert!(
ranges
.windows(2)
.all(|w| w[0].start_lba + w[0].sectors <= w[1].start_lba),
"planned data ranges must not overlap"
);
let data_bytes = layout::total_data_bytes(&plan.root);
tracing::info!(
target: "freemkv::dirimage",
volume_id = %plan.volume_id,
files = files.len(),
dirs = plan.dir_count,
meta_blocks = layout::metadata_block_count(&plan.root),
total_sectors = plan.total_sectors,
"synthesized UDF image over directory"
);
Ok(Self {
meta,
ranges,
files,
open: Vec::new(),
total_sectors: plan.total_sectors,
volume_id: plan.volume_id,
data_bytes,
})
}
/// UDF volume identifier the image declares (the folder's own name).
pub fn volume_id(&self) -> &str {
&self.volume_id
}
/// Total bytes of real file content the image carries — the folder's size,
/// not the image's (which also counts metadata and inter-file gaps).
pub fn data_bytes(&self) -> u64 {
self.data_bytes
}
/// The range covering `lba`, if any.
fn range_at(&self, lba: u32) -> Option<&DataRange> {
let i = self.ranges.partition_point(|r| r.start_lba <= lba);
let r = self.ranges.get(i.checked_sub(1)?)?;
(lba < r.start_lba + r.sectors).then_some(r)
}
/// Borrow an open handle for `file`, opening it (and evicting the
/// least-recently-used handle) if necessary.
///
/// Opening is also where the plan is revalidated. A folder is not a disc:
/// a file can be shortened or replaced between planning and reading, and
/// zero-filling the difference would turn "the user deleted something"
/// into corrupt output at exit 0. The size is re-checked here, and a
/// truncation that happens while the handle is already open is caught by
/// the short read in [`Self::fill`].
fn handle(&mut self, file: usize) -> Result<&mut File> {
if let Some(pos) = self.open.iter().position(|(i, _)| *i == file) {
// `open` is ordered most-recently-used first.
let entry = self.open.remove(pos);
self.open.insert(0, entry);
return Ok(&mut self.open[0].1);
}
let f = File::open(&self.files[file].host).map_err(Error::from)?;
let md = f.metadata().map_err(Error::from)?;
// Size AND mtime. Size alone is content-blind, and this plan depends on
// content: a DVD's VOB placement comes from bytes 0xC0/0xC4 of its IFO,
// and an IFO rewritten in place keeps its length because IFOs occupy a
// whole number of sectors. The size check would pass while every title
// extent pointed at the wrong sectors — corrupt video behind an intact
// structure, reported complete at exit 0.
//
// Only compared when both sides have a timestamp; a platform or
// filesystem that reports none simply falls back to the size check
// rather than failing every read.
let changed_size = md.len() != self.files[file].size;
let changed_mtime = match (self.files[file].mtime, md.modified().ok()) {
(Some(planned), Some(live)) => planned != live,
_ => false,
};
if changed_size || changed_mtime {
return Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
});
}
if self.open.len() >= HANDLE_CACHE {
self.open.pop();
}
self.open.insert(0, (file, f));
Ok(&mut self.open[0].1)
}
/// Fill `out` (a whole number of sectors) from one data range, starting at
/// `lba`. `out` is already zeroed, so a file's tail sector comes back
/// zero-padded — which is exactly what `file_extents`' `div_ceil(2048)`
/// (`udf.rs:816`) makes every consumer expect.
fn fill(&mut self, r: &DataRange, lba: u32, out: &mut [u8]) -> Result<()> {
let within = (lba - r.start_lba) as u64 * SECTOR as u64;
let want = (r.bytes.saturating_sub(within)).min(out.len() as u64) as usize;
if want == 0 {
return Ok(());
}
let at = r.offset + within;
let file = r.file;
let h = self.handle(file)?;
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Release the window just read, every time.
//
// The ISO source accumulates and drops in chunks because it reads
// one file linearly, so a running start offset always names the
// bytes it has consumed. Reads here jump between files, so there is
// no single cursor to accumulate against — an accumulated byte
// count paired with one read's offset names 1/Nth of what was
// actually consumed and leaves the rest pinned, which is how the
// first version of this got it wrong.
//
// Dropping per read costs one advisory syscall per batch (4-16 MiB),
// which is nothing against the read itself, and it is correct
// regardless of how reads interleave across files.
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
}
match res {
Ok(()) => Ok(()),
// The file shrank while the handle was open. Same verdict as the
// size check in `handle`, reached the other way.
Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
})
}
Err(e) => Err(Error::from(e)),
}
}
}
impl SectorSource for DirImage {
fn capacity_sectors(&self) -> u32 {
self.total_sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let need = count as usize * SECTOR;
if buf.len() < need {
return Err(Error::UdfBufferTooSmall);
}
buf[..need].fill(0);
// Walk the request in RUNS, not sector by sector. A mux batch is 8192
// sectors and almost always lands entirely inside one stream file's
// extent; per-sector seek+read would issue 8192 syscalls for what is
// one 16 MiB sequential read.
let mut i = 0u32;
while i < count as u32 {
// Checked: callers saturate their LBAs (`disc/dvd.rs` builds a cell
// start as `vob_start_sector.saturating_add(cell.first_sector)`, and
// the prefetcher adds an offset the same way), so a crafted IFO can
// present a request at the very top of the address space. Wrapping
// here would fold `at` back to a LOW sector and hand the muxer a
// different file's bytes with nothing reported.
let Some(at) = lba.checked_add(i) else {
break;
};
let off = i as usize * SECTOR;
if let Some(s) = self.meta.get(&at) {
buf[off..off + SECTOR].copy_from_slice(&s[..]);
i += 1;
continue;
}
// Metadata blocks all sit below the data floor, so a data range is
// never interrupted by one.
match self.range_at(at).cloned() {
Some(r) => {
let run = (r.start_lba + r.sectors - at).min(count as u32 - i);
let end = off + run as usize * SECTOR;
self.fill(&r, at, &mut buf[off..end])?;
i += run;
}
// A gap between planned extents. Reads as zeros, exactly as an
// unrecorded sector of a real image does.
None => i += 1,
}
}
Ok(need)
}
}
#[cfg(test)]
mod tests;
File diff suppressed because it is too large Load Diff
+1377 -121
View File
File diff suppressed because it is too large Load Diff
+214 -22
View File
@@ -7,13 +7,32 @@ use crate::udf;
impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Vec<DiscTitle> {
halt: Option<&crate::halt::Halt>,
) -> Result<Vec<DiscTitle>> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(_) => return Vec::new(),
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok(Vec::new()),
};
let mut titles = Vec::new();
@@ -179,7 +198,10 @@ impl Disc {
// the coded video frame the subpicture was authored against
// (720x480 NTSC / 720x576 PAL) so players place and scale the
// bitmap correctly.
let (vid_w, vid_h) = ts.video.resolution.pixels();
// format_palette guards on (0, 0) and omits its `size:` line,
// so an unresolved resolution degrades to a palette-only .idx
// rather than one claiming a 0x0 frame.
let (vid_w, vid_h) = ts.video.resolution.pixels().unwrap_or((0, 0));
let codec_data = dvd_title
.palette
.as_ref()
@@ -240,7 +262,14 @@ impl Disc {
}
}
titles
// Polled again AFTER the loop: a cancel raised during the IFO reads
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok(titles)
}
}
@@ -525,6 +554,118 @@ mod tests {
// Tests
// ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|ts| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|ts| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test]
@@ -532,7 +673,11 @@ mod tests {
let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]);
assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.is_empty()
);
}
/// Single VTS, single title, one cell. Extent absolute LBA =
@@ -568,7 +713,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -624,7 +769,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -682,7 +827,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -740,7 +885,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -781,7 +926,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -836,7 +981,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -897,7 +1042,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -908,7 +1053,7 @@ mod tests {
.collect();
assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "en");
assert_eq!(audios[0].language, "eng");
assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -967,7 +1112,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1024,7 +1169,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1037,7 +1182,7 @@ mod tests {
// Languages preserved in order.
assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["en", "fr", "de"]
vec!["eng", "fra", "deu"]
);
// PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1084,7 +1229,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let sub = t
.streams
.iter()
@@ -1094,7 +1239,7 @@ mod tests {
})
.expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "en");
assert_eq!(sub.language, "eng");
assert!(
sub.codec_data.is_some(),
"non-zero palette must yield codec_data"
@@ -1138,7 +1283,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1175,7 +1320,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1260,7 +1405,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1312,7 +1457,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1320,4 +1465,51 @@ mod tests {
// Chapter 0 stays at 0.0 (no shift).
assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01);
}
/// Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio
/// stream has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio,
/// per `ifo::assign_audio_sub_stream_ids` — the PID falls back to
/// `0xBD00 + i` where `i` is the stream's positional index in the IFO
/// audio-attribute table. Two MPEG-audio (coding_mode 2) streams must
/// land on two DISTINCT, correctly-offset PIDs: 0xBD00 and 0xBD01. This
/// pins the `+` (not `-`/`*`) so the second stream doesn't collide with,
/// or wrap under, the first.
#[test]
fn scan_dvd_titles_mp2_audio_pid_fallback_is_additive() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// coding_mode bits are b0>>5 & 0x7; mode 2 = MPEG-1 Layer II (Mp2),
// which `assign_audio_sub_stream_ids` leaves at `sub_stream_id: None`.
// b0 = 0b010_00000 = 0x40. b1 = 0 (mono, sample rate 48k).
let audio = [(0x40u8, 0x00u8, [0u8, 0u8]), (0x40u8, 0x00u8, [0u8, 0u8])];
let vts = build_vts(1000, 0x00, &audio, &[], &[(10, 109)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts,
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
.iter()
.filter_map(|s| match s {
Stream::Audio(a) => Some(a.pid),
_ => None,
})
.collect();
assert_eq!(audio_pids, vec![0xBD00u16, 0xBD01u16]);
}
}
+173 -5
View File
@@ -104,10 +104,10 @@ fn max_substream_channels(data: &[u8]) -> Option<u8> {
};
let start = pos + rel;
let frame = &data[start..];
if let Some(ch) = ac3::acmod_channels(frame) {
if ch > 0 {
best = Some(best.map_or(ch, |b| b.max(ch)));
}
if let Some(ch) = ac3::acmod_channels(frame)
&& ch > 0
{
best = Some(best.map_or(ch, |b| b.max(ch)));
}
// Advance past this frame by its declared size when that is mappable;
// otherwise step 2 bytes past the sync and re-scan for the next one.
@@ -242,7 +242,11 @@ pub fn probe_and_remap<S: SectorSource + ?Sized>(
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate};
use crate::disc::{
AudioChannels, AudioStream, Codec, ContentFormat, DiscTitle, Extent, LabelPurpose,
SampleRate,
};
use crate::sector::SectorSource;
/// Build a single, correctly-SIZED AC-3 frame whose `acmod`/`lfeon` encode a
/// known channel count. `byte4` is `fscod=0 | frmsizecod=0`, so
@@ -463,4 +467,168 @@ mod tests {
};
assert_eq!(a.pid, 0xBD80, "no probe data → keep ordinal");
}
/// `max_substream_channels` must locate the sync at its true ABSOLUTE
/// position (`pos + rel`) when it is preceded by non-sync bytes, not just
/// when the sync sits at offset 0. Regression guard for a hand-checked
/// mutation (`+` → `-` at the `pos + rel` offset computation): with `pos`
/// starting at 0 and the first sync found 3 bytes in, `pos - rel` would
/// underflow a `usize` and panic, or (if it somehow didn't) index the
/// wrong start entirely. `pos + rel` is the only computation that is
/// always in-bounds, since `rel` is itself bounded by the length of the
/// slice searched from `pos`.
#[test]
fn max_substream_channels_locates_sync_after_leading_non_sync_bytes() {
let mut data = vec![0xAA, 0xAA, 0xAA]; // no 0x0B77 pattern in here
data.extend(ac3_frame(2, false)); // real 2.0 frame, sync at absolute offset 3
assert_eq!(
max_substream_channels(&data),
Some(2),
"must find and decode the frame whose sync is NOT at offset 0"
);
}
/// When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
/// `fscod == 3`), `max_substream_channels` must fall back to stepping
/// `start + 2` bytes past the sync to re-lock onto the next genuine sync,
/// and must keep making forward progress doing so (never revisit the same
/// sync, which would loop forever, and never jump so far that it skips
/// the very next real frame). This lays a bogus-sized header at absolute
/// offset 4 (so `start == 4`, `start + 2 == 6`) immediately followed, at
/// offset 6, by a real, fully decodable 2.0 frame — the position the
/// `+ 2` fallback must land on exactly.
#[test]
fn max_substream_channels_unmappable_size_steps_forward_by_two() {
let mut real = ac3_frame(2, false);
// Overwrite the (unchecked) CRC bytes of the real frame — these double
// as byte4/byte5 of the bogus header 2 bytes earlier, at absolute
// offset 4: byte4 = 0xC0 (fscod=3 reserved -> ac3_frame_size == 0,
// unmappable), byte5 = 0xF8 (bsid=31 >= 11 -> acmod_channels == None,
// so the bogus header itself never contributes a spurious channel
// count).
real[2] = 0xC0;
real[3] = 0xF8;
let mut data = vec![0xAA, 0xAA, 0xAA, 0xAA]; // offsets 0..4, no sync
data.push(0x0B); // offset 4: bogus header sync byte 0
data.push(0x77); // offset 5: bogus header sync byte 1
data.extend(real); // offset 6..: the real frame (also serves as the
// bogus header's byte4/byte5 at offsets 8/9)
assert_eq!(
max_substream_channels(&data),
Some(2),
"must recover the real frame 2 bytes after the unmappable-size sync, not lose it"
);
}
/// Same fallback as above, but with the unmappable-size sync at absolute
/// offset 0 (`start == 0`) so that stepping backward instead of forward
/// (`start - 2`) would underflow rather than merely land on the wrong
/// byte. Also proves the real frame is still found 6 bytes further in,
/// confirming forward progress past the bogus header.
#[test]
fn max_substream_channels_unmappable_size_at_start_steps_forward_not_back() {
let mut data = vec![0x0B, 0x77, 0x00, 0x00, 0xC0, 0xF8]; // bogus header, offsets 0..6
data.extend(ac3_frame(2, false)); // real 2.0 frame at offset 6
assert_eq!(
max_substream_channels(&data),
Some(2),
"must step forward past the bogus header at offset 0 and find the real frame at offset 6"
);
}
/// `remap_audio_pids` must read a stream's CURRENT physical sub-stream id
/// from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
/// `0x00FF`, both of which force the low byte to `0xFF` regardless of the
/// real PID and so always miss the "already matches" shortcut. That
/// matters observably when TWO physical sub-streams share the same probed
/// channel count: with a correct read, a stream already sitting on a
/// matching sub-stream is left alone (conservative, per the module's
/// documented behaviour); with the low byte forced to `0xFF`,
/// `probed.get(&0xFF)` is always `None`, so the code falls through to the
/// "find any unclaimed match" path and picks the FIRST (lowest-keyed,
/// BTreeMap-ordered) matching physical sub-stream instead — which here is
/// a *different* sub-stream (0x80) than the one the PID already correctly
/// names (0x81), producing a spurious PID change.
#[test]
fn remap_reads_current_substream_via_and_not_or_or_xor() {
let mut probed = BTreeMap::new();
probed.insert(0x80u8, 6u8);
probed.insert(0x81u8, 6u8); // ambiguous: two physical 6ch sub-streams
let mut streams = vec![ac3_stream(0xBD81, AudioChannels::Surround51)];
let changed = remap_audio_pids(&mut streams, &probed);
assert_eq!(
changed, 0,
"already sitting on a matching physical sub-stream (0x81) must be left alone"
);
let Stream::Audio(a) = &streams[0] else {
panic!()
};
assert_eq!(
a.pid, 0xBD81,
"must not be bumped to the other matching sub-stream (0x80)"
);
}
/// A `SectorSource` stub that hands back fixed bytes regardless of the
/// requested LBA/count, for exercising `probe_and_remap`'s end-to-end
/// wiring (format/AC-3/extent/count guards -> read -> probe -> remap).
struct FixedSource {
data: Vec<u8>,
}
impl SectorSource for FixedSource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let n = self.data.len().min(buf.len());
buf[..n].copy_from_slice(&self.data[..n]);
Ok(n)
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
/// stream to 0xBD81. It also, by construction, proves each of the guards
/// along the way lets a real, positive case through: the content-format
/// check must NOT bail on `MpegPs` (only on non-`MpegPs`), the AC-3
/// presence check must NOT bail when AC-3 IS present, and the
/// sector-count check must NOT bail when the count is nonzero — any one
/// of those inverted would skip the probe entirely and leave the PID at
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
playlist: "00001.ifo".into(),
playlist_id: 1,
duration_secs: 60.0,
size_bytes: bytes.len() as u64,
clips: Vec::new(),
streams: vec![ac3_stream(0xBD80, AudioChannels::Surround51)],
chapters: Vec::new(),
extents: vec![Extent {
start_lba: 0,
sector_count: 2,
}],
content_format: ContentFormat::MpegPs,
codec_privates: vec![None],
};
let mut source = FixedSource { data: bytes };
probe_and_remap(&mut source, &mut title);
let Stream::Audio(a) = &title.streams[0] else {
panic!("audio")
};
assert_eq!(
a.pid, 0xBD81,
"declared 5.1 stream must be re-routed to the physical 5.1 sub-stream 0x81"
);
}
}
+65 -34
View File
@@ -181,6 +181,24 @@ fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::trace::UnlockOutco
}
}
/// Did the cert handshake actually carry a Volume ID?
///
/// Extracted so it can be tested as a VALUE. It only ever reaches an operator
/// as the `has_volume_id` field of the `bus_key_unavailable` warn, and
/// asserting on a `tracing` field means installing a capturing subscriber —
/// which is thread-local, while `tracing`'s callsite-interest cache is global.
/// Those two facts race: the test failed roughly one run in ten under the full
/// parallel suite while passing every time in isolation, and serialising the
/// captures was not enough because the cache can be re-evaluated against the
/// process default rather than the thread-local dispatch.
///
/// A predicate this small does not need a subscriber to verify. The polarity is
/// the whole point: an `==` here would tell an operator a VID was absent on
/// exactly the discs where one was present.
fn handshake_has_volume_id(h: &HandshakeResult) -> bool {
h.volume_id != [0u8; 16]
}
impl Disc {
/// SCSI handshake — drives the VID-acquisition flow and returns
/// a structured `HandshakeResult` for downstream key resolution.
@@ -369,7 +387,7 @@ impl Disc {
// file/ISO, drive unlock, cert bus key). The gate enumerates nothing.
if !bus_encryption_removed(bus_encryption, handshake) {
let (rdk_err, has_vid) = handshake
.map(|h| (h.read_data_key_err, h.volume_id != [0u8; 16]))
.map(|h| (h.read_data_key_err, handshake_has_volume_id(h)))
.unwrap_or((None, false));
tracing::warn!(
target: "freemkv::disc",
@@ -949,41 +967,54 @@ mod tests {
/// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy
/// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`).
/// This is the damaged-primary recovery path real discs rely on.
#[test]
fn resolve_vid_only_falls_back_to_duplicate_unit_key_ro() {
let mut disc = MemDisc::new();
// Build AACS dir with a DUPLICATE subdir holding Unit_Key_RO.inf.
let uk = vec![0x55u8; 48];
let mut dup_fids = Vec::new();
push_fid(&mut dup_fids, "", 70, true, true);
push_fid(&mut dup_fids, "Unit_Key_RO.inf", 72, false, false);
disc.put(PART_START + 72, build_file_icb(uk.len() as u32, 9000));
disc.put_bytes(PART_START + 9000, &uk);
disc.put(PART_START + 70, build_file_icb(dup_fids.len() as u32, 71));
disc.put_bytes(PART_START + 71, &dup_fids);
// AACS dir: only a DUPLICATE subdir (no primary Unit_Key_RO.inf).
let mut aacs_fids = Vec::new();
push_fid(&mut aacs_fids, "", 50, true, true);
push_fid(&mut aacs_fids, "DUPLICATE", 70, true, false);
disc.put(PART_START + 50, build_file_icb(aacs_fids.len() as u32, 51));
disc.put_bytes(PART_START + 51, &aacs_fids);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "AACS", 50, true, false);
disc.put(PART_START + 10, build_file_icb(root_fids.len() as u32, 11));
disc.put_bytes(PART_START + 11, &root_fids);
build_udf_skeleton(&mut disc, 10);
let udf = udf::read_filesystem(&mut disc).expect("fs");
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("DUPLICATE fallback");
// disc_hash must be computed over the DUPLICATE bytes.
assert_eq!(
st.disc_hash,
aacs::inf::disc_hash_hex(&aacs::inf::disc_hash(&uk)),
"fallback must hash the DUPLICATE Unit_Key_RO.inf"
fn handshake_has_volume_id_reports_presence_not_absence() {
let with_vid = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
assert!(
super::handshake_has_volume_id(&with_vid),
"a non-zero Volume ID must report as PRESENT"
);
assert_eq!(st.uk_ro, uk);
let without = HandshakeResult {
volume_id: [0u8; 16],
..with_vid
};
assert!(
!super::handshake_has_volume_id(&without),
"an all-zero Volume ID is the absent case"
);
// One bit of difference is still a VID: the check is != all-zero, not a
// heuristic about how much of it looks populated.
let mut barely = [0u8; 16];
barely[15] = 1;
assert!(
super::handshake_has_volume_id(&HandshakeResult {
volume_id: barely,
..with_vid
}),
"any non-zero byte makes a Volume ID present"
);
}
/// The gate itself still hard-errors — the property the log line annotates.
#[test]
fn resolve_vid_only_bus_key_gate_hard_errors_without_a_read_data_key() {
let (mut disc, udf) = disc_with_cert(0x01, true);
let hs = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
let err = Disc::resolve_vid_only(&udf, &mut disc, Some(&hs))
.expect_err("bus-encrypted, no read_data_key must still hard-error");
assert!(matches!(err, Error::AacsBusKeyUnavailable));
}
// ---------------------------------------------------------------
+966 -83
View File
File diff suppressed because it is too large Load Diff
+2303 -61
View File
File diff suppressed because it is too large Load Diff
-1670
View File
File diff suppressed because it is too large Load Diff
+2794 -2547
View File
File diff suppressed because it is too large Load Diff
-1666
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-218
View File
@@ -1,218 +0,0 @@
//! `Disc::sweep`'s consumer-side `Sink<WorkItem>`.
//!
//! Background: the original sweep loop runs strictly serialised —
//! SCSI read → decrypt → seek + write → mapfile.record → next iter.
//! On a healthy disc the SCSI read costs ~5-12 ms per 64 KB batch and
//! the post-read work (decrypt 1-3 ms + file write + mapfile fsync
//! 5-15 ms) adds another batch's worth of latency. The drive idles
//! during the post-read work; throughput tops out at the *sum* of
//! both costs.
//!
//! A producer/consumer split overlaps the two stages on the generic
//! [`crate::io::Pipeline`] + [`crate::io::Sink`] primitive. This module
//! is the sweep-specific `Sink` impl; the producer-side state machine
//! (read_error context, decrypt, set_speed, halt) stays in
//! `Disc::sweep` in `disc/mod.rs`.
//!
//! Correctness invariants preserved:
//! - Mapfile is single-writer (consumer-only). No locking.
//! - All `read_error::ReadCtx` state stays on the producer thread.
//! - `set_speed` calls happen on the producer thread (same thread that
//! owns the `SectorSource`). No new SCSI concurrency.
//! - Per-iteration ordering of file-write → mapfile-record is kept
//! intact in the consumer (write before record), so the on-disk
//! invariant "mapfile only marks Finished what the file has
//! received" survives a crash mid-pass.
//! - Only one SCSI command is in flight at a time; error-path timing
//! is identical and no new retry logic is introduced.
use std::io::{Seek, SeekFrom, Write};
use std::sync::mpsc::{Receiver, SyncSender, sync_channel};
use crate::error::Error;
use crate::io::{Flow, Sink};
use super::mapfile::{MapStats, Mapfile, SectorStatus};
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. 64 KB
/// matches the existing zero_gap chunk size used by the pre-split
/// sweep loop.
const ZERO_CHUNK: usize = 64 * 1024;
/// Producer → Consumer messages. The consumer applies these in FIFO
/// order; ordering of file writes and mapfile records across items is
/// preserved.
pub(super) enum WorkItem {
/// Successful batch read. Producer has already decrypted `buf` if
/// `opts.decrypt` was set. Consumer writes `buf` at `pos` and
/// records the range as `Finished`.
Good { pos: u64, buf: Vec<u8> },
/// Bisect inner-loop good single sector (already decrypted by the
/// producer). 2048 bytes.
BisectGood { pos: u64, buf: Box<[u8; 2048]> },
/// Bisect inner-loop bad single sector. Consumer writes 2048
/// zeros at `pos` and records the sector as `NonTrimmed`.
BisectBad { pos: u64 },
/// Whole-batch zero-fill (failed batch on `SkipBlock`, or the
/// failed batch portion of `JumpAhead`). Consumer streams zeros
/// across `[pos, pos+len)` and records the range as `NonTrimmed`.
SkipFill { pos: u64, len: u64 },
/// Gap fill following a `JumpAhead`. Same effect as `SkipFill`;
/// distinguished only so future logging / instrumentation can
/// tell them apart without parsing a flag.
GapFill { pos: u64, len: u64 },
/// Producer wants the latest mapfile stats for the progress
/// callback. Consumer responds on `prog_tx` with a fresh
/// [`ProgressSnapshot`]. Best-effort: if the producer hasn't
/// drained the previous snapshot, the new one is silently
/// dropped — the producer's local cache stays current enough.
StatsRequest,
}
/// Snapshot the consumer sends back to the producer for the progress
/// callback.
pub(super) struct ProgressSnapshot {
pub stats: MapStats,
pub bad_ranges: Vec<(u64, u64)>,
}
/// Final summary returned by the consumer thread on shutdown — what
/// `SweepSink::close` produces, surfaced to the producer via
/// `Pipeline::finish`.
pub(super) struct ConsumerSummary {
pub stats: MapStats,
}
/// Drain any pending progress snapshots from the consumer. Returns
/// the most recent one, if any. The producer caches it and uses it
/// for subsequent progress callbacks until a fresh one arrives.
pub(super) fn try_recv_progress(rx: &Receiver<ProgressSnapshot>) -> Option<ProgressSnapshot> {
let mut latest = None;
while let Ok(snap) = rx.try_recv() {
latest = Some(snap);
}
latest
}
/// `Sink<WorkItem>` for sweep. Owns the writeback file + mapfile +
/// progress back-channel. `apply` carries the file-write +
/// mapfile.record per item; `close` drains the writeback pipeline,
/// fsyncs the ISO, and flushes the mapfile.
pub(super) struct SweepSink {
file: crate::io::WritebackFile,
map: Mapfile,
/// `sync_all`-on-failure-is-an-error iff the output is a regular
/// file. `/dev/null` and pipes always fail `sync_all`; that's not
/// a real error.
is_regular: bool,
/// Back-channel for `StatsRequest` responses. The producer caches
/// the latest snapshot and uses it for the progress callback;
/// dropped sends on a full channel are by design.
prog_tx: SyncSender<ProgressSnapshot>,
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. Held
/// in the sink so each apply call doesn't reallocate.
zero: Box<[u8; ZERO_CHUNK]>,
}
impl SweepSink {
/// Construct a new `SweepSink` plus the matching progress
/// receiver. Channel depth on the back-channel is `1` — the
/// producer's cache is the source of truth between snapshots.
pub(super) fn new(
file: crate::io::WritebackFile,
map: Mapfile,
is_regular: bool,
) -> (Self, Receiver<ProgressSnapshot>) {
let (prog_tx, prog_rx) = sync_channel::<ProgressSnapshot>(1);
let sink = SweepSink {
file,
map,
is_regular,
prog_tx,
zero: Box::new([0u8; ZERO_CHUNK]),
};
(sink, prog_rx)
}
}
impl Sink<WorkItem> for SweepSink {
type Output = ConsumerSummary;
fn apply(&mut self, item: WorkItem) -> Result<Flow, Error> {
match item {
WorkItem::Good { pos, buf } => {
// Decrypt is on the producer; consumer assumes plaintext.
let len = buf.len() as u64;
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&buf)?;
self.map.record(pos, len, SectorStatus::Finished)?;
}
WorkItem::BisectGood { pos, buf } => {
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&buf[..])?;
self.map.record(pos, 2048, SectorStatus::Finished)?;
}
WorkItem::BisectBad { pos } => {
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&self.zero[..2048])?;
self.map.record(pos, 2048, SectorStatus::NonTrimmed)?;
}
WorkItem::SkipFill { pos, len } | WorkItem::GapFill { pos, len } => {
self.file.seek(SeekFrom::Start(pos))?;
// Subsequent writes are sequential; `WritebackFile`'s
// seek-elision keeps them on the writeback pipeline path.
let mut filled = 0u64;
while filled < len {
let chunk = (len - filled).min(self.zero.len() as u64) as usize;
self.file.write_all(&self.zero[..chunk])?;
filled += chunk as u64;
}
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::StatsRequest => {
let stats = self.map.stats();
// DAMAGE only — NOT NonTried. NonTried is the unread remainder
// ahead of the sweep head, not damage; including it made the live
// located drilldown (at-risk movie time + range count) treat the
// whole unread disc as confirmed damage, so at sweep start it
// showed ~full-movie at-risk and melted to 0 as the sweep
// progressed. Matches the one-shot progress path, which already
// excludes NonTried.
let bad_ranges = self.map.ranges_with(&[
SectorStatus::NonTrimmed,
SectorStatus::Unreadable,
SectorStatus::NonScraped,
]);
// Best-effort: drop on backpressure; producer's cache
// stays current enough.
let _ = self
.prog_tx
.try_send(ProgressSnapshot { stats, bad_ranges });
}
}
Ok(Flow::Continue)
}
fn close(mut self) -> Result<Self::Output, Error> {
// Drain the writeback pipeline + fsync the ISO, then persist
// any pending mapfile state. Same finalisation order as the
// pre-Pipeline consumer loop.
if let Err(e) = self.file.sync_all() {
if self.is_regular {
return Err(Error::IoError { source: e });
}
// Non-regular outputs (/dev/null, pipes) always fail
// sync_all; that's not a real error.
}
self.map.flush()?;
Ok(ConsumerSummary {
stats: self.map.stats(),
})
}
}
+6 -8
View File
@@ -23,14 +23,12 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
if !std::path::Path::new(&path).exists() {
continue;
}
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path)) {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((path, id));
}
}
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path))
&& let Ok(id) = DriveId::from_drive(transport.as_mut())
&& !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((path, id));
}
}
drives
+35 -6
View File
@@ -25,12 +25,11 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
let path = std::path::Path::new(&info.path);
match crate::scsi::open(path) {
Ok(mut transport) => {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((info.path.clone(), id));
}
if let Ok(id) = DriveId::from_drive(transport.as_mut())
&& !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((info.path.clone(), id));
}
}
Err(_) => {
@@ -53,3 +52,33 @@ pub fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
}
Ok((path.to_string(), DeviceResolution::Direct))
}
#[cfg(test)]
mod resolve_device_tests {
use super::*;
/// An existing path resolves unchanged as `Direct` — macOS has no
/// `sr`->`sg` substitution, so the returned path must be byte-identical
/// to the input, not some canonicalised/mutated form.
#[test]
fn existing_path_resolves_direct_unchanged() {
// Use the test binary's own executable path: guaranteed to exist,
// no fixture file needed.
let exe = std::env::current_exe().unwrap();
let path = exe.to_str().unwrap();
let (resolved, kind) = resolve_device(path).expect("existing path must resolve");
assert_eq!(resolved, path, "path must be returned unchanged");
assert_eq!(kind, DeviceResolution::Direct);
}
/// A path that does not exist must error with `DeviceNotFound` carrying
/// the original path, never silently succeed.
#[test]
fn missing_path_is_device_not_found() {
let path = "/dev/freemkv-definitely-does-not-exist-0xdead";
match resolve_device(path) {
Err(Error::DeviceNotFound { path: p }) => assert_eq!(p, path),
other => panic!("expected DeviceNotFound, got {other:?}"),
}
}
}
+1068 -134
View File
File diff suppressed because it is too large Load Diff
+823 -123
View File
File diff suppressed because it is too large Load Diff
+267
View File
@@ -0,0 +1,267 @@
//! Seeded robustness harness for the untrusted-input parsers.
//!
//! Every parser reached from here takes bytes that came off a disc, and this
//! crate's primary boundary is that the disc is untrusted: a malformed, damaged
//! or hostile image must never crash the library. These tests assert exactly
//! that one property — **the parser returns `Ok` or `Err`, and never panics.**
//!
//! # Why this exists rather than `cargo-fuzz`
//!
//! `cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
//! for libFuzzer's coverage feedback) and this project pins stable. So the
//! generator lives here instead. It gives up coverage-guided mutation — the real
//! loss — and keeps everything else: millions of cases, structure-aware input,
//! and a crash corpus. It also gains determinism, which a fuzzer does not have:
//! the same seed replays the same cases on any machine.
//!
//! # Why no `proptest` or `arbitrary`
//!
//! This crate has exactly one dev-dependency. That is a deliberate posture, and
//! a randomness crate is not worth ten transitive dependencies when the parsers
//! take plain `&[u8]` and a good enough generator is forty lines.
//!
//! # Budget
//!
//! `FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
//! enough that the per-commit gate stays under a second). The overnight run sets
//! it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
//! fixed so a failure in CI reproduces locally verbatim.
//!
//! # On failure
//!
//! The panic message carries the seed, generator and case index. Re-run with
//! `FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
//! into `tests/corpus/` as a permanent regression fixture — discovery happens
//! here, defence happens there.
#![cfg(test)]
/// Marsaglia xorshift64. Not cryptographic and does not need to be: the job is
/// a reproducible spread of bytes, and a named algorithm beats an ad-hoc LCG
/// whose period nobody has checked.
struct Rng(u64);
impl Rng {
fn new(seed: u64) -> Self {
// A zero seed is a fixed point of xorshift — it would emit zeros forever
// and every generated case would be identical.
Self(if seed == 0 {
0x2545_F491_4F6C_DD1D
} else {
seed
})
}
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
fn byte(&mut self) -> u8 {
(self.next() >> 24) as u8
}
/// Uniform-ish in `0..n`. The modulo bias is irrelevant at these magnitudes.
fn below(&mut self, n: usize) -> usize {
if n == 0 {
0
} else {
(self.next() % n as u64) as usize
}
}
fn fill(&mut self, len: usize) -> Vec<u8> {
(0..len).map(|_| self.byte()).collect()
}
}
/// Budget per generator per target.
fn cases() -> usize {
std::env::var("FREEMKV_HARNESS_CASES")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(256)
}
fn seed() -> u64 {
std::env::var("FREEMKV_HARNESS_SEED")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(0x5EED_1234_ABCD_0001)
}
/// Largest generated input. Big enough to carry a plausible header plus a body,
/// small enough that millions of cases stay quick.
const MAX_LEN: usize = 4096;
/// Drive `f` over three generators and report which case broke it.
///
/// A panic inside `f` fails the test on its own — nothing is caught here,
/// because catching would risk reporting a pass on an input that aborted. The
/// wrapper exists to make the failing case *identifiable*: the harness prints
/// the seed, generator and index before each call, so the last line before a
/// panic names the exact case to reproduce.
fn sweep<F: FnMut(&[u8])>(target: &str, magic: &[u8], f: F) {
sweep_n(target, magic, cases(), f)
}
/// `sweep` with an explicit budget. The budget is a PARAMETER rather than read
/// from the environment inside the loop: the meta-tests below need a small,
/// fixed count, and `std::env::set_var` is unsound once the test harness runs
/// tests in parallel — two tests setting the same variable race, which is
/// exactly what happened on the first run of this file.
fn sweep_n<F: FnMut(&[u8])>(target: &str, magic: &[u8], n: usize, mut f: F) {
let s = seed();
// 1. Pure random bytes. Cheap, and almost always rejected at the magic
// number — it exercises the entry guards and little else. Kept because
// the entry guards are themselves worth exercising.
let mut rng = Rng::new(s);
for i in 0..n {
let len = rng.below(MAX_LEN);
let buf = rng.fill(len);
run(target, "random", s, i, &buf, &mut f);
}
// 2. Valid magic, random body. THE generator that matters: pure random
// input dies at the magic check and never reaches the parser body, so
// without this the sweep only ever tests the first few lines.
let mut rng = Rng::new(s ^ 0xA5A5_A5A5_A5A5_A5A5);
for i in 0..n {
let mut buf = magic.to_vec();
let tail = rng.below(MAX_LEN.saturating_sub(magic.len()));
buf.extend(rng.fill(tail));
run(target, "magic+noise", s, i, &buf, &mut f);
}
// 3. Structured mutation of a plausible record: a valid magic, then mostly
// zeroes, with a handful of bytes corrupted and a truncation. Length and
// offset fields live in those early bytes, so this is what reaches the
// arithmetic — the offsets, counts and sizes a hostile image would lie
// about.
let mut rng = Rng::new(s ^ 0x1234_5678_9ABC_DEF0);
for i in 0..n {
let mut buf = vec![0u8; 512];
buf[..magic.len().min(512)].copy_from_slice(&magic[..magic.len().min(512)]);
for _ in 0..rng.below(24) + 1 {
let at = rng.below(buf.len());
buf[at] = rng.byte();
}
buf.truncate(rng.below(buf.len()) + 1);
run(target, "mutate", s, i, &buf, &mut f);
}
}
fn run<F: FnMut(&[u8])>(target: &str, generator: &str, seed: u64, i: usize, buf: &[u8], f: &mut F) {
// Printed, not asserted: `cargo test` swallows stdout for passing tests and
// shows it for failing ones, so this line is invisible until it is the last
// thing before a panic — at which point it is exactly what is needed.
println!(
"harness {target}/{generator} seed={seed:#x} case={i} len={} :: \
FREEMKV_HARNESS_SEED={seed} to reproduce",
buf.len()
);
f(buf);
}
#[test]
fn mpls_parse_never_panics() {
sweep("mpls", b"MPLS", |b| {
let _ = crate::mpls::parse(b);
});
}
#[test]
fn clpi_parse_never_panics() {
sweep("clpi", b"HDMV", |b| {
let _ = crate::clpi::parse(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
// input to reject, so the "magic" is a byte the sweep will mutate anyway.
sweep("udf_name", &[8], |b| {
let _ = crate::udf::parse_udf_name(b);
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
// each case is fed to a FRESH demuxer and then a shared one. The shared pass
// is what exercises cross-feed state — a start code split over a boundary,
// a held PES completed by later bytes, the carry-over cap.
let mut shared = crate::mux::ps::PsDemuxer::new();
sweep("ps_demux", &[0x00, 0x00, 0x01, 0xBA], |b| {
let mut fresh = crate::mux::ps::PsDemuxer::new();
let _ = fresh.feed(b);
let _ = shared.feed(b);
});
}
#[test]
fn mkv_lacing_split_never_panics() {
// All four lacing modes, including the reserved bit pattern. A degenerate
// fixed lace was a real defect found by audit round 5.
sweep("mkv_lacing", &[0x00], |b| {
for lacing in 0u8..=3 {
let _ = crate::mux::mkvstream::split_lacing(lacing, b);
}
});
}
/// The generators must actually differ, or the sweep is one generator run three
/// times and the coverage claim is false.
#[test]
fn the_three_generators_produce_different_inputs() {
let mut seen: Vec<Vec<u8>> = Vec::new();
sweep_n("probe", b"MPLS", 1, |b| seen.push(b.to_vec()));
assert_eq!(seen.len(), 3, "one case per generator");
assert_ne!(seen[0], seen[1], "random and magic+noise must differ");
assert_ne!(seen[1], seen[2], "magic+noise and mutate must differ");
assert!(
seen[1].starts_with(b"MPLS"),
"the magic+noise generator must actually carry the magic, or it never \
reaches the parser body"
);
}
/// The same seed must replay the same bytes, or a reported failure cannot be
/// reproduced and the harness is worthless as a regression tool.
#[test]
fn a_seed_replays_identically() {
let mut a = Vec::new();
let mut b = Vec::new();
sweep_n("probe", b"MPLS", 4, |x| a.push(x.to_vec()));
sweep_n("probe", b"MPLS", 4, |x| b.push(x.to_vec()));
assert_eq!(a, b, "the same seed must produce the same cases");
}
/// The harness is worthless if its cases die at the entry guards, so this
/// MEASURES how deep they actually reach instead of assuming. A generator that
/// never gets past a length or magic check exercises the first ten lines and
/// nothing else — the fuzzing equivalent of a test that cannot fail.
#[test]
fn the_generators_actually_reach_the_parser_bodies() {
// mpls::parse rejects at: len < 40, bad magic, then playlist_start + 10 >
// len. Anything that returns Ok got all the way through the play-item loop.
let mut ok = 0usize;
let mut total = 0usize;
sweep_n("reach", b"MPLS", 20000, |b| {
total += 1;
if crate::mpls::parse(b).is_ok() {
ok += 1;
}
});
assert!(
ok > 0,
"not one of {total} generated cases parsed successfully — the generators \
are all being rejected at the entry guards, so this harness is testing \
the guards and nothing behind them"
);
println!("mpls reach: {ok}/{total} cases parsed to completion");
}
+1 -1
View File
@@ -20,7 +20,7 @@ pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
let bytes = body.as_bytes();
// Empty → empty Vec (a legitimately-empty variable-length field); odd length
// is malformed. (`parse_hex_fixed` enforces a concrete length separately.)
if bytes.len() % 2 != 0 {
if !bytes.len().is_multiple_of(2) {
return None;
}
let mut out = Vec::with_capacity(bytes.len() / 2);
+193 -1
View File
@@ -49,13 +49,30 @@ pub struct DriveId {
pub raw_gc_010c: Vec<u8>,
}
/// SPC-4 standard INQUIRY data: 36 bytes through `product_revision`. Anything
/// shorter cannot populate the identity fields this type promises.
const INQUIRY_STANDARD_LEN: usize = 36;
impl DriveId {
/// Probe a real drive via SCSI and build its identity.
pub fn from_drive(transport: &mut dyn ScsiTransport) -> Result<Self> {
// INQUIRY — SPC-4 §6.4
let mut inquiry = vec![0u8; 96];
let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00];
transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
let inq = transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
// `bytes_transferred` is device-reported and untrusted — the same rule
// the two GET CONFIGURATION calls below already apply. It was ignored
// here, and the buffer is pre-zeroed, so a drive answering GOOD with a
// short or empty data phase (a USB-SATA bridge mid-wedge does exactly
// this) decoded to blank identity strings and a byte 0 of 0x00. Every
// platform enumerator gates on `raw_inquiry[0] & 0x1F == OPTICAL`, so
// 0x00 reads as DIRECT ACCESS and the drive silently disappears from
// the device list instead of reporting a failed probe.
if inq.bytes_transferred < INQUIRY_STANDARD_LEN {
return Err(crate::error::Error::DriveInquiryShort);
}
// Never decode past what the drive actually sent.
inquiry.truncate(inq.bytes_transferred.min(inquiry.len()));
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6.
// Best-effort: 010Ch (Firmware Information) is an optional feature.
@@ -262,6 +279,54 @@ mod tests {
/// Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer
/// (e.g. a device that reports fewer than 8 bytes) must not panic.
/// Mutation: removing the `data.len() > start` guard makes it panic on short inputs.
/// A drive that answers INQUIRY with GOOD status but a short or empty
/// data phase must fail the probe, not present as a blank drive.
///
/// The buffer is pre-zeroed, so decoding it unconditionally yielded empty
/// vendor/product/revision strings and a byte 0 of 0x00. Every platform
/// enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
/// and 0x00 is DIRECT ACCESS — so the drive silently vanished from the
/// device list rather than reporting that its identity probe failed. A
/// USB-SATA bridge mid-wedge does exactly this.
///
/// The two GET CONFIGURATION calls in the same function already clamped on
/// `bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
/// lines above them, discarded it.
#[test]
fn inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive() {
/// GOOD status, no sense, and only `n` bytes written.
struct ShortInquiry(usize);
impl ScsiTransport for ShortInquiry {
fn execute(
&mut self,
_cdb: &[u8],
_dir: DataDirection,
_buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Ok(ScsiResult {
status: 0,
sense: [0u8; 32],
bytes_transferred: self.0,
})
}
}
// Empty data phase — the case that made a real drive disappear.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(0)),
Err(crate::error::Error::DriveInquiryShort)
));
// One byte short of the SPC-4 standard 36-byte header.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(35)),
Err(crate::error::Error::DriveInquiryShort)
));
// Exactly the standard length is acceptable: the optional
// vendor-specific tail past byte 36 is allowed to be absent.
assert!(DriveId::from_drive(&mut ShortInquiry(36)).is_ok());
}
#[test]
fn ascii_field_short_buffer_returns_empty() {
// Buffer of length 5: start=8 is beyond the end → empty string.
@@ -339,9 +404,136 @@ mod tests {
);
}
/// `ascii_field`'s guard is `data.len() > start` (strictly greater), not
/// `>=`: a buffer whose length is exactly `start` has NO byte at that
/// offset, so it must still yield empty, not attempt to slice.
/// Mutation: `>` -> `>=` would try to slice `data[start..]` when
/// `data.len() == start`, which panics (empty range at the very end is
/// fine, but the guard's job is the `< start` case below it — pinning the
/// exact boundary catches an off-by-one either direction).
#[test]
fn ascii_field_boundary_len_equals_start_is_empty() {
let buf = vec![0u8; 8];
assert_eq!(ascii_field(&buf, 8, 16), "");
}
/// One byte past the boundary: `data.len() == start + 1` must extract
/// that single byte (clamped to `end`), proving the guard is `>` and not
/// off by one in the other direction.
#[test]
fn ascii_field_boundary_len_one_past_start_extracts_one_byte() {
let mut buf = vec![0u8; 9];
buf[8] = b'X';
assert_eq!(ascii_field(&buf, 8, 16), "X");
}
/// `Display` renders the four trimmed identity fields space-separated —
/// the human-readable counterpart of `match_key`'s pipe-separated form.
/// Not exercised anywhere else in this test module.
/// Mutation: replacing the `fmt` body with `Ok(Default::default())`
/// writes nothing at all, so formatting any `DriveId` yields "".
#[test]
fn display_formats_trimmed_fields_space_separated() {
let mut inquiry = vec![0u8; 96];
inquiry[8..16].copy_from_slice(b"PIONEER ");
inquiry[16..32].copy_from_slice(b"BD-RW BDR-S09 ");
inquiry[32..36].copy_from_slice(b"1.34");
inquiry[36..43].copy_from_slice(b" 16/04/");
let id = DriveId::from_inquiry(&inquiry, "201604250000");
assert_eq!(id.to_string(), "PIONEER BD-RW BDR-S09 1.34 16/04/");
}
/// GET CONFIGURATION failure (transport error) must not abort the
/// identity probe — firmware_date is empty, raw_gc_010c is empty.
/// Mutation: propagating the GET_CONFIGURATION error with `?` aborts from_drive.
/// Transport whose GET CONFIGURATION responses report an exact,
/// caller-chosen `bytes_transferred` for each of the two GC features
/// (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
/// boundary guards can be pinned precisely. INQUIRY always succeeds.
struct FixedGcCountTransport {
firmware_bytes: usize,
serial_bytes: usize,
}
impl ScsiTransport for FixedGcCountTransport {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
for b in buf.iter_mut() {
*b = b'Z';
}
let bytes_transferred = match cdb.first() {
Some(&0x12) => buf.len(),
Some(&0x46) if cdb[3] == 0x0C => self.firmware_bytes,
Some(&0x46) if cdb[3] == 0x08 => self.serial_bytes,
_ => buf.len(),
};
Ok(ScsiResult {
status: 0,
bytes_transferred,
sense: [0u8; 32],
})
}
}
/// `end > 12` in the firmware-date branch (`from_drive`) is a strict
/// inequality: `bytes_transferred == 12` reports the field absent
/// (offset 12 is the first byte of the 12-char date; a count of exactly
/// 12 covers bytes 0..12, none of which is the date), so `firmware_date`
/// must be empty, not the mutant's off-by-one read.
/// Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an
/// empty but non-panicking slice — silently reporting "present" data
/// that is actually all outside the transferred count.
#[test]
fn from_drive_firmware_date_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 12,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "");
}
/// One byte past the boundary (`bytes_transferred == 13`) must extract
/// exactly the one available date byte (offset 12), proving the guard
/// is `>` and the slice end is clamped to `end`, not always to 24.
#[test]
fn from_drive_firmware_date_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 13,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "Z");
}
/// Same `> 12` boundary for the serial-number branch: exactly 12
/// transferred bytes must yield an empty serial.
#[test]
fn from_drive_serial_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 12,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "");
}
/// One byte past the serial boundary extracts exactly that byte.
#[test]
fn from_drive_serial_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 13,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "Z");
}
#[test]
fn from_drive_gc_failure_yields_empty_firmware_date() {
struct GcFailTransport;
+1168 -107
View File
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -133,10 +133,10 @@ where
match rx.recv_timeout(slice) {
Ok(v) => return Ok(v),
Err(RecvTimeoutError::Timeout) => {
if let Some(h) = halt {
if h.is_cancelled() {
return Err(BoundedError::Halted);
}
if let Some(h) = halt
&& h.is_cancelled()
{
return Err(BoundedError::Halted);
}
if Instant::now() >= deadline {
return Err(BoundedError::Timeout);
+21
View File
@@ -249,6 +249,27 @@ impl Drop for BytePrefetcher {
mod tests {
use super::*;
/// `RECYCLE_DEPTH` must be one MORE than `FORWARD_DEPTH` per its own
/// doc comment: the producer needs at least one buffer to fill while
/// the consumer holds the other `FORWARD_DEPTH`-worth in flight. A
/// `+` -> `*`/`-` mutation on `FORWARD_DEPTH + 1` would under-size the
/// recycle channel (e.g. `FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one
/// short), which starves the producer of a spare buffer.
#[test]
fn recycle_depth_is_forward_depth_plus_one() {
assert_eq!(RECYCLE_DEPTH, FORWARD_DEPTH + 1);
assert_eq!(RECYCLE_DEPTH, 3, "FORWARD_DEPTH is 2, so recycle must be 3");
}
/// `DEFAULT_CHUNK_BYTES` is documented as 16 MiB. Pins the literal so a
/// `*` -> `+`/`/` mutation on either factor (16 * 1024 * 1024) is
/// caught by a concrete, spec-derived expected value rather than by
/// recomputing the same expression.
#[test]
fn default_chunk_bytes_is_16_mib() {
assert_eq!(DEFAULT_CHUNK_BYTES, 16_777_216, "documented as 16 MiB");
}
/// Endless reader: every `read` fills the whole buffer and never
/// hits EOF, so the producer keeps trying to push batches forward
/// until the forward channel disconnects. Exactly the shape that
+3 -3
View File
@@ -23,7 +23,7 @@
use std::fs::File;
use std::os::unix::io::AsRawFd;
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
pub(crate) fn hint_sequential(file: &File, _len_bytes: u64) {
// Best-effort: return value ignored. A fadvise failure has no
// user-observable consequence.
unsafe {
@@ -34,7 +34,7 @@ pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
/// Drop pages in the half-open byte range `[start, start+len)` from
/// the page cache. Called periodically by `read_sectors` to bound the
/// read-side page cache pressure.
pub(super) fn drop_window(file: &File, start: u64, len: u64) {
pub(crate) fn drop_window(file: &File, start: u64, len: u64) {
unsafe {
libc::posix_fadvise(
file.as_raw_fd(),
@@ -58,7 +58,7 @@ pub(super) fn drop_window(file: &File, start: u64, len: u64) {
/// can only pre-stage a tiny slice of the next batch. An explicit
/// `readahead()` of the same size as the current batch tells the
/// kernel to queue the full next-batch read now.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) {
unsafe {
libc::readahead(file.as_raw_fd(), offset as i64, len as usize);
}
+3 -3
View File
@@ -15,7 +15,7 @@ use std::os::unix::io::AsRawFd;
/// pipeline depth.
const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024;
pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
pub(crate) fn hint_sequential(file: &File, len_bytes: u64) {
let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory {
ra_offset: 0,
@@ -33,14 +33,14 @@ pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
/// approximation: no-op. macOS's unified buffer cache is generally
/// less prone to the pin-everything pathology that triggers the
/// regression on Linux NFS clients.
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Async-prefetch the byte range `[offset, offset+len)`. macOS uses
/// the same `fcntl(F_RDADVISE, &radvisory)` primitive as the open-
/// time sequential hint, just targeted at a moving window instead of
/// the whole file. The kernel queues I/O for the requested range and
/// returns immediately.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) {
let bytes = (len as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory {
ra_offset: offset as libc::off_t,
+131 -21
View File
@@ -48,22 +48,25 @@
//! far smaller than our 16 MiB app-level batch.
#[cfg(target_os = "linux")]
mod linux;
pub(crate) mod linux;
#[cfg(target_os = "macos")]
mod macos;
pub(crate) mod macos;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
mod other;
pub(crate) mod other;
#[cfg(target_os = "windows")]
mod windows;
pub(crate) mod windows;
// The page-cache hints are shared with any other file-backed sector source:
// `dirimage` reads host files the same way and needs the same eviction, or a
// large rip pins every byte it has read (see this module's DONTNEED note).
#[cfg(target_os = "linux")]
use linux as platform;
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
use macos as platform;
pub(crate) use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use other as platform;
pub(crate) use other as platform;
#[cfg(target_os = "windows")]
use windows as platform;
pub(crate) use windows as platform;
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
@@ -85,11 +88,27 @@ use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// pressure concurrent writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
const READ_DROP_CHUNK_BYTES_DEFAULT: u64 = 32 * 1024 * 1024;
/// Upper bound (in MiB) accepted from `FREEMKV_READ_DROP_CHUNK_MIB`. 64 GiB —
/// generous for any real medium, and small enough that `n * 1024 * 1024` cannot
/// wrap `u64`. Mirrors `WRITEBACK_CHUNK_MIB_MAX`, whose identical multiply is
/// bounded for exactly this reason: without the bound, a value above 2^44
/// overflows — a panic on the first ISO open in an overflow-checked build, and in
/// release a wrap to a near-zero window that fires `drop_window` on every read.
/// Out-of-range values fall back to the default.
const READ_DROP_CHUNK_MIB_MAX: u64 = 64 * 1024;
fn read_drop_chunk_bytes() -> u64 {
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok())
.filter(|&n| n > 0)
resolve_read_drop_chunk(
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok()),
)
}
/// The pure part of [`read_drop_chunk_bytes`], split out so the bound is
/// testable without mutating process environment.
fn resolve_read_drop_chunk(mib: Option<u64>) -> u64 {
mib.filter(|&n| n > 0 && n <= READ_DROP_CHUNK_MIB_MAX)
.map(|n| n * 1024 * 1024)
.unwrap_or(READ_DROP_CHUNK_BYTES_DEFAULT)
}
@@ -171,12 +190,20 @@ impl SectorSource for FileSectorSource {
) -> Result<usize> {
let count = count as u32;
let bytes = count as usize * SECTOR_BYTES;
debug_assert!(
out.len() >= bytes,
"FileSectorSource::read_sectors: out len {} < requested {}",
out.len(),
bytes
);
// A real check, not a debug_assert: this is a public `SectorSource` impl,
// so an undersized `out` is caller input, and `out[..bytes]` below would
// panic with 'range end index out of range' in release where the assert is
// compiled away. `Drive::read_fua` already carries exactly this guard, with
// a comment recording the same panic being fixed there — this impl was
// simply never given it, and `PrefetchedSectorSource` has a regression test
// for the case that this one lacked.
if out.len() < bytes {
return Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
});
}
if count == 0 {
return Ok(0);
}
@@ -219,6 +246,40 @@ mod tests {
use std::io::Write;
use tempfile::tempdir;
/// An undersized output buffer must return an error, never panic. This is a
/// public `SectorSource` impl, so buffer length is caller input, and the guard
/// used to be a `debug_assert!` — compiled out in release, where the
/// `out[..bytes]` slice then panicked with 'range end index out of range'.
///
/// `Drive::read_fua` already carries this exact guard with a comment recording
/// the same panic being fixed there, and `PrefetchedSectorSource` has
/// `direct_read_too_small_buffer_errors` for the same case; this impl had
/// neither.
#[test]
fn read_sectors_with_an_undersized_buffer_errors_rather_than_panicking() {
let dir = tempdir().unwrap();
let iso = dir.path().join("t.iso");
make_iso(&iso, 8);
let mut src = FileSectorSource::open(&iso).expect("iso opens");
// Ask for four sectors but supply room for barely more than one.
let mut out = vec![0u8; SECTOR_BYTES + 1];
let err = src
.read_sectors(0, 4, &mut out, false)
.expect_err("an undersized buffer must be an error, not a panic");
assert!(
matches!(err, Error::DiscRead { .. }),
"expected DiscRead, got {err:?}"
);
// Exactly-sized still works, so the guard is not off by one.
let mut out = vec![0u8; 4 * SECTOR_BYTES];
assert_eq!(
src.read_sectors(0, 4, &mut out, false).unwrap(),
4 * SECTOR_BYTES
);
}
/// Build a deterministic ISO of `sectors` sectors where sector `n`
/// is filled with the byte pattern `((n & 0xff) as u8)`. Lets us
/// verify any sector by content alone.
@@ -376,19 +437,36 @@ mod tests {
// Additional coverage.
// ---------------------------------------------------------------
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or
/// reading, even at an out-of-range LBA — the early-return guard
/// runs before any I/O. Grounding: `if count == 0 { return Ok(0) }`.
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or reading,
/// even at an out-of-range LBA. Grounding: `if count == 0 { return Ok(0) }`.
///
/// The `Ok(0)` return alone proves nothing: with the guard deleted, a seek
/// past EOF succeeds (POSIX permits seeking beyond the end of a file) and a
/// zero-length `read_exact` returns `Ok(())` immediately, so the call still
/// returns `Ok(0)`. The observable difference is the file's cursor — the
/// seek MOVES it to `lba * 2048`. Assert on that, so the guard is what the
/// test is actually measuring.
#[test]
fn zero_count_returns_zero_no_io() {
let dir = tempdir().unwrap();
let path = dir.path().join("zc.iso");
make_iso(&path, 4);
let mut src = FileSectorSource::open(&path).unwrap();
let before = src.file.stream_position().expect("cursor readable");
assert_eq!(before, 0, "a freshly opened file starts at offset 0");
// LBA far past EOF — must not matter because count==0 returns early.
let mut buf = [0u8; 1];
let n = src.read_sectors(1_000_000, 0, &mut buf, false).unwrap();
assert_eq!(n, 0);
assert_eq!(
src.file.stream_position().expect("cursor readable"),
before,
"count == 0 must return before the seek — an unmoved cursor is the \
only observable proof that no I/O was issued"
);
// And the drop-window accounting must not have advanced either.
assert_eq!(src.bytes_read_since_drop, 0);
assert_eq!(src.drop_window_start, 0);
}
/// Reading past EOF must ERROR (read_exact's UnexpectedEof), never
@@ -518,4 +596,36 @@ mod tests {
lba += batch as u32;
}
}
/// `FREEMKV_READ_DROP_CHUNK_MIB` must be BOUNDED before the MiB→byte
/// multiply, exactly as its writeback twin bounds the identical multiply.
/// Unbounded, any value above 2^44 overflowed `n * 1024 * 1024`: a panic on
/// the first ISO open in an overflow-checked build, and in release a wrap to
/// a near-zero window that fires `drop_window` on essentially every read.
#[test]
fn read_drop_chunk_env_is_bounded_before_the_multiply() {
// Default when unset / zero / out of range.
assert_eq!(resolve_read_drop_chunk(None), READ_DROP_CHUNK_BYTES_DEFAULT);
assert_eq!(
resolve_read_drop_chunk(Some(0)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// The overflow value: `u64::MAX * 1024 * 1024` panicked here.
assert_eq!(
resolve_read_drop_chunk(Some(u64::MAX)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX + 1)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// In-range values convert MiB→bytes. Mutation: `* 1024` breaks this.
assert_eq!(resolve_read_drop_chunk(Some(1)), 1024 * 1024);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX)),
READ_DROP_CHUNK_MIB_MAX * 1024 * 1024
);
// And the bound itself keeps the multiply inside u64.
assert!((READ_DROP_CHUNK_MIB_MAX as u128) * 1024 * 1024 <= u64::MAX as u128);
}
}
+3 -3
View File
@@ -4,8 +4,8 @@
use std::fs::File;
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+3 -3
View File
@@ -9,7 +9,7 @@ use std::fs::File;
/// No-op stub. `FILE_FLAG_SEQUENTIAL_SCAN` can only be set at
/// `CreateFile` open time, which the plain `File::open` path does not
/// do, so there is no post-open hint to issue here.
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {
tracing::debug!(
target: "mux",
"FileSectorSource hint_sequential: windows no-op stub"
@@ -19,10 +19,10 @@ pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
/// Windows page-cache eviction is not exposed via a posix_fadvise
/// equivalent. The kernel does its own working-set management. No-op
/// for now.
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Windows async-prefetch hint. With FILE_FLAG_SEQUENTIAL_SCAN at
/// open the kernel already prefetches aggressively, so there's no
/// per-range hint we'd add on top. No-op stub for parity with the
/// posix platforms.
pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+283
View File
@@ -0,0 +1,283 @@
//! `write_image` — write an image-level source out as a sector image.
//!
//! This is the plain image writer: sectors in from any [`SectorSource`], bytes
//! out to a file, in order, once. It is what an `iso://` DESTINATION means when
//! the source is not a physical drive.
//!
//! # Why this is not `freemkv_engine::copy`
//!
//! The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
//! sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
//! exists because an optical drive returns read errors on marginal media. A
//! file-backed or synthesized source has no marginal media: a read either
//! succeeds or the underlying file is broken, and retrying it is pointless.
//!
//! Routing a non-drive source through the recovery path is not merely wasteful,
//! it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
//! both of which are empty for an already-decrypted source, so identity passes
//! for ANY such source: a second run with a different input to the same output
//! path would resume over the previous image and produce wrong content at exit
//! zero. Keeping the two paths separate makes that unrepresentable.
//!
//! So: drive sources get `freemkv_engine::copy`. Everything else gets this.
use crate::consts::SECTOR_BYTES;
use crate::error::{Error, Result};
use crate::halt::Halt;
use crate::sector::SectorSource;
use std::fs::File;
use std::io::{BufWriter, Write};
use std::path::Path;
/// Sectors per read/write batch. 4 MiB — large enough that per-call overhead
/// disappears against a file-backed source, small enough that the buffer is not
/// a notable allocation and cancellation stays responsive.
const BATCH_SECTORS: u32 = 2048;
/// Write `total_sectors` sectors from `reader` to `dest`.
///
/// Reads sequentially from LBA 0 and writes in order, so the output is a faithful
/// image of whatever the source presents — decrypted if the caller wrapped the
/// source in a [`DecryptingSectorSource`](crate::sector::decrypting::DecryptingSectorSource),
/// ciphertext if it did not. This function performs no decryption itself and makes
/// no decryption decision; that belongs to the caller, which knows whether the run
/// is `--raw`.
///
/// `on_progress` is called after each batch with the cumulative byte count, for
/// front-end progress reporting. It must not block.
///
/// `halt` is checked once per batch. On cancellation the partial file is left in
/// place — the caller decides whether a partial image is worth keeping, and
/// deleting a multi-gigabyte file the user may want to inspect is not this
/// function's call to make.
///
/// Returns the number of bytes written.
///
/// # Errors
///
/// - [`Error::Halted`] if `halt` was cancelled.
/// - [`Error::IoError`] if the destination cannot be created or written.
/// - Whatever the source's `read_sectors` returns. A short read is an error, not
/// a zero-fill: silently padding a truncated source produces an image that
/// looks complete and is not.
pub fn write_image(
reader: &mut dyn SectorSource,
dest: &Path,
total_sectors: u32,
halt: &Halt,
mut on_progress: impl FnMut(u64),
) -> Result<u64> {
if total_sectors == 0 {
return Err(Error::EmptyImage);
}
let file = File::create(dest).map_err(|source| Error::IoError { source })?;
let mut out = BufWriter::with_capacity(BATCH_SECTORS as usize * SECTOR_BYTES, file);
let mut buf = vec![0u8; BATCH_SECTORS as usize * SECTOR_BYTES];
let mut written: u64 = 0;
let mut lba: u32 = 0;
while lba < total_sectors {
if halt.is_cancelled() {
return Err(Error::Halted);
}
let count = BATCH_SECTORS.min(total_sectors - lba);
let want = count as usize * SECTOR_BYTES;
// `recovery = false`: a file-backed source ignores the flag, and a
// retry loop over a local file would only re-read the same bytes.
let got = reader.read_sectors(lba, count as u16, &mut buf[..want], false)?;
if got != want {
return Err(Error::ShortImageRead {
lba,
expected: want as u32,
got: got as u32,
});
}
out.write_all(&buf[..want])
.map_err(|source| Error::IoError { source })?;
written += want as u64;
lba += count;
on_progress(written);
}
// flush() only pushes the BufWriter's bytes into the kernel via write(2).
// It makes no durability promise at all, so returning Ok here would report
// a finished image while up to several gigabytes of it still sit in the
// page cache. A crash, a power loss, or yanking the removable/network
// volume the image was written to then leaves a truncated or empty file
// that the caller was told was complete.
//
// For a 6-90 GB image that is exactly the failure this crate treats as
// worst: success reported over wrong output. `into_inner` is used rather
// than `flush` so a buffered-write error is surfaced instead of being
// dropped on the floor by BufWriter's Drop.
let file = out.into_inner().map_err(|e| Error::IoError {
source: e.into_error(),
})?;
file.sync_all()
.map_err(|source| Error::IoError { source })?;
Ok(written)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::error::Result as FmResult;
/// A source that yields a deterministic byte per sector, so the written
/// image can be checked positionally rather than just by length.
struct PatternSource {
sectors: u32,
/// Sectors after which `read_sectors` reports a short read.
short_after: Option<u32>,
}
impl SectorSource for PatternSource {
fn capacity_sectors(&self) -> u32 {
self.sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> FmResult<usize> {
let want = count as usize * SECTOR_BYTES;
if self.short_after.is_some_and(|after| lba >= after) {
return Ok(want - 1);
}
for s in 0..count as usize {
let byte = ((lba as usize + s) % 251) as u8;
buf[s * SECTOR_BYTES..(s + 1) * SECTOR_BYTES].fill(byte);
}
Ok(want)
}
}
fn tmp(name: &str) -> std::path::PathBuf {
let mut p = std::env::temp_dir();
p.push(format!("fmkv-image-writer-{name}-{}", std::process::id()));
p
}
/// The written image is byte-for-byte what the source presented, at the
/// right offsets — not merely the right length.
#[test]
fn writes_every_sector_in_order() {
let dest = tmp("order");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 5000 * SECTOR_BYTES as u64);
let data = std::fs::read(&dest).expect("read back");
assert_eq!(data.len(), 5000 * SECTOR_BYTES);
// Spot-check across batch boundaries (BATCH_SECTORS = 2048): the last
// sector of batch 0, the first of batch 1, and the final sector.
for lba in [0usize, 2047, 2048, 4095, 4096, 4999] {
let want = (lba % 251) as u8;
assert_eq!(
data[lba * SECTOR_BYTES],
want,
"sector {lba} head byte wrong — batching lost or duplicated a sector"
);
assert_eq!(
data[(lba + 1) * SECTOR_BYTES - 1],
want,
"sector {lba} tail"
);
}
let _ = std::fs::remove_file(&dest);
}
/// A tail shorter than a full batch must still be written whole — the
/// classic off-by-one when `total_sectors` is not a batch multiple.
#[test]
fn writes_a_partial_final_batch() {
let dest = tmp("tail");
let mut src = PatternSource {
sectors: 2049,
short_after: None,
};
let n = write_image(&mut src, &dest, 2049, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 2049 * SECTOR_BYTES as u64);
assert_eq!(
std::fs::metadata(&dest).expect("stat").len(),
2049 * SECTOR_BYTES as u64
);
let _ = std::fs::remove_file(&dest);
}
/// A short read is an error. Zero-filling would yield an image that looks
/// complete and is not — the single worst outcome for an archival copy.
#[test]
fn short_read_is_an_error_not_a_zero_fill() {
let dest = tmp("short");
let mut src = PatternSource {
sectors: 4096,
short_after: Some(2048),
};
let err = write_image(&mut src, &dest, 4096, &Halt::new(), |_| {}).expect_err("must fail");
assert!(
matches!(err, Error::ShortImageRead { lba: 2048, .. }),
"got {err:?}"
);
let _ = std::fs::remove_file(&dest);
}
/// Cancellation stops the run and reports it, rather than finishing quietly
/// or reporting success on a partial image.
#[test]
fn cancellation_halts_and_reports() {
let dest = tmp("halt");
let mut src = PatternSource {
sectors: 100_000,
short_after: None,
};
let halt = Halt::new();
halt.cancel();
let err = write_image(&mut src, &dest, 100_000, &halt, |_| {}).expect_err("must halt");
assert!(matches!(err, Error::Halted), "got {err:?}");
let _ = std::fs::remove_file(&dest);
}
/// Progress is cumulative and monotonic, and its final value equals the
/// returned byte count — a front-end that trusts the callback must not end
/// up disagreeing with the return value.
#[test]
fn progress_is_cumulative_and_ends_at_the_total() {
let dest = tmp("progress");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let mut seen: Vec<u64> = Vec::new();
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |b| seen.push(b)).expect("write");
assert!(
seen.windows(2).all(|w| w[1] > w[0]),
"not monotonic: {seen:?}"
);
assert_eq!(*seen.last().expect("at least one callback"), n);
let _ = std::fs::remove_file(&dest);
}
/// A zero-sector source is a caller error, not a zero-byte image: an empty
/// ISO is never what anyone wanted, and failing here names the problem.
#[test]
fn zero_sectors_is_an_error() {
let dest = tmp("empty");
let mut src = PatternSource {
sectors: 0,
short_after: None,
};
let err = write_image(&mut src, &dest, 0, &Halt::new(), |_| {}).expect_err("must fail");
assert!(matches!(err, Error::EmptyImage), "got {err:?}");
// The destination must not have been created — a failed run leaves no
// stub for a later run to mistake for output.
assert!(!dest.exists(), "empty run created a file");
}
}
+3 -3
View File
@@ -31,6 +31,7 @@ pub(crate) mod bounded;
pub mod byte_prefetcher;
pub mod file_sector_source;
pub mod fsync;
pub mod image_writer;
pub mod sink;
mod writeback;
mod writeback_file;
@@ -40,9 +41,8 @@ pub(crate) mod platform_macos;
pub mod pipeline;
pub(crate) use writeback_file::WritebackFile;
pub use writeback_file::WritebackFile;
pub use pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, READ_PIPELINE_DEPTH, Sink, WRITE_PIPELINE_DEPTH,
WRITE_THROUGH_DEPTH,
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
+352 -65
View File
@@ -33,7 +33,7 @@
//! consumer lag detection). This is critical for diagnosing stalls.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU8, Ordering};
use std::thread::{self, JoinHandle};
use std::time::{Duration, Instant};
@@ -117,8 +117,32 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
Error::PipelineConsumerPanicked
}
/// Consumer lifecycle state, shared between the caller and the consumer thread.
///
/// A plain `AtomicBool` could not make "the caller abandons" and "the consumer
/// commits to finalising" mutually exclusive: the consumer loaded the flag, the
/// caller stored it, and the consumer then finalised the container anyway — the
/// caller reporting the rip as interrupted while a fully finalised MKV (Cues
/// written, Segment size patched) landed on disk, indistinguishable from a
/// complete one. The two transitions are therefore a single compare-exchange each,
/// out of [`state::RUNNING`]: whoever wins decides, and the loser observes the
/// winner. (`ST_RUNNING` does not exist anywhere in the crate — the constants are
/// `state::RUNNING` / `state::ABANDONED` / `state::CLOSING` below, and both
/// compare-exchange sites that must stay in step with this argument name them.)
mod state {
/// Consumer is running; neither side has committed yet.
pub const RUNNING: u8 = 0;
/// The caller gave up on the consumer and will report failure — the consumer
/// must NOT finalise the output.
pub const ABANDONED: u8 = 1;
/// The consumer has committed to `close()` (finalising the output). The caller
/// can no longer abandon it; it must wait for the result it is about to
/// produce.
pub const CLOSING: u8 = 2;
}
/// After a halt or deadline fires, spin-poll `handle.is_finished()` for
/// [`FINISH_GRACE_SECS`] before accepting the thread leak. This converts
/// `grace` before accepting the thread leak. This converts
/// the common "nearly-done" consumer (whose own bounded_syscall just
/// returned and is about to drop its output file) into a clean join,
/// releasing the file handle without waiting the full grace period.
@@ -132,11 +156,12 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
/// syscall itself; that still returns on its own (or at process exit).
fn finish_with_grace<R: Send + 'static>(
handle: thread::JoinHandle<Result<R, Error>>,
abandoned: &Arc<AtomicBool>,
state: &Arc<AtomicU8>,
grace: Duration,
leak_err: Error,
) -> Result<R, Error> {
let grace = Instant::now() + Duration::from_secs(FINISH_GRACE_SECS);
while Instant::now() < grace {
let deadline = Instant::now() + grace;
while Instant::now() < deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
@@ -145,16 +170,50 @@ fn finish_with_grace<R: Send + 'static>(
}
thread::sleep(POLL_INTERVAL);
}
// Grace expired. Signal abandonment, then log and leak. Setting the
// flag BEFORE dropping the handle guarantees the leaked consumer
// observes it the moment its wedged syscall returns: it then skips
// any further `apply` and skips `close()`, rather than running on to
// finalise the abandoned output file.
// `Release` here pairs with the `Acquire` loads in the consumer loop so
// the leaked consumer reliably observes the flag the moment its wedged
// syscall returns, even on weak memory models (ARM64/POWER) where
// `Relaxed` gives no cross-thread visibility guarantee.
abandoned.store(true, Ordering::Release);
// Grace expired. CLAIM abandonment, then log and leak. Claiming BEFORE
// dropping the handle guarantees the leaked consumer observes it the moment
// its wedged syscall returns: it then skips any further `apply` and skips
// `close()`, rather than running on to finalise the abandoned output file.
//
// A compare-exchange, not a store, because the consumer may have committed to
// `close()` in the instant between our last `is_finished()` poll and now. It
// then cannot be stopped — the finalise IS happening — so abandoning it would
// report the rip as interrupted while a valid, fully finalised container
// lands on disk. Losing the race means waiting for the result the consumer is
// already producing instead. `AcqRel` pairs with the consumer's own
// compare-exchange and with the `Acquire` loads in its drain loop, so the flag
// is reliably observed even on weak memory models (ARM64/POWER).
if state
.compare_exchange(
state::RUNNING,
state::ABANDONED,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_close_in_flight",
"pipeline consumer had already committed to finalising the output; \
waiting for it rather than reporting an unfinalised output"
);
let close_deadline = Instant::now() + grace;
while Instant::now() < close_deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
Err(payload) => Err(consumer_panicked(payload)),
};
}
thread::sleep(POLL_INTERVAL);
}
// Still finalising after a second grace window: leak and report the wedge.
// The output may end up finalised by the leaked thread — but that is now a
// wedged-`close()` case, not the check-then-finalise race.
drop(handle);
return Err(leak_err);
}
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_grace_expired",
@@ -173,14 +232,9 @@ fn finish_with_grace<R: Send + 'static>(
/// Default channel depth for callers without a specific reason to
/// pick another value. Kept conservative (4) — most callers should
/// use READ_PIPELINE_DEPTH or WRITE_PIPELINE_DEPTH instead.
/// use WRITE_PIPELINE_DEPTH instead.
pub const DEFAULT_PIPELINE_DEPTH: usize = 4;
/// Read pipeline depth. Larger buffer compensates for drive variability
/// and NFS sync_file_range stalls; keeps ISO reader thread fed even when
/// consumer blocks on write.
pub const READ_PIPELINE_DEPTH: usize = 32;
/// Write pipeline depth. Smaller buffer reduces backpressure risk when
/// sync_file_range blocks; prevents producer from accumulating too much
/// work while consumer waits for NFS to drain.
@@ -189,7 +243,8 @@ pub const WRITE_PIPELINE_DEPTH: usize = 16;
/// Channel depth for write-through pipelines. Each `send` fully
/// drains before the next can enqueue. Use this when the producer
/// must observe consumer side-effects (e.g. mapfile state) before
/// emitting the next item. Currently used by `disc::patch`.
/// emitting the next item. Used by `freemkv_engine::recovery::patch` — the
/// recovery strategy moved to that crate in 1.6.0, so there is no `patch` here.
pub const WRITE_THROUGH_DEPTH: usize = 1;
/// Outcome of [`Sink::apply`]: either keep feeding items
@@ -247,7 +302,21 @@ pub struct Pipeline<I: Send + 'static, R: Send + 'static> {
/// a syscall the consumer is currently wedged in, but it does bound
/// the damage to "whatever write is already in flight" once that
/// syscall returns, instead of running on to a clean finalise.
abandoned: Arc<AtomicBool>,
///
/// One of [`state::RUNNING`] / [`state::ABANDONED`] / [`state::CLOSING`];
/// both transitions are compare-exchanges so abandoning and finalising are
/// mutually exclusive rather than racing.
state: Arc<AtomicU8>,
/// Set by the consumer the moment an `apply` returns `Err`. The consumer keeps
/// draining the channel after that (so the producer never blocks on a dead
/// receiver) — which means a producer watching only `send`'s return value
/// cannot tell the difference between "being consumed" and "being discarded
/// after a fatal write error", and would go on reading the whole remaining
/// disc before `finish()` finally surfaced the error. This flag is that
/// missing edge: [`Pipeline::send_with_halt`] fails fast on it, and
/// [`Pipeline::consumer_failed`] exposes it to producers that use plain
/// [`Pipeline::send`].
failed: Arc<AtomicBool>,
}
impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
@@ -256,17 +325,21 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
///
/// The thread is named `freemkv-pipeline-consumer` so it shows up
/// distinctly in stack traces and `top -H`. Callers that want a
/// more specific name (e.g. `freemkv-sweep-consumer`) should use
/// [`Pipeline::spawn_named`] instead. Returns an `Error::IoError`
/// if the OS refuses the thread spawn (resource exhaustion);
/// callers already operate in fallible context, so this is
/// propagated rather than panicked.
/// more specific name should use [`Pipeline::spawn_named`] instead.
/// Returns an `Error::IoError` if the OS refuses the thread spawn
/// (resource exhaustion); callers already operate in fallible context, so
/// this is propagated rather than panicked.
///
/// Sweep uses [`Pipeline::spawn_named`] directly so the consumer
/// thread shows up as `freemkv-sweep-consumer`; mux uses
/// `freemkv-mux-consumer`. `Pipeline::spawn` (this function, with
/// the default name) is used by `disc::patch` and by the unit
/// tests in this module.
/// Inside this crate the only [`Pipeline::spawn_named`] caller is the mux
/// driver, which names its thread `freemkv-mux-consumer`. `Pipeline::spawn`
/// (this function, with the default name) is used only by the unit tests in
/// this module.
///
/// This paragraph twice named a caller that had left the crate: first
/// `disc::patch`, then Sweep and its `freemkv-sweep-consumer` thread. Both
/// went to freemkv-engine with the recovery passes in 1.6.0, and each in
/// turn sent readers hunting a component that is not here. Name callers
/// that live in THIS crate, or none.
pub fn spawn<S: Sink<I, Output = R>>(depth: usize, sink: S) -> Result<Self, Error> {
Self::spawn_named("freemkv-pipeline-consumer", depth, sink)
}
@@ -274,15 +347,17 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Like [`Pipeline::spawn`] but lets the caller supply the
/// consumer thread's name. Useful when several pipelines run in
/// the same process and stack traces / `top -H` need to tell them
/// apart (e.g. `freemkv-sweep-consumer`, `freemkv-mux-consumer`).
/// apart (e.g. `freemkv-mux-consumer`).
pub fn spawn_named<S: Sink<I, Output = R>>(
name: &str,
depth: usize,
sink: S,
) -> Result<Self, Error> {
let (tx, rx) = bounded::<I>(depth);
let abandoned = Arc::new(AtomicBool::new(false));
let abandoned_consumer = abandoned.clone();
let state = Arc::new(AtomicU8::new(state::RUNNING));
let state_consumer = state.clone();
let failed = Arc::new(AtomicBool::new(false));
let failed_consumer = failed.clone();
let handle = thread::Builder::new()
.name(name.into())
.spawn(move || -> Result<R, Error> {
@@ -313,7 +388,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// dead receiver, but we touch the output no further. The
// final post-loop abandonment check returns the error
// and skips `close()`.
if abandoned_consumer.load(Ordering::Acquire) {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
continue;
}
@@ -342,6 +417,12 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
}
first_err = Some(e);
// Publish the failure so the producer can stop
// FEEDING a dead write side instead of only learning
// about it at `finish()` — by which time it has read
// the rest of the disc. `Release` pairs with the
// `Acquire` load in `send_with_halt`.
failed_consumer.store(true, Ordering::Release);
}
}
@@ -402,13 +483,38 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// MKV Cues + patching the segment header) on a file the
// caller already reported as failed is exactly the
// write race we must not run.
if abandoned_consumer.load(Ordering::Acquire) {
return Err(Error::Halted);
}
match first_err {
Some(e) => Err(e),
None => sink.close(),
// No `close()` on this path, so there is nothing to claim —
// just report, unless the caller has already given up on us.
Some(e) => {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
Err(Error::Halted)
} else {
Err(e)
}
}
// CLAIM the finalise. A plain load here left a window in which
// the caller stored `abandoned` AFTER we read it as clear, so
// `close()` ran anyway and finalised (Cues + Segment-size
// patch) an output the caller had already reported as
// interrupted — a truncated rip indistinguishable from a
// complete one. The compare-exchange closes that window: if the
// caller got there first we skip `close()`, and if we get there
// first the caller waits for us instead of abandoning.
None => {
if state_consumer
.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
return Err(Error::Halted);
}
sink.close()
}
}
})
.map_err(|e| Error::IoError { source: e })?;
@@ -416,10 +522,24 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Ok(Pipeline {
tx,
handle,
abandoned,
state,
failed,
})
}
/// Whether the consumer's `apply` has already failed fatally.
///
/// The consumer keeps draining the channel after an `apply` error (so the
/// producer never blocks on a dead receiver), which means `send` keeps
/// succeeding and a producer has no other way to tell that everything it feeds
/// is being discarded. A long-running producer — the mux frame pump reading a
/// 60 GB title off an optical drive — should check this and unwind instead of
/// reading the rest of the disc for a write that has already failed.
/// [`Pipeline::send_with_halt`] checks it automatically.
pub fn consumer_failed(&self) -> bool {
self.failed.load(Ordering::Acquire)
}
/// Push one item. Blocks if the channel is full — that's the
/// back-pressure the whole primitive exists to provide. Returns
/// the item back if the consumer thread is gone (panicked or
@@ -512,11 +632,34 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// wedged inside an unkillable syscall, the producer can still
/// observe `/api/stop` and unwind within
/// [`SEND_HALT_CHECK_INTERVAL`].
/// NOT a `foo_with_X` variant of [`Pipeline::send`], despite the name.
/// The two encode OPPOSITE policies on the same event, each with its own
/// test: after the consumer's `apply` has failed, `send` still succeeds
/// (the consumer keeps draining, so the channel accepts the item), while
/// this one hands the item straight back — so a producer does not read an
/// hour of disc for a write that died on the first frame. Collapsing them
/// into one Option-parameterised method deletes one of those behaviours;
/// it was tried and `apply_error_drains_then_propagates` caught it.
pub fn send_with_halt(&self, item: I, halt: &Halt, deadline: Duration) -> Result<(), I> {
use crossbeam_channel::SendTimeoutError;
let end = Instant::now() + deadline;
let mut pending = item;
loop {
// The consumer's `apply` has failed fatally: everything sent from here
// is drained and discarded, so hand the item back at once. Without this
// the producer saw every send succeed (the channel is always being
// drained) and went on reading the whole remaining title — an hour of
// drive time on a UHD — for a write that died on the first frame, only
// learning about it at `finish()`.
if self.consumer_failed() {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: consumer apply failed, returning item={}",
std::any::type_name::<I>()
);
}
return Err(pending);
}
// Pre-check the cheap exit conditions before parking.
if halt.is_cancelled() {
if debug_enabled() {
@@ -571,7 +714,8 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let Pipeline {
tx,
handle,
abandoned: _,
state: _,
failed: _,
} = self;
// Explicit drop, although the destructure already drops `tx`
// at end-of-scope. Being explicit keeps the intent obvious.
@@ -607,11 +751,18 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Plain [`Pipeline::finish`] is preserved for callers without a
/// halt-token plumbed through; that path still blocks indefinitely
/// on `join()`, matching pre-0.20.8 behaviour.
/// Also not a `foo_with_X` variant: [`Pipeline::finish`] joins and waits
/// however long the consumer needs, while this one gives up after
/// `JOIN_TIMEOUT_SECS` and reports halted. Which is right depends on
/// whether the caller has a user waiting to cancel — the mux driver does
/// and uses this; the unit tests do not and use the plain join. Merging
/// them means picking one of those policies for both.
pub fn finish_with_halt(self, halt: Option<&Halt>) -> Result<R, Error> {
let Pipeline {
tx,
handle,
abandoned,
state,
failed: _,
} = self;
drop(tx);
let deadline = Instant::now() + Duration::from_secs(JOIN_TIMEOUT_SECS);
@@ -622,13 +773,23 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Err(payload) => Err(consumer_panicked(payload)),
};
}
if let Some(h) = halt {
if h.is_cancelled() {
return finish_with_grace(handle, &abandoned, Error::Halted);
}
if let Some(h) = halt
&& h.is_cancelled()
{
return finish_with_grace(
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::Halted,
);
}
if Instant::now() >= deadline {
return finish_with_grace(handle, &abandoned, Error::PipelineJoinTimeout);
return finish_with_grace(
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::PipelineJoinTimeout,
);
}
thread::sleep(POLL_INTERVAL);
}
@@ -1080,20 +1241,6 @@ mod tests {
/// A sink that records the exact order of items it receives, so we
/// can prove the channel is FIFO (no reordering). `close` returns
/// the recorded vector.
struct OrderSink {
seen: Vec<u64>,
}
impl Sink<u64> for OrderSink {
type Output = Vec<u64>;
fn apply(&mut self, item: u64) -> Result<Flow, Error> {
self.seen.push(item);
Ok(Flow::Continue)
}
fn close(self) -> Result<Vec<u64>, Error> {
Ok(self.seen)
}
}
/// Zero items sent: closing the pipeline immediately must still
/// call `close()` exactly once and return its Output. The consumer
/// loop's `while let Ok = rx.recv()` exits on the dropped tx with
@@ -1598,4 +1745,144 @@ mod tests {
let res = pipe.finish_with_halt(None);
assert!(matches!(res, Ok(190)), "expected Ok(190), got {res:?}");
}
/// A fatal `apply` error must become visible to the PRODUCER, not only to
/// `finish()`. The consumer keeps draining after the error (so the producer
/// never blocks on a dead receiver), which meant every `send_with_halt`
/// returned `Ok` for the rest of the run: on a 60 GB mkv:// mux that hit
/// ENOSPC on the first frame, the mux driver read the entire remaining title —
/// an hour of optical-drive time — before learning the write had died.
#[test]
fn send_with_halt_fails_fast_once_apply_has_failed() {
struct FailFirst {
failed: Arc<AtomicUsize>,
}
impl Sink<u64> for FailFirst {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
self.failed.fetch_add(1, Ordering::SeqCst);
Err(Error::DecryptFailed)
}
fn close(self) -> Result<(), Error> {
Ok(())
}
}
let applied = Arc::new(AtomicUsize::new(0));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
FailFirst {
failed: applied.clone(),
},
)
.expect("spawn");
let halt = crate::halt::Halt::new();
let deadline = Duration::from_secs(5);
// Feed one item and wait until the consumer has actually applied (and
// failed on) it, so the check below is deterministic rather than racy.
pipe.send_with_halt(0u64, &halt, deadline)
.expect("the first send lands");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && applied.load(Ordering::SeqCst) == 0 {
std::thread::sleep(Duration::from_millis(5));
}
assert_eq!(applied.load(Ordering::SeqCst), 1, "apply ran and failed");
assert!(pipe.consumer_failed(), "the failure must be observable");
// The very next send must hand the item straight back — the producer's
// signal to stop reading the disc.
assert_eq!(
pipe.send_with_halt(1u64, &halt, deadline),
Err(1u64),
"send_with_halt must fail fast once the consumer's apply has failed"
);
// The halt was never fired, so this is not a cancellation: the real error
// still comes out of finish().
assert!(matches!(pipe.finish(), Err(Error::DecryptFailed)));
assert_eq!(
applied.load(Ordering::SeqCst),
1,
"no further item was applied"
);
}
/// The abandon/finalise race. A consumer that has ALREADY committed to
/// `close()` when the grace period expires cannot be stopped — the finalise is
/// happening — so the caller must wait for its result instead of reporting the
/// output as un-finalised. With a plain flag the consumer read it as clear, the
/// caller then stored it, and the caller returned `Err(Halted)`
/// (`completed = false`) while a fully finalised MKV (Cues written, Segment
/// size patched) landed on disk — a truncated rip indistinguishable from a
/// complete one.
#[test]
fn abandon_loses_to_a_close_already_committed() {
let state = Arc::new(AtomicU8::new(state::RUNNING));
let release = Arc::new(AtomicBool::new(false));
let in_close = Arc::new(AtomicBool::new(false));
let (st, rel, inc) = (state.clone(), release.clone(), in_close.clone());
let handle = thread::Builder::new()
.name("test-consumer".into())
.spawn(move || -> Result<u64, Error> {
// Exactly what the consumer does before finalising: claim the
// right to close.
assert!(
st.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire
)
.is_ok(),
"the consumer claims the finalise first"
);
inc.store(true, Ordering::SeqCst);
// Inside `close()`, finalising the container.
while !rel.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
Ok(42)
})
.expect("spawn");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && !in_close.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
assert!(in_close.load(Ordering::SeqCst), "consumer reached close()");
// Finish the close only AFTER the first grace window has expired, so the
// caller genuinely reaches the abandon decision with a close in flight.
let rel = release.clone();
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
//
// These intervals used to be 600 ms against a 300 ms grace, which
// left NO margin: two 300 ms windows end at 600 ms, and the 250 ms
// poll cadence can push the observation later still, so on a loaded
// runner the second window expired first and the caller abandoned —
// failing with Err(Halted) against a race, not a defect.
//
// Scaled up so the jitter is small relative to the intervals: the
// first window ends at ~1.0-1.25 s and the second at ~2.0-2.25 s,
// so releasing at 1.6 s sits well inside the second with roughly
// 350 ms of slack on either side. The ordering under test is
// unchanged; only the margin is.
thread::sleep(Duration::from_millis(1600));
rel.store(true, Ordering::SeqCst);
});
let grace = Duration::from_secs(1);
let res = finish_with_grace(handle, &state, grace, Error::Halted);
assert!(
matches!(res, Ok(42)),
"a finalise already in flight must be waited for, not abandoned: {res:?}"
);
assert_eq!(
state.load(Ordering::SeqCst),
state::CLOSING,
"the caller must not have overwritten the consumer's claim"
);
}
}
+1 -1
View File
@@ -272,7 +272,7 @@ impl WritebackPipeline {
self.chunk_bytes,
self.skip_wait(),
);
if self.chunk_count % SIZE_LOG_INTERVAL == 0 {
if self.chunk_count.is_multiple_of(SIZE_LOG_INTERVAL) {
tracing::debug!(
target: "mux",
"WritebackPipeline chunk_bytes={} after {} chunks is_nfs={} degraded={}",
+84 -29
View File
@@ -30,14 +30,15 @@ pub(super) fn preallocate(file: &File, size_bytes: u64) {
);
}
/// Run `fsync` on `file` with a 60 s deadline. On timeout — and
/// likewise on halt or a lost worker we log and return `Ok(())`: the
/// kernel will still flush on close, so the data is best-effort durable.
/// The alternative (trap the thread for the rest of the rip, or return
/// an error that aborts an otherwise-complete mux) is worse, so all
/// three fallbacks return `Ok(())`. `Ok(())` from these paths is NOT a
/// durability barrier — the durable flush did not complete; only the
/// hang is bounded.
/// Run `fsync` on `file` with a 60 s deadline. On timeout, halt or a lost
/// worker we log and return `Err` — matching macOS. POSIX gives `fsync`
/// exactly one way to say "the data is on stable storage" and that is a zero
/// return; a call that never reached the device has not earned it, so `Ok(())`
/// from here means the flush completed and nothing else.
///
/// The kernel will still flush on close, so the data is usually durable
/// anyway — but that is a probability, not a barrier, and a caller that needs
/// crash-consistency has to be able to tell the difference.
///
/// ## fd-reuse safety
///
@@ -78,27 +79,7 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; kernel will flush on close (best-effort)"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::WorkerLost) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(e) => bounded_failure_to_result(e),
}
}
@@ -143,3 +124,77 @@ mod tests {
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `fsync` onto the
/// `io::Error` `durable_sync` returns.
///
/// Every arm means the same thing: **no sync observably ran**. All three used to
/// return `Ok(())`, so `WritebackFile::sync_all` reported success for a
/// durability barrier that never happened. POSIX gives `fsync` one way to say
/// "the data is on stable storage" — a zero return — and a call that never
/// reached the device has not earned it.
///
/// This mirrors the macOS `F_FULLFSYNC` mapping exactly. The two were found
/// carrying the identical defect, and a platform disagreeing with its sibling
/// about whether a failed sync is an error is the "works on my platform" class
/// this crate has been bitten by before — most recently an over-length SCSI CDB
/// that macOS rejected and the other two silently truncated.
///
/// No message text (this crate ships no user-facing English): the kind, and
/// `EIO` for the worker-lost case, are the signal; `tracing` carries the detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
// EIO, matching the macOS sibling: a consumer distinguishing these
// three failures does so on the same value on every platform.
// ErrorKind::Other carries nothing a caller can branch on.
Err(crate::error::Error::SyncWorkerLost.into())
}
}
}
#[cfg(test)]
mod bounded_failure_tests {
use super::*;
use crate::io::bounded::BoundedError;
/// Every bounded-fsync failure must be an error. Asserted per variant rather
/// than as a loop so a new variant defaulting to Ok cannot slip through.
#[test]
fn no_bounded_fsync_failure_maps_to_ok() {
assert_eq!(
bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out fsync must be an error")
.kind(),
io::ErrorKind::TimedOut
);
assert_eq!(
bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted fsync must be an error")
.kind(),
io::ErrorKind::Interrupted
);
assert!(
bounded_failure_to_result(BoundedError::WorkerLost).is_err(),
"a lost fsync worker must be an error"
);
}
}
+107 -5
View File
@@ -105,15 +105,46 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
Err(e) => bounded_failure_to_result(e),
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `F_FULLFSYNC`
/// onto the `io::Error` `durable_sync` returns.
///
/// Every arm here means the same thing: **no sync observably ran**. All three
/// previously returned `Ok(())`, so `WritebackFile::sync_all` reported success
/// for a durability barrier that never happened — a total failure exiting 0,
/// with only a log line to distinguish it. POSIX gives `fsync` exactly one way
/// to say "the data is on stable storage" and that is a zero return; a call
/// that never reached the device has not earned it.
///
/// The errors carry no message text (this crate ships no user-facing English):
/// the kind, and `EIO` for the worker-lost case, are the whole signal, and the
/// `tracing` lines above/below carry the operator detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; kernel will flush on close (best-effort)"
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Ok(())
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncWorkerLost.into())
}
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
}
}
@@ -156,4 +187,75 @@ mod tests {
// durable_sync must complete without error on the local tempfile.
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
}
/// Every `BoundedError` arm of the bounded `F_FULLFSYNC` means no sync
/// observably ran. All three returned `Ok(())`, so `sync_all` reported a
/// durability barrier that never happened — the caller could not tell a
/// completed flush from a skipped one by any means except reading a log.
///
/// Asserted on the concrete `ErrorKind` / `errno` each arm must produce,
/// so a future arm that quietly reverts to `Ok(())` fails here.
#[test]
fn every_bounded_failure_is_reported_as_an_error() {
use crate::io::bounded::BoundedError;
let timeout = bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out F_FULLFSYNC must be an error");
assert_eq!(
timeout.kind(),
io::ErrorKind::TimedOut,
"a timed-out F_FULLFSYNC must not be reported as a completed sync"
);
let halted = bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted F_FULLFSYNC must be an error");
assert_eq!(
halted.kind(),
io::ErrorKind::Interrupted,
"a halted F_FULLFSYNC must not be reported as a completed sync"
);
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each
// carries its own numeric code through the "E<code>" prefix that
// `From<Error> for io::Error` mints — the only shape `error_code`
// recognises. A bare `ErrorKind` cannot be classified, which is how a
// user cancel here used to read as a hard I/O failure.
let lost = bounded_failure_to_result(BoundedError::WorkerLost)
.expect_err("a lost F_FULLFSYNC worker must be an error");
assert!(
lost.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_WORKER_LOST)),
"a lost worker must be identifiable, got {lost}"
);
assert!(
timeout
.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_TIMEOUT)),
"a timeout must be distinguishable from a lost worker, got {timeout}"
);
assert!(
crate::error::is_halt(&halted),
"a halt must satisfy the crate's own is_halt(), or the CLI reports a \
user cancel as a failure; got {halted}"
);
}
/// The failure path must be reachable through the public surface: a
/// `WritebackFile::sync_all` that hits any of these arms must surface an
/// `Err`, not a silent `Ok`. Pinned at the mapping boundary because the
/// timeout itself is not deterministically inducible in a unit test.
#[test]
fn bounded_failures_are_never_mapped_to_ok() {
use crate::io::bounded::BoundedError;
for e in [
BoundedError::Timeout,
BoundedError::Halted,
BoundedError::WorkerLost,
] {
assert!(
bounded_failure_to_result(e).is_err(),
"a bounded F_FULLFSYNC failure must never map to Ok"
);
}
}
}
+21 -15
View File
@@ -97,7 +97,7 @@ fn writeback_chunk_bytes() -> u64 {
.unwrap_or(WRITEBACK_CHUNK_BYTES_DEFAULT)
}
pub(crate) struct WritebackFile {
pub struct WritebackFile {
file: File,
pipeline: WritebackPipeline,
pos: u64,
@@ -115,7 +115,7 @@ impl WritebackFile {
/// once so the pipeline starts tracking from wherever the file
/// already is (typically 0 for fresh files; non-zero for resumed
/// or appended files).
pub(crate) fn new(mut file: File) -> io::Result<Self> {
pub fn new(mut file: File) -> io::Result<Self> {
let pos = file.stream_position()?;
let pipeline = WritebackPipeline::new(&file, pos, writeback_chunk_bytes());
Ok(Self {
@@ -136,7 +136,7 @@ impl WritebackFile {
/// [`Self::create_with_size_hint`] so the kernel can pre-reserve
/// extents.
#[allow(dead_code)]
pub(crate) fn create(path: &Path) -> io::Result<Self> {
pub fn create(path: &Path) -> io::Result<Self> {
let file = File::create(path)?;
Self::new(file)
}
@@ -153,7 +153,7 @@ impl WritebackFile {
/// On platforms without an extent-preallocation primitive this is
/// equivalent to `create` — the size hint is dropped after a debug
/// log.
pub(crate) fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
pub fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
let file = File::create(path)?;
platform::preallocate(&file, size_bytes);
Self::new(file)
@@ -163,7 +163,7 @@ impl WritebackFile {
/// wrap it. Mirrors `File::open` semantics for the writable case
/// — used by patch / resume paths that mutate an existing ISO in
/// place.
pub(crate) fn open(path: &Path) -> io::Result<Self> {
pub fn open(path: &Path) -> io::Result<Self> {
let file = OpenOptions::new().write(true).open(path)?;
Self::new(file)
}
@@ -178,13 +178,20 @@ impl WritebackFile {
/// is left to the kernel's normal flush-on-close path — best
/// effort, but bounded.
///
/// IMPORTANT: on Linux/macOS a successful `Ok(())` does NOT
/// guarantee the data is durable if the bounded fsync timed out or
/// was halted — only the hang is bounded, the fsync may not have
/// completed. Callers needing crash-consistency (e.g. mux-finish
/// then external commit/DB update) must not treat `Ok(())` as a
/// durability barrier.
pub(crate) fn sync_all(&mut self) -> io::Result<()> {
/// A bounded-fsync failure is returned as an `Err` on BOTH platforms, so
/// `Ok(())` means the flush completed and a caller needing
/// crash-consistency can treat it as a durability barrier.
///
/// The three causes are DISTINGUISHABLE by numeric code, because a caller
/// should not retry a lost worker the way it retries a timeout, and must
/// not report a user cancel as a failure:
///
/// * [`E_SYNC_TIMEOUT`](crate::error::E_SYNC_TIMEOUT) — deadline expired
/// * [`E_HALTED`](crate::error::E_HALTED) — cancelled;
/// [`is_halt`](crate::error::is_halt) recognises it
/// * [`E_SYNC_WORKER_LOST`](crate::error::E_SYNC_WORKER_LOST) — the worker
/// thread died before reporting
pub fn sync_all(&mut self) -> io::Result<()> {
if self.seek_count > 0 {
tracing::debug!(
target: "mux",
@@ -256,9 +263,8 @@ impl super::sink::SequentialSink for WritebackFile {
/// the same work [`Self::sync_all`] does. Implemented explicitly (no
/// blanket impl) so a `dyn SequentialSink` / `dyn RandomAccessSink`
/// `finish()` actually finalises + fsyncs instead of hitting a no-op
/// default. Note the bounded-fsync caveat from [`Self::sync_all`]
/// applies: `Ok(())` is not a durability barrier if the fsync timed
/// out or was halted.
/// default. A bounded-fsync failure surfaces as an `Err` here, on every
/// platform, exactly as it does from [`Self::sync_all`].
fn finish(&mut self) -> io::Result<()> {
self.sync_all()
}
+345 -31
View File
@@ -53,9 +53,30 @@ pub const MIN_SAMPLE_UNITS: usize = 8;
/// units it yields); the *requested* count is a caller-side compile-time constant that
/// callers pin to `MIN_SAMPLE_UNITS` (see e.g. autorip's `SAMPLE_UNITS`). Together the
/// two make under-sampling unrepresentable at the request boundary.
#[derive(Debug, Clone)]
///
/// The wrapped samples are on-disc AACS ciphertext — the same bytes the sibling
/// [`DiscInputs::samples`] redacts as key MATERIAL — so [`Debug`] is hand-written
/// and redacting; see the impl below.
#[derive(Clone)]
pub struct DecodeSampleSet(Vec<Vec<u8>>);
impl std::fmt::Debug for DecodeSampleSet {
/// Prints the SHAPE only. A derived `Debug` dumped every wrapped sample
/// verbatim: a `DecodeSampleSet` carries at least [`MIN_SAMPLE_UNITS`]
/// 6144-byte aligned units (≥ 49 KiB, in practice multi-MB) of AACS
/// ciphertext plus each unit's clear 16-byte derivation seed, so one
/// `tracing::debug!("{set:?}")` on a failed `/decode` request — or an
/// `assert_eq!` whose panic message formats it — wrote all of it to the log
/// that gets attached to a bug report. Same policy and same shape as
/// [`DiscInputs`]'s impl below.
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DecodeSampleSet")
.field("units", &"<redacted>")
.field("units_len", &self.0.len())
.finish()
}
}
impl DecodeSampleSet {
/// Wrap `units` iff it carries at least [`MIN_SAMPLE_UNITS`] samples; `None`
/// otherwise (the caller then skips the online source rather than sending an
@@ -82,9 +103,12 @@ impl DecodeSampleSet {
}
/// The public AACS inputs a key source needs to look a disc up. Captured at
/// scan; contains no secrets — only the disc identity and the on-disc AACS
/// structures a source or key server may key on.
#[derive(Debug, Clone)]
/// scan; carries no DERIVED secrets (no media key, VUK or plaintext unit key) —
/// only the disc identity and the on-disc AACS structures a source or key server
/// may key on. The on-disc structures are nonetheless key MATERIAL (the encrypted
/// title keys live in `unit_key_ro`), so [`Debug`] is hand-written and redacting;
/// see the impl below.
#[derive(Clone)]
pub struct DiscInputs {
/// SHA-1 of `Unit_Key_RO.inf`, `0x`-prefixed hex. The value a keydb keys
/// its per-disc entries by, and a key server identifies the disc with.
@@ -115,6 +139,30 @@ pub struct DiscInputs {
pub volume_label: Option<String>,
}
/// Redacting `Debug`, per the policy `aacs::types` documents (and which
/// `aacs::types::Vid` already applies to this very Volume ID). `DiscInputs` is
/// public and returned by [`crate::Disc::inputs`], so a consumer's
/// `tracing::debug!("{inputs:?}")` used to print the Volume ID, the whole
/// `Unit_Key_RO.inf` (the encrypted title keys), the entire MKB and every
/// ciphertext sample verbatim into a log that ends up attached to a bug report.
/// Only non-secret identity and shape (presence, lengths) is printed.
impl std::fmt::Debug for DiscInputs {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscInputs")
.field("disc_hash", &self.disc_hash)
.field("volume_id", &"<redacted>")
.field("version", &self.version)
.field("mkb", &"<redacted>")
.field("mkb_len", &self.mkb.len())
.field("unit_key_ro", &"<redacted>")
.field("unit_key_ro_len", &self.unit_key_ro.len())
.field("samples", &"<redacted>")
.field("samples_len", &self.samples.len())
.field("volume_label", &self.volume_label)
.finish()
}
}
/// A lazy view of a disc's AACS material, handed to [`KeySource::get_unit_keys`] so a
/// source can drive the derivation chain without holding the disc reader.
///
@@ -307,8 +355,23 @@ pub fn resolve_and_apply(
/// from [`crate::aacs::derive::decrypt_unit_key`]; the library's canonical CPS-unit number is
/// `position + 1` (matching [`crate::aacs::inf::parse_unit_key_ro`]'s `(i + 1)`), so
/// the committed `AacsState.unit_keys` is byte-identical to the library-resolved
/// path. The number is cosmetic for descramble (the decrypt path strips it and
/// tries every key) but is kept faithful to the resolver's convention.
/// path.
///
/// The NUMBER itself is not what descramble indexes by — but the ORDER is
/// load-bearing, so a source must return its keys in CPS-unit order. Trial
/// decrypt-and-check was deliberately deleted (see
/// [`crate::decrypt::AacsKeyMap`]: decryption is driven by the disc's CPS-unit /
/// FMTS-segment structure, "never by trial-decrypt-and-check per unit"), and
/// `decrypt_sectors_mapped` indexes the committed pool POSITIONALLY —
/// `unit_keys[key_idx].1`, where `key_idx` is a POSITION in the Vec a source
/// returned, recorded by `resolve_mux_key_map_cached` / `resolve_fmts_key_map`.
/// Return the same keys in a different order and every `AacsKeyMap` points at the
/// wrong key: the whole title decrypts under a neighbour's key, or a forensic
/// range trips the `is_clean` net into `DecryptFailed`. (The doc used to say the
/// number "is cosmetic for descramble (the decrypt path strips it and tries every
/// key)", which is what the DELETED trial-decrypt path did; the only place that
/// still tries every key is `Disc::decrypt_with`'s sample VALIDATION, which does
/// not descramble content.)
pub fn resolve_and_apply_traced(
sources: &[Box<dyn KeySource>],
inputs: &DiscInputs,
@@ -318,6 +381,14 @@ pub fn resolve_and_apply_traced(
let mut trace = crate::aacs::trace::ResolutionTrace::new();
// The FIRST source failure seen, if any. A source that returns `Err` did not
// answer "no key for this disc" — it could not answer at all — and that
// reason is stamped onto `disc.aacs_error` below so the decrypt gate reports
// THAT instead of the generic `NoDiscKey`. First-wins (not last) so the
// ordered sources' most-preferred failure is the one the operator is told
// about, matching the first-valid-wins rule for successes.
let mut source_failure: Option<crate::error::Error> = None;
// The ctx parses Unit_Key_RO.inf at the stride for `inputs.version` (the
// disc's own AACS major), so the stride is the disc's single source of truth.
let ctx = DiscInputsCtx::new(inputs);
@@ -350,17 +421,56 @@ pub fn resolve_and_apply_traced(
outcome: KeyOutcome::NoKey,
});
}
// Empty (no key here) or a source failure — both are "no key from
// this source"; move on to the next.
Ok(_) | Err(_) => {
// The source ANSWERED and holds nothing for this disc. This — and
// only this — is `NoEntry`: the claim "I looked, it is not there".
Ok(_) => {
trace.keys.push(KeyStep {
who,
path: vec![KeyNode::NoEntry],
outcome: KeyOutcome::NoKey,
});
}
// The source could NOT answer — it was unreachable, it errored, or it
// refused. Nothing is known about whether a key exists, so the path
// is EMPTY: recording `NoEntry` here is exactly the conflation that
// made a seven-hour run of HTTP 502s render as
// `key: online > no entry > NO KEY` + `E7022 No key source has a
// decryption key for this disc`, and sent operators hunting for a VUK
// that was never missing.
//
// The reason itself rides out on `disc.aacs_error` (below), the
// channel `Disc::ensure_decryptable_keys` already reads for the
// E7017-vs-E7022 split — so the decrypt gate raises the SOURCE's code
// (`KeyServiceUnavailable` / `KeyServiceUnauthorized` /
// `KeyServiceRateLimited`) instead of the generic `NoDiscKey`.
//
// `KeyOutcome` deliberately gains no variant: it is matched
// exhaustively by every front-end's trace renderer (freemkv's
// `pipe::render_resolution_trace`, autorip's
// `keysource::render_resolution_trace`), and this fix must not turn
// into a breaking change across four repos to say something the error
// code already says precisely.
Err(e) => {
if source_failure.is_none() {
source_failure = Some(e);
}
trace.keys.push(KeyStep {
who,
path: Vec::new(),
outcome: KeyOutcome::NoKey,
});
}
}
}
// Nothing resolved. If a source FAILED rather than answered, stamp that
// reason onto the disc so the decrypt gate can report it — but never clobber
// a reason the scan already captured (e.g. `AacsVidUnavailable`), which is
// closer to the disc itself than a source outage is.
if let Some(e) = source_failure
&& disc.aacs_error.is_none()
{
disc.aacs_error = Some(e);
}
(false, trace)
}
@@ -375,14 +485,42 @@ pub fn resolve_and_apply_traced(
/// [`resolve_and_apply`] this does not validate/commit to a disc — the read's
/// decorator re-decrypts with the returned keys, which is the validation.
pub fn fetch_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
drive_unit_keys(sources, ctx).keys
}
/// Whether a driver run resolved keys, and — when it did NOT — whether the miss
/// was a genuine "no source holds this key" (`errored == false`) or at least one
/// source FAILED (`errored == true`, e.g. a network source was unreachable). The
/// distinction gates negative-result memoization: an empty-because-absent result
/// is safe to cache, an empty-because-a-source-was-down result is transient and
/// must NOT be cached (the key may resolve once the source recovers).
struct FetchOutcome {
keys: Vec<UnitKey>,
errored: bool,
}
/// [`fetch_unit_keys`] plus the error signal: drive `sources` in order, return the
/// first source's non-empty Unit Keys, and flag whether any source that failed to
/// answer did so with an `Err` (a source failure) rather than an empty `Ok`
/// (genuine absence — see [`KeySource::get_unit_keys`]).
fn drive_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> FetchOutcome {
let mut errored = false;
for source in sources {
if let Ok(uks) = source.get_unit_keys(ctx) {
if !uks.is_empty() {
return uks;
match source.get_unit_keys(ctx) {
Ok(uks) if !uks.is_empty() => {
return FetchOutcome {
keys: uks,
errored: false,
};
}
Ok(_) => {}
Err(_) => errored = true,
}
}
Vec::new()
FetchOutcome {
keys: Vec::new(),
errored,
}
}
/// The forensic counterpart to [`fetch_unit_keys`]: drive `sources` in order and
@@ -392,14 +530,29 @@ pub fn fetch_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) ->
/// keying on `disc_hash`) ignores them. Whatever the winning source returns —
/// ≥ 1 key — is trusted as the COMPLETE ordered set; no fixed count is assumed.
pub fn fetch_fmts_indexes(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
drive_fmts_indexes(sources, ctx).keys
}
/// [`fetch_fmts_indexes`] plus the error signal (see [`drive_unit_keys`]): the
/// forensic counterpart that flags whether any source `Err`ed during the miss.
fn drive_fmts_indexes(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> FetchOutcome {
let mut errored = false;
for source in sources {
if let Ok(uks) = source.get_fmts_indexes(ctx) {
if !uks.is_empty() {
return uks;
match source.get_fmts_indexes(ctx) {
Ok(uks) if !uks.is_empty() => {
return FetchOutcome {
keys: uks,
errored: false,
};
}
Ok(_) => {}
Err(_) => errored = true,
}
}
Vec::new()
FetchOutcome {
keys: Vec::new(),
errored,
}
}
/// Build the read-time [`crate::sector::KeyFetch`] from the disc's public AACS
@@ -426,12 +579,17 @@ pub fn key_fetch(
// batch: the resolved keys are disc-level (a clip's index / CPS keys are
// identical for every title that references it), so the first batch resolves
// over the network and every repeat is answered from the cache with no
// request. Empty replies are cached too — a key the service lacks for a batch
// won't appear on a re-ask, so re-hitting the network buys nothing. Each
// operation gets its OWN cache: a base batch and a forensic anchor never
// collide, and the same bytes could legitimately resolve differently per op.
// The per-kind driver: `fetch_unit_keys` or `fetch_fmts_indexes`.
type FetchDriver = fn(&[Box<dyn KeySource>], &dyn ResolveCtx) -> Vec<UnitKey>;
// request. A GENUINELY-empty reply (every source ran and none held the key)
// is cached too — the key the service lacks for a batch won't appear on a
// re-ask, so re-hitting the network buys nothing. But an empty reply caused
// by a source FAILURE (network down, source unreachable) is NOT cached: that
// is a transient miss, and caching it would permanently drop a unit that
// could be recovered once the source recovers — the `errored` flag on
// `FetchOutcome` draws exactly that line. Each operation gets its OWN cache:
// a base batch and a forensic anchor never collide, and the same bytes could
// legitimately resolve differently per op.
// The per-kind driver: `drive_unit_keys` or `drive_fmts_indexes`.
type FetchDriver = fn(&[Box<dyn KeySource>], &dyn ResolveCtx) -> FetchOutcome;
fn make_op(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
@@ -460,16 +618,22 @@ pub fn key_fetch(
// derives unit keys from `enc_title_keys`, which a V10 disc parses at
// the 48-byte stride — hardcoding the V20 stride here corrupted them.
let ctx = DiscInputsCtx::new(&di);
let keys: Vec<[u8; 16]> = drive(&sources, &ctx).into_iter().map(|u| u.key).collect();
cache
.lock()
.unwrap_or_else(|e| e.into_inner())
.insert(fp, keys.clone());
let outcome = drive(&sources, &ctx);
let keys: Vec<[u8; 16]> = outcome.keys.into_iter().map(|u| u.key).collect();
// Memoize a positive result always; memoize a NEGATIVE (empty) result
// only when it is a genuine absence, never when a source errored — a
// transient outage must not permanently poison this fingerprint.
if !keys.is_empty() || !outcome.errored {
cache
.lock()
.unwrap_or_else(|e| e.into_inner())
.insert(fp, keys.clone());
}
keys
})
}
let unit = make_op(inputs.clone(), make_sources.clone(), fetch_unit_keys);
let fmts = make_op(inputs, make_sources, fetch_fmts_indexes);
let unit = make_op(inputs.clone(), make_sources.clone(), drive_unit_keys);
let fmts = make_op(inputs, make_sources, drive_fmts_indexes);
crate::sector::KeyFetch::new(unit, fmts)
}
@@ -867,6 +1031,97 @@ mod tests {
);
}
/// A transient source outage must NOT be memoized as a permanent "no key":
/// a fingerprint whose first fetch failed because the source errored must be
/// re-asked, and once the source recovers the key resolves. Regression guard
/// for the negative-result memoization fix — caching the errored empty would
/// permanently drop a recoverable unit for the rest of the op.
#[test]
fn errored_empty_is_not_cached_and_retries_when_source_recovers() {
use std::sync::atomic::{AtomicUsize, Ordering};
let key = [0x77u8; 16];
// Shared across every `make_sources()` rebuild: call 0 errors (source
// down), every later call succeeds (source recovered).
let calls = Arc::new(AtomicUsize::new(0));
struct Flaky {
calls: Arc<AtomicUsize>,
key: [u8; 16],
}
impl KeySource for Flaky {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
if self.calls.fetch_add(1, Ordering::SeqCst) == 0 {
Err(Error::AacsNoKeys) // first attempt: source unreachable
} else {
Ok(vec![UnitKey::new(0, self.key)])
}
}
}
let calls_c = Arc::clone(&calls);
let make: Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync> = Arc::new(move || {
vec![Box::new(Flaky {
calls: Arc::clone(&calls_c),
key,
}) as Box<dyn KeySource>]
});
let cb = key_fetch(empty_inputs(), make);
let samples = vec![vec![0xCDu8; 8]];
// First fetch: the source errors → empty, but the miss must NOT be cached.
assert!(
cb.unit_keys(&samples).is_empty(),
"source down → empty this time"
);
// Second fetch, SAME samples: not blocked by a cached empty → the now-
// recovered source resolves the key.
assert_eq!(
cb.unit_keys(&samples),
vec![key],
"recovered source resolves — errored empty was not memoized"
);
}
/// A GENUINE absence (a source that runs and returns an empty `Ok`) is still
/// memoized — the benefit the fix preserves. A source counting its calls must
/// be asked exactly once for a fingerprint whose first (clean) reply was empty.
#[test]
fn genuine_empty_is_still_memoized() {
use std::sync::atomic::{AtomicUsize, Ordering};
let calls = Arc::new(AtomicUsize::new(0));
struct AlwaysEmpty {
calls: Arc<AtomicUsize>,
}
impl KeySource for AlwaysEmpty {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
self.calls.fetch_add(1, Ordering::SeqCst);
Ok(Vec::new()) // ran fine, genuinely holds no key
}
}
let calls_c = Arc::clone(&calls);
let make: Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync> = Arc::new(move || {
vec![Box::new(AlwaysEmpty {
calls: Arc::clone(&calls_c),
}) as Box<dyn KeySource>]
});
let cb = key_fetch(empty_inputs(), make);
let samples = vec![vec![0xEFu8; 8]];
assert!(cb.unit_keys(&samples).is_empty());
assert!(cb.unit_keys(&samples).is_empty());
assert_eq!(
calls.load(Ordering::SeqCst),
1,
"a clean empty reply is cached — the source is asked only once"
);
}
/// The two `KeyFetch` operations route to the two DISTINCT trait methods:
/// `unit_keys` drives `get_unit_keys`, `fmts_indexes` drives
/// `get_fmts_indexes`. A source that returns different keys per method proves
@@ -1068,7 +1323,7 @@ mod tests {
break;
}
let abs = (lba - self.ext_start) / ALIGNED_UNIT_SECTORS + i as u32;
if abs % 2 == 0 {
if abs.is_multiple_of(2) {
chunk.fill(0x11); // CPI-clear (0x11 & 0xC0 == 0), no TS sync
} else {
chunk.fill(0xAB); // scrambled body (no TS sync)
@@ -1166,4 +1421,63 @@ mod tests {
assert_eq!(k10[1], [0x10; 16], "V10 reads the 2nd key at +48");
assert_ne!(k20[1], k10[1], "the parse stride follows inputs.version");
}
/// `DiscInputs` is public and returned by `Disc::inputs`, so any consumer's
/// `tracing::debug!("{inputs:?}")` prints it. A derived `Debug` printed the
/// Volume ID (the value `aacs::types::Vid` deliberately renders as
/// `Vid(<redacted>)`), the whole `Unit_Key_RO.inf` (the encrypted title keys),
/// the entire MKB and every ciphertext sample verbatim. Sentinel byte
/// 0xD5 = decimal 213, matching `aacs::types::redaction_tests`. Mutation
/// guard: restoring `#[derive(Debug)]` fails this.
#[test]
fn disc_inputs_debug_is_redacted() {
let inputs = DiscInputs {
disc_hash: "0xAA".into(),
volume_id: [0xD5; 16],
version: 2,
mkb: vec![0xD5; 64],
unit_key_ro: vec![0xD5; 48],
samples: vec![vec![0xD5; 6144]],
volume_label: Some("TITLE_2024".into()),
};
let dbg = format!("{inputs:?}");
assert!(
!dbg.contains("213"),
"DiscInputs Debug leaked key material (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"DiscInputs Debug missing redaction marker: {dbg}"
);
// Non-secret identity and shape stay printable for diagnostics.
assert!(dbg.contains("0xAA"), "{dbg}");
assert!(dbg.contains("mkb_len: 64"), "{dbg}");
assert!(dbg.contains("unit_key_ro_len: 48"), "{dbg}");
assert!(dbg.contains("samples_len: 1"), "{dbg}");
assert!(dbg.contains("TITLE_2024"), "{dbg}");
}
/// `DecodeSampleSet` is public and wraps the SAME on-disc ciphertext the
/// sibling `DiscInputs` redacts, so a derived `Debug` dumped ≥ MIN_SAMPLE_UNITS
/// × 6144 bytes of verbatim AACS ciphertext (plus every unit's clear 16-byte
/// derivation seed) into any log that formatted it. Sentinel byte 0xD5 =
/// decimal 213, matching `aacs::types::redaction_tests` and the
/// `DiscInputs` test above. Mutation guard: restoring `#[derive(Debug)]`
/// fails this.
#[test]
fn decode_sample_set_debug_is_redacted() {
let set = DecodeSampleSet::new(vec![vec![0xD5; 6144]; MIN_SAMPLE_UNITS])
.expect("MIN_SAMPLE_UNITS units is a valid set");
let dbg = format!("{set:?}");
assert!(
!dbg.contains("213"),
"DecodeSampleSet Debug leaked ciphertext (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"DecodeSampleSet Debug missing redaction marker: {dbg}"
);
// Non-secret shape stays printable for diagnostics.
assert!(dbg.contains("units_len: 8"), "{dbg}");
}
}
+45 -16
View File
@@ -100,10 +100,10 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<DiscMetadata>
// Disc-set position is disc-global; first one we successfully
// read wins. (All bdmt_*.xml on a given disc carry the same
// value in practice.)
if out.disc_number.is_none() {
if let Some(ds) = disc_set {
out.disc_number = Some(ds);
}
if out.disc_number.is_none()
&& let Some(ds) = disc_set
{
out.disc_number = Some(ds);
}
}
@@ -184,20 +184,20 @@ fn extract_title(xml_text: &str) -> Option<String> {
// xml::text already trims its result, so an empty string after
// extraction means a genuinely empty element.
for tag in ["name", "title"] {
if let Some(s) = xml::text(xml_text, tag) {
if !s.is_empty() {
return Some(s);
}
if let Some(s) = xml::text(xml_text, tag)
&& !s.is_empty()
{
return Some(s);
}
}
// tableOfContents/titleName: search inside the toc block so we
// don't accidentally pick a stray <titleName> from elsewhere.
if let Some((s, e)) = xml::find_element(xml_text, "tableOfContents", 0) {
let block = &xml_text[s..e];
if let Some(t) = xml::text(block, "titleName") {
if !t.is_empty() {
return Some(t);
}
if let Some(t) = xml::text(block, "titleName")
&& !t.is_empty()
{
return Some(t);
}
}
None
@@ -370,10 +370,10 @@ mod tests {
if let Some(d) = desc {
meta.descriptions.insert(lang.to_string(), d);
}
if meta.disc_number.is_none() {
if let Some(d) = ds {
meta.disc_number = Some(d);
}
if meta.disc_number.is_none()
&& let Some(d) = ds
{
meta.disc_number = Some(d);
}
}
@@ -566,4 +566,33 @@ mod tests {
let (title, _, _) = parse_bdmt_xml(xml).unwrap();
assert_eq!(title, "Real Title");
}
/// `is_bdmt_filename` must recognize the `bdmt_<lang>.xml` convention
/// and reject everything else — it drives `detect`'s directory scan.
/// Mutation: stub the return to a constant `true`/`false` → every
/// directory listing (or none) would match regardless of filename.
#[test]
fn is_bdmt_filename_matches_convention_only() {
assert!(is_bdmt_filename("bdmt_eng.xml"));
assert!(is_bdmt_filename("BDMT_FRA.XML"));
assert!(!is_bdmt_filename("bdmt_engl.xml"));
assert!(!is_bdmt_filename("index.bdmv"));
assert!(!is_bdmt_filename("foo.xml"));
}
/// Spec: "Disc 1 of 1" (a single-disc release whose bdmt XML still
/// carries `<di:numSets>1</di:numSets>`) is a valid, non-nonsensical
/// pair — `total < 1` must reject only `total == 0`, not `total == 1`.
/// Mutation: `total < 1` -> `total == 1` or `total <= 1` would reject
/// this legitimate (1, 1) pair as if it were malformed.
#[test]
fn disc_set_allows_single_disc_release() {
let xml = r#"<discInfo xmlns:di="urn:BDA:bdmv;disclibmeta">
<di:name>Film</di:name>
<di:discNumber>1</di:discNumber>
<di:numSets>1</di:numSets>
</discInfo>"#;
let (_, _, set) = parse_bdmt_xml(xml).unwrap();
assert_eq!(set, Some((1, 1)));
}
}
+387 -1
View File
@@ -989,7 +989,18 @@ impl<'a> Reader<'a> {
}
fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> {
if self.pos + n > self.data.len() {
// `n` is attacker-supplied: it comes from a JVMS `u4` attribute_length
// / code_length (§4.7, §4.7.3) or a `u2` Utf8 length (§4.4.7). Unlike
// the fixed-width readers above, whose `self.pos + k` cannot leave the
// buffer's own address range, `self.pos + n` can wrap — on a 32-bit
// target a `u4` length near 0xFFFF_FFFF plus a non-zero `pos` panics
// in debug and in release wraps to a SMALL end offset that passes the
// bounds check, after which the slice index itself panics. Checked, so
// an out-of-range length is the EOF error it always should have been.
let Some(end) = self.pos.checked_add(n) else {
return Err(Error::UnexpectedEof { needed });
};
if end > self.data.len() {
return Err(Error::UnexpectedEof { needed });
}
let s = &self.data[self.pos..self.pos + n];
@@ -1006,6 +1017,46 @@ impl<'a> Reader<'a> {
mod tests {
use super::*;
/// `Reader::slice` takes an attacker-supplied length: a JVMS `u4`
/// `attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8
/// length (§4.4.7). Adding it to `pos` without a wrap check panics on
/// overflow in debug and, in release, wraps to a small end offset that
/// slips past the bounds check and then panics inside the slice index.
/// Both are panics escaping a parser whose whole input is untrusted disc
/// bytes; the contract is an EOF error.
#[test]
fn slice_rejects_a_length_that_would_wrap_pos() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
r.u64("advance pos").expect("8 bytes available");
// pos is now 8; usize::MAX would wrap the end offset to 7.
match r.slice(usize::MAX, "wrapping length") {
Err(Error::UnexpectedEof { .. }) => {}
Err(other) => panic!("expected UnexpectedEof, got {other:?}"),
Ok(s) => panic!("expected UnexpectedEof, got a {}-byte slice", s.len()),
}
// The reader must not have consumed anything.
match r.slice(8, "remaining bytes") {
Ok(s) => assert_eq!(s.len(), 8, "pos moved on the rejected slice"),
Err(e) => panic!("the remaining 8 bytes must still be readable: {e:?}"),
}
}
/// The ordinary out-of-range case (no wrap) must keep returning EOF, and
/// an exactly-fitting length must still succeed — the check is `>`, not
/// `>=`.
#[test]
fn slice_boundary_is_inclusive_of_the_final_byte() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
assert_eq!(r.slice(16, "whole buffer").expect("exact fit").len(), 16);
let mut r = Reader::new(&data);
assert!(matches!(
r.slice(17, "one past"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn rejects_non_class_bytes() {
match ClassFile::parse(b"\x00\x01\x02\x03DEAD") {
@@ -1337,4 +1388,339 @@ mod tests {
let _ = decode_modified_utf8(&buf);
}
}
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness
//
// These exercise plain data accessors on an already-parsed pool
// (built via the test-only `from_entries` constructor) — not the
// untrusted-bytes parsing path, just "does the right variant map to
// the right Option value."
// -----------------------------------------------------------------
fn sample_pool() -> ConstantPool {
// index: 0=Empty (reserved), 1=Utf8("Hello"), 2=Integer(42),
// 3=String{string_index:1}, 4=Class{name_index:1}, 5=Float(1.5),
// 6=Long(9), 7=Empty (2-slot tail), 8=Double(2.5), 9=Empty (tail).
ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("Hello".to_string()),
CpInfo::Integer(42),
CpInfo::String { string_index: 1 },
CpInfo::Class { name_index: 1 },
CpInfo::Float(1.5),
CpInfo::Long(9),
CpInfo::Empty,
CpInfo::Double(2.5),
CpInfo::Empty,
])
}
#[test]
fn constant_pool_string_resolves_through_string_index() {
let pool = sample_pool();
// index 3 is CpInfo::String{string_index: 1} -> utf8(1) = "Hello".
assert_eq!(pool.string(3), Some("Hello"));
// Wrong variant (Integer at index 2) must not resolve as a string.
assert_eq!(pool.string(2), None);
// Out of range index.
assert_eq!(pool.string(999), None);
}
#[test]
fn constant_pool_integer_resolves_only_integer_entries() {
let pool = sample_pool();
assert_eq!(pool.integer(2), Some(42));
// Wrong variant (Utf8 at index 1) must not resolve as an integer.
assert_eq!(pool.integer(1), None);
assert_eq!(pool.integer(999), None);
}
#[test]
fn constant_pool_load_constant_display_covers_ldc_operand_kinds() {
let pool = sample_pool();
assert_eq!(
pool.load_constant_display(1),
Some("utf8:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(2), Some("int:42".to_string()));
assert_eq!(
pool.load_constant_display(3),
Some("str:\"Hello\"".to_string())
);
assert_eq!(
pool.load_constant_display(4),
Some("class:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(5), Some("float:1.5".to_string()));
assert_eq!(pool.load_constant_display(6), Some("long:9".to_string()));
assert_eq!(
pool.load_constant_display(8),
Some("double:2.5".to_string())
);
// A variant with no display arm (e.g. reserved Empty slot) -> None.
assert_eq!(pool.load_constant_display(0), None);
assert_eq!(pool.load_constant_display(999), None);
}
#[test]
fn constant_pool_len_and_is_empty() {
let pool = sample_pool();
assert_eq!(pool.len(), 10);
assert!(!pool.is_empty());
let empty = ConstantPool::from_entries(vec![]);
assert_eq!(empty.len(), 0);
assert!(empty.is_empty());
}
#[test]
fn constant_pool_iter_yields_index_and_entry_pairs() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("A".to_string()),
CpInfo::Integer(7),
]);
let indices: Vec<u16> = pool.iter().map(|(i, _)| i).collect();
assert_eq!(indices, vec![0, 1, 2]);
// Confirm the entries themselves come through, not an empty iterator.
let utf8_at_1 = pool.iter().find(|(i, _)| *i == 1).map(|(_, e)| match e {
CpInfo::Utf8(s) => s.as_str(),
_ => "?",
});
assert_eq!(utf8_at_1, Some("A"));
}
fn class_file_with(this_class: u16, super_class: u16, pool: ConstantPool) -> ClassFile {
ClassFile {
minor_version: 0,
major_version: 0,
constant_pool: pool,
access_flags: 0,
this_class,
super_class,
interfaces: Vec::new(),
fields: Vec::new(),
methods: Vec::new(),
attributes: Vec::new(),
}
}
#[test]
fn this_class_name_and_super_class_name_resolve_distinct_indices() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("com/example/Foo".to_string()),
CpInfo::Utf8("com/example/Bar".to_string()),
CpInfo::Class { name_index: 1 },
CpInfo::Class { name_index: 2 },
]);
let cf = class_file_with(3, 4, pool);
assert_eq!(cf.this_class_name(), Some("com/example/Foo"));
assert_eq!(cf.super_class_name(), Some("com/example/Bar"));
// this_class index pointing at a non-Class entry must not resolve.
let pool2 = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("not a class ref".to_string()),
]);
let cf2 = class_file_with(1, 1, pool2);
assert_eq!(cf2.this_class_name(), None);
assert_eq!(cf2.super_class_name(), None);
}
#[test]
fn member_descriptor_resolves_the_descriptor_not_the_name() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("doStuff".to_string()), // index 1: name
CpInfo::Utf8("()V".to_string()), // index 2: descriptor
]);
let cf = class_file_with(0, 0, pool);
let m = Member {
access_flags: 0,
name_index: 1,
descriptor_index: 2,
attributes: Vec::new(),
};
assert_eq!(cf.member_descriptor(&m), Some("()V"));
assert_ne!(cf.member_descriptor(&m), Some("doStuff"));
}
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness
//
// Mirrors `slice_boundary_is_inclusive_of_the_final_byte`: an
// exact-fit read must succeed, one byte short must fail. Plus
// positive-value tests so a scrambled byte assembly (not just an
// out-of-bounds read) would be caught.
// -----------------------------------------------------------------
#[test]
fn u16_boundary_is_inclusive_of_the_final_byte() {
let data = [0xAB, 0xCD];
let mut r = Reader::new(&data);
assert_eq!(r.u16("exact fit").expect("2 bytes available"), 0xABCD);
let data = [0xAB];
let mut r = Reader::new(&data);
assert!(matches!(
r.u16("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u16_decodes_big_endian_value() {
let data = [0x01, 0x02];
let mut r = Reader::new(&data);
assert_eq!(r.u16("value").unwrap(), 0x0102);
}
#[test]
fn u32_boundary_is_inclusive_of_the_final_byte() {
let data = [0x00, 0x00, 0x00, 0x2A];
let mut r = Reader::new(&data);
assert_eq!(r.u32("exact fit").expect("4 bytes available"), 42);
let data = [0x00, 0x00, 0x00];
let mut r = Reader::new(&data);
assert!(matches!(
r.u32("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u32_decodes_big_endian_value() {
let data = [0x00, 0x00, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u32("value").unwrap(), 1337);
}
#[test]
fn u64_boundary_is_inclusive_of_the_final_byte() {
// pos == 0, buffer exactly 8 bytes: must succeed.
let data = [0, 0, 0, 0, 0, 0, 0, 0x7B]; // 123
let mut r = Reader::new(&data);
assert_eq!(r.u64("exact fit").expect("8 bytes available"), 123);
// pos == 0, buffer one byte short of 8: must fail cleanly, not
// panic on the internal self.data[self.pos + 7] index.
let data = [0u8; 7];
let mut r = Reader::new(&data);
assert!(matches!(
r.u64("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u64_decodes_big_endian_value() {
let data = [0, 0, 0, 0, 0, 0, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u64("value").unwrap(), 1337);
}
// -----------------------------------------------------------------
// decode_modified_utf8: 3-byte (BMP) decode path
// -----------------------------------------------------------------
#[test]
fn modified_utf8_three_byte_cjk() {
// U+3042 (hiragana あ) in modified UTF-8: 1110xxxx 10xxxxxx 10xxxxxx
// = 0xE3 0x81 0x82.
let s = decode_modified_utf8(&[0xE3, 0x81, 0x82]).unwrap();
assert_eq!(s, "\u{3042}");
}
#[test]
fn modified_utf8_three_byte_rejects_bad_first_continuation() {
// Valid lead byte (0xE3), but the first continuation byte is not
// 10xxxxxx (0x01 instead) — must be rejected, proving the first
// `& 0xC0 != 0x80` check is live.
assert!(decode_modified_utf8(&[0xE3, 0x01, 0x82]).is_err());
}
#[test]
fn modified_utf8_three_byte_rejects_bad_second_continuation() {
// Valid lead + first continuation, but the second continuation
// byte is not 10xxxxxx — proves the second check is independently
// live (not short-circuited by the first).
assert!(decode_modified_utf8(&[0xE3, 0x81, 0x01]).is_err());
}
// -----------------------------------------------------------------
// read_constant_pool: Long/Double two-slot skip, real byte parsing
// -----------------------------------------------------------------
#[test]
fn constant_pool_long_entry_occupies_two_slots_via_real_parse() {
// Real class-file bytes (not the `from_entries` synthetic ctor):
// magic + minor/major + cp_count=4 + tag=5 (Long, 8-byte payload
// at index 1, reserved slot at index 2) + tag=1 (Utf8 at index 3)
// + empty access_flags/this/super/interfaces/fields/methods/attrs.
let mut buf = vec![
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, // minor
0x00, 0x34, // major
0x00, 0x04, // cp_count = 4 (0=Empty,1=Long,2=Empty tail,3=Utf8)
5, // Long tag
];
buf.extend_from_slice(&0x1122_3344_5566_7788u64.to_be_bytes()); // 8-byte payload
buf.push(1); // Utf8 tag
let name = b"marker";
buf.extend_from_slice(&(name.len() as u16).to_be_bytes());
buf.extend_from_slice(name);
// access_flags, this_class, super_class, interfaces_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0, 0, 0]);
// fields_count, methods_count, attributes_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0]);
let cf = ClassFile::parse(&buf).expect("well-formed synthetic class file");
assert_eq!(cf.constant_pool.len(), 4);
// The Long occupies indices 1 AND 2 (its reserved tail slot).
// The Utf8 must resolve at index 3 = long_index(1) + 2, NOT +1.
assert_eq!(cf.constant_pool.utf8(3), Some("marker"));
// Index 2 is the reserved tail slot: not a Utf8, must not
// resolve as one (guards against the Utf8 landing one slot early).
assert_eq!(cf.constant_pool.utf8(2), None);
match cf.constant_pool.get(1) {
Some(CpInfo::Long(v)) => assert_eq!(*v, 0x1122_3344_5566_7788u64 as i64),
other => panic!("expected Long at index 1, got {:?}", other),
}
}
// -----------------------------------------------------------------
// instruction_size: tableswitch/lookupswitch with non-degenerate
// low/high/npairs (the existing tests only cover low==high==0 and
// npairs==0, which can't distinguish `-` from `+` in the entry-count
// arithmetic).
// -----------------------------------------------------------------
#[test]
fn instruction_size_tableswitch_non_degenerate_range() {
// low=1, high=4 -> 4 entries (high-low+1 = 4). A `-`->`+` mutation
// on that arithmetic would instead compute high+low+1 = 6.
let mut code = vec![TABLESWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default offset
code.extend_from_slice(&1i32.to_be_bytes()); // low = 1
code.extend_from_slice(&4i32.to_be_bytes()); // high = 4
code.extend_from_slice(&[0; 16]); // 4 jump entries * 4 bytes
// total = 1 (opcode) + 3 (pad) + 12 (default/low/high) + 16 (entries) = 32
assert_eq!(instruction_size(&code, 0), Some(32));
}
#[test]
fn instruction_size_lookupswitch_non_degenerate_npairs() {
// npairs = 3 -> 3 * 8 = 24 bytes of pairs.
let mut code = vec![LOOKUPSWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default
code.extend_from_slice(&3i32.to_be_bytes()); // npairs = 3
code.extend_from_slice(&[0; 24]); // 3 pairs
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
}
+208 -36
View File
@@ -36,17 +36,18 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// Stream number mapping from playbackconfig.xml
let mut stream_map: HashMap<String, u16> = HashMap::new();
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml") {
if let Ok(pc_text) = std::str::from_utf8(&pc_data) {
parse_playback_config(pc_text, &mut stream_map);
}
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml")
&& let Ok(pc_text) = std::str::from_utf8(&pc_data)
{
parse_playback_config(pc_text, &mut stream_map);
}
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map);
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map)?;
let mut labels = Vec::new();
for (info, &stream_num) in stream_infos.iter().zip(stream_nums.iter()) {
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type: info.stream_type,
language: info.language.clone(),
@@ -76,7 +77,29 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// map-assigned one. (Both numbering domains are 1-based per type, and
/// `apply_labels` matches on `(type, stream_number)`, so a collision
/// would mislabel tracks.)
fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>) -> Vec<u16> {
///
/// Returns `None` when the 1-based stream-number space is exhausted — every
/// number in `1..=u16::MAX` for that type is either already claimed by the map
/// or already synthesized. That is unreachable on real media: the BD STN_table
/// carries at most 32 primary audio and 32 PG streams per playlist, so the
/// 65535-wide space leaves >2000x headroom. It IS reachable from a crafted
/// `streamproperties.xml` listing >65535 stream entries, and the only correct
/// answers there are "fail the parse" or "emit colliding numbers"; we fail.
///
/// The skip search is bounded by the numbering space itself: a `u16`
/// `saturating_add` here parked the counter at `u16::MAX` forever whenever the
/// map also claimed `u16::MAX`, turning an overflow guard into a hang that
/// `apply()`'s `catch_unwind` cannot interrupt. The counters are therefore
/// widened to `u32` so the skip loop strictly increases toward a fixed ceiling
/// (guaranteeing termination) and exhaustion is reported rather than absorbed.
fn assign_stream_numbers(
infos: &[StreamInfo],
stream_map: &HashMap<String, u16>,
) -> Option<Vec<u16>> {
/// One past the last assignable stream number, as a `u32` so the
/// counters can step off the end of the `u16` domain without wrapping.
const NUMBER_SPACE_END: u32 = u16::MAX as u32 + 1;
// Numbers already claimed by the map, per type. A map value of 0 is NOT a
// claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable.
// Treat 0 as "unmapped" here (defense in depth — parse_playback_config also
@@ -96,8 +119,8 @@ fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>
}
}
let mut audio_idx: u16 = 1;
let mut sub_idx: u16 = 1;
let mut audio_idx: u32 = 1;
let mut sub_idx: u32 = 1;
let mut out = Vec::with_capacity(infos.len());
for info in infos {
let n = match stream_map.get(&info.id).copied() {
@@ -107,21 +130,31 @@ fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>
StreamLabelType::Audio => (&mut audio_idx, &taken_audio),
StreamLabelType::Subtitle => (&mut sub_idx, &taken_sub),
};
// Advance past any number already claimed via the map.
// saturating: a crafted XML with >65k stream entries must
// not overflow (panic in debug, wrap-to-0 in release) on
// untrusted disc bytes.
while taken.contains(idx) {
*idx = idx.saturating_add(1);
// Advance past any number already claimed via the map. The
// counter strictly increases and NUMBER_SPACE_END is fixed, so
// this terminates in at most 65535 steps for any input.
while *idx < NUMBER_SPACE_END && taken.contains(&(*idx as u16)) {
*idx += 1;
}
let n = *idx;
*idx = idx.saturating_add(1);
if *idx >= NUMBER_SPACE_END {
// Numbering space exhausted. Emitting anything here would
// either wrap to 0 (unmatchable) or duplicate a number
// already bound to a different stream, so the parse fails.
tracing::warn!(
streams = infos.len(),
"criterion: 1-based u16 stream-number space exhausted; \
refusing to synthesize a colliding stream number"
);
return None;
}
let n = *idx as u16;
*idx += 1;
n
}
};
out.push(n);
}
out
Some(out)
}
struct StreamInfo {
@@ -189,14 +222,13 @@ fn parse_playback_config(text: &str, map: &mut HashMap<String, u16>) {
if let (Some(stream_id_str), Some(info_id)) = (
xml::text(block, "StreamID"),
xml::text(block, "StreamInfo_ID"),
) {
if let Ok(stream_num) = stream_id_str.parse::<u16>() {
// Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it.
if stream_num != 0 {
map.insert(info_id, stream_num);
}
) && let Ok(stream_num) = stream_id_str.parse::<u16>()
{
// Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it.
if stream_num != 0 {
map.insert(info_id, stream_num);
}
}
from = end;
@@ -226,11 +258,89 @@ mod tests {
info("a1", StreamLabelType::Audio),
info("s0", StreamLabelType::Subtitle),
];
let nums = assign_stream_numbers(&infos, &HashMap::new());
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Per-type 1-based: audio 1,2 ; subtitle 1.
assert_eq!(nums, vec![1, 2, 1]);
}
/// Immunity pin. `parse_stream_infos` emits one `StreamInfo` per
/// `*StreamInfos` element unconditionally — no filter, no `continue` — so
/// an element whose fields are missing or unrecognized still occupies its
/// position, and `assign_stream_numbers` still spends a number on it.
///
/// That is the property that keeps this parser out of the failure mode
/// where a skipped entry pulls every later label one stream forward. It
/// is load-bearing for the fallback path specifically: with no
/// `playbackconfig.xml` the numbers come purely from position in this
/// list, so dropping an element there would shift the rest.
///
/// Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
/// real audio streams renumber to 1 and 2.
/// Immunity pin, section-boundary half. Each stream here is one closed XML
/// element, and every field is read out of `&text[start..end]` — the range
/// `xml::find_element` returned — so one element can never absorb the next
/// one's fields, however the document is malformed around it. Contrast the
/// flat-string walk in pixelogic, where a section whose end marker is
/// missing keeps consuming entries as STN slots.
///
/// The missing-boundary case fails closed. An element with no close tag of
/// its own ends at the NEXT close tag, so it absorbs the element behind it
/// — the list comes back SHORTER. It cannot come back longer: nothing
/// outside a returned range is ever read as a stream, and `find_element`
/// yields `None` rather than a range running to EOF when no close tag
/// exists at all. A malformed document can cost this parser a slot; it can
/// never invent one.
///
/// Mutation: read fields from the document rather than the element's
/// range, or let a close-less element run to EOF → the trailing elements
/// re-enter the list as extra streams.
#[test]
fn an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it() {
let sp = concat!(
"<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID></AudioStreamInfos>",
// No `</AudioStreamInfos>` for this one.
"<AudioStreamInfos><ID>a1</ID><LangInfoID>FRA</LangInfoID>",
"<AudioStreamInfos><ID>a2</ID><LangInfoID>DEU</LangInfoID></AudioStreamInfos>",
);
let infos = parse_stream_infos(sp);
assert_eq!(
infos.iter().map(|i| i.id.as_str()).collect::<Vec<_>>(),
vec!["a0", "a1"],
"the close-less element absorbs the one behind it — two slots, not \
three, and never four"
);
assert_eq!(infos[1].language, "fra", "and keeps its own leading fields");
// With no close tag anywhere behind it, the element is not returned at
// all and the walk ends — the tail of the document never becomes a
// stream list.
let no_close = "<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID>";
assert!(parse_stream_infos(no_close).is_empty());
}
#[test]
fn unusable_stream_element_still_occupies_its_position() {
let sp = r#"
<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG_US</LangInfoID></AudioStreamInfos>
<AudioStreamInfos></AudioStreamInfos>
<AudioStreamInfos><ID>a2</ID><LangInfoID>FRA</LangInfoID><Content>COMMENTARY</Content></AudioStreamInfos>
<SubtitleStreamInfos><ID>s0</ID><LangInfoID></LangInfoID><Qualifier>WAT</Qualifier></SubtitleStreamInfos>
<SubtitleStreamInfos><ID>s1</ID><LangInfoID>ENG</LangInfoID><Qualifier>SDH</Qualifier></SubtitleStreamInfos>
"#;
let infos = parse_stream_infos(sp);
assert_eq!(infos.len(), 5, "every element yields a StreamInfo");
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
assert_eq!(
nums,
vec![1, 2, 3, 1, 2],
"the blank element owns audio slot 2, so the commentary is slot 3"
);
assert_eq!(infos[2].purpose, LabelPurpose::Commentary);
assert_eq!(infos[4].qualifier, LabelQualifier::Sdh);
}
#[test]
fn fallback_does_not_collide_with_partial_map() {
// Map claims audio "a1" -> 1. The unmapped audio "a0" must NOT
@@ -242,7 +352,7 @@ mod tests {
info("a1", StreamLabelType::Audio), // mapped → 1
info("a2", StreamLabelType::Audio), // unmapped → fallback
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// a0 skips the taken 1 → 2; a1 keeps 1; a2 → 3. All distinct.
assert_eq!(nums, vec![2, 1, 3]);
let mut sorted = nums.clone();
@@ -260,7 +370,10 @@ mod tests {
info("a0", StreamLabelType::Audio),
info("a1", StreamLabelType::Audio),
];
assert_eq!(assign_stream_numbers(&infos, &map), vec![5, 9]);
assert_eq!(
assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"),
vec![5, 9]
);
}
// ── Additional hardening tests ─────────────────────────────────────────
@@ -276,7 +389,8 @@ mod tests {
info("a1", StreamLabelType::Audio),
info("s1", StreamLabelType::Subtitle),
];
let nums = assign_stream_numbers(&infos, &HashMap::new());
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Audio: 1, 2; Subtitle: 1, 2 — each counter resets at 1 per type.
assert_eq!(nums[0], 1); // audio 1
assert_eq!(nums[1], 1); // subtitle 1
@@ -293,7 +407,7 @@ mod tests {
let mut map = HashMap::new();
map.insert("a0".to_string(), 0u16); // 0 must not be treated as a claim
let infos = vec![info("a0", StreamLabelType::Audio)];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// 0 is treated as unmapped → the fallback counter assigns 1.
assert_eq!(nums[0], 1);
}
@@ -309,7 +423,7 @@ mod tests {
info("real", StreamLabelType::Audio),
info("bad", StreamLabelType::Audio),
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
assert_eq!(nums[0], 1); // the genuinely-mapped stream keeps 1
assert_eq!(nums[1], 2); // the 0-stream is synthesized to the next free slot
}
@@ -326,16 +440,74 @@ mod tests {
info("a0", StreamLabelType::Audio), // fallback
info("s0", StreamLabelType::Subtitle), // mapped → 2
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// Audio fallback for a0 → 1 (subtitle's taken-2 doesn't block it).
assert_eq!(nums[0], 1);
assert_eq!(nums[1], 2);
}
/// Spec: saturating_add prevents overflow when many streams are listed.
/// Mutation: use wrapping_add → counter wraps to 0 and collides.
/// A crafted `streamproperties.xml` can drive the fallback counter to the
/// top of the 1-based u16 stream-number space and then present one more
/// unmapped stream whose successor number is also claimed by the map.
///
/// This must TERMINATE. The bound is the numbering space itself, so the
/// assertion is on the spec-derived exhaustion behaviour (`None`), not on
/// any tunable constant. Run on a worker thread with a deadline so a
/// non-terminating loop fails the test in 20 s instead of hanging CI.
#[test]
fn assign_stream_numbers_saturation_on_overflow() {
fn exhausted_numbering_terminates_instead_of_looping() {
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
// One mapped audio stream claims the last number in the space.
let mut map = HashMap::new();
map.insert("claims_max".to_string(), u16::MAX);
let mut infos = vec![info("claims_max", StreamLabelType::Audio)];
// Enough unmapped audio streams to walk the counter to the top.
for i in 0..=(u16::MAX as u32) {
infos.push(info(&format!("u{i}"), StreamLabelType::Audio));
}
let _ = tx.send(assign_stream_numbers(&infos, &map));
});
match rx.recv_timeout(std::time::Duration::from_secs(20)) {
Ok(result) => {
worker.join().expect("worker panicked");
assert!(
result.is_none(),
"an exhausted 1-based u16 numbering space must fail the parse, \
not emit colliding or wrapped stream numbers"
);
}
Err(_) => panic!(
"assign_stream_numbers did not terminate within 20s — \
non-terminating skip loop on crafted stream_map"
),
}
}
/// The whole 1-based u16 space must remain usable: 65535 unmapped audio
/// streams get 65535 distinct numbers with no panic and no wrap. The
/// literals here are the JVMS-independent, spec-derived size of a u16
/// 1-based numbering domain, not a tunable cap.
#[test]
fn full_u16_numbering_space_is_usable_and_unique() {
let infos: Vec<StreamInfo> = (0..65_535u32)
.map(|i| info(&format!("a{i}"), StreamLabelType::Audio))
.collect();
let nums = assign_stream_numbers(&infos, &HashMap::new()).expect("space is not exhausted");
assert_eq!(nums.len(), 65_535);
assert_eq!(nums[0], 1);
assert_eq!(nums[65_534], 65_535);
let mut sorted = nums.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), 65_535, "stream numbers must all be distinct");
}
/// Spec: a partially-mapped playlist with many claimed numbers must still
/// synthesize past every claim without panicking or colliding.
/// Mutation: drop the skip loop → the fallback reuses a claimed number.
#[test]
fn fallback_skips_a_dense_block_of_claimed_numbers() {
// Force the counter past u16::MAX by pre-taking all values 1..=u16::MAX.
// Doing that for real would be slow; instead inject u16::MAX into taken.
let mut map = HashMap::new();
@@ -362,7 +534,7 @@ mod tests {
qualifier: LabelQualifier::None,
});
// This must not panic.
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
assert_eq!(nums.len(), 501);
// The last (unmapped) entry's number must be > 500 (skipped all taken).
assert!(nums[500] > 500);
+276 -119
View File
@@ -50,10 +50,10 @@ fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
if let Some(mb_match) = mb
.iter()
.find(|m| m.stream_type == label.stream_type && m.stream_number == label.stream_number)
&& label.name.is_empty()
&& !mb_match.name.is_empty()
{
if label.name.is_empty() && !mb_match.name.is_empty() {
label.name = mb_match.name.clone();
}
label.name = mb_match.name.clone();
}
}
// Append any menu_base-only stream (present in mb but not in ls by
@@ -87,7 +87,21 @@ fn prefix_is_commentary(prefix: &str) -> bool {
fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
let data = super::read_jar_file(reader, udf, "language_streams.txt")?;
let text = std::str::from_utf8(&data).ok()?;
let labels = parse_language_streams_text(text);
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels.
///
/// This is the shipping parser: [`parse_language_streams`] does the UDF read
/// and UTF-8 decode and then delegates here. It is split out — rather than
/// duplicated under `#[cfg(test)]`, which is what it used to be — so the unit
/// tests below exercise production code. A test that re-implements the
/// function it guards cannot fail when the real function breaks.
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
@@ -193,122 +207,7 @@ fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<
}
labels.push(StreamLabel {
stream_number: stream_num,
stream_type,
language,
name: String::new(),
purpose: final_purpose,
qualifier,
codec_hint,
variant: variant_code,
});
}
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels. Split
/// out from [`parse_language_streams`] so unit tests exercise the real parsing
/// logic without needing a SectorSource / UdfFs.
#[cfg(test)]
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let parts: Vec<&str> = line.split(',').map(|s| s.trim()).collect();
if parts.len() < 4 {
continue;
}
let type_str = parts[1];
let stream_num: u16 = match parts[2].parse() {
Ok(n) if n > 0 => n,
_ => continue,
};
let language = parts[3].to_string();
let variant = if parts.len() > 4 {
parts[4].to_string()
} else {
String::new()
};
let (stream_type, purpose, qualifier) = match type_str {
"audio_production" => (
StreamLabelType::Audio,
LabelPurpose::Normal,
LabelQualifier::None,
),
"audio_commentary" => (
StreamLabelType::Audio,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"audio_ime" => (
StreamLabelType::Audio,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_production" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_commentary" => (
StreamLabelType::Subtitle,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"subtitle_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::Forced,
),
"subtitle_dual" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_bonus" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_ime" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_ime_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::Forced,
),
_ => continue,
};
let mut codec_hint = String::new();
let mut variant_code = String::new();
let mut final_purpose = purpose;
if !variant.is_empty() {
match variant.as_str() {
"eda" => final_purpose = LabelPurpose::Descriptive,
"csp" | "cs" | "lsp" | "ls" | "cf" | "pf" | "bp" | "pp" => {
variant_code = variant.clone();
}
_ => codec_hint = vocab::codec(&variant).to_string(),
}
}
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type,
language,
@@ -426,6 +325,68 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `menu_base.prop` lines are skipped when `is_empty() ||
/// starts_with('#')` — either alone is sufficient. A commented-out
/// key=value line must never be parsed into an entry.
/// Mutation: `||` -> `&&` requires both, which a non-empty comment
/// line can't satisfy, so it falls through to `line.find('=')` and
/// gets parsed as a real property.
#[test]
fn menu_base_comment_line_with_equals_is_still_skipped() {
let labels = parse_props(
"#audio_1.class=AudioButton\n\
#audio_1.streamNumber=9\n\
#audio_1.name=Should Not Appear\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=1\n\
audio_2.name=Real Track\n",
);
assert_eq!(labels.len(), 1, "commented-out entry must not be parsed");
assert_eq!(labels[0].name, "Real Track");
}
/// Spec: `menu_base.prop` streamNumber (or audioStream/subtitleStream)
/// must be strictly positive — `0` means "no STN entry" and must be
/// skipped, matching the `n > 0` guard on the language_streams side.
/// Mutation: `n > 0` -> `n >= 0` (or the guard deleted) would let a
/// stream_num of 0 through, emitting a dead label apply_labels can
/// never match (its counter starts at 1).
#[test]
fn menu_base_zero_stream_number_skipped() {
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_1.name=Disabled Slot\n",
);
assert!(
labels.is_empty(),
"streamNumber=0 must be skipped, got {labels:?}"
);
}
/// Spec: `is_subtitle` is `class.contains("SubtitleButton") ||
/// prefix.starts_with("subtitle_")` — EITHER signal alone is
/// sufficient to classify (and keep) a subtitle entry whose prefix
/// doesn't follow the `subtitle_` naming convention.
/// Mutation: `||` -> `&&` would require BOTH signals; an entry whose
/// class says SubtitleButton but whose prefix is something else
/// (e.g. a vendor-specific button id) would then satisfy neither
/// `is_audio` nor `is_subtitle` and get dropped entirely.
#[test]
fn menu_base_subtitle_class_alone_is_sufficient() {
let labels = parse_props(
"menuBtn7.class=SubtitleButton\n\
menuBtn7.streamNumber=1\n\
menuBtn7.name=English SDH\n",
);
assert_eq!(
labels.len(),
1,
"class=SubtitleButton alone must classify as subtitle, not be dropped"
);
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
}
#[test]
fn prefix_commentary_segment_match_not_substring() {
// Genuine commentary group segments match.
@@ -441,6 +402,7 @@ mod tests {
fn lbl(t: StreamLabelType, n: u16, name: &str) -> StreamLabel {
StreamLabel {
stream_id: None,
stream_number: n,
stream_type: t,
language: String::new(),
@@ -452,6 +414,39 @@ mod tests {
}
}
/// Spec: `merge`'s `mb.iter().find(...)` must match an mb entry by
/// (stream_type AND stream_number) TOGETHER — either alone is not a
/// unique key (there can be an audio #1 and a subtitle #1, or two
/// different audio streams).
/// Mutation: `&&` -> `||` inside the closure would match on type OR
/// number alone, so `.find` (which returns the FIRST match) can pick
/// an mb entry with the right type but the WRONG stream number.
#[test]
fn merge_matches_mb_entry_by_type_and_number_together() {
// ls wants audio #2 (empty name, so it will borrow from mb).
let ls = vec![lbl(StreamLabelType::Audio, 2, "")];
// mb's FIRST audio entry is #1 (wrong number); its #2 entry (the
// real match) comes second.
let mb = vec![
lbl(StreamLabelType::Audio, 1, "Wrong Number Match"),
lbl(StreamLabelType::Audio, 2, "Correct Match"),
];
let merged = merge(ls, mb);
assert_eq!(
merged.len(),
2,
"mb's own audio #1 must also survive as its own entry"
);
let a2 = merged
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio && l.stream_number == 2)
.unwrap();
assert_eq!(
a2.name, "Correct Match",
"must match mb by (type AND number), not type or number alone"
);
}
#[test]
fn merge_preserves_menu_base_only_streams() {
// language_streams covers audio 1; menu_base has audio 1 (name)
@@ -521,6 +516,47 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::Forced);
}
/// Immunity pin against the defect measured in the `paramount` parser,
/// where a vendor `forced_sub` cell hung off a FULL dialogue track's own
/// slot to say "this track also contains forced signs", and reading that
/// cell as "this track is forced" flagged 30 MB dialogue tracks forced.
///
/// This format cannot express that. The forced signal is not a flag beside
/// a track's entry — it IS the entry's stream-kind token, drawn from a
/// closed vocabulary in which `subtitle_production` (the full dialogue
/// track) and `subtitle_narrative` (the forced-narrative track) are
/// mutually exclusive alternatives in the same position. A row is one or
/// the other; there is no cell a full track can carry to acquire the
/// qualifier, so the paramount failure mode has no encoding here.
///
/// Mutation: give `subtitle_production` a `Forced` qualifier, or add a
/// forced side-flag that both kinds may carry.
#[test]
fn a_full_subtitle_track_kind_can_never_carry_the_forced_qualifier() {
// Every subtitle kind in the vocabulary, one row each.
let text = "id1,subtitle_production,1,eng\n\
id2,subtitle_commentary,2,eng\n\
id3,subtitle_dual,3,eng\n\
id4,subtitle_bonus,4,eng\n\
id5,subtitle_ime,5,kor\n\
id6,subtitle_narrative,6,eng\n\
id7,subtitle_ime_narrative,7,kor\n";
let labels = parse_language_streams_text(text);
let forced: Vec<&str> = labels
.iter()
.filter(|l| l.qualifier == LabelQualifier::Forced)
.map(|l| l.language.as_str())
.collect();
assert_eq!(
forced.len(),
2,
"only the two narrative kinds are forced, got {forced:?}"
);
// The full dialogue kind specifically.
let production = parse_language_streams_text("id,subtitle_production,1,eng\n");
assert_eq!(production[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_commentary` → Subtitle / Commentary.
/// Mutation: treat as Normal → subtitle commentary not flagged.
#[test]
@@ -564,6 +600,72 @@ mod tests {
assert!(labels.is_empty());
}
/// Immunity pin. `language_streams.txt` states each stream's number in
/// field 3, so a row the parser cannot use is simply dropped — it can
/// never renumber the rows behind it. This is the property that keeps
/// this parser out of the STN-slot-shifting failure mode that bites
/// parsers which count positionally: there, a skipped entry silently
/// pulls every later label one stream forward.
///
/// Mutation: replace `parts[2]` with a running per-type counter → the
/// three unusable rows here collapse the survivors onto 1/2 and 1.
#[test]
fn ls_stream_numbers_come_from_the_row_not_a_counter() {
let labels = parse_language_streams_text(
"id,audio_production,4,eng\n\
id,audio_bonus_extended,5,eng\n\
id,audio_production,0,fra\n\
id,audio_production,7,fra\n\
id,subtitle_production\n\
id,subtitle_narrative,9,deu\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 4),
(StreamLabelType::Audio, 7),
(StreamLabelType::Subtitle, 9),
],
"an unusable row drops out without shifting the numbering"
);
assert_eq!(labels[2].qualifier, LabelQualifier::Forced);
}
/// Immunity pin, `menu_base.prop` side: the number comes from the
/// entry's own `streamNumber` property, so a skipped entry (commented
/// out, `streamNumber=0`, neither audio nor subtitle) leaves the
/// surviving entries on their authored slots.
///
/// Mutation: number by iteration order → the survivors collapse to 1/2.
#[test]
fn menu_base_stream_numbers_come_from_the_entry_not_a_counter() {
let labels = parse_props(
"#audio_0.class=AudioButton\n\
#audio_0.streamNumber=1\n\
audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=6\n\
other_1.class=SomeOtherButton\n\
other_1.streamNumber=2\n\
subtitle_1.class=SubtitleButton\n\
subtitle_1.streamNumber=11\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![(StreamLabelType::Audio, 6), (StreamLabelType::Subtitle, 11),],
"skipped entries must not renumber the ones that survive"
);
}
/// Spec: `eda` variant → `Descriptive` purpose.
/// Mutation: miss the `eda` branch → purpose stays Normal.
#[test]
@@ -608,6 +710,60 @@ mod tests {
assert_eq!(labels[0].language, "eng");
}
/// Spec: the skip test is `is_empty() || starts_with('#')` — EITHER
/// condition alone must skip the line. A commented-out line that
/// happens to look like valid CSV (a real authoring pattern for
/// disabling a stream entry) must never produce a label.
/// Mutation: `||` -> `&&` requires BOTH conditions, which a non-empty
/// comment line can never satisfy, so it would fall through to the
/// CSV parser and (since it has >= 4 comma fields) emit a spurious
/// label instead of being skipped.
#[test]
fn ls_comment_line_with_csv_shape_is_still_skipped() {
let labels =
parse_language_streams_text("#id,audio_production,1,eng\nid2,audio_production,2,fra\n");
assert_eq!(
labels.len(),
1,
"the commented-out CSV-shaped line must not parse"
);
assert_eq!(labels[0].language, "fra");
}
/// Spec: `subtitle_dual` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → falls to the
/// catch-all `_ => continue`, silently dropping the stream.
#[test]
fn ls_subtitle_dual_parsed() {
let labels = parse_language_streams_text("id,subtitle_dual,1,eng\n");
assert_eq!(labels.len(), 1, "subtitle_dual must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_bonus` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_bonus_parsed() {
let labels = parse_language_streams_text("id,subtitle_bonus,2,eng\n");
assert_eq!(labels.len(), 1, "subtitle_bonus must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
}
/// Spec: `subtitle_ime` maps to Subtitle/Ime (no Forced qualifier,
/// unlike `subtitle_ime_narrative`).
/// Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_ime_parsed() {
let labels = parse_language_streams_text("id,subtitle_ime,3,jpn\n");
assert_eq!(labels.len(), 1, "subtitle_ime must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Ime);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: multiple valid lines produce multiple labels.
/// Mutation: stop after first label → only 1 label returned.
#[test]
@@ -756,6 +912,7 @@ fn parse_menu_base_text(text: &str) -> Vec<StreamLabel> {
.unwrap_or_default();
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type,
language,
+265 -8
View File
@@ -93,6 +93,45 @@ fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
out
}
/// Cap on the bytes retained for one stream label.
///
/// The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
/// whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
/// constant contributes up to 65535 bytes, and the `u16` stream-number
/// keyspace admits 65536 of them per type.
///
/// Headroom: real dbp menu labels are short display names — "English Dolby
/// Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
/// retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
/// bytes. 256 leaves >5x headroom over that, and any string past it is menu
/// geometry or padding, never a language name — `vocab::lang` would not
/// resolve it anyway.
const MAX_LABEL_BYTES: usize = 256;
/// Cap on retained stream slots per type.
///
/// The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
/// type are reachable; paired with [`MAX_LABEL_BYTES`] this bounds the whole
/// scan at 2 x 512 x 256 bytes.
///
/// Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
/// streams per playlist, and dbp emits one menu TextField per stream. 512
/// leaves 16x headroom over the spec maximum.
const MAX_LABELS_PER_TYPE: usize = 512;
/// Record `label` for stream `n`, honouring the retention caps. Existing
/// slots are still overwritten at the cap so the documented last-write-wins
/// behaviour is preserved; only NEW slots are refused.
fn retain_label(map: &mut BTreeMap<u16, String>, n: u16, label: &str) {
if label.len() > MAX_LABEL_BYTES {
return;
}
if map.len() >= MAX_LABELS_PER_TYPE && !map.contains_key(&n) {
return;
}
map.insert(n, label.to_string());
}
fn collect_textfield(
s: &str,
audios: &mut BTreeMap<u16, String>,
@@ -112,15 +151,15 @@ fn collect_textfield(
}
if let Some(rest) = kind_n.strip_prefix("Audio") {
if let Ok(n) = rest.parse::<u16>() {
audios.insert(n, label.to_string());
retain_label(audios, n, label);
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle") {
if let Ok(n) = rest.parse::<u16>() {
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
subs.insert(n, label.to_string());
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle")
&& let Ok(n) = rest.parse::<u16>()
{
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
retain_label(subs, n, label);
}
}
}
@@ -132,6 +171,7 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
let qualifier = vocab::qualifier(&label);
let purpose = vocab::purpose(&label);
StreamLabel {
stream_id: None,
stream_number: num,
stream_type,
language,
@@ -147,6 +187,223 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
mod tests {
use super::super::{LabelPurpose, LabelQualifier};
use super::*;
use std::io::{Cursor, Write as _};
/// Build a minimal, structurally valid `.class` file (JVMS §4.1) whose
/// constant pool holds exactly the given `Utf8` strings (indices 1..=N,
/// no long/double slot padding needed for plain strings). No fields,
/// methods, interfaces, or attributes — `scan_jar`'s only interest is
/// the constant pool.
fn build_class(utf8_entries: &[&str]) -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(&0xCAFEBABEu32.to_be_bytes()); // magic
out.extend_from_slice(&0u16.to_be_bytes()); // minor_version
out.extend_from_slice(&52u16.to_be_bytes()); // major_version (Java 8)
out.extend_from_slice(&((utf8_entries.len() + 1) as u16).to_be_bytes()); // cp_count
for s in utf8_entries {
out.push(1); // CONSTANT_Utf8 tag
out.extend_from_slice(&(s.len() as u16).to_be_bytes());
out.extend_from_slice(s.as_bytes());
}
out.extend_from_slice(&0u16.to_be_bytes()); // access_flags
out.extend_from_slice(&0u16.to_be_bytes()); // this_class
out.extend_from_slice(&0u16.to_be_bytes()); // super_class
out.extend_from_slice(&0u16.to_be_bytes()); // interfaces_count
out.extend_from_slice(&0u16.to_be_bytes()); // fields_count
out.extend_from_slice(&0u16.to_be_bytes()); // methods_count
out.extend_from_slice(&0u16.to_be_bytes()); // attributes_count
out
}
/// Zip `entries` (name -> bytes) into an in-memory, Stored (uncompressed)
/// `jar::Jar` via the `zip` crate's own writer — a real archive, not a
/// hand-rolled central directory.
fn build_jar(entries: &[(&str, Vec<u8>)]) -> jar::Jar {
let mut buf = Vec::new();
{
let mut writer = zip::ZipWriter::new(Cursor::new(&mut buf));
let opts = zip::write::SimpleFileOptions::default()
.compression_method(zip::CompressionMethod::Stored);
for (name, data) in entries {
writer.start_file(*name, opts).expect("start_file");
writer.write_all(&data[..]).expect("write class bytes");
}
writer.finish().expect("finish zip");
}
zip::ZipArchive::new(Cursor::new(buf)).expect("valid zip")
}
/// `scan_jar` wires together `for_each_class`, constant-pool iteration,
/// `collect_textfield`, and `make_label` into the actual per-jar scan
/// used by `parse`. The pure `collect_textfield`/`make_label` unit
/// tests above don't exercise this wiring at all.
///
/// Mutation: replace the whole function body with `vec![]` — every
/// dbp disc would silently lose all its stream labels regardless of
/// what's in the jar.
#[test]
fn scan_jar_extracts_labels_from_real_class_entries() {
let class_bytes = build_class(&[
"com/dbp/Whatever", // unrelated string — must be ignored
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
"HTextField,Subtitle1,English SDH,Fontstrip_Composite,1312,763",
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843", // disable button, skipped
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
assert_eq!(
labels.len(),
2,
"expected one audio + one real subtitle label"
);
let audio = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio)
.expect("audio label present");
assert_eq!(audio.stream_number, 1);
assert_eq!(audio.language, "eng");
let sub = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Subtitle)
.expect("subtitle label present");
assert_eq!(sub.stream_number, 1);
assert_eq!(sub.qualifier, LabelQualifier::Sdh);
}
/// Immunity pin. Every dbp label states its own slot in the `AudioN` /
/// `SubtitleN` token, so the numbering survives gaps and skipped entries
/// intact. Nothing here counts positionally, which is what keeps this
/// parser out of the failure mode where a skipped entry pulls every later
/// label one stream forward.
///
/// Mutation: number by iteration order → `Audio4` becomes 2 and
/// `Subtitle3` becomes 1, silently rebinding both to other streams.
#[test]
fn stream_numbers_come_from_the_token_not_iteration_order() {
let class_bytes = build_class(&[
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
// Slots 2 and 3 have no menu TextField authored.
"LTextField,Audio4,French 5.1 Dolby Digital,Fontstrip_Composite,296,803",
// Not a stream: the disable-subtitles button.
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843",
// Unparseable slot token — dropped, and must shift nothing.
"HTextField,SubtitleX,German,Fontstrip_Composite,1312,883",
"HTextField,Subtitle3,English SDH,Fontstrip_Composite,1312,763",
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 1),
(StreamLabelType::Audio, 4),
(StreamLabelType::Subtitle, 3),
],
"unlabelled and unusable slots leave the authored numbers alone"
);
}
/// A `CONSTANT_Utf8_info` carries a `u16` length (JVMS §4.4.7), so one
/// crafted constant contributes up to 65535 bytes and the `u16` stream
/// keyspace admits 65536 slots per type — ~4 GiB of retained `String` per
/// map from a jar that is orders of magnitude smaller.
///
/// Boundary literals, not the constant: a 256-byte label is kept, 257 and
/// the JVMS maximum 65535 are refused.
#[test]
fn oversized_labels_are_not_retained() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
collect_textfield(
&format!("XTextField,Audio1,{},rest", "A".repeat(256)),
&mut audios,
&mut subs,
);
assert_eq!(
audios.get(&1).map(String::len),
Some(256),
"a 256-byte label must still be retained"
);
collect_textfield(
&format!("XTextField,Audio2,{},rest", "A".repeat(257)),
&mut audios,
&mut subs,
);
assert!(!audios.contains_key(&2), "a 257-byte label must be refused");
collect_textfield(
&format!("XTextField,Subtitle1,{},rest", "B".repeat(65_535)),
&mut audios,
&mut subs,
);
assert!(
!subs.contains_key(&1),
"a JVMS-maximum 65535-byte Utf8 label must be refused"
);
}
/// The stream-slot keyspace is the full `u16` on both maps. Offer 600
/// distinct audio slots; exactly 512 are retained.
#[test]
fn retained_stream_slots_are_capped_per_type() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
assert_eq!(
audios.len(),
512,
"600 audio slots offered, {} retained — the slot count is unbounded",
audios.len()
);
}
/// Reaching the slot cap must not break the documented last-write-wins
/// behaviour for slots already held.
#[test]
fn existing_slot_is_still_overwritten_at_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
collect_textfield("XTextField,Audio1,Spanish,rest", &mut audios, &mut subs);
assert_eq!(audios.get(&1).map(String::as_str), Some("Spanish"));
}
/// Headroom: the longest plausible retail label must survive untouched.
#[test]
fn longest_realistic_label_survives_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
let real = "Portuguese (Brazilian) 5.1 Dolby Digital Plus";
assert_eq!(real.len(), 45, "fixture length changed");
collect_textfield(
&format!("XTextField,Audio1,{real},Fontstrip_Composite,296,763"),
&mut audios,
&mut subs,
);
assert_eq!(audios.get(&1).map(String::as_str), Some(real));
}
#[test]
fn collect_extracts_audio_and_subtitle_indices() {
+1632 -39
View File
File diff suppressed because it is too large Load Diff
+21
View File
@@ -216,6 +216,27 @@ mod tests {
ZipArchive::new(Cursor::new(bytes)).expect("valid zip")
}
/// The doc comment states the cap is 64 MiB. Pin the exact numeric
/// value (not derived from the same `64 * 1024 * 1024` expression
/// under test — a hardcoded literal) so a mutation of the arithmetic
/// (e.g. `*` -> `+`) is caught even though no test builds an actual
/// 64 MiB buffer.
#[test]
fn max_class_bytes_is_64_mebibytes() {
assert_eq!(MAX_CLASS_BYTES, 67_108_864);
}
#[test]
fn has_path_prefix_matches_only_declared_prefix() {
let jar = open(build_stored_zip(
"com/dbp/Loader.class",
MINIMAL_CLASS,
MINIMAL_CLASS.len() as u32,
));
assert!(has_path_prefix(&jar, "com/dbp/"));
assert!(!has_path_prefix(&jar, "com/bydeluxe/"));
}
#[test]
fn try_each_class_reads_minimal_class() {
let mut jar = open(build_stored_zip(
+1928 -224
View File
File diff suppressed because it is too large Load Diff
+245 -119
View File
@@ -61,22 +61,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
let mut labels: Vec<StreamLabel> = Vec::new();
// (stream_type_tag, language, codec_hint, pid) — PID is the
// canonical "same physical stream" key; type+lang+codec round
// out the rare case where two distinct logical streams happen
// to share a PID across playlists with different metadata.
let mut seen: Vec<(StreamLabelType, String, String, u16)> = Vec::new();
// Global 1-based counters keyed by StreamLabelType. Incremented
// only when an entry survives dedup, so stream_numbers are dense
// (1, 2, 3, ...) per type across the whole disc — not reset per
// playlist. A disc with 2 MPLS files that each list the same
// 8 audio streams ends up with audio_1..audio_8, not audio_1..
// audio_16 or audio_1..audio_8 with audio_1 duplicated.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
let mut playlists: Vec<crate::mpls::Playlist> = Vec::new();
for name in &mpls_names {
let path = format!("/BDMV/PLAYLIST/{}", name);
let Ok(data) = udf.read_file(reader, &path) else {
@@ -85,28 +70,66 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let Ok(playlist) = crate::mpls::parse(&data) else {
continue;
};
playlists.push(playlist);
}
let labels = build_labels(&playlists);
if labels.is_empty() {
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
Some(ParseResult::low(labels))
}
/// Convert every stream entry across `playlists` into one [`StreamLabel`] per
/// physical stream. Factored out of [`parse`] so unit tests can drive the
/// actual conversion logic (stream-type mapping, identity, slot numbering)
/// directly from already-parsed [`crate::mpls::Playlist`] values, without
/// needing a synthetic on-disc UDF image.
///
/// Identity is `(clip, PID)` — what the STN entry states — and it is both the
/// dedup key and the label's [`StreamId`]. A stream twenty playlists list is
/// one label; two clips that both open their first audio at 0x1100 are two.
/// This replaced a disc-global dense counter that numbered surviving entries
/// 1, 2, 3, … in playlist-directory order: that number was not an STN slot in
/// anything, but it was handed to a binder that reads `stream_number` as one.
fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
use std::collections::HashSet;
let mut labels: Vec<StreamLabel> = Vec::new();
let mut seen: HashSet<super::StreamId> = HashSet::new();
for playlist in playlists {
// `Playlist::streams` is the FIRST play item's STN table, so every
// entry here is a stream of that play item's clip — the same clip
// `disc::bluray` records as the title's `clips[0]`. That pairing is
// what makes the PID an identity rather than a 16-bit number.
//
// Streams cannot be non-empty without a play item to have read them
// from, so the empty case is unreachable on a real disc; entries we
// cannot identify are skipped rather than emitted as unbindable
// labels.
let Some(clip_id) = playlist.play_items.first().map(|pi| pi.clip_id.clone()) else {
continue;
};
// 1-based STN slot within THIS playlist's table, per type — the
// `stream_number` field's documented meaning, counted the same way
// `disc::bluray` counts the stream list it builds from these entries.
// Nothing binds through it (these labels bind by id); it is stated
// truthfully rather than invented so that a reader of the label list
// sees where on its own playlist each stream sits.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for entry in &playlist.streams {
let label_type = match entry.stream_type {
2 | 5 => StreamLabelType::Audio, // primary + secondary audio
3 => StreamLabelType::Subtitle, // PG subtitle
// 1 = primary video, 6 = secondary video, 7 = DV EL
// → no StreamLabelType variant for video, skip.
// 4 = IG (interactive graphics) — not a user-facing
// stream, skip.
_ => continue,
};
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
let key = (label_type, language.clone(), codec_hint.clone(), entry.pid);
if seen.contains(&key) {
let Some(label_type) = label_type_for(entry) else {
continue;
}
seen.push(key);
};
let stream_number = match label_type {
StreamLabelType::Audio => {
audio_idx += 1;
@@ -118,7 +141,20 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
};
let stream_id = super::StreamId {
clip_id: clip_id.clone(),
pid: entry.pid,
};
if !seen.insert(stream_id.clone()) {
continue;
}
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
labels.push(StreamLabel {
stream_id: Some(stream_id),
stream_number,
stream_type: label_type,
language,
@@ -130,17 +166,38 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
});
}
}
labels
}
if labels.is_empty() {
/// Which per-type numbering list an STN entry belongs to, or `None` when it
/// is not a labellable stream at all.
///
/// This MUST agree with the stream list `disc::bluray` builds from the same
/// entries, because that list is what `labels::apply_labels` counts against
/// when it binds `stream_number`. The two counters run over the same STN
/// entries in the same order, so any entry one side keeps and the other drops
/// — or files under a different type — shifts every later label of that type
/// onto the wrong stream. Three rules, all mirroring `disc::bluray`:
///
/// * `coding_type == 0` is the STN table's empty/padding slot. Not a
/// stream on either side.
/// * a PG coding_type in an audio STN slot is a subtitle, not audio.
/// `mpls::parse_stream_entry` has a dedicated arm for this layout, so it
/// is an authored shape rather than a corruption.
/// * video (1 / 6 / 7 = primary, secondary, Dolby Vision EL) and IG (4)
/// have no `StreamLabelType`; they are numbered in their own STN lists
/// and never interleave with the audio or PG lists.
fn label_type_for(entry: &crate::mpls::StreamEntry) -> Option<StreamLabelType> {
use crate::consts::coding_type as c;
if entry.coding_type == 0 {
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
Some(ParseResult::low(labels))
match entry.stream_type {
2 | 5 if entry.coding_type == c::PG => Some(StreamLabelType::Subtitle),
2 | 5 => Some(StreamLabelType::Audio),
3 => Some(StreamLabelType::Subtitle),
_ => None,
}
}
fn has_mpls_extension(name: &str) -> bool {
@@ -327,69 +384,37 @@ mod tests {
}
}
/// A playlist over clip "00001". `Playlist::streams` is read out of the
/// first play item's STN table, so a playlist that has streams always has
/// a play item to have read them from — the fixture carries one so tests
/// exercise the shape production sees, and so each label gets the
/// `(clip, PID)` identity it is bound by.
fn playlist_with(streams: Vec<StreamEntry>) -> Playlist {
playlist_on("00001", streams)
}
fn playlist_on(clip_id: &str, streams: Vec<StreamEntry>) -> Playlist {
Playlist {
version: "0200".to_string(),
play_items: Vec::new(),
play_items: vec![crate::mpls::PlayItem {
clip_id: clip_id.to_string(),
in_time: 0,
out_time: 0,
connection_condition: 1,
}],
streams,
marks: Vec::new(),
}
}
/// Drive the same conversion logic that `parse()` runs on real
/// disc data, but starting from already-parsed Playlists so we
/// don't have to synthesize valid MPLS bytes.
/// Drive the actual production conversion logic (`build_labels`, the
/// function `parse()` calls) starting from already-parsed Playlists,
/// so tests don't have to synthesize valid on-disc MPLS/UDF bytes.
/// This calls the *real* code under test rather than a hand-written
/// re-implementation, so mutations inside `build_labels` (stream-type
/// mapping, dedup key, counters) are actually caught here.
fn labels_from_playlists(playlists: &[Playlist]) -> Vec<StreamLabel> {
let mut labels: Vec<StreamLabel> = Vec::new();
let mut seen: Vec<(StreamLabelType, String, String, u16)> = Vec::new();
// Global counters hoisted OUT of the playlist loop to match
// production `parse()` (lines 77-78): stream_numbers are dense
// per type across the whole disc, not reset per playlist.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for playlist in playlists {
for entry in &playlist.streams {
let label_type = match entry.stream_type {
2 | 5 => StreamLabelType::Audio,
3 => StreamLabelType::Subtitle,
_ => continue,
};
// Dedup BEFORE consuming a counter value, matching prod
// parse() ordering so a deduped duplicate does not burn a
// stream number.
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
let key = (label_type, language.clone(), codec_hint.clone(), entry.pid);
if seen.contains(&key) {
continue;
}
seen.push(key);
let stream_number = match label_type {
StreamLabelType::Audio => {
audio_idx += 1;
audio_idx
}
StreamLabelType::Subtitle => {
sub_idx += 1;
sub_idx
}
};
labels.push(StreamLabel {
stream_number,
stream_type: label_type,
language,
name,
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::None,
codec_hint,
variant: String::new(),
});
}
}
labels
build_labels(playlists)
}
#[test]
@@ -445,31 +470,114 @@ mod tests {
assert_eq!(labels[2].language, "fra");
}
/// `stream_number` is bound by `labels::apply_labels` against the title's
/// own stream list, which `disc::bluray` builds from these same STN
/// entries. That builder DROPS an entry whose `coding_type` is 0 — the
/// STN table's empty/padding slot — so it must not be counted here
/// either. Counting it advances the audio counter past a stream that
/// never materializes, and every label behind it binds one stream late.
#[test]
fn dedup_streams_across_playlists() {
// Two playlists, same English TrueHD 7.1 PID 0x1100 in both.
// Expect one Audio label, not two.
let pl1 = playlist_with(vec![
fn padding_stn_entry_does_not_consume_a_label_slot() {
let pl = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
audio_entry(0x1101, 0x81, 6, 1, "fra"),
// coding_type 0: STN padding. Not a stream.
audio_entry(0x1101, 0x00, 0, 0, ""),
audio_entry(0x1102, 0x81, 6, 1, "fra"),
]);
let pl2 = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), // duplicate
audio_entry(0x1102, 0x82, 6, 1, "deu"), // new
]);
let labels = labels_from_playlists(&[pl1, pl2]);
// Expected: eng@0x1100, fra@0x1101, deu@0x1102 — three uniques.
assert_eq!(labels.len(), 3);
// PID isn't stored on StreamLabel, so assert on the surviving
// language set instead.
let mut langs: Vec<String> = labels.iter().map(|l| l.language.clone()).collect();
langs.sort();
assert_eq!(langs, vec!["deu", "eng", "fra"]);
let labels = labels_from_playlists(&[pl]);
assert_eq!(labels.len(), 2, "the padding slot yields no label");
assert_eq!(labels[0].language, "eng");
assert_eq!(labels[0].stream_number, 1);
assert_eq!(labels[1].language, "fra");
assert_eq!(
labels[1].stream_number, 2,
"padding is absent from the title's stream list, so `fra` is \
audio stream 2"
);
}
// Stream numbers must be DENSE and GLOBAL across playlists, not
// reset per playlist. eng (pl1) = 1, fra (pl1) = 2, the duplicate
// eng in pl2 is deduped (no number consumed), and deu (pl2) = 3.
// Regression guard for the per-playlist counter-reset divergence.
/// A PG coding_type sitting in an audio STN slot is a real, documented
/// shape — `mpls::parse_stream_entry` has an explicit arm for it, and
/// `disc::bluray` builds it as a Subtitle stream, not an Audio one. This
/// module must classify it the same way, or the audio counter runs one
/// ahead and the subtitle counter one behind for every later stream.
#[test]
fn pg_coding_type_in_an_audio_slot_counts_as_a_subtitle() {
let mut misplaced = audio_entry(0x1200, 0x90, 0, 0, "spa");
misplaced.stream_type = 2;
let pl = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
misplaced,
audio_entry(0x1101, 0x81, 6, 1, "fra"),
pg_entry(0x1201, "deu"),
]);
let labels = labels_from_playlists(&[pl]);
let audio: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.map(|l| (l.language.as_str(), l.stream_number))
.collect();
assert_eq!(
audio,
vec![("eng", 1), ("fra", 2)],
"the PG entry is not an audio stream and must not number one"
);
let sub: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.map(|l| (l.language.as_str(), l.stream_number))
.collect();
assert_eq!(
sub,
vec![("spa", 1), ("deu", 2)],
"it is subtitle stream 1, ahead of the PG-slot entry"
);
}
/// Two playlists over the SAME clip that both list PID 0x1100: one
/// physical stream, so one label. Identity is `(clip, PID)`, and each
/// label states the STN slot it holds in its own playlist.
#[test]
fn one_label_per_stream_across_playlists_on_the_same_clip() {
let pl1 = playlist_on(
"00001",
vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
audio_entry(0x1101, 0x81, 6, 1, "fra"),
],
);
let pl2 = playlist_on(
"00001",
vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), // same stream
audio_entry(0x1102, 0x82, 6, 1, "deu"), // new
],
);
let labels = labels_from_playlists(&[pl1, pl2]);
assert_eq!(
labels.len(),
3,
"eng/fra/deu — the duplicate eng is one stream"
);
let id = |lang: &str| {
labels
.iter()
.find(|l| l.language == lang)
.and_then(|l| l.stream_id.clone())
.map(|i| (i.clip_id, i.pid))
};
assert_eq!(id("eng"), Some(("00001".into(), 0x1100)));
assert_eq!(id("fra"), Some(("00001".into(), 0x1101)));
assert_eq!(id("deu"), Some(("00001".into(), 0x1102)));
// `stream_number` is the entry's slot in ITS OWN playlist's STN table
// — deu is pl2's second audio, so 2, not "the third distinct stream
// seen while scanning the disc". It used to be the latter: a dense
// disc-global counter that named no table anyone could count against,
// handed to a binder that reads the field as an STN slot.
let num = |lang: &str| {
labels
.iter()
@@ -478,7 +586,25 @@ mod tests {
};
assert_eq!(num("eng"), Some(1));
assert_eq!(num("fra"), Some(2));
assert_eq!(num("deu"), Some(3));
assert_eq!(num("deu"), Some(2), "pl2's second audio slot");
}
/// The same PID in two DIFFERENT clips is two different streams — a PID is
/// only unique within one clip. Deduping on the PID alone (as the old
/// key's `(type, language, codec_hint, pid)` did across clips) collapses
/// them into one label, and the second clip's stream is then described by
/// the first clip's.
#[test]
fn same_pid_in_two_clips_is_two_streams() {
let pl1 = playlist_on("00001", vec![audio_entry(0x1100, 0x83, 12, 1, "eng")]);
let pl2 = playlist_on("00002", vec![audio_entry(0x1100, 0x83, 12, 1, "eng")]);
let labels = labels_from_playlists(&[pl1, pl2]);
assert_eq!(labels.len(), 2, "different clips: two distinct streams");
let clips: Vec<String> = labels
.iter()
.filter_map(|l| l.stream_id.as_ref().map(|i| i.clip_id.clone()))
.collect();
assert_eq!(clips, vec!["00001", "00002"]);
}
#[test]
+563 -59
View File
@@ -3,18 +3,28 @@
//! Richest structured format. Complete language lists with forced flags
//! and commentary indices per playlist, all in XML attributes.
//!
//! NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this
//! file is one authoring house's internal metadata that happens to ship on
//! the pressing. There is nothing to look up: every field meaning here was
//! derived by measuring real discs and cross-checking against per-display-set
//! content. Treat an unfamiliar value as unknown rather than guessing — the
//! disc's own `forced_on_flag` is the only authoritative forced signal.
//!
//! ```xml
//! <playlist name="Feature" id="00222"
//! aud="eng,deu,spa,spa,fra"
//! sub="eng,eng,zho,ces,dan"
//! forced_sub="0,0,0,1,0"
//! forced_sub="0,0,0,1,3"
//! aud_com1_idx="10"
//! sub_com1_idx="23,24,25" />
//! ```
//!
//! `forced_sub` is an ENUMERATION, not a boolean — see [`ForcedSub`].
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorSource;
use crate::udf::UdfFs;
use std::collections::HashSet;
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "playlists.xml")
@@ -32,11 +42,138 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
if labels.is_empty() {
return None;
}
// High confidence: paramount's playlists.xml is fully structured
// and we extract every documented field.
// High confidence: this format is fully structured and we extract
// every field whose meaning the corpus establishes. "Documented" would
// be the wrong word — see the module note; nothing about it is.
Some(ParseResult::high(labels))
}
/// One cell of the `forced_sub` CSV.
///
/// The attribute reads like a boolean and was parsed as one (`cell == "1"` →
/// forced). It is not. Every image in the corpus carrying this vendor's
/// `playlists.xml` — seven distinct discs — uses four values, and decoding
/// three of those discs' feature subtitle tracks and counting every PGS
/// display set separates them into two populations two orders of magnitude
/// apart:
///
/// * `0` — a subtitle track with no forced-narrative content. On the two
/// discs measured that use the flag at all, not one `0` track carried a
/// single `forced_on_flag` display set.
/// * `1` — a FULL DIALOGUE track that additionally contains some
/// forced-narrative signs. On one measured disc, all nine `1` cells are
/// full tracks of 949-1411 display sets, eight of them carrying 5-14
/// flagged sets and the ninth none; that disc has no dedicated forced
/// track at all. On another, all seven `1` cells are full tracks of
/// 1602-1651 display sets carrying 0-31 flagged sets. Reading `1` as
/// forced is what made one language present as two identical full
/// subtitle tracks with one of them flagged forced.
/// * `2` and `3` — a DEDICATED forced-narrative track. These take their own
/// trailing STN slots, one per localized language, duplicating a language
/// that already holds a full track earlier in the list. Measured: the two
/// `2` slots on one disc are 15 and 10 display sets, EVERY one flagged
/// forced, against ~1600 on that disc's full tracks; the four `3` slots on
/// another are 7, 14, 23 and 59 display sets against 1216-2655. What
/// distinguishes `2` from `3` the corpus does not reveal — both sit in the
/// same trailing position, both measure the same shape, and one disc uses
/// each for a different language — so both map alike.
///
/// So the old reading was wrong in BOTH directions: it flagged full dialogue
/// tracks forced, and it discarded the cells that name the real forced tracks.
///
/// The `1` case is deliberately NOT carried through as a weaker "contains
/// forced segments" hint. There is no qualifier for that, and the asymmetry
/// argues against inventing one here: a wrong forced flag on a 30 MB dialogue
/// track is the user-visible defect, while a missing hint costs nothing.
///
/// An unrecognised cell maps to [`ForcedSub::None`] — the conservative
/// direction, since asserting forced is the expensive mistake.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum ForcedSub {
/// No forced-narrative content, or an unrecognised cell.
None,
/// A full dialogue track that also carries forced-narrative segments.
ContainsForcedSegments,
/// A dedicated forced-narrative track.
ForcedNarrative,
}
/// The one number behind every cap in this parser: the highest CSV cell
/// position that can ever be addressed.
///
/// The labelling loops number cells 1-based into a `u16` and `break` at
/// `u16::try_from(i + 1)`, so cell `MAX_COM_INDICES` and everything past it is
/// never visited. Two different things are measured against that, and they are
/// not the same bound:
///
/// - **A VALUE at or beyond it cannot match any cell.** This is what caps the
/// set: values are filtered before insertion, so at most `MAX_COM_INDICES`
/// distinct entries can ever be stored, however long the attribute is. The
/// `HashSet` that replaced a linear scan fixed the LOOKUP cost; this is what
/// fixes the ALLOCATION, and a disc declaring half a billion indices no
/// longer costs half a billion entries.
/// - **A POSITION at or beyond it describes nothing new.** This caps the WORK,
/// not the memory — the value filter already made the set small, but without
/// it every one of those half a billion cells is still split and parsed. It
/// is an early exit at the first position whose contents provably cannot
/// matter, and it is why `forced_sub` — which holds no set at all, and so
/// gets no protection from the value rule — is bounded too.
///
/// Real authoring is nowhere near either limit: the BD STN table admits at
/// most 32 streams per playlist, so nothing legitimate is lost.
const MAX_COM_INDICES: usize = u16::MAX as usize;
/// Parse a `*_com1_idx` attribute into the set the labelling loops query.
///
/// Extracted so the BOUND is observable. Asserting it through
/// `labels_from_feature` is not possible: a `HashSet` collapses repeated
/// values, and an out-of-range index changes no label either way, so such a
/// test passes whether or not the cap exists — an assertion that cannot fail.
/// Returning the set lets a test hand in tens of thousands of DISTINCT
/// unaddressable indices and see them refused.
fn com_indices(attr: Option<String>) -> HashSet<usize> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.filter_map(|i| i.trim().parse().ok())
.filter(|&i| i < MAX_COM_INDICES)
.collect()
})
.unwrap_or_default()
}
/// Parse the `forced_sub` attribute into the cell list the subtitle loop
/// queries — the third attacker-controlled CSV in this file, and the last one
/// that was still unbounded.
///
/// Bounded by POSITION, and it has no other choice: a `*_com1_idx` list holds
/// values that can be filtered, and that filter is what caps its set, but a
/// `forced_sub` cell is a classification of the position it sits at, so there
/// is nothing to filter and nothing else would ever cap this. The Vec is read
/// only as `forced.get(i)` from a loop that stops at `MAX_COM_INDICES`, so
/// every cell past that is unreachable by construction.
///
/// Extracted, like [`com_indices`], so the bound is OBSERVABLE. Through
/// `labels_from_feature` it is not: the subtitle loop cannot reach those cells
/// either, so a label-level assertion passes whether or not the cap exists.
fn forced_subs(attr: Option<String>) -> Vec<ForcedSub> {
attr.map(|s| {
s.split(',')
.take(MAX_COM_INDICES)
.map(forced_sub_cell)
.collect()
})
.unwrap_or_default()
}
fn forced_sub_cell(cell: &str) -> ForcedSub {
match cell.trim() {
"1" => ForcedSub::ContainsForcedSegments,
"2" | "3" => ForcedSub::ForcedNarrative,
_ => ForcedSub::None,
}
}
/// Build the stream labels from a single `<playlist .../>` feature
/// element. Split out from `parse` so the per-type numbering and
/// commentary/forced-index logic is unit-testable without a
@@ -49,17 +186,33 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// aud_com1_idx is a trimmed, comma-separated list of CSV positions
// (some authoring tools emit whitespace, and multiple commentary
// tracks are possible) — symmetric with sub_com1_idx below.
let com_indices: Vec<usize> = xml::attr(feature, "aud_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
// A HashSet, not a Vec: `com_indices` is parsed straight out of an
// attacker-controlled attribute with no length bound and was scanned
// linearly once per stream, so `aud="..."` and `aud_com1_idx="..."`
// both grown large make this quadratic in the size of one XML file.
// Membership is the only operation performed on it.
let com_indices = com_indices(xml::attr(feature, "aud_com1_idx"));
// stream_number must match apply_labels' monotonic 1-based
// per-type counter, which increments once per *real* stream — so
// it counts only non-empty slots, not the raw CSV index. The
// commentary index comparison stays on the raw CSV index `i`,
// since aud_com1_idx is positional against the original CSV.
let mut audio_num: u16 = 0;
// The CSV *is* the STN list: one cell per stream, in stream order,
// and `aud_com1_idx` is a 0-based index into those same cells. So
// `stream_number` is the cell's own 1-based position — NOT a counter
// that only advances on cells carrying a language.
//
// A cell with an empty language still occupies its STN slot; it just
// has nothing to label. Renumbering the surviving cells 1..N shifts
// every label behind an empty cell one slot forward, which is how a
// marker authored for one stream ends up written onto the stream in
// front of it (see the subtitle side, where the marker is `forced`).
//
// `u16::try_from` rather than `saturating_add`: past the 1-based u16
// numbering space every cell would collapse onto `u16::MAX`, binding
// several streams to one label. Stop emitting instead. Unreachable on
// real media — the BD STN_table admits at most 32 primary audio
// streams per playlist.
for (i, lang) in aud.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
};
let lang = lang.trim();
if lang.is_empty() {
continue;
@@ -69,9 +222,9 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
} else {
LabelPurpose::Normal
};
audio_num = audio_num.saturating_add(1);
labels.push(StreamLabel {
stream_number: audio_num,
stream_id: None,
stream_number,
stream_type: StreamLabelType::Audio,
language: lang.to_string(),
name: String::new(),
@@ -85,18 +238,21 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse subtitle streams
if let Some(sub) = xml::attr(feature, "sub") {
let forced: Vec<bool> = xml::attr(feature, "forced_sub")
.map(|s| s.split(',').map(|f| f.trim() == "1").collect())
.unwrap_or_default();
let forced = forced_subs(xml::attr(feature, "forced_sub"));
let com_indices: Vec<usize> = xml::attr(feature, "sub_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
// HashSet for the same reason as the audio side above: unbounded
// parsed input, membership-only use, linear scan once per stream.
let com_indices = com_indices(xml::attr(feature, "sub_com1_idx"));
// As with audio: count only non-empty slots for stream_number,
// but keep com/forced lookups on the raw CSV index `i`.
let mut sub_num: u16 = 0;
// As with audio: the cell position IS the STN slot. `forced_sub` and
// `sub_com1_idx` are indexed against those same cells, so an empty
// cell must not renumber the cells behind it — a forced marker
// authored for one PG slot would otherwise be written onto an
// earlier, full-dialogue subtitle track.
for (i, lang) in sub.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
};
let lang = lang.trim();
if lang.is_empty() {
continue;
@@ -108,15 +264,17 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
LabelPurpose::Normal
};
let qualifier = if forced.get(i).copied().unwrap_or(false) {
LabelQualifier::Forced
} else {
LabelQualifier::None
// Only a DEDICATED forced-narrative slot earns the forced flag.
// A cell marking a full track as merely containing forced segments
// is dropped, not weakened into a forced label (see [`ForcedSub`]).
let qualifier = match forced.get(i).copied().unwrap_or(ForcedSub::None) {
ForcedSub::ForcedNarrative => LabelQualifier::Forced,
ForcedSub::ContainsForcedSegments | ForcedSub::None => LabelQualifier::None,
};
sub_num = sub_num.saturating_add(1);
labels.push(StreamLabel {
stream_number: sub_num,
stream_id: None,
stream_number,
stream_type: StreamLabelType::Subtitle,
language: lang.to_string(),
name: String::new(),
@@ -142,10 +300,10 @@ fn find_feature_playlist(text: &str) -> Option<String> {
let element = &text[start..end];
// Prefer name="Feature" explicitly.
if let Some(name) = xml::attr(element, "name") {
if name.eq_ignore_ascii_case("Feature") {
return Some(element.to_string());
}
if let Some(name) = xml::attr(element, "name")
&& name.eq_ignore_ascii_case("Feature")
{
return Some(element.to_string());
}
// Otherwise pick the one with the most audio streams. Count only
@@ -168,6 +326,206 @@ fn find_feature_playlist(text: &str) -> Option<String> {
mod tests {
use super::*;
/// Immunity pin, section-boundary half. The pixelogic parser walks a flat
/// string sequence and recognises its feature section's END by marker
/// alone, so a section with no marker behind it runs off into whatever
/// follows and counts it as more STN slots. Nothing here can do that: the
/// stream list is one attribute of one XML element, so its length is the
/// CSV's own cell count and its scope is the element's byte range that
/// `xml::find_element` returns. Text after the element — including the
/// next playlist's own `aud` — is not reachable from it.
///
/// And when the boundary is MISSING the failure is closed, not open:
/// `xml::find_element` needs a matching close tag and yields `None`
/// without one, so an unterminated element ends the walk rather than
/// swallowing the rest of the document.
///
/// Mutation: hand `labels_from_feature` the document instead of the
/// element, or let an unterminated element run to EOF → the bonus
/// playlist's languages join the feature's stream list.
#[test]
fn a_playlists_stream_list_cannot_run_into_the_next_playlist() {
let doc = r#"
<playlist name="Feature" aud="eng,fra" sub="eng,spa" forced_sub="0,1"/>
<playlist name="Bonus" aud="deu,ita,jpn" sub="deu,ita,jpn"/>
"#;
let feature = find_feature_playlist(doc).expect("feature playlist found");
let labels = labels_from_feature(&feature);
let got: Vec<(StreamLabelType, u16, &str)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number, l.language.as_str()))
.collect();
assert_eq!(
got,
vec![
(StreamLabelType::Audio, 1, "eng"),
(StreamLabelType::Audio, 2, "fra"),
(StreamLabelType::Subtitle, 1, "eng"),
(StreamLabelType::Subtitle, 2, "spa"),
],
"the CSV's own cells are the whole stream list"
);
// Same document with the feature element left unterminated.
let unterminated = r#"
<playlist name="Feature" aud="eng,fra">
<playlist name="Bonus" aud="deu,ita,jpn"/>
"#;
assert!(
find_feature_playlist(unterminated).is_none(),
"a missing element boundary truncates the walk, never extends it"
);
}
/// `sub_com1_idx` is parsed straight out of the disc's `playlists.xml`,
/// which is attacker-controlled and has no length bound of its own.
///
/// This replaces a WALL-CLOCK test. That one built a 200 000 x 1 000 001
/// fixture and failed if it took over 10 s, to prove the membership test
/// was a set rather than a linear scan. Measured on the machine that
/// wrote this: 1.62 s alone, and OVER 10 s — a real failure — when the
/// suite's other 3 347 tests were running concurrently. A 6x margin
/// against a shared CPU is not a margin; it is a CI failure that looks
/// like a flake and gets re-run until it passes.
///
/// It also measured the wrong thing. Making the lookup O(1) bounded the
/// QUERY, not the PARSE: the set was still built from every entry the
/// disc declared, so a hostile playlist could still force an unbounded
/// allocation before any lookup happened. `MAX_COM_INDICES` bounds that.
///
/// What THIS test guards is that bounding did not change what a
/// legitimate playlist MEANS: it goes red if the bound is set too LOW
/// (verified at 2 — the real indices `0,2,4` stop resolving and the
/// purposes change). It does NOT go red if the bound is deleted
/// entirely, because the out-of-range filler is unobservable at the
/// label level and a `HashSet` collapses the repeats. Enforcement is
/// proven separately, by
/// `distinct_unaddressable_indices_are_refused_not_stored`, which reads
/// the set itself. Two tests, two properties; neither pretends to the
/// other's job.
#[test]
fn bounding_the_parse_does_not_change_a_legitimate_playlist() {
// Three real indices, then far more entries than can address a cell.
const OVERSIZED: usize = MAX_COM_INDICES + 10_000;
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(8));
feature.pop();
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(OVERSIZED));
feature.pop();
feature.push_str(r#"" />"#);
let labels = labels_from_feature(&feature);
// The fixture's real indices still decide the purposes: bounding the
// parse must not change what a legitimate playlist means.
assert_eq!(labels.len(), 8);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
/// The set REFUSES unaddressable indices, so a hostile playlist cannot
/// inflate it. DISTINCT values on purpose: a `HashSet` collapses repeats,
/// so a million copies of one index costs one entry and would prove
/// nothing. Fifty thousand distinct out-of-range indices cost fifty
/// thousand entries without the filter, and none with it — so this test
/// goes red if the bound is removed, which the label-level assertions
/// below cannot do.
#[test]
fn distinct_unaddressable_indices_are_refused_not_stored() {
let hostile: String = (MAX_COM_INDICES..MAX_COM_INDICES + 50_000)
.map(|i| i.to_string())
.collect::<Vec<_>>()
.join(",");
let set = com_indices(Some(hostile));
assert!(
set.is_empty(),
"kept {} unaddressable indices — the parse is still unbounded",
set.len()
);
// The addressable ones are still kept.
assert_eq!(com_indices(Some("0,2,4".to_string())).len(), 3);
}
/// `forced_sub` is bounded too — the third CSV in the same function, and
/// the one that had no value filter to hide behind.
///
/// Read through `forced_subs` rather than through the labels for the same
/// reason the two tests above read the set: the subtitle loop stops at
/// `MAX_COM_INDICES`, so a label-level assertion cannot tell a bounded
/// parse from an unbounded one.
#[test]
fn forced_sub_cells_past_the_last_addressable_one_are_not_parsed() {
let hostile = "0,".repeat(MAX_COM_INDICES + 50_000);
let cells = forced_subs(Some(hostile));
assert_eq!(
cells.len(),
MAX_COM_INDICES,
"parsed {} cells — the forced_sub parse is still unbounded",
cells.len()
);
}
/// Bounding it must not change what a legitimate playlist means: the
/// cells that CAN address a stream still classify exactly as before.
#[test]
fn bounding_forced_sub_leaves_the_addressable_cells_alone() {
let cells = forced_subs(Some("0,1,3,2".to_string()));
assert_eq!(
cells,
vec![
ForcedSub::None,
ForcedSub::ContainsForcedSegments,
ForcedSub::ForcedNarrative,
ForcedSub::ForcedNarrative,
]
);
}
/// An index that cannot address any cell is dropped rather than STORED.
///
/// Asserted through `com_indices`, not through the labels: the labelling
/// loop never queries a cell position that high, so at the label level
/// retaining the value is unobservable and the assertion could not fail.
/// Reading the set is what makes the claim checkable.
#[test]
fn an_index_that_cannot_address_any_cell_is_not_retained() {
let set = com_indices(Some(format!(
"1,{},{}",
MAX_COM_INDICES,
MAX_COM_INDICES + 1
)));
assert_eq!(
set.len(),
1,
"only the addressable index belongs in the set, got {set:?}"
);
assert!(set.contains(&1));
}
/// Headroom: the BD STN_table admits at most 32 PG streams per playlist,
/// and a real `sub_com1_idx` lists a handful of commentary tracks. The set
/// must behave identically to the old scan on real-shaped input.
#[test]
fn commentary_indices_still_match_on_real_shaped_input() {
let feature = r#"<playlist name="Feature" sub="eng,eng,zho,ces,dan" sub_com1_idx="1,3" />"#;
let labels = labels_from_feature(feature);
let purposes: Vec<LabelPurpose> = labels.iter().map(|l| l.purpose).collect();
assert_eq!(
purposes,
vec![
LabelPurpose::Normal,
LabelPurpose::Commentary,
LabelPurpose::Normal,
LabelPurpose::Commentary,
LabelPurpose::Normal,
]
);
}
fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
@@ -182,11 +540,56 @@ mod tests {
.collect()
}
/// The `aud` / `sub` CSVs are the vendor's STN-ordered stream lists: one
/// slot per stream, and `aud_com1_idx` / `forced_sub` are indexed against
/// those same slot positions. A slot whose language cell is empty carries
/// nothing to label but still OCCUPIES its slot, so it must not renumber
/// the slots behind it.
///
/// Numbering only the slots that carry a language collapsed every later
/// label one position forward per empty cell, which is how a forced
/// marker authored for one STN slot lands on the full-subtitle track in
/// front of it.
#[test]
fn empty_middle_slot_does_not_inflate_stream_number() {
// aud="eng,,fra": the empty middle slot is skipped, and the
// second real stream (fra) must be numbered 2, matching
// apply_labels' monotonic counter — not 3 (its raw CSV index).
fn empty_csv_slot_still_occupies_its_stn_slot() {
// Audio: slot 2 is empty; `fra` is STN slot 3 and is the commentary
// the vendor pointed at with the 0-based CSV index 2.
let feature = r#"<playlist name="Feature" aud="eng,,fra" aud_com1_idx="2" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
assert_eq!(a.len(), 2, "the empty slot carries no label");
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(
a[1].stream_number, 3,
"an empty CSV cell occupies STN slot 2, so `fra` is slot 3"
);
assert_eq!(a[1].purpose, LabelPurpose::Commentary);
// Subtitles: same shape, and the consequence is a misplaced forced
// flag. `forced_sub` index 2 is the forced-narrative track; with the
// empty slot renumbered away it would be written onto STN slot 2.
let feature = r#"<playlist name="Feature" sub="eng,,fra" forced_sub="0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 2);
assert_eq!(s[0].language, "eng");
assert_eq!(s[0].stream_number, 1);
assert_eq!(s[0].qualifier, LabelQualifier::None);
assert_eq!(s[1].language, "fra");
assert_eq!(
s[1].stream_number, 3,
"the forced marker belongs to STN slot 3, not slot 2"
);
assert_eq!(s[1].qualifier, LabelQualifier::Forced);
}
#[test]
fn empty_middle_slot_carries_no_label_but_keeps_its_slot() {
// aud="eng,,fra": the empty middle cell yields no label — there is
// nothing to label — but it still owns STN slot 2, so `fra` is slot
// 3. (This test previously asserted 2, pinning the renumbering bug.)
let feature = r#"<playlist name="Feature" aud="eng,,fra" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -194,7 +597,7 @@ mod tests {
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 2);
assert_eq!(a[1].stream_number, 3);
}
#[test]
@@ -202,7 +605,7 @@ mod tests {
// Whitespace around the index, and a multi-value list, must both
// resolve. com index is positional against the raw CSV, so with
// an empty slot at position 1, " 2 " marks the 'fra' track
// (CSV index 2) as commentary.
// (CSV index 2, STN slot 3) as commentary.
let feature = r#"<playlist aud="eng,,fra" aud_com1_idx=" 2 " />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -214,10 +617,10 @@ mod tests {
#[test]
fn forced_sub_aligns_with_raw_csv_index() {
// sub="eng,eng,zho,ces" forced_sub="0,0,0,1": the forced flag is
// sub="eng,eng,zho,ces" forced_sub="0,0,0,3": the forced marker is
// positional on the raw CSV, so 'ces' (index 3) is forced; its
// stream_number is its non-empty position (4 here, no gaps).
let feature = r#"<playlist sub="eng,eng,zho,ces" forced_sub="0,0,0,1" />"#;
// stream_number is its 1-based cell position, 4.
let feature = r#"<playlist sub="eng,eng,zho,ces" forced_sub="0,0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 4);
@@ -262,10 +665,12 @@ mod tests {
assert!(feature.contains(r#"name="MainMovie""#));
}
/// Spec: stream_number for audio is 1-based and increments only on non-empty slots.
/// Mutation: increment for empty slots too → stream numbers inflate.
/// Spec: stream_number for audio is the cell's own 1-based CSV position,
/// because the CSV is the STN list and empty cells are slots too.
/// Mutation: count only non-empty cells → every label behind an empty
/// cell shifts one slot forward.
#[test]
fn audio_stream_numbering_skips_empty_slots() {
fn audio_stream_numbering_uses_raw_csv_slot_position() {
let feature = r#"<playlist name="Feature" aud="eng,,fra,,spa" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -273,9 +678,9 @@ mod tests {
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 2);
assert_eq!(a[1].stream_number, 3);
assert_eq!(a[2].language, "spa");
assert_eq!(a[2].stream_number, 3);
assert_eq!(a[2].stream_number, 5);
}
/// Spec: forced subtitle at the last position with gaps in between.
@@ -283,9 +688,9 @@ mod tests {
/// Mutation: use stream_number (dense) instead of raw index → wrong subtitle forced.
#[test]
fn forced_sub_uses_raw_csv_index_with_gaps() {
// sub="eng,,fra,,spa" forced_sub="0,0,0,0,1"
// raw CSV index 4 = "spa"; stream_number for spa = 3 (3rd non-empty).
let feature = r#"<playlist name="Feature" sub="eng,,fra,,spa" forced_sub="0,0,0,0,1" />"#;
// sub="eng,,fra,,spa" forced_sub="0,0,0,0,3"
// raw CSV index 4 = "spa", i.e. STN slot 5.
let feature = r#"<playlist name="Feature" sub="eng,,fra,,spa" forced_sub="0,0,0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 3);
@@ -295,6 +700,7 @@ mod tests {
assert_eq!(s[1].qualifier, LabelQualifier::None);
assert_eq!(s[2].language, "spa");
assert_eq!(s[2].qualifier, LabelQualifier::Forced);
assert_eq!(s[2].stream_number, 5);
}
/// Spec: aud_com1_idx is positional against the raw CSV.
@@ -303,13 +709,14 @@ mod tests {
/// Mutation: use stream_number instead of raw CSV index → wrong stream is commentary.
#[test]
fn audio_commentary_index_raw_csv_position() {
// aud="eng,,fra,spa" aud_com1_idx="2" → CSV index 2 = "fra".
// "fra" is stream_number 2 (second non-empty slot, skipping the empty).
// aud="eng,,fra,spa" aud_com1_idx="2" → CSV index 2 = "fra",
// which is STN slot 3.
let feature = r#"<playlist name="Feature" aud="eng,,fra,spa" aud_com1_idx="2" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
assert_eq!(a.len(), 3);
assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 3);
assert_eq!(a[1].purpose, LabelPurpose::Commentary);
assert_eq!(a[0].purpose, LabelPurpose::Normal);
assert_eq!(a[2].purpose, LabelPurpose::Normal);
@@ -352,10 +759,11 @@ mod tests {
assert!(s.is_empty(), "no subtitle labels when sub is absent");
}
/// Spec: audio stream_number uses saturating_add on overflow (per u16 cap).
/// Mutation: use wrapping_add → stream numbers wrap to 0, skipping apply.
/// Spec: audio stream_number is the cell's 1-based position and never
/// wraps; past the u16 space the parser stops emitting.
/// Mutation: cast `i + 1` to u16 → stream numbers wrap to 0, skipping apply.
#[test]
fn audio_stream_number_saturates_not_wraps() {
fn audio_stream_number_never_wraps() {
// 65535 audio tracks is impossible on a real disc but the parser must
// not panic or produce 0. Build a comma-separated list of 65535 "eng"s.
// We only run the number-assignment logic via labels_from_feature.
@@ -375,15 +783,92 @@ mod tests {
assert_eq!(last, 300);
}
/// Spec: forced_sub with whitespace around "1" must still parse as true.
/// Mutation: use `== "1"` instead of `trim() == "1"` → " 1 " fails.
/// Spec: a `forced_sub` cell with surrounding whitespace still classifies.
/// Mutation: drop the `trim()` → " 3 " falls through to the unrecognised
/// arm and the disc's forced-narrative track loses its label.
#[test]
fn forced_sub_whitespace_around_one() {
let feature = r#"<playlist name="Feature" sub="eng,fra" forced_sub="0, 1" />"#;
fn forced_sub_cells_are_trimmed_before_classification() {
let feature = r#"<playlist name="Feature" sub="eng,fra,spa" forced_sub="0, 3 , 1 " />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s[0].qualifier, LabelQualifier::None);
assert_eq!(s[1].qualifier, LabelQualifier::Forced);
assert_eq!(s[2].qualifier, LabelQualifier::None);
}
/// `forced_sub` is an enumeration, and `1` is its "full dialogue track that
/// also carries forced signs" value — NOT "this track is forced".
///
/// Measured on a disc whose feature declares nine `1` cells among 32
/// subtitle slots: all nine are full dialogue tracks of 949-1411 display
/// sets, and the disc has no dedicated forced track at all. Reading `1` as
/// forced is what produced two identical full subtitle tracks for one
/// language with one of them flagged forced.
///
/// Nothing downstream can undo this on the discs that need it most:
/// `mux::codec::pgs::demotable` may only clear a vendor forced label where
/// some track on the disc demonstrably sets `forced_on_flag`, and measured
/// discs using this label format never set it.
///
/// Mutation: `"1" => ForcedNarrative` (the old reading) → red.
#[test]
fn a_contains_forced_segments_cell_is_not_a_forced_track() {
let feature = r#"<playlist name="Feature" sub="eng,ces,deu" forced_sub="0,1,1" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 3);
assert!(
s.iter().all(|l| l.qualifier == LabelQualifier::None),
"a `1` marks a full track containing forced signs, not a forced track"
);
}
/// `2` and `3` are the cells that DO name a dedicated forced-narrative
/// track, and the old boolean reading discarded both.
///
/// Measured: these cells occupy their own trailing STN slots, one per
/// localized language, duplicating a language that already holds a full
/// track earlier in the list. On one measured disc the four `3` slots carry
/// 7, 14, 23 and 59 display sets against 1216-2655 on the full tracks they
/// duplicate — and not one display set anywhere on that disc carries
/// `forced_on_flag`, so neither the scan probe nor the muxer can promote
/// them from content. The vendor cell is the only evidence there is.
///
/// Mutation: drop either arm of the `"2" | "3"` match → red.
#[test]
fn a_dedicated_forced_narrative_cell_is_a_forced_track() {
// The measured shape: full tracks first, their forced companions in
// trailing slots of the same languages.
let feature =
r#"<playlist name="Feature" sub="eng,cat,jpn,cat,jpn" forced_sub="0,0,0,2,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 5);
assert_eq!(s[1].qualifier, LabelQualifier::None, "the full cat track");
assert_eq!(s[2].qualifier, LabelQualifier::None, "the full jpn track");
assert_eq!(s[3].qualifier, LabelQualifier::Forced, "cat forced slot");
assert_eq!(s[3].stream_number, 4);
assert_eq!(s[4].qualifier, LabelQualifier::Forced, "jpn forced slot");
assert_eq!(s[4].stream_number, 5);
}
/// An unrecognised cell must fall to NOT forced. Asserting forced is the
/// expensive mistake (a full dialogue track a player then burns on screen),
/// so an unknown value from a future authoring revision must not be able to
/// make that claim.
///
/// Mutation: `_ => ForcedNarrative`, or treating "any non-zero" as forced.
#[test]
fn an_unrecognised_forced_sub_cell_is_not_forced() {
let feature = r#"<playlist name="Feature" sub="eng,fra,spa,ita" forced_sub="4,x,,-1" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 4);
assert!(s.iter().all(|l| l.qualifier == LabelQualifier::None));
// ...and so must a cell the CSV simply does not reach.
let feature = r#"<playlist name="Feature" sub="eng,fra" forced_sub="0" />"#;
let labels = labels_from_feature(feature);
assert_eq!(subs(&labels)[1].qualifier, LabelQualifier::None);
}
/// Spec: `find_feature_playlist` returns None when XML has no `<playlist>` elements.
@@ -393,4 +878,23 @@ mod tests {
assert!(find_feature_playlist("").is_none());
assert!(find_feature_playlist("<root />").is_none());
}
/// Spec: on a tie in audio-slot count, the FIRST playlist encountered
/// wins (consistent with `select_result`'s first-wins tiebreak
/// elsewhere in the registry) — later playlists only displace the
/// current best on a STRICTLY greater count.
/// Mutation: `count > best_aud_count` -> `count >= best_aud_count`
/// would let a later tied playlist silently displace the first.
#[test]
fn find_feature_first_wins_on_audio_count_tie() {
let xml = r#"
<playlist name="A" aud="eng,fra" />
<playlist name="B" aud="deu,spa" />
"#;
let feature = find_feature_playlist(xml).expect("a feature is found");
assert!(
feature.contains(r#"name="A""#),
"first playlist must win a tie, got: {feature}"
);
}
}
+785 -64
View File
File diff suppressed because it is too large Load Diff
+19 -12
View File
@@ -1,8 +1,8 @@
//! Menu-graphic filename language hints.
//!
//! Some BD-J discs encode per-language menu artwork with the language in the
//! filename, e.g. `Dune_UHD01_Eng_Composite1.png`,
//! `VForVendetta_UHD01_FRE_Composite2.png`. The `_UHD01_{LANG}_Composite`
//! filename, e.g. `Feature_UHD01_Eng_Composite1.png`,
//! `AltFeature_UHD01_FRE_Composite2.png`. The `_UHD01_{LANG}_Composite`
//! marker is authored deliberately, so the set of `{LANG}` tokens is the set
//! of menu languages the disc ships.
//!
@@ -41,15 +41,16 @@ pub fn parse(_reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
fn labels_from_filenames(names: &[String]) -> Vec<StreamLabel> {
let mut seen: Vec<&'static str> = Vec::new();
for name in names {
if let Some(code) = filename_lang(name) {
if !seen.contains(&code) {
seen.push(code);
}
if let Some(code) = filename_lang(name)
&& !seen.contains(&code)
{
seen.push(code);
}
}
seen.into_iter()
.enumerate()
.map(|(i, code)| StreamLabel {
stream_id: None,
stream_number: (i as u16).saturating_add(1),
stream_type: StreamLabelType::Audio,
language: code.to_string(),
@@ -91,10 +92,16 @@ mod tests {
#[test]
fn extracts_confirmed_samples() {
assert_eq!(filename_lang("Dune_UHD01_Eng_Composite1.png"), Some("eng"));
assert_eq!(filename_lang("Dune_UHD01_Ger_Composite2.png"), Some("deu"));
assert_eq!(
filename_lang("VForVendetta_UHD01_FRE_Composite2.png"),
filename_lang("Feature_UHD01_Eng_Composite1.png"),
Some("eng")
);
assert_eq!(
filename_lang("Feature_UHD01_Ger_Composite2.png"),
Some("deu")
);
assert_eq!(
filename_lang("AltFeature_UHD01_FRE_Composite2.png"),
Some("fra")
);
}
@@ -120,9 +127,9 @@ mod tests {
#[test]
fn dedups_and_numbers_distinct_languages() {
let names = vec![
"Dune_UHD01_Eng_Composite1.png".to_string(),
"Dune_UHD01_Eng_Composite2.png".to_string(),
"Dune_UHD01_Ger_Composite1.png".to_string(),
"Feature_UHD01_Eng_Composite1.png".to_string(),
"Feature_UHD01_Eng_Composite2.png".to_string(),
"Feature_UHD01_Ger_Composite1.png".to_string(),
"LoadingComposite1.png".to_string(),
];
let labels = labels_from_filenames(&names);
+1 -1
View File
@@ -169,7 +169,7 @@ mod tests {
/// Mutation: skip the final `if !current.is_empty()` emit → trailing run lost.
#[test]
fn large_buffer_trailing_run_emitted() {
let buf: Vec<u8> = (0..1000u32).map(|i| (0x41u8 + (i % 26) as u8)).collect();
let buf: Vec<u8> = (0..1000u32).map(|i| 0x41u8 + (i % 26) as u8).collect();
let got = extract_ascii_strings(&buf, 1);
// All printable, so one big run at the end.
assert!(!got.is_empty());
+431 -1
View File
@@ -177,7 +177,7 @@ const BARE_LANGS: &[(&str, &str)] = &[
];
/// Map a short menu-graphic language token (as embedded in authoring
/// filenames like `Dune_UHD01_Eng_Composite1.png`) to an ISO-639-2/T code.
/// filenames like `Feature_UHD01_Eng_Composite1.png`) to an ISO-639-2/T code.
///
/// These filename tokens are compact 2/3-letter abbreviations, NOT the full
/// language names [`lang`] handles, so they get their own certain table.
@@ -219,6 +219,237 @@ pub fn menu_lang(token: &str) -> Option<&'static str> {
Some(code)
}
// ── ISO 639-1 → ISO 639-2 ────────────────────────────────────────────────────
/// The complete ISO 639-1 set, paired with its ISO 639-2/**T** (terminological)
/// code. Every two-letter code ISO 639-1 defines appears exactly once.
///
/// /T is the variant the rest of this crate uses — [`lang`] and [`menu_lang`]
/// both normalize to it (`deu` not `ger`, `fra` not `fre`, `zho` not `chi`,
/// `ces`, `nld`, `ell`, `ron`, `slk`, `isl`, `eus`, `hrv`) — so the three
/// tables cannot disagree. `iso639_1_agrees_with_menu_lang` pins that.
///
/// For the 165 codes where 639-2/B and /T are identical this distinction does
/// not arise; it only matters for the 20-odd languages with a distinct
/// bibliographic code.
const ISO_639_1_TO_2: &[(&str, &str)] = &[
("aa", "aar"),
("ab", "abk"),
("ae", "ave"),
("af", "afr"),
("ak", "aka"),
("am", "amh"),
("an", "arg"),
("ar", "ara"),
("as", "asm"),
("av", "ava"),
("ay", "aym"),
("az", "aze"),
("ba", "bak"),
("be", "bel"),
("bg", "bul"),
("bh", "bih"),
("bi", "bis"),
("bm", "bam"),
("bn", "ben"),
("bo", "bod"),
("br", "bre"),
("bs", "bos"),
("ca", "cat"),
("ce", "che"),
("ch", "cha"),
("co", "cos"),
("cr", "cre"),
("cs", "ces"),
("cu", "chu"),
("cv", "chv"),
("cy", "cym"),
("da", "dan"),
("de", "deu"),
("dv", "div"),
("dz", "dzo"),
("ee", "ewe"),
("el", "ell"),
("en", "eng"),
("eo", "epo"),
("es", "spa"),
("et", "est"),
("eu", "eus"),
("fa", "fas"),
("ff", "ful"),
("fi", "fin"),
("fj", "fij"),
("fo", "fao"),
("fr", "fra"),
("fy", "fry"),
("ga", "gle"),
("gd", "gla"),
("gl", "glg"),
("gn", "grn"),
("gu", "guj"),
("gv", "glv"),
("ha", "hau"),
("he", "heb"),
("hi", "hin"),
("ho", "hmo"),
("hr", "hrv"),
("ht", "hat"),
("hu", "hun"),
("hy", "hye"),
("hz", "her"),
("ia", "ina"),
("id", "ind"),
("ie", "ile"),
("ig", "ibo"),
("ii", "iii"),
("ik", "ipk"),
("io", "ido"),
("is", "isl"),
("it", "ita"),
("iu", "iku"),
("ja", "jpn"),
("jv", "jav"),
("ka", "kat"),
("kg", "kon"),
("ki", "kik"),
("kj", "kua"),
("kk", "kaz"),
("kl", "kal"),
("km", "khm"),
("kn", "kan"),
("ko", "kor"),
("kr", "kau"),
("ks", "kas"),
("ku", "kur"),
("kv", "kom"),
("kw", "cor"),
("ky", "kir"),
("la", "lat"),
("lb", "ltz"),
("lg", "lug"),
("li", "lim"),
("ln", "lin"),
("lo", "lao"),
("lt", "lit"),
("lu", "lub"),
("lv", "lav"),
("mg", "mlg"),
("mh", "mah"),
("mi", "mri"),
("mk", "mkd"),
("ml", "mal"),
("mn", "mon"),
("mr", "mar"),
("ms", "msa"),
("mt", "mlt"),
("my", "mya"),
("na", "nau"),
("nb", "nob"),
("nd", "nde"),
("ne", "nep"),
("ng", "ndo"),
("nl", "nld"),
("nn", "nno"),
("no", "nor"),
("nr", "nbl"),
("nv", "nav"),
("ny", "nya"),
("oc", "oci"),
("oj", "oji"),
("om", "orm"),
("or", "ori"),
("os", "oss"),
("pa", "pan"),
("pi", "pli"),
("pl", "pol"),
("ps", "pus"),
("pt", "por"),
("qu", "que"),
("rm", "roh"),
("rn", "run"),
("ro", "ron"),
("ru", "rus"),
("rw", "kin"),
("sa", "san"),
("sc", "srd"),
("sd", "snd"),
("se", "sme"),
("sg", "sag"),
("si", "sin"),
("sk", "slk"),
("sl", "slv"),
("sm", "smo"),
("sn", "sna"),
("so", "som"),
("sq", "sqi"),
("sr", "srp"),
("ss", "ssw"),
("st", "sot"),
("su", "sun"),
("sv", "swe"),
("sw", "swa"),
("ta", "tam"),
("te", "tel"),
("tg", "tgk"),
("th", "tha"),
("ti", "tir"),
("tk", "tuk"),
("tl", "tgl"),
("tn", "tsn"),
("to", "ton"),
("tr", "tur"),
("ts", "tso"),
("tt", "tat"),
("tw", "twi"),
("ty", "tah"),
("ug", "uig"),
("uk", "ukr"),
("ur", "urd"),
("uz", "uzb"),
("ve", "ven"),
("vi", "vie"),
("vo", "vol"),
("wa", "wln"),
("wo", "wol"),
("xh", "xho"),
("yi", "yid"),
("yo", "yor"),
("za", "zha"),
("zh", "zho"),
("zu", "zul"),
];
/// The three two-letter codes ISO 639-1 has since withdrawn, mapped to their
/// replacements. DVD-Video froze its language list on the 1988 edition, so
/// discs authored to the spec carry these spellings and no other table sees
/// them: `iw` Hebrew (now `he`), `in` Indonesian (now `id`), `ji` Yiddish
/// (now `yi`).
const ISO_639_1_DEPRECATED: &[(&str, &str)] = &[("iw", "he"), ("in", "id"), ("ji", "yi")];
/// Map an ISO 639-1 two-letter language code to its ISO 639-2/T three-letter
/// code, accepting the withdrawn DVD-era spellings (`iw`, `in`, `ji`) as
/// aliases for their replacements.
///
/// Covers the WHOLE of ISO 639-1, unlike [`menu_lang`], whose table only spans
/// the languages that show up in Blu-ray menu-graphic filenames. Callers that
/// convert a spec field — a DVD IFO attribute block, say — need the whole set:
/// narrowing it to the menu vocabulary would fold every other language onto
/// one value and make a disc's tracks indistinguishable from each other.
///
/// Case-insensitive and trimmed. Returns `None` for anything that is not an
/// ISO 639-1 code, so callers decide the fallback rather than getting a guess.
pub fn iso639_1_to_iso639_2(code: &str) -> Option<&'static str> {
let c = code.trim().to_ascii_lowercase();
let c = ISO_639_1_DEPRECATED
.iter()
.find(|(old, _)| *old == c)
.map_or(c.as_str(), |(_, new)| new);
ISO_639_1_TO_2
.iter()
.find(|(two, _)| *two == c)
.map(|(_, three)| *three)
}
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
@@ -685,4 +916,203 @@ mod tests {
fn codec_empty_passes_through() {
assert_eq!(codec(""), "");
}
/// `purpose()`'s multi-word-compound fast path ORs two independent
/// phrase checks ("audio description" / "descriptive service"). Each
/// phrase, when it appears as a *word*-bounded match, is independently
/// caught by the has_word fallback further down — so the OR only
/// matters when a phrase appears as a *substring inside a larger word*
/// (no boundary), which .contains() still catches but has_word() would
/// reject.
///
/// Mutation: replace `||` with `&&` at line 238 → since "audio
/// description" is absent here, the AND fails, the fast path doesn't
/// fire, and the fallback has_word("descriptive") also fails (no word
/// boundary before "descriptive" in "nondescriptive"), so purpose()
/// wrongly returns Normal instead of Descriptive.
#[test]
fn purpose_descriptive_service_substring_without_word_boundary() {
assert_eq!(
purpose("nondescriptive service track"),
LabelPurpose::Descriptive
);
}
/// `menu_lang()` maps every authoring-filename token in its table
/// (ISO-639-2/B and /T spellings, plus ISO-639-1) to the canonical
/// /T code used by the rest of the pipeline. Exhaustive per-arm check:
/// deleting any single match arm makes that arm's tokens return None
/// instead of the documented code.
#[test]
fn menu_lang_covers_every_table_entry() {
let cases: &[(&str, &str)] = &[
("eng", "eng"),
("en", "eng"),
("ger", "deu"),
("deu", "deu"),
("de", "deu"),
("fre", "fra"),
("fra", "fra"),
("fr", "fra"),
("spa", "spa"),
("es", "spa"),
("ita", "ita"),
("it", "ita"),
("por", "por"),
("pt", "por"),
("jpn", "jpn"),
("jap", "jpn"),
("ja", "jpn"),
("kor", "kor"),
("ko", "kor"),
("chi", "zho"),
("zho", "zho"),
("zh", "zho"),
("rus", "rus"),
("ru", "rus"),
("dut", "nld"),
("nld", "nld"),
("nl", "nld"),
("pol", "pol"),
("pl", "pol"),
("cze", "ces"),
("ces", "ces"),
("cs", "ces"),
("dan", "dan"),
("da", "dan"),
("fin", "fin"),
("fi", "fin"),
("nor", "nor"),
("no", "nor"),
("swe", "swe"),
("sv", "swe"),
("hun", "hun"),
("hu", "hun"),
("gre", "ell"),
("ell", "ell"),
("el", "ell"),
("tur", "tur"),
("tr", "tur"),
("ara", "ara"),
("ar", "ara"),
("hin", "hin"),
("hi", "hin"),
("tha", "tha"),
("th", "tha"),
("ukr", "ukr"),
("uk", "ukr"),
("cat", "cat"),
("ca", "cat"),
];
for (token, expected) in cases {
assert_eq!(
menu_lang(token),
Some(*expected),
"menu_lang({:?}) should map to {:?}",
token,
expected
);
}
// Case-insensitive and trimmed.
assert_eq!(menu_lang("ENG"), Some("eng"));
assert_eq!(menu_lang(" Eng "), Some("eng"));
// Unrecognized token -> None, never a guess.
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
/// Structural invariants of `ISO_639_1_TO_2`: it must hold the complete
/// ISO 639-1 set (184 codes), every key a distinct pair of lowercase
/// letters and every value three lowercase letters. A typo'd or duplicated
/// row fails here rather than silently mislabelling a track.
#[test]
fn iso639_1_table_is_complete_and_well_formed() {
assert_eq!(
ISO_639_1_TO_2.len(),
184,
"ISO 639-1 defines 184 two-letter codes; the table must hold all \
of them"
);
let mut keys: Vec<&str> = ISO_639_1_TO_2.iter().map(|(two, _)| *two).collect();
keys.sort_unstable();
let unique = keys.len();
keys.dedup();
assert_eq!(unique, keys.len(), "no ISO 639-1 code may appear twice");
for (two, three) in ISO_639_1_TO_2 {
assert!(
two.len() == 2 && two.bytes().all(|b| b.is_ascii_lowercase()),
"{two:?} is not a two-letter lowercase ISO 639-1 code"
);
assert!(
three.len() == 3 && three.bytes().all(|b| b.is_ascii_lowercase()),
"{three:?} is not a three-letter lowercase ISO 639-2 code"
);
}
// The withdrawn DVD-era spellings resolve, and are not themselves
// rows in the main table (they are aliases, not codes).
for (old, new) in ISO_639_1_DEPRECATED {
assert!(
!ISO_639_1_TO_2.iter().any(|(two, _)| two == old),
"withdrawn code {old:?} must not be a table row"
);
assert_eq!(
iso639_1_to_iso639_2(old),
iso639_1_to_iso639_2(new),
"withdrawn code {old:?} must resolve exactly as {new:?}"
);
}
}
/// The two tables must not disagree. Every two-letter token `menu_lang`
/// accepts has to yield the same ISO 639-2/T code through
/// `iso639_1_to_iso639_2`, so a DVD-sourced language and a Blu-ray
/// menu-label language for the same tongue never produce different
/// `Language` elements.
#[test]
fn iso639_1_agrees_with_menu_lang() {
for (two, three) in ISO_639_1_TO_2 {
if let Some(via_menu) = menu_lang(two) {
assert_eq!(
via_menu, *three,
"menu_lang({two:?}) = {via_menu:?} disagrees with the ISO \
639-1 table's {three:?}"
);
}
}
// Spot-check the /T choice itself, on the languages where /B differs.
for (two, t_code) in [
("de", "deu"),
("fr", "fra"),
("zh", "zho"),
("cs", "ces"),
("nl", "nld"),
("el", "ell"),
("ro", "ron"),
("sk", "slk"),
("is", "isl"),
("hy", "hye"),
("ka", "kat"),
("fa", "fas"),
] {
assert_eq!(
iso639_1_to_iso639_2(two),
Some(t_code),
"the crate standardises on ISO 639-2/T, so {two:?} is \
{t_code:?} and never the bibliographic form"
);
}
}
/// Trimming, case-insensitivity, and the no-guess contract.
#[test]
fn iso639_1_normalizes_input_and_never_guesses() {
assert_eq!(iso639_1_to_iso639_2("RO"), Some("ron"));
assert_eq!(iso639_1_to_iso639_2(" Ro "), Some("ron"));
assert_eq!(iso639_1_to_iso639_2("IW"), Some("heb"));
assert_eq!(iso639_1_to_iso639_2("zz"), None);
assert_eq!(iso639_1_to_iso639_2(""), None);
assert_eq!(iso639_1_to_iso639_2("e"), None);
// A three-letter code is not ISO 639-1 input — that is menu_lang's job.
assert_eq!(iso639_1_to_iso639_2("eng"), None);
}
}
+132
View File
@@ -634,4 +634,136 @@ mod tests {
let (s, e) = find_element(xml, "name", 0).unwrap();
assert_eq!(&xml[s..e], "<di:name>Title</di:name>");
}
// ── Malformed / truncated input (untrusted on-disc XML) ────────────────
//
// These scrapers run on XML lifted out of BD-J jar entries, which is
// attacker-controllable. Every scan in this module must terminate and
// stay in bounds on truncated or unbalanced input rather than panic.
// XML 1.0 §2.3 defines the Name production these boundary rules model.
/// A quoted attribute value that is never closed must terminate the
/// scan at EOF rather than reading past the end of the buffer.
#[test]
fn attr_unterminated_quoted_value_scan_stops_at_eof() {
// The scanner enters the `y="` value and runs off the end looking
// for the closing quote; `name` is never found.
assert_eq!(attr(r#"<x y="oops"#, "name"), None);
assert_eq!(attr("<x y='oops", "name"), None);
// The truncated attribute itself has no terminated value either.
assert_eq!(attr(r#"<x y="oops"#, "y"), None);
}
/// An attribute name at EOF followed only by whitespace (no `=`) must
/// return None, not read past the buffer while skipping that whitespace.
#[test]
fn attr_name_with_trailing_whitespace_and_no_equals_returns_none() {
assert_eq!(attr("<x name ", "name"), None);
}
/// `name=` followed only by whitespace to EOF has no value to return.
#[test]
fn attr_equals_with_trailing_whitespace_and_no_value_returns_none() {
assert_eq!(attr("<x name= ", "name"), None);
}
/// A quoted attribute value is opaque: a `name="..."` pair that appears
/// *inside* another attribute's value must never be reported, even when
/// it is preceded by whitespace so it would otherwise clear the
/// word-boundary check.
#[test]
fn attr_decoy_name_after_space_inside_quoted_value_is_skipped() {
assert_eq!(attr(r#"<x y=" name='decoy'" />"#, "name"), None);
// The real attribute after the decoy still resolves.
assert_eq!(
attr(r#"<x y=" name='decoy'" name="real" />"#, "name"),
Some("real".into())
);
}
/// XML 1.0 §2.3 NameChar includes `-`, `_` and `.`, so `q-a`, `q_a` and
/// `q.a` are each a single attribute name distinct from `a`. Searching
/// for `a` must not match the tail of any of them.
#[test]
fn attr_name_char_boundary_covers_hyphen_underscore_and_dot() {
assert_eq!(
attr(r#"<x q-a="decoy" a="real" />"#, "a"),
Some("real".into())
);
assert_eq!(
attr(r#"<x q_a="decoy" a="real" />"#, "a"),
Some("real".into())
);
assert_eq!(
attr(r#"<x q.a="decoy" a="real" />"#, "a"),
Some("real".into())
);
}
/// An open tag truncated mid-attribute never terminates, so no element
/// can be returned — and the attribute walk must not read past EOF.
#[test]
fn find_element_unterminated_open_tag_returns_none() {
assert_eq!(find_element("<x attr=", "x", 0), None);
}
/// A `/` as the final byte of the buffer is not a self-closing marker;
/// probing for the `>` that would follow it must stay in bounds.
#[test]
fn find_element_trailing_slash_at_eof_returns_none() {
assert_eq!(find_element("<a /", "a", 0), None);
}
/// `/>` inside a quoted attribute value does not close the element.
#[test]
fn find_element_quoted_self_close_marker_does_not_end_element() {
let xml = r#"<x a="/>"/>"#;
let (s, e) = find_element(xml, "x", 0).unwrap();
assert_eq!(&xml[s..e], r#"<x a="/>"/>"#);
}
/// An attribute value whose quote is never closed leaves the open tag
/// unterminated; the scan must end at EOF and report no element.
#[test]
fn find_element_unterminated_quoted_attr_returns_none() {
assert_eq!(find_element(r#"<x a="oops"#, "x", 0), None);
}
/// A `/` in the middle of an unquoted attribute value is not a
/// self-closing marker — only `/>` is.
#[test]
fn find_element_unquoted_slash_is_not_self_closing() {
let xml = "<a href=x/y>body</a>";
let (s, e) = find_element(xml, "a", 0).unwrap();
assert_eq!(&xml[s..e], "<a href=x/y>body</a>");
}
/// `text` must locate the real end of the open tag: a bare `/` inside
/// an unquoted attribute value must not be treated as `/>`, which would
/// shift the body start and leak tag bytes into the returned text.
#[test]
fn text_unquoted_slash_in_attr_does_not_truncate_body() {
assert_eq!(text("<x a=b/c>hello</x>", "x"), Some("hello".into()));
}
/// A `>` inside a quoted attribute value must not be mistaken for the
/// end of the open tag when `text` computes the body start.
#[test]
fn text_quoted_gt_in_attr_does_not_truncate_body() {
assert_eq!(text(r#"<x a="b>c">hello</x>"#, "x"), Some("hello".into()));
}
/// A close tag truncated mid-name (`</x` with no `>`) is not a close
/// tag; matching it must stay in bounds and report no text.
#[test]
fn text_truncated_close_tag_returns_none() {
assert_eq!(text("<x>body</x", "x"), None);
}
/// A `/` in element content is only a close tag when preceded by `<`.
/// Body text containing `a/x>` must not be mistaken for `</x>`.
#[test]
fn text_slash_in_body_is_not_a_close_tag() {
assert_eq!(text("<x>a/x> </x>", "x"), Some("a/x>".into()));
}
}
+46 -28
View File
@@ -32,7 +32,7 @@
//! let opts = libfreemkv::InputOptions::default();
//! let mut input = libfreemkv::input("iso://disc.iso", &opts)?;
//! let title = input.info().clone();
//! let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
//! let mut output = libfreemkv::output("mkv://Movie.mkv", &title, None)?;
//! // Propagate read errors instead of silently stopping on the first one.
//! while let Some(frame) = input.read()? {
//! output.write(&frame)?;
@@ -106,12 +106,15 @@ pub mod consts;
pub mod css;
pub mod decrypt;
pub mod diag;
pub mod dirimage;
pub mod disc;
pub mod drive;
pub mod dvdnav;
pub mod error;
pub mod event;
pub mod halt;
#[cfg(test)]
mod harness;
pub mod hex;
pub(crate) mod identity;
pub(crate) mod ifo;
@@ -126,7 +129,8 @@ pub mod progress;
pub mod scsi;
pub mod sector;
pub mod session;
pub(crate) mod speed;
#[cfg(test)]
pub(crate) mod testlog;
pub(crate) mod udf;
pub(crate) mod unlock_bridge;
@@ -138,7 +142,7 @@ pub(crate) mod unlock_bridge;
pub use drive::capture::{
CapturedFeature, DriveCapture, capture_drive_data, mask_bytes, mask_string,
};
pub use drive::{Drive, DriveStatus, find_drive};
pub use drive::{Drive, DriveStatus, extract_scsi_context, find_drive};
// ─── Disc session (drive open + SCSI bring-up hoist) ─────────────────────────
//
@@ -147,7 +151,8 @@ pub use drive::{Drive, DriveStatus, find_drive};
// Owns the `Drive` by value; forwards consumer-built key material into
// `ScanOptions` (the library derives no certs — see `KeySpec`).
pub use session::{
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_iso,
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_dir,
scan_iso,
};
// ─── Errors ─────────────────────────────────────────────────────────────────
@@ -155,31 +160,47 @@ pub use session::{
// All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a
// numeric `code()`; **no English text in the library** — applications map
// codes to localized messages. See `error.rs` for the full taxonomy.
pub use error::{Error, Result};
pub use error::{
Error, Result, error_code, is_disc_level_no_key, is_halt, is_skippable_title_stub,
};
// ─── Cooperative cancellation ───────────────────────────────────────────────
//
// One-bit cooperative cancellation token, shared by every long-running loop
// in libfreemkv (sweep, patch, mux). Clone it cheaply; pass it by value into
// each component; poll `is_cancelled()` inside the loop body.
// One-bit cooperative cancellation token, shared by every long-running loop
// libfreemkv's mux, and the recovery passes (sweep/patch) that now live in the
// freemkv-engine crate. Clone it cheaply; pass it by value into each component;
// poll `is_cancelled()` inside the loop body.
pub use halt::Halt;
// Generic bounded producer/consumer primitive used by sweep, patch, and
// mux to overlap reads with writes via a dedicated consumer thread.
// Generic bounded producer/consumer primitive used by the mux pipeline (and,
// via this re-export, by the engine's sweep/patch recovery passes) to overlap
// reads with writes via a dedicated consumer thread.
// `Pipeline::spawn(name, depth, sink)` spawns a named consumer; `pipe.send(item)`
// pushes one item with back-pressure; `pipe.finish()` joins the
// consumer and surfaces its `close()` output. Callers implement `Sink`
// to define per-item behaviour and end-of-stream finalisation.
//
// `DEFAULT_PIPELINE_DEPTH` (=4) is for callers without specific needs;
// most should use READ_PIPELINE_DEPTH or WRITE_PIPELINE_DEPTH instead.
// most should use WRITE_PIPELINE_DEPTH instead.
// Patch uses `WRITE_THROUGH_DEPTH` (=1). Returning `Flow::Stop` from
// `apply` ends the consumer cleanly (still calls `close()`).
pub use io::pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, READ_PIPELINE_DEPTH, Sink, WRITE_PIPELINE_DEPTH,
WRITE_THROUGH_DEPTH,
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
// ─── Bounded-cache buffered file writer ─────────────────────────────────────
//
// Drop-in `std::fs::File` replacement used everywhere the lib writes large
// sequential output (mux, extract, sweep, patch) — drains dirty pages
// continuously instead of bursting. General I/O infra, not recovery policy;
// promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too.
pub use io::WritebackFile;
/// Write an image-level source out as a sector image — what an `iso://`
/// DESTINATION means for any source that is not a physical drive. Drive sources
/// go through `freemkv_engine::copy`, which is the recovery path; see
/// [`io::image_writer`] for why the two are deliberately separate.
pub use io::image_writer::write_image;
// ─── Drive events (low-level callbacks) ─────────────────────────────────────
pub use event::{BatchSizeReason, Event, EventKind};
pub use identity::DriveId;
@@ -198,10 +219,7 @@ pub use identity::DriveId;
// don't touch `DecryptKeys` directly — `DiscStream::new(reader, title, keys, …)`
// accepts whatever `Disc::decrypt_keys()` returned. `decrypt_sectors()` is
// for callers that operate on raw sector buffers (e.g. ISO patching).
pub use decrypt::{
AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_sectors_mapped, decrypt_threads,
set_decrypt_threads,
};
pub use decrypt::{AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_threads};
// ─── Disc structure ─────────────────────────────────────────────────────────
//
@@ -214,12 +232,12 @@ pub use decrypt::{
// — not the `pes::Stream` trait re-exported below as `PesStream`. Two
// different concepts, the same short name; the trait gets the `Pes`
// prefix at the crate root to keep both addressable.
pub use dirimage::DirImage;
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, DamageSeverity,
Disc, DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult,
FileResult, FrameRate, HdrFormat, Key, KeyOrigin, LabelPurpose, LabelQualifier, PatchOptions,
PatchOutcome, Resolution, SampleRate, ScanOptions, Stream, SubtitleStream, SweepOptions,
VideoStream, classify_damage,
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult,
FileResult, FrameRate, HdrFormat, Key, KeyOrigin, LabelPurpose, LabelQualifier, Resolution,
SampleRate, ScanOptions, Stream, SubtitleStream, VideoStream,
};
pub use keysource::{DiscInputs, KeySource, read_encrypted_units, resolve_and_apply};
@@ -252,7 +270,8 @@ pub use mux::NullStream;
pub use mux::StdioStream;
pub use mux::WriteSeek;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
pub use mux::{Mp4FitReport, Mp4SkipReason, mp4_fit_report};
pub use mux::{Medium, SourceInfo};
pub use mux::{Mp4FitReport, Mp4Sink, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
@@ -264,11 +283,10 @@ pub use mux::{Mp4FitReport, Mp4SkipReason, mp4_fit_report};
// `SectorSource` to get plaintext sectors out.
pub use mux::build_iso_pipeline;
pub use mux::resolve_mux_key_map;
pub use mux::{MuxEvents, MuxInput, MuxOptions, MuxOutcome, NoopEvents, mux_stream};
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, drive_has_disc, list_drives};
pub use mux::select::{PidFilter, StreamSelection};
pub use mux::{MuxEvents, MuxInput, MuxOptions, MuxOutcome, mux_stream};
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, SenseFamily, drive_has_disc, list_drives};
pub use sector::{
DecryptingSectorSource, FileSectorSink, FileSectorSource, KeyFetch, PrefetchedSectorSource,
SectorSink, SectorSource,
DecryptingSectorSource, FileSectorSource, KeyFetch, PrefetchedSectorSource, SectorSource,
};
pub use speed::DriveSpeed;
pub use udf::{UdfFs, read_filesystem};
+316 -3
View File
@@ -39,6 +39,18 @@ pub(crate) struct PlaylistMark {
pub timestamp: u32,
}
impl PlaylistMark {
/// Is this mark a chapter entry point?
///
/// Only `mark_type == 1` counts. Type 0 is reserved and type 2 is a link
/// point, and neither is a chapter. Every chapter filter in the crate goes
/// through here: two hand-rolled copies had already drifted, one testing
/// `<= 1` and silently counting reserved marks as chapters.
pub(crate) fn is_chapter_mark(&self) -> bool {
self.mark_type == 1
}
}
/// A play item — one clip reference with in/out times.
#[derive(Debug)]
pub(crate) struct PlayItem {
@@ -432,14 +444,13 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
}
}
}
STREAM_CATEGORY_PG_SUBTITLE => {
STREAM_CATEGORY_PG_SUBTITLE
// PG: coding_type(1) + language(3).
// IG is parsed only to advance spos and is then discarded by the
// caller, so it deliberately has no arm here.
if sa.len() >= 4 {
if sa.len() >= 4 => {
language = String::from_utf8_lossy(&sa[1..4]).to_string();
}
}
_ => {}
}
@@ -1417,4 +1428,306 @@ mod tests {
data[8..12].copy_from_slice(&40u32.to_be_bytes()); // playlist_start = 40 = len
assert!(parse(&data).is_err());
}
// ─────────────────────────────────────────────────────────────────────
// Added: STN-table block alignment and section-boundary hardening.
// ─────────────────────────────────────────────────────────────────────
/// Build an MPLS from raw PlayItem bodies, with no PlayListMark section
/// (mark_start = 0). Lets a test control item_length exactly.
fn build_mpls_raw_items(items: &[Vec<u8>]) -> Vec<u8> {
let playlist_start: u32 = 40;
let mut buf = Vec::new();
buf.extend_from_slice(b"MPLS0200");
buf.extend_from_slice(&playlist_start.to_be_bytes());
buf.extend_from_slice(&[0u8; 28]); // mark_start = 0, then padding
let pl_start = buf.len();
buf.extend_from_slice(&[0u8; 4]); // PlayList length placeholder
buf.extend_from_slice(&[0u8; 2]); // reserved
buf.extend_from_slice(&(items.len() as u16).to_be_bytes());
buf.extend_from_slice(&[0u8; 2]); // num_sub_paths
for it in items {
buf.extend_from_slice(&(it.len() as u16).to_be_bytes());
buf.extend_from_slice(it);
}
let pl_len = (buf.len() - pl_start - 4) as u32;
buf[pl_start..pl_start + 4].copy_from_slice(&pl_len.to_be_bytes());
buf
}
/// The 20 bytes a PlayItem needs for clip_id(5) + codec_id(4) +
/// connection_condition(1) + reserved(2) + IN_time(4) + OUT_time(4).
fn play_item_20(clip: &[u8; 5], cc: u8, in_t: u32, out_t: u32) -> Vec<u8> {
let mut it = Vec::new();
it.extend_from_slice(clip);
it.extend_from_slice(b"M2TS");
it.push(cc);
it.extend_from_slice(&[0u8; 2]);
it.extend_from_slice(&in_t.to_be_bytes());
it.extend_from_slice(&out_t.to_be_bytes());
assert_eq!(it.len(), 20);
it
}
/// A PlayItem body of exactly 20 bytes carries every field the parser
/// reads (the last is OUT_time at [16..20]), so it must be RECORDED,
/// not skipped — and it has no STN table, which starts at byte 32.
#[test]
fn play_item_of_exactly_20_bytes_is_recorded_without_stn() {
let data = build_mpls_raw_items(&[play_item_20(b"00007", 5, 90_000, 180_000)]);
let pl = parse(&data).expect("a 20-byte PlayItem must parse");
assert_eq!(pl.play_items.len(), 1);
assert_eq!(pl.play_items[0].clip_id, "00007");
assert_eq!(pl.play_items[0].in_time, 90_000);
assert_eq!(pl.play_items[0].out_time, 180_000);
assert_eq!(pl.play_items[0].connection_condition, 5);
assert!(pl.streams.is_empty(), "no STN table exists below byte 32");
}
/// A 40-byte MPLS whose PlayList section is exactly its 10-byte header
/// (length(4)+reserved(2)+num_play_items(2)+num_sub_paths(2)) ending at
/// EOF is structurally complete, not truncated: nothing the parser reads
/// lies past the buffer, so it must parse to an empty playlist.
#[test]
fn minimum_size_mpls_with_empty_playlist_header_parses() {
let mut data = vec![0u8; 40];
data[0..4].copy_from_slice(b"MPLS");
data[4..8].copy_from_slice(b"0200");
data[8..12].copy_from_slice(&30u32.to_be_bytes()); // playlist_start + 10 == 40
// mark_start (12..16) stays 0; num_play_items at data[36..38] is 0.
let pl = parse(&data).expect("40-byte MPLS with a complete PlayList header must parse");
assert!(pl.play_items.is_empty());
assert!(pl.streams.is_empty());
assert!(pl.marks.is_empty());
}
/// A mark_start of 0 means "no PlayListMark section". The file header
/// bytes at offset 0 must not be decoded as one — data[4..6] is the
/// version string "02", which as a big-endian num_marks would be 12338.
#[test]
fn mark_start_zero_does_not_parse_header_as_marks() {
let data = build_mpls_raw_items(&[play_item_20(b"00007", 1, 0, 90_000)]);
assert_eq!(
&data[12..16],
&[0, 0, 0, 0],
"fixture must have mark_start 0"
);
let pl = parse(&data).expect("should parse");
assert!(
pl.marks.is_empty(),
"mark_start == 0 must mean absent, got {} marks",
pl.marks.len()
);
}
/// Full STN table walk with every category populated and DISTINCT
/// counts, so no count byte can be read from a neighbour's offset
/// without changing the result.
///
/// Each secondary block is followed by its reference block(s), which
/// per the BD STN table are num_refs(1) + reserved(1) + one byte per
/// ref + one padding byte when the ref count is odd. Every ref count
/// here is 1 — the value that distinguishes `n % 2` (=1) from `n / 2`
/// (=0) — so a wrong skip length misaligns the cursor and every
/// following stream decodes from the wrong offset. IG entries are
/// consumed to keep the cursor aligned but never retained.
#[test]
fn full_stn_table_block_alignment() {
let mut entries: Vec<Vec<u8>> = vec![
build_stream_entry_video(0x1011, 0x1B, 6, 1, None),
build_stream_entry_audio(0x1100, 0x83, 6, 1, b"eng"),
build_stream_entry_audio(0x1101, 0x86, 3, 1, b"fra"),
build_stream_entry_pg(0x1200, 0x90, b"eng"),
build_stream_entry_pg(0x1201, 0x90, b"fra"),
build_stream_entry_pg(0x1202, 0x90, b"deu"),
];
for i in 0..4u16 {
entries.push(build_stream_entry_pg(0x1400 + i, 0x91, b"eng"));
}
// secondary audio + its secondary-audio ref block (1 ref → 1 pad)
let mut sec_audio = build_stream_entry_audio(0x1A00, 0x83, 3, 1, b"spa");
sec_audio.extend_from_slice(&[1, 0, 0x55, 0x00]);
entries.push(sec_audio);
// secondary video + audio-ref block + PiP-PG-ref block
let mut sec_video = build_stream_entry_video(0x1B00, 0x1B, 4, 1, None);
sec_video.extend_from_slice(&[1, 0, 0x55, 0x00]);
sec_video.extend_from_slice(&[1, 0, 0x66, 0x00]);
entries.push(sec_video);
// PiP PG + its ref block
let mut pip_pg = build_stream_entry_pg(0x1C00, 0x90, b"jpn");
pip_pg.extend_from_slice(&[1, 0, 0x77, 0x00]);
entries.push(pip_pg);
// Dolby Vision enhancement layer
entries.push(build_stream_entry_video(0x1015, 0x24, 8, 1, Some(0x12)));
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 2, 3, 4, 1, 1, 1, 1),
&entries,
);
let pl = parse(&data).expect("should parse");
let got: Vec<(u8, u16, bool)> = pl
.streams
.iter()
.map(|s| (s.stream_type, s.pid, s.secondary))
.collect();
assert_eq!(
got,
vec![
(1, 0x1011, false), // primary video
(2, 0x1100, false), // primary audio ×2
(2, 0x1101, false),
(3, 0x1200, false), // PG ×3
(3, 0x1201, false),
(3, 0x1202, false),
// the 4 IG entries are consumed and discarded
(5, 0x1A00, true), // secondary audio
(6, 0x1B00, true), // secondary video
(3, 0x1C00, true), // PiP PG
(7, 0x1015, true), // Dolby Vision EL
]
);
// Languages prove each entry was decoded at its own offset.
assert_eq!(pl.streams[1].language, "eng");
assert_eq!(pl.streams[2].language, "fra");
assert_eq!(pl.streams[6].language, "spa");
assert_eq!(pl.streams[8].language, "jpn");
}
/// A secondary block whose stream entry ends exactly at the end of the
/// PlayItem has no reference block at all; the count byte must not be
/// read from one-past-the-end. Covers all three secondary blocks that
/// carry reference data.
#[test]
fn secondary_ref_block_at_item_end_is_not_read() {
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
// Secondary audio is the last entry, with no ref bytes following.
let sec_audio = build_stream_entry_audio(0x1A00, 0x83, 3, 1, b"eng");
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 1, 0, 0, 0),
&[video.clone(), sec_audio],
);
let pl = parse(&data).expect("secondary audio at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1A00);
// Secondary video is the last entry, with no ref bytes following.
let sec_video = build_stream_entry_video(0x1B00, 0x1B, 4, 1, None);
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 1, 0, 0),
&[video.clone(), sec_video.clone()],
);
let pl = parse(&data).expect("secondary video at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1B00);
// Secondary video whose audio-ref block ends exactly at item end, so
// the PiP-PG ref count byte would sit one past it.
let mut sec_video_arefs = sec_video;
sec_video_arefs.extend_from_slice(&[0, 0]); // n_arefs = 0, reserved
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 1, 0, 0),
&[video.clone(), sec_video_arefs],
);
let pl = parse(&data).expect("secondary video aref block at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1B00);
// PiP PG is the last entry, with no ref bytes following.
let pip_pg = build_stream_entry_pg(0x1C00, 0x90, b"jpn");
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 0, 1, 0),
&[video, pip_pg],
);
let pl = parse(&data).expect("PiP PG at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1C00);
}
// ─────────────────────────────────────────────────────────────────────
// parse_stream_entry bounds, exercised directly.
// ─────────────────────────────────────────────────────────────────────
/// Fewer than 2 bytes remain for the stream_entry header
/// (length(1) + stream_entry_type(1)) → None, without reading either.
#[test]
fn stream_entry_header_past_end_is_none() {
let item = [0u8; 8];
for pos in 7..12usize {
assert!(
parse_stream_entry(&item, pos, STREAM_CATEGORY_VIDEO).is_none(),
"pos={pos}"
);
}
}
/// The stream_attributes header (length(1) + coding_type(1)) lies past
/// the end of the PlayItem → None, without reading the length byte.
#[test]
fn stream_attributes_header_past_end_is_none() {
// se_len = 3 → se_end = 4 == item.len(); the sa length byte would be
// at item[4] and the coding type at item[5].
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11];
assert!(parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).is_none());
}
/// A declared stream_attributes length of 0 has no coding_type byte and
/// must be rejected — even when the (empty) attribute region is itself
/// in bounds.
#[test]
fn zero_length_attributes_in_bounds_is_none() {
// se_len = 3 → se_end = 4; sa_len = item[4] = 0 → sa_end = 5 ≤ 6.
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11, 0, 0];
assert!(parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).is_none());
}
/// stream_attributes of exactly 1 byte carries only the coding_type.
/// That is the minimum the parser accepts, so the entry is returned
/// with its PID and coding_type and no format-specific fields — for a
/// PG stream the 3-byte language must NOT be read past the attributes.
#[test]
fn one_byte_stream_attributes_yields_bare_entry() {
// se_len = 3 → se_end = 4; sa_len = 1 → sa_end = 6 == item.len().
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11, 1, 0x1B];
let (entry, next) =
parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).expect("1-byte attrs are valid");
assert_eq!(entry.pid, 0x1011);
assert_eq!(entry.coding_type, 0x1B);
assert_eq!(entry.video_format, 0);
assert_eq!(entry.video_rate, 0);
assert_eq!(next, 6);
let pg = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x12, 0x00, 1, 0x90];
let (entry, _) = parse_stream_entry(&pg, 0, STREAM_CATEGORY_PG_SUBTITLE)
.expect("1-byte PG attrs are valid");
assert_eq!(entry.pid, 0x1200);
assert_eq!(entry.coding_type, 0x90);
assert_eq!(entry.language, "");
}
/// Type 0 is reserved and type 2 is a link point; neither is a chapter.
/// `labels::collect_chapter_summary` used to filter on `mark_type <= 1`,
/// which counted reserved marks and inflated the public `chapter_count`
/// (and let a playlist whose only marks are reserved pass the
/// `chapter_count == 0` skip). Both call sites now share this predicate.
#[test]
fn only_entry_marks_count_as_chapters() {
let mk = |mark_type| PlaylistMark {
mark_type,
play_item_ref: 0,
timestamp: 0,
};
assert!(
!mk(0).is_chapter_mark(),
"type 0 is reserved, not a chapter"
);
assert!(mk(1).is_chapter_mark());
assert!(!mk(2).is_chapter_mark(), "type 2 is a link point");
}
}

Some files were not shown because too many files have changed in this diff Show More