447 Commits
Author SHA1 Message Date
Matthew Jackson 7a63bac709 v1.6.6: bump version (freemkv-unlock git-pinned for the tag)
Release / verify (push) Successful in 5s
leak-guard / leak-guard (push) Successful in 35s
Release / release (push) Failing after 10s
Release / test (push) Successful in 2m15s
2026-08-20 20:03:44 -07:00
Matthew Jackson 14eafa0692 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
freemkv release 9560248c49 v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew 14408b75a0 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson e7b6f727d9 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 040bc8b14d drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson 7abcfaab72 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 341e079e1e Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 25893f1be4 CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit 4cd9b7b ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-4cd9b7b
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson efb69e3ba5 Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson 4cd9b7baa1 CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 02e7bc605d Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson 0563b58f2e Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 313460c97f ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson 68a1a55958 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson 0955730045 restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
Matthew Jackson d3103453f1 v1.6.4: bump version (freemkv-unlock git-pinned for the tag) 2026-08-15 09:13:26 -07:00
Matthew Jackson b64a96042e Trim single-clip titles to their playlist marks (1.6.4) 2026-08-15 08:14:31 -07:00
Matthew Jackson 074b1ee829 Stop an uncrackable VTS borrowing another VTS's title key
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.

`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.

Also, three things nothing could catch:

- The demux output filename took the stream's language raw while the
  base beside it was sanitised. A language code is three raw STN bytes
  through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
  encoding on real discs — a NUL in a path fails `File::create` with
  InvalidInput, taking the whole export down before one track file
  opened. `sanitize` now maps control characters too; it did not.

- `css::crack_key_scan`'s short-read handling was dead code under test:
  every source in the module returned the full request, so reverting
  `advance` to the requested count, or dropping the `.max(1)`, left the
  suite green. The `.max(1)` is load-bearing — without it a source
  returning `Ok(0)` never moves the cursor and never increments the
  budget, so the scan spins forever. That mutation now HANGS the test
  rather than failing it, which is the honest demonstration.

- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
  `#[cfg(test)] mod tests`, so it compiled in no configuration and could
  never fire — the dead gate the test above it was written to replace.
  Moved to module scope, and verified it now rejects a wrong constant at
  compile time.
2026-08-11 17:47:56 -07:00
Matthew Jackson 8f9bde9b9a Bound the third hostile CSV, and stop two docs overclaiming
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.

The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.

`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
2026-08-11 16:59:33 -07:00
Matthew Jackson 2d1563c63a Round 2: stop two of round 1's tests claiming more than they prove
Both are mine, and both pass with the bound they "cover" deleted.

`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.

`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.

Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
2026-08-11 11:54:28 -07:00
Matthew Jackson cbb127a175 Bound the commentary-index parse, and stop proving it with a clock
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.

It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.

`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.

The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.

Applies to both `sub_com1_idx` and `aud_com1_idx`.

NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
2026-08-11 11:30:53 -07:00
Matthew Jackson 9c6b7baf83 Cover the dir:// input door, and correct the CSS scramble-gate docs
Three round-1 audit findings on v1.6.0..HEAD.

Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.

Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.

Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
2026-08-11 10:23:37 -07:00
Matthew Jackson 3d738af58f restore freemkv-unlock path dep for local dev (post-v1.6.3) 2026-08-10 09:04:40 -07:00
Matthew Jackson 3e400edd4f v1.6.3: bump version (freemkv-unlock git-pinned for the tag) 2026-08-10 09:04:35 -07:00
Matthew Jackson 9c9de0e095 Carry the same licence and community files as the other repos
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.

SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
2026-08-09 22:08:22 -07:00
Matthew Jackson 539dd0131b Keep the cross-platform jobs for the release candidate
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.

Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.

They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
2026-08-09 20:24:07 -07:00
Matthew Jackson 5677f42c69 Release 1.6.3
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
2026-08-09 20:01:33 -07:00
Matthew Jackson f434b9cf2c Drop six unused crates, and align the rest with the workspace
Two problems, both invisible until the whole graph is looked at together.

DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are
declared here and referenced nowhere -- not in src, tests or benches.
They were being compiled, audited and offered version bumps forever for
no reason. Removing beats bumping.

cbc nearly survived the sweep: a substring search for "cbc" matches 44
occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked
used. Only a word-boundary search exposed it.

SKEW: this crate was the outlier on every shared dependency -- aes 0.8,
rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8
elsewhere. Cargo cannot unify across a major version, so it compiled
BOTH: 32 duplicated crates in the freemkv binary's graph, including two
complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two
digest stacks and two getrandom. Two crypto stacks in one product is
worth removing on its own.

The aes bump is an API rename -- BlockCipher-prefixed traits, Array for
GenericArray -- and the obvious translation uses Array::from_slice,
which the new version deprecates and clippy's -D warnings would reject.
These use the From<[T; N]> conversion the crate points at instead.

3441 tests pass in debug and release. The AACS crypto here is covered by
known-answer tests, so a byte-order or sizing mistake in that rename
could not have passed.
2026-08-09 18:37:24 -07:00
dependabot[bot]andGitHub b17761a24e Bump softprops/action-gh-release from 2 to 3 (#4)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 17:14:30 -07:00
dependabot[bot]andGitHub f2dd1d2e34 Bump actions/checkout from 5 to 7 (#5)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:58:27 -07:00
Matthew Jackson 671c3c7c8c Have Dependabot watch the dependencies, and land its PRs on dev
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".

Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.

Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.

The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
2026-08-09 16:48:31 -07:00
Matthew Jackson c6be942d1a Retire the README version bot
It fired on `release: published` and committed to main, the branch the
release points at its tag, so it could only ever leave main ahead of the
tag. release.sh now makes the same edit inside the release commit.

In this repo it was a no-op regardless: the pattern it rewrote does not
appear in this README, so it has been running and changing nothing.
2026-08-09 07:20:44 -07:00
Matthew Jackson 90d20304cd restore freemkv-unlock path dep for local dev (post-v1.6.2) 2026-08-08 22:18:33 -07:00
Matthew Jackson 50dfe877b9 v1.6.2: bump version (freemkv-unlock git-pinned for the tag) 2026-08-08 22:18:28 -07:00
Matthew Jackson 17622a1b59 Stop trusting a byte offset when the feed is not what was measured
A clip's span is measured over the title's full extents when the disc is
scanned, and a frame is placed by the offset it was read from. Those only
agree while the mux reads every byte the scan counted.

On a disc carrying a forensic segment it does not: the read plan omits the
units belonging to another device group, so fewer bytes are fed than the
spans describe, and the shortfall grows through the title. Every frame past
the first segment then looks earlier than it is — placed in a clip it did
not come from, or dropped at a join for failing marks it was never inside.
The spans still tile one another perfectly, so the check that asks whether
they can be trusted cannot see any of it.

The plan is now compared against the extents it was built from. When they
differ the spans are dropped and placement falls back to timestamps, which
is what that path is for and what the surrounding comment already promised
would happen when an offset stops meaning anything. Ordinary discs are
untouched: with no forensic segment the plan IS the extents.
2026-08-08 19:42:53 -07:00
Matthew Jackson 32824fba5b Make four guards testable, and one guard see both spellings
The provenance guard knew one way to lose a frame's source offset —
writing it as absent. Omitting the field entirely does the same thing,
because the frame type fills it in by default, and the guard read straight
past that. A parser rewritten into the second spelling would have kept its
green light while its track silently went back to being placed by guesswork.
It now reports which spelling it found and where.

The title-count clamp was asserted over a fixture too small to hold more
titles than the cap allows, so the walk stopped when the buffer ran out and
the clamp was never what bounded it — the assertion held with the clamp
deleted. The fixture now carries more entries than the cap.

The subdirectory limit was checked by restating the constant's own
definition; the guard itself had never run, and deleting it changed
nothing. The limit is lowered under test so a real folder can exceed it,
and the test now walks one and requires the refusal.

One test also carried two unrelated grounding notes while the test they
described had none, so reading the note above a test told you about a
different one.
2026-08-08 19:06:37 -07:00
Matthew Jackson 109afcdcf7 Keep both halves of a clip file that two play items share
A playlist may point at the same .m2ts from several play items — a
seamless split, a looped segment, multiple angles. The file has one set of
bytes, so it has one span, and the byte a frame was read from therefore
identifies the FILE, not which play item's range it falls in. Placement
took the first of them and judged every frame against its marks, so
everything past that range's end was treated as material the playlist
excludes and dropped: half the clip missing from the rip, with the
timeline still charged for its full duration.

The offset narrows a frame to the run; only its timestamp can finish the
job, and each play item carries its own marks. A frame is now matched
against the marks of the entry it actually falls in, falling back to the
first when it falls in none, which leaves genuinely-excluded material
dropped as before.

The test for this construction asserted only that such a playlist is
trusted, and never placed a frame from the second range — so the loss it
described in prose was invisible to it. It now places one from every range
and requires them all to survive.

The deferred-mux replay of buffered frames wrote them without their source
offset, quietly sending the head of every such title down the timestamp
heuristic the rest of this work exists to retire. It passes the offset
through now, and the frame writer that omits it is compiled out of the
library entirely: no production path can discard provenance any more.

Also: the AC-3 buffer-reuse test could not see the regression it named.
Feeding equal-sized packets, a fresh allocation per call yields the same
capacity as a reused one. It now feeds a large packet then a small one,
where only a reused buffer keeps the larger capacity.
2026-08-08 17:18:35 -07:00
Matthew Jackson dd749132d5 Audit round: an ADTS frame that cannot hold its own CRC, and three drifts
An ADTS header that declares a CRC follows must be at least nine bytes —
seven of header plus the two the CRC occupies — because the declared frame
length counts them. The structural gate compared against a flat seven and
never read the bit that says whether a CRC is there at all, so a frame
whose own header describes something impossible was accepted and handed to
the muxer as decodable.

The pipeline's spawn doc named Sweep, and the thread Sweep would have
created, as callers to look for. Neither has been in this crate since the
recovery passes moved out. The same paragraph already records fixing this
once, for a different departed caller — it simply drifted again a sentence
later, so it now says to name callers that live here or name none.

Whether a disc is structurally AACS-encrypted was spelled out by hand in
both the fast identify and the full scan. They agreed today; nothing made
them agree tomorrow, and disagreeing would mean the same disc reported
encrypted by one path and clear by the other. There is one definition now,
and the comment that pointed at it by line number points at its name.

The AC-3 parser built a fresh buffer on every packet — of the order of a
hundred thousand times per title — to work around a borrow it cannot
avoid. The copy stays; the allocation does not. The buffer is now lent out
and handed back, and a test pins that, because reverting it would be
invisible in behaviour.

Left alone deliberately: send/send_with_halt and finish/finish_with_halt
look like one action under two names, and are not. After the consumer
fails, one must still accept items and the other must refuse them; that
difference is what stops a producer reading an entire disc for a write
that died on its first frame. Collapsing them was tried here and the
existing test caught it. Both now say so where the choice is made.
2026-08-08 16:55:55 -07:00
Matthew Jackson 3ff2abbff5 Add an error for an image shorter than its recovery data
A recovery resumed against an image that has been truncated since the
previous pass — a full disk, an interrupted transfer, a remount — cannot
repair it: the pass only revisits the ranges the recovery data calls bad,
so everything past the cut stays a hole while the counts still describe a
whole disc. There was no way to say that. Reusing the recovery-data error
would have been wrong, because the recovery data is intact; the image is
not.

Registered in the uniqueness list, the code table and the range check, so
a future variant cannot silently reuse 6015 or map to the wrong code.
2026-08-08 13:35:44 -07:00
Matthew Jackson 03c6d20447 changelog: 1.6.2 2026-08-08 08:53:13 -07:00
Matthew Jackson 237794a6ea ci: run the dependents' test suites, not just a type-check
The `consumers` job checked out all five dependents against this commit and
ran `cargo check --all-targets`. That proves they still COMPILE, which
catches a changed signature and nothing else. The failures worth catching
here keep every signature intact and change behaviour: the library still
builds, the dependent still builds, and the dependent's tests are what go
red. Those never ran.

Run their suites instead. Same checkouts, same patched paths — only the
verb changes.
2026-08-08 08:46:17 -07:00
Matthew Jackson 835e97ce71 Give a frame that outruns its epoch's video a provisional offset
The demuxer interleaves, so at a boundary the streams do not reset on the
same frame: audio for the next segment can reach the muxer before the video
frame that opens the epoch it belongs to. Only the primary video track may
open one, so those frames rode the just-ended offset and landed a whole
segment in the past. Downstream the strictly-monotonic block nudge then
crushed the entire run onto one instant a tick apart, which is audible.

Such a frame is recognised on its OWN raw PTS, which within an epoch only
advances, so a large backward step is unambiguous — a different signal from
the shared frontier, which is what the old false-positive ratchet keyed on.
It then rides a provisional offset private to its track, computed the same
way the video path computes a real one, and drops it the moment the video
retires an epoch, so the run rejoins with no seam.

The offset is deliberately private: it never writes offset_ns, never
advances the frontier and never retires an epoch, so it cannot move the
video timeline. Letting a passive track open a REAL epoch was tried first
and inflated a 476.776 s title to 656.216 s, because every track meets a
boundary at its own pace and the video path rebased again on top.

Measured on a real DVD title with 8 cell boundaries: audio frames stamped
inside one cadence 120 -> 0, outsized gaps 7 -> 0, subtitle 1 -> 0, and the
video span byte-identical at 476.484. On the HD-DVD title the remaining
boundary gap on its first audio track falls from 0.999 s to 0.275 s.
Acceptance: 76 pass, 0 fail, 0 skip.
2026-08-08 00:48:10 -07:00
Matthew Jackson 70e1807e61 Place a straggler by its own epoch's end, not the current frontier
A non-video frame can reach the muxer after the video that opened the next
epoch has already been processed. It carries an old-epoch PTS, so adding
the new offset flings it forward by a whole clip. The remap that exists to
catch this compared the frame against the CURRENT frontier and only
accepted it within one backstep, which is the wrong yardstick: by the time
a straggler is seen the frontier has moved on into a new epoch, and how far
below it the frame lands says nothing about where it belongs.

Retiring an epoch now records the frontier it closed at alongside its
offset, and a straggler is judged against the end of the epoch its PTS came
from. Keeping the whole history rather than one previous offset also fixes
the case a single `prev_offset_ns` cannot express at all: a frame from two
or more epochs back.

Measured on a real HD-DVD title whose second audio track put its final
packet at 12834.587 s in a 6434.100 s file. That frame sits 23 s below the
current frontier — outside the old window, so it was refused — but 0.15 s
below the end of the clip it actually came from. It now lands at 6417.216,
one 32 ms cadence step after its neighbour, and the track spans 6416.160
with no outsized gaps. The other two streams are unchanged.

A title that never rebases retires no epoch, so the lookup finds nothing
and the mapping is bit-identical to before.
2026-08-08 00:02:57 -07:00
Matthew Jackson 418abfe79e test: cover the 1.6.1 provenance work where it was assumed, not asserted
Four parsers (dvdsub, flac, lpcm, mpegaudio) stamp a source offset on
every frame but had no test that read one back, so a regression to
`source: None` would have been caught only by the brace-balanced audit
in codec/mod.rs — a lint, not a behavioural check. Each now asserts the
emitted frame carries the offset of the packet that supplied its first
byte.

The Blu-ray feed spans had no direct test at all. Add one that walks a
multi-item playlist and requires the spans to tile the feed with no gap
or overlap; it catches a one-sector-per-clip drift, which is exactly the
error class that would misattribute frames near a seam.

`no_provenance_still_places_by_marks` asserted only that placement
returned something, which passes for a frame placed in the wrong clip.
Its probe timestamp lands in an overlap between two clips in the real
mark table, so pinning one clip would assert a coin-flip; instead
require the offset to be one that a clip actually containing that
timestamp would produce.
2026-08-07 22:16:02 -07:00
Matthew Jackson 42c62d46b6 Test the DVD title table's sector-offset arithmetic
The TT_SRPT pointer at 0xC4 is a SECTOR offset from the start of
VIDEO_TS.IFO. Reading it as a byte offset lands 1/2048th of the way in and
does not error — it silently returns a different title list, which is how
a disc enumerating 38 titles and an image enumerating 10 looked like a
scan difference rather than a failure.

The table already had tests for its entry layout, dedup and the 99-title
clamp. What it did not have was anything pinning the offset unit itself,
or the grouping of titles by title set, or a truncated table keeping the
rows it can read rather than failing a whole disc.
2026-08-07 22:01:37 -07:00
Matthew Jackson 06d0f9ef8b restore freemkv-unlock path dep for local dev (post-v1.6.1) 2026-08-07 19:08:46 -07:00
Matthew Jackson b3887be90f v1.6.1: bump version (freemkv-unlock git-pinned for the tag) 2026-08-07 19:08:42 -07:00
Matthew Jackson 3ecebf5b40 changelog: mark 1.6.1 UNRELEASED so release.sh can date it
release.sh dates the heading when it cuts the tag, and refuses a bare
'## [1.6.1]' because it cannot tell an undated heading from one it has
already stamped. Shipping a tag whose public changelog says UNRELEASED is
permanent and cannot be corrected inside that tag, so the check is right
to stop rather than guess.
2026-08-07 17:57:29 -07:00
Matthew Jackson 282651186c Guard the invariant that provenance is universal
Every emitted frame must carry the source byte offset of the packet it
came from. That held only for video for as long as it existed, and
nothing asserted it, so nothing caught it: ten parsers built frames with
`source: None` and a multi-clip title could not place audio or subtitles
by byte at all.

Finding them took a brace-balanced scan of the tree by hand — a regex
cannot do it, because a Frame literal contains nested braces and a
non-greedy match stops at the first `}`, which is how five sites survived
the first pass. This is that scan, as a test.

A second test fails if a codec module is added and not listed, since an
unchecked parser is exactly how the gap persists. Modules that emit no
Frame are named explicitly rather than skipped silently.

The guard tripped on itself twice while being written — first on its own
doc comment, then on its own string literals — so it strips line comments
and excludes its own module. A check that matches prose about the defect
rather than the defect is the same mistake in a different place.
2026-08-07 13:37:27 -07:00
Matthew Jackson c3c37380ae changelog: tighten 1.6.1, and record two entries it was missing
The entries had grown into narrative paragraphs; each is now the outcome
first and the mechanism in a sentence or two. The website changelog page
mirrors libfreemkv's, so it inherits this directly.

Two things that shipped in this cycle were not in it at all:

dir:// as a source. It was pulled from 1.6.1 on 2026-08-05 because the
folder reader produced a wrong title list on a real DVD. That defect was
the CSS descramble bug below, fixed since — the acceptance suite now
shows a folder reporting the same 38 titles as its ISO, with matching
streams, languages and runtime, and the CLI has been wired for it all
along. It ships, so it is listed.

The CSS title-table corruption itself: a disc enumerating 38 titles
produced a decrypted image enumerating 10, silently, at exit 0. That is
a data-integrity fix a user needs to know about.
2026-08-07 13:15:36 -07:00
Matthew Jackson 8ec71834dd Stamp the last frames that were still leaving without provenance
A real rip caught these. The warning added for a track placed from
timestamps under a seam plan fired once, on one subtitle track of a
23-clip title, eighteen minutes in — a display set that arrives as a lone
non-PCS segment is emitted straight through rather than accumulated, and
that path still built its frame with source: None.

Auditing the rest the same way found four more: flac, lpcm, mpegaudio and
the passthrough parser. All are one-PES-one-frame, so the packet's own
facts are the unit's facts.

Every Frame construction in every codec parser now carries a source,
checked by walking balanced braces rather than by eye — the earlier
count was taken with a regex that cannot see a block containing nested
braces, which is how these survived the first pass.
2026-08-07 10:01:39 -07:00
Matthew Jackson 991977f297 Place a non-advancing mark table by provenance
Marks that do not advance across a title are normal, not a defect: each
clip file carries its own STC, so one clip's IN has no ordering
relationship to the previous clip's. Refusing those tables dropped
exactly the branched titles this type exists for onto the inference path
that cannot read them, and that is the overrun.

Lifting it was tried once before every track carried a source offset, and
nine audio and subtitle tracks with nothing to place them by pinned
themselves to clip 0 and dropped most of the title. Every parser stamps
provenance now, so the clip comes from the byte offset and the marks keep
one job: whether a frame lies inside its own clip's [in, out].

Still gated on spans_trusted. Without usable spans there is no offset to
place by and inference is all that is left, so a table inference cannot
read is still refused rather than silently truncating a title.

A frame reaching the timestamp path under a plan now says so once per
track. It is not expected any more, and it is how a mostly-dropped track
reached a user without a single line in the log.
2026-08-07 07:49:53 -07:00
Matthew Jackson d4a32d3a26 Every parser now carries provenance, on the one shared buffer
dts, ac3 and truehd assembled access units across PES packets with three
private implementations of the same bookkeeping, and none of them carried
the source byte offset. They now hold a PesBuf, so a unit takes the
timestamp AND the source of the packet covering its first byte, from the
same mark, and no codec can answer that question its own way again.

dts is the reference case: its pts_marks already implemented the rule
correctly, and all 61 of its existing tests -- including the PTS
attribution ones -- pass unchanged on the shared type. That is the
evidence the type preserves the behaviour dts had right.

ac3 kept a single carry-over timestamp and one anchor offset, so it could
only attribute the first unit in a call; it now resolves each unit at its
own offset. truehd had no attribution at all beyond a running clock.

New tests cover the case that motivated this: a unit whose first bytes
arrive in one packet and whose remainder arrives in the next keeps the
FIRST packet's offset. At a clip seam those two packets belong to
different clips, and taking the later one places the audio in the wrong
one.

Clippy on the pinned toolchain caught an empty `if` block left where dts
used to clear stale marks -- restored as an explicit clear, with why it
is still needed once drain keeps the covering mark.
2026-08-07 07:45:32 -07:00
Matthew Jackson 813edd0965 One rule for which packet a unit's facts come from
A parser that assembles an access unit across PES packets has to answer
one question for every unit it emits: which packet carried this unit's
FIRST byte? Its timestamp comes from that packet, and so does the source
byte offset that says which clip of a multi-clip title it belongs to.
The packets that complete the unit carry later values that must not
override it.

That question was being answered three ways. DTS kept a deque of
(offset, pts) markers and took the one covering offset 0. AC-3 kept a
single carry-over timestamp. TrueHD kept its own. PGS and DVD subtitles
each held a pending unit with just a start time. None of them carried
the source offset at all, which is why provenance existed only for video
and why nine audio and subtitle tracks on a branched title had nothing
to place them by.

So it lives in one place now. PesBuf owns the bytes AND the marks;
PesFacts returns a packet's timestamp, source and discontinuity
together, so a parser cannot take one from one packet and another from
the next, because it does not assemble them itself.

The type answers WHICH packet. How a codec reads a timestamp out of that
packet stayed the codec's business at first, and that turned out to be
the same drift one level down: dvdsub read pts alone and returned 0 for
a packet carrying only dts, while everything else took pts.or(dts). Now
there is one derivation. It is not a choice between two fields — for
audio and subtitles there is no reordering, so dts IS the presentation
time and reading it is reading the same value from whichever field the
packet used. Reordering video never calls it; that path reconstructs
display order instead.

Migrated: adts, pgs, dvdsub. dts, ac3 and truehd follow.
2026-08-07 07:19:58 -07:00
Matthew Jackson b16eacd6b4 Restore the refusal: only the video parsers carry provenance
Accepting a table whose marks do not advance was wrong, and a real
22-clip title showed exactly how. The rip failed the drop-volume gate at
84 percent with more frames dropped than kept.

The cause is a fact the code never stated: provenance is VIDEO-ONLY.
Every audio and subtitle parser -- dts, ac3, adts, truehd, pgs -- builds
its frames with `source: None`, while hevc, h264 and vc1 propagate it.
So on that title the video track placed correctly by byte offset and
nine audio and subtitle tracks arrived with nothing to place them by.

The branch added to keep an unprovenanced frame from stranding did the
stranding itself. It held such a frame on its track's current clip,
reasoning that the cursor only ever advances under provenance and so
could not be wrong -- which is true only for a track that eventually
receives some. A track that never receives any stays pinned to clip 0
for the whole title, and every frame past clip 0's OUT mark is dropped.
That is what the disc showed: nine tracks pinned to clip 0, the first
drop nine milliseconds past its OUT.

So the refusal goes back, now with the real reason recorded, and the
test asserts it rather than asserting the behaviour that failed.

Lifting it needs the audio and subtitle parsers to carry provenance
first, stamped from the PES that STARTED each access unit -- the same
rule au_assembly already applies to video. The diagnostics that found
this are kept.
2026-08-07 03:21:21 -07:00
Matthew Jackson 776a4fd6eb Say which frame and which marks caused a provenance drop
The heuristic path has always logged its first drop per track with the
frame's timestamp and the clip marks it was judged against. The
provenance path did not log at all, so a title that dropped MOST of its
frames there failed the volume gate without a single line saying which
frame, which clip, or which marks — a full-length rip produced zero drop
events and an error, which is not a diagnosable failure.

Also pins the all-identical-spans hazard as a test. Every PlayItem
referencing one clip file gives every clip the same span, which passes
the tiling check and is therefore "trusted" while carrying no
information at all about which PlayItem a byte belongs to. That is not
what the discs on hand do — theirs have one distinct span per clip — but
the check conflates "the spans tile" with "the spans distinguish", and
only the second justifies placing a frame by its byte offset.
2026-08-07 03:05:32 -07:00
Matthew Jackson a0f76a4f18 Say how many DISTINCT feed spans a title's clips share
A seamlessly branched title re-references one clip file from several
PlayItems with different mark ranges, and those references share a single
feed span because the bytes are read once. A byte offset alone cannot
then tell them apart.

Whether that is happening on a given title is the fact that decides
whether provenance can identify a clip on its own, and it was not
observable. distinct_spans < clips says it directly.
2026-08-07 02:45:57 -07:00
Matthew Jackson afa0a213d6 Say which placement strategy a title got
Whether a title was placed by the seam plan or fell back to inference is
the single most useful fact about a branched rip, and it was invisible.
The two looked identical in the output, so telling them apart meant
rebuilding and re-ripping — which is exactly what it cost to find that
most branched discs were silently on the inference path.

Log both flags once at plan construction, and log each refusal with the
reason that caused it.
2026-08-07 02:01:47 -07:00
Matthew Jackson e5f92e591a Place a restarting-clock playlist by provenance instead of refusing it
A playlist whose clips each restart their own STC has marks that all
cover the same low values, so they are not points on one title-wide
clock. from_clips refused those tables outright and fell back to PTS-jump
inference.

That refusal predates provenance, and it is only about inference. With
the feed spans tiling the title the clip comes from the frame's byte
offset and the marks are never read across clips: each clip's offset_ns
maps its own private clock onto the output timeline, which is exactly
the right operation for a restarting STC.

Refusing unconditionally therefore turned the seam plan off on most of
the branched discs on hand — the very titles it was written for. They
ran on inference with a timeline minutes past the title's real length:
one 2h29m title came out with its audio and video both spanning 3h15m.

So the refusal now applies only when the spans cannot be trusted, where
it is still load-bearing: without a usable clock or a usable byte offset
there is nothing to place with.

Accepting these tables opens one hole, and it is the same stranding the
refusal guarded against. A frame arriving with no byte offset would fall
through to the mark heuristics, which on this kind of table are
meaningless and can strand a track on a clip it has already left,
dropping the rest of the title. Such a frame now stays on its track's
current clip; the cursor only ever advances under provenance, which is
never wrong about which clip a byte came from.
2026-08-07 01:17:04 -07:00
Matthew Jackson 79dbb2cf84 Resolve a frame clip by binary search, not a linear scan
The provenance lookup runs once per frame per track. A linear scan over the
clip list is fine for the 11-PlayItem fixture and is not fine for the real
hoard: two discs there carry 900 PlayItems in one title, and several carry
250-450, so a scan would be hundreds of comparisons on every one of millions of
frames.

spans_trusted already guarantees the spans tile the feed contiguously and in
order, which makes this a partition point. A repeated clip reuses its first
reference span, so the search walks back to the FIRST entry sharing it — the
bytes are read once, so the material is emitted once, at that entry offset, and
the answer is stable regardless of which duplicate the partition lands on.

Also adds a test at the real 900-clip scale: every clip places, output stays
monotonic across all 900, an arbitrary byte resolves to the right clip out of
ascending order, and a byte past the end belongs to no clip.
2026-08-06 23:22:38 -07:00
Matthew Jackson b9ca75f471 Identify a frame clip by provenance, not by inferring it from timestamps
Four audit rounds each fixed one rule in SeamPlan::place and broke another,
because the question the rules were answering has no answer. Inside a
seamless-branching overlap clip k OUT comes AFTER clip k+1 IN — 57.8s of
overlap on the real fixture table — so a single timestamp is legitimately
inside two clips, and a clip file is not trimmed to its marks, so it also
carries material from before its own IN. No rule over timestamps can say which
clip a frame came from, and each attempt was right for one disc layout and
silently wrong for another: 65s of rewind, 17 minutes stranded, 28 minutes
dropped, 55s refused.

Frames already carry the byte offset they were read from (PesFrame::source,
stamped by the TS demuxer). Clip now carries the byte span its stream occupies
in the title feed, recorded while the extents are gathered. So the clip is a
LOOKUP: the offset falls in exactly one span. There is no decision to get wrong.

Every track of a clip lives in the same stream file and therefore shares one
span, so video, audio and subtitles agree by construction. Divergence between
them — each track guessing separately under its own tolerance — is how audio
and video ended up on different clips and drifted apart in the first place.

spans_trusted gates the whole path: unless the spans tile the feed contiguously
from zero, an offset means nothing and provenance is ignored in favour of the
mark heuristics, which is the 1.6.0 behaviour. A broken map degrades instead of
confidently selecting a wrong clip for every frame. A clip referenced twice
reuses its first span (the bytes are read once) and is still trusted.

Sources that stamp no provenance — a mkv:// remux, the deserialize hop — take
the heuristics, which is what they have always used and where they have always
been right, because they have no overlapping clips to be ambiguous about.

36 timeline tests, six of them new and covering: the overlap case marks cannot
see, all tracks agreeing, out-of-marks material dropped AND counted, a holed
span map, a discontiguous one, a repeated clip, and no provenance at all.
2026-08-06 23:08:13 -07:00
Matthew Jackson adc8ee37be Say when an entry is omitted for not being a plain file
A symlink to a directory — a normal way to keep tens of gigabytes of streams
off the system disk — was dropped from the plan silently. `entry.file_type()`
reports the LINK, so such a subtree never enters `dirs`, and `metadata()` then
follows it and reports a directory, so it fails the is_file() test and was
skipped with nothing said.

The result is the worst class this release exists to close: PLAYLIST and
CLIPINF still synthesize, so the folder scans and enumerates titles, every clip
resolves to no extents, the mux takes its clean-EOF path, and a near-empty MKV
is written at exit 0.

It is still skipped rather than followed — following link targets invites
cycles and escapes from the folder — but at the same volume as the
unrepresentable-name skip eight lines above.
2026-08-06 22:48:27 -07:00
Matthew Jackson c3e82f28b5 Enforce the no-rewind invariant instead of guessing at clip crossings
Round 7 enumerated the state space of SeamPlan::place and found three more
holes in the mark heuristics — the third consecutive round to find a defect
here, each time in the guard the previous round added:

- a clip file opening ON its own IN mark, after a tail frame past the previous
  clip OUT had already advanced the cursor, read as another crossing and jumped
  a clip too far: 28 minutes of clip 6 dropped;
- the last clip was unguarded entirely (`clip + 1 < len` skipped the check), so
  the 9->10 seam rewound 75.6s with dropped_total() at 0 — the exact round-5
  signature, reachable without any corrupt input;
- a glitch mid-clip satisfies "inside the current clip", so the round-6
  stranding survived for the rest of that clip.

Three patches to the same heuristic, three wrong in a new way. So stop patching
it. Everything above the placement is still heuristics over marks, but the
property they exist to serve is now checked DIRECTLY: a track output must not
run backwards. A placement that would rewind by more than
DISCONTINUITY_BACKSTEP_NS is refused; the scan then looks for a LATER clip
containing the frame that does not rewind, which is exactly what a genuine
overlap crossing looks like; and if none exists the frame is dropped and
counted, because emitting it anywhere moves the track backwards.

The tolerance is the backstep constant because B-frame reorder legitimately
emits out of order by a fraction of a second, while a rewind that matters is
orders of magnitude larger.

The cursor moves ONLY when a later clip actually accepts the frame. My first
version of this advanced on a failed search, which strands a track exactly as
before — caught immediately by the round-6 regression test.

All 29 previous timeline tests still pass, including
continuity_preserves_forward_gap. The three round-7 scenarios now have a test
each, with the real 00801.mpls numbers.
2026-08-06 12:31:23 -07:00
Matthew Jackson 6b67c52249 Round 6: a large backstep only crosses when it would be misplaced
The round-5 fix advanced the clip cursor on ANY backward step over 3s. That
also fires for a corrupt PTS, and for a legitimate STC discontinuity inside one
clip — and nothing moves the cursor back, because a forward step matches
neither past_out nor stepped_back. Every later frame then sits below the new
clip IN and is dropped: on the fixture table that is ~17 minutes of one track
gone, and the only volume gate compares total drops against ALL tracks frames,
so it exits 0.

The branch now also requires the frame to be INSIDE the current clip marks,
which is the only case that would otherwise be silently placed at the old
offset — the rewind. A frame outside them needs no help: the containment check
drops and counts it, the cursor stays put, and the next good frame is placed
normally. Both behaviours have a test, each confirmed to fail without the guard.

Separately, the dir:// PES input path scanned the folder and never applied the
encryption verdict scan_dir exists to produce, so the same folder ripped
through one door and failed through the other asking for a key it does not
need. That logic now lives in one function, session::apply_folder_encryption_verdict,
called by both.
2026-08-06 11:02:48 -07:00
Matthew Jackson dd9e92ed52 Retry a short read instead of skipping past it
The previous commit bounded the crack scan inspection by the bytes actually
read, which stopped it examining stale buffer bytes from an earlier batch. But
the cursor still advanced by the REQUESTED count, so those sectors were skipped
outright — trading "scans the wrong data" for "silently scans less than it
thinks", on exactly the damaged media where a title key is hardest to find.

The cursor now advances by what was read, so the next iteration resumes where
the read stopped. Floored at one sector so a source returning Ok(0) cannot spin.

Found by reading the fix again rather than by the next audit round.
2026-08-06 10:15:53 -07:00
Matthew Jackson 65ccbcbd92 Scan only the sectors the crack actually read
read_sectors returns the number of bytes written and a source may return Ok
with fewer than asked — a recovery read over a damaged region does exactly
that. The crack loop discarded the count and inspected all n sectors of a
buffer that is REUSED across batches, so the tail still held the previous
batch sectors.

Cracking a key from those means cracking from data belonging to a different
extent, possibly a different VTS, while crack_span records the CURRENT one. A
key that opens nothing in this region is then installed, and its wrong
descrambles are only partly caught by the per-sector crib.
2026-08-06 09:31:37 -07:00
Matthew Jackson aa149099ec Make the public per-sector descramble refuse a non-pack sector
descramble_region was fixed to require the MPEG-2 pack start code, but
descramble_sector — the PUBLIC per-sector entry point, and the one the
module-level example tells callers to use — still keyed on the byte 0x14 flag
bits alone. The crate own documented guidance therefore led straight back into
the defect this release exists to fix: a VIDEO_TS.IFO sector holding 0x15 at
0x14 while starting 00 26 00 00 loses 1912 of its 2048 bytes, and because that
sector carries TT_SRPT the disc enumerates 38 titles while an image decrypted
from it enumerates 10, at exit 0.

It has no callers inside the crate, which is exactly why it survived three
rounds: nothing exercised it. Putting the guard inside the function rather than
in each caller is what keeps the safe path the easy one.

The integration test that covers it built its sector from the flag byte alone,
which no real scrambled sector looks like, so it stopped representing the path
it names — the same fixture-realism gap already fixed in four other places this
release.
2026-08-06 09:26:15 -07:00
Matthew Jackson 7deacf1761 A clip join must never rewind the output timeline
A hole in the seam fix itself, found in audit round 5 and confirmed by
measurement against the real 00801.mpls marks already in the test file.

Nothing upstream trims a clip stream to its marks, so a clip file can open with
material from BEFORE its IN. With the cursor on clip 5 (7708.99..7910.79) a
frame at 7845.00 — clip 6 pre-mark lead-in, 8s below clip 6 IN of 7853.00 —
sits further from that mark than the 250ms tolerance, so neither crossing rule
fires. The cursor stays on clip 5, and 7845.00 IS inside clip 5 range, so the
frame is PLACED with clip 5 offset.

Measured: output went from 3714.51s to 3649.51s. Backwards 65 seconds, with
dropped_total() still 0 — no counter, no gate, nothing noticed — while the
whole 57.8s overlap band was emitted a second time over clip 5 written tail.
That is the collide-and-flatten symptom this type exists to remove.

A track own PTS only ever runs forward inside a clip, so a backward step larger
than DISCONTINUITY_BACKSTEP_NS is always a clip change. Such a frame now
advances to the first clip that could contain it — which terminates on its own,
rather than running to the end of the list, which is what the next_in bound
exists to prevent — and a frame still below that clip IN is material the
playlist excludes, so the containment check drops and COUNTS it.

All 27 pre-existing timeline tests still pass, including
continuity_preserves_forward_gap, which is the one that catches an over-eager
heuristic here. The new test asserts the output never moves backwards at a
join and was confirmed to FAIL before the fix.
2026-08-06 09:20:39 -07:00
Matthew Jackson 47789f71bc Test the demux seam gates too
The same gap as the MKV muxer: DemuxSink builds its timeline with
TimelineContinuity::with_clips, but no test ever handed it a title carrying
usable PlayItem marks, so frames_mapped and dropped_total() were always zero
and both gates in finish() were unreachable.

Deleting either left the whole suite green while a demux:// export of a
seamless-branching title wrote a directory of zero-byte track files beside a
populated chapters document and reported success at exit 0.

The test gives the title two clips with marks that exclude every frame, and was
confirmed to FAIL with the gate removed.
2026-08-06 09:05:30 -07:00
Matthew Jackson c31d9fc88e Test the seam gate that stops a drift-broken title being skipped
Audit finding: both seam gates in MkvMuxer::finish were dead code under test.
set_clips was never called anywhere in the suite, so continuity.dropped_total()
was always zero and neither branch could be reached.

The ordering of the two zero-frame checks is load-bearing and the comment says
so: a title the seam plan emptied also has a zero frame count, and MkvInvalid
is classified by is_skippable_title_stub as an empty nav/menu stub — so
reporting it that way makes an all-titles rip drop a real feature and finish the
rest at exit 0. Swapping the checks left the entire suite green.

The new test builds a two-clip plan (from_clips needs at least two, strictly
increasing) whose marks exclude every frame, and asserts both that the error is
SinkWroteNothing and that it is NOT classified as skippable. Confirmed to FAIL
with the two checks swapped.
2026-08-06 08:45:58 -07:00
Matthew Jackson 0e8c31a9c3 Round 3: fix three defects introduced by the round-2 fixes
Auditing my own fixes found all three. None were in the original code.

The VTS crack sort was byte-wise case-SENSITIVE while the filters that select
those files (vts_group_of / is_title_vob) are case-insensitive. On a
case-sensitive volume a set holding vts_01_1.vob beside VTS_01_2.VOB sorted
part 2 first, because V (0x56) precedes v (0x76) — reintroducing exactly the
budget-exhaustion the ordering exists to prevent. Now sorted on the same
uppercase normalisation the filters apply.

Refusing a name that round-trips to empty aborted the WHOLE plan. A sidecar
folder named with a single emoji made a backup un-rippable that 1.6.0 handled
fine, and reported it as a collision with a file that does not exist. It is now
skipped with a warning: the entry is unaddressable either way, but one
irrelevant file should not cost the user their rip.

The mtime check now applies only to files whose CONTENT the plan read — the
IFOs, whose bytes 0xC0/0xC4 place every VOB. Everything else is planned from
size alone, which is already checked, so comparing mtime there bought nothing
and risked a real false positive: disc backups commonly live on exFAT/FAT32,
which stores local time, so a long rip spanning a DST transition would see a
whole-hour shift on an untouched multi-gigabyte VOB and abort hours in.
2026-08-06 08:25:54 -07:00
Matthew Jackson 03d088abfc Catch names the reader cannot tell apart, and in-place content changes
Two dirimage findings, both silent-wrong-output.

The per-directory uniqueness check compared raw host names, but the reader does
not see raw host names: parse_udf_name trims leading and trailing whitespace
and drops any code unit char::from_u32 rejects — which is every half of the
surrogate pairs the encoder emits for non-BMP characters. So " 00000.m2ts" and
"00000.m2ts", or "A<astral>.m2ts" and "A.m2ts", were two entries at plan time
and ONE name at read time. find/read_file take the first match, so a title
resolved to the wrong file extents and muxed the wrong bytes at exit 0 — the
exact shadowing DirNameCollision exists to prevent.

The key is now derived by round-tripping the name through the very encoder and
parser that will be used, so it cannot drift from them. A name that survives
that round trip as empty is refused outright: it would exist in the image and
be addressable by nothing. The new test builds a real folder, calls plan, and
was confirmed to FAIL with the fix reverted.

Separately, the plan-vs-read revalidation compared file LENGTH only, while the
plan depends on CONTENT: a DVD VOB placement comes from bytes 0xC0/0xC4 of its
IFO, and IFOs occupy a whole number of sectors so an in-place rewrite keeps the
length. A re-authoring tool touching the folder mid-rip would pass the size
check while every title extent pointed at stale sectors. mtime is now compared
alongside size, and only when both sides report one, so a filesystem without
timestamps falls back to the old behaviour rather than failing every read.
2026-08-06 08:00:23 -07:00
Matthew Jackson 84e0ba9fa8 Anchor the folder encryption probe on a unit boundary
probe_folder_encryption sampled AACS units starting from the largest extent
anywhere in the disc. AACS units are 3 sectors and a unit boundary is only
guaranteed at the START of a clip, so that anchor is only correct when the
largest extent happens to be a clip first extent.

For any clip over ~2 GiB it is not. The planner caps an allocation descriptor
at MAX_AD_BYTES = 524287 sectors, so every full piece of a split file ties on
sector_count and max_by_key returns the LAST tie — an extent beginning
(k-1)*524287 sectors into the file. 524287 % 3 == 1, so that start misses the
unit boundary for two file sizes in three.

The 6144-byte sample windows then begin mid-source-packet and the byte read as
the CPI flag is content. Both verdicts fail in a costly direction: a decrypted
folder is rejected as encrypted (DirImageEncrypted on something perfectly
rippable — the exact case scan_dir was added to rescue), or real ciphertext
reads as clear and the mux writes it out as video at exit 0. is_unit_aligned
cannot catch it: it measures against the same wrong base.

Now the largest TITLE first extent, which is unit-aligned by construction and
is also the more meaningful sample — the main feature rather than whichever
fragment happened to be biggest.

The existing tests could not reach this: their fixture m2ts is 786,432 bytes,
a single extent, which is always its own first.
2026-08-06 07:37:59 -07:00
Matthew Jackson bc5e3453b4 Order a VTS crack by filename, not by directory order
Round 1 removed a largest-first sort from resolve_vts_key because
largest-first is the 1.5.1 garbage bug. Auditing that fix showed it was only
half right: `planned` comes from walking the UDF directory, which yields File
Identifier Descriptors in on-disc authoring order with nothing sorting them. So
deleting the sort did not restore playback order, it left the order undefined —
whatever the disc happened to list first.

DVD-Video numbers a title sets VOBs in playback order by spec (VTS_xx_1.VOB ..
VTS_xx_9.VOB, single digit), so ascending filename IS playback order and is
deterministic regardless of how the directory is laid out.

Order decides correctness here: crack_key shares one sector budget across the
whole extent list, a CSS DVDs biggest cell opens with a long clear run, and CSS
recovers the title key from scrambled data itself. Starting in the wrong place
can exhaust the budget without ever meeting scrambled data, whereupon the
caller falls back to the disc-wide key and the whole VTS is descrambled wrongly
— corrupt PES behind an intact header, written out at exit 0.
2026-08-06 07:31:35 -07:00
Matthew Jackson 6d9791affc Audit round 1: playback order, silent title drops, image durability
Four fixes from the first audit round. Every finding was verified against a
pinned tree and read directly before being accepted.

resolve_vts_key sorted a VTS title-VOB extents largest-first. That is the 1.5.1
garbage bug, and it grew back in a new code path: the comment claimed it
"matched the scan heuristic", but that heuristic WAS the bug and had already
been fixed in decrypt_keys_for_title, which documents the rule (PLAYBACK ORDER,
never largest-cell-first) and pins it with a regression test. A CSS DVDs biggest
cell opens with a long clear run and crack_key shares one sector budget across
the extent list, so starting there can exhaust it without ever MEETING
scrambled data — and CSS recovers the key from scrambled data itself. The crack
then returns None, the caller falls back to the disc-wide key, and every VOB in
that VTS is descrambled wrongly: corrupt PES behind an intact header, written
out as a complete extract at exit 0.

parse_pgcit dropped titles silently in THREE places — an unparseable PGC, an
out-of-range PGC index, and a truncated entry table. The finder caught one; the
other two turned up on reading the function. parse_vmg already counts and warns
per skipped title SET for exactly this reason, and this was the last place a
disc could quietly report fewer titles than it has.

write_image called flush() and returned Ok. flush() only pushes bytes into the
page cache and promises nothing about durability, so a 6-90 GB image could be
reported complete while still unwritten — a crash or an unmounted volume then
leaves a truncated file the caller was told was finished. Now into_inner (so a
buffered-write error surfaces instead of being dropped by BufWriter::drop)
followed by sync_all.

timeline used abs() on a saturating_sub result. Every other comparison in that
module is saturating because the timestamps come off a disc and are not
trusted; abs() panics on i64::MIN, which saturating_sub can produce.
2026-08-06 07:11:52 -07:00
Matthew Jackson d1551eb588 Rename is_scrambled to has_scramble_flag_bits
The 1.6.1 DVD fix removed the USE of the loose predicate but left the
predicate sitting there with the better name. Anyone asking "is this sector
scrambled?" finds is_scrambled before is_scrambled_pack, and reintroduces the
defect that destroyed 1912 bytes of a real VIDEO_TS.IFO — the sector carrying
TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
enumerated 10, silently, at exit 0.

Byte 0x14 only means "scrambling control" inside an MPEG-2 pack. In an IFO,
UDF or ISO 9660 sector it is whatever that format stores there. The new name
says what the function actually tests and nothing more, so the honest question
has the obvious name and the dangerous one has to be asked for deliberately.

Its doc comment also claimed a caller, decrypt::decrypt_sectors, that does not
exist — so the name was an invitation and the documentation was an argument
for accepting it. It has no production callers at all; it stays public because
an integration test asserts the flag extraction directly.
2026-08-05 22:27:04 -07:00
Matthew Jackson c875df49e3 Scope the qa matrix to the platforms each crate actually supports
The first cut ran release-profile tests on all three platforms everywhere,
which invented coverage no crate had ever claimed. bdemu is Linux-only — its
ci.yml has no macOS or Windows job at all, and its tests call
ExitStatus::signal(), which does not exist on Windows, so they do not compile
there let alone run. The rest follow the policy ci.yml already set: tests
execute on Linux and macOS, Windows compiles them so the cfg(windows) halves
still get codegen before release time.

That policy is now explicit in qa.yml as a windows-build job rather than
implied by which jobs happen to exist.

Recorded, not papered over: running the suites on Windows DOES find real
defects — autorip has five Windows path-handling failures in mover.rs and
freemkv-engine has three tests hardcoding /dev/null. freemkv ships a Windows
GUI, so they matter. They are pre-existing rather than 1.6.1 regressions, and
fixing them is its own piece of work.

Also build the binary before cli-parity.sh. cli-integration.sh builds its own;
cli-parity.sh only checks for one and exits 2 with "build it first", so both
legs died before running a single case.
2026-08-05 21:22:30 -07:00
Matthew Jackson 93ad1f4854 Give Windows a free-space gate, and stop two tests timing the scheduler
Three release-profile failures on macOS and Windows, all found by the qa gate
on its first run. Release-profile tests on those platforms had never run
before it existed, so none of these were regressions — they had simply never
been visible.

available_space returned None on every non-unix target. That did not merely
skip a test, it skipped the GATE: a Windows user extracting a disc to a full
volume got a confusing failure part-way through instead of a clear refusal up
front, and Windows is where the GUI ships. GetDiskFreeSpaceExW is declared
directly against kernel32, matching how scsi::windows already reaches Win32
rather than pulling in a binding crate for one call. It asks for
FreeBytesAvailableToCaller, which accounts for per-user quotas — the same
question f_bavail answers on unix.

The other two asserted on wall-clock timing with no margin:

- sleep_until_halted_wakes_mid_sleep bounded the wait at 350 ms and measured
  377 ms on a loaded runner. That bound measures the scheduler, not the wake.
  What the test is for is distinguishing "woke because the flag flipped" from
  "woke because the 10 s timeout expired", and 2 s does that just as well.

- abandon_loses_to_a_close_already_committed released at 600 ms against two
  300 ms grace windows plus a 250 ms poll cadence, so the windows could expire
  first and the caller abandoned — a race, not a defect. The intervals are
  scaled up so jitter is small relative to them; the ordering under test is
  unchanged, only the margin.
2026-08-05 21:20:29 -07:00
Matthew Jackson d0393fb629 Stamp every qa push as a release candidate
Every push to qa now tags v<version>-rc<N>, N incrementing, before the gates
run. That answers "which build is on qa, and is it the one I tested?" without
anyone having to remember it.

The tag lands whether the run goes green or red, deliberately. A red candidate
needs a name more than a green one does: "rc3 failed release-tests on windows"
is a sentence you can act on, "qa is red" is not. Red on qa is the gate doing
its job — the branch saying this is not production worth yet.

release.yml now excludes v*-rc*. Its trigger was v*, which matches the
candidate tags, so without this every push to qa would have built and PUBLISHED
a GitHub release — including for the candidates that failed.
2026-08-05 21:03:01 -07:00
Matthew Jackson c64bc311ff Add the qa gate: dev for speed, qa for proof
dev -> qa -> main, across every public repo.

dev is where work lands and is meant to be pushed to often, so ci.yml stays
the fast answer: fmt, clippy, unit tests, leak-guard. qa is the release
candidate, and qa.yml is the claim that a commit is production worth —
release-profile tests on all three platforms, and the Linux cross-target
clippy that the local precommit gate has always run but CI never did.

Release profile matters as its own gate: overflow checks are off, debug_assert
is compiled out, and inlining changes what the optimiser can prove. A test
that only passes in debug never guarded the binary anyone ships.

qa.yml checks siblings out at qa rather than dev. This is not a monorepo, and
a qa run resolving its dependencies from unreleased dev tips would go green on
a combination that is not the one shipping — the exact mismatch the branch
model exists to prevent. ci.yml now tracks whichever branch triggered it for
the same reason, since it also fires on qa.

The consequence is that repos move to qa together, in dependency order. qa is
backfilled from main so an unchanged crate still presents working code to the
crates built on it.
2026-08-05 20:59:34 -07:00
Matthew Jackson a018e1adc4 Require a pack start code before descrambling a sector
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a
sound test once a caller has committed to a title's VOB data, where every
sector is an MPEG-2 PS pack and byte 0x14 always means what it says.
descramble_region is not such a caller: it is handed arbitrary regions of a
disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where
byte 0x14 is whatever that format happens to store there.

Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at
offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test
read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes.
That sector carries TT_SRPT, so the title table went with it — the disc
enumerated 38 titles and an image decrypted from it enumerated 10, silently,
at exit 0.

is_scrambled_pack already existed with the right predicate. Use it here. It
costs nothing: a genuinely scrambled VOB sector always carries the pack start
code, and no IFO sector does.

Verified end to end — the decrypted image's `info` output is now identical to
the source disc's, 38 titles both, differing only in the CSS: Encrypted line.

The fixtures moved with it. Four of them built a sector by setting byte 0x14
alone, which no real scrambled sector looks like; they now build packs.
2026-08-05 20:59:14 -07:00
Matthew Jackson 1d35dcf7c6 Report the resolved extent alongside each IFO read
The diagnostic hashed IFO contents but not where they came from, so a
content difference could not be told apart from a path resolving to a
different place. It now prints file_start_lba and file_extents next to the
hash.

That measurement is what inverted this investigation: both a CSS disc and
its decrypted copy resolve VIDEO_TS.IFO to the same extent, and only one
sector of the fourteen differs — by exactly a descrambled payload with the
scrambling-control bits cleared. So the decrypted copy holds the correct
bytes, and an iso:// scan of a CSS disc is parsing a still-scrambled IFO
sector, because scan_iso opens a plain FileSectorSource.
2026-08-05 20:24:20 -07:00
Matthew Jackson 77ad147563 Diagnostics that localise the decrypted-DVD title loss
Two opt-in dumps, driven by FMKV_IMAGE, that narrow where a decrypted DVD
image loses titles:

dump_title_sets_for_an_image reports what survives parse_vmg. On one disc
the CSS image yields 13 title sets and 38 titles; its decrypted copy
yields 8 and 10. Sets 8, 9, 10, 12 and 13 are dropped outright, set 11
parses but returns no titles at all, and set 7 returns 3 of 5.

dump_vts_ifo_reads_for_an_image reads every VTS IFO and hashes the
CONTENT. All thirteen are byte-identical across the two images, so
parse_vts is handed the same bytes and the same TT_SRPT info and still
fails on one of them — the divergence is in what it reads from the READER
afterwards, which is file_start_lba and the PGCIT.

An earlier version of the second dump compared only length and magic and
so wrongly reported the images as identical; it hashes the bytes now.
2026-08-05 20:14:46 -07:00
Matthew Jackson 5c64662213 Say when a title set is dropped from a DVD scan
parse_vmg skipped any title set whose parse failed, with no log and no
counter. A real disc enumerated 38 titles from one image and 10 from
another, and the 28 discarded failures were invisible — the symptom read
as a scan difference rather than as dropped reads, which is most of why
it took so long to localise.

Behaviour is unchanged: a disc may legitimately carry placeholder TT_SRPT
entries, so one failure is still not fatal. It now warns per skip with the
title set and the error, and once at the end with kept-versus-declared.

Also adds an opt-in diagnostic that reads every VTS IFO from an image, to
separate a read failure from a parse failure. It reports all 13 sets
reading identically from both a CSS image and its decrypted copy, which is
what proves the 38-to-10 loss is downstream of these reads.
2026-08-05 19:41:43 -07:00
Matthew Jackson 1f70398774 Two opt-in scan diagnostics for the DVD decrypted-image defect
dvd_placement_invariant_on_a_real_folder checks, per title set, the sum
ifo.rs relies on: file_start_lba(VTS_nn_0.IFO) + vtstt_vobs must land on
VTS_nn_1.VOB. It reports all 13 sets correct on a real DVD folder, which
is what excluded placement as the cause.

dump_titles_for_an_image prints every title a scan produces with the
numbers canonical_title_order sorts on. It is what showed the real shape:
the same disc scans to 38 titles as a CSS image and 10 once decrypted,
with identical capacity and a byte-complete image.

Both are #[ignore]d and read their target from the environment, so they
cost the gate nothing and are there for whoever picks the defect up.
2026-08-05 19:33:58 -07:00
Matthew Jackson 35c5eedc20 Round 5: fix the gates added in round 4, and two placement holes
The zero-frame check ran before the seam gate, and its error is
classified as a skippable nav stub — so a title the plan dropped
ENTIRELY was reported as an empty stub and an all-titles rip would omit
a real feature and finish the rest at exit 0. The seam case is decided
first now, with a code that is not skippable.

The demux sink read a frame's track kind out of the FILTERED slot, which
is empty for a class the export drops. On an audio:// or sub:// export
the video track was therefore called non-video and handed the permissive
crossing rule — the same defect round 4 fixed for a Dolby Vision layer,
reintroduced one file over. Video tracks are now recorded before the kind
filter, beside the primary-video reference that exists for this reason.

Its frame counter counted frames PLACED, not written, while its name and
doc claimed otherwise. Renamed and documented for what it is, including
that it cannot see a single lost track among many.

Placement: files in a subdirectory of VIDEO_TS were never given data.
They were declared at full size with no extents, so they appeared in the
tree and read as nothing. The same folder under BDMV was always placed
correctly. And the duplicate title-set guard keyed on the constraint maps,
so an IFO declaring no offsets inserted nothing and a colliding second IFO
went undetected — it keys on the groups seen now.

Display for SeamPlanDroppedMost and ShortImageRead discarded their
payloads, and four new variants were missing from the code-uniqueness
test.
2026-08-05 18:40:14 -07:00
Matthew Jackson f4b95b3dea Test the image-size cap against a real folder
The cap was added without a test. Its companion — a modest oversize is
honoured as a gap — already existed, so this pins the other side: an
offset past the ceiling is refused rather than grown into, which is what
keeps a rewritten IFO from planning a multi-terabyte image.
2026-08-05 18:30:23 -07:00
Matthew Jackson 0f61be00b7 Make the drop count gate the verdict, not just the log
Round 3 counted frames the clip marks excluded and reported them at
finish. Counting is not bounding: the only other gate was a global
zero-frame check, and its error is additionally classified as a skippable
nav stub, so a title whose marks do not line up with its PES clock could
discard almost all of itself and still exit 0 — a two-hour feature
emitting seconds, which is the defect this change set already shipped
once. Dropping more than was kept is never a real join, so it now fails.

The demux sink had no zero-output guard at all, so a fully-dropped title
finished cleanly: a directory of zero-byte track files beside a populated
chapters document. It now refuses, keyed on frames having been OFFERED —
a chapters-only export, or a track class the title does not carry,
legitimately writes none, and two existing tests correctly said so.

A title set's placement group is the parsed number, so VTS_01_0.IFO and
VTS_1_0.IFO land on one key and the second silently overwrote the first's
constraint, placing a VOB where the IFO the reader uses does not point.
Refused rather than resolved by arrival order.
2026-08-05 18:25:47 -07:00
Matthew Jackson f4fb5c65e0 Key the seam-crossing rule on reorder, not on driving epochs
A Dolby Vision enhancement layer is a second video track: it does not
drive epochs, but it does carry B-frame reorder. The crossing rule was
keyed on driving epochs, so the EL took the branch whose premise is that
the track has no reorder — and its ordinary reorder dip near the end of a
clip, which during an overlap also lands inside the next clip's range,
was read as a join. The EL was then placed on the next clip's offset, out
of step with the base-layer frame it must be co-timed with by the width
of the overlap: the same desync the per-track cursor was added to remove,
reintroduced for one track.

The property the rule actually depends on is whether a backward step can
be reorder, so it is now keyed on that. Both sinks derive it from the
track kind rather than from the epoch driver.
2026-08-05 17:52:13 -07:00
Matthew Jackson c8fafec393 Keep the seam plan to Blu-ray, and stop a missed crossing truncating a title
Two findings from the same escalation, both silent-wrong-output.

The plan was built for every multi-clip title. Only a Blu-ray PlayItem's
IN/OUT are positions in the clock the PES PTS runs on. HD-DVD fills the
same fields from the XPL's title-relative times and a DVD's come from
cell tables, so a plan built from them is an identity map with a drop
filter: it suppresses the layer-break rebase inference performs, and
drops whatever falls outside marks the PTS was never measured against. An
earlier reading of this called HD-DVD safe because its marks are
contiguous and every computed offset was zero — true, and irrelevant,
because they were zero in the wrong clock. Gated on the content format,
with a test using an HD-DVD-shaped table that the clock check alone
accepts.

The crossing test was also one-shot. A table whose clips restart their
own bases could miss it, and a missed crossing STRANDS the track: every
later frame falls outside the stranded clip's marks and is dropped for
the rest of the title. Counting drops, which is all the previous round
added, does not bound them. A table that is not one advancing clock is
now refused outright and falls back to inference, which is the documented
safe path for those titles.

Also from the same round: read_sectors added an unchecked lba + i, where
callers deliberately saturate their LBAs — a wrap folds the read back to
a low sector and hands the muxer another file's bytes. classify added 1
to two numbers parsed verbatim out of a filename. read_head used a single
read() where a short read on a network mount silently records no
placement constraint at all. And the page-cache eviction added last round
released only the read that crossed its threshold rather than everything
accumulated, so seven eighths of what was read stayed pinned.
2026-08-05 17:31:43 -07:00
Matthew Jackson 764535bb7d Make the name-cap tests exercise the planner, not the constants
The round-1 tests asserted arithmetic about MAX_CS0_NAME_BYTES and never
called plan(), so both would have passed with the guard deleted — which
is the failure mode this audit exists to catch, committed by the audit's
own fix. They now build a real folder containing a 255-byte name and
require the planner to refuse it, plus a companion proving a name at the
cap is still accepted so the guard is not merely refusing everything.

The subdirectory cap keeps its arithmetic-only test — creating 65,535
directories is not reasonable in a unit test — but now says so instead of
implying coverage it does not have.
2026-08-05 17:21:43 -07:00
Matthew Jackson 60d9cc1bac Audit round 2 fixes: an unsafe default, four omissions, and two swallowed errors
The folder encryption probe returned "not encrypted" when it had sampled
nothing at all — a title shorter than one aligned unit skipped the loop
entirely. That verdict CLEARS the structural one an AACS directory
raised, so a genuinely encrypted folder would have been ripped as clear
and written ciphertext as video at exit 0. With no evidence it now keeps
the structural verdict, and its bounds arithmetic no longer trusts
disc-derived values not to wrap.

Reading an IFO header swallowed every I/O error and returned an empty
buffer, which sent each placement offset through unwrap_or(0) and
recorded no constraint at all — a permission error on one file produced a
silently misplaced VOB. The directory walk swallowed the same class while
claiming to skip only vanished files. Both now propagate; only NotFound
is skipped.

Four things the round-1 changes left inconsistent: two new error codes had
no doc comments, were absent from the io::Error mapping, printed no path
in Display, and were missing from the test that proves codes are distinct.
The demux sink dropped frames silently while the MKV muxer reported them.
And set_clips had been inserted INTO write_frame's doc comment, leaving
write_frame undocumented and its paragraphs describing the wrong function.

uid/gid used 0 as "not specified"; UDF's sentinel is 0xFFFFFFFF, and 0 is
root.
2026-08-05 17:16:03 -07:00
Matthew Jackson cbb3517afe Bound the synthesized image, and stop it pinning the page cache
A DVD title set records where its VOBS begins as an offset inside its own
IFO, and the planner honours that offset because honouring it is what
makes a real backup readable. Nothing bounded it: a regenerated .BUP or a
hand-assembled folder naming an offset far past the content grew the
image to wherever it pointed — a u32 sector count reaches ~8.8 TB, and
writing that to an iso:// destination fills a disk with zeros before
anything notices. Capped at 128 GiB, which clears BD-100 with room.

Metadata is materialized up front and held for the life of the image, at
a 2 KiB File Entry per node, so the 100,000-entry cap alone permitted
~205 MB of it for content of no size at all — and the mux holds two
images at once while probing. The module claimed a budget of a few MiB;
that budget is now enforced rather than asserted.

Host reads had no page-cache eviction. The ISO source documents what that
costs, measured: an 85 GB read pins the whole file, starves the writer,
and collapses the mux to 2.7 MB/s against 70 MB/s isolated. A folder
source reads host files the same way, so it now uses the same eviction —
the hints move from private-to-that-module to crate-internal rather than
being reimplemented.
2026-08-05 17:10:19 -07:00
Matthew Jackson 3980aa8976 The VMG's 0xC4 is TT_SRPT, not VMGM_C_ADT
The placement code skips that field for the Video Manager, which is
right, but said it was skipping it because the field is the menu cell
address table. It is the title search pointer table, and it is an offset
inside the IFO rather than a pointer to another file — which is the
actual reason it constrains nothing. ifo.rs reads the same offset under
the correct name, so the two would have drifted.
2026-08-05 17:03:19 -07:00
Matthew Jackson ffbc1d8399 Audit round 1 fixes: sparse-track joins, silent drops, and two encoder wraps
A sparse passive track — a subtitle with no event near a clip's mark —
was held to the dense-video crossing window, so it stayed on the previous
clip's offset until its PTS passed that clip's OUT and every event in
between was mistimed by the overlap. Video keeps the tight window,
because its backward steps are also B-frame reorder; passive tracks have
no reorder, so any backward step into the next clip's range is a join.

Frames the marks exclude were dropped without a trace. Dropping is right
at a join, but this codebase has shipped complete-looking wrong output
before, so the count is kept per track and reported when the mux
finishes, alongside the pre-cluster counter that exists for the same
reason.

A File Identifier Descriptor records its name length in one byte, and the
length was narrowed with a cast: a 255-byte name — POSIX NAME_MAX,
entirely ordinary — encodes to 256 and wrote zero, which would read every
later entry in that directory from the wrong offset. A directory's link
count is 16 bits and was computed as 1 + subdirectory count, which the
global entry cap alone permits overflowing. Both are refused while
planning, where the tree can still be rejected cleanly.

The module and struct docs described inference as the whole algorithm;
they now say which path decides what.
2026-08-05 16:58:49 -07:00
Matthew Jackson 9247e7da2f changelog: dir:// is not in 1.6.1
The folder reader produces a wrong title list on a real DVD — the title
table is read at a sector offset relative to VIDEO_TS.IFO's start, and a
synthesized layout does not put that file where the offset resolves the
same way. Output is silently wrong at exit 0, so the feature is not
wired to the CLI in this release. The library module stays, unreachable,
and ships once both disc families are verified against real content.
2026-08-05 16:44:23 -07:00
Matthew Jackson 6c92370013 changelog: dir:// as a source, and the multi-clip Blu-ray timing fix 2026-08-05 15:47:47 -07:00
Matthew Jackson 3090314717 Cross a clip join per track, not on the video's frame
The first cut placed every track with the cursor the video had moved. At
an overlap join the previous clip's audio is still arriving after video
has crossed, and those tail frames sit inside both clips' mark ranges —
so they took the new clip's offset, jumped forward by the overlap, and
collided with the new clip's own audio, which the muxer's monotonic nudge
then flattened. A remux confirmed it: the timeline length was already
correct and the original symptom was still there, 169 audio packets on
the tick floor.

A track's PTS only runs forward inside a clip, so its own backward step
to the next clip's IN is its crossing. That is per track, so the cursor
is too.
2026-08-05 15:46:40 -07:00
Matthew Jackson ce246a1c87 rustfmt the seam-plan tests
Line-wrapping only, no behaviour change.
2026-08-05 15:28:40 -07:00
Matthew Jackson 617532e6ff Merge branch 'fix/mpls-seam-timeline' into dev 2026-08-05 15:24:37 -07:00
Matthew Jackson dfd2f023d0 Place Blu-ray clips by the playlist's marks, not by guessing at PTS jumps
A seamless-branching title's PlayItems do not chain contiguously: one
clip's OUT can sit after the next clip's IN, where the disc stores the
join twice, or before it, where the playlist skips material. The mux
never saw those marks — its own header said so — and inferred seams from
PTS jumps instead.

Inference cannot recover this. A forward jump is ambiguous: it means the
playlist skipped, or it means frames were lost to damaged media, and
compressing the latter would falsify timing on exactly the rips that most
need it faithful. An overlap smaller than the B-frame reorder threshold
is invisible to inference entirely, and its duplicate content then
collided in the muxer, where the monotonic nudge flattened a run of audio
onto the tick floor and put sound ahead of picture for the rest of the
film.

Measured on one 11-PlayItem title: the file declared 7893.385 s, which is
what the playlist says the title is, and carried packets to 8029.298 s.
Both numbers came from the same program on the same disc. Four skips
totalling 135.9 s became dead timeline, and a 1.79 s overlap put audio
1.8 s ahead at the half-hour mark. Five of forty-seven titles were
affected; every single-clip title was exact.

So the marks are read. Each clip contributes exactly out - in, laid end to
end, so the output runs as long as the playlist says and a join never
rewinds. Titles without usable marks — DVD, HD-DVD, file sources — keep
the inference path unchanged, and clips that already chain contiguously
produce a constant offset, which is pinned by a test.
2026-08-05 15:24:31 -07:00
Matthew Jackson bd2ba08bb7 Read a disc folder as an input: dir:// becomes a source
Users keep discs as extracted folders — a DVD VIDEO_TS or a Blu-ray BDMV,
usually a backup that is already decrypted. dir:// could only ever be a
destination, so those folders could be produced and never read back.

Everything above the sector layer wants a UdfFs over a SectorSource, and
every UdfFs read re-reads the ICB off that source at call time, so a
folder has to present itself as sectors. It does: dirimage plans a block
layout over the real files, encodes a UDF 1.02 filesystem for the
metadata, and serves data straight from disk. read_filesystem then parses
it exactly as it parses a disc, so nothing above changes — and the
iso:// arm of input() is now shared rather than duplicated, so dir://
inherits its decrypt gates, title selection and stream pruning.

The encoder is validated by more than its own reader: macOS mounts the
synthesized image and the mounted files compare byte-identical to the
originals. A round-trip through our own parser could not have shown that
— the tag CRC seeds at zero, and a wrong seed would satisfy us and no
real driver.

DVD placement is not free packing: a VTS IFO records where its title
VOBS begins relative to itself, so the VOB has to land exactly there.
Unsatisfiable marks fail loudly rather than misplace the file. 3D folders
are refused for now: the scanner detects SSIF and the planner cannot
alias its extents yet, so accepting them would produce quiet nonsense.

Left for later: metadata capture, HD-DVD, FMTS, encrypted folders.
2026-08-05 15:24:18 -07:00
Matthew Jackson dc7c3a7db5 Make iso:// a destination for any image source, not just a drive
Writing an iso:// meant a raw sector copy off a drive, so an existing
image could only ever be a source. Decrypting one you already had meant
finding the disc again.

The engine's copy is the recovery path — mapfile, multipass, damage-jump,
auto-resume — and all of it exists because optical media returns read
errors. A file does not, so a non-drive source gets write_image instead:
sectors in, bytes out, once, no recovery machinery. Keeping them apart is
not just tidiness. The mapfile identity check compares AACS unit keys and
the VID, both empty for an already-decrypted source, so identity passes
for any such source — a second run with different input to the same
output path would resume over the previous image and report success.

A short read is an error rather than a zero-fill: padding a truncated
source yields an image that looks complete and is not, which is the worst
outcome for a copy someone means to keep.
2026-08-05 14:45:01 -07:00
AnimeFNandMatthew Jackson cfce270186 Fix NTSC chapter/duration drift: dvd_time_t is a timecode, not seconds
`dvd_time_t` stores H:M:S:F as non-drop-frame timecode with a rate flag,
not elapsed wall-clock time. On NTSC the seconds field advances every 30
frames, but a frame lasts 1001/30000 s, so 30 frames occupy 1.001 s of
real time. Reading H:M:S as literal seconds under-reports real time by
exactly 0.1% — 3.6 s per hour, growing with elapsed time, which is what
made chapter marks drift ~4 s by the 67-minute mark.

Convert the whole timecode through an integer frame count and apply the
exact 1001/30000 fraction once, instead of reading H:M:S literally and
dividing only the frame remainder by a decimal 29.97. Chapter marks sum
per-cell frame counts as integers and convert once per chapter, so every
mark lands on an exact frame boundary instead of accumulating f64
rounding across a long title.

PAL is arithmetically unaffected: 25 frames = 1.000 s exactly, so the
old and new paths agree to within one ULP. Sweeping all 900k H:M:S:F
combinations under 10 hours, 12 differ, by at most 8.9e-16 s — the new
path divides one exact integer instead of adding a rounded fraction to a
large one, so where they differ it is the more accurate of the two.

Five existing NTSC fixtures asserted the old literal-seconds values and
were updated to real seconds (each is exactly 1.001x its old figure);
`bcd_secs` now documents that its argument is timecode, not real time.

Fixes #1
Reported-by: AnimeFN <admin@animefn.com>
2026-08-05 13:48:11 -07:00
Matthew Jackson 4ab8303a65 changelog: write it for users, not for the people who fixed it
Entries had grown into investigative narratives — root causes, disc
counts, PIDs, percentages, function names — which is the right level for
a post-mortem and the wrong one for a public changelog. Each entry is now
one to three sentences leading with what a user observes.

Halves the file (1538 -> 735 lines) without dropping a version. API
signature changes stay explicit, since a consumer migrates against those.
2026-08-03 08:31:04 -07:00
Matthew Jackson 80564e0470 restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-03 05:57:31 -07:00
Matthew Jackson bab566da40 v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-03 05:57:28 -07:00
Matthew Jackson 4fe05ba4c0 restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-02 22:35:49 -07:00
Matthew Jackson 7199ee497a v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-02 22:35:47 -07:00
Matthew Jackson 9ccb3c8444 Merge branch 'fix/label-pid-binding' into dev 2026-08-02 18:56:10 -07:00
Matthew Jackson 4959b48386 Bind borrowed stream labels by the stream they name, not by a slot
A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.

Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check 94377c7
added; 301 are applied. Those labels carry no editorial payload, so the
direct damage is confined to codec text — but the polluted list is also
what the anchor gate reads, and on 11 disc/stream-type pairs it is what
decides the anchor, which is how it reaches the vendor's forced and SDH
flags. 53 anchor facts are harvested off a merged slot. The clip-info
orphans had the same shape, numbered from `max + 1` of a list they share
no coordinate system with.

A label now either NAMES its stream — `StreamId { clip, pid }`, read out
of the very table `disc::bluray` builds the stream from — or it does
not, and only the ones that do not are ever reached by counting:

  * `label_at` returns vendor labels only, so the two numberings can no
    longer be confused by construction.
  * Playlist and clip-info labels bind by id. Exact, no anchor, no
    sequence, no ordinal.
  * A named stream outranks a guessed one, so an editorial flag reaches
    a stream only where the disc's own numbering puts it there.
  * The presence of an id is the provenance the anchor gate was missing.
    It reads the vendor's slots alone now, so a slot the vendor never
    named no longer breaks the sequence — under-yield is the normal
    shape of these blobs — and a title with fewer streams than the list
    has slots is no longer eligible to hold it. Ranking prefers the
    title that positively confirms most of the list.
  * An orphan is in no playlist, hence in no title, so it binds to
    nothing rather than to whatever counted its way.

The MPLS floor is one label per physical stream keyed by `(clip, PID)`,
not by `(type, language, codec, pid)`: the same PID in two clips is two
streams, and the old key collapsed them. Its `stream_number` is now the
entry's real slot in its own playlist's table.

41 of 44 images are byte-identical; all three that move lose a label
they should not have had, and no feature title changes on any image. A
dozen featurette playlists stop reporting a feature subtitle's SDH
marking on their own unrelated subtitle; eleven menu and bonus titles
stop advertising the feature's object-audio format on plain stereo; and
on a disc whose every title carries a single audio stream — too short a
table to anchor anything — a regional-variant tag asserted on all
seventeen titles is now asserted on none, in exchange for every title
stating the codec it actually carries, which none of them did.

Six tests had been asserting the invented numbering, including one
pinning the disc-global counter as a deliberate property.

Also fixes a defect in the same family that the corpus work surfaced:
`pgs_forced_probe::apply_verdicts` set `forced` but left `qualifier`,
so a demoted track shipped with a metadata sidecar calling it forced
next to a Matroska header saying it is not. Only a forced claim is
cleared; an SDH marking is not the probe's to touch.
2026-08-02 18:52:40 -07:00
Matthew Jackson 37e056f070 Say plainly that the vendor playlist format is not a specification 2026-08-02 18:27:01 -07:00
Matthew Jackson 1df36c78b2 Merge branch 'fix/paramount-forced-sub-semantics' into dev 2026-08-02 18:22:25 -07:00
Matthew Jackson 6d2ff4d1fc Read the vendor forced-subtitle field as the enumeration it is
One vendor's playlists.xml carries a per-subtitle-slot cell that looks
like a boolean, and it was parsed as one: value 1 meant forced, anything
else meant not forced. Across every image in the corpus that uses this
format the cell takes four values, and 1 is not the forced one.

Decoding three of those discs and counting every PGS display set:

  * 1 marks a FULL dialogue track that additionally contains some
    forced-narrative signs. All nine cells bearing it on one disc are
    full tracks of 949-1411 display sets; all seven on another are full
    tracks of 1602-1651. Neither disc has a small track among them.
  * 2 and 3 mark a DEDICATED forced-narrative track, in its own trailing
    stream slot, duplicating a language that already holds a full track.
    The two 2 slots measured are 15 and 10 display sets with every one
    flagged forced; the four 3 slots are 7, 14, 23 and 59 against
    1216-2655 on the tracks they duplicate.

So the old reading was wrong in both directions — it flagged full
dialogue tracks forced, which is how one language came to present as two
identical full subtitle tracks with one of them marked forced, and it
threw away the cells naming the real forced tracks.

Content could not have corrected this afterwards. Clearing a wrong
forced label needs a disc whose authoring sets forced_on_flag, and on
the measured disc carrying four genuine forced tracks not one display
set anywhere sets it — there, the vendor cell is the only evidence there
is. The classification is now explicit: only a dedicated forced slot
earns the flag, an unrecognised value never does, and the
contains-forced-signs value is dropped rather than weakened into a
forced label, since a wrong forced flag on a full dialogue track is the
user-visible defect while a missing hint costs nothing.

Four of the crate's own tests had been asserting the boolean reading;
their subject was positional alignment, so they keep it and now use a
real forced value. The other two parsers that emit a forced qualifier
from vendor metadata were audited and are structurally immune — in both,
the forced marker names a slot of its own rather than hanging off a full
track's entry, so the failure has no encoding there — and each is now
pinned by a test saying so.
2026-08-02 18:21:32 -07:00
Matthew Jackson 94377c75fd Bind vendor stream labels by PID, not by per-title ordinal
A vendor label list describes ONE playlist's stream table, but
apply_labels re-numbered it from 1 inside every title. Where sibling
playlists cover the identical feature clip and enumerate different
subtitle sets, identical ordinals resolve to different PIDs, and the
same physical stream came out flagged forced in one title and not in
the other. On the title such a disc offers as its rip target that put
`forced` on an 873 MB full-dialogue English subtitle track — the
reported "I see English and English (Forced), they are identical".

The blobs are not wrong; the binding was. The list carries no playlist
id, but it carries a language per slot, and that sequence is a
fingerprint: on the corpus exactly one title's per-type language
sequence reproduces the list position for position, and content
confirms that title's binding is the correct one. So binding is now
two-tier:

  * The title whose whole per-type language sequence sits on the list
    (>= 2 streams, longest wins) is the ANCHOR — the table the list is
    describing. Each of its slots yields a `(clip, PID) -> label` fact,
    and a PID is the same elementary stream in every playlist that
    plays that clip, so sibling playlists bind through the map. A slot
    the anchor never showed us is not bound at all.

  * Streams no anchor fact reaches still bind by the STN ordinal, but a
    label whose language contradicts the stream it would land on is
    dropped. Subtitle labels carry nothing but the qualifier, so an
    unverifiable one is all risk and no gain: off the authoritative
    path they additionally require both sides to STATE a language and
    state the same one. Unlabelled beats mislabelled — the muxer's
    demotable() guard can only clear a wrong `forced` on discs whose
    authoring uses forced_on_flag, and half the measured discs never
    set it.

Measured over 44 disc images, 9 change and every cross-title label
conflict goes away: 171 forced flags that contradicted a sibling
playlist are cleared, 61 correct ones are recovered on playlists that
had been missing them, 5 subtitle qualifiers and 1 audio purpose bound
against a contradicting language are dropped. No title gains a label it
did not have.

Known residual: on one disc the featurette playlists keep two forced
flags (down from nine) where a shifted list happens to coincide on
language. Ruling those out needs the list's provenance — which slots
are the vendor's and which were merged in by the MPLS gap-fill — and a
whole-sequence gate without it costs correct flags on discs whose
vendor slots are interleaved with gap-filled ones.
2026-08-02 17:50:20 -07:00
Matthew Jackson 0d4aab99df Stop naming specific commercial discs in the AACS and codec comments
The same scrub as the previous commit, over the files it did not reach:
the variant-MKB layout notes, the 2.1 segment index observations, the
PPS-revert regressions and the playlist-twin tiebreak.

Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16"
is the useful part, and the title it came from never was.
2026-08-02 17:10:43 -07:00
Matthew Jackson b93d10082d Merge branch 'fix/pgs-forced-probe-sampling' into dev 2026-08-02 17:07:47 -07:00
Matthew Jackson 17bb13b077 Do not call a track forced off one display set of a sample
A forced verdict is an absence claim, and a sampled run sees a fraction
of a track. Measured on real discs: tracks exist that flag about a
quarter of their display sets and leave the rest unflagged, so a sample
that catches one flagged set and nothing else promotes a full dialogue
track to forced -- which players then force on screen. A sampled run now
needs two display sets; a run that read every extent end to end has no
unread gap and may still promote off one.
2026-08-02 17:05:28 -07:00
Matthew Jackson b68765fe84 Stop naming specific commercial discs in comments and tests
Fifteen references across six files named the discs a defect was first
seen on. The parser leak found earlier was not an isolated slip — the
same habit runs through the mux comments, the changelog and the AACS
content verdict, where a title name was standing in for the shape of
the problem.

Every one is replaced with the property that actually mattered: a
multi-clip title, a UHD Dolby Vision profile 7 dual-layer stream, a
disc carrying an authored-bad TS packet. The comments are more useful
for it — the reader needs to recognise the shape on a disc they have,
not the one we happened to have.

`SEG_MainFeature` stays: the parser matches on that literal, so it is
a format token rather than a title.
2026-08-02 17:00:10 -07:00
Matthew Jackson abffa4235a Halve the seeks: eight 32 MiB windows, not sixteen 16 MiB ones
Measured on a real title: the same 256 MiB budget cut into sixteen
windows per extent took 41.8s against the head-first read's 6.6s, because
every jump collapsed the source's read batching into three-sector calls.
Expected observations depend on total bytes read, not on how finely they
are cut, and measured subtitle density (a display set every ~30-50 MB of
clip) makes a 32 MiB window about even money on its own.
2026-08-02 16:55:55 -07:00
Matthew Jackson c94e9f4fb7 End a label section where the next one's stream list begins
The pixelogic walk finds the feature playlist's section by name and ends
it at the next `SEG_`/`SF_`/`FPL_` marker. Those markers are section
NAMES, and a project's trailing sections — the per-language notice,
disclaimer and dub-credit cards — carry none. On 8 of the 11
affected-format discs in the corpus the feature playlist is the last
NAMED section in the blob, so the terminator never fires and the walk
consumes the whole tail of the file as more of the feature's stream
list.

The card names are `{lang3}_{card}`, which passes `is_stream_token`, so
each one advances an STN counter, and a card whose name collides with a
catalogued component emits a label outright. Measured on the worst disc:
95 entries past the end of a 9-audio/21-PG list, five phantom audio
labels at STN 10-14 from `*_AC` notice cards (`AC` reads as the AC-3
codec), and 94 uncatalogued-component occurrences — which also took the
parse from High to Medium confidence and fired the vocabulary-gap
warning on four components that are deliberately not catalogued. A
second disc fabricated one subtitle label from a token in a following
playlist section named `FP_SingAlong`, which `FPL_` does not match.

What every section has, named or not, is a stream list that opens with
its video slots. So a `Video Stream N` entry repeating one this section
already listed is the first entry of the NEXT section, and ends this
one. Distinct video entries are kept, since a section may legitimately
list a secondary video stream; the memo of them is bounded at the BD STN
table's ceiling so disc bytes cannot grow it.

Replaying all 11 blobs through `assign_labels` before and after: the two
discs above lose exactly their phantom labels (11→6 and 5→4), the other
nine are byte-identical.

One residue is pinned rather than papered over: a card's name precedes
its own section's video slot, so a forward-only walk can still count the
FIRST card after the last real slot. It sits at the tail of a list
nothing follows in, so it can renumber nothing — at worst it costs a
parse its High confidence.

No other parser in src/labels/ walks a flat entry sequence with a
terminator set; the rest scope each stream to a structural range or read
its number off the entry itself. paramount and criterion gain immunity
pins for the boundary property specifically: a stream list cannot run
into the next element's, and a missing element boundary shortens the
list rather than extending it.
2026-08-02 16:55:08 -07:00
Matthew Jackson 28d5897b86 Apply the shape test to a mixed track too
A track that flags some of its own display sets and not others proves the
authoring house makes the distinction, so it needs no sibling to
corroborate the flag being in use -- but it still has to look like a full
dialogue track before a forced label is cleared. A small track with a
couple of flagged signs is a forced track, and demoting it is the mistake
the shape test exists to prevent.
2026-08-02 16:44:10 -07:00
Matthew Jackson 3ed8630535 Take a lone sample window from the middle of its extent
A title cut into 50-odd clips gives each extent one window's worth of
budget. At the extent's head, every clip is sampled at the same relative
position and the first clip's window lands on the opening of the feature
-- the one stretch with no subtitles in it.
2026-08-02 16:31:39 -07:00
Matthew Jackson d0d8e2c9bf Record the label numbering and vocabulary fixes in the changelog 2026-08-02 16:28:02 -07:00
Matthew Jackson 01d4a1ba00 Catalogue the DUB forced-narrative marker and report vocabulary gaps once
Two halves of the same subtitle-labelling bug class. The numbering half
landed already: an entry the parser could not parse still occupies an STN
slot, so skipping it shifted every later label onto the wrong stream. This
is the other half — an entry the parser counts correctly but cannot
INTERPRET.

`DUB` names the forced-narrative subtitle track authored to accompany a
language's dubbed audio presentation: the signs and on-screen-text pass a
viewer still needs once the dialogue itself is dubbed. It is the same
editorial class as `*_TXT_FOR_`, spelled differently by some authoring
runs. Uncatalogued, it matched no component arm, so the token signalled
neither audio nor subtitle and the whole stream record was dropped at the
domain guard — a genuine forced track left with no forced qualifier even
after the numbering was right.

Evidence, from two independent discs in the corpus: the token appears only
inside the PG list, embedded in an otherwise contiguous run of
`{lang}[_{region}]_TXT_FOR_` siblings — one forced-narrative slot per
localized language — and takes exactly the slot where that language's
forced entry belongs. Both discs also carry that language's FULL subtitle
track as a separate, separately-labelled slot, so the DUB entry is not it.
The stream it lands on is a sparse PG track, the signature of a forced
pass rather than full dialogue. Deliberately not added to vocab::qualifier:
that maps free-form English label text, where a bare "dub" means dubbed
AUDIO. The forced-subtitle reading is specific to this token grammar.

The corpus sweep that found it also produced four components that are
deliberately NOT catalogued. They are per-language notice and disclaimer
clip names that merely collide with the `{lang3}_{component}` token shape;
each occurrence sits in a one-video-stream section next to the disclaimer
entry it names. They carry no editorial meaning, and mapping them would
attach a qualifier to a stream on the strength of a filename.

Which is also why an unmapped component now reports once per parse rather
than once per occurrence. It was a debug line nobody reads, and that is how
this gap survived to a user complaint; but a per-occurrence warn would bury
the signal under dozens of routine collisions on an ordinary disc. One
bounded, deduplicated line names the distinct components and says plainly
that any forced/SDH/commentary meaning they carry went unapplied. The
backing set is capped and truncates by chars, not bytes — the components
come from untrusted disc bytes, and a byte-offset slice can split a
multi-byte sequence and panic.

Two existing tests encoded the wrong behaviour and are corrected: the STN
numbering test expected the forced run to skip the DUB slot, and the
unclassifiable-slot test used DUB as its example of a token with no
meaning. The latter now uses a genuinely uncatalogued component.
2026-08-02 16:25:04 -07:00
Matthew Jackson 5c8b4dc7c5 Note the bounded display-count over-count on a stalled read retry 2026-08-02 16:18:19 -07:00
Matthew Jackson 841aa1a1c6 Drain a sampled run's tail and merge repeated extent memos
A window's last PES stays open in the demuxer until the next PUSI, which
under sampling is in a different window or nowhere — so the last display
set of every window was discarded, worst where the sample is thinnest.
Flush the demuxer at the end of each window.

A playlist that lists the same clip twice now merges the second read into
that extent's memo (facts OR'd, display count MAXed, coverage the larger
of the two) instead of overwriting it.
2026-08-02 16:16:20 -07:00
Matthew Jackson 14049bb477 Document the forced-probe redesign in the changelog 2026-08-02 16:09:27 -07:00
Matthew Jackson 8ffce6b621 Sample PGS across the title instead of reading its head
The content-based forced-subtitle probe spent its whole 256 MiB budget
on the first sectors of a title. A feature's subtitles begin minutes in,
so the probe read the opening logos, hit the budget, observed no display
set at all and contributed nothing to any verdict — the vendor label was
always the only input.

The forced predicate is asymmetric: one non-forced display set disproves
forced permanently, while proving forced needs the whole track, and
genuine forced tracks are tiny where full tracks are huge. So the same
budget is now SPREAD over each extent in ~16 MiB windows placed on the
AACS unit grid, sized in proportion to the extent, ending at the extent's
end. Cost is unchanged; placement is not.

Also:

  * Per-track early exit. A track that is disproven (and whose label
    needs no correcting) stops asking for budget; an extent that owes
    evidence only for such tracks is skipped outright, and evidence
    already in the cache is never demuxed a second time.

  * Content may now DEMOTE a wrong vendor forced flag, in the probe and
    in the muxer, behind one shared guard: absence of forced_on_flag
    only means something if some other track demonstrably uses it, and
    the track must have the shape of a full dialogue track rather than
    of a forced-narrative one. On a disc where no track sets the flag,
    nothing is demotable.

  * A sampled or budget-cut extent's evidence is memoised with the
    COVERAGE behind it. It used to be filed under the extent's full key
    and replayed to playlists that would have read far more of the clip,
    turning a prefix into an absence claim about the whole extent.
2026-08-02 16:07:02 -07:00
Matthew Jackson 9fda690d00 Number vendor label streams by STN slot, not by parsed entry
Sweep of every parser in src/labels/ for the numbering bug fixed in
pixelogic: a blob lists one entry per STN slot, but the parser advances
its per-kind counter only for entries it can use, so every entry it
skips shifts all later labels onto the wrong stream.

Three parsers were affected; the rest key each label off a number the
blob states outright and are immune.

  * paramount — `aud` / `sub` are the STN-ordered stream lists, and
    `forced_sub` / `*_com1_idx` index those same cells. A cell with an
    empty language was skipped without consuming its slot, so every
    label behind it bound one stream early while the vendor's own
    positional indices still pointed at the raw cell. `stream_number`
    is now the cell's 1-based position. The forced flag is the payload
    here, so the shift lands `forced` on a full-dialogue track.

  * mpls_universal — its counters must agree with the stream list
    `disc::bluray` builds from the same STN entries, since that list is
    what `apply_labels` counts against. They disagreed twice: a
    `coding_type == 0` padding entry was counted here and dropped
    there, and a PG coding_type in an audio STN slot (a layout
    `mpls::parse_stream_entry` has a dedicated arm for) was counted as
    audio here and built as a subtitle there. Both rules now live in
    one `label_type_for`.

  * deluxe — a binding construction whose Language `getstatic` did not
    resolve was skipped outright. It is still an STN slot; it just has
    nothing to label. It now advances the counter, with the list it
    belongs to taken from its CodingType argument or, failing that,
    from what its binding type's resolved siblings showed.

Two paramount tests asserted the renumbering as if it were the spec
(`empty_middle_slot_does_not_inflate_stream_number`,
`audio_stream_numbering_skips_empty_slots`) and are rewritten. Immunity
pins added for ctrm, dbp and criterion so the property cannot rot.

Also scrubs two commercial disc titles from the menu-graphic filename
examples in png_filenames and vocab.
2026-08-02 15:51:09 -07:00
Matthew Jackson f53abfe0d4 Stop naming a specific commercial disc in the label parser
Two comments identified the disc the STN-numbering bug was found on by its
vendor project name. The reproduction does not need it: what matters is the
SHAPE of the token list — placeholder slots, region-only tokens, an
uncatalogued component — not which release happened to exhibit it. Both now
describe the shape.

The remaining `SEG_MainFeature` references stay. That is a vendor section name
the parser matches on at pixelogic.rs:88, not a disc identifier — it is the
format's vocabulary, like `FPL_` or the `eng_MLP_` stream tokens beside it, and
removing it would break the parser.

Also drops the last prohibited citation from the changelog: an mp4:// bullet
said "no ffmpeg". The website changelog page is REGENERATED from this file at
release time, so a scrub of the site alone would have been reverted by the next
release.
2026-08-02 15:30:56 -07:00
Matthew Jackson 80be34bff5 Number pixelogic subtitle labels by STN slot, not by parsed token
A pixelogic feature section lists one entry per STN slot. Only the
entries that parse were advancing the counters, so every surviving
label was renumbered 1..N and applied to the wrong stream.

Three kinds of entry were being skipped:

  * `PG Stream N` placeholders — the subtitle twin of `Audio Stream N`,
    which was already counted. The old comment claimed the corpus showed
    subtitle tokens align without counting them; on a disc that has both
    placeholders and later editorial tokens they do not.
  * region-only tokens (`fra_CF_`, `spa_LS_`) — REGIONS sets `variant`
    but neither `is_audio` nor `is_subtitle`, so the token is dropped.
  * tokens whose distinguishing component is uncatalogued (`jpn_DUB_`).

On UHD_Crime101_WW_150728 the PG list is 18 slots: five placeholders,
four region-only tokens, and `jpn_DUB_`, with the seven `*_TXT_FOR_`
forced-narrative tokens at STN 11-16 and 18. Counting only the ten
that parse put `eng_SDH_` on STN 1 and the seven forced markers on STN
2-8 — the disc's FULL subtitle tracks. `labels::apply_labels` turns a
Forced qualifier into `SubtitleStream::forced`, which the muxer writes
as Matroska `FlagForced`, so the output offered an "English (forced)"
track carrying 1731 display sets of complete English dialogue while the
real 19-set forced track went unflagged.

Unclassifiable entries carry no stream type, so they advance the list
currently being enumerated; sections run video -> audio -> PG and
`domain` follows the last entry whose type was known.
2026-08-02 15:27:28 -07:00
Matthew Jackson 4447bd60ce Record the 1.6.0 fixes in the changelog
The fvi provenance fix, the key-service outage codes, the settings BOM
data-loss fix and the per-track-kind picker rows all landed on dev without a
changelog line. A release that ships undocumented fixes is one nobody can
audit later.
2026-08-02 15:08:27 -07:00
Matthew Jackson c054893540 Merge branch 'fix/key-service-outage-not-missing-key' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson 6758370f8d Merge branch 'fix/fvi-source-provenance' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson b2a274782a Report a key source that could not answer as its own failure, not as "no key"
The online key service returned HTTP 502 for about seven hours. Every rip in
that window ended with

    key: online > no entry > NO KEY
    Error: E7022 No key source has a decryption key for this disc (id: 422EB...)

which reads as "this disc is not in the key database". Operators went hunting
for a VUK that was never missing; the correct action was to wait.

E7022 is a claim about the WORLD: every source answered, and none holds a key
for this disc. A source that could not be reached made no such claim -- nothing
at all was learned. `resolve_and_apply_traced` collapsed the two anyway, with
its own comment naming the incident and pointing at the fix.

Three codes for the three different operator actions, each a variant with a
number and no English (the library ships none):

    E7028 KeyServiceUnavailable   unreachable / DNS / timeout / 5xx -> wait
    E7029 KeyServiceUnauthorized  401 or 403                        -> fix token
    E7030 KeyServiceRateLimited   429                               -> back off

None carries a payload: the key-service URL and its resolved address are
operator-confidential and must not ride out in a Display an operator pastes
into a bug report.

`resolve_and_apply_traced` now keeps `Err` and `Ok(empty)` apart. A source that
answered and holds nothing still records `KeyNode::NoEntry` -- that is the one
true "I looked, it is not there". A source that FAILED records an empty path,
and its reason is stamped onto `Disc::aacs_error`, the channel
`ensure_decryptable_keys` already reads for the E7017-vs-E7022 split. The gate
gained three arms alongside `AacsVidUnavailable` and raises the source's own
code.

`KeyOutcome` deliberately gains no variant. It is matched exhaustively by every
front-end's trace renderer (freemkv's `pipe::render_resolution_trace`,
autorip's `keysource::render_resolution_trace`), and this fix must not become a
breaking change across four repos to say something the error code already says
precisely. Dropping the false `NoEntry` node is enough for the trace line.

The three codes join `is_disc_level_no_key`: a service that is down, refusing
the token or throttling is down for every title, so a rip loop must stop rather
than issue N doomed requests -- and on 429, dig the hole deeper.

`FetchOutcome::errored`, documented as unreachable-in-production, now fires for
real: the negative-result cache stops memoising a transient outage.

Red before green: `key_source_failure_is_not_reported_as_a_missing_disc_key`
drives KeySource -> resolve -> aacs_error -> ensure_decryptable twice over the
same disc and asserts the verdicts differ. With the old conflation restored it
fails on the trace node, and with that assertion removed it fails
`left: 7022, right: 7028` at the gate.
2026-08-02 11:53:09 -07:00
Matthew Jackson cf7ee69fd5 Record the source, not the destination, in the FVI header
The `fvi://` arm of `output()` passed the destination `.fvi` path as
`FviSink::create`'s `source_path`, so every index named itself as its
own source. `SourceInfo::default()` supplied the rest, making
`source.medium` always "file" and `source.title` always 0 — three
header members wrong, where FVI_FORMAT.md §6.2 defines `source` as
describing the input.

Beyond the wrong data, it made the output unreproducible: two machines
indexing identical bytes emitted different files purely from where
they wrote them, and a local filesystem path leaked into a shareable
file.

`output()` cannot see the source, so thread the provenance down from
the driver, which can: `mux_stream` derives a `SourceInfo` per
`MuxInput` arm and passes it through `drive_mux` to `output()`. Per the
one-method-per-action rule this is a signature change, not an
`output_with_source()` variant; the parameter is `Option<&SourceInfo>`
so a caller with no provenance declares none rather than back-filling
the destination. `SourceInfo`/`Medium` become public API.

What each arm can honestly reach:

- Session: everything — device path, the caller's title index, the
  title's playlist, the scanned volume id.
- Url: the source URL, its scheme's medium, `title_index`, and the
  playlist off the opened stream's scanned title.
- Iso: the image path and playlist. The title index is not in
  `MuxInput::Iso` (it carries a scanned `DiscTitle`, which has no
  index), so it stays 0.
- Live: medium and playlist. The reader is an opaque
  `Box<dyn SectorSource>` with no path, and again no title index.

Unreachable members are left empty rather than guessed — the sink
already omits the empty ones.
2026-08-02 11:19:54 -07:00
Matthew Jackson e008e71a17 Add the missing direct coverage for parse_stss
Every other sample-table parser (stco, stsc, stts, ctts) had a "count lie"
test proving the declared count is bounded by what the box actually holds,
and stco/stsc had their own arithmetic pinned. parse_stss had neither - no
test in this file ever called it with real entries, only with a too-short
buffer. Added the same two: two distinct entries read from their own
offsets (catching the o = 8 + i*4 arithmetic and the per-entry bounds
check), and a declared count of 3 backed by only 2 real entries (catching
the same "trust the box, not the count" contract the other four parsers
already had).
2026-08-01 16:37:10 -07:00
Matthew Jackson c59e1e3342 Pin the sample-table parsers' short-buffer safety and byte arithmetic
parse_stco, parse_stsc, parse_stts, parse_ctts and parse_stss all open with
the same "if b.len() < 8" guard before reading count = be32(b, 4), but no
fixture anywhere in this file ever called any of them with a buffer shorter
than 8 bytes - every test builds a complete box. A <-to-== mutant of that
guard only rejects a buffer of EXACTLY 8 bytes and lets everything shorter
fall through to an out-of-bounds be32 read, and nothing was exercising that
fall-through to notice. One test now drives all five through every length
from 0 to 8.

Also: co64's 8-byte offsets were never actually built by any co64 fixture
in this file (only the 32-bit stco path was exercised), so its manual
byte-by-byte u64 assembly was unconstrained the same way parse_elst's was.
And sample_offsets's sidx only advanced correctly by coincidence in every
existing fixture, because none of them placed three or more samples in a
single chunk back to back - the only shape where reusing the wrong sample's
size becomes observable.

Documented the <=8 boundary as equivalent across all five parsers (and the
matching start < end in sample_offsets): the per-entry guard immediately
below always breaks on the first entry at that exact boundary, so both
branches converge on the same empty result. Confirmed by re-running each
mutation against the full suite.
2026-08-01 16:33:17 -07:00
Matthew Jackson 39d9714ae7 Pin parse_esds_asc's two independent boundary checks
asc_len == 0 and end > b.len() reject for different reasons - a useless
zero-length ASC, and a truncated one - and nothing distinguished the || from
an && that would only reject when both are true simultaneously, or the >
from a < that would reject the common case of an esds with bytes after the
ASC (more child boxes, padding) instead of only a genuine truncation.

Documented the | in read_descriptor_len's accumulator as the same
shift-then-mask equivalent already recorded in audio.rs's BitReader::read:
the shift always vacates exactly the bits the mask fills, so | and ^ can't
disagree.
2026-08-01 16:23:10 -07:00
Matthew Jackson dd940583c7 Pin mdhd_language's length boundary and three unasserted stsd codec paths
mdhd_language's guard was written as "b.len() < off + 2" (reject too short)
but nothing distinguished that from "reject anything not exactly off + 2" -
a buffer one byte longer than the minimum has to keep working, and a buffer
missing the field entirely has to return None rather than read past the end.

parse_stsd's mp4a path had no test at all: the AAC codec_private extraction
depends on both codec == Aac and body.len() >= 28 being true together, and
with no mp4a fixture anywhere in this file neither half of that condition,
nor the boundary itself, was constrained. Also added the same header-length
boundary check parse_elst already had (< 8 vs <= 8 - proved equivalent this
time, since the very next guard on the empty slice catches the <= 8 case
too), and one test walking every recognised audio fourcc (ac-3/ec-3/mp4a/the
four dtsX variants) so a deleted match arm for any of them fails loudly
instead of silently dropping that track.
2026-08-01 16:18:20 -07:00
Matthew Jackson 4b7e4ddbb3 Pin find_boxes_capped's cap boundary and its size-field byte offsets
Nothing asserted the scan actually STOPS at cap rather than one match past
it, or that the declared box size is decoded from its own four bytes rather
than an adjacent one - every existing fixture used sizes small enough that
all but the last size byte are zero, so an index slip reading the wrong byte
would read the same zero and go unnoticed.
2026-08-01 16:08:11 -07:00
Matthew Jackson 5222458411 Pin Stream::read's MAX_ALLOC_BYTES boundary and that write always rejects
Stream::read has its own s.size > MAX_ALLOC_BYTES cap, a separate call site
from read_moov's over the same policy - checked they agree (both reject
strictly greater than the cap, exact cap allowed) and they do, so this is not
one of tonight's one-policy-two-copies bugs. But the boundary itself and the
one-byte-over case were unasserted, and Mp4Reader::write returning an error
(mp4:// is read-only) had no test either.

Building Mp4Reader directly in the test (its fields are private but visible
within this module) over the existing FakeBigReader avoids a real 256 MiB
backing file for the boundary case.
2026-08-01 16:04:03 -07:00
Matthew Jackson dd5118ee9c Pin per-track handler routing, PID arithmetic and the shared sample budget
Nothing asserted that a hdlr other than vide/soun gets dropped rather than
folded into the audio branch, that the per-track PID formulas
(0x1011/0x1100 + track_idx) use the right operator and the right operand,
that a sample-less track still advances track_idx for the next one, or that
the cross-track sample_budget is actually decremented (as opposed to grown or
divided) by each track's real count. All four were reachable with a single
track_idx == 0, which made every existing fixture blind to +/-/* confusion on
these sites - track_idx never moved past 0 in any of them.

Also let audio_trak_missing omit stsz, needed to build a sample-less track for
the track_idx test.
2026-08-01 15:59:53 -07:00
Matthew Jackson a8db2435cd Pin elst byte-offset decoding and a zero-timescale boundary in elst_offset_ticks
parse_elst's existing tests used segment_duration/media_time values that were
almost all zero or 0xFF bytes, so an index slip in the version-1 byte
extraction (reading a neighbouring byte, or one outside the entry entirely)
could return the same value by coincidence and the test wouldn't notice. Added
a fixture with every byte distinct and nonzero so any wrong offset is caught.

elst_offset_ticks's `empty_movie_ticks > 0` guard on the Some(mts) arm looked
like a pure optimisation, but shifting its boundary lets an
empty_movie_ticks == 0 call fall into the division instead of skipping it -
and a zero movie timescale (unreachable through from_reader, which filters it,
but not through this function's own contract) makes that division panic.
Pinned the boundary directly so the function stays safe on its own terms.

Documented nine further mutants as equivalent rather than chasing them:
media_edits/odd_rate and the None-arm's empty_movie_ticks check only gate a
tracing::warn!, never the returned offset, and parse_elst's b.len() < 8 vs
<= 8 boundary computes the same empty Vec either way once available = 0 is
worked through. Confirmed by re-running each mutation against the full test
suite.
2026-08-01 15:54:24 -07:00
Matthew Jackson ff18d4c3c8 Close the MEDIUM mutation gaps across transport, labels and codecs
The remaining triage items after tonight's HIGH fixes: 1,290 lines, almost
all tests. Covers disc/mod.rs's DVD scan path (with real minimal VMG/VTS IFO
fixtures rather than mocks), drive/mod.rs, labels/class_reader.rs and
labels/mod.rs — the two biggest untriaged survivor clusters in the crate —
plus hevc.rs and ps.rs.

One production change, and it is an extraction rather than a behaviour
change: MacScsiTransport::open mapped the shim's negative failure sentinels
to typed errors inline, where nothing could reach it without a real IOKit
FFI call. It is now map_shim_open_error, so the mapping can be pinned. It
matters because collapsing -5 into the DeviceNotFound catch-all turns
"another process holds the drive" into "no such drive", and an operator
chasing the wrong problem is worse than a blunt error.

Gate green on the pinned toolchain including the secrets scanner.
2026-08-01 15:00:01 -07:00
Matthew Jackson f8ed0b99f4 Pin read_moov's box-size boundaries and MAX_ALLOC_BYTES exactly
read_moov's forward-progress guard (box_size < header_len, OR'd with the
EOF check) and the MAX_ALLOC_BYTES cap were only exercised on inputs well
away from their boundaries, so a mutation testing pass found the exact
edges unasserted: a size-8 (header-only) moov, a box that overruns the
file by exactly the amount the OR/AND distinction can see, and a payload
of precisely MAX_ALLOC_BYTES. Added a shared FakeBigReader (lifted out of
an existing test's local struct so a new test can reuse it) to exercise
the MAX_ALLOC_BYTES boundary without a multi-hundred-MiB backing file.
2026-08-01 14:39:39 -07:00
Matthew Jackson 8189da1b0c Document why audio.rs's bit-packing | mutants are equivalent
Every mutation-testing survivor in this file is a | with ^ flip inside a
bitstream packer: BitReader::read's accumulate step, the push closures in
dac3_box/dec3_box/ddts_box, and the multi-field extractions in parse_eac3
and parse_dts. All nine are the same shape: shift an accumulator left by
exactly the width of the next field, then OR it in, so the two operands
never share a set bit and | and ^ agree on every input. Confirmed by
running cargo-mutants against just these nine mutations after the existing
test suite (which already exercises each function's field values) - all
nine still survive, as expected for a genuinely equivalent mutant. Recorded
the reasoning once at BitReader::read so nobody spends time chasing it
site by site.
2026-08-01 14:24:47 -07:00
Matthew Jackson 698ba36ae4 Document the pack_language and detect_rate equivalent mutants
Three pack_language mutants and two detect_rate boundary mutants survive
mutation testing with no test able to close them, and it's not for lack
of trying: they're equivalent by construction. Recording the proofs next
to the code so nobody re-chases them:

- (b[0] - 0x60) as u16, shifted << 10 then truncated to u16, is congruent
  mod 65536 to (b[0] + 0x60) as u16 shifted the same way, because
  0x60 * 2 * 1024 is an exact multiple of 65536. The same swap on the
  second letter (shifted only << 5) is NOT equivalent, which is why only
  the first letter's mutant survives.
- The two | with ^ mutations that OR the three packed fields together
  are equivalent because the fields (a lowercase letter minus 0x60, so
  1..=26) always fit in 5 bits and never share a set bit once shifted
  into their 0/5/10 positions.
- detect_rate's tolerance and tie-break comparisons only diverge from
  their <= mutants on an exact 0.5 fps distance or an exact tie, and a
  brute-force search over every achievable integer-nanosecond median
  found no case that lands on either boundary bit-exactly.
2026-08-01 14:20:13 -07:00
Matthew Jackson 1eb8bdc9c7 Merge branch 'mux-mp4' into dev 2026-08-01 14:06:24 -07:00
Matthew Jackson 90a7fe2ff1 Assert the MP4 timing arithmetic, and name the faststart slack rule
MP4 track timing was numerically unasserted. Every operator in the
PTS-to-ticks, duration, tkhd_dur and ctts chain could be flipped and the
whole suite stayed green, because no test decoded an output file and checked
a concrete number — the existing tests assert box presence and gross
container shape only. That is the crate's worst failure mode: a title muxes
"successfully" with silently wrong A/V sync or total duration, and nothing
above can tell.

The new tests build tracks with known PTS deltas and compare the emitted
stts, ctts and tkhd.duration against computed values.

Also lifted the faststart slack rule out of the match guard into
faststart_fits(). A leftover hole of 1-7 bytes cannot be expressed as any
ISO-BMFF box, since a box header is 8 bytes, so finish() must fall back to
moov-at-end rather than write a free box that lies about its own size. The
condition now has a name and a test instead of being an unexplained
`g == 0 || g >= 8` inside a pattern guard.
2026-08-01 14:06:16 -07:00
Matthew Jackson 4e70d9a5c5 Assert the undersized-buffer guard on the chunked read path
Drive::read and read_fua split any request larger than the transport's
transfer limit into chunks and slice the caller's buffer by count * 2048.
The up-front length check is the only thing between an undersized buffer and
a "range end index out of range" panic out of a public API, and the comment
above it records that this was once a live panic.

Every existing read test stays on the single-chunk path, where an undersized
buffer is already tolerated and returns Err(DiscRead), so the guard itself
had no coverage at all and a mutation run flipped its arithmetic freely.

The test drives a mock transport with a small transfer limit and asserts the
two paths agree: an undersized buffer is an error either way, and behaviour
on a caller mistake does not depend on the drive's transfer limit. Confirmed
by hand that both the reported * -> + mutation and a < -> > flip now fail.
2026-08-01 13:56:58 -07:00
Matthew Jackson 1b95d346bb Cite the H.264 spec directly, not a reference implementation
The escape-stripper comments named a third-party decoder as the authority
for the cumulative-zero rule. This repo is public and does not cite other
implementations; the rule is specified in ITU-T H.264 §7.3.1, which is the
citation that belongs here anyway.

No behaviour change — comments only. The scan-secrets gate caught it.
2026-08-01 13:49:47 -07:00
Matthew Jackson 48663c6a2f Merge branch 'mux-codec' into dev 2026-08-01 13:47:15 -07:00
Matthew Jackson a05f1d4498 Merge branch 'mux-mkv' into dev 2026-08-01 13:45:04 -07:00
Matthew Jackson 3ecb2510e8 Assert what the MKV mux and demux actually produce
A mutation run over mux/mkv.rs and mux/mkvstream.rs left 148 survivors.
Reading them turned up no wrong code, but a lot of code whose output
nothing ever looked at. Most of that is on the read side: parse_track
had a dedicated arm for Language, TrackName, FlagForced, Video and
Channels and not one of them was checked, so a re-mux could have lost
the audio language, the subtitle forced flag, every track label, the
resolution and the channel layout with the suite still green. Ten of
the eleven CodecID comparisons were unasserted too — only HEVC was
pinned — so any of them could have been mis-wired and the stream would
have gone to the wrong parser. The round-trip test now writes a real
three-track title through the muxer and reads it back through the
reader, and a separate test walks every registered CodecID.

The BPS statistics tag was the worst of the write side. Its test
asserted `file_bytes.contains("800")`, which a wrong bitrate passes
trivially — 80000 contains "800". Both the tag and the back-patched
Segment duration are now decoded and compared to a computed number, on
a title that declares no duration so the whole max_block_ticks →
seconds → bits chain is exercised. Cue points get the same treatment:
their CueTrack and CueClusterPosition were never read back, on either
the keyframe path or the i16-forced-split path, which are two hand-
written copies of the same three fields.

The rest closes arithmetic that only a bad disc reaches: a zero frame
rate or zero display-aspect denominator (both divisions), a
TimestampScale that does not fit an i64, a cluster timestamp of exactly
i64::MAX, a TrackNumber of 65537 that truncates onto the valid track 1,
and the shortest legal Block at both VINT widths. Two tests separate a
clean end of stream from a device failure: swallowing the second one
truncates the output at a bad sector and reports the rip complete.

Also pinned: only the first video and first audio track may be default
(the de-duplication lives in MkvStream::create and had no test at all),
the activation trigger is the first VIDEO track rather than track 0,
the measured field order reaches the file rather than just the helper
that computes it, and a Blu-ray 3D base/dependent pair builds the merge
instead of shipping two unrelated H.264 tracks.

96 of the 148 mutants verified killed by hand. Of the remainder, most
are equivalent — disjoint-bit `|` that `^` cannot change, delete-arm
mutants whose fallback is the same constant, guards on tracing calls —
and the write_frame branch at 1452 is unreachable: a cluster is always
open by the time it can be entered.
2026-08-01 13:44:08 -07:00
Matthew Jackson 048f125879 Kill codec mutation survivors and unify H.264's duplicated escape stripper
The mux/codec parsers (startcode, h264, hevc, dts) had 300 surviving
mutants between them, and it turned out to be for the reason you'd
fear: the exp-Golomb readers and the AU-boundary bitstream scanners had
essentially no direct unit coverage, only indirect exercise through
full-frame parse() calls that never touched the actual edge cases.

Direct fixes to test gaps:

- The shared BitReader's read_ue truncation guard (`leading_zeros >
  31`) and skip_start_code's 4-byte-vs-3-byte boundary check had no
  test at their exact boundary. Added tests that hit the boundary
  precisely; a `>=`/`==`/`<=` typo either rejects a legal 31-leading-
  zero code or reads one byte past the buffer.
- H.264's private SpsReader duplicates the same read_bits/read_ue
  shapes with no tests of its own at all (only reached through
  multi-field SPS parsing, several fields deep). Added direct tests.
- HEVC's per-AU trailing-zero strip after the last NAL (no start code
  following) walks `end` down to trim padding; a wrong-direction typo
  there walks off the end of the buffer instead of terminating -
  exactly the "loop must make positive progress on malformed input"
  class. Added a test with a zero-padded trailing NAL.
- HEVC's SEI match guards (`sei_mastering.is_none()` /
  `sei_content_light.is_none()`) implement "first HDR10 value in the
  title wins" - untested, and a naive test using both-messages-per-AU
  can't even exercise the guards because the whole-scan early return
  above them already handles that case. Split into single-message-
  per-AU tests that actually reach the arms.
- parse_mastering_display/parse_content_light_level's length guards
  were `< N` with no boundary test; one-byte-short input now confirmed
  to return None instead of indexing out of bounds.
- DTS's drain_front collapses duplicate offset-0 PTS markers after
  rebasing; untested, and the visible effect (front_pts()) can't tell
  a working collapse from a broken one since it already returns the
  right marker either way - the actual defect is unbounded growth of
  pts_marks over a long recording, so the new test asserts the bound
  directly across repeated drains.
- DTS's dts_core_samples/dts_core_sample_rate header-length guard and
  next_core_boundary's syncword-length guard got exact-boundary tests
  the same way; also caught a nblks `<<`/`>>` direction bug candidate
  in the mutant (confirmed the real code is correct, just untested).

Real bug found and fixed, not just a test gap:

H.264's parse_sps_high_profile_ext re-implemented emulation-prevention
byte stripping inline (a window scan: match `00 00 03` at position i,
advance 3, else advance 1) instead of calling the existing
unescape_ebsp_prefix used by slice-header parsing. On a run of 3+ real
zero bytes ahead of an 0x03 - non-conformant, but this is disc bytes,
not a spec-clean encoder - the two disagreed: unescape_ebsp_prefix's
cumulative zero counter (matching the H.264 reference decode process
and libavcodec's RBSP extractor) treats it as an escape and drops the
0x03; the window scan treats it as real payload and keeps it,
corrupting the SPS bits read after it. Extracted the shared rule into
`unescape_ebsp` (parameterized on output length so both the 16-byte
slice-header prefix and the unbounded SPS case can share it) and
pointed both call sites at the one implementation. Added a regression
test pinning the shared function's behaviour on the input that used to
separate them.

All new tests hand-verified against the actual mutation (operator
flipped or guard replaced by hand, confirmed red, then restored) per
the mutation-testing brief, not just written and trusted.
2026-08-01 13:35:43 -07:00
Matthew Jackson 65dbcb1ca6 Close mutation-testing gaps in the TS/PS mux (ts.rs, ps.rs, tsmux.rs)
A 12,330-mutant run left 159 survivors across these three files, all from
missing assertions rather than wrong code — every gap here is a test, no
production logic changed.

Two shapes accounted for most of them:

- Buffer-cap constants (MAX_PES_BUFFER_TOTAL, MAX_PS_BUFFER,
  MAX_BD_PES_PAYLOAD, PES_BUFFER_INIT_CAP) were only ever read by tests
  through their own symbol, so a mutated `*`/`-` in the constant's
  definition changes what the symbol itself evaluates to and every
  self-referential assertion still passes. Pinned each against a literal
  computed independently in the test.

- Several `>`/`==` boundary checks on framing lengths (MPEG-2 pack header,
  system header, BD-TS adaptation field) were only ever exercised with
  slack in the buffer, never at the exact byte the check exists for.
  Added exact-fit cases for the pack header, system header, and
  psi_payload_base's AF-consumes-everything boundary.

Real, higher-value gaps closed along the way:

- ts.rs's per-PID discontinuity_flag and the NULL-TS concealment marker
  both require adaptation_field_length > 0 before trusting the AF flags
  byte; neither branch had a test proving af_len == 0 (no flags byte at
  all, ordinary payload underneath) is left alone.
- header_remaining (PES header spillover across TS packets) only had
  single-continuation-packet coverage, which can't distinguish `-=` from
  `+=`/`*=` because the corrupted value never gets read again. Added a
  case spanning two continuations.
- ps.rs's parse_stream_id_extension (used for HD-DVD 0xFD routing) walks
  nine optional PES-header/extension fields with a `pos +=` each; only
  the PTS/DTS pair had ever been exercised. One test now arms every
  field and checks the walk lands on the right byte.
- find_ps_boundary's `sc + 3 >= len` guard had no test at sc == 0 with a
  bare 3-byte start code, the case a `+` -> `-` mutation turns into a
  debug-mode subtract-overflow panic on ordinary tail-of-buffer input.
- tsmux.rs: an oversized video access unit must go out as a single
  unbounded-length PES; the `is_video || small-enough` guard that
  enforces this had no test with a video frame actually over the
  bounded-PES threshold, so a `||` -> `&&` mutant survived (it would
  silently split a keyframe across several look-alike-independent PES
  units). Also pinned the PES-length and PTS big-endian encodes at
  values above 255 / with bit 29+ set, where a `>>`/`<<` swap first
  becomes observable.

Every test above was verified by hand: applied the exact mutation,
confirmed the test fails (or the specific panic fires), then reverted.

Left unclosed, all confirmed equivalent by hand-tracing rather than
just left alone:
- Every `<<8 | byte` PID/length bit-combine (ts.rs pid/PAT/PMT parsing,
  ps.rs dvd_audio_pid/hddvd_extended_pid/parse_pts): the two halves
  never share a bit, so `|` and `^` produce identical output for every
  input - no test can tell them apart.
- ts.rs's `af_len > 183` check in process_packet: fully subsumed by the
  `payload_start >= TS_PACKET_BYTES` check three lines later for every
  af_len that could trip it.
- ts.rs's out-of-range `pid_index` sentinel (-1 vs 1): unreachable, since
  a TS PID is masked to 13 bits (max 8191) and the table is always sized
  to at least 8192.
- A cluster of "push an empty slice on an exact boundary" mutants in
  tsmux.rs's write_pes_chain (offset < hdr_len, af_bytes stuffing
  guards): the guarded write becomes a length-0 write_all, a no-op
  either way.

Not reached this pass, for lack of a clean seam within the time
available - ps.rs's extract_packets bounded-PES-length exact-fit
checks (lines 278/282/303, the `sc+6>len` / `sc+6+pes_len>len` /
force-flush cap arithmetic). The first two need a scenario where
"proceed vs. wait one more byte" is observable in the packet list, and
the third only shows up at a start-code offset (sc) that survives to
the moment the cap check runs - in this code path sc is always 0 once
an unbounded PES buffer starts accumulating, since nothing before it
ever drains. Didn't find a construction in the time available; flagged
rather than papered over with a self-referential assert.
2026-08-01 13:34:48 -07:00
Matthew Jackson b002da4221 Fail the identity probe when INQUIRY returns a short data phase
DriveId::from_drive issues three data-in commands. The two GET CONFIGURATION
calls both clamp on bytes_transferred, with a comment noting it is
device-reported and untrusted. INQUIRY, three lines above them, discarded it
and decoded bytes 8..43 unconditionally.

The buffer is pre-zeroed, so a drive answering GOOD status with a short or
empty data phase — a USB-SATA bridge mid-wedge does exactly this — produced
blank vendor, product and revision strings and a byte 0 of 0x00. Every
platform enumerator gates on raw_inquiry[0] & 0x1F == the optical peripheral
type, and 0x00 is DIRECT ACCESS, so the drive silently disappeared from the
device list instead of reporting that its identity probe had failed. The
operator sees no drive at all rather than an error.

Anything shorter than the SPC-4 standard 36-byte header is now
E9058 DriveInquiryShort, and the buffer is truncated to what actually
arrived so nothing decodes past it. Exactly 36 bytes is still accepted: the
vendor-specific tail is optional.

This is the same defect as the READ CAPACITY short-transfer bug fixed
earlier today, in the same crate, found the same way.
2026-08-01 12:21:15 -07:00
Matthew Jackson 51d2b14d03 Give extract and analyze one label-parser tie-break, not two
mod.rs had two independent implementations of "pick the winning label
parser". select_result owns the rule — highest confidence wins, and on a tie
the earlier entry in PARSERS wins — and carries a regression test for a past
bug where analyze() picked the LAST equal-confidence parser instead of the
first. extract(), the path that actually ships, re-derived the same rule with
its own inline scan and had no test of its own.

The tie-break is load-bearing rather than incidental: array order encodes a
trust ordering, with the hand-vetted parsers registered ahead of the ones
that detect on any BD-J disc. A later parser winning a tie means the disc
gets labels from a less trusted source, silently.

extract now collects its candidates and calls select_result. Confirmed by
flipping the tie-break to prefer the later entry: the shared test fails,
where before the fix that mutation was invisible to the whole suite.

This is the seventh instance tonight of one policy implemented twice with
only one copy hardened, and the second in this file after the chapter
mark_type filter.
2026-08-01 12:15:48 -07:00
Matthew Jackson c4ad4184ec Make the FMTS phase gate reachable by tests
An FMTS forensic segment interleaves two variants at the unit level: the
disc carries both halves and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving it as
ciphertext is correct, because the muxer drops untouched ciphertext cleanly.

The decision lived inline in apply_aacs_map's per-unit closure, where no test
could reach it. A mutation run flipped the subtraction to an addition and the
parity comparison, and every test still passed — so the gate that decides
which half of a forensic segment we decrypt was entirely unasserted.

It is now unit_is_our_phase(). The index arithmetic also saturates and
clamps: both inputs come from the key map, so a unit below its own range
start or a zero unit size means a malformed map, and neither may panic on
debug overflow or a divide by zero inside a library a long-running service
depends on.

Four of the five surviving mutants now fail. The fifth, dividing by the unit
size versus multiplying, is equivalent rather than uncovered: aligned offsets
are exact multiples of the unit size, so the two differ only by a factor of
unit_sectors squared, and unit_sectors is the constant 3 — odd, so parity is
preserved on every reachable input. The proof is recorded in the test instead
of a test that pretends to cover it.
2026-08-01 11:48:39 -07:00
Matthew Jackson 528a6b7345 Walk past empty extents iteratively instead of recursing
fill_extents skipped an exhausted or zero-sector extent by calling itself,
which costs a stack frame per skipped extent. Nothing filters
sector_count == 0 out of a UDF or MPLS extent list, so a malformed disc
declaring a long run of empty extents recursed once per extent before
reading a single sector. Rust does not guarantee tail-call elimination, so
that overflows the stack — which aborts the process rather than returning
an io::Error, taking a long-running service down with it.

The skip is now a loop. The regression test runs on a 256 KiB stack, where
the recursive version dies and the loop finishes immediately.
2026-08-01 11:05:12 -07:00
Matthew Jackson fb321f51eb Reject a short READ CAPACITY reply, and count only entry marks as chapters
Two cases of the same shape: one policy implemented twice, with only one
copy hardened.

Disc::read_capacity decoded buf[0..4] from READ CAPACITY (10) without
checking that the transport actually delivered four bytes, even though its
comment claims to mirror decode_read_capacity — which has exactly that
check, and documents why. A drive answering GOOD with an empty data phase
leaves the buffer zeroed, so last_lba decodes to 0 and the probe reports a
one-sector disc instead of an error. It now calls the shared decoder rather
than re-deriving it.

collect_chapter_summary filtered chapters on mark_type <= 1, counting the
reserved type 0. PlaylistMark's own doc says filters must test == 1, and
disc/bluray.rs did; the labels path did not, inflating the public
chapter_count and letting a playlist whose only marks are reserved pass the
chapter_count == 0 skip. Both sites now share PlaylistMark::is_chapter_mark
so the copies cannot drift again.

Both fixes were confirmed red before green.
2026-08-01 11:00:49 -07:00
Matthew Jackson e0ff0cfeb4 ci: make libfreemkv prove its five dependents still compile
Every job above proves libfreemkv builds; none proved anything built on
it does. That gap bit today one level up — an engine signature change
broke autorip and went unnoticed, because consumer CI only fires on a
push to that consumer, and nobody pushed one.

libfreemkv sits underneath all five dependents, so a break here costs
more than a break anywhere else in the project. It is now the place the
question gets asked, since it is the place the change happened.

cargo check --all-targets only: each dependent has its own suite for its
own behaviour. This answers the narrower question that went unanswered.
2026-07-31 18:40:24 -07:00
Matthew Jackson 71686f1407 Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv,
the crate the other seven build against and the one held up as the
reference workflow, was the last one still linting the library only — so
its ~3,000 tests, by far the largest body of test code in the project,
had never been linted at all. Turning the flag on surfaced 74 findings.

Most were mechanical and applied with clippy --fix. The rest, by hand:

- Four discarded Results in decrypt.rs. css::descramble_region returns a
  Result and four CSS tests threw it away, so a descramble that FAILED
  would have surfaced as a confusing buffer-comparison mismatch instead
  of the actual error. They expect() now.
- A dead `kp` field on the PlantedWalk fixture. The test deliberately
  asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4
  rather than against a stored value — its doc comment says so — which
  makes the field not just unused but a trap: the obvious "fix" of
  asserting against it would quietly weaken the test to comparing the
  fixture with itself. Removed.
- Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut,
  three vec!s that only ever needed arrays, a filter_map whose every arm
  was Some, and a Vec::new()+push chain.
- Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering
  in the generated docs.
- A five-[u8; 16]-tuple return type named FourLevelParts.

Three lints are allowed at the specific sites, with reasons, because
they are wrong for this domain: the underscores in the bitstream-header
literals mark BITFIELD boundaries, not digit groups, so regrouping them
uniformly would satisfy the lint by destroying the only thing they
encode; and in three table-validation loops the loop variable is the
domain value under test (a DTS SFREQ code, an AMODE value, a palette
entry number), which is what the assertion messages name.
2026-07-31 15:08:37 -07:00
Matthew Jackson d50a7173ad Expose error_code so consumers can read a code instead of parsing one
io_error_code was private, so the predicates built on it (is_halt,
is_skippable_title_stub, is_disc_level_no_key) were the only way to ask
anything about an io::Error's origin. A consumer that needs the code
itself — to report WHY a title failed rather than to branch on one of
three known cases — had no route to it: mux_stream returns an io::Error,
the typed Error is gone by then, and only the E<code> string prefix
survives.

That left every front-end to re-implement the prefix parse by hand,
which is precisely the string-matching 1.5.x spent its time removing.
One parser, exported.

No behaviour change: the function is unchanged and the three predicates
still call it.
2026-07-31 14:46:39 -07:00
Matthew Jackson e9811a1e01 ci: make the branch tip actually buildable
CI checked out one repo, but libfreemkv path-deps ../freemkv-unlock on
the branch tip — release.sh swaps that to a git tag only inside the
TAGGED commit, then restores the path dep on the branch. So the branch
tip has never been buildable in CI by construction, and every green run
we have ever had was a tag build. Windows and Linux were first compiled
at release time, which is the worst moment to discover a build error.

Both repos now check out into subdirectories (actions/checkout refuses a
`path:` outside $GITHUB_WORKSPACE, and ../freemkv-unlock is outside), so
the path dep resolves exactly as it does on a developer's machine. Every
cargo step runs with working-directory: libfreemkv, and rust-cache is
pointed at the same workspace.

The sibling is taken from `dev`. On main/tag builds the Cargo.toml in
that commit carries the git-tag dep instead, so the extra checkout is
simply unused there.

This is what makes the new branch policy's "dev must be green"
achievable rather than aspirational.
2026-07-31 13:27:05 -07:00
Matthew Jackson f3841c8aca ci: build dev as well as main
Work now lands on dev and CI must be green there; main only moves at
release time, to the tagged commit, so a push to main is the release
validation run rather than day-to-day feedback.

leak-guard already runs on every branch (on: [push, pull_request]) and
release.yml stays tag-triggered, so neither needed a change.
2026-07-31 13:16:56 -07:00
Matthew Jackson 7d48d820e5 fix(css): revert the hard-fail — it made real DVDs unrippable
I broke DVD ripping earlier today and the real-media acceptance gate
caught it on its first full run. Greenland.iso failed with E7013
"Decryption failed"; reverting only this change made it rip clean in 8
seconds. That is a regression I introduced, not a pre-existing defect.

WHAT I GOT WRONG.

Round 9's crypto lens reported that descramble_region "descrambles with
a key it just proved wrong" when the crib check rejects the cached key
and the re-crack also fails. I agreed, and made it Error::DecryptFailed
to match the AACS path, on the reasoning that CSS has no external key
source so a failed crack on a readable sector should never happen.

The premise was wrong. `attack_crib` is a HEURISTIC, not a proof: it
finds a periodic run in the unscrambled header and predicts the run
continues past 0x80. When that prediction does not hold, the crib
reports a mismatch even for a CORRECT key — and the re-crack then fails
BECAUSE the crib was never valid. So crib mismatch plus crack failure is
the signature of a crib false positive, not of a stale key. The cached
key is not proven wrong; it remains the best available evidence, and on
a real DVD it is very probably right. Real discs hit this constantly.

The deeper error was treating "no key" as one thing across schemes. An
AACS unit key either opens a unit or it does not — the Verify-Media-Key
relation decides it, and a wrong key is provable. A CSS title key is
recovered from the data itself by an attack whose success varies sector
by sector, so "the crack failed here" says something about THIS SECTOR's
plaintext, not about the key. Unifying the policy was right for the
schemes that can prove a key wrong. CSS cannot, and I folded it in
anyway.

decrypt_span keeps its shape and the cross-scheme test keeps its two
AACS arms, with CSS now explicitly excluded and the reason stated.

Three tests asserted the wrong behaviour and are corrected, including
one I rewrote earlier today to pin exactly this. Every one of them
passed the whole time the code was broken — because none of them had
ever seen a real disc.

The lesson is the one I kept stating and then did not act on: 3,013 unit
tests, ~400 mutants killed and nine audit rounds did not catch this, and
one acceptance run did. Synthetic media cannot reproduce what a real
disc does.
2026-07-30 22:07:32 -07:00
Matthew Jackson 42591c77fc test: constrain the AC-3/E-AC-3/DTS header decode and the boxes it emits
All 59 measured survivors in mp4/audio.rs: 50 killed, 9 proven
equivalent, none left unaddressed. No production change — every
extraction reads correct against ETSI TS 102 366 (5.3.2, 5.4.2, Annex
E.1.3, F.4, F.6.1) and TS 102 114 5.3.1.

It was a fixture gap, not a code defect, and a specific one: the
existing fixtures gave several fields the SAME value (fscod=0, bsmod=0,
lfeon=1, acmod=7) and asserted only derived channel counts. Nothing
asserted the emitted dac3/dec3/ddts payload BYTES at all, so the
packer's shifts and masks were entirely unconstrained. A wrong mask
there does not crash — it writes a box declaring the wrong channel
configuration, and a player believes it.

Several kills needed fixtures designed to discriminate rather than
merely exercise:

  the reduced-rate branch needed fscod == 3, which no test in the file
  reached — and `== -> !=` survived on an fscod=0 fixture only because
  the reduced table happens to return 48000 there too

  the DTS LFF mask needed a value where XOR and AND differ in MEANING:
  on the 5.1 fixtures LFF flips 1 -> 2 and BOTH codes mean "LFE present"

  the acmod mix-level skips needed three different acmods, because `^`
  is a no-op unless acmod == 4 exactly

  the ddts bitrate needed 44100/512, which does not divide evenly — at
  48000 the rounding is invisible and `/ -> %` on den/2 survives

The 9 equivalents are one pattern: OR-ing a shifted high part with a
masked low part on disjoint bit lanes, where the mask is on the same
line as the OR. Since cargo-mutants applies one mutation at a time, no
single mutant can break both. Each was applied and observed green.

FILED, not fixed: reserved sample-rate codes are silently guessed as
48 kHz (audio.rs:133, :20/:153) while a reserved DTS AMODE is refused
with a comment explaining why. Same silent-wrong-metadata class,
opposite answer. Refusing would make such a stream unmuxable, which is a
product call.
2026-07-30 21:51:26 -07:00
Matthew Jackson 2efe1425d6 refactor(decrypt): one orchestrator owns the no-key decision
There were TWO top-level decrypt paths: decrypt_sectors_impl for CSS and
clear media, whose AACS arm was a bare `return Err` stub, and a wholly
separate decrypt_sectors_mapped for AACS. Each scheme therefore decided
its own answer to "there is no key for these bytes", and nothing held
them to the same one.

They drifted, in opposite directions, within a single release:

  css::descramble_region descrambled with a key the sector's own crib had
  just proven stale — garbage behind an intact clear header, reported Ok.

  the mapped path returned early for any LBA outside every range, before
  ever asking whether those bytes were ciphertext, so an unkeyable
  encrypted unit passed through and extract counted it as good.

Both were fixed individually earlier today. This removes the shape that
allowed them.

decrypt_span is now the single orchestrator: it owns the loop, the
refusal, and the loss count, and each scheme supplies only what
genuinely differs. apply_aacs_map is a scheme step that reports what it
could not open; it no longer decides what that means. The public
wrappers (decrypt_sectors, _in_content, _mapped) are unchanged in
signature and all funnel through it.

Adding a scheme now means adding an arm here, which means answering the
refusal question. That is the point.

The new test asserts ONE verdict across all three schemes — AACS with no
map, AACS with an encrypted unit outside every range, and CSS whose
re-crack failed — plus that clear media is NOT a refusal. A per-scheme
test cannot hold this: each would keep passing while the two disagreed.
Flipping the AACS arm back to pass-through reds it.

Also removes the last of the tracing-capture scaffolding. Serialising
those captures crate-wide did not fix the 1-in-10 flake, and asserting
the predicates directly made the helper, both capture subscribers and
an unrelated dead OrderSink unused. Deleted rather than left behind.
2026-07-30 21:35:33 -07:00
Matthew Jackson b86f7aef17 fix(session): delete two dead accessors, make into_drive fallible
drive() and drive_mut() had ZERO callers — not in libfreemkv, freemkv,
autorip, bdemu, keysources or kdb. Deleted rather than converted: dead
public API that panics is not an API worth preserving the shape of.

into_drive() had two callers and now returns Result. The empty-slot
state is reachable through ordinary public use — stage_drive_as_reader
moves the drive into the reader slot, and calling into_drive twice moves
it out — so the panic was not guarding a caller error. identify() was
converted for exactly this reason in this same release; the fix went to
one of four public sinks and the other three were left.

I deferred this on the assumption the blast radius was large. It was
three call sites. Checking beats assuming.

Also fixes a REAL FLAKE in the gate, which is worth more than the above.
resolve_vid_only_bus_key_gate_reports_true_has_volume_id... failed about
one full-suite run in ten while passing every time in isolation. It
installed a capturing tracing subscriber to read back the has_volume_id
field of a warn.

That cannot be made reliable: dispatcher::set_default is THREAD-LOCAL
while tracing's callsite-interest cache is GLOBAL. The original author
knew, and called rebuild_interest_cache() — necessary but not
sufficient. I first serialised every capture in the crate behind one
lock (harness::with_captured_tracing, which also removed the same
hand-rolled dance from three other sites). Still 1-in-10, because the
cache can be re-evaluated against the process-default dispatch rather
than the thread-local one.

So the predicate is now a named function, handshake_has_volume_id, and
the test asserts the VALUE. A boolean does not need a subscriber to
check. The gate's hard-error behaviour keeps its own test.

Measured: 14 consecutive full-suite runs, 2994 passed, 0 failed.

A flaky gate is worse than a missing one — every green after it means
less, and this one had been eroding trust in the whole suite.
2026-07-30 21:18:13 -07:00
Matthew Jackson 5559987325 test(mux): replace a false-green discontinuity test with the property it named
a_signalled_discontinuity_survives_a_backstop_discard asserted that a
SOURCE-signalled discontinuity on discarded bytes still reaches the AU
that follows. It did not test that. Deleting the disc_marks push, or the
mark-retirement loop inside discard_gap_before, left it passing.

The mechanism: the `discontinuity = true` rode the FIRST over-cap push,
which still has the next AU's delimiter at buf[0] — so it force-flushes
as an over-long AU rather than discarding, and THAT AU consumes the
mark. The assertion's `.find(|x| x.data.contains(&0x22))` then filters
it out, and the flag it reads comes entirely from `pending_gap`, set by
the second push's backstop. Behaviourally identical to the test 40 lines
above it, under a name promising something else.

I wrote it this morning, in the same commit that fixed a different test
for having a fixture that never reached the code it named, while
cataloguing that exact shape. Third instance today of writing the bug I
was hunting.

The two mechanisms cannot be isolated in one fixture — a fragment that
trips the backstop sets pending_gap regardless — so they now get one
test each. The replacement drives disc_marks end to end with no backstop
involved: a flagged fragment that carries a complete AU and is emitted,
not discarded. Nothing else pinned that path. Removing the disc_marks
push reds it.

Found by the round-9 opus escalation over test quality, dispatched
because the sonnet pass over the same 17,000 lines of new test code
returned zero findings.
2026-07-30 20:07:11 -07:00
Matthew Jackson 72bcc371fb fix(io): a halted fsync is recognisable as a halt, not a hard failure
The three bounded-fsync failures returned bare io::ErrorKind values —
TimedOut, Interrupted, Other/EIO. `is_halt()` matches on
`io_error_code(e) == Some(E_HALTED)`, i.e. the "E<code>" prefix that
`From<Error> for io::Error` mints, and that is documented as the ONLY
recognised shape. A bare ErrorKind carries no prefix.

So cancelling a rip while sync_all / finish was inside the bounded fsync
made `is_halt()` return false, and the CLI reported a clean user cancel
as a hard I/O failure at the end of an otherwise complete mux.

All three were also mutually unclassifiable, which is the same
information-loss the numeric-code scheme exists to prevent: a caller
could not tell "cancelled" from "NFS wedged" from "worker died", and
should not retry the third the way it retries the first. And their
Display text is std English — "timed out", "operation interrupted" —
reaching a user from library code, which this crate does not do.

Now Error::Halted, Error::SyncTimeout (E_SYNC_TIMEOUT 9056) and
Error::SyncWorkerLost (E_SYNC_WORKER_LOST 9057), on both platforms.

E_HALTED also now maps to ErrorKind::Interrupted rather than falling into
the 6000..=6999 InvalidData bucket. A stop is an interruption, not
invalid data. Nothing branched on the old kind — every consumer uses
is_halt() — so this is safe as well as more accurate.

Two things worth recording. I first placed the E_HALTED arm AFTER the
6000..=6999 range arm and wrote a comment claiming it preceded it; match
arms are ordered, so the range won and the comment was simply false. The
test caught it. And the macOS test asserted only that each arm was
non-Ok with a particular ErrorKind — it passed throughout the period the
three were indistinguishable. It now asserts they can be TOLD APART,
which is the property that actually matters.

Found by the round-9 opus escalation over the API contract.
2026-07-30 20:04:08 -07:00
Matthew Jackson 54d038e478 fix(udf): file_start_lba must skip a leading unrecorded extent
Regression from the round-8 change that started RETAINING ECMA-167
4/14.14.1.1 type-1 (allocated, not recorded) descriptors. Retaining them
is correct — dropping one slides every later extent's data down by the
hole's length, corrupting the file silently. But read_icb_extent still
took extents.first(), so the value it returns can now be a hole.

A type-1 extent's lba is where SPACE is allocated, not where bytes live.
IcbExtent's own doc says exactly that. file_start_lba hands the value
out as "the absolute starting LBA of a file's first data extent", and
ifo.rs uses it as the base for every VTS VOB extent:

    file_start_lba(IFO) + vtstt_vobs + cell.first_sector

So a DVD whose IFO's first descriptor is type-1 reads its entire video
title set from the wrong place on disc. No error anywhere — the reads
succeed, they just land on unrelated sectors. Verified: the test reports
2900 instead of 2040 with the old code.

Same shape as file_extents/extents_abs_at dropping the recorded flag,
and the same root cause: one change taught read_icb_extents about a new
extent type and did not visit the accessors that consume its output.
Three of them; two are still open (task filed).

Found by the round-9 opus escalation over the API contract, dispatched
because the sonnet pass over the same scope returned zero findings.
Seven of its eight items were absences rather than wrong lines — the
class a wrong-line scan structurally cannot see.
2026-07-30 20:00:15 -07:00
Matthew Jackson 6868b93b7e fix(udf): retry the customary VDS location when the recorded one holds nothing
The Main Volume Descriptor Sequence was selected from the anchor's
declared extent whenever that extent's SHAPE was usable — length >= 16
sectors (ECMA-167 3/10.2.1), non-zero location, no address wrap — and
the customary location was tried only when the shape failed.

Shape is a property of the field, not of what is there. An anchor can
pass all three checks and point at nothing: a mastering tool that wrote
the reserve location, a stale anchor on a rewritten volume, or
deliberate corruption on an untrusted disc. The sweep then finds no
Partition Descriptor, partition_start stays 0, and the volume is
rejected as UdfNotFilesystem — while the real sequence sits unread at
the location the old fixed sweep would have found.

So the branch a DAMAGED disc actually takes had no recovery path, which
is backwards: that is the branch recovery exists for.

Now both are candidates and the fallback is retried on OUTCOME. This is
the same principle the Metadata File Location chain in this same
function already uses — treat the recorded value as a candidate, fall
back when it does not pan out — applied one level up. The sibling was
added in d5a9e70 and the asymmetry has been sitting there since.

A read fault inside a sweep no longer aborts before the next candidate
is tried, but it does not vanish either: it is held and returned if no
candidate yields a partition. "Could not read" stays distinct from "read
fine and the bytes say no", because mux::resolve caches the second for
the whole disc.

Not addressed, and pre-existing: the RESERVE sequence at avdp[24..32]
exists in ECMA-167 3/8.4.2 for exactly a damaged main sequence and is
still not consulted. Worth doing; a bigger change than this one.
2026-07-30 19:49:03 -07:00
Matthew Jackson 8d39ccc613 fix(decrypt): an encrypted unit outside every key range fails, not passes
decrypt_sectors_mapped returned early for any LBA no map range covers,
on the reasoning that unmapped means clear filesystem or nav. "The map
has no key here" and "there is nothing to decrypt here" are different
statements, and only the second makes passing the unit through correct.

On a multi-CPS disc an orphan clip — referenced by no playlist, so in no
title extent and therefore in no range — hits the first and was treated
as the second. The early return fired BEFORE the aacs_unit_encrypted
gate below it, so nothing ever asked whether those bytes were
ciphertext. extract_tree then counted them as bytes_good, dropped the
.partial suffix, set complete = true and exited 0: a scrambled file on
disk with a clean bill of health.

FileResult's own doc already stated the intended contract — "unreadable
sectors AND undecryptable units both land here (extract fails a bad
decrypt loud)". The code did not implement it on this path.

The fix is one line of symmetry. The split-unit branch immediately above
already makes exactly this distinction, checking aacs_unit_seed_encrypted
before refusing. This branch did not, so the same question got two
answers eight lines apart — the duplication shape this release keeps
finding.

Deliberately NOT changed: the decrypt decision on an orphan is still to
refuse rather than guess. Blind trial-decrypt is what the keymap-only
model exists to remove, and extract has no CPS/forensic fetch source.
The defect was never that we declined to key it; it was that declining
looked like success.

The test asserts BOTH directions — a clear out-of-range unit must still
pass through byte-identical, because that is the ordinary whole-disc
read and breaking it would trade one silent defect for a loud one.
2026-07-30 19:44:37 -07:00
Matthew Jackson 8c0de5711e fix(mux): resync-gate drops reach errors(), including after the gap resolves
ResyncGate::dropped is zeroed the moment a keyframe disarms the gate,
and the only EOF warning fires for gates STILL armed. So a mid-title gap
that resolves left no trace anywhere — and most gaps do resolve. A rip
with several concealed gaps reported 0 errors and 0 lost bytes while
whole GOPs had been discarded, which disc.rs's own test comment calls
the ONLY channel through which loss is reported.

This is the other half of e99b634. Arming the gate after an 8 MiB
backstop discard is right — a picture with dangling references must not
ship — but until the drop is counted that trades silent corruption for
silent loss.

The gate now carries dropped_total alongside dropped: per-run answers
"how expensive was this gap", cumulative answers "what did the caller
lose". errors() sums the gates.

Summed in ONE place rather than counted at the three admit call sites.
Three copies of the same increment is how the mux-flush path ends up
counting and the main path not, or the reverse — the duplication shape
this release has been removing. The gate already knows its own total;
the accessor just has to ask.

Found independently by two round-9 lenses, which is what raised it from
plausible to worth acting on.

Both halves are pinned: removing the dropped_total increment reds the
resync test, and removing the sum from errors() reds the DiscStream one.
The second test asserts the ACCESSOR rather than the gate's counter,
because a test on the counter would have passed throughout the entire
period the defect existed.
2026-07-30 19:40:48 -07:00
Matthew Jackson 9f25a4c454 fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote
it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry
(12.1.3). Both fields are MANDATORY there, so unlike Matroska — which
omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to
leave out. The result was a structurally complete file that passes every
container check, declares a 0x0 video track, cannot be rendered, and is
written with no error anywhere.

WHY IT WAS POSSIBLE, which is the part worth keeping:

pixels() previously fabricated 1920x1080 for Unknown. That was wrong but
playable, so this sink never needed a guard and the absence of one was
invisible. Changing the sentinel to (0, 0) moved the defect instead of
removing it — a zero PAIR still reads as a usable value, so the sink
stored it and serialised it.

The accessor's doc comment then ENUMERATED the callers it believed were
safe: "the Matroska sink omits the optional elements, the VobSub writer
omits its size: line, and no caller divides by either dimension." Two of
those three are true. MP4 was not on the list because MP4 has no guard
at all, and a prose list cannot enforce itself. mkv.rs's own comment
even states the principle — "the check belongs in the one accessor
rather than in each caller that remembered to write it" — and
labels/mod.rs still carried its own duplicate Unknown test long after
the accessor took that job over.

So: pixels() now returns Option. Not because Option is tidier, but
because every caller genuinely needs a DIFFERENT answer and the compiler
is the only thing that reliably makes them choose one. Matroska and the
metadata sinks take unwrap_or((0, 0)) with the reason stated at each
site; the VobSub path degrades to a palette-only .idx; MP4 fails with
E_MP4_UNKNOWN_RESOLUTION (9055).

Six call sites, not the five my first grep showed — I piped it through
`head` and acted on a truncated list. The compiler caught the sixth.
That is the same mistake as trusting a lens that reported silence.
2026-07-30 19:34:54 -07:00
Matthew Jackson 30bea12392 fix(css): no provable key is a hard failure, matching AACS
descramble_region descrambled with the key a sector's own crib had just
proven stale, whenever the re-crack from that sector also failed. The
clear header is not scrambled, so it survives intact: the sector still
opens with a valid pack start and passes every structural check the PS
demuxer applies. Only the payload is corrupted — exactly where nothing
looks. Ok(0) dropped, exit 0.

CSS has no external key source. The title key comes only from cracking
the data, so on a READABLE sector "no key" is not a missing input, it is
recovery failing on bytes we can see. That should never happen, and when
it does the answer is not to emit something.

Now Error::DecryptFailed — the same verdict the AACS path already gives
for a unit no held key opens. Both alternatives to failing are bad data
reported as success: descrambled with a rejected key it is garbage
behind a valid header, and passed through untouched it is ciphertext
where plaintext is meant to be.

WHY IT WAS POSSIBLE, which matters more than the fix:

There is no single place that owns "what do we do when there is no key".
decrypt_sectors_impl looks like the central dispatch, but its AACS arm
is a `return Err` stub — AACS decrypts entirely through
decrypt_sectors_mapped, a separate top-level path. So CSS decided its
own policy inside css/, AACS decided in decrypt.rs and mux/resolve.rs,
and nothing held them to the same answer. The asymmetry was not an
oversight; it was structurally permitted.

How a disc decrypts is one process — resolve a key for this data, apply
it, refuse if it cannot be proven. Only the resolve-and-apply step is
scheme-specific. Filed as a task: the policy belongs in one orchestrator
with the schemes supplying only what genuinely differs.

Two tests changed rather than added, both of which pinned the old
behaviour: the unit test asserted the sector was descrambled, and the
integration test asserted the scramble flag was cleared, which is what
descrambling-with-any-key does. Neither established that the result was
CORRECT — the fourth bad-test shape.
2026-07-30 19:10:32 -07:00
Matthew Jackson b2b611fa3b fix(udf): a read fault locating the Metadata File is not a non-UDF disc
Regression I introduced in d5a9e70 earlier today. The Metadata File
Location fix replaced

    read_sector(reader, meta_file_lba, &mut meta_icb)?;

with a candidate loop guarded by `.is_ok()`, which discards the error.
A transient read fault — marginal sector, drive re-read, an ECC recovery
that reports failure once — then falls through to the block-0 fallback,
the File Set Descriptor read there finds the wrong tag, and the volume
comes back as Error::UdfNotFilesystem: the deterministic verdict "this
is not a UDF disc" for a retryable I/O event.

mux::resolve MEMOISES that negative for the whole disc, so one flaky
read silently demotes every remaining title to the base-Unit-Key-only
path — the AACS 2.1 forensic units garble and the demux drops them. The
mux completes with less content and no error, which resolve.rs's own
comment says must never happen.

This file already draws the same distinction twice in prose, at the AVDP
check and the FSD check: read fine but the bytes say no is structural;
could not read is transient. The loop erased it.

Two things I got wrong on the way, both worth recording:

The first guard was `meta_tag == 0` — nothing read at all. That misses
the damaging case, which is the RECORDED candidate faulting while block 0
reads fine and holds some other descriptor. meta_tag is then non-zero and
not 266, and the fault is still laundered into a structural negative. The
guard is now `meta_tag != 266`: any unread candidate leaves the verdict
unproven, so the fault wins.

The first test denied BOTH candidate LBAs, and passed WITHOUT the fix —
the block-0 read fails too, so a read error propagates from further down
either way. A test whose fixture never reaches the changed line. That is
the eleventh bad-test shape this audit catalogued, and I walked straight
into it while fixing a defect found by looking for it. The fixture now
denies only the recorded location, and removing the guard reds it.

Found by the round-9 opus escalation over correctness, dispatched
because the sonnet pass over the same scope returned zero findings.
Silence from a cheap model is not evidence.
2026-07-30 18:57:13 -07:00
Matthew Jackson 4f4b1ed222 fix(labels): make deluxe master-enum selection deterministic
identify_master_enums picks, for each fingerprint, the best candidate
class out of CandidatePool. Its tie-break only prefers an exact ldc
count over an inexact one, so two candidates that are BOTH inexact but
both within LDC_COUNT_TOLERANCE are decided purely by iteration order —
and the pool was a HashMap.

Rust seeds HashMap per instance, so this is not merely unstable across
runs: the new test resolves the SAME jar to both Alpha and Beta within a
single process, across 16 iterations. The same disc could emit different
commentary/SDH/descriptive labels on consecutive rips of unchanged
input, with nothing in the output saying the choice was arbitrary.

BTreeMap fixes it by construction rather than by a sort someone can
forget to keep. The pool is capped at MAX_CANDIDATE_CLASSES, so the
ordering cost is irrelevant.

The test runs the whole identification sixteen times and asserts one
distinct winner. A single run cannot distinguish deterministic from
lucky, and the seed does not change within a process — so repetition is
what makes this a test rather than a hope.

Found by the round-9 labels pass, which was dispatched specifically
because every one of the ten lenses had reported leaving deluxe.rs
unread. 1,159 new lines that nobody had opened.
2026-07-30 18:49:59 -07:00
Matthew Jackson 944e6a8b09 fix: align the Linux fsync error with macOS, and clear three stale docs
Round 9 findings, triaged and verified against the pinned tree.

writeback_file: a bounded-fsync WorkerLost returned bare ErrorKind::Other
on Linux where macOS returns EIO. Round 8 fixed the Linux arm to return
Err at all — the right fix — but stopped short of matching the value, so
a consumer distinguishing timeout / halt / lost-worker had nothing to
branch on for the third case on one platform. Now EIO on both.

Three doc comments described the pre-fix behaviour, one of them for
longer than the bug existed:

  linux.rs durable_sync still said "all three fallbacks return Ok(())"
  mod.rs sync_all still said Linux silently swallows fsync failures and
    callers must not treat Ok(()) as a durability barrier
  mod.rs SequentialSink::finish repeated the same caveat

All three now say what the code does: a bounded-fsync failure is an Err
on every platform, so Ok(()) IS a durability barrier. A doc that
describes a fixed bug is worse than no doc — it tells a caller to write
a workaround for something that no longer exists.

au_assembly: discard_gap_before duplicated drop_marks_before's
mark-retirement body verbatim and added one statement. Mine, from
earlier today. It now calls it. Two copies of the same retirement loop
is exactly how the two call sites would drift back together.

clpi: ClpiStream's audio_format / audio_rate / video_format / video_rate
are decoded from untrusted on-disc bytes on every parse and read by
nothing. The identically-named fields consumed in disc/bluray.rs belong
to mpls::StreamEntry, not to this struct — checked, because an earlier
round wrongly called a live function dead. Deleted, along with the seven
test assertions that pinned them; the tests that pin pid, coding_type
and language remain. Also removed a section-header comment orphaned by
the get_extents deletion, describing a fixture that no longer exists.
2026-07-30 18:39:47 -07:00
Matthew Jackson 5360f8d309 test: salvage the orphaned labels/disc triage, and extract build_labels
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to freemkv-private first, because a
worktree is not a backup and this one had already nearly been lost.

One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.

Two collisions resolved by hand:

A second mod pass_progress_tests, written independently against the
same survivors as the one committed in c610285. Kept mine — it covers
the distinct-counters case and the Progress blanket impl, which theirs
does not — but theirs had three clamp tests mine lacked: good_pct,
bad_pct and pending_pct also clamp an overshoot, and I had only tested
that for work_pct. Merged those in as one test and proved each of the
three clamps load-bearing by removing them individually.

An unused_parens warning in a new fixture.

Method note, recorded because it cost real time: git apply --3way
STAGES its result, so `git diff` reads empty and the tree looks
untouched. I nearly concluded the patch had silently failed. Worse, the
first attempt piped through `head -20`, so `echo exit=$?` reported
head's status rather than git's — the same mistake this audit has
already documented once. Check the real exit status, and check
--cached, not just the working tree.
2026-07-30 16:36:13 -07:00
Matthew Jackson 8b8bcff106 test: pin five untrusted-input guards in the AACS 2.1 and CSS paths
Second pass over src/aacs and src/css. No production change; the only
non-test edits are two fixture bytes and one test rename.

Five latent panics on untrusted data, every guard correct and none
tested — so each was free to be deleted:

  variant.rs:224  a 0x04 record not a multiple of 5 indexes p_uv[0..4]
                  off a one-byte tail
  variant.rs:269  a 0x0c record shorter than the 0x04 slot count
                  slices past the cvalue table
  stevenson.rs:177  short sector read -> index 138 into a 129-byte slice
  stevenson.rs:208  a crib longer than the 1920-byte encrypted region
                    -> index 2058 into 2048
  stevenson.rs:272  a header periodic all the way to offset 0 ->
                    subtract with overflow

That last one is reachable from ORDINARY DVD data — constant or padding
bytes are periodic. Verified on HEAD: widening the guard to <= 0x80
passes all 64 css tests unmutated.

media_key_variant_from_kp had only a soft-correction test, so every
step past that early return was unexecuted. The new two-slot fixture
puts the covering slot at index 1, so the uvs[1 + 5*idx] and
cvalues[idx*16] strides stop multiplying by zero.

derive.rs:319 + -> - confirmed killable, as the first pass predicted:
p == 0 makes (p-1)..32 underflow. Every prior fixture used a uv whose
lowest set bit was 4, 10 or 11, so trailing_zeros() was never 0.

One fixture bug caught and fixed rather than papered over: a |= mutant
first SURVIVED because mk[14]'s 0x04 bit happened to be set, making OR
and XOR agree. The byte is now clear and an assert_eq! pins it, so the
fixture cannot drift back into agreeing with the mutation it exists to
catch.

walk_mkb_be24_high_byte_is_honored renamed to
walk_mkb_be24_middle_byte_is_honored. Its 0x00_0110 length exercises
the << 8 term only, which is why << 16 -> >> 16 survived it. The name
was the lie; both framings are worth having, and the comment now points
at the genuine high-byte test at 0x01_0004.

derive.rs 146:32 and 154:30 stay untested, now with a proof rather than
a judgement: bit_pos == -1 requires current_v_mask == 0xFFFF_FFFF, and
calc_v_mask can never return that — its loop condition holds at
!v_mask == 0, so it always shifts at least once. Both branches are
reachable only after the walk has gone non-convergent and is heading
for the bounded exit, where the return value is undefined. Termination
is already pinned.

Equivalents proven by observing green, including six more OR/XOR pairs
on provably disjoint bit fields, and the two KEY_CORRECTION_DATA sites
where the constant is the documented all-zero placeholder so x ^ 0 ==
x | 0. Those become killable only if a real per-licensee KCD is wired
in.

A partial confirmation sweep (138 of 415 mutants before the box
saturated) found 135 caught, one timeout that is itself a detection,
and exactly one survivor — the KEY_CORRECTION_DATA equivalent above.
2026-07-30 16:27:03 -07:00
Matthew Jackson d5a9e70700 fix(udf): read the Metadata File Location from the partition map
read_filesystem hardcoded the Metadata File's File Entry at block 0 of
the physical partition — 'the metadata file ICB is at physical
partition lba 0'. UDF 2.50 2.2.10 records where it actually lives, as a
partition-relative Uint32 at offset 40 of the Metadata Partition Map.
That field is the only thing on the volume that says where the entry
is; block 0 is merely where authoring tools usually put it.

On a conformant volume that recorded it elsewhere, block 0 holds
something that is not a File Entry, metadata_start falls back to
partition_start, the File Set Descriptor read there carries the wrong
tag, and the volume is rejected as UdfNotFilesystem. Worse, a volume
with a decoy file set at block 0 — as a rewritten or dual-structure
volume can have — does not error at all: it mounts a different
filesystem and reports success.

Verified on HEAD: reverting the lookup reds four tests, e.g. the
metadata partition beginning at 2000 where the map records 33754069.

The recorded location is trusted only when the map's partition type
identifier reads '*UDF Metadata Partition'. A Virtual (2.2.8) or
Sparable (2.2.9) map is ALSO ECMA-167 3/10.7.3 Type 2 and records
unrelated fields at offset 40, so its bytes must never be read as a
location. Deleting that guard reds its own test.

Block 0 stays in the candidate chain, so a volume whose map is absent
or wrong but whose Metadata File does sit there keeps mounting exactly
as before. This is additive, not a behaviour swap.

Also 30 tests and ~55 more mutants across read_icb_extents,
read_file_limited, read_inline_data, the prefetch stubs, parse_dstring
and parse_udf_name. The metadata-partition branch — the branch EVERY
real BD-ROM takes — had no test at all; nothing in the crate built a
two-partition-map volume.

Closes the max_bytes gap flagged earlier: 259 > -> == and > -> < now
die on both the declared-size and the inline-ICB paths.

Equivalents proven by application, notably two guards that read as
protective but are unreachable: pm1_len is a single byte so
440 + pm1_len < 2048 always holds, and ad_offset + l_ad <= 2048 is
enforced upstream so off + ad_size never exceeds the block.

Bit 0 (Existence) stays unread, deliberately. ECMA-167 4/14.4.4 makes
it a display hint, not a statement that the file is absent, and UDF
2.50 2.3.4.2 carries it through as the DOS hidden attribute. For a
ripper the consequences are asymmetric: honouring it can silently drop
a real .m2ts from the title list, ignoring it costs an extra name in a
listing.

Known structural limit, not fixed: read_filesystem takes only the FIRST
extent of the Metadata File, so a fragmented metadata partition would
map every sector past that extent to the wrong place. metadata_start
being a single base LBA is what forbids the fix.
2026-07-30 16:09:35 -07:00
Matthew Jackson 0bc8d7af9c test: constrain the AACS key-map gap fill, the PSI walk, and MP4 field offsets
Third pass over src/mux/. 40 survivors killed, no production change.

resolve.rs — the deleted-statement cluster is now fully constrained.
All 14 deletable statements probed; 9 were already caught, 5 survived:

  c.sort_unstable() in fill_base_key_gaps. Every existing case handed
  it cuts already in LBA order, but IndividualSegment.tbl is a record
  list. Verified on HEAD: deleting the sort passes all 54 resolve
  tests. The mutant lays a base-key fill straight over a forensic
  segment.

  last_idx = idx (FMTS gap fill) and last_idx = hit (multi-CPS cache
  hit). An extent with nothing to sample must inherit its neighbour's
  CPS unit; the mutants fall back to the first unit's key. Exactly the
  shape this file's own comments name — wrong key, no error,
  lost_bytes == 0.

  Both check_halt()? polls in probe_fmts_index_keys. These cannot be
  killed by outcome, since a later poll returns Halted too. The tests
  count reads instead, which is what the don't-hammer-a-struggling-
  drive rule actually says: after a Stop the drive is asked for zero
  content sectors.

The four unresolved += 1 arms each got a test, and deleting each fails
exactly one — one-to-one, so no fixture passes for the wrong reason. A
control test pins that the baseline table resolves, so an expect_err
cannot succeed for an unrelated reason.

ts.rs::scan_streams was never entered. Six killed, two of which return
wrong answers that look right: reading the PAT/PMT CRC as a table entry
invents a stream on PID 546 out of CRC bytes, and dropping the
ES_info_length skip decodes a descriptor as an entry and loses the one
after it. Every existing PMT fixture declares ES_info_length = 0; a
real BD PMT carries a registration descriptor on essentially every
entry. Also ISO/IEC 13818-1 2.4.4.3 program_number == 0 is the network
PID, not a program.

mp4/read.rs — 13. Height read as the width beside it; channelcount;
the 4-byte base-128 descriptor varint (every existing esds fixture uses
a single byte); all three optional ES_Descriptor fields, whose loss is
silent (an AAC track just loses its CodecPrivate); first-vs-last media
edit, which is A/V desync of the difference; and the version-1 mvhd
timescale offset, emitted by any writer whose duration exceeds 32 bits.

dts.rs — 7, from a real cargo-mutants run over the file rather than
guesswork. Including a buffer that IS the syncword, which is the state
a sync split across PES packets lands in the moment its last byte
arrives.

Equivalents proven by application, not argued: the sample_encrypted_units
guard pair is mutually redundant by construction (total*p/9 < total for
p <= 8), so either alone is equivalent and both together are not; the
PMT section_len guard is dead code where its PAT twin panics; three of
the seven EXSS_HEADER_MIN_BYTES arithmetic mutants still sum to 10.
2026-07-30 16:05:37 -07:00
Matthew Jackson e99b634635 fix(mux): a backstop discard is a discontinuity; a stream-start trim is not
drop_marks_before retired discontinuity marks alongside timing marks at
both of its call sites. At stream start that is right. At the
MAX_AU_BUFFER backstop it is not, and the two are now separate.

The backstop fires when 8 MiB accumulate with no AU start code in them
— corrupt or hostile input — and throws the run away. There IS a prior
AU in that case, and whatever emits next definitively does not continue
it: a decoder handed that picture resolves its references against
frames separated from it by megabytes of discarded data. Retiring the
flag meant the resync gate (resync.rs, driven from mux/disc.rs) never
armed, so the broken picture went out looking sound. Silent corruption
is the one class of loss this crate refuses to have.

At stream start the opposite holds. Bytes ahead of the first
access-unit delimiter are the tail of an AU that began before sync, and
there is no prior AU to be discontinuous from. Marking it would arm the
gate at the head of every title and drop its opening GOP. That risk is
why this was a decision rather than a fix, and splitting the call sites
is what avoids paying it.

Recorded as a sticky flag, not an offset mark. A mark placed at the new
base is retired moments later by the pre-sync trim that follows resync
— the gap has to outlive the bytes that caused it. I found that by
writing the test first and watching it fail with the mark approach.

The discard is a discontinuity whether or not the source signalled one,
and a signalled one on discarded bytes still reaches the AU that
follows; both directions are tested.

Note the first over-cap run is NOT a discard: the next AU's delimiter
is still at buf[0], so it force-flushes as an over-long access unit and
loses nothing. Only a run with no opener at all reaches the backstop.
The tests push twice for that reason — the single-push version passes
without the fix.

Swapping either call site for the other fails: reverting the backstop
reds the two gap tests, and arming the gate at stream start reds the
third.
2026-07-30 14:52:39 -07:00
Matthew Jackson f9d081ed45 test: drive the AACS 2.1 variant chain to a Media Key, and pin AES-G3
163 of 322 surviving mutants across src/aacs and src/css. No production
line changed — every function read correct; the finding was always an
absent test.

Two structural holes, both verified against HEAD before landing.

variant.rs had no test that ever produced a Media Key. Every terminal
assertion in the module was an Err classification — NotVariantMkb,
SoftCorrectionRequired, OnlineChallengeRequired. So the entire 2.1
success path (VARIANTS lookup, VKD selection, Kpnew, the final unwrap,
the verify gate) was pinned by nothing, and that path produces the
Media Key that becomes the VUK that decrypts every byte of a 2.1 disc.
Built the first complete planted variant MKB: the VARIANTS entry is
chosen as Kvn ^ 1 so the real VKD sits behind a decoy at table index 1,
making the lookup load-bearing rather than incidentally correct. That
one fixture kills 23 operator mutants across three functions.

aesg3 — the subset-difference tree node function — was in the survivor
list as replaceable by [0; 16], meaning every device key in the crate
would derive the same Processing Key. It is caught today only as a side
effect of a negative test added after the mutation run; nothing asserted
the relation itself. Pinned now via the spec relation ([C] 3.2.2) using
the FORWARD primitive, with s0 transcribed independently rather than
read back from AESG3_SEED, so the test cannot agree with a mutated
constant.

Same shape in derive.rs: plant_mkb was one slot with zero descent, so
slot indexing was the identity permutation and the ancestor-descent
branch never ran — which is why 39 of recover_dk_position's mutants
survived. Added a 3-slot fixture keyed at index 2 and a four-level
descent fixture whose expected Processing Key is written out as an
explicit aesg3 chain rather than computed by calc_pk_from_dk; a fixture
built by the function under test moves with its own mutations.

Two latent panics on untrusted input now have tests: a 0x05 cvalue
table shorter than the 0x04 slot index, and a drive declaring more
payload than the 32772-byte response buffer holds.

23 equivalents claimed with reasoning, and confirmed empirically where
possible — all eight css/lfsr mutants were run and exactly the seven
disjoint-bit-lane ones survived.

Explicitly NOT claimed equivalent: derive.rs 146:32 and 154:30 are
reachable, but only on the non-convergent bounded-exit path where the
function's sole contract is termination. A test there would pin
defined-but-meaningless output.

Noted for the next pass: the pre-existing walk_mkb_be24_high_byte_is_honored
used total length 0x0110, whose high byte is zero — it exercised the
middle byte only, which is why << 16 -> >> 16 survived it. Left in
place; a real one was added at 0x01_0004.
2026-07-30 14:44:35 -07:00
Matthew Jackson 3e13a155fa refactor(clpi): delete the unused EP-map to sector-extent path
get_extents had no caller anywhere in the ecosystem, and neither did
anything feeding it. Removed: get_extents, resolved_ep_map, full_pts,
full_spn, parse_cpi, EpCoarse, EpFine, the ep_coarse/ep_fine fields,
the unused version field, and the 35 tests that exercised them.

This reverses the fix in b2e1982, deliberately. That commit corrected a
real truncation — out_time past the last EP entry resolved to
last_ep_spn + 1, dropping everything from the final I-frame to EOF —
and the fix stands in history as the record of what was wrong. But the
defect existed for eight audit rounds precisely because the code had no
caller: nothing exercised it, so nothing noticed. Keeping speculative
infrastructure alive on the strength of a comment saying it is reserved
for a path that does not exist is how that happens again.

ClipInfo keeps source_packet_count (read by disc/bluray.rs) and streams
(the CLPI/MPLS cross-validation in labels/clpi_audit.rs). 1,139 lines
out, gate green in debug and release.
2026-07-30 14:39:22 -07:00
Matthew Jackson b2e1982051 fix(clpi): resolve out_time past the last EP entry to the end of the clip
ClipInfo::get_extents fell back to `last EP SPN + 1` whenever out_time
lay past the last entry-point. EP entries mark I-frames (BD-ROM Part 3,
CPI / EP map) and a clip's final GOP lies after the last one, so a
PlayItem covering a whole clip — whose OUT_time is the presentation end
— always lands in that arm. The extent then stopped one source packet
after the last I-frame.

Measured on a fixture with 200,000 source packets and the last EP at
SPN 131,072: sector_count came back 12,289 where covering the clip
needs 18,750. Everything from the last entry point to EOF is outside
the returned extent.

Scope, stated plainly: get_extents has NO callers anywhere in the
ecosystem today — it is #[allow(dead_code)] and documented as reserved
for the timestamp-range read path. Nothing ships this loss. It is
fixed now because a latent truncation in extent arithmetic is far
cheaper to correct before it has callers than after.

The SPN at-or-after an out-of-range out_time is the end of the clip,
source_packet_count, with .max(last + 1) so a disc that under-declares
its own packet count against its own EP map still yields a sane bound.

Also 174 mutants killed across clpi, mpls, ifo and ebml — the first
time any of these four files has been examined. And ebml's 8-byte VINT
back-patch was duplicated verbatim in end_master and end_master_buf
with its top four payload octets unreachable through either (they need
a 16 MiB..256 TiB buffer); extracted to fixed_width_vint8 and tested
across the full 56-bit payload, no behaviour change.

38 of ebml's 46 survivors are one equivalence cluster: every | in
write_size / read_id / read_size / read_uint_val ORs into disjoint bit
lanes, where ^ is the identical operation. Applied all 38 at once —
green — then spot-checked four individually.
2026-07-30 14:24:57 -07:00
Matthew Jackson 84a77f6a0e test: pin FMTS read_plan unit indexing to an unaligned range start
Five surviving mutants in AacsKeyMap::read_plan, all in the arithmetic
that decides which half of a forensic segment this disc's key opens.

The existing coverage used a forensic range starting exactly on the
extent's first unit. Under that shape several wrong formulas agree with
the right one by arithmetic accident: (lba + range_start) / us and
(lba - range_start) * us both produce the correct kept set.

Real ranges are not shaped like that. A range start comes from a source
packet number — start_spn * 192 through clip_byte_to_lba in
mux/resolve.rs — and 192-byte packets bear no relation to the 3-sector
aligned unit, so range_start % 3 is whatever the disc says.

Getting the parity wrong does not crash. It reads and decrypts the
ALTERNATE variant's half: the units this key does not open decrypt to
garbage, the units it does open are skipped. AACS 2.1 forensic marking
is precisely what makes the two halves differ, so the failure is silent
— a full-length rip carrying the wrong variant.

Two fixtures are needed because no single one kills both: an unaligned
range start with the extent beginning on it inverts the halves under
the + form, and an extent offset one unit-remainder from the range
start inverts them under the * form.

Also pinned the short-tail guard.  is for a remnant
SMALLER than a unit — bytes with no following unit to desync. Widened
to <=, the last whole unit of every extent bypasses the phase gate, so
a forensic segment ending at an extent boundary contributes one
alternate-variant unit to the rip.
2026-07-30 14:18:39 -07:00
Matthew Jackson 9de88969ca test: constrain the DiscStream loss surface and the empty-title guards
Second mutation pass over src/mux/. 26 survivors killed, no production
change. Verified on HEAD before landing: each mutation below passes all
1,237 mux tests unmutated-suite.

The priority item was the honest-loss-reporting surface. Both
DiscStream::errors and DiscStream::lost_bytes could return a constant
with nothing failing — a rip that lost sectors would report zero loss
to the caller. This project has already shipped one defect of that
shape (a total decryption failure reported as an empty title, exit 0).
Driven now through two short-read fills so both land on values that are
neither 0 nor 1 and differ from each other; no constant and no field
swap survives.

MkvStream::finish -> Ok(()) also survived. MkvMuxer::finish has the
zero-frame MkvInvalid guard and two tests cover it, but the Stream
wrapper above it could return Ok unconditionally and bypass the guard
entirely — the empty-title defence was one layer thinner than it looked.

au_assembly: pinned au_opener_from behaviourally to the normative byte
values for all four modes, with negative cases for codes that are
explicitly not openers (MPEG-2 slice 0x01..0xAF, user data 0xB2,
extension 0xB5, sequence end 0xB7 per 13818-2 Table 6-1; VC-1
0x0A/0x0B/0x0C; H.264 SPS/PPS/IDR-slice). au_assembly and codec/ hold
independent copies of these constants; they agree today, and comparing
constants would not catch logic drifting apart, so both sides are now
pinned to the spec instead of to each other.

demux_sink::sanitize: every filename component demux:// writes comes
from disc-controlled text, so the path-separator arm is a traversal
guard. Deleting it now fails, including an end-to-end case where
base = "../evil/Title" must produce exactly one file inside the
chosen directory.

stts_and_ctts_expand renamed to stts_expands_runs_to_per_sample_deltas_in_order
and given runs with distinct deltas AND distinct lengths. Its old name
claimed ctts coverage it never had, which is why the composition-time
chain went unconstrained for eight rounds; the doc comment now points
at the tests that do cover ctts.

Correction to the previous pass: codec/truehd.rs flush -> vec![] IS
equivalent. Applied it, full mux suite green. TrueHD buffers across PES
but parse emits every complete unit immediately, so a residual buffer
at EOF is a truncated access unit and is correctly discarded. The
vec![Default::default()] variants are genuinely different and are
killed.

Deliberately not constrained: mkv::set_opening_capture (diagnostics
behind a process-global tracing check, flaky under the parallel
runner), and the three stdio.rs header paths (StdioStream holds
concrete io::Stdin/Stdout and cannot be driven without a production
refactor to injectable Read/Write).
2026-07-30 14:13:33 -07:00
Matthew Jackson 170fd0c064 test: constrain MP4 composition timing, MLP substream directory, and codec-private absence
Mutation testing over src/mux/. No production change — 49 survivors
killed, all proven red before green.

The MP4 composition-time chain was entirely unconstrained: VideoTiming::ctts,
build_ctts and parse_ctts could each return a constant and the suite
stayed green. Confirmed on HEAD: build_ctts -> vec![] passes all 1,220
mux tests. A demuxed B-frame title presenting in decode order would
have shipped.

The cause is a test whose name asserts coverage its body does not
deliver — stts_and_ctts_expand builds an stts box and never touches
ctts, and write_then_read_round_trip asserts sample sizes and keyframe
flags but not one PTS. Same shape as the set_speed forwarding finding,
different disguise.

mlp_num_substreams / mlp_substr_header_size: every TrueHD fixture in
the crate uses one substream and no extraword, so both could return a
constant and agree with all of them. These position mlp_parity_ok's
window over the AU header, so a constant mis-windows the parity check
on exactly the multi-substream AUs that carry 7.1 and Atmos.

CodecPrivate absent vs empty: mkv.rs writes Some(bytes) verbatim and
omits the element on None (RFC 9559 5.1.4.1.24), so a zero-length Some
emits a track header asserting the config IS empty. Four parsers could
return Some(vec![]) before any frame.

Also: mandatory ISO/IEC 14496-12 boxes (tkhd, vmhd, smhd, dinf, mdhd)
could each build empty; HEVC num_extra_slice_header_bits (H.265 7.3.2.3)
was never non-zero in any fixture, so the slice-type offset skip was
unexercised; chapter names from the disc go straight into
<ChapterString> and the & escape must run first; a stray 0x47 in a
payload must not latch a TS resync.

Documented as equivalent rather than killed: CodecParser::flush and the
three parser flush bodies that differ from the mutant only by a tracing
call, and DropTally::log_summary.
2026-07-30 13:39:02 -07:00
Matthew Jackson 55b97ac576 fix(udf): skip deleted File Identifier Descriptors
read_directory decoded file-characteristics bit 1 (Directory) and bit 3
(Parent) but never bit 2 (Deleted) — ECMA-167 4/14.4.4. It followed the
ICB of a descriptor naming a file that no longer exists.

4/14.4.3 permits a deleted FID's ICB field to specify an extent of
length zero, so it need not point at a File Entry at all. Following it
reads whatever descriptor occupies that metadata LBA:

- deleted DIRECTORY FID: recursion lands on the File Set Descriptor
  (tag 256), hits the not-a-File-Entry arm and returns Err. One stale
  descriptor in one directory fails enumeration of the entire volume.
- deleted FILE FID: read_file_size returns Ok(0) for a non-File-Entry
  tag, so a deleted name is reported as a real zero-byte file.

Both directions of the same shape at once: a recoverable condition
becoming a hard failure, and a non-existent entry becoming a plausible
success.

Bit 0 (Existence) is still unread. Skipping hidden files could hide
real content, so it is left alone deliberately rather than folded in.

Found by mutation testing: 48 of the 51 surviving mutants in
read_directory are killed by the 17 tests added here, covering the
short_ad decode byte by byte (ECMA-167 4/14.14.1), the extent-type
mask, the AD bounds guard at the exact sector end, the tag-261 File
Entry directory arm that no test reached at all, the multi-sector
read offset, the FID stride including L_IU, and the nesting cap.

Three survivors are genuine equivalents and are documented as such:
l_fi > 0 vs >= 0 (the emptiness guard below reaches the same state),
and the << 32 / | in the visited-set key (meta_start is constant
across a walk, and the two halves are disjoint).
2026-07-30 13:32:06 -07:00
Matthew Jackson c610285910 test: constrain SectorSource speed forwarding and PassProgress percentages
Mutation testing left both unconstrained.

sector/mod.rs — set_speed on the Box<dyn> and &mut dyn forwarding impls
could be replaced with an empty body and nothing failed. This one hides
better than the read methods because the trait's own default body is
already a no-op, so a forwarder that swallowed the call is
indistinguishable from a source with no speed control. Consequence is a
silently absent value, not a wrong one: the recovery path lowers read
speed through a damaged region, and a swallowed call leaves the drive
at full speed while the caller believes it slowed down. Routed through
a generic S: SectorSource bound, since a direct call on a &mut dyn
receiver auto-derefs to the vtable and never enters the forwarding body.

progress.rs — 42 survivors. All four percentage accessors could return
a constant, read the wrong byte counter, or have their divide-by-zero
guard inverted. Added exact-value tests (25%, not 'some percentage'),
both sides of each guard, the overshoot clamp, and one test setting all
three disc counters to distinct values at once — without it, a swapped
field still passes every single-counter test.

The Progress blanket impl for closures could return a constant true.
That return value is the cancellation signal, so a constant-true body
makes every closure-based consumer uncancellable.

Each mutation applied, observed red, reverted.
2026-07-30 13:26:31 -07:00
Matthew Jackson e4b1e5b19e docs: correct info invocation and read timeouts
TROUBLESHOOTING step 2 said `freemkv info`, which needs a source URL;
the drive route is `freemkv info disc://`.

architecture.md quoted 1.5 s / 30 s for the read timeouts. The
constants are READ_TIMEOUT_MS = 10_000 and READ_RECOVERY_TIMEOUT_MS =
60_000 (src/scsi/mod.rs:72,94).
2026-07-30 13:21:04 -07:00
Matthew Jackson 8d4a6d54a4 Constrain five behaviours that mutation testing showed nothing constrained
Fifteen surviving mutants killed, from the highest-risk class: functions a
mutant could replace wholesale with a constant while all 2,555 tests
passed. None of the code was wrong. In every case a test was absent, which
is why eight rounds of reading never found any of them.

The one that generalises is in sector/mod.rs. Its existing test READS as
covering `read_sectors` on the `&mut dyn SectorSource` forwarding impl —
it takes a `&mut dyn`, calls the method, checks the spy. But the receiver
auto-derefs and dispatches through the vtable straight to the spy, so the
forwarding body is never entered. An earlier round hit this exact trap on
`set_unit_base` and fixed it with a generic helper; the read path kept the
test that looked right. Verified by stubbing the forwarding impl to Ok(0):
the new test fails, the old one passes. That makes a tenth distinct shape
of bad test in this audit, and the mutation list is how to find the rest —
any forwarding-impl method in it has the same problem.

decrypt.rs's two existing gate tests assert only `dropped == 0`, which is
precisely what the `Ok(0)` mutant returns; one asserts nothing else at all.
A wrapper that decrypts nothing therefore looked correct while the caller
muxed scrambled MPEG. Now pinned by descrambling a real CSS sector and
comparing against the plaintext it was built from — not against a
re-derived descramble, which would only assert the code agrees with
itself.

css/mod.rs's `is_scrambled_uncracked` turns out to have no production
callers at all; the enum is matched directly. Its three tests all assert
only the true direction, which is exactly why the `-> true` mutant
survived. It is public API, so a consumer routing on it would, under that
mutant, refuse to rip every clear DVD.

aacs/inf.rs's MKB drive read had no test whatsoever. Now pinned
byte-for-byte across multi-pack concatenation, the single-pack case, a
genuinely empty response, and error propagation — an unreadable MKB must
surface as an error, not as an empty one.

aacs/derive.rs's nine mutants are killed with planted MKBs built by
inverting the AACS relations, so no real key material is involved. The
assertions land on the derived Media Key rather than the intermediate
positions: a recovered position that does not actually walk to the planted
key is no better than None. A fixture-guard test asserts the planted MKB
parses, since an unparseable one would make every `-> None` body look
right.

2570 lib tests, debug and release.
2026-07-30 12:41:28 -07:00
Matthew Jackson 93e1436fc0 Test that the Media Key verifier actually rejects a wrong key
`km_verifies` is the gate deciding whether a candidate Media Key belongs
to the disc. The MK-pool brute force in resolve.rs runs every candidate
through it, so a version that said yes to everything would accept
whichever candidate it tried first and the rip would continue with a wrong
Media Key — wrong VUK, wrong title keys, garbage plaintext, and no error
raised anywhere.

Nothing tested it. Whole-crate mutation testing reported
`replace km_verifies -> bool with true` as SURVIVING: the body could be
replaced with `true` and all 2,556 tests still passed. A verification
routine whose verification was itself unverified, in the most
safety-critical function in the crate.

The implementation is correct — it matches the AACS relation
`AES-D(km, mk_dv)[0..8] == 01 23 45 67 89 AB CD EF`. Only the defence was
missing.

No real key material is required to test it. That relation means a valid
record for any chosen km is just `AES-E(km, <the constant> || anything)`,
so the fixture is self-contained. The test asserts three things: the key
the record was built for verifies; a key differing by ONE BIT does not,
which is the assertion that kills the mutant and is a near-miss rather
than a random key; and an MKB carrying no verify record does not default
to yes, because unverifiable and verified are different answers.

Confirmed by reintroducing the exact reported mutation and watching this
test fail.

This is the first defect found by mutation testing rather than by reading.
Eight audit rounds and a security lens that read this file in full all
missed it, because it is not a wrong line — it is an absent test, and only
an instrument that asks "would anything notice if this were broken?" can
see that.
2026-07-30 12:14:14 -07:00
Matthew Jackson 18f8b285c4 Bound the BD-J label parsers, and stop a crafted disc hanging the scan
Ten defects in code no previous round had ever scoped. `src/labels/`
identifies a disc's studio by parsing jar archives and JVM class files off
untrusted media, so every byte here is attacker-controllable — and 813 of
its lines were executed by no test at all.

The worst is a non-terminating loop. A fallback stream-number scan
advanced with `saturating_add`, and the comment says why: a crafted XML
"must not overflow (panic in debug, wrap-to-0 in release)". Once the
counter pins at u16::MAX and that number is taken, the loop cannot exit.
So a fix for an overflow panic produced an unbounded hang, which is
strictly worse — a panic is observable and catchable, and catch_unwind
cannot interrupt a live loop. Reachable from about 8 MB of XML.

Where the same overflow appears in the deluxe decoder the fix is
checked_add and stop, NOT saturation — twice wrong there, because
saturating would peg every stream past the ceiling at one number and
apply_labels binds on (type, number), silently mislabelling tracks. A
correctness bug wearing the costume of success.

Round 7 capped the ldc-string retention per class; nothing capped the
aggregate, so a 64 MiB jar held that budget for every class at once. Same
defect one level up, which is the shape that keeps recurring in this
directory. Four other amplifications are bounded the same way, each with a
stated headroom and a paired test proving real media passes untouched —
the tightest is 5x on a label length, the loosest 2000x on the stream
numbering space, against BD's 32-per-type STN_table limit.

Two are not caps at all: a quadratic membership scan became a set, and an
attacker-derived length added to a cursor without saturation now cannot
wrap. Nothing is excluded by either.

A `#[cfg(test)]` hand-copy of a shipping parser was the ninth bad test
this audit has found, and the first proven by mutation rather than
inspection: deleting the guard from the REAL function left all 26 tests
green, including the one named for that guard. Pointed at the real
function, the same mutation fails.

Separately, all three failure arms of the bounded fsync returned Ok(()) on
both macOS and Linux, so sync_all reported success for a durability
barrier that never ran. Only macOS was in scope; the Linux twin is fixed
here too, because a platform disagreeing with its sibling about whether a
failed sync is an error is the class that already produced an over-length
SCSI CDB macOS rejected and the other two truncated. Note the behaviour
change: a mux whose final sync times out on a wedged mount now fails
rather than exiting 0.

Three of the caps are proven by wall-clock deadline rather than an
operation count, with 18-80x margin on the passing side. On a heavily
oversubscribed machine those could flake.
2026-07-30 11:30:24 -07:00
Matthew Jackson c63dafcf1a Key each FMTS extent from its own CPS unit, and stop a bad ICB tag reading
as an empty directory

On an AACS 2.1 disc the non-forensic gap fill hardcoded pool slot 0 as
"the" base Unit Key, so every content LBA outside a forensic segment was
keyed with CPS unit 1's key even on a disc carrying several CPS units. It
does not fail loudly — it produces garbage plaintext. The gap fill now
resolves each extent's own base key from its ciphertext, sharing the
sampling and slot-picking the multi-CPS path already had rather than
adding a second copy, and memoised in the existing per-disc cache. A disc
with one base key still short-circuits with zero extra reads, which the
existing probe-cost test pins. Forcing the slot back to a constant fails
four tests, so the choice is load-bearing rather than incidental.

A directory ICB whose descriptor tag is neither File Entry nor Extended
File Entry (ECMA-167 4/14.9, 4/14.17) was turned into a successfully-read
EMPTY directory, indistinguishable from a genuinely empty one, while the
same tag on a file ICB was already a hard error. Fifth instance in this
audit of a failure converted into a plausible success value, and the
second in this very function — round 5 fixed a read error becoming a file
size of zero here.

An unrecorded extent (ECMA-167 4/14.14.1.1: allocated but not recorded,
logically zeros that still occupy file space) was dropped entirely rather
than contributing its length, so every later extent landed at the wrong
file offset. Silent corruption, not an error. Extents now carry a recorded
flag and the hole emits zeros without touching the media.

The Volume Descriptor Sequence was swept at hardcoded sectors 32..64 while
the anchor's own Main VDS Extent pointer was parsed into a comment and
ignored; ECMA-167 3/10.2.1 defines that extent by the field, not by
position, so a conformant volume placing it elsewhere failed to mount.

drive_status decoded byte 5 as Media Status without checking the event
header's NEA bit or notification class (MMC-6 §6.7), so a reply carrying
no media event descriptor decoded as "no disc". The drive is untrusted
input here, and this is the works-on-my-drive class.

Two more tests were found asserting the defects they sit next to — one
requiring unrecorded extents to be dropped, one that four drive fixtures
built non-conformant replies the corrected decoder rightly rejects. Both
rewritten. Combined with the DTS one in the previous commit that makes
three tests this round that locked a bug in as intended behaviour, which
is a different and worse failure than the tautological tests found so far:
a tautology fails to catch a regression, these actively defend the defect.

Not fixed, adjacent: extract_one_file streams extents sequentially and
will now READ an unrecorded extent's sectors rather than writing
guaranteed zeros. Offsets are right, and pressed media reads as zeros
there, but it is not zero-guaranteed the way read_file now is; that needs
a recorded flag through PlannedFile.
2026-07-30 11:15:52 -07:00
Matthew Jackson 46eb88c51f Feed the CSS crack the canonical extent order, and stop Resolution faking 1080p
Seven defects in the code the test suite executes least — 913 lines of
disc/mod.rs alone are run by no test at all, which is why this round scoped
from coverage rather than from what previous rounds said they had read.

Disc::scan_image kept its own copy of the crack's extent ordering and fed
crack_key_outcome largest-cell-first. That is the fifth instance in this
audit of a local reimplementation drifting from the canonical one, and the
cost here is a key that does not descramble the feature: picking by sector
count bypasses the capacity gate and can select a different VTS entirely.
The copy is gone — which title comes from the canonical order the scan
already applied, and the extents are handed over in playback order,
exactly as decrypt_keys_for_title does. Its doc records why the duplicate
existed so it cannot grow back.

Resolution::pixels returned 1920x1080 for Unknown. That is the FOURTH
instance of one trap and the other three were in this same file, two of
them fixed hours earlier — without sweeping for siblings, which is the
whole reason this one survived. It now returns (0, 0), and the sweep was
done properly this time: every remaining Unknown arm across the crate is
honest, and the two ColorSpace sites that look like fabrication are
emitting H.273 code point 2, which is the spec's own "unspecified". Two
callers carried local Unknown-to-zero workarounds — precisely the cost of
making callers responsible for a lie — and one is now redundant.

BD-ROM Part 3 code 0xA2 is the lossy secondary DTS stream, not lossless
Master Audio. A test asserted the wrong mapping as intended behaviour, so
correcting the code failed it; the test is deleted with a note pointing at
its replacement. That is a NEW failure mode for this audit: not a test
that cannot fail, but one that locks the defect in. There is no
DtsExpress variant to map to, so it takes the lossy DTS-HD member and the
approximation is documented.

Also: DiscSession::identify could panic through drive_mut once the public
API allows an absent drive — two siblings were converted in an earlier
round and this one was missed; an extent end that added without saturating
where the rest of the crate saturates; a diag reason string restating the
comparator's sort keys and drifting from them, now derived from them; and
a short read that advanced the offset by the full request, silently
skipping the gap. That last one existed twice, in two reads with the same
shape, now merged so they cannot drift apart.

The short-read policy is a judgement call I could not derive from a spec:
no skip_errors is a hard error, with skip_errors zero-fills and charges
the loss. It deliberately does not retry mid-unit, because resuming inside
an AACS aligned unit would trade a silent gap for a silent decrypt
desync — the worse of the two.
2026-07-30 11:12:55 -07:00
Matthew Jackson dea968f32b Stop AudioChannels and SampleRate fabricating a value for Unknown
Three copies of the same two mappings existed. The canonical accessors
returned 6 channels and 48000 Hz for Unknown; a third copy in diag.rs
returned 0. The honest one was the copy.

A plausible wrong answer is worse than an obvious one. Six channels at
48 kHz is indistinguishable from a real 5.1 track, so every caller became
responsible for remembering to check the variant first — and this crate
walked into exactly that: the json:// sink reported a confident 5.1 for
audio whose neighbouring fields said "unknown". That was fixed at the call
site earlier in this audit; this fixes it at the source.

The accessors now return 0, which is what both in-crate call sites already
coerced Unknown to by hand, so their guards are gone and the behaviour is
unchanged. Zero is also obviously wrong if it ever reaches output, where
six is not.

The diag.rs duplicates are deleted rather than corrected — a fourth copy
would have drifted too. Their only caller was a trace line in the same
file, now on the canonical accessors. Their tests moved across and gained
the Unknown case, which is the point: restoring either fabricated value
fails both.

Found by the round-7 correctness agent while fixing the json:// sink; it
flagged the third copy as out of its scope rather than touching it.
2026-07-30 10:09:01 -07:00
Matthew Jackson 079c9b1327 Add a seeded robustness harness for the untrusted-input parsers
Five parsers that take bytes straight off a disc are now swept with
generated input asserting one property: they return Ok or Err and never
panic. That is this crate's own hard rule, and the class seven rounds of
reading is worst at.

Written in-crate rather than with cargo-fuzz, which needs a nightly
toolchain this project does not use, and without proptest or arbitrary,
because one dev-dependency is a deliberate posture and the parsers take
plain byte slices. What is given up is coverage-guided mutation, which is
the real loss. What is gained is determinism: the same seed replays the
same cases anywhere, so a CI failure reproduces locally verbatim.

Three generators, and the second is the one that matters. Pure random
bytes die at the magic check and exercise the entry guards only; prefixing
valid magic is what reaches the parser body; mutating a mostly-zero record
is what reaches the offset and count arithmetic a hostile image would lie
about.

That claim is MEASURED, not asserted. A harness whose cases all bounce off
the entry guards is the fuzzing equivalent of a test that cannot fail, so
one test counts how many generated cases parse to completion: 15,606 of
60,000, about 26%. If a future change to a guard drops that to zero, the
test fails rather than continuing to report a meaningless pass. Two further
tests pin that the three generators produce different bytes and that a seed
replays identically.

1.2M cases across all five targets found nothing. On this evidence that is
a real negative rather than an empty one.

The first version of this file was itself broken in the way this audit
keeps finding: its two meta-tests set FREEMKV_HARNESS_CASES and raced,
because the test harness runs them in parallel and env mutation is unsound
there. The budget is a parameter now, and the environment is read once at
the call site.

Two crate-internal parsers widened from private to pub(crate) so the
harness can reach them. No public API change.
2026-07-30 09:45:56 -07:00
Matthew Jackson 327087c70e Make five tests capable of failing, and stop the presence probe unmounting the disc
The worst of the five was a regression suite that never touched the code
it guarded: nine batch-count tests called `safe_batch_count` and
`buggy_batch_count`, both defined in the test file itself. The u16
truncation they exist to prevent could be reintroduced in
sector/prefetched.rs with every one of them green. They now drive the real
producer through the public API, and reinstating the truncation fails five
of the nine. Worth recording that the symptom has changed since the
original fix: the unit-alignment clamp below floors a zero batch at three
sectors, so the bug is now a twenty-fold throughput cliff rather than the
stall it once was.

The MP4 reserve test's only numeric case was dominated by the floor and
the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now
has a case where the per-sample term dominates. The zero-count guard in
FileSectorSource was likewise unfalsifiable — seek-past-EOF and a
zero-length read both succeed — so the test now observes the file cursor.
The AACS media-key ambiguity guard had no test at all; the pool scan is
extracted so the verifier can be injected, because a genuine two-key
collision needs one ciphertext decrypting under two AES-128 keys to
plaintexts sharing a 64-bit magic, which is a 2^64 search and not a
fixture.

macOS implemented the documented cheap, side-effect-free presence probe by
building a full exclusive transport — which force-unmounts the disc. Linux
and Windows issue one TEST UNIT READY with no unmount; macOS was the
outlier. It now walks the IOKit registry for the media object instead.

The C shim's registry reads assumed CoreFoundation types the registry does
not guarantee, so a driver publishing a CFNumber where a CFString was
expected aborted the process from inside public API. Types are checked and
a wrong type treated as absent. The unbounded waitpid on the unmount child
is now a polled deadline, and the last-resort match gained the NULL check
its two siblings already had.

The empty-CDB guard existed only on Linux while a shared helper's comment
claimed all three backends had it. Moved into the helper, so the comment
is now true and macOS and Windows are covered.

One finding was REJECTED with evidence rather than fixed. The TrueHD
buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be
unreachable by any input: the parser only retains data when the buffer is
shorter than the declared AU, and that declaration is twelve bits, so the
worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over
all 65536 AU headers confirmed it. The fixture now sits at the reachable
ceiling and asserts that instead. The cap itself is left in place as
defence, unreachable by construction, matching how the AC-3 resync guard
was handled earlier in this audit.

Two behaviour changes worth naming: Linux's empty-CDB error becomes
InvalidCdbLength rather than a transport failure, and an unknown device
now reports absent media rather than a not-found error, because the
registry cannot tell an empty drive from a missing one. The latter is a
conflation of the kind this audit has fixed three times; it is recorded
for the next round rather than left silent.
2026-07-30 09:26:56 -07:00
Matthew Jackson b8fa5e74dc Stop the live rip path muxing Blu-ray 3D differently from the ISO path
Five defects, four of them the same shape: a local reimplementation of
logic the crate already had, which had drifted from it. Each is now fixed
by calling the canonical version rather than by patching the copy.

DiscStream::new — the live disc:// path — built every parser through the
plain codec lookup and never asked whether a video stream was an MVC
dependent view, though resolve::build_demux_state does. The same 3D disc
therefore muxed correctly from an ISO and incorrectly ripped live. The
open-coded loop is gone; both paths now call build_demux_state.

collect_psi_section reimplemented the continuity-counter gap test and
disagreed with process_packet in the same file: it tolerated neither a
duplicate packet nor an adaptation-field-only packet, which per ISO/IEC
13818-1 §2.4.3.3 does not increment the counter. A spec-legal PMT
continuation was read as desync and the title's stream list came back
empty. Both callers now share one `cc_is_gap`, and a duplicate packet's
payload is no longer appended twice — doing so would have corrupted the
section the check exists to protect.

The json:// sink called the channel-count and sample-rate accessors
unconditionally, and both fabricate a concrete value for Unknown, so it
reported a confident 5.1 at 48 kHz for audio whose format was unknown
while its own neighbouring string fields said "unknown". The keys are now
omitted, matching mkv.rs. This matters more than it did: a sample-rate
ladder fixed earlier in this audit means Unknown now reaches consumers
that used to receive a wrong-but-concrete value.

For an audio:// or sub:// sink the reference video track's output is
filtered out, so its first PTS was never recorded and every delay was
computed against zero — baking a wrong DELAY into the filename. The
reference is now recorded whenever a frame is on the reference track,
independent of whether that track has an output, so a normal title gets a
correct delay; where no reference is ever observed the tag is omitted
rather than guessed.

A third copy of the channel/sample-rate mapping exists in src/diag.rs and
was left alone as outside the confirmed set. It is the same drift shape
and is recorded for the next round.
2026-07-30 09:18:33 -07:00
Matthew Jackson 3f7d7af472 Bound three allocations an untrusted disc can drive without limit
The Program Stream demuxer appended every fed byte and enforced its 4 MiB
cap only inside a branch reached once a start code had been found. Input
containing no start code anywhere therefore hit no cap at all, and since a
whole title is fed through this demuxer, a zero-filled or ciphertext VOB
extent buffered the entire title — up to ~90 GB. When the buffer holds no
start code, only a two-byte `00 00` prefix can begin a PS unit on the next
feed, so that is kept and the rest dropped. The bound is exact rather than
a heuristic: a start code can straddle a feed boundary by at most its
first two bytes, so no real byte is discarded, and a test feeding
`FF FF 00 00` then `01 E0 ...` pins that.

The existing test named for this case fed a real start code first, so the
cap it exercised was the in-PES one. Renamed to say what it covers.

The BD-J label path had a different shape to anything found so far: the
cap is on the COMPRESSED size of a disc file while the allocation scales
with the decompressed size. A `.class` gated only by a path prefix
inflates to the 64 MiB ceiling, yielding ~33M retained strings from `ldc`
operands or ~67M pushes onto a symbolic stack whose depth was unbounded
despite the Code attribute's own `max_stack` being parsed and then
ignored. Bounded both, the stack by `max_stack` itself (JVMS §4.7.3).

The VMG TT_SRPT title count is an untrusted u16 with no de-duplication,
so ~800 KB of crafted IFO re-parsed one PGC 65535 times. Capped at 99, the
DVD-Video maximum, so no conformant disc is clipped.

Every cap carries stated headroom against real media, and each has a test
locking that real media still passes.

I rewrote three of the new assertions before landing them. They compared
the result against the very constant under test — `total <= MAX_TT_SRPT_TITLES`
— which passes vacuously the moment someone raises the constant, the most
likely future regression and the seventh instance of this tautology shape
in this audit. They now assert literals derived from the spec.

The TT_SRPT fixture also had to change: with 65535 identical entries the
de-duplication collapsed them on its own and the cap was never what
bounded the result, so the test passed with the cap removed entirely.
Distinct entries defeat dedup and leave the cap as the only guard;
de-duplication now has its own fixture. Verified by raising each of the
three constants and confirming all three tests fail.
2026-07-30 09:17:16 -07:00
Matthew Jackson fdd473d7e9 Remove emulation-prevention bytes before reading the H.264 slice header
The bytes after a NAL header are EBSP, not RBSP: ISO/IEC 14496-10 §7.4.1
has the encoder insert 0x03 after any 0x00 0x00, and §7.3.1 removes it
before parsing. The measured-picture-type parse read the raw NAL instead,
on the stated reasoning that slice_type is too early for an escape to
intervene.

That holds only up to a point. first_mb_in_slice is ue(v), so a value of
65535 or more needs sixteen leading zero bits and opens the payload with
0x00 0x00, which an encoder must then escape. A UHD frame is ~32,400
macroblocks, so a conforming Blu-ray never reaches it — but 8K does, and
the disc is untrusted input. Such a stream decoded slice_type against a
byte the encoder had inserted and reported the wrong picture type: a wrong
result rather than an error, which is the class this lens exists for.

The prefix is un-escaped into a 16-octet buffer rather than the whole NAL:
the two ue(v) fields are at most 32 bits each, so nothing longer can be
needed, and it keeps a per-frame allocation proportional to the frame off
the path.

The test pins both directions. It asserts the un-escaped prefix decodes to
first_mb_in_slice = 65535 and slice_type = 2, AND that the raw EBSP does
NOT — without that second assertion the test would pass whether or not the
fix were present, which is the failure mode this audit has now found four
times. The bit string was derived independently rather than by hand: my
first attempt at the fixture was wrong by one nibble and the test caught
it.

Also covers the cases that must NOT be unescaped: a 0x03 not preceded by
00 00 is ordinary payload, and 00 00 03 03 keeps its second 0x03 because
the escape resets the zero run.
2026-07-30 08:49:51 -07:00
Matthew Jackson 05fed1b0e0 Log the OS error when a SCSI command fails on Windows and macOS
Both backends discarded the platform's own error code on the execute hot
path — the one every READ(10) of a rip goes through — and collapsed every
cause to the same status-0xFF transport failure.

On Windows, open() and reset() in the same file both capture the Win32
error; execute() did not. That left ERROR_INVALID_PARAMETER (a struct
layout regression, the exact class this file's SDK-layout tests exist to
catch), ERROR_ACCESS_DENIED and ERROR_GEN_FAILURE (a genuinely wedged
drive) indistinguishable, with nothing in the log to tell a code bug from
a hardware one.

On macOS the same, and worse: the file had no tracing calls at all, where
the Linux and Windows backends both log their execute failures. Its open()
carefully decodes the shim's sentinel into typed variants instead of
flattening them, but execute() threw the IOKit return away — so another
process taking exclusive access mid-rip and a real hardware wedge produced
identical, empty diagnostics.

Logged rather than added to the error type: the typed variant is public
API, and the recovery classification is deliberately the same for all of
these. What was missing is the breadcrumb, not the distinction.

Two further findings from the same sweep were rejected. Windows reset()
always returning Ok(()) and macOS ignoring timeout_ms are both already
documented in the code as deliberate, and the reporter flagged them for
completeness rather than as defects.

Neither fix has a test: reaching either branch needs a failing ioctl or a
failing IOKit call, and both files are compiled only on their own platform.
2026-07-30 08:42:02 -07:00
Matthew Jackson d444afbdfc Turn a release-only slice panic into an error, and stop calling 32 kHz 48 kHz
Four round-6 findings.

FileSectorSource::read_sectors guarded its output buffer with a
debug_assert, which is compiled out in release — so an undersized buffer
panicked with 'range end index out of range' instead of returning an
error, out of a public SectorSource impl where the length is caller input.
Drive::read_fua already carries this exact guard, with a comment recording
the same panic being fixed there, and PrefetchedSectorSource has a
regression test for the same case; this impl had been given neither. The
new test is red in release for precisely the predicted reason: 'range end
index 8192 out of range for slice of length 2049'.

parse_track's sample-rate ladder ended in an unconditional S48, so any
SamplingFrequency below 44100 was recorded as 48 kHz. A 32000 Hz AC-3 or
DTS track is legal and common in broadcast-sourced content, and the wrong
rate then propagated into the reconstructed AudioStream. Anything below
the lowest mapped rate is now Unknown, which is what the crate's canonical
SampleRate::from_hz already returned — the ladder disagreed with it. The
ladder itself stays, because the MKV element is a float and wants
tolerance rather than exact equality.

shim_open_exclusive used the mach port from IOMainPort without checking
the return; on failure the port is left untouched and every IOKit call
below ran against an uninitialised value. shim_list_drives in the same
file does check it.

build.rs treated cc and ar as successful if the process merely SPAWNED, so
a genuine compile error in the macOS C shim produced no object file and
surfaced later as an unexplained link failure against a missing symbol.
The shim is macOS-only and is neither linted nor compiled on the other two
platforms, so a mistake in it has exactly one chance to be noticed.

The last two have no test: one needs IOMainPort to fail, the other needs a
deliberately broken C shim, and neither is reachable from the test
harness. Both mirror a correct sibling in the same file, which is the
evidence available.
2026-07-29 22:56:33 -07:00
Matthew Jackson 921404d135 Fix five clippy errors that only appear on the target CI lints
CI's clippy job runs on ubuntu-latest, so cfg(target_os = "linux") code is
what the gate actually compiles — and none of it is built by clippy on a
Mac. Five `-D warnings` errors were sitting in drive/linux.rs,
scsi/linux.rs, io/writeback/linux.rs and the Linux arm of drive/mod.rs:
four collapsible let-chains and one manual `% n == 0`. CI was red on the
lint job while the local gate reported all green.

Collapsed into let-chains, which the declared toolchain supports, and
folded drive/mod.rs's length precondition into its chain so the body no
longer needs a nested block.

Found because an agent working on the SCSI backends reported the lints in
passing while checking that a Windows-only file compiled. Worth noting how
it stayed hidden: every one of these files is cfg-gated to a platform this
machine is not, so no amount of local gating would have surfaced them. The
companion change to the precommit script closes that hole.
2026-07-29 22:37:47 -07:00
Matthew Jackson 399c3d2769 Reject an over-length CDB on every transport, splice H.264 param sets in place
Two fixes from round 5.

The Linux and Windows backends truncated a CDB longer than 16 bytes
(`cdb.len().min(16)`) where macOS returned InvalidCdbLength. Under SPC-4 a
command's length is fixed by its opcode group code, so a shortened CDB is
not a shorter form of the same command — it is a DIFFERENT command, and
the drive executes it and answers GOOD with data for a request nobody
made. A silently wrong result on the layer everything else sits on.

Rather than mirror the guard a third time it now lives in scsi::mod as
checked_cdb_len, with all three backends routed through it, so it cannot
drift per platform again. That also makes it testable everywhere: each
platform module is cfg-gated to its own host, so a guard inlined into
linux.rs and windows.rs would have had no test coverage on any single
machine. The shared helper is the only place the behaviour can be
asserted on every platform's CI.

The two existing macOS tests were tautological — they replicated the
guard's logic inline instead of calling it, so they would have passed with
the guard deleted. They now call the real helper.

Separately, the H.264 keyframe parameter-set re-assert grew a
few-hundred-byte prefix buffer to the full access-unit size, copied the
whole frame into it, and dropped the presized buffer: one extra
whole-frame allocation and copy per keyframe. A UHD title is ~200,000
frames of 150-400 KB with a keyframe every second or two, so that is
thousands of avoidable multi-hundred-KB copies per title, each large
enough to go through mmap. It now splices into the reserved headroom in
place. This mirrors the identical fix already made in hevc.rs, which the
H.264 path had drifted from.

Byte-for-byte equivalence is pinned by a test whose expected literals
were captured from the pre-change implementation, and which I confirmed
still passes when the old build-and-copy code is restored. The
no-reallocation claim is measured rather than argued: a counter over 30
bare keyframes, which reports 30 of 30 against the old path and 0 with
the splice.

The reallocation test initially passed even with PARAM_REASSERT_HEADROOM
set to zero, because a small parameter set fits in the presize's
incidental slack — it proved the fixture did not reallocate, not that the
headroom prevented it. Its SPS is now large enough that the constant is
load-bearing, so zeroing it fails the test.

Not verified: no runtime behaviour on Linux or Windows: no drive, no
ioctl. Both files were confirmed to compile for their own targets.
2026-07-29 22:34:22 -07:00
Matthew Jackson dc5b67ed46 Stop reporting an uncrackable CSS disc as N empty titles
Same shape as the mkv:// conflation fixed earlier in this round, found by
looking for it deliberately. E7023 carried two conditions with opposite
correct responses: one title on a multi-VTS DVD failing its own re-crack,
where skipping it and finishing the rest is right, and the main feature's
crack failing outright, which is disc-wide and dooms every title
identically. Because both raised the same code and that code is in
is_skippable_title_stub, an uncrackable disc walked all N titles printing
"title skipped, it was empty" and exited 0.

The disc-wide condition gets E7027 CssNoDiscKey, mirroring the AACS-side
E7022 NoDiscKey it is the analogue of, and joins is_disc_level_no_key.
The per-title raise keeps E7023 and stays skippable. Because the engine's
classifier already tests is_disc_level_no_key before the skippable
branch, this reaches the right outcome downstream with no change there:
such a disc now stops on the first title and reports no-key instead of
returning success with nothing written.

Disc::css_error deliberately still stores CssKeyMissing — autorip matches
that variant on the field to pick the CSS rather than AACS message, and
what consumers classify on is the gate's returned verdict, which is the
only thing that changed.

Two neighbouring CSS raises were examined and deliberately left alone:
the no-key branch in the same function is genuinely unreachable via
ensure_decryptable and documented as defensive, and resolve_dvd_title_key
is per-title on both of its call paths.

Verified by removing the new code from is_disc_level_no_key, which fails
both new tests; each pins both directions so neither can silently flip.
Not proven end to end against a real uncrackable disc — none available.
2026-07-29 22:29:51 -07:00
Matthew Jackson 5c6a6d0785 Round 5: reject a degenerate fixed lace, bound the pending buffer by bytes
Five fixes. Three are real defects with regression tests; two are bounds
that were expressible but not expressed.

A fixed-size lace (RFC 9559 §10.3.4) whose body is empty declared n
frames and carried none. The divisibility check passed, because 0 % n is
0, and `chunks` yields nothing on an empty slice whatever width it is
given — so the clamp that existed to avoid chunks(0) returned zero frames
where the Lacing Head said n. The whole lace vanished with no error
raised and the caller saw a clean short block. A zero-size frame cannot
be a valid frame, so it is now malformed.

A disc read failure while fetching a directory entry's ICB became a file
size of zero rather than an error. Zero is indistinguishable from a
genuinely empty file, so an unreadable ICB on a damaged disc silently
changed which titles a caller saw as present — read_directory already
fails hard on its entry-budget guard, so propagating is also what the
surrounding code does. read_file_size still returns Ok(0) for an ICB
whose tag is neither File Entry nor Extended File Entry, which is a real
zero and not a failure.

The pending-frame buffer was capped at 4096 frames, which does not bound
memory: frames are arbitrarily large and a UHD video frame runs to a few
hundred KB, so the existing cap permitted over a gigabyte. Now bounded by
bytes as well, at 64 MiB.

round_up_grain overflowed for inputs within one grain of u64::MAX —
div_ceil then multiply — and the wrapped product is small, turning the
largest possible estimate into a negligible reserve. It saturates, and
the reserve is clamped to what a `free` box's 32-bit size field can
actually hold, since writing a larger one truncated the size and left
mdat beyond a box claiming to be far shorter. No real title comes close;
a 90 GB UHD title estimates a few MiB.

The AC-3 resync guard now advances the PTS cadence like both of its
sibling branches, so the three paths out of that block cannot disagree.
This one is defensive and has NO test: reaching it needs input that both
parses frames and leaves a megabyte of residue, and the parser's own
carry rules drop pre-sync junk and cap a partial frame at 8192 bytes, so
no such input was found. Stated here rather than covered by a test that
would pass either way.

Two findings from this round were rejected on inspection. A reported
panic in the .mpls suffix check does not exist: the `.get(..)` on the
line above returns None off a char boundary and `filter` never runs its
closure, so the byte index is unreachable. A test written for it passed
against the unfixed code, which is what surfaced the error.
2026-07-29 22:28:43 -07:00
Matthew Jackson f5e169efb3 Stop reporting a corrupt mkv:// source as an empty title
E6008 meant two unrelated things: "this title produced no muxable
frames", which is a benign stub worth skipping, and "the source file is
malformed", which is not. Because a single code carried both,
is_skippable_title_stub answered yes to the second one — so feeding a
truncated or corrupt mkv:// input made the engine classify it
SkippableStub, print a notice saying the title was empty, and exit 0.
Silent data loss reported as success.

Split into E9053 MkvSourceInvalid for the read path (25 raise sites
across mkvstream.rs and ebml.rs's read primitives) and E9054
MkvUnencodable for the four write-side sites, which are the encoder
refusing to emit a body at or above the 56-bit VINT limit — an output
limit with no input involved, so calling it a corrupt source would be
wrong in the other direction. E6008 keeps only the zero-frame guard it
was documented to mean.

Kept one code for the whole read path rather than one per raise site:
nothing a consumer does differs between a bad VINT, a non-UTF-8 string
element, a truncated body and a child overrunning its parent. E9052 is
the model for when a carve-out earns its keep — laced blocks name one
specific RFC 9559 §10.3 feature with its own diagnosis.

Also fixed meta_sink.rs raising MkvInvalid for a serde_json encode
failure in the json:// sink, where no MKV is involved at all; it now
matches the identical guard in mux/meta.rs.

Reverting the split at the single code() arm reproduces the old
classification: 22 tests fail, including both new assertions. The
opposite direction is pinned too — dropping E6008 from the predicate
fails the genuine-stub test, which drives the real muxer end to end.
2026-07-29 22:11:53 -07:00
Matthew Jackson 0bbceed985 Round 4: fix 26 defects across crypto, resource use and codec paths
Twenty-six confirmed findings from the fourth audit round, landed as one
cluster because they were found by agents working over disjoint file sets.

The one worth calling out is a pair of AACS tests that could not fail.
Both asserted CBC behaviour against a hand-rolled expectation that
happened to be IV-independent, so replacing AACS_IV with sixteen zero
bytes left them passing — they were pinning the code's own arithmetic,
not the published constant. Replaced with a literal witness of the
published IV plus the NIST SP 800-38A F.2.2 CBC-AES128 vector, and
verified the other way round: zeroing AACS_IV now fails three tests.

The rest are allocation and correctness work on hot paths: the Annex-B
writer in demux_sink allocated and freed a whole-frame Vec per frame,
which for a UHD title is ~200,000 allocations over the mmap threshold
plus the page faults to first-touch each one; it now reuses a buffer on
the writer, and still takes the NAL prefix width from the configuration
record rather than assuming four.

Six findings whose real fix lives in a consumer crate are recorded for
re-filing rather than patched here.
2026-07-29 22:09:52 -07:00
Matthew Jackson 4fcd28b487 Parse MKV lacing, route by real TrackNumber, honour NAL length size and edit lists
Four conformance defects in the read paths, two of them silent corruption.

**Lacing was ignored entirely.** RFC 9559 §10.2 defines Xiph, EBML and
fixed-size lacing, where one Block carries several frames; the reader took the
Block payload verbatim, so a laced Block became a single "frame" consisting of a
lacing header followed by concatenated frames — garbage to the codec parser, no
error. Audio tracks from other muxers commonly use lacing, so an ordinary
foreign MKV was silently mangled.

All three modes are now parsed: Xiph 255-run sizes including the trailing-zero
rule for exact multiples of 255, EBML unsigned first size plus SIGNED VINT deltas
with the 2^((7*n)-1)-1 bias of §10.3.3, and fixed-size even division, with the
last frame's size deduced from the remainder. Laced timestamps follow §10.3.5:
the first frame takes the Block timestamp and the rest are spaced by the track's
DefaultDuration, else BlockDuration/count, else shared with a warn.

Parsing was chosen over refusing because refusal would leave freemkv unable to
remux common foreign audio at all, and each mode is about fifteen lines.

A malformed lacing header now raises a NEW code, E_MKV_LACING_INVALID = 9052,
deliberately NOT MkvInvalid — because is_skippable_title_stub classifies
MkvInvalid as a skippable nav stub, so reusing it would have recreated the exact
conflation that is still open as a separate finding. A test asserts the new code
is not skippable.

**TrackNumber was assumed to be 1..N in TrackEntry order.** RFC 9559 §5.1.4.1.1
only requires it to be non-zero and unique, so sparse or unordered numbers are
legal. Block routing and codec_private both computed track + 1. A real
TrackNumber map is now built, recorded only for TrackEntries that yield a stream
so dropped track types no longer shift the mapping.

Verified red here independently, and the failure mode is worse than mis-routing:
with track + 1 restored, a buttons track's payload was attributed to the AUDIO
stream — wrong payload into the wrong codec parser.

**The NAL length prefix was hardcoded to 4 bytes.** lengthSizeMinusOne lives in
avcC byte 4 and hvcC byte 21 (ISO/IEC 14496-15 §5.3.3.1.2, §8.3.3.1.2) and was
never read, so a source declaring 1- or 2-byte prefixes had its raw prefixed
bytes emitted verbatim with no start codes. All four conversion sites now derive
the width from the track's own configuration record.

**Edit lists were ignored.** No edts/elst was parsed, so the presentation
timeline an edit list defines (ISO/IEC 14496-12 §8.6.5/§8.6.6) was dropped —
which is how encoder delay is normally expressed. Leading empty edits and the
first media edit's media_time are now applied to both dts and pts, with the movie
vs media timescale distinction respected. A list needing more than a constant
shift applies the leading edit and warns rather than presenting the result as
faithful.

17 tests. I reproduced the lacing mutant independently: returning the body whole
kills five of them, including the exact-payload and malformed-header cases.

Still open and deliberately untouched: the MkvInvalid / is_skippable_title_stub
conflation across ~20 reader raise sites. It is a cross-cutting error.rs change
and E_MKV_LACING_INVALID is the template for it.
2026-07-29 21:53:19 -07:00
Matthew Jackson 9527bc1e13 Anchor forensic segments to the forensic clip, and stop CPS branching on resolve order
Two correctness defects in AACS 2.1 / FMTS key-map resolution, both order- or
anchor-dependent, and both able to abort a whole disc or silently garble it.

**The single-CPS short-circuit depended on which title resolved first.**
`pool_len` counted the WHOLE unit-key pool, and resolve_fmts_key_map appends the
disc's forensic index keys to that same caller-owned pool. The count was captured
before THIS call's FMTS branch but not before earlier titles', so once any forensic
title resolved, every later title saw a pool larger than one and fell into
multi-CPS sampling — 8 random reads per extent, and a whole-disc DecryptFailed if
no pooled key opened a menu extent's samples. A disc that ripped fine when a
non-forensic playlist sorted first failed when a forensic one did.

Forensic keys are now tagged FMTS_POOL_TAG_BASE = 1 << 24 and the short-circuit
asks single_base_key_slot(), which excludes them. The old 1000 tag was NOT kept,
and the reasoning is worth recording: base CPS ids are Unit_Key_RO.inf position + 1
and that count is a BE16, so 1000 sits inside a genuinely reachable id space. 1<<24
cannot collide. The id field is cosmetic — decrypt.rs indexes the pool by slot and
reads only the key — so widening the tag is safe.

**Forensic segment SPNs were anchored to the wrong clip.** They live in the
forensic feature clip's byte space, but were mapped through
clip_byte_to_lba(&title.extents, ..), which treats byte 0 as the start of the
title's FIRST extent. Any playlist not beginning with the forensic clip mapped
every segment to the wrong LBA: either the anchor probe sampled the wrong clip and
the whole-disc resolve aborted with FmtsKeyMissing, or — worse — a forensic index
key was applied to non-forensic sectors while the real forensic units kept the base
key, giving silently garbled output with no error at all.

The correct anchor turns out to be a DISC fact, not title data: an AACS 2.1 disc
names its forensic feature BDMV/STREAM/<clip>.fmts, and carries one
IndividualSegment.tbl, so the SPNs are in that one clip's byte space. A new
forensic_clip_extents() finds the unique .fmts in the already-walked UDF tree, and
those extents now drive the segment arithmetic, the addressability filter, and the
index probe — whose title parameter is gone, since its reads were mis-anchored too.
"Does this title carry forensic content" is now "does it read the forensic clip's
sectors" rather than "do the segment bytes land somewhere in the concatenation".

Where the clip is NOT identifiable — no .fmts, or several, making the SPN space
ambiguous — on a disc that does carry a non-empty table, the resolve now fails loud
with FmtsKeyMissing rather than guessing an anchor. That is a deliberate behaviour
change: a hypothetical disc with two .fmts clips hard-fails where it previously
produced a possibly-wrong map. Failing loud beats silently garbled output, and
inventing an anchor was not acceptable.

This site had been flagged independently three times — by the agent that added the
FMTS per-disc memo, by the round-4 correctness lens with a concrete scenario, and
by the round-4 conformance pass.

Verified red here independently: reverting single_base_key_slot to count the whole
pool fails both new tests. The agent's own evidence was probe_reads 48 vs 40 (the
8 extra sampling reads) and an E7013 DecryptFailed whole-disc abort, and for the
anchor an E7026 FmtsKeyMissing on a [trailer, forensic] extent list.

All six pre-existing FMTS tests pass unchanged through the new anchor, including
the exact-cost assertions (40 probe reads, one key-service call per disc, one UDF
walk for 60 titles), so the round-3 memoisation wins are intact.
2026-07-29 21:37:50 -07:00
Matthew Jackson 58bdb42f8e Group whole E-AC-3 frame sets, and keep short reads unit-aligned
Two defects in fixes landed the same day, both found by round 4 auditing round 3's
work rather than trusting it.

**The E-AC-3 grouping ignored substreamid, so the timeline still doubled.**
Confirmed independently by the correctness and conformance lenses and verified by
hand: substreamid appeared only in test helpers, never in the production path. Per
ETSI TS 102 366 (A/52) Annex E a frame set is independent substream 0 — mandatory,
always first — with its dependents, then the OPTIONAL additional independent
substreams 1..7 with theirs, all covering the same time period. Treating an
additional independent substream as a new access unit advanced the clock a second
time for the same 32 ms, which is exactly the doubling the grouping fix existed to
prevent. No fixture caught it because every fixture used substreamid 0.

is_dependent_substream becomes substream_role -> Starts | Extends: strmtyp 1
extends; strmtyp 0/2 with substreamid != 0 now extends (this was the bug); strmtyp
0/2 with substreamid == 0, legacy AC-3, and reserved strmtyp 3 start. Reserved 3
starts regardless of its id bits, because its BSI layout is undefined so those bits
cannot be trusted — an unknown frame is neither merged into an unrelated programme
nor silently discarded.

The frame set stays ONE sample rather than being split into a separate track for
the associated service, and the reasoning is in the module doc: a substream
numbered 1..7 with no substream 0 is not conforming, so extracting one would mean
renumbering ids and rebuilding frame sets — a transcode, not a remux. Programme
selection is the player's job.

A stream joined mid-frame-set (first sync is substreamid 3) is skipped with a debug
and resyncs at the next id-0, mirroring the orphan-dependent rule: its mandatory
id-0 substream was never seen, so it is neither decodable alone nor timeable.

MAX_AC3_BUF 128 KiB -> 1 MiB, because an AU is now a whole frame set: worst case 8
independent x 9 substreams x 8192 B = 576 KiB, which the old cap could have dropped
mid-hold.

**The forced probe's two round-3 fixes cancelled each other.** CHUNK_SECTORS = 1023
exists (with a const assert) so every read starts on a 3-sector AACS aligned-unit
boundary; the short-read fix advanced by actual bytes, making the advance a
non-multiple of 3. Every later read was then misaligned, DecryptingSectorSource
refused it before reading, the stop became ReadFailed, and no verdict was asserted
— so content-based forced detection silently fell back to the vendor label on
exactly the encrypted discs the 1023 change was written for.

A partially-satisfied read now advances only by whole aligned units and re-reads
the <=2 residue sectors from the next boundary, feeding only the aligned prefix so
nothing is double-fed and no partial unit reaches the parsers. A read that fully
satisfies its request still advances by all of it. When less than one aligned unit
comes back the bytes are fed and the same LBA is retried twice before stopping, so
a starved source cannot spin — verified by raising the retry limit and watching the
test hang.

Verified red independently here: reverting substream_role to strmtyp-only fails
eac3_additional_independent_substream_stays_in_the_frame_set (6 access units where
3 are correct — the doubling, literally) and the mid-frame-set resync test.

Reported, not fixed: dec3_box still hardcodes num_ind_sub - 1 = 0 and
num_dep_sub = 0, so it under-declares any stream carrying additional independent
or dependent substreams now that frame sets arrive whole. DolbyConfig has no
fields for either; a real fix needs the parser to surface observed substream
counts. That file is another lens's this round.

Unverified: no real multi-programme DD+ stream exists here, so defect 1 rests on
synthetic Annex-E fixtures. The retail DD+ check (No Time to Die, all substreamid
0) confirms single-programme discs are unaffected.
2026-07-29 21:33:18 -07:00
Matthew Jackson 62450e19bd Make two public-API panics return errors, and drop two shipped citations
**The session panics are reachable, and my earlier triage of them was wrong.**
`DiscSession::scan` and `resolve_keys` both did
`self.drive.as_mut().expect(..)`. I previously downgraded these to LOW on the
grounds that no shipped consumer calls them after the drive has been staged into
the reader slot. That is the wrong test: `stage_drive_as_reader` is a PUBLIC
method that empties the drive slot, so the public surface permits the sequence,
and a library must not panic from public API regardless of what current callers
happen to do. Both now return Error::DeviceNotReady.

**A shipped doc comment cited a third-party source FILE** as the authority for
the CLPI ProgramInfo layout ("Layout per the BD CLPI spec clpi_parse.c"). Now
cites the Blu-ray Disc Read-Only Format Part 3 CLIPINF specification.

**The CHANGELOG justified a muxer decision by naming a commercial competitor**
("MakeMKV's rip of the same disc omits it", "matching MakeMKV"). Reworded to
stand on its own terms: the element is optional in RFC 9559, nothing requires it
for interlaced SD, and the 40 ms DefaultDuration is the frame rate the source
actually carries.

The leak gate is extended for both new classes — third-party `*_parse.c` /
`*_dec.c` / `*_demux.c` style filenames, and a competitor named as authority.

Narrowing that rule took two attempts, which is worth recording. A bare
`\.(c|cpp|cc)` pattern produced eight false positives: this repo has its own C
shim (`macos_shim.c`) that build.rs and the docs legitimately reference, and the
pattern also matched the Rust field access `p.cc`. It now matches only the
suffixes typical of third-party media-library sources. This is the second
false-positive round on this rule — the first flagged ffmpeg INVOCATIONS in the
test harness — so the lesson is that a hygiene pattern needs testing in both
directions before it lands, exactly like any other code.
2026-07-29 21:21:29 -07:00
Matthew Jackson 013881ac06 Restore the MP4 conformance fixes I clobbered while landing another agent's work
detect_rate's nearest-match fix, the colr HLG/BT.470 fix and their four tests
were silently reverted. Cause: the r3fix-silent worktree was cut BEFORE the
conformance commit landed, and I landed its work by copying whole files into the
main tree. mp4/mod.rs was in both agents' file sets, so silent's copy — built on
the older base — overwrote the conformance changes wholesale. The gate stayed
green throughout, because reverting a fix and its tests together is perfectly
consistent.

Re-applied the conformance commit's diff for that file with a three-way merge;
both agents' changes to mp4/mod.rs now coexist (final_report, UndescribableAudio
and the max(track_id) id fix are all still present alongside RATE_TOLERANCE_FPS
and the colr resolver delegation).

Only mp4/mod.rs was affected. mp4/audio.rs and mkv.rs were in no other agent's
file set and were intact.

Process lesson, recorded because I would otherwise repeat it: NEVER land a
parallel agent's work by copying whole files, when its worktree was cut at an
older base than HEAD. Apply its DIFF (git apply -3), or rebase its worktree
first. Copying files silently discards anything committed to those files in the
interim, and no test can catch it because the tests disappear with the code.
2026-07-29 21:03:26 -07:00
Matthew Jackson 5f8dc392c0 Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.

The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.

Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.

Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.

The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
2026-07-29 21:00:55 -07:00
Matthew Jackson a32373ff40 Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files
another agent held this round, 0 rejected.

**A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and
1024 % 3 == 1 — verified — so every chunk after the first was misaligned against
the 6144-byte AACS aligned unit and would be REJECTED by
DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle
probe I added last round would have read almost nothing past its first chunk.
Now 1023 sectors (341 aligned units) with a const assertion that fails the build
if it stops dividing, plus set_unit_base per extent so the source's gate is
anchored where the extent actually starts.

**The same probe skipped sectors on a short read**, advancing by the REQUESTED
count rather than the bytes actually returned, so a partial read silently left a
gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n
to the buffer.

**Its cache key omitted the PGS PID set**, so a playlist declaring an extra
subtitle PID got another playlist's verdict for a track that had never been
probed. And the key was the whole extent list, so partial clip sharing missed
entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent
keying was shown SOUND rather than assumed: ForcedTracker is two monotone
booleans, so per-extent evidence composes by field-wise OR, order- and
grouping-independently. Making that honest required per-extent demux state, so an
extent's evidence comes only from its own bytes, and memoising only extents whose
read reached a designed stop.

**A reachable panic in the timeline.** mkvstream::parse_block accepts a
TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the
next passive frame panicked adding the backstep. In release it wrapped negative
instead, firing the straggler clamp for essentially every passive frame — audio
and subtitles rewritten onto the wrong point of the output timeline. All four
sites saturate.

**A public constructor divided by zero**: PrefetchedSectorSource::new_with_events
with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling.

**Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro,
samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as
prevention, because the next tracing::debug! someone adds is the leak. A doc claim
that DiscInputs "contains no secrets" was false and is corrected.

**An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB
like its writeback sibling, with the parse split out so the bound is testable
without touching process env.

**The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification.
Now file_len/16, since only vide/soun tracks are indexed and the shortest legal
AC-3 frame is 128 bytes.

**Two pipeline concurrency defects**: a consumer apply() error was invisible to the
producer, and abandon/finalise had a TOCTOU where a caller could report an
unfinalised output. Both fixed with compare-exchange state rather than a bool.

**Two per-frame copies removed**, both MEASURED rather than reasoned: the AU
assembler now hands its allocation to the frame (same pointer, unchanged capacity,
proven by asserting the pointer) and tsmux reuses one Annex-B buffer across
frames. Both keep capacity deliberately — a naive split_off would have cost more
than it saved.

**A comment pointed at the wrong file** for a mirrored constant; the mirror is now
compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift
fails the build.

Deferred to another agent's files, all confirmed: detect_rate's fractional-twin
snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic
base-key gap fill, and MkvStream's frame cap counting frames rather than bytes.

Every fix verified red by reverting it. Also noted for later:
DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext.
2026-07-29 20:47:31 -07:00
Matthew Jackson 3efa6211f3 Make six silent mux failures observable
All six confirmed against the code. The governing rule this cluster serves: a
lossy or degraded outcome is never silent, because a corrupt rip the user does
not know about is the worst failure available.

**A 3D MKV re-mux silently lost one eye.** The BlockGroup read path had arms for
BLOCK / BLOCK_DURATION / REFERENCE_BLOCK only, so BLOCK_ADDITIONS fell into the
skip arm — while the writer does emit BlockAdditions > BlockMore > BlockAdditional
for the MVC dependent view. Reconstruction was judged out of scope and the
reasoning is recorded: PesFrame has no side-payload field and the header parser
never reads BlockAdditionMapping, so there is no dependent-view track to route the
AU to. Instead the loss is now LOUD — counted in bytes and events, warned once,
and surfaced through MkvStream's errors()/lost_bytes(), which the driver already
samples into MuxOutcome. One detail in the finding was wrong and is corrected: the
re-mux does NOT still advertise the mvcC mapping, because the header parser
ignores that element, so the output is a plain 2D H.264 track.

**An all-titles rip silently skipped real titles.** The header-buffer-cap
overflow returned Error::MkvInvalid, and is_skippable_title_stub matches exactly
E_MKV_INVALID | E_CSS_KEY_MISSING — verified here — so a 512 MiB-of-frames title
was classified as an empty nav/menu PGC stub and dropped. It now has its own
E9051 / MuxHeaderBufferExceeded { bytes }, outside the skippable set.

**The public pre-mux report contradicted the file.** Mp4Sink::finish() drops an
audio track it cannot describe, which I chose last round over failing an export
whose video is fine — but mp4_fit_report still listed that stream as included, so
the application's plan and the actual output disagreed. Fixed at both levels:
Mp4SkipReason is now non_exhaustive with NoSamples and UndescribableAudio,
Mp4Sink::final_report() describes the FILE rather than the plan, and for the
boxed dyn Stream path a defaulted Stream::undelivered_streams() carries the
information out to MuxOutcome::undelivered_streams with a driver-side warn.

**MP4 track ids could collide.** ids were assigned before the retain that drops
sample-less tracks, while next_id came from the post-retain count, so [1,3]
yielded next_id 3. Now max(track_id) + 1, saturating.

**Stream selection silently skipped its codec_privates prune** when the lists were
not the same length — but codec_privates is consumed POSITIONALLY and trailing
extras are documented as benign, so the length-equality guard was itself the bug.
The prune now runs unconditionally by index.

**The m2ts_mux scaffolding armed params_written on both the absent and the
unparseable codec_private arms** — the same defect already fixed in tsmux.rs.
Split into params_attempted (a latch, since retrying identical bytes cannot help)
and params_emitted, with a warn on each failure arm and an accessor so the
eventual wiring and its test can observe it.

Each fix verified red by mutating back to the prior behaviour: errors() 0 vs 1,
E6008 vs E9051, final_report [0,1] vs [0], next_track_id 3 vs [1,3], and the
selection prune resolving index 1 to the wrong track's record.

API surface deliberately widened: MuxOutcome gains a public field and Mp4Sink
becomes public. Nothing in-repo breaks. Note a behaviour change on the mkv://
input path — a 3D re-mux now reports non-zero loss, so a consumer treating
errors > 0 as disc damage will trip on it. That is intended: the outcome IS
degraded.
2026-07-29 20:46:10 -07:00
Matthew Jackson 9ad68dd092 Fix four MP4 conformance defects against the standards
**dec3 declared a 0 kbit/s AC-3 substream.** parse_dolby routes bsid < 11 to
parse_ac3, which leaves data_rate_kbps = 0 and keeps the AC-3 bsid, yet
dolby_sample_entry wrapped that config in ec-3/dec3 for any Codec::Ac3Plus track.
ETSI TS 102 366 Annex F.4 assigns ac-3/dac3 to an AC-3 bitstream and F.6 assigns
ec-3/dec3 to an Enhanced AC-3 one, so the entry now follows the SYNCFRAME that was
actually parsed, not the playlist's codec label. Computing an AC-3 data rate and
keeping ec-3 was rejected: it fixes one field while bit_stream_identification and
num_dep_sub keep misdescribing the stream. The bsid threshold is hoisted into one
constant so parser and entry-chooser cannot drift. Adjacent defect fixed in the
same box: data_rate is 13 bits from a u16 source, so push's mask WRAPPED anything
above 8191 (9000 became 808); it now saturates.

**colr tagged HLG as PQ, and PAL as BT.601.** video_colr carried a second,
drifted copy of the ColorSpace-to-CICP map: transfer 16 (PQ) for every BT.2020
stream with no HdrFormat override, and 6 (BT.601) for Bt470bg. Per ITU-T H.273
Table 3, HLG is 18 and BT.470-6 System B/G is 5 — and mkv::cicp_for_video already
got both right. video_colr now delegates to that shared resolver, so the
duplicated table is gone and cannot drift again. It keeps only its own decision
about WHETHER to emit the box, since an absent colr and an all-unspecified colr
mean the same thing per ISO/IEC 14496-12.

**Exact 24.000 / 30.000 / 60.000 fps was declared 23.976 / 29.97 / 59.94.**
detect_rate took the FIRST STD_RATES entry within 0.5 fps, and every 1000/1001
entry precedes its integer twin 0.024 fps away — a 0.1% error across the whole
track's mdhd and stts. Fixed as nearest-wins rather than by reordering the table:
reordering fixes today's table and re-breaks the moment someone appends a rate,
while nearest-wins is order-independent. Verified red here independently by
reverting to first-match, which fails exactly the two timing tests.

**ddts MultiAssetFlag was set from has_extension.** In the DTSSpecificBox
(ETSI TS 102 114) that flag signals more than one audio ASSET. A DTS-HD MA/HRA
track is one asset whose extension substream carries the XLL/XBR component, so
setting it from "an EXSS sync follows the core" sent a parser looking for a second
asset descriptor while StreamConstruction simultaneously said there was no
extension — the box contradicting itself. This module parses the core header only
and never reads the EXSS asset table, so 0 is the only honest declaration. A
StreamConstruction index for core+EXSS was deliberately NOT invented: that field
is a table lookup that could not be confirmed against the standard, and a wrong
index is worse than an under-declaration.

DTS_AMODE_LAYOUT's masks were independently re-derived against ETSI TS 102 114
§5.3.1 and all 16 are CORRECT — only three adjacent comments were wrong (the
AMODE 2/3/4 annotations were rotated by one, and AMODE 9's said "5.1 with LFE"
when 0x0007 is the 5.0 mask and LFE is OR'd in separately). Comments corrected.

Every one of the eight new tests decodes the field back OUT of the emitted bytes —
data_rate from the dec3 body's leading 13 bits, MultiAssetFlag from bit 48 of the
ddts tail, colr from the nclx payload inside a real stsd, and the frame rate from
mdhd.timescale plus stts.sample_delta of a fully muxed MP4 — rather than
restating arithmetic. This audit has already caught one of my own tests doing the
latter.

NOT fixed, root-caused and recorded at the site instead: an A_PCM/INT/BIG track
ships with no BitDepth, which the Matroska Codec Specifications make a MUST. The
width exists on disc (BD LPCM signals it in the ES header byte 3, DVD in the IFO
audio attribute byte 1) but neither source reaches MkvTrack::audio, and the fix
needs a new AudioStream member plus a deferred setter in files another agent held
this round. Guessing 16 was rejected — it would confidently misdecode every
24-bit disc — as was refusing the track, which would regress the 16-bit majority
that currently plays by accident.
2026-07-29 20:29:17 -07:00
Matthew Jackson 13897e14f0 Resolve the forensic key map once per disc, not once per playlist
resolve_content_key_map loops every title into resolve_mux_key_map, which called
resolve_fmts_key_map FIRST — before the CpsUnitCache — and on an FMTS disc
returned immediately. So every playlist re-derived facts that belong to the DISC:
a full UDF walk plus /AACS/IndividualSegment.tbl, and on an FMTS disc the anchor
probe, the 32-index phase probe, and a fetch.fmts_indexes round trip.

On a 60-playlist disc, measured on a synthetic fixture: 840 -> 14 metadata reads,
2,400 -> 40 probe reads, and 60 -> 1 key-service calls. Worst case before was up
to 256 probe reads and 32 key-service calls per title. The 60 redundant
key-service round trips are a strong candidate for the keyserver storm seen in
the field.

Two memos behind a pub(crate) DiscKeyCache. The table memo (UDF walk + tbl parse)
is disc-invariant outright — nothing in that path mentions the title — and runs on
EVERY disc, so a plain BD benefits too. Only the deterministic negatives are
memoised as "not FMTS"; a DiscRead fault propagates uncached so a later title
retries.

A blind once-per-disc hoist of the PROBES was rejected as unsafe, and this is the
load-bearing reasoning: the title enters through clip_byte_to_lba, which decides
which segments are addressable and which LBA every probed clip byte reads from, so
two titles with different extent lists probe different physical bytes. A hoist
would serve title B an answer derived from title A's media and could silently turn
a per-title FmtsKeyMissing into a success. The extent list is the ONLY per-title
input, so keying on it is exactly sufficient — matching the ForcedProbeCache
precedent.

Result-identity was proved, not assumed: NEITHER probe reads the key pool.
Verified here independently — probe_fmts_index_keys takes no keys parameter at
all; index keys come from `fetch`, and the anchor's reply feeds the phase probe.
So the pool's growth across titles, the one thing that does change between calls,
cannot move a memoised value, and the result is order-independent. A test resolves
three titles through a shared memo and through fresh memos and asserts both the
per-title ranges and the final key pool (keys, slots, order) are identical.

Not memoised, deliberately: fail-loud FmtsKeyMissing, and any run where an index
hit a read fault — that is a property of a transient drive fault, not of the
extents, and caching it would spread one bad read across 59 playlists.

A fully-memoised title now does zero I/O, which made the old in-loop halt polls
unreachable for it, so a check_halt on entry was added with a test that cancels
after warming the memos.

Also corrects my own overstatement from last round: the CpsUnitCache doc now says
plainly that on an FMTS disc it removes NO reads, because this function returns
before the extent loop ever runs.

Five mutations, all verified red. Pre-existing bug flagged but not fixed:
filter_addressable_segments only checks that a segment's START byte maps to some
LBA in the title, so a play-all playlist can pass the filter while mapping segment
bytes into the wrong clip, whose anchor then returns empty and aborts the sweep.
2026-07-29 20:27:50 -07:00
Matthew Jackson b4bf0daa82 Group E-AC-3 dependent substreams into one access unit
The AC-3 parser's own module doc stated the assumption: "AC3 frames are
self-contained and always start with syncword 0x0B77". True for legacy AC-3,
false for E-AC-3 above 5.1. Per ETSI TS 102 366 (A/52) Annex E, byte 2 of an
E-AC-3 syncframe is strmtyp(2) | substreamid(3) | frmsiz[10:8], and an access
unit is one INDEPENDENT substream plus every DEPENDENT substream that follows it
until the next independent one. The parser emitted one PES frame per syncframe,
so a decoder saw each dependent substream as a standalone frame with no parent —
including the AC-3-core + E-AC-3-dependent form Blu-ray uses for Dolby Digital
Plus. The extra channels were lost and the timeline ran at 2x.

The bit position is cross-checked against code already in the tree: the existing
frmsiz parse takes byte2 & 0x07 as its high bits, which is only consistent with
strmtyp occupying byte2's top two bits. Legacy AC-3 is excluded by bsid < 11,
where byte 2 is crc1 and reading strmtyp there would be nonsense. Reserved
strmtyp 3 is treated as INDEPENDENT so an unknown type starts a fresh AU rather
than merging into an unrelated one.

The AU carries the INDEPENDENT substream's PTS, and only the independent
substream advances the clock — dependents cover the same time period and add zero
duration. That is what removes the doubled timeline.

A trailing AU that can still grow is HELD across the PES boundary, because the
boundary is unknowable until the next independent sync; the whole AU is re-scanned
next call, so there is no shift and no double-count in the loss tally. Plain AC-3
is never held, which keeps DVD/AC-3 latency and behaviour unchanged.

A latent pre-existing bug surfaced while testing this: a new PES's PTS was
re-stamping an AU that began in an earlier PES, a constant one-frame shift. Fixed
with a PtsAnchor so a PES timestamp applies to the first AU that STARTS in that
PES's own bytes, while a genuine PTS jump is still adopted.

Nine tests. Verified red against five mutations, each killing a specific set:
reverting to the pre-fix behaviour kills 8 while
plain_ac3_frames_are_not_grouped_or_delayed SURVIVES as the no-regression guard —
reproduced independently here. Stamping the dependent's PTS kills 6; not holding
across PES kills 4; holding plain AC-3 too kills 15; neutering the PTS anchor
kills exactly the 2 split-across-PES timing tests.

Three sibling defects found and deliberately NOT fixed, all in mp4/audio.rs:
dec3 hardcodes num_dep_sub = 0 (and a nonzero value changes the box LAYOUT, not
just a field, per Annex F/G); parse_eac3 ignores strmtyp/substreamid entirely;
and a 7.1 DD+ track is still labelled 5.1 because the channel count comes from
the independent substream while the extra channels are described by the
dependent's chanmap, which nothing parses.

Not verified: no real E-AC-3-with-dependents sample exists here, so all evidence
is synthetic frames plus the spec layout. The multi-independent-substream case
(num_ind_sub > 1, main + associated audio in one PID) is deliberately treated as
one AU per independent substream and is untested.
2026-07-29 20:26:51 -07:00
Matthew Jackson ef36b452ad Fix three defects in last round's own fixes
Round 3 audited the round-1/2 fix commits rather than trusting them, and found
three defects in that new code. This is why the pin moves each round.

1. LICENCE REGRESSION, and it was mine. Reverting the "distinguish a failed key
   source" commit also restored a verbatim reference-decoder table citation in
   src/mux/codec/dts.rs, because both changes were in that one commit. The MIT
   licence cleanup was silently undone at HEAD and nothing caught it.

   The citation is replaced with ETSI TS 102 114 §5.3.1 again, and — more
   importantly — the rule now lives in the leak gate instead of in my memory.
   scan-secrets.sh gains LICENCE_RE, which flags ff_dca*, dcadec, l-smash,
   libav*, and bare ffmpeg/FFmpeg as REF-IMPL-CITATION. `no ffmpeg` is
   explicitly allowed via negative lookbehind: stating what this project does
   NOT depend on carries no risk and is a genuine selling point. Verified by
   re-introducing the citation (gate fails) and removing it (gate clean).

2. TsMuxer armed params_written even when the avcC/hvcC parser returned None, so
   a track whose codec_private exists but will not parse was muxed to BD-TS with
   no VPS/SPS/PPS ever emitted — undecodable video, reported as success, with no
   log line. Last round's fix corrected WHICH parser is used and left this half
   untouched. Arming the flag is still right (retrying identical bytes cannot
   succeed) but it is no longer silent: it now warns with the track, codec and
   codec_private length.

3. test_aes_cbc_roundtrip defined a LOCAL fn aes_cbc_encrypt that SHADOWED the
   production primitive, so it round-tripped a copy of the algorithm against
   itself and never touched crypto::aes_cbc_encrypt — the function this cycle
   added. Any mutation to the shipped code passed it. The shadow is deleted and
   the test now calls the real primitive; verified by mutating
   crypto::aes_cbc_encrypt, which now fails it and previously would not have.
2026-07-29 20:08:09 -07:00
Matthew Jackson f338552969 Pin the toolchain to Rust 1.87
The Windows UI needs winsafe, whose current release requires rustc 1.87. The
alternative was pinning winsafe back to an older release, which would bake a
stale API surface into a brand-new UI permanently to dodge one minor version.

The pin's purpose is to sit BELOW the Mac default so clippy drift is caught
locally before CI, not to stay on 1.86 specifically, so 1.87 preserves the
discipline exactly.

Verified before moving anything, not after: `cargo +1.87 clippy -- -D warnings`
and `cargo +1.87 fmt --check` are clean across all eight repos, and the full
precommit gate (fmt + clippy + tests) passes on libfreemkv, autorip, bdemu,
freemkv-engine and freemkv-keysources. Zero new lints, zero formatting drift.
2026-07-29 20:02:02 -07:00
Matthew Jackson 38aa895038 Memoise multi-CPS key-map sampling per extent, not per title
resolve_content_key_map calls resolve_mux_key_map once per title, and on a
multi-CPS disc that path issues 8 random single-unit reads per extent. A disc's
playlists overwhelmingly reference the same few clips — main feature, play-all,
per-chapter and seamless-branch variants — so the same physical extents were
re-sampled from the drive once per playlist. On a 60-playlist / 15-clip disc
that is ~2,400 non-sequential 6144-byte reads, roughly 8 minutes of pure seeking
at 200 ms per seek, before the mux starts. Now ~600 reads.

Keyed per EXTENT — (format, start_lba, sector_count) — rather than per title's
whole extent list, which is finer-grained than the forced-subtitle probe's cache
and strictly better here: a play-all playlist sharing 4 of 5 extents with the
main feature still hits on those 4.

Why a cached pool index is provably identical to a recomputed one, verified
rather than assumed:

  * `pick` iterates the pool IN ORDER and returns the FIRST index whose key
    decrypts a sample to clean.
  * The pool is APPEND-ONLY. Checked across the whole crate: only `push`, with no
    insert/remove/clear/retain/sort/dedup/truncate/drain/swap/reverse anywhere.
    So appended keys can only land AFTER a matched index, and the first match for
    the same samples cannot shift.
  * The samples are a pure function of the three values in the key, read from
    read-only optical media.

Two outcomes are deliberately NOT cached, which is what makes this safe rather
than merely faster:

  * the inherited index (`None if samples.is_empty() => last_idx`) is per-TITLE
    state, not a property of the extent — caching it would let one title's
    carry-in index leak into another title's clear extent, i.e. a WRONG key;
  * the fail-loud DecryptFailed verdict, so a retry after a key source banks the
    missing key re-samples instead of inheriting a stale answer.

Halt is still polled before the cache lookup, so cancellation is unchanged.
resolve_mux_key_map keeps its exact signature and delegates with a fresh cache,
so there is no public API change. ContentFormat gains Eq + Hash (additive).

Four tests, and the two mutants that matter both verified red: disabling the
cache short-circuit fails the hit and recompute-equivalence tests, and wrongly
caching the inherited index fails multi_cps_inherited_index_is_not_cached.
2026-07-29 19:49:55 -07:00
Matthew Jackson e3676e7cdf Build BlockGroups in memory so the hot path never seeks
Every BlockGroup frame back-patched its element size via ebml::end_master, which
does two stream_position() calls and two real seeks. BufWriter does not override
Seek::stream_position, so each position query is seek(Current(0)) = flush_buf +
lseek — and each of those flushed the 4 MiB BufWriter while every position-moving
seek reset WritebackPipeline::last_flush_pos. The buffer never got to do its job.

An MPEG-2 title takes this path for EVERY frame (the parser stamps a per-frame
duration, so I, P and B all become BlockGroups) — roughly 350,000 per feature.

A BlockGroup's size is knowable before writing, so there is no need to back-patch
at all. New seek-free twins start_master_buf / end_master_buf patch a placeholder
by buffer INDEX instead of file offset, and build_block_group assembles the whole
element into a persistent buffer that write_block_group and its MVC sibling take,
fill, write once, and hand back — including on the error path — so the allocation
is made once rather than per frame.

Measured with a counting writer that, like BufWriter, does not override
stream_position, over 200 frames:

              seek calls   position-moving
  plain   before 912              451
  plain   after  112               51
  MVC     before 2516            1253
  MVC     after  116               53

Per-frame cost is now zero; the residual is the header, the per-cluster
back-patch and Cues. For 350k BlockGroups that is 1.4M seek calls removed on the
plain path, 4.2M on an MVC title.

end_master is deliberately NOT changed for its other callers. The Cluster master
genuinely streams — frames are appended to an open cluster over time, so its body
cannot be buffered without holding a whole cluster in memory — and the rest
(EBML header, Tracks, Info, Chapters, Cues) run once, not per frame.

Byte-identity is the safety property, and it is structural: end_master always
patches a FIXED-width 8-byte VINT, and the buffered pair writes and patches
exactly that same placeholder, so the encodings cannot differ. Verified by
capture-then-compare over 200 frames (17 keyframes / 183 non-keyframes, multiple
clusters, BlockDuration present and absent, both reference branches) — output
byte-for-byte identical.

Three tests now pin it permanently: buffered vs seeking output byte-for-byte for
empty/tiny/multi-byte bodies, the same for NESTED masters (the MVC path nests
BlockAdditions > BlockMore, where an index-arithmetic slip would surface), and
end_master_buf erroring rather than panicking on a position outside the buffer.
All three verified red against a deliberately divergent placeholder width.

A note on verifying this kind of change: comparing emitted MKV across two
worktrees at different commits shows a spurious 14-byte difference, because
MuxingApp/WritingApp embed the build's git SHA twice. Compare at the same base.
2026-07-29 19:33:31 -07:00
Matthew Jackson 807eb053ca Only assert a forced-subtitle verdict the read actually supports
probe_and_set_forced broke out of its read loop on a read error but still
applied whatever partial observation it had accumulated as an authoritative
verdict, overwriting the vendor-label-derived forced flag. A disc that faults
early could have a correct flag replaced by a guess from a fraction of the data.

The file already got the zero-observation case right — it deliberately leaves
the vendor flag alone rather than "assert not-forced from having seen nothing".
The defect was that a PARTIAL observation cut short by a fault was treated as
complete.

The fix rests on the two verdicts not being symmetric evidence.
settled_not_forced() is POSITIVE evidence — a non-forced display set was
actually seen, and no unread data can retract it. is_forced() is an ABSENCE
claim — display sets were seen and none was non-forced — which is only sound if
the read got far enough for the absence to mean something.

So every loop exit now yields a named StopReason, and absence claims are
asserted only for a designed stop:

  * Exhausted / Budget → conclusive. The budget is deliberately conclusive: the
    natural exit is "every track settled not-forced", which a genuinely forced
    track never satisfies, so the budget is the ONLY path by which a real forced
    verdict is ever reached. Treating it as inconclusive would disable forced
    detection entirely.
  * Halted / ReadFailed → inconclusive. A cancelled probe's cut-off point is as
    arbitrary as a faulted one, so its absence claim is worth no more.

Evaluated per track, matching the existing observed() gate: a track that already
saw a non-forced set keeps its sound verdict even on a truncated run, while a
forced-so-far sibling keeps the vendor flag.

An inconclusive run is also NOT memoised. The cache key is the extent list and a
disc's playlists share clips, so caching a truncated run would replay one read
fault onto every playlist referencing those extents and deny any later title the
chance to re-read them.

Four tests; three of them verified red by forcing absence_is_conclusive() back
to always-true (the old semantics), while the budget test correctly stays green
either way — confirming the guard was not over-corrected.

Also moves the function doc comment back onto probe_and_set_forced; the earlier
probe commit left it attached to the ForcedProbeCache type alias.
2026-07-29 19:26:58 -07:00
Matthew Jackson 7322f4dd8a Record that the failed-vs-absent key source arm is unreachable
The `Ok(_) | Err(_)` arm in resolve_and_apply_traced conflates "this source
had no entry for the disc" with "this source failed", and reports both as
KeyNode::NoEntry. An operator whose key server is returning 502s is therefore
told their disc is not in the database.

The conflation is real but LATENT, and fixing it here would change nothing an
operator can see, because no shipped KeySource ever returns Err:
KeydbSource::get_unit_keys maps a load/parse failure to Ok(Vec::new()),
OnlineSource::get_unit_keys is Ok(self.query(ctx)) where query returns empty on
transport error, HTTP status, oversize body and bad JSON alike, and MultiSource
discards inner Errs. Only test doubles return Err. FetchOutcome::errored in
drive_unit_keys / drive_fmts_indexes is dead for the same reason — the right
contract, honoured by no source.

autorip already works around the missing signal by re-probing the service over
HTTP (probe_online_reachability / key_service_transient_status), and its own
comment names the incident: "the online keysource swallows every failure
(transport error, 502, timeout)".

So the fix belongs at the source boundary in freemkv-keysources, with
Disc::aacs_error as the channel the operator actually reads — not in this
trace. Documented here so the next reader does not assume the arm works, and
does not "fix" a dead path as I nearly did twice.
2026-07-29 19:24:58 -07:00
Matthew Jackson 4ed245868e Revert "Distinguish a failed key source from one with no entry"
This reverts commit 22a3e3fd01.
2026-07-29 19:15:21 -07:00
Matthew Jackson 50f37462db Remove reference-decoder citations from a public MIT-licensed repo
This crate is MIT licensed. Comments citing another decoder's internal symbols
and reproducing its tables verbatim create licence risk that no engineering
benefit justifies, so every such citation is replaced with the primary source:
ETSI TS 102 114 §5.3.1.

Eleven sites across src/mux/codec/dts.rs and src/mux/mp4/audio.rs. The technical
substance is unchanged in every case — the deficit-sample-count semantics, the
reserved-field skips, the invalid LFF value, and the 16 legal AMODE codes are all
spec facts and are now attributed as such. Four CHANGELOG entries that named a
validator are reworded; the "no a reference decoder" dependency claim stays, since stating
what this project does NOT depend on carries no risk.

I initially argued this was a false positive on the grounds that the project's
hygiene rules name internal infrastructure and reverse-engineering material, not
open-source citations, and that a channel-count table from a standard is fact
rather than expression. That reasoning missed the point: the exposure is MIT
distributing text derived from GPL/LGPL sources, and that is the maintainer's
risk to weigh, not mine. Reversed in full.
2026-07-29 19:09:10 -07:00
Matthew Jackson 22a3e3fd01 Distinguish a failed key source from one with no entry
resolve_and_apply_traced collapsed `Ok(_) | Err(_)` into a single
KeyNode::NoEntry step, so a key source that FAILED — server unreachable, keydb
unreadable, malformed entry — was recorded identically to one that simply had no
entry for this disc. The front-end renders that trace, so it told the operator
their disc is not in the database when the real cause was a fixable
infrastructure problem. drive_unit_keys and drive_fmts_indexes were refactored
this cycle to preserve exactly this distinction; this path had not been.

KeyNode gains a SourceFailed variant and the two arms are split. freemkv's
trace renderer matches KeyNode exhaustively with no catch-all, so its arm is
added in the same change — otherwise the consumer would not build.

Also made ETSI TS 102 114 the primary authority for DTS_AMODE_COUNT's comment
rather than a reference decoder internal symbol, and pointed it at this
crate's own cross-checked DTS_AMODE_LAYOUT / DTS_AMODE_CH tables.

A round-2 finding asked for every a reference decoder and a reference decoder citation in the DTS parser
to be stripped as a public-repo hygiene violation. Rejected: the project's rules
(scan-secrets.sh, CLAUDE.md) prohibit internal infrastructure references and
reverse-engineering material, and a reference-decoder citation is neither. The
AMODE channel-count table is a factual table from the standard, not expression
copied from an implementation. Citing the spec plus a corroborating
implementation is how a decodability gate should be justified.
2026-07-29 19:07:15 -07:00
Matthew Jackson 99c5fd3500 Reference keyframes per track, size the DTS reserve, correct two claims
The ReferenceBlock offset was computed for ANY video track, but the keyframe tick
it measures against was recorded in a single global slot gated to the PRIMARY
video track. On a title with two video tracks — an MVC base plus secondary view,
or a multi-angle disc — a secondary track's non-keyframe therefore referenced a
keyframe on a different track, or 0 (a self-reference) when the primary had not
produced one yet. The tick is now recorded per track, so a non-keyframe can only
reference a keyframe on its own track.

The faststart moov-hole estimate modelled every audio track as (E-)AC-3 at 1536
samples per frame. 1.6.0 added DTS to the writer's carried set, and a DTS core AU
is commonly 512 samples — a third of that — so a DTS track's sample table was
under-reserved threefold and the mux fell back to moov-at-end, losing faststart
on exactly the files 1.6.0 newly supports.

mvc_frame_emits_blockgroup_additional_and_reference asserted only that the
non-keyframe's ReferenceBlock was Some(_). Its non-MVC sibling, added in the same
commit, pins the exact offset; this one now does too, so a mutant emitting a
constant or wrong-signed offset no longer passes.

The comment above the mp4 sample budget claimed file_len stops a crafted file
inflating allocations "past the file's own size". Each indexed sample costs ~52
bytes, so the real ceiling is ~52x file_len (still capped by MAX_SAMPLE_COUNT).
The bound is real; the comment overstated how tight it is.
2026-07-29 19:03:59 -07:00
Matthew Jackson d4c913e0d3 Validate stream-selection PIDs per class, not across both
StreamSelection::apply validated a listed PID by scanning ALL streams, so a PID
named in the wrong class's filter passed validation — an audio filter listing a
subtitle PID, say. `keeps` then matched it against the audio streams only, so
the requested track was silently absent from the output. That is precisely the
outcome this validation documents itself as preventing: "fail loud rather than
silently emit an MKV missing a requested track".

Each filter is now checked against its own stream class. The `listed_pids`
helper existed only for the cross-class scan and is removed rather than left
behind as dead code.

Test covers both directions plus the sanity case, and asserts a rejected
selection leaves the title unpruned.
2026-07-29 19:00:53 -07:00
Matthew Jackson 0e23a6b291 Halve the per-frame allocation and copy on the m2ts NAL video path
The NAL path called length_prefixed_to_annex_b, which allocates a whole-frame
Vec of its own, then copied the result into a second whole-frame Vec — two
full-frame allocations and two full-frame copies per video frame. The crate
already has append_length_prefixed_as_annex_b, which writes the conversion
straight into a destination buffer; it is the same code path with the
intermediate removed.

The destination is also sized once up front instead of starting from Vec::new(),
which re-grew from zero capacity inside every conversion.

On a UHD HEVC title muxed to m2ts:// — ~200k video frames averaging ~310 KB of
ES at 60 Mb/s — that removes roughly 62 GB of allocation and 62 GB of memcpy.

Behaviour is unchanged: the existing tsmux conversion tests, including the
non-NAL passthrough and Annex-B default pair added last round, all still pass.

A first attempt reused a persistent scratch buffer across frames, which does not
work: the buffer is handed out as Cow::Owned and so can never be returned. A
right-sized single allocation gets most of the win without restructuring the
function around the borrow.
2026-07-29 18:58:39 -07:00
Matthew Jackson bcf47cc4ca Fix ddts numeric truncation and the decrypt pool's poison asymmetry
ddts CoreSize wrapped to zero on a maximum-size core. core_size is FSIZE + 1 and
FSIZE is itself 14 bits, so the maximum is 16384 — one past what the 14-bit
CoreSize field holds — and push()'s mask turned that into 0, declaring an empty
core frame. Clamped to 16383 instead: one byte short beats telling a decoder
there is no core. Proven red first (the field read back as 0).

ddts avg/max bitrate under-declared every non-integral frame rate. It computed
sample_rate / frame_samples first, so a 512-sample core at 48 kHz truncated
93.75 frames/s to 93. Multiplying before dividing, with round-to-nearest, keeps
the precision.

set_decrypt_threads skipped the pool swap on a poisoned lock while
DECRYPT_THREADS had already been updated, so the new thread count was reported
as taking effect while the stale pool kept serving. decrypt_pool() deliberately
recovers from poisoning for exactly this reason; the setter now does the same.
The pool Arc is immutable once stored, so a prior panic cannot have left it
half-written.

The CoreSize test decodes the value back out of the emitted box rather than
restating the clamp — a first draft asserted the clamp arithmetic against
itself, which would have passed against the unfixed writer.
2026-07-29 18:56:48 -07:00
Matthew Jackson a9dc3d7244 Make encrypt_unit report a refused slice, and expand its key once
Two defects in the encrypt_unit promoted to public API last round, both found by
round 2 auditing that new code.

It returned silently without encrypting when the slice was shorter than
ALIGNED_UNIT_LEN. Its own contract requires the caller to set the container's
encrypted flag BEFORE calling — the header is the key seed — so a silent no-op
leaves a unit advertised as encrypted while still carrying plaintext, with
nothing for an authoring caller to check. It now returns bool and is
#[must_use], so ignoring the refusal is a compile-time warning; every call site
was updated to assert on it.

bool rather than Result deliberately: a wrong buffer length is a programming
error at a library boundary, not a disc condition, and a new Error variant would
mean a new numeric code plus its rendering in another repo.

It also drove CBC from the single-block aes_ecb_encrypt, rebuilding the AES key
schedule for each of the 383 blocks in a unit — an order of magnitude slower
than its inverse, which expands the key once via aes_cbc_decrypt. The missing
counterpart aes_cbc_encrypt now exists alongside it, and encrypt_unit calls it,
so the two directions are symmetric in structure as well as in result. For an
authoring caller encrypting a 90 GB image that removes ~5.6 billion redundant
key expansions.

New test pins the boundary: ALIGNED_UNIT_LEN - 1 returns false and leaves the
buffer byte-identical, ALIGNED_UNIT_LEN succeeds. The existing round-trip and
padding-asymmetry tests still pass, so the CBC rewrite is provably the same
transform.
2026-07-29 18:54:19 -07:00
Matthew Jackson 94a876664b Correct six stale comments and doc claims
All six describe code that does something different from what they say, which
is the class of defect that gets a maintainer to write a bug on purpose.

docs/clpi.md presented the CLPI stream-PID entry as byte-aligned 2/2/2/4/4-byte
fields with a 32-bit fine-entry count. It is one 80-bit packed block —
reserved(10) + EP_stream_type(4) + num_EP_coarse(16) + num_EP_fine(18) +
EP_map_start_address(32) — and num_EP_fine is 18 bits. Anyone parsing to the
doc's offsets would read garbage. Replaced with the real bit layout.

docs/udf.md said read_directory()'s recursion cap is 3; MAX_DIR_DEPTH is 8.

TROUBLESHOOTING.md called Pass 1 `recovery::copy`. The engine's `sweep` is
documented as "Pass 1 of a multipass rip"; `copy` is the dispatch verb that
chooses between sweep and patch. This inconsistency was mine, introduced in the
1.6.0 doc rewrite. docs/drive-access.md already said `sweep` and was right — a
round-2 finding claimed the opposite on the grounds that `recovery::sweep`
appears nowhere else in this crate, which it cannot, being in another crate.

io/pipeline.rs cited `disc::patch` as WRITE_THROUGH_DEPTH's caller; that moved
to freemkv-engine in 1.6.0 and no `patch` exists here.

truehd.rs's doc on mlp_major_sync_crc_ok said the trailer is compared
big-endian while the body compares u16::from_le_bytes — and a big-endian
compare was the bug the function was fixed for, so the comment described the
defect rather than the code.

sector/decrypting.rs claimed the decorator owns "the only mutable state (its
call-count cap and spent flag)". DecryptingSectorSource has no such fields and
no KeyFetch field at all in this revision.
2026-07-29 18:50:53 -07:00
Matthew Jackson 7030de4ec9 Apply stream selection on the live path, and treat a halt as a clean stop
Three defects around MuxOptions in the mux driver.

MuxInput::Live never applied MuxOptions.selection, so a caller's audio/subtitle
selection was silently ignored on the live-drive path while the field's own
documentation said it was applied before the demux pipeline is built. The Iso
and Session arms both apply it; Live now does the same, in the same place —
before resolve_inline_base_map, which is keyed on extents and so unaffected by
pruning the stream list.

The header gate returned Error::MkvInvalid whenever headers had not resolved.
On the prefetch-highway path a halt landing while the pump is blocked in a read
can end the stream as Ok(None) rather than Err(Halted), so the loop breaks with
headers unresolved through no fault of the data. Reporting that as MkvInvalid
tells the consumer its disc is malformed and skips the stop-preserves-staging
path that a clean completed=false triggers. The gate now re-checks halt first.

MuxOptions.selection's doc claimed it was applied without naming the one input
it is not applied to. That exception lived only in an internal comment at the
Url match arm, where a caller reading the public field docs would never see it.
It is now on the field, pointing at InputOptions::selection instead.
2026-07-29 18:48:51 -07:00
Matthew Jackson c0434e87de Fix the DVD MPEG-audio codec mapping and the fabricated AACS docs
parse_audio_attr mapped DVD audio_coding_mode 2 to Codec::Mpeg1 — the MPEG-1
VIDEO variant. Codec::kind() reports Video for it, so a DVD MPEG-audio stream
was classified and handled as video everywhere downstream. Modes 2 and 3 are
both MPEG audio Layer II (3 adds the MPEG-2 multichannel extension), so both
map to Codec::Mp2. A test now walks every coding mode and asserts each result's
kind() is Audio, so no mode can map to a non-audio codec again.

docs/aacs.md documented an entire keydb-resolving API that does not exist:
ScanOptions::with_keydb, Disc::open_title, reader.read_unit(). None of those
symbols appear anywhere in the crate, and ScanOptions has no keydb field — its
own doc comment says "libfreemkv is lookup-free — it resolves no keys". A
reader following that page would conclude the library reads keydb.cfg, which
inverts the actual design: the caller resolves keys out-of-band through a
KeySource and applies them with Disc::decrypt_with.

The section is rewritten against the real API, and the AacsState table's
`key_source` type corrected from KeySource to KeyOrigin.

Worth recording: the first replacement example I wrote was itself wrong. It
used `input("disc://...")`, which resolve.rs explicitly rejects with
Error::DiscUrlNotDirect — live disc must go through Drive::open + Disc::scan +
DiscStream::new. Every symbol and signature in the committed example was
checked against the source rather than assumed.
2026-07-29 18:47:06 -07:00
Matthew Jackson ec5cd31ae1 Stop Mp4Sink losing audio frames and writing an empty sample entry
Mp4Sink::write returned Ok(()) without recording the sample whenever an audio
track's frame would not parse into a sample entry. Two consequences, both
silent: leading audio frames were lost until one frame parsed, and a track
whose frames never parsed disappeared from the output entirely — finish()'s
retain() removed the sample-less trak and the run reported success. That
contradicts this crate's stated policy that a skipped track is never silently
dropped.

The drop was never necessary. audio_entry is read in exactly one place,
build_trak, reached only from build_moov inside finish() — nothing on the write
path consumes it. So write() now records every sample and derives the entry
opportunistically from whichever frame parses first.

That makes build_trak's `audio_entry.unwrap_or_default()` reachable, which
would emit an stsd declaring entry_count=1 around an EMPTY sample entry: a
structurally invalid mp4 returned as success. finish() therefore drops any
audio track it cannot describe, with a tracing::warn! naming the codec and
sample count.

Dropping rather than erroring is deliberate. It matches finish()'s existing
treatment of sample-less tracks, keeps an export whose video is fine from
failing outright, and needs no new error code — a new code would mean a new
i18n key across 29 locale files in another repo, which is not this change's
scope. The track's bytes stay unreferenced in mdat: wasted space in a valid
file, which is the cheaper failure.

Test pins both halves — moov describes only the video track, and the
unparseable audio bytes still reach mdat rather than being discarded at write
time.
2026-07-29 18:44:49 -07:00
Matthew Jackson a1304f9e78 Stop the mp4 demuxer dropping tracks silently or inventing sample offsets
Three defects in the mp4:// read path, all of the same family: a damaged
source was remuxed minus a track, or with fabricated data, and the run
reported success.

Silent drops. Eight paths dropped a whole track on malformed input with no
report of any kind, so an mp4:// source missing its audio looked like a clean
run. Each now emits a tracing::warn! naming the track and the missing or
inconsistent table (tracing English is permitted in this crate; the numeric
error codes are unchanged). The non-A/V handler case is debug!, since skipping
a timecode or hint track is normal.

Fabricated offsets. sample_offsets ended with a `while offsets.len() <
sizes.len()` loop that packed unplaced samples after the last known offset.
Those samples have no known location, so the invented offsets made the reader
pull frame data from arbitrary file bytes — the exact "emit garbage" outcome
the stco/stsc presence guards refuse. It now returns the short list and the
caller drops the track.

Short stts. `durations.get(i).unwrap_or(0)` gave every sample past the end of a
short stts a duration of 0, collapsing the whole tail onto one timestamp. That
is the same degenerate timing the `durations.is_empty()` guard was written to
refuse, so the guard now refuses both cases.

Two shared test fixtures were internally inconsistent and only passed because
the reader was lenient: stsz declared 3 samples while stsc placed 1, and the
hostile-stsz fixture's stsc/stts covered a single sample. Both are now
consistent. The hostile fixture keeps its lying stsz count — that lie is what
it tests — but its stsc and stts now cover whatever count survives the
file_len bound, so it exercises the allocation bound rather than the
inconsistency guards.

Two new tests pin the new refusals by mutating the consistent fixture: an stsc
that places 1 of 3 samples, and an stts that covers 1 of 3.
2026-07-29 18:42:38 -07:00
Matthew Jackson a39045adf1 Bound the forced-subtitle probe, make it cancellable, and stop re-reading clips
The probe's only natural exit was "every PGS track has shown a non-forced
display set". A genuinely FORCED track never satisfies that, so on the common
authoring — a forced-narrative track for foreign dialogue — the loop read the
title's entire extent set at 2 MiB per call with no byte cap, no time cap and
no halt check. It was also invoked once per title rather than once per distinct
clip, and a disc's playlists overwhelmingly reference the same few clips (main
feature, play-all, seamless-branch variants), so the same physical extents were
re-read 30-150 times. The two defects multiplied: tens of GB, times the
playlist count, off an optical drive.

Reached via ScanOptions::probe_forced_subtitles, whose only consumer is
`freemkv info -v` (freemkv/src/disc_info.rs). The rip path leaves it off. So
the symptom is `info -v` never returning on an ordinary UHD, not a corrupt rip.

Three changes:

  * PROBE_BUDGET_SECTORS caps a probe at 256 MiB. A forced track's display sets
    appear throughout the title, so a bounded prefix classifies it; the budget
    only decides how long we keep looking for a non-forced set before accepting
    the forced verdict.
  * ScanOptions::halt is now plumbed in and checked per chunk, so `info -v` is
    cancellable. The probe previously took no halt at all.
  * A ForcedProbeCache memoises verdicts against the title's exact extent list.
    Keying on byte-identical input means a hit cannot change any result — it
    only removes the re-read.

Verdict application is factored into apply_verdicts so the cached and freshly
probed paths cannot diverge.

Three tests pin the behaviour, each against a reader that counts sectors and
never ends: the budget stops at exactly PROBE_BUDGET_SECTORS, a cancelled halt
reads zero sectors, and a second title with identical extents costs no further
reads while a different extent list still misses the cache.

Also worth recording: the CLI acceptance harness never exercised `info -v`
against an optical drive — it reads ISOs from local SSD, where a full-extent
read is fast enough to hide both defects.
2026-07-29 18:38:25 -07:00
Matthew Jackson ea72e6df5f Give every audio and video codec its own registered Matroska CodecID
MkvTrack::audio's catch-all was `_ => CODEC_AC3`, and ebml.rs defined no
A_AAC, A_MPEG/L2, A_MPEG/L3, A_FLAC or A_OPUS constant at all. A CodecID
names the payload, so any of those codecs was written into the MKV declaring
AC-3 while carrying something else — a player either refuses the track or
decodes noise.

Reachable through two ordinary paths, both verified: ifo.rs:577 maps DVD
audio_coding_mode 3 to Codec::Mp2, so a DVD with MPEG audio muxed to mkv://
produced a track declaring A_AC3 over MP2 bytes; and mp4/read.rs:479 maps the
`mp4a` sample entry to Codec::Aac for an mp4:// source. codec/mod.rs already
has working parsers for MP2, MP3, AAC, FLAC and Opus, so the pipeline carried
these codecs end to end and only the container label was wrong.

MkvTrack::video had the same shape: `_ => CODEC_MPEG2` announced Codec::Mpeg1
and Codec::Av1 as MPEG-2 video. V_MPEG1 and V_AV1 added.

Both catch-alls stay, because MkvTrack::audio/video return Self and have no
error channel, but they are now reachable only by a non-audio/non-video or
Unknown codec routed there in error. Two tests enumerate every real codec of
each kind and cross-check each against Codec::kind(), so a codec added to the
enum later cannot silently inherit another codec's ID.

Both proven red first: Aac declared A_AC3 before the fix.
2026-07-29 18:34:53 -07:00
Matthew Jackson b2c7490b5b Parse H.264 parameter sets with the avcC parser, not the hvcC one
TsMuxer::write_frame handed every NAL video track's codec_private to
hvcc_to_annex_b. An H.264 track carries an avcC record, whose box layout is
different, so the parser returned None and no parameter sets were emitted —
and params_written was set unconditionally, so it never retried. H.264
muxed to m2ts:// reached the player with no SPS/PPS and was undecodable,
silently: frame_count still advanced and the mux reported success. AVC is
the dominant Blu-ray video codec, so this was not an edge case.

The correct dispatch already existed and was already used by
demux_sink::annexb_param_sets. tsmux simply never got the codec: it knew
only PIDs and a NAL-or-not bool, so it could not tell hvcC from avcC.

Rather than add a second setter, set_nal_video(track, bool) becomes
set_video_codec(track, Codec). One fact decides both the ES framing and the
parameter-set parser, so the two can no longer disagree — and that
disagreement is precisely this defect. The default stays Codec::Hevc, which
is the behaviour the bool's `true` default encoded, so a caller that never
calls it is unaffected.

Proven red first, end-to-end through M2tsStream::create with a real avcC
record: before the fix neither the SPS nor the PPS reached the transport
stream.
2026-07-29 18:32:21 -07:00
Matthew Jackson 3db4106253 Stop documenting the recovery API that 1.6.0 deleted
Disc::sweep, Disc::patch, Disc::copy, SweepOptions and PatchOptions have
zero occurrences in src/ — recovery moved to freemkv-engine — but they were
still documented in 30 places across README.md, TROUBLESHOOTING.md, six
files under docs/, seven src/ doc comments and a Cargo.toml comment.
README.md is the crate's GitHub front page and carried a full multi-pass
code example that cannot compile.

Two of the src/ references were intra-doc LINKS to deleted items
([`disc::Disc::copy`], [`disc::Disc::patch`] in scsi/mod.rs). They produced
no warning on a normal `cargo doc` only because they sit on pub(crate)
items; `--document-private-items` reports both, and they are gone now.

The README example is deleted rather than rewritten against the engine's
API: libfreemkv documenting a downstream crate's API on its own front page
is the drift that produced this, and it cannot even depend on it. The src/
references become plain code spans naming freemkv_engine::recovery::* —
deliberately not links, for the same reason.

docs/rip-recovery.md was 202 lines about relocated code. It now documents
only what this crate owns — Drive::read, SenseFamily, DiscStream's adaptive
batch halving — plus the read-path design constraints, which belong with the
code that enforces them, and points at freemkv-engine/src/recovery/ for the
strategy. api-design.md's module tree is regenerated from the real src/disc/
and src/drive/ layouts instead of hand-patched; it had listed sweep.rs,
patch.rs, mapfile.rs and read_error.rs, none of which exist.

Three stale facts surfaced while rewriting and are corrected: the read
timeouts are 10 s / 60 s, not the documented 1.5 s / 30 s; Drive::reset and
SgIoTransport::reset no longer exist at all, so "no SCSI reset from any read
path" is now stated as the stronger fact it has become; and verify_title,
listed as a progress-emitting operation, was removed entirely.

CHANGELOG.md keeps its references — those are the historical record of the
releases that shipped the API.
2026-07-29 18:04:51 -07:00
Matthew Jackson d34979ac57 Assert ReferenceBlock per block instead of scanning for the byte 0xFB
Two tests checked only find_id(&data, ebml::REFERENCE_BLOCK).is_some().
ReferenceBlock's EBML ID is the single byte 0xFB, so that asks whether one
exists ANYWHERE in the output — not which blocks carry one. Inside a
BlockGroup, keyframe-ness is signalled by the ABSENCE of ReferenceBlock, so
"which" is the entire question.

Measured rather than assumed. A mutant emitting a ReferenceBlock on every
BlockGroup, keyframes included — which reintroduces the exact 1.6.0 defect
these tests were added for, every frame reading as a non-keyframe — passed
mvc_frame_emits_blockgroup_additional_and_reference untouched. The
duration-bearing roundtrip test did catch it, via its MkvStream read-back
rather than via the presence check. A mutant corrupting the offset VALUE
was invisible to both.

all_block_groups() parses every BlockGroup in emission order with its
Block flags, relative timestamp, duration and decoded signed ReferenceBlock.
first_block_group() now delegates to it, so there is one walker rather than
two. Both tests assert per block: the keyframe carries NO ReferenceBlock,
each non-keyframe carries one, the offset equals the distance back to the
keyframe, and the SimpleBlock-only 0x80 flag is clear throughout.

Both tests previously took muxer.writer.into_inner() WITHOUT calling
finish(), leaving cluster sizes unwritten — an unparseable file, which is
why they could only byte-scan in the first place. They now write through
SharedWriter and finish(), so the assertions run against a well-formed
Matroska file.

Verified: with the fixes in place, the every-block mutant and the
off-by-one-offset mutant both fail.
2026-07-29 18:00:06 -07:00
Matthew Jackson bf2d15f39c Cover the non-NAL video path, including the wiring that selects it
set_nal_video had exactly one production caller and zero test callers. The
branch it gates decides whether a video track's ES goes through
length_prefixed_to_annex_b, and MPEG-2 and VC-1 must not: they are already
start-code ES. Getting it wrong is silent — frame_count still increments,
so the mux reports success while emitting a video-less file.

Five tests, each verified against a real mutant:

  * non-NAL ES passes through byte-for-byte, and the default path still
    converts. Both use a deliberately length-prefix-SHAPED payload so a
    wrongly-applied conversion rewrites the leading four bytes into a start
    code — a payload the converter happened to leave alone would let a
    mutant pass.
  * a non-NAL keyframe arms params_written, so following non-keyframes are
    not dropped by the pre-keyframe guard.
  * set_nal_video rejects an out-of-range track instead of panicking.
  * M2tsStream::create wires a VC-1 track to the non-NAL path.

That last one matters more than it looks. The four TsMuxer-level tests set
the flag themselves, so deleting the set_nal_video loop from
M2tsStream::create left all 2366 tests passing — the exact mutant the
finding named went undetected until a test drove the real wiring. It now
also catches the subtler mutant of widening the matches! arm to include
Vc1.
2026-07-29 17:54:27 -07:00
Matthew Jackson d09ed76e07 Promote AACS unit encryption to real API, and assert decrypt byte-exactly
encrypt_unit becomes public library API rather than a #[cfg(test)] helper.
Authoring an encrypted disc image is a legitimate use of this crate, and
the capability was already written four times over: a pub(crate) test-only
copy in aacs/content.rs plus three hand-rolled duplicates in decrypt.rs,
sector/decrypting.rs and disc/extract.rs. All four now call one function,
removing ~110 lines of duplicated cipher code that could drift from
decrypt_unit independently.

It mirrors decrypt_unit's purity contract: crypto only, no encrypted-flag
handling, because where that flag lives is container-specific (CPI bits in
byte 0 for BD-TS, elsewhere for HD-DVD-PS). Callers set the flag BEFORE
encrypting — bytes 0..16 are the key seed left in plaintext, so touching a
header byte afterwards changes the key a decryptor derives. That footgun is
documented at the function and at every call site.

Two tests pin it: an exact round trip through both directions, and the one
place the pair is deliberately asymmetric — decrypt_unit restores
all-zero-on-disc packets to zero, and the test proves an all-zero plaintext
packet enciphers to non-zero bytes so it is never mistaken for padding.
That asymmetry was previously only prose.

Two decrypt tests were also weaker than their own names:

  * aacs_clear_trailing_partial_passes_through asserted only is_ok(), so a
    mutant corrupting the clear partial while returning Ok passed. It now
    snapshots the buffer and asserts byte equality, matching the
    none_keys_is_noop pattern already in the file.
  * aacs_decorator_decrypts_encrypted_unit_via_map checked only that 0x47
    reappeared at the 192-byte stride, leaving corruption in the other 6112
    bytes undetected. The plaintext is fully known, so it now asserts
    byte-exact recovery against it.

Both were verified red first by mutating the production path.
2026-07-29 17:50:24 -07:00
Matthew Jackson f76688a0dc Make the ddts speaker mask agree with its own channel count
The ddts box declares both a channel count and a 16-bit speaker mask, and
a decoder may trust either. dts_channel_layout ended in a `_ => 0x0007`
catch-all describing five speakers (C + L/R + Ls/Rs), so for AMODE 6, 7,
and 10 through 15 the box contradicted the count DTS_AMODE_CH declared
alongside it — provoking a downmix or an outright decode error.

AMODE 6 (L + R + S) is the reachable case: its S is a single
centre-surround, not the Ls/Rs pair, so it is three channels and 0x0012,
not four and 0x0006. AMODE 13/14/15 do not occur on retail media, which
ships a 5.1 AMODE-9 core plus an extension substream.

Replace the catch-all with the full 16-entry ETSI TS 102 114 mask table.
Every entry is cross-checked against DTS_AMODE_CH by counting the
speakers its bits imply: sixteen independent constraints, all satisfied,
and that table is itself pinned to the per-AMODE channel counts in ETSI TS 102 114.

AMODE is a 6-bit field, so the reserved 16..=63 are reachable from a
malformed stream. The old `unwrap_or(6)` invented a channel count for
them that no mask could match; parse_dts now refuses those frames rather
than guessing a layout it cannot name.

One existing fixture placed the ext-sync pattern at f[4..8], which
incidentally set f[7]=0x25 → AMODE 20. That test is about where the
pattern sits, not about AMODE, so the frame is now spec-legal (AMODE 9,
48 kHz) with the pattern moved into the payload proper.
2026-07-29 17:43:38 -07:00
Matthew Jackson 840cb9aef0 Drop the comment that promised tests this crate no longer has
The bisect ReadAction regression tests moved to freemkv-engine with the
recovery strategy, but their explanatory block stayed behind — seventeen lines
describing a `let _ = handle_read_error(..)` bug and asserting "the tests below
prove the required ReadAction values are produced". There are no tests below it;
handle_read_error is not even resolvable here any more. Anyone auditing whether
that bug is still guarded would read this and conclude yes. The block moved to
the engine alongside the tests it describes.
2026-07-29 16:36:22 -07:00
Matthew Jackson f3e80c8499 Route conduct reports through GitHub instead of a private address
The enforcement contact was an address on a domain used for internal
infrastructure, which does not belong in a public repository. GitHub needs no
mailbox to exist and is reachable for anyone who can read this file.
2026-07-29 16:07:14 -07:00
Matthew Jackson c812a32f3d mux: fix keyframe signalling for BlockGroup frames
Inside a Matroska BlockGroup the SimpleBlock 0x80 keyframe bit is
reserved and is always written as 0; keyframe-ness is carried only by
the presence or absence of a ReferenceBlock child. Both halves of the
round-trip got this wrong:

  - the reader skipped past ReferenceBlock and read the reserved bit,
    so every BlockGroup frame came back as a non-keyframe;
  - write_block_group discarded its `keyframe` argument and never
    emitted a ReferenceBlock, on the assumption that only intra frames
    (PGS subtitles) reached that path.

The MPEG-2 parser stamps a per-frame duration on I, P and B pictures
alike, so all MPEG-2 video is written as a BlockGroup. That makes the
assumption false and left no video frame looking like a keyframe on
read-back. Downstream, mkv:// -> m2ts:// dropped every video frame (the
TS muxer discards non-key video until the first keyframe) while still
reporting success, and mkv:// -> mkv:// and the stdio round-trip failed
E6008, because the MKV muxer opens a cluster only on a track-0 video
keyframe and so wrote nothing at all. HEVC was unaffected: it carries no
per-frame duration, so it takes the SimpleBlock path where the flag bit
is authoritative.

Verified on a real CSS DVD: 841 keyframes out of 11440 video packets
survive a re-mux, matching the I-picture count in the source bitstream.

Also stop running non-NAL video through the Annex-B converter. MPEG-2
and VC-1 elementary streams are already start-code framed, so
length-prefix conversion corrupts them. TsMuxer takes a per-track
nal_video flag, defaulting to the previous behaviour, which M2tsStream
sets from each video stream's codec.
2026-07-29 15:29:48 -07:00
Matthew Jackson eedd27e352 mux: document that the Url arm selects via InputOptions.selection
The Url mux path prunes streams inside input() via InputOptions.selection;
MuxOptions.selection only applies on the File/Session arms. A Url-source caller
must set InputOptions.selection — noting it so a future caller does not put the
selection on MuxOptions and silently keep every track (the bug the GUI hit).
2026-07-28 18:40:47 -07:00
Matthew Jackson d8e5b97c86 1.6.0: remove recovery strategy (moved to freemkv-engine) + trim dead surface
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).

- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
  Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
  classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
  three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
  READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
  consumer), and the DriveSpeed enum (its one live use — set max drive
  speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
  decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
2026-07-28 15:35:19 -07:00
MattJackson 0151e199ef changelog: 1.6.0 section (layering API, mux fixes, engine relocation, stream selection); date 1.5.2 2026-07-28 13:54:42 -07:00
MattJackson ea99c82e32 mux: end-to-end test that stream selection drops excluded-PID frames
A title declaring two audio PIDs, pruned to one via StreamSelection::apply
before build_iso_pipeline, must never surface a frame from the excluded PID.
Proves the declaration-driven seam end-to-end through the full highway (read ->
demux -> codec parse): the demuxer is built from the pruned title.streams, so
the excluded PID is untracked and skipped, and both track headers and frames
follow the pruned list. Builds on the existing synthetic-TS harness.
2026-07-28 13:36:36 -07:00
MattJackson 8822c29905 disc: add DiscTitle audio_streams/subtitle_streams/video_streams accessors
Typed iterators over the audio/subtitle/video streams, cleaner than matching on
the Stream enum for the common iterate-the-tracks case (stream selection, the
desktop UI info panel, disc-info). Additive; all lib tests pass on 1.86.
2026-07-28 13:35:12 -07:00
MattJackson 0ee8341aea mux: StreamSelection primitive + apply sites for per-title stream selection
The demux pipeline is declaration-driven off DiscTitle.streams (build_demux_state,
DiscStream::new, and the MKV writer all key off that list), so 'which streams to
keep' is already a pipeline capability with no public knob. This adds the knob:

- mux/select.rs: StreamSelection { audio, subtitle: PidFilter::All | Only(Vec<u16>) }
  + apply(&mut DiscTitle): keep Video always, keep Audio/Subtitle whose PID the
  filter lists, prune the rest (and the parallel codec_privates in lockstep);
  error SelectionPidUnknown on a listed PID absent from the title (fail loud, not
  a silently-missing track). Pure; 6 unit tests. Re-exported at crate root.
- Error::SelectionPidUnknown (E6014).
- MuxOptions gains  (+ derives Default now) applied in mux_stream's
  Iso/Session arms before the highway/DiscStream builds demux state (and before
  probe_and_remap's DVD AC-3 PID rewrite). InputOptions gains  applied
  in input()'s iso arm right after the title-index bounds check.

PIDs not languages -- language->PID is caller/engine policy. Default All/All is a
no-op (apply gated on !is_all()), so the no-selection path is byte-identical:
nothing below the title-finalization line changes (ts/ps/demux_thread/
pipelined_stream/mkv/disc untouched). All 2488 lib tests pass on 1.86.
2026-07-28 13:14:40 -07:00
MattJackson d22d09c898 error: add is_disc_level_no_key classifier (re-exported)
Distinguishes a WHOLE-DISC key failure (E_NO_DISC_KEY / E_KEYDB_LOAD /
E_AACS_NO_KEYS -- every title fails identically) from a per-title skippable
stub. The engine's multi-title loop uses it to fail-fast on the first no-key
title instead of iterating all N. Additive.
2026-07-28 12:45:58 -07:00
MattJackson 43564d5752 error: re-export is_halt + is_skippable_title_stub at crate root
The engine's multi-title rip loop classifies per-title mux failures (halt vs
skippable stub vs hard) using these typed classifiers instead of E-code string
matching. Additive; no behavior change.
2026-07-28 12:31:40 -07:00
MattJackson 508a2c3376 disc: promote locate_ranges to pub (engine multipass reads it)
Small pub promotion + fmt one-lining. The relocated multipass progress
reporting in freemkv-engine needs locate_ranges externally. No behavior change.
2026-07-28 12:09:38 -07:00
MattJackson 45a16991ff drive: promote extract_scsi_context to pub
Small pure error->(status,sense) introspection helper the relocated
sweep/patch will need externally. Same category as the prior WritebackFile/
resolve_content_key_map promotions -- infra, not policy. No behavior change.
2026-07-28 11:41:49 -07:00
MattJackson ca0daaea07 io: promote WritebackFile to pub
Bounded-cache buffered File replacement used across mux/extract/sweep/patch
-- general I/O infrastructure, not recovery policy. freemkv-engine's
relocated sweep/patch need to construct it directly once those methods
leave this crate. No behavior change.
2026-07-28 11:36:43 -07:00
MattJackson a02bbbdd24 scsi: promote SenseFamily to a lib-level SCSI-fact primitive
Moved SenseFamily::from_sense_key + is_wedge_family from disc/read_error.rs
into scsi/mod.rs (with its own tests) and re-exported at the crate root.
This is pure SCSI sense-code classification -- objective hardware fact, zero
recovery-policy opinion -- so it belongs in the library primitives, unlike
the retry-DECISION state machine (ReadCtx/PassSummary/ReadAction/
handle_read_error) built on top of it, which is freemkv's specific recovery
strategy and is moving to freemkv-engine next.

disc/read_error.rs and disc/section_recover.rs now import SenseFamily from
crate::scsi instead of defining/re-exporting their own copy. No behavior
change. Precommit green on Rust 1.86 (fmt+clippy+test).
2026-07-28 11:32:13 -07:00
MattJackson ba8114f29b disc: promote resolve_content_key_map + encrypted_content_ranges to pub
The upcoming freemkv-engine crate needs both to build the multipass
sweep/patch recovery strategy externally over Disc's public API. Everything
else sweep/patch touch on Disc was already pub; these were the only two
gaps. No behavior change -- visibility only.
2026-07-28 11:26:37 -07:00
Matthew Jackson 8421c227cd mux: fix DTS core-header false-drops + close TrueHD/mux gate coverage
DTS core decodability gate (core_header_drop_reason) — full ETSI TS 102 114
spec-conformance sweep against a reference decoder the spec core-header rules and
a reference decoder parse_frame_header:

- deficit_samples: only require ==32 for NORMAL frames (FTYPE==1). A
  TERMINATION frame (FTYPE==0, the last frame of a stream) legitimately
  carries fewer and is fully decodable; the old unconditional check dropped
  it on every stream that ends on one — a guaranteed per-track silence gap.
  Matches a reference decoder (normal_frame && deficit != DCA_PCMBLOCK_SAMPLES) and
  a reference decoder (branches on normal_frame).
- reserved bit (after RATE): both reference decoders SKIP it (a reference decoder
  skip_bits1, a reference decoder bits_skip1 "Reserved field") and never reject on it.
  Rejecting was a false-drop that silenced any real stream whose encoder
  set the bit. Relaxed to read-and-discard; DropReason::ReservedBit removed.

Swept and confirmed spec-correct as-is (no change): npcmblocks multiple-of-8,
frame_size>=96, audio_mode>=16 (a reference decoder-permissive), sample-rate validity
table (matches avpriv_dca_sample_rates incl 96k/192k at 14/15), LFE flag==3
invalid, PCMR bits table (matches a reference decoder sample_res {16,16,20,20,0,24,24,0}).
Bit-read order verified field-by-field against a reference decoder. bit_rate is left
unvalidated (lenient, never-false-drop direction) as before.

Tests: termination frame with small deficit is kept; normal frame with bad
deficit is dropped; reserved-bit-set frame is kept. make_bad_dts_core now
uses an invalid LFE flag (duration-neutral) instead of the relaxed reserved
bit.

TrueHD: add coverage for the EXTENDED major-sync header CRC path (ms[25]&1,
mshdr=28+2+2n) — previously zero-tested, the exact path a shipped endianness
bug once used to silently drop whole 7.1/Atmos tracks. Trailer is an
independently-computed oracle (separate CRC-16/0x2D, anchored to the 0x4FF7
catalogue value, NOT crc16_mlp), stored little-endian; test asserts accept,
body-corruption reject, and big-endian-trailer reject.

mux driver: extract the finish completion mapping into pure mux_run_completed
so the finalize_failed -> completed=false branch (reachable only via real
write-thread wedge timing) is unit-tested; add an out-of-range
MuxInput::Session title_index test asserting a clean Error::MuxTrackRange
(E9011) instead of a panic.
2026-07-24 10:28:21 -07:00
Matthew Jackson b79ff71b43 Fix read-fault misclassification, DTS AMODE channel table, and untestable guards
- resolve_fmts_key_map: distinguish a genuinely-not-FMTS disc from a
  transient live-drive read fault. read_filesystem now returns the new
  Error::UdfNotFilesystem for a deterministic tag/format mismatch (no AVDP,
  no partition descriptor, no FSD); resolve maps only UdfNotFilesystem (fs)
  and UdfNotFound (.tbl absent) to Ok(None), and PROPAGATES DiscRead / other
  I/O faults so a marginal AACS 2.1 disc fails loud instead of silently
  dropping forensic content under a base-Unit-Key-only map.

- DTS_AMODE_CH (mp4/audio.rs): extend 10→16 entries
  {1,2,2,2,2,3,3,4,4,5,6,6,6,7,8,8} (the spec per-AMODE channel table / ETSI TS 102 114) so
  the spec-legal high AMODEs that now pass the decodability gate declare
  their true channelcount (AMODE 13→7, 14/15→8) instead of a truncated 6.

- session.rs resolve_keys "called before scan" guard is now testable:
  from_parts_for_test takes Option<Disc>; added a test that a disc-less
  session returns a clean DeviceNotReady Err rather than panicking.

- mp4/read.rs: a track with samples but a missing/malformed stts (mandatory
  per ISO/IEC 14496-12) is dropped rather than emitting all-zero timestamps,
  matching the existing stco/stsc guards; all-tracks-dropped → Mp4Invalid.

- Remove the inert MuxInput::Iso.key_map field (the Iso path re-derives its
  map inside build_iso_pipeline); the live path keeps Live.key_map.

All four fixes are mutation-verified.
2026-07-24 09:59:33 -07:00
Matthew Jackson 9b3e281f4d mux: distinguish FMTS phase-probe read fault from wrong key; cover multi-CPS
Fix 1 (correctness): resolve_fmts_key_map's per-index phase probe read a
single representative segment via read_unit (whose read_sectors(...).ok()?
swallows read errors into None) with no fault fallback. A transient live-drive
read fault (e.g. NOT READY 2/04/3E on the BU40N) made every probe read return
None, giving even==0 && odd==0 — indistinguishable from a genuine wrong key —
so resolve_tie_phase returned FmtsKeyMissing and aborted the entire multi-title
rip even though the forensic index keys were valid and in hand.

Extract the probe into probe_index_phase, which returns Phase / WrongKey /
ReadFault. It mirrors the anchor loop's tolerance: it tries multiple same-index
segments and only concludes WrongKey once a read actually succeeded and decrypted
to no clean parity. If every read of every same-index segment faults it returns
ReadFault; the caller then leaves the phase unresolved so the range-builder
defaults to Phase::All (decrypt both parities, demux drops the garbled alternate
half) instead of aborting. A wrong key can never be masked as a read fault:
ReadFault requires that not a single read succeeded, so zero decrypt evidence.

Fix 2 (test coverage): resolve_mux_key_map's multi-CPS branch was only exercised
with an all-zero source, so pick(), the KeyFetch cold path, and the fail-loud
DecryptFailed guard never ran. Add tests over real AACS ciphertext (via
aacs_encrypt_unit_for_test) covering pick() selecting the correct pool index,
the DecryptFailed guard firing when a clean sample matches no held/fetched key,
and the fetch cold path recovering a missing unit key. Mutation-verified.

Fix 3 (doc): move the reader_event_fn EventKind->MuxEvents mapping doc off
session_mux_keys onto reader_event_fn.
2026-07-24 09:28:09 -07:00
Matthew Jackson e0456c72da mux: thread halt into live AACS key-map resolution; cover Session arm
Round-2 follow-ups to 6d6e60f (inline base-map resolve on the live
single-pass Session/Live mux arms).

Fix 1 (halt threading) — the inline resolve chain sampled ciphertext off
the LIVE drive with no cancel token, so an operator /api/stop during key
resolution was not honored (the FMTS probe can issue hundreds of reads,
each able to stall to the 60s SCSI recovery timeout — violating the
"don't hammer a struggling live drive" rule). Add an optional
`halt: Option<&Halt>` to `resolve_mux_key_map`, `resolve_fmts_key_map`,
`resolve_inline_base_map`, and `Disc::resolve_content_key_map`, and poll
it at each loop boundary (FMTS anchor + per-index probe loops, multi-CPS
extent loop) — returning Err(Halted) promptly. Live/Session arms pass the
driver's halt; sweep/patch pass their own token (via Halt::from_arc);
file-backed probe/ISO callers pass None. Tested with a pre-cancelled halt
(Err Halted, no extent sampling) and a None-halt no-abort case;
mutation-verified (dropping the extent-loop check → Ok, not Err).

Fix 2 (Session-arm coverage) — the MuxInput::Session arm ran the same
resolve→install→decrypt sequence as Live but had NO end-to-end test
(DiscSession only exposed open(), which needs live hardware). Add a
#[cfg(test)] DiscSession::from_parts_for_test (injected reader + scanned
disc, no Drive), an end-to-end AACS decrypt test through the Session arm
(mutation-verified: dropping with_key_map → mux aborts), and a
missing-reader clean-error (not panic) test.

Fix 3 (cleanups) — io_error_code: remove the unreachable typed-Error
downcast branch (From<Error> for io::Error stringifies; no path builds an
io::Error holding a typed Error), keeping the stringify parse is_halt /
is_skippable_title_stub rely on. Add a resolve_keys_for test covering the
largest-title sampling branch. Document the patch wedge-exit coverage gap
(TODO) in passn_handler_ab.rs.
2026-07-24 09:04:19 -07:00
Matthew Jackson 1c7ccd5a85 mux: resolve+install AACS key map on live single-pass (Session/Live)
Under the map-only decrypt model an AACS DecryptingSectorSource decrypts
nothing until a key map is installed; with no map the AACS arm fails loud
with DecryptFailed on the first content unit. The two inline live-mux arms
in mux_stream did not install one:

  - MuxInput::Session (freemkv `rip disc://…mkv`) installed NO map at all.
  - MuxInput::Live (autorip non-FMTS single-pass) installed only a
    caller-supplied forensic FMTS map, which is None for a plain AACS disc.

So EVERY plain AACS Blu-ray/UHD ripped via the live single-pass path failed
DecryptFailed on the first content read. This predates the mux_stream
refactor: the bug was introduced with the map-only decrypt model, and the
pre-refactor CLI likewise built DiscStream::new without with_key_map.

Fix: add resolve_inline_base_map, the inline counterpart to what
build_iso_pipeline does for the file highway. Both arms now resolve the
AACS map off the reader (borrow to sample, then move into DiscStream) and
install it via with_key_map before any read. DVD/CSS keeps DecryptKeys::None
(DiscStream's per-title CSS crack owns it); clear/raw resolve to no map.
Session passes session.key_fetch() so a multi-CPS/orphan unit can still be
recovered; a caller-supplied FMTS map (autorip) is used verbatim, never
re-resolved.

Tests: an end-to-end MuxInput::Live mux over a genuinely-AACS-encrypted
synthetic unit now decrypts and finalises (mutation-verified: dropping the
resolve/install makes the mux abort). Adds a pub(crate) test-only AACS
encrypt helper so the mux test can build a real encrypted fixture, and a
gating test for resolve_inline_base_map (AACS→map, CSS/clear/raw→none).
2026-07-24 08:34:51 -07:00
Matthew Jackson 3bd2fd23b0 Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just
  0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements;
  only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are
  decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal
  multichannel core frames as undecodable, silencing recoverable audio. Add a
  regression test (literal 0..16 range) that fails if the bound reverts to 10.

- keysource: only memoize a NEGATIVE (empty) key-fetch result when every source
  genuinely ran and none held the key — never when a source Err'd (network down,
  unreachable). A transient outage was being cached as a permanent "no key" for
  the fingerprint, permanently dropping a unit that could be recovered once the
  source came back. Thread an `errored` flag out of the drivers and gate the
  cache insert on it. Tests cover both the recover-after-outage case and that a
  genuine absence is still memoized.

- pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS
  display sets through the full demux -> parse -> observe -> apply path, both a
  forced verdict landing and a non-forced verdict clearing a vendor flag.

- mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD).

- scan_iso test: add independent fixture expectations (volume id) so the parity
  test is no longer purely tautological against a re-run of the same composition.
2026-07-24 08:32:37 -07:00
Matthew Jackson c00384d4df mux/driver: map a mid-read halt to completed=false (Stop is not a failure)
The finish stage and the highway path already treat Error::Halted as a clean
operator stop (completed=false), but the header pump (`stream.read()?`) and the
frame pump (`Err(e) => return Err(e)`) propagated it as a hard error. Since slow
recovery reads dominate wall-clock, a Stop almost always lands mid-read, so it
surfaced as a mux failure instead of a resumable incomplete outcome — breaking the
driver's own "a clean operator stop is not an error" contract and diverging from
the ISO/Url highway (which returns Ok(None) on halt).

Both read arms now route Error::Halted to the completed=false path. Adds
error::is_halt(&io::Error) (typed, mirrors is_skippable_title_stub) as the check.
Two regression tests cover a halt landing mid-header-read and mid-frame-read.
2026-07-24 07:52:42 -07:00
Matthew Jackson e8c151792d mux: make write-pipeline send deadline per-call via MuxOptions
The 60s per-frame send deadline the mux driver applies in drive_mux was
hardcoded. That is correct for autorip (its hard watchdog + container-restart
model wants a wedged sink surfaced as a bounded per-frame timeout), but wrong
for the CLI's interactive stdout:// / network sinks: the old inline
output.write() had no deadline, so a slow-but-alive downstream (paused pager,
backpressured pipe, slow peer) was spuriously reported as an interrupted mux
after 60s of backpressure.

Add MuxOptions.send_deadline: Option<Duration>. Some(d) keeps a hard d timeout;
None means no backpressure timeout (block on a live-but-slow sink). None is
resolved to an effectively-unbounded-but-Instant-safe NO_SEND_DEADLINE at the
effective_send_deadline seam. Ctrl-C / halt stays responsive under None because
send_with_halt slices its wait at POLL_INTERVAL and re-checks the halt token
every slice regardless of the deadline value.

Unit-test the routing seam (Some(d) -> d, None -> unbounded, never collapses to
60s) and that MuxOptions carries the knob.
2026-07-24 02:38:30 -07:00
Matthew Jackson 20b36229a5 mux: restore three drive_mux regressions from the mux hoist
The mux drive loop hoisted out of autorip/CLI into libfreemkv's
drive_mux dropped three guarantees the hand-rolled loops enforced:

A. Header-buffer OOM cap. The header pump buffered every PES frame
   until codec_private resolved with no byte accounting, so a title
   whose video header never resolves (damaged/undecryptable HEVC on
   marginal UHD) buffered the whole 30-90 GB title into RAM until
   OOM-kill. Restore the 512 MiB HEADER_BUFFER_CAP_BYTES cap autorip's
   run_mux enforced; once exceeded, fail fast with the same
   Error::MkvInvalid the headers-never-resolved gate already returns.

B. Watchdog feed during the header-frame drain. After headers resolve
   the buffered header frames are flushed to the sink, but
   on_write_progress (the sole feed for autorip's hard watchdog) fired
   only in the steady-state loop, leaving the watchdog unfed during a
   long/slow drain and able to false-escalate. Feed on_write_progress
   per buffered frame during the drain, matching the steady-state path.

C. Per-title CSS key on MuxInput::Session. The Session arm set keys via
   disc.decrypt_keys() unconditionally, which returns the largest
   title's VTS CSS key; muxing a bonus title in a different VTS
   descrambled with the wrong key (DiscStream's per-title crack only
   fires on DecryptKeys::None). Special-case DVD to None so the pipeline
   cracks the correct per-title key, matching resolve.rs::input(). AACS
   and clear discs still resolve from decrypt_keys().

Adds mutation-verified synthetic tests for each (no live drive).
2026-07-24 02:27:57 -07:00
Matthew Jackson 71aad385c7 mux: add MuxInput::Live for the inline live-drive single-pass path
STEP 4c-ii (layering refactor): give `mux_stream` a live-drive source so
autorip's live single-pass mux stops hand-rolling `DiscStream::new`.

`MuxInput::Live` is the live analogue of `MuxInput::Iso`: it wraps the
raw `Box<dyn SectorSource>` in the INLINE `DiscStream` (the same
constructor the `Session` arm uses — NOT `build_iso_pipeline`, the
prefetch highway), so a consumer's adaptive batch-retry in
`DiscStream::fill_extents` still fires on a bad live-drive sector. Unlike
`Session`, `Live` carries an optional pre-resolved forensic `AacsKeyMap`
that is applied via `DiscStream::with_key_map` before any read — required
for single-pass FMTS correctness (read only our-phase units; decrypt the
forensic segment with the mapped key). The consumer resolves keys as its
own policy and hands the banked material in; the driver re-resolves
nothing.

Unit test drives `MuxInput::Live` with a recording `SectorSource` and a
phased forensic key map, asserting the alternate-phase (odd) units are
never fetched — proof the arm builds the inline `DiscStream` and applies
the key map. Mutation-verified: dropping the `with_key_map` call reads the
dropped odd-phase LBAs and fails the assertion.
2026-07-24 01:53:36 -07:00
Matthew Jackson ec5b10f83a mux/driver: split on_progress; stage live drive as session reader
Split the ambiguous MuxEvents::on_progress(bytes, total) into two
callbacks — on_read_progress(bytes_read, total) and
on_write_progress(bytes_written, total) — so a consumer no longer has
to guess which side of the pipeline a progress figure came from. The
CLI drives its bar from the write side; autorip from the read side.
The reader EventFn's BytesRead now maps to on_read_progress; the
per-frame emit in drive_mux to on_write_progress. Both keep empty
defaults; NoopEvents and the driver tests are updated to match.

Add DiscSession::stage_drive_as_reader so the live single-pass path can
run through MuxInput::Session: the owned Drive (itself a SectorSource)
moves into the reader slot for mux_stream to take. The drive is now
held as Option<Drive> with the device path cached up front, so the
mux driver can still name the device after the drive has been staged;
the driver's Session arm uses the new device_path() accessor.
2026-07-24 01:01:34 -07:00
Matthew Jackson 5181f6ce19 mux/driver: forward reader-side events through Arc<dyn MuxEvents>
mux_stream took events as &dyn MuxEvents and passed None for the
constructors' EventFn, because that hook is Box<dyn Fn(Event)+Send+
'static> and a borrowed handle cannot satisfy 'static. The reader-side
events (BytesRead progress, SectorSkipped, BatchSizeChanged, ReadError)
therefore never fired.

Take events as Arc<dyn MuxEvents> (MuxEvents: Send+Sync+'static) and
clone it into a 'static EventFn (reader_event_fn) that translates each
real EventKind into the matching MuxEvents call. Wire it into
build_iso_pipeline (ISO path) and DiscStream::on_event (live path).
The write-side on_progress in drive_mux is unchanged.

Add a translation unit test (all four variants) and an end-to-end
mux_stream ISO test that asserts the read-side BytesRead reaches the
Arc; passing None regresses the latter.
2026-07-24 00:37:03 -07:00
Matthew Jackson a711ee1d00 mux: add high-level mux_stream driver (layering step 4a)
Add the shared decrypt+mux driver both consumers hand-roll today, as a
library-only API. `mux_stream(input, dest_url, opts, halt, events)` runs
the construct -> headers gate -> open sink -> pump -> finish pipeline:

- MuxInput::{Session, Iso, Url} selects the source. The file/ISO path
  calls the untouched build_iso_pipeline highway (zero added copies —
  the driver reads frames exactly where consumers call stream.read());
  the live path builds a DiscStream; a URL source goes through input().
- chapters:// / json:// metadata sinks short-circuit BEFORE the header
  pump/gate. They write their whole file from the scanned title and need
  no codec headers; the CLI placed this after the gate, so a metadata
  export on a title whose video headers never resolved failed with
  MkvInvalid. Fixed by construction.
- The header gate refuses a stream with no resolved codec_private
  (MkvInvalid); the zero-output gate refuses an empty/undecryptable drain
  (NoStreams); a halt mid-pump yields completed=false, never a success.
- Frames are written through a WRITE_PIPELINE_DEPTH consumer pipeline so
  the latency-bound sink write overlaps the next read.

Add error::is_skippable_title_stub(&io::Error) so consumers can drop the
E7023/E6008 string-match, and DiscSession::take_reader for the live arm.

Driver body unit-tested in isolation via a synthetic Stream against
null:// / chapters:// / json:// sinks (short-circuit, header gate,
zero-output gate, halt, happy path); each gate mutation-verified.
Consumers are NOT migrated yet (steps 4b/4c).
2026-07-24 00:27:47 -07:00
Matthew Jackson f2b7cc9bdd Add DiscSession::resolve_keys and resolve_keys_for for base AACS keys
Hoist the AACS base-unit-key resolution glue the CLI and autorip hand-rolled
around the existing primitives into one place: sample the largest title's
ciphertext, run the ordered key sources first-valid-wins, bank the winning
unit keys onto the disc, and build the read-time KeyFetch.

- resolve_keys_for(reader, disc, sources): the free-function core, over any
  SectorSource (live drive or a scan_iso file reader). Returns the structured
  ResolutionTrace plus the KeyFetch; a non-AACS disc is a no-op (empty trace,
  no fetch).
- DiscSession::resolve_keys(sources): samples through the session's reader
  (staged file reader if present, else the live drive), banks onto the
  scanned disc, and retains the KeyFetch on the session (key_fetch()
  accessor) for a later mux.
- KeySourceFactory: the Arc source factory the consumer supplies (libfreemkv
  builds no key sources itself).

Sampling is skipped when the factory yields no sources (resolution is a miss
regardless) — no wasted disc read. Tests cover banking, the no-key path
still building a fetch, and the CSS/None non-AACS no-op.
2026-07-24 00:09:28 -07:00
Matthew Jackson 6f53767e8b Add scan_iso entry point for file-backed ISO scans
Introduce libfreemkv::scan_iso(path, opts) -> (Disc, Box<dyn SectorSource>),
the file-backed counterpart to DiscSession::scan. It is the single place
that opens a FileSectorSource, reads its capacity, and runs Disc::scan_image,
returning the scanned Disc plus a reusable reader over the same image so
consumers stop hand-rolling that triple.

Add an integration test that materialises a minimal synthetic UDF image to a
real file, asserts scan_iso matches the manual open+scan_image composition,
and confirms the returned reader is still usable (capacity + sector read).
Also covers open-failure and scan-failure error propagation.
2026-07-23 23:48:50 -07:00
Matthew Jackson 7ed798e386 libfreemkv: add DiscSession (drive open/bring-up hoist, step 1)
New src/session.rs providing DiscSession, DeviceTarget, and KeySpec,
re-exported from lib.rs. DiscSession::open runs the drive open +
advisory wait_ready/init/probe_disc bring-up (owning the Drive by
value); identify()/scan() are split so consumers keep their own
UI/TMDB sequencing. KeySpec carries consumer-built credentials +
key_sources which scan() forwards into ScanOptions without clobbering
caller-set fields (the library derives no certs and reads no keydb).
Unit tests cover the KeySpec->ScanOptions forwarding.
2026-07-23 23:37:20 -07:00
Matthew Jackson b9568242df libfreemkv: 10-phase release audit fixes (v1.5.2..HEAD)
Multi-round audit of the decrypt/AACS/mux-codec refactor. Fixes, in
descending severity:

- mux/mp4/read.rs: bound untrusted-input allocations. `sample_budget`
  now also capped by file_len (a fixed-size stsz claiming count=u32::MAX
  can't inflate the Vec<SampleRef> past the file's own size); trak scan
  capped at MAX_TRACKS matches; find_box() takes only the first match
  (cap=1) instead of materializing every match. Removes dead find_boxes
  wrapper.
- disc/mod.rs: merge_content_key_ranges now UNIONS same-key overlapping
  ranges (coverage-preserving) instead of dropping the non-overlapping
  tail, which silently left encrypted LBAs uncovered -> ciphertext
  passthrough in the whole-disc sweep/patch map. Different-key overlap
  (malformed) still dropped to keep the set disjoint.
- sector/decrypting.rs: remove dead unit_key_idx field + with_unit_key_idx
  setter (vestigial from the pre-keymap trial-decrypt design; AACS is
  map-only now). Fix stale docs.
- decrypt.rs / resolve.rs / error.rs / extract.rs: doc/comment drift from
  the refactor (AacsKeyMap positive-map semantics, resolve_mux_key_map doc
  reattachment, decrypt_sectors_in_content legacy-alias, E_MP4_INVALID
  meaning, multi-CPS orphan by-design note).

Test coverage (all mutation-verified real):
- DTS NeedMore force-flush buffer bound; FLAC/MPEG-audio PTS carry-forward;
  mp4 mdhd timescale=0 divide-by-zero guard, MAX_TRACKS cap, sample-count
  file_len bound, MAX_ALLOC_BYTES cap under inflated file_len.
- resolve_fmts_key_map: extracted filter_addressable_segments,
  resolve_tie_phase, fill_base_key_gaps as pure behavior-preserving
  helpers, each unit-tested (segment filter, phase-tie arms, gap-fill
  gaplessness over every extent).
2026-07-23 22:09:04 -07:00
Matthew Jackson 8ac18fa631 changelog: document 1.5.2 TrueHD/FMTS/extract/trailing-partial fixes 2026-07-23 16:09:03 -07:00
Matthew Jackson 197489fb7c Cache AacsKeyMap key indices; extract + test whole-disc range merge
- AacsKeyMap now derives its distinct key-index set once at construction
  (from_ranges_phased) instead of re-allocating/sorting it on every
  decrypt batch; key_indices() returns the cached slice.
- Extract the whole-disc content-map range merge out of
  resolve_content_key_map into merge_content_key_ranges and cover it:
  sort/disjoint, shared-clip dedup, overlap drop, adjacent-kept.
2026-07-23 16:00:00 -07:00
Matthew Jackson dc7dfc6041 Test hygiene: real AACS decrypt coverage, drop dead recovery fixtures
- Repoint three tautological AACS tests (which passed only via the
  unconditional-Err AACS arm of decrypt_sectors) at the real shipping
  path decrypt_sectors_mapped: out-of-range mapped key index, empty pool
  vs a non-empty map, and a scrambled encrypted trailing partial (the new
  guard) — plus a clear-trailing-partial pass-through and an explicit
  fail-loud safety-net test for AACS reaching the unmapped wrapper.
- Add the missing end-to-end AACS round-trip through the decorator +
  key map (aacs_decorator_decrypts_encrypted_unit_via_map) and a
  mapless-AACS-fails-loud decorator test — the exact class of bug the
  TrueHD probe shipped. Fills the // TODO: AACS round-trip test gap.
- Delete the now-dead recovery test fixtures (encrypt_aacs_unit_bad,
  AnyLbaUnit) left after the reactive key-fetch path was removed; keep
  encrypt_aacs_unit + FixedUnit, now used by the round-trip test.
2026-07-23 15:45:17 -07:00
Matthew Jackson 0acb326079 Fix FMTS per-title resolve, extract multi-CPS keying, trailing-partial guard
- resolve_fmts_key_map: filter segments to those addressable within THIS
  title's extents; a title with no forensic content (menu/extras playlist,
  or a different clip) returns Ok(None) and takes the base Unit-Key/CPS
  path instead of hard-failing FmtsKeyMissing. Previously the first
  non-forensic title aborted the entire whole-disc sweep
  (resolve_content_key_map iterates every title) and blocked muxing any
  non-main title.
- FMTS phase probe: an even/odd is_clean tie now only fails loud when
  BOTH halves are 0 (no clean decrypt). A both-clean tie is source-zero
  padding (is_clean is true for any key on all-zero content) — the key is
  valid, default Even, never abort the rip on a padding-heavy sample.
- extract_tree: multi-CPS discs now build the exact per-CPS content map
  (resolve_content_key_map) instead of a blanket key-0 map that silently
  mis-decrypted every secondary-CPS file into garbage. Single-CPS keeps
  the blanket key-0 map (one key opens every unit, incl. orphan clips).
- decrypt_sectors_mapped: a trailing partial unit that is inside a mapped
  range AND flagged encrypted in its clear seed now fails loud (a CBC
  fragment split across a boundary can't be decrypted) instead of being
  emitted as clear. New aacs_unit_seed_encrypted reads the flag on a
  partial.
- Correct the stale decrypt_sectors doc (AACS arm now always errors;
  AACS decrypts only via decrypt_sectors_mapped).
2026-07-23 15:36:15 -07:00
Matthew Jackson e632874665 Fix TrueHD channel-correction probe on AACS discs; delete dead reactive key-fetch path
The mux's TrueHD 7.1/Atmos channel-correction probe built its
DecryptingSectorSource with no key map. Since AACS now decrypts only via
the resolved map, the mapless probe read failed loud on the first unit
and the correction was silently skipped on every AACS disc — 7.1/Atmos
stayed understated as the MPLS-declared 5.1. The probe now resolves the
same up-front key map the mux read installs (non-fatal on failure).

Delete the reactive per-read key-fetch recovery path
(DecryptingSectorSource::with_key_fetch, the recovery field/cipher
scratch, and sector/recovery.rs). It was unreachable: AACS resolves its
full key map up front, so a mapless AACS read is a bug, not a
recover-mid-read case. KeyFetch itself stays — it is the up-front
resolver fetch used by resolve_mux_key_map.
2026-07-23 14:03:08 -07:00
Matthew Jackson 88e58bfc95 Decrypt is keymap-only: sweep/patch/extract, no AACS trial-decrypt
Every AACS decrypt now goes through the resolved key map (decrypt_sectors_
mapped): the map keys each content unit up front and a missing key fails at
resolve time. The old trial-decrypt path — try each held key per unit, keep
the first-tried plaintext on a miss — is gone; decrypt_sectors_impl's AACS
arm now fails loud (reaching it means a reader was built without its map,
which would silently apply a wrong key). CSS (self-descramble) and the clear
no-op path are unchanged.

Disc::sweep and Disc::patch resolve a whole-disc key map up front for a
decrypting pass (the fetch secures any missing CPS-unit key, fail-loud) and
decrypt via the map — clear nav/filesystem sectors are in no range and pass
through, so the separate content-range gate and the reactive per-unit
key-fetch recovery are no longer needed. extract_tree keys every unit with
the base Unit Key through the map (its encrypted-flag gate skips clear
files). Multipass sweeps stay --raw.

Removes the obsolete non-mapped-AACS trial/gate/recovery tests (the mapped
path and resolve fail-loud are tested directly).
2026-07-23 13:24:36 -07:00
Matthew Jackson 279ba0dd7c AacsKeyMap: a positive map — no key for a sector means pass through
The key map is now purely "these sectors use this key": entry_for returns
Option and an LBA in no range is left untouched (no default-decrypt-
everything fallback). resolve_mux_key_map builds explicit content ranges
for every case — single-CPS keys each content extent, multi-CPS keys each
extent with the key that opens it, and FMTS fills the non-segment content
with the base Unit Key so a whole-disc read decrypts content and passes
nav/filesystem through. Combined with the fail-loud resolve, a map can
never silently apply a wrong key, and clear sectors are never scrambled.

decrypt_sectors_mapped skips a unit with no map entry; read_plan keeps an
unmapped unit (pass-through content) and drops only alternate-phase
forensic units. Tests updated to the Option semantics.
2026-07-23 12:58:35 -07:00
Matthew Jackson 1eb6910bdb Harden mux + decrypt paths; fail-loud on unresolvable keys
mp4 demuxer (untrusted input): bound every allocation sized from a box
field (stsz/stco/stsc counts, stts/ctts run-lengths, per-sample and moov
sizes, plus an absolute cap so a sparse file can't inflate file_len);
guard the parse_stsd slice and a zero mdhd timescale; cap track count so
the per-track PID can't overflow; rewrite read_moov to handle size==0 /
size<8 / 64-bit largesize; parse esds/AudioSpecificConfig for AAC; write
tkhd duration in the movie timescale.

decrypt: resolve_mux_key_map now fails loud on an extent no key can
classify instead of inheriting the previous extent's key, so a keymap
never silently carries a wrong key; the sweep/patch key-fetch recovery
fails loud when a unit is still unresolved after the retry.

AACS: reject inverted forensic segments in both range builders; compare
the forensic index in u16 space so an out-of-range value can't truncate
onto a valid u8 index. RECOVERED_ERROR no longer latches the damage zone,
preserving the 30s wedge cooldown for a following hard error.

audio: AAC/MP2/MP3/FLAC carry the last PTS across a PES with no timestamp;
the DTS-HD extension-sync search is bounded to after the core; the MP4
16.16 sample-rate field saturates. demux_sink records the video reference
before the kind filter so audio:// / sub:// keep multi-clip PTS continuity
and the DELAY tag.

Remove a dead error variant and the AACS-unsupported-video code; codec
comments cite the primary format specs; assorted doc/naming fixes and
regression tests throughout.
2026-07-23 12:02:43 -07:00
Matthew Jackson e380e3b7c8 restore freemkv-unlock path dep for local dev (post-v1.5.2) 2026-07-22 22:19:42 -07:00
Matthew Jackson ff349fa61b v1.5.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-22 22:19:39 -07:00
Matthew Jackson 52fd0f733a CSS DVD: resolve the per-title key at read time, drop the scan-time crack
Every DVD read path — the file-backed mux highway (build_iso_pipeline) and
the live-drive single-pass DiscStream — now resolves the per-VTS CSS title
key through one shared step, css::resolve_dvd_title_key, cracked keylessly in
playback order from the title's own extents. Removes the earlier design that
reused a single scan-time key (meaningless for a per-VTS scheme) and muxed a
detection-miss disc's scrambled sectors as garbage.

- Disc::scan no longer cracks a key up front; it does only the CSS bus-auth
  read-unlock, hoisted before the UDF prefetch so scrambled small/menu VOBs
  no longer cost a rejected read each (CSS-DVD scan ~25s -> ~6s).
- An uncrackable title hard-fails (E7023) instead of passing ciphertext as
  plaintext; --raw skips the crack entirely; a Stop mid-crack surfaces Halted.
- DiscStream::new is now fallible and threads raw + halt.
- Fix a stale codec-parser doc claim (TrueHD/FLAC/MP2/AAC do gate via DropTally).
2026-07-22 22:15:27 -07:00
Matthew Jackson 5b03fd8ebc libfreemkv: 1.5.2 changelog reflects the shipped playback-order crack
The prior wording described the round-1 'self-cracking per-sector' approach
that the audit replaced; rewrite it to match what ships (per-title crack in
playback order + scan-key reuse + loud hard-fail on an uncrackable title).
2026-07-22 13:48:56 -07:00
Matthew Jackson c635190b0d libfreemkv: restore freemkv-unlock path dep on the branch tip
The last release left this pinned to the git tag v1.5.1; release.sh expects
a committed path dep on the branch tip (it swaps path -> git tag only in the
tagged commit, matching bdemu). Restores that state so a local full-workspace
build links the sibling freemkv-unlock, and release.sh's path->tag swap works.
2026-07-22 13:43:32 -07:00
Matthew Jackson 34c5293704 CSS DVD mux: per-title key via scan-key reuse + playback-order crack
A CSS DVD whose main title was mis-detected as unencrypted (the up-front
crack scanned the largest cell first and starved its budget in that cell's
clear prefix) muxed scrambled sectors as plaintext at exit 0. CSS leaves
the pack/PES header clear, so an un-descrambled sector muxes as a
structurally-valid but corrupt PES packet with zero loss reported.

decrypt_keys_for_title resolves a DVD title's CSS key two ways:
- Fast path: reuse the scan's cracked key when its crack_span covers this
  title's VTS (no re-read; on a live drive no second bus-auth).
- Crack: on a detection miss or a different VTS, crack from the title's OWN
  extents in a SINGLE scan in natural PLAYBACK ORDER (never largest-first).
  One scan = one CSS-locked early-bail, so a locked title is not re-hammered
  per cell against a live drive (hard rule #2); the 50k-sector budget is the
  same accepted bound the disc-wide scan uses. Cracked -> key; Unencrypted
  -> clear; ScrambledUncracked -> hard-fail.
ensure_title_decryptable hard-fails an uncrackable DVD title even when
detection missed, and passes a title that resolved its OWN valid key
regardless of the disc-wide css_error. descramble_region is unchanged from
v1.5.1 (validated-key seed).

Also rename the unlocker report's DVD entry CSS -> DVD. Bump 1.5.2.
2026-07-22 13:07:26 -07:00
Matthew Jackson bb59166e48 v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:04:24 -07:00
Matthew Jackson ea047b57d6 restore freemkv-unlock path dep for local dev (post-v1.5.1) 2026-07-20 17:02:14 -07:00
Matthew Jackson 909fe48628 v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:02:11 -07:00
Matthew Jackson da19280950 mux/codec/truehd: fix MLP major-sync checksum endianness (was dropping the whole TrueHD track)
Regression since the previous release, which added an MLP major-sync checksum
gate to drop genuinely-undecodable audio frames. The checksum itself was computed
with mismatched byte order: `crc16_mlp` is the correct crc_2D table (poly 0x2D,
MSB-first) but returns its two bytes in the OPPOSITE order to libavutil's
`av_crc`, and `mlp_major_sync_crc_ok` then folded in the pre-trailer word
little-endian while comparing the trailer big-endian. The net result never
matched a real major sync, so EVERY major sync was judged corrupt. That armed the
drop-forward on the first AU and, since no major sync ever validated to clear it,
collateral-dropped every following AU forever — the entire TrueHD track was
silently dropped. Its AUs then flushed only at mux end, so the track's blocks
landed physically after all the video: a decoder reading video+TrueHD had to
buffer the whole title to reach the first audio block and spiralled into an
unbounded memory runaway ("decoder ran out of memory"). Every TrueHD title
produced after the gate landed was affected; a title from the release before it
is clean. (The header-size parse — a frequent suspect for extended 7.1/Atmos
headers — is NOT the bug; it already matches ffmpeg's `mlp_get_major_sync_size`
byte-for-byte.)

Fix: compute the checksum exactly as ffmpeg's `ff_mlp_checksum16` —
`crc16_mlp(body).swap_bytes() ^ AV_RL16(word) == AV_RL16(trailer)`. Cross-verified
byte-exact against two real discs (a 7.1/Atmos title and a 5.1 title, independent
32-byte headers both validate). With the checksum correct, major syncs validate
and the drop-forward corruption protection works as intended.

Defence in depth: a major-sync checksum that STILL can't be validated (a genuinely
corrupt or as-yet-unparsed header) no longer arms the drop-forward until we hold a
validated baseline (`num_substreams` from a prior clean major sync) — so a single
bad header can never again silently drop an entire track.

Tests: the `finalize_major_sync` fixture now builds the checksum the corrected way;
a synthetic checksum-failed head major sync is kept, not dropped; the existing
baseline-then-corrupt drop-forward tests still pass. Verified end to end against a
real disc: the TrueHD track demuxes to a full, cleanly-decodable 48 kHz 8-channel
stream, interleaved with the video, instead of 0 bytes.
2026-07-20 16:58:06 -07:00
Matthew Jackson 2274423a6f aacs: parse HD DVD VTKF title keys at the spec's 36-byte stride
The HD DVD Title Key File (VTKF*.AACS) stores 64 title-key entries of 36 bytes
each — 1-byte BIFO + 3 reserved + 16-byte encrypted key + 16-byte binding MAC —
per AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, confirmed byte-exact
against real discs (Freedom VTKF090, Dukes VTKF000: every 36-byte slot has
BIFO=0x80, a clean key, and a 0xFF binding MAC).

The parser used a 32-byte stride (a 12-byte pad in place of the 16-byte binding
MAC) with flag-based termination. That aligns entry #1 (key at offset 132, where
both strides agree) but drifts +4 bytes per entry after it and never terminates
(the previous entry's 0xFF MAC reads as a set present-flag), so it recovered a
correct key only for single-CPS-unit discs and garbage for CPS unit >=2. Every
multi-title HD DVD (Freedom, Harry Potter) was affected.

Fix: 36-byte stride, iterate the fixed 64 slots, take slots whose BIFO AV_FLG
(bit 7) is set, key at offset 4, slot index = CPS unit (skip empty slots rather
than terminate so a gap can't renumber later keys), and never read the trailing
16-byte TKF MAC as a key. Tests rebuilt on the real layout, including a full
64-entry file.

Also correct the VTKF-selection TODO in mod.rs: the AACS HD DVD Book gives the
selector explicitly (match the TKF's PLAYLIST_NAME field to the active
playlist), not the "validate against an encrypted unit" placeholder.

Reconciled against the new HD DVD reference (freemkv.org/docs/hddvd/); the spec
source is archived in freemkv-private/spec/.
2026-07-20 11:57:13 -07:00
Matthew Jackson c1f1593003 aacs: discover HD DVD AACS dir + title-key files instead of hardcoding /ANY!/VTKF000
The HD DVD AACS directory name and title-key filename are chosen by the
authoring house, but the resolver hardcoded a single spelling
(/ANY!/VTKF000.AACS, /ANY!/MKBROM.AACS, /ANY!/CONTENT_CERT.AACS). Real discs
diverge: Freedom (Memory-Tech) names its AACS dir AAC! and ships VTKF090.AACS
+ VTKF100.AACS; Harry Potter carries VTKF000/001/002/099. On such a disc the
hardcoded path finds nothing, so no MKB/title-key/cert is read and decryption
silently can't engage.

Replace the fixed HD DVD path constants with structural discovery:
- find_hddvd_aacs_dir() locates the AACS dir as the root child dir ending in
  '!' that contains MKBROM.AACS (so the ..._BAK mirror is skipped; the dozens
  of decoy advanced-content '!' dirs are excluded by the MKBROM.AACS guard).
- role_paths(udf, role) builds the ordered candidate list per role: the static
  BD/UHD /AACS/ paths first, then the discovered HD DVD files — MKBROM.AACS,
  CONTENT_CERT.AACS, and every VTKF*.AACS (sorted), not just VTKF000.
- read_first() is now generic over &str / String so it takes the Vec<String>.

BD/UHD unaffected (no '!' dir → discovery returns None, list is the /AACS/
constants exactly as before). Verified on real Freedom (AAC!/VTKF090+100) and
Dukes (ANY!/VTKF000) ISOs; unit tests cover both shapes.

Open item (TODO(hddvd-encrypted)): when a disc has multiple VTKF variants the
correct one must be chosen by validating its VUK-derived key against a real
encrypted unit rather than first-that-reads. Blocked on obtaining a genuinely
encrypted HD DVD image — all HD DVD ISOs on hand are already-decrypted rips.
2026-07-20 11:29:28 -07:00
Matthew Jackson 6718c9cdb2 release: tee up 1.5.1 (marginal-read reporting fix) 2026-07-19 20:47:52 -07:00
Matthew Jackson 9fdd5edb65 drive: report marginal reads (PER=1) so a dirty disc can't pass clean
A read was only ever judged "good" by the drive's SCSI GOOD status — with no
integrity check on the bytes. On dirty/marginal media a drive can silently
return best-effort ECC data (occasionally mis-corrected) with GOOD status, so a
rip could "pass clean" yet decode with errors, pushing corruption downstream to
the mux instead of catching it at the read.

Enable recovered-error REPORTING at drive-prep: MODE SELECT the Read-Write
Error Recovery page with PER=1 (TB on / DTE off so the data still comes back),
preserving the drive's own retry count. Marginal reads now surface as
CHECK CONDITION / RECOVERED ERROR. Best-effort — a drive that doesn't honor it
keeps its defaults (no regression), and on a clean disc nothing changes.

A recovered error is distrusted: it marks just that ECC block NonTrimmed for a
Pass N re-read (a clean re-read wins; a persistent marginal becomes an honest
concealed gap), and is counted in the pass summary. Crucially it takes a
per-block SkipBlock, NOT the damage-jump path — a single marginal sector must
not trigger a 64 MB skip that would discard good data on a lightly-smudged disc.
2026-07-19 20:30:21 -07:00
Matthew Jackson 8a5a26f2a5 changelog: 1.5.0 release date + trim to app-level detail 2026-07-19 17:34:21 -07:00
Matthew Jackson 97ce0b7fab restore freemkv-unlock path dep for local dev (post-v1.5.0) 2026-07-19 15:37:27 -07:00
Matthew Jackson e194ef1585 v1.5.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-19 15:37:25 -07:00
Matthew Jackson 4d1b922232 changelog: drop-on-undecodable audio + forced PGS detection (1.5.0) 2026-07-19 14:29:57 -07:00
Matthew Jackson 3841ae2250 disc: detect forced PGS subtitles from stream content for info
Give `info` the same forced-subtitle verdict the muxer derives during a
rip, so the two agree. A shared classifier (mux::codec::pgs::ForcedTracker)
folds a PGS track's display sets — forced iff every one carries the
forced_on_flag — and is used by BOTH the MKV writer and a new scan-time
probe that reads the title's PGS streams (reusing the TS demuxer and PGS
parser). The probe only overrides a track it actually observed content
for, so an undecrypted/unread stream keeps its vendor-derived flag. Gated
behind ScanOptions::probe_forced_subtitles (off for the rip path, which
detects forced while muxing without a second read).
2026-07-19 14:23:04 -07:00
Matthew Jackson 2ccb5c9d01 mux/codec/mpegaudio: keep free-format frames
Free-format MPEG-audio (bitrate_index 0) is a legal, decodable mode — the
decoder derives the frame size from the sync spacing. Dropping it was a
false positive on a clean stream, so it now passes the gate.
2026-07-19 14:23:04 -07:00
Matthew Jackson b2bd5f8b3e mux/codec/truehd: fix drop-forward poison + rate/resync validation
Three TrueHD state-machine fixes from an adversarial audit:

- A corrupt access unit drops forward to the next major sync, but only
  the individually-verified corruption now feeds the whole-track poison
  verdict; the resync run is collateral. A couple of transient errors can
  no longer poison and discard an otherwise-good multi-hour track. The
  shared drop tally gains a verified/collateral split for this.
- The per-AU PTS rate is refined only from a CRC-validated major sync, so
  a corrupt major sync whose rate nibble decodes to another rate family
  can no longer shift the resumed audio — a drop stays a silence gap.
- The resync clears only on a CRC-validated major sync, never on a runt
  too short to hold and validate its header.
2026-07-19 14:23:04 -07:00
Matthew Jackson 6f055394c7 release: 1.5.0 2026-07-19 13:42:48 -07:00
Matthew Jackson 98f3dc513f mux: detect forced PGS subtitles from the stream
Flag a PGS subtitle track FlagForced when it displays subtitles and every
one carries the HDMV forced_on_flag (a dedicated forced/narrative track),
independent of the disc's vendor label metadata. The track header
reserves a FlagForced byte up front and it is promoted at finish() from
the accumulated display-set state. Only ever promotes — a track already
forced from the playlist metadata is never demoted.
2026-07-19 13:42:48 -07:00
Matthew Jackson 5ecfe7c69a mux/codec: drop undecodable audio frames, keep A/V sync
A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch. Sync is preserved — a drop becomes a silence
gap, never a shift — and every drop is logged. Detection is per-codec,
each mirroring the format's authoritative integrity check:

  DTS          core-header validity gates
  AC-3/E-AC-3  native frame CRC-16 + bitstream-id range
  FLAC         whole-frame CRC-16 residue
  MP2/MP3      header sanity + free-format reject
  AAC-ADTS     header sanity (raw AAC passes through untouched)
  TrueHD/MLP   major-sync CRC-16 + AU parity; corrupt AUs drop forward
               to the next major sync, since decode state carries
               across access units

LPCM and video are excluded by design (no in-frame integrity data;
inter-frame prediction). A shared DropTally handles counting, logging,
and a whole-track fallback for a mostly-undecodable track.
2026-07-19 13:42:48 -07:00
Matthew Jackson f255361683 dts: emit clean core alone when the extension boundary is garbage
DTS-HD MA access units are a lossy core frame followed by trailing
extension substreams up to the next core sync. On source-damaged discs
(observed on the Bourne UHDs) the bytes where the XLL extension belongs
are neither a core sync nor an extension sync -- pure garbage -- which
desyncs ffmpeg's XLL decoder and cascades into 'Read past end of XLL
band data' / 'DSYNC check failed' across the whole track.

next_core_boundary now distinguishes three boundary states via a new
ext_clean flag on NextCore::Found:
  - precise/recognized extension sync  -> ext_clean=true  (keep full AU)
  - garbage at the boundary byte       -> ext_clean=false (drop the ext)

When ext_clean is false we emit the DTS core alone (drop the smallest
junk piece, keep the frame and its PTS) and drain past the garbage to
the next core. Recognized-but-unsizeable extensions still ride the
heuristic scan and are kept intact, so lossless tracks are unaffected --
only genuinely corrupt extension bytes are dropped.

Bourne s1.dts: 1606 damaged frames / 691620 (0.232%), 93% isolated
single frames, worst run 3 in a row (~32ms lossy blip).
2026-07-19 09:27:28 -07:00
Matthew Jackson c6e6bb9f4b changelog: faststart default + DTS/DTS-HD audio (mp4://) 2026-07-18 22:35:37 -07:00
Matthew Jackson f7edd4e6a9 mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box
(sample rate, channel layout mask, core size, computed bitrate, LFE);
whole access units (core + DTS-HD extension substreams) pass through, so
an HD decoder finds the extension. dtsh when an extension sync follows the
core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR.

Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is
byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1)
and decodes clean (exit 0). Channel layout correct.
2026-07-18 22:18:35 -07:00
Matthew Jackson a947439171 mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free
box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample ×
est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes
mdat, sample offsets are fixed from the start — no rewrite, no offset
patch. finish() writes moov into the hole and pads the slack with a free
box; if the estimate is blown it falls back to moov-at-end. Streams over
HTTP without a pre-fetch. Reserve-math + box-order tests added.
2026-07-18 21:43:44 -07:00
Matthew Jackson 8e6114cd2a changelog: mp4:// input demuxer (1.5.0) 2026-07-18 21:22:41 -07:00
Matthew Jackson 8f55cb78d2 mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel
info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/
composition timing, stss → sync), rebuild a DiscTitle, and emit samples
as PesFrames in global decode order. Video NALs are length-prefixed in
MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into
input() so mp4:// flows to every sink (mkv://, audio://, json://, …).
In-memory write→read round-trip test proves symmetry (streams, hvcC,
sample sizes survive). Progressive MP4 only; fragmented (moof) is future.
2026-07-18 21:10:15 -07:00
Matthew Jackson 65e14fe3b7 mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps
the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow /
NFS staging avoids the dirty-page burst. The mdat backpatch is an
ordinary seek WritebackFile already handles (seek_then_patch_roundtrip).
Tests switched to in-memory Cursor writers.
2026-07-18 20:57:44 -07:00
Matthew Jackson 1e3610fd75 changelog: mp4:// native MP4 muxer (1.5.0) 2026-07-18 20:44:18 -07:00
Matthew Jackson 0c9d375548 mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio
track. Audio sample entries built from the first frame's bitstream —
AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for
fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio
durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle
(mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3;
TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the
CLI can report exclusions — never a silent drop.

Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3
both frame-exact on every track (video 2650/4270, audio 5567/3454),
duration and colour identical, clean decode. On a pathological 4-clip
title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more
of them) — source-inherent, not a muxer defect.
2026-07-18 20:37:10 -07:00
Matthew Jackson 8aff7fe708 mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into
a 64-bit mdat and building the sample tables in memory, patched at
finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs
(already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts,
CFR-derived decode timeline (pipeline carries presentation PTS only), and
a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the
hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping.

Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and
HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact
(bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land
in M2.
2026-07-18 20:16:19 -07:00
Matthew Jackson 489545c865 mux: add video:// sink (video-only per-track elementary streams)
Completes the per-track-class trio with audio:// / sub://. video:// is
the demux path with a TrackKind::Video kind filter — each video track to
its own native elementary stream (.hevc/.h264/.vc1/.m2v/.obu), no audio
or subtitles. Reuses the existing kind_filter machinery and extension
map; scheme/parse/output/write-only-guard arms added symmetrically.
2026-07-18 19:24:59 -07:00
Matthew Jackson 281d8baed6 changelog: 1.5.0 — extraction sinks + complete json:// model 2026-07-18 19:13:30 -07:00
Matthew Jackson 6a4ac97a33 mux: json:// emits the complete title model (no dropped fields)
The json:// sink previously emitted only codec + language + pid per
stream. Fill it out to the full DiscTitle: video carries resolution
(+ pixel dims, interlaced), frame rate (+ fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channels (+ count),
sample rate (+ Hz), and editorial purpose; subtitles carry the
qualifier. Add the clip list and chapter names. This is everything the
scan resolved, so json:// is a lossless machine-readable view of a
title rather than a summary.
2026-07-18 18:14:11 -07:00
Matthew Jackson a8563e9fa3 test: pin audio://sub://chapters://json:// in the scheme round-trip 2026-07-18 17:46:08 -07:00
Matthew Jackson 63f6909ff0 mux: chapters:// and json:// metadata sinks
Two write-only file sinks that ignore the PES stream and emit the title
metadata at construction (fvi-style, wired through output()):
- chapters:// — chapter markers in the format the extension picks: .xml
  (Matroska, default), .txt/.ogm (OGM simple), .vtt (WebVTT). Reuses the
  existing chapters_xml/chapters_ogm writers; adds a WebVTT writer.
- json:// — one title's model (playlist, duration, size, format, streams,
  chapters) as pretty JSON via serde_json.

New mux/meta_sink.rs; StreamUrl gains Chapters/Json; codec_label +
chapters_xml/ogm promoted to pub(crate) for reuse.

Tests: chapters_format_selected_by_extension, title_json_carries_streams_and_chapters.
2026-07-18 17:45:39 -07:00
Matthew Jackson 9f33306a0a mux: audio:// and sub:// sinks (demux filtered by track class)
Two extraction sinks built on the existing demux machinery: audio:// keeps
only audio tracks (native containers .thd/.dts/.ac3/.eac3/.pcm...), sub://
only subtitle tracks (PGS .sup, VobSub .idx+.sub, text .srt) — one file per
track, no video, no chapters sidecar. DemuxOptions gains a kind_filter; the
DemuxSink writers (already complete) are reused verbatim. StreamUrl gains
Audio/Sub variants; output() builds the filtered DemuxSink.

Test: kind_filter_keeps_only_the_selected_class.
2026-07-18 17:24:28 -07:00
Matthew Jackson 43cdc1351d restore freemkv-unlock path dep for local dev (post-v1.4.5) 2026-07-18 16:08:03 -07:00
Matthew Jackson 5728a7c577 v1.4.5: bump version (freemkv-unlock git-pinned for the tag) 2026-07-18 16:08:01 -07:00
Matthew Jackson f85d91a17a changelog: 1.4.5 (FMTS clean single-variant mux + key-Debug redaction + hex fix) 2026-07-18 13:30:36 -07:00
Matthew Jackson a688e2c642 mux: FMTS read-plan on the inline live-drive path too (DiscStream::with_key_map)
The read_plan fix landed on the file-backed highway (build_iso_pipeline)
but the inline DiscStream (live single-pass) still passed the alternate
device-group half to the demux — a known FMTS bug on rip_mode="single".

DiscStream::with_key_map installs the proactive map and rewrites the
extent walk to the read plan, so the live single-pass path reads ONLY
our-phase units, exactly like the highway. DecryptingSectorSource gains
set_key_map for the already-constructed decorator. Non-forensic maps
return the extents unchanged, so every non-FMTS disc is byte-identical.

Test: with_key_map_reads_only_our_phase_units — a forensic Even segment
drops exactly its alternate units from the extent walk. Precommit green
on 1.86.
2026-07-18 13:27:48 -07:00
Matthew Jackson 05fb632d7c mux: FMTS reads only our-phase units (AacsKeyMap::read_plan)
An FMTS forensic segment interleaves our device group's variant with a
foreign group's at the aligned-unit level. The mux was decrypting only
our phase but leaving the alternate (foreign) units in the buffer as
ciphertext, trusting the demux to 'drop untouched ciphertext cleanly'.
It doesn't: random 0x47 bytes at the 192-byte stride hit tracked PIDs,
mis-parse, and trip the demux's concealed-gap keyframe-resync — which
drops GOOD frames of ours around every segment (349 resyncs / ~6391
packets on Stand by Me, visible as playback flaws).

The map already knows which unit each LBA is and, for a forensic
segment, which phase is ours. AacsKeyMap::read_plan turns that into the
title's read plan: every default/CPS unit, plus inside a segment ONLY
our-phase units. The alternate units are never fetched, decrypted, or
handed to the demux — the demux sees one gapless our-variant stream.

- read_plan is general (single-CPS, multi-CPS, FMTS): a map with no
  Even/Odd range returns the extents unchanged, so DVD/CSS, single-CPS
  UHD and multi-CPS Blu-ray read byte-for-byte as before.
- Wired into build_iso_pipeline (the file-backed highway that muxes
  resumed ISOs). Producer re-anchors unit_base per extent, so per-unit
  segment reads stay unit-aligned and decrypt correctly.
- Extent gains PartialEq/Eq for the read_plan tests.

Tests: read_plan non-forensic unchanged; forensic omits exactly the
alternate units, kept units match the decrypt gate unit-for-unit. All
2314 lib tests pass; precommit green on 1.86.

Pending: end-to-end ISO re-mux validation (concealed gaps 349 -> ~0).
2026-07-18 12:06:20 -07:00
Matthew Jackson 3cb0a8f41c remove user-facing English from the library (KeyOrigin::name, hddvd Title)
Library holds ZERO user-facing English (CLAUDE.md). Removed KeyOrigin::name()'s
English prose — apps map the typed enum (freemkv gets key_origin_label); diag
logs the enum's Debug repr. hddvd unnamed-title fallback 'Title N' -> neutral
TITLE_N identifier (UDF volume-label style).
2026-07-17 21:50:09 -07:00
Matthew Jackson 9dbfb70f7e narrow leaked-internal pub surface to pub(crate)
Verified zero callers across all consumer crates AND libfreemkv integration tests:
MUX_APP, Disc::{aacs_disc_hash,encrypted_content_ranges,inject_unit_keys},
locate_ranges, mapfile::{MapEntry,entries}, diag::dump_mkv_track,
DiscStream::{errors,lost_bytes} (read via accessors). Removed the unused mux
DemuxSink/FviSink crate-root re-exports (constructed internally by output() via
the direct module path). Staged demux option variants marked allow(dead_code).
2026-07-17 21:45:33 -07:00
Matthew Jackson 9af3f7da7a disc: collapse decrypt_keys_for_title twin into one method
Removed the test-only unchecked wrapper and renamed decrypt_keys_for_title_checked
-> decrypt_keys_for_title (one method returning (keys, title_is_clear)). Production
already used the checked form; the _checked suffix had no counterpart. Callers +
docs updated; two tests take .0.
2026-07-17 21:41:10 -07:00
Matthew Jackson 2638c3075e css: remove redundant crack_key_halt wrapper
crack_key_halt had no caller except the crack_key wrapper — a needless middle
layer. crack_key now calls crack_key_scan directly; crack_key (Option) and
crack_key_outcome (full CrackOutcome + halt) remain as the two real entry points.
2026-07-17 21:34:37 -07:00
Matthew Jackson 3661942bdb mux: collapse FviSink::create/create_with_source into one create
create_with_source (full SourceInfo) was only ever called by the create wrapper
with a partial SourceInfo — the extra capability was unused (YAGNI). Inlined the
constructor body into create and deleted the _with_source variant; callers
unchanged.
2026-07-17 21:32:39 -07:00
Matthew Jackson 43c1f9bda0 mux: collapse MkvStream::create/create_at into one create(.., Option<&Path>)
One method per action (CLAUDE.md), not a foo_with_X pair — create_at was the real
constructor and create was a None wrapper. Callers pass Option<&Path> directly.
2026-07-17 21:31:29 -07:00
Matthew Jackson 37832ac2dd hex: canonical hex->integer parsers + public strip_hex_prefix
Adds parse_hex_u16/u32/u8 and exposes strip_hex_prefix so callers stop hand-rolling
from_str_radix(trim_start_matches("0x")) — a case-sensitive strip that this module
exists to prevent. disc::aacs_disc_hash now uses strip_hex_prefix.
2026-07-17 21:25:59 -07:00
Matthew Jackson 2263d2cc4e disc: redact Debug for AacsState/Key/HandshakeResult (test-guarded)
AacsState (public via Disc.aacs) and Key (the decrypt_with key-transport enum)
are crate-root re-exported and carried VUK/unit/read-data keys + volume id on
#[derive(Debug)]; HandshakeResult carried the VID + AACS 2.0 bus key. Manual
Debug impls print shape only, guarded by red->green tests.
2026-07-17 21:10:51 -07:00
Matthew Jackson 3546648faa aacs: redact Debug for ResolvedChain/ResolvedKeys/ProcessingKeyMatch (test-guarded)
These carry raw unit-key / VUK / processing-key bytes on their Debug; manual impls
print shape only (unit_keys_len, redacted markers). Each has a red→green test.
2026-07-17 21:03:31 -07:00
Matthew Jackson 98000869b2 css: redact CssState Debug (test-guarded)
CssState is reachable via the public Disc.css field; #[derive(Debug)] leaked the
raw CSS title key on any {:?} of a Disc. Manual Debug prints crack_span only.
2026-07-17 21:00:34 -07:00
Matthew Jackson e308c5b825 aacs/types: redact Debug for all key-bearing types (test-guarded)
DeviceKey/HostCert/Vid/MediaKey/Vuk/ProcessingKey/UnitKey/DiscEntry carried key
material on #[derive(Debug)] — a stray {:?} would leak device/host-private/media/
volume/unit keys. Manual Debug impls print shape only. redaction_tests asserts no
key bytes appear and a redaction marker is present (fails if a derive returns).
2026-07-17 20:56:38 -07:00
Matthew Jackson 5e1f880f6e libfreemkv: phase-aware FMTS decode + two-operation KeyFetch/KeySource seam
FMTS (AACS 2.1) now decodes per (LBA, phase): Phase enum + AacsKeyMap::
from_ranges_phased, decrypt only the variant's parity half. resolve_fmts_key_map
does a 2-phase index-1 anchor then per-index phase probe, and sizes the forensic
set to whatever the source returns (no hardcoded 32). KeyFetch is now two explicit
operations (unit_keys / fmts_indexes) and KeySource splits get_uk into
get_unit_keys + get_fmts_indexes. BYPASS_FMTS_KEY gate removed (first-class format).

Teed up for 1.4.5. Local WIP baseline.
2026-07-17 20:51:15 -07:00
Matthew Jackson ffe8ee8684 restore freemkv-unlock path dep for local dev (post-v1.4.4)
leak-guard / leak-guard (push) Successful in 22s
CI / test (push) Failing after 1m31s
CI / lint (push) Failing after 1m33s
CI / check-macos (push) Has been cancelled
CI / check-windows (push) Has been cancelled
2026-07-16 21:44:08 -07:00
Matthew Jackson 93571d9181 v1.4.4: bump version (freemkv-unlock git-pinned for the tag)
leak-guard / leak-guard (push) Successful in 17s
Release / verify (push) Successful in 5s
Release / release (push) Failing after 16s
Release / test (push) Successful in 2m55s
2026-07-16 21:44:05 -07:00
Matthew Jackson 89af9876ae keysource: add DecodeSampleSet — a >=MIN_SAMPLE_UNITS-by-construction sample set so an online request can't be built under-sized 2026-07-16 21:43:02 -07:00
Matthew Jackson 0471e0ca40 restore freemkv-unlock path dep for local dev (post-v1.4.3) 2026-07-16 21:06:04 -07:00
Matthew Jackson 38207d2272 v1.4.3: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:06:01 -07:00
Matthew Jackson add9d8e0cd aacs: sample only index-1 forensic segments for the online key query; hoist MIN_SAMPLE_UNITS into the base crate 2026-07-16 21:01:16 -07:00
Matthew Jackson edc60582ec FMTS: resolve the index key map from one forensic keyserver query
The keyserver protocol now returns all 32 index keys as an array for a
forensic content sample (and a single-element array for plain content).
resolve_fmts_key_map sends one forensic batch and maps array element i to
segment index i+1, replacing the per-index blind-probe collection loop
that repeatedly hit the key service. Segment/index parsing and the
aligned-unit content classification are reworked to support this:

- rename variant_select -> index_select (per-index, not per-variant)
- content classification moves to is_clean(buf, ContentFormat) so the
  unit selector emits only units the key service accepts
- segment.rs: parse IndividualSegment.tbl index tags + SPN ranges,
  build contiguous LBA key ranges from the resolved 32-key array
- decrypt/decorator plumbing for the resolved per-index keys

Fail loud (FmtsKeyMissing) when the forensic query returns < 32 keys or
any segment index stays unresolved.
2026-07-16 19:41:44 -07:00
Matthew Jackson ccb7cafc68 restore freemkv-unlock path dep for local dev (post-v1.4.2)
leak-guard / leak-guard (push) Successful in 20s
CI / test (push) Failing after 45s
CI / lint (push) Failing after 45s
CI / check-macos (push) Has been cancelled
CI / check-windows (push) Has been cancelled
2026-07-15 19:37:54 -07:00
Matthew Jackson 0183bfb58c v1.4.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-15 19:37:51 -07:00
Matthew Jackson 830d1e360c Mux decrypt/verify redesign, HD DVD first-class, MVC 3D
decrypt:
- decrypt_sectors is now a pure decrypt (apply key, leave plaintext, report
  unverified bytes); TS-structure is a separate primitive (is_clean_ts/ps) used
  only for key selection and read-verify. The mux passes decrypted bytes through
  (the demuxer drops non-conforming packets), ending the NULL-TS conceal loop and
  the per-unit key-server refetch storm. Key-proof floor replaces the 75%
  supermajority.

recovery:
- Removed the post-read decrypt-verify gate (verify.rs) that mis-aligned the
  disc-absolute unit grid against clip-anchored AACS units and false-failed good
  clips (e.g. Dunkirk's orphan-CPS clip). Bad sectors are marked by physical read
  result; decryptability is proven at scan + mux time.

HD DVD (first-class AACS):
- Role-based candidate-list file sourcing so an HD DVD's /ANY!/ files
  (MKBROM.AACS, VTKF000.AACS, CONTENT_CERT.AACS) are found with no disc-type
  branch. parse_vtkf parses VTKF000.AACS into the same UnitKeyFile as a BD
  Unit_Key_RO.inf, so the shared VUK unwrap applies unchanged. set_unit_base
  clip-anchoring. Two decrypt-axis assumptions remain UNVERIFIED-HDDVD-DECRYPT
  (no encrypted disc to test).

mux:
- MVC (Blu-ray 3D) track signals unified into one MVCDecoderConfigurationRecord;
  release-safe track_vint (3-byte VINT) and pid_index (i32) guards.

hardening:
- Container-aware is_clean / encryption detection; bytes_bad_in_title fail-safe
  on a corrupt mapfile; CSS crack gated on DiscFormat::Dvd (HD DVD excluded);
  non-vacuous CSS tests; patch NOT_READY/HARDWARE/ILLEGAL_REQUEST/ABORTED
  sense-path tests.
2026-07-15 19:35:12 -07:00
Matthew Jackson 04728d7d94 Mux: pure decrypt, policy at the caller (no null, no key-server storm)
decrypt_sectors is now a pure decrypt — apply the CPS unit key, leave the
plaintext, report how many bytes did not reach clean TS ("unverified"). It
never restores ciphertext, nulls, or re-fetches. "Did a key produce clean TS?"
is a key-selection / read-verify signal, not the verdict "did we decrypt?": a
correct key can decrypt a bad-encoded region, and broken TS is a muxer concern
(the demuxer drops the packet and resyncs).

Callers own the policy:
- mux (read > decrypt > mux): pass the decrypted bytes to the muxer, whatever
  they are; fail loud only on a genuine can't-decrypt (no key / misaligned).
- sweep/patch (reading from a disc): an unverified unit is a bad read — recover
  a fresh key and retry, or fail loud so disc-recovery re-reads it.

Removes three duplicated decisions — the decrypt-time ciphertext restore, the
mux NULL-TS conceal loop, and the per-unit key-server refetch — plus the dead
aacs_unit_still_ciphertext predicate. Key-fetch recovery now samples the on-disc
ciphertext explicitly (a pure decrypt leaves the buffer plaintext) and lives
only on the rip/verify path, never the mux.

Fixes the 30-90s/region mux stalls and key-server storm on bad-encoded UHD runs
that 1.4.1 left behind (it relaxed the gate but not the surrounding machinery).
2026-07-14 21:04:24 -07:00
Matthew Jackson e62ffed2b1 restore freemkv-unlock path dep for local dev (post-v1.4.1) 2026-07-14 14:45:33 -07:00
Matthew Jackson 9d37043b3e v1.4.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-14 14:45:30 -07:00
Matthew Jackson 6858cd064d Fix mux concealing decryptable video over a single defective packet
AACS content decryption rejected a whole 6144-byte aligned unit unless
EVERY content packet was conformant MPEG-TS. One authored-bad packet (a
pressing/encoding defect or an AACS 2.1 forensic-variant frame) made the
mux conceal the entire unit as NULL TS — destroying up to 31/32 good
packets and tallying them as loss, surfacing as false "corruption" on
otherwise-clean discs (observed across two UHD titles).

decrypt_unit now asks only "did a key OPEN this unit?" — a padding-aware
>=75% supermajority of content packets restoring their 0x47 sync, a gate
no wrong key can reach (uniform-AES noise floor) yet one that tolerates a
minority of authored-bad packets. Opened units pass through VERBATIM; a
non-conforming packet is left for the demuxer to drop on sync-loss and
resync past. TS-sync conformance is a muxer concern, never a decryption
verdict. The post-read verify/sweep gate now shares the same primitive so
it can never disagree with the mux decrypt.

Also unify the MVC (Blu-ray 3D) track signals: the mvcC CodecPrivate
extension, the BlockAdditionMapping, and each per-frame BlockAdditional
all derive from one MVCDecoderConfigurationRecord built once per track, so
a malformed dependent-view parameter set can no longer orphan a BlockAddID.
2026-07-14 14:43:06 -07:00
Matthew Jackson f99670ceaa restore freemkv-unlock path dep for local dev (post-v1.4.0) 2026-07-13 19:28:55 -07:00
Matthew Jackson 75b0e68b85 v1.4.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-13 19:28:52 -07:00
Matthew Jackson 4a341331e2 Changelog: 1.4.0 (Blu-ray 3D / MVC) 2026-07-13 12:47:03 -07:00
Matthew Jackson 422f2b6bcf 3D MVC mux: audit round 2 (converged)
Second audit round converged (severity collapsed 6 HIGH -> 1; the one
HIGH was a bounded 32-element scan, not a defect; the sole spec MEDIUM
was the same false-positive re-raised — 0xBF matches ISO/IEC 14496-15
§7.6.2 verbatim). One genuine robustness fix plus coverage:

- extract_mvc_params: skip a zero-length NAL instead of abandoning the
  scan, so a stray length prefix before the subset SPS/PPS no longer
  silently drops 3D signalling. Test proves params after a zero-length
  NAL are still found.
- Tests: parser_for_mvc_dependent routes H.264 to a passthrough parser;
  passthrough with an IDR does not re-assert param sets (the keyframe &&
  !mvc branch).
- Document the per-playlist (not per-clip) is_3d latching as a known
  limitation (real main-feature playlists are uniformly 3D).
2026-07-13 11:39:07 -07:00
Matthew Jackson d4021114cd Harden 3D MVC mux: robustness + tests (audit round 1)
Triage of a 10-lens code audit of the 3D branch. Fixes for real defects;
rejected three spec false-positives that matched the ISO/IEC 14496-15
§7.6.2 record verbatim.

Robustness / correctness:
- Never panic when a title's only video is the MVC dependent view: the
  base is now the first NON-dependent video, so a dependent-only title
  sets up no merge (muxed as an ordinary track) instead of hitting an
  `expect` on the skipped track slot.
- Drop a per-frame BlockAdditional (BlockAddID=2) when the track declared
  no mvcC mapping (dependent params not captured before the header) — a
  plain block keeps the file conforming instead of an orphaned add.
- A non-keyframe MVC base frame always carries a ReferenceBlock (fall back
  to a 0 offset in the pre-first-keyframe corner) so it is never mistaken
  for a seek point.
- Reference the last keyframe on the PRIMARY video track only, so a
  secondary video track's keyframe can't become a cross-track reference.
- dep_by_pts overflow: bound BEFORE inserting so the just-arrived
  dependent survives the drift-clear; count a displaced duplicate-PTS
  dependent as an orphan instead of losing it silently.

API / docs:
- Fold write_frame_with_additional into write_frame(..., Option<&[u8]>)
  per the "no foo_with_X" convention.
- Fix mvc_params doc (StereoMode is intentionally not emitted); remove a
  stale PAT/PMT comment describing an approach that was never taken.

Tests: MVCDecoderConfigurationRecord over-length guards; write_int minimal
two's-complement widths; BlockGroup/BlockAdditions/BlockAdditional +
ReferenceBlock emission; additional dropped without a mapping; h264 MVC
passthrough keeps param sets in-band; extract_mvc_params no-panic on
truncated/empty input; pairing window + dep-overflow edges; no-panic on a
dependent-only title.
2026-07-13 11:24:23 -07:00
Matthew Jackson fd6dfbe5b0 Mux Blu-ray 3D (MVC) as a single MVC video track
Fold the MVC dependent (right-eye) view into the base H.264 track as a
per-frame BlockAdditional under an mvcC BlockAdditionMapping, so a 3D
title produces one MVC video track instead of two independent H.264
tracks.

- h264: MVC-passthrough parser mode keeps the dependent view's subset
  SPS/PPS in-band, so each emitted frame is a self-contained dependent
  access unit for a BlockAdditional
- resolve: route the dependent stream through the passthrough parser
- mkvstream: detect the dependent view, pair it to the base frame by
  PTS (bounded FIFO), attach it as a BlockAdditional (BlockAddID=2),
  and skip building its own track; build the mvcC
  MVCDecoderConfigurationRecord from the captured subset SPS/PPS and set
  it on the base track at activation
- mkv: emit the mvcC BlockAdditionMapping and BlockGroup/BlockAdditions,
  with a ReferenceBlock on non-keyframe base frames
- ebml: add BlockAdditions/BlockMore/BlockAdditional/BlockAddID/
  BlockAddIDValue/ReferenceBlock elements and a signed-int writer

Verified against a Blu-ray 3D ISO: ffprobe shows a single MVC track,
the mvcC mapping is present, ~144k BlockAdditionals carry the dependent
view (8.7 GB), and the base view decodes cleanly with no regression.
MVCDecoderConfigurationRecord follows ISO/IEC 14496-15 7.6.2; StereoMode
is intentionally omitted (no enum value describes MVC-in-BlockAdditional;
the mvcC mapping is the primary 3D signal per RFC 9559).
2026-07-13 11:00:04 -07:00
Matthew Jackson 573d2f46c4 feat(3d): read the SSIF and enumerate the MVC dependent view
Blu-ray 3D as a mux-path variant:
- Detect 3D (a clip has STREAM/SSIF/<clip>.ssif — note the SSIF/ subdir, which
  the old .ssif fallback path got wrong) and use the SSIF extents for the mux:
  one transport stream carrying both the base (left) and MVC dependent (right)
  eyes on distinct PIDs, so muxing it captures the full 3D.
- Add the dependent-view video stream. The base STN table lists only the left
  eye; the on-disc PAT/PMT are AACS-encrypted (unreadable pre-key) and the base
  STN omits the dependent view (it lives in the MPLS STN_table_SS), so use the
  BD-3D PID convention: dependent = base video PID + 1 (0x1011 -> 0x1012),
  coding 0x20 -> H.264. The demux routes its packets from the SSIF.

info now reports two video streams for a 3D title; the mux writes both eyes.
2026-07-12 23:41:47 -07:00
Matthew Jackson ef39674194 feat(3d): enumerate MVC dependent view (stream_type 0x20 -> H.264 video)
from_coding_type maps 0x20 (MVC dependent view) to Codec::H264, so the existing
PAT/PMT scan surfaces the SSIF right-eye substream as a second video stream on
its own PID instead of dropping it as Unknown. No new parser: the dependent eye
rides the same demux path as any other TS video stream. Removes the throwaway
3D-structure probes (findings captured in prior commit messages).
2026-07-12 22:16:13 -07:00
Matthew Jackson 9973849408 feat(3d): add MVC dependent-view coding type (0x20)
First step of Blu-ray 3D as a mux-path format variant: name stream_type 0x20
(H.264/MVC dependent view, the SSIF right-eye substream). Recognition in the
scan/demux + dependent-stream enumeration follow.
2026-07-12 22:11:01 -07:00
Matthew Jackson d9db268b06 wip(3d): prove SSIF de-interleave = ssif−base, dependent decrypts on same UK
Empirical (300, clip 00042): base .m2ts (1567 extents, 25.85 GB) is entirely
inside the SSIF LBA span; dependent = SSIF − base = 1567 ranges, 9.59 GB
(exact vs 35.43−25.85). 200/200 sampled dependent-view units decrypt under the
base UK. => de-interleave needs no SSIF parsing (complement of the base extents
we already resolve), and the dependent eye needs no extra key.
2026-07-12 20:40:58 -07:00
Matthew Jackson 129c34b002 wip(3d): read-only 3D-structure probe (SSIF/m2ts + MPLS ExtensionData)
Dev tool for the 3D build: uses the library UDF reader (7z cannot parse these
ISOs) to dump base .m2ts vs SSIF sizes and the MPLS ExtensionData entries
(STN_table_SS + SS sub-path). Not product code.
2026-07-12 20:38:37 -07:00
Matthew Jackson ebf30a679e fix(udf): follow allocation-descriptor continuation past the AED header
read_icb_extents parsed a continuation block's allocation descriptors from
offset 0, but a continuation block begins with a 24-byte Allocation Extent
Descriptor (ECMA-167 4/14.5): 16-byte descriptor tag + prev-loc(4) +
length_of_allocation_descriptors(4); the real ADs start at offset 24. Reading
from 0 mis-parsed the AED tag as a descriptor -> one garbage extent, then an
unknown-type break, silently truncating any file whose extents spill into a
continuation block.

Few-extent files store their ADs inline and never hit this path, so it stayed
hidden; a heavily fragmented file (~1600 fragments) truncated to 113 extents +
a bogus non-unit-aligned fragment that then aborted the mux
(ExtentNotUnitAligned). Skip the 24-byte AED header, read l_ad bytes of
descriptors from offset 24. Adds a regression test with a non-zero AED tag.
2026-07-12 20:37:55 -07:00
Matthew Jackson 09a8dd183d restore freemkv-unlock path dep for local dev (post-v1.3.2) 2026-07-10 14:26:11 -07:00
Matthew Jackson 057c878831 v1.3.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 14:26:08 -07:00
Matthew Jackson 43cbfa07f5 1.3.2: AACS 2.1 FMTS variant-decode foundation
Add UnitKey.variant_number (0 = ordinary, 1..32 = forensic variant) with new/variant constructors, and aacs::variant_select — resolve a disc's single variant and classify each aligned unit (default / variant / drop foreign / conceal keyless). Correct IndividualSegment.tbl: the per-record field is the variant (cycles 1..32 on a retail disc), not a segment number — Segment.number -> Segment.variant.
2026-07-10 14:00:28 -07:00
Matthew Jackson 0e0967795e changelog: trim the 1.3.1 relicense note to just the relicense fact 2026-07-10 12:53:17 -07:00
Matthew Jackson 68b5372415 restore freemkv-unlock path dep for local dev (post-v1.3.1) 2026-07-10 12:35:12 -07:00
Matthew Jackson 9e8f196b20 v1.3.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 12:35:09 -07:00
Matthew Jackson b8f0af9ef5 1.3.1: relicense to MIT (clean-room CSS + drop copyleft-lib references)
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL). The CSS
content cipher and Stevenson title-key attack are attributed to their published
cryptanalysis (not libdvdcss); all libaacs/libbluray/libdvdread/libdvdnav name
references were dropped from comments while keeping the standard format/spec
descriptions. Also bumps to 1.3.1.
2026-07-10 12:31:19 -07:00
Matthew Jackson 24e2bc33cf hddvd: authoritative title composition from the Advanced-Content playlist
Parse ADV_OBJ/VPLST000.XPL (the real HD-DVD player playlist) with roxmltree
into one DiscTitle per <Title>: its PrimaryAudioVideoClip clips in order (EVO
via the .MAP sidecar), the titleDuration, the displayName, and the ChapterList.
A layer-break split (FEATURE_1+FEATURE_2, feature/feature_Divide) composes into
ONE title with the two parts as clips, each carrying its title-time in/out
points (45kHz ticks) for seamless-join splicing. Falls back to the clip-name
heuristic when no playlist is present.

Fixes unknown runtimes (real durations), poor names (FEATURE -> 'Main Movie'),
and gives authoritative composition + chapters. Validated on ANCHORMAN (97m),
SHAUN (99m), HARRY_POTTER (152m).
2026-07-10 10:28:18 -07:00
Matthew Jackson a7df91b92b ci: drop the dead crates.io publish job (libfreemkv is git-tag-only)
libfreemkv is publish=false (it git-deps the firmware crate freemkv-unlock,
never on crates.io), so the Release workflow's 'cargo publish' step failed
hard on every tag. Consumers git-tag-pin libfreemkv; the git tag is the
release artifact. Removed the publish job.
2026-07-10 08:52:57 -07:00
Matthew Jackson 0eb0188ba7 restore freemkv-unlock path dep for local dev (post-v1.3.0) 2026-07-10 08:43:47 -07:00
Matthew Jackson e2f595d558 v1.3.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 08:43:42 -07:00
Matthew Jackson 18082d0df1 audit: void empty-timeline duration, cover sniff overlap
Round-11 findings from the 10-phase release audit (no real HIGH):

- When a no-declared-duration source (HD-DVD) muxes a degenerate single
  frame at tick 0 with no per-frame duration, max_block_ticks stays 0 and
  the reserved DURATION placeholder was left as a literal 0.0 (players read
  that as a zero-length file). Void the element instead, so the Segment
  omits DURATION as an unknown-duration source did before the back-patch.
- Add a regression test for the sniff_video_codec overlap fix (a
  picture_start_code whose payload begins 00 00 followed by a real start
  code) so the i+=4 marker skip can't silently regress to i+=3.
2026-07-09 20:40:54 -07:00
Matthew Jackson 640502d5a8 audit: lock DTS rate table, fix sniff overflow-scan, cover decrypt loss
Round-10 findings from the 10-phase release audit:

- A finder claimed the DTS SFREQ→rate table was wrong at 11/12; verified
  it against ffmpeg's avpriv_dca_sample_rates (12k/24k/48k/96k/192k at
  11-15) — the table is CORRECT. Added a test that locks the full table so
  it can't be mis-"fixed".
- sniff_video_codec advanced 3 bytes after a matched start code, re-reading
  the code byte as an overlapping start code; skip the full 4-byte marker.
- Guard the HD-DVD next_id title counter with saturating_add so a crafted
  disc with >65536 clips can't overflow (panic in debug).
- Add a test that an undecryptable unit (DecryptFailed) is zero-filled and
  counted as loss through ExtractResult (complete=false, bytes_lost>0) —
  the recovery-seam consolidation folded that bucket into bytes_unreadable.
2026-07-09 20:17:12 -07:00
Matthew Jackson 270f9d88b3 audit: drop dead DTS marks cap, lazy passthrough buf, doc corrections
Round-9 findings from the 10-phase release audit (no HIGH):

- Remove the MAX_PTS_MARKS backstop and its tautological test: an empty
  DTS PES returns before recording a mark, and a non-empty run is already
  bounded by the MAX_AU_BYTES buffer clear (which clears pts_marks) — so
  the deque cannot grow unbounded and the cap was dead code.
- AuAssembler::for_codec no longer reserves 256 KiB for a Passthrough
  stream (audio/subtitle, and every TS/BD stream) whose buf is never
  written; only the reassembling modes reserve.
- Correct the scan comment that claimed region is computed (it is a
  Region-free stub until region detection lands) and drop a public-repo
  reference to internal "private refactor notes" in the mkb module doc.
2026-07-09 19:50:54 -07:00
Matthew Jackson 6a0e61d415 audit: clamp BD format fallback, running GOP byte counter, O(1) DTS marks
Round-8 findings from the 10-phase release audit:

- detect_disc_format's BDMV fallback passed detect_format's result
  through unchanged, so an SD bonus/menu title could tag a BD-tree disc as
  DVD (mis-sizing the ECC sweep) — violating its own "never below Blu-ray"
  invariant. Clamp anything but UHD up to Blu-ray.
- Track the MPEG-2 GOP byte total incrementally instead of re-summing the
  whole gop_buf on every pushed picture (was O(pictures²) on any MPEG-2
  disc, not just adversarial input).
- Back the DTS pts_marks deque with a VecDeque so the over-cap prune is an
  O(1) pop_front, not an O(n) Vec::remove(0).
- Add a test exercising parse_stream_id_extension's PTS/DTS skip branches
  (the real AU-opening 0xFD video PES path) — previously untested.
2026-07-09 19:27:01 -07:00
Matthew Jackson 92e3b41468 audit: bound DTS marks, align disc-format tree order, doc/test cleanups
Round-7 findings from the 10-phase release audit (no HIGH; convergence):

- Cap DtsParser.pts_marks (MAX_PTS_MARKS): a run of zero-length timed PES
  packets grew no buffer bytes, so the drain_front mark-prune never ran —
  the deque could accumulate without bound on hostile PS input.
- detect_disc_format tested HVDVD_TS before BDMV while the title-scan
  dispatch tests BDMV first, so a disc with both trees would be classified
  HD-DVD but enumerated as Blu-ray. Align both to BDMV → HVDVD_TS →
  VIDEO_TS.
- Document why the DTS new-PES re-base can emit a locally-decreasing PTS
  (the muxer's block_ts applies the strictly-monotonic audio nudge, tested
  in mkv.rs) — this is by design, not a mux defect.
- Fix stale aacs/keys.rs comment references (functions moved to
  aacs/inf.rs / aacs::resolve/derive in the module split).
2026-07-09 19:01:56 -07:00
Matthew Jackson 7d852419b5 audit: byte caps on GOP buffers, opener-scan resume, honest video codec
Round-6 findings from the 10-phase release audit:

- Wire the documented MAX_PENDING_BYTES byte cap into the MPEG-2 GOP
  buffer (it was dead code) and add an equivalent MAX_GOP_BYTES cap to the
  sparse-PTS reorder, so a crafted stream of few-but-huge access units
  cannot over-allocate — both were bounded only by frame count before.
- probe_evo_streams defaulted an unsniffable HD-DVD video stream to H.264,
  which mis-parses a VC-1 (or still-encrypted) clip into a corrupt track.
  Emit the video stream only when the codec is actually identified — the
  honest outcome, matching the audio path (a real clear clip always carries
  its sequence header at the head).
- Resume the AU-opener search from a cursor (like the boundary search), so
  a long unsynced junk run is O(bytes), not O(buffer) per push.
- Mark mpeg2's now-dead MAX_AU_BUFFER test-only; restore #[doc(hidden)] on
  the aacs probe harness module.
- Add regression tests: the 0xFD video-routing guard, the FMTS-is-UHD key
  state, and the GOP byte caps.
2026-07-09 18:31:47 -07:00
Matthew Jackson 9066433c29 audit: guard 0xFD video routing, carry frame duration, add cap tests
Round-5 findings from the 10-phase release audit:

- collect_es routed EVERY extended-stream-id (0xFD) PES into the video ES
  buffer, so a 0xFD HD-audio sub-stream (MLP/TrueHD) could pollute the
  video sample and — if it preceded the video PES — stamp the video track
  with the audio PID, losing the video. Only the VC-1 extension (0x55) is
  now treated as video; routing 0xFD audio to its own track is deferred to
  the HD-DVD program-chain follow-up.
- The sparse-PTS reorder now carries its calibrated per-frame duration onto
  each frame, so the muxer emits a BlockDuration and the back-patched
  Segment Duration covers the final frame instead of understating it.
- Add regression tests for the MAX_MARKS and MAX_VTI_HITS caps (promote
  MAX_VTI_HITS to module scope); make the differential-test factory array a
  named type; drop an identity-op in a reorder test.
2026-07-09 17:59:04 -07:00
Matthew Jackson c81a6e05cd audit: fix AU mark-field loss, VTI tie determinism, and mark/perf issues
Round-4 findings from the 10-phase release audit (the first fully clean
round; it dug into the new #22/#18 refactor code):

- AuAssembler closed each AU from only the FRONT mark's fields, so when
  one PES fragment carried the source and a later fragment of the same AU
  carried the PTS, the second field was dropped — a regression vs the old
  separate pts/source mark deques. Now merge the first Some of each field
  across all in-range marks.
- parse_vti_clip_order picked the largest residue bucket with
  HashMap::into_values().max_by_key(), nondeterministic on a size tie
  (randomized HashMap iteration) — could select a different clip table
  run-to-run. Break ties by smallest offset.
- Bound the marks/disc_marks deques (MAX_MARKS): the buf-size cap prunes
  marks only when bytes accumulate, so a run of zero-length timed
  fragments could grow them without bound on hostile input.
- Add push_owned so the PS path moves the PES payload into a passthrough
  AU with no copy (MPEG-2 video + all audio), removing a per-PES
  malloc+memcpy the refactor had introduced on the DVD path.
- Back-patch the MKV duration from the block END (start + its own
  duration) so it covers the final frame instead of understating by one.
- Add direct tests for the MKB record-framing walker; drop a stale
  drain_complete_aus doc comment left on process_au.
2026-07-09 17:30:41 -07:00
Matthew Jackson 0a9bdf08f6 docs: drop Renesas from changelog (detection-only stub) 2026-07-09 16:36:47 -07:00
Matthew Jackson 2b74a9b21f docs: soften HD-DVD changelog to partial support (PGC parse deferred) 2026-07-09 16:33:40 -07:00
Matthew Jackson b5a5138569 mux: resume AU-boundary scans from a cursor (O(n) not O(n²))
AuAssembler::drain rescanned the whole buffered access unit from a fixed
offset on every push, so reassembling one AU split across N program-stream
PES fragments cost O(bytes²/fragment) — amplified on the HD-DVD PS path
where H.264/HEVC/VC-1 frames are large and now flow through this shared
assembler (unlike the TS path, which delivers one AU per PES).

Carry a scan_pos cursor (and, for the stateful VC-1/MPEG-2 rules, a
seen_unit flag) so each push resumes the boundary search where the last
one stopped instead of restarting. Total scan work for one AU is now
O(AU bytes). The from-scratch scanners are retained as a #[cfg(test)]
oracle; a new differential test asserts the resumable path yields
byte-identical AUs at every fragment granularity for all three modes.
2026-07-09 16:22:54 -07:00
Matthew Jackson 26423187d3 audit: bound the VTI clip-table scan; fix stale aacs doc links
Round-2 findings from the 10-phase release audit:

- parse_vti_clip_order bucketed hits by residue with an O(stride*hits)
  rescan and no hit cap, so a crafted HD-DVD VTI packed with millions of
  `.EVO` tokens (up to the 64 MiB UDF read cap) could burn seconds of CPU
  on a routine scan. Bucket in a single O(hits) pass and cap collected
  hits at MAX_VTI_HITS (a real table holds a few dozen).
- Fix the stale `super::keys::…` intra-doc links left by the aacs module
  rename: the referenced fns live in `super::derive`.
2026-07-09 14:41:49 -07:00
Matthew Jackson a94f78d090 audit: cap the sparse-PTS reorder buffer, FMTS key state, zero KCD
Round-1 findings from the 10-phase release audit:

- SparsePtsReorder buffered its current GOP with no bound, draining only on
  a keyframe — an open-GOP or crafted program stream that never signals one
  could hold the whole title in RAM. Force-complete the GOP at
  MAX_GOP_FRAMES, matching the MPEG-2 parser's backstop.
- inject_unit_keys labelled a 2.1 FMTS disc as AACS 1.0 / bus-encryption
  off; FMTS is UHD-family, so synthesize the UHD version + bus encryption.
- The compiled Key Correction Data was a non-zero 16-byte constant fed into
  the Media Key derivation. Per the no-compiled-keys rule it is now all-zero;
  the chain still cannot complete on a real disc (documented), so this is
  behaviour-neutral — all variant tests pass unchanged.
- Fix stale doc references (broken `super::variants` intra-doc links, and
  `aacs::keys` comments) left by the module rename.
2026-07-09 14:14:20 -07:00
Matthew Jackson 14c4227292 mux: back-patch the MKV duration from the timeline when the source has none
A title whose scan yields no duration (HD-DVD — its `.MAP` timemaps are
not parsed, so DiscTitle.duration_secs is 0) produced an MKV with no
Segment Duration element, so players/MediaInfo reported an unknown
runtime.

Reserve a DURATION placeholder when the source declares none, track the
highest block timestamp written, and back-patch the placeholder at
finish() with the real muxed runtime (also enabling the per-track BPS
tags for these titles). Gated on duration_secs == 0, so BD/UHD/DVD —
which carry a real mpls/IFO duration — write it up-front exactly as
before, unchanged.
2026-07-09 12:52:48 -07:00
Matthew Jackson fc3e1dd003 docs: changelog for HD-DVD feature composition + MPEG-2 AuAssembler 2026-07-09 09:12:56 -07:00
Matthew Jackson 5090ddab6c disc: compose the HD-DVD feature from its VTI clip table
HD-DVD Standard Content splits the main feature across clips at the
layer break (FEATURE_1/FEATURE_2, or feature/feature_Divide). The scanner
enumerated one title per .evo, so main-title selection picked only part 1
(e.g. Shaun's 11 GB FEATURE_1, missing the 6.8 GB FEATURE_2).

Parse the HVA*.VTI navigation file's clip table — a fixed-stride record
list naming every clip in authored order, isolated by residue-mod-stride
rather than the imprecise header pointer — and concatenate the feature
clips (matched by the feature* naming convention) into one title whose
extents run in authored order. Every other clip stays its own title.
Falls back to one-title-per-clip when the VTI is absent or unparseable,
so nothing regresses on a disc with no readable navigation.

Validated on real discs: Shaun 17.8 GB / Anchorman 20.1 GB / Harry Potter
24.4 GB now enumerate as one 2-clip FEATURE title (largest = the movie).
2026-07-09 09:12:13 -07:00
Matthew Jackson 3633882d6c mux: reassemble MPEG-2 access units via the shared AuAssembler
The MPEG-2 parser hand-rolled its own PES reassembly — a byte buffer plus
parallel PTS / source / discontinuity mark queues keyed by absolute
offset — duplicating what AuAssembler already does for H.264/HEVC/VC-1.

Add a Mode::Mpeg2 to AuAssembler (picture 0x00 with preceding sequence
0xB3 / GOP 0xB8 headers — the same headers-precede-picture shape as the
VC-1 mode) and have the MPEG-2 parser own one via AuAssembler::mpeg2().
parse() now feeds fragments to the assembler and processes each complete
access unit; the buffer, base offset, and three mark queues are gone. The
GOP-buffered temporal_reference reorder and PTS origin-locking are
unchanged. The parser's external contract is unchanged, so all existing
MPEG-2 parser tests pass as-is; new AuAssembler tests cover the MPEG-2
boundary rule directly.
2026-07-09 08:44:01 -07:00
Matthew Jackson ae27a097b9 docs: changelog for HD-DVD VC-1, sparse-PTS, variant layout (1.3.0) 2026-07-09 08:21:09 -07:00
Matthew Jackson 5fdff5664f mux: reconstruct display-order PTS for sparse-PTS program streams
HD-DVD EVO (and DVD VOB) program streams timestamp video at GOP
granularity: only one access unit per GOP carries a PES PTS. The H.264 /
HEVC / VC-1 parsers collapsed a missing PTS to 0, so on such a source
every non-anchor frame landed on the same block timestamp and a decoder
reported "non monotonically increasing dts".

Add a shared SparsePtsReorder that rebuilds a display-order PTS per frame
from the coded picture type (I/P/B) plus the sparse anchor PTS, with a
per-frame duration self-calibrated from the spacing between consecutive
GOP anchors (no external frame-rate needed). Display order is derived via
the classic single-anchor-delay rule (an anchor displays only after the
previously-held anchor; a B displays immediately), exact for the
non-hierarchical GOP structures HD-DVD H.264/VC-1 use. It mirrors the
MPEG-2 parser's GOP-buffered origin-locking.

Gated to the program-stream path only: the three parsers enable it via
with_ps_reorder(is_dvd_ps), so the BD/UHD transport path (per-frame PTS)
is byte-identical and untouched.
2026-07-08 21:54:09 -07:00
Matthew Jackson f8bea78db5 aacs(2.1): pin variant record layout against reference MKBs
Correct the variant Media Key chain against two real variant MKBs
(Zombieland v70, Stand By Me v70): C for Kmp is the per-slot block of
the 0x0c cvalue table indexed by the matched subset-difference, not the
0x2d head. The 0x2d record is the VARIANTS table (leading body-16 bytes,
one big-endian u16 per subset-difference) followed by a trailing 16-byte
Nonce, with no leading header. VKD stays at 0x2f.

Route record-type selection through the named REC_* consts in mkb rather
than bare hex, and document that Key Correction Data is per-licensee: no
universal constant exists and none is compiled in, so on a real disc the
chain yields a wrong Media Key that the Verify-Media-Key gate rejects
rather than emitting a bad key. This is a key-acquisition gap, not a
code gap.
2026-07-08 21:01:57 -07:00
Matthew Jackson 48bec4cc03 mux: HD-DVD VC-1 demux via extended stream id 0xFD
VC-1 HD-DVDs (e.g. Shaun of the Dead) carry video on MPEG-PS extended
stream id 0xFD, with the real stream selector in stream_id_extension
inside the PES extension. Parse that field so the video routes to a
distinct track (pid 0xFD00|ext) instead of being dropped.

Reframe VC-1 access units in AuAssembler with a dedicated Mode::Vc1:
an AU is delimited by the next frame BDU (0x0D) once a frame has already
been seen, so the sequence (0x0F) and entry-point (0x0E) headers that
precede an I-frame stay attached to the frame they describe. The old
single-start-code split stranded those headers on the prior AU, which
the decoder reported as bits-overconsumption and hard decode failures.

hddvd probe now tracks the video pid it detects and emits VC-1 on 0xFD.
2026-07-08 21:01:48 -07:00
Matthew Jackson bfe88d2673 release: prepare 1.3.0
Bump to 1.3.0 and add the 1.3.0 changelog entry (FMTS/HD-DVD formats, AACS 2.1 variant chain, recovery seam, aacs module split, main-title-by-size, and the fixes since v1.2.2). Not tagged or pushed.
2026-07-08 14:48:50 -07:00
Matthew Jackson 0d587d1154 fix: assorted correctness fixes and dead-code cleanup
- aacs/resolve: a media-keys-only provider missing the VID classifies as
  VidUnavailable, not NoMaterial (an MK derives the VUK once the VID
  arrives).
- disc/bluray: mark a clip seen only after its .clpi parses, so a
  transient parse failure on the first PlayItem cannot suppress the
  clip's extents for a later PlayItem referencing it that succeeds.
- disc/patch: log rather than swallow mapfile record/flush failures on a
  reverify downgrade, so a failed persist cannot silently mismark a bad
  unit good on resume.
- mux/ts: flag a discontinuity when a partial PES is dropped, matching
  the other partial-drop paths.
- mux/demux_thread: the no-demuxer branch forwards an empty batch for
  early consumer-disconnect detection instead of reading the whole disc.
- io/pipeline: correct the send-timing log (as_secs_f64, not as_micros
  printed as ms).
- aacs/derive, aacs/variant, disc/read_error, keysource: comment/doc
  accuracy. sector/prefetched, udf: remove dead fields/functions.
- mux/disc: assert unit-aligned read counts in the test.
2026-07-08 14:44:15 -07:00
Matthew Jackson 67aba17173 sector: generic recovery seam; FMTS forensic segments as decrypt loss
Replace the AACS-specific inline key-fetch in the decrypt decorator with
a scheme-neutral recovery seam: the input stream (L3) installs a Recover
closure (none / AACS key-fetch) and the decorator (L2) runs it at the
single decrypt-miss point. FMTS (AACS 2.1) forensic-segment units that no
key opens are just undecryptable units, concealed and counted as ordinary
decrypt loss with no FMTS-specific branch ("a loss is a loss"), so the
separate bytes_undecryptable bucket collapses into one loss count.

- sector/recovery.rs: the seam (MissOutcome, none/key_fetch factories),
  naming no encryption scheme in its type.
- FMTS: segment routing primitives + BYPASS_FMTS_KEY, and an upfront
  ensure_forensic_segments_decryptable gate (Error::FmtsKeyMissing) in
  the mux input path, parallel to the unit-key gate.
- CSS descramble/rekey moves from decrypt_sectors into
  css::descramble_region: CSS self-recovers from the data itself, so it
  stays OFF the seam (which is only for external inputs).
- disc/mod.rs also: main-title selection aligned to largest physical
  size; is_regular read from the open file handle, not metadata(path),
  fixing a swallowed sync_all on a fresh-rip ISO. decrypt_threads()
  resolved once via OnceLock off the per-buffer hot path.
2026-07-08 14:44:03 -07:00
Matthew Jackson 45c12fc5ce labels: reader-backed detection, Criterion fix, menu-language fallback
- DetectFn now takes a SectorSource so a parser can inspect a jar's
  central directory in detect() instead of firing on "any BD-J jar".
  dbp/deluxe do the real com/<vendor>/ prefix check up front, so each
  claims only its own discs (foundational for scaling the registry).
- criterion: treat a stream-map value of 0 as unmapped and synthesize a
  real 1-based number, so a 0 can't shadow or collide with a genuine
  stream 1 (with regression tests).
- png_filenames: new Low-confidence, last-resort parser reading menu
  language from {title}_UHD01_{LANG}_Composite artwork; sits below the
  MPLS floor so a real framework parser always wins.
- vocab: add menu_lang() for 639-2/B to 639-2/T menu-token normalization.
2026-07-08 14:43:52 -07:00
Matthew Jackson 3da8228068 aacs: parse FMTS SegmentKey.tbl container (16-bit variant-indexed)
New aacs::segment_key: parses the AACS 2.1 SegmentKeyNNNNN.tbl container, confirmed against a retail disc as an 8-byte header + 65536 records of 536 bytes, indexed by the 16-bit variant selector the Media Key Variant chain produces. This is the confirmed link between the two 2.1 variant layers (selector picks the device's per-segment variant). The per-record 528-byte payload layout is not yet reversed.
2026-07-07 19:18:33 -07:00
Matthew Jackson 974e886742 aacs: parse FMTS forensic segment map (IndividualSegment.tbl)
New aacs::segment: parses the AACS 2.1 IndividualSegment.tbl into the source-packet ranges of the forensic variant segments (validated against a retail disc: 792 segments, 2560 packets each). First piece of the FMTS variant decoder — the segments' variants are encrypted under segment keys, not the unit key, so a unit-key rip corrupts them (broken HEVC refs).
2026-07-07 18:59:09 -07:00
Matthew Jackson 1f3f52d225 disc: select main title by largest size, not clip count
canonical_title_order keyed on clip-count ascending before duration, so a short 1-clip bonus reel outranked the real feature — which is often chaptered into many clips (one per chapter). Make physical size the primary key: the main feature is the largest non-oversize title; decoy 'play-all' playlists run long but tiny (reused clips) and self-eliminate.

Validated across 23 UHD/BD discs: fixes t1 on F9, Fast Five, Fast & Furious 6, and Furious 7 (feature was ranked #13-36); no regressions on the 19 already correct.
2026-07-07 18:36:36 -07:00
Matthew Jackson 85347597cc disc: first-class FMTS + HD-DVD formats; CPI sample selection
Add DiscFormat::Fmts (AACS 2.1) and DiscFormat::HdDvd as first-class peers. Format derives from the AACS MKB generation (mkb_type().generation(): V10=BD, V20=UHD, V21=FMTS), reusing existing AACS code, and from the on-disc tree for HD-DVD/DVD. One detector (detect_disc_format) shared by the coarse DiscId probe and the full scan — no more 'default BluRay, defer to full scan'.

FMTS is a BD-tree stream variant: parse_playlist resolves the clip stream via CLIP_STREAM_EXTS (.m2ts -> .fmts -> .ssif), so the .fmts main feature yields real extents (previously silently empty). HD-DVD is a tree-level peer with its own enumerator (disc/hddvd.rs): HVDVD_TS/*.evo -> MpegPs titles with real extents (playlist/stream parsing honestly stubbed).

Sample selection for key resolution now uses the authoritative AACS CPI flag (aacs_unit_encrypted, byte-0 & 0xC0) not the ts_sync_destroyed heuristic — container-agnostic (M2TS/FMTS/EVO; TS-sync is meaningless on HD-DVD program streams) and stops the decode-server '0 encrypted units' rejection.

Tests live with each format (bluray/hddvd/mod); generic UDF fixture builders extracted to a shared udf::fixture module.
2026-07-07 18:16:04 -07:00
Matthew Jackson bc04ee7bd2 unlock_bridge: run_features/run_bus dispatch; report LibreDrive vs Renesas
Adopt freemkv-unlock's split Unlocker trait: run_features drives the drive-prep capability, run_bus the content bus removal, each iterating unlockers until one doesn't decline (NotApplicable = try next; Ok or a real error stops).

unlocker_matrix now reports which drive-prep unlocker actually ran — LibreDrive removes the bus at the drive; Renesas unlocks features but leaves the bus to the cert. Wire product_id through to fu::DriveId.

Bump to 1.2.3.
2026-07-07 16:36:05 -07:00
Matthew Jackson 122a03b23d aacs: remove boil veneer; move key newtypes to aacs::types
leak-guard / leak-guard (push) Successful in 8s
aacs::boil was a thin newtype veneer over aacs::derive — a duplicate layer.
Delete it: move the shared key newtypes (Vid, MediaKey, Vuk, ProcessingKey,
UnitKey) into aacs::types, and expose resolve_candidate + KeyCandidate from
aacs::derive directly. Downstream (keysource, disc::encrypt) now import from
aacs::{derive,types}. Pure API consolidation, no behaviour change; full test
suite green on Rust 1.86.
2026-07-05 12:09:39 -07:00
Matthew Jackson dcc553a716 aacs: keep flat re-exports for the typed key primitives
leak-guard / leak-guard (push) Successful in 8s
The module structure is the public API, but the typed key primitives
(MediaKey/UnitKey/Vid/Vuk, mk_from_dk/mk_from_pk/uk_from_vuk/vuk_from_mk),
derive_vuk, the aligned-unit decrypt entry points, and DeviceKey/HostCert
are load-bearing names that downstream key-source crates import through the
flat `aacs::` path. Re-export them here so those crates can track the
module refactor without a lockstep re-pin.
2026-07-04 22:19:39 -07:00
Matthew Jackson 3b06a4c844 aacs(2.1): variant Media Key as a clean PK -> Km primitive
leak-guard / leak-guard (push) Successful in 19s
Redesign derive_media_key_variant to the minimal derivation surface:

  derive_media_key_variant(mkb_records, pk) -> Km

- PK-input only. Deriving Kp from device keys (DK -> PK) is a separate
  concern (walk_processing_key); a leaked 2.1 key is a PK, and the chain
  starts at Kp. A bare PK arrives without its subset-difference slot, so
  the primitive tries it against every slot and returns the Km for the
  slot that passes the MKB's Verify-Media-Key record -- mirroring the
  classical bare-PK derive_media_key_from_pk, gated by the chain's own
  verify so an unverified key is never returned.
- VID-free: the Media Key is MKB-scoped. VUK stays the separate
  derive_vuk(Km, VID) step.
- KCD is a fixed algorithm constant compiled in, not a caller parameter
  (removes the kcd argument, the placeholder const, and KcdNotProvided).
- Soft-correction / online-challenge slots are treated as non-covering;
  surfaced over the generic miss so a disc needing those modes is
  distinguishable from a non-covering key.

resolve_keys_v21 updated to walk DK -> PK first, then call the primitive
and derive the VUK from Km + VID. Module + helper docs refreshed to the
pinned record layout; tests reworked for the PK-only signature.
2026-07-04 19:16:40 -07:00
Matthew Jackson f9d112e481 aacs(2.1): wire variants_for_uv from the 0x2d record; chain runs end-to-end
variants_for_uv previously returned None (placeholder), dead-stopping the
Media Key Variant chain at VariantsTableUnavailable on any real disc. Layout
now pinned against two real 2.1 variant MKBs (Zombieland v70, Stand By Me v70):
the 0x2d Encrypted-Media-Key-Variant-Data body is sd_count u16 VARIANTS entries
(1:1 with 0x0c cvalues / 0x04 subset-diffs) followed by the 16-byte per-disc
Nonce at the tail. variants_for_uv reads the sd_slot_index-th u16.

With this the chain runs fully: Kmp -> Kpnew(=Kmp^KCD, extracted CyberLink
constant) -> VKD(0x2f) -> Km -> Kvu. The only remaining input is a covering
2.1 Processing Key to validate against a known answer; until then the final
Verify-Media-Key (0x86) gate rejects any wrong layout pick, so a bad key can
never be emitted. Tests updated to reflect the wired lookup; 1.86 precommit
green (fmt+clippy+tests).
2026-07-04 18:26:09 -07:00
Matthew Jackson f4fe651cb9 aacs: point integration tests at real module paths
The facade removal (5ff0464) left tests/ calling aacs::disc_hash,
aacs::decrypt_unit, aacs::AacsVersion, etc. at the old flat paths,
so the branch did not compile its integration tests. Repoint each
to its real module (inf/derive/content/mkb). No logic change.
2026-07-04 18:25:59 -07:00
Matthew Jackson 5ff04649ba aacs: remove the mod.rs facade — module paths are the public API
Delete the pub-use re-export facade; consumers now import from the owning
module (aacs::content::decrypt_unit, aacs::mkb::MkbType, aacs::derive::derive_vuk,
aacs::boil::mk_from_dk, aacs::resolve::resolve_keys_v2, ...). Internal callers
repointed accordingly. Path-only change; logic hash identical (95fb9924); 2210
tests green.
2026-07-04 14:19:40 -07:00
Matthew Jackson 84aaceceb7 aacs: merge media_key + volume_key into derive.rs
The volume_key module was only 34 lines and is just the tail of the same
DK/PK -> MK -> VUK -> UK derivation ladder as media_key. Fold both into one
derive module so every aacs module is a substantial, distinct responsibility
(crypto/mkb/derive/inf/content/variant/resolve). Relocation only; logic hash
identical (95fb9924); 2210 tests green.
2026-07-04 14:14:56 -07:00
Matthew Jackson cdee9739fd aacs: split keys.rs god-module into media_key/volume_key/inf/resolve
Break the 2800-line keys.rs into four responsibility-scoped modules:
- media_key.rs: DK/PK -> Media Key subset-difference walk (+ probe harness)
- volume_key.rs: VUK derivation, unit-key unwrap
- inf.rs: Unit_Key_RO.inf parsing, disc_hash, content cert, in-drive MKB read
- resolve.rs: the resolve_keys_* orchestration (keys.rs renamed)

Relocation only; the (white-box) test suite stays in resolve.rs and pulls
the moved items via glob imports. Proven byte-identical to the pre-refactor
state via the logic hash (95fb9924); 2210 tests green.
2026-07-04 14:06:57 -07:00
Matthew Jackson f55f11d043 aacs: extract mkb.rs (MKB record format, MkbType, finders)
Relocate the shared MKB machinery into a single mkb module: the record
framing walker + MkbRecord view (from variant), the MkbType/AacsVersion
classification, the MKB-file utilities, and the record-body finders (from
keys). Fixes the inversion where the MKB parser lived in the 2.1-only
variant module. variant.rs keeps its local MkbRecord-based mkb_find_mk_dv
(name collision with the raw one; unified in the dedup follow-up).

Relocation only. Proven byte-identical to the pre-refactor state via the
function-body logic hash (imports normalized out); 2210 tests green.
2026-07-04 13:55:06 -07:00
Matthew Jackson f55d8f7acd aacs: extract crypto.rs (shared AES primitives + constants)
Relocate the shared low-level primitives into a single crypto module:
aes_ecb_encrypt/decrypt, aes_cbc_decrypt, aes_g (from content/variant) and
aesg3 + AESG3_SEED (from keys), plus AACS_IV. Fixes the scatter where AES-G
lived in the 2.1 file and AES-G3 in keys. Relocation only — no rename, no
logic change (logic-hash identical to baseline; 277 items; 2210 tests green).
2026-07-04 13:37:53 -07:00
Matthew Jackson 31b0ba323a aacs: rename decrypt.rs->content.rs, variants.rs->variant.rs (no logic change)
Pure file+module-path rename. 'content' names the AACS unit-decrypt layer
(distinct from the top-level sector-decrypt driver crate::decrypt), and
'variant' (singular, spec term 'Media Key Variant') names the 2.1 chain.
Logic-hash identical to baseline; 277 items intact; tests green.
2026-07-04 13:24:26 -07:00
Matthew Jackson 188baced39 aacs: annotate crypto with AACS spec section citations
Add [C]/[PR]/[BD]/[libaacs] §x.y provenance markers across the AACS
crypto so each primitive links to the spec section it implements, with a
source-tag legend in mod.rs. Doc-comments only — no logic, constant, or
signature changes.

Also: correct two stale record-type comments in variants.rs (0x82/0x83 →
the real 0x2d/0x2f) and document the Variant Number width (spec lsb_10 vs
the 2.1 chain's lsb_16, driven by the 65,535-entry VKD table).
2026-07-04 13:15:53 -07:00
Matthew Jackson bce11a2de0 restore freemkv-unlock path dep for local dev (post-v1.2.2) 2026-07-03 20:50:17 -07:00
Matthew Jackson 7a79577343 v1.2.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-03 20:50:14 -07:00
Matthew Jackson f07251c2d4 changelog: 1.2.2 (AACS 2.1 variant MKB, mk_from_dk fix, PK-path perf, autorip online-outage + move-error clear) 2026-07-03 20:48:04 -07:00
Matthew Jackson eb23ab4586 aacs: target real AACS 2.1 Variant MKB records (0x2d/0x2f/0x0c)
The Media Key Variant scheme is detected and parsed via the actual MKB
record types found on a real variant disc:
  - 0x2d  Encrypted Media Key Variant Data (C)
  - 0x2f  Variant Key Data table (65,535 x 16)
  - 0x0c  variant cvalues (one per 0x04 subset-difference slot)

Replaces the earlier placeholder 0x82/0x83 record types, which were a
guess and appear on no real MKB. is_variant_mkb, the record finders, and
the subset-difference cvalue source (now 0x0c, falling back to 0x07/0x05)
are updated accordingly, along with the V20->V21 upgrade detection in
resolve_keys_v2 and its fixtures.

The variant chain still halts at variants_for_uv (the VARIANTS[uv] /
Nonce sub-field offsets need a covering key to confirm end-to-end
against the 0x86 verify), so a best-effort offset is never silently
trusted. All 27 variant tests pass on Rust 1.86.
2026-07-03 13:25:01 -07:00
Matthew Jackson 61b6070f03 aacs: make the PK path terminal (no SD-tree walk)
A Processing Key is the key at its Subset-Difference node — one AES-G from
the Media Key — so it is tried directly against the MKB cvalue tables,
matching libaacs _calc_mk_pks (iterate PKs × cvalues). The prior code
treated every PK entry as a device-node label at unknown depth and BFS-walked
the SD tree (depth 3, capped 5), which was both wrong for terminal PKs and
~15x slower on a large UHD MKB (~181k cvalues): PK derivation on UHD dropped
from ~37s to ~2.4s.

The Subset-Difference tree walk now lives solely in the device-key path
(derive_media_key_from_dk), which owns per-node path bits; the PK path never
descends. Removed PK_WALK_MAX_DEPTH / _CAP / _walked / walk_pk_against_tables_impl;
renamed the core scan try_pk_against_tables and its probe test.

Precommit (Rust 1.86): fmt + clippy + tests green.
2026-07-02 23:17:20 -07:00
matthew 125a8e5bf0 aacs: full-ladder resolve_candidate + correct mk_from_dk
leak-guard / leak-guard (push) Successful in 24s
resolve_candidate(candidate, mkb, unit_key_ro, vid) -> ResolvedChain: one
composed, PURE-DERIVATION boil-down for a candidate key at any ladder rung
(KeyCandidate::{Uk,Vuk,Mk,Pk,Dk}, each carrying the module's newtype). Walks
DK/PK/MK/VUK -> terminal unit keys, parsing Unit_Key_RO.inf at the version
the disc's MKB declares, and returns every CPS unit key as (cps_unit, key)
(matching ResolvedKeys) plus the intermediate chain. No sampling/validation
(that's the caller's unit_key_validates) and no position recovery (Dk is a
positioned DeviceKey; recover_dk_position first). Adds ProcessingKey newtype.
Consumers stop re-composing the ladder; every client hardens one impl.

Correct mk_from_dk: real Subset-Difference walk (derive_media_key_and_pk_from_dk)
instead of the Media-Key-Variant path, which needed integrator KCD absent
in-tree and Err'd for every real disc -- dead for both consumers. Drops the
now-unused vid arg.
2026-07-01 22:34:52 -07:00
Matthew Jackson 741c1ea11d restore freemkv-unlock path dep for local dev (post-v1.2.1) 2026-07-01 19:08:56 -07:00
Matthew Jackson e713b26b87 v1.2.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-01 19:08:53 -07:00
Matthew Jackson fcbd667add changelog: describe the shipped re-base DTS fix + faithful-passthrough note 2026-07-01 19:02:04 -07:00
Matthew Jackson 88c03152e2 mux/dts: re-base PTS per PES instead of a running clock (fix long-title drift)
The first cut used a global running clock (max(next, own-PES PTS) + advance),
which fixed the same-PES collision but DRIFTED: once accumulated frame
durations exceeded the PES-timestamp spacing, it never re-based, so a
feature-long DVD DTS track ran minutes past its real length (2h44 for a 2h03
film) while AC-3 from the same source stayed exact.

Match the AC-3 path: re-base to each PES's own container timestamp, and advance
by one frame duration ONLY within a run of AUs sharing one PES. Fixes the DVD
multi-frame-per-PES collision without drift; the UHD DTS-HD MA per-PES
attribution (da85f56) is preserved (each AU still takes its own core PES's PTS).
Adds new_pes_rebases_to_its_own_pts_no_drift; full mux suite green (905).
2026-07-01 18:21:25 -07:00
Matthew Jackson c8e7ad5e56 v1.2.1: bump version + changelog (DVD DTS timestamp fix) 2026-07-01 17:30:25 -07:00
Matthew Jackson f122f08628 mux/dts: monotonic per-frame PTS + real frame duration (fixes DVD DTS)
DVD packs several DTS core frames into one PES; the parser stamped every
access unit with that single PES PTS and duration_ns=None, so consecutive
frames collided on one timestamp — ffmpeg rejected the output as 'non
monotonically increasing dts to muxer: X >= X' (deep-decode = corrupt,
e.g. The Punisher). The UHD DTS-HD MA path (one AU per PES, distinct PTS)
was unaffected, which is why this only surfaced on DVD.

Parse the DTS core header for samples ((NBLKS+1)*32) and sample rate
(SFREQ, 48kHz fallback) to derive each AU's duration, and stamp a running
monotonic PTS: max(next_clock, own-core-PES PTS), then advance by the
frame duration. A later PES whose PTS is ahead of the clock still wins
(preserves the UHD per-PES attribution from da85f56/c49a180); frames
sharing one PES advance frame-by-frame instead of colliding.

Tests: the 3 that encoded 'same PES -> same PTS' now assert monotonic
advance; new dvd_many_cores_one_pes_are_strictly_monotonic reproduces the
Punisher bug; duration/SFREQ-fallback unit tests added.
2026-07-01 17:17:35 -07:00
Matthew Jackson b2c3540989 restore freemkv-unlock path dep for local dev (post-v1.2.0) 2026-07-01 16:53:06 -07:00
Matthew Jackson 3a4307def6 v1.2.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-01 16:53:03 -07:00
Matthew Jackson 917026d566 changelog: DVD DTS/LPCM silent-audio fix + Pass-N recovery specialists + flat scheduler 2026-07-01 16:50:08 -07:00
187 changed files with 89945 additions and 26762 deletions
+63
View File
@@ -0,0 +1,63 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+188 -10
View File
@@ -2,50 +2,228 @@ name: CI
on:
push:
branches: [main]
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
pull_request:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
- run: cargo clippy -- -D warnings
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
working-directory: libfreemkv
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo check
working-directory: libfreemkv
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
# Build the tests (not just `cargo check`): catches errors in test
# code and forces full codegen of the Windows-only SPTI transport
# (src/scsi/windows.rs), which never compiles on the Linux/macOS dev
# hosts. We don't `cargo test` here — the suite needs no drive but the
# extra build is the value; running tests is covered by the Linux job.
- run: cargo build --tests
working-directory: libfreemkv
# ── Did this change break anything downstream? ──────────────────────────────
#
# Every job above proves libfreemkv builds. None proved its DEPENDENTS do,
# and that gap is real: an engine signature change broke autorip today and
# went unnoticed because consumer CI only fires on a push to that consumer.
# libfreemkv sits below all five of them, so a break here is worth strictly
# more than a break anywhere else in the project.
#
# `cargo check --all-targets` only — each dependent owns its own behaviour
# and has its own suite. The question here is just "does everything built on
# me still compile against this commit".
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo test --tests
working-directory: freemkv
- run: cargo test --tests
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
+133
View File
@@ -0,0 +1,133 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
on:
push:
branches: [qa]
workflow_dispatch:
jobs:
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
working-directory: libfreemkv
+16 -29
View File
@@ -4,6 +4,11 @@ on:
push:
tags:
- 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions:
contents: write
@@ -12,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -24,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.86, same commit). Binary consumers
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -32,36 +37,18 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
- run: cargo test
# crates.io publish is an INDEPENDENT job: it serves EXTERNAL consumers only.
# The freemkv binaries no longer depend on it (they git-tag-pin libfreemkv via
# a committed [patch.crates-io]), so this publish runs in parallel with their
# release builds rather than gating them. It `needs: [verify, test]` so a
# failing test suite still blocks publication to crates.io — external
# consumers who `cargo add libfreemkv` must never receive a release whose
# tests were failing. (The two upstream jobs run in parallel, so this gate
# does not serialize publish behind test beyond their own completion.)
publish:
needs: [verify, test]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: Swatinem/rust-cache@v2
# --no-verify: CI already compiled this exact commit (in the `test` job
# and on every push via ci.yml). cargo publish's default re-verify does a
# full cold release build of the packaged tarball, which here is pure
# redundant work (~a cold lib build). Skip it.
- name: Publish to crates.io
run: cargo publish --no-verify
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
# (`package.publish = false` — it git-deps the firmware crate freemkv-unlock,
# which never ships to crates.io). Every consumer git-tag-pins libfreemkv via
# a committed [patch.crates-io]; the git tag itself IS the release artifact.
# A `cargo publish` here fails hard on `publish = false`, so it was removed.
release:
# Only needs `verify`; the GitHub Release can be cut as soon as the version
@@ -69,8 +56,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
-36
View File
@@ -1,36 +0,0 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push
+9
View File
@@ -14,3 +14,12 @@ scratch/
# internal agent context — never publish (path AND dir; leak-guard blocks both)
CLAUDE.md
.claude/
# Nightly harness output. Written into the repo it audits, and it embeds
# absolute paths from the machine that ran it — which must never reach a public
# repo. Ignored rather than relocated so a run from any working copy is safe.
.nightly/
# cargo-mutants working output: large, machine-specific, never committed
mutants.out/
mutants.out.old/
+859 -496
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -36,7 +36,7 @@ This Code of Conduct applies within all community spaces, and also applies when
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at matthew@pq.io. All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+1 -1
View File
@@ -24,4 +24,4 @@ cargo test
## License
By contributing, you agree your code will be licensed under AGPL-3.0.
By contributing, you agree your code will be licensed under MIT.
+16 -15
View File
@@ -1,9 +1,9 @@
[package]
name = "libfreemkv"
version = "1.2.0"
version = "1.6.6"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
rust-version = "1.97"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
keywords = ["bluray", "uhd", "optical", "scsi", "disc"]
@@ -22,21 +22,22 @@ codegen-units = 1
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { path = "../freemkv-unlock" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.5" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
+17 -12
View File
@@ -1,16 +1,21 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
MIT License
Copyright (C) 2026 FreeMKV Contributors
Copyright (c) 2026 Matthew Jackson & Contributors
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, version 3 of the License.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+11 -44
View File
@@ -1,4 +1,4 @@
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
# libfreemkv
@@ -55,48 +55,15 @@ output.finish()?;
### Multi-pass recovery rip
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md).
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the
ddrescue mapfile, damage classification and the multipass loop now live in the
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`.
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
libfreemkv keeps the layers underneath: the raw single-shot read
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's
strategy is built on. The dependency runs engine → libfreemkv, so this crate
cannot call into it; front-ends get recovery from the engine directly. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
## What It Does
@@ -114,7 +81,7 @@ loop {
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
@@ -190,4 +157,4 @@ Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/fre
## License
AGPL-3.0-only
MIT
+22
View File
@@ -0,0 +1,22 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
+3 -3
View File
@@ -141,8 +141,8 @@ If your machine has a free SATA port, use it.
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
@@ -302,7 +302,7 @@ Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not adver
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
2. **Drive model:** from the drive label, or from `freemkv info disc://`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
+15 -8
View File
@@ -22,7 +22,7 @@ fn main() {
&target_arch // x86_64 → x86_64
};
std::process::Command::new("cc")
let cc_status = std::process::Command::new("cc")
.args([
"-arch",
clang_arch,
@@ -38,12 +38,19 @@ fn main() {
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
.expect("failed to spawn cc for macos_shim.c");
// `.status()` succeeding only means the process RAN. A real compile error
// exits non-zero, and ignoring that left no object file, which surfaced
// much later as an unexplained link failure against a missing symbol. The
// shim is macOS-only and is neither linted nor compiled on the other two
// platforms, so a mistake in it has exactly one chance to be noticed.
assert!(cc_status.success(), "cc failed to compile macos_shim.c");
std::process::Command::new("ar")
let ar_status = std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
.expect("failed to spawn ar");
assert!(ar_status.success(), "ar failed to create the static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
@@ -77,10 +84,10 @@ fn emit_git_suffix() {
// Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD") {
if let Some(ref_path) = head.strip_prefix("ref: ") {
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
if let Ok(head) = std::fs::read_to_string(".git/HEAD")
&& let Some(ref_path) = head.strip_prefix("ref: ")
{
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
+27 -16
View File
@@ -65,27 +65,38 @@ if disc.encrypted {
}
}
// Read content -- decryption is automatic
let mut reader = disc.open_title(&mut session, 0).unwrap();
while let Some(unit) = reader.read_unit().unwrap() {
// decrypted content
// Read content -- decryption is applied on read by the DiscStream decorator.
// Live disc does NOT go through the URL resolver: `input("disc://...")` returns
// Error::DiscUrlNotDirect by design.
let keys = disc.decrypt_keys();
let mut stream = DiscStream::new(
Box::new(drive),
disc.titles[0].clone(),
keys,
batch_sectors,
disc.titles[0].content_format,
false, // raw: false → decrypt on read
None, // halt
)?;
while let Ok(Some(frame)) = stream.read() {
// decrypted PES frames
}
```
The application never touches keys, never calls decryption functions, and never
manages handshakes. All of that is internal to `Disc::scan()` and the content
reader.
The application never calls decryption functions and never manages the
drive-level handshake. It DOES own key resolution — see below.
### KEYDB Location
### Key resolution is the caller's job
`ScanOptions` controls where the keydb is loaded from. If no explicit path is
set, the library checks the standard config locations. To specify an explicit
path:
`libfreemkv` is **lookup-free: it resolves no keys and reads no keydb.** There is
no `ScanOptions::with_keydb`, and `ScanOptions` has no keydb field — its only
scan input is the optional drive credentials for the live-drive authenticated
handshake.
```rust
let opts = ScanOptions::with_keydb("/path/to/keydb.cfg");
let disc = Disc::scan(&mut session, &opts).unwrap();
```
The caller resolves a key out-of-band through a key source and applies it with
[`Disc::decrypt_with`]. `freemkv-keysources` is the crate that implements the
keydb and key-server sources; `ScanOptions::key_sources` takes them as
`Box<dyn KeySource>`.
### AacsState
@@ -97,7 +108,7 @@ After a successful scan, `disc.aacs` contains an `AacsState`:
| `bus_encryption` | `bool` | Whether bus encryption is active |
| `mkb_version` | `Option<u32>` | MKB version from disc |
| `disc_hash` | `String` | Identifier for the disc's key-input files |
| `key_source` | `KeySource` | How the disc's key was resolved |
| `key_source` | `KeyOrigin` | How the disc's key was resolved |
## keydb.cfg
+10 -7
View File
@@ -170,17 +170,20 @@ libfreemkv/src/
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── mod.rs Drive struct, init, read (single-shot), eject
│ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
│ ├── mod.rs Disc struct, scan, titles, formats; Disc::copy + Disc::sweep (Pass 1)
│ ├── sweep.rs Pass 1 internal helpers (pub(super))
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── mapfile.rs ddrescue-format mapfile
── read_error.rs ReadCtx / ReadAction state machine
├── disc/ Disc (scan, titles, AACS setup, per-format parsing)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── bluray.rs Blu-ray / UHD scanning (MPLS/CLPI-driven)
│ ├── dvd.rs DVD-Video scanning (IFO-driven)
│ ├── hddvd.rs HD-DVD scanning
── extract.rs Per-extent content extraction
│ ├── encrypt.rs Encrypted-range mapping for content reads
│ ├── dvd_audio_probe.rs DVD audio-stream probing
│ └── pgs_forced_probe.rs PGS forced-subtitle probing
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── unlock.rs Unlocker trait + registry (pluggable unlock seam)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
+4 -3
View File
@@ -7,7 +7,7 @@ AACS decryption requires an external `keydb.cfg` (default
material is compiled in; DVD CSS player keys are the only compiled-in keys.
**Repository:** <https://github.com/freemkv/libfreemkv>
**License:** AGPL-3.0-only
**License:** MIT
---
@@ -96,12 +96,13 @@ After open:
- `init()` -- routes to the matching registered unlocker (if any); otherwise
a no-op and the cert handshake carries the disc
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (`READ_TIMEOUT_MS` 10 s vs. `READ_RECOVERY_TIMEOUT_MS` 60 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
(`freemkv_engine::recovery::patch`, in the engine crate) handles bad-range
retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
+15 -5
View File
@@ -69,13 +69,23 @@ Each stream PID entry header (14 bytes):
```
Offset Size Field
------ ---- -----
0 2 Stream PID
2 2 Reserved + EP stream type
4 2 Number of coarse entries
6 4 Number of fine entries (note: 32-bit, can be large)
10 4 EP map start offset (relative to EP map start)
2 2 stream_PID (byte-aligned)
4 10 Bit-packed block, 80 bits total (see below)
```
The stream PID entry is **not** byte-aligned past `stream_PID`. Bytes 4..14 are one
80-bit packed field, read as a `u64` plus a trailing `u16`:
Bits Width Field
---- ----- -----
0-9 10 reserved
10-13 4 EP_stream_type
14-29 16 num_EP_coarse
30-47 18 num_EP_fine
48-79 32 EP_map_start_address (relative to the EP map start)
Note `num_EP_fine` is **18 bits**, not 32. See `parse_cpi` in `src/clpi.rs`.
libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking.
### Two-Level Index
+2 -2
View File
@@ -79,7 +79,7 @@ Insert disc
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — Disc::patch re-runs against the mapfile.
│ 1 above this — freemkv_engine::recovery::patch re-runs against the mapfile.
PES frames → output stream (MKV, M2TS, network, etc.)
@@ -123,7 +123,7 @@ output.finish()?;
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan (sweep/patch/mapfile moved to freemkv-engine in 1.6.0) |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
+4 -3
View File
@@ -58,15 +58,16 @@ selects the per-CDB timeout:
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
| `false` | 10 s | `freemkv_engine::recovery::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 60 s | `freemkv_engine::recovery::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
- **Layer 1 — `freemkv_engine::recovery::patch`** (in the engine crate, not
here) loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
+64 -167
View File
@@ -1,135 +1,38 @@
# Rip recovery — three-layer architecture
# Rip recovery — what libfreemkv owns
`libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
discs: a fast forward sweep that tolerates read failures, in-loop request-size
adaptation that survives transient drive trouble without bailing, and targeted
retry passes against a persistent bad-range map. The stream pipeline
(`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
the mux stage never touches the drive.
**Recovery strategy moved OUT of this crate in 1.6.0.** The forward sweep, the
targeted retry pass, the ddrescue mapfile, damage classification and the
multipass loop now live in the **`freemkv-engine`** crate as
`freemkv_engine::recovery::{copy, sweep, patch}`. The dependency runs
engine → libfreemkv, so this crate cannot call into the engine; front-ends
(`freemkv` CLI, autorip) get recovery from the engine directly.
Recovery is layered cleanly. Each layer has one responsibility and does not
reach into the others.
What stayed here are the two layers underneath the strategy: the single-shot
read primitive, and the in-stream request-size adaptation that sits in front of
it. This document covers those, plus the design constraints they encode — the
constraints are the reason the strategy above them looks the way it does, so
they belong with the code that enforces them.
For the strategy itself — damage-jump thresholds, pass ordering, mapfile status
state machine, wedge detection — read `freemkv-engine/src/recovery/`.
| Layer | Where it lives | What it does |
|-------|---------------|--------------|
| 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 1 — Bad-range retry | **`freemkv-engine`** (`recovery::patch`) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` in `src/mux/disc.rs` | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
The library exposes flat verbs; the caller drives the multipass loop. Autorip
runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
clean or the configured retry budget is exhausted, then hands the ISO off to
the mux pipeline. The `freemkv` CLI does the same shape with a
terminal-output progress sink. Layer 3 runs inside any consumer of
`DiscStream` (direct PES pipeline, ISO playback, etc.) without caller
involvement.
Layer 2 also translates drive facts: [`SenseFamily`](../src/scsi/mod.rs)
classifies SCSI sense data into the categories the engine's strategy routes on
(marginal vs. hardware vs. not-ready). Getting that classification wrong
silently misroutes recovery, which is why it lives next to the transport rather
than in the strategy.
Three primitives compose the disc-side flow:
Layer 3 runs inside any consumer of `DiscStream` — direct PES pipeline, ISO
playback — without caller involvement, and applies whether or not the engine's
recovery is in play.
| Primitive | What it does |
|---------------------------|-----------------------------------------------------------------------|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
## Data model
### Mapfile
Format: [ddrescue](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)-compatible
plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
so a crashed rip loses at most one block.
```
# Rescue Logfile. Created by libfreemkv v0.13.6
# Current pos / status / pass / pass_time
0x000000000 ? 1 0
# pos size status
0x000000000 0x12a35d000 +
0x12a35d000 0x000003000 -
0x12a360000 0x009c4a000 +
0x12d00a000 0x000064000 *
```
Status characters match ddrescue:
| Char | Meaning |
|------|----------------------------------------------------|
| `?` | Not yet attempted |
| `*` | Fast-pass failed; needs edge-trim |
| `/` | Trimmed; interior needs sector scrape |
| `-` | Unreadable this session |
| `+` | Finished (good) |
Position and size are hex byte offsets into the ISO.
### `SweepOptions` and `PatchOptions`
The library no longer dispatches between sweep and patch internally — the
caller picks the verb explicitly per pass. The two option structs are flat
and have no overlap:
```rust
SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true, // damage-jump + zero-fill on read failure
progress: Some(&reporter),
halt: Some(flag),
}
PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true, // walk bad ranges high → low LBA
wedged_threshold: 50,
progress: Some(&reporter),
halt: Some(flag),
}
```
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
- No mapfile → `sweep` (fresh Pass 1).
- Mapfile with `?` ranges → `sweep` with `resume: true`.
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
- Mapfile clean → done; no further pass needed.
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
lines of `Mapfile::stats()` checks.
## Algorithm
### Pass 1 — fast sweep (`Disc::sweep`)
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
**damage-jump**: skip ahead by `1024×batch×multiplier` sectors (64 MB base for
UHD). Double the multiplier on each jump (64→128→256→512 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
## In-stream — adaptive batch halving (`DiscStream::fill_extents`)
When a consumer reads a `DiscStream` directly (no ISO intermediate),
`fill_extents` runs an adaptive sizer in front of `Drive::read`:
@@ -143,59 +46,53 @@ When a consumer reads a `DiscStream` directly (no ISO intermediate),
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
`Err(DiscRead)`.
This is layer 3. It exists so a transient single-sector glitch in a 32-sector
batch can be isolated and read individually without the caller needing to
implement retry logic.
This exists so a transient single-sector glitch inside a 32-sector batch can be
isolated and read individually without the caller implementing retry logic. See
[`src/event.rs`](../src/event.rs) for the emitted events.
## Design choices
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
handles ranges, layer 3 handles request size, neither touches the
wedge-prone reset path.
These are constraints on the read path, enforced here and relied on by the
engine's strategy.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue
nor any consumer ripper does this. Drive firmware has access to raw analog signal, laser
power control, and drive-specific ECC tuning that userspace can't replicate —
disabling it throws away recovery headroom on marginal sectors. We fail fast
via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
in pass 2 / patch.
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset, no
eject cycle. The `recovery` flag controls only the per-CDB timeout (10 s vs.
60 s); on any failure it returns `Err(DiscRead)` immediately. Inline recovery
(5× gentle retry → close + SCSI reset + reopen → 5× more) was removed in
0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale: the inline
reset on the LG BU40N (Initio USB-SATA bridge) wedged drive firmware below the
bridge without ever recovering a sector, and the gentle-retry phase produced
long stretches of 0 KB/s with nothing to show for it. Recovery responsibility
is layered instead: layer 1 handles ranges, layer 3 handles request size,
neither touches the wedge-prone reset path.
**No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
`SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
The macOS reset (which had been a no-op) was removed entirely. The top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
also removed (no callers). The remaining `Drive::reset()` is only invoked
explicitly by callers that need an eject-cycle escape hatch — it is never
reached from a read path.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue nor any
consumer ripper does this. Drive firmware has access to raw analog signal,
laser power control and drive-specific ECC tuning that userspace cannot
replicate — disabling it throws away recovery headroom on marginal sectors. The
fast pass fails quickly via short SG_IO timeouts and lets the firmware work the
long timeout during retry.
**ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
mux stage reads the ISO via `FileSectorSource`. For single-pass (no retries),
this adds ~2-3 min (local disk mux) but gains resumability across crashes,
**No SCSI reset from any read path.** There is no reset escape hatch on
`Drive` at all: the `SG_SCSI_RESET` ioctl and STOP/START UNIT escalation went in
0.13.6, the macOS reset (always a no-op) was removed entirely, and the
top-level `scsi::reset()` wrappers went with their last callers. The only
remaining reset is a Windows-specific device-level helper in
[`src/scsi/windows.rs`](../src/scsi/windows.rs), never reached from a read.
**ISO intermediate, even for single-pass.** The engine's Pass 1 always writes
an ISO, and the mux stage reads it back via `FileSectorSource`. For a
no-retry rip this costs a few minutes but buys resumability across crashes,
re-muxability without re-ripping, and a persistent forensic artifact. Callers
who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
still applies there.
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
coalesce on adjacent same-status ranges so files stay small.
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
ddrescue's but `patch` collapses the distinction — it just tries every
non-finished range with the long timeout. Future work can specialize (trim vs.
scrape vs. retry with direction reversal) if there's measured benefit.
who need pure speed can bypass it with `DiscStream::new(Box::new(drive), …)`
nothing forbids it, and layer 3 still applies there.
## References
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
- Recovery strategy and mapfile: `freemkv-engine/src/recovery/`
- In this crate: [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`),
[`src/scsi/mod.rs`](../src/scsi/mod.rs) (`SenseFamily`),
[`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`),
[`src/event.rs`](../src/event.rs) (progress events).
+1 -1
View File
@@ -114,7 +114,7 @@ The `read_filesystem()` function in `src/udf.rs` follows the pointer chain above
2. Scans sectors 32-63 for the Partition Descriptor and Logical Volume Descriptor.
3. If two partition maps exist and the second is Type 2, reads the metadata file ICB at partition_start to find metadata_start.
4. Reads the FSD at metadata_start, extracts the root directory ICB LBA.
5. Calls `read_directory()` recursively (max depth 3) to build the full file tree.
5. Calls `read_directory()` recursively (max depth `MAX_DIR_DEPTH` = 8) to build the full file tree.
Each directory read involves two sector reads: one for the ICB, then one or more for the directory data. File sizes are read from info_length in each file's ICB.
-83
View File
@@ -1,83 +0,0 @@
// Minimal ISO dumper — find exact stall point
use libfreemkv::Drive;
use std::io::{BufWriter, Write};
use std::path::Path;
use std::time::Instant;
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("Usage: iso_dump <device> <output>");
std::process::exit(1);
}
let mut drive = Drive::open(Path::new(&args[1])).unwrap();
drive.wait_ready().unwrap();
let _ = drive.init();
let _ = drive.probe_disc();
// AACS handshake — required to read past the protected area
eprint!("Scanning disc... ");
let _ = libfreemkv::Disc::scan(&mut drive, &libfreemkv::ScanOptions::default());
eprintln!("OK");
let cap = drive.read_capacity().unwrap();
let batch = libfreemkv::disc::detect_max_batch_sectors(drive.device_path());
eprintln!("Device: {} | {} sectors | batch {}", args[1], cap, batch);
let file = std::fs::File::create(&args[2]).unwrap();
let mut w = BufWriter::with_capacity(4 * 1024 * 1024, file);
let mut buf = vec![0u8; batch as usize * 2048];
let mut lba: u32 = 0;
let start = Instant::now();
let mut last = Instant::now();
let mut bytes: u64 = 0;
let mut last_bytes: u64 = 0;
while lba < cap {
let count = ((cap - lba) as u16).min(batch);
let n = count as usize * 2048;
// Tiny yield between reads — test if pacing prevents firmware throttle
std::thread::yield_now();
let t0 = Instant::now();
let ok = drive.read(lba, count, &mut buf[..n], true).is_ok();
let read_ms = t0.elapsed().as_millis();
// Flag slow reads
if read_ms > 2000 {
eprintln!("\n SLOW READ: LBA {} took {}ms (ok={})", lba, read_ms, ok);
}
if !ok {
buf[..n].fill(0);
}
w.write_all(&buf[..n]).unwrap();
lba += count as u32;
bytes += n as u64;
if last.elapsed().as_millis() >= 1000 {
let delta = bytes - last_bytes;
let speed = delta as f64 / last.elapsed().as_secs_f64() / 1_048_576.0;
let avg = bytes as f64 / start.elapsed().as_secs_f64() / 1_048_576.0;
let pct = bytes as f64 / (cap as f64 * 2048.0) * 100.0;
eprint!(
"\r {:.1}% LBA {} | {:.0} MB/s (avg {:.0}) | {:.1} GB ",
pct,
lba,
speed,
avg,
bytes as f64 / 1e9
);
last_bytes = bytes;
last = Instant::now();
}
}
w.flush().unwrap();
eprintln!(
"\nDone: {:.1} GB in {:.0}s",
bytes as f64 / 1e9,
start.elapsed().as_secs_f64()
);
}
-308
View File
@@ -1,308 +0,0 @@
//! AACS derivation "boil-down" — one public home for the key chain.
//!
//! Thin newtypes at the API boundary and three wrapper functions over the
//! existing crypto. Nothing here re-implements a primitive: every function
//! delegates to the already-audited code in [`super::keys`] and
//! [`super::variants`], so the boil-down cannot drift from production math.
//!
//! The newtypes wrap bare `[u8; 16]` ONLY at this boundary — the crypto
//! internals continue to operate on raw arrays. They exist so a caller threads
//! the chain `DK → MK → VUK → UK` without confusing one 16-byte secret for
//! another, not to refactor the resolver.
//!
//! Chain (matches `aacs::keys::resolve_keys_classical` path 1 and
//! `aacs::keys::resolve_keys_v21` path 1 byte-for-byte):
//!
//! ```text
//! mk_from_dk(device_keys, mkb, vid) → MediaKey (Km)
//! mk_from_pk(processing_keys, mkb) → MediaKey (Km)
//! vuk_from_mk(MediaKey, Vid) → Vuk (= AES-G(Km, VID))
//! uk_from_vuk(Vuk, enc_title_keys) → [UnitKey] (decrypt_unit_key each)
//! ```
//!
//! `mk_from_dk` and `mk_from_pk` are two entry points to the SAME Media Key:
//! the device-key path walks the MKB's Media-Key-Variant chain, the
//! processing-key path walks the MKB's Subset-Difference cvalue tables. Neither
//! needs a VID (the VID enters at `vuk_from_mk`).
use super::keys::{decrypt_unit_key, derive_media_key_from_pk, derive_vuk};
use super::types::DeviceKey;
use super::variants::{KEY_CORRECTION_DATA_PLACEHOLDER, derive_media_key_variant, walk_mkb};
/// Volume ID (16 bytes) — read from the disc via the SCSI handshake / OEM path.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Vid(pub [u8; 16]);
/// Media Key (Km, 16 bytes) — the MKB-scoped key derived from device keys.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct MediaKey(pub [u8; 16]);
/// Volume Unique Key (VUK / Kvu, 16 bytes) — derived from `MediaKey` + `Vid`,
/// decrypts the per-disc encrypted title keys in `Unit_Key_RO.inf`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Vuk(pub [u8; 16]);
/// One decrypted per-CPS-unit AACS title key.
///
/// `idx` is the POSITIONAL index of the encrypted title key within the slice
/// handed to [`uk_from_vuk`] (i.e. its order in `Unit_Key_RO.inf`'s key-storage
/// area). The CPS-unit *number* association (the `u32` in
/// `ResolvedKeys::unit_keys`) is a higher-level concern owned by
/// [`super::keys::parse_unit_key_ro`], which pairs each positional key with its
/// declared CPS unit; this primitive only does the AES, so it surfaces position.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct UnitKey {
pub idx: u32,
pub key: [u8; 16],
}
/// Derive the Volume Unique Key from a Media Key and Volume ID.
///
/// Wraps [`derive_vuk`] verbatim: `VUK = AES-128-ECB-DECRYPT(MK, VID) XOR VID`.
/// This is byte-identical to the inline `derive_vuk(&mk, ctx.volume_id)` call in
/// every classical resolver path AND to the `Kvu = AES-G(Km, VID)` step inside
/// [`derive_media_key_variant`] (AES-G and `derive_vuk` are the same math), so
/// `vuk_from_mk(mk_from_dk(..)?, vid)` reproduces the V21 variant VUK exactly.
pub fn vuk_from_mk(mk: MediaKey, vid: Vid) -> Vuk {
Vuk(derive_vuk(&mk.0, &vid.0))
}
/// Decrypt the disc's encrypted title keys with a VUK.
///
/// Wraps [`decrypt_unit_key`] (AES-128-ECB-DECRYPT) per entry, mirroring the
/// `derive_uks` closure in `resolve_keys_classical` / `resolve_keys_v21`. The
/// returned `UnitKey::idx` is the slice position; pair with CPS-unit numbers via
/// [`super::keys::parse_unit_key_ro`] when the numbering matters.
pub fn uk_from_vuk(vuk: Vuk, enc_title_keys: &[[u8; 16]]) -> Vec<UnitKey> {
enc_title_keys
.iter()
.enumerate()
.map(|(i, enc)| UnitKey {
idx: i as u32,
key: decrypt_unit_key(&vuk.0, enc),
})
.collect()
}
/// Derive the Media Key (Km) from device keys via the Media Key Variant chain.
///
/// Wraps [`walk_mkb`] + [`derive_media_key_variant`] with exactly the arguments
/// `resolve_keys_v21` path 1 passes: the placeholder Key Correction Data and the
/// disc Volume ID. Returns the FIRST tuple element `Km` (the Media Key) — the
/// resolver treats `Km` as the media key and derives the VUK from it as
/// `Kvu = AES-G(Km, VID)`, which equals [`vuk_from_mk`]`(MediaKey(km), vid)`. The
/// variant fn's second element is that already-derived `Kvu`; returning `Km`
/// keeps this primitive at the "media key" level so the chain composes.
///
/// Because the integrator KCD is unavailable in-tree (the placeholder is
/// rejected by the variant chain), this returns `Err` for every real disc today
/// — byte-for-byte identical to `resolve_keys_v21` path 1, which the resolver
/// also leaves unreachable in production. All variant-chain failures collapse to
/// [`Error::AacsMkUnavailable`] (E7018): no numeric distinction is load-bearing
/// at this boundary, and the variant error carries no English to preserve.
pub fn mk_from_dk(
device_keys: &[DeviceKey],
mkb: &[u8],
vid: Vid,
) -> Result<MediaKey, crate::error::Error> {
let records = walk_mkb(mkb);
match derive_media_key_variant(
&records,
device_keys,
&KEY_CORRECTION_DATA_PLACEHOLDER,
&vid.0,
) {
Ok((km, _kvu)) => Ok(MediaKey(km)),
Err(_) => Err(crate::error::Error::AacsMkUnavailable),
}
}
/// Derive the Media Key (Km) from one or more Processing Keys and the disc MKB.
///
/// Wraps [`derive_media_key_from_pk`] — the Subset-Difference PK→MK walk: each
/// processing key is validated (and tree-walked) against the MKB's cvalue tables
/// (records `0x04`/`0x05`) until one yields the Media Key whose verify record
/// (`0x81`/`0x86`) matches. Unlike [`mk_from_dk`] this path is reachable for
/// real discs — a leaked/precomputed AACS Processing Key in the keydb resolves
/// the Media Key directly. No VID is involved at this step; the VID enters at
/// [`vuk_from_mk`].
///
/// Returns [`Error::AacsMkUnavailable`] (E7018) when no processing key resolves
/// the MKB — the same terminal error as [`mk_from_dk`]; no numeric distinction
/// is load-bearing at this boundary.
///
/// [`Error::AacsMkUnavailable`]: crate::error::Error::AacsMkUnavailable
pub fn mk_from_pk(
processing_keys: &[[u8; 16]],
mkb: &[u8],
) -> Result<MediaKey, crate::error::Error> {
match derive_media_key_from_pk(mkb, processing_keys) {
Some(km) => Ok(MediaKey(km)),
None => Err(crate::error::Error::AacsMkUnavailable),
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::aacs::decrypt::aes_ecb_encrypt;
use crate::aacs::keys::{decrypt_unit_key, derive_vuk};
/// `vuk_from_mk` must equal the inline `derive_vuk` path bit-for-bit, for
/// several known (MK, VID) vectors.
#[test]
fn vuk_from_mk_matches_inline_derive_vuk() {
let cases: [([u8; 16], [u8; 16]); 3] = [
([0x5A; 16], [0xA5; 16]),
([0x11; 16], [0x22; 16]),
(
[
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C,
0x0D, 0x0E, 0x0F,
],
[
0xF0, 0xE1, 0xD2, 0xC3, 0xB4, 0xA5, 0x96, 0x87, 0x78, 0x69, 0x5A, 0x4B, 0x3C,
0x2D, 0x1E, 0x0F,
],
),
];
for (mk, vid) in cases {
let inline = derive_vuk(&mk, &vid);
let boiled = vuk_from_mk(MediaKey(mk), Vid(vid));
assert_eq!(boiled.0, inline, "vuk_from_mk must equal derive_vuk");
}
}
/// `uk_from_vuk` must equal the inline `decrypt_unit_key` path bit-for-bit
/// and carry positional indices 0..n. Built by encrypting known plaintext
/// title keys under the VUK (the same primitive the resolver inverts).
#[test]
fn uk_from_vuk_matches_inline_decrypt_unit_key() {
let vuk = [0x5Au8; 16];
let plain_keys = [[0x11u8; 16], [0x22u8; 16], [0xCDu8; 16]];
let enc: Vec<[u8; 16]> = plain_keys
.iter()
.map(|k| aes_ecb_encrypt(&vuk, k))
.collect();
let boiled = uk_from_vuk(Vuk(vuk), &enc);
assert_eq!(boiled.len(), enc.len());
for (i, uk) in boiled.iter().enumerate() {
assert_eq!(uk.idx, i as u32, "idx must be the positional index");
// Matches the inline derive_uks closure: decrypt_unit_key(vuk, enc).
assert_eq!(uk.key, decrypt_unit_key(&vuk, &enc[i]));
// And recovers the original plaintext title key.
assert_eq!(
uk.key, plain_keys[i],
"VUK roundtrip recovers the title key"
);
}
}
/// `uk_from_vuk` on an empty slice yields no keys (no panic, no phantom idx).
#[test]
fn uk_from_vuk_empty_is_empty() {
assert!(uk_from_vuk(Vuk([0u8; 16]), &[]).is_empty());
}
/// `mk_from_dk` returns `Err(AacsMkUnavailable)` for the placeholder-KCD
/// path that production also leaves unreachable — never a wrong key, never a
/// panic — on both an empty MKB and a non-variant MKB.
#[test]
fn mk_from_dk_errors_without_integrator_kcd() {
let dk = DeviceKey {
key: [0x11; 16],
node: 1,
uv: 1,
u_mask_shift: 0,
};
// Empty MKB → not a variant MKB → Err.
let e = mk_from_dk(std::slice::from_ref(&dk), &[], Vid([0x09; 16]));
assert!(matches!(e, Err(crate::error::Error::AacsMkUnavailable)));
// A variant-looking MKB (0x82 record) still cannot complete without the
// integrator KCD, so it also errors — never silently yields a key.
let mut mkb: Vec<u8> = Vec::new();
mkb.extend_from_slice(&[0x82, 0x00, 0x00, 0x14]); // variant data record
mkb.extend_from_slice(&[0xAB; 16]);
let e2 = mk_from_dk(&[dk], &mkb, Vid([0x09; 16]));
assert!(matches!(e2, Err(crate::error::Error::AacsMkUnavailable)));
}
/// Build a 4-byte MKB record header (type + 3-byte big-endian total length,
/// header included) and append `body`. Mirrors the MKB record framing the
/// parser expects; no crypto.
fn mkb_record(rec_type: u8, body: &[u8]) -> Vec<u8> {
let total = 4 + body.len();
let mut rec = Vec::with_capacity(total);
rec.push(rec_type);
rec.push(((total >> 16) & 0xFF) as u8);
rec.push(((total >> 8) & 0xFF) as u8);
rec.push((total & 0xFF) as u8);
rec.extend_from_slice(body);
rec
}
/// `mk_from_pk` resolves a planted Processing Key against a synthetic MKB and
/// drives the FULL boil chain PK → MK → VUK → UK. The MKB is built with the
/// same (pk, cv, mk_dv, uv) construction the production SD walk validates, so
/// this proves a PK entry yields real Unit Keys — not just an `Ok`.
#[test]
fn mk_from_pk_drives_full_chain_to_uks() {
let pk: [u8; 16] = [
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE,
0xFF, 0x00,
];
let mk: [u8; 16] = [
0xA0, 0xA1, 0xA2, 0xA3, 0xA4, 0xA5, 0xA6, 0xA7, 0xA8, 0xA9, 0xAA, 0xAB, 0xAC, 0xAD,
0xAE, 0xAF,
];
let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00];
// cv = AES-E(pk, mk_raw), where mk_raw is mk with the last-4-bytes-uv XOR
// pre-undone, so the validate step XORs uv back in and recovers mk.
let mut mk_raw = mk;
for a in 0..4 {
mk_raw[12 + a] ^= uv[a];
}
let cv = aes_ecb_encrypt(&pk, &mk_raw);
// mk_dv = AES-E(mk, magic||pad): AES-D(mk, mk_dv) starts with the AACS
// verify sentinel.
let mut vd = [0x11u8; 16];
vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]);
let mk_dv = aes_ecb_encrypt(&mk, &vd);
// Synthetic MKB: type/version (0x10), verify record (0x86 = mk_dv),
// one-entry SD index (0x04 = [u_mask_shift=0][uv]), one-entry cvalue
// table (0x05 = cv).
let mut sd = vec![0u8];
sd.extend_from_slice(&uv);
let mut mkb = Vec::new();
mkb.extend_from_slice(&mkb_record(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
mkb.extend_from_slice(&mkb_record(0x86, &mk_dv));
mkb.extend_from_slice(&mkb_record(0x04, &sd));
mkb.extend_from_slice(&mkb_record(0x05, &cv));
// PK → MK.
let got_mk = mk_from_pk(std::slice::from_ref(&pk), &mkb).expect("planted PK resolves MK");
assert_eq!(got_mk, MediaKey(mk), "mk_from_pk recovers the planted MK");
// MK → VUK → UK over an encrypted title key.
let vid = Vid([0x42u8; 16]);
let plain_uk = [0x7Eu8; 16];
let vuk = vuk_from_mk(got_mk, vid);
let enc = aes_ecb_encrypt(&vuk.0, &plain_uk);
let uks = uk_from_vuk(vuk, std::slice::from_ref(&enc));
assert_eq!(uks.len(), 1);
assert_eq!(uks[0].key, plain_uk, "PK chain recovers the title key");
// A corrupt PK resolves nothing.
let mut bad = pk;
bad[0] ^= 0xFF;
assert!(matches!(
mk_from_pk(std::slice::from_ref(&bad), &mkb),
Err(crate::error::Error::AacsMkUnavailable)
));
}
}
+1669
View File
File diff suppressed because it is too large Load Diff
+268
View File
@@ -0,0 +1,268 @@
//! AACS common cryptographic primitives — [C] Chapter 2 / §3.2.2.
//!
//! Source: `[C]` = AACS Introduction and Common Cryptographic Elements Book,
//! Rev 0.953. The shared low-level building blocks — AES-128 ECB E/D, AES-G,
//! the AES-G3 Triple Generator, AES-CBC decrypt — and their fixed constants
//! (`iv0`, `s0`). Used by every AACS generation; relocated here so the
//! primitives live in one place instead of being scattered across the
//! content / keys / variant modules.
use aes::Aes128;
use aes::cipher::{Array, BlockCipherDecrypt, BlockCipherEncrypt, KeyInit};
/// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`).
pub(crate) const AACS_IV: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78,
];
// Per-thread count of AES-128 key schedules built through `new_cipher`.
// Test-only instrumentation: an AES-128 key expansion is 10 round-key
// derivations, and the CBC helpers here run on the per-aligned-unit decrypt hot
// path of a whole disc read, so "how many times was the schedule built for one
// loop-invariant key" is a property worth asserting rather than reasoning about.
// THREAD-LOCAL, not a global atomic: `cargo test` runs tests concurrently, so a
// shared counter would see every other test's expansions. See
// `content::tests::decrypt_bus_expands_the_read_data_key_once_per_unit`.
#[cfg(test)]
thread_local! {
pub(crate) static KEY_EXPANSIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Build an AES-128 key schedule for a caller that will drive
/// [`cbc_decrypt_blocks`] over several regions under one key.
pub(crate) fn new_cipher_for(key: &[u8; 16]) -> Aes128 {
new_cipher(key)
}
/// Build an AES-128 key schedule. The single construction site for the CBC
/// helpers, so [`KEY_EXPANSIONS`] can count them under test.
fn new_cipher(key: &[u8; 16]) -> Aes128 {
#[cfg(test)]
KEY_EXPANSIONS.with(|c| c.set(c.get() + 1));
Aes128::new(&(*key).into())
}
/// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`).
pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
cipher.encrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
out
}
/// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`).
pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(&(*key).into());
let mut block: Array<u8, _> = (*data).into();
cipher.decrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
out
}
/// AES-128-CBC ENCRYPT in place under the fixed [`AACS_IV`] — the forward
/// direction of [`aes_cbc_decrypt`], and its exact inverse. [C] §2.1.2
/// (`AES-128CBCE`).
///
/// Precondition: `data.len()` is a multiple of 16; the assert
/// documents/enforces that contract.
///
/// Constructs the cipher ONCE for the whole slice. Driving this from the
/// single-block [`aes_ecb_encrypt`] instead rebuilds the AES key schedule per
/// 16-byte block, which for a 6144-byte aligned unit is 383 redundant key
/// expansions.
pub(crate) fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len().is_multiple_of(16),
"aes_cbc_encrypt requires a block-aligned slice"
);
let cipher = new_cipher(key);
let num_blocks = data.len() / 16;
let mut prev = AACS_IV;
// Forward order: each block is XORed with the PRECEDING ciphertext block.
for i in 0..num_blocks {
let offset = i * 16;
let mut block = [0u8; 16];
for j in 0..16 {
block[j] = data[offset + j] ^ prev[j];
}
let mut ga: Array<u8, _> = block.into();
cipher.encrypt_block(&mut ga);
data[offset..offset + 16].copy_from_slice(&ga);
prev.copy_from_slice(&ga);
}
}
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. [C] §2.1.2
/// (`AES-128CBCD`).
///
/// Precondition: `data.len()` is a multiple of 16. Any trailing partial
/// block is silently ignored; all callers pass aligned regions (6128 and
/// 2032 bytes), and the assert documents/enforces that contract.
///
/// (This doc block was orphaned onto `aes_cbc_encrypt` above when that function
/// was inserted directly after it with no separating blank line, so rustdoc
/// rendered the crate's only forward-direction AACS primitive as "decrypt" and
/// cited the spec's DECRYPT clause for it, while this function had no doc at
/// all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in `content.rs` pins
/// the directions behaviourally so a maintainer 'fixing' the contradiction by
/// swapping the two bodies fails the suite instead of shipping a second
/// decryptor behind an already-set encrypted flag.)
pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len().is_multiple_of(16),
"aes_cbc_decrypt requires a block-aligned slice"
);
cbc_decrypt_blocks(&new_cipher(key), data);
}
/// AES-128-CBC decrypt in place under the fixed [`AACS_IV`] with an ALREADY
/// EXPANDED key schedule.
///
/// Split out of [`aes_cbc_decrypt`] so a caller that decrypts several regions
/// under one loop-invariant key expands the schedule once. `decrypt_bus`
/// ([`super::content::decrypt_bus`]) is that caller: bus encryption
/// ([C] §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// 2048-byte sector, so a 6144-byte aligned unit is three regions under one
/// `read_data_key` — three key schedules where one suffices, on the per-unit
/// decrypt hot path of a whole 90 GB read.
pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
let num_blocks = data.len() / 16;
// Process blocks in reverse to avoid clobbering ciphertext needed for XOR
for i in (0..num_blocks).rev() {
let offset = i * 16;
let prev = if i == 0 {
AACS_IV
} else {
let mut p = [0u8; 16];
p.copy_from_slice(&data[(i - 1) * 16..i * 16]);
p
};
let mut chunk = [0u8; 16];
chunk.copy_from_slice(&data[offset..offset + 16]);
let mut block: Array<u8, _> = chunk.into();
cipher.decrypt_block(&mut block);
for j in 0..16 {
data[offset + j] = block[j] ^ prev[j];
}
}
}
/// AES-G(x1, x2) = AES-128D(x1, x2) XOR x2. [C] §2.1.3 (note: uses AES-128**D**).
///
/// The Media Key Variant chain uses AES-G to derive both the variant
/// number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
/// (`Kvu = AES-G(Km, VID)`). See [`super::derive::derive_vuk`] for the
/// classical VUK form — the math is identical, this exposes it as a
/// neutral primitive for the variant chain.
pub(crate) fn aes_g(x1: &[u8; 16], x2: &[u8; 16]) -> [u8; 16] {
let mut out = aes_ecb_decrypt(x1, x2);
for i in 0..16 {
out[i] ^= x2[i];
}
out
}
/// AACS-G3 seed constant (`s0`). [C] §3.2.2.
pub(crate) const AESG3_SEED: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B, 0xD9,
];
/// AACS-G3: derive a subkey from a parent key. [C] §3.2.2 (Triple AES Generator:
/// left=`D(k,s0)⊕s0` inc 0, pk=`D(k,s0+1)⊕(s0+1)` inc 1, right=`D(k,s0+2)⊕(s0+2)` inc 2).
/// seed[15] += inc, then AES-DEC(key, seed) XOR seed.
///
/// Shared with [`super::variant`] (its variant chain runs the same SD
/// tree); a single definition keeps the two walks byte-identical.
pub(crate) fn aesg3(key: &[u8; 16], inc: u8) -> [u8; 16] {
let mut seed = AESG3_SEED;
seed[15] = seed[15].wrapping_add(inc);
let mut out = aes_ecb_decrypt(key, &seed);
for i in 0..16 {
out[i] ^= seed[i];
}
out
}
#[cfg(test)]
mod tests {
use super::*;
/// The AACS-G3 seed `s0`, transcribed independently from [C] §3.2.2 rather
/// than read from [`AESG3_SEED`] — a test that sourced the seed from the
/// production constant would assert that constant against itself and would
/// still pass if it were edited.
const S0: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B,
0xD9,
];
/// An arbitrary non-degenerate key. Nothing about it is secret or special;
/// the AES-G3 relation holds for every key, and a constant-returning body
/// cannot satisfy it for any.
const K: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2, 0xE1,
0xF0,
];
/// `aesg3` is the node function of the AACS subset-difference tree: every
/// Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every
/// descent step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that
/// returned a fixed block would make every device key in the crate derive
/// the SAME Processing Key, and a `^` that became `|` or `&` would derive a
/// wrong-but-plausible one — in both cases the MKB walk simply stops
/// finding Media Keys, with no error to say why.
///
/// Pinned through the spec relation rather than a re-implementation:
/// [C] §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `s = s0 + inc` (added into the last seed byte), so applying the
/// FORWARD primitive [`aes_ecb_encrypt`] — a different function from the
/// one under test — to `aesg3(k, inc) XOR s` must reproduce `s` exactly.
#[test]
fn aesg3_inverts_to_the_spec_seed_under_aes_encrypt() {
for inc in 0u8..=2 {
let mut seed = S0;
seed[15] = seed[15].wrapping_add(inc);
let out = aesg3(&K, inc);
// out == AES-128D(K, seed) XOR seed, so out XOR seed is the raw
// decryption and re-encrypting it must land back on the seed.
let mut pre = [0u8; 16];
for i in 0..16 {
pre[i] = out[i] ^ seed[i];
}
assert_eq!(
aes_ecb_encrypt(&K, &pre),
seed,
"AES-G3 inc={inc} must satisfy out = AES-128D(k, s0+inc) XOR (s0+inc)"
);
}
}
/// The Triple Generator's three outputs ([C] §3.2.2: left = inc 0, the
/// Processing Key = inc 1, right = inc 2) are the two child node keys and
/// the Processing Key of ONE tree node. They must be three different keys —
/// if `inc` were ignored, a descent would revisit its own parent and the
/// walk would derive the same key at every level of the tree.
#[test]
fn aesg3_yields_three_distinct_subkeys_for_the_three_increments() {
let left = aesg3(&K, 0);
let pk = aesg3(&K, 1);
let right = aesg3(&K, 2);
assert_ne!(left, pk, "left child and Processing Key must differ");
assert_ne!(pk, right, "Processing Key and right child must differ");
assert_ne!(left, right, "left and right children must differ");
}
/// Distinct parent keys must yield distinct subkeys — the tree would
/// collapse otherwise.
#[test]
fn aesg3_separates_distinct_parent_keys() {
let mut other = K;
other[0] ^= 0x01;
assert_ne!(aesg3(&K, 1), aesg3(&other, 1));
}
}
-1501
View File
File diff suppressed because it is too large Load Diff
+1769
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -10,8 +10,8 @@
pub fn collect_host_certs(
opts: &crate::disc::ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::HostCert> {
let mut host_certs: Vec<crate::aacs::HostCert> = Vec::new();
) -> Vec<crate::aacs::types::HostCert> {
let mut host_certs: Vec<crate::aacs::types::HostCert> = Vec::new();
if let Some(c) = &opts.credentials {
host_certs.extend(c.host_certs.iter().cloned());
}
+198
View File
@@ -0,0 +1,198 @@
//! FMTS index selection — the pure decode-time decision for a 2.1 disc.
//!
//! A 2.1 disc resolves to exactly one forensic index (1..=32) for a given
//! rip. `IndividualSegment.tbl` tags each forensic segment with an index (see
//! [`super::segment`]); the decode keeps the segments matching our index,
//! drops the other 31, and treats everything outside a segment as ordinary
//! (index-0) content. This module owns that classification and nothing else —
//! no I/O, no keys, no cipher — so it is fully testable in isolation. The
//! decrypt pipeline consumes the [`UnitDisposition`] it returns.
//!
//! Where the resolved index comes from is a separate concern
//! ([`resolve_disc_index`]): today it is read off the index keys the key
//! source handed us; when Processing Keys are available it will come from the
//! VK derivation instead. Either way the disposition logic below is identical.
use super::segment::{Segment, segment_for_unit};
use super::types::UnitKey;
/// What the decode should do with one AACS aligned unit.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UnitDisposition {
/// Outside every forensic segment: ordinary content, decrypt with the
/// default (index-0) unit key.
Default,
/// Inside a forensic segment tagged with OUR resolved index: decrypt with
/// that index's key.
Index(u8),
/// Inside a forensic segment tagged with a DIFFERENT index: not our
/// watermark, so it is not part of our output — drop it.
DropForeignIndex(u8),
/// Inside a forensic segment but no index key is held (the disc's index
/// was never resolved): the segment cannot be decoded, so it is concealed
/// as loss. Carries the segment's index for diagnostics.
ForensicNoKey(u8),
}
/// Resolve the disc's single forensic index from the keys we hold.
///
/// Scans for an index key (`index_number` in `1..=32`) and returns its
/// index. `None` when only default (index-0) keys are held — i.e. no
/// index source answered, so forensic segments are not decodable. A disc has
/// exactly one index, so the first non-zero key decides; if several distinct
/// index keys were somehow supplied the lowest wins (deterministic), which is
/// only a defensive tiebreak — the probe/derivation yields one.
pub fn resolve_disc_index(unit_keys: &[UnitKey]) -> Option<u8> {
unit_keys
.iter()
.map(|k| k.index_number)
.filter(|&v| v != 0)
.min()
}
/// Classify the AACS aligned unit at `unit_offset` (clip-relative bytes) given
/// the forensic segment map and the disc's resolved index (`None` if no
/// index key is held).
pub fn unit_disposition(
unit_offset: u64,
segments: &[Segment],
disc_index: Option<u8>,
) -> UnitDisposition {
match segment_for_unit(segments, unit_offset) {
// Not in any forensic segment → ordinary content.
None => UnitDisposition::Default,
// In a forensic segment → decide by whether it is our index.
Some(seg) => {
// `seg.index` is an untrusted u16 from IndividualSegment.tbl; a real
// forensic index is 1..=32. Compare in u16 space so a corrupt/crafted
// index above 255 can't truncate into a valid u8 and alias our index.
// The disposition carries a u8 for diagnostics (saturated — an
// out-of-range index is never ours anyway).
let seg_index = seg.index;
let diag = seg_index.min(u8::MAX as u16) as u8;
match disc_index {
Some(v) if u16::from(v) == seg_index => UnitDisposition::Index(v),
Some(_) => UnitDisposition::DropForeignIndex(diag),
None => UnitDisposition::ForensicNoKey(diag),
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::aacs::content::ALIGNED_UNIT_LEN;
use crate::aacs::segment::{SOURCE_PACKET_LEN, parse_individual_segments};
/// Build a one-record segment table (index, start_spn, end_spn).
fn tbl(recs: &[(u16, u32, u32)]) -> Vec<Segment> {
let mut v = Vec::new();
v.extend_from_slice(&0x0100_0000u32.to_be_bytes());
v.extend_from_slice(&(recs.len() as u16).to_be_bytes());
v.extend_from_slice(&16u16.to_be_bytes());
for &(n, s, e) in recs {
v.extend_from_slice(&0x0100_0000u32.to_be_bytes());
v.extend_from_slice(&n.to_be_bytes());
v.extend_from_slice(&1u16.to_be_bytes());
v.extend_from_slice(&s.to_be_bytes());
v.extend_from_slice(&e.to_be_bytes());
}
parse_individual_segments(&v).expect("parse")
}
fn uk(idx: u32, index: u8) -> UnitKey {
if index == 0 {
UnitKey::new(idx, [0u8; 16])
} else {
UnitKey::forensic(idx, [index; 16], index)
}
}
#[test]
fn resolve_picks_the_single_index_key() {
// Default keys only → no index resolved.
assert_eq!(resolve_disc_index(&[uk(0, 0)]), None);
assert_eq!(resolve_disc_index(&[]), None);
// One index key among defaults → that index.
assert_eq!(resolve_disc_index(&[uk(0, 0), uk(1, 7)]), Some(7));
// Defensive: lowest of several distinct indexes (deterministic).
assert_eq!(resolve_disc_index(&[uk(0, 9), uk(1, 3)]), Some(3));
}
#[test]
fn unit_outside_segments_is_default() {
let segs = tbl(&[(1, 343680, 346239)]);
let off = 1000u64 * SOURCE_PACKET_LEN; // well before the segment
assert_eq!(
unit_disposition(off, &segs, Some(1)),
UnitDisposition::Default
);
// With no segments at all (1.0 / 2.0), everything is Default.
assert_eq!(
unit_disposition(off, &[], Some(1)),
UnitDisposition::Default
);
}
#[test]
fn unit_in_our_index_decrypts() {
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(7)),
UnitDisposition::Index(7)
);
}
#[test]
fn unit_in_foreign_index_drops() {
// Segment tagged index 7, but our disc index is 3 → drop it.
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(3)),
UnitDisposition::DropForeignIndex(7)
);
}
#[test]
fn forensic_unit_with_no_key_is_concealed() {
// A forensic segment but we never resolved an index → conceal as loss.
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, None),
UnitDisposition::ForensicNoKey(7)
);
}
#[test]
fn out_of_range_index_does_not_truncate_into_ours() {
// A crafted/corrupt segment index of 288 (0x0120) truncates to 32 in a
// u8. With our disc index resolved as 32, the old `seg.index as u8`
// compare would alias it to OUR index and decrypt with the wrong key.
// The u16 compare must instead classify it as foreign.
let segs = tbl(&[(288, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(32)),
UnitDisposition::DropForeignIndex(255)
);
}
#[test]
fn straddling_unit_still_classified_as_its_segment() {
// A unit whose 32-packet span only tails into the segment still routes
// to the segment (matches segment_for_unit's span test).
let segs = tbl(&[(5, 100, 200)]);
let unit_packets = (ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32; // 32
// Start so the unit covers [80, 80+31] = [80, 111]: overlaps at 100.
let off = 80u64 * SOURCE_PACKET_LEN;
assert!(80 + unit_packets > 100, "sanity: unit tails into seg");
assert_eq!(
unit_disposition(off, &segs, Some(5)),
UnitDisposition::Index(5)
);
}
}
+881
View File
@@ -0,0 +1,881 @@
//! AACS on-disc key-input files: `Unit_Key_RO.inf` parsing, the disc-hash
//! keydb lookup key, the Content Certificate, and the in-drive MKB read.
//! These turn raw disc files into the structures the key paths consume.
use super::mkb::*;
/// Parsed Unit_Key_RO.inf file.
pub struct UnitKeyFile {
/// Disc hash (SHA1 of the entire file) — used as KEYDB lookup key
pub disc_hash: [u8; 20],
/// Application type (1 = BD-ROM)
pub app_type: u8,
/// Number of BDMV directories
pub num_bdmv_dir: u8,
/// Whether SKB MKB is used
pub use_skb_mkb: bool,
/// AACS generation this file's stride matches
pub version: AacsVersion,
/// Encrypted unit keys (CPS unit number, encrypted key)
pub encrypted_keys: Vec<(u32, [u8; 16])>,
/// Title → CPS unit index mapping (title_idx → unit_key_idx)
pub title_cps_unit: Vec<u16>,
}
/// Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
/// the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
/// — plus the disc hash they are looked up by. A derived `Debug` printed every key
/// byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
/// message, a future `tracing::debug!` in this module) leaked them. Only
/// non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
impl std::fmt::Debug for UnitKeyFile {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKeyFile")
// The disc hash is the public keydb lookup key, printed as hex the
// same way `DiscEntry` prints its own — never as raw bytes.
.field("disc_hash", &disc_hash_hex(&self.disc_hash))
.field("app_type", &self.app_type)
.field("num_bdmv_dir", &self.num_bdmv_dir)
.field("use_skb_mkb", &self.use_skb_mkb)
.field("version", &self.version)
.field("encrypted_keys", &"<redacted>")
.field("encrypted_keys_len", &self.encrypted_keys.len())
.field("title_cps_unit", &self.title_cps_unit)
.finish()
}
}
/// Compute disc hash (SHA1 of Unit_Key_RO.inf content).
pub fn disc_hash(data: &[u8]) -> [u8; 20] {
use sha1::{Digest, Sha1};
let hash = Sha1::digest(data);
let mut out = [0u8; 20];
out.copy_from_slice(&hash);
out
}
/// Format disc hash as hex string with 0x prefix (for KEYDB lookup).
pub fn disc_hash_hex(hash: &[u8; 20]) -> String {
let mut s = String::with_capacity(42);
s.push_str("0x");
for b in hash {
s.push_str(&format!("{b:02X}"));
}
s
}
/// Parse Unit_Key_RO.inf from raw bytes.
///
/// Format (from AACS spec):
/// [0..4] BE32: offset to key storage area (uk_pos)
/// [16] app_type (1 = BD-ROM)
/// [17] num_bdmv_dir
/// [18] bit 7: use_skb_mkb
/// [20..22] BE16: first_play CPS unit
/// [22..24] BE16: top_menu CPS unit
/// [24..26] BE16: num_titles
/// [26..] title entries: 2 bytes padding + 2 bytes CPS unit, × num_titles
///
/// Key storage at uk_pos:
/// [uk_pos..uk_pos+2] BE16: num_unit_keys
/// [uk_pos+48..] encrypted keys, 16 bytes each
/// AACS 1.0: 48-byte stride
/// AACS 2.0 / 2.1: 64-byte stride (48 + 16 extra)
pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() < 20 {
return None;
}
let hash = disc_hash(data);
// Header
let app_type = data[16];
let num_bdmv_dir = data[17];
let use_skb_mkb = (data[18] >> 7) & 1 == 1;
// Key storage offset
let uk_pos = u32::from_be_bytes([data[0], data[1], data[2], data[3]]) as usize;
if uk_pos + 2 > data.len() {
return None;
}
// Number of unit keys
let num_uk = u16::from_be_bytes([data[uk_pos], data[uk_pos + 1]]) as usize;
if num_uk == 0 {
return Some(UnitKeyFile {
disc_hash: hash,
app_type,
num_bdmv_dir,
use_skb_mkb,
version,
encrypted_keys: Vec::new(),
title_cps_unit: Vec::new(),
});
}
// Stride between keys
let stride = version.unit_key_stride();
// Validate size
let keys_start = uk_pos + 48; // first key at uk_pos + 48
if keys_start + 16 > data.len() {
return None;
}
// Extract encrypted keys
let mut encrypted_keys = Vec::with_capacity(num_uk);
let mut pos = keys_start;
for i in 0..num_uk {
if pos + 16 > data.len() {
break;
}
let mut key = [0u8; 16];
key.copy_from_slice(&data[pos..pos + 16]);
encrypted_keys.push(((i + 1) as u32, key));
pos += stride;
}
// The loop above `break`s if the buffer runs out mid-key. A short list
// means the .inf is malformed/truncated — reject it rather than silently
// accepting fewer keys than the header declared, which would later map
// title CPS units to nonexistent keys.
if encrypted_keys.len() != num_uk {
return None;
}
// Title → CPS unit mapping (AACS Unit_Key_RO format): each on-disc CPS
// value is in `1..=num_uk` (else zeroes it) and converts the 1-based on-disc
// index to a 0-based key index. We mirror that so the stored value is a safe,
// ready-to-use key index rather than a raw 1-based number.
let to_key_idx = |cps: u16| -> u16 {
if cps >= 1 && cps as usize <= num_uk {
cps - 1
} else {
0
}
};
let mut title_cps_unit = Vec::new();
if data.len() >= 26 {
let first_play = u16::from_be_bytes([data[20], data[21]]);
let top_menu = u16::from_be_bytes([data[22], data[23]]);
let num_titles = u16::from_be_bytes([data[24], data[25]]) as usize;
title_cps_unit.push(to_key_idx(first_play));
title_cps_unit.push(to_key_idx(top_menu));
for i in 0..num_titles {
let off = 26 + i * 4 + 2; // 2 bytes padding + 2 bytes CPS unit
if off + 2 <= data.len() {
let cps = u16::from_be_bytes([data[off], data[off + 1]]);
title_cps_unit.push(to_key_idx(cps));
}
}
}
Some(UnitKeyFile {
disc_hash: hash,
app_type,
num_bdmv_dir,
use_skb_mkb,
version,
encrypted_keys,
title_cps_unit,
})
}
/// HD DVD Video Title Key File (`VTKF%%%.AACS`) magic — "DVD_HD_V_TKF".
pub const VTKF_MAGIC: &[u8; 12] = b"DVD_HD_V_TKF";
/// Fixed header length before the first Title Key Entry (AACS HD DVD Book,
/// Table 3-8).
const VTKF_HEADER_LEN: usize = 0x80;
/// Title Key Entry stride (Table 3-8): 1-byte `BIFO` + 3 reserved + 16-byte
/// encrypted title key + 16-byte binding MAC = 36 bytes.
const VTKF_ENTRY_LEN: usize = 0x24;
/// Byte offset of the encrypted title key within an entry (after `BIFO` + 3
/// reserved).
const VTKF_KEY_OFF: usize = 4;
/// Number of Title Key Entry slots in a VTKF (Table 3-8): a fixed 64.
const VTKF_MAX_ENTRIES: usize = 64;
/// `BIFO` bit 7 (`AV_FLG`): set = this slot carries an available title key.
const VTKF_AV_FLG: u8 = 0x80;
/// Parse an HD DVD `VTKF%%%.AACS` into the SAME [`UnitKeyFile`] a BD/UHD
/// `Unit_Key_RO.inf` yields — so the shared AACS crypto (`derive_unit_keys` →
/// `decrypt_unit_key(vuk, …)`) unwraps HD DVD title keys with no change. Only
/// the on-disc CONTAINER differs between BD and HD DVD; the title-key unwrap is
/// the identical AES-128 VUK step (`Kt = AES-128D(Kvu, Kte)`).
///
/// Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
/// 2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
/// Dukes of Hazzard `VTKF000`):
/// ```text
/// [0x00..0x0C] magic "DVD_HD_V_TKF"
/// [0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
/// [0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
/// [0x1C..0x80] reserved
/// [0x80..] 64 entries × 36 bytes:
/// BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
/// BIFO bit 7 (AV_FLG) set = this slot holds a title key
/// (pre-recorded discs fill the binding MAC with 0xFF)
/// [0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
/// ```
/// The slot index (1-based) is the CPS unit number, so an absent slot is
/// SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
/// hand the wrong title key to CPS unit N+1. The title→CPS mapping is
/// playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
/// `title_cps_unit` is left empty here.
///
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
}
// SHA1 of the WHOLE file — the KEYDB lookup key. BackupHDDVD-family key
// databases index an HD DVD disc by SHA1(VTKF000.AACS), the same role the
// BD disc_hash plays for `Unit_Key_RO.inf`.
let hash = disc_hash(data);
let mut encrypted_keys = Vec::new();
for n in 0..VTKF_MAX_ENTRIES {
let pos = VTKF_HEADER_LEN + n * VTKF_ENTRY_LEN;
if pos + VTKF_ENTRY_LEN > data.len() {
break;
}
// AV_FLG clear = empty slot: skip it, but keep the slot index as the CPS
// number (do NOT break — a gap must not renumber the keys that follow).
if data[pos] & VTKF_AV_FLG == 0 {
continue;
}
let mut key = [0u8; 16];
key.copy_from_slice(&data[pos + VTKF_KEY_OFF..pos + VTKF_KEY_OFF + 16]);
encrypted_keys.push((n as u32 + 1, key));
}
if encrypted_keys.is_empty() {
return None;
}
Some(UnitKeyFile {
disc_hash: hash,
app_type: 0, // HD DVD VTKF carries no BD-ROM app_type
num_bdmv_dir: 0, // BD-only concept
use_skb_mkb: false,
version: AacsVersion::V10, // HD DVD is always AACS 1.0
encrypted_keys,
title_cps_unit: Vec::new(),
})
}
/// Parse a disc's title-key file, dispatching on the self-describing magic:
/// an HD DVD `VTKF000.AACS` (`DVD_HD_V_TKF`) → [`parse_vtkf`]; anything else is a
/// BD/UHD `Unit_Key_RO.inf` → [`parse_unit_key_ro`]. Both return the same
/// [`UnitKeyFile`], so every downstream AACS derivation stays container-agnostic
/// — the single seam where BD-vs-HD-DVD key layout is resolved (mirrors the key
/// service, which classifies HD DVD by the very same magic).
pub fn parse_title_keys(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() >= 12 && &data[..12] == VTKF_MAGIC {
parse_vtkf(data)
} else {
parse_unit_key_ro(data, version)
}
}
/// MKB disc structure format code.
const MKB_DISC_STRUCTURE_FORMAT: u8 = 0x83;
/// MKB pack buffer size.
const MKB_PACK_SIZE: usize = 32772;
/// Read MKB from drive via SCSI (REPORT DISC STRUCTURE format 0x83).
/// Returns the concatenated MKB data from all packs.
pub fn read_mkb_from_drive(
session: &mut dyn crate::scsi::ScsiTransport,
) -> crate::error::Result<Vec<u8>> {
use crate::scsi::{DataDirection, SCSI_READ_DISC_STRUCTURE};
let cdb = [
SCSI_READ_DISC_STRUCTURE,
0x01,
0x00,
0x00,
0x00,
0x00,
0x00,
MKB_DISC_STRUCTURE_FORMAT,
(MKB_PACK_SIZE >> 8) as u8,
(MKB_PACK_SIZE & 0xFF) as u8,
0x00,
0x00,
];
let mut buf = vec![0u8; 32772];
session.execute(&cdb, DataDirection::FromDevice, &mut buf, 10_000)?;
let data_len = u16::from_be_bytes([buf[0], buf[1]]) as usize;
if data_len < 2 {
return Ok(Vec::new());
}
let len = data_len - 2;
let num_packs = buf[3] as usize;
let mut mkb = Vec::with_capacity(32768 * num_packs.max(1));
if len > 0 && len <= 32768 {
mkb.extend_from_slice(&buf[4..4 + len]);
}
// Read remaining packs
for pack in 1..num_packs {
let mut cdb = [
SCSI_READ_DISC_STRUCTURE,
0x01,
0x00,
0x00,
0x00,
0x00,
0x00,
MKB_DISC_STRUCTURE_FORMAT,
(MKB_PACK_SIZE >> 8) as u8,
(MKB_PACK_SIZE & 0xFF) as u8,
0x00,
0x00,
];
// Pack number goes in address field
cdb[2] = ((pack >> 24) & 0xFF) as u8;
cdb[3] = ((pack >> 16) & 0xFF) as u8;
cdb[4] = ((pack >> 8) & 0xFF) as u8;
cdb[5] = (pack & 0xFF) as u8;
let mut buf = vec![0u8; 32772];
if session
.execute(&cdb, DataDirection::FromDevice, &mut buf, 10_000)
.is_ok()
{
let len = u16::from_be_bytes([buf[0], buf[1]]) as usize;
if len > 2 && len - 2 <= 32768 {
mkb.extend_from_slice(&buf[4..4 + len - 2]);
}
}
}
Ok(mkb)
}
/// AACS Content Certificate — identifies disc AACS version and features.
#[derive(Debug)]
pub struct ContentCert {
/// Bus encryption enabled flag
pub bus_encryption: bool,
/// Content Certificate ID (6 bytes)
pub cc_id: [u8; 6],
/// AACS generation indicated by the certificate type byte.
///
/// Cert type `0x00` → [`AacsVersion::V10`]; any other value →
/// [`AacsVersion::V20`]. The certificate alone cannot distinguish
/// V20 from V21 — Variant detection happens after the MKB walk.
pub version: AacsVersion,
}
/// Parse a Content Certificate (ContentXXX.cer) file.
pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
if data.len() < 20 {
return None;
}
// Content Certificate layout (per the AACS content-cert format):
// [0] certificate type (0x00 = AACS1, 0x10 = AACS2)
// [1] bit7 bus_encryption_enabled_flag (`p[1] >> 7`)
// [14..20] cc_id (6 bytes) (`p + 14`)
let version = if data[0] == 0x00 {
AacsVersion::V10
} else {
AacsVersion::V20
};
// The flag is bit 7 of byte 1, NOT bit 0. Reading bit 0 (the prior bug) made
// a bus-encrypted cert (byte1=0x80) read as `false`, defeating the
// AacsBusKeyUnavailable fail-loud gate in disc/encrypt.rs.
let bus_encryption = (data[1] >> 7) & 1 == 1;
let mut cc_id = [0u8; 6];
cc_id.copy_from_slice(&data[14..20]);
Some(ContentCert {
bus_encryption,
cc_id,
version,
})
}
#[cfg(test)]
mod vtkf_tests {
use super::*;
/// Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
/// HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
/// `VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
/// entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
/// set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
fn synth_vtkf(keys: &[[u8; 16]]) -> Vec<u8> {
const FILE_LEN: usize = 2480;
let mut v = Vec::new();
v.extend_from_slice(VTKF_MAGIC); // 0x00
v.extend_from_slice(&(FILE_LEN as u32).to_be_bytes()); // 0x0C HD_VTKF_SIZE
v.extend_from_slice(b"VPLST000.XPL"); // 0x10 playlist name
v.resize(VTKF_HEADER_LEN, 0); // reserve to first entry (0x80)
for n in 0..VTKF_MAX_ENTRIES {
if let Some(k) = keys.get(n) {
v.push(VTKF_AV_FLG); // BIFO: AV_FLG set (present)
v.extend_from_slice(&[0, 0, 0]); // reserved
v.extend_from_slice(k); // 16-byte encrypted title key
v.extend_from_slice(&[0xFFu8; 16]); // binding MAC (0xFF, pre-recorded)
} else {
v.extend_from_slice(&[0u8; VTKF_ENTRY_LEN]); // empty slot (AV_FLG clear)
}
}
v.resize(FILE_LEN - 16, 0); // reserved gap before the trailer
v.extend_from_slice(&[0xABu8; 16]); // TKF MAC (must NOT be read as a key)
v
}
#[test]
fn parse_vtkf_reads_present_entries_skips_empty_ignores_mac() {
let k1 = [0x11u8; 16];
let k2 = [0x22u8; 16];
let k3 = [0x33u8; 16];
let data = synth_vtkf(&[k1, k2, k3]);
let ukf = parse_vtkf(&data).expect("valid VTKF must parse");
// Exactly the three present entries — the empty slots and the trailing
// 16-byte TKF MAC are NOT mistaken for keys. Critically, k2/k3 are read
// at the 36-byte stride (offsets 0xA4, 0xC8); the old 32-byte stride
// misread them from inside the previous entry's binding MAC.
assert_eq!(ukf.encrypted_keys.len(), 3);
assert_eq!(
ukf.encrypted_keys[0],
(1, k1),
"CPS units = 1-based slot index"
);
assert_eq!(ukf.encrypted_keys[1], (2, k2));
assert_eq!(ukf.encrypted_keys[2], (3, k3));
assert_eq!(ukf.version, AacsVersion::V10, "HD DVD is AACS 1.0");
// disc_hash is SHA1 of the whole file (the KEYDB lookup key).
assert_eq!(ukf.disc_hash, disc_hash(&data));
}
#[test]
fn parse_vtkf_reads_a_full_64_entry_file() {
// Real discs (Freedom, Dukes) carry all 64 slots present. Every key must
// come back, none dropped and none drifted — the regression the 32-byte
// stride failed.
let keys: Vec<[u8; 16]> = (0..VTKF_MAX_ENTRIES).map(|n| [n as u8; 16]).collect();
let ukf = parse_vtkf(&synth_vtkf(&keys)).expect("64-entry VTKF");
assert_eq!(ukf.encrypted_keys.len(), 64);
assert_eq!(
ukf.encrypted_keys[63],
(64, [63u8; 16]),
"entry 64 at 0x{:x}",
VTKF_HEADER_LEN + 63 * VTKF_ENTRY_LEN
);
}
#[test]
fn parse_vtkf_rejects_non_magic() {
let mut data = synth_vtkf(&[[0x11u8; 16]]);
data[0] = b'X'; // corrupt magic
assert!(
parse_vtkf(&data).is_none(),
"non-VTKF magic must be rejected"
);
assert!(
parse_vtkf(&[0u8; 4]).is_none(),
"too short must be rejected"
);
}
#[test]
fn parse_title_keys_dispatches_by_magic() {
// VTKF magic → parse_vtkf.
let data = synth_vtkf(&[[0x44u8; 16], [0x55u8; 16]]);
let ukf = parse_title_keys(&data, AacsVersion::V10).expect("VTKF dispatch");
assert_eq!(ukf.encrypted_keys.len(), 2);
// Non-VTKF → parse_unit_key_ro (a 2-byte buffer is not a valid inf, so
// this proves it ROUTED to the BD parser rather than parse_vtkf).
assert!(
parse_title_keys(&[0x00, 0x00], AacsVersion::V10).is_none(),
"non-magic input must route to parse_unit_key_ro"
);
}
/// The whole point of the seam: a parsed VTKF feeds the SHARED VUK→title-key
/// crypto (`decrypt_unit_key`) exactly like a BD `Unit_Key_RO.inf` would —
/// no HD-DVD-specific crypto path.
#[test]
fn vtkf_encrypted_keys_feed_shared_vuk_unwrap() {
let enc = [0x9Au8; 16];
let data = synth_vtkf(&[enc]);
let ukf = parse_vtkf(&data).unwrap();
let vuk = [0x5Cu8; 16];
let derived = super::super::derive::decrypt_unit_key(&vuk, &ukf.encrypted_keys[0].1);
// Same as applying the shared unwrap directly to the stored enc key.
assert_eq!(derived, super::super::derive::decrypt_unit_key(&vuk, &enc));
}
/// `UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
/// printed every byte; the hand-written impl must not. Sentinel key byte
/// 0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
/// same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
/// `#[derive(Debug)]` back fails this.
#[test]
fn unit_key_file_debug_is_redacted() {
let f = UnitKeyFile {
disc_hash: [0xD5; 20],
app_type: 1,
num_bdmv_dir: 1,
use_skb_mkb: false,
version: AacsVersion::V20,
encrypted_keys: vec![(0, [0xD5; 16]), (1, [0xD5; 16])],
title_cps_unit: vec![0, 1],
};
let dbg = format!("{f:?}");
assert!(
!dbg.contains("213"),
"UnitKeyFile Debug leaked key bytes (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"UnitKeyFile Debug missing redaction marker: {dbg}"
);
// Non-secret shape is still useful for diagnostics.
assert!(dbg.contains("encrypted_keys_len: 2"), "{dbg}");
}
}
#[cfg(test)]
mod read_mkb_tests {
use super::*;
use crate::scsi::{DataDirection, SCSI_READ_DISC_STRUCTURE, ScsiResult, ScsiTransport};
/// A drive that answers READ DISC STRUCTURE format 0x83 from a scripted set
/// of packs and records every CDB it was handed.
struct MkbDrive {
/// One entry per pack: the pack's MKB payload bytes.
packs: Vec<Vec<u8>>,
cdbs: Vec<Vec<u8>>,
}
impl ScsiTransport for MkbDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
self.cdbs.push(cdb.to_vec());
// Pack number is carried in the CDB address field (bytes 2..6),
// MMC-6 READ DISC STRUCTURE.
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]) as usize;
let body = self.packs.get(pack).cloned().unwrap_or_default();
// Header: BE16 data length (counts the 2 header bytes that follow
// it plus the payload), reserved byte, pack count, then payload.
let data_len = body.len() + 2;
data[0..2].copy_from_slice(&(data_len as u16).to_be_bytes());
data[2] = 0x00;
data[3] = self.packs.len() as u8;
data[4..4 + body.len()].copy_from_slice(&body);
Ok(ScsiResult {
status: 0,
bytes_transferred: 4 + body.len(),
sense: [0u8; 32],
})
}
}
/// `read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
/// downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
/// Media Key ladder) consumes exactly what it returns. An empty return is
/// not a benign "no MKB" — it is a total read failure reported as success,
/// and every derivation then fails with a key-not-found code that points
/// the operator at their keydb rather than at the drive.
///
/// This pins the CONTENT: the concatenated payload of all packs, in pack
/// order, byte for byte.
#[test]
fn read_mkb_from_drive_returns_the_concatenated_pack_payload() {
let pack0: Vec<u8> = (0..600u32).map(|i| (i % 251) as u8).collect();
let pack1: Vec<u8> = (0..300u32).map(|i| (i % 253) as u8 ^ 0xA5).collect();
let mut drive = MkbDrive {
packs: vec![pack0.clone(), pack1.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
let mut expected = pack0.clone();
expected.extend_from_slice(&pack1);
assert_eq!(
mkb.len(),
expected.len(),
"every pack's payload must be concatenated, none dropped"
);
assert!(
mkb == expected,
"MKB bytes must be the drive's payload in pack order; first \
mismatch at {:?}",
(0..expected.len()).find(|&i| mkb[i] != expected[i])
);
// MMC-6 READ DISC STRUCTURE with the AACS MKB format code, one command
// per pack, pack number in the address field.
assert_eq!(drive.cdbs.len(), 2, "one command per declared pack");
for (i, cdb) in drive.cdbs.iter().enumerate() {
assert_eq!(cdb[0], SCSI_READ_DISC_STRUCTURE, "opcode");
assert_eq!(cdb[7], 0x83, "AACS MKB disc-structure format code");
assert_eq!(
u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]),
i as u32,
"pack {i} must be requested by number"
);
}
}
/// The CDB is what the drive actually acts on, and every byte of it is
/// load-bearing: a wrong format code returns a different disc structure
/// entirely, and a wrong allocation length truncates the pack. The existing
/// test above pins the opcode, the format code and the pack number; this
/// pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
///
/// Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
/// `[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
/// (BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
/// BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
#[test]
fn read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack() {
let mut drive = MkbDrive {
packs: vec![vec![0x11u8; 64], vec![0x22u8; 64], vec![0x33u8; 64]],
cdbs: Vec::new(),
};
read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(drive.cdbs.len(), 3, "one command per declared pack");
for (pack, cdb) in drive.cdbs.iter().enumerate() {
let p = pack as u32;
let expected: [u8; 12] = [
SCSI_READ_DISC_STRUCTURE,
0x01,
(p >> 24) as u8,
(p >> 16) as u8,
(p >> 8) as u8,
p as u8,
0x00,
0x83, // AACS MKB disc-structure format
0x80, // allocation length 32772 = 0x8004, high byte
0x04, // …low byte
0x00,
0x00,
];
assert_eq!(
cdb.as_slice(),
&expected[..],
"CDB for pack {pack} must match the MMC-6 READ DISC STRUCTURE layout"
);
}
}
/// A pack payload filling the FULL 32768-byte window must come back whole.
/// The `len > 0 && len <= 32768` bound is what stands between a maximal
/// pack and a silently dropped one, and the small payloads used elsewhere
/// in this module never reach it.
#[test]
fn read_mkb_from_drive_accepts_a_full_size_pack() {
let full: Vec<u8> = (0..32768u32).map(|i| (i % 251) as u8).collect();
let other: Vec<u8> = (0..32768u32).map(|i| (i % 241) as u8 ^ 0x5A).collect();
// TWO maximal packs: the first-pack read and the per-pack loop carry
// separate bounds, so both must accept a full-window payload.
let mut drive = MkbDrive {
packs: vec![full.clone(), other.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
65536,
"neither maximal pack may be dropped at the size bound"
);
let mut expected = full.clone();
expected.extend_from_slice(&other);
assert!(mkb == expected, "both maximal packs' bytes must be intact");
}
/// A pack that declares only the 2-byte header and NO payload contributes
/// nothing, and must not push a phantom byte into the MKB — an off-by-one
/// at the zero-length boundary corrupts every following pack's alignment.
#[test]
fn read_mkb_from_drive_zero_length_pack_contributes_nothing() {
let mut drive = MkbDrive {
packs: vec![Vec::new(), vec![0xABu8; 32]],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
32,
"an empty pack adds no bytes; only pack 1's payload is present"
);
assert!(mkb == vec![0xABu8; 32], "and the bytes are pack 1's");
}
/// A drive that DECLARES more payload than it returned must not be
/// believed. The BE16 length in the response header is drive-supplied data:
/// a firmware bug, a short transfer, or a hostile device can put a value in
/// it that runs past the 32772-byte buffer. Copying `len` bytes on that word
/// alone panics the rip thread mid-scan.
///
/// Both the first-pack read and the per-pack loop carry the same bound, so
/// both are exercised here: the over-declared pack contributes nothing and
/// the honest pack still comes through.
#[test]
fn read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds() {
/// Pack 0 is honest; pack 1 declares a 60000-byte payload it never sent.
struct LyingDrive {
honest: Vec<u8>,
}
impl ScsiTransport for LyingDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]);
data[3] = 2; // two packs declared
if pack == 0 {
let dl = self.honest.len() + 2;
data[0..2].copy_from_slice(&(dl as u16).to_be_bytes());
data[4..4 + self.honest.len()].copy_from_slice(&self.honest);
} else {
// A length far beyond the 32772-byte response buffer.
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
}
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let honest = vec![0xC7u8; 256];
let mut drive = LyingDrive {
honest: honest.clone(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("an over-declared pack is not an error");
assert_eq!(
mkb.len(),
honest.len(),
"only the honest pack's bytes may be taken; the over-declared pack \
contributes nothing and must not be read past the buffer"
);
assert!(mkb == honest, "and those bytes are pack 0's");
}
/// The same over-declaration on the FIRST pack, which uses a separate bound
/// from the loop's.
#[test]
fn read_mkb_from_drive_ignores_a_first_pack_declaring_more_than_the_buffer() {
struct LyingFirst;
impl ScsiTransport for LyingFirst {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
data[3] = 1;
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut LyingFirst).expect("not an error");
assert!(
mkb.is_empty(),
"a first pack declaring more than the buffer holds yields no bytes"
);
}
/// A single-pack disc still yields that pack's bytes — the common case, and
/// the one where a body returning an empty vector looks most plausible.
#[test]
fn read_mkb_from_drive_returns_a_single_packs_payload() {
let pack: Vec<u8> = (0..1024u32).map(|i| (i * 7 % 256) as u8).collect();
let mut drive = MkbDrive {
packs: vec![pack.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(mkb.len(), pack.len(), "single pack payload length");
assert!(mkb == pack, "single pack payload bytes");
}
/// A drive that reports a header-only response (`data_len < 2`) has no MKB
/// to give. That must be an EMPTY vec, not a partial one — the distinction
/// matters because the AACS paths treat a non-empty MKB as parseable.
#[test]
fn read_mkb_from_drive_empty_response_is_empty() {
struct NoMkb;
impl ScsiTransport for NoMkb {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&0u16.to_be_bytes());
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut NoMkb).expect("no-MKB drive still returns Ok");
assert!(
mkb.is_empty(),
"a header-only response carries no MKB bytes"
);
}
/// A transport failure on the FIRST pack must propagate as an error — the
/// MKB is the root of the whole AACS ladder, so an unreadable one cannot be
/// downgraded to "an MKB with no records".
#[test]
fn read_mkb_from_drive_propagates_the_first_pack_failure() {
struct DeadDrive;
impl ScsiTransport for DeadDrive {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
Err(crate::error::Error::ScsiError {
opcode: SCSI_READ_DISC_STRUCTURE,
status: 0x02,
sense: None,
})
}
}
assert!(
read_mkb_from_drive(&mut DeadDrive).is_err(),
"an unreadable MKB must surface as an error, not an empty MKB"
);
}
}
+540
View File
@@ -0,0 +1,540 @@
//! AACS Media Key Block — [C] Chapter 3.
//!
//! The MKB record format (framing walker, the `MkbRecord` view, record-body
//! finders), the MKBType / AACS-generation classification, and MKB-file
//! utilities (content length, trimming, version). Consolidated here so the one
//! place that understands MKB bytes is `mkb`. Some duplicate record finders
//! still live side by side pending a follow-up that collapses them.
// ── MKB record types ([C] Chapter 3) ──────────────────────────────────────
// The ONE canonical set. Every record-type comparison in the `aacs` module
// references these, so a type byte is never a bare literal scattered across
// files (the `0x0c` variant-data record in particular used to appear in several
// hand-rolled forms).
/// Type-and-Version — carries the 32-bit MKBType / AACS generation.
pub(crate) const REC_TYPE_AND_VERSION: u8 = 0x10;
/// Subset-Difference index — the per-slot `(u_mask_shift, uv)` table.
pub(crate) const REC_SUBSET_DIFFERENCE: u8 = 0x04;
/// Media Key Data — the classical (1.0 / 2.0) per-subset cvalue table.
pub(crate) const REC_MEDIA_KEY_DATA: u8 = 0x05;
/// Explicit Subset-Difference — the smaller cvalue table some MKBs use.
pub(crate) const REC_EXPLICIT_SUBSET_DIFF: u8 = 0x07;
/// Media Key Variant Data (AACS 2.1) — the per-subset-difference `C` table
/// (one 16-byte C per slot); the `Kmp` step reads C from HERE, not `0x2d`.
pub(crate) const REC_MEDIA_KEY_VARIANT_DATA: u8 = 0x0c;
/// Variant Data + Nonce (AACS 2.1) — the `VARIANTS[uv]` table (leading bytes)
/// with the 16-byte `Kvn` Nonce at the tail.
pub(crate) const REC_VARIANT_DATA_AND_NONCE: u8 = 0x2d;
/// Variant Key Data table (AACS 2.1) — 65,535×16, indexed by the resolved VKD index.
pub(crate) const REC_VKD_TABLE: u8 = 0x2f;
/// Verify-Media-Key — AACS 1.0.
pub(crate) const REC_VERIFY_MEDIA_KEY_V1: u8 = 0x81;
/// Verify-Media-Key — AACS 2.x.
pub(crate) const REC_VERIFY_MEDIA_KEY_V2: u8 = 0x86;
/// A single MKB record produced by [`walk_mkb`].
#[derive(Debug, Clone)]
pub struct MkbRecord {
/// Byte offset of the record within the MKB.
pub offset: usize,
/// Record type byte.
pub rec_type: u8,
/// Record length in bytes (includes the 4-byte header).
pub rec_len: usize,
/// Record body (the bytes after the 4-byte header).
pub body: Vec<u8>,
}
/// Walk an MKB into a flat list of records.
///
/// MKB record framing per AACS: 1 byte type, 3 bytes BE length
/// INCLUDING the 4-byte header, followed by payload. The walker stops
/// at the first `(type=0, len=0)` end marker or at end of buffer.
pub fn walk_mkb(mkb: &[u8]) -> Vec<MkbRecord> {
mkb_records(mkb)
.map(|(offset, rec_type, rec_len)| MkbRecord {
offset,
rec_type,
rec_len,
body: mkb[offset + 4..offset + rec_len].to_vec(),
})
.collect()
}
/// THE single MKB record-framing walker: yields `(offset, rec_type, rec_len)`
/// for each record — a 4-byte header (type byte + big-endian 24-bit length)
/// then the body — stopping at the `00 000000` end marker or a
/// malformed/out-of-bounds length. Lazy (no body clone), so a find-one-record
/// caller never materialises the multi-MB cvalue table. [`walk_mkb`] and every
/// MKB record walk in `aacs::resolve`/`aacs::derive` are built on this, so the framing rules — and
/// any future fix to them — live in exactly one place (they had drifted across
/// six hand-rolled copies).
pub(crate) fn mkb_records(mkb: &[u8]) -> impl Iterator<Item = (usize, u8, usize)> + '_ {
let mut pos = 0usize;
std::iter::from_fn(move || {
if pos + 4 > mkb.len() {
return None;
}
let rec_type = mkb[pos];
let rec_len = ((mkb[pos + 1] as usize) << 16)
| ((mkb[pos + 2] as usize) << 8)
| (mkb[pos + 3] as usize);
if rec_type == 0 && rec_len == 0 {
return None;
}
if rec_len < 4 || pos + rec_len > mkb.len() {
return None;
}
let here = pos;
pos += rec_len;
Some((here, rec_type, rec_len))
})
}
pub(crate) fn mkb_find_body(records: &[MkbRecord], rec_type: u8) -> Option<&[u8]> {
records
.iter()
.find(|r| r.rec_type == rec_type && !r.body.is_empty())
.map(|r| r.body.as_slice())
}
/// AACS protection generation a disc carries.
///
/// The content cert byte distinguishes V10 (`0x00`) from V20 (`0x01`). V21
/// cannot be detected from the cert alone — a V21 disc carries a V20 cert
/// and is upgraded to `V21` only after the MKB walk turns up the real Variant
/// records `0x2d` / `0x2f` (Encrypted Media Key Variant Data and the Variant
/// Key Data table).
///
/// Key-storage stride in `Unit_Key_RO.inf` is 48 bytes for V10 and 64
/// bytes for V20 / V21.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AacsVersion {
/// AACS 1.0 — original BD-ROM.
V10,
/// AACS 2.0 — UHD-BD, classical Media Key derivation.
V20,
/// AACS 2.1 — UHD-BD with Media Key Variant chain on top of V20.
V21,
}
/// AACS major version as the small integer threaded through the scan / key
/// paths (`AacsState.version`, `DiscInputs.version`, `DiscInputsCtx::new`):
/// 1 = AACS 1.0 (BD), 2 = AACS 2.x (UHD). Centralised so the bare `1`/`2` — and
/// the V10-vs-else stride choice it drives — lives in exactly one place.
pub const AACS_MAJOR_BD: u8 = 1;
pub const AACS_MAJOR_UHD: u8 = 2;
impl AacsVersion {
/// Stride (in bytes) between successive encrypted unit keys in
/// `Unit_Key_RO.inf`.
pub(crate) fn unit_key_stride(self) -> usize {
match self {
AacsVersion::V10 => 48,
AacsVersion::V20 | AacsVersion::V21 => 64,
}
}
/// This version as the major integer ([`AACS_MAJOR_BD`] / [`AACS_MAJOR_UHD`]).
pub fn major(self) -> u8 {
match self {
AacsVersion::V10 => AACS_MAJOR_BD,
AacsVersion::V20 | AacsVersion::V21 => AACS_MAJOR_UHD,
}
}
/// The version a bare major integer selects for stride purposes: only the
/// BD major is V10; every other value takes the V20/V21 64-byte stride.
pub fn from_major(major: u8) -> Self {
if major == AACS_MAJOR_BD {
AacsVersion::V10
} else {
AacsVersion::V20
}
}
}
/// Find Verify Media Key Record (type 0x81 for AACS 1.0, 0x86 for AACS 2.0/2.1) in MKB.
/// 0x81: [C] §3.2.5.1.4. 0x86 (AACS 2.x): [RE] — not in the public spec (from real 2.x MKBs).
pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
// Verify-Media-Key record (0x81 for AACS 1.0, 0x86 for AACS 2.x): mk_dv is
// the 16 bytes at record offset 4 (body offset 0). Needs rec_len >= 20.
let found = mkb_records(mkb).find(|&(_, rt, len)| {
(rt == REC_VERIFY_MEDIA_KEY_V1 || rt == REC_VERIFY_MEDIA_KEY_V2) && len >= 20
});
match found {
Some((o, rec_type, rec_len)) => {
let mut dv = [0u8; 16];
dv.copy_from_slice(&mkb[o + 4..o + 20]);
tracing::debug!(
target: "freemkv::disc",
phase = "mkb_mk_dv_found",
rec_type,
pos = o,
rec_len,
"mk_dv extracted from MKB"
);
Some(dv)
}
None => {
tracing::warn!(
target: "freemkv::disc",
phase = "mkb_mk_dv_not_found",
"no 0x81/0x86 record with rec_len>=20 found"
);
None
}
}
}
/// Find Subset-Difference records (type 0x04) in MKB. [C] §3.2.5.1.5.
pub(crate) fn mkb_find_subdiff_records(mkb: &[u8]) -> Option<Vec<u8>> {
find_record_body(mkb, 0x04)
}
/// Find the Media Key Data Record (cvalues table) in an MKB. [C] §3.2.4 / §3.2.5.1.7.
///
/// The cvalue table is record type `0x05` (Media Key Data) on BOTH AACS
/// 1.0 and AACS 2.x MKBs — its 16-byte cvalue entries are 1:1 with the
/// 5-byte Subset-Difference index entries in record `0x04` — the standard AACS
/// MKB layout (`0x05` cvalues 1:1 with the `0x04` subset-difference index).
///
/// On AACS 2.x in-drive UHD MKBs the `0x05` table is large (the full
/// subset-difference cvalue set: ~181k entries on a retail MKB, 1:1 with
/// the giant `0x04` index), while record `0x07` (Explicit
/// Subset-Difference Record) is a much smaller structure (~96 entries) and
/// is NOT the cvalue table. An earlier version of this function preferred
/// `0x07`, which under-tested the Subset-Difference walk on UHD discs and
/// prevented the DK→walk path from ever finding the matching uv. The
/// selection MUST therefore be `0x05`-first; `0x07` is only a fallback for
/// malformed/legacy MKBs that somehow lack a `0x05` record.
pub(crate) fn mkb_find_cvalues(mkb: &[u8]) -> Option<Vec<u8>> {
if let Some(body) = find_record_body(mkb, 0x05) {
return Some(body);
}
find_record_body(mkb, 0x07)
}
/// Walk an MKB and return the payload (header stripped) of the first
/// record matching `rec_type`. Returns `None` if no such record exists or
/// the record is empty.
pub(crate) fn find_record_body(mkb: &[u8], rec_type_wanted: u8) -> Option<Vec<u8>> {
mkb_records(mkb)
.find(|&(_, rt, len)| rt == rec_type_wanted && len > 4)
.map(|(o, _, len)| mkb[o + 4..o + len].to_vec())
}
/// Real content length of an MKB: the byte offset where the record stream
/// ends. MKB files (especially `MKB_RW.inf`, but `MKB_RO.inf` too on some
/// discs) are allocated to a fixed size — often ~128 MiB — with the records at
/// the front and the rest zero padding. Walking records (type+len) and stopping
/// at the first padding byte (`type == 0` / zero-length / overrun) gives the
/// actual size so callers can trim off megabytes of zeros before sending or
/// archiving. Returns `mkb.len()` only if the whole buffer parsed as records.
pub fn mkb_content_len(mkb: &[u8]) -> usize {
// End of the last framed record = where the fixed-region zero padding begins.
// (The `00 000000` terminator / overrun stops the walk; real MKBs pad with
// zeros, so this matches the prior "stop at the first padding byte".)
mkb_records(mkb)
.last()
.map(|(o, _, len)| o + len)
.unwrap_or(0)
}
/// Trim an MKB's trailing fixed-region padding to its real content length —
/// but ONLY when [`mkb_content_len`] actually found one. It returns 0 for an
/// MKB whose first record cannot be parsed; truncating to 0 in that case would
/// hand downstream consumers (and the online key service) an EMPTY MKB that can
/// never resolve. So a 0 (or a length that isn't strictly inside the buffer)
/// leaves the MKB untouched. A 0.31.0 regression dropped this guard and
/// `truncate`-d unconditionally, zeroing unrecognised MKBs.
pub fn trim_mkb(mut mkb: Vec<u8>) -> Vec<u8> {
let n = mkb_content_len(&mkb);
if n > 0 && n < mkb.len() {
mkb.truncate(n);
}
mkb
}
/// Get MKB version from Type and Version Record (type 0x10).
/// Layout: 4-byte record header at `pos` (type + BE24 length), then the
/// record body starts at `pos + 4`. The body holds the BE u32 Type field at
/// body offset 0 (`pos + 4`), then the BE u32 version at body offset 4
/// (`pos + 8`).
pub fn mkb_version(mkb: &[u8]) -> Option<u32> {
// Type-and-Version record (0x10): version is the BE u32 at body offset 4
// (record offset 8). Needs rec_len >= 12 (4 header + 4 type + 4 version).
mkb_records(mkb)
.find(|&(_, rt, len)| rt == REC_TYPE_AND_VERSION && len >= 12)
.map(|(o, _, _)| u32::from_be_bytes([mkb[o + 8], mkb[o + 9], mkb[o + 10], mkb[o + 11]]))
}
/// `0x00031003` — recordable media MKB (Class I & II compute Km directly).
pub const MKB_TYPE_3_RECORDABLE: u32 = 0x0003_1003;
/// `0x00041003` — AACS 1.0 pre-recorded content MKB (KCD-based). Standard BD.
pub const MKB_TYPE_4_PRERECORDED: u32 = 0x0004_1003;
/// `0x000A1003` — Class II / Unified MKB (Sequence-Key-Block functionality).
pub const MKB_TYPE_10_CLASS_II: u32 = 0x000A_1003;
/// `0x48141003` — AACS 2.0 Category C (UHD content) MKB type value.
pub const MKB_20_CATEGORY_C: u32 = 0x4814_1003;
/// `0x48151003` — AACS 2.1 Category C (UHD content) MKB type value.
pub const MKB_21_CATEGORY_C: u32 = 0x4815_1003;
/// The AACS MKB Type field, decoded.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MkbType {
/// Type 3 — recordable media.
Recordable,
/// Type 4 — AACS 1.0 pre-recorded content (KCD). Standard Blu-ray.
Prerecorded,
/// Type 10 — Class II / Unified (SKB).
ClassII,
/// AACS 2.0 Category C — UHD content.
CategoryC20,
/// AACS 2.1 Category C — UHD content.
CategoryC21,
/// Unrecognized MKBType value (raw field preserved).
Other(u32),
}
impl MkbType {
pub(crate) fn from_raw(raw: u32) -> Self {
match raw {
MKB_TYPE_3_RECORDABLE => MkbType::Recordable,
MKB_TYPE_4_PRERECORDED => MkbType::Prerecorded,
MKB_TYPE_10_CLASS_II => MkbType::ClassII,
MKB_20_CATEGORY_C => MkbType::CategoryC20,
MKB_21_CATEGORY_C => MkbType::CategoryC21,
other => MkbType::Other(other),
}
}
/// AACS generation this MKB belongs to (Category C → 2.0/2.1, else 1.0).
pub fn generation(self) -> AacsVersion {
match self {
MkbType::CategoryC21 => AacsVersion::V21,
MkbType::CategoryC20 => AacsVersion::V20,
_ => AacsVersion::V10,
}
}
/// `true` for UHD (AACS 2.x Category C); `false` for Blu-ray (AACS 1.x).
pub fn is_uhd(self) -> bool {
matches!(self, MkbType::CategoryC20 | MkbType::CategoryC21)
}
}
/// The raw 32-bit MKBType field from the Type-and-Version record (0x10), bytes
/// 4-7. `None` if no 0x10 record is present. [C] §3.2.5.1.1 Table 3-2.
pub fn mkb_type_raw(mkb: &[u8]) -> Option<u32> {
// Type-and-Version record (0x10): the 32-bit MKBType is bytes 4-7 (body
// offset 0). Needs rec_len >= 8 (4 header + 4 type).
mkb_records(mkb)
.find(|&(_, rt, len)| rt == REC_TYPE_AND_VERSION && len >= 8)
.map(|(o, _, _)| u32::from_be_bytes([mkb[o + 4], mkb[o + 5], mkb[o + 6], mkb[o + 7]]))
}
/// Decode an MKB's Type field. `None` if no Type-and-Version record is present.
pub fn mkb_type(mkb: &[u8]) -> Option<MkbType> {
mkb_type_raw(mkb).map(MkbType::from_raw)
}
/// `Some(true)` if this MKB is a UHD (AACS 2.x Category C) block, `Some(false)`
/// for Blu-ray (AACS 1.x), `None` if the Type record is absent.
pub fn mkb_is_uhd(mkb: &[u8]) -> Option<bool> {
mkb_type(mkb).map(MkbType::is_uhd)
}
#[cfg(test)]
mod tests {
use super::*;
/// One MKB record: 1 type byte + big-endian 24-bit total length + body.
fn rec(rec_type: u8, body: &[u8]) -> Vec<u8> {
let len = 4 + body.len();
let mut v = vec![rec_type, (len >> 16) as u8, (len >> 8) as u8, len as u8];
v.extend_from_slice(body);
v
}
/// Type-and-Version record (0x10): body = 4-byte MKBType + 4-byte version.
fn type_and_version(mkb_type: u32, version: u32) -> Vec<u8> {
let mut body = mkb_type.to_be_bytes().to_vec();
body.extend_from_slice(&version.to_be_bytes());
rec(REC_TYPE_AND_VERSION, &body)
}
#[test]
fn walker_frames_records_and_stops_at_end_marker() {
let mut mkb = type_and_version(MKB_20_CATEGORY_C, 77);
mkb.extend(rec(REC_VKD_TABLE, &[0xAA; 16]));
mkb.extend([0x00, 0x00, 0x00, 0x00]); // end marker
mkb.extend(rec(0x99, &[0xFF; 8])); // must NOT be walked (past the marker)
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "walk stops at the 00 000000 end marker");
assert_eq!(recs[0].rec_type, REC_TYPE_AND_VERSION);
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].body, vec![0xAA; 16]);
}
#[test]
fn walker_stops_on_malformed_or_out_of_bounds_length() {
// A record whose declared length runs past the buffer end must terminate
// the walk rather than panic or read OOB.
let mkb = vec![REC_VKD_TABLE, 0x00, 0xFF, 0xFF, 0x01, 0x02]; // len=0xFFFF, only 6 bytes
assert!(
walk_mkb(&mkb).is_empty(),
"over-long record yields no records"
);
// A sub-4 length (shorter than the header itself) is also rejected.
let short = vec![REC_VKD_TABLE, 0x00, 0x00, 0x02];
assert!(walk_mkb(&short).is_empty(), "sub-4 length is rejected");
// A truncated header (< 4 bytes) yields nothing.
assert!(walk_mkb(&[0x10, 0x00]).is_empty());
}
#[test]
fn mkb_type_and_version_decode_from_the_type_record() {
let mut mkb = type_and_version(MKB_21_CATEGORY_C, 100);
mkb.extend([0x00, 0x00, 0x00, 0x00]);
assert_eq!(mkb_type_raw(&mkb), Some(MKB_21_CATEGORY_C));
assert_eq!(mkb_version(&mkb), Some(100));
assert_eq!(mkb_is_uhd(&mkb), Some(true), "2.1 Category C is UHD");
let bd = type_and_version(MKB_TYPE_4_PRERECORDED, 68);
assert_eq!(
mkb_is_uhd(&bd),
Some(false),
"AACS 1.0 prerecorded is not UHD"
);
// No Type record → None (not a panic, not a fabricated value).
assert_eq!(mkb_version(&rec(REC_VKD_TABLE, &[0; 16])), None);
assert_eq!(mkb_type_raw(&[]), None);
}
#[test]
fn trim_mkb_keeps_only_the_framed_records() {
let mut mkb = type_and_version(MKB_20_CATEGORY_C, 1);
let content_len = mkb.len(); // the single framed record, no end marker
mkb.extend([0x00, 0x00, 0x00, 0x00]); // end marker
mkb.extend([0xDE; 4096]); // trailing padding past the end marker
let trimmed = trim_mkb(mkb);
assert_eq!(
trimmed.len(),
content_len,
"trim keeps the framed records, dropping the end marker and padding"
);
}
// ── BE24 length field: all THREE bytes ────────────────────────────────
/// The record length is a big-endian **24-bit** field, so the high byte
/// carries lengths of 64 KiB and up. The MKB records that matter most are
/// exactly that size — a real UHD cvalue table is `46_101 * 16` bytes and a
/// `0x2d` variant record is ~92 KiB — so a walker that dropped the high
/// byte would mis-frame every record of a real MKB from the first big one
/// onward, and every downstream key lookup would read the wrong bytes.
///
/// (The pre-existing high-byte test used total length `0x0110`, whose high
/// byte is ZERO — it exercised the middle byte only. This one puts a
/// non-zero value in the high byte.)
#[test]
fn mkb_records_honors_the_high_byte_of_the_be24_length() {
const TOTAL: usize = 0x0001_0004; // 65_540 — high byte 0x01
let mut mkb = vec![REC_VKD_TABLE, 0x01, 0x00, 0x04];
mkb.resize(TOTAL, 0xAB);
// A second record follows, so a walker that mis-read the length would
// frame a different number of records rather than merely a short one.
mkb.extend(rec(REC_TYPE_AND_VERSION, &[0x11; 8]));
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the big record must be framed as ONE record");
assert_eq!(
recs[0].rec_len, TOTAL,
"rec_len must include the high BE24 byte"
);
assert_eq!(recs[0].body.len(), TOTAL - 4);
assert_eq!(
recs[1].rec_type, REC_TYPE_AND_VERSION,
"the following record must start where the big one ends"
);
}
// ── Header-only records and the exact end marker ──────────────────────
/// `rec_len == 4` is a well-formed HEADER-ONLY record (the minimum the
/// walker accepts), including one sitting at the very end of the buffer
/// with no bytes after it. Rejecting either — the `pos + 4` bound or the
/// `rec_len < 4` floor being off by one — silently drops the MKB's last
/// record, and "the record isn't there" is indistinguishable from "the disc
/// doesn't carry it".
#[test]
fn mkb_records_yields_a_header_only_record_at_the_buffer_end() {
let mut mkb = rec(REC_TYPE_AND_VERSION, &[0xAA, 0xBB]);
mkb.extend([REC_VKD_TABLE, 0x00, 0x00, 0x04]); // 4-byte, empty body, at EOF
assert_eq!(
mkb.len(),
10,
"sanity: the last record ends at the buffer end"
);
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the trailing header-only record is a record");
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].rec_len, 4);
assert!(recs[1].body.is_empty());
}
/// ONLY the exact `00 00 00 00` marker ends the walk. A record whose TYPE
/// happens to be `0x00` but which declares a real length is a record, not
/// the end of the MKB — stopping there would truncate everything after it,
/// including the cvalue and verify records the key derivation needs.
#[test]
fn mkb_records_stops_only_on_the_all_zero_end_marker() {
// A type-0 record of length 8, then a normal record, then the marker.
let mut mkb = vec![0x00, 0x00, 0x00, 0x08, 1, 2, 3, 4];
mkb.extend(rec(REC_VKD_TABLE, &[0x55; 16]));
mkb.extend([0x00, 0x00, 0x00, 0x00]); // the real end marker
mkb.extend(rec(0x99, &[0xFF; 4])); // past the marker: not walked
let recs = walk_mkb(&mkb);
assert_eq!(
recs.len(),
2,
"a type-0 record with a non-zero length is a record, not the end"
);
assert_eq!(recs[0].rec_type, 0x00);
assert_eq!(recs[0].rec_len, 8);
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].body, vec![0x55; 16]);
}
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim ([C] §3.2.5.1.1
/// Table 3-2), including a value this build does not recognise — the caller
/// uses it to tell "unknown MKB generation" from "no Type record at all".
/// All four bytes must come from the record body; reading any of them from
/// the wrong offset yields a type that silently classifies as a different
/// AACS generation.
///
/// The recognised constants all share bytes with the `0x10` record-type
/// header byte (e.g. `MKB_21_CATEGORY_C` is `48 15 10 03`), so this uses a
/// value with four distinct bytes, none of them `0x10`.
#[test]
fn mkb_type_raw_reads_all_four_body_bytes() {
const RAW: u32 = 0xDEAD_BEEF;
let mkb = type_and_version(RAW, 7);
assert_eq!(
mkb_type_raw(&mkb),
Some(RAW),
"every byte of the MKBType field must come from the record body"
);
assert_eq!(mkb_version(&mkb), Some(7));
}
}
+345 -60
View File
@@ -13,20 +13,52 @@
//!
//! The VUK decrypts title keys from AACS/Unit_Key_RO.inf on disc.
//! Title keys decrypt m2ts stream content (AES-128-CBC).
//!
//! ## Spec provenance
//!
//! The crypto below carries `[TAG] §x.y` citations back to the published AACS
//! specification (Final Rev 0.953), so each primitive links to the section it
//! implements:
//! - `[C]` — AACS Introduction and Common Cryptographic Elements Book (primitives, MKB/key-management).
//! - `[PR]` — AACS Pre-recorded Video Book (Volume/Title Key layer).
//! - `[BD]` — AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit, Block Key).
//! - `[RE]` — reverse-engineered from real discs, cited only where the public
//! spec is silent (the `0x86` verify record and the Category-C MKB type values).
pub mod boil;
pub mod decrypt;
pub mod content;
pub mod crypto;
pub mod derive;
pub mod host_certs;
pub mod keys;
pub mod index_select;
pub mod inf;
pub mod mkb;
pub mod provider;
pub mod resolve;
pub mod segment;
pub mod segment_key;
pub mod trace;
pub mod types;
pub mod variants;
pub mod variant;
/// On-disc UDF paths to the AACS key-input files (with their fallbacks).
/// Centralised so every reader (`resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, `read_aacs_version`) walks the exact same files — adding
/// or changing a fallback in one place can then never silently diverge the
/// On-disc UDF paths to the AACS key-input files, plus HD DVD AACS-directory
/// discovery.
///
/// BD and UHD keep their key material under a fixed `/AACS/…` tree, so those
/// paths are constants. HD DVD keeps the equivalents in a reserved root
/// directory whose NAME is authoring-house-specific — observed `ANY!` (Dukes
/// of Hazzard) and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK`
/// mirror — and whose title-key file is NOT always `VTKF000.AACS` (Freedom
/// ships `VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED
/// from the parsed UDF tree ([`find_hddvd_aacs_dir`] + [`role_paths`]), never
/// hardcoded.
///
/// Each key ROLE ([`AacsRole`]) resolves to an ordered candidate list — the
/// BD/UHD constants first, then whatever the HD DVD directory actually holds —
/// which every reader walks with [`read_first`], first-that-reads. No reader
/// ever branches on disc type: a BD/UHD disc has the `/AACS/` files so those
/// win; an HD DVD has none of them, so it falls through to the discovered
/// entries. Centralised so `resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, and `read_aacs_version` can never silently diverge the
/// disc_hash / MKB / VID that another reader feeds a key service.
pub const PATH_UNIT_KEY_RO: &str = "/AACS/Unit_Key_RO.inf";
pub const PATH_UNIT_KEY_RO_DUPLICATE: &str = "/AACS/DUPLICATE/Unit_Key_RO.inf";
@@ -35,50 +67,135 @@ pub const PATH_MKB_RW: &str = "/AACS/MKB_RW.inf";
pub const PATH_CONTENT_CERT: &str = "/AACS/Content000.cer";
pub const PATH_CONTENT_CERT_ALT: &str = "/AACS/Content001.cer";
// Boil-down derivation primitives (thin newtypes + wrappers over the crypto).
pub use boil::{MediaKey, UnitKey, Vid, Vuk, mk_from_dk, mk_from_pk, uk_from_vuk, vuk_from_mk};
// Structured, English-free resolution trace.
pub use trace::{KeyNode, KeyOutcome, KeyStep, ResolutionTrace, UnlockOutcome, UnlockStep};
/// An AACS key-input role. [`role_paths`] maps it to an ordered candidate path
/// list (BD/UHD constants, then the discovered HD DVD files).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AacsRole {
/// Title-key file: BD/UHD `Unit_Key_RO.inf`, HD DVD `VTKF*.AACS`
/// (magic `DVD_HD_V_TKF`). The disc_hash is `SHA1` of this file.
UnitKey,
/// Media Key Block: BD/UHD `MKB_RO/RW.inf`, HD DVD `MKBROM.AACS`.
Mkb,
/// Content certificate: BD/UHD `Content000/001.cer`, HD DVD
/// `CONTENT_CERT.AACS` (byte 0 gives the AACS major).
ContentCert,
}
// Explicit re-exports — only items needed by external consumers and sibling crate modules.
// AES primitives (aes_ecb_encrypt, aes_ecb_decrypt, aes_cbc_decrypt) are pub(crate) in decrypt.rs.
pub use decrypt::{
ALIGNED_UNIT_LEN, ALIGNED_UNIT_SECTORS, UnitKeyResult, aacs_unit_encrypted,
aacs_unit_needs_decrypt, aacs_unit_still_ciphertext, decrypt_bus, decrypt_unit,
decrypt_unit_checked, decrypt_unit_full, decrypt_unit_try_keys, fill_null_ts_unit,
is_unit_aligned, ts_packet_total, ts_sync_count, ts_sync_destroyed, unit_is_clean_ps,
unit_is_clean_ts, unit_key_validates,
};
// `probe` is a reproduction-harness helper (see keys.rs), not part of the
// documented 1.0 surface; keep it reachable but off the rendered docs so we
// don't commit semver stability to test primitives.
#[doc(hidden)]
pub use keys::probe;
pub use keys::{
AACS_MAJOR_BD, AACS_MAJOR_UHD, AacsVersion, ContentCert, MKB_20_CATEGORY_C, MKB_21_CATEGORY_C,
MKB_TYPE_3_RECORDABLE, MKB_TYPE_4_PRERECORDED, MKB_TYPE_10_CLASS_II, MkbType, ResolveContext,
ResolveFailure, ResolvedKeys, UnitKeyFile, decrypt_unit_key, derive_media_key_and_pk_from_dk,
derive_media_key_from_dk, derive_media_key_from_pk, derive_vuk, disc_hash, disc_hash_hex,
mkb_content_len, mkb_is_uhd, mkb_type, mkb_type_raw, mkb_version, parse_content_cert,
parse_unit_key_ro, read_mkb_from_drive, recover_dk_position, resolve_keys_v1, resolve_keys_v2,
resolve_keys_v21, resolve_keys_with_reason, trim_mkb,
};
pub use provider::KeyProvider;
pub use types::{DeviceKey, DiscEntry, HostCert};
pub use variants::{
KEY_CORRECTION_DATA_PLACEHOLDER, MediaKeyVariantError, MkbRecord, ProcessingKeyMatch,
derive_media_key_variant, is_variant_mkb, variant_nonce, walk_mkb, walk_processing_key,
};
/// The HD DVD AACS directory in a parsed UDF tree, if present.
///
/// Identified structurally, NOT by a hardcoded name: the root child directory
/// whose name ends in `!` (so the `<name>!_BAK` backup mirror, which also ends
/// in a non-`!` char, is not mistaken for it) and which contains `MKBROM.AACS`.
/// Observed real names: `ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD
/// disc has no such directory → `None`.
pub(crate) fn find_hddvd_aacs_dir(udf: &crate::udf::UdfFs) -> Option<&crate::udf::DirEntry> {
udf.root.entries.iter().find(|e| {
e.is_dir
&& e.name.ends_with('!')
&& e.entries
.iter()
.any(|c| !c.is_dir && c.name.eq_ignore_ascii_case("MKBROM.AACS"))
})
}
/// Ordered candidate paths for an AACS key [`AacsRole`]: the fixed BD/UHD
/// `/AACS/…` paths first, then the actual HD DVD files discovered in the disc's
/// AACS directory (see [`find_hddvd_aacs_dir`]). A disc has only one family, so
/// the other family's entries simply never read.
///
/// For [`AacsRole::UnitKey`] every `VTKF*.AACS` in the directory is appended in
/// sorted name order — a disc may carry more than one variant (Freedom:
/// `VTKF090` + `VTKF100`), not just `VTKF000`.
pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String> {
let mut v: Vec<String> = match role {
AacsRole::UnitKey => vec![PATH_UNIT_KEY_RO, PATH_UNIT_KEY_RO_DUPLICATE],
AacsRole::Mkb => vec![PATH_MKB_RO, PATH_MKB_RW],
AacsRole::ContentCert => vec![PATH_CONTENT_CERT, PATH_CONTENT_CERT_ALT],
}
.into_iter()
.map(String::from)
.collect();
if let Some(dir) = find_hddvd_aacs_dir(udf) {
let d = &dir.name;
match role {
AacsRole::Mkb => v.push(format!("/{d}/MKBROM.AACS")),
AacsRole::ContentCert => v.push(format!("/{d}/CONTENT_CERT.AACS")),
AacsRole::UnitKey => {
// Glob VTKF*.AACS — the title-key filename is not fixed at
// VTKF000 (Freedom ships VTKF090 + VTKF100). Sorted for a
// deterministic try order.
//
// Each VTKF%%%.AACS is bound to ONE playlist (VPLST%%%.XPL): the
// TKF's 12-byte PLAYLIST_NAME field (bytes 0x10..0x1C) names the
// playlist whose Title Keys it carries, and keys from a TKF whose
// name does not match the title's playlist must not be used. The
// caller resolves this by trying candidates in sorted order and
// decrypting with the one whose keys verify — correct for a
// single-playlist disc; a name-matched selection keyed on the
// active playlist is the precise form for multi-playlist discs.
let mut names: Vec<&str> = dir
.entries
.iter()
.filter(|e| !e.is_dir)
.filter(|e| {
let u = e.name.to_ascii_uppercase();
u.starts_with("VTKF") && u.ends_with(".AACS")
})
.map(|e| e.name.as_str())
.collect();
names.sort_unstable();
v.extend(names.into_iter().map(|n| format!("/{d}/{n}")));
}
}
}
v
}
/// Walk an AACS role's candidate paths (from [`role_paths`]) and return the
/// first that reads.
///
/// `read` performs the actual per-path read (full file or bounded prefix), so
/// callers share the same first-present walk regardless of read style. Returns
/// [`Error::AacsNoKeys`] if no candidate is present. Generic over the path
/// element (`&str` or owned `String`) so it accepts the `Vec<String>` that
/// [`role_paths`] builds from the discovered HD DVD directory.
pub(crate) fn read_first<S, F>(candidates: &[S], mut read: F) -> crate::error::Result<Vec<u8>>
where
S: AsRef<str>,
F: FnMut(&str) -> crate::error::Result<Vec<u8>>,
{
for path in candidates {
if let Ok(buf) = read(path.as_ref()) {
return Ok(buf);
}
}
Err(crate::error::Error::AacsNoKeys)
}
// The module structure IS the public API — consumers import from the owning
// module directly (e.g. `aacs::content::decrypt_unit`, `aacs::mkb::MkbType`,
// `aacs::derive::{derive_vuk, resolve_candidate}`, `aacs::resolve::resolve_keys_v2`).
// The `derive::probe` reproduction harness stays reachable via its module path.
//
// A small set of flat re-exports is kept for the typed key primitives and the
// content-decrypt entry points that downstream key-source crates import through
// the `aacs::` path. These are the stable, load-bearing names; keeping them here
// lets those crates track the module refactor without a lockstep re-pin.
pub use content::ALIGNED_UNIT_LEN;
pub use derive::derive_vuk;
pub use types::{DeviceKey, HostCert, MediaKey, ProcessingKey, UnitKey, Vid, Vuk};
#[cfg(test)]
mod tests {
//! Re-export surface guards. The module's public API is the set of
//! `pub use` items above. A regression that drops or renames an export
//! (the class of bug that shipped in 0.31.0 by silently changing a
//! surface) breaks compilation of these references, so they act as a
//! compile-time contract for the crate's AACS surface.
//! Surface guards. The public API is the module tree itself (no facade).
//! Touching one representative item per module keeps these as a
//! compile-time contract that the module paths stay stable.
use super::*;
use super::content::ALIGNED_UNIT_LEN;
use super::inf::{disc_hash, disc_hash_hex};
use super::mkb::{AacsVersion, mkb_content_len, walk_mkb};
use super::variant::is_variant_mkb;
#[test]
fn aligned_unit_len_is_three_2048_byte_sectors() {
@@ -100,21 +217,189 @@ mod tests {
}
#[test]
fn key_correction_data_placeholder_is_all_zero() {
// The variant chain refuses to run against this all-zero placeholder
// KCD; the public constant must therefore be exactly 16 zero bytes.
assert_eq!(KEY_CORRECTION_DATA_PLACEHOLDER, [0u8; 16]);
}
#[test]
fn public_helpers_are_callable_through_the_facade() {
// Touch a representative function from each re-export group so a
// dropped/renamed export fails to compile. These are smoke calls, not
// behavioural assertions (behaviour is covered in each module).
let _ = ts_sync_destroyed(&[0u8; ALIGNED_UNIT_LEN]);
fn public_helpers_are_callable_by_module_path() {
// Touch a representative function from each module so a dropped/renamed
// item fails to compile. Smoke calls, not behavioural assertions.
let _ = !crate::aacs::content::is_clean(
&[0u8; ALIGNED_UNIT_LEN],
crate::disc::ContentFormat::BdTs,
);
let _ = mkb_content_len(&[]);
let _ = is_variant_mkb(&walk_mkb(&[]));
let _ = disc_hash_hex(&disc_hash(b"x"));
let _ = mk_from_pk(&[[0u8; 16]], &[]);
let _ = super::derive::resolve_candidate(
&super::derive::KeyCandidate::Uk(super::types::UnitKey::new(0, [0u8; 16])),
&[],
&[],
None,
);
}
// ── HD DVD AACS directory / filename discovery ────────────────────────
//
// The HD DVD AACS dir name and title-key filename are authoring-specific
// and were previously hardcoded to `/ANY!/VTKF000.AACS`. These verify the
// discovery replacement against both real-disc shapes: Freedom (`AAC!` +
// `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
#[test]
fn role_paths_discovers_hddvd_dir_and_globs_all_vtkf_variants() {
use crate::udf::fixture::*;
// Freedom-shaped: an `AAC!` dir (NOT `ANY!`) holding MKBROM + two VTKF
// variants (090/100, NOT 000) + a VTUF usage file (must be excluded),
// plus the `AAC!_BAK` mirror (must NOT be picked as the AACS dir).
let mut disc = MemDisc::new();
let aacs_files = vec![
file("MKBROM.AACS", 100, 5000, 4096, true),
file("CONTENT_CERT.AACS", 101, 5100, 2048, true),
file("VTKF100.AACS", 102, 5200, 2048, true),
file("VTKF090.AACS", 103, 5300, 2048, true),
file("VTUF090.AACS", 104, 5400, 2048, true),
];
let bak_files = vec![file("MKBROM.AACS", 110, 6000, 4096, true)];
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![
DirSpec {
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: aacs_files,
subdirs: vec![],
},
DirSpec {
name: "AAC!_BAK".to_string(),
icb_lba: 30,
dir_data_lba: 31,
files: bak_files,
subdirs: vec![],
},
],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
// Discovered structurally (ends in '!', holds MKBROM.AACS) — the real
// AACS dir, never the `_BAK` mirror.
let dir = super::find_hddvd_aacs_dir(&udf).expect("aacs dir");
assert_eq!(dir.name, "AAC!");
// UnitKey: BD/UHD paths first, then EVERY VTKF*.AACS in sorted order
// (090 before 100) — NOT hardcoded VTKF000; VTUF (usage) excluded.
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
"/AAC!/VTKF090.AACS".to_string(),
"/AAC!/VTKF100.AACS".to_string(),
]
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::Mkb)
.last()
.unwrap(),
"/AAC!/MKBROM.AACS"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::ContentCert)
.last()
.unwrap(),
"/AAC!/CONTENT_CERT.AACS"
);
}
/// The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required —
/// the discovery is a conjunction, not a disjunction.
///
/// The existing fixtures only ever present a directory that satisfies both
/// (`AAC!` with `MKBROM.AACS`) alongside one that satisfies neither
/// (`AAC!_BAK` — which contains `MKBROM.AACS` but is ALSO reached only after
/// the real dir), so either half of the conjunction could be dropped and the
/// same directory would still be found. Here a directory satisfies the name
/// half and NOT the contents half: it must not be picked.
///
/// If it were, the HD DVD path would resolve `MKBROM.AACS`,
/// `CONTENT_CERT.AACS` and the title-key file under a directory that holds
/// none of them — the disc reports "no AACS key files" and never rips.
#[test]
fn a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
// Ends in '!' — but carries no MKBROM.AACS, so it is not the
// HD DVD AACS directory.
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTKF090.AACS", 102, 5200, 2048, true),
file("CONTENT_CERT.AACS", 103, 5300, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
super::find_hddvd_aacs_dir(&udf).is_none(),
"a '!' directory without MKBROM.AACS is not the AACS directory"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
],
"no HD DVD candidates may be appended from a directory that was \
never identified as the AACS directory"
);
}
#[test]
fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() {
use crate::udf::fixture::*;
// A `/AACS/` tree (BD/UHD) has no '!' directory → discovery finds none
// and the candidate list is exactly the static BD/UHD paths.
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "AACS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("Unit_Key_RO.inf", 100, 5000, 2048, true),
file("MKB_RO.inf", 101, 5100, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(super::find_hddvd_aacs_dir(&udf).is_none());
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
]
);
}
}
+48 -1
View File
@@ -42,7 +42,7 @@ use super::types::{DeviceKey, DiscEntry, HostCert};
/// Source of AACS key material.
///
/// Implementors return raw material only — the resolver in
/// `aacs::keys` owns all the crypto (DK→PK walking, PK validation,
/// `aacs::resolve` and `aacs::derive` own the crypto (DK→PK walking, PK validation,
/// MK→VUK→TK derivation). See module docs for method semantics.
pub trait KeyProvider: Send + Sync {
/// Device keys (top-of-tree, walked by the resolver).
@@ -197,6 +197,17 @@ mod tests {
}
}
/// A host cert whose (non-secret) certificate body and private key are both
/// filled with `byte`, so a cert is identifiable in an aggregated list.
fn cert(byte: u8) -> HostCert {
HostCert {
private_key: [byte; 20],
certificate: vec![byte; 92],
private_key_v2: None,
certificate_v2: None,
}
}
fn dk(byte: u8, node: u16) -> DeviceKey {
DeviceKey {
key: [byte; 16],
@@ -357,6 +368,42 @@ mod tests {
assert_eq!(got.disc_hash, "vid-a");
}
/// `Providers::host_certs` is the union across the provider array. It is not
/// wired into the handshake today (see the module docs), so nothing else in
/// the crate would notice a body that dropped every cert on the floor — and
/// the day it IS wired in, a silently-empty cert list means the drive AACS
/// authentication finds no host certificate to present and every disc fails
/// to open, with no indication that the caller's certs were discarded.
///
/// Unlike the bulk key unions this one does NOT dedup (HostCert is not
/// Ord/Hash), so the assertion is on the full concatenation in array order.
#[test]
fn providers_host_certs_unions_every_providers_certs_in_array_order() {
struct Certs(Vec<HostCert>);
impl KeyProvider for Certs {
fn host_certs(&self) -> Vec<HostCert> {
self.0.clone()
}
}
// Distinguish certs by their (non-secret) certificate body, so the
// assertion lands on WHICH certs came back, not merely how many.
let a = Certs(vec![cert(0xA1), cert(0xA2)]);
let b = Certs(vec![cert(0xB1)]);
let arr: &[&dyn KeyProvider] = &[&a, &b];
let got = Providers(arr).host_certs();
let bodies: Vec<Vec<u8>> = got.iter().map(|c| c.certificate.clone()).collect();
assert_eq!(
bodies,
vec![vec![0xA1u8; 92], vec![0xA2u8; 92], vec![0xB1u8; 92]],
"every provider's certs must survive the union, in array order"
);
// The private key travels with the cert — a union that returned default
// certs would still have the right count.
assert_eq!(got[0].private_key, [0xA1u8; 20]);
assert_eq!(got[2].private_key, [0xB1u8; 20]);
}
#[test]
fn providers_empty_array_yields_nothing() {
let arr: &[&dyn KeyProvider] = &[];
File diff suppressed because it is too large Load Diff
+454
View File
@@ -0,0 +1,454 @@
//! AACS 2.1 FMTS forensic segment map — `AACS/IndividualSegment.tbl`.
//!
//! An FMTS main feature interleaves short forensic **segments** — the sequence-key
//! / forensic-watermark mechanism. Each segment carries an **index** (1..32): a
//! tag in `IndividualSegment.tbl` that selects which of the 32 forensic **index
//! keys** decrypts that segment's units, in place of the ordinary CPS Unit Key.
//!
//! Terminology (see the project AACS reference): the **index** here is NOT the
//! AACS 2.1 *Media Key Variant* — that is the 65536-value device selector in the
//! MKB that decides *which set* of index keys a device receives, a layer this
//! module does not deal with. All the index keys belong to one variant, whose
//! number is unknown and irrelevant to the segment map. Decrypting a segment with
//! the Unit Key yields garbage — broken HEVC reference frames (empirically:
//! `Could not find ref with POC …` on a plain unit-key rip).
//!
//! This table says WHERE the segments live and which index each carries, so a
//! decoder can decrypt them with the matching index key instead of muxing
//! unit-key garbage.
//!
//! Format (validated against a retail AACS 2.1 disc):
//! ```text
//! header (8 bytes): u32 type | u16 count | u16 record_size (= 16)
//! record[count] (16 bytes each):
//! u32 marker (= 0x01000000) | u16 index | u16 flag (= 1)
//! u32 start_spn | u32 end_spn (source-packet numbers, inclusive)
//! ```
//! `index` is the 1..32 forensic index tag, NOT a sequential segment id: measured
//! on a retail 2.1 disc it cycles 1,2,…,32,1,2,… across records in
//! file order — 24 full cycles of 32 plus a final partial cycle of 24 = 792
//! records. Source-packet numbers are the 192-byte BDAV packet index: byte offset
//! = `spn * 192`. Each segment is ~2560 packets (~480 KB) = 80 aligned units,
//! spread across the entire 54 GB feature (one roughly every 67 MB). Inside a
//! segment the 80 units interleave in two stride-2 halves: applying the segment's
//! index key decrypts ~40 of them to clean TS and garbles the other ~40 (a second
//! interleaved half, unidentified), which the demux then drops — leaving one
//! coherent stream. Confirmed by decoding a retail disc with a full set of 32
//! index keys.
/// Fixed size of one `IndividualSegment.tbl` record.
pub const SEGMENT_RECORD_LEN: usize = 16;
/// Bytes per BDAV source packet (188-byte TS + 4-byte arrival-time header).
pub const SOURCE_PACKET_LEN: u64 = 192;
/// One forensic segment: the inclusive source-packet range it occupies in the
/// FMTS clip.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Segment {
/// Forensic index tag, 1..=32 (field@4 of the record). Cycles across the
/// table rather than counting up — it selects WHICH of the 32 index keys
/// decrypts this range. (`0` is not used here; the default/non-forensic
/// content carries no segment record at all.)
pub index: u16,
/// First source packet of the segment (inclusive).
pub start_spn: u32,
/// Last source packet of the segment (inclusive).
pub end_spn: u32,
}
impl Segment {
/// Source-packet count in this (inclusive) segment.
pub fn packet_count(&self) -> u32 {
self.end_spn
.saturating_sub(self.start_spn)
.saturating_add(1)
}
/// Byte offset of the segment start within the clip (`start_spn * 192`).
pub fn start_byte(&self) -> u64 {
self.start_spn as u64 * SOURCE_PACKET_LEN
}
/// Byte length of the segment (`packet_count * 192`).
pub fn byte_len(&self) -> u64 {
self.packet_count() as u64 * SOURCE_PACKET_LEN
}
/// True when source packet `spn` falls inside this segment.
pub fn contains_spn(&self, spn: u32) -> bool {
spn >= self.start_spn && spn <= self.end_spn
}
/// True when the inclusive source-packet span `[first, last]` overlaps this
/// segment. Used to decide whether an aligned unit (which spans several
/// packets) touches the segment at all, not just whether one packet does.
pub fn overlaps_spn(&self, first: u32, last: u32) -> bool {
first <= self.end_spn && last >= self.start_spn
}
}
/// Source packets spanned by one AACS aligned unit: `6144 / 192 = 32`.
pub const PACKETS_PER_UNIT: u32 =
(crate::aacs::content::ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32;
/// Byte offset within the clip of a clip-relative 2048-byte sector `lba`. The
/// FMTS decode reads the clip file directly, so `lba` 0 is the clip's first
/// byte and this offset lines up with the source-packet grid the segment map
/// uses.
pub fn lba_byte_offset(lba: u32) -> u64 {
lba as u64 * 2048
}
/// The forensic segment an AACS aligned unit belongs to, if any, given the
/// unit's clip-relative byte offset.
///
/// This is the routing decision behind a 2.1 decrypt-miss: a unit that
/// overlaps a forensic segment must be opened with that segment's **index key**
/// (selected by the segment's `index`), not the CPS Unit Key. Opening it with
/// the Unit Key is exactly what yields the broken-reference-frame garbage a
/// plain unit-key rip produces. A unit outside every segment is ordinary
/// content and a miss on it is a Unit-Key miss, so this returns `None` and the
/// caller falls back to the normal unit-key fetch.
///
/// The unit is tested as a packet *span* (`[off/192, (off+6144-1)/192]`) so a
/// unit that only partly overlaps a segment edge is still classified as
/// forensic; on the observed disc segments are unit-aligned, but the span test
/// does not rely on that.
pub fn segment_for_unit(segments: &[Segment], unit_offset: u64) -> Option<&Segment> {
let unit_len = crate::aacs::content::ALIGNED_UNIT_LEN as u64;
let first = (unit_offset / SOURCE_PACKET_LEN) as u32;
let last = ((unit_offset + unit_len - 1) / SOURCE_PACKET_LEN) as u32;
segments.iter().find(|s| s.overlaps_spn(first, last))
}
/// Parse `IndividualSegment.tbl` into its forensic segments, in table
/// order. Returns `None` when the header is malformed, the record size is not
/// [`SEGMENT_RECORD_LEN`], or the declared record count overruns the buffer —
/// so a truncated / foreign table degrades to "no segment map" rather than
/// yielding bogus ranges.
pub fn parse_individual_segments(tbl: &[u8]) -> Option<Vec<Segment>> {
if tbl.len() < 8 {
return None;
}
let count = u16::from_be_bytes([tbl[4], tbl[5]]) as usize;
let record_size = u16::from_be_bytes([tbl[6], tbl[7]]) as usize;
if record_size != SEGMENT_RECORD_LEN {
return None;
}
if 8usize.checked_add(count.checked_mul(record_size)?)? > tbl.len() {
return None;
}
let mut segments = Vec::with_capacity(count);
for i in 0..count {
let o = 8 + i * record_size;
// o+4..o+8 = index (u16, 1..32) + flag (u16); o+8..o+16 = start/end SPN.
let index = u16::from_be_bytes([tbl[o + 4], tbl[o + 5]]);
let start_spn = u32::from_be_bytes([tbl[o + 8], tbl[o + 9], tbl[o + 10], tbl[o + 11]]);
let end_spn = u32::from_be_bytes([tbl[o + 12], tbl[o + 13], tbl[o + 14], tbl[o + 15]]);
segments.push(Segment {
index,
start_spn,
end_spn,
});
}
Some(segments)
}
/// Map a clip-relative byte offset to the absolute LBA that holds it, by walking
/// the title's extents (the `.fmts` clip's sectors in file order). Segment
/// offsets in [`Segment`] are clip-relative source-packet numbers, so this is how
/// a segment's `spn` range becomes disc LBAs. `None` if the offset is past the
/// clip.
pub fn clip_byte_to_lba(extents: &[crate::disc::Extent], clip_byte: u64) -> Option<u32> {
let mut cum = 0u64;
for e in extents {
let len = e.sector_count as u64 * crate::consts::SECTOR_BYTES as u64;
if clip_byte < cum + len {
let sector_in_ext = ((clip_byte - cum) / crate::consts::SECTOR_BYTES as u64) as u32;
return Some(e.start_lba.saturating_add(sector_in_ext));
}
cum += len;
}
None
}
/// Build the `[start_lba, end_lba) → key_idx` ranges for an FMTS forensic key map.
///
/// Each forensic segment's clip-relative source-packet span becomes an absolute
/// LBA range tagged with the key its `index` selects (via `index_to_key_idx`,
/// e.g. `|i| i as usize` when the pool is `[base, idx1, idx2, …]`). Applying that
/// one key across the whole segment decodes the ~40 units of its interleave half
/// to clean TS and garbles the other ~40 (the second interleaved half), which the
/// demux then drops — yielding one coherent stream. Ranges outside every segment
/// are left for the map's default (the ordinary Unit Key). A segment that straddles
/// a UDF extent boundary is emitted as one range per whole-sector slice it covers.
///
/// The result feeds [`AacsKeyMap::from_ranges`](crate::decrypt::AacsKeyMap::from_ranges)
/// with the Unit-Key index as the default — the same structure the CPS map uses,
/// only finer-grained.
pub fn fmts_key_ranges(
segments: &[Segment],
extents: &[crate::disc::Extent],
index_to_key_idx: &dyn Fn(u16) -> usize,
) -> Vec<(u32, u32, usize)> {
let mut ranges = Vec::new();
for s in segments {
// SPNs are untrusted (from IndividualSegment.tbl); an inverted record
// (start_spn > end_spn) would underflow `end_byte - 1 - start_byte` below.
if s.start_spn > s.end_spn {
continue;
}
let start_byte = s.start_spn as u64 * SOURCE_PACKET_LEN;
let end_byte = (s.end_spn as u64 + 1) * SOURCE_PACKET_LEN; // exclusive
// A segment is unit-aligned and contiguous in clip bytes; map its first
// and last sector to LBAs. Segments are ~480 KB and extents are GB-sized,
// so a segment almost never crosses an extent boundary — but if the two
// ends land in different extents (non-contiguous LBAs), skip rather than
// emit a wrong span; the units there fall to the Unit Key (garble+drop),
// never a mis-decrypt.
let (Some(a), Some(b)) = (
clip_byte_to_lba(extents, start_byte),
clip_byte_to_lba(extents, end_byte - 1),
) else {
continue;
};
if b >= a
&& (b - a) as u64 == (end_byte - 1 - start_byte) / crate::consts::SECTOR_BYTES as u64
{
ranges.push((a, b + 1, index_to_key_idx(s.index)));
}
}
ranges
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a table with the real on-disc layout: 8-byte header + N 16-byte
/// records. `recs` are `(index, start_spn, end_spn)`.
fn build_tbl(recs: &[(u16, u32, u32)]) -> Vec<u8> {
let mut v = Vec::new();
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // type
v.extend_from_slice(&(recs.len() as u16).to_be_bytes()); // count
v.extend_from_slice(&(SEGMENT_RECORD_LEN as u16).to_be_bytes()); // record_size
for &(n, s, e) in recs {
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // marker
v.extend_from_slice(&n.to_be_bytes());
v.extend_from_slice(&1u16.to_be_bytes()); // flag
v.extend_from_slice(&s.to_be_bytes());
v.extend_from_slice(&e.to_be_bytes());
}
v
}
#[test]
fn fmts_key_ranges_maps_segments_to_lba_by_index() {
use crate::disc::Extent;
// One big clip extent starting at LBA 1000. Clip byte B lives at
// LBA 1000 + B/2048.
let extents = vec![Extent {
start_lba: 1000,
sector_count: 1_000_000,
}];
// Two segments, indexes 5 and 7 (spn ranges as on a real disc).
let segs = vec![
Segment {
index: 5,
start_spn: 100,
end_spn: 199,
},
Segment {
index: 7,
start_spn: 10_000,
end_spn: 10_099,
},
];
// Pool layout [base, idx1, idx2, …] → index N uses key slot N.
let ranges = fmts_key_ranges(&segs, &extents, &|v| v as usize);
assert_eq!(ranges.len(), 2, "one LBA range per segment");
// Segment 0: spn 100..=199 → clip bytes [19200, 38400) → sectors 9..=18
// → LBA 1009..1019, key index 5.
assert_eq!(ranges[0], (1009, 1019, 5));
// Segment 1: spn 10000..=10099 → bytes [1_920_000, 1_939_200) →
// sectors 937..=946 → LBA 1937..1947, key index 7.
assert_eq!(ranges[1], (1937, 1947, 7));
// The ranges drive a positive AacsKeyMap: an LBA in no range has no key.
let map = crate::decrypt::AacsKeyMap::from_ranges(ranges);
assert_eq!(map.key_idx_for(500), None, "outside any segment → no key");
assert_eq!(
map.key_idx_for(1012),
Some(5),
"inside index-5 segment → key 5"
);
assert_eq!(
map.key_idx_for(1940),
Some(7),
"inside index-7 segment → key 7"
);
assert_eq!(
map.key_idx_for(1019),
None,
"segment end is exclusive → no key"
);
}
#[test]
fn fmts_key_ranges_skips_inverted_segment_without_underflow() {
use crate::disc::Extent;
let extents = vec![Extent {
start_lba: 1000,
sector_count: 1_000_000,
}];
// start_spn == end_spn + 1: `end_byte - 1 - start_byte` would underflow.
// The record must be skipped rather than panic (debug) / wrap (release).
let segs = vec![Segment {
index: 5,
start_spn: 200,
end_spn: 199,
}];
let ranges = fmts_key_ranges(&segs, &extents, &|v| v as usize);
assert!(ranges.is_empty(), "inverted segment yields no range");
}
#[test]
fn clip_byte_to_lba_walks_extents() {
use crate::disc::Extent;
let extents = vec![
Extent {
start_lba: 100,
sector_count: 10,
}, // clip bytes [0, 20480)
Extent {
start_lba: 500,
sector_count: 10,
}, // clip bytes [20480, 40960)
];
assert_eq!(clip_byte_to_lba(&extents, 0), Some(100));
assert_eq!(clip_byte_to_lba(&extents, 2048), Some(101));
assert_eq!(clip_byte_to_lba(&extents, 20480), Some(500)); // second extent
assert_eq!(clip_byte_to_lba(&extents, 22528), Some(501));
assert_eq!(clip_byte_to_lba(&extents, 40960), None); // past the clip
}
#[test]
fn parses_real_disc_layout() {
// First three records observed on retail 2.1: the variant
// field counts 1,2,3,… (it wraps at 32 further into the table — see
// `index_field_cycles_one_to_thirty_two`), segments are 2560 packets.
let tbl = build_tbl(&[
(1, 343680, 346239),
(2, 695616, 698175),
(3, 1051840, 1054399),
]);
let segs = parse_individual_segments(&tbl).expect("parse");
assert_eq!(segs.len(), 3);
assert_eq!(segs[0].index, 1);
assert_eq!(segs[1].index, 2);
assert_eq!(segs[2].index, 3);
assert_eq!(segs[0].start_spn, 343680);
assert_eq!(segs[0].end_spn, 346239);
assert_eq!(segs[0].packet_count(), 2560);
assert_eq!(segs[0].byte_len(), 2560 * 192);
assert_eq!(segs[0].start_byte(), 343680 * 192);
assert!(segs[0].contains_spn(345000));
assert!(!segs[0].contains_spn(343679));
assert!(!segs[0].contains_spn(346240));
}
#[test]
fn rejects_wrong_record_size() {
let mut tbl = build_tbl(&[(1, 0, 10)]);
tbl[6..8].copy_from_slice(&20u16.to_be_bytes()); // record_size != 16
assert!(parse_individual_segments(&tbl).is_none());
}
#[test]
fn rejects_truncated_and_overrun() {
assert!(parse_individual_segments(&[0u8; 4]).is_none()); // < header
let mut tbl = build_tbl(&[(1, 0, 10)]);
tbl[4..6].copy_from_slice(&99u16.to_be_bytes()); // claims 99 recs, has 1
assert!(parse_individual_segments(&tbl).is_none());
}
#[test]
fn empty_table_is_empty_not_none() {
let tbl = build_tbl(&[]);
assert_eq!(parse_individual_segments(&tbl), Some(Vec::new()));
}
#[test]
fn packets_per_unit_is_thirty_two() {
// 6144-byte aligned unit / 192-byte source packet.
assert_eq!(PACKETS_PER_UNIT, 32);
}
#[test]
fn unit_inside_segment_routes_to_index() {
// A real first-record segment: packets [343680, 346239].
let segs = parse_individual_segments(&build_tbl(&[(1, 343680, 346239)])).unwrap();
// A unit sitting squarely inside: start at packet 344000 → byte 344000*192.
let off = 344000u64 * SOURCE_PACKET_LEN;
let hit = segment_for_unit(&segs, off).expect("inside the segment");
assert_eq!(hit.index, 1);
}
#[test]
fn index_field_cycles_one_to_thirty_two() {
// Reality on a retail 2.1 disc: field@4 is the index, cycling 1..=32 in file
// order (NOT a sequential segment id). Reproduce one-and-a-bit cycles.
let mut recs = Vec::new();
let mut spn = 1000u32;
for row in 0..2 {
for v in 1..=32u16 {
recs.push((v, spn, spn + 2559));
spn += 50_000; // ~one segment every ~67 MB
}
let _ = row;
}
let segs = parse_individual_segments(&build_tbl(&recs)).unwrap();
assert_eq!(segs.len(), 64);
assert_eq!(segs[31].index, 32); // end of first cycle
assert_eq!(segs[32].index, 1); // wraps, does not become 33
assert!(segs.iter().all(|s| (1..=32).contains(&s.index)));
}
#[test]
fn unit_outside_every_segment_is_unit_key_miss() {
let segs = parse_individual_segments(&build_tbl(&[(1, 343680, 346239)])).unwrap();
// A unit well before the segment is ordinary content → None (unit-key path).
let off = 1000u64 * SOURCE_PACKET_LEN;
assert!(segment_for_unit(&segs, off).is_none());
}
#[test]
fn unit_straddling_a_segment_edge_counts_as_forensic() {
// Segment starts at packet 100. A unit that ENDS just inside it (its 32
// packets straddle the boundary) must still route to the index key,
// because part of its ciphertext is forensic-encrypted.
let segs = parse_individual_segments(&build_tbl(&[(7, 100, 200)])).unwrap();
// Unit covering packets [80, 111]: overlaps [100,200] at the tail.
let off = 80u64 * SOURCE_PACKET_LEN;
let hit = segment_for_unit(&segs, off).expect("straddles the start edge");
assert_eq!(hit.index, 7);
// A unit ending exactly at packet 99 (offset s.t. last = 99) does NOT overlap.
let before = 68u64 * SOURCE_PACKET_LEN; // [68, 99]
assert!(segment_for_unit(&segs, before).is_none());
}
#[test]
fn no_segments_never_routes_to_index() {
// The 1.0 / 2.0 case: no forensic map, so every miss is a unit-key miss.
assert!(segment_for_unit(&[], lba_byte_offset(0)).is_none());
assert!(segment_for_unit(&[], lba_byte_offset(9_999_999)).is_none());
}
#[test]
fn lba_maps_to_the_packet_grid() {
// A unit is 3 sectors (6144 bytes) = 32 packets. Clip-relative LBA 3 is
// the second aligned unit, which starts at packet 32.
let off = lba_byte_offset(3);
assert_eq!(off / SOURCE_PACKET_LEN, 32);
}
}
+165
View File
@@ -0,0 +1,165 @@
//! AACS 2.1 FMTS forensic segment keys, `AACS/SegmentKeyNNNNN.tbl`.
//!
//! One file per CPS unit (`SegmentKey00001.tbl`, ...). It is the on-disc key
//! store for the forensic variant segments mapped by [`super::segment`]. A
//! device does not read a segment key directly. It derives a **16-bit variant
//! selector** from the Media Key Variant chain (see [`super::variant`]) and uses
//! that selector to index this table, which is how the device's position in the
//! key tree decides which variant it can decrypt (the traitor-tracing link).
//!
//! Container format (confirmed against a retail AACS 2.1 disc):
//! ```text
//! header (8 bytes): u32 tag | u16 index_space | u16 record_size
//! record[index_space] (record_size bytes each)
//! ```
//! On the reference disc: `index_space` = `0xffff` (the full 16-bit selector
//! space, 65536 records), `record_size` = `0x0218` = 536. Total
//! `8 + 65536 * 536 = 35,127,304` bytes, which matches the file exactly. Each
//! record begins with an 8-byte sub-header, then 528 bytes of encrypted key
//! material.
//!
//! **Not yet reversed:** the internal layout of a record's 528-byte payload, and
//! how it maps onto the segments of [`super::segment`]. One numeric coincidence
//! worth noting for whoever cracks it: the reference disc has 792 segments and
//! `528 = 33 * 16`, with `792 = 24 * 33`, so `33` appears on both sides. Until
//! the mapping and the key derivation are pinned, this module exposes only the
//! confirmed container: locate the record for a given 16-bit selector.
/// Bytes of the fixed file header.
pub const HEADER_LEN: usize = 8;
/// The on-disc segment-key table container. Borrows the file bytes; a record is
/// looked up by the 16-bit variant selector.
#[derive(Debug, Clone, Copy)]
pub struct SegmentKeyTable<'a> {
data: &'a [u8],
/// Number of records (the selector index space, e.g. 65536).
count: usize,
/// Bytes per record (e.g. 536).
record_size: usize,
}
impl<'a> SegmentKeyTable<'a> {
/// Parse and validate the container header against the buffer length.
///
/// Returns `None` when the buffer is too small, or the declared
/// `count * record_size` (plus header) does not match the buffer, so a
/// truncated or foreign table degrades to "no segment keys" rather than
/// handing back bogus records. `index_space` of `0xffff` is read as the full
/// 65536-entry space (a device selector is a full 16-bit value).
pub fn parse(data: &'a [u8]) -> Option<Self> {
if data.len() < HEADER_LEN {
return None;
}
let index_space = u16::from_be_bytes([data[4], data[5]]);
let record_size = u16::from_be_bytes([data[6], data[7]]) as usize;
// 0xffff means the full 16-bit selector space (65536 records).
let count = if index_space == 0xffff {
0x1_0000
} else {
index_space as usize
};
if record_size == 0 {
return None;
}
let body = count.checked_mul(record_size)?;
if HEADER_LEN.checked_add(body)? != data.len() {
return None;
}
Some(Self {
data,
count,
record_size,
})
}
/// Number of records (the selector index space).
pub fn record_count(&self) -> usize {
self.count
}
/// Bytes per record.
pub fn record_size(&self) -> usize {
self.record_size
}
/// The raw record for a 16-bit variant `selector`, including its 8-byte
/// sub-header. `None` if the selector is past the table (only possible when
/// `index_space` was not the full 16-bit space).
pub fn record(&self, selector: u16) -> Option<&'a [u8]> {
let idx = selector as usize;
if idx >= self.count {
return None;
}
let start = HEADER_LEN + idx * self.record_size;
self.data.get(start..start + self.record_size)
}
/// The encrypted key payload for a selector: the record with its 8-byte
/// sub-header stripped. The internal layout of these bytes is not yet
/// reversed (see module docs).
pub fn record_payload(&self, selector: u16) -> Option<&'a [u8]> {
self.record(selector).and_then(|r| r.get(HEADER_LEN..))
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a container with `record_size` and the given `index_space`, filling
/// each record with a distinguishable byte so lookups can be checked.
fn build(index_space: u16, record_size: u16) -> Vec<u8> {
let count = if index_space == 0xffff {
0x1_0000
} else {
index_space as usize
};
let mut v = Vec::with_capacity(HEADER_LEN + count * record_size as usize);
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // tag
v.extend_from_slice(&index_space.to_be_bytes());
v.extend_from_slice(&record_size.to_be_bytes());
for i in 0..count {
let mut rec = vec![(i & 0xff) as u8; record_size as usize];
// sub-header, as seen on disc
rec[..8].copy_from_slice(&[0x01, 0x00, 0x00, 0x00, 0x00, 0x20, 0x01, 0x02]);
v.extend_from_slice(&rec);
}
v
}
#[test]
fn parses_retail_container_geometry() {
// The real disc: 0xffff index space, 536-byte records, 35,127,304 total.
let data = build(0xffff, 536);
assert_eq!(
data.len(),
35_127_304,
"matches the retail file size exactly"
);
let t = SegmentKeyTable::parse(&data).expect("parse");
assert_eq!(t.record_count(), 65_536);
assert_eq!(t.record_size(), 536);
let rec = t.record(0x1234).expect("record");
assert_eq!(rec.len(), 536);
assert_eq!(&rec[..8], &[0x01, 0x00, 0x00, 0x00, 0x00, 0x20, 0x01, 0x02]);
assert_eq!(t.record_payload(0x1234).unwrap().len(), 528);
}
#[test]
fn small_index_space_bounds_lookups() {
let data = build(4, 32);
let t = SegmentKeyTable::parse(&data).expect("parse");
assert_eq!(t.record_count(), 4);
assert!(t.record(3).is_some());
assert!(t.record(4).is_none(), "selector past the table is None");
}
#[test]
fn rejects_size_mismatch_and_truncation() {
assert!(SegmentKeyTable::parse(&[0u8; 4]).is_none());
let mut data = build(4, 32);
data.truncate(data.len() - 1); // body no longer matches header
assert!(SegmentKeyTable::parse(&data).is_none());
}
}
+296 -3
View File
@@ -6,7 +6,7 @@
//! owns only the crypto and these value types that flow through it.
/// A device key for MKB subset-difference tree processing.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DeviceKey {
pub key: [u8; 16],
pub node: u16,
@@ -15,7 +15,7 @@ pub struct DeviceKey {
}
/// Host certificate + private key for AACS SCSI authentication.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct HostCert {
/// AACS 1.0: 20 bytes. AACS 2.0: 32 bytes.
pub private_key: [u8; 20],
@@ -27,8 +27,75 @@ pub struct HostCert {
pub certificate_v2: Option<Vec<u8>>,
}
/// Volume ID (16 bytes) — read from the disc via the SCSI handshake / OEM path.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vid(pub [u8; 16]);
/// Media Key (Km, 16 bytes) — the MKB-scoped key derived from device keys.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct MediaKey(pub [u8; 16]);
/// Volume Unique Key (VUK / Kvu, 16 bytes) — derived from `MediaKey` + `Vid`,
/// decrypts the per-disc encrypted title keys in `Unit_Key_RO.inf`.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vuk(pub [u8; 16]);
/// Processing Key (Kp, 16 bytes) — an MKB Subset-Difference key that yields the
/// Media Key. A leaked/precomputed PK in the keydb, or the intermediate PK a
/// device-key walk derives at its matching SD node.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct ProcessingKey(pub [u8; 16]);
/// One decrypted per-CPS-unit AACS title key.
///
/// `idx` is the POSITIONAL index of the encrypted title key within the slice
/// handed to the VUK→UK step (i.e. its order in `Unit_Key_RO.inf`'s key-storage
/// area). The CPS-unit *number* association is a higher-level concern owned by
/// [`super::inf::parse_unit_key_ro`], which pairs each positional key with its
/// declared CPS unit; this primitive only does the AES, so it surfaces position.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct UnitKey {
pub idx: u32,
pub key: [u8; 16],
/// AACS 2.1 (FMTS) forensic **index** tag (see [`crate::aacs::segment`]).
///
/// `0` = ordinary (non-forensic) content — the value for every 1.0 / 2.0
/// key and for the bulk of a 2.1 title. `1..=32` = a forensic index key that
/// decrypts the `IndividualSegment.tbl` segments tagged with that same index.
/// This is the per-segment index (1..32), NOT the AACS 2.1 Media Key Variant
/// (the 65536-value device selector), which is a separate MKB-layer concern.
pub index_number: u8,
}
impl UnitKey {
/// An ordinary (non-forensic) unit key: `index_number == 0`. The value
/// for every AACS 1.0 / 2.0 key and the bulk of a 2.1 title.
pub const fn new(idx: u32, key: [u8; 16]) -> Self {
Self {
idx,
key,
index_number: 0,
}
}
/// A forensic index key: `index_number` in `1..=32`, decrypting the
/// `IndividualSegment.tbl` segments tagged with that index.
pub const fn forensic(idx: u32, key: [u8; 16], index_number: u8) -> Self {
Self {
idx,
key,
index_number,
}
}
/// Whether this key decrypts ordinary (non-forensic) content (index 0).
pub const fn is_default_index(&self) -> bool {
self.index_number == 0
}
}
/// A per-disc entry from the key database.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DiscEntry {
/// Disc hash (20 bytes, hex)
pub disc_hash: String,
@@ -43,3 +110,229 @@ pub struct DiscEntry {
/// Unit keys (title keys) indexed by CPS unit number
pub unit_keys: Vec<(u32, [u8; 16])>,
}
// ── Redacting `Debug` impls ──────────────────────────────────────────────────
//
// Every type above carries AACS secret material (device keys, host PRIVATE keys,
// media/volume/processing/unit keys). `#[derive(Debug)]` would print those bytes
// verbatim, so a stray `debug!("{:?}", …)` or a panic message would leak the
// keys. These hand-written impls print only NON-secret shape (presence, lengths,
// tree coordinates, indices) — never key bytes. `decrypt::DecryptKeys` follows
// the same policy by omitting `Debug` entirely; here we keep `Debug` because
// these are `PartialEq`/`Eq` value types used in `assert_eq!` and nested inside
// other `#[derive(Debug)]` structs, so the trait must exist — just not leak.
// Guarded by `redaction_tests` below.
impl std::fmt::Debug for DeviceKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DeviceKey")
.field("key", &"<redacted>")
.field("node", &self.node)
.field("uv", &self.uv)
.field("u_mask_shift", &self.u_mask_shift)
.finish()
}
}
impl std::fmt::Debug for HostCert {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HostCert")
.field("private_key", &"<redacted>")
.field("certificate_len", &self.certificate.len())
.field("private_key_v2", &self.private_key_v2.map(|_| "<redacted>"))
.field(
"certificate_v2_len",
&self.certificate_v2.as_ref().map(|c| c.len()),
)
.finish()
}
}
impl std::fmt::Debug for Vid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vid(<redacted>)")
}
}
impl std::fmt::Debug for MediaKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("MediaKey(<redacted>)")
}
}
impl std::fmt::Debug for Vuk {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vuk(<redacted>)")
}
}
impl std::fmt::Debug for ProcessingKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("ProcessingKey(<redacted>)")
}
}
impl std::fmt::Debug for UnitKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKey")
.field("idx", &self.idx)
.field("key", &"<redacted>")
.field("index_number", &self.index_number)
.finish()
}
}
impl std::fmt::Debug for DiscEntry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscEntry")
.field("disc_hash", &self.disc_hash)
.field("title", &self.title)
.field("media_key", &self.media_key.map(|_| "<redacted>"))
.field("disc_id", &self.disc_id.map(|_| "<redacted>"))
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.finish()
}
}
#[cfg(test)]
mod unit_key_tests {
use super::*;
/// `is_default_index` is the public predicate that separates ordinary
/// (index-0) content keys from FMTS forensic index keys ([`UnitKey`] docs;
/// AACS 2.1 `IndividualSegment.tbl` tagging). A body answering `true` for
/// everything would present a forensic index key as an ordinary content
/// key — the caller would decrypt the bulk of the title with a key that
/// only opens 1/32nd of it; answering `false` for everything would hide
/// every ordinary key.
///
/// Pinned against the two NAMED constructors, which are the contract:
/// [`UnitKey::new`] builds the ordinary key, [`UnitKey::forensic`] builds
/// an index key for `1..=32`.
#[test]
fn is_default_index_separates_the_two_constructors() {
let ordinary = UnitKey::new(0, [0xAA; 16]);
assert!(
ordinary.is_default_index(),
"UnitKey::new builds the ordinary (index-0) key"
);
// Every forensic index the spec allows must be reported as NOT default.
for n in 1u8..=32 {
let k = UnitKey::forensic(0, [0xAA; 16], n);
assert!(
!k.is_default_index(),
"UnitKey::forensic({n}) is an index key, not the default key"
);
}
}
/// The predicate must agree with the one consumer of `index_number` in the
/// crate: [`crate::aacs::index_select::resolve_disc_index`] resolves the
/// disc's forensic index from exactly the keys that are NOT default. If
/// the two disagree, a disc resolves an index whose key the rest of the
/// pipeline treats as ordinary (or vice versa).
#[test]
fn is_default_index_agrees_with_the_forensic_index_resolver() {
use crate::aacs::index_select::resolve_disc_index;
let keys = [
UnitKey::new(0, [0x11; 16]),
UnitKey::forensic(1, [0x22; 16], 7),
];
assert_eq!(
resolve_disc_index(&keys),
Some(7),
"sanity: the resolver picks the forensic key's index"
);
let non_default: Vec<u8> = keys
.iter()
.filter(|k| !k.is_default_index())
.map(|k| k.index_number)
.collect();
assert_eq!(
non_default,
vec![7],
"exactly the key the resolver picked must be non-default"
);
// An all-ordinary key set resolves no index, and every key must report
// itself default.
let plain = [UnitKey::new(0, [0x11; 16]), UnitKey::new(1, [0x22; 16])];
assert_eq!(resolve_disc_index(&plain), None);
assert!(plain.iter().all(|k| k.is_default_index()));
}
}
#[cfg(test)]
mod redaction_tests {
use super::*;
// Sentinel key byte 0xD5 = decimal 213. A derived `Debug` prints `[u8;N]`
// as decimal, so a leaked key surfaces the substring "213"; the redacting
// impls must not. No non-secret field below is 213, so "213" appearing means
// key bytes leaked. Each type must also carry a "redacted" marker (or omit
// the secret entirely) so re-adding `#[derive(Debug)]` fails this test.
const S: u8 = 0xD5;
fn assert_redacted(what: &str, dbg: &str) {
assert!(
!dbg.contains("213"),
"{what}: Debug leaked key bytes (found decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"{what}: Debug missing redaction marker: {dbg}"
);
}
#[test]
fn device_key_debug_is_redacted() {
let d = DeviceKey {
key: [S; 16],
node: 1,
uv: 2,
u_mask_shift: 3,
};
assert_redacted("DeviceKey", &format!("{d:?}"));
}
#[test]
fn host_cert_debug_is_redacted() {
let h = HostCert {
private_key: [S; 20],
certificate: vec![0u8; 92],
private_key_v2: Some([S; 32]),
certificate_v2: None,
};
assert_redacted("HostCert", &format!("{h:?}"));
}
#[test]
fn newtype_keys_debug_is_redacted() {
assert_redacted("Vid", &format!("{:?}", Vid([S; 16])));
assert_redacted("MediaKey", &format!("{:?}", MediaKey([S; 16])));
assert_redacted("Vuk", &format!("{:?}", Vuk([S; 16])));
assert_redacted("ProcessingKey", &format!("{:?}", ProcessingKey([S; 16])));
}
#[test]
fn unit_key_debug_is_redacted() {
assert_redacted("UnitKey", &format!("{:?}", UnitKey::new(0, [S; 16])));
}
#[test]
fn disc_entry_debug_is_redacted() {
let e = DiscEntry {
disc_hash: "0xAA".into(),
title: "T".into(),
media_key: Some([S; 16]),
disc_id: Some([S; 16]),
vuk: Some([S; 16]),
unit_keys: vec![(1, [S; 16])],
};
assert_redacted("DiscEntry", &format!("{e:?}"));
}
}
+2137
View File
File diff suppressed because it is too large Load Diff
-1045
View File
File diff suppressed because it is too large Load Diff
+102 -841
View File
File diff suppressed because it is too large Load Diff
+13 -1
View File
@@ -66,6 +66,10 @@ pub mod coding_type {
pub const MPEG2_VIDEO: u8 = 0x02;
/// H.264 / AVC video (ISO/IEC 13818-1 Table 2-34).
pub const H264: u8 = 0x1B;
/// H.264 / MVC dependent view (Blu-ray 3D right-eye substream). Carried in
/// the SSIF interleaved stream under its own PID; the base view is [`H264`].
/// ISO/IEC 13818-1 stream_type 0x20 (MVC video sub-bitstream).
pub const H264_MVC: u8 = 0x20;
/// HEVC / H.265 video (ISO/IEC 13818-1 Table 2-34, 2015 amendment).
pub const HEVC: u8 = 0x24;
/// SMPTE VC-1 video (BD-ROM convention, ISO user-private range).
@@ -95,7 +99,11 @@ pub mod coding_type {
/// Secondary Dolby Digital Plus audio (BD-ROM convention).
pub const AC3_PLUS_SECONDARY: u8 = 0xA1;
/// Secondary DTS-HD audio (lossless MA, not lossy HR) (BD-ROM convention).
/// Secondary DTS-HD audio — DTS Express / DTS-HD LBR, a LOSSY low-bitrate
/// stream for picture-in-picture and BD-J mixing (BD-ROM Part 3
/// `stream_coding_type` table). The lossless primary is [`DTS_HD_MA`]
/// (0x86); this code is its lossy secondary counterpart, parallel to
/// [`AC3_PLUS_SECONDARY`] (0xA1) on the Dolby side.
pub const DTS_HD_SECONDARY: u8 = 0xA2;
}
@@ -106,6 +114,10 @@ pub mod coding_type {
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// system_header start code — the MPEG-PS `00 00 01 BB` structural header
/// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it
/// follows the pack header, so it lands at sector offset 0x11.
pub const SYSTEM_HEADER: u8 = 0xBB;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
+163 -134
View File
@@ -1,58 +1,46 @@
//! CSS cipher implementation based on the Stevenson 1999 analysis.
//! CSS content cipher — an independent implementation of the publicly
//! documented Content Scramble System stream cipher.
//!
//! The CSS cipher uses two table-driven feedback circuits:
//! - LFSR1: 17-bit state (9-bit lo + 8-bit hi register, seeded from
//! key[0..2]), driven by TAB2/TAB3
//! - LFSR0: 24-bit feedback register (seeded from key[2..5] XOR seed[2..5],
//! masked to 0xFFFFFF), driven by a feedback polynomial through TAB4
//! The algorithm is the one recovered and published in Frank A. Stevenson's
//! 1999 cryptanalysis ("Cryptanalysis of Contents Scrambling System") and
//! described in the open CSS literature. It is implemented here from that public
//! description; its constants (see [`super::tables`]) are the cipher's own
//! defined values. Nothing in this file is copied or translated from any
//! particular CSS software.
//!
//! The keystream is the bytewise sum (with carry) of both LFSR outputs.
//! Content descrambling computes plain = TAB1[cipher] ^ keystream — a TAB1
//! substitution of each ciphertext byte followed by an XOR with the keystream
//! (NOT a plain XOR; the cipher is not its own inverse).
//! The cipher uses two table-driven linear-feedback circuits:
//! - **LFSR1** — a 17-bit register (a 9-bit and an 8-bit half seeded from
//! `key[0..2] XOR seed[0..2]`), stepped through `TAB2`/`TAB3`/`TAB5`.
//! - **LFSR0** — a 24-bit feedback register (seeded from `key[2..5] XOR
//! seed[2..5]`), stepped through a feedback polynomial and `TAB4`.
//!
//! Algorithm: Frank A. Stevenson's divide-and-conquer attack (1999).
//! Tables: CSS specification constants.
//! Each output byte is the sum-with-carry of the two register outputs. A body
//! byte is recovered as `plain = TAB1[cipher] ^ keystream` — a `TAB1`
//! substitution of the ciphertext byte followed by an XOR with the keystream
//! (so the cipher is deliberately not its own inverse).
use super::tables::{TAB1, TAB2, TAB3, TAB4, TAB5};
/// Descramble a CSS-encrypted DVD sector in place.
///
/// Exact port of libdvdcss `dvdcss_unscramble` (css.c). The two content
/// LFSRs are seeded **directly** from `title_key XOR sector_seed` — there is
/// no `decrypt_key` mangling on this path (that is the disc/title-key
/// hierarchy, not the content cipher). Bytes 0x80..0x800 are recovered with
/// `*p = TAB1[*p] ^ (i_t5 & 0xff)`.
/// The two feedback registers are seeded **directly** from
/// `title_key XOR sector_seed` (bytes `0x54..0x59`) — there is no title-key
/// mangling on the content path (that belongs to the disc/title-key hierarchy,
/// not the sector cipher). Only the body, bytes `0x80..0x800`, is transformed:
/// `body[i] = TAB1[body[i]] ^ (keystream & 0xff)`.
///
/// The scramble flag at byte 0x14 (bits 4-5) indicates encryption. Like
/// libdvdcss, the flag byte is NOT modified here — the caller treats a
/// nonzero `sector[0x14] & 0x30` as "needs unscrambling" and the descramble
/// is its own inverse, so re-running it on plaintext would re-scramble.
/// (freemkv historically cleared the flag; we keep clearing it so callers
/// and the existing tests can distinguish a descrambled sector. This does
/// not affect the recovered body.)
/// The scramble flag at byte `0x14` (bits 4-5) marks an encrypted sector. This
/// routine CLEARS that flag after unscrambling, so a descrambled sector reads as
/// `sector[0x14] & 0x30 == 0`; callers and tests use that to tell it from
/// ciphertext, and re-running descramble on an already-cleared sector is a no-op
/// (the flag guard below skips it). Clearing does not affect the recovered body.
///
/// No-op (returns without modifying `sector`) in two cases:
/// - `sector.len() < 2048`: the encrypted region (0x80..0x800) is not
/// fully present. Callers chunk by 2048, so a trailing partial chunk is
/// left untouched. The `debug_assert!` flags this misuse in debug/test
/// builds; a DVD sector is always exactly 2048 bytes.
/// - `sector.len() < 2048`: the encrypted region (`0x80..0x800`) is not fully
/// present. Callers chunk by 2048, so a trailing partial chunk is left
/// untouched. The `debug_assert!` flags this misuse in debug/test builds; a
/// DVD sector is always exactly 2048 bytes.
/// - scramble flags are zero: the sector is not CSS-encrypted.
///
/// Design reference: libdvdcss `dvdcss_unscramble`. The combiner mirrors
/// `css.c` line-for-line:
/// ```text
/// i_t1 = (key[0] ^ sec[0x54]) | 0x100;
/// i_t2 = key[1] ^ sec[0x55];
/// i_t3 = (key[2]|key[3]<<8|key[4]<<16) ^ (sec[0x56]|sec[0x57]<<8|sec[0x58]<<16);
/// i_t4 = i_t3 & 7; i_t3 = i_t3*2 + 8 - i_t4;
/// // per byte over 0x80..0x800:
/// i_t4 = TAB2[i_t2] ^ TAB3[i_t1];
/// i_t2 = i_t1 >> 1; i_t1 = ((i_t1 & 1) << 8) ^ i_t4; i_t4 = TAB5[i_t4];
/// i_t6 = (((((((i_t3>>3)^i_t3)>>1)^i_t3)>>8)^i_t3)>>5) & 0xff;
/// i_t3 = (i_t3 << 8) | i_t6; i_t6 = TAB4[i_t6];
/// i_t5 += i_t6 + i_t4; *p = TAB1[*p] ^ (i_t5 & 0xff); i_t5 >>= 8;
/// ```
pub fn descramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
debug_assert!(
sector.len() >= 2048,
@@ -62,102 +50,103 @@ pub fn descramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
return;
}
// libdvdcss: `if( !(p_sec[0x14] & 0x30) ) return;`
// Not scrambled (flag bits 4-5 clear) → nothing to do.
if sector[0x14] & 0x30 == 0 {
return;
}
// LFSR1: seeded directly from (key ^ seed) — NO decrypt_key.
let mut i_t1: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut i_t2: u32 = (title_key[1] ^ sector[0x55]) as u32;
// LFSR1 halves, seeded from (key ^ seed) bytes 0-1. The 9-bit half carries a
// set bit 8 (`| 0x100`) as its running marker.
let mut r1a: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut r1b: u32 = (title_key[1] ^ sector[0x55]) as u32;
// LFSR0 (i_t3): 24-bit feedback register seeded from the remaining three
// key/seed bytes, then transformed `i_t3 = i_t3*2 + 8 - (i_t3 & 7)`.
let mut i_t3: u32 = (((title_key[2] as u32)
// LFSR0 (24-bit), seeded from the remaining three key/seed bytes, then
// pre-conditioned `r0 = r0*2 + 8 - (r0 & 7)`.
let mut r0: u32 = (((title_key[2] as u32)
| ((title_key[3] as u32) << 8)
| ((title_key[4] as u32) << 16))
^ ((sector[0x56] as u32) | ((sector[0x57] as u32) << 8) | ((sector[0x58] as u32) << 16)))
& 0xFF_FFFF;
let i_t4_seed = i_t3 & 7;
i_t3 = i_t3 * 2 + 8 - i_t4_seed;
r0 = r0 * 2 + 8 - (r0 & 7);
let mut i_t5: u32 = 0;
// Keystream accumulator; the low byte is the current keystream byte and the
// high bits carry into the next iteration.
let mut acc: u32 = 0;
for byte in sector.iter_mut().take(2048).skip(128) {
// Advance LFSR1.
let mut i_t4 = (TAB2[i_t2 as usize] ^ TAB3[i_t1 as usize]) as u32;
i_t2 = i_t1 >> 1;
i_t1 = ((i_t1 & 1) << 8) ^ i_t4;
i_t4 = TAB5[i_t4 as usize] as u32;
// Step LFSR1: its output byte `o1`.
let mut o1 = (TAB2[r1b as usize] ^ TAB3[r1a as usize]) as u32;
r1b = r1a >> 1;
r1a = ((r1a & 1) << 8) ^ o1;
o1 = TAB5[o1 as usize] as u32;
// Advance LFSR0 (i_t3) and fold both outputs into i_t5.
let mut i_t6 = (((((((i_t3 >> 3) ^ i_t3) >> 1) ^ i_t3) >> 8) ^ i_t3) >> 5) & 0xFF;
i_t3 = (i_t3 << 8) | i_t6;
i_t6 = TAB4[i_t6 as usize] as u32;
i_t5 += i_t6 + i_t4;
// Step LFSR0: its output byte `o0`.
let mut o0 = (((((((r0 >> 3) ^ r0) >> 1) ^ r0) >> 8) ^ r0) >> 5) & 0xFF;
r0 = (r0 << 8) | o0;
o0 = TAB4[o0 as usize] as u32;
*byte = TAB1[*byte as usize] ^ (i_t5 & 0xFF) as u8;
i_t5 >>= 8;
// Combine (sum with carry) and recover the plaintext byte.
acc += o0 + o1;
*byte = TAB1[*byte as usize] ^ (acc & 0xFF) as u8;
acc >>= 8;
}
// libdvdcss leaves byte 0x14 untouched; freemkv clears the scramble bits
// so downstream code and tests can tell a sector was descrambled.
// Clear the scramble bits so downstream code and tests can tell a sector was
// descrambled; bits 6-7 of byte 0x14 are preserved.
sector[0x14] &= 0xCF;
}
/// Exact inverse of [`descramble_sector`]: turn a plaintext sector body into
/// CSS ciphertext under `title_key`.
///
/// Descramble computes `plain = TAB1[cipher] ^ (i_t5 & 0xff)`, so the
/// inverse is `cipher = TAB1_INV[plain ^ (i_t5 & 0xff)]` with the identical
/// LFSR keystream. The keystream derivation is byte-for-byte the same as
/// `descramble_sector` (libdvdcss `dvdcss_unscramble`); only the final
/// substitution differs. Bytes 0x80..0x800 are rewritten in place; the
/// scramble flag is set to 0x10 so a subsequent descramble runs.
/// Descramble computes `plain = TAB1[cipher] ^ (keystream & 0xff)`, so the
/// inverse is `cipher = TAB1_INV[plain ^ (keystream & 0xff)]` with the identical
/// keystream. The keystream derivation is the same as [`descramble_sector`];
/// only the final substitution differs. Bytes `0x80..0x800` are rewritten in
/// place; the scramble flag is set to `0x10` so a subsequent descramble runs.
///
/// Not on any production read path — it exists so the key-recovery tests
/// (and any caller that needs to produce a known CSS-encrypted sector) can
/// build genuine ciphertext rather than approximating it.
/// Not on any production read path — it exists so the key-recovery tests (and
/// any caller that needs a known CSS-encrypted sector) can build genuine
/// ciphertext rather than approximating it.
#[cfg(test)]
pub(crate) fn scramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
if sector.len() < 2048 {
return;
}
let mut i_t1: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut i_t2: u32 = (title_key[1] ^ sector[0x55]) as u32;
let mut i_t3: u32 = (((title_key[2] as u32)
let mut r1a: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut r1b: u32 = (title_key[1] ^ sector[0x55]) as u32;
let mut r0: u32 = (((title_key[2] as u32)
| ((title_key[3] as u32) << 8)
| ((title_key[4] as u32) << 16))
^ ((sector[0x56] as u32) | ((sector[0x57] as u32) << 8) | ((sector[0x58] as u32) << 16)))
& 0xFF_FFFF;
let i_t4_seed = i_t3 & 7;
i_t3 = i_t3 * 2 + 8 - i_t4_seed;
r0 = r0 * 2 + 8 - (r0 & 7);
let mut i_t5: u32 = 0;
let mut acc: u32 = 0;
for byte in sector.iter_mut().take(2048).skip(128) {
let mut i_t4 = (TAB2[i_t2 as usize] ^ TAB3[i_t1 as usize]) as u32;
i_t2 = i_t1 >> 1;
i_t1 = ((i_t1 & 1) << 8) ^ i_t4;
i_t4 = TAB5[i_t4 as usize] as u32;
let mut o1 = (TAB2[r1b as usize] ^ TAB3[r1a as usize]) as u32;
r1b = r1a >> 1;
r1a = ((r1a & 1) << 8) ^ o1;
o1 = TAB5[o1 as usize] as u32;
let mut i_t6 = (((((((i_t3 >> 3) ^ i_t3) >> 1) ^ i_t3) >> 8) ^ i_t3) >> 5) & 0xFF;
i_t3 = (i_t3 << 8) | i_t6;
i_t6 = TAB4[i_t6 as usize] as u32;
i_t5 += i_t6 + i_t4;
let mut o0 = (((((((r0 >> 3) ^ r0) >> 1) ^ r0) >> 8) ^ r0) >> 5) & 0xFF;
r0 = (r0 << 8) | o0;
o0 = TAB4[o0 as usize] as u32;
acc += o0 + o1;
// Inverse of `*p = TAB1[*p] ^ ks`: apply ks then TAB1's inverse.
*byte = (*TAB1_INV)[(*byte ^ (i_t5 & 0xFF) as u8) as usize];
i_t5 >>= 8;
*byte = (*TAB1_INV)[(*byte ^ (acc & 0xFF) as u8) as usize];
acc >>= 8;
}
// Mark the sector scrambled so the descrambler will process it.
sector[0x14] = (sector[0x14] & 0xCF) | 0x10;
}
/// Inverse permutation of [`TAB1`], built at first use. `TAB1` is a
/// bijection on 0..256, so `TAB1_INV[TAB1[x]] == x`.
/// Inverse permutation of [`TAB1`], built at first use. `TAB1` is a bijection on
/// `0..256`, so `TAB1_INV[TAB1[x]] == x`.
#[cfg(test)]
static TAB1_INV: std::sync::LazyLock<[u8; 256]> = std::sync::LazyLock::new(|| {
let mut inv = [0u8; 256];
@@ -181,14 +170,15 @@ mod tests {
assert_eq!(sector, original);
}
/// Cross-check `descramble_sector` against the EXACT output of libdvdcss
/// `dvdcss_unscramble` (css.c) for a fixed sector, computed from the
/// reference C semantics with the reference tables. Pins the content
/// cipher to libdvdcss byte-for-byte.
/// Regression vector: the deterministic output of the CSS content cipher for
/// a fixed key/seed/body. The value is generated by this implementation and
/// is self-consistent with the scramble/descramble round-trip below — any
/// correct CSS descrambler yields the same bytes, since the cipher is
/// deterministic. Pins the implementation against accidental change.
///
/// key = 42 13 37 BE EF, seed (0x54..0x59) = DE AD BE EF 42, body = 0xAA.
#[test]
fn descramble_matches_libdvdcss_unscramble_vector() {
fn descramble_produces_the_reference_css_vector() {
let key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let mut sector = vec![0xAAu8; 2048];
sector[0x14] = 0x30;
@@ -200,12 +190,12 @@ mod tests {
0x81, 0x92, 0x24, 0xA2, 0x46, 0x70, 0x3C, 0x64, 0xA6, 0x91, 0x84, 0xF5, 0x1F, 0x98,
0xA0, 0x31
],
"descramble body head must match libdvdcss dvdcss_unscramble"
"descramble body head must match the reference CSS vector"
);
assert_eq!(
&sector[0x7F8..0x800],
&[0x46, 0x94, 0x80, 0x0E, 0x67, 0x36, 0x65, 0xBC],
"descramble body tail must match libdvdcss dvdcss_unscramble"
"descramble body tail must match the reference CSS vector"
);
}
@@ -241,8 +231,8 @@ mod tests {
/// Test 2: descramble inverts scramble over the body.
///
/// The content cipher is NOT a plain XOR involution (it applies TAB1 to
/// the ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
/// The content cipher is NOT a plain XOR involution (it applies TAB1 to the
/// ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
/// [`scramble_sector`]. Scrambling a plaintext body and then descrambling
/// with the same key must reproduce the original body exactly.
#[test]
@@ -279,9 +269,9 @@ mod tests {
/// css_tab1_relationship
///
/// Verify the structure of TAB1: it is a substitution table used in
/// key mangling. Check that no two inputs map to the same output
/// (TAB1 is a permutation of 0..255).
/// Verify the structure of TAB1: it is a substitution table used in key
/// mangling. Check that no two inputs map to the same output (TAB1 is a
/// permutation of 0..255).
#[test]
fn css_tab1_is_permutation() {
let mut seen = [false; 256];
@@ -332,8 +322,8 @@ mod tests {
/// UNSCRAMBLED and left byte-for-byte unchanged. This guards against a
/// too-wide mask silently "descrambling" (and thus corrupting) clear data.
///
/// Grounding: CSS sector header byte 0x14 — copyright/scramble bits live
/// in bits 4-5; the masked value 0 means not scrambled.
/// Grounding: CSS sector header byte 0x14 — copyright/scramble bits live in
/// bits 4-5; the masked value 0 means not scrambled.
/// Mutation: widen the mask `0x30` to `0x70`/`0xF0` -> 0x40/0x80 would be
/// seen as scrambled and the body would change.
#[test]
@@ -352,11 +342,10 @@ mod tests {
}
}
/// Each individual scramble bit (4 and 5) independently marks the sector
/// as encrypted: 0x10 and 0x20 must both trigger descrambling.
/// Each individual scramble bit (4 and 5) independently marks the sector as
/// encrypted: 0x10 and 0x20 must both trigger descrambling.
///
/// Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both
/// nonzero.
/// Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both nonzero.
/// Mutation: change `!= 0` early-return condition to `== 3` -> a sector
/// flagged only 0x10 or 0x20 would be skipped and left scrambled.
#[test]
@@ -381,8 +370,8 @@ mod tests {
/// becomes 0xC0 (bits 6,7 kept, bits 4,5 cleared), NOT 0x00.
///
/// Grounding: code does `sector[0x14] &= 0xCF`; 0xF0 & 0xCF == 0xC0.
/// Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved
/// high bits assert fails.
/// Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved high
/// bits assert fails.
#[test]
fn descramble_clear_preserves_high_bits_of_0x14() {
let key = [0x01, 0x02, 0x03, 0x04, 0x05];
@@ -398,11 +387,11 @@ mod tests {
// ── header / body boundary (encrypted region is 0x80..0x800) ───────────
/// The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80
/// (the header) must NOT be modified by the keystream — except byte 0x14
/// whose flag is cleared. In particular the sector-seed bytes 0x54..0x59
/// (which live inside the header) must survive untouched, since the
/// descrambler reads them but never writes them.
/// The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80 (the
/// header) must NOT be modified by the keystream — except byte 0x14 whose
/// flag is cleared. In particular the sector-seed bytes 0x54..0x59 (which
/// live inside the header) must survive untouched, since the descrambler
/// reads them but never writes them.
///
/// Grounding: loop is `sector.iter_mut().take(2048).skip(128)` -> indices
/// 128..2048 only.
@@ -429,16 +418,15 @@ mod tests {
assert_eq!(&sector[0x54..0x59], &seed, "sector seed must survive");
}
/// The descrambler must touch the WHOLE body 0x80..0x800, not just a
/// prefix. With a constant body and constant key, the keystream is
/// non-degenerate enough that the very last sector byte (index 2047) is
/// altered. This guards the loop bound `.take(2048)` against an
/// off-by-one that would leave the final byte(s) scrambled.
/// The descrambler must touch the WHOLE body 0x80..0x800, not just a prefix.
/// With a constant body and constant key, the keystream is non-degenerate
/// enough that the very last sector byte (index 2047) is altered. This guards
/// the loop bound `.take(2048)` against an off-by-one that would leave the
/// final byte(s) scrambled.
///
/// Grounding: encrypted region end is 0x800 == 2048 (exclusive).
/// Mutation: change `.take(2048)` to `.take(2047)` -> last byte unchanged,
/// assert fires (keystream byte for the last position is verified nonzero
/// below by the round-trip, and this body is all-zero so any XOR shows).
/// assert fires (this body is all-zero so any keystream XOR shows).
#[test]
fn descramble_covers_final_body_byte() {
let key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
@@ -447,9 +435,7 @@ mod tests {
sector[0x54..0x59].copy_from_slice(&[0x11, 0x22, 0x33, 0x44, 0x55]);
descramble_sector(&key, &mut sector);
// Body was all zero; any nonzero in [0x80,0x800) is keystream. Confirm
// the keystream reaches the final byte. (If the last keystream byte
// happened to be 0 this could be a flaky test, so assert the run-end
// region as a whole differs from zero.)
// the keystream reaches the final byte.
assert_ne!(
&sector[2040..2048],
&[0u8; 8][..],
@@ -457,14 +443,57 @@ mod tests {
);
}
/// The length guard is a FLOOR, not a ceiling: `descramble_sector` is a
/// no-op below one sector, and processes the FIRST sector of anything at
/// least that long (the loop is `.take(2048)`). `css::descramble_sector` is
/// a public entry taking `&mut [u8]` of any length, so a caller handing it a
/// multi-sector buffer must get its first sector descrambled — a guard that
/// rejected over-long buffers would hand that caller its ciphertext back
/// unchanged, with the scramble flag cleared as if it had worked.
#[test]
fn descramble_processes_the_first_sector_of_an_over_long_buffer() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
// Two sectors' worth of buffer; only the first is a sector.
let mut buf = vec![0xAAu8; 4096];
buf[0x14] = 0x30;
buf[0x54..0x59].copy_from_slice(&seed);
let original = buf.clone();
descramble_sector(&title_key, &mut buf);
assert_ne!(
&buf[0x80..0x800],
&original[0x80..0x800],
"the first sector's body must be descrambled"
);
assert_eq!(buf[0x14] & 0x30, 0x00, "and its scramble flag cleared");
assert_eq!(
&buf[2048..4096],
&original[2048..4096],
"bytes past the first sector must be left untouched"
);
// The result must equal what a caller gets by passing exactly one
// sector — the same transform, not a length-dependent one.
let mut one = original[..2048].to_vec();
descramble_sector(&title_key, &mut one);
assert_eq!(
&buf[..2048],
&one[..],
"the first sector must descramble identically either way"
);
}
/// Descramble is keyed by `title_key XOR seed`: two different title keys
/// produce two different bodies for the same scrambled input. A cipher
/// that ignored the title key (or mixed it in wrongly) would yield
/// identical output — silent wrong-key decryption.
/// produce two different bodies for the same scrambled input. A cipher that
/// ignored the title key (or mixed it in wrongly) would yield identical
/// output — silent wrong-key decryption.
///
/// Grounding: per-sector key = title_key[i] ^ sector[0x54+i].
/// Mutation: in the `key` array drop the `title_key[i] ^` term -> both
/// keys give the same body, assert fires.
/// Mutation: in the `key` array drop the `title_key[i] ^` term -> both keys
/// give the same body, assert fires.
#[test]
fn descramble_output_depends_on_title_key() {
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
+1020 -83
View File
File diff suppressed because it is too large Load Diff
+608 -61
View File
@@ -1,48 +1,37 @@
//! CSS title-key recovery — Frank A. Stevenson's divide-and-conquer attack
//! (1999), ported exactly from libdvdcss `RecoverTitleKey` + `AttackPattern`
//! (css.c).
//!
//! Recovers the 5-byte CSS title key from a single scrambled DVD sector with
//! no player keys and no disc-key crack, using only known plaintext.
//! (1999), implemented from his published cryptanalysis ("Cryptanalysis of
//! Contents Scrambling System"). It recovers the 5-byte CSS title key from a
//! single scrambled DVD sector with no player keys and no disc-key crack, using
//! only known plaintext. Implemented from that public description; nothing here
//! is copied or translated from any particular CSS software.
//!
//! # The cipher this attacks
//!
//! The content descrambler ([`super::lfsr::descramble_sector`], = libdvdcss
//! `dvdcss_unscramble`) seeds its two LFSRs **directly** from
//! `key = title_key XOR sector_seed` (seed = `sector[0x54..0x59]`):
//!
//! ```text
//! i_t1 = (key[0] ^ sec[0x54]) | 0x100; // LFSR1 low (9-bit)
//! i_t2 = key[1] ^ sec[0x55]; // LFSR1 high
//! i_t3 = (key[2]|key[3]<<8|key[4]<<16) ^ seed3; // LFSR0 (24-bit feedback)
//! i_t3 = i_t3*2 + 8 - (i_t3 & 7);
//! // per byte: *p = TAB1[*p] ^ (i_t5 & 0xff)
//! ```
//!
//! There is NO `decrypt_key` mangling on the content path. So the recovery
//! is a single inversion of `dvdcss_unscramble`, not the multi-stage
//! working-key inversion the previous (non-CSS) implementation used.
//! The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
//! LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
//! `sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
//! bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
//! byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
//! title-key mangling on the content path, so the recovery is a single inversion
//! of the sector cipher.
//!
//! # The attack
//!
//! 1. **Known plaintext → keystream.** Because the descramble applies TAB1
//! to the ciphertext, the per-byte keystream is
//! `buf[i] = TAB1[cipher[i]] ^ plain[i]` (matching libdvdcss
//! `RecoverTitleKey`'s `p_buffer`).
//! 1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
//! ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
//! 2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
//! forward; for the first four steps deduce the LFSR0 output bytes from
//! the keystream (carry-tracked), reconstructing `i_t3`. For the next six
//! steps clock LFSR0 normally and check it reproduces the keystream — a
//! wrong LFSR1 seed fails fast.
//! 3. **Back-clock LFSR0.** Run four backward `i_t3` steps (each a 256-way
//! search for the byte shifted in) to reach the initial state, then undo
//! `i_t3 = i_t3*2 + 8 - (i_t3 & 7)` to recover key[2..5].
//! 4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]` (plain XOR —
//! the descramble seeds directly, so there is no inversion).
//! forward; for the first four steps deduce the LFSR0 output bytes from the
//! keystream (carry-tracked), reconstructing LFSR0's state. For the next six
//! steps clock LFSR0 normally and check it reproduces the keystream — a wrong
//! LFSR1 seed fails fast.
//! 3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
//! the byte shifted in) to reach the initial state, then undo the
//! `r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
//! 4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
//!
//! `AttackPattern` finds known plaintext for step 1: the longest periodic
//! run in the cleartext `sec[0x00..0x80]`, assumed to continue into the
//! encrypted region at 0x80.
//! Known plaintext for step 1 comes from the longest periodic run in the
//! cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
//! 0x80.
use super::lfsr::descramble_sector;
use super::tables::{TAB1, TAB2, TAB3, TAB4, TAB5};
@@ -52,13 +41,11 @@ const ENCRYPTED_START: usize = 0x80; // byte 128
const SEED_OFFSET: usize = 0x54; // sector seed at bytes 0x54-0x58
const FLAG_BYTE: usize = 0x14;
/// RecoverTitleKey: recover the title key from cipher + known plaintext.
///
/// Exact port of libdvdcss `RecoverTitleKey` (css.c). `crypted` is the
/// ciphertext starting at sector byte 0x80; `decrypted` is the matching
/// known plaintext; `seed` is `sector[0x54..0x59]`. On success returns the
/// recovered 5-byte title key; `None` if no LFSR seed reproduces the
/// keystream.
/// Recover the title key from cipher + known plaintext (the core of Stevenson's
/// attack). `crypted` is the ciphertext starting at sector byte 0x80;
/// `decrypted` is the matching known plaintext; `seed` is `sector[0x54..0x59]`.
/// On success returns the recovered 5-byte title key; `None` if no LFSR seed
/// reproduces the keystream.
///
/// At least 10 bytes of `crypted`/`decrypted` are required (the cipher is
/// iterated 10 times: 4 to reconstruct LFSR0, 6 to validate).
@@ -222,16 +209,13 @@ fn descramble_matches(sector: &[u8], title: &[u8; 5], plain: &[u8]) -> bool {
test[ENCRYPTED_START..ENCRYPTED_START + n] == plain[..n]
}
/// AttackPattern: find a repeating pattern just before the encrypted region
/// and assume the plaintext at 0x80 continues it.
///
/// Functionally-equivalent port of libdvdcss `AttackPattern` (css.c) — finds the
/// same periodic cribs on real DVD data, though its byte-comparison anchor
/// differs from the C on phase-misaligned runs. Scans cleartext
/// `sec[0x00..0x80]` for the longest run that repeats with a cycle length in
/// 2..0x2F. If the run is long enough (`plen > 3` and at least two full
/// cycles), the known plaintext at 0x80 is taken to be the periodic run
/// continuing forward, and [`recover_title_key_from_plain`] is applied.
/// Find a repeating pattern just before the encrypted region and assume the
/// plaintext at 0x80 continues it — the known-plaintext step of Stevenson's
/// attack. Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// with a cycle length in 2..0x2F. If the run is long enough (`plen > 3` and at
/// least two full cycles), the known plaintext at 0x80 is taken to be the
/// periodic run continuing forward, and [`recover_title_key_from_plain`] is
/// applied.
pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
if sector.len() < SECTOR_BYTES {
return None;
@@ -260,10 +244,7 @@ pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
result
}
/// Inner body of [`crack_title_key`] — the actual AttackPattern search. Split
/// out so the public entry point can wall-clock the whole attempt for the
/// runaway guard without threading a timer through every return path.
/// AttackPattern crib: the predicted 10-byte plaintext at byte 0x80.
/// Crib: the predicted 10-byte plaintext at byte 0x80.
///
/// Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
/// run that repeats with a cycle length in 2..0x2F. If the run is long enough
@@ -369,7 +350,7 @@ mod tests {
/// Build a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends
/// in a periodic run that continues into the encrypted region — the case
/// `AttackPattern` (crack_title_key) is designed to crack.
/// `crack_title_key` is designed to crack.
fn synth_periodic_sector(
title_key: &[u8; 5],
seed: &[u8; 5],
@@ -382,7 +363,7 @@ mod tests {
// (RUN_START..0x80) and continuing into the encrypted region. This
// mirrors a real VOB: a periodic data run just before the scrambled
// part. The run must NOT overlap the seed bytes (0x54..0x59), or the
// AttackPattern detector would break mid-run. The phase is anchored to
// the crib detector would break mid-run. The phase is anchored to
// offset 0 so the run is consistent across the 0x80 boundary.
// Just above the seed (0x54..0x59); gives a 39-byte run (0x59..0x80)
// — enough for >=2 cycles of every tested period (<=19).
@@ -469,7 +450,65 @@ mod tests {
}
}
/// MANDATORY (Task C.1): the AttackPattern entry point crack_title_key
/// `descramble_matches` is the ONLY gate between the LFSR search and a key
/// handed back to the caller: both [`recover_title_key`] and the crib-driven
/// `crack_title_key_inner` return a candidate only if this says the key
/// really descrambles the sector to the known plaintext. A body that always
/// answered `true` would let the first spurious LFSR-seed match through as
/// the title key — the ripper would then descramble the whole title with a
/// key that opens nothing, producing garbage rather than a "no key" error.
///
/// Pinned both directions: the genuine key is accepted, and EVERY key one
/// bit away from it is rejected. The one-bit neighbours are the strongest
/// form of wrong key — a gate that only rejects wildly different keys would
/// still pass a near-miss out of the 2^16 seed search.
#[test]
fn descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
assert!(
descramble_matches(&sector, &title_key, &PES),
"the key the sector was scrambled with must be accepted"
);
for byte in 0..5usize {
for bit in 0..8u32 {
let mut wrong = title_key;
wrong[byte] ^= 1u8 << bit;
assert!(
!descramble_matches(&sector, &wrong, &PES),
"key differing only in byte {byte} bit {bit} must be rejected"
);
}
}
}
/// The gate is applied to a COPY: verifying a candidate must not modify the
/// caller's sector. `recover_title_key` runs the gate and then hands the
/// sector on to be descrambled for real — if verification descrambled in
/// place, that second descramble would run over already-transformed bytes
/// (and, worse, a rejected candidate would leave the sector corrupted).
#[test]
fn descramble_matches_does_not_disturb_the_caller_s_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
let before = sector.clone();
assert!(descramble_matches(&sector, &title_key, &PES));
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
assert_eq!(
sector, before,
"verification must leave the sector byte-for-byte unchanged"
);
}
/// MANDATORY (Task C.1): the crib-based entry point crack_title_key —
/// no plaintext supplied — recovers a round-tripping key when the
/// cleartext ends in a periodic run that continues into 0x80.
#[test]
@@ -491,7 +530,7 @@ mod tests {
}
}
/// recover_title_key_from_plain inverts dvdcss_unscramble exactly: scramble
/// recover_title_key_from_plain inverts descramble_sector exactly: scramble
/// a known body, hand back the keystream-derived key, and the recovered
/// key (XOR-back included) reproduces the plaintext.
#[test]
@@ -590,4 +629,512 @@ mod tests {
let _ = crack_title_key(&sector);
}
}
// ── entry-point guards on caller- and disc-supplied lengths ────────────
/// A sector buffer that ENDS inside the encrypted region must be refused,
/// not sliced.
///
/// `recover_title_key` slices `sector[0x80..0x8A]` unconditionally after its
/// length guard. The existing short-sector test uses `SECTOR_BYTES - 1`,
/// which is still long enough for that slice to succeed — so the guard was
/// never the thing producing the `None`, and dropping it (or weakening the
/// `||` to `&&`, which a full-length crib satisfies) changed nothing
/// observable. On a real short read this is an out-of-bounds panic on the
/// rip thread.
#[test]
fn recover_rejects_a_sector_that_ends_inside_the_encrypted_region() {
for len in [0x81usize, 0x85, 0x89] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so no other guard fires first
assert!(
recover_title_key(&sector, &PES).is_none(),
"a {len}-byte buffer cannot supply ten ciphertext bytes at 0x80"
);
}
}
/// A buffer LONGER than one sector is still one sector: both entry points
/// read the first `SECTOR_BYTES` and must recover the key from it.
///
/// Callers read DVD data in multi-sector blocks, so an over-long slice is
/// the normal case, not an exotic one. A length guard that rejected it
/// (`len > SECTOR_BYTES` instead of `<`) would make every block-read caller
/// silently unable to crack anything.
#[test]
fn a_buffer_longer_than_one_sector_still_yields_its_key() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let mut padded = sector.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
recover_title_key(&padded, &PES),
Some(title_key),
"a two-and-a-bit-sector buffer must still recover the first sector's key"
);
let (periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
let mut padded = periodic.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
crack_title_key(&padded),
crack_title_key(&periodic),
"padding past the sector must not change the crack result"
);
assert!(crack_title_key(&padded).is_some());
}
/// `recover_title_key` accepts MORE than ten bytes of known plaintext, and
/// uses all of it: the extra bytes tighten the `descramble_matches` gate.
/// The ten-byte figure is a MINIMUM (the cipher is iterated ten times), not
/// an exact requirement — a guard reading it as an upper bound would reject
/// every caller that knows a longer crib.
#[test]
fn recover_accepts_more_than_ten_bytes_of_known_plaintext() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let long_plain: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(37).wrapping_add(5))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &long_plain);
assert_eq!(
recover_title_key(&sector, &long_plain),
Some(title_key),
"64 bytes of known plaintext must be accepted, not rejected as \
'more than ten'"
);
}
/// The scramble-flag gate on a sector whose BODY really is ciphertext.
///
/// Both entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
/// unscrambled sector has no title key to recover, and its bytes at 0x80
/// are already plaintext. Every prior test of this gate used an all-zero or
/// all-`0x11` sector, where the recovery would have found nothing anyway —
/// so widening the mask test (`&` to `|`, which makes it true for EVERY
/// flag byte) produced the same `None` and went unseen.
///
/// Here the sector is genuinely scrambled and its key IS recoverable; only
/// the cleared flag stands in the way. If the gate stops working, both
/// functions start returning keys for sectors the disc says are in the
/// clear.
#[test]
fn a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key(&sector, &PES),
Some(title_key),
"fixture check: with the flag set this sector's key IS recoverable"
);
sector[FLAG_BYTE] = 0x00;
assert_eq!(
recover_title_key(&sector, &PES),
None,
"scramble bits clear → no title key, even though one could be found"
);
let (mut periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
assert!(
crack_title_key(&periodic).is_some(),
"fixture check: with the flag set this sector cracks"
);
assert!(
attack_crib(&periodic).is_some(),
"fixture check: with the flag set this sector has a usable crib"
);
periodic[FLAG_BYTE] = 0x00;
assert_eq!(
crack_title_key(&periodic),
None,
"scramble bits clear → no crack, even though one would succeed"
);
// `attack_crib` carries its own copy of the same gate, and it is the one
// that actually stops the crack (`crack_title_key`'s is defensive
// duplication). The crib doubles as the decrypt path's cached-key
// oracle, so a widened mask there would hand that path a "predicted
// plaintext" for sectors that were never scrambled.
assert_eq!(
attack_crib(&periodic),
None,
"an unscrambled sector has no predicted plaintext to offer"
);
}
// ── descramble_matches: the verification gate's own mechanics ──────────
/// The gate must verify a candidate against the sector's CIPHERTEXT
/// regardless of what the sector's own flag byte says.
///
/// `descramble_matches` forces `0x10` on its copy precisely because
/// [`super::lfsr::descramble_sector`] is a no-op when the scramble bits are
/// clear — without that, verifying a scrambled-but-unflagged sector
/// compares raw ciphertext against the crib, and every candidate key is
/// rejected. Nothing exercised it: every fixture already had the flag set,
/// where forcing the bit is a no-op.
#[test]
fn descramble_matches_forces_the_scramble_flag_on_its_own_copy() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
sector[FLAG_BYTE] = 0x00;
assert!(
descramble_matches(&sector, &title_key, &PES),
"the body is ciphertext and the key is right — the gate must \
descramble it even though the flag byte says otherwise"
);
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
}
/// The gate compares the WHOLE supplied plaintext, clamped to the encrypted
/// region.
///
/// Two properties in one, because they are the two halves of
/// `plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
///
/// - it must compare beyond the first sixteen bytes, or a key that opens
/// only the head of the crib is accepted; and
/// - it must never compare past the end of the sector — a caller that
/// knows more plaintext than the 1920-byte encrypted region holds
/// otherwise indexes off the end of the buffer and panics.
#[test]
fn descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let body: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(29).wrapping_add(3))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &body);
assert!(descramble_matches(&sector, &title_key, &body));
// A crib agreeing for the first 16 bytes and diverging after must be
// rejected: the comparison window is the crib's length, not a fixed
// prefix.
let mut tail_wrong = body.clone();
tail_wrong[40] ^= 0xFF;
assert!(
!descramble_matches(&sector, &title_key, &tail_wrong),
"a crib that diverges at byte 40 must not match"
);
assert_eq!(
tail_wrong[..16],
body[..16],
"fixture check: the first 16 bytes are identical, so only a \
comparison that runs past them can tell these apart"
);
// A crib LONGER than the encrypted region: the comparison is clamped to
// the sector, not run off the end of it.
let plain_len = SECTOR_BYTES - ENCRYPTED_START;
let mut over_long = vec![0u8; plain_len + 10];
let (full_sector, full_body) = synth_sector(&title_key, &seed, &[0x00u8; 10]);
over_long[..plain_len].copy_from_slice(&full_body[ENCRYPTED_START..]);
assert!(
descramble_matches(&full_sector, &title_key, &over_long),
"a crib longer than the encrypted region must be clamped, not \
compared past the end of the sector"
);
}
// ── attack_crib: known-answer vectors ──────────────────────────────────
//
// `attack_crib` is BOTH the cracker's known plaintext and the decrypt
// path's "did the cached key descramble correctly?" oracle. Until now it
// was only ever exercised end-to-end through `crack_title_key`, on a
// fixture whose periodic run covered 39 bytes (0x59..0x80) — long enough
// that the run start, the cycle count and the `i % best_p` wrap were all
// slack. A crib that silently drifts costs a rip its title key.
/// Build a sector whose clear header ends in a `period`-length repeating
/// run of exactly `run_len` bytes immediately before 0x80.
///
/// The run is anchored to ABSOLUTE sector offset (`sec[x] = pat[x % period]`),
/// which is what makes "the run continues past 0x80" a statement independent
/// of the code under test: the byte at `0x80 + i` of the underlying
/// plaintext is `pat[(0x80 + i) % period]`.
///
/// Everything before the run is `0x00` (the pattern bytes are all >= 0xD0,
/// so the run cannot be extended backwards by accident), and the encrypted
/// region is filled with `0xFF` — so a crib that reads past 0x80 into
/// "ciphertext" is immediately visible.
fn sector_with_trailing_run(period: usize, run_len: usize) -> Vec<u8> {
assert!(
run_len < ENCRYPTED_START,
"the run lives in the clear header"
);
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for x in (ENCRYPTED_START - run_len)..ENCRYPTED_START {
sector[x] = pat[x % period];
}
sector
}
/// The crib the run PREDICTS: the periodic pattern continued past 0x80.
fn expected_crib(period: usize) -> [u8; 10] {
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
let mut out = [0u8; 10];
for (i, o) in out.iter_mut().enumerate() {
*o = pat[(ENCRYPTED_START + i) % period];
}
out
}
/// KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly
/// at 0x80, the crib is the run continued forward — the same ten bytes for
/// every run length, because the prediction depends only on the pattern and
/// the phase, never on how many cycles happened to be visible.
///
/// The short lengths are the load-bearing ones: at `run_len = 11` the crib
/// window starts at 0x76 and is only 10 bytes from the end of the header, so
/// any drift in `plain_start`, in `cycles * best_p`, or in the `i % best_p`
/// wrap reads the 0xFF "ciphertext" instead of the run.
#[test]
fn attack_crib_predicts_the_periodic_run_continuing_past_0x80() {
for &run_len in &[11usize, 12, 13, 14, 15, 16, 20, 31] {
let sector = sector_with_trailing_run(5, run_len);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(5)),
"period-5 run of {run_len} bytes must predict the run continuing"
);
}
}
/// The same known answer across several periods, including a period that
/// does NOT divide 0x80 (so the crib's phase is non-zero and a body that
/// restarted the pattern at index 0 gives a different answer).
#[test]
fn attack_crib_recovers_the_run_period_and_phase() {
// 0x80 % period: 3 for 5, 2 for 6, 2 for 7, 8 for 0x18 — all non-zero,
// so the predicted first byte is NOT pat[0] in any of these cases.
for &period in &[5usize, 6, 7, 0x18] {
let sector = sector_with_trailing_run(period, 3 * period + 1);
let crib =
attack_crib(&sector).unwrap_or_else(|| panic!("no crib for period {period}"));
assert_eq!(crib, expected_crib(period), "period {period}");
assert_ne!(
crib[0], 0xD0,
"period {period} does not divide 0x80, so the crib must not \
start at pattern index 0"
);
assert!(
crib.iter().all(|&b| b != 0xFF),
"period {period}: the crib must never contain a byte read from \
the encrypted region"
);
}
}
/// A run of exactly ONE cycle (plus the trivial tail the detector counts) is
/// not enough to predict forward: [`attack_crib`] requires at least two full
/// cycles. Weakening that guard would let a one-off byte sequence be
/// declared periodic and produce a confidently wrong crib — which the
/// decrypt path uses as its "is my cached key still right?" oracle.
#[test]
fn attack_crib_refuses_a_run_shorter_than_two_cycles() {
// period 8, run of 9 bytes: best_plen = 8, 8 / 8 == 1 cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 9)), None);
// period 0x18, run of 0x19 bytes: one cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(0x18, 0x19)), None);
// ...and one more byte of run does not conjure a second cycle either.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 10)), None);
}
/// A header with no repeating tail at all yields no crib. Asserted on a
/// header whose bytes are pairwise distinct right up to 0x80, so no cycle
/// length in 2..0x2F can match even one byte.
#[test]
fn attack_crib_refuses_a_header_with_no_periodic_tail() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
// 0x00..0x80 strictly increasing: sec[a] == sec[b] iff a == b, so the
// detector's `sec[0x7f - (j % i)] == sec[0x7f - j]` needs j % i == j,
// which the scan's starting `j = i + 1` already excludes.
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = x as u8;
}
assert_eq!(attack_crib(&sector), None);
// And the cracker built on it reports no key rather than guessing.
assert_eq!(crack_title_key(&sector), None);
}
/// `attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
/// own length guard is the only thing between a short buffer and an
/// out-of-bounds read. Nothing reached it: every caller-level test used a
/// full sector, and the entry points' guards fire first.
#[test]
fn attack_crib_refuses_a_buffer_shorter_than_a_sector() {
for len in [0x15usize, 0x40, 0x7F, SECTOR_BYTES - 1] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so the flag half cannot fire
assert_eq!(
attack_crib(&sector),
None,
"a {len}-byte buffer is not a sector"
);
}
}
/// A header that is periodic ALL THE WAY to offset 0 must not walk the
/// backward scan off the front of the sector.
///
/// The detector counts backwards from 0x7f while `j < 0x80`. On a fully
/// periodic header the run never breaks, so `j` reaches 0x7f and the bound
/// is the ONLY thing that stops it — one step further and `0x7f - j`
/// underflows a `usize` and panics. A constant or fully-patterned 128-byte
/// header is ordinary DVD data (padding, a run of zeros), not a crafted
/// input, and every existing fixture had a filler/run boundary well before
/// offset 0 that stopped the scan early.
#[test]
fn attack_crib_survives_a_header_that_is_periodic_to_offset_zero() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x30;
let period = 5usize;
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = pat[x % period];
}
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
// The FLAG byte sits inside the header at 0x14, so it interrupts the
// pattern there; re-lay it and accept that 0x14 breaks the run — the
// scan still reaches offset 0x15 - 1 = 0x14 going backwards, i.e.
// j = 0x7f - 0x14 = 0x6b, well short of the bound. Instead put the
// scramble flag bits into a byte value that IS the pattern's.
sector[FLAG_BYTE] = pat[FLAG_BYTE % period];
assert_ne!(
sector[FLAG_BYTE] & 0x30,
0,
"fixture check: the pattern byte at 0x14 must itself carry \
scramble bits, so the header stays unbroken"
);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(period)),
"a fully periodic header must predict its own continuation, and \
the backward scan must stop at offset 0"
);
}
/// The crib is read from the CLEAR header only. A run that reaches 0x80 must
/// predict from the header bytes, never from the encrypted region — the
/// previously-fixed bug this function's doc comment records. Pinned by
/// rewriting the encrypted region and requiring the crib not to move.
#[test]
fn attack_crib_is_independent_of_the_encrypted_region() {
let base = sector_with_trailing_run(5, 11);
let crib = attack_crib(&base).expect("crib");
for fill in [0x00u8, 0x5A, 0xD1, 0xFF] {
let mut s = base.clone();
for b in s[ENCRYPTED_START..].iter_mut() {
*b = fill;
}
assert_eq!(
attack_crib(&s),
Some(crib),
"the crib must not depend on the encrypted region (fill {fill:#04x})"
);
}
}
// ── recover_title_key_from_plain: input-length guard ───────────────────
/// `recover_title_key_from_plain` unconditionally builds a 10-byte keystream
/// buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard
/// is the only thing standing between a short slice and an
/// index-out-of-bounds PANIC.
///
/// Nothing reached that guard before: `recover_title_key` rejects
/// `plain.len() < 10` at its own door and always hands on exactly ten
/// ciphertext bytes, and `crack_title_key_inner` always passes a fixed
/// `[u8; 10]` crib. The guard is a live contract for any future caller and
/// was executed by no test at either boundary.
#[test]
fn recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input() {
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let full = [0xA5u8; 10];
for n in 0..10usize {
assert_eq!(
recover_title_key_from_plain(&full[..n], &full, &seed),
None,
"{n} ciphertext bytes is fewer than the ten the cipher iterates"
);
assert_eq!(
recover_title_key_from_plain(&full, &full[..n], &seed),
None,
"{n} plaintext bytes is fewer than the ten the cipher iterates"
);
}
// Exactly ten of each is ACCEPTED as far as the search — the boundary is
// `< 10`, not `<= 10`. (Whether this particular keystream has a seed is
// immaterial; what must not happen is an early `None` from the guard.)
// Proven through the round-trip fixture, whose inputs are exactly ten
// bytes and which does recover its key.
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key_from_plain(
&sector[ENCRYPTED_START..ENCRYPTED_START + 10],
&PES,
&seed
),
Some(title_key),
"exactly ten bytes of each input must run the search, not trip the guard"
);
}
/// The seed XOR-back ([`recover_title_key_from_plain`]'s last step) is what
/// turns the recovered LFSR key into the TITLE key: `key ^= sector_seed`.
/// Pinned as a known answer across seeds that differ only in one byte — the
/// same ciphertext/plaintext pair therefore must yield title keys differing
/// in exactly that byte.
///
/// Without this, a body that ORed the seed in (or dropped the step) still
/// round-trips on any fixture whose seed is zero, and on the non-zero ones
/// the failure looks like "no key found" rather than a wrong step.
#[test]
fn recover_title_key_from_plain_xors_the_sector_seed_back_out() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let crypted = &sector[ENCRYPTED_START..ENCRYPTED_START + 10];
// The cipher is seeded from `title_key XOR seed`, so re-running the SAME
// ciphertext/plaintext against a seed differing in one byte must return
// a title key differing in exactly that byte — the XOR is a bijection.
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &seed),
Some(title_key)
);
for byte in 0..5usize {
for bit in [0u32, 3, 7] {
let mut alt_seed = seed;
alt_seed[byte] ^= 1u8 << bit;
let mut expected = title_key;
expected[byte] ^= 1u8 << bit;
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &alt_seed),
Some(expected),
"seed byte {byte} bit {bit} must XOR straight through to the \
title key"
);
}
}
}
}
+11 -12
View File
@@ -24,9 +24,8 @@ pub const TAB1: [u8; 256] = [
0xb7, 0xf7, 0xbf, 0xa2, 0xe7, 0xa7, 0xef, 0xf2, 0xba, 0xfa, 0xb2, 0xaf, 0xea, 0xaa, 0xe2, 0xff,
];
/// Table 2: LFSR1 high-byte feedback permutation.
///
/// Byte-identical to libdvdcss `p_css_tab2` (csstables.h).
/// Table 2: LFSR1 high-byte feedback permutation — a fixed constant of the CSS
/// cipher (per the published algorithm).
pub const TAB2: [u8; 256] = [
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x09, 0x08, 0x0b, 0x0a, 0x0d, 0x0c, 0x0f, 0x0e,
0x12, 0x13, 0x10, 0x11, 0x16, 0x17, 0x14, 0x15, 0x1b, 0x1a, 0x19, 0x18, 0x1f, 0x1e, 0x1d, 0x1c,
@@ -46,12 +45,12 @@ pub const TAB2: [u8; 256] = [
0xed, 0xec, 0xef, 0xee, 0xe9, 0xe8, 0xeb, 0xea, 0xe4, 0xe5, 0xe6, 0xe7, 0xe0, 0xe1, 0xe2, 0xe3,
];
/// Table 3: LFSR1 9-bit low-word feedback table (512 entries).
/// Table 3: LFSR1 9-bit low-word feedback table (512 entries) — a fixed constant
/// of the CSS cipher (per the published algorithm).
///
/// Byte-identical to libdvdcss `p_css_tab3` (csstables.h): the 8-value
/// block `BASE[i & 7]` repeated 64 times. The CSS LFSR1 step indexes this
/// table with the 9-bit low register (0x100..=0x1FF), but only the low 3
/// bits select the output — the high bits are ignored, hence the constant
/// It is the 8-value block `BASE[i & 7]` repeated 64 times. The CSS LFSR1 step
/// indexes this table with the 9-bit low register (0x100..=0x1FF), but only the
/// low 3 bits select the output — the high bits are ignored, hence the constant
/// blocks. The 512-entry width simply lets the 9-bit index be used without
/// masking.
pub const TAB3: [u8; 512] = [
@@ -197,12 +196,12 @@ mod tests {
}
}
/// TAB3 is the libdvdcss `p_css_tab3`: the 8-value feedback block
/// TAB3 is the CSS LFSR1 low-word table: the 8-value feedback block
/// BASE = [0x00,0x24,0x49,0x6d,0x92,0xb6,0xdb,0xff]
/// repeated 64 times — `TAB3[i] == BASE[i & 7]`. The high bits of the
/// 9-bit index do not affect the output (libdvdcss's LFSR1 step indexes
/// with the full 9-bit low register but only `& 7` matters). This pins
/// all 512 entries to the published table.
/// 9-bit index do not affect the output (the LFSR1 step indexes with the
/// full 9-bit low register but only `& 7` matters). This pins all 512
/// entries to the published cipher's table.
///
/// Mutation: flip any single byte in the TAB3 literal -> the formula
/// check fails at that index.
+1331 -783
View File
File diff suppressed because it is too large Load Diff
+105 -47
View File
@@ -22,10 +22,7 @@
//! `Disc`-level dump ([`dump_disc`]) covers everything that survives
//! lowering: titles, streams, the picked main feature, and AACS state.
use crate::disc::{
AudioChannels, ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, SampleRate,
Stream,
};
use crate::disc::{ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, Stream};
use crate::ifo::{CellCategory, DvdTitle};
const DIAG: &str = "freemkv::diag";
@@ -95,36 +92,12 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
}
}
/// Channel count from an [`AudioChannels`] layout (what lands in the MKV
/// `Channels` element).
pub fn channel_count(ch: AudioChannels) -> u8 {
match ch {
AudioChannels::Mono => 1,
AudioChannels::Stereo => 2,
AudioChannels::Stereo21 => 3,
AudioChannels::Quad => 4,
AudioChannels::Surround50 => 5,
AudioChannels::Surround51 => 6,
AudioChannels::Surround61 => 7,
AudioChannels::Surround71 => 8,
AudioChannels::Unknown => 0,
}
}
/// Sample-rate in Hz for a [`SampleRate`].
pub fn sample_rate_hz(s: SampleRate) -> u32 {
match s {
SampleRate::S44_1 => 44100,
SampleRate::S48 => 48000,
SampleRate::S88_2 => 88200,
SampleRate::S96 => 96000,
SampleRate::S176_4 => 176400,
SampleRate::S192 => 192000,
SampleRate::S48_96 => 96000,
SampleRate::S48_192 => 192000,
SampleRate::Unknown => 0,
}
}
// `channel_count` and `sample_rate_hz` lived here as a third copy of the
// AudioChannels/SampleRate mappings. They were the only HONEST copy — returning
// 0 for Unknown where the canonical accessors fabricated 6 channels at 48 kHz —
// and their only caller was the trace line below, in this same file. The
// canonical accessors are honest now, so the duplicates are gone rather than
// left to drift a fourth time.
// ── DVD cell-category dump (from the IFO scan, pre-lowering) ─────────────────
@@ -336,7 +309,7 @@ fn frame_record(track_idx: usize, pts_ns: i64, keyframe: bool, data: &[u8]) -> V
/// `track_number` is the 1-based MKV track number; `track` is the built
/// [`crate::mux::mkv::MkvTrack`] whose fields map one-to-one onto the emitted
/// elements (see `MkvMuxer::new`). No-op unless the diag target is on.
pub fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
pub(crate) fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
if !diag_enabled() {
return;
}
@@ -499,15 +472,31 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!(
target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason=canonical_title_order(fits-disc, fewest-clips, longest, richest-audio)",
size={}B clips={} reason={}",
main.playlist,
main.duration_secs,
main.size_bytes,
main.clips.len(),
main_feature_reason(),
);
}
}
/// The `reason=` token on the main-feature decision row.
///
/// DERIVED from [`Disc::CANONICAL_TITLE_ORDER_KEYS`], which lives beside the
/// comparator that actually implements them — never restated here. The previous
/// hand-written copy drifted (it advertised a `fewest-clips` key the comparator
/// had replaced with largest-physical-size), which made the self-diagnosing log
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
}
fn dump_aacs(disc: &Disc) {
let Some(a) = disc.aacs.as_ref() else {
if disc.css.is_some() {
@@ -530,7 +519,7 @@ fn dump_aacs(disc: &Disc) {
a.bus_encryption,
a.mkb_version,
a.disc_hash,
a.key_source.name(),
a.key_source,
a.vuk.is_some(),
a.unit_keys.len(),
a.uk_ro.len(),
@@ -610,8 +599,8 @@ fn dump_title(ti: usize, title: &DiscTitle) {
a.pid,
a.codec,
a.channels,
channel_count(a.channels),
sample_rate_hz(a.sample_rate),
a.channels.count(),
a.sample_rate.hz(),
a.language,
a.secondary,
),
@@ -631,6 +620,60 @@ fn dump_title(ti: usize, title: &DiscTitle) {
#[cfg(test)]
mod tests {
use super::*;
// Needed only by the tests: the production code in this file no longer names
// these types directly, since the local channel/sample-rate duplicates were
// deleted in favour of the canonical accessors.
use crate::disc::{AudioChannels, SampleRate};
/// The main-feature decision row must NAME `canonical_title_order`'s sort
/// keys, not restate them from memory. The restated copy had drifted: it
/// still advertised a "fewest-clips" key long after the comparator replaced
/// clip-count with largest-physical-size, so a bug report read at
/// `--log-level 3` explained the pick with a rule the code does not apply.
///
/// The behavioural half is asserted first — against the comparator itself,
/// with literals — so the key names are checked against what the code
/// actually does, not against the string that names them.
#[test]
fn main_feature_reason_names_the_comparators_real_keys() {
use crate::disc::{Clip, Disc, DiscTitle};
let sized = |size_bytes: u64, n_clips: usize| DiscTitle {
size_bytes,
clips: (0..n_clips)
.map(|i| Clip {
feed_span: None,
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 0,
duration_secs: 0.0,
source_packets: 0,
})
.collect(),
..DiscTitle::empty()
};
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's
// primary key among disc-fitting titles is LARGEST SIZE. "fewest clips"
// would predict the opposite, so the drifted string described a rule
// the comparator does not implement.
let many_clips_big = sized(8_000_000_000, 40);
let one_clip_small = sized(1_000_000_000, 1);
assert_eq!(
Disc::canonical_title_order(&many_clips_big, &one_clip_small, 25_000_000_000),
std::cmp::Ordering::Less,
"largest size wins regardless of clip count"
);
let reason = main_feature_reason();
assert!(
!reason.contains("clips"),
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
);
}
#[test]
fn res_str_keeps_interlace_marker() {
@@ -656,18 +699,33 @@ mod tests {
assert_eq!(hdr_str(HdrFormat::Sdr), "SDR");
}
/// Moved from the deleted local duplicates onto the canonical accessors,
/// with the Unknown case added — which is the whole point of the change.
#[test]
fn channel_count_matches_layout() {
assert_eq!(channel_count(AudioChannels::Mono), 1);
assert_eq!(channel_count(AudioChannels::Stereo), 2);
assert_eq!(channel_count(AudioChannels::Surround51), 6);
assert_eq!(channel_count(AudioChannels::Surround71), 8);
fn channel_count_matches_layout_and_is_zero_when_unknown() {
assert_eq!(AudioChannels::Mono.count(), 1);
assert_eq!(AudioChannels::Stereo.count(), 2);
assert_eq!(AudioChannels::Surround51.count(), 6);
assert_eq!(AudioChannels::Surround71.count(), 8);
// The one that matters. This used to return 6, which is indistinguishable
// from a real 5.1 track and left every caller responsible for checking
// the variant first.
assert_eq!(
AudioChannels::Unknown.count(),
0,
"an unknown layout must not report a plausible channel count"
);
}
#[test]
fn sample_rate_hz_values() {
assert_eq!(sample_rate_hz(SampleRate::S48), 48000);
assert_eq!(sample_rate_hz(SampleRate::S96), 96000);
fn sample_rate_hz_values_and_zero_when_unknown() {
assert_eq!(SampleRate::S48.hz(), 48000.0);
assert_eq!(SampleRate::S96.hz(), 96000.0);
assert_eq!(
SampleRate::Unknown.hz(),
0.0,
"an unknown sample rate must not report a plausible 48 kHz"
);
}
#[test]
+743
View File
@@ -0,0 +1,743 @@
//! ECMA-167 / UDF 1.02 descriptor encoder.
//!
//! Turns a [`Layout`](super::layout::Layout) — a directory tree with every
//! ICB, directory-data and file-data block already assigned — into the set of
//! metadata sectors a real UDF volume would carry. Nothing here touches the
//! filesystem: it is a pure function from layout to sectors, which is what
//! makes it testable against the production parser in `udf.rs`.
//!
//! What is emitted, in volume order:
//!
//! | sector | descriptor |
//! |---|---|
//! | 16, 17, 18 | Volume Recognition Sequence — `BEA01`, `NSR02`, `TEA01` (ECMA-167 2/9.1) |
//! | 32… | Main Volume Descriptor Sequence — PVD, IUVD, PD, LVD, USD, TD |
//! | 48… | Reserve VDS (byte-identical but for the tag locations) |
//! | 64, 65 | Logical Volume Integrity Sequence — LVID, TD |
//! | 256 | Anchor Volume Descriptor Pointer |
//! | `part_start` + 0, +1 | File Set Descriptor, TD |
//! | `part_start` + … | File Entries (ICBs) and directory data (FIDs) |
//! | last sector | Anchor Volume Descriptor Pointer (copy) |
//!
//! UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
//! the DVD-Video profile, it is the shape `read_filesystem` takes when
//! `num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
//! entirely. That also means a synthetic image never exercises the Metadata
//! Partition path in `udf.rs` (`:946-991`) — see the module docs on `dirimage`.
use super::layout::{DirNode, Layout};
use crate::error::{Error, Result};
use std::collections::BTreeMap;
/// Logical block / sector size. Fixed for every optical profile this crate
/// reads, and the same quantity as [`crate::consts::SECTOR_BYTES`] — aliased
/// rather than re-declared so the two cannot drift apart. The short name is
/// kept because it appears in ~25 extent and offset expressions across
/// `dirimage`, where the longer one would bury the arithmetic.
pub(super) use crate::consts::SECTOR_BYTES as SECTOR;
/// Descriptor version recorded in every tag. 2 = ECMA-167 2nd edition, which
/// is what UDF revisions up to and including 2.00 require.
const DESC_VERSION: u16 = 2;
/// UDF revision recorded in the domain EntityID suffix (1.02, BCD-ish u16).
const UDF_REVISION: u16 = 0x0102;
/// A fixed recording timestamp, so an image synthesized from the same folder
/// twice is byte-identical. Real mtimes would make every test golden-file
/// comparison and every `dir:// -> iso://` re-run differ for no benefit.
const FIXED_TIME: Timestamp = Timestamp {
year: 2000,
month: 1,
day: 1,
};
struct Timestamp {
year: i16,
month: u8,
day: u8,
}
/// The synthesized metadata: absolute LBA → sector contents. Data sectors are
/// NOT here; they are served from the backing files.
pub(super) type MetaSectors = BTreeMap<u32, Box<[u8; SECTOR]>>;
/// The descriptor-tag CRC of ECMA-167 7.2.4: polynomial 0x1021, initial value
/// ZERO, no reflection, no final XOR — the variant catalogued as CRC-16/XMODEM
/// (check value 0x31C3), NOT CCITT-FALSE, which seeds at 0xFFFF and would make
/// every descriptor this crate writes fail a conformant driver's validation.
fn crc16(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x1021
} else {
crc << 1
};
}
}
crc
}
/// Write an ECMA-167 3/7.2 descriptor tag over `buf[0..16]`.
///
/// `tag_loc` is the block number of the sector holding the descriptor —
/// ABSOLUTE for the volume-space descriptors (AVDP, VDS, LVID) and
/// PARTITION-RELATIVE for everything inside the partition (FSD, File Entries).
/// Getting that wrong is the classic reason a hand-built volume mounts nowhere:
/// a driver that validates the tag location rejects the descriptor outright.
///
/// `desc_len` is the descriptor's total length including the tag; the CRC
/// covers `buf[16..desc_len]`.
fn finish_tag(buf: &mut [u8], tag_id: u16, tag_loc: u32, desc_len: usize) {
buf[0..2].copy_from_slice(&tag_id.to_le_bytes());
buf[2..4].copy_from_slice(&DESC_VERSION.to_le_bytes());
buf[4] = 0; // checksum, filled below
buf[5] = 0; // reserved
buf[6..8].copy_from_slice(&0u16.to_le_bytes()); // tag serial number
let crc_len = desc_len - 16;
let crc = crc16(&buf[16..desc_len]);
buf[8..10].copy_from_slice(&crc.to_le_bytes());
buf[10..12].copy_from_slice(&(crc_len as u16).to_le_bytes());
buf[12..16].copy_from_slice(&tag_loc.to_le_bytes());
// ECMA-167 3/7.2.3: sum of bytes 0..16 EXCLUDING byte 4, modulo 256.
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
buf[4] = (sum % 256) as u8;
}
/// ECMA-167 1/7.2.1 charspec: type 0 (CS0) + "OSTA Compressed Unicode".
fn put_charspec(buf: &mut [u8]) {
buf[0] = 0;
let id = b"OSTA Compressed Unicode";
buf[1..1 + id.len()].copy_from_slice(id);
}
/// ECMA-167 1/7.4 EntityID: flags byte, 23 identifier bytes, 8 suffix bytes.
fn put_entity_id(buf: &mut [u8], id: &[u8], suffix: &[u8]) {
buf[0] = 0;
let n = id.len().min(23);
buf[1..1 + n].copy_from_slice(&id[..n]);
let m = suffix.len().min(8);
buf[24..24 + m].copy_from_slice(&suffix[..m]);
}
/// The `*OSTA UDF Compliant` domain EntityID suffix: UDF revision, domain
/// flags (0 = neither hard nor soft write-protected), reserved.
fn domain_suffix() -> [u8; 8] {
let mut s = [0u8; 8];
s[0..2].copy_from_slice(&UDF_REVISION.to_le_bytes());
s
}
/// This crate's implementation EntityID suffix: OS class / OS identifier
/// (0 = undefined, deliberately — the image is not OS-specific) + 6 free bytes.
fn impl_suffix() -> [u8; 8] {
[0u8; 8]
}
fn put_impl_id(buf: &mut [u8]) {
put_entity_id(buf, b"*freemkv", &impl_suffix());
}
fn put_domain_id(buf: &mut [u8]) {
put_entity_id(buf, b"*OSTA UDF Compliant", &domain_suffix());
}
/// OSTA CS0 d-string: a compression-ID byte, the characters, then the used
/// length in the FIELD'S LAST byte (ECMA-167 1/7.2.12 + UDF 2.1.3). An
/// all-zero field is the empty string.
fn put_dstring(buf: &mut [u8], s: &str) {
if s.is_empty() {
return;
}
let encoded = encode_cs0(s);
// Leave room for the trailing length byte.
let room = buf.len() - 1;
let n = encoded.len().min(room);
buf[..n].copy_from_slice(&encoded[..n]);
buf[buf.len() - 1] = n as u8;
}
/// OSTA CS0: compression ID 8 (one byte per character) when every character
/// is ASCII, otherwise compression ID 16 (UTF-16BE).
///
/// ASCII rather than Latin-1 for the 8-bit form on purpose: `parse_udf_name`
/// (`udf.rs:1467`) decodes a compression-8 name with `from_utf8_lossy`, so a
/// 0x80-0xFF byte — legal CS0 — would come back as U+FFFD. Every character
/// above 0x7F therefore takes the 16-bit form, which that parser decodes
/// correctly.
pub(super) fn encode_cs0(s: &str) -> Vec<u8> {
if s.is_ascii() {
let mut v = Vec::with_capacity(1 + s.len());
v.push(8u8);
v.extend_from_slice(s.as_bytes());
v
} else {
let mut v = vec![16u8];
for u in s.encode_utf16() {
v.extend_from_slice(&u.to_be_bytes());
}
v
}
}
/// ECMA-167 1/7.3 timestamp, 12 bytes. Type 1 (local time) with a zero
/// offset, i.e. UTC.
fn put_timestamp(buf: &mut [u8]) {
buf[0..2].copy_from_slice(&0x1000u16.to_le_bytes());
buf[2..4].copy_from_slice(&FIXED_TIME.year.to_le_bytes());
buf[4] = FIXED_TIME.month;
buf[5] = FIXED_TIME.day;
}
/// ECMA-167 3/7.1 extent_ad: length in BYTES, then location.
fn put_extent_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
/// ECMA-167 4/14.14.2 long_ad: length+type, then lb_addr (block, partition
/// reference), then 6 implementation-use bytes.
fn put_long_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
buf[8..10].copy_from_slice(&0u16.to_le_bytes()); // partition reference 0
}
/// ECMA-167 4/14.14.1 short_ad. The top two bits of the length word are the
/// extent TYPE (0 = recorded and allocated), which is exactly why `udf.rs`
/// masks with `0x3FFF_FFFF` when it reads one back — the mask is the field
/// boundary, not a truncation bug.
fn put_short_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
debug_assert!(len_bytes <= 0x3FFF_FFFF, "AD length must fit 30 bits");
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
fn blank() -> Box<[u8; SECTOR]> {
Box::new([0u8; SECTOR])
}
// ── Volume-space descriptors ────────────────────────────────────────────────
/// ECMA-167 2/9.1 Volume Structure Descriptor: the three-sector recognition
/// sequence an OS looks for before it will even consider the volume UDF.
fn volume_recognition(id: &[u8; 5]) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[0] = 0; // structure type
s[1..6].copy_from_slice(id);
s[6] = 1; // structure version
s
}
/// ECMA-167 3/10.1 Primary Volume Descriptor.
fn primary_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes()); // PVD number
put_dstring(&mut s[24..56], volume_id);
s[56..58].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[58..60].copy_from_slice(&1u16.to_le_bytes()); // max volume sequence number
s[60..62].copy_from_slice(&2u16.to_le_bytes()); // interchange level
s[62..64].copy_from_slice(&2u16.to_le_bytes()); // max interchange level
s[64..68].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[68..72].copy_from_slice(&1u32.to_le_bytes()); // max character set list
// UDF 2.2.2.5: the first 8 characters of the volume set identifier must be
// unique. A fixed hex prefix plus the volume id is sufficient here — the
// image is single-volume and never joins a real volume set.
put_dstring(&mut s[72..200], &format!("46524D4B{volume_id}"));
put_charspec(&mut s[200..264]); // descriptor character set
put_charspec(&mut s[264..328]); // explanatory character set
put_timestamp(&mut s[376..388]);
put_impl_id(&mut s[388..420]);
finish_tag(&mut s[..], 1, lba, 512);
s
}
/// ECMA-167 3/10.4 + UDF 2.2.7 Implementation Use Volume Descriptor
/// (`*UDF LV Info`). Not read by `udf.rs`, required by the spec.
fn impl_use_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_entity_id(&mut s[20..52], b"*UDF LV Info", &domain_suffix());
put_charspec(&mut s[52..116]); // LVI charset
put_dstring(&mut s[116..244], volume_id); // logical volume identifier
put_impl_id(&mut s[352..384]);
finish_tag(&mut s[..], 4, lba, 512);
s
}
/// ECMA-167 3/10.5 Partition Descriptor — the descriptor `read_filesystem`
/// takes `partition_start` from (offset 188).
fn partition(part_start: u32, part_sectors: u32, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..22].copy_from_slice(&1u16.to_le_bytes()); // partition flags: allocated
s[22..24].copy_from_slice(&0u16.to_le_bytes()); // partition number
put_entity_id(&mut s[24..56], b"+NSR02", &[]);
// s[56..184] partition contents use = Partition Header Descriptor. All
// zero: a read-only partition records no unallocated/freed space tables.
s[184..188].copy_from_slice(&1u32.to_le_bytes()); // access type: read only
s[188..192].copy_from_slice(&part_start.to_le_bytes());
s[192..196].copy_from_slice(&part_sectors.to_le_bytes());
put_impl_id(&mut s[196..228]);
finish_tag(&mut s[..], 5, lba, 512);
s
}
/// ECMA-167 3/10.6 Logical Volume Descriptor. Carries the FSD long_ad and the
/// partition map table; `read_filesystem` reads `num_partition_maps` at 268
/// and takes the single-partition path when it is 1.
fn logical_volume(
volume_id: &str,
fsd_lba: u32,
integrity_lba: u32,
integrity_sectors: u32,
lba: u32,
seq: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_charspec(&mut s[20..84]);
put_dstring(&mut s[84..212], volume_id);
s[212..216].copy_from_slice(&(SECTOR as u32).to_le_bytes()); // logical block size
put_domain_id(&mut s[216..248]);
// Logical volume contents use = long_ad of the File Set Descriptor,
// partition-relative. One sector.
put_long_ad(&mut s[248..264], SECTOR as u32, fsd_lba);
s[264..268].copy_from_slice(&6u32.to_le_bytes()); // map table length
s[268..272].copy_from_slice(&1u32.to_le_bytes()); // number of partition maps
put_impl_id(&mut s[272..304]);
put_extent_ad(
&mut s[432..440],
integrity_sectors * SECTOR as u32,
integrity_lba,
);
// ECMA-167 3/10.7.2 Type 1 partition map.
s[440] = 1; // map type
s[441] = 6; // map length
s[442..444].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[444..446].copy_from_slice(&0u16.to_le_bytes()); // partition number
finish_tag(&mut s[..], 6, lba, 446);
s
}
/// ECMA-167 3/10.8 Unallocated Space Descriptor with zero extents — the whole
/// volume is accounted for by the partition.
fn unallocated_space(lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes());
finish_tag(&mut s[..], 7, lba, 24);
s
}
/// ECMA-167 3/10.9 Terminating Descriptor.
fn terminating(lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
finish_tag(&mut s[..], 8, lba, 512);
s
}
/// ECMA-167 3/10.10 + UDF 2.2.6 Logical Volume Integrity Descriptor, closed.
fn integrity(
part_sectors: u32,
files: u32,
dirs: u32,
next_uid: u64,
lba: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..32].copy_from_slice(&1u32.to_le_bytes()); // integrity type: close
// s[32..40] next integrity extent: none.
s[40..48].copy_from_slice(&next_uid.to_le_bytes()); // logical volume contents use: next unique id
s[72..76].copy_from_slice(&1u32.to_le_bytes()); // number of partitions
s[76..80].copy_from_slice(&46u32.to_le_bytes()); // length of implementation use
s[80..84].copy_from_slice(&0u32.to_le_bytes()); // free space: none (read-only)
s[84..88].copy_from_slice(&part_sectors.to_le_bytes()); // size table
put_impl_id(&mut s[88..120]);
s[120..124].copy_from_slice(&files.to_le_bytes());
s[124..128].copy_from_slice(&dirs.to_le_bytes());
s[128..130].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min read revision
s[130..132].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min write revision
s[132..134].copy_from_slice(&UDF_REVISION.to_le_bytes()); // max write revision
finish_tag(&mut s[..], 9, lba, 134);
s
}
/// ECMA-167 3/10.2 Anchor Volume Descriptor Pointer. `read_filesystem` reads
/// the main VDS extent from offsets 16..24 and sweeps it.
fn anchor(main_lba: u32, reserve_lba: u32, vds_sectors: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_extent_ad(&mut s[16..24], vds_sectors * SECTOR as u32, main_lba);
put_extent_ad(&mut s[24..32], vds_sectors * SECTOR as u32, reserve_lba);
finish_tag(&mut s[..], 2, lba, 512);
s
}
/// ECMA-167 4/14.1 File Set Descriptor. `read_filesystem` requires tag 256 at
/// the first block of the (metadata =) partition and reads the root ICB block
/// from offset 404.
fn file_set(volume_id: &str, root_icb: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..30].copy_from_slice(&3u16.to_le_bytes()); // interchange level
s[30..32].copy_from_slice(&3u16.to_le_bytes()); // max interchange level
s[32..36].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[36..40].copy_from_slice(&1u32.to_le_bytes()); // max character set list
s[40..44].copy_from_slice(&0u32.to_le_bytes()); // file set number
s[44..48].copy_from_slice(&0u32.to_le_bytes()); // file set descriptor number
put_charspec(&mut s[48..112]);
put_dstring(&mut s[112..240], volume_id);
put_charspec(&mut s[240..304]);
put_dstring(&mut s[304..336], volume_id);
put_long_ad(&mut s[400..416], SECTOR as u32, root_icb);
put_domain_id(&mut s[416..448]);
finish_tag(&mut s[..], 256, lba, 512);
s
}
// ── Partition-space descriptors ─────────────────────────────────────────────
/// UDF permission word: read + execute for owner, group and other. No write
/// bit anywhere — the volume is read-only.
const PERM_R_X: u32 = 0x0000_1000 | 0x0000_0400 | 0x0000_0080 | 0x0000_0020 | 0x4 | 0x1;
/// ECMA-167 4/14.9 File Entry (tag 261).
///
/// Tag 261 rather than the Extended File Entry (266) real BD-ROMs use: an EFE
/// requires UDF 2.00+, and this image declares 1.02. `udf.rs` reads both — the
/// 261 field offsets it uses (l_ea 168, l_ad 172, ADs at 176 + l_ea) are the
/// ones written here.
///
/// `extents` are partition-relative (block, byte-length) pairs, already split
/// so no single one exceeds the 30-bit AD length field.
fn file_entry(
is_dir: bool,
info_len: u64,
extents: &[(u32, u32)],
link_count: u16,
unique_id: u64,
lba: u32,
) -> Result<Box<[u8; SECTOR]>> {
let mut s = blank();
// ICB tag (ECMA-167 4/14.6) at offset 16.
s[16..20].copy_from_slice(&0u32.to_le_bytes()); // prior recorded direct entries
s[20..22].copy_from_slice(&4u16.to_le_bytes()); // strategy type 4
s[24..26].copy_from_slice(&1u16.to_le_bytes()); // max number of entries
s[27] = if is_dir { 4 } else { 5 }; // file type: directory / byte sequence
// s[28..34] parent ICB location: not recorded (permitted).
// s[34..36] ICB flags: 0 => short allocation descriptors. `udf.rs:601`
// reads exactly this word to pick its AD stride.
s[34..36].copy_from_slice(&0u16.to_le_bytes());
// UDF's sentinel for "not specified" is 0xFFFFFFFF, not 0 — 0 is a real
// uid/gid (root). A synthesized image has no meaningful owner, and a driver
// that maps these through would otherwise report every file as root-owned.
s[36..40].copy_from_slice(&u32::MAX.to_le_bytes()); // uid: not specified
s[40..44].copy_from_slice(&u32::MAX.to_le_bytes()); // gid: not specified
s[44..48].copy_from_slice(&PERM_R_X.to_le_bytes());
s[48..50].copy_from_slice(&link_count.to_le_bytes());
s[56..64].copy_from_slice(&info_len.to_le_bytes());
let blocks: u64 = extents
.iter()
.map(|(_, len)| (*len as u64).div_ceil(SECTOR as u64))
.sum();
s[64..72].copy_from_slice(&blocks.to_le_bytes()); // logical blocks recorded
put_timestamp(&mut s[72..84]); // access
put_timestamp(&mut s[84..96]); // modification
put_timestamp(&mut s[96..108]); // attribute
s[108..112].copy_from_slice(&1u32.to_le_bytes()); // checkpoint
put_impl_id(&mut s[128..160]);
s[160..168].copy_from_slice(&unique_id.to_le_bytes());
s[168..172].copy_from_slice(&0u32.to_le_bytes()); // length of EAs
let l_ad = extents.len() * 8;
// A short AD is 8 bytes and the entry has 2048 - 176 = 1872 bytes for
// them, i.e. 234 extents — over 200 GiB at the per-AD ceiling. Beyond
// that an Allocation Extent Descriptor chain would be required; refuse
// rather than write a truncated list.
if 176 + l_ad > SECTOR {
return Err(Error::DirImageTooLarge);
}
s[172..176].copy_from_slice(&(l_ad as u32).to_le_bytes());
for (i, (elba, len)) in extents.iter().enumerate() {
let off = 176 + i * 8;
put_short_ad(&mut s[off..off + 8], *len, *elba);
}
finish_tag(&mut s[..], 261, lba, 176 + l_ad);
Ok(s)
}
/// ECMA-167 4/14.4 File Identifier Descriptor, appended to `buf`.
///
/// FIDs are packed with no inter-descriptor padding beyond the 4-byte
/// alignment the spec mandates, and they are allowed to span logical blocks —
/// which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
/// directory extent as one flat byte run and STOPS at the first non-257 tag,
/// so any block-alignment gap would truncate the directory.
fn push_fid(buf: &mut Vec<u8>, name: &str, icb_lba: u32, is_dir: bool, is_parent: bool) {
let start = buf.len();
let name_field: Vec<u8> = if is_parent {
Vec::new()
} else {
encode_cs0(name)
};
let l_fi = name_field.len();
let mut fid = vec![0u8; 38];
fid[16..18].copy_from_slice(&1u16.to_le_bytes()); // file version number
let mut chars = 0u8;
if is_dir {
chars |= 0x02;
}
if is_parent {
chars |= 0x08;
}
fid[18] = chars;
// The planner refuses any name whose encoding exceeds what this byte can
// hold (`layout::MAX_CS0_NAME_BYTES`), so this cannot wrap in practice. The
// assert states the invariant where it is relied on rather than trusting a
// check three files away; a wrap here would desynchronise the directory.
debug_assert!(
l_fi <= u8::MAX as usize,
"FID name length must fit one byte"
);
fid[19] = l_fi as u8;
put_long_ad(&mut fid[20..36], SECTOR as u32, icb_lba);
fid[36..38].copy_from_slice(&0u16.to_le_bytes()); // length of implementation use
buf.extend_from_slice(&fid);
buf.extend_from_slice(&name_field);
let unpadded = buf.len() - start;
let padded = unpadded.div_ceil(4) * 4;
buf.resize(start + padded, 0);
// The tag is written last: its CRC covers the descriptor body, which the
// padding is not part of (ECMA-167 4/14.4.9 counts padding outside the
// CRC'd length).
let tag_loc_placeholder = 0;
finish_tag(
&mut buf[start..start + unpadded],
257,
tag_loc_placeholder,
unpadded,
);
}
/// Serialize one directory's FID list (parent entry first, then children).
pub(super) fn dir_fids(dir: &DirNode) -> Vec<u8> {
let mut buf = Vec::new();
push_fid(&mut buf, "", dir.parent_icb_lba, true, true);
for sub in &dir.dirs {
push_fid(&mut buf, &sub.name, sub.icb_lba, true, false);
}
for f in &dir.files {
push_fid(&mut buf, &f.name, f.icb_lba, false, false);
}
buf
}
/// Patch every FID's tag location to the block it actually lands in. ECMA-167
/// 3/7.2.2 makes the tag location the block of the descriptor, and a FID that
/// spans two blocks records the block it STARTS in.
fn fix_fid_tag_locations(buf: &mut [u8], first_block: u32) {
let mut pos = 0usize;
while pos + 38 <= buf.len() {
let l_fi = buf[pos + 19] as usize;
let l_iu = u16::from_le_bytes([buf[pos + 36], buf[pos + 37]]) as usize;
let unpadded = 38 + l_iu + l_fi;
if pos + unpadded > buf.len() {
break;
}
let block = first_block + (pos / SECTOR) as u32;
finish_tag(&mut buf[pos..pos + unpadded], 257, block, unpadded);
pos += unpadded.div_ceil(4) * 4;
}
}
// ── Whole-image assembly ────────────────────────────────────────────────────
/// Volume-space block of the Volume Recognition Sequence.
const VRS_START: u32 = 16;
/// Volume-space block of the Main Volume Descriptor Sequence.
pub(super) const MAIN_VDS_START: u32 = 32;
/// Volume-space block of the Reserve Volume Descriptor Sequence.
pub(super) const RESERVE_VDS_START: u32 = 48;
/// Sectors reserved for each VDS. ECMA-167 3/10.2.1 requires an anchor to
/// record at least 16.
pub(super) const VDS_SECTORS: u32 = 16;
/// Volume-space block of the Logical Volume Integrity Sequence.
pub(super) const LVID_START: u32 = 64;
/// Sectors reserved for the integrity sequence (LVID + TD).
pub(super) const LVID_SECTORS: u32 = 2;
/// The mandatory anchor block (ECMA-167 3/10.2).
pub(super) const ANCHOR_LBA: u32 = 256;
/// First block a partition may start at. Everything above is volume space.
pub(super) const MIN_PART_START: u32 = 320;
/// Emit the six-descriptor Volume Descriptor Sequence at `start`.
fn write_vds(out: &mut MetaSectors, layout: &Layout, start: u32) {
let vid = &layout.volume_id;
out.insert(start, primary_volume(vid, start, 1));
out.insert(start + 1, impl_use_volume(vid, start + 1, 2));
out.insert(
start + 2,
partition(layout.part_start, layout.part_sectors, start + 2, 3),
);
out.insert(
start + 3,
logical_volume(vid, 0, LVID_START, LVID_SECTORS, start + 3, 4),
);
out.insert(start + 4, unallocated_space(start + 4, 5));
out.insert(start + 5, terminating(start + 5));
}
/// Recursively emit one directory's File Entry and FID list, then its
/// children's.
fn write_dir(out: &mut MetaSectors, layout: &Layout, dir: &DirNode) -> Result<()> {
let mut fids = dir_fids(dir);
fix_fid_tag_locations(&mut fids, dir.data_lba);
debug_assert_eq!(fids.len(), dir.data_bytes as usize);
// A directory's link count is 1 (its own FID in the parent) plus one for
// each child directory's parent FID pointing back at it.
// The planner caps subdirectory fan-out (`layout::MAX_SUBDIRS`) so this
// cannot overflow; saturating rather than wrapping keeps a future change to
// that cap from silently producing a wrong count.
let link_count = (dir.dirs.len() as u16).saturating_add(1);
let fe = file_entry(
true,
fids.len() as u64,
&[(dir.data_lba, fids.len() as u32)],
link_count,
dir.unique_id,
dir.icb_lba,
)?;
out.insert(layout.part_start + dir.icb_lba, fe);
for (i, chunk) in fids.chunks(SECTOR).enumerate() {
let mut s = blank();
s[..chunk.len()].copy_from_slice(chunk);
out.insert(layout.part_start + dir.data_lba + i as u32, s);
}
for f in &dir.files {
let extents: Vec<(u32, u32)> = f.extents.iter().map(|e| (e.lba, e.bytes)).collect();
let fe = file_entry(false, f.size, &extents, 1, f.unique_id, f.icb_lba)?;
out.insert(layout.part_start + f.icb_lba, fe);
}
for sub in &dir.dirs {
write_dir(out, layout, sub)?;
}
Ok(())
}
/// Build every metadata sector of the synthesized volume.
pub(super) fn encode(layout: &Layout) -> Result<MetaSectors> {
let mut out = MetaSectors::new();
out.insert(VRS_START, volume_recognition(b"BEA01"));
out.insert(VRS_START + 1, volume_recognition(b"NSR02"));
out.insert(VRS_START + 2, volume_recognition(b"TEA01"));
write_vds(&mut out, layout, MAIN_VDS_START);
write_vds(&mut out, layout, RESERVE_VDS_START);
out.insert(
LVID_START,
integrity(
layout.part_sectors,
layout.file_count,
layout.dir_count,
layout.next_unique_id,
LVID_START,
),
);
out.insert(LVID_START + 1, terminating(LVID_START + 1));
let avdp = anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, ANCHOR_LBA);
out.insert(ANCHOR_LBA, avdp);
let last = layout.total_sectors - 1;
out.insert(
last,
anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, last),
);
// Partition block 0 must hold the File Set Descriptor: `read_filesystem`
// reads exactly `metadata_start` (== partition start on a single-partition
// volume) and rejects the volume outright if the tag there is not 256.
out.insert(
layout.part_start,
file_set(&layout.volume_id, layout.root.icb_lba, 0),
);
out.insert(layout.part_start + 1, terminating(1));
write_dir(&mut out, layout, &layout.root)?;
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
/// The reference check value for CRC-16/XMODEM — poly 0x1021 seeded at 0,
/// which is what ECMA-167 7.2.4 specifies: "123456789" → 0x31C3. Seeding
/// at 0xFFFF instead (CCITT-FALSE) yields 0x29B1, and that mutant is
/// invisible to `udf.rs`, which never verifies a tag CRC — it would only
/// show up as a volume no operating system will mount.
#[test]
fn crc16_matches_the_ecma167_check_value() {
assert_eq!(crc16(b"123456789"), 0x31C3);
assert_ne!(crc16(b"123456789"), 0x29B1, "not the 0xFFFF-seeded variant");
}
/// ECMA-167 3/7.2.3: the checksum is the sum of the tag's first 16 bytes
/// EXCLUDING the checksum byte itself, modulo 256.
#[test]
fn tag_checksum_excludes_its_own_byte() {
let mut buf = [0u8; 512];
buf[16..24].copy_from_slice(&[1, 2, 3, 4, 5, 6, 7, 8]);
finish_tag(&mut buf, 261, 0x1234, 512);
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
assert_eq!(buf[4] as u32, sum % 256);
// And the recorded CRC covers the body, not the tag.
let crc = u16::from_le_bytes([buf[8], buf[9]]);
assert_eq!(crc, crc16(&buf[16..512]));
assert_eq!(u16::from_le_bytes([buf[10], buf[11]]), 496);
assert_eq!(
u32::from_le_bytes([buf[12], buf[13], buf[14], buf[15]]),
0x1234
);
}
/// ASCII takes compression ID 8; anything above takes 16 (UTF-16BE),
/// because `parse_udf_name` decodes compression-8 bytes as UTF-8.
#[test]
fn cs0_picks_the_encoding_the_parser_can_decode() {
assert_eq!(encode_cs0("AB"), vec![8, b'A', b'B']);
let e = encode_cs0("Ä");
assert_eq!(e[0], 16);
assert_eq!(&e[1..], &[0x00, 0xC4]);
assert_eq!(crate::udf::parse_udf_name(&e), "Ä");
}
/// A d-string records its used length in the field's LAST byte, and the
/// production parser must read the same string back.
#[test]
fn dstring_round_trips_through_the_production_parser() {
let mut field = [0u8; 32];
put_dstring(&mut field, "FREEMKV");
assert_eq!(field[31], 8, "compid byte + 7 characters");
assert_eq!(crate::udf::parse_dstring_for_test(&field), "FREEMKV");
}
}
File diff suppressed because it is too large Load Diff
+362
View File
@@ -0,0 +1,362 @@
//! `dir://` as an image-level SOURCE: a synthetic UDF volume over a folder.
//!
//! A user's extracted disc — a DVD `VIDEO_TS/` or a Blu-ray `BDMV/`, typically
//! a MakeMKV-style backup — has files but no sectors, and everything above the
//! sector layer in this crate wants sectors: `Disc::scan_image`, `UdfFs`,
//! `ifo.rs`, `mpls.rs`, `clpi.rs` and the mux all read through a
//! [`SectorSource`]. [`DirImage`] supplies one.
//!
//! The trick is that nothing is emulated. A real, minimal, valid UDF 1.02
//! volume is synthesized over the folder:
//!
//! * **Metadata sectors** (anchors, the volume descriptor sequences, the File
//! Set Descriptor, every File Entry, every directory's FID list) are encoded
//! into RAM by [`encode`] — a few MiB even for a large Blu-ray.
//! * **Data sectors** are not materialized at all. Each one maps to a byte
//! range of a real file, read on demand.
//!
//! So `udf::read_filesystem` parses this image by exactly the same code path it
//! parses a real disc with, and every consumer above it is unchanged. The cost
//! is that a single-partition synthetic volume never exercises the UDF 2.50
//! Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
//! this module's tests do not cover that block and must not be read as if they
//! did.
//!
//! What this module deliberately does NOT do:
//!
//! * **3D / SSIF** — rejected up front ([`Error::DirImageSsifUnsupported`]).
//! An SSIF aliases the same sectors as its base and dependent `.m2ts`; the
//! planner allocates disjoint extents, so a 3D folder would produce silently
//! wrong output.
//! * **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
//! * **Encrypted folders** — a folder whose content is still AACS-scrambled is
//! rejected by the caller-side probe, not decrypted here.
mod encode;
mod layout;
use crate::error::{Error, Result};
#[cfg(target_os = "linux")]
use crate::io::file_sector_source::linux::drop_window;
#[cfg(target_os = "macos")]
use crate::io::file_sector_source::macos::drop_window;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use crate::io::file_sector_source::other::drop_window;
#[cfg(target_os = "windows")]
use crate::io::file_sector_source::windows::drop_window;
use crate::sector::SectorSource;
use encode::{MetaSectors, SECTOR};
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::{Path, PathBuf};
/// How many host files may be held open at once.
///
/// A Blu-ray `BDMV/` can exceed a thousand files while macOS `RLIMIT_NOFILE`
/// defaults to 256, so "open every file up front" is not available. Reads are
/// overwhelmingly sequential through one large stream file at a time, so a
/// small LRU keeps the hit rate near 1 while bounding descriptors.
const HANDLE_CACHE: usize = 16;
/// One file's bytes at one place in the image.
#[derive(Debug, Clone)]
struct DataRange {
/// Absolute first block.
start_lba: u32,
/// Blocks covered (the last one may be partially used, and is zero-padded).
sectors: u32,
/// Index into [`DirImage::files`].
file: usize,
/// Byte offset within the file at which this range's bytes begin.
offset: u64,
/// Byte length of the range.
bytes: u64,
}
/// A file the image reads through.
#[derive(Debug)]
struct FileRef {
host: PathBuf,
disc_path: String,
size: u64,
/// Host mtime at plan time — see `layout::FileNode::mtime` for why size
/// alone is not enough.
mtime: Option<std::time::SystemTime>,
}
/// A synthesized UDF disc image over a host directory.
///
/// Owns everything it reads through (`PathBuf`s and its own file handles), so
/// it is `Send + 'static` and can be moved into `build_iso_pipeline`, which
/// hands it to `PrefetchedSectorSource`'s producer thread.
pub struct DirImage {
meta: MetaSectors,
/// Sorted by `start_lba`, non-overlapping.
ranges: Vec<DataRange>,
files: Vec<FileRef>,
open: Vec<(usize, File)>,
total_sectors: u32,
volume_id: String,
data_bytes: u64,
}
impl std::fmt::Debug for DirImage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DirImage")
.field("volume_id", &self.volume_id)
.field("total_sectors", &self.total_sectors)
.field("files", &self.files.len())
.field("meta_sectors", &self.meta.len())
.finish()
}
}
impl DirImage {
/// Plan and encode an image over `root`.
///
/// Every error is decided here, at plan time, where it can name the file
/// responsible — the read path is deliberately left with nothing to decide
/// except "this file changed underneath me".
pub fn open(root: &Path) -> Result<Self> {
let plan = layout::plan(root)?;
let meta = encode::encode(&plan)?;
let mut nodes = Vec::new();
layout::flatten(&plan.root, &mut nodes);
let mut files = Vec::with_capacity(nodes.len());
let mut ranges = Vec::new();
for (idx, node) in nodes.iter().enumerate() {
// Carry the plan-time mtime ONLY for files whose CONTENT the plan
// read — the DVD IFOs, whose bytes 0xC0/0xC4 decide where every VOB
// is placed (`layout::place_video_ts` -> `read_head`).
//
// For every other file the plan depends on the SIZE alone, and size
// is already checked. Comparing mtime on those buys nothing and
// costs real false positives: disc backups commonly live on
// exFAT/FAT32, which stores local time, so a long rip spanning a
// DST transition sees a whole-hour shift on a file nobody touched
// and would abort hours in, blaming a change that did not happen.
// The multi-gigabyte VOBs are exactly the files a long rip re-opens
// after the handle cache evicts them.
let content_sensitive = node
.disc_path
.rsplit('.')
.next()
.is_some_and(|e| e.eq_ignore_ascii_case("IFO"));
files.push(FileRef {
host: node.host.clone(),
disc_path: node.disc_path.clone(),
size: node.size,
mtime: content_sensitive.then_some(node.mtime).flatten(),
});
let mut offset = 0u64;
for e in &node.extents {
ranges.push(DataRange {
start_lba: plan.part_start + e.lba,
sectors: (e.bytes as u64).div_ceil(SECTOR as u64) as u32,
file: idx,
offset,
bytes: e.bytes as u64,
});
offset += e.bytes as u64;
}
}
ranges.sort_by_key(|r| r.start_lba);
debug_assert!(
ranges
.windows(2)
.all(|w| w[0].start_lba + w[0].sectors <= w[1].start_lba),
"planned data ranges must not overlap"
);
let data_bytes = layout::total_data_bytes(&plan.root);
tracing::info!(
target: "freemkv::dirimage",
volume_id = %plan.volume_id,
files = files.len(),
dirs = plan.dir_count,
meta_blocks = layout::metadata_block_count(&plan.root),
total_sectors = plan.total_sectors,
"synthesized UDF image over directory"
);
Ok(Self {
meta,
ranges,
files,
open: Vec::new(),
total_sectors: plan.total_sectors,
volume_id: plan.volume_id,
data_bytes,
})
}
/// UDF volume identifier the image declares (the folder's own name).
pub fn volume_id(&self) -> &str {
&self.volume_id
}
/// Total bytes of real file content the image carries — the folder's size,
/// not the image's (which also counts metadata and inter-file gaps).
pub fn data_bytes(&self) -> u64 {
self.data_bytes
}
/// The range covering `lba`, if any.
fn range_at(&self, lba: u32) -> Option<&DataRange> {
let i = self.ranges.partition_point(|r| r.start_lba <= lba);
let r = self.ranges.get(i.checked_sub(1)?)?;
(lba < r.start_lba + r.sectors).then_some(r)
}
/// Borrow an open handle for `file`, opening it (and evicting the
/// least-recently-used handle) if necessary.
///
/// Opening is also where the plan is revalidated. A folder is not a disc:
/// a file can be shortened or replaced between planning and reading, and
/// zero-filling the difference would turn "the user deleted something"
/// into corrupt output at exit 0. The size is re-checked here, and a
/// truncation that happens while the handle is already open is caught by
/// the short read in [`Self::fill`].
fn handle(&mut self, file: usize) -> Result<&mut File> {
if let Some(pos) = self.open.iter().position(|(i, _)| *i == file) {
// `open` is ordered most-recently-used first.
let entry = self.open.remove(pos);
self.open.insert(0, entry);
return Ok(&mut self.open[0].1);
}
let f = File::open(&self.files[file].host).map_err(Error::from)?;
let md = f.metadata().map_err(Error::from)?;
// Size AND mtime. Size alone is content-blind, and this plan depends on
// content: a DVD's VOB placement comes from bytes 0xC0/0xC4 of its IFO,
// and an IFO rewritten in place keeps its length because IFOs occupy a
// whole number of sectors. The size check would pass while every title
// extent pointed at the wrong sectors — corrupt video behind an intact
// structure, reported complete at exit 0.
//
// Only compared when both sides have a timestamp; a platform or
// filesystem that reports none simply falls back to the size check
// rather than failing every read.
let changed_size = md.len() != self.files[file].size;
let changed_mtime = match (self.files[file].mtime, md.modified().ok()) {
(Some(planned), Some(live)) => planned != live,
_ => false,
};
if changed_size || changed_mtime {
return Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
});
}
if self.open.len() >= HANDLE_CACHE {
self.open.pop();
}
self.open.insert(0, (file, f));
Ok(&mut self.open[0].1)
}
/// Fill `out` (a whole number of sectors) from one data range, starting at
/// `lba`. `out` is already zeroed, so a file's tail sector comes back
/// zero-padded — which is exactly what `file_extents`' `div_ceil(2048)`
/// (`udf.rs:816`) makes every consumer expect.
fn fill(&mut self, r: &DataRange, lba: u32, out: &mut [u8]) -> Result<()> {
let within = (lba - r.start_lba) as u64 * SECTOR as u64;
let want = (r.bytes.saturating_sub(within)).min(out.len() as u64) as usize;
if want == 0 {
return Ok(());
}
let at = r.offset + within;
let file = r.file;
let h = self.handle(file)?;
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Release the window just read, every time.
//
// The ISO source accumulates and drops in chunks because it reads
// one file linearly, so a running start offset always names the
// bytes it has consumed. Reads here jump between files, so there is
// no single cursor to accumulate against — an accumulated byte
// count paired with one read's offset names 1/Nth of what was
// actually consumed and leaves the rest pinned, which is how the
// first version of this got it wrong.
//
// Dropping per read costs one advisory syscall per batch (4-16 MiB),
// which is nothing against the read itself, and it is correct
// regardless of how reads interleave across files.
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
}
match res {
Ok(()) => Ok(()),
// The file shrank while the handle was open. Same verdict as the
// size check in `handle`, reached the other way.
Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
})
}
Err(e) => Err(Error::from(e)),
}
}
}
impl SectorSource for DirImage {
fn capacity_sectors(&self) -> u32 {
self.total_sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let need = count as usize * SECTOR;
if buf.len() < need {
return Err(Error::UdfBufferTooSmall);
}
buf[..need].fill(0);
// Walk the request in RUNS, not sector by sector. A mux batch is 8192
// sectors and almost always lands entirely inside one stream file's
// extent; per-sector seek+read would issue 8192 syscalls for what is
// one 16 MiB sequential read.
let mut i = 0u32;
while i < count as u32 {
// Checked: callers saturate their LBAs (`disc/dvd.rs` builds a cell
// start as `vob_start_sector.saturating_add(cell.first_sector)`, and
// the prefetcher adds an offset the same way), so a crafted IFO can
// present a request at the very top of the address space. Wrapping
// here would fold `at` back to a LOW sector and hand the muxer a
// different file's bytes with nothing reported.
let Some(at) = lba.checked_add(i) else {
break;
};
let off = i as usize * SECTOR;
if let Some(s) = self.meta.get(&at) {
buf[off..off + SECTOR].copy_from_slice(&s[..]);
i += 1;
continue;
}
// Metadata blocks all sit below the data floor, so a data range is
// never interrupted by one.
match self.range_at(at).cloned() {
Some(r) => {
let run = (r.start_lba + r.sectors - at).min(count as u32 - i);
let end = off + run as usize * SECTOR;
self.fill(&r, at, &mut buf[off..end])?;
i += run;
}
// A gap between planned extents. Reads as zeros, exactly as an
// unrecorded sector of a real image does.
None => i += 1,
}
}
Ok(need)
}
}
#[cfg(test)]
mod tests;
File diff suppressed because it is too large Load Diff
+1465 -331
View File
File diff suppressed because it is too large Load Diff
+214 -22
View File
@@ -7,13 +7,32 @@ use crate::udf;
impl Disc {
/// Scan DVD titles from IFO files (VIDEO_TS.IFO + VTS_XX_0.IFO).
///
/// Cancellation: `halt` is polled before the IFO tree is read, and an IFO
/// read that fails with [`Error::Halted`] — how a live drive reports a
/// Stop, since `Drive::checked_exec` fails EVERY command once its flag is
/// set — is propagated rather than swallowed. Every other IFO failure
/// keeps its best-effort `Ok(vec![])`.
///
/// It has to be an error and not an empty title list, for the same reason
/// spelled out on [`Disc::scan_hddvd_titles`]: a cancelled enumeration
/// that returned `Ok` would be indistinguishable from a disc that
/// genuinely holds fewer titles. This one was the worst of the three
/// enumerators — a bare `Err(_) => return Vec::new()` turned an operator
/// Stop into ZERO titles at rc=0, a disc reported as carrying no video at
/// all.
pub(super) fn scan_dvd_titles(
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Vec<DiscTitle> {
halt: Option<&crate::halt::Halt>,
) -> Result<Vec<DiscTitle>> {
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
let dvd_info = match ifo::parse_vmg(reader, udf_fs) {
Ok(info) => info,
Err(_) => return Vec::new(),
Err(Error::Halted) => return Err(Error::Halted),
Err(_) => return Ok(Vec::new()),
};
let mut titles = Vec::new();
@@ -179,7 +198,10 @@ impl Disc {
// the coded video frame the subpicture was authored against
// (720x480 NTSC / 720x576 PAL) so players place and scale the
// bitmap correctly.
let (vid_w, vid_h) = ts.video.resolution.pixels();
// format_palette guards on (0, 0) and omits its `size:` line,
// so an unresolved resolution degrades to a palette-only .idx
// rather than one claiming a 0x0 frame.
let (vid_w, vid_h) = ts.video.resolution.pixels().unwrap_or((0, 0));
let codec_data = dvd_title
.palette
.as_ref()
@@ -240,7 +262,14 @@ impl Disc {
}
}
titles
// Polled again AFTER the loop: a cancel raised during the IFO reads
// that `parse_vmg` performs per title set has nothing left to poll,
// so without this a partially enumerated disc could still be handed
// back as success.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(Error::Halted);
}
Ok(titles)
}
}
@@ -525,6 +554,118 @@ mod tests {
// Tests
// ---------------------------------------------------------------
/// A `SectorSource` that fails every read at or above `halt_at` with
/// [`Error::Halted`] — how a LIVE DRIVE behaves once the operator presses
/// Stop: `Drive::checked_exec` fails every SCSI command with `Halted` from
/// then on, and `Drive::read` deliberately preserves the variant. Reads
/// below the threshold still succeed, so the scan gets far enough to have
/// something to truncate.
struct HaltingReader<'a> {
inner: &'a mut MemDisc,
halt_at: u32,
}
impl SectorSource for HaltingReader<'_> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> crate::error::Result<usize> {
if lba >= self.halt_at {
return Err(crate::error::Error::Halted);
}
self.inner.read_sectors(lba, count, buf, recovery)
}
}
/// A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has
/// its own flag and `checked_exec` fails every SCSI command with
/// [`Error::Halted`] once it is set. The DVD enumerator must not swallow
/// that into a successful scan.
///
/// This was the worst of the three enumerators. RED BEFORE GREEN, two
/// distinct swallows, both measured with the fix reverted:
/// * `ifo::parse_vmg` treats a failed title set as a placeholder entry
/// and continues, so a cancel landing on VTS_02's IFO returned
/// `Ok([VTS_01_1.VOB])` — one title from a two-title disc.
/// * `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel
/// landing on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc
/// reported as holding no video at all.
///
/// Both are indistinguishable from a real disc, and both are now
/// `Err(Error::Halted)`.
#[test]
fn halted_ifo_read_is_not_reported_as_a_shorter_disc() {
// Two title sets: VTS_01's IFO data at PART_START+6000, VTS_02's at
// PART_START+7000. Both ICBs sit far below, so the filesystem
// metadata resolves and only the second title set's CONTENT is
// cancelled — the truncation case.
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1), (1, 2, 1)]);
let vts1 = build_vts(100, 0x00, &[], &[], &[(0, 9)], false);
let vts2 = build_vts(200, 0x00, &[], &[], &[(0, 19)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts1,
},
FileSpec {
name: "VTS_02_0.IFO".into(),
icb_lba: 64,
data_lba: 7000,
contents: vts2,
},
],
);
// Sanity: both title sets enumerate when nothing is cancelled, so a
// short list below can only be the cancel.
assert_eq!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.len(),
2,
"fixture must offer two title sets"
);
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 7000, // VTS_02_0.IFO's data extent
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled title-set read must surface as a cancelled scan, not \
as a disc with fewer titles; got {:?}",
res.map(|ts| ts.iter().map(|t| t.playlist.clone()).collect::<Vec<_>>())
);
// The same cancel one level up: VIDEO_TS.IFO itself. This is the
// `Err(_) => Vec::new()` path — a cancel that used to report a DVD as
// carrying no titles whatsoever.
let mut reader = HaltingReader {
inner: &mut disc,
halt_at: PART_START + 5000,
};
let res = Disc::scan_dvd_titles(&mut reader, &udf, None);
assert!(
matches!(res, Err(crate::error::Error::Halted)),
"a cancelled VMG read must surface as a cancelled scan, not as an \
empty disc; got {:?}",
res.map(|ts| ts.len())
);
}
/// scan_dvd_titles returns empty when VIDEO_TS.IFO can't be parsed
/// (dvd.rs: `parse_vmg(...) Err → return Vec::new()`). Never panics.
#[test]
@@ -532,7 +673,11 @@ mod tests {
let mut disc = MemDisc::new();
// VIDEO_TS exists but VIDEO_TS.IFO is missing.
let udf = build_video_ts_fs(&mut disc, &[]);
assert!(Disc::scan_dvd_titles(&mut disc, &udf).is_empty());
assert!(
Disc::scan_dvd_titles(&mut disc, &udf, None)
.expect("scan")
.is_empty()
);
}
/// Single VTS, single title, one cell. Extent absolute LBA =
@@ -568,7 +713,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -624,7 +769,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 1);
let t = &titles[0];
assert_eq!(t.extents.len(), 1);
@@ -682,7 +827,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
@@ -740,7 +885,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100);
@@ -781,7 +926,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -836,7 +981,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let v = t
.streams
.iter()
@@ -897,7 +1042,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -908,7 +1053,7 @@ mod tests {
.collect();
assert_eq!(audios.len(), 2);
assert_eq!(audios[0].codec, Codec::Ac3);
assert_eq!(audios[0].language, "en");
assert_eq!(audios[0].language, "eng");
assert_eq!(audios[1].codec, Codec::Dts);
// Real channel layouts survive the scan (not a 1ch placeholder): the
// AC-3 is 5.1 (6ch), the DTS is 2.0 (2ch).
@@ -967,7 +1112,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let audios: Vec<_> = t
.streams
.iter()
@@ -1024,7 +1169,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let subs: Vec<_> = t
.streams
.iter()
@@ -1037,7 +1182,7 @@ mod tests {
// Languages preserved in order.
assert_eq!(
subs.iter().map(|s| s.language.as_str()).collect::<Vec<_>>(),
vec!["en", "fr", "de"]
vec!["eng", "fra", "deu"]
);
// PIDs are 0x20 + ordinal, all distinct.
let pids: Vec<u16> = subs.iter().map(|s| s.pid).collect();
@@ -1084,7 +1229,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
let sub = t
.streams
.iter()
@@ -1094,7 +1239,7 @@ mod tests {
})
.expect("subtitle stream");
assert_eq!(sub.codec, Codec::DvdSub);
assert_eq!(sub.language, "en");
assert_eq!(sub.language, "eng");
assert!(
sub.codec_data.is_some(),
"non-zero palette must yield codec_data"
@@ -1138,7 +1283,7 @@ mod tests {
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
assert_eq!(titles.len(), 2);
// title_number is a running counter across all title sets.
assert_eq!(titles[0].playlist_id, 1);
@@ -1175,7 +1320,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// One program in the program map → one chapter time (0.0 for the
// first program). Name is the ordinal from chapter_name(0).
assert_eq!(t.chapters.len(), 1);
@@ -1260,7 +1405,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
@@ -1312,7 +1457,7 @@ mod tests {
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
let t = &Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan")[0];
// Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
@@ -1320,4 +1465,51 @@ mod tests {
// Chapter 0 stays at 0.0 (no shift).
assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01);
}
/// Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio
/// stream has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio,
/// per `ifo::assign_audio_sub_stream_ids` — the PID falls back to
/// `0xBD00 + i` where `i` is the stream's positional index in the IFO
/// audio-attribute table. Two MPEG-audio (coding_mode 2) streams must
/// land on two DISTINCT, correctly-offset PIDs: 0xBD00 and 0xBD01. This
/// pins the `+` (not `-`/`*`) so the second stream doesn't collide with,
/// or wrap under, the first.
#[test]
fn scan_dvd_titles_mp2_audio_pid_fallback_is_additive() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// coding_mode bits are b0>>5 & 0x7; mode 2 = MPEG-1 Layer II (Mp2),
// which `assign_audio_sub_stream_ids` leaves at `sub_stream_id: None`.
// b0 = 0b010_00000 = 0x40. b1 = 0 (mono, sample rate 48k).
let audio = [(0x40u8, 0x00u8, [0u8, 0u8]), (0x40u8, 0x00u8, [0u8, 0u8])];
let vts = build_vts(1000, 0x00, &audio, &[], &[(10, 109)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts,
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf, None).expect("scan");
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
.iter()
.filter_map(|s| match s {
Stream::Audio(a) => Some(a.pid),
_ => None,
})
.collect();
assert_eq!(audio_pids, vec![0xBD00u16, 0xBD01u16]);
}
}
+173 -5
View File
@@ -104,10 +104,10 @@ fn max_substream_channels(data: &[u8]) -> Option<u8> {
};
let start = pos + rel;
let frame = &data[start..];
if let Some(ch) = ac3::acmod_channels(frame) {
if ch > 0 {
best = Some(best.map_or(ch, |b| b.max(ch)));
}
if let Some(ch) = ac3::acmod_channels(frame)
&& ch > 0
{
best = Some(best.map_or(ch, |b| b.max(ch)));
}
// Advance past this frame by its declared size when that is mappable;
// otherwise step 2 bytes past the sync and re-scan for the next one.
@@ -242,7 +242,11 @@ pub fn probe_and_remap<S: SectorSource + ?Sized>(
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate};
use crate::disc::{
AudioChannels, AudioStream, Codec, ContentFormat, DiscTitle, Extent, LabelPurpose,
SampleRate,
};
use crate::sector::SectorSource;
/// Build a single, correctly-SIZED AC-3 frame whose `acmod`/`lfeon` encode a
/// known channel count. `byte4` is `fscod=0 | frmsizecod=0`, so
@@ -463,4 +467,168 @@ mod tests {
};
assert_eq!(a.pid, 0xBD80, "no probe data → keep ordinal");
}
/// `max_substream_channels` must locate the sync at its true ABSOLUTE
/// position (`pos + rel`) when it is preceded by non-sync bytes, not just
/// when the sync sits at offset 0. Regression guard for a hand-checked
/// mutation (`+` → `-` at the `pos + rel` offset computation): with `pos`
/// starting at 0 and the first sync found 3 bytes in, `pos - rel` would
/// underflow a `usize` and panic, or (if it somehow didn't) index the
/// wrong start entirely. `pos + rel` is the only computation that is
/// always in-bounds, since `rel` is itself bounded by the length of the
/// slice searched from `pos`.
#[test]
fn max_substream_channels_locates_sync_after_leading_non_sync_bytes() {
let mut data = vec![0xAA, 0xAA, 0xAA]; // no 0x0B77 pattern in here
data.extend(ac3_frame(2, false)); // real 2.0 frame, sync at absolute offset 3
assert_eq!(
max_substream_channels(&data),
Some(2),
"must find and decode the frame whose sync is NOT at offset 0"
);
}
/// When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
/// `fscod == 3`), `max_substream_channels` must fall back to stepping
/// `start + 2` bytes past the sync to re-lock onto the next genuine sync,
/// and must keep making forward progress doing so (never revisit the same
/// sync, which would loop forever, and never jump so far that it skips
/// the very next real frame). This lays a bogus-sized header at absolute
/// offset 4 (so `start == 4`, `start + 2 == 6`) immediately followed, at
/// offset 6, by a real, fully decodable 2.0 frame — the position the
/// `+ 2` fallback must land on exactly.
#[test]
fn max_substream_channels_unmappable_size_steps_forward_by_two() {
let mut real = ac3_frame(2, false);
// Overwrite the (unchecked) CRC bytes of the real frame — these double
// as byte4/byte5 of the bogus header 2 bytes earlier, at absolute
// offset 4: byte4 = 0xC0 (fscod=3 reserved -> ac3_frame_size == 0,
// unmappable), byte5 = 0xF8 (bsid=31 >= 11 -> acmod_channels == None,
// so the bogus header itself never contributes a spurious channel
// count).
real[2] = 0xC0;
real[3] = 0xF8;
let mut data = vec![0xAA, 0xAA, 0xAA, 0xAA]; // offsets 0..4, no sync
data.push(0x0B); // offset 4: bogus header sync byte 0
data.push(0x77); // offset 5: bogus header sync byte 1
data.extend(real); // offset 6..: the real frame (also serves as the
// bogus header's byte4/byte5 at offsets 8/9)
assert_eq!(
max_substream_channels(&data),
Some(2),
"must recover the real frame 2 bytes after the unmappable-size sync, not lose it"
);
}
/// Same fallback as above, but with the unmappable-size sync at absolute
/// offset 0 (`start == 0`) so that stepping backward instead of forward
/// (`start - 2`) would underflow rather than merely land on the wrong
/// byte. Also proves the real frame is still found 6 bytes further in,
/// confirming forward progress past the bogus header.
#[test]
fn max_substream_channels_unmappable_size_at_start_steps_forward_not_back() {
let mut data = vec![0x0B, 0x77, 0x00, 0x00, 0xC0, 0xF8]; // bogus header, offsets 0..6
data.extend(ac3_frame(2, false)); // real 2.0 frame at offset 6
assert_eq!(
max_substream_channels(&data),
Some(2),
"must step forward past the bogus header at offset 0 and find the real frame at offset 6"
);
}
/// `remap_audio_pids` must read a stream's CURRENT physical sub-stream id
/// from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
/// `0x00FF`, both of which force the low byte to `0xFF` regardless of the
/// real PID and so always miss the "already matches" shortcut. That
/// matters observably when TWO physical sub-streams share the same probed
/// channel count: with a correct read, a stream already sitting on a
/// matching sub-stream is left alone (conservative, per the module's
/// documented behaviour); with the low byte forced to `0xFF`,
/// `probed.get(&0xFF)` is always `None`, so the code falls through to the
/// "find any unclaimed match" path and picks the FIRST (lowest-keyed,
/// BTreeMap-ordered) matching physical sub-stream instead — which here is
/// a *different* sub-stream (0x80) than the one the PID already correctly
/// names (0x81), producing a spurious PID change.
#[test]
fn remap_reads_current_substream_via_and_not_or_or_xor() {
let mut probed = BTreeMap::new();
probed.insert(0x80u8, 6u8);
probed.insert(0x81u8, 6u8); // ambiguous: two physical 6ch sub-streams
let mut streams = vec![ac3_stream(0xBD81, AudioChannels::Surround51)];
let changed = remap_audio_pids(&mut streams, &probed);
assert_eq!(
changed, 0,
"already sitting on a matching physical sub-stream (0x81) must be left alone"
);
let Stream::Audio(a) = &streams[0] else {
panic!()
};
assert_eq!(
a.pid, 0xBD81,
"must not be bumped to the other matching sub-stream (0x80)"
);
}
/// A `SectorSource` stub that hands back fixed bytes regardless of the
/// requested LBA/count, for exercising `probe_and_remap`'s end-to-end
/// wiring (format/AC-3/extent/count guards -> read -> probe -> remap).
struct FixedSource {
data: Vec<u8>,
}
impl SectorSource for FixedSource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let n = self.data.len().min(buf.len());
buf[..n].copy_from_slice(&self.data[..n]);
Ok(n)
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
/// stream to 0xBD81. It also, by construction, proves each of the guards
/// along the way lets a real, positive case through: the content-format
/// check must NOT bail on `MpegPs` (only on non-`MpegPs`), the AC-3
/// presence check must NOT bail when AC-3 IS present, and the
/// sector-count check must NOT bail when the count is nonzero — any one
/// of those inverted would skip the probe entirely and leave the PID at
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
playlist: "00001.ifo".into(),
playlist_id: 1,
duration_secs: 60.0,
size_bytes: bytes.len() as u64,
clips: Vec::new(),
streams: vec![ac3_stream(0xBD80, AudioChannels::Surround51)],
chapters: Vec::new(),
extents: vec![Extent {
start_lba: 0,
sector_count: 2,
}],
content_format: ContentFormat::MpegPs,
codec_privates: vec![None],
};
let mut source = FixedSource { data: bytes };
probe_and_remap(&mut source, &mut title);
let Stream::Audio(a) = &title.streams[0] else {
panic!("audio")
};
assert_eq!(
a.pid, 0xBD81,
"declared 5.1 stream must be re-routed to the physical 5.1 sub-stream 0x81"
);
}
}
+140 -72
View File
@@ -7,7 +7,6 @@ use crate::udf;
/// Result of SCSI AACS handshake (ECDH authentication).
/// Only available when scanning from a real drive, not ISO images.
#[derive(Debug)]
pub(super) struct HandshakeResult {
pub volume_id: [u8; 16],
pub read_data_key: Option<[u8; 16]>,
@@ -29,6 +28,19 @@ pub(super) struct HandshakeResult {
pub drive_unlocked: bool,
}
// Redacting `Debug`: `volume_id` and `read_data_key` (the AACS 2.0 bus key) are
// secret; print only shape. Guarded by `handshake_result_debug_is_redacted`.
impl std::fmt::Debug for HandshakeResult {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HandshakeResult")
.field("volume_id", &"<redacted>")
.field("read_data_key", &self.read_data_key.map(|_| "<redacted>"))
.field("read_data_key_err", &self.read_data_key_err)
.field("drive_unlocked", &self.drive_unlocked)
.finish()
}
}
/// Single source of truth for "is AACS bus encryption gone for this scan?". The
/// gate asks ONLY this — `if !removed { error }` — never enumerating cases. Bus
/// encryption is gone when ANY of these holds:
@@ -86,9 +98,9 @@ impl AacsCertUnlocker<'_> {
// MKB generation (best-effort) — forwarded to each source's
// `host_certs(mkb)` so a source MAY select a generation-appropriate cert
// (the default impl ignores it). A read failure leaves it `None`.
let mkb_gen = aacs::read_mkb_from_drive(session.scsi_mut())
let mkb_gen = aacs::inf::read_mkb_from_drive(session.scsi_mut())
.ok()
.and_then(|m| aacs::mkb_version(&m));
.and_then(|m| aacs::mkb::mkb_version(&m));
// Host certs are keysource-served, never compiled in — unioned from the
// explicit `DriveCredentials` and the key-source layer. With ZERO certs
@@ -111,13 +123,13 @@ impl AacsCertUnlocker<'_> {
// through method calls, so clone the (cheap) identity first.
let drive_id = session.drive_id.clone();
let fu_certs = crate::unlock_bridge::map_host_certs(&host_certs);
let unlocked = crate::unlock_bridge::run_unlockers(
let (_, unlock_res) = crate::unlock_bridge::run_bus(
session.scsi_mut(),
&drive_id,
freemkv_unlock::DiscKind::Aacs,
&fu_certs,
)
.map_err(CertUnlockFailure::Unlock)?;
);
let unlocked = unlock_res.map_err(CertUnlockFailure::Unlock)?;
// The cert handshake yields a VID on success; its absence is VidUnavailable.
let Some(volume_id) = unlocked.vid else {
return Err(CertUnlockFailure::Unlock(UnlockError::VidUnavailable));
@@ -152,10 +164,10 @@ fn unlock_error_to_error(e: &CertUnlockFailure) -> Error {
}
}
/// Map a [`CertUnlockFailure`] to a structured [`crate::aacs::UnlockOutcome`]
/// Map a [`CertUnlockFailure`] to a structured [`crate::aacs::trace::UnlockOutcome`]
/// for the resolution trace (English-free).
fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::UnlockOutcome {
use crate::aacs::UnlockOutcome;
fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::trace::UnlockOutcome {
use crate::aacs::trace::UnlockOutcome;
use freemkv_unlock::UnlockError;
match e {
CertUnlockFailure::NoHostCert { mkb } => UnlockOutcome::NoUsableHostCert { mkb: *mkb },
@@ -169,6 +181,24 @@ fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::UnlockOutcome {
}
}
/// Did the cert handshake actually carry a Volume ID?
///
/// Extracted so it can be tested as a VALUE. It only ever reaches an operator
/// as the `has_volume_id` field of the `bus_key_unavailable` warn, and
/// asserting on a `tracing` field means installing a capturing subscriber —
/// which is thread-local, while `tracing`'s callsite-interest cache is global.
/// Those two facts race: the test failed roughly one run in ten under the full
/// parallel suite while passing every time in isolation, and serialising the
/// captures was not enough because the cache can be re-evaluated against the
/// process default rather than the thread-local dispatch.
///
/// A predicate this small does not need a subscriber to verify. The polarity is
/// the whole point: an `==` here would tell an operator a VID was absent on
/// exactly the discs where one was present.
fn handshake_has_volume_id(h: &HandshakeResult) -> bool {
h.volume_id != [0u8; 16]
}
impl Disc {
/// SCSI handshake — drives the VID-acquisition flow and returns
/// a structured `HandshakeResult` for downstream key resolution.
@@ -230,7 +260,10 @@ impl Disc {
/// `mkb` is the disc's MKB generation when known, forwarded to each source's
/// [`crate::KeySource::host_certs`] so a source MAY return only
/// generation-appropriate certs (the default ignores it).
fn collect_host_certs(opts: &ScanOptions, mkb: Option<u32>) -> Vec<crate::aacs::HostCert> {
fn collect_host_certs(
opts: &ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::types::HostCert> {
// Delegates to the shared cert primitive (the external freemkv-unlock-aacs
// plugin uses the same one). Kept as a thin Disc method so the existing
// collect_host_certs_* unit tests and call sites are unchanged.
@@ -308,18 +341,19 @@ impl Disc {
) -> Result<AacsState> {
use crate::aacs;
let uk_ro_data = udf_fs
.read_file(reader, crate::aacs::PATH_UNIT_KEY_RO)
.or_else(|_| udf_fs.read_file(reader, crate::aacs::PATH_UNIT_KEY_RO_DUPLICATE))
.map_err(|_| Error::AacsNoKeys)?;
let dh = aacs::disc_hash(&uk_ro_data);
let uk_ro_data =
aacs::read_first(&aacs::role_paths(udf_fs, aacs::AacsRole::UnitKey), |p| {
udf_fs.read_file(reader, p)
})?;
let dh = aacs::inf::disc_hash(&uk_ro_data);
let cc = udf_fs
.read_file(reader, crate::aacs::PATH_CONTENT_CERT)
.or_else(|_| udf_fs.read_file(reader, crate::aacs::PATH_CONTENT_CERT_ALT))
.ok()
.as_deref()
.and_then(aacs::parse_content_cert);
let cc = aacs::read_first(
&aacs::role_paths(udf_fs, aacs::AacsRole::ContentCert),
|p| udf_fs.read_file(reader, p),
)
.ok()
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let bus_encryption = cc.as_ref().map(|c| c.bus_encryption).unwrap_or(false);
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so the
// scanned `AacsState.version` and the out-of-band fetch agree. A wrong
@@ -328,7 +362,7 @@ impl Disc {
let version = cc
.as_ref()
.map(|c| c.version.major())
.unwrap_or(aacs::AACS_MAJOR_UHD);
.unwrap_or(aacs::mkb::AACS_MAJOR_UHD);
// Bus-encryption gate (wrong-keys guard). A bus-encrypted disc (Content
// Certificate bus-encryption bit set) carries bus encryption on its
@@ -353,7 +387,7 @@ impl Disc {
// file/ISO, drive unlock, cert bus key). The gate enumerates nothing.
if !bus_encryption_removed(bus_encryption, handshake) {
let (rdk_err, has_vid) = handshake
.map(|h| (h.read_data_key_err, h.volume_id != [0u8; 16]))
.map(|h| (h.read_data_key_err, handshake_has_volume_id(h)))
.unwrap_or((None, false));
tracing::warn!(
target: "freemkv::disc",
@@ -394,12 +428,12 @@ impl Disc {
Vec::new()
}
};
let mkb_ver = aacs::mkb_version(&mkb_bytes);
let mkb_ver = aacs::mkb::mkb_version(&mkb_bytes);
tracing::debug!(
target: "freemkv::disc",
phase = "scan_aacs_vid_only",
disc_hash = %aacs::disc_hash_hex(&dh),
disc_hash = %aacs::inf::disc_hash_hex(&dh),
version,
bus_encryption,
has_vid = handshake.is_some(),
@@ -410,7 +444,7 @@ impl Disc {
version,
bus_encryption,
mkb_version: mkb_ver,
disc_hash: aacs::disc_hash_hex(&dh),
disc_hash: aacs::inf::disc_hash_hex(&dh),
key_source: KeyOrigin::ExternalUk,
vuk: None,
unit_keys: vec![],
@@ -429,6 +463,27 @@ mod tests {
use crate::sector::SectorSource;
use std::collections::HashMap;
/// `HandshakeResult` carries the Volume ID and the AACS 2.0 bus (read-data)
/// key; `Debug` must redact both. Sentinel 213 (0xD5).
#[test]
fn handshake_result_debug_is_redacted() {
let hs = HandshakeResult {
volume_id: [0xD5; 16],
read_data_key: Some([0xD5; 16]),
read_data_key_err: None,
drive_unlocked: false,
};
let d = format!("{hs:?}");
assert!(
!d.contains("213"),
"HandshakeResult leaked VID/bus key: {d}"
);
assert!(
d.contains("redacted"),
"HandshakeResult missing marker: {d}"
);
}
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical
// partition (metadata_start == partition_start). Offsets cited
@@ -569,7 +624,7 @@ mod tests {
}
/// A content certificate: type byte@0 (0x00 = V10, else V20),
/// bus_encryption bit7@1, cc_id@14..20 (aacs/keys.rs parse_content_cert,
/// bus_encryption bit7@1, cc_id@14..20 (aacs/inf.rs parse_content_cert,
/// which requires ≥20 bytes and reads the bus flag from `data[1] >> 7`).
fn build_content_cert(cert_type: u8, bus_encryption: bool) -> Vec<u8> {
let mut v = vec![0u8; 20];
@@ -581,7 +636,7 @@ mod tests {
/// An MKB with one Type-and-Version record (type 0x10) carrying the
/// version as BE u32 at record offset 8, followed by a recorded EOF
/// record then trailing zero padding. mkb_content_len walks records
/// and stops at the first padding (type 0) byte (aacs/keys.rs).
/// and stops at the first padding (type 0) byte (aacs/inf.rs).
fn build_mkb(version: u32, pad_to: usize) -> Vec<u8> {
let mut v = Vec::new();
// Type 0x10 record, length 16 (>= 12 so version is read).
@@ -687,14 +742,14 @@ mod tests {
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("state");
assert_eq!(
st.version,
aacs::AACS_MAJOR_UHD,
aacs::mkb::AACS_MAJOR_UHD,
"no cert → default UHD (major 2)"
);
assert!(!st.bus_encryption);
}
/// disc_hash is SHA1 of the Unit_Key_RO.inf bytes, hex with 0x prefix
/// and uppercase (aacs::disc_hash + disc_hash_hex). The state's
/// and uppercase (aacs::inf::disc_hash + disc_hash_hex). The state's
/// disc_hash must match independently computing it over the same bytes.
#[test]
fn resolve_vid_only_disc_hash_is_sha1_of_unit_key_ro() {
@@ -710,7 +765,7 @@ mod tests {
}],
);
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("state");
let expected = aacs::disc_hash_hex(&aacs::disc_hash(&uk));
let expected = aacs::inf::disc_hash_hex(&aacs::inf::disc_hash(&uk));
assert_eq!(st.disc_hash, expected);
assert!(st.disc_hash.starts_with("0x"));
// uk_ro must be stashed verbatim for the external resolver.
@@ -746,7 +801,7 @@ mod tests {
// Real record stream is the single 16-byte type-0x10 record.
assert_eq!(
st.mkb.len(),
aacs::mkb_content_len(&mkb),
aacs::mkb::mkb_content_len(&mkb),
"MKB must be trimmed to record-stream length, not the zero-pad"
);
assert_eq!(st.mkb.len(), 16);
@@ -912,41 +967,54 @@ mod tests {
/// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy
/// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`).
/// This is the damaged-primary recovery path real discs rely on.
#[test]
fn resolve_vid_only_falls_back_to_duplicate_unit_key_ro() {
let mut disc = MemDisc::new();
// Build AACS dir with a DUPLICATE subdir holding Unit_Key_RO.inf.
let uk = vec![0x55u8; 48];
let mut dup_fids = Vec::new();
push_fid(&mut dup_fids, "", 70, true, true);
push_fid(&mut dup_fids, "Unit_Key_RO.inf", 72, false, false);
disc.put(PART_START + 72, build_file_icb(uk.len() as u32, 9000));
disc.put_bytes(PART_START + 9000, &uk);
disc.put(PART_START + 70, build_file_icb(dup_fids.len() as u32, 71));
disc.put_bytes(PART_START + 71, &dup_fids);
// AACS dir: only a DUPLICATE subdir (no primary Unit_Key_RO.inf).
let mut aacs_fids = Vec::new();
push_fid(&mut aacs_fids, "", 50, true, true);
push_fid(&mut aacs_fids, "DUPLICATE", 70, true, false);
disc.put(PART_START + 50, build_file_icb(aacs_fids.len() as u32, 51));
disc.put_bytes(PART_START + 51, &aacs_fids);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "AACS", 50, true, false);
disc.put(PART_START + 10, build_file_icb(root_fids.len() as u32, 11));
disc.put_bytes(PART_START + 11, &root_fids);
build_udf_skeleton(&mut disc, 10);
let udf = udf::read_filesystem(&mut disc).expect("fs");
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("DUPLICATE fallback");
// disc_hash must be computed over the DUPLICATE bytes.
assert_eq!(
st.disc_hash,
aacs::disc_hash_hex(&aacs::disc_hash(&uk)),
"fallback must hash the DUPLICATE Unit_Key_RO.inf"
fn handshake_has_volume_id_reports_presence_not_absence() {
let with_vid = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
assert!(
super::handshake_has_volume_id(&with_vid),
"a non-zero Volume ID must report as PRESENT"
);
assert_eq!(st.uk_ro, uk);
let without = HandshakeResult {
volume_id: [0u8; 16],
..with_vid
};
assert!(
!super::handshake_has_volume_id(&without),
"an all-zero Volume ID is the absent case"
);
// One bit of difference is still a VID: the check is != all-zero, not a
// heuristic about how much of it looks populated.
let mut barely = [0u8; 16];
barely[15] = 1;
assert!(
super::handshake_has_volume_id(&HandshakeResult {
volume_id: barely,
..with_vid
}),
"any non-zero byte makes a Volume ID present"
);
}
/// The gate itself still hard-errors — the property the log line annotates.
#[test]
fn resolve_vid_only_bus_key_gate_hard_errors_without_a_read_data_key() {
let (mut disc, udf) = disc_with_cert(0x01, true);
let hs = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
let err = Disc::resolve_vid_only(&udf, &mut disc, Some(&hs))
.expect_err("bus-encrypted, no read_data_key must still hard-error");
assert!(matches!(err, Error::AacsBusKeyUnavailable));
}
// ---------------------------------------------------------------
@@ -965,8 +1033,8 @@ mod tests {
// the route fails gracefully (AacsNoHostCert), never panics.
// ---------------------------------------------------------------
fn fake_cert(tag: u8) -> aacs::HostCert {
aacs::HostCert {
fn fake_cert(tag: u8) -> aacs::types::HostCert {
aacs::types::HostCert {
private_key: [tag; 20],
certificate: vec![tag; 92],
private_key_v2: None,
@@ -975,15 +1043,15 @@ mod tests {
}
/// A minimal in-test KeySource that yields no keys but a fixed cert list.
struct CertSource(Vec<aacs::HostCert>);
struct CertSource(Vec<aacs::types::HostCert>);
impl crate::KeySource for CertSource {
fn get_uk(
fn get_unit_keys(
&self,
_ctx: &dyn crate::keysource::ResolveCtx,
) -> Result<Vec<crate::aacs::UnitKey>> {
) -> Result<Vec<crate::aacs::types::UnitKey>> {
Ok(Vec::new())
}
fn host_certs(&self, _mkb: Option<u32>) -> Vec<aacs::HostCert> {
fn host_certs(&self, _mkb: Option<u32>) -> Vec<aacs::types::HostCert> {
self.0.clone()
}
}
@@ -1077,7 +1145,7 @@ mod tests {
#[test]
fn cert_unlock_outcome_maps_to_structured_trace_step() {
use crate::aacs::UnlockOutcome;
use crate::aacs::trace::UnlockOutcome;
use freemkv_unlock::UnlockError;
// The libfreemkv-side no-cert case carries the MKB generation.
assert_eq!(
+1083 -177
View File
File diff suppressed because it is too large Load Diff
+3639
View File
File diff suppressed because it is too large Load Diff
-1670
View File
File diff suppressed because it is too large Load Diff
+3859 -2938
View File
File diff suppressed because it is too large Load Diff
-1705
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-232
View File
@@ -1,232 +0,0 @@
//! `Disc::sweep`'s consumer-side `Sink<WorkItem>`.
//!
//! Background: the original sweep loop runs strictly serialised —
//! SCSI read → decrypt → seek + write → mapfile.record → next iter.
//! On a healthy disc the SCSI read costs ~5-12 ms per 64 KB batch and
//! the post-read work (decrypt 1-3 ms + file write + mapfile fsync
//! 5-15 ms) adds another batch's worth of latency. The drive idles
//! during the post-read work; throughput tops out at the *sum* of
//! both costs.
//!
//! A producer/consumer split overlaps the two stages on the generic
//! [`crate::io::Pipeline`] + [`crate::io::Sink`] primitive. This module
//! is the sweep-specific `Sink` impl; the producer-side state machine
//! (read_error context, decrypt, set_speed, halt) stays in
//! `Disc::sweep` in `disc/mod.rs`.
//!
//! Correctness invariants preserved:
//! - Mapfile is single-writer (consumer-only). No locking.
//! - All `read_error::ReadCtx` state stays on the producer thread.
//! - `set_speed` calls happen on the producer thread (same thread that
//! owns the `SectorSource`). No new SCSI concurrency.
//! - Per-iteration ordering of file-write → mapfile-record is kept
//! intact in the consumer (write before record), so the on-disk
//! invariant "mapfile only marks Finished what the file has
//! received" survives a crash mid-pass.
//! - Only one SCSI command is in flight at a time; error-path timing
//! is identical and no new retry logic is introduced.
use std::io::{Seek, SeekFrom, Write};
use std::sync::mpsc::{Receiver, SyncSender, sync_channel};
use crate::error::Error;
use crate::io::{Flow, Sink};
use super::mapfile::{MapStats, Mapfile, SectorStatus};
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. 64 KB
/// matches the existing zero_gap chunk size used by the pre-split
/// sweep loop.
const ZERO_CHUNK: usize = 64 * 1024;
/// Producer → Consumer messages. The consumer applies these in FIFO
/// order; ordering of file writes and mapfile records across items is
/// preserved.
pub(super) enum WorkItem {
/// Successful batch read. Producer has already decrypted `buf` if
/// `opts.decrypt` was set. Consumer writes `buf` at `pos` and
/// records the range as `Finished`.
Good { pos: u64, buf: Vec<u8> },
/// Bisect inner-loop good single sector (already decrypted by the
/// producer). 2048 bytes.
BisectGood { pos: u64, buf: Box<[u8; 2048]> },
/// Bisect inner-loop bad single sector. Consumer writes 2048
/// zeros at `pos` and records the sector as `NonTrimmed`.
BisectBad { pos: u64 },
/// Whole-batch zero-fill (failed batch on `SkipBlock`, or the
/// failed batch portion of `JumpAhead`). Consumer streams zeros
/// across `[pos, pos+len)` and records the range as `NonTrimmed`.
SkipFill { pos: u64, len: u64 },
/// Gap fill following a `JumpAhead`. Same effect as `SkipFill`;
/// distinguished only so future logging / instrumentation can
/// tell them apart without parsing a flag.
GapFill { pos: u64, len: u64 },
/// Post-read verify downgrade. The producer's `UnitVerifier` found that the
/// just-`Finished` clip unit at `[pos, pos+len)` is confidently undecryptable
/// (a silent bad read). The consumer re-records the range as `NonTrimmed` so
/// the patch pass re-reads it — the ISO bytes (ciphertext) already written by
/// the preceding `Good` are left in place for the patch to overwrite. FIFO
/// pipe ordering guarantees this arrives AFTER the `Good` that wrote them.
MarkBad { pos: u64, len: u64 },
/// Producer wants the latest mapfile stats for the progress
/// callback. Consumer responds on `prog_tx` with a fresh
/// [`ProgressSnapshot`]. Best-effort: if the producer hasn't
/// drained the previous snapshot, the new one is silently
/// dropped — the producer's local cache stays current enough.
StatsRequest,
}
/// Snapshot the consumer sends back to the producer for the progress
/// callback.
pub(super) struct ProgressSnapshot {
pub stats: MapStats,
pub bad_ranges: Vec<(u64, u64)>,
}
/// Final summary returned by the consumer thread on shutdown — what
/// `SweepSink::close` produces, surfaced to the producer via
/// `Pipeline::finish`.
pub(super) struct ConsumerSummary {
pub stats: MapStats,
}
/// Drain any pending progress snapshots from the consumer. Returns
/// the most recent one, if any. The producer caches it and uses it
/// for subsequent progress callbacks until a fresh one arrives.
pub(super) fn try_recv_progress(rx: &Receiver<ProgressSnapshot>) -> Option<ProgressSnapshot> {
let mut latest = None;
while let Ok(snap) = rx.try_recv() {
latest = Some(snap);
}
latest
}
/// `Sink<WorkItem>` for sweep. Owns the writeback file + mapfile +
/// progress back-channel. `apply` carries the file-write +
/// mapfile.record per item; `close` drains the writeback pipeline,
/// fsyncs the ISO, and flushes the mapfile.
pub(super) struct SweepSink {
file: crate::io::WritebackFile,
map: Mapfile,
/// `sync_all`-on-failure-is-an-error iff the output is a regular
/// file. `/dev/null` and pipes always fail `sync_all`; that's not
/// a real error.
is_regular: bool,
/// Back-channel for `StatsRequest` responses. The producer caches
/// the latest snapshot and uses it for the progress callback;
/// dropped sends on a full channel are by design.
prog_tx: SyncSender<ProgressSnapshot>,
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. Held
/// in the sink so each apply call doesn't reallocate.
zero: Box<[u8; ZERO_CHUNK]>,
}
impl SweepSink {
/// Construct a new `SweepSink` plus the matching progress
/// receiver. Channel depth on the back-channel is `1` — the
/// producer's cache is the source of truth between snapshots.
pub(super) fn new(
file: crate::io::WritebackFile,
map: Mapfile,
is_regular: bool,
) -> (Self, Receiver<ProgressSnapshot>) {
let (prog_tx, prog_rx) = sync_channel::<ProgressSnapshot>(1);
let sink = SweepSink {
file,
map,
is_regular,
prog_tx,
zero: Box::new([0u8; ZERO_CHUNK]),
};
(sink, prog_rx)
}
}
impl Sink<WorkItem> for SweepSink {
type Output = ConsumerSummary;
fn apply(&mut self, item: WorkItem) -> Result<Flow, Error> {
match item {
WorkItem::Good { pos, buf } => {
// Decrypt is on the producer; consumer assumes plaintext.
let len = buf.len() as u64;
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&buf)?;
self.map.record(pos, len, SectorStatus::Finished)?;
}
WorkItem::BisectGood { pos, buf } => {
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&buf[..])?;
self.map.record(pos, 2048, SectorStatus::Finished)?;
}
WorkItem::BisectBad { pos } => {
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&self.zero[..2048])?;
self.map.record(pos, 2048, SectorStatus::NonTrimmed)?;
}
WorkItem::SkipFill { pos, len } | WorkItem::GapFill { pos, len } => {
self.file.seek(SeekFrom::Start(pos))?;
// Subsequent writes are sequential; `WritebackFile`'s
// seek-elision keeps them on the writeback pipeline path.
let mut filled = 0u64;
while filled < len {
let chunk = (len - filled).min(self.zero.len() as u64) as usize;
self.file.write_all(&self.zero[..chunk])?;
filled += chunk as u64;
}
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::MarkBad { pos, len } => {
// Verify downgrade: the ISO bytes are already written by the
// preceding Good; only the mapfile status changes so patch
// re-reads this range. No file write.
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::StatsRequest => {
let stats = self.map.stats();
// DAMAGE only — NOT NonTried. NonTried is the unread remainder
// ahead of the sweep head, not damage; including it made the live
// located drilldown (at-risk movie time + range count) treat the
// whole unread disc as confirmed damage, so at sweep start it
// showed ~full-movie at-risk and melted to 0 as the sweep
// progressed. Matches the one-shot progress path, which already
// excludes NonTried.
let bad_ranges = self.map.ranges_with(&[
SectorStatus::NonTrimmed,
SectorStatus::Unreadable,
SectorStatus::NonScraped,
]);
// Best-effort: drop on backpressure; producer's cache
// stays current enough.
let _ = self
.prog_tx
.try_send(ProgressSnapshot { stats, bad_ranges });
}
}
Ok(Flow::Continue)
}
fn close(mut self) -> Result<Self::Output, Error> {
// Drain the writeback pipeline + fsync the ISO, then persist
// any pending mapfile state. Same finalisation order as the
// pre-Pipeline consumer loop.
if let Err(e) = self.file.sync_all() {
if self.is_regular {
return Err(Error::IoError { source: e });
}
// Non-regular outputs (/dev/null, pipes) always fail
// sync_all; that's not a real error.
}
self.map.flush()?;
Ok(ConsumerSummary {
stats: self.map.stats(),
})
}
}
-1021
View File
File diff suppressed because it is too large Load Diff
+6 -8
View File
@@ -23,14 +23,12 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
if !std::path::Path::new(&path).exists() {
continue;
}
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path)) {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((path, id));
}
}
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path))
&& let Ok(id) = DriveId::from_drive(transport.as_mut())
&& !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((path, id));
}
}
drives
+35 -6
View File
@@ -25,12 +25,11 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
let path = std::path::Path::new(&info.path);
match crate::scsi::open(path) {
Ok(mut transport) => {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((info.path.clone(), id));
}
if let Ok(id) = DriveId::from_drive(transport.as_mut())
&& !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((info.path.clone(), id));
}
}
Err(_) => {
@@ -53,3 +52,33 @@ pub fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
}
Ok((path.to_string(), DeviceResolution::Direct))
}
#[cfg(test)]
mod resolve_device_tests {
use super::*;
/// An existing path resolves unchanged as `Direct` — macOS has no
/// `sr`->`sg` substitution, so the returned path must be byte-identical
/// to the input, not some canonicalised/mutated form.
#[test]
fn existing_path_resolves_direct_unchanged() {
// Use the test binary's own executable path: guaranteed to exist,
// no fixture file needed.
let exe = std::env::current_exe().unwrap();
let path = exe.to_str().unwrap();
let (resolved, kind) = resolve_device(path).expect("existing path must resolve");
assert_eq!(resolved, path, "path must be returned unchanged");
assert_eq!(kind, DeviceResolution::Direct);
}
/// A path that does not exist must error with `DeviceNotFound` carrying
/// the original path, never silently succeed.
#[test]
fn missing_path_is_device_not_found() {
let path = "/dev/freemkv-definitely-does-not-exist-0xdead";
match resolve_device(path) {
Err(Error::DeviceNotFound { path: p }) => assert_eq!(p, path),
other => panic!("expected DeviceNotFound, got {other:?}"),
}
}
}
+1266 -142
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -6,7 +6,7 @@
//!
//! Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against libdvdnav's decoder.
//! verified against real discs.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
+5 -5
View File
@@ -2,7 +2,7 @@
//!
//! An 8-byte navigation command as found in PGC command tables (pre/post/cell)
//! and PCI button info. Decoded per the DVD-Video VM instruction set and
//! verified against libdvdnav's command decoder.
//! verified against real discs.
//!
//! Bit model: the 8 bytes are a big-endian 64-bit word. `byte0` bits 7-5 are the
//! command **type**; for type 1, `byte0` bit 4 selects Link (0) vs Jump (1), and
@@ -133,7 +133,7 @@ const JP_JUMP_SS: u8 = 6;
const JP_CALL_SS: u8 = 8;
// Link (type 1, direct=0) sub-commands. NOTE: sub-op 0 is NOP/no-link and 1 is
// the LinkSub form (libdvdnav `decoder.c` `eval_link_instruction`).
// the LinkSub form (the DVD-Video VM link instruction).
const LK_SUB: u8 = 1;
const LK_PGCN: u8 = 4;
const LK_PTTN: u8 = 5;
@@ -159,7 +159,7 @@ fn be16(b: &[u8; 8], o: usize) -> u16 {
((b[o] as u16) << 8) | b[o + 1] as u16
}
// Compare-operand layouts ("if_version"s) per libdvdnav `decoder.c`. The op
// Compare-operand layouts ("if_version"s) per the DVD-Video VM. The op
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
@@ -205,7 +205,7 @@ pub fn decode(b: &[u8; 8]) -> Command {
let cmd = b[1] & 0x0F;
// Compare predicate, with the operand layout for this command family
// (libdvdnav `decoder.c` `vm_eval_command` type dispatch).
// (the DVD-Video VM command type dispatch).
let compare = match (typ, direct) {
(TYPE_SPECIAL, _) => if_v1(b),
(TYPE_LINK_JUMP, 1) => if_v2(b), // jump
@@ -372,7 +372,7 @@ mod tests {
}
}
// Regression for the libdvdnav cross-check: link sub-op 0 = NOP, 1 = LinkSub.
// Regression for the link sub-op decode: 0 = NOP, 1 = LinkSub.
#[test]
fn link_subop_zero_is_nop_one_is_linksub() {
assert_eq!(decode(&h("2000000000000000")).instr, Instr::Nop);
+901 -101
View File
File diff suppressed because it is too large Load Diff
+267
View File
@@ -0,0 +1,267 @@
//! Seeded robustness harness for the untrusted-input parsers.
//!
//! Every parser reached from here takes bytes that came off a disc, and this
//! crate's primary boundary is that the disc is untrusted: a malformed, damaged
//! or hostile image must never crash the library. These tests assert exactly
//! that one property — **the parser returns `Ok` or `Err`, and never panics.**
//!
//! # Why this exists rather than `cargo-fuzz`
//!
//! `cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
//! for libFuzzer's coverage feedback) and this project pins stable. So the
//! generator lives here instead. It gives up coverage-guided mutation — the real
//! loss — and keeps everything else: millions of cases, structure-aware input,
//! and a crash corpus. It also gains determinism, which a fuzzer does not have:
//! the same seed replays the same cases on any machine.
//!
//! # Why no `proptest` or `arbitrary`
//!
//! This crate has exactly one dev-dependency. That is a deliberate posture, and
//! a randomness crate is not worth ten transitive dependencies when the parsers
//! take plain `&[u8]` and a good enough generator is forty lines.
//!
//! # Budget
//!
//! `FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
//! enough that the per-commit gate stays under a second). The overnight run sets
//! it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
//! fixed so a failure in CI reproduces locally verbatim.
//!
//! # On failure
//!
//! The panic message carries the seed, generator and case index. Re-run with
//! `FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
//! into `tests/corpus/` as a permanent regression fixture — discovery happens
//! here, defence happens there.
#![cfg(test)]
/// Marsaglia xorshift64. Not cryptographic and does not need to be: the job is
/// a reproducible spread of bytes, and a named algorithm beats an ad-hoc LCG
/// whose period nobody has checked.
struct Rng(u64);
impl Rng {
fn new(seed: u64) -> Self {
// A zero seed is a fixed point of xorshift — it would emit zeros forever
// and every generated case would be identical.
Self(if seed == 0 {
0x2545_F491_4F6C_DD1D
} else {
seed
})
}
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
fn byte(&mut self) -> u8 {
(self.next() >> 24) as u8
}
/// Uniform-ish in `0..n`. The modulo bias is irrelevant at these magnitudes.
fn below(&mut self, n: usize) -> usize {
if n == 0 {
0
} else {
(self.next() % n as u64) as usize
}
}
fn fill(&mut self, len: usize) -> Vec<u8> {
(0..len).map(|_| self.byte()).collect()
}
}
/// Budget per generator per target.
fn cases() -> usize {
std::env::var("FREEMKV_HARNESS_CASES")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(256)
}
fn seed() -> u64 {
std::env::var("FREEMKV_HARNESS_SEED")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(0x5EED_1234_ABCD_0001)
}
/// Largest generated input. Big enough to carry a plausible header plus a body,
/// small enough that millions of cases stay quick.
const MAX_LEN: usize = 4096;
/// Drive `f` over three generators and report which case broke it.
///
/// A panic inside `f` fails the test on its own — nothing is caught here,
/// because catching would risk reporting a pass on an input that aborted. The
/// wrapper exists to make the failing case *identifiable*: the harness prints
/// the seed, generator and index before each call, so the last line before a
/// panic names the exact case to reproduce.
fn sweep<F: FnMut(&[u8])>(target: &str, magic: &[u8], f: F) {
sweep_n(target, magic, cases(), f)
}
/// `sweep` with an explicit budget. The budget is a PARAMETER rather than read
/// from the environment inside the loop: the meta-tests below need a small,
/// fixed count, and `std::env::set_var` is unsound once the test harness runs
/// tests in parallel — two tests setting the same variable race, which is
/// exactly what happened on the first run of this file.
fn sweep_n<F: FnMut(&[u8])>(target: &str, magic: &[u8], n: usize, mut f: F) {
let s = seed();
// 1. Pure random bytes. Cheap, and almost always rejected at the magic
// number — it exercises the entry guards and little else. Kept because
// the entry guards are themselves worth exercising.
let mut rng = Rng::new(s);
for i in 0..n {
let len = rng.below(MAX_LEN);
let buf = rng.fill(len);
run(target, "random", s, i, &buf, &mut f);
}
// 2. Valid magic, random body. THE generator that matters: pure random
// input dies at the magic check and never reaches the parser body, so
// without this the sweep only ever tests the first few lines.
let mut rng = Rng::new(s ^ 0xA5A5_A5A5_A5A5_A5A5);
for i in 0..n {
let mut buf = magic.to_vec();
let tail = rng.below(MAX_LEN.saturating_sub(magic.len()));
buf.extend(rng.fill(tail));
run(target, "magic+noise", s, i, &buf, &mut f);
}
// 3. Structured mutation of a plausible record: a valid magic, then mostly
// zeroes, with a handful of bytes corrupted and a truncation. Length and
// offset fields live in those early bytes, so this is what reaches the
// arithmetic — the offsets, counts and sizes a hostile image would lie
// about.
let mut rng = Rng::new(s ^ 0x1234_5678_9ABC_DEF0);
for i in 0..n {
let mut buf = vec![0u8; 512];
buf[..magic.len().min(512)].copy_from_slice(&magic[..magic.len().min(512)]);
for _ in 0..rng.below(24) + 1 {
let at = rng.below(buf.len());
buf[at] = rng.byte();
}
buf.truncate(rng.below(buf.len()) + 1);
run(target, "mutate", s, i, &buf, &mut f);
}
}
fn run<F: FnMut(&[u8])>(target: &str, generator: &str, seed: u64, i: usize, buf: &[u8], f: &mut F) {
// Printed, not asserted: `cargo test` swallows stdout for passing tests and
// shows it for failing ones, so this line is invisible until it is the last
// thing before a panic — at which point it is exactly what is needed.
println!(
"harness {target}/{generator} seed={seed:#x} case={i} len={} :: \
FREEMKV_HARNESS_SEED={seed} to reproduce",
buf.len()
);
f(buf);
}
#[test]
fn mpls_parse_never_panics() {
sweep("mpls", b"MPLS", |b| {
let _ = crate::mpls::parse(b);
});
}
#[test]
fn clpi_parse_never_panics() {
sweep("clpi", b"HDMV", |b| {
let _ = crate::clpi::parse(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
// input to reject, so the "magic" is a byte the sweep will mutate anyway.
sweep("udf_name", &[8], |b| {
let _ = crate::udf::parse_udf_name(b);
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
// each case is fed to a FRESH demuxer and then a shared one. The shared pass
// is what exercises cross-feed state — a start code split over a boundary,
// a held PES completed by later bytes, the carry-over cap.
let mut shared = crate::mux::ps::PsDemuxer::new();
sweep("ps_demux", &[0x00, 0x00, 0x01, 0xBA], |b| {
let mut fresh = crate::mux::ps::PsDemuxer::new();
let _ = fresh.feed(b);
let _ = shared.feed(b);
});
}
#[test]
fn mkv_lacing_split_never_panics() {
// All four lacing modes, including the reserved bit pattern. A degenerate
// fixed lace was a real defect found by audit round 5.
sweep("mkv_lacing", &[0x00], |b| {
for lacing in 0u8..=3 {
let _ = crate::mux::mkvstream::split_lacing(lacing, b);
}
});
}
/// The generators must actually differ, or the sweep is one generator run three
/// times and the coverage claim is false.
#[test]
fn the_three_generators_produce_different_inputs() {
let mut seen: Vec<Vec<u8>> = Vec::new();
sweep_n("probe", b"MPLS", 1, |b| seen.push(b.to_vec()));
assert_eq!(seen.len(), 3, "one case per generator");
assert_ne!(seen[0], seen[1], "random and magic+noise must differ");
assert_ne!(seen[1], seen[2], "magic+noise and mutate must differ");
assert!(
seen[1].starts_with(b"MPLS"),
"the magic+noise generator must actually carry the magic, or it never \
reaches the parser body"
);
}
/// The same seed must replay the same bytes, or a reported failure cannot be
/// reproduced and the harness is worthless as a regression tool.
#[test]
fn a_seed_replays_identically() {
let mut a = Vec::new();
let mut b = Vec::new();
sweep_n("probe", b"MPLS", 4, |x| a.push(x.to_vec()));
sweep_n("probe", b"MPLS", 4, |x| b.push(x.to_vec()));
assert_eq!(a, b, "the same seed must produce the same cases");
}
/// The harness is worthless if its cases die at the entry guards, so this
/// MEASURES how deep they actually reach instead of assuming. A generator that
/// never gets past a length or magic check exercises the first ten lines and
/// nothing else — the fuzzing equivalent of a test that cannot fail.
#[test]
fn the_generators_actually_reach_the_parser_bodies() {
// mpls::parse rejects at: len < 40, bad magic, then playlist_start + 10 >
// len. Anything that returns Ok got all the way through the play-item loop.
let mut ok = 0usize;
let mut total = 0usize;
sweep_n("reach", b"MPLS", 20000, |b| {
total += 1;
if crate::mpls::parse(b).is_ok() {
ok += 1;
}
});
assert!(
ok > 0,
"not one of {total} generated cases parsed successfully — the generators \
are all being rejected at the entry guards, so this harness is testing \
the guards and nothing behind them"
);
println!("mpls reach: {ok}/{total} cases parsed to completion");
}
+47 -5
View File
@@ -16,11 +16,11 @@
/// (case-insensitive), then requires an even run of ASCII hex digits. Any
/// non-hex byte, or an odd length, yields `None`.
pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
// Empty → empty Vec (a legitimately-empty variable-length field); odd length
// is malformed. (`parse_hex_fixed` enforces a concrete length separately.)
if bytes.len() % 2 != 0 {
if !bytes.len().is_multiple_of(2) {
return None;
}
let mut out = Vec::with_capacity(bytes.len() / 2);
@@ -34,7 +34,7 @@ pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
/// prefix; requires EXACTLY `2*N` ASCII hex digits after it. `None` on any
/// non-hex byte or a length mismatch.
pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
if bytes.len() != 2 * N {
return None;
@@ -46,8 +46,33 @@ pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
Some(out)
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive).
fn strip_prefix(s: &str) -> &str {
/// Parse a hex string into a `u16`. Accepts an optional `0x`/`0X` prefix
/// (case-insensitive) via the same [`strip_hex_prefix`] the byte parsers use.
/// `None` on any non-hex content or overflow.
///
/// Exists so callers never hand-roll `from_str_radix(s.trim_start_matches("0x"), 16)`
/// — a **case-sensitive** strip that silently dropped an uppercase-`0X` value.
/// (That reintroduced-in-keydb bug is exactly what this module was built to kill;
/// the integer fields now share the one prefix rule.)
pub fn parse_hex_u16(s: &str) -> Option<u16> {
u16::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u32`. See [`parse_hex_u16`].
pub fn parse_hex_u32(s: &str) -> Option<u32> {
u32::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u8`. See [`parse_hex_u16`].
pub fn parse_hex_u8(s: &str) -> Option<u8> {
u8::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive). Public so
/// callers that only need the prefix rule (e.g. normalizing a disc hash) reuse
/// the one definition instead of hand-rolling a case-sensitive
/// `trim_start_matches("0x")`.
pub fn strip_hex_prefix(s: &str) -> &str {
s.strip_prefix("0x")
.or_else(|| s.strip_prefix("0X"))
.unwrap_or(s)
@@ -95,6 +120,23 @@ mod tests {
assert_eq!(parse_hex_fixed::<16>(&s), None);
}
#[test]
fn hex_ints_accept_both_prefix_cases_and_bare() {
// The regression the keydb device-key bug hit: uppercase `0X` must parse
// identically to `0x` and to a bare value.
assert_eq!(parse_hex_u16("0x0001"), Some(1));
assert_eq!(parse_hex_u16("0X0001"), Some(1));
assert_eq!(parse_hex_u16("0001"), Some(1));
assert_eq!(parse_hex_u16(" 0XABCD "), Some(0xABCD));
assert_eq!(parse_hex_u32("0X00000002"), Some(2));
assert_eq!(parse_hex_u32("deadbeef"), Some(0xDEAD_BEEF));
assert_eq!(parse_hex_u8("0X03"), Some(3));
assert_eq!(parse_hex_u8("ff"), Some(0xFF));
// Overflow / non-hex → None.
assert_eq!(parse_hex_u8("0x1FF"), None);
assert_eq!(parse_hex_u16("0xzz"), None);
}
#[test]
fn bytes_variable_length_and_odd_rejected() {
assert_eq!(parse_hex_bytes("0xAABBCC"), Some(vec![0xAA, 0xBB, 0xCC]));
+193 -1
View File
@@ -49,13 +49,30 @@ pub struct DriveId {
pub raw_gc_010c: Vec<u8>,
}
/// SPC-4 standard INQUIRY data: 36 bytes through `product_revision`. Anything
/// shorter cannot populate the identity fields this type promises.
const INQUIRY_STANDARD_LEN: usize = 36;
impl DriveId {
/// Probe a real drive via SCSI and build its identity.
pub fn from_drive(transport: &mut dyn ScsiTransport) -> Result<Self> {
// INQUIRY — SPC-4 §6.4
let mut inquiry = vec![0u8; 96];
let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00];
transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
let inq = transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
// `bytes_transferred` is device-reported and untrusted — the same rule
// the two GET CONFIGURATION calls below already apply. It was ignored
// here, and the buffer is pre-zeroed, so a drive answering GOOD with a
// short or empty data phase (a USB-SATA bridge mid-wedge does exactly
// this) decoded to blank identity strings and a byte 0 of 0x00. Every
// platform enumerator gates on `raw_inquiry[0] & 0x1F == OPTICAL`, so
// 0x00 reads as DIRECT ACCESS and the drive silently disappears from
// the device list instead of reporting a failed probe.
if inq.bytes_transferred < INQUIRY_STANDARD_LEN {
return Err(crate::error::Error::DriveInquiryShort);
}
// Never decode past what the drive actually sent.
inquiry.truncate(inq.bytes_transferred.min(inquiry.len()));
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6.
// Best-effort: 010Ch (Firmware Information) is an optional feature.
@@ -262,6 +279,54 @@ mod tests {
/// Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer
/// (e.g. a device that reports fewer than 8 bytes) must not panic.
/// Mutation: removing the `data.len() > start` guard makes it panic on short inputs.
/// A drive that answers INQUIRY with GOOD status but a short or empty
/// data phase must fail the probe, not present as a blank drive.
///
/// The buffer is pre-zeroed, so decoding it unconditionally yielded empty
/// vendor/product/revision strings and a byte 0 of 0x00. Every platform
/// enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
/// and 0x00 is DIRECT ACCESS — so the drive silently vanished from the
/// device list rather than reporting that its identity probe failed. A
/// USB-SATA bridge mid-wedge does exactly this.
///
/// The two GET CONFIGURATION calls in the same function already clamped on
/// `bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
/// lines above them, discarded it.
#[test]
fn inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive() {
/// GOOD status, no sense, and only `n` bytes written.
struct ShortInquiry(usize);
impl ScsiTransport for ShortInquiry {
fn execute(
&mut self,
_cdb: &[u8],
_dir: DataDirection,
_buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Ok(ScsiResult {
status: 0,
sense: [0u8; 32],
bytes_transferred: self.0,
})
}
}
// Empty data phase — the case that made a real drive disappear.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(0)),
Err(crate::error::Error::DriveInquiryShort)
));
// One byte short of the SPC-4 standard 36-byte header.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(35)),
Err(crate::error::Error::DriveInquiryShort)
));
// Exactly the standard length is acceptable: the optional
// vendor-specific tail past byte 36 is allowed to be absent.
assert!(DriveId::from_drive(&mut ShortInquiry(36)).is_ok());
}
#[test]
fn ascii_field_short_buffer_returns_empty() {
// Buffer of length 5: start=8 is beyond the end → empty string.
@@ -339,9 +404,136 @@ mod tests {
);
}
/// `ascii_field`'s guard is `data.len() > start` (strictly greater), not
/// `>=`: a buffer whose length is exactly `start` has NO byte at that
/// offset, so it must still yield empty, not attempt to slice.
/// Mutation: `>` -> `>=` would try to slice `data[start..]` when
/// `data.len() == start`, which panics (empty range at the very end is
/// fine, but the guard's job is the `< start` case below it — pinning the
/// exact boundary catches an off-by-one either direction).
#[test]
fn ascii_field_boundary_len_equals_start_is_empty() {
let buf = vec![0u8; 8];
assert_eq!(ascii_field(&buf, 8, 16), "");
}
/// One byte past the boundary: `data.len() == start + 1` must extract
/// that single byte (clamped to `end`), proving the guard is `>` and not
/// off by one in the other direction.
#[test]
fn ascii_field_boundary_len_one_past_start_extracts_one_byte() {
let mut buf = vec![0u8; 9];
buf[8] = b'X';
assert_eq!(ascii_field(&buf, 8, 16), "X");
}
/// `Display` renders the four trimmed identity fields space-separated —
/// the human-readable counterpart of `match_key`'s pipe-separated form.
/// Not exercised anywhere else in this test module.
/// Mutation: replacing the `fmt` body with `Ok(Default::default())`
/// writes nothing at all, so formatting any `DriveId` yields "".
#[test]
fn display_formats_trimmed_fields_space_separated() {
let mut inquiry = vec![0u8; 96];
inquiry[8..16].copy_from_slice(b"PIONEER ");
inquiry[16..32].copy_from_slice(b"BD-RW BDR-S09 ");
inquiry[32..36].copy_from_slice(b"1.34");
inquiry[36..43].copy_from_slice(b" 16/04/");
let id = DriveId::from_inquiry(&inquiry, "201604250000");
assert_eq!(id.to_string(), "PIONEER BD-RW BDR-S09 1.34 16/04/");
}
/// GET CONFIGURATION failure (transport error) must not abort the
/// identity probe — firmware_date is empty, raw_gc_010c is empty.
/// Mutation: propagating the GET_CONFIGURATION error with `?` aborts from_drive.
/// Transport whose GET CONFIGURATION responses report an exact,
/// caller-chosen `bytes_transferred` for each of the two GC features
/// (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
/// boundary guards can be pinned precisely. INQUIRY always succeeds.
struct FixedGcCountTransport {
firmware_bytes: usize,
serial_bytes: usize,
}
impl ScsiTransport for FixedGcCountTransport {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
for b in buf.iter_mut() {
*b = b'Z';
}
let bytes_transferred = match cdb.first() {
Some(&0x12) => buf.len(),
Some(&0x46) if cdb[3] == 0x0C => self.firmware_bytes,
Some(&0x46) if cdb[3] == 0x08 => self.serial_bytes,
_ => buf.len(),
};
Ok(ScsiResult {
status: 0,
bytes_transferred,
sense: [0u8; 32],
})
}
}
/// `end > 12` in the firmware-date branch (`from_drive`) is a strict
/// inequality: `bytes_transferred == 12` reports the field absent
/// (offset 12 is the first byte of the 12-char date; a count of exactly
/// 12 covers bytes 0..12, none of which is the date), so `firmware_date`
/// must be empty, not the mutant's off-by-one read.
/// Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an
/// empty but non-panicking slice — silently reporting "present" data
/// that is actually all outside the transferred count.
#[test]
fn from_drive_firmware_date_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 12,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "");
}
/// One byte past the boundary (`bytes_transferred == 13`) must extract
/// exactly the one available date byte (offset 12), proving the guard
/// is `>` and the slice end is clamped to `end`, not always to 24.
#[test]
fn from_drive_firmware_date_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 13,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "Z");
}
/// Same `> 12` boundary for the serial-number branch: exactly 12
/// transferred bytes must yield an empty serial.
#[test]
fn from_drive_serial_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 12,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "");
}
/// One byte past the serial boundary extracts exactly that byte.
#[test]
fn from_drive_serial_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 13,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "Z");
}
#[test]
fn from_drive_gc_failure_yields_empty_firmware_date() {
struct GcFailTransport;
+1176 -115
View File
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -133,10 +133,10 @@ where
match rx.recv_timeout(slice) {
Ok(v) => return Ok(v),
Err(RecvTimeoutError::Timeout) => {
if let Some(h) = halt {
if h.is_cancelled() {
return Err(BoundedError::Halted);
}
if let Some(h) = halt
&& h.is_cancelled()
{
return Err(BoundedError::Halted);
}
if Instant::now() >= deadline {
return Err(BoundedError::Timeout);
+21
View File
@@ -249,6 +249,27 @@ impl Drop for BytePrefetcher {
mod tests {
use super::*;
/// `RECYCLE_DEPTH` must be one MORE than `FORWARD_DEPTH` per its own
/// doc comment: the producer needs at least one buffer to fill while
/// the consumer holds the other `FORWARD_DEPTH`-worth in flight. A
/// `+` -> `*`/`-` mutation on `FORWARD_DEPTH + 1` would under-size the
/// recycle channel (e.g. `FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one
/// short), which starves the producer of a spare buffer.
#[test]
fn recycle_depth_is_forward_depth_plus_one() {
assert_eq!(RECYCLE_DEPTH, FORWARD_DEPTH + 1);
assert_eq!(RECYCLE_DEPTH, 3, "FORWARD_DEPTH is 2, so recycle must be 3");
}
/// `DEFAULT_CHUNK_BYTES` is documented as 16 MiB. Pins the literal so a
/// `*` -> `+`/`/` mutation on either factor (16 * 1024 * 1024) is
/// caught by a concrete, spec-derived expected value rather than by
/// recomputing the same expression.
#[test]
fn default_chunk_bytes_is_16_mib() {
assert_eq!(DEFAULT_CHUNK_BYTES, 16_777_216, "documented as 16 MiB");
}
/// Endless reader: every `read` fills the whole buffer and never
/// hits EOF, so the producer keeps trying to push batches forward
/// until the forward channel disconnects. Exactly the shape that
+3 -3
View File
@@ -23,7 +23,7 @@
use std::fs::File;
use std::os::unix::io::AsRawFd;
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
pub(crate) fn hint_sequential(file: &File, _len_bytes: u64) {
// Best-effort: return value ignored. A fadvise failure has no
// user-observable consequence.
unsafe {
@@ -34,7 +34,7 @@ pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
/// Drop pages in the half-open byte range `[start, start+len)` from
/// the page cache. Called periodically by `read_sectors` to bound the
/// read-side page cache pressure.
pub(super) fn drop_window(file: &File, start: u64, len: u64) {
pub(crate) fn drop_window(file: &File, start: u64, len: u64) {
unsafe {
libc::posix_fadvise(
file.as_raw_fd(),
@@ -58,7 +58,7 @@ pub(super) fn drop_window(file: &File, start: u64, len: u64) {
/// can only pre-stage a tiny slice of the next batch. An explicit
/// `readahead()` of the same size as the current batch tells the
/// kernel to queue the full next-batch read now.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) {
unsafe {
libc::readahead(file.as_raw_fd(), offset as i64, len as usize);
}
+3 -3
View File
@@ -15,7 +15,7 @@ use std::os::unix::io::AsRawFd;
/// pipeline depth.
const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024;
pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
pub(crate) fn hint_sequential(file: &File, len_bytes: u64) {
let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory {
ra_offset: 0,
@@ -33,14 +33,14 @@ pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
/// approximation: no-op. macOS's unified buffer cache is generally
/// less prone to the pin-everything pathology that triggers the
/// regression on Linux NFS clients.
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Async-prefetch the byte range `[offset, offset+len)`. macOS uses
/// the same `fcntl(F_RDADVISE, &radvisory)` primitive as the open-
/// time sequential hint, just targeted at a moving window instead of
/// the whole file. The kernel queues I/O for the requested range and
/// returns immediately.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) {
let bytes = (len as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory {
ra_offset: offset as libc::off_t,
+133 -23
View File
@@ -48,22 +48,25 @@
//! far smaller than our 16 MiB app-level batch.
#[cfg(target_os = "linux")]
mod linux;
pub(crate) mod linux;
#[cfg(target_os = "macos")]
mod macos;
pub(crate) mod macos;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
mod other;
pub(crate) mod other;
#[cfg(target_os = "windows")]
mod windows;
pub(crate) mod windows;
// The page-cache hints are shared with any other file-backed sector source:
// `dirimage` reads host files the same way and needs the same eviction, or a
// large rip pins every byte it has read (see this module's DONTNEED note).
#[cfg(target_os = "linux")]
use linux as platform;
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
use macos as platform;
pub(crate) use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use other as platform;
pub(crate) use other as platform;
#[cfg(target_os = "windows")]
use windows as platform;
pub(crate) use windows as platform;
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
@@ -78,18 +81,34 @@ use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// read side. Mirrors `WRITEBACK_CHUNK_BYTES` so the read-side page
/// cache stays bounded the same way the write side does.
///
/// 32 MiB is the empirically tuned value on the rip1 test bed (single
/// 7200rpm HDD via SATA): smaller windows (8 / 16 MiB) shorten the
/// 32 MiB is the empirically tuned value on a 7200rpm HDD via SATA:
/// smaller windows (8 / 16 MiB) shorten the
/// kernel-readahead overlap and slow the producer; larger windows
/// (64 / 128 MiB) let the page cache pin enough of the ISO to
/// pressure concurrent writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
const READ_DROP_CHUNK_BYTES_DEFAULT: u64 = 32 * 1024 * 1024;
/// Upper bound (in MiB) accepted from `FREEMKV_READ_DROP_CHUNK_MIB`. 64 GiB —
/// generous for any real medium, and small enough that `n * 1024 * 1024` cannot
/// wrap `u64`. Mirrors `WRITEBACK_CHUNK_MIB_MAX`, whose identical multiply is
/// bounded for exactly this reason: without the bound, a value above 2^44
/// overflows — a panic on the first ISO open in an overflow-checked build, and in
/// release a wrap to a near-zero window that fires `drop_window` on every read.
/// Out-of-range values fall back to the default.
const READ_DROP_CHUNK_MIB_MAX: u64 = 64 * 1024;
fn read_drop_chunk_bytes() -> u64 {
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok())
.filter(|&n| n > 0)
resolve_read_drop_chunk(
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok()),
)
}
/// The pure part of [`read_drop_chunk_bytes`], split out so the bound is
/// testable without mutating process environment.
fn resolve_read_drop_chunk(mib: Option<u64>) -> u64 {
mib.filter(|&n| n > 0 && n <= READ_DROP_CHUNK_MIB_MAX)
.map(|n| n * 1024 * 1024)
.unwrap_or(READ_DROP_CHUNK_BYTES_DEFAULT)
}
@@ -171,12 +190,20 @@ impl SectorSource for FileSectorSource {
) -> Result<usize> {
let count = count as u32;
let bytes = count as usize * SECTOR_BYTES;
debug_assert!(
out.len() >= bytes,
"FileSectorSource::read_sectors: out len {} < requested {}",
out.len(),
bytes
);
// A real check, not a debug_assert: this is a public `SectorSource` impl,
// so an undersized `out` is caller input, and `out[..bytes]` below would
// panic with 'range end index out of range' in release where the assert is
// compiled away. `Drive::read_fua` already carries exactly this guard, with
// a comment recording the same panic being fixed there — this impl was
// simply never given it, and `PrefetchedSectorSource` has a regression test
// for the case that this one lacked.
if out.len() < bytes {
return Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
});
}
if count == 0 {
return Ok(0);
}
@@ -219,6 +246,40 @@ mod tests {
use std::io::Write;
use tempfile::tempdir;
/// An undersized output buffer must return an error, never panic. This is a
/// public `SectorSource` impl, so buffer length is caller input, and the guard
/// used to be a `debug_assert!` — compiled out in release, where the
/// `out[..bytes]` slice then panicked with 'range end index out of range'.
///
/// `Drive::read_fua` already carries this exact guard with a comment recording
/// the same panic being fixed there, and `PrefetchedSectorSource` has
/// `direct_read_too_small_buffer_errors` for the same case; this impl had
/// neither.
#[test]
fn read_sectors_with_an_undersized_buffer_errors_rather_than_panicking() {
let dir = tempdir().unwrap();
let iso = dir.path().join("t.iso");
make_iso(&iso, 8);
let mut src = FileSectorSource::open(&iso).expect("iso opens");
// Ask for four sectors but supply room for barely more than one.
let mut out = vec![0u8; SECTOR_BYTES + 1];
let err = src
.read_sectors(0, 4, &mut out, false)
.expect_err("an undersized buffer must be an error, not a panic");
assert!(
matches!(err, Error::DiscRead { .. }),
"expected DiscRead, got {err:?}"
);
// Exactly-sized still works, so the guard is not off by one.
let mut out = vec![0u8; 4 * SECTOR_BYTES];
assert_eq!(
src.read_sectors(0, 4, &mut out, false).unwrap(),
4 * SECTOR_BYTES
);
}
/// Build a deterministic ISO of `sectors` sectors where sector `n`
/// is filled with the byte pattern `((n & 0xff) as u8)`. Lets us
/// verify any sector by content alone.
@@ -376,19 +437,36 @@ mod tests {
// Additional coverage.
// ---------------------------------------------------------------
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or
/// reading, even at an out-of-range LBA — the early-return guard
/// runs before any I/O. Grounding: `if count == 0 { return Ok(0) }`.
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or reading,
/// even at an out-of-range LBA. Grounding: `if count == 0 { return Ok(0) }`.
///
/// The `Ok(0)` return alone proves nothing: with the guard deleted, a seek
/// past EOF succeeds (POSIX permits seeking beyond the end of a file) and a
/// zero-length `read_exact` returns `Ok(())` immediately, so the call still
/// returns `Ok(0)`. The observable difference is the file's cursor — the
/// seek MOVES it to `lba * 2048`. Assert on that, so the guard is what the
/// test is actually measuring.
#[test]
fn zero_count_returns_zero_no_io() {
let dir = tempdir().unwrap();
let path = dir.path().join("zc.iso");
make_iso(&path, 4);
let mut src = FileSectorSource::open(&path).unwrap();
let before = src.file.stream_position().expect("cursor readable");
assert_eq!(before, 0, "a freshly opened file starts at offset 0");
// LBA far past EOF — must not matter because count==0 returns early.
let mut buf = [0u8; 1];
let n = src.read_sectors(1_000_000, 0, &mut buf, false).unwrap();
assert_eq!(n, 0);
assert_eq!(
src.file.stream_position().expect("cursor readable"),
before,
"count == 0 must return before the seek — an unmoved cursor is the \
only observable proof that no I/O was issued"
);
// And the drop-window accounting must not have advanced either.
assert_eq!(src.bytes_read_since_drop, 0);
assert_eq!(src.drop_window_start, 0);
}
/// Reading past EOF must ERROR (read_exact's UnexpectedEof), never
@@ -518,4 +596,36 @@ mod tests {
lba += batch as u32;
}
}
/// `FREEMKV_READ_DROP_CHUNK_MIB` must be BOUNDED before the MiB→byte
/// multiply, exactly as its writeback twin bounds the identical multiply.
/// Unbounded, any value above 2^44 overflowed `n * 1024 * 1024`: a panic on
/// the first ISO open in an overflow-checked build, and in release a wrap to
/// a near-zero window that fires `drop_window` on essentially every read.
#[test]
fn read_drop_chunk_env_is_bounded_before_the_multiply() {
// Default when unset / zero / out of range.
assert_eq!(resolve_read_drop_chunk(None), READ_DROP_CHUNK_BYTES_DEFAULT);
assert_eq!(
resolve_read_drop_chunk(Some(0)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// The overflow value: `u64::MAX * 1024 * 1024` panicked here.
assert_eq!(
resolve_read_drop_chunk(Some(u64::MAX)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX + 1)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// In-range values convert MiB→bytes. Mutation: `* 1024` breaks this.
assert_eq!(resolve_read_drop_chunk(Some(1)), 1024 * 1024);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX)),
READ_DROP_CHUNK_MIB_MAX * 1024 * 1024
);
// And the bound itself keeps the multiply inside u64.
assert!((READ_DROP_CHUNK_MIB_MAX as u128) * 1024 * 1024 <= u64::MAX as u128);
}
}
+3 -3
View File
@@ -4,8 +4,8 @@
use std::fs::File;
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+3 -3
View File
@@ -9,7 +9,7 @@ use std::fs::File;
/// No-op stub. `FILE_FLAG_SEQUENTIAL_SCAN` can only be set at
/// `CreateFile` open time, which the plain `File::open` path does not
/// do, so there is no post-open hint to issue here.
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {
tracing::debug!(
target: "mux",
"FileSectorSource hint_sequential: windows no-op stub"
@@ -19,10 +19,10 @@ pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
/// Windows page-cache eviction is not exposed via a posix_fadvise
/// equivalent. The kernel does its own working-set management. No-op
/// for now.
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Windows async-prefetch hint. With FILE_FLAG_SEQUENTIAL_SCAN at
/// open the kernel already prefetches aggressively, so there's no
/// per-range hint we'd add on top. No-op stub for parity with the
/// posix platforms.
pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+283
View File
@@ -0,0 +1,283 @@
//! `write_image` — write an image-level source out as a sector image.
//!
//! This is the plain image writer: sectors in from any [`SectorSource`], bytes
//! out to a file, in order, once. It is what an `iso://` DESTINATION means when
//! the source is not a physical drive.
//!
//! # Why this is not `freemkv_engine::copy`
//!
//! The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
//! sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
//! exists because an optical drive returns read errors on marginal media. A
//! file-backed or synthesized source has no marginal media: a read either
//! succeeds or the underlying file is broken, and retrying it is pointless.
//!
//! Routing a non-drive source through the recovery path is not merely wasteful,
//! it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
//! both of which are empty for an already-decrypted source, so identity passes
//! for ANY such source: a second run with a different input to the same output
//! path would resume over the previous image and produce wrong content at exit
//! zero. Keeping the two paths separate makes that unrepresentable.
//!
//! So: drive sources get `freemkv_engine::copy`. Everything else gets this.
use crate::consts::SECTOR_BYTES;
use crate::error::{Error, Result};
use crate::halt::Halt;
use crate::sector::SectorSource;
use std::fs::File;
use std::io::{BufWriter, Write};
use std::path::Path;
/// Sectors per read/write batch. 4 MiB — large enough that per-call overhead
/// disappears against a file-backed source, small enough that the buffer is not
/// a notable allocation and cancellation stays responsive.
const BATCH_SECTORS: u32 = 2048;
/// Write `total_sectors` sectors from `reader` to `dest`.
///
/// Reads sequentially from LBA 0 and writes in order, so the output is a faithful
/// image of whatever the source presents — decrypted if the caller wrapped the
/// source in a [`DecryptingSectorSource`](crate::sector::decrypting::DecryptingSectorSource),
/// ciphertext if it did not. This function performs no decryption itself and makes
/// no decryption decision; that belongs to the caller, which knows whether the run
/// is `--raw`.
///
/// `on_progress` is called after each batch with the cumulative byte count, for
/// front-end progress reporting. It must not block.
///
/// `halt` is checked once per batch. On cancellation the partial file is left in
/// place — the caller decides whether a partial image is worth keeping, and
/// deleting a multi-gigabyte file the user may want to inspect is not this
/// function's call to make.
///
/// Returns the number of bytes written.
///
/// # Errors
///
/// - [`Error::Halted`] if `halt` was cancelled.
/// - [`Error::IoError`] if the destination cannot be created or written.
/// - Whatever the source's `read_sectors` returns. A short read is an error, not
/// a zero-fill: silently padding a truncated source produces an image that
/// looks complete and is not.
pub fn write_image(
reader: &mut dyn SectorSource,
dest: &Path,
total_sectors: u32,
halt: &Halt,
mut on_progress: impl FnMut(u64),
) -> Result<u64> {
if total_sectors == 0 {
return Err(Error::EmptyImage);
}
let file = File::create(dest).map_err(|source| Error::IoError { source })?;
let mut out = BufWriter::with_capacity(BATCH_SECTORS as usize * SECTOR_BYTES, file);
let mut buf = vec![0u8; BATCH_SECTORS as usize * SECTOR_BYTES];
let mut written: u64 = 0;
let mut lba: u32 = 0;
while lba < total_sectors {
if halt.is_cancelled() {
return Err(Error::Halted);
}
let count = BATCH_SECTORS.min(total_sectors - lba);
let want = count as usize * SECTOR_BYTES;
// `recovery = false`: a file-backed source ignores the flag, and a
// retry loop over a local file would only re-read the same bytes.
let got = reader.read_sectors(lba, count as u16, &mut buf[..want], false)?;
if got != want {
return Err(Error::ShortImageRead {
lba,
expected: want as u32,
got: got as u32,
});
}
out.write_all(&buf[..want])
.map_err(|source| Error::IoError { source })?;
written += want as u64;
lba += count;
on_progress(written);
}
// flush() only pushes the BufWriter's bytes into the kernel via write(2).
// It makes no durability promise at all, so returning Ok here would report
// a finished image while up to several gigabytes of it still sit in the
// page cache. A crash, a power loss, or yanking the removable/network
// volume the image was written to then leaves a truncated or empty file
// that the caller was told was complete.
//
// For a 6-90 GB image that is exactly the failure this crate treats as
// worst: success reported over wrong output. `into_inner` is used rather
// than `flush` so a buffered-write error is surfaced instead of being
// dropped on the floor by BufWriter's Drop.
let file = out.into_inner().map_err(|e| Error::IoError {
source: e.into_error(),
})?;
file.sync_all()
.map_err(|source| Error::IoError { source })?;
Ok(written)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::error::Result as FmResult;
/// A source that yields a deterministic byte per sector, so the written
/// image can be checked positionally rather than just by length.
struct PatternSource {
sectors: u32,
/// Sectors after which `read_sectors` reports a short read.
short_after: Option<u32>,
}
impl SectorSource for PatternSource {
fn capacity_sectors(&self) -> u32 {
self.sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> FmResult<usize> {
let want = count as usize * SECTOR_BYTES;
if self.short_after.is_some_and(|after| lba >= after) {
return Ok(want - 1);
}
for s in 0..count as usize {
let byte = ((lba as usize + s) % 251) as u8;
buf[s * SECTOR_BYTES..(s + 1) * SECTOR_BYTES].fill(byte);
}
Ok(want)
}
}
fn tmp(name: &str) -> std::path::PathBuf {
let mut p = std::env::temp_dir();
p.push(format!("fmkv-image-writer-{name}-{}", std::process::id()));
p
}
/// The written image is byte-for-byte what the source presented, at the
/// right offsets — not merely the right length.
#[test]
fn writes_every_sector_in_order() {
let dest = tmp("order");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 5000 * SECTOR_BYTES as u64);
let data = std::fs::read(&dest).expect("read back");
assert_eq!(data.len(), 5000 * SECTOR_BYTES);
// Spot-check across batch boundaries (BATCH_SECTORS = 2048): the last
// sector of batch 0, the first of batch 1, and the final sector.
for lba in [0usize, 2047, 2048, 4095, 4096, 4999] {
let want = (lba % 251) as u8;
assert_eq!(
data[lba * SECTOR_BYTES],
want,
"sector {lba} head byte wrong — batching lost or duplicated a sector"
);
assert_eq!(
data[(lba + 1) * SECTOR_BYTES - 1],
want,
"sector {lba} tail"
);
}
let _ = std::fs::remove_file(&dest);
}
/// A tail shorter than a full batch must still be written whole — the
/// classic off-by-one when `total_sectors` is not a batch multiple.
#[test]
fn writes_a_partial_final_batch() {
let dest = tmp("tail");
let mut src = PatternSource {
sectors: 2049,
short_after: None,
};
let n = write_image(&mut src, &dest, 2049, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 2049 * SECTOR_BYTES as u64);
assert_eq!(
std::fs::metadata(&dest).expect("stat").len(),
2049 * SECTOR_BYTES as u64
);
let _ = std::fs::remove_file(&dest);
}
/// A short read is an error. Zero-filling would yield an image that looks
/// complete and is not — the single worst outcome for an archival copy.
#[test]
fn short_read_is_an_error_not_a_zero_fill() {
let dest = tmp("short");
let mut src = PatternSource {
sectors: 4096,
short_after: Some(2048),
};
let err = write_image(&mut src, &dest, 4096, &Halt::new(), |_| {}).expect_err("must fail");
assert!(
matches!(err, Error::ShortImageRead { lba: 2048, .. }),
"got {err:?}"
);
let _ = std::fs::remove_file(&dest);
}
/// Cancellation stops the run and reports it, rather than finishing quietly
/// or reporting success on a partial image.
#[test]
fn cancellation_halts_and_reports() {
let dest = tmp("halt");
let mut src = PatternSource {
sectors: 100_000,
short_after: None,
};
let halt = Halt::new();
halt.cancel();
let err = write_image(&mut src, &dest, 100_000, &halt, |_| {}).expect_err("must halt");
assert!(matches!(err, Error::Halted), "got {err:?}");
let _ = std::fs::remove_file(&dest);
}
/// Progress is cumulative and monotonic, and its final value equals the
/// returned byte count — a front-end that trusts the callback must not end
/// up disagreeing with the return value.
#[test]
fn progress_is_cumulative_and_ends_at_the_total() {
let dest = tmp("progress");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let mut seen: Vec<u64> = Vec::new();
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |b| seen.push(b)).expect("write");
assert!(
seen.windows(2).all(|w| w[1] > w[0]),
"not monotonic: {seen:?}"
);
assert_eq!(*seen.last().expect("at least one callback"), n);
let _ = std::fs::remove_file(&dest);
}
/// A zero-sector source is a caller error, not a zero-byte image: an empty
/// ISO is never what anyone wanted, and failing here names the problem.
#[test]
fn zero_sectors_is_an_error() {
let dest = tmp("empty");
let mut src = PatternSource {
sectors: 0,
short_after: None,
};
let err = write_image(&mut src, &dest, 0, &Halt::new(), |_| {}).expect_err("must fail");
assert!(matches!(err, Error::EmptyImage), "got {err:?}");
// The destination must not have been created — a failed run leaves no
// stub for a later run to mistake for output.
assert!(!dest.exists(), "empty run created a file");
}
}
+3 -3
View File
@@ -31,6 +31,7 @@ pub(crate) mod bounded;
pub mod byte_prefetcher;
pub mod file_sector_source;
pub mod fsync;
pub mod image_writer;
pub mod sink;
mod writeback;
mod writeback_file;
@@ -40,9 +41,8 @@ pub(crate) mod platform_macos;
pub mod pipeline;
pub(crate) use writeback_file::WritebackFile;
pub use writeback_file::WritebackFile;
pub use pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, READ_PIPELINE_DEPTH, Sink, WRITE_PIPELINE_DEPTH,
WRITE_THROUGH_DEPTH,
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
+360 -67
View File
@@ -33,7 +33,7 @@
//! consumer lag detection). This is critical for diagnosing stalls.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU8, Ordering};
use std::thread::{self, JoinHandle};
use std::time::{Duration, Instant};
@@ -117,8 +117,32 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
Error::PipelineConsumerPanicked
}
/// Consumer lifecycle state, shared between the caller and the consumer thread.
///
/// A plain `AtomicBool` could not make "the caller abandons" and "the consumer
/// commits to finalising" mutually exclusive: the consumer loaded the flag, the
/// caller stored it, and the consumer then finalised the container anyway — the
/// caller reporting the rip as interrupted while a fully finalised MKV (Cues
/// written, Segment size patched) landed on disk, indistinguishable from a
/// complete one. The two transitions are therefore a single compare-exchange each,
/// out of [`state::RUNNING`]: whoever wins decides, and the loser observes the
/// winner. (`ST_RUNNING` does not exist anywhere in the crate — the constants are
/// `state::RUNNING` / `state::ABANDONED` / `state::CLOSING` below, and both
/// compare-exchange sites that must stay in step with this argument name them.)
mod state {
/// Consumer is running; neither side has committed yet.
pub const RUNNING: u8 = 0;
/// The caller gave up on the consumer and will report failure — the consumer
/// must NOT finalise the output.
pub const ABANDONED: u8 = 1;
/// The consumer has committed to `close()` (finalising the output). The caller
/// can no longer abandon it; it must wait for the result it is about to
/// produce.
pub const CLOSING: u8 = 2;
}
/// After a halt or deadline fires, spin-poll `handle.is_finished()` for
/// [`FINISH_GRACE_SECS`] before accepting the thread leak. This converts
/// `grace` before accepting the thread leak. This converts
/// the common "nearly-done" consumer (whose own bounded_syscall just
/// returned and is about to drop its output file) into a clean join,
/// releasing the file handle without waiting the full grace period.
@@ -132,11 +156,12 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
/// syscall itself; that still returns on its own (or at process exit).
fn finish_with_grace<R: Send + 'static>(
handle: thread::JoinHandle<Result<R, Error>>,
abandoned: &Arc<AtomicBool>,
state: &Arc<AtomicU8>,
grace: Duration,
leak_err: Error,
) -> Result<R, Error> {
let grace = Instant::now() + Duration::from_secs(FINISH_GRACE_SECS);
while Instant::now() < grace {
let deadline = Instant::now() + grace;
while Instant::now() < deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
@@ -145,16 +170,50 @@ fn finish_with_grace<R: Send + 'static>(
}
thread::sleep(POLL_INTERVAL);
}
// Grace expired. Signal abandonment, then log and leak. Setting the
// flag BEFORE dropping the handle guarantees the leaked consumer
// observes it the moment its wedged syscall returns: it then skips
// any further `apply` and skips `close()`, rather than running on to
// finalise the abandoned output file.
// `Release` here pairs with the `Acquire` loads in the consumer loop so
// the leaked consumer reliably observes the flag the moment its wedged
// syscall returns, even on weak memory models (ARM64/POWER) where
// `Relaxed` gives no cross-thread visibility guarantee.
abandoned.store(true, Ordering::Release);
// Grace expired. CLAIM abandonment, then log and leak. Claiming BEFORE
// dropping the handle guarantees the leaked consumer observes it the moment
// its wedged syscall returns: it then skips any further `apply` and skips
// `close()`, rather than running on to finalise the abandoned output file.
//
// A compare-exchange, not a store, because the consumer may have committed to
// `close()` in the instant between our last `is_finished()` poll and now. It
// then cannot be stopped — the finalise IS happening — so abandoning it would
// report the rip as interrupted while a valid, fully finalised container
// lands on disk. Losing the race means waiting for the result the consumer is
// already producing instead. `AcqRel` pairs with the consumer's own
// compare-exchange and with the `Acquire` loads in its drain loop, so the flag
// is reliably observed even on weak memory models (ARM64/POWER).
if state
.compare_exchange(
state::RUNNING,
state::ABANDONED,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_close_in_flight",
"pipeline consumer had already committed to finalising the output; \
waiting for it rather than reporting an unfinalised output"
);
let close_deadline = Instant::now() + grace;
while Instant::now() < close_deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
Err(payload) => Err(consumer_panicked(payload)),
};
}
thread::sleep(POLL_INTERVAL);
}
// Still finalising after a second grace window: leak and report the wedge.
// The output may end up finalised by the leaked thread — but that is now a
// wedged-`close()` case, not the check-then-finalise race.
drop(handle);
return Err(leak_err);
}
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_grace_expired",
@@ -173,14 +232,9 @@ fn finish_with_grace<R: Send + 'static>(
/// Default channel depth for callers without a specific reason to
/// pick another value. Kept conservative (4) — most callers should
/// use READ_PIPELINE_DEPTH or WRITE_PIPELINE_DEPTH instead.
/// use WRITE_PIPELINE_DEPTH instead.
pub const DEFAULT_PIPELINE_DEPTH: usize = 4;
/// Read pipeline depth. Larger buffer compensates for drive variability
/// and NFS sync_file_range stalls; keeps ISO reader thread fed even when
/// consumer blocks on write.
pub const READ_PIPELINE_DEPTH: usize = 32;
/// Write pipeline depth. Smaller buffer reduces backpressure risk when
/// sync_file_range blocks; prevents producer from accumulating too much
/// work while consumer waits for NFS to drain.
@@ -189,7 +243,8 @@ pub const WRITE_PIPELINE_DEPTH: usize = 16;
/// Channel depth for write-through pipelines. Each `send` fully
/// drains before the next can enqueue. Use this when the producer
/// must observe consumer side-effects (e.g. mapfile state) before
/// emitting the next item. Currently used by `disc::patch`.
/// emitting the next item. Used by `freemkv_engine::recovery::patch` — the
/// recovery strategy moved to that crate in 1.6.0, so there is no `patch` here.
pub const WRITE_THROUGH_DEPTH: usize = 1;
/// Outcome of [`Sink::apply`]: either keep feeding items
@@ -247,7 +302,21 @@ pub struct Pipeline<I: Send + 'static, R: Send + 'static> {
/// a syscall the consumer is currently wedged in, but it does bound
/// the damage to "whatever write is already in flight" once that
/// syscall returns, instead of running on to a clean finalise.
abandoned: Arc<AtomicBool>,
///
/// One of [`state::RUNNING`] / [`state::ABANDONED`] / [`state::CLOSING`];
/// both transitions are compare-exchanges so abandoning and finalising are
/// mutually exclusive rather than racing.
state: Arc<AtomicU8>,
/// Set by the consumer the moment an `apply` returns `Err`. The consumer keeps
/// draining the channel after that (so the producer never blocks on a dead
/// receiver) — which means a producer watching only `send`'s return value
/// cannot tell the difference between "being consumed" and "being discarded
/// after a fatal write error", and would go on reading the whole remaining
/// disc before `finish()` finally surfaced the error. This flag is that
/// missing edge: [`Pipeline::send_with_halt`] fails fast on it, and
/// [`Pipeline::consumer_failed`] exposes it to producers that use plain
/// [`Pipeline::send`].
failed: Arc<AtomicBool>,
}
impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
@@ -256,17 +325,21 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
///
/// The thread is named `freemkv-pipeline-consumer` so it shows up
/// distinctly in stack traces and `top -H`. Callers that want a
/// more specific name (e.g. `freemkv-sweep-consumer`) should use
/// [`Pipeline::spawn_named`] instead. Returns an `Error::IoError`
/// if the OS refuses the thread spawn (resource exhaustion);
/// callers already operate in fallible context, so this is
/// propagated rather than panicked.
/// more specific name should use [`Pipeline::spawn_named`] instead.
/// Returns an `Error::IoError` if the OS refuses the thread spawn
/// (resource exhaustion); callers already operate in fallible context, so
/// this is propagated rather than panicked.
///
/// Sweep uses [`Pipeline::spawn_named`] directly so the consumer
/// thread shows up as `freemkv-sweep-consumer`; mux uses
/// `freemkv-mux-consumer`. `Pipeline::spawn` (this function, with
/// the default name) is used by `disc::patch` and by the unit
/// tests in this module.
/// Inside this crate the only [`Pipeline::spawn_named`] caller is the mux
/// driver, which names its thread `freemkv-mux-consumer`. `Pipeline::spawn`
/// (this function, with the default name) is used only by the unit tests in
/// this module.
///
/// This paragraph twice named a caller that had left the crate: first
/// `disc::patch`, then Sweep and its `freemkv-sweep-consumer` thread. Both
/// went to freemkv-engine with the recovery passes in 1.6.0, and each in
/// turn sent readers hunting a component that is not here. Name callers
/// that live in THIS crate, or none.
pub fn spawn<S: Sink<I, Output = R>>(depth: usize, sink: S) -> Result<Self, Error> {
Self::spawn_named("freemkv-pipeline-consumer", depth, sink)
}
@@ -274,15 +347,17 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Like [`Pipeline::spawn`] but lets the caller supply the
/// consumer thread's name. Useful when several pipelines run in
/// the same process and stack traces / `top -H` need to tell them
/// apart (e.g. `freemkv-sweep-consumer`, `freemkv-mux-consumer`).
/// apart (e.g. `freemkv-mux-consumer`).
pub fn spawn_named<S: Sink<I, Output = R>>(
name: &str,
depth: usize,
sink: S,
) -> Result<Self, Error> {
let (tx, rx) = bounded::<I>(depth);
let abandoned = Arc::new(AtomicBool::new(false));
let abandoned_consumer = abandoned.clone();
let state = Arc::new(AtomicU8::new(state::RUNNING));
let state_consumer = state.clone();
let failed = Arc::new(AtomicBool::new(false));
let failed_consumer = failed.clone();
let handle = thread::Builder::new()
.name(name.into())
.spawn(move || -> Result<R, Error> {
@@ -313,7 +388,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// dead receiver, but we touch the output no further. The
// final post-loop abandonment check returns the error
// and skips `close()`.
if abandoned_consumer.load(Ordering::Acquire) {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
continue;
}
@@ -342,6 +417,12 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
}
first_err = Some(e);
// Publish the failure so the producer can stop
// FEEDING a dead write side instead of only learning
// about it at `finish()` — by which time it has read
// the rest of the disc. `Release` pairs with the
// `Acquire` load in `send_with_halt`.
failed_consumer.store(true, Ordering::Release);
}
}
@@ -402,13 +483,38 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// MKV Cues + patching the segment header) on a file the
// caller already reported as failed is exactly the
// write race we must not run.
if abandoned_consumer.load(Ordering::Acquire) {
return Err(Error::Halted);
}
match first_err {
Some(e) => Err(e),
None => sink.close(),
// No `close()` on this path, so there is nothing to claim —
// just report, unless the caller has already given up on us.
Some(e) => {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
Err(Error::Halted)
} else {
Err(e)
}
}
// CLAIM the finalise. A plain load here left a window in which
// the caller stored `abandoned` AFTER we read it as clear, so
// `close()` ran anyway and finalised (Cues + Segment-size
// patch) an output the caller had already reported as
// interrupted — a truncated rip indistinguishable from a
// complete one. The compare-exchange closes that window: if the
// caller got there first we skip `close()`, and if we get there
// first the caller waits for us instead of abandoning.
None => {
if state_consumer
.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
return Err(Error::Halted);
}
sink.close()
}
}
})
.map_err(|e| Error::IoError { source: e })?;
@@ -416,10 +522,24 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Ok(Pipeline {
tx,
handle,
abandoned,
state,
failed,
})
}
/// Whether the consumer's `apply` has already failed fatally.
///
/// The consumer keeps draining the channel after an `apply` error (so the
/// producer never blocks on a dead receiver), which means `send` keeps
/// succeeding and a producer has no other way to tell that everything it feeds
/// is being discarded. A long-running producer — the mux frame pump reading a
/// 60 GB title off an optical drive — should check this and unwind instead of
/// reading the rest of the disc for a write that has already failed.
/// [`Pipeline::send_with_halt`] checks it automatically.
pub fn consumer_failed(&self) -> bool {
self.failed.load(Ordering::Acquire)
}
/// Push one item. Blocks if the channel is full — that's the
/// back-pressure the whole primitive exists to provide. Returns
/// the item back if the consumer thread is gone (panicked or
@@ -449,7 +569,10 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
} else {
// Benign per-item OK: trace-level (L4) only; the
// apply-side rolling summary carries throughput.
tracing::trace!("Pipeline send: OK in {:.3}ms", elapsed.as_micros());
tracing::trace!(
"Pipeline send: OK in {:.3}ms",
elapsed.as_secs_f64() * 1000.0
);
}
}
Ok(())
@@ -464,7 +587,10 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
std::any::type_name::<I>()
);
} else {
tracing::debug!("Pipeline send: failed after {:.3}ms", elapsed.as_micros());
tracing::debug!(
"Pipeline send: failed after {:.3}ms",
elapsed.as_secs_f64() * 1000.0
);
}
}
Err(e.0)
@@ -506,11 +632,34 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// wedged inside an unkillable syscall, the producer can still
/// observe `/api/stop` and unwind within
/// [`SEND_HALT_CHECK_INTERVAL`].
/// NOT a `foo_with_X` variant of [`Pipeline::send`], despite the name.
/// The two encode OPPOSITE policies on the same event, each with its own
/// test: after the consumer's `apply` has failed, `send` still succeeds
/// (the consumer keeps draining, so the channel accepts the item), while
/// this one hands the item straight back — so a producer does not read an
/// hour of disc for a write that died on the first frame. Collapsing them
/// into one Option-parameterised method deletes one of those behaviours;
/// it was tried and `apply_error_drains_then_propagates` caught it.
pub fn send_with_halt(&self, item: I, halt: &Halt, deadline: Duration) -> Result<(), I> {
use crossbeam_channel::SendTimeoutError;
let end = Instant::now() + deadline;
let mut pending = item;
loop {
// The consumer's `apply` has failed fatally: everything sent from here
// is drained and discarded, so hand the item back at once. Without this
// the producer saw every send succeed (the channel is always being
// drained) and went on reading the whole remaining title — an hour of
// drive time on a UHD — for a write that died on the first frame, only
// learning about it at `finish()`.
if self.consumer_failed() {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: consumer apply failed, returning item={}",
std::any::type_name::<I>()
);
}
return Err(pending);
}
// Pre-check the cheap exit conditions before parking.
if halt.is_cancelled() {
if debug_enabled() {
@@ -565,7 +714,8 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let Pipeline {
tx,
handle,
abandoned: _,
state: _,
failed: _,
} = self;
// Explicit drop, although the destructure already drops `tx`
// at end-of-scope. Being explicit keeps the intent obvious.
@@ -601,11 +751,18 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Plain [`Pipeline::finish`] is preserved for callers without a
/// halt-token plumbed through; that path still blocks indefinitely
/// on `join()`, matching pre-0.20.8 behaviour.
/// Also not a `foo_with_X` variant: [`Pipeline::finish`] joins and waits
/// however long the consumer needs, while this one gives up after
/// `JOIN_TIMEOUT_SECS` and reports halted. Which is right depends on
/// whether the caller has a user waiting to cancel — the mux driver does
/// and uses this; the unit tests do not and use the plain join. Merging
/// them means picking one of those policies for both.
pub fn finish_with_halt(self, halt: Option<&Halt>) -> Result<R, Error> {
let Pipeline {
tx,
handle,
abandoned,
state,
failed: _,
} = self;
drop(tx);
let deadline = Instant::now() + Duration::from_secs(JOIN_TIMEOUT_SECS);
@@ -616,13 +773,23 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Err(payload) => Err(consumer_panicked(payload)),
};
}
if let Some(h) = halt {
if h.is_cancelled() {
return finish_with_grace(handle, &abandoned, Error::Halted);
}
if let Some(h) = halt
&& h.is_cancelled()
{
return finish_with_grace(
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::Halted,
);
}
if Instant::now() >= deadline {
return finish_with_grace(handle, &abandoned, Error::PipelineJoinTimeout);
return finish_with_grace(
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::PipelineJoinTimeout,
);
}
thread::sleep(POLL_INTERVAL);
}
@@ -1074,20 +1241,6 @@ mod tests {
/// A sink that records the exact order of items it receives, so we
/// can prove the channel is FIFO (no reordering). `close` returns
/// the recorded vector.
struct OrderSink {
seen: Vec<u64>,
}
impl Sink<u64> for OrderSink {
type Output = Vec<u64>;
fn apply(&mut self, item: u64) -> Result<Flow, Error> {
self.seen.push(item);
Ok(Flow::Continue)
}
fn close(self) -> Result<Vec<u64>, Error> {
Ok(self.seen)
}
}
/// Zero items sent: closing the pipeline immediately must still
/// call `close()` exactly once and return its Output. The consumer
/// loop's `while let Ok = rx.recv()` exits on the dropped tx with
@@ -1592,4 +1745,144 @@ mod tests {
let res = pipe.finish_with_halt(None);
assert!(matches!(res, Ok(190)), "expected Ok(190), got {res:?}");
}
/// A fatal `apply` error must become visible to the PRODUCER, not only to
/// `finish()`. The consumer keeps draining after the error (so the producer
/// never blocks on a dead receiver), which meant every `send_with_halt`
/// returned `Ok` for the rest of the run: on a 60 GB mkv:// mux that hit
/// ENOSPC on the first frame, the mux driver read the entire remaining title —
/// an hour of optical-drive time — before learning the write had died.
#[test]
fn send_with_halt_fails_fast_once_apply_has_failed() {
struct FailFirst {
failed: Arc<AtomicUsize>,
}
impl Sink<u64> for FailFirst {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
self.failed.fetch_add(1, Ordering::SeqCst);
Err(Error::DecryptFailed)
}
fn close(self) -> Result<(), Error> {
Ok(())
}
}
let applied = Arc::new(AtomicUsize::new(0));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
FailFirst {
failed: applied.clone(),
},
)
.expect("spawn");
let halt = crate::halt::Halt::new();
let deadline = Duration::from_secs(5);
// Feed one item and wait until the consumer has actually applied (and
// failed on) it, so the check below is deterministic rather than racy.
pipe.send_with_halt(0u64, &halt, deadline)
.expect("the first send lands");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && applied.load(Ordering::SeqCst) == 0 {
std::thread::sleep(Duration::from_millis(5));
}
assert_eq!(applied.load(Ordering::SeqCst), 1, "apply ran and failed");
assert!(pipe.consumer_failed(), "the failure must be observable");
// The very next send must hand the item straight back — the producer's
// signal to stop reading the disc.
assert_eq!(
pipe.send_with_halt(1u64, &halt, deadline),
Err(1u64),
"send_with_halt must fail fast once the consumer's apply has failed"
);
// The halt was never fired, so this is not a cancellation: the real error
// still comes out of finish().
assert!(matches!(pipe.finish(), Err(Error::DecryptFailed)));
assert_eq!(
applied.load(Ordering::SeqCst),
1,
"no further item was applied"
);
}
/// The abandon/finalise race. A consumer that has ALREADY committed to
/// `close()` when the grace period expires cannot be stopped — the finalise is
/// happening — so the caller must wait for its result instead of reporting the
/// output as un-finalised. With a plain flag the consumer read it as clear, the
/// caller then stored it, and the caller returned `Err(Halted)`
/// (`completed = false`) while a fully finalised MKV (Cues written, Segment
/// size patched) landed on disk — a truncated rip indistinguishable from a
/// complete one.
#[test]
fn abandon_loses_to_a_close_already_committed() {
let state = Arc::new(AtomicU8::new(state::RUNNING));
let release = Arc::new(AtomicBool::new(false));
let in_close = Arc::new(AtomicBool::new(false));
let (st, rel, inc) = (state.clone(), release.clone(), in_close.clone());
let handle = thread::Builder::new()
.name("test-consumer".into())
.spawn(move || -> Result<u64, Error> {
// Exactly what the consumer does before finalising: claim the
// right to close.
assert!(
st.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire
)
.is_ok(),
"the consumer claims the finalise first"
);
inc.store(true, Ordering::SeqCst);
// Inside `close()`, finalising the container.
while !rel.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
Ok(42)
})
.expect("spawn");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && !in_close.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
assert!(in_close.load(Ordering::SeqCst), "consumer reached close()");
// Finish the close only AFTER the first grace window has expired, so the
// caller genuinely reaches the abandon decision with a close in flight.
let rel = release.clone();
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
//
// These intervals used to be 600 ms against a 300 ms grace, which
// left NO margin: two 300 ms windows end at 600 ms, and the 250 ms
// poll cadence can push the observation later still, so on a loaded
// runner the second window expired first and the caller abandoned —
// failing with Err(Halted) against a race, not a defect.
//
// Scaled up so the jitter is small relative to the intervals: the
// first window ends at ~1.0-1.25 s and the second at ~2.0-2.25 s,
// so releasing at 1.6 s sits well inside the second with roughly
// 350 ms of slack on either side. The ordering under test is
// unchanged; only the margin is.
thread::sleep(Duration::from_millis(1600));
rel.store(true, Ordering::SeqCst);
});
let grace = Duration::from_secs(1);
let res = finish_with_grace(handle, &state, grace, Error::Halted);
assert!(
matches!(res, Ok(42)),
"a finalise already in flight must be waited for, not abandoned: {res:?}"
);
assert_eq!(
state.load(Ordering::SeqCst),
state::CLOSING,
"the caller must not have overwritten the consumer's claim"
);
}
}
+1 -1
View File
@@ -272,7 +272,7 @@ impl WritebackPipeline {
self.chunk_bytes,
self.skip_wait(),
);
if self.chunk_count % SIZE_LOG_INTERVAL == 0 {
if self.chunk_count.is_multiple_of(SIZE_LOG_INTERVAL) {
tracing::debug!(
target: "mux",
"WritebackPipeline chunk_bytes={} after {} chunks is_nfs={} degraded={}",
+84 -29
View File
@@ -30,14 +30,15 @@ pub(super) fn preallocate(file: &File, size_bytes: u64) {
);
}
/// Run `fsync` on `file` with a 60 s deadline. On timeout — and
/// likewise on halt or a lost worker we log and return `Ok(())`: the
/// kernel will still flush on close, so the data is best-effort durable.
/// The alternative (trap the thread for the rest of the rip, or return
/// an error that aborts an otherwise-complete mux) is worse, so all
/// three fallbacks return `Ok(())`. `Ok(())` from these paths is NOT a
/// durability barrier — the durable flush did not complete; only the
/// hang is bounded.
/// Run `fsync` on `file` with a 60 s deadline. On timeout, halt or a lost
/// worker we log and return `Err` — matching macOS. POSIX gives `fsync`
/// exactly one way to say "the data is on stable storage" and that is a zero
/// return; a call that never reached the device has not earned it, so `Ok(())`
/// from here means the flush completed and nothing else.
///
/// The kernel will still flush on close, so the data is usually durable
/// anyway — but that is a probability, not a barrier, and a caller that needs
/// crash-consistency has to be able to tell the difference.
///
/// ## fd-reuse safety
///
@@ -78,27 +79,7 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; kernel will flush on close (best-effort)"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::WorkerLost) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(e) => bounded_failure_to_result(e),
}
}
@@ -143,3 +124,77 @@ mod tests {
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `fsync` onto the
/// `io::Error` `durable_sync` returns.
///
/// Every arm means the same thing: **no sync observably ran**. All three used to
/// return `Ok(())`, so `WritebackFile::sync_all` reported success for a
/// durability barrier that never happened. POSIX gives `fsync` one way to say
/// "the data is on stable storage" — a zero return — and a call that never
/// reached the device has not earned it.
///
/// This mirrors the macOS `F_FULLFSYNC` mapping exactly. The two were found
/// carrying the identical defect, and a platform disagreeing with its sibling
/// about whether a failed sync is an error is the "works on my platform" class
/// this crate has been bitten by before — most recently an over-length SCSI CDB
/// that macOS rejected and the other two silently truncated.
///
/// No message text (this crate ships no user-facing English): the kind, and
/// `EIO` for the worker-lost case, are the signal; `tracing` carries the detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
// EIO, matching the macOS sibling: a consumer distinguishing these
// three failures does so on the same value on every platform.
// ErrorKind::Other carries nothing a caller can branch on.
Err(crate::error::Error::SyncWorkerLost.into())
}
}
}
#[cfg(test)]
mod bounded_failure_tests {
use super::*;
use crate::io::bounded::BoundedError;
/// Every bounded-fsync failure must be an error. Asserted per variant rather
/// than as a loop so a new variant defaulting to Ok cannot slip through.
#[test]
fn no_bounded_fsync_failure_maps_to_ok() {
assert_eq!(
bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out fsync must be an error")
.kind(),
io::ErrorKind::TimedOut
);
assert_eq!(
bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted fsync must be an error")
.kind(),
io::ErrorKind::Interrupted
);
assert!(
bounded_failure_to_result(BoundedError::WorkerLost).is_err(),
"a lost fsync worker must be an error"
);
}
}
+107 -5
View File
@@ -105,15 +105,46 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
Err(e) => bounded_failure_to_result(e),
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `F_FULLFSYNC`
/// onto the `io::Error` `durable_sync` returns.
///
/// Every arm here means the same thing: **no sync observably ran**. All three
/// previously returned `Ok(())`, so `WritebackFile::sync_all` reported success
/// for a durability barrier that never happened — a total failure exiting 0,
/// with only a log line to distinguish it. POSIX gives `fsync` exactly one way
/// to say "the data is on stable storage" and that is a zero return; a call
/// that never reached the device has not earned it.
///
/// The errors carry no message text (this crate ships no user-facing English):
/// the kind, and `EIO` for the worker-lost case, are the whole signal, and the
/// `tracing` lines above/below carry the operator detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; kernel will flush on close (best-effort)"
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Ok(())
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncWorkerLost.into())
}
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
}
}
@@ -156,4 +187,75 @@ mod tests {
// durable_sync must complete without error on the local tempfile.
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
}
/// Every `BoundedError` arm of the bounded `F_FULLFSYNC` means no sync
/// observably ran. All three returned `Ok(())`, so `sync_all` reported a
/// durability barrier that never happened — the caller could not tell a
/// completed flush from a skipped one by any means except reading a log.
///
/// Asserted on the concrete `ErrorKind` / `errno` each arm must produce,
/// so a future arm that quietly reverts to `Ok(())` fails here.
#[test]
fn every_bounded_failure_is_reported_as_an_error() {
use crate::io::bounded::BoundedError;
let timeout = bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out F_FULLFSYNC must be an error");
assert_eq!(
timeout.kind(),
io::ErrorKind::TimedOut,
"a timed-out F_FULLFSYNC must not be reported as a completed sync"
);
let halted = bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted F_FULLFSYNC must be an error");
assert_eq!(
halted.kind(),
io::ErrorKind::Interrupted,
"a halted F_FULLFSYNC must not be reported as a completed sync"
);
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each
// carries its own numeric code through the "E<code>" prefix that
// `From<Error> for io::Error` mints — the only shape `error_code`
// recognises. A bare `ErrorKind` cannot be classified, which is how a
// user cancel here used to read as a hard I/O failure.
let lost = bounded_failure_to_result(BoundedError::WorkerLost)
.expect_err("a lost F_FULLFSYNC worker must be an error");
assert!(
lost.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_WORKER_LOST)),
"a lost worker must be identifiable, got {lost}"
);
assert!(
timeout
.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_TIMEOUT)),
"a timeout must be distinguishable from a lost worker, got {timeout}"
);
assert!(
crate::error::is_halt(&halted),
"a halt must satisfy the crate's own is_halt(), or the CLI reports a \
user cancel as a failure; got {halted}"
);
}
/// The failure path must be reachable through the public surface: a
/// `WritebackFile::sync_all` that hits any of these arms must surface an
/// `Err`, not a silent `Ok`. Pinned at the mapping boundary because the
/// timeout itself is not deterministically inducible in a unit test.
#[test]
fn bounded_failures_are_never_mapped_to_ok() {
use crate::io::bounded::BoundedError;
for e in [
BoundedError::Timeout,
BoundedError::Halted,
BoundedError::WorkerLost,
] {
assert!(
bounded_failure_to_result(e).is_err(),
"a bounded F_FULLFSYNC failure must never map to Ok"
);
}
}
}
+21 -15
View File
@@ -97,7 +97,7 @@ fn writeback_chunk_bytes() -> u64 {
.unwrap_or(WRITEBACK_CHUNK_BYTES_DEFAULT)
}
pub(crate) struct WritebackFile {
pub struct WritebackFile {
file: File,
pipeline: WritebackPipeline,
pos: u64,
@@ -115,7 +115,7 @@ impl WritebackFile {
/// once so the pipeline starts tracking from wherever the file
/// already is (typically 0 for fresh files; non-zero for resumed
/// or appended files).
pub(crate) fn new(mut file: File) -> io::Result<Self> {
pub fn new(mut file: File) -> io::Result<Self> {
let pos = file.stream_position()?;
let pipeline = WritebackPipeline::new(&file, pos, writeback_chunk_bytes());
Ok(Self {
@@ -136,7 +136,7 @@ impl WritebackFile {
/// [`Self::create_with_size_hint`] so the kernel can pre-reserve
/// extents.
#[allow(dead_code)]
pub(crate) fn create(path: &Path) -> io::Result<Self> {
pub fn create(path: &Path) -> io::Result<Self> {
let file = File::create(path)?;
Self::new(file)
}
@@ -153,7 +153,7 @@ impl WritebackFile {
/// On platforms without an extent-preallocation primitive this is
/// equivalent to `create` — the size hint is dropped after a debug
/// log.
pub(crate) fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
pub fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
let file = File::create(path)?;
platform::preallocate(&file, size_bytes);
Self::new(file)
@@ -163,7 +163,7 @@ impl WritebackFile {
/// wrap it. Mirrors `File::open` semantics for the writable case
/// — used by patch / resume paths that mutate an existing ISO in
/// place.
pub(crate) fn open(path: &Path) -> io::Result<Self> {
pub fn open(path: &Path) -> io::Result<Self> {
let file = OpenOptions::new().write(true).open(path)?;
Self::new(file)
}
@@ -178,13 +178,20 @@ impl WritebackFile {
/// is left to the kernel's normal flush-on-close path — best
/// effort, but bounded.
///
/// IMPORTANT: on Linux/macOS a successful `Ok(())` does NOT
/// guarantee the data is durable if the bounded fsync timed out or
/// was halted — only the hang is bounded, the fsync may not have
/// completed. Callers needing crash-consistency (e.g. mux-finish
/// then external commit/DB update) must not treat `Ok(())` as a
/// durability barrier.
pub(crate) fn sync_all(&mut self) -> io::Result<()> {
/// A bounded-fsync failure is returned as an `Err` on BOTH platforms, so
/// `Ok(())` means the flush completed and a caller needing
/// crash-consistency can treat it as a durability barrier.
///
/// The three causes are DISTINGUISHABLE by numeric code, because a caller
/// should not retry a lost worker the way it retries a timeout, and must
/// not report a user cancel as a failure:
///
/// * [`E_SYNC_TIMEOUT`](crate::error::E_SYNC_TIMEOUT) — deadline expired
/// * [`E_HALTED`](crate::error::E_HALTED) — cancelled;
/// [`is_halt`](crate::error::is_halt) recognises it
/// * [`E_SYNC_WORKER_LOST`](crate::error::E_SYNC_WORKER_LOST) — the worker
/// thread died before reporting
pub fn sync_all(&mut self) -> io::Result<()> {
if self.seek_count > 0 {
tracing::debug!(
target: "mux",
@@ -256,9 +263,8 @@ impl super::sink::SequentialSink for WritebackFile {
/// the same work [`Self::sync_all`] does. Implemented explicitly (no
/// blanket impl) so a `dyn SequentialSink` / `dyn RandomAccessSink`
/// `finish()` actually finalises + fsyncs instead of hitting a no-op
/// default. Note the bounded-fsync caveat from [`Self::sync_all`]
/// applies: `Ok(())` is not a durability barrier if the fsync timed
/// out or was halted.
/// default. A bounded-fsync failure surfaces as an `Err` here, on every
/// platform, exactly as it does from [`Self::sync_all`].
fn finish(&mut self) -> io::Result<()> {
self.sync_all()
}
+807 -91
View File
File diff suppressed because it is too large Load Diff
+45 -16
View File
@@ -100,10 +100,10 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<DiscMetadata>
// Disc-set position is disc-global; first one we successfully
// read wins. (All bdmt_*.xml on a given disc carry the same
// value in practice.)
if out.disc_number.is_none() {
if let Some(ds) = disc_set {
out.disc_number = Some(ds);
}
if out.disc_number.is_none()
&& let Some(ds) = disc_set
{
out.disc_number = Some(ds);
}
}
@@ -184,20 +184,20 @@ fn extract_title(xml_text: &str) -> Option<String> {
// xml::text already trims its result, so an empty string after
// extraction means a genuinely empty element.
for tag in ["name", "title"] {
if let Some(s) = xml::text(xml_text, tag) {
if !s.is_empty() {
return Some(s);
}
if let Some(s) = xml::text(xml_text, tag)
&& !s.is_empty()
{
return Some(s);
}
}
// tableOfContents/titleName: search inside the toc block so we
// don't accidentally pick a stray <titleName> from elsewhere.
if let Some((s, e)) = xml::find_element(xml_text, "tableOfContents", 0) {
let block = &xml_text[s..e];
if let Some(t) = xml::text(block, "titleName") {
if !t.is_empty() {
return Some(t);
}
if let Some(t) = xml::text(block, "titleName")
&& !t.is_empty()
{
return Some(t);
}
}
None
@@ -370,10 +370,10 @@ mod tests {
if let Some(d) = desc {
meta.descriptions.insert(lang.to_string(), d);
}
if meta.disc_number.is_none() {
if let Some(d) = ds {
meta.disc_number = Some(d);
}
if meta.disc_number.is_none()
&& let Some(d) = ds
{
meta.disc_number = Some(d);
}
}
@@ -566,4 +566,33 @@ mod tests {
let (title, _, _) = parse_bdmt_xml(xml).unwrap();
assert_eq!(title, "Real Title");
}
/// `is_bdmt_filename` must recognize the `bdmt_<lang>.xml` convention
/// and reject everything else — it drives `detect`'s directory scan.
/// Mutation: stub the return to a constant `true`/`false` → every
/// directory listing (or none) would match regardless of filename.
#[test]
fn is_bdmt_filename_matches_convention_only() {
assert!(is_bdmt_filename("bdmt_eng.xml"));
assert!(is_bdmt_filename("BDMT_FRA.XML"));
assert!(!is_bdmt_filename("bdmt_engl.xml"));
assert!(!is_bdmt_filename("index.bdmv"));
assert!(!is_bdmt_filename("foo.xml"));
}
/// Spec: "Disc 1 of 1" (a single-disc release whose bdmt XML still
/// carries `<di:numSets>1</di:numSets>`) is a valid, non-nonsensical
/// pair — `total < 1` must reject only `total == 0`, not `total == 1`.
/// Mutation: `total < 1` -> `total == 1` or `total <= 1` would reject
/// this legitimate (1, 1) pair as if it were malformed.
#[test]
fn disc_set_allows_single_disc_release() {
let xml = r#"<discInfo xmlns:di="urn:BDA:bdmv;disclibmeta">
<di:name>Film</di:name>
<di:discNumber>1</di:discNumber>
<di:numSets>1</di:numSets>
</discInfo>"#;
let (_, _, set) = parse_bdmt_xml(xml).unwrap();
assert_eq!(set, Some((1, 1)));
}
}
+387 -1
View File
@@ -989,7 +989,18 @@ impl<'a> Reader<'a> {
}
fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> {
if self.pos + n > self.data.len() {
// `n` is attacker-supplied: it comes from a JVMS `u4` attribute_length
// / code_length (§4.7, §4.7.3) or a `u2` Utf8 length (§4.4.7). Unlike
// the fixed-width readers above, whose `self.pos + k` cannot leave the
// buffer's own address range, `self.pos + n` can wrap — on a 32-bit
// target a `u4` length near 0xFFFF_FFFF plus a non-zero `pos` panics
// in debug and in release wraps to a SMALL end offset that passes the
// bounds check, after which the slice index itself panics. Checked, so
// an out-of-range length is the EOF error it always should have been.
let Some(end) = self.pos.checked_add(n) else {
return Err(Error::UnexpectedEof { needed });
};
if end > self.data.len() {
return Err(Error::UnexpectedEof { needed });
}
let s = &self.data[self.pos..self.pos + n];
@@ -1006,6 +1017,46 @@ impl<'a> Reader<'a> {
mod tests {
use super::*;
/// `Reader::slice` takes an attacker-supplied length: a JVMS `u4`
/// `attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8
/// length (§4.4.7). Adding it to `pos` without a wrap check panics on
/// overflow in debug and, in release, wraps to a small end offset that
/// slips past the bounds check and then panics inside the slice index.
/// Both are panics escaping a parser whose whole input is untrusted disc
/// bytes; the contract is an EOF error.
#[test]
fn slice_rejects_a_length_that_would_wrap_pos() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
r.u64("advance pos").expect("8 bytes available");
// pos is now 8; usize::MAX would wrap the end offset to 7.
match r.slice(usize::MAX, "wrapping length") {
Err(Error::UnexpectedEof { .. }) => {}
Err(other) => panic!("expected UnexpectedEof, got {other:?}"),
Ok(s) => panic!("expected UnexpectedEof, got a {}-byte slice", s.len()),
}
// The reader must not have consumed anything.
match r.slice(8, "remaining bytes") {
Ok(s) => assert_eq!(s.len(), 8, "pos moved on the rejected slice"),
Err(e) => panic!("the remaining 8 bytes must still be readable: {e:?}"),
}
}
/// The ordinary out-of-range case (no wrap) must keep returning EOF, and
/// an exactly-fitting length must still succeed — the check is `>`, not
/// `>=`.
#[test]
fn slice_boundary_is_inclusive_of_the_final_byte() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
assert_eq!(r.slice(16, "whole buffer").expect("exact fit").len(), 16);
let mut r = Reader::new(&data);
assert!(matches!(
r.slice(17, "one past"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn rejects_non_class_bytes() {
match ClassFile::parse(b"\x00\x01\x02\x03DEAD") {
@@ -1337,4 +1388,339 @@ mod tests {
let _ = decode_modified_utf8(&buf);
}
}
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness
//
// These exercise plain data accessors on an already-parsed pool
// (built via the test-only `from_entries` constructor) — not the
// untrusted-bytes parsing path, just "does the right variant map to
// the right Option value."
// -----------------------------------------------------------------
fn sample_pool() -> ConstantPool {
// index: 0=Empty (reserved), 1=Utf8("Hello"), 2=Integer(42),
// 3=String{string_index:1}, 4=Class{name_index:1}, 5=Float(1.5),
// 6=Long(9), 7=Empty (2-slot tail), 8=Double(2.5), 9=Empty (tail).
ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("Hello".to_string()),
CpInfo::Integer(42),
CpInfo::String { string_index: 1 },
CpInfo::Class { name_index: 1 },
CpInfo::Float(1.5),
CpInfo::Long(9),
CpInfo::Empty,
CpInfo::Double(2.5),
CpInfo::Empty,
])
}
#[test]
fn constant_pool_string_resolves_through_string_index() {
let pool = sample_pool();
// index 3 is CpInfo::String{string_index: 1} -> utf8(1) = "Hello".
assert_eq!(pool.string(3), Some("Hello"));
// Wrong variant (Integer at index 2) must not resolve as a string.
assert_eq!(pool.string(2), None);
// Out of range index.
assert_eq!(pool.string(999), None);
}
#[test]
fn constant_pool_integer_resolves_only_integer_entries() {
let pool = sample_pool();
assert_eq!(pool.integer(2), Some(42));
// Wrong variant (Utf8 at index 1) must not resolve as an integer.
assert_eq!(pool.integer(1), None);
assert_eq!(pool.integer(999), None);
}
#[test]
fn constant_pool_load_constant_display_covers_ldc_operand_kinds() {
let pool = sample_pool();
assert_eq!(
pool.load_constant_display(1),
Some("utf8:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(2), Some("int:42".to_string()));
assert_eq!(
pool.load_constant_display(3),
Some("str:\"Hello\"".to_string())
);
assert_eq!(
pool.load_constant_display(4),
Some("class:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(5), Some("float:1.5".to_string()));
assert_eq!(pool.load_constant_display(6), Some("long:9".to_string()));
assert_eq!(
pool.load_constant_display(8),
Some("double:2.5".to_string())
);
// A variant with no display arm (e.g. reserved Empty slot) -> None.
assert_eq!(pool.load_constant_display(0), None);
assert_eq!(pool.load_constant_display(999), None);
}
#[test]
fn constant_pool_len_and_is_empty() {
let pool = sample_pool();
assert_eq!(pool.len(), 10);
assert!(!pool.is_empty());
let empty = ConstantPool::from_entries(vec![]);
assert_eq!(empty.len(), 0);
assert!(empty.is_empty());
}
#[test]
fn constant_pool_iter_yields_index_and_entry_pairs() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("A".to_string()),
CpInfo::Integer(7),
]);
let indices: Vec<u16> = pool.iter().map(|(i, _)| i).collect();
assert_eq!(indices, vec![0, 1, 2]);
// Confirm the entries themselves come through, not an empty iterator.
let utf8_at_1 = pool.iter().find(|(i, _)| *i == 1).map(|(_, e)| match e {
CpInfo::Utf8(s) => s.as_str(),
_ => "?",
});
assert_eq!(utf8_at_1, Some("A"));
}
fn class_file_with(this_class: u16, super_class: u16, pool: ConstantPool) -> ClassFile {
ClassFile {
minor_version: 0,
major_version: 0,
constant_pool: pool,
access_flags: 0,
this_class,
super_class,
interfaces: Vec::new(),
fields: Vec::new(),
methods: Vec::new(),
attributes: Vec::new(),
}
}
#[test]
fn this_class_name_and_super_class_name_resolve_distinct_indices() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("com/example/Foo".to_string()),
CpInfo::Utf8("com/example/Bar".to_string()),
CpInfo::Class { name_index: 1 },
CpInfo::Class { name_index: 2 },
]);
let cf = class_file_with(3, 4, pool);
assert_eq!(cf.this_class_name(), Some("com/example/Foo"));
assert_eq!(cf.super_class_name(), Some("com/example/Bar"));
// this_class index pointing at a non-Class entry must not resolve.
let pool2 = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("not a class ref".to_string()),
]);
let cf2 = class_file_with(1, 1, pool2);
assert_eq!(cf2.this_class_name(), None);
assert_eq!(cf2.super_class_name(), None);
}
#[test]
fn member_descriptor_resolves_the_descriptor_not_the_name() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("doStuff".to_string()), // index 1: name
CpInfo::Utf8("()V".to_string()), // index 2: descriptor
]);
let cf = class_file_with(0, 0, pool);
let m = Member {
access_flags: 0,
name_index: 1,
descriptor_index: 2,
attributes: Vec::new(),
};
assert_eq!(cf.member_descriptor(&m), Some("()V"));
assert_ne!(cf.member_descriptor(&m), Some("doStuff"));
}
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness
//
// Mirrors `slice_boundary_is_inclusive_of_the_final_byte`: an
// exact-fit read must succeed, one byte short must fail. Plus
// positive-value tests so a scrambled byte assembly (not just an
// out-of-bounds read) would be caught.
// -----------------------------------------------------------------
#[test]
fn u16_boundary_is_inclusive_of_the_final_byte() {
let data = [0xAB, 0xCD];
let mut r = Reader::new(&data);
assert_eq!(r.u16("exact fit").expect("2 bytes available"), 0xABCD);
let data = [0xAB];
let mut r = Reader::new(&data);
assert!(matches!(
r.u16("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u16_decodes_big_endian_value() {
let data = [0x01, 0x02];
let mut r = Reader::new(&data);
assert_eq!(r.u16("value").unwrap(), 0x0102);
}
#[test]
fn u32_boundary_is_inclusive_of_the_final_byte() {
let data = [0x00, 0x00, 0x00, 0x2A];
let mut r = Reader::new(&data);
assert_eq!(r.u32("exact fit").expect("4 bytes available"), 42);
let data = [0x00, 0x00, 0x00];
let mut r = Reader::new(&data);
assert!(matches!(
r.u32("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u32_decodes_big_endian_value() {
let data = [0x00, 0x00, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u32("value").unwrap(), 1337);
}
#[test]
fn u64_boundary_is_inclusive_of_the_final_byte() {
// pos == 0, buffer exactly 8 bytes: must succeed.
let data = [0, 0, 0, 0, 0, 0, 0, 0x7B]; // 123
let mut r = Reader::new(&data);
assert_eq!(r.u64("exact fit").expect("8 bytes available"), 123);
// pos == 0, buffer one byte short of 8: must fail cleanly, not
// panic on the internal self.data[self.pos + 7] index.
let data = [0u8; 7];
let mut r = Reader::new(&data);
assert!(matches!(
r.u64("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u64_decodes_big_endian_value() {
let data = [0, 0, 0, 0, 0, 0, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u64("value").unwrap(), 1337);
}
// -----------------------------------------------------------------
// decode_modified_utf8: 3-byte (BMP) decode path
// -----------------------------------------------------------------
#[test]
fn modified_utf8_three_byte_cjk() {
// U+3042 (hiragana あ) in modified UTF-8: 1110xxxx 10xxxxxx 10xxxxxx
// = 0xE3 0x81 0x82.
let s = decode_modified_utf8(&[0xE3, 0x81, 0x82]).unwrap();
assert_eq!(s, "\u{3042}");
}
#[test]
fn modified_utf8_three_byte_rejects_bad_first_continuation() {
// Valid lead byte (0xE3), but the first continuation byte is not
// 10xxxxxx (0x01 instead) — must be rejected, proving the first
// `& 0xC0 != 0x80` check is live.
assert!(decode_modified_utf8(&[0xE3, 0x01, 0x82]).is_err());
}
#[test]
fn modified_utf8_three_byte_rejects_bad_second_continuation() {
// Valid lead + first continuation, but the second continuation
// byte is not 10xxxxxx — proves the second check is independently
// live (not short-circuited by the first).
assert!(decode_modified_utf8(&[0xE3, 0x81, 0x01]).is_err());
}
// -----------------------------------------------------------------
// read_constant_pool: Long/Double two-slot skip, real byte parsing
// -----------------------------------------------------------------
#[test]
fn constant_pool_long_entry_occupies_two_slots_via_real_parse() {
// Real class-file bytes (not the `from_entries` synthetic ctor):
// magic + minor/major + cp_count=4 + tag=5 (Long, 8-byte payload
// at index 1, reserved slot at index 2) + tag=1 (Utf8 at index 3)
// + empty access_flags/this/super/interfaces/fields/methods/attrs.
let mut buf = vec![
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, // minor
0x00, 0x34, // major
0x00, 0x04, // cp_count = 4 (0=Empty,1=Long,2=Empty tail,3=Utf8)
5, // Long tag
];
buf.extend_from_slice(&0x1122_3344_5566_7788u64.to_be_bytes()); // 8-byte payload
buf.push(1); // Utf8 tag
let name = b"marker";
buf.extend_from_slice(&(name.len() as u16).to_be_bytes());
buf.extend_from_slice(name);
// access_flags, this_class, super_class, interfaces_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0, 0, 0]);
// fields_count, methods_count, attributes_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0]);
let cf = ClassFile::parse(&buf).expect("well-formed synthetic class file");
assert_eq!(cf.constant_pool.len(), 4);
// The Long occupies indices 1 AND 2 (its reserved tail slot).
// The Utf8 must resolve at index 3 = long_index(1) + 2, NOT +1.
assert_eq!(cf.constant_pool.utf8(3), Some("marker"));
// Index 2 is the reserved tail slot: not a Utf8, must not
// resolve as one (guards against the Utf8 landing one slot early).
assert_eq!(cf.constant_pool.utf8(2), None);
match cf.constant_pool.get(1) {
Some(CpInfo::Long(v)) => assert_eq!(*v, 0x1122_3344_5566_7788u64 as i64),
other => panic!("expected Long at index 1, got {:?}", other),
}
}
// -----------------------------------------------------------------
// instruction_size: tableswitch/lookupswitch with non-degenerate
// low/high/npairs (the existing tests only cover low==high==0 and
// npairs==0, which can't distinguish `-` from `+` in the entry-count
// arithmetic).
// -----------------------------------------------------------------
#[test]
fn instruction_size_tableswitch_non_degenerate_range() {
// low=1, high=4 -> 4 entries (high-low+1 = 4). A `-`->`+` mutation
// on that arithmetic would instead compute high+low+1 = 6.
let mut code = vec![TABLESWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default offset
code.extend_from_slice(&1i32.to_be_bytes()); // low = 1
code.extend_from_slice(&4i32.to_be_bytes()); // high = 4
code.extend_from_slice(&[0; 16]); // 4 jump entries * 4 bytes
// total = 1 (opcode) + 3 (pad) + 12 (default/low/high) + 16 (entries) = 32
assert_eq!(instruction_size(&code, 0), Some(32));
}
#[test]
fn instruction_size_lookupswitch_non_degenerate_npairs() {
// npairs = 3 -> 3 * 8 = 24 bytes of pairs.
let mut code = vec![LOOKUPSWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default
code.extend_from_slice(&3i32.to_be_bytes()); // npairs = 3
code.extend_from_slice(&[0; 24]); // 3 pairs
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
}
+242 -56
View File
@@ -16,7 +16,7 @@ use std::collections::HashMap;
/// Cheap signature check: a Criterion disc ships `streamproperties.xml`
/// inside a `/BDMV/JAR/*` archive.
pub fn detect(udf: &UdfFs) -> bool {
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "streamproperties.xml")
}
@@ -36,17 +36,18 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// Stream number mapping from playbackconfig.xml
let mut stream_map: HashMap<String, u16> = HashMap::new();
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml") {
if let Ok(pc_text) = std::str::from_utf8(&pc_data) {
parse_playback_config(pc_text, &mut stream_map);
}
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml")
&& let Ok(pc_text) = std::str::from_utf8(&pc_data)
{
parse_playback_config(pc_text, &mut stream_map);
}
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map);
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map)?;
let mut labels = Vec::new();
for (info, &stream_num) in stream_infos.iter().zip(stream_nums.iter()) {
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type: info.stream_type,
language: info.language.clone(),
@@ -76,12 +77,41 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// map-assigned one. (Both numbering domains are 1-based per type, and
/// `apply_labels` matches on `(type, stream_number)`, so a collision
/// would mislabel tracks.)
fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>) -> Vec<u16> {
// Numbers already claimed by the map, per type.
///
/// Returns `None` when the 1-based stream-number space is exhausted — every
/// number in `1..=u16::MAX` for that type is either already claimed by the map
/// or already synthesized. That is unreachable on real media: the BD STN_table
/// carries at most 32 primary audio and 32 PG streams per playlist, so the
/// 65535-wide space leaves >2000x headroom. It IS reachable from a crafted
/// `streamproperties.xml` listing >65535 stream entries, and the only correct
/// answers there are "fail the parse" or "emit colliding numbers"; we fail.
///
/// The skip search is bounded by the numbering space itself: a `u16`
/// `saturating_add` here parked the counter at `u16::MAX` forever whenever the
/// map also claimed `u16::MAX`, turning an overflow guard into a hang that
/// `apply()`'s `catch_unwind` cannot interrupt. The counters are therefore
/// widened to `u32` so the skip loop strictly increases toward a fixed ceiling
/// (guaranteeing termination) and exhaustion is reported rather than absorbed.
fn assign_stream_numbers(
infos: &[StreamInfo],
stream_map: &HashMap<String, u16>,
) -> Option<Vec<u16>> {
/// One past the last assignable stream number, as a `u32` so the
/// counters can step off the end of the `u16` domain without wrapping.
const NUMBER_SPACE_END: u32 = u16::MAX as u32 + 1;
// Numbers already claimed by the map, per type. A map value of 0 is NOT a
// claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable.
// Treat 0 as "unmapped" here (defense in depth — parse_playback_config also
// filters it) so such a stream gets a real synthesized number instead of an
// orphan 0 that collides with / shadows a genuine stream 1.
let mut taken_audio: Vec<u16> = Vec::new();
let mut taken_sub: Vec<u16> = Vec::new();
for info in infos {
if let Some(&n) = stream_map.get(&info.id) {
if n == 0 {
continue;
}
match info.stream_type {
StreamLabelType::Audio => taken_audio.push(n),
StreamLabelType::Subtitle => taken_sub.push(n),
@@ -89,32 +119,42 @@ fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>
}
}
let mut audio_idx: u16 = 1;
let mut sub_idx: u16 = 1;
let mut audio_idx: u32 = 1;
let mut sub_idx: u32 = 1;
let mut out = Vec::with_capacity(infos.len());
for info in infos {
let n = match stream_map.get(&info.id).copied() {
Some(n) => n,
None => {
Some(n) if n != 0 => n,
_ => {
let (idx, taken) = match info.stream_type {
StreamLabelType::Audio => (&mut audio_idx, &taken_audio),
StreamLabelType::Subtitle => (&mut sub_idx, &taken_sub),
};
// Advance past any number already claimed via the map.
// saturating: a crafted XML with >65k stream entries must
// not overflow (panic in debug, wrap-to-0 in release) on
// untrusted disc bytes.
while taken.contains(idx) {
*idx = idx.saturating_add(1);
// Advance past any number already claimed via the map. The
// counter strictly increases and NUMBER_SPACE_END is fixed, so
// this terminates in at most 65535 steps for any input.
while *idx < NUMBER_SPACE_END && taken.contains(&(*idx as u16)) {
*idx += 1;
}
let n = *idx;
*idx = idx.saturating_add(1);
if *idx >= NUMBER_SPACE_END {
// Numbering space exhausted. Emitting anything here would
// either wrap to 0 (unmatchable) or duplicate a number
// already bound to a different stream, so the parse fails.
tracing::warn!(
streams = infos.len(),
"criterion: 1-based u16 stream-number space exhausted; \
refusing to synthesize a colliding stream number"
);
return None;
}
let n = *idx as u16;
*idx += 1;
n
}
};
out.push(n);
}
out
Some(out)
}
struct StreamInfo {
@@ -182,14 +222,13 @@ fn parse_playback_config(text: &str, map: &mut HashMap<String, u16>) {
if let (Some(stream_id_str), Some(info_id)) = (
xml::text(block, "StreamID"),
xml::text(block, "StreamInfo_ID"),
) {
if let Ok(stream_num) = stream_id_str.parse::<u16>() {
// Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it.
if stream_num != 0 {
map.insert(info_id, stream_num);
}
) && let Ok(stream_num) = stream_id_str.parse::<u16>()
{
// Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it.
if stream_num != 0 {
map.insert(info_id, stream_num);
}
}
from = end;
@@ -219,11 +258,89 @@ mod tests {
info("a1", StreamLabelType::Audio),
info("s0", StreamLabelType::Subtitle),
];
let nums = assign_stream_numbers(&infos, &HashMap::new());
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Per-type 1-based: audio 1,2 ; subtitle 1.
assert_eq!(nums, vec![1, 2, 1]);
}
/// Immunity pin. `parse_stream_infos` emits one `StreamInfo` per
/// `*StreamInfos` element unconditionally — no filter, no `continue` — so
/// an element whose fields are missing or unrecognized still occupies its
/// position, and `assign_stream_numbers` still spends a number on it.
///
/// That is the property that keeps this parser out of the failure mode
/// where a skipped entry pulls every later label one stream forward. It
/// is load-bearing for the fallback path specifically: with no
/// `playbackconfig.xml` the numbers come purely from position in this
/// list, so dropping an element there would shift the rest.
///
/// Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
/// real audio streams renumber to 1 and 2.
/// Immunity pin, section-boundary half. Each stream here is one closed XML
/// element, and every field is read out of `&text[start..end]` — the range
/// `xml::find_element` returned — so one element can never absorb the next
/// one's fields, however the document is malformed around it. Contrast the
/// flat-string walk in pixelogic, where a section whose end marker is
/// missing keeps consuming entries as STN slots.
///
/// The missing-boundary case fails closed. An element with no close tag of
/// its own ends at the NEXT close tag, so it absorbs the element behind it
/// — the list comes back SHORTER. It cannot come back longer: nothing
/// outside a returned range is ever read as a stream, and `find_element`
/// yields `None` rather than a range running to EOF when no close tag
/// exists at all. A malformed document can cost this parser a slot; it can
/// never invent one.
///
/// Mutation: read fields from the document rather than the element's
/// range, or let a close-less element run to EOF → the trailing elements
/// re-enter the list as extra streams.
#[test]
fn an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it() {
let sp = concat!(
"<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID></AudioStreamInfos>",
// No `</AudioStreamInfos>` for this one.
"<AudioStreamInfos><ID>a1</ID><LangInfoID>FRA</LangInfoID>",
"<AudioStreamInfos><ID>a2</ID><LangInfoID>DEU</LangInfoID></AudioStreamInfos>",
);
let infos = parse_stream_infos(sp);
assert_eq!(
infos.iter().map(|i| i.id.as_str()).collect::<Vec<_>>(),
vec!["a0", "a1"],
"the close-less element absorbs the one behind it — two slots, not \
three, and never four"
);
assert_eq!(infos[1].language, "fra", "and keeps its own leading fields");
// With no close tag anywhere behind it, the element is not returned at
// all and the walk ends — the tail of the document never becomes a
// stream list.
let no_close = "<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID>";
assert!(parse_stream_infos(no_close).is_empty());
}
#[test]
fn unusable_stream_element_still_occupies_its_position() {
let sp = r#"
<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG_US</LangInfoID></AudioStreamInfos>
<AudioStreamInfos></AudioStreamInfos>
<AudioStreamInfos><ID>a2</ID><LangInfoID>FRA</LangInfoID><Content>COMMENTARY</Content></AudioStreamInfos>
<SubtitleStreamInfos><ID>s0</ID><LangInfoID></LangInfoID><Qualifier>WAT</Qualifier></SubtitleStreamInfos>
<SubtitleStreamInfos><ID>s1</ID><LangInfoID>ENG</LangInfoID><Qualifier>SDH</Qualifier></SubtitleStreamInfos>
"#;
let infos = parse_stream_infos(sp);
assert_eq!(infos.len(), 5, "every element yields a StreamInfo");
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
assert_eq!(
nums,
vec![1, 2, 3, 1, 2],
"the blank element owns audio slot 2, so the commentary is slot 3"
);
assert_eq!(infos[2].purpose, LabelPurpose::Commentary);
assert_eq!(infos[4].qualifier, LabelQualifier::Sdh);
}
#[test]
fn fallback_does_not_collide_with_partial_map() {
// Map claims audio "a1" -> 1. The unmapped audio "a0" must NOT
@@ -235,7 +352,7 @@ mod tests {
info("a1", StreamLabelType::Audio), // mapped → 1
info("a2", StreamLabelType::Audio), // unmapped → fallback
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// a0 skips the taken 1 → 2; a1 keeps 1; a2 → 3. All distinct.
assert_eq!(nums, vec![2, 1, 3]);
let mut sorted = nums.clone();
@@ -253,7 +370,10 @@ mod tests {
info("a0", StreamLabelType::Audio),
info("a1", StreamLabelType::Audio),
];
assert_eq!(assign_stream_numbers(&infos, &map), vec![5, 9]);
assert_eq!(
assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"),
vec![5, 9]
);
}
// ── Additional hardening tests ─────────────────────────────────────────
@@ -269,7 +389,8 @@ mod tests {
info("a1", StreamLabelType::Audio),
info("s1", StreamLabelType::Subtitle),
];
let nums = assign_stream_numbers(&infos, &HashMap::new());
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Audio: 1, 2; Subtitle: 1, 2 — each counter resets at 1 per type.
assert_eq!(nums[0], 1); // audio 1
assert_eq!(nums[1], 1); // subtitle 1
@@ -277,27 +398,34 @@ mod tests {
assert_eq!(nums[3], 2); // subtitle 2
}
/// Spec: map stream_num=0 is explicitly rejected (apply_labels uses 1-based).
/// This is documented in parse_playback_config: `if stream_num != 0`.
/// Mutation: remove the `!= 0` guard → zero is stored in map.
/// Spec: a map value of 0 is unmatchable (apply_labels is 1-based), so
/// assign_stream_numbers must treat it as unmapped and synthesize a real
/// 1-based number rather than emit an orphan 0.
/// Mutation: read the map value verbatim → stream_number 0 leaks out.
#[test]
fn map_zero_stream_num_is_skipped() {
// parse_playback_config skips zero; simulate that: the zero shouldn't
// end up in the map. We test assign_stream_numbers with a zero-containing
// map to verify it won't freeze the fallback counter at 1 forever.
fn map_zero_stream_num_is_synthesized_not_emitted() {
let mut map = HashMap::new();
map.insert("a0".to_string(), 0u16); // zero — per spec, was filtered by parse_playback_config
map.insert("a0".to_string(), 0u16); // 0 must not be treated as a claim
let infos = vec![info("a0", StreamLabelType::Audio)];
// If 0 IS in the map and assign_stream_numbers uses it, stream_number=0
// is not matchable (apply_labels is 1-based). The fallback counter
// would assign 1 instead. Test both paths:
let nums = assign_stream_numbers(&infos, &map);
// If the map has 0 for a0, assign_stream_numbers returns 0 (map wins).
// This is a known limitation — the guard lives in parse_playback_config.
// The test documents the ACTUAL behavior so a code change that introduces
// the guard in assign_stream_numbers would be caught.
// Current behavior: map wins → 0.
assert_eq!(nums[0], 0);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// 0 is treated as unmapped → the fallback counter assigns 1.
assert_eq!(nums[0], 1);
}
/// A stream genuinely mapped to 1 plus another stream whose map value is 0
/// must NOT both land on 1: the 0-stream is synthesized past the claimed 1.
#[test]
fn map_zero_does_not_collide_with_a_real_stream_one() {
let mut map = HashMap::new();
map.insert("real".to_string(), 1u16);
map.insert("bad".to_string(), 0u16);
let infos = vec![
info("real", StreamLabelType::Audio),
info("bad", StreamLabelType::Audio),
];
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
assert_eq!(nums[0], 1); // the genuinely-mapped stream keeps 1
assert_eq!(nums[1], 2); // the 0-stream is synthesized to the next free slot
}
/// Spec: collision-avoidance works across audio AND subtitle independently.
@@ -312,16 +440,74 @@ mod tests {
info("a0", StreamLabelType::Audio), // fallback
info("s0", StreamLabelType::Subtitle), // mapped → 2
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// Audio fallback for a0 → 1 (subtitle's taken-2 doesn't block it).
assert_eq!(nums[0], 1);
assert_eq!(nums[1], 2);
}
/// Spec: saturating_add prevents overflow when many streams are listed.
/// Mutation: use wrapping_add → counter wraps to 0 and collides.
/// A crafted `streamproperties.xml` can drive the fallback counter to the
/// top of the 1-based u16 stream-number space and then present one more
/// unmapped stream whose successor number is also claimed by the map.
///
/// This must TERMINATE. The bound is the numbering space itself, so the
/// assertion is on the spec-derived exhaustion behaviour (`None`), not on
/// any tunable constant. Run on a worker thread with a deadline so a
/// non-terminating loop fails the test in 20 s instead of hanging CI.
#[test]
fn assign_stream_numbers_saturation_on_overflow() {
fn exhausted_numbering_terminates_instead_of_looping() {
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
// One mapped audio stream claims the last number in the space.
let mut map = HashMap::new();
map.insert("claims_max".to_string(), u16::MAX);
let mut infos = vec![info("claims_max", StreamLabelType::Audio)];
// Enough unmapped audio streams to walk the counter to the top.
for i in 0..=(u16::MAX as u32) {
infos.push(info(&format!("u{i}"), StreamLabelType::Audio));
}
let _ = tx.send(assign_stream_numbers(&infos, &map));
});
match rx.recv_timeout(std::time::Duration::from_secs(20)) {
Ok(result) => {
worker.join().expect("worker panicked");
assert!(
result.is_none(),
"an exhausted 1-based u16 numbering space must fail the parse, \
not emit colliding or wrapped stream numbers"
);
}
Err(_) => panic!(
"assign_stream_numbers did not terminate within 20s — \
non-terminating skip loop on crafted stream_map"
),
}
}
/// The whole 1-based u16 space must remain usable: 65535 unmapped audio
/// streams get 65535 distinct numbers with no panic and no wrap. The
/// literals here are the JVMS-independent, spec-derived size of a u16
/// 1-based numbering domain, not a tunable cap.
#[test]
fn full_u16_numbering_space_is_usable_and_unique() {
let infos: Vec<StreamInfo> = (0..65_535u32)
.map(|i| info(&format!("a{i}"), StreamLabelType::Audio))
.collect();
let nums = assign_stream_numbers(&infos, &HashMap::new()).expect("space is not exhausted");
assert_eq!(nums.len(), 65_535);
assert_eq!(nums[0], 1);
assert_eq!(nums[65_534], 65_535);
let mut sorted = nums.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), 65_535, "stream numbers must all be distinct");
}
/// Spec: a partially-mapped playlist with many claimed numbers must still
/// synthesize past every claim without panicking or colliding.
/// Mutation: drop the skip loop → the fallback reuses a claimed number.
#[test]
fn fallback_skips_a_dense_block_of_claimed_numbers() {
// Force the counter past u16::MAX by pre-taking all values 1..=u16::MAX.
// Doing that for real would be slow; instead inject u16::MAX into taken.
let mut map = HashMap::new();
@@ -348,7 +534,7 @@ mod tests {
qualifier: LabelQualifier::None,
});
// This must not panic.
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
assert_eq!(nums.len(), 501);
// The last (unmapped) entry's number must be > 500 (skipped all taken).
assert!(nums[500] > 500);
+277 -120
View File
@@ -11,7 +11,7 @@ use std::collections::HashMap;
/// Cheap signature check: a CTRM disc ships `menu_base.prop` and/or
/// `language_streams.txt` inside a `/BDMV/JAR/*` archive.
pub fn detect(udf: &UdfFs) -> bool {
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "menu_base.prop")
|| super::jar_file_exists(udf, "language_streams.txt")
}
@@ -50,10 +50,10 @@ fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
if let Some(mb_match) = mb
.iter()
.find(|m| m.stream_type == label.stream_type && m.stream_number == label.stream_number)
&& label.name.is_empty()
&& !mb_match.name.is_empty()
{
if label.name.is_empty() && !mb_match.name.is_empty() {
label.name = mb_match.name.clone();
}
label.name = mb_match.name.clone();
}
}
// Append any menu_base-only stream (present in mb but not in ls by
@@ -87,7 +87,21 @@ fn prefix_is_commentary(prefix: &str) -> bool {
fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
let data = super::read_jar_file(reader, udf, "language_streams.txt")?;
let text = std::str::from_utf8(&data).ok()?;
let labels = parse_language_streams_text(text);
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels.
///
/// This is the shipping parser: [`parse_language_streams`] does the UDF read
/// and UTF-8 decode and then delegates here. It is split out — rather than
/// duplicated under `#[cfg(test)]`, which is what it used to be — so the unit
/// tests below exercise production code. A test that re-implements the
/// function it guards cannot fail when the real function breaks.
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
@@ -193,122 +207,7 @@ fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<
}
labels.push(StreamLabel {
stream_number: stream_num,
stream_type,
language,
name: String::new(),
purpose: final_purpose,
qualifier,
codec_hint,
variant: variant_code,
});
}
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels. Split
/// out from [`parse_language_streams`] so unit tests exercise the real parsing
/// logic without needing a SectorSource / UdfFs.
#[cfg(test)]
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let parts: Vec<&str> = line.split(',').map(|s| s.trim()).collect();
if parts.len() < 4 {
continue;
}
let type_str = parts[1];
let stream_num: u16 = match parts[2].parse() {
Ok(n) if n > 0 => n,
_ => continue,
};
let language = parts[3].to_string();
let variant = if parts.len() > 4 {
parts[4].to_string()
} else {
String::new()
};
let (stream_type, purpose, qualifier) = match type_str {
"audio_production" => (
StreamLabelType::Audio,
LabelPurpose::Normal,
LabelQualifier::None,
),
"audio_commentary" => (
StreamLabelType::Audio,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"audio_ime" => (
StreamLabelType::Audio,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_production" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_commentary" => (
StreamLabelType::Subtitle,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"subtitle_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::Forced,
),
"subtitle_dual" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_bonus" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_ime" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_ime_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::Forced,
),
_ => continue,
};
let mut codec_hint = String::new();
let mut variant_code = String::new();
let mut final_purpose = purpose;
if !variant.is_empty() {
match variant.as_str() {
"eda" => final_purpose = LabelPurpose::Descriptive,
"csp" | "cs" | "lsp" | "ls" | "cf" | "pf" | "bp" | "pp" => {
variant_code = variant.clone();
}
_ => codec_hint = vocab::codec(&variant).to_string(),
}
}
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type,
language,
@@ -426,6 +325,68 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `menu_base.prop` lines are skipped when `is_empty() ||
/// starts_with('#')` — either alone is sufficient. A commented-out
/// key=value line must never be parsed into an entry.
/// Mutation: `||` -> `&&` requires both, which a non-empty comment
/// line can't satisfy, so it falls through to `line.find('=')` and
/// gets parsed as a real property.
#[test]
fn menu_base_comment_line_with_equals_is_still_skipped() {
let labels = parse_props(
"#audio_1.class=AudioButton\n\
#audio_1.streamNumber=9\n\
#audio_1.name=Should Not Appear\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=1\n\
audio_2.name=Real Track\n",
);
assert_eq!(labels.len(), 1, "commented-out entry must not be parsed");
assert_eq!(labels[0].name, "Real Track");
}
/// Spec: `menu_base.prop` streamNumber (or audioStream/subtitleStream)
/// must be strictly positive — `0` means "no STN entry" and must be
/// skipped, matching the `n > 0` guard on the language_streams side.
/// Mutation: `n > 0` -> `n >= 0` (or the guard deleted) would let a
/// stream_num of 0 through, emitting a dead label apply_labels can
/// never match (its counter starts at 1).
#[test]
fn menu_base_zero_stream_number_skipped() {
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_1.name=Disabled Slot\n",
);
assert!(
labels.is_empty(),
"streamNumber=0 must be skipped, got {labels:?}"
);
}
/// Spec: `is_subtitle` is `class.contains("SubtitleButton") ||
/// prefix.starts_with("subtitle_")` — EITHER signal alone is
/// sufficient to classify (and keep) a subtitle entry whose prefix
/// doesn't follow the `subtitle_` naming convention.
/// Mutation: `||` -> `&&` would require BOTH signals; an entry whose
/// class says SubtitleButton but whose prefix is something else
/// (e.g. a vendor-specific button id) would then satisfy neither
/// `is_audio` nor `is_subtitle` and get dropped entirely.
#[test]
fn menu_base_subtitle_class_alone_is_sufficient() {
let labels = parse_props(
"menuBtn7.class=SubtitleButton\n\
menuBtn7.streamNumber=1\n\
menuBtn7.name=English SDH\n",
);
assert_eq!(
labels.len(),
1,
"class=SubtitleButton alone must classify as subtitle, not be dropped"
);
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
}
#[test]
fn prefix_commentary_segment_match_not_substring() {
// Genuine commentary group segments match.
@@ -441,6 +402,7 @@ mod tests {
fn lbl(t: StreamLabelType, n: u16, name: &str) -> StreamLabel {
StreamLabel {
stream_id: None,
stream_number: n,
stream_type: t,
language: String::new(),
@@ -452,6 +414,39 @@ mod tests {
}
}
/// Spec: `merge`'s `mb.iter().find(...)` must match an mb entry by
/// (stream_type AND stream_number) TOGETHER — either alone is not a
/// unique key (there can be an audio #1 and a subtitle #1, or two
/// different audio streams).
/// Mutation: `&&` -> `||` inside the closure would match on type OR
/// number alone, so `.find` (which returns the FIRST match) can pick
/// an mb entry with the right type but the WRONG stream number.
#[test]
fn merge_matches_mb_entry_by_type_and_number_together() {
// ls wants audio #2 (empty name, so it will borrow from mb).
let ls = vec![lbl(StreamLabelType::Audio, 2, "")];
// mb's FIRST audio entry is #1 (wrong number); its #2 entry (the
// real match) comes second.
let mb = vec![
lbl(StreamLabelType::Audio, 1, "Wrong Number Match"),
lbl(StreamLabelType::Audio, 2, "Correct Match"),
];
let merged = merge(ls, mb);
assert_eq!(
merged.len(),
2,
"mb's own audio #1 must also survive as its own entry"
);
let a2 = merged
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio && l.stream_number == 2)
.unwrap();
assert_eq!(
a2.name, "Correct Match",
"must match mb by (type AND number), not type or number alone"
);
}
#[test]
fn merge_preserves_menu_base_only_streams() {
// language_streams covers audio 1; menu_base has audio 1 (name)
@@ -521,6 +516,47 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::Forced);
}
/// Immunity pin against the defect measured in the `paramount` parser,
/// where a vendor `forced_sub` cell hung off a FULL dialogue track's own
/// slot to say "this track also contains forced signs", and reading that
/// cell as "this track is forced" flagged 30 MB dialogue tracks forced.
///
/// This format cannot express that. The forced signal is not a flag beside
/// a track's entry — it IS the entry's stream-kind token, drawn from a
/// closed vocabulary in which `subtitle_production` (the full dialogue
/// track) and `subtitle_narrative` (the forced-narrative track) are
/// mutually exclusive alternatives in the same position. A row is one or
/// the other; there is no cell a full track can carry to acquire the
/// qualifier, so the paramount failure mode has no encoding here.
///
/// Mutation: give `subtitle_production` a `Forced` qualifier, or add a
/// forced side-flag that both kinds may carry.
#[test]
fn a_full_subtitle_track_kind_can_never_carry_the_forced_qualifier() {
// Every subtitle kind in the vocabulary, one row each.
let text = "id1,subtitle_production,1,eng\n\
id2,subtitle_commentary,2,eng\n\
id3,subtitle_dual,3,eng\n\
id4,subtitle_bonus,4,eng\n\
id5,subtitle_ime,5,kor\n\
id6,subtitle_narrative,6,eng\n\
id7,subtitle_ime_narrative,7,kor\n";
let labels = parse_language_streams_text(text);
let forced: Vec<&str> = labels
.iter()
.filter(|l| l.qualifier == LabelQualifier::Forced)
.map(|l| l.language.as_str())
.collect();
assert_eq!(
forced.len(),
2,
"only the two narrative kinds are forced, got {forced:?}"
);
// The full dialogue kind specifically.
let production = parse_language_streams_text("id,subtitle_production,1,eng\n");
assert_eq!(production[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_commentary` → Subtitle / Commentary.
/// Mutation: treat as Normal → subtitle commentary not flagged.
#[test]
@@ -564,6 +600,72 @@ mod tests {
assert!(labels.is_empty());
}
/// Immunity pin. `language_streams.txt` states each stream's number in
/// field 3, so a row the parser cannot use is simply dropped — it can
/// never renumber the rows behind it. This is the property that keeps
/// this parser out of the STN-slot-shifting failure mode that bites
/// parsers which count positionally: there, a skipped entry silently
/// pulls every later label one stream forward.
///
/// Mutation: replace `parts[2]` with a running per-type counter → the
/// three unusable rows here collapse the survivors onto 1/2 and 1.
#[test]
fn ls_stream_numbers_come_from_the_row_not_a_counter() {
let labels = parse_language_streams_text(
"id,audio_production,4,eng\n\
id,audio_bonus_extended,5,eng\n\
id,audio_production,0,fra\n\
id,audio_production,7,fra\n\
id,subtitle_production\n\
id,subtitle_narrative,9,deu\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 4),
(StreamLabelType::Audio, 7),
(StreamLabelType::Subtitle, 9),
],
"an unusable row drops out without shifting the numbering"
);
assert_eq!(labels[2].qualifier, LabelQualifier::Forced);
}
/// Immunity pin, `menu_base.prop` side: the number comes from the
/// entry's own `streamNumber` property, so a skipped entry (commented
/// out, `streamNumber=0`, neither audio nor subtitle) leaves the
/// surviving entries on their authored slots.
///
/// Mutation: number by iteration order → the survivors collapse to 1/2.
#[test]
fn menu_base_stream_numbers_come_from_the_entry_not_a_counter() {
let labels = parse_props(
"#audio_0.class=AudioButton\n\
#audio_0.streamNumber=1\n\
audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=6\n\
other_1.class=SomeOtherButton\n\
other_1.streamNumber=2\n\
subtitle_1.class=SubtitleButton\n\
subtitle_1.streamNumber=11\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![(StreamLabelType::Audio, 6), (StreamLabelType::Subtitle, 11),],
"skipped entries must not renumber the ones that survive"
);
}
/// Spec: `eda` variant → `Descriptive` purpose.
/// Mutation: miss the `eda` branch → purpose stays Normal.
#[test]
@@ -608,6 +710,60 @@ mod tests {
assert_eq!(labels[0].language, "eng");
}
/// Spec: the skip test is `is_empty() || starts_with('#')` — EITHER
/// condition alone must skip the line. A commented-out line that
/// happens to look like valid CSV (a real authoring pattern for
/// disabling a stream entry) must never produce a label.
/// Mutation: `||` -> `&&` requires BOTH conditions, which a non-empty
/// comment line can never satisfy, so it would fall through to the
/// CSV parser and (since it has >= 4 comma fields) emit a spurious
/// label instead of being skipped.
#[test]
fn ls_comment_line_with_csv_shape_is_still_skipped() {
let labels =
parse_language_streams_text("#id,audio_production,1,eng\nid2,audio_production,2,fra\n");
assert_eq!(
labels.len(),
1,
"the commented-out CSV-shaped line must not parse"
);
assert_eq!(labels[0].language, "fra");
}
/// Spec: `subtitle_dual` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → falls to the
/// catch-all `_ => continue`, silently dropping the stream.
#[test]
fn ls_subtitle_dual_parsed() {
let labels = parse_language_streams_text("id,subtitle_dual,1,eng\n");
assert_eq!(labels.len(), 1, "subtitle_dual must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_bonus` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_bonus_parsed() {
let labels = parse_language_streams_text("id,subtitle_bonus,2,eng\n");
assert_eq!(labels.len(), 1, "subtitle_bonus must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
}
/// Spec: `subtitle_ime` maps to Subtitle/Ime (no Forced qualifier,
/// unlike `subtitle_ime_narrative`).
/// Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_ime_parsed() {
let labels = parse_language_streams_text("id,subtitle_ime,3,jpn\n");
assert_eq!(labels.len(), 1, "subtitle_ime must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Ime);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: multiple valid lines produce multiple labels.
/// Mutation: stop after first label → only 1 label returned.
#[test]
@@ -756,6 +912,7 @@ fn parse_menu_base_text(text: &str) -> Vec<StreamLabel> {
.unwrap_or_default();
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type,
language,
+275 -16
View File
@@ -35,14 +35,16 @@ use crate::sector::SectorSource;
use crate::udf::UdfFs;
use std::collections::BTreeMap;
/// dbp detect can't peek inside a jar without a SectorSource (the
/// trait function only takes `&UdfFs`), so we trigger on the cheap
/// signal "any top-level .jar in /BDMV/JAR/." That fires on every
/// BD-J disc, but parse() does the real `com/dbp/` check and
/// returns None on a mismatch — so this parser only ever consumes
/// time on discs that fell through every earlier parser.
pub fn detect(udf: &UdfFs) -> bool {
jar::has_any_top_level_jar(udf)
/// The real dbp signal is the `com/dbp/` package prefix inside a top-level
/// jar's central directory. With a reader in `detect`, we check that directly
/// (a cheap central-directory scan, no class decode) so this parser claims
/// only dbp discs instead of firing on every BD-J disc. `parse()` repeats the
/// check as belt-and-suspenders.
pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
jar::for_each_jar(reader, udf, |_entry, archive| {
jar::has_path_prefix(archive, "com/dbp/").then_some(())
})
.is_some()
}
/// Scan every top-level `/BDMV/JAR/*.jar` for the dbp framework and
@@ -91,6 +93,45 @@ fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
out
}
/// Cap on the bytes retained for one stream label.
///
/// The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
/// whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
/// constant contributes up to 65535 bytes, and the `u16` stream-number
/// keyspace admits 65536 of them per type.
///
/// Headroom: real dbp menu labels are short display names — "English Dolby
/// Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
/// retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
/// bytes. 256 leaves >5x headroom over that, and any string past it is menu
/// geometry or padding, never a language name — `vocab::lang` would not
/// resolve it anyway.
const MAX_LABEL_BYTES: usize = 256;
/// Cap on retained stream slots per type.
///
/// The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
/// type are reachable; paired with [`MAX_LABEL_BYTES`] this bounds the whole
/// scan at 2 x 512 x 256 bytes.
///
/// Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
/// streams per playlist, and dbp emits one menu TextField per stream. 512
/// leaves 16x headroom over the spec maximum.
const MAX_LABELS_PER_TYPE: usize = 512;
/// Record `label` for stream `n`, honouring the retention caps. Existing
/// slots are still overwritten at the cap so the documented last-write-wins
/// behaviour is preserved; only NEW slots are refused.
fn retain_label(map: &mut BTreeMap<u16, String>, n: u16, label: &str) {
if label.len() > MAX_LABEL_BYTES {
return;
}
if map.len() >= MAX_LABELS_PER_TYPE && !map.contains_key(&n) {
return;
}
map.insert(n, label.to_string());
}
fn collect_textfield(
s: &str,
audios: &mut BTreeMap<u16, String>,
@@ -110,15 +151,15 @@ fn collect_textfield(
}
if let Some(rest) = kind_n.strip_prefix("Audio") {
if let Ok(n) = rest.parse::<u16>() {
audios.insert(n, label.to_string());
retain_label(audios, n, label);
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle") {
if let Ok(n) = rest.parse::<u16>() {
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
subs.insert(n, label.to_string());
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle")
&& let Ok(n) = rest.parse::<u16>()
{
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
retain_label(subs, n, label);
}
}
}
@@ -130,6 +171,7 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
let qualifier = vocab::qualifier(&label);
let purpose = vocab::purpose(&label);
StreamLabel {
stream_id: None,
stream_number: num,
stream_type,
language,
@@ -145,6 +187,223 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
mod tests {
use super::super::{LabelPurpose, LabelQualifier};
use super::*;
use std::io::{Cursor, Write as _};
/// Build a minimal, structurally valid `.class` file (JVMS §4.1) whose
/// constant pool holds exactly the given `Utf8` strings (indices 1..=N,
/// no long/double slot padding needed for plain strings). No fields,
/// methods, interfaces, or attributes — `scan_jar`'s only interest is
/// the constant pool.
fn build_class(utf8_entries: &[&str]) -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(&0xCAFEBABEu32.to_be_bytes()); // magic
out.extend_from_slice(&0u16.to_be_bytes()); // minor_version
out.extend_from_slice(&52u16.to_be_bytes()); // major_version (Java 8)
out.extend_from_slice(&((utf8_entries.len() + 1) as u16).to_be_bytes()); // cp_count
for s in utf8_entries {
out.push(1); // CONSTANT_Utf8 tag
out.extend_from_slice(&(s.len() as u16).to_be_bytes());
out.extend_from_slice(s.as_bytes());
}
out.extend_from_slice(&0u16.to_be_bytes()); // access_flags
out.extend_from_slice(&0u16.to_be_bytes()); // this_class
out.extend_from_slice(&0u16.to_be_bytes()); // super_class
out.extend_from_slice(&0u16.to_be_bytes()); // interfaces_count
out.extend_from_slice(&0u16.to_be_bytes()); // fields_count
out.extend_from_slice(&0u16.to_be_bytes()); // methods_count
out.extend_from_slice(&0u16.to_be_bytes()); // attributes_count
out
}
/// Zip `entries` (name -> bytes) into an in-memory, Stored (uncompressed)
/// `jar::Jar` via the `zip` crate's own writer — a real archive, not a
/// hand-rolled central directory.
fn build_jar(entries: &[(&str, Vec<u8>)]) -> jar::Jar {
let mut buf = Vec::new();
{
let mut writer = zip::ZipWriter::new(Cursor::new(&mut buf));
let opts = zip::write::SimpleFileOptions::default()
.compression_method(zip::CompressionMethod::Stored);
for (name, data) in entries {
writer.start_file(*name, opts).expect("start_file");
writer.write_all(&data[..]).expect("write class bytes");
}
writer.finish().expect("finish zip");
}
zip::ZipArchive::new(Cursor::new(buf)).expect("valid zip")
}
/// `scan_jar` wires together `for_each_class`, constant-pool iteration,
/// `collect_textfield`, and `make_label` into the actual per-jar scan
/// used by `parse`. The pure `collect_textfield`/`make_label` unit
/// tests above don't exercise this wiring at all.
///
/// Mutation: replace the whole function body with `vec![]` — every
/// dbp disc would silently lose all its stream labels regardless of
/// what's in the jar.
#[test]
fn scan_jar_extracts_labels_from_real_class_entries() {
let class_bytes = build_class(&[
"com/dbp/Whatever", // unrelated string — must be ignored
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
"HTextField,Subtitle1,English SDH,Fontstrip_Composite,1312,763",
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843", // disable button, skipped
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
assert_eq!(
labels.len(),
2,
"expected one audio + one real subtitle label"
);
let audio = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio)
.expect("audio label present");
assert_eq!(audio.stream_number, 1);
assert_eq!(audio.language, "eng");
let sub = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Subtitle)
.expect("subtitle label present");
assert_eq!(sub.stream_number, 1);
assert_eq!(sub.qualifier, LabelQualifier::Sdh);
}
/// Immunity pin. Every dbp label states its own slot in the `AudioN` /
/// `SubtitleN` token, so the numbering survives gaps and skipped entries
/// intact. Nothing here counts positionally, which is what keeps this
/// parser out of the failure mode where a skipped entry pulls every later
/// label one stream forward.
///
/// Mutation: number by iteration order → `Audio4` becomes 2 and
/// `Subtitle3` becomes 1, silently rebinding both to other streams.
#[test]
fn stream_numbers_come_from_the_token_not_iteration_order() {
let class_bytes = build_class(&[
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
// Slots 2 and 3 have no menu TextField authored.
"LTextField,Audio4,French 5.1 Dolby Digital,Fontstrip_Composite,296,803",
// Not a stream: the disable-subtitles button.
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843",
// Unparseable slot token — dropped, and must shift nothing.
"HTextField,SubtitleX,German,Fontstrip_Composite,1312,883",
"HTextField,Subtitle3,English SDH,Fontstrip_Composite,1312,763",
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 1),
(StreamLabelType::Audio, 4),
(StreamLabelType::Subtitle, 3),
],
"unlabelled and unusable slots leave the authored numbers alone"
);
}
/// A `CONSTANT_Utf8_info` carries a `u16` length (JVMS §4.4.7), so one
/// crafted constant contributes up to 65535 bytes and the `u16` stream
/// keyspace admits 65536 slots per type — ~4 GiB of retained `String` per
/// map from a jar that is orders of magnitude smaller.
///
/// Boundary literals, not the constant: a 256-byte label is kept, 257 and
/// the JVMS maximum 65535 are refused.
#[test]
fn oversized_labels_are_not_retained() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
collect_textfield(
&format!("XTextField,Audio1,{},rest", "A".repeat(256)),
&mut audios,
&mut subs,
);
assert_eq!(
audios.get(&1).map(String::len),
Some(256),
"a 256-byte label must still be retained"
);
collect_textfield(
&format!("XTextField,Audio2,{},rest", "A".repeat(257)),
&mut audios,
&mut subs,
);
assert!(!audios.contains_key(&2), "a 257-byte label must be refused");
collect_textfield(
&format!("XTextField,Subtitle1,{},rest", "B".repeat(65_535)),
&mut audios,
&mut subs,
);
assert!(
!subs.contains_key(&1),
"a JVMS-maximum 65535-byte Utf8 label must be refused"
);
}
/// The stream-slot keyspace is the full `u16` on both maps. Offer 600
/// distinct audio slots; exactly 512 are retained.
#[test]
fn retained_stream_slots_are_capped_per_type() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
assert_eq!(
audios.len(),
512,
"600 audio slots offered, {} retained — the slot count is unbounded",
audios.len()
);
}
/// Reaching the slot cap must not break the documented last-write-wins
/// behaviour for slots already held.
#[test]
fn existing_slot_is_still_overwritten_at_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
collect_textfield("XTextField,Audio1,Spanish,rest", &mut audios, &mut subs);
assert_eq!(audios.get(&1).map(String::as_str), Some("Spanish"));
}
/// Headroom: the longest plausible retail label must survive untouched.
#[test]
fn longest_realistic_label_survives_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
let real = "Portuguese (Brazilian) 5.1 Dolby Digital Plus";
assert_eq!(real.len(), 45, "fixture length changed");
collect_textfield(
&format!("XTextField,Audio1,{real},Fontstrip_Composite,296,763"),
&mut audios,
&mut subs,
);
assert_eq!(audios.get(&1).map(String::as_str), Some(real));
}
#[test]
fn collect_extracts_audio_and_subtitle_indices() {
+1641 -44
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More