Compare commits

..
332 Commits
Author SHA1 Message Date
Matthew Jackson 0158112c35 v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-03 05:57:28 -07:00
Matthew Jackson b55884675c restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-02 22:35:49 -07:00
Matthew Jackson 455d953144 v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-02 22:35:47 -07:00
Matthew Jackson c8eafad5ef Merge branch 'fix/label-pid-binding' into dev 2026-08-02 18:56:10 -07:00
Matthew Jackson fe795fb472 Bind borrowed stream labels by the stream they name, not by a slot
A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.

Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check 4da6d3a
added; 301 are applied. Those labels carry no editorial payload, so the
direct damage is confined to codec text — but the polluted list is also
what the anchor gate reads, and on 11 disc/stream-type pairs it is what
decides the anchor, which is how it reaches the vendor's forced and SDH
flags. 53 anchor facts are harvested off a merged slot. The clip-info
orphans had the same shape, numbered from `max + 1` of a list they share
no coordinate system with.

A label now either NAMES its stream — `StreamId { clip, pid }`, read out
of the very table `disc::bluray` builds the stream from — or it does
not, and only the ones that do not are ever reached by counting:

  * `label_at` returns vendor labels only, so the two numberings can no
    longer be confused by construction.
  * Playlist and clip-info labels bind by id. Exact, no anchor, no
    sequence, no ordinal.
  * A named stream outranks a guessed one, so an editorial flag reaches
    a stream only where the disc's own numbering puts it there.
  * The presence of an id is the provenance the anchor gate was missing.
    It reads the vendor's slots alone now, so a slot the vendor never
    named no longer breaks the sequence — under-yield is the normal
    shape of these blobs — and a title with fewer streams than the list
    has slots is no longer eligible to hold it. Ranking prefers the
    title that positively confirms most of the list.
  * An orphan is in no playlist, hence in no title, so it binds to
    nothing rather than to whatever counted its way.

The MPLS floor is one label per physical stream keyed by `(clip, PID)`,
not by `(type, language, codec, pid)`: the same PID in two clips is two
streams, and the old key collapsed them. Its `stream_number` is now the
entry's real slot in its own playlist's table.

41 of 44 images are byte-identical; all three that move lose a label
they should not have had, and no feature title changes on any image. A
dozen featurette playlists stop reporting a feature subtitle's SDH
marking on their own unrelated subtitle; eleven menu and bonus titles
stop advertising the feature's object-audio format on plain stereo; and
on a disc whose every title carries a single audio stream — too short a
table to anchor anything — a regional-variant tag asserted on all
seventeen titles is now asserted on none, in exchange for every title
stating the codec it actually carries, which none of them did.

Six tests had been asserting the invented numbering, including one
pinning the disc-global counter as a deliberate property.

Also fixes a defect in the same family that the corpus work surfaced:
`pgs_forced_probe::apply_verdicts` set `forced` but left `qualifier`,
so a demoted track shipped with a metadata sidecar calling it forced
next to a Matroska header saying it is not. Only a forced claim is
cleared; an SDH marking is not the probe's to touch.
2026-08-02 18:52:40 -07:00
Matthew Jackson 5b095c56b6 Say plainly that the vendor playlist format is not a specification 2026-08-02 18:27:01 -07:00
Matthew Jackson 383adacec3 Merge branch 'fix/paramount-forced-sub-semantics' into dev 2026-08-02 18:22:25 -07:00
Matthew Jackson 5115eb14da Read the vendor forced-subtitle field as the enumeration it is
One vendor's playlists.xml carries a per-subtitle-slot cell that looks
like a boolean, and it was parsed as one: value 1 meant forced, anything
else meant not forced. Across every image in the corpus that uses this
format the cell takes four values, and 1 is not the forced one.

Decoding three of those discs and counting every PGS display set:

  * 1 marks a FULL dialogue track that additionally contains some
    forced-narrative signs. All nine cells bearing it on one disc are
    full tracks of 949-1411 display sets; all seven on another are full
    tracks of 1602-1651. Neither disc has a small track among them.
  * 2 and 3 mark a DEDICATED forced-narrative track, in its own trailing
    stream slot, duplicating a language that already holds a full track.
    The two 2 slots measured are 15 and 10 display sets with every one
    flagged forced; the four 3 slots are 7, 14, 23 and 59 against
    1216-2655 on the tracks they duplicate.

So the old reading was wrong in both directions — it flagged full
dialogue tracks forced, which is how one language came to present as two
identical full subtitle tracks with one of them marked forced, and it
threw away the cells naming the real forced tracks.

Content could not have corrected this afterwards. Clearing a wrong
forced label needs a disc whose authoring sets forced_on_flag, and on
the measured disc carrying four genuine forced tracks not one display
set anywhere sets it — there, the vendor cell is the only evidence there
is. The classification is now explicit: only a dedicated forced slot
earns the flag, an unrecognised value never does, and the
contains-forced-signs value is dropped rather than weakened into a
forced label, since a wrong forced flag on a full dialogue track is the
user-visible defect while a missing hint costs nothing.

Four of the crate's own tests had been asserting the boolean reading;
their subject was positional alignment, so they keep it and now use a
real forced value. The other two parsers that emit a forced qualifier
from vendor metadata were audited and are structurally immune — in both,
the forced marker names a slot of its own rather than hanging off a full
track's entry, so the failure has no encoding there — and each is now
pinned by a test saying so.
2026-08-02 18:21:32 -07:00
Matthew Jackson 4da6d3ab70 Bind vendor stream labels by PID, not by per-title ordinal
A vendor label list describes ONE playlist's stream table, but
apply_labels re-numbered it from 1 inside every title. Where sibling
playlists cover the identical feature clip and enumerate different
subtitle sets, identical ordinals resolve to different PIDs, and the
same physical stream came out flagged forced in one title and not in
the other. On the title such a disc offers as its rip target that put
`forced` on an 873 MB full-dialogue English subtitle track — the
reported "I see English and English (Forced), they are identical".

The blobs are not wrong; the binding was. The list carries no playlist
id, but it carries a language per slot, and that sequence is a
fingerprint: on the corpus exactly one title's per-type language
sequence reproduces the list position for position, and content
confirms that title's binding is the correct one. So binding is now
two-tier:

  * The title whose whole per-type language sequence sits on the list
    (>= 2 streams, longest wins) is the ANCHOR — the table the list is
    describing. Each of its slots yields a `(clip, PID) -> label` fact,
    and a PID is the same elementary stream in every playlist that
    plays that clip, so sibling playlists bind through the map. A slot
    the anchor never showed us is not bound at all.

  * Streams no anchor fact reaches still bind by the STN ordinal, but a
    label whose language contradicts the stream it would land on is
    dropped. Subtitle labels carry nothing but the qualifier, so an
    unverifiable one is all risk and no gain: off the authoritative
    path they additionally require both sides to STATE a language and
    state the same one. Unlabelled beats mislabelled — the muxer's
    demotable() guard can only clear a wrong `forced` on discs whose
    authoring uses forced_on_flag, and half the measured discs never
    set it.

Measured over 44 disc images, 9 change and every cross-title label
conflict goes away: 171 forced flags that contradicted a sibling
playlist are cleared, 61 correct ones are recovered on playlists that
had been missing them, 5 subtitle qualifiers and 1 audio purpose bound
against a contradicting language are dropped. No title gains a label it
did not have.

Known residual: on one disc the featurette playlists keep two forced
flags (down from nine) where a shifted list happens to coincide on
language. Ruling those out needs the list's provenance — which slots
are the vendor's and which were merged in by the MPLS gap-fill — and a
whole-sequence gate without it costs correct flags on discs whose
vendor slots are interleaved with gap-filled ones.
2026-08-02 17:50:20 -07:00
Matthew Jackson 5f0cd2e3fe Stop naming specific commercial discs in the AACS and codec comments
The same scrub as the previous commit, over the files it did not reach:
the variant-MKB layout notes, the 2.1 segment index observations, the
PPS-revert regressions and the playlist-twin tiebreak.

Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16"
is the useful part, and the title it came from never was.
2026-08-02 17:10:43 -07:00
Matthew Jackson 308063196a Merge branch 'fix/pgs-forced-probe-sampling' into dev 2026-08-02 17:07:47 -07:00
Matthew Jackson b14497dfba Do not call a track forced off one display set of a sample
A forced verdict is an absence claim, and a sampled run sees a fraction
of a track. Measured on real discs: tracks exist that flag about a
quarter of their display sets and leave the rest unflagged, so a sample
that catches one flagged set and nothing else promotes a full dialogue
track to forced -- which players then force on screen. A sampled run now
needs two display sets; a run that read every extent end to end has no
unread gap and may still promote off one.
2026-08-02 17:05:28 -07:00
Matthew Jackson f6e167ff2d Stop naming specific commercial discs in comments and tests
Fifteen references across six files named the discs a defect was first
seen on. The parser leak found earlier was not an isolated slip — the
same habit runs through the mux comments, the changelog and the AACS
content verdict, where a title name was standing in for the shape of
the problem.

Every one is replaced with the property that actually mattered: a
multi-clip title, a UHD Dolby Vision profile 7 dual-layer stream, a
disc carrying an authored-bad TS packet. The comments are more useful
for it — the reader needs to recognise the shape on a disc they have,
not the one we happened to have.

`SEG_MainFeature` stays: the parser matches on that literal, so it is
a format token rather than a title.
2026-08-02 17:00:10 -07:00
Matthew Jackson f982ce1046 Halve the seeks: eight 32 MiB windows, not sixteen 16 MiB ones
Measured on a real title: the same 256 MiB budget cut into sixteen
windows per extent took 41.8s against the head-first read's 6.6s, because
every jump collapsed the source's read batching into three-sector calls.
Expected observations depend on total bytes read, not on how finely they
are cut, and measured subtitle density (a display set every ~30-50 MB of
clip) makes a 32 MiB window about even money on its own.
2026-08-02 16:55:55 -07:00
Matthew Jackson f21cbdef80 End a label section where the next one's stream list begins
The pixelogic walk finds the feature playlist's section by name and ends
it at the next `SEG_`/`SF_`/`FPL_` marker. Those markers are section
NAMES, and a project's trailing sections — the per-language notice,
disclaimer and dub-credit cards — carry none. On 8 of the 11
affected-format discs in the corpus the feature playlist is the last
NAMED section in the blob, so the terminator never fires and the walk
consumes the whole tail of the file as more of the feature's stream
list.

The card names are `{lang3}_{card}`, which passes `is_stream_token`, so
each one advances an STN counter, and a card whose name collides with a
catalogued component emits a label outright. Measured on the worst disc:
95 entries past the end of a 9-audio/21-PG list, five phantom audio
labels at STN 10-14 from `*_AC` notice cards (`AC` reads as the AC-3
codec), and 94 uncatalogued-component occurrences — which also took the
parse from High to Medium confidence and fired the vocabulary-gap
warning on four components that are deliberately not catalogued. A
second disc fabricated one subtitle label from a token in a following
playlist section named `FP_SingAlong`, which `FPL_` does not match.

What every section has, named or not, is a stream list that opens with
its video slots. So a `Video Stream N` entry repeating one this section
already listed is the first entry of the NEXT section, and ends this
one. Distinct video entries are kept, since a section may legitimately
list a secondary video stream; the memo of them is bounded at the BD STN
table's ceiling so disc bytes cannot grow it.

Replaying all 11 blobs through `assign_labels` before and after: the two
discs above lose exactly their phantom labels (11→6 and 5→4), the other
nine are byte-identical.

One residue is pinned rather than papered over: a card's name precedes
its own section's video slot, so a forward-only walk can still count the
FIRST card after the last real slot. It sits at the tail of a list
nothing follows in, so it can renumber nothing — at worst it costs a
parse its High confidence.

No other parser in src/labels/ walks a flat entry sequence with a
terminator set; the rest scope each stream to a structural range or read
its number off the entry itself. paramount and criterion gain immunity
pins for the boundary property specifically: a stream list cannot run
into the next element's, and a missing element boundary shortens the
list rather than extending it.
2026-08-02 16:55:08 -07:00
Matthew Jackson 7ad5cdaf41 Apply the shape test to a mixed track too
A track that flags some of its own display sets and not others proves the
authoring house makes the distinction, so it needs no sibling to
corroborate the flag being in use -- but it still has to look like a full
dialogue track before a forced label is cleared. A small track with a
couple of flagged signs is a forced track, and demoting it is the mistake
the shape test exists to prevent.
2026-08-02 16:44:10 -07:00
Matthew Jackson af64089978 Take a lone sample window from the middle of its extent
A title cut into 50-odd clips gives each extent one window's worth of
budget. At the extent's head, every clip is sampled at the same relative
position and the first clip's window lands on the opening of the feature
-- the one stretch with no subtitles in it.
2026-08-02 16:31:39 -07:00
Matthew Jackson a2e6debac4 Record the label numbering and vocabulary fixes in the changelog 2026-08-02 16:28:02 -07:00
Matthew Jackson fd1918ef08 Catalogue the DUB forced-narrative marker and report vocabulary gaps once
Two halves of the same subtitle-labelling bug class. The numbering half
landed already: an entry the parser could not parse still occupies an STN
slot, so skipping it shifted every later label onto the wrong stream. This
is the other half — an entry the parser counts correctly but cannot
INTERPRET.

`DUB` names the forced-narrative subtitle track authored to accompany a
language's dubbed audio presentation: the signs and on-screen-text pass a
viewer still needs once the dialogue itself is dubbed. It is the same
editorial class as `*_TXT_FOR_`, spelled differently by some authoring
runs. Uncatalogued, it matched no component arm, so the token signalled
neither audio nor subtitle and the whole stream record was dropped at the
domain guard — a genuine forced track left with no forced qualifier even
after the numbering was right.

Evidence, from two independent discs in the corpus: the token appears only
inside the PG list, embedded in an otherwise contiguous run of
`{lang}[_{region}]_TXT_FOR_` siblings — one forced-narrative slot per
localized language — and takes exactly the slot where that language's
forced entry belongs. Both discs also carry that language's FULL subtitle
track as a separate, separately-labelled slot, so the DUB entry is not it.
The stream it lands on is a sparse PG track, the signature of a forced
pass rather than full dialogue. Deliberately not added to vocab::qualifier:
that maps free-form English label text, where a bare "dub" means dubbed
AUDIO. The forced-subtitle reading is specific to this token grammar.

The corpus sweep that found it also produced four components that are
deliberately NOT catalogued. They are per-language notice and disclaimer
clip names that merely collide with the `{lang3}_{component}` token shape;
each occurrence sits in a one-video-stream section next to the disclaimer
entry it names. They carry no editorial meaning, and mapping them would
attach a qualifier to a stream on the strength of a filename.

Which is also why an unmapped component now reports once per parse rather
than once per occurrence. It was a debug line nobody reads, and that is how
this gap survived to a user complaint; but a per-occurrence warn would bury
the signal under dozens of routine collisions on an ordinary disc. One
bounded, deduplicated line names the distinct components and says plainly
that any forced/SDH/commentary meaning they carry went unapplied. The
backing set is capped and truncates by chars, not bytes — the components
come from untrusted disc bytes, and a byte-offset slice can split a
multi-byte sequence and panic.

Two existing tests encoded the wrong behaviour and are corrected: the STN
numbering test expected the forced run to skip the DUB slot, and the
unclassifiable-slot test used DUB as its example of a token with no
meaning. The latter now uses a genuinely uncatalogued component.
2026-08-02 16:25:04 -07:00
Matthew Jackson 8636497505 Note the bounded display-count over-count on a stalled read retry 2026-08-02 16:18:19 -07:00
Matthew Jackson e78c8e9802 Drain a sampled run's tail and merge repeated extent memos
A window's last PES stays open in the demuxer until the next PUSI, which
under sampling is in a different window or nowhere — so the last display
set of every window was discarded, worst where the sample is thinnest.
Flush the demuxer at the end of each window.

A playlist that lists the same clip twice now merges the second read into
that extent's memo (facts OR'd, display count MAXed, coverage the larger
of the two) instead of overwriting it.
2026-08-02 16:16:20 -07:00
Matthew Jackson d6542eda10 Document the forced-probe redesign in the changelog 2026-08-02 16:09:27 -07:00
Matthew Jackson 08bdd51588 Sample PGS across the title instead of reading its head
The content-based forced-subtitle probe spent its whole 256 MiB budget
on the first sectors of a title. A feature's subtitles begin minutes in,
so the probe read the opening logos, hit the budget, observed no display
set at all and contributed nothing to any verdict — the vendor label was
always the only input.

The forced predicate is asymmetric: one non-forced display set disproves
forced permanently, while proving forced needs the whole track, and
genuine forced tracks are tiny where full tracks are huge. So the same
budget is now SPREAD over each extent in ~16 MiB windows placed on the
AACS unit grid, sized in proportion to the extent, ending at the extent's
end. Cost is unchanged; placement is not.

Also:

  * Per-track early exit. A track that is disproven (and whose label
    needs no correcting) stops asking for budget; an extent that owes
    evidence only for such tracks is skipped outright, and evidence
    already in the cache is never demuxed a second time.

  * Content may now DEMOTE a wrong vendor forced flag, in the probe and
    in the muxer, behind one shared guard: absence of forced_on_flag
    only means something if some other track demonstrably uses it, and
    the track must have the shape of a full dialogue track rather than
    of a forced-narrative one. On a disc where no track sets the flag,
    nothing is demotable.

  * A sampled or budget-cut extent's evidence is memoised with the
    COVERAGE behind it. It used to be filed under the extent's full key
    and replayed to playlists that would have read far more of the clip,
    turning a prefix into an absence claim about the whole extent.
2026-08-02 16:07:02 -07:00
Matthew Jackson 9d19781b8a Number vendor label streams by STN slot, not by parsed entry
Sweep of every parser in src/labels/ for the numbering bug fixed in
pixelogic: a blob lists one entry per STN slot, but the parser advances
its per-kind counter only for entries it can use, so every entry it
skips shifts all later labels onto the wrong stream.

Three parsers were affected; the rest key each label off a number the
blob states outright and are immune.

  * paramount — `aud` / `sub` are the STN-ordered stream lists, and
    `forced_sub` / `*_com1_idx` index those same cells. A cell with an
    empty language was skipped without consuming its slot, so every
    label behind it bound one stream early while the vendor's own
    positional indices still pointed at the raw cell. `stream_number`
    is now the cell's 1-based position. The forced flag is the payload
    here, so the shift lands `forced` on a full-dialogue track.

  * mpls_universal — its counters must agree with the stream list
    `disc::bluray` builds from the same STN entries, since that list is
    what `apply_labels` counts against. They disagreed twice: a
    `coding_type == 0` padding entry was counted here and dropped
    there, and a PG coding_type in an audio STN slot (a layout
    `mpls::parse_stream_entry` has a dedicated arm for) was counted as
    audio here and built as a subtitle there. Both rules now live in
    one `label_type_for`.

  * deluxe — a binding construction whose Language `getstatic` did not
    resolve was skipped outright. It is still an STN slot; it just has
    nothing to label. It now advances the counter, with the list it
    belongs to taken from its CodingType argument or, failing that,
    from what its binding type's resolved siblings showed.

Two paramount tests asserted the renumbering as if it were the spec
(`empty_middle_slot_does_not_inflate_stream_number`,
`audio_stream_numbering_skips_empty_slots`) and are rewritten. Immunity
pins added for ctrm, dbp and criterion so the property cannot rot.

Also scrubs two commercial disc titles from the menu-graphic filename
examples in png_filenames and vocab.
2026-08-02 15:51:09 -07:00
Matthew Jackson 030f7c5f34 Stop naming a specific commercial disc in the label parser
Two comments identified the disc the STN-numbering bug was found on by its
vendor project name. The reproduction does not need it: what matters is the
SHAPE of the token list — placeholder slots, region-only tokens, an
uncatalogued component — not which release happened to exhibit it. Both now
describe the shape.

The remaining `SEG_MainFeature` references stay. That is a vendor section name
the parser matches on at pixelogic.rs:88, not a disc identifier — it is the
format's vocabulary, like `FPL_` or the `eng_MLP_` stream tokens beside it, and
removing it would break the parser.

Also drops the last prohibited citation from the changelog: an mp4:// bullet
said "no ffmpeg". The website changelog page is REGENERATED from this file at
release time, so a scrub of the site alone would have been reverted by the next
release.
2026-08-02 15:30:56 -07:00
Matthew Jackson 4dd0848d62 Number pixelogic subtitle labels by STN slot, not by parsed token
A pixelogic feature section lists one entry per STN slot. Only the
entries that parse were advancing the counters, so every surviving
label was renumbered 1..N and applied to the wrong stream.

Three kinds of entry were being skipped:

  * `PG Stream N` placeholders — the subtitle twin of `Audio Stream N`,
    which was already counted. The old comment claimed the corpus showed
    subtitle tokens align without counting them; on a disc that has both
    placeholders and later editorial tokens they do not.
  * region-only tokens (`fra_CF_`, `spa_LS_`) — REGIONS sets `variant`
    but neither `is_audio` nor `is_subtitle`, so the token is dropped.
  * tokens whose distinguishing component is uncatalogued (`jpn_DUB_`).

On UHD_Crime101_WW_150728 the PG list is 18 slots: five placeholders,
four region-only tokens, and `jpn_DUB_`, with the seven `*_TXT_FOR_`
forced-narrative tokens at STN 11-16 and 18. Counting only the ten
that parse put `eng_SDH_` on STN 1 and the seven forced markers on STN
2-8 — the disc's FULL subtitle tracks. `labels::apply_labels` turns a
Forced qualifier into `SubtitleStream::forced`, which the muxer writes
as Matroska `FlagForced`, so the output offered an "English (forced)"
track carrying 1731 display sets of complete English dialogue while the
real 19-set forced track went unflagged.

Unclassifiable entries carry no stream type, so they advance the list
currently being enumerated; sections run video -> audio -> PG and
`domain` follows the last entry whose type was known.
2026-08-02 15:27:28 -07:00
Matthew Jackson 49d5ca2a67 Record the 1.6.0 fixes in the changelog
The fvi provenance fix, the key-service outage codes, the settings BOM
data-loss fix and the per-track-kind picker rows all landed on dev without a
changelog line. A release that ships undocumented fixes is one nobody can
audit later.
2026-08-02 15:08:27 -07:00
Matthew Jackson f39762e09b Merge branch 'fix/key-service-outage-not-missing-key' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson 126765ca19 Merge branch 'fix/fvi-source-provenance' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson bbc07ba437 Report a key source that could not answer as its own failure, not as "no key"
The online key service returned HTTP 502 for about seven hours. Every rip in
that window ended with

    key: online > no entry > NO KEY
    Error: E7022 No key source has a decryption key for this disc (id: 422EB...)

which reads as "this disc is not in the key database". Operators went hunting
for a VUK that was never missing; the correct action was to wait.

E7022 is a claim about the WORLD: every source answered, and none holds a key
for this disc. A source that could not be reached made no such claim -- nothing
at all was learned. `resolve_and_apply_traced` collapsed the two anyway, with
its own comment naming the incident and pointing at the fix.

Three codes for the three different operator actions, each a variant with a
number and no English (the library ships none):

    E7028 KeyServiceUnavailable   unreachable / DNS / timeout / 5xx -> wait
    E7029 KeyServiceUnauthorized  401 or 403                        -> fix token
    E7030 KeyServiceRateLimited   429                               -> back off

None carries a payload: the key-service URL and its resolved address are
operator-confidential and must not ride out in a Display an operator pastes
into a bug report.

`resolve_and_apply_traced` now keeps `Err` and `Ok(empty)` apart. A source that
answered and holds nothing still records `KeyNode::NoEntry` -- that is the one
true "I looked, it is not there". A source that FAILED records an empty path,
and its reason is stamped onto `Disc::aacs_error`, the channel
`ensure_decryptable_keys` already reads for the E7017-vs-E7022 split. The gate
gained three arms alongside `AacsVidUnavailable` and raises the source's own
code.

`KeyOutcome` deliberately gains no variant. It is matched exhaustively by every
front-end's trace renderer (freemkv's `pipe::render_resolution_trace`,
autorip's `keysource::render_resolution_trace`), and this fix must not become a
breaking change across four repos to say something the error code already says
precisely. Dropping the false `NoEntry` node is enough for the trace line.

The three codes join `is_disc_level_no_key`: a service that is down, refusing
the token or throttling is down for every title, so a rip loop must stop rather
than issue N doomed requests -- and on 429, dig the hole deeper.

`FetchOutcome::errored`, documented as unreachable-in-production, now fires for
real: the negative-result cache stops memoising a transient outage.

Red before green: `key_source_failure_is_not_reported_as_a_missing_disc_key`
drives KeySource -> resolve -> aacs_error -> ensure_decryptable twice over the
same disc and asserts the verdicts differ. With the old conflation restored it
fails on the trace node, and with that assertion removed it fails
`left: 7022, right: 7028` at the gate.
2026-08-02 11:53:09 -07:00
Matthew Jackson b97b419f76 Record the source, not the destination, in the FVI header
The `fvi://` arm of `output()` passed the destination `.fvi` path as
`FviSink::create`'s `source_path`, so every index named itself as its
own source. `SourceInfo::default()` supplied the rest, making
`source.medium` always "file" and `source.title` always 0 — three
header members wrong, where FVI_FORMAT.md §6.2 defines `source` as
describing the input.

Beyond the wrong data, it made the output unreproducible: two machines
indexing identical bytes emitted different files purely from where
they wrote them, and a local filesystem path leaked into a shareable
file.

`output()` cannot see the source, so thread the provenance down from
the driver, which can: `mux_stream` derives a `SourceInfo` per
`MuxInput` arm and passes it through `drive_mux` to `output()`. Per the
one-method-per-action rule this is a signature change, not an
`output_with_source()` variant; the parameter is `Option<&SourceInfo>`
so a caller with no provenance declares none rather than back-filling
the destination. `SourceInfo`/`Medium` become public API.

What each arm can honestly reach:

- Session: everything — device path, the caller's title index, the
  title's playlist, the scanned volume id.
- Url: the source URL, its scheme's medium, `title_index`, and the
  playlist off the opened stream's scanned title.
- Iso: the image path and playlist. The title index is not in
  `MuxInput::Iso` (it carries a scanned `DiscTitle`, which has no
  index), so it stays 0.
- Live: medium and playlist. The reader is an opaque
  `Box<dyn SectorSource>` with no path, and again no title index.

Unreachable members are left empty rather than guessed — the sink
already omits the empty ones.
2026-08-02 11:19:54 -07:00
Matthew Jackson eac0040b95 Add the missing direct coverage for parse_stss
Every other sample-table parser (stco, stsc, stts, ctts) had a "count lie"
test proving the declared count is bounded by what the box actually holds,
and stco/stsc had their own arithmetic pinned. parse_stss had neither - no
test in this file ever called it with real entries, only with a too-short
buffer. Added the same two: two distinct entries read from their own
offsets (catching the o = 8 + i*4 arithmetic and the per-entry bounds
check), and a declared count of 3 backed by only 2 real entries (catching
the same "trust the box, not the count" contract the other four parsers
already had).
2026-08-01 16:37:10 -07:00
Matthew Jackson a939b5b2e1 Pin the sample-table parsers' short-buffer safety and byte arithmetic
parse_stco, parse_stsc, parse_stts, parse_ctts and parse_stss all open with
the same "if b.len() < 8" guard before reading count = be32(b, 4), but no
fixture anywhere in this file ever called any of them with a buffer shorter
than 8 bytes - every test builds a complete box. A <-to-== mutant of that
guard only rejects a buffer of EXACTLY 8 bytes and lets everything shorter
fall through to an out-of-bounds be32 read, and nothing was exercising that
fall-through to notice. One test now drives all five through every length
from 0 to 8.

Also: co64's 8-byte offsets were never actually built by any co64 fixture
in this file (only the 32-bit stco path was exercised), so its manual
byte-by-byte u64 assembly was unconstrained the same way parse_elst's was.
And sample_offsets's sidx only advanced correctly by coincidence in every
existing fixture, because none of them placed three or more samples in a
single chunk back to back - the only shape where reusing the wrong sample's
size becomes observable.

Documented the <=8 boundary as equivalent across all five parsers (and the
matching start < end in sample_offsets): the per-entry guard immediately
below always breaks on the first entry at that exact boundary, so both
branches converge on the same empty result. Confirmed by re-running each
mutation against the full suite.
2026-08-01 16:33:17 -07:00
Matthew Jackson 2ccbb9d417 Pin parse_esds_asc's two independent boundary checks
asc_len == 0 and end > b.len() reject for different reasons - a useless
zero-length ASC, and a truncated one - and nothing distinguished the || from
an && that would only reject when both are true simultaneously, or the >
from a < that would reject the common case of an esds with bytes after the
ASC (more child boxes, padding) instead of only a genuine truncation.

Documented the | in read_descriptor_len's accumulator as the same
shift-then-mask equivalent already recorded in audio.rs's BitReader::read:
the shift always vacates exactly the bits the mask fills, so | and ^ can't
disagree.
2026-08-01 16:23:10 -07:00
Matthew Jackson 2d06639963 Pin mdhd_language's length boundary and three unasserted stsd codec paths
mdhd_language's guard was written as "b.len() < off + 2" (reject too short)
but nothing distinguished that from "reject anything not exactly off + 2" -
a buffer one byte longer than the minimum has to keep working, and a buffer
missing the field entirely has to return None rather than read past the end.

parse_stsd's mp4a path had no test at all: the AAC codec_private extraction
depends on both codec == Aac and body.len() >= 28 being true together, and
with no mp4a fixture anywhere in this file neither half of that condition,
nor the boundary itself, was constrained. Also added the same header-length
boundary check parse_elst already had (< 8 vs <= 8 - proved equivalent this
time, since the very next guard on the empty slice catches the <= 8 case
too), and one test walking every recognised audio fourcc (ac-3/ec-3/mp4a/the
four dtsX variants) so a deleted match arm for any of them fails loudly
instead of silently dropping that track.
2026-08-01 16:18:20 -07:00
Matthew Jackson 56921af7c9 Pin find_boxes_capped's cap boundary and its size-field byte offsets
Nothing asserted the scan actually STOPS at cap rather than one match past
it, or that the declared box size is decoded from its own four bytes rather
than an adjacent one - every existing fixture used sizes small enough that
all but the last size byte are zero, so an index slip reading the wrong byte
would read the same zero and go unnoticed.
2026-08-01 16:08:11 -07:00
Matthew Jackson 322381c429 Pin Stream::read's MAX_ALLOC_BYTES boundary and that write always rejects
Stream::read has its own s.size > MAX_ALLOC_BYTES cap, a separate call site
from read_moov's over the same policy - checked they agree (both reject
strictly greater than the cap, exact cap allowed) and they do, so this is not
one of tonight's one-policy-two-copies bugs. But the boundary itself and the
one-byte-over case were unasserted, and Mp4Reader::write returning an error
(mp4:// is read-only) had no test either.

Building Mp4Reader directly in the test (its fields are private but visible
within this module) over the existing FakeBigReader avoids a real 256 MiB
backing file for the boundary case.
2026-08-01 16:04:03 -07:00
Matthew Jackson 309997f09f Pin per-track handler routing, PID arithmetic and the shared sample budget
Nothing asserted that a hdlr other than vide/soun gets dropped rather than
folded into the audio branch, that the per-track PID formulas
(0x1011/0x1100 + track_idx) use the right operator and the right operand,
that a sample-less track still advances track_idx for the next one, or that
the cross-track sample_budget is actually decremented (as opposed to grown or
divided) by each track's real count. All four were reachable with a single
track_idx == 0, which made every existing fixture blind to +/-/* confusion on
these sites - track_idx never moved past 0 in any of them.

Also let audio_trak_missing omit stsz, needed to build a sample-less track for
the track_idx test.
2026-08-01 15:59:53 -07:00
Matthew Jackson dc65517f55 Pin elst byte-offset decoding and a zero-timescale boundary in elst_offset_ticks
parse_elst's existing tests used segment_duration/media_time values that were
almost all zero or 0xFF bytes, so an index slip in the version-1 byte
extraction (reading a neighbouring byte, or one outside the entry entirely)
could return the same value by coincidence and the test wouldn't notice. Added
a fixture with every byte distinct and nonzero so any wrong offset is caught.

elst_offset_ticks's `empty_movie_ticks > 0` guard on the Some(mts) arm looked
like a pure optimisation, but shifting its boundary lets an
empty_movie_ticks == 0 call fall into the division instead of skipping it -
and a zero movie timescale (unreachable through from_reader, which filters it,
but not through this function's own contract) makes that division panic.
Pinned the boundary directly so the function stays safe on its own terms.

Documented nine further mutants as equivalent rather than chasing them:
media_edits/odd_rate and the None-arm's empty_movie_ticks check only gate a
tracing::warn!, never the returned offset, and parse_elst's b.len() < 8 vs
<= 8 boundary computes the same empty Vec either way once available = 0 is
worked through. Confirmed by re-running each mutation against the full test
suite.
2026-08-01 15:54:24 -07:00
Matthew Jackson 20fd5c3f63 Close the MEDIUM mutation gaps across transport, labels and codecs
The remaining triage items after tonight's HIGH fixes: 1,290 lines, almost
all tests. Covers disc/mod.rs's DVD scan path (with real minimal VMG/VTS IFO
fixtures rather than mocks), drive/mod.rs, labels/class_reader.rs and
labels/mod.rs — the two biggest untriaged survivor clusters in the crate —
plus hevc.rs and ps.rs.

One production change, and it is an extraction rather than a behaviour
change: MacScsiTransport::open mapped the shim's negative failure sentinels
to typed errors inline, where nothing could reach it without a real IOKit
FFI call. It is now map_shim_open_error, so the mapping can be pinned. It
matters because collapsing -5 into the DeviceNotFound catch-all turns
"another process holds the drive" into "no such drive", and an operator
chasing the wrong problem is worse than a blunt error.

Gate green on the pinned toolchain including the secrets scanner.
2026-08-01 15:00:01 -07:00
Matthew Jackson 284c19f15a Pin read_moov's box-size boundaries and MAX_ALLOC_BYTES exactly
read_moov's forward-progress guard (box_size < header_len, OR'd with the
EOF check) and the MAX_ALLOC_BYTES cap were only exercised on inputs well
away from their boundaries, so a mutation testing pass found the exact
edges unasserted: a size-8 (header-only) moov, a box that overruns the
file by exactly the amount the OR/AND distinction can see, and a payload
of precisely MAX_ALLOC_BYTES. Added a shared FakeBigReader (lifted out of
an existing test's local struct so a new test can reuse it) to exercise
the MAX_ALLOC_BYTES boundary without a multi-hundred-MiB backing file.
2026-08-01 14:39:39 -07:00
Matthew Jackson 1f1a17d23f Document why audio.rs's bit-packing | mutants are equivalent
Every mutation-testing survivor in this file is a | with ^ flip inside a
bitstream packer: BitReader::read's accumulate step, the push closures in
dac3_box/dec3_box/ddts_box, and the multi-field extractions in parse_eac3
and parse_dts. All nine are the same shape: shift an accumulator left by
exactly the width of the next field, then OR it in, so the two operands
never share a set bit and | and ^ agree on every input. Confirmed by
running cargo-mutants against just these nine mutations after the existing
test suite (which already exercises each function's field values) - all
nine still survive, as expected for a genuinely equivalent mutant. Recorded
the reasoning once at BitReader::read so nobody spends time chasing it
site by site.
2026-08-01 14:24:47 -07:00
Matthew Jackson 1ca47ce406 Document the pack_language and detect_rate equivalent mutants
Three pack_language mutants and two detect_rate boundary mutants survive
mutation testing with no test able to close them, and it's not for lack
of trying: they're equivalent by construction. Recording the proofs next
to the code so nobody re-chases them:

- (b[0] - 0x60) as u16, shifted << 10 then truncated to u16, is congruent
  mod 65536 to (b[0] + 0x60) as u16 shifted the same way, because
  0x60 * 2 * 1024 is an exact multiple of 65536. The same swap on the
  second letter (shifted only << 5) is NOT equivalent, which is why only
  the first letter's mutant survives.
- The two | with ^ mutations that OR the three packed fields together
  are equivalent because the fields (a lowercase letter minus 0x60, so
  1..=26) always fit in 5 bits and never share a set bit once shifted
  into their 0/5/10 positions.
- detect_rate's tolerance and tie-break comparisons only diverge from
  their <= mutants on an exact 0.5 fps distance or an exact tie, and a
  brute-force search over every achievable integer-nanosecond median
  found no case that lands on either boundary bit-exactly.
2026-08-01 14:20:13 -07:00
Matthew Jackson 6889e6a032 Merge branch 'mux-mp4' into dev 2026-08-01 14:06:24 -07:00
Matthew Jackson 794e757889 Assert the MP4 timing arithmetic, and name the faststart slack rule
MP4 track timing was numerically unasserted. Every operator in the
PTS-to-ticks, duration, tkhd_dur and ctts chain could be flipped and the
whole suite stayed green, because no test decoded an output file and checked
a concrete number — the existing tests assert box presence and gross
container shape only. That is the crate's worst failure mode: a title muxes
"successfully" with silently wrong A/V sync or total duration, and nothing
above can tell.

The new tests build tracks with known PTS deltas and compare the emitted
stts, ctts and tkhd.duration against computed values.

Also lifted the faststart slack rule out of the match guard into
faststart_fits(). A leftover hole of 1-7 bytes cannot be expressed as any
ISO-BMFF box, since a box header is 8 bytes, so finish() must fall back to
moov-at-end rather than write a free box that lies about its own size. The
condition now has a name and a test instead of being an unexplained
`g == 0 || g >= 8` inside a pattern guard.
2026-08-01 14:06:16 -07:00
Matthew Jackson ddb36e2c15 Assert the undersized-buffer guard on the chunked read path
Drive::read and read_fua split any request larger than the transport's
transfer limit into chunks and slice the caller's buffer by count * 2048.
The up-front length check is the only thing between an undersized buffer and
a "range end index out of range" panic out of a public API, and the comment
above it records that this was once a live panic.

Every existing read test stays on the single-chunk path, where an undersized
buffer is already tolerated and returns Err(DiscRead), so the guard itself
had no coverage at all and a mutation run flipped its arithmetic freely.

The test drives a mock transport with a small transfer limit and asserts the
two paths agree: an undersized buffer is an error either way, and behaviour
on a caller mistake does not depend on the drive's transfer limit. Confirmed
by hand that both the reported * -> + mutation and a < -> > flip now fail.
2026-08-01 13:56:58 -07:00
Matthew Jackson 530b4a1925 Cite the H.264 spec directly, not a reference implementation
The escape-stripper comments named a third-party decoder as the authority
for the cumulative-zero rule. This repo is public and does not cite other
implementations; the rule is specified in ITU-T H.264 §7.3.1, which is the
citation that belongs here anyway.

No behaviour change — comments only. The scan-secrets gate caught it.
2026-08-01 13:49:47 -07:00
Matthew Jackson 6153690a90 Merge branch 'mux-codec' into dev 2026-08-01 13:47:15 -07:00
Matthew Jackson 65dd9ce9dc Merge branch 'mux-mkv' into dev 2026-08-01 13:45:04 -07:00
Matthew Jackson f72a3d59a3 Assert what the MKV mux and demux actually produce
A mutation run over mux/mkv.rs and mux/mkvstream.rs left 148 survivors.
Reading them turned up no wrong code, but a lot of code whose output
nothing ever looked at. Most of that is on the read side: parse_track
had a dedicated arm for Language, TrackName, FlagForced, Video and
Channels and not one of them was checked, so a re-mux could have lost
the audio language, the subtitle forced flag, every track label, the
resolution and the channel layout with the suite still green. Ten of
the eleven CodecID comparisons were unasserted too — only HEVC was
pinned — so any of them could have been mis-wired and the stream would
have gone to the wrong parser. The round-trip test now writes a real
three-track title through the muxer and reads it back through the
reader, and a separate test walks every registered CodecID.

The BPS statistics tag was the worst of the write side. Its test
asserted `file_bytes.contains("800")`, which a wrong bitrate passes
trivially — 80000 contains "800". Both the tag and the back-patched
Segment duration are now decoded and compared to a computed number, on
a title that declares no duration so the whole max_block_ticks →
seconds → bits chain is exercised. Cue points get the same treatment:
their CueTrack and CueClusterPosition were never read back, on either
the keyframe path or the i16-forced-split path, which are two hand-
written copies of the same three fields.

The rest closes arithmetic that only a bad disc reaches: a zero frame
rate or zero display-aspect denominator (both divisions), a
TimestampScale that does not fit an i64, a cluster timestamp of exactly
i64::MAX, a TrackNumber of 65537 that truncates onto the valid track 1,
and the shortest legal Block at both VINT widths. Two tests separate a
clean end of stream from a device failure: swallowing the second one
truncates the output at a bad sector and reports the rip complete.

Also pinned: only the first video and first audio track may be default
(the de-duplication lives in MkvStream::create and had no test at all),
the activation trigger is the first VIDEO track rather than track 0,
the measured field order reaches the file rather than just the helper
that computes it, and a Blu-ray 3D base/dependent pair builds the merge
instead of shipping two unrelated H.264 tracks.

96 of the 148 mutants verified killed by hand. Of the remainder, most
are equivalent — disjoint-bit `|` that `^` cannot change, delete-arm
mutants whose fallback is the same constant, guards on tracing calls —
and the write_frame branch at 1452 is unreachable: a cluster is always
open by the time it can be entered.
2026-08-01 13:44:08 -07:00
Matthew Jackson efae55fa2f Kill codec mutation survivors and unify H.264's duplicated escape stripper
The mux/codec parsers (startcode, h264, hevc, dts) had 300 surviving
mutants between them, and it turned out to be for the reason you'd
fear: the exp-Golomb readers and the AU-boundary bitstream scanners had
essentially no direct unit coverage, only indirect exercise through
full-frame parse() calls that never touched the actual edge cases.

Direct fixes to test gaps:

- The shared BitReader's read_ue truncation guard (`leading_zeros >
  31`) and skip_start_code's 4-byte-vs-3-byte boundary check had no
  test at their exact boundary. Added tests that hit the boundary
  precisely; a `>=`/`==`/`<=` typo either rejects a legal 31-leading-
  zero code or reads one byte past the buffer.
- H.264's private SpsReader duplicates the same read_bits/read_ue
  shapes with no tests of its own at all (only reached through
  multi-field SPS parsing, several fields deep). Added direct tests.
- HEVC's per-AU trailing-zero strip after the last NAL (no start code
  following) walks `end` down to trim padding; a wrong-direction typo
  there walks off the end of the buffer instead of terminating -
  exactly the "loop must make positive progress on malformed input"
  class. Added a test with a zero-padded trailing NAL.
- HEVC's SEI match guards (`sei_mastering.is_none()` /
  `sei_content_light.is_none()`) implement "first HDR10 value in the
  title wins" - untested, and a naive test using both-messages-per-AU
  can't even exercise the guards because the whole-scan early return
  above them already handles that case. Split into single-message-
  per-AU tests that actually reach the arms.
- parse_mastering_display/parse_content_light_level's length guards
  were `< N` with no boundary test; one-byte-short input now confirmed
  to return None instead of indexing out of bounds.
- DTS's drain_front collapses duplicate offset-0 PTS markers after
  rebasing; untested, and the visible effect (front_pts()) can't tell
  a working collapse from a broken one since it already returns the
  right marker either way - the actual defect is unbounded growth of
  pts_marks over a long recording, so the new test asserts the bound
  directly across repeated drains.
- DTS's dts_core_samples/dts_core_sample_rate header-length guard and
  next_core_boundary's syncword-length guard got exact-boundary tests
  the same way; also caught a nblks `<<`/`>>` direction bug candidate
  in the mutant (confirmed the real code is correct, just untested).

Real bug found and fixed, not just a test gap:

H.264's parse_sps_high_profile_ext re-implemented emulation-prevention
byte stripping inline (a window scan: match `00 00 03` at position i,
advance 3, else advance 1) instead of calling the existing
unescape_ebsp_prefix used by slice-header parsing. On a run of 3+ real
zero bytes ahead of an 0x03 - non-conformant, but this is disc bytes,
not a spec-clean encoder - the two disagreed: unescape_ebsp_prefix's
cumulative zero counter (matching the H.264 reference decode process
and libavcodec's RBSP extractor) treats it as an escape and drops the
0x03; the window scan treats it as real payload and keeps it,
corrupting the SPS bits read after it. Extracted the shared rule into
`unescape_ebsp` (parameterized on output length so both the 16-byte
slice-header prefix and the unbounded SPS case can share it) and
pointed both call sites at the one implementation. Added a regression
test pinning the shared function's behaviour on the input that used to
separate them.

All new tests hand-verified against the actual mutation (operator
flipped or guard replaced by hand, confirmed red, then restored) per
the mutation-testing brief, not just written and trusted.
2026-08-01 13:35:43 -07:00
Matthew Jackson ef934ed308 Close mutation-testing gaps in the TS/PS mux (ts.rs, ps.rs, tsmux.rs)
A 12,330-mutant run left 159 survivors across these three files, all from
missing assertions rather than wrong code — every gap here is a test, no
production logic changed.

Two shapes accounted for most of them:

- Buffer-cap constants (MAX_PES_BUFFER_TOTAL, MAX_PS_BUFFER,
  MAX_BD_PES_PAYLOAD, PES_BUFFER_INIT_CAP) were only ever read by tests
  through their own symbol, so a mutated `*`/`-` in the constant's
  definition changes what the symbol itself evaluates to and every
  self-referential assertion still passes. Pinned each against a literal
  computed independently in the test.

- Several `>`/`==` boundary checks on framing lengths (MPEG-2 pack header,
  system header, BD-TS adaptation field) were only ever exercised with
  slack in the buffer, never at the exact byte the check exists for.
  Added exact-fit cases for the pack header, system header, and
  psi_payload_base's AF-consumes-everything boundary.

Real, higher-value gaps closed along the way:

- ts.rs's per-PID discontinuity_flag and the NULL-TS concealment marker
  both require adaptation_field_length > 0 before trusting the AF flags
  byte; neither branch had a test proving af_len == 0 (no flags byte at
  all, ordinary payload underneath) is left alone.
- header_remaining (PES header spillover across TS packets) only had
  single-continuation-packet coverage, which can't distinguish `-=` from
  `+=`/`*=` because the corrupted value never gets read again. Added a
  case spanning two continuations.
- ps.rs's parse_stream_id_extension (used for HD-DVD 0xFD routing) walks
  nine optional PES-header/extension fields with a `pos +=` each; only
  the PTS/DTS pair had ever been exercised. One test now arms every
  field and checks the walk lands on the right byte.
- find_ps_boundary's `sc + 3 >= len` guard had no test at sc == 0 with a
  bare 3-byte start code, the case a `+` -> `-` mutation turns into a
  debug-mode subtract-overflow panic on ordinary tail-of-buffer input.
- tsmux.rs: an oversized video access unit must go out as a single
  unbounded-length PES; the `is_video || small-enough` guard that
  enforces this had no test with a video frame actually over the
  bounded-PES threshold, so a `||` -> `&&` mutant survived (it would
  silently split a keyframe across several look-alike-independent PES
  units). Also pinned the PES-length and PTS big-endian encodes at
  values above 255 / with bit 29+ set, where a `>>`/`<<` swap first
  becomes observable.

Every test above was verified by hand: applied the exact mutation,
confirmed the test fails (or the specific panic fires), then reverted.

Left unclosed, all confirmed equivalent by hand-tracing rather than
just left alone:
- Every `<<8 | byte` PID/length bit-combine (ts.rs pid/PAT/PMT parsing,
  ps.rs dvd_audio_pid/hddvd_extended_pid/parse_pts): the two halves
  never share a bit, so `|` and `^` produce identical output for every
  input - no test can tell them apart.
- ts.rs's `af_len > 183` check in process_packet: fully subsumed by the
  `payload_start >= TS_PACKET_BYTES` check three lines later for every
  af_len that could trip it.
- ts.rs's out-of-range `pid_index` sentinel (-1 vs 1): unreachable, since
  a TS PID is masked to 13 bits (max 8191) and the table is always sized
  to at least 8192.
- A cluster of "push an empty slice on an exact boundary" mutants in
  tsmux.rs's write_pes_chain (offset < hdr_len, af_bytes stuffing
  guards): the guarded write becomes a length-0 write_all, a no-op
  either way.

Not reached this pass, for lack of a clean seam within the time
available - ps.rs's extract_packets bounded-PES-length exact-fit
checks (lines 278/282/303, the `sc+6>len` / `sc+6+pes_len>len` /
force-flush cap arithmetic). The first two need a scenario where
"proceed vs. wait one more byte" is observable in the packet list, and
the third only shows up at a start-code offset (sc) that survives to
the moment the cap check runs - in this code path sc is always 0 once
an unbounded PES buffer starts accumulating, since nothing before it
ever drains. Didn't find a construction in the time available; flagged
rather than papered over with a self-referential assert.
2026-08-01 13:34:48 -07:00
Matthew Jackson 2c3e372e9b Fail the identity probe when INQUIRY returns a short data phase
DriveId::from_drive issues three data-in commands. The two GET CONFIGURATION
calls both clamp on bytes_transferred, with a comment noting it is
device-reported and untrusted. INQUIRY, three lines above them, discarded it
and decoded bytes 8..43 unconditionally.

The buffer is pre-zeroed, so a drive answering GOOD status with a short or
empty data phase — a USB-SATA bridge mid-wedge does exactly this — produced
blank vendor, product and revision strings and a byte 0 of 0x00. Every
platform enumerator gates on raw_inquiry[0] & 0x1F == the optical peripheral
type, and 0x00 is DIRECT ACCESS, so the drive silently disappeared from the
device list instead of reporting that its identity probe had failed. The
operator sees no drive at all rather than an error.

Anything shorter than the SPC-4 standard 36-byte header is now
E9058 DriveInquiryShort, and the buffer is truncated to what actually
arrived so nothing decodes past it. Exactly 36 bytes is still accepted: the
vendor-specific tail is optional.

This is the same defect as the READ CAPACITY short-transfer bug fixed
earlier today, in the same crate, found the same way.
2026-08-01 12:21:15 -07:00
Matthew Jackson 00abead473 Give extract and analyze one label-parser tie-break, not two
mod.rs had two independent implementations of "pick the winning label
parser". select_result owns the rule — highest confidence wins, and on a tie
the earlier entry in PARSERS wins — and carries a regression test for a past
bug where analyze() picked the LAST equal-confidence parser instead of the
first. extract(), the path that actually ships, re-derived the same rule with
its own inline scan and had no test of its own.

The tie-break is load-bearing rather than incidental: array order encodes a
trust ordering, with the hand-vetted parsers registered ahead of the ones
that detect on any BD-J disc. A later parser winning a tie means the disc
gets labels from a less trusted source, silently.

extract now collects its candidates and calls select_result. Confirmed by
flipping the tie-break to prefer the later entry: the shared test fails,
where before the fix that mutation was invisible to the whole suite.

This is the seventh instance tonight of one policy implemented twice with
only one copy hardened, and the second in this file after the chapter
mark_type filter.
2026-08-01 12:15:48 -07:00
Matthew Jackson f655befb51 Make the FMTS phase gate reachable by tests
An FMTS forensic segment interleaves two variants at the unit level: the
disc carries both halves and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving it as
ciphertext is correct, because the muxer drops untouched ciphertext cleanly.

The decision lived inline in apply_aacs_map's per-unit closure, where no test
could reach it. A mutation run flipped the subtraction to an addition and the
parity comparison, and every test still passed — so the gate that decides
which half of a forensic segment we decrypt was entirely unasserted.

It is now unit_is_our_phase(). The index arithmetic also saturates and
clamps: both inputs come from the key map, so a unit below its own range
start or a zero unit size means a malformed map, and neither may panic on
debug overflow or a divide by zero inside a library a long-running service
depends on.

Four of the five surviving mutants now fail. The fifth, dividing by the unit
size versus multiplying, is equivalent rather than uncovered: aligned offsets
are exact multiples of the unit size, so the two differ only by a factor of
unit_sectors squared, and unit_sectors is the constant 3 — odd, so parity is
preserved on every reachable input. The proof is recorded in the test instead
of a test that pretends to cover it.
2026-08-01 11:48:39 -07:00
Matthew Jackson a6585df923 Walk past empty extents iteratively instead of recursing
fill_extents skipped an exhausted or zero-sector extent by calling itself,
which costs a stack frame per skipped extent. Nothing filters
sector_count == 0 out of a UDF or MPLS extent list, so a malformed disc
declaring a long run of empty extents recursed once per extent before
reading a single sector. Rust does not guarantee tail-call elimination, so
that overflows the stack — which aborts the process rather than returning
an io::Error, taking a long-running service down with it.

The skip is now a loop. The regression test runs on a 256 KiB stack, where
the recursive version dies and the loop finishes immediately.
2026-08-01 11:05:12 -07:00
Matthew Jackson 823ec82872 Reject a short READ CAPACITY reply, and count only entry marks as chapters
Two cases of the same shape: one policy implemented twice, with only one
copy hardened.

Disc::read_capacity decoded buf[0..4] from READ CAPACITY (10) without
checking that the transport actually delivered four bytes, even though its
comment claims to mirror decode_read_capacity — which has exactly that
check, and documents why. A drive answering GOOD with an empty data phase
leaves the buffer zeroed, so last_lba decodes to 0 and the probe reports a
one-sector disc instead of an error. It now calls the shared decoder rather
than re-deriving it.

collect_chapter_summary filtered chapters on mark_type <= 1, counting the
reserved type 0. PlaylistMark's own doc says filters must test == 1, and
disc/bluray.rs did; the labels path did not, inflating the public
chapter_count and letting a playlist whose only marks are reserved pass the
chapter_count == 0 skip. Both sites now share PlaylistMark::is_chapter_mark
so the copies cannot drift again.

Both fixes were confirmed red before green.
2026-08-01 11:00:49 -07:00
Matthew Jackson 5049e6515f ci: make libfreemkv prove its five dependents still compile
Every job above proves libfreemkv builds; none proved anything built on
it does. That gap bit today one level up — an engine signature change
broke autorip and went unnoticed, because consumer CI only fires on a
push to that consumer, and nobody pushed one.

libfreemkv sits underneath all five dependents, so a break here costs
more than a break anywhere else in the project. It is now the place the
question gets asked, since it is the place the change happened.

cargo check --all-targets only: each dependent has its own suite for its
own behaviour. This answers the narrower question that went unanswered.
2026-07-31 18:40:24 -07:00
Matthew Jackson ba3ce1e6a5 Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv,
the crate the other seven build against and the one held up as the
reference workflow, was the last one still linting the library only — so
its ~3,000 tests, by far the largest body of test code in the project,
had never been linted at all. Turning the flag on surfaced 74 findings.

Most were mechanical and applied with clippy --fix. The rest, by hand:

- Four discarded Results in decrypt.rs. css::descramble_region returns a
  Result and four CSS tests threw it away, so a descramble that FAILED
  would have surfaced as a confusing buffer-comparison mismatch instead
  of the actual error. They expect() now.
- A dead `kp` field on the PlantedWalk fixture. The test deliberately
  asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4
  rather than against a stored value — its doc comment says so — which
  makes the field not just unused but a trap: the obvious "fix" of
  asserting against it would quietly weaken the test to comparing the
  fixture with itself. Removed.
- Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut,
  three vec!s that only ever needed arrays, a filter_map whose every arm
  was Some, and a Vec::new()+push chain.
- Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering
  in the generated docs.
- A five-[u8; 16]-tuple return type named FourLevelParts.

Three lints are allowed at the specific sites, with reasons, because
they are wrong for this domain: the underscores in the bitstream-header
literals mark BITFIELD boundaries, not digit groups, so regrouping them
uniformly would satisfy the lint by destroying the only thing they
encode; and in three table-validation loops the loop variable is the
domain value under test (a DTS SFREQ code, an AMODE value, a palette
entry number), which is what the assertion messages name.
2026-07-31 15:08:37 -07:00
Matthew Jackson 94f3d106f9 Expose error_code so consumers can read a code instead of parsing one
io_error_code was private, so the predicates built on it (is_halt,
is_skippable_title_stub, is_disc_level_no_key) were the only way to ask
anything about an io::Error's origin. A consumer that needs the code
itself — to report WHY a title failed rather than to branch on one of
three known cases — had no route to it: mux_stream returns an io::Error,
the typed Error is gone by then, and only the E<code> string prefix
survives.

That left every front-end to re-implement the prefix parse by hand,
which is precisely the string-matching 1.5.x spent its time removing.
One parser, exported.

No behaviour change: the function is unchanged and the three predicates
still call it.
2026-07-31 14:46:39 -07:00
Matthew Jackson a24bf3b96d ci: make the branch tip actually buildable
CI checked out one repo, but libfreemkv path-deps ../freemkv-unlock on
the branch tip — release.sh swaps that to a git tag only inside the
TAGGED commit, then restores the path dep on the branch. So the branch
tip has never been buildable in CI by construction, and every green run
we have ever had was a tag build. Windows and Linux were first compiled
at release time, which is the worst moment to discover a build error.

Both repos now check out into subdirectories (actions/checkout refuses a
`path:` outside $GITHUB_WORKSPACE, and ../freemkv-unlock is outside), so
the path dep resolves exactly as it does on a developer's machine. Every
cargo step runs with working-directory: libfreemkv, and rust-cache is
pointed at the same workspace.

The sibling is taken from `dev`. On main/tag builds the Cargo.toml in
that commit carries the git-tag dep instead, so the extra checkout is
simply unused there.

This is what makes the new branch policy's "dev must be green"
achievable rather than aspirational.
2026-07-31 13:27:05 -07:00
Matthew Jackson cc35b7af66 ci: build dev as well as main
Work now lands on dev and CI must be green there; main only moves at
release time, to the tagged commit, so a push to main is the release
validation run rather than day-to-day feedback.

leak-guard already runs on every branch (on: [push, pull_request]) and
release.yml stays tag-triggered, so neither needed a change.
2026-07-31 13:16:56 -07:00
Matthew Jackson 1faf2ce51b fix(css): revert the hard-fail — it made real DVDs unrippable
I broke DVD ripping earlier today and the real-media acceptance gate
caught it on its first full run. Greenland.iso failed with E7013
"Decryption failed"; reverting only this change made it rip clean in 8
seconds. That is a regression I introduced, not a pre-existing defect.

WHAT I GOT WRONG.

Round 9's crypto lens reported that descramble_region "descrambles with
a key it just proved wrong" when the crib check rejects the cached key
and the re-crack also fails. I agreed, and made it Error::DecryptFailed
to match the AACS path, on the reasoning that CSS has no external key
source so a failed crack on a readable sector should never happen.

The premise was wrong. `attack_crib` is a HEURISTIC, not a proof: it
finds a periodic run in the unscrambled header and predicts the run
continues past 0x80. When that prediction does not hold, the crib
reports a mismatch even for a CORRECT key — and the re-crack then fails
BECAUSE the crib was never valid. So crib mismatch plus crack failure is
the signature of a crib false positive, not of a stale key. The cached
key is not proven wrong; it remains the best available evidence, and on
a real DVD it is very probably right. Real discs hit this constantly.

The deeper error was treating "no key" as one thing across schemes. An
AACS unit key either opens a unit or it does not — the Verify-Media-Key
relation decides it, and a wrong key is provable. A CSS title key is
recovered from the data itself by an attack whose success varies sector
by sector, so "the crack failed here" says something about THIS SECTOR's
plaintext, not about the key. Unifying the policy was right for the
schemes that can prove a key wrong. CSS cannot, and I folded it in
anyway.

decrypt_span keeps its shape and the cross-scheme test keeps its two
AACS arms, with CSS now explicitly excluded and the reason stated.

Three tests asserted the wrong behaviour and are corrected, including
one I rewrote earlier today to pin exactly this. Every one of them
passed the whole time the code was broken — because none of them had
ever seen a real disc.

The lesson is the one I kept stating and then did not act on: 3,013 unit
tests, ~400 mutants killed and nine audit rounds did not catch this, and
one acceptance run did. Synthetic media cannot reproduce what a real
disc does.
2026-07-30 22:07:32 -07:00
Matthew Jackson d0ca927a9e test: constrain the AC-3/E-AC-3/DTS header decode and the boxes it emits
All 59 measured survivors in mp4/audio.rs: 50 killed, 9 proven
equivalent, none left unaddressed. No production change — every
extraction reads correct against ETSI TS 102 366 (5.3.2, 5.4.2, Annex
E.1.3, F.4, F.6.1) and TS 102 114 5.3.1.

It was a fixture gap, not a code defect, and a specific one: the
existing fixtures gave several fields the SAME value (fscod=0, bsmod=0,
lfeon=1, acmod=7) and asserted only derived channel counts. Nothing
asserted the emitted dac3/dec3/ddts payload BYTES at all, so the
packer's shifts and masks were entirely unconstrained. A wrong mask
there does not crash — it writes a box declaring the wrong channel
configuration, and a player believes it.

Several kills needed fixtures designed to discriminate rather than
merely exercise:

  the reduced-rate branch needed fscod == 3, which no test in the file
  reached — and `== -> !=` survived on an fscod=0 fixture only because
  the reduced table happens to return 48000 there too

  the DTS LFF mask needed a value where XOR and AND differ in MEANING:
  on the 5.1 fixtures LFF flips 1 -> 2 and BOTH codes mean "LFE present"

  the acmod mix-level skips needed three different acmods, because `^`
  is a no-op unless acmod == 4 exactly

  the ddts bitrate needed 44100/512, which does not divide evenly — at
  48000 the rounding is invisible and `/ -> %` on den/2 survives

The 9 equivalents are one pattern: OR-ing a shifted high part with a
masked low part on disjoint bit lanes, where the mask is on the same
line as the OR. Since cargo-mutants applies one mutation at a time, no
single mutant can break both. Each was applied and observed green.

FILED, not fixed: reserved sample-rate codes are silently guessed as
48 kHz (audio.rs:133, :20/:153) while a reserved DTS AMODE is refused
with a comment explaining why. Same silent-wrong-metadata class,
opposite answer. Refusing would make such a stream unmuxable, which is a
product call.
2026-07-30 21:51:26 -07:00
Matthew Jackson 093c90481b refactor(decrypt): one orchestrator owns the no-key decision
There were TWO top-level decrypt paths: decrypt_sectors_impl for CSS and
clear media, whose AACS arm was a bare `return Err` stub, and a wholly
separate decrypt_sectors_mapped for AACS. Each scheme therefore decided
its own answer to "there is no key for these bytes", and nothing held
them to the same one.

They drifted, in opposite directions, within a single release:

  css::descramble_region descrambled with a key the sector's own crib had
  just proven stale — garbage behind an intact clear header, reported Ok.

  the mapped path returned early for any LBA outside every range, before
  ever asking whether those bytes were ciphertext, so an unkeyable
  encrypted unit passed through and extract counted it as good.

Both were fixed individually earlier today. This removes the shape that
allowed them.

decrypt_span is now the single orchestrator: it owns the loop, the
refusal, and the loss count, and each scheme supplies only what
genuinely differs. apply_aacs_map is a scheme step that reports what it
could not open; it no longer decides what that means. The public
wrappers (decrypt_sectors, _in_content, _mapped) are unchanged in
signature and all funnel through it.

Adding a scheme now means adding an arm here, which means answering the
refusal question. That is the point.

The new test asserts ONE verdict across all three schemes — AACS with no
map, AACS with an encrypted unit outside every range, and CSS whose
re-crack failed — plus that clear media is NOT a refusal. A per-scheme
test cannot hold this: each would keep passing while the two disagreed.
Flipping the AACS arm back to pass-through reds it.

Also removes the last of the tracing-capture scaffolding. Serialising
those captures crate-wide did not fix the 1-in-10 flake, and asserting
the predicates directly made the helper, both capture subscribers and
an unrelated dead OrderSink unused. Deleted rather than left behind.
2026-07-30 21:35:33 -07:00
Matthew Jackson cb8cd21b21 fix(session): delete two dead accessors, make into_drive fallible
drive() and drive_mut() had ZERO callers — not in libfreemkv, freemkv,
autorip, bdemu, keysources or kdb. Deleted rather than converted: dead
public API that panics is not an API worth preserving the shape of.

into_drive() had two callers and now returns Result. The empty-slot
state is reachable through ordinary public use — stage_drive_as_reader
moves the drive into the reader slot, and calling into_drive twice moves
it out — so the panic was not guarding a caller error. identify() was
converted for exactly this reason in this same release; the fix went to
one of four public sinks and the other three were left.

I deferred this on the assumption the blast radius was large. It was
three call sites. Checking beats assuming.

Also fixes a REAL FLAKE in the gate, which is worth more than the above.
resolve_vid_only_bus_key_gate_reports_true_has_volume_id... failed about
one full-suite run in ten while passing every time in isolation. It
installed a capturing tracing subscriber to read back the has_volume_id
field of a warn.

That cannot be made reliable: dispatcher::set_default is THREAD-LOCAL
while tracing's callsite-interest cache is GLOBAL. The original author
knew, and called rebuild_interest_cache() — necessary but not
sufficient. I first serialised every capture in the crate behind one
lock (harness::with_captured_tracing, which also removed the same
hand-rolled dance from three other sites). Still 1-in-10, because the
cache can be re-evaluated against the process-default dispatch rather
than the thread-local one.

So the predicate is now a named function, handshake_has_volume_id, and
the test asserts the VALUE. A boolean does not need a subscriber to
check. The gate's hard-error behaviour keeps its own test.

Measured: 14 consecutive full-suite runs, 2994 passed, 0 failed.

A flaky gate is worse than a missing one — every green after it means
less, and this one had been eroding trust in the whole suite.
2026-07-30 21:18:13 -07:00
Matthew Jackson 1fd612954b test(mux): replace a false-green discontinuity test with the property it named
a_signalled_discontinuity_survives_a_backstop_discard asserted that a
SOURCE-signalled discontinuity on discarded bytes still reaches the AU
that follows. It did not test that. Deleting the disc_marks push, or the
mark-retirement loop inside discard_gap_before, left it passing.

The mechanism: the `discontinuity = true` rode the FIRST over-cap push,
which still has the next AU's delimiter at buf[0] — so it force-flushes
as an over-long AU rather than discarding, and THAT AU consumes the
mark. The assertion's `.find(|x| x.data.contains(&0x22))` then filters
it out, and the flag it reads comes entirely from `pending_gap`, set by
the second push's backstop. Behaviourally identical to the test 40 lines
above it, under a name promising something else.

I wrote it this morning, in the same commit that fixed a different test
for having a fixture that never reached the code it named, while
cataloguing that exact shape. Third instance today of writing the bug I
was hunting.

The two mechanisms cannot be isolated in one fixture — a fragment that
trips the backstop sets pending_gap regardless — so they now get one
test each. The replacement drives disc_marks end to end with no backstop
involved: a flagged fragment that carries a complete AU and is emitted,
not discarded. Nothing else pinned that path. Removing the disc_marks
push reds it.

Found by the round-9 opus escalation over test quality, dispatched
because the sonnet pass over the same 17,000 lines of new test code
returned zero findings.
2026-07-30 20:07:11 -07:00
Matthew Jackson 139abb721b fix(io): a halted fsync is recognisable as a halt, not a hard failure
The three bounded-fsync failures returned bare io::ErrorKind values —
TimedOut, Interrupted, Other/EIO. `is_halt()` matches on
`io_error_code(e) == Some(E_HALTED)`, i.e. the "E<code>" prefix that
`From<Error> for io::Error` mints, and that is documented as the ONLY
recognised shape. A bare ErrorKind carries no prefix.

So cancelling a rip while sync_all / finish was inside the bounded fsync
made `is_halt()` return false, and the CLI reported a clean user cancel
as a hard I/O failure at the end of an otherwise complete mux.

All three were also mutually unclassifiable, which is the same
information-loss the numeric-code scheme exists to prevent: a caller
could not tell "cancelled" from "NFS wedged" from "worker died", and
should not retry the third the way it retries the first. And their
Display text is std English — "timed out", "operation interrupted" —
reaching a user from library code, which this crate does not do.

Now Error::Halted, Error::SyncTimeout (E_SYNC_TIMEOUT 9056) and
Error::SyncWorkerLost (E_SYNC_WORKER_LOST 9057), on both platforms.

E_HALTED also now maps to ErrorKind::Interrupted rather than falling into
the 6000..=6999 InvalidData bucket. A stop is an interruption, not
invalid data. Nothing branched on the old kind — every consumer uses
is_halt() — so this is safe as well as more accurate.

Two things worth recording. I first placed the E_HALTED arm AFTER the
6000..=6999 range arm and wrote a comment claiming it preceded it; match
arms are ordered, so the range won and the comment was simply false. The
test caught it. And the macOS test asserted only that each arm was
non-Ok with a particular ErrorKind — it passed throughout the period the
three were indistinguishable. It now asserts they can be TOLD APART,
which is the property that actually matters.

Found by the round-9 opus escalation over the API contract.
2026-07-30 20:04:08 -07:00
Matthew Jackson 2d9b648486 fix(udf): file_start_lba must skip a leading unrecorded extent
Regression from the round-8 change that started RETAINING ECMA-167
4/14.14.1.1 type-1 (allocated, not recorded) descriptors. Retaining them
is correct — dropping one slides every later extent's data down by the
hole's length, corrupting the file silently. But read_icb_extent still
took extents.first(), so the value it returns can now be a hole.

A type-1 extent's lba is where SPACE is allocated, not where bytes live.
IcbExtent's own doc says exactly that. file_start_lba hands the value
out as "the absolute starting LBA of a file's first data extent", and
ifo.rs uses it as the base for every VTS VOB extent:

    file_start_lba(IFO) + vtstt_vobs + cell.first_sector

So a DVD whose IFO's first descriptor is type-1 reads its entire video
title set from the wrong place on disc. No error anywhere — the reads
succeed, they just land on unrelated sectors. Verified: the test reports
2900 instead of 2040 with the old code.

Same shape as file_extents/extents_abs_at dropping the recorded flag,
and the same root cause: one change taught read_icb_extents about a new
extent type and did not visit the accessors that consume its output.
Three of them; two are still open (task filed).

Found by the round-9 opus escalation over the API contract, dispatched
because the sonnet pass over the same scope returned zero findings.
Seven of its eight items were absences rather than wrong lines — the
class a wrong-line scan structurally cannot see.
2026-07-30 20:00:15 -07:00
Matthew Jackson 08da31883b fix(udf): retry the customary VDS location when the recorded one holds nothing
The Main Volume Descriptor Sequence was selected from the anchor's
declared extent whenever that extent's SHAPE was usable — length >= 16
sectors (ECMA-167 3/10.2.1), non-zero location, no address wrap — and
the customary location was tried only when the shape failed.

Shape is a property of the field, not of what is there. An anchor can
pass all three checks and point at nothing: a mastering tool that wrote
the reserve location, a stale anchor on a rewritten volume, or
deliberate corruption on an untrusted disc. The sweep then finds no
Partition Descriptor, partition_start stays 0, and the volume is
rejected as UdfNotFilesystem — while the real sequence sits unread at
the location the old fixed sweep would have found.

So the branch a DAMAGED disc actually takes had no recovery path, which
is backwards: that is the branch recovery exists for.

Now both are candidates and the fallback is retried on OUTCOME. This is
the same principle the Metadata File Location chain in this same
function already uses — treat the recorded value as a candidate, fall
back when it does not pan out — applied one level up. The sibling was
added in 8e70f2a and the asymmetry has been sitting there since.

A read fault inside a sweep no longer aborts before the next candidate
is tried, but it does not vanish either: it is held and returned if no
candidate yields a partition. "Could not read" stays distinct from "read
fine and the bytes say no", because mux::resolve caches the second for
the whole disc.

Not addressed, and pre-existing: the RESERVE sequence at avdp[24..32]
exists in ECMA-167 3/8.4.2 for exactly a damaged main sequence and is
still not consulted. Worth doing; a bigger change than this one.
2026-07-30 19:49:03 -07:00
Matthew Jackson 871471edcf fix(decrypt): an encrypted unit outside every key range fails, not passes
decrypt_sectors_mapped returned early for any LBA no map range covers,
on the reasoning that unmapped means clear filesystem or nav. "The map
has no key here" and "there is nothing to decrypt here" are different
statements, and only the second makes passing the unit through correct.

On a multi-CPS disc an orphan clip — referenced by no playlist, so in no
title extent and therefore in no range — hits the first and was treated
as the second. The early return fired BEFORE the aacs_unit_encrypted
gate below it, so nothing ever asked whether those bytes were
ciphertext. extract_tree then counted them as bytes_good, dropped the
.partial suffix, set complete = true and exited 0: a scrambled file on
disk with a clean bill of health.

FileResult's own doc already stated the intended contract — "unreadable
sectors AND undecryptable units both land here (extract fails a bad
decrypt loud)". The code did not implement it on this path.

The fix is one line of symmetry. The split-unit branch immediately above
already makes exactly this distinction, checking aacs_unit_seed_encrypted
before refusing. This branch did not, so the same question got two
answers eight lines apart — the duplication shape this release keeps
finding.

Deliberately NOT changed: the decrypt decision on an orphan is still to
refuse rather than guess. Blind trial-decrypt is what the keymap-only
model exists to remove, and extract has no CPS/forensic fetch source.
The defect was never that we declined to key it; it was that declining
looked like success.

The test asserts BOTH directions — a clear out-of-range unit must still
pass through byte-identical, because that is the ordinary whole-disc
read and breaking it would trade one silent defect for a loud one.
2026-07-30 19:44:37 -07:00
Matthew Jackson 0890d84d4d fix(mux): resync-gate drops reach errors(), including after the gap resolves
ResyncGate::dropped is zeroed the moment a keyframe disarms the gate,
and the only EOF warning fires for gates STILL armed. So a mid-title gap
that resolves left no trace anywhere — and most gaps do resolve. A rip
with several concealed gaps reported 0 errors and 0 lost bytes while
whole GOPs had been discarded, which disc.rs's own test comment calls
the ONLY channel through which loss is reported.

This is the other half of 41e0982. Arming the gate after an 8 MiB
backstop discard is right — a picture with dangling references must not
ship — but until the drop is counted that trades silent corruption for
silent loss.

The gate now carries dropped_total alongside dropped: per-run answers
"how expensive was this gap", cumulative answers "what did the caller
lose". errors() sums the gates.

Summed in ONE place rather than counted at the three admit call sites.
Three copies of the same increment is how the mux-flush path ends up
counting and the main path not, or the reverse — the duplication shape
this release has been removing. The gate already knows its own total;
the accessor just has to ask.

Found independently by two round-9 lenses, which is what raised it from
plausible to worth acting on.

Both halves are pinned: removing the dropped_total increment reds the
resync test, and removing the sum from errors() reds the DiscStream one.
The second test asserts the ACCESSOR rather than the gate's counter,
because a test on the counter would have passed throughout the entire
period the defect existed.
2026-07-30 19:40:48 -07:00
Matthew Jackson f1c2b2c158 fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote
it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry
(12.1.3). Both fields are MANDATORY there, so unlike Matroska — which
omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to
leave out. The result was a structurally complete file that passes every
container check, declares a 0x0 video track, cannot be rendered, and is
written with no error anywhere.

WHY IT WAS POSSIBLE, which is the part worth keeping:

pixels() previously fabricated 1920x1080 for Unknown. That was wrong but
playable, so this sink never needed a guard and the absence of one was
invisible. Changing the sentinel to (0, 0) moved the defect instead of
removing it — a zero PAIR still reads as a usable value, so the sink
stored it and serialised it.

The accessor's doc comment then ENUMERATED the callers it believed were
safe: "the Matroska sink omits the optional elements, the VobSub writer
omits its size: line, and no caller divides by either dimension." Two of
those three are true. MP4 was not on the list because MP4 has no guard
at all, and a prose list cannot enforce itself. mkv.rs's own comment
even states the principle — "the check belongs in the one accessor
rather than in each caller that remembered to write it" — and
labels/mod.rs still carried its own duplicate Unknown test long after
the accessor took that job over.

So: pixels() now returns Option. Not because Option is tidier, but
because every caller genuinely needs a DIFFERENT answer and the compiler
is the only thing that reliably makes them choose one. Matroska and the
metadata sinks take unwrap_or((0, 0)) with the reason stated at each
site; the VobSub path degrades to a palette-only .idx; MP4 fails with
E_MP4_UNKNOWN_RESOLUTION (9055).

Six call sites, not the five my first grep showed — I piped it through
`head` and acted on a truncated list. The compiler caught the sixth.
That is the same mistake as trusting a lens that reported silence.
2026-07-30 19:34:54 -07:00
Matthew Jackson a8e7d46400 fix(css): no provable key is a hard failure, matching AACS
descramble_region descrambled with the key a sector's own crib had just
proven stale, whenever the re-crack from that sector also failed. The
clear header is not scrambled, so it survives intact: the sector still
opens with a valid pack start and passes every structural check the PS
demuxer applies. Only the payload is corrupted — exactly where nothing
looks. Ok(0) dropped, exit 0.

CSS has no external key source. The title key comes only from cracking
the data, so on a READABLE sector "no key" is not a missing input, it is
recovery failing on bytes we can see. That should never happen, and when
it does the answer is not to emit something.

Now Error::DecryptFailed — the same verdict the AACS path already gives
for a unit no held key opens. Both alternatives to failing are bad data
reported as success: descrambled with a rejected key it is garbage
behind a valid header, and passed through untouched it is ciphertext
where plaintext is meant to be.

WHY IT WAS POSSIBLE, which matters more than the fix:

There is no single place that owns "what do we do when there is no key".
decrypt_sectors_impl looks like the central dispatch, but its AACS arm
is a `return Err` stub — AACS decrypts entirely through
decrypt_sectors_mapped, a separate top-level path. So CSS decided its
own policy inside css/, AACS decided in decrypt.rs and mux/resolve.rs,
and nothing held them to the same answer. The asymmetry was not an
oversight; it was structurally permitted.

How a disc decrypts is one process — resolve a key for this data, apply
it, refuse if it cannot be proven. Only the resolve-and-apply step is
scheme-specific. Filed as a task: the policy belongs in one orchestrator
with the schemes supplying only what genuinely differs.

Two tests changed rather than added, both of which pinned the old
behaviour: the unit test asserted the sector was descrambled, and the
integration test asserted the scramble flag was cleared, which is what
descrambling-with-any-key does. Neither established that the result was
CORRECT — the fourth bad-test shape.
2026-07-30 19:10:32 -07:00
Matthew Jackson 94f087d5ab fix(udf): a read fault locating the Metadata File is not a non-UDF disc
Regression I introduced in 8e70f2a earlier today. The Metadata File
Location fix replaced

    read_sector(reader, meta_file_lba, &mut meta_icb)?;

with a candidate loop guarded by `.is_ok()`, which discards the error.
A transient read fault — marginal sector, drive re-read, an ECC recovery
that reports failure once — then falls through to the block-0 fallback,
the File Set Descriptor read there finds the wrong tag, and the volume
comes back as Error::UdfNotFilesystem: the deterministic verdict "this
is not a UDF disc" for a retryable I/O event.

mux::resolve MEMOISES that negative for the whole disc, so one flaky
read silently demotes every remaining title to the base-Unit-Key-only
path — the AACS 2.1 forensic units garble and the demux drops them. The
mux completes with less content and no error, which resolve.rs's own
comment says must never happen.

This file already draws the same distinction twice in prose, at the AVDP
check and the FSD check: read fine but the bytes say no is structural;
could not read is transient. The loop erased it.

Two things I got wrong on the way, both worth recording:

The first guard was `meta_tag == 0` — nothing read at all. That misses
the damaging case, which is the RECORDED candidate faulting while block 0
reads fine and holds some other descriptor. meta_tag is then non-zero and
not 266, and the fault is still laundered into a structural negative. The
guard is now `meta_tag != 266`: any unread candidate leaves the verdict
unproven, so the fault wins.

The first test denied BOTH candidate LBAs, and passed WITHOUT the fix —
the block-0 read fails too, so a read error propagates from further down
either way. A test whose fixture never reaches the changed line. That is
the eleventh bad-test shape this audit catalogued, and I walked straight
into it while fixing a defect found by looking for it. The fixture now
denies only the recorded location, and removing the guard reds it.

Found by the round-9 opus escalation over correctness, dispatched
because the sonnet pass over the same scope returned zero findings.
Silence from a cheap model is not evidence.
2026-07-30 18:57:13 -07:00
Matthew Jackson d706d4b49e fix(labels): make deluxe master-enum selection deterministic
identify_master_enums picks, for each fingerprint, the best candidate
class out of CandidatePool. Its tie-break only prefers an exact ldc
count over an inexact one, so two candidates that are BOTH inexact but
both within LDC_COUNT_TOLERANCE are decided purely by iteration order —
and the pool was a HashMap.

Rust seeds HashMap per instance, so this is not merely unstable across
runs: the new test resolves the SAME jar to both Alpha and Beta within a
single process, across 16 iterations. The same disc could emit different
commentary/SDH/descriptive labels on consecutive rips of unchanged
input, with nothing in the output saying the choice was arbitrary.

BTreeMap fixes it by construction rather than by a sort someone can
forget to keep. The pool is capped at MAX_CANDIDATE_CLASSES, so the
ordering cost is irrelevant.

The test runs the whole identification sixteen times and asserts one
distinct winner. A single run cannot distinguish deterministic from
lucky, and the seed does not change within a process — so repetition is
what makes this a test rather than a hope.

Found by the round-9 labels pass, which was dispatched specifically
because every one of the ten lenses had reported leaving deluxe.rs
unread. 1,159 new lines that nobody had opened.
2026-07-30 18:49:59 -07:00
Matthew Jackson d7bf829b01 fix: align the Linux fsync error with macOS, and clear three stale docs
Round 9 findings, triaged and verified against the pinned tree.

writeback_file: a bounded-fsync WorkerLost returned bare ErrorKind::Other
on Linux where macOS returns EIO. Round 8 fixed the Linux arm to return
Err at all — the right fix — but stopped short of matching the value, so
a consumer distinguishing timeout / halt / lost-worker had nothing to
branch on for the third case on one platform. Now EIO on both.

Three doc comments described the pre-fix behaviour, one of them for
longer than the bug existed:

  linux.rs durable_sync still said "all three fallbacks return Ok(())"
  mod.rs sync_all still said Linux silently swallows fsync failures and
    callers must not treat Ok(()) as a durability barrier
  mod.rs SequentialSink::finish repeated the same caveat

All three now say what the code does: a bounded-fsync failure is an Err
on every platform, so Ok(()) IS a durability barrier. A doc that
describes a fixed bug is worse than no doc — it tells a caller to write
a workaround for something that no longer exists.

au_assembly: discard_gap_before duplicated drop_marks_before's
mark-retirement body verbatim and added one statement. Mine, from
earlier today. It now calls it. Two copies of the same retirement loop
is exactly how the two call sites would drift back together.

clpi: ClpiStream's audio_format / audio_rate / video_format / video_rate
are decoded from untrusted on-disc bytes on every parse and read by
nothing. The identically-named fields consumed in disc/bluray.rs belong
to mpls::StreamEntry, not to this struct — checked, because an earlier
round wrongly called a live function dead. Deleted, along with the seven
test assertions that pinned them; the tests that pin pid, coding_type
and language remain. Also removed a section-header comment orphaned by
the get_extents deletion, describing a fixture that no longer exists.
2026-07-30 18:39:47 -07:00
Matthew Jackson cb5a918daa test: salvage the orphaned labels/disc triage, and extract build_labels
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to private-repo first, because a
worktree is not a backup and this one had already nearly been lost.

One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.

Two collisions resolved by hand:

A second mod pass_progress_tests, written independently against the
same survivors as the one committed in 337b57e. Kept mine — it covers
the distinct-counters case and the Progress blanket impl, which theirs
does not — but theirs had three clamp tests mine lacked: good_pct,
bad_pct and pending_pct also clamp an overshoot, and I had only tested
that for work_pct. Merged those in as one test and proved each of the
three clamps load-bearing by removing them individually.

An unused_parens warning in a new fixture.

Method note, recorded because it cost real time: git apply --3way
STAGES its result, so `git diff` reads empty and the tree looks
untouched. I nearly concluded the patch had silently failed. Worse, the
first attempt piped through `head -20`, so `echo exit=$?` reported
head's status rather than git's — the same mistake this audit has
already documented once. Check the real exit status, and check
--cached, not just the working tree.
2026-07-30 16:36:13 -07:00
Matthew Jackson 66a3506df3 test: pin five untrusted-input guards in the AACS 2.1 and CSS paths
Second pass over src/aacs and src/css. No production change; the only
non-test edits are two fixture bytes and one test rename.

Five latent panics on untrusted data, every guard correct and none
tested — so each was free to be deleted:

  variant.rs:224  a 0x04 record not a multiple of 5 indexes p_uv[0..4]
                  off a one-byte tail
  variant.rs:269  a 0x0c record shorter than the 0x04 slot count
                  slices past the cvalue table
  stevenson.rs:177  short sector read -> index 138 into a 129-byte slice
  stevenson.rs:208  a crib longer than the 1920-byte encrypted region
                    -> index 2058 into 2048
  stevenson.rs:272  a header periodic all the way to offset 0 ->
                    subtract with overflow

That last one is reachable from ORDINARY DVD data — constant or padding
bytes are periodic. Verified on HEAD: widening the guard to <= 0x80
passes all 64 css tests unmutated.

media_key_variant_from_kp had only a soft-correction test, so every
step past that early return was unexecuted. The new two-slot fixture
puts the covering slot at index 1, so the uvs[1 + 5*idx] and
cvalues[idx*16] strides stop multiplying by zero.

derive.rs:319 + -> - confirmed killable, as the first pass predicted:
p == 0 makes (p-1)..32 underflow. Every prior fixture used a uv whose
lowest set bit was 4, 10 or 11, so trailing_zeros() was never 0.

One fixture bug caught and fixed rather than papered over: a |= mutant
first SURVIVED because mk[14]'s 0x04 bit happened to be set, making OR
and XOR agree. The byte is now clear and an assert_eq! pins it, so the
fixture cannot drift back into agreeing with the mutation it exists to
catch.

walk_mkb_be24_high_byte_is_honored renamed to
walk_mkb_be24_middle_byte_is_honored. Its 0x00_0110 length exercises
the << 8 term only, which is why << 16 -> >> 16 survived it. The name
was the lie; both framings are worth having, and the comment now points
at the genuine high-byte test at 0x01_0004.

derive.rs 146:32 and 154:30 stay untested, now with a proof rather than
a judgement: bit_pos == -1 requires current_v_mask == 0xFFFF_FFFF, and
calc_v_mask can never return that — its loop condition holds at
!v_mask == 0, so it always shifts at least once. Both branches are
reachable only after the walk has gone non-convergent and is heading
for the bounded exit, where the return value is undefined. Termination
is already pinned.

Equivalents proven by observing green, including six more OR/XOR pairs
on provably disjoint bit fields, and the two KEY_CORRECTION_DATA sites
where the constant is the documented all-zero placeholder so x ^ 0 ==
x | 0. Those become killable only if a real per-licensee KCD is wired
in.

A partial confirmation sweep (138 of 415 mutants before the box
saturated) found 135 caught, one timeout that is itself a detection,
and exactly one survivor — the KEY_CORRECTION_DATA equivalent above.
2026-07-30 16:27:03 -07:00
Matthew Jackson 8e70f2a973 fix(udf): read the Metadata File Location from the partition map
read_filesystem hardcoded the Metadata File's File Entry at block 0 of
the physical partition — 'the metadata file ICB is at physical
partition lba 0'. UDF 2.50 2.2.10 records where it actually lives, as a
partition-relative Uint32 at offset 40 of the Metadata Partition Map.
That field is the only thing on the volume that says where the entry
is; block 0 is merely where authoring tools usually put it.

On a conformant volume that recorded it elsewhere, block 0 holds
something that is not a File Entry, metadata_start falls back to
partition_start, the File Set Descriptor read there carries the wrong
tag, and the volume is rejected as UdfNotFilesystem. Worse, a volume
with a decoy file set at block 0 — as a rewritten or dual-structure
volume can have — does not error at all: it mounts a different
filesystem and reports success.

Verified on HEAD: reverting the lookup reds four tests, e.g. the
metadata partition beginning at 2000 where the map records 33754069.

The recorded location is trusted only when the map's partition type
identifier reads '*UDF Metadata Partition'. A Virtual (2.2.8) or
Sparable (2.2.9) map is ALSO ECMA-167 3/10.7.3 Type 2 and records
unrelated fields at offset 40, so its bytes must never be read as a
location. Deleting that guard reds its own test.

Block 0 stays in the candidate chain, so a volume whose map is absent
or wrong but whose Metadata File does sit there keeps mounting exactly
as before. This is additive, not a behaviour swap.

Also 30 tests and ~55 more mutants across read_icb_extents,
read_file_limited, read_inline_data, the prefetch stubs, parse_dstring
and parse_udf_name. The metadata-partition branch — the branch EVERY
real BD-ROM takes — had no test at all; nothing in the crate built a
two-partition-map volume.

Closes the max_bytes gap flagged earlier: 259 > -> == and > -> < now
die on both the declared-size and the inline-ICB paths.

Equivalents proven by application, notably two guards that read as
protective but are unreachable: pm1_len is a single byte so
440 + pm1_len < 2048 always holds, and ad_offset + l_ad <= 2048 is
enforced upstream so off + ad_size never exceeds the block.

Bit 0 (Existence) stays unread, deliberately. ECMA-167 4/14.4.4 makes
it a display hint, not a statement that the file is absent, and UDF
2.50 2.3.4.2 carries it through as the DOS hidden attribute. For a
ripper the consequences are asymmetric: honouring it can silently drop
a real .m2ts from the title list, ignoring it costs an extra name in a
listing.

Known structural limit, not fixed: read_filesystem takes only the FIRST
extent of the Metadata File, so a fragmented metadata partition would
map every sector past that extent to the wrong place. metadata_start
being a single base LBA is what forbids the fix.
2026-07-30 16:09:35 -07:00
Matthew Jackson fc1f4db3a2 test: constrain the AACS key-map gap fill, the PSI walk, and MP4 field offsets
Third pass over src/mux/. 40 survivors killed, no production change.

resolve.rs — the deleted-statement cluster is now fully constrained.
All 14 deletable statements probed; 9 were already caught, 5 survived:

  c.sort_unstable() in fill_base_key_gaps. Every existing case handed
  it cuts already in LBA order, but IndividualSegment.tbl is a record
  list. Verified on HEAD: deleting the sort passes all 54 resolve
  tests. The mutant lays a base-key fill straight over a forensic
  segment.

  last_idx = idx (FMTS gap fill) and last_idx = hit (multi-CPS cache
  hit). An extent with nothing to sample must inherit its neighbour's
  CPS unit; the mutants fall back to the first unit's key. Exactly the
  shape this file's own comments name — wrong key, no error,
  lost_bytes == 0.

  Both check_halt()? polls in probe_fmts_index_keys. These cannot be
  killed by outcome, since a later poll returns Halted too. The tests
  count reads instead, which is what the don't-hammer-a-struggling-
  drive rule actually says: after a Stop the drive is asked for zero
  content sectors.

The four unresolved += 1 arms each got a test, and deleting each fails
exactly one — one-to-one, so no fixture passes for the wrong reason. A
control test pins that the baseline table resolves, so an expect_err
cannot succeed for an unrelated reason.

ts.rs::scan_streams was never entered. Six killed, two of which return
wrong answers that look right: reading the PAT/PMT CRC as a table entry
invents a stream on PID 546 out of CRC bytes, and dropping the
ES_info_length skip decodes a descriptor as an entry and loses the one
after it. Every existing PMT fixture declares ES_info_length = 0; a
real BD PMT carries a registration descriptor on essentially every
entry. Also ISO/IEC 13818-1 2.4.4.3 program_number == 0 is the network
PID, not a program.

mp4/read.rs — 13. Height read as the width beside it; channelcount;
the 4-byte base-128 descriptor varint (every existing esds fixture uses
a single byte); all three optional ES_Descriptor fields, whose loss is
silent (an AAC track just loses its CodecPrivate); first-vs-last media
edit, which is A/V desync of the difference; and the version-1 mvhd
timescale offset, emitted by any writer whose duration exceeds 32 bits.

dts.rs — 7, from a real cargo-mutants run over the file rather than
guesswork. Including a buffer that IS the syncword, which is the state
a sync split across PES packets lands in the moment its last byte
arrives.

Equivalents proven by application, not argued: the sample_encrypted_units
guard pair is mutually redundant by construction (total*p/9 < total for
p <= 8), so either alone is equivalent and both together are not; the
PMT section_len guard is dead code where its PAT twin panics; three of
the seven EXSS_HEADER_MIN_BYTES arithmetic mutants still sum to 10.
2026-07-30 16:05:37 -07:00
Matthew Jackson 41e0982ec9 fix(mux): a backstop discard is a discontinuity; a stream-start trim is not
drop_marks_before retired discontinuity marks alongside timing marks at
both of its call sites. At stream start that is right. At the
MAX_AU_BUFFER backstop it is not, and the two are now separate.

The backstop fires when 8 MiB accumulate with no AU start code in them
— corrupt or hostile input — and throws the run away. There IS a prior
AU in that case, and whatever emits next definitively does not continue
it: a decoder handed that picture resolves its references against
frames separated from it by megabytes of discarded data. Retiring the
flag meant the resync gate (resync.rs, driven from mux/disc.rs) never
armed, so the broken picture went out looking sound. Silent corruption
is the one class of loss this crate refuses to have.

At stream start the opposite holds. Bytes ahead of the first
access-unit delimiter are the tail of an AU that began before sync, and
there is no prior AU to be discontinuous from. Marking it would arm the
gate at the head of every title and drop its opening GOP. That risk is
why this was a decision rather than a fix, and splitting the call sites
is what avoids paying it.

Recorded as a sticky flag, not an offset mark. A mark placed at the new
base is retired moments later by the pre-sync trim that follows resync
— the gap has to outlive the bytes that caused it. I found that by
writing the test first and watching it fail with the mark approach.

The discard is a discontinuity whether or not the source signalled one,
and a signalled one on discarded bytes still reaches the AU that
follows; both directions are tested.

Note the first over-cap run is NOT a discard: the next AU's delimiter
is still at buf[0], so it force-flushes as an over-long access unit and
loses nothing. Only a run with no opener at all reaches the backstop.
The tests push twice for that reason — the single-push version passes
without the fix.

Swapping either call site for the other fails: reverting the backstop
reds the two gap tests, and arming the gate at stream start reds the
third.
2026-07-30 14:52:39 -07:00
Matthew Jackson 0469845c2d test: drive the AACS 2.1 variant chain to a Media Key, and pin AES-G3
163 of 322 surviving mutants across src/aacs and src/css. No production
line changed — every function read correct; the finding was always an
absent test.

Two structural holes, both verified against HEAD before landing.

variant.rs had no test that ever produced a Media Key. Every terminal
assertion in the module was an Err classification — NotVariantMkb,
SoftCorrectionRequired, OnlineChallengeRequired. So the entire 2.1
success path (VARIANTS lookup, VKD selection, Kpnew, the final unwrap,
the verify gate) was pinned by nothing, and that path produces the
Media Key that becomes the VUK that decrypts every byte of a 2.1 disc.
Built the first complete planted variant MKB: the VARIANTS entry is
chosen as Kvn ^ 1 so the real VKD sits behind a decoy at table index 1,
making the lookup load-bearing rather than incidentally correct. That
one fixture kills 23 operator mutants across three functions.

aesg3 — the subset-difference tree node function — was in the survivor
list as replaceable by [0; 16], meaning every device key in the crate
would derive the same Processing Key. It is caught today only as a side
effect of a negative test added after the mutation run; nothing asserted
the relation itself. Pinned now via the spec relation ([C] 3.2.2) using
the FORWARD primitive, with s0 transcribed independently rather than
read back from AESG3_SEED, so the test cannot agree with a mutated
constant.

Same shape in derive.rs: plant_mkb was one slot with zero descent, so
slot indexing was the identity permutation and the ancestor-descent
branch never ran — which is why 39 of recover_dk_position's mutants
survived. Added a 3-slot fixture keyed at index 2 and a four-level
descent fixture whose expected Processing Key is written out as an
explicit aesg3 chain rather than computed by calc_pk_from_dk; a fixture
built by the function under test moves with its own mutations.

Two latent panics on untrusted input now have tests: a 0x05 cvalue
table shorter than the 0x04 slot index, and a drive declaring more
payload than the 32772-byte response buffer holds.

23 equivalents claimed with reasoning, and confirmed empirically where
possible — all eight css/lfsr mutants were run and exactly the seven
disjoint-bit-lane ones survived.

Explicitly NOT claimed equivalent: derive.rs 146:32 and 154:30 are
reachable, but only on the non-convergent bounded-exit path where the
function's sole contract is termination. A test there would pin
defined-but-meaningless output.

Noted for the next pass: the pre-existing walk_mkb_be24_high_byte_is_honored
used total length 0x0110, whose high byte is zero — it exercised the
middle byte only, which is why << 16 -> >> 16 survived it. Left in
place; a real one was added at 0x01_0004.
2026-07-30 14:44:35 -07:00
Matthew Jackson d72084e077 refactor(clpi): delete the unused EP-map to sector-extent path
get_extents had no caller anywhere in the ecosystem, and neither did
anything feeding it. Removed: get_extents, resolved_ep_map, full_pts,
full_spn, parse_cpi, EpCoarse, EpFine, the ep_coarse/ep_fine fields,
the unused version field, and the 35 tests that exercised them.

This reverses the fix in f2ca0e4, deliberately. That commit corrected a
real truncation — out_time past the last EP entry resolved to
last_ep_spn + 1, dropping everything from the final I-frame to EOF —
and the fix stands in history as the record of what was wrong. But the
defect existed for eight audit rounds precisely because the code had no
caller: nothing exercised it, so nothing noticed. Keeping speculative
infrastructure alive on the strength of a comment saying it is reserved
for a path that does not exist is how that happens again.

ClipInfo keeps source_packet_count (read by disc/bluray.rs) and streams
(the CLPI/MPLS cross-validation in labels/clpi_audit.rs). 1,139 lines
out, gate green in debug and release.
2026-07-30 14:39:22 -07:00
Matthew Jackson f2ca0e48b0 fix(clpi): resolve out_time past the last EP entry to the end of the clip
ClipInfo::get_extents fell back to `last EP SPN + 1` whenever out_time
lay past the last entry-point. EP entries mark I-frames (BD-ROM Part 3,
CPI / EP map) and a clip's final GOP lies after the last one, so a
PlayItem covering a whole clip — whose OUT_time is the presentation end
— always lands in that arm. The extent then stopped one source packet
after the last I-frame.

Measured on a fixture with 200,000 source packets and the last EP at
SPN 131,072: sector_count came back 12,289 where covering the clip
needs 18,750. Everything from the last entry point to EOF is outside
the returned extent.

Scope, stated plainly: get_extents has NO callers anywhere in the
ecosystem today — it is #[allow(dead_code)] and documented as reserved
for the timestamp-range read path. Nothing ships this loss. It is
fixed now because a latent truncation in extent arithmetic is far
cheaper to correct before it has callers than after.

The SPN at-or-after an out-of-range out_time is the end of the clip,
source_packet_count, with .max(last + 1) so a disc that under-declares
its own packet count against its own EP map still yields a sane bound.

Also 174 mutants killed across clpi, mpls, ifo and ebml — the first
time any of these four files has been examined. And ebml's 8-byte VINT
back-patch was duplicated verbatim in end_master and end_master_buf
with its top four payload octets unreachable through either (they need
a 16 MiB..256 TiB buffer); extracted to fixed_width_vint8 and tested
across the full 56-bit payload, no behaviour change.

38 of ebml's 46 survivors are one equivalence cluster: every | in
write_size / read_id / read_size / read_uint_val ORs into disjoint bit
lanes, where ^ is the identical operation. Applied all 38 at once —
green — then spot-checked four individually.
2026-07-30 14:24:57 -07:00
Matthew Jackson b79015f822 test: pin FMTS read_plan unit indexing to an unaligned range start
Five surviving mutants in AacsKeyMap::read_plan, all in the arithmetic
that decides which half of a forensic segment this disc's key opens.

The existing coverage used a forensic range starting exactly on the
extent's first unit. Under that shape several wrong formulas agree with
the right one by arithmetic accident: (lba + range_start) / us and
(lba - range_start) * us both produce the correct kept set.

Real ranges are not shaped like that. A range start comes from a source
packet number — start_spn * 192 through clip_byte_to_lba in
mux/resolve.rs — and 192-byte packets bear no relation to the 3-sector
aligned unit, so range_start % 3 is whatever the disc says.

Getting the parity wrong does not crash. It reads and decrypts the
ALTERNATE variant's half: the units this key does not open decrypt to
garbage, the units it does open are skipped. AACS 2.1 forensic marking
is precisely what makes the two halves differ, so the failure is silent
— a full-length rip carrying the wrong variant.

Two fixtures are needed because no single one kills both: an unaligned
range start with the extent beginning on it inverts the halves under
the + form, and an extent offset one unit-remainder from the range
start inverts them under the * form.

Also pinned the short-tail guard.  is for a remnant
SMALLER than a unit — bytes with no following unit to desync. Widened
to <=, the last whole unit of every extent bypasses the phase gate, so
a forensic segment ending at an extent boundary contributes one
alternate-variant unit to the rip.
2026-07-30 14:18:39 -07:00
Matthew Jackson c7e43b2bc4 test: constrain the DiscStream loss surface and the empty-title guards
Second mutation pass over src/mux/. 26 survivors killed, no production
change. Verified on HEAD before landing: each mutation below passes all
1,237 mux tests unmutated-suite.

The priority item was the honest-loss-reporting surface. Both
DiscStream::errors and DiscStream::lost_bytes could return a constant
with nothing failing — a rip that lost sectors would report zero loss
to the caller. This project has already shipped one defect of that
shape (a total decryption failure reported as an empty title, exit 0).
Driven now through two short-read fills so both land on values that are
neither 0 nor 1 and differ from each other; no constant and no field
swap survives.

MkvStream::finish -> Ok(()) also survived. MkvMuxer::finish has the
zero-frame MkvInvalid guard and two tests cover it, but the Stream
wrapper above it could return Ok unconditionally and bypass the guard
entirely — the empty-title defence was one layer thinner than it looked.

au_assembly: pinned au_opener_from behaviourally to the normative byte
values for all four modes, with negative cases for codes that are
explicitly not openers (MPEG-2 slice 0x01..0xAF, user data 0xB2,
extension 0xB5, sequence end 0xB7 per 13818-2 Table 6-1; VC-1
0x0A/0x0B/0x0C; H.264 SPS/PPS/IDR-slice). au_assembly and codec/ hold
independent copies of these constants; they agree today, and comparing
constants would not catch logic drifting apart, so both sides are now
pinned to the spec instead of to each other.

demux_sink::sanitize: every filename component demux:// writes comes
from disc-controlled text, so the path-separator arm is a traversal
guard. Deleting it now fails, including an end-to-end case where
base = "../evil/Title" must produce exactly one file inside the
chosen directory.

stts_and_ctts_expand renamed to stts_expands_runs_to_per_sample_deltas_in_order
and given runs with distinct deltas AND distinct lengths. Its old name
claimed ctts coverage it never had, which is why the composition-time
chain went unconstrained for eight rounds; the doc comment now points
at the tests that do cover ctts.

Correction to the previous pass: codec/truehd.rs flush -> vec![] IS
equivalent. Applied it, full mux suite green. TrueHD buffers across PES
but parse emits every complete unit immediately, so a residual buffer
at EOF is a truncated access unit and is correctly discarded. The
vec![Default::default()] variants are genuinely different and are
killed.

Deliberately not constrained: mkv::set_opening_capture (diagnostics
behind a process-global tracing check, flaky under the parallel
runner), and the three stdio.rs header paths (StdioStream holds
concrete io::Stdin/Stdout and cannot be driven without a production
refactor to injectable Read/Write).
2026-07-30 14:13:33 -07:00
Matthew Jackson 5b81c33c9a test: constrain MP4 composition timing, MLP substream directory, and codec-private absence
Mutation testing over src/mux/. No production change — 49 survivors
killed, all proven red before green.

The MP4 composition-time chain was entirely unconstrained: VideoTiming::ctts,
build_ctts and parse_ctts could each return a constant and the suite
stayed green. Confirmed on HEAD: build_ctts -> vec![] passes all 1,220
mux tests. A demuxed B-frame title presenting in decode order would
have shipped.

The cause is a test whose name asserts coverage its body does not
deliver — stts_and_ctts_expand builds an stts box and never touches
ctts, and write_then_read_round_trip asserts sample sizes and keyframe
flags but not one PTS. Same shape as the set_speed forwarding finding,
different disguise.

mlp_num_substreams / mlp_substr_header_size: every TrueHD fixture in
the crate uses one substream and no extraword, so both could return a
constant and agree with all of them. These position mlp_parity_ok's
window over the AU header, so a constant mis-windows the parity check
on exactly the multi-substream AUs that carry 7.1 and Atmos.

CodecPrivate absent vs empty: mkv.rs writes Some(bytes) verbatim and
omits the element on None (RFC 9559 5.1.4.1.24), so a zero-length Some
emits a track header asserting the config IS empty. Four parsers could
return Some(vec![]) before any frame.

Also: mandatory ISO/IEC 14496-12 boxes (tkhd, vmhd, smhd, dinf, mdhd)
could each build empty; HEVC num_extra_slice_header_bits (H.265 7.3.2.3)
was never non-zero in any fixture, so the slice-type offset skip was
unexercised; chapter names from the disc go straight into
<ChapterString> and the & escape must run first; a stray 0x47 in a
payload must not latch a TS resync.

Documented as equivalent rather than killed: CodecParser::flush and the
three parser flush bodies that differ from the mutant only by a tracing
call, and DropTally::log_summary.
2026-07-30 13:39:02 -07:00
Matthew Jackson 3ccee19101 fix(udf): skip deleted File Identifier Descriptors
read_directory decoded file-characteristics bit 1 (Directory) and bit 3
(Parent) but never bit 2 (Deleted) — ECMA-167 4/14.4.4. It followed the
ICB of a descriptor naming a file that no longer exists.

4/14.4.3 permits a deleted FID's ICB field to specify an extent of
length zero, so it need not point at a File Entry at all. Following it
reads whatever descriptor occupies that metadata LBA:

- deleted DIRECTORY FID: recursion lands on the File Set Descriptor
  (tag 256), hits the not-a-File-Entry arm and returns Err. One stale
  descriptor in one directory fails enumeration of the entire volume.
- deleted FILE FID: read_file_size returns Ok(0) for a non-File-Entry
  tag, so a deleted name is reported as a real zero-byte file.

Both directions of the same shape at once: a recoverable condition
becoming a hard failure, and a non-existent entry becoming a plausible
success.

Bit 0 (Existence) is still unread. Skipping hidden files could hide
real content, so it is left alone deliberately rather than folded in.

Found by mutation testing: 48 of the 51 surviving mutants in
read_directory are killed by the 17 tests added here, covering the
short_ad decode byte by byte (ECMA-167 4/14.14.1), the extent-type
mask, the AD bounds guard at the exact sector end, the tag-261 File
Entry directory arm that no test reached at all, the multi-sector
read offset, the FID stride including L_IU, and the nesting cap.

Three survivors are genuine equivalents and are documented as such:
l_fi > 0 vs >= 0 (the emptiness guard below reaches the same state),
and the << 32 / | in the visited-set key (meta_start is constant
across a walk, and the two halves are disjoint).
2026-07-30 13:32:06 -07:00
Matthew Jackson 337b57e9dc test: constrain SectorSource speed forwarding and PassProgress percentages
Mutation testing left both unconstrained.

sector/mod.rs — set_speed on the Box<dyn> and &mut dyn forwarding impls
could be replaced with an empty body and nothing failed. This one hides
better than the read methods because the trait's own default body is
already a no-op, so a forwarder that swallowed the call is
indistinguishable from a source with no speed control. Consequence is a
silently absent value, not a wrong one: the recovery path lowers read
speed through a damaged region, and a swallowed call leaves the drive
at full speed while the caller believes it slowed down. Routed through
a generic S: SectorSource bound, since a direct call on a &mut dyn
receiver auto-derefs to the vtable and never enters the forwarding body.

progress.rs — 42 survivors. All four percentage accessors could return
a constant, read the wrong byte counter, or have their divide-by-zero
guard inverted. Added exact-value tests (25%, not 'some percentage'),
both sides of each guard, the overshoot clamp, and one test setting all
three disc counters to distinct values at once — without it, a swapped
field still passes every single-counter test.

The Progress blanket impl for closures could return a constant true.
That return value is the cancellation signal, so a constant-true body
makes every closure-based consumer uncancellable.

Each mutation applied, observed red, reverted.
2026-07-30 13:26:31 -07:00
Matthew Jackson 0e50b74211 docs: correct info invocation and read timeouts
TROUBLESHOOTING step 2 said `freemkv info`, which needs a source URL;
the drive route is `freemkv info disc://`.

architecture.md quoted 1.5 s / 30 s for the read timeouts. The
constants are READ_TIMEOUT_MS = 10_000 and READ_RECOVERY_TIMEOUT_MS =
60_000 (src/scsi/mod.rs:72,94).
2026-07-30 13:21:04 -07:00
Matthew Jackson c56b1d0b85 Constrain five behaviours that mutation testing showed nothing constrained
Fifteen surviving mutants killed, from the highest-risk class: functions a
mutant could replace wholesale with a constant while all 2,555 tests
passed. None of the code was wrong. In every case a test was absent, which
is why eight rounds of reading never found any of them.

The one that generalises is in sector/mod.rs. Its existing test READS as
covering `read_sectors` on the `&mut dyn SectorSource` forwarding impl —
it takes a `&mut dyn`, calls the method, checks the spy. But the receiver
auto-derefs and dispatches through the vtable straight to the spy, so the
forwarding body is never entered. An earlier round hit this exact trap on
`set_unit_base` and fixed it with a generic helper; the read path kept the
test that looked right. Verified by stubbing the forwarding impl to Ok(0):
the new test fails, the old one passes. That makes a tenth distinct shape
of bad test in this audit, and the mutation list is how to find the rest —
any forwarding-impl method in it has the same problem.

decrypt.rs's two existing gate tests assert only `dropped == 0`, which is
precisely what the `Ok(0)` mutant returns; one asserts nothing else at all.
A wrapper that decrypts nothing therefore looked correct while the caller
muxed scrambled MPEG. Now pinned by descrambling a real CSS sector and
comparing against the plaintext it was built from — not against a
re-derived descramble, which would only assert the code agrees with
itself.

css/mod.rs's `is_scrambled_uncracked` turns out to have no production
callers at all; the enum is matched directly. Its three tests all assert
only the true direction, which is exactly why the `-> true` mutant
survived. It is public API, so a consumer routing on it would, under that
mutant, refuse to rip every clear DVD.

aacs/inf.rs's MKB drive read had no test whatsoever. Now pinned
byte-for-byte across multi-pack concatenation, the single-pack case, a
genuinely empty response, and error propagation — an unreadable MKB must
surface as an error, not as an empty one.

aacs/derive.rs's nine mutants are killed with planted MKBs built by
inverting the AACS relations, so no real key material is involved. The
assertions land on the derived Media Key rather than the intermediate
positions: a recovered position that does not actually walk to the planted
key is no better than None. A fixture-guard test asserts the planted MKB
parses, since an unparseable one would make every `-> None` body look
right.

2570 lib tests, debug and release.
2026-07-30 12:41:28 -07:00
Matthew Jackson 20e3aa26ca Test that the Media Key verifier actually rejects a wrong key
`km_verifies` is the gate deciding whether a candidate Media Key belongs
to the disc. The MK-pool brute force in resolve.rs runs every candidate
through it, so a version that said yes to everything would accept
whichever candidate it tried first and the rip would continue with a wrong
Media Key — wrong VUK, wrong title keys, garbage plaintext, and no error
raised anywhere.

Nothing tested it. Whole-crate mutation testing reported
`replace km_verifies -> bool with true` as SURVIVING: the body could be
replaced with `true` and all 2,556 tests still passed. A verification
routine whose verification was itself unverified, in the most
safety-critical function in the crate.

The implementation is correct — it matches the AACS relation
`AES-D(km, mk_dv)[0..8] == 01 23 45 67 89 AB CD EF`. Only the defence was
missing.

No real key material is required to test it. That relation means a valid
record for any chosen km is just `AES-E(km, <the constant> || anything)`,
so the fixture is self-contained. The test asserts three things: the key
the record was built for verifies; a key differing by ONE BIT does not,
which is the assertion that kills the mutant and is a near-miss rather
than a random key; and an MKB carrying no verify record does not default
to yes, because unverifiable and verified are different answers.

Confirmed by reintroducing the exact reported mutation and watching this
test fail.

This is the first defect found by mutation testing rather than by reading.
Eight audit rounds and a security lens that read this file in full all
missed it, because it is not a wrong line — it is an absent test, and only
an instrument that asks "would anything notice if this were broken?" can
see that.
2026-07-30 12:14:14 -07:00
Matthew Jackson 7e4c22e36c Bound the BD-J label parsers, and stop a crafted disc hanging the scan
Ten defects in code no previous round had ever scoped. `src/labels/`
identifies a disc's studio by parsing jar archives and JVM class files off
untrusted media, so every byte here is attacker-controllable — and 813 of
its lines were executed by no test at all.

The worst is a non-terminating loop. A fallback stream-number scan
advanced with `saturating_add`, and the comment says why: a crafted XML
"must not overflow (panic in debug, wrap-to-0 in release)". Once the
counter pins at u16::MAX and that number is taken, the loop cannot exit.
So a fix for an overflow panic produced an unbounded hang, which is
strictly worse — a panic is observable and catchable, and catch_unwind
cannot interrupt a live loop. Reachable from about 8 MB of XML.

Where the same overflow appears in the deluxe decoder the fix is
checked_add and stop, NOT saturation — twice wrong there, because
saturating would peg every stream past the ceiling at one number and
apply_labels binds on (type, number), silently mislabelling tracks. A
correctness bug wearing the costume of success.

Round 7 capped the ldc-string retention per class; nothing capped the
aggregate, so a 64 MiB jar held that budget for every class at once. Same
defect one level up, which is the shape that keeps recurring in this
directory. Four other amplifications are bounded the same way, each with a
stated headroom and a paired test proving real media passes untouched —
the tightest is 5x on a label length, the loosest 2000x on the stream
numbering space, against BD's 32-per-type STN_table limit.

Two are not caps at all: a quadratic membership scan became a set, and an
attacker-derived length added to a cursor without saturation now cannot
wrap. Nothing is excluded by either.

A `#[cfg(test)]` hand-copy of a shipping parser was the ninth bad test
this audit has found, and the first proven by mutation rather than
inspection: deleting the guard from the REAL function left all 26 tests
green, including the one named for that guard. Pointed at the real
function, the same mutation fails.

Separately, all three failure arms of the bounded fsync returned Ok(()) on
both macOS and Linux, so sync_all reported success for a durability
barrier that never ran. Only macOS was in scope; the Linux twin is fixed
here too, because a platform disagreeing with its sibling about whether a
failed sync is an error is the class that already produced an over-length
SCSI CDB macOS rejected and the other two truncated. Note the behaviour
change: a mux whose final sync times out on a wedged mount now fails
rather than exiting 0.

Three of the caps are proven by wall-clock deadline rather than an
operation count, with 18-80x margin on the passing side. On a heavily
oversubscribed machine those could flake.
2026-07-30 11:30:24 -07:00
Matthew Jackson d628e24eaa Key each FMTS extent from its own CPS unit, and stop a bad ICB tag reading
as an empty directory

On an AACS 2.1 disc the non-forensic gap fill hardcoded pool slot 0 as
"the" base Unit Key, so every content LBA outside a forensic segment was
keyed with CPS unit 1's key even on a disc carrying several CPS units. It
does not fail loudly — it produces garbage plaintext. The gap fill now
resolves each extent's own base key from its ciphertext, sharing the
sampling and slot-picking the multi-CPS path already had rather than
adding a second copy, and memoised in the existing per-disc cache. A disc
with one base key still short-circuits with zero extra reads, which the
existing probe-cost test pins. Forcing the slot back to a constant fails
four tests, so the choice is load-bearing rather than incidental.

A directory ICB whose descriptor tag is neither File Entry nor Extended
File Entry (ECMA-167 4/14.9, 4/14.17) was turned into a successfully-read
EMPTY directory, indistinguishable from a genuinely empty one, while the
same tag on a file ICB was already a hard error. Fifth instance in this
audit of a failure converted into a plausible success value, and the
second in this very function — round 5 fixed a read error becoming a file
size of zero here.

An unrecorded extent (ECMA-167 4/14.14.1.1: allocated but not recorded,
logically zeros that still occupy file space) was dropped entirely rather
than contributing its length, so every later extent landed at the wrong
file offset. Silent corruption, not an error. Extents now carry a recorded
flag and the hole emits zeros without touching the media.

The Volume Descriptor Sequence was swept at hardcoded sectors 32..64 while
the anchor's own Main VDS Extent pointer was parsed into a comment and
ignored; ECMA-167 3/10.2.1 defines that extent by the field, not by
position, so a conformant volume placing it elsewhere failed to mount.

drive_status decoded byte 5 as Media Status without checking the event
header's NEA bit or notification class (MMC-6 §6.7), so a reply carrying
no media event descriptor decoded as "no disc". The drive is untrusted
input here, and this is the works-on-my-drive class.

Two more tests were found asserting the defects they sit next to — one
requiring unrecorded extents to be dropped, one that four drive fixtures
built non-conformant replies the corrected decoder rightly rejects. Both
rewritten. Combined with the DTS one in the previous commit that makes
three tests this round that locked a bug in as intended behaviour, which
is a different and worse failure than the tautological tests found so far:
a tautology fails to catch a regression, these actively defend the defect.

Not fixed, adjacent: extract_one_file streams extents sequentially and
will now READ an unrecorded extent's sectors rather than writing
guaranteed zeros. Offsets are right, and pressed media reads as zeros
there, but it is not zero-guaranteed the way read_file now is; that needs
a recorded flag through PlannedFile.
2026-07-30 11:15:52 -07:00
Matthew Jackson 9bea6ab52b Feed the CSS crack the canonical extent order, and stop Resolution faking 1080p
Seven defects in the code the test suite executes least — 913 lines of
disc/mod.rs alone are run by no test at all, which is why this round scoped
from coverage rather than from what previous rounds said they had read.

Disc::scan_image kept its own copy of the crack's extent ordering and fed
crack_key_outcome largest-cell-first. That is the fifth instance in this
audit of a local reimplementation drifting from the canonical one, and the
cost here is a key that does not descramble the feature: picking by sector
count bypasses the capacity gate and can select a different VTS entirely.
The copy is gone — which title comes from the canonical order the scan
already applied, and the extents are handed over in playback order,
exactly as decrypt_keys_for_title does. Its doc records why the duplicate
existed so it cannot grow back.

Resolution::pixels returned 1920x1080 for Unknown. That is the FOURTH
instance of one trap and the other three were in this same file, two of
them fixed hours earlier — without sweeping for siblings, which is the
whole reason this one survived. It now returns (0, 0), and the sweep was
done properly this time: every remaining Unknown arm across the crate is
honest, and the two ColorSpace sites that look like fabrication are
emitting H.273 code point 2, which is the spec's own "unspecified". Two
callers carried local Unknown-to-zero workarounds — precisely the cost of
making callers responsible for a lie — and one is now redundant.

BD-ROM Part 3 code 0xA2 is the lossy secondary DTS stream, not lossless
Master Audio. A test asserted the wrong mapping as intended behaviour, so
correcting the code failed it; the test is deleted with a note pointing at
its replacement. That is a NEW failure mode for this audit: not a test
that cannot fail, but one that locks the defect in. There is no
DtsExpress variant to map to, so it takes the lossy DTS-HD member and the
approximation is documented.

Also: DiscSession::identify could panic through drive_mut once the public
API allows an absent drive — two siblings were converted in an earlier
round and this one was missed; an extent end that added without saturating
where the rest of the crate saturates; a diag reason string restating the
comparator's sort keys and drifting from them, now derived from them; and
a short read that advanced the offset by the full request, silently
skipping the gap. That last one existed twice, in two reads with the same
shape, now merged so they cannot drift apart.

The short-read policy is a judgement call I could not derive from a spec:
no skip_errors is a hard error, with skip_errors zero-fills and charges
the loss. It deliberately does not retry mid-unit, because resuming inside
an AACS aligned unit would trade a silent gap for a silent decrypt
desync — the worse of the two.
2026-07-30 11:12:55 -07:00
Matthew Jackson 0a88b61421 Stop AudioChannels and SampleRate fabricating a value for Unknown
Three copies of the same two mappings existed. The canonical accessors
returned 6 channels and 48000 Hz for Unknown; a third copy in diag.rs
returned 0. The honest one was the copy.

A plausible wrong answer is worse than an obvious one. Six channels at
48 kHz is indistinguishable from a real 5.1 track, so every caller became
responsible for remembering to check the variant first — and this crate
walked into exactly that: the json:// sink reported a confident 5.1 for
audio whose neighbouring fields said "unknown". That was fixed at the call
site earlier in this audit; this fixes it at the source.

The accessors now return 0, which is what both in-crate call sites already
coerced Unknown to by hand, so their guards are gone and the behaviour is
unchanged. Zero is also obviously wrong if it ever reaches output, where
six is not.

The diag.rs duplicates are deleted rather than corrected — a fourth copy
would have drifted too. Their only caller was a trace line in the same
file, now on the canonical accessors. Their tests moved across and gained
the Unknown case, which is the point: restoring either fabricated value
fails both.

Found by the round-7 correctness agent while fixing the json:// sink; it
flagged the third copy as out of its scope rather than touching it.
2026-07-30 10:09:01 -07:00
Matthew Jackson ef0cf028e4 Add a seeded robustness harness for the untrusted-input parsers
Five parsers that take bytes straight off a disc are now swept with
generated input asserting one property: they return Ok or Err and never
panic. That is this crate's own hard rule, and the class seven rounds of
reading is worst at.

Written in-crate rather than with cargo-fuzz, which needs a nightly
toolchain this project does not use, and without proptest or arbitrary,
because one dev-dependency is a deliberate posture and the parsers take
plain byte slices. What is given up is coverage-guided mutation, which is
the real loss. What is gained is determinism: the same seed replays the
same cases anywhere, so a CI failure reproduces locally verbatim.

Three generators, and the second is the one that matters. Pure random
bytes die at the magic check and exercise the entry guards only; prefixing
valid magic is what reaches the parser body; mutating a mostly-zero record
is what reaches the offset and count arithmetic a hostile image would lie
about.

That claim is MEASURED, not asserted. A harness whose cases all bounce off
the entry guards is the fuzzing equivalent of a test that cannot fail, so
one test counts how many generated cases parse to completion: 15,606 of
60,000, about 26%. If a future change to a guard drops that to zero, the
test fails rather than continuing to report a meaningless pass. Two further
tests pin that the three generators produce different bytes and that a seed
replays identically.

1.2M cases across all five targets found nothing. On this evidence that is
a real negative rather than an empty one.

The first version of this file was itself broken in the way this audit
keeps finding: its two meta-tests set FREEMKV_HARNESS_CASES and raced,
because the test harness runs them in parallel and env mutation is unsound
there. The budget is a parameter now, and the environment is read once at
the call site.

Two crate-internal parsers widened from private to pub(crate) so the
harness can reach them. No public API change.
2026-07-30 09:45:56 -07:00
Matthew Jackson 766fb68e04 Make five tests capable of failing, and stop the presence probe unmounting the disc
The worst of the five was a regression suite that never touched the code
it guarded: nine batch-count tests called `safe_batch_count` and
`buggy_batch_count`, both defined in the test file itself. The u16
truncation they exist to prevent could be reintroduced in
sector/prefetched.rs with every one of them green. They now drive the real
producer through the public API, and reinstating the truncation fails five
of the nine. Worth recording that the symptom has changed since the
original fix: the unit-alignment clamp below floors a zero batch at three
sectors, so the bug is now a twenty-fold throughput cliff rather than the
stall it once was.

The MP4 reserve test's only numeric case was dominated by the floor and
the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now
has a case where the per-sample term dominates. The zero-count guard in
FileSectorSource was likewise unfalsifiable — seek-past-EOF and a
zero-length read both succeed — so the test now observes the file cursor.
The AACS media-key ambiguity guard had no test at all; the pool scan is
extracted so the verifier can be injected, because a genuine two-key
collision needs one ciphertext decrypting under two AES-128 keys to
plaintexts sharing a 64-bit magic, which is a 2^64 search and not a
fixture.

macOS implemented the documented cheap, side-effect-free presence probe by
building a full exclusive transport — which force-unmounts the disc. Linux
and Windows issue one TEST UNIT READY with no unmount; macOS was the
outlier. It now walks the IOKit registry for the media object instead.

The C shim's registry reads assumed CoreFoundation types the registry does
not guarantee, so a driver publishing a CFNumber where a CFString was
expected aborted the process from inside public API. Types are checked and
a wrong type treated as absent. The unbounded waitpid on the unmount child
is now a polled deadline, and the last-resort match gained the NULL check
its two siblings already had.

The empty-CDB guard existed only on Linux while a shared helper's comment
claimed all three backends had it. Moved into the helper, so the comment
is now true and macOS and Windows are covered.

One finding was REJECTED with evidence rather than fixed. The TrueHD
buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be
unreachable by any input: the parser only retains data when the buffer is
shorter than the declared AU, and that declaration is twelve bits, so the
worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over
all 65536 AU headers confirmed it. The fixture now sits at the reachable
ceiling and asserts that instead. The cap itself is left in place as
defence, unreachable by construction, matching how the AC-3 resync guard
was handled earlier in this audit.

Two behaviour changes worth naming: Linux's empty-CDB error becomes
InvalidCdbLength rather than a transport failure, and an unknown device
now reports absent media rather than a not-found error, because the
registry cannot tell an empty drive from a missing one. The latter is a
conflation of the kind this audit has fixed three times; it is recorded
for the next round rather than left silent.
2026-07-30 09:26:56 -07:00
Matthew Jackson 45fbaa8ee2 Stop the live rip path muxing Blu-ray 3D differently from the ISO path
Five defects, four of them the same shape: a local reimplementation of
logic the crate already had, which had drifted from it. Each is now fixed
by calling the canonical version rather than by patching the copy.

DiscStream::new — the live disc:// path — built every parser through the
plain codec lookup and never asked whether a video stream was an MVC
dependent view, though resolve::build_demux_state does. The same 3D disc
therefore muxed correctly from an ISO and incorrectly ripped live. The
open-coded loop is gone; both paths now call build_demux_state.

collect_psi_section reimplemented the continuity-counter gap test and
disagreed with process_packet in the same file: it tolerated neither a
duplicate packet nor an adaptation-field-only packet, which per ISO/IEC
13818-1 §2.4.3.3 does not increment the counter. A spec-legal PMT
continuation was read as desync and the title's stream list came back
empty. Both callers now share one `cc_is_gap`, and a duplicate packet's
payload is no longer appended twice — doing so would have corrupted the
section the check exists to protect.

The json:// sink called the channel-count and sample-rate accessors
unconditionally, and both fabricate a concrete value for Unknown, so it
reported a confident 5.1 at 48 kHz for audio whose format was unknown
while its own neighbouring string fields said "unknown". The keys are now
omitted, matching mkv.rs. This matters more than it did: a sample-rate
ladder fixed earlier in this audit means Unknown now reaches consumers
that used to receive a wrong-but-concrete value.

For an audio:// or sub:// sink the reference video track's output is
filtered out, so its first PTS was never recorded and every delay was
computed against zero — baking a wrong DELAY into the filename. The
reference is now recorded whenever a frame is on the reference track,
independent of whether that track has an output, so a normal title gets a
correct delay; where no reference is ever observed the tag is omitted
rather than guessed.

A third copy of the channel/sample-rate mapping exists in src/diag.rs and
was left alone as outside the confirmed set. It is the same drift shape
and is recorded for the next round.
2026-07-30 09:18:33 -07:00
Matthew Jackson 40ac369fb1 Bound three allocations an untrusted disc can drive without limit
The Program Stream demuxer appended every fed byte and enforced its 4 MiB
cap only inside a branch reached once a start code had been found. Input
containing no start code anywhere therefore hit no cap at all, and since a
whole title is fed through this demuxer, a zero-filled or ciphertext VOB
extent buffered the entire title — up to ~90 GB. When the buffer holds no
start code, only a two-byte `00 00` prefix can begin a PS unit on the next
feed, so that is kept and the rest dropped. The bound is exact rather than
a heuristic: a start code can straddle a feed boundary by at most its
first two bytes, so no real byte is discarded, and a test feeding
`FF FF 00 00` then `01 E0 ...` pins that.

The existing test named for this case fed a real start code first, so the
cap it exercised was the in-PES one. Renamed to say what it covers.

The BD-J label path had a different shape to anything found so far: the
cap is on the COMPRESSED size of a disc file while the allocation scales
with the decompressed size. A `.class` gated only by a path prefix
inflates to the 64 MiB ceiling, yielding ~33M retained strings from `ldc`
operands or ~67M pushes onto a symbolic stack whose depth was unbounded
despite the Code attribute's own `max_stack` being parsed and then
ignored. Bounded both, the stack by `max_stack` itself (JVMS §4.7.3).

The VMG TT_SRPT title count is an untrusted u16 with no de-duplication,
so ~800 KB of crafted IFO re-parsed one PGC 65535 times. Capped at 99, the
DVD-Video maximum, so no conformant disc is clipped.

Every cap carries stated headroom against real media, and each has a test
locking that real media still passes.

I rewrote three of the new assertions before landing them. They compared
the result against the very constant under test — `total <= MAX_TT_SRPT_TITLES`
— which passes vacuously the moment someone raises the constant, the most
likely future regression and the seventh instance of this tautology shape
in this audit. They now assert literals derived from the spec.

The TT_SRPT fixture also had to change: with 65535 identical entries the
de-duplication collapsed them on its own and the cap was never what
bounded the result, so the test passed with the cap removed entirely.
Distinct entries defeat dedup and leave the cap as the only guard;
de-duplication now has its own fixture. Verified by raising each of the
three constants and confirming all three tests fail.
2026-07-30 09:17:16 -07:00
Matthew Jackson f5e65cf2da Remove emulation-prevention bytes before reading the H.264 slice header
The bytes after a NAL header are EBSP, not RBSP: ISO/IEC 14496-10 §7.4.1
has the encoder insert 0x03 after any 0x00 0x00, and §7.3.1 removes it
before parsing. The measured-picture-type parse read the raw NAL instead,
on the stated reasoning that slice_type is too early for an escape to
intervene.

That holds only up to a point. first_mb_in_slice is ue(v), so a value of
65535 or more needs sixteen leading zero bits and opens the payload with
0x00 0x00, which an encoder must then escape. A UHD frame is ~32,400
macroblocks, so a conforming Blu-ray never reaches it — but 8K does, and
the disc is untrusted input. Such a stream decoded slice_type against a
byte the encoder had inserted and reported the wrong picture type: a wrong
result rather than an error, which is the class this lens exists for.

The prefix is un-escaped into a 16-octet buffer rather than the whole NAL:
the two ue(v) fields are at most 32 bits each, so nothing longer can be
needed, and it keeps a per-frame allocation proportional to the frame off
the path.

The test pins both directions. It asserts the un-escaped prefix decodes to
first_mb_in_slice = 65535 and slice_type = 2, AND that the raw EBSP does
NOT — without that second assertion the test would pass whether or not the
fix were present, which is the failure mode this audit has now found four
times. The bit string was derived independently rather than by hand: my
first attempt at the fixture was wrong by one nibble and the test caught
it.

Also covers the cases that must NOT be unescaped: a 0x03 not preceded by
00 00 is ordinary payload, and 00 00 03 03 keeps its second 0x03 because
the escape resets the zero run.
2026-07-30 08:49:51 -07:00
Matthew Jackson ade13a35de Log the OS error when a SCSI command fails on Windows and macOS
Both backends discarded the platform's own error code on the execute hot
path — the one every READ(10) of a rip goes through — and collapsed every
cause to the same status-0xFF transport failure.

On Windows, open() and reset() in the same file both capture the Win32
error; execute() did not. That left ERROR_INVALID_PARAMETER (a struct
layout regression, the exact class this file's SDK-layout tests exist to
catch), ERROR_ACCESS_DENIED and ERROR_GEN_FAILURE (a genuinely wedged
drive) indistinguishable, with nothing in the log to tell a code bug from
a hardware one.

On macOS the same, and worse: the file had no tracing calls at all, where
the Linux and Windows backends both log their execute failures. Its open()
carefully decodes the shim's sentinel into typed variants instead of
flattening them, but execute() threw the IOKit return away — so another
process taking exclusive access mid-rip and a real hardware wedge produced
identical, empty diagnostics.

Logged rather than added to the error type: the typed variant is public
API, and the recovery classification is deliberately the same for all of
these. What was missing is the breadcrumb, not the distinction.

Two further findings from the same sweep were rejected. Windows reset()
always returning Ok(()) and macOS ignoring timeout_ms are both already
documented in the code as deliberate, and the reporter flagged them for
completeness rather than as defects.

Neither fix has a test: reaching either branch needs a failing ioctl or a
failing IOKit call, and both files are compiled only on their own platform.
2026-07-30 08:42:02 -07:00
Matthew Jackson 816a41a01f Turn a release-only slice panic into an error, and stop calling 32 kHz 48 kHz
Four round-6 findings.

FileSectorSource::read_sectors guarded its output buffer with a
debug_assert, which is compiled out in release — so an undersized buffer
panicked with 'range end index out of range' instead of returning an
error, out of a public SectorSource impl where the length is caller input.
Drive::read_fua already carries this exact guard, with a comment recording
the same panic being fixed there, and PrefetchedSectorSource has a
regression test for the same case; this impl had been given neither. The
new test is red in release for precisely the predicted reason: 'range end
index 8192 out of range for slice of length 2049'.

parse_track's sample-rate ladder ended in an unconditional S48, so any
SamplingFrequency below 44100 was recorded as 48 kHz. A 32000 Hz AC-3 or
DTS track is legal and common in broadcast-sourced content, and the wrong
rate then propagated into the reconstructed AudioStream. Anything below
the lowest mapped rate is now Unknown, which is what the crate's canonical
SampleRate::from_hz already returned — the ladder disagreed with it. The
ladder itself stays, because the MKV element is a float and wants
tolerance rather than exact equality.

shim_open_exclusive used the mach port from IOMainPort without checking
the return; on failure the port is left untouched and every IOKit call
below ran against an uninitialised value. shim_list_drives in the same
file does check it.

build.rs treated cc and ar as successful if the process merely SPAWNED, so
a genuine compile error in the macOS C shim produced no object file and
surfaced later as an unexplained link failure against a missing symbol.
The shim is macOS-only and is neither linted nor compiled on the other two
platforms, so a mistake in it has exactly one chance to be noticed.

The last two have no test: one needs IOMainPort to fail, the other needs a
deliberately broken C shim, and neither is reachable from the test
harness. Both mirror a correct sibling in the same file, which is the
evidence available.
2026-07-29 22:56:33 -07:00
Matthew Jackson ae33563587 Fix five clippy errors that only appear on the target CI lints
CI's clippy job runs on ubuntu-latest, so cfg(target_os = "linux") code is
what the gate actually compiles — and none of it is built by clippy on a
Mac. Five `-D warnings` errors were sitting in drive/linux.rs,
scsi/linux.rs, io/writeback/linux.rs and the Linux arm of drive/mod.rs:
four collapsible let-chains and one manual `% n == 0`. CI was red on the
lint job while the local gate reported all green.

Collapsed into let-chains, which the declared toolchain supports, and
folded drive/mod.rs's length precondition into its chain so the body no
longer needs a nested block.

Found because an agent working on the SCSI backends reported the lints in
passing while checking that a Windows-only file compiled. Worth noting how
it stayed hidden: every one of these files is cfg-gated to a platform this
machine is not, so no amount of local gating would have surfaced them. The
companion change to the precommit script closes that hole.
2026-07-29 22:37:47 -07:00
Matthew Jackson 9a88efcc84 Reject an over-length CDB on every transport, splice H.264 param sets in place
Two fixes from round 5.

The Linux and Windows backends truncated a CDB longer than 16 bytes
(`cdb.len().min(16)`) where macOS returned InvalidCdbLength. Under SPC-4 a
command's length is fixed by its opcode group code, so a shortened CDB is
not a shorter form of the same command — it is a DIFFERENT command, and
the drive executes it and answers GOOD with data for a request nobody
made. A silently wrong result on the layer everything else sits on.

Rather than mirror the guard a third time it now lives in scsi::mod as
checked_cdb_len, with all three backends routed through it, so it cannot
drift per platform again. That also makes it testable everywhere: each
platform module is cfg-gated to its own host, so a guard inlined into
linux.rs and windows.rs would have had no test coverage on any single
machine. The shared helper is the only place the behaviour can be
asserted on every platform's CI.

The two existing macOS tests were tautological — they replicated the
guard's logic inline instead of calling it, so they would have passed with
the guard deleted. They now call the real helper.

Separately, the H.264 keyframe parameter-set re-assert grew a
few-hundred-byte prefix buffer to the full access-unit size, copied the
whole frame into it, and dropped the presized buffer: one extra
whole-frame allocation and copy per keyframe. A UHD title is ~200,000
frames of 150-400 KB with a keyframe every second or two, so that is
thousands of avoidable multi-hundred-KB copies per title, each large
enough to go through mmap. It now splices into the reserved headroom in
place. This mirrors the identical fix already made in hevc.rs, which the
H.264 path had drifted from.

Byte-for-byte equivalence is pinned by a test whose expected literals
were captured from the pre-change implementation, and which I confirmed
still passes when the old build-and-copy code is restored. The
no-reallocation claim is measured rather than argued: a counter over 30
bare keyframes, which reports 30 of 30 against the old path and 0 with
the splice.

The reallocation test initially passed even with PARAM_REASSERT_HEADROOM
set to zero, because a small parameter set fits in the presize's
incidental slack — it proved the fixture did not reallocate, not that the
headroom prevented it. Its SPS is now large enough that the constant is
load-bearing, so zeroing it fails the test.

Not verified: no runtime behaviour on Linux or Windows: no drive, no
ioctl. Both files were confirmed to compile for their own targets.
2026-07-29 22:34:22 -07:00
Matthew Jackson 719a9077f3 Stop reporting an uncrackable CSS disc as N empty titles
Same shape as the mkv:// conflation fixed earlier in this round, found by
looking for it deliberately. E7023 carried two conditions with opposite
correct responses: one title on a multi-VTS DVD failing its own re-crack,
where skipping it and finishing the rest is right, and the main feature's
crack failing outright, which is disc-wide and dooms every title
identically. Because both raised the same code and that code is in
is_skippable_title_stub, an uncrackable disc walked all N titles printing
"title skipped, it was empty" and exited 0.

The disc-wide condition gets E7027 CssNoDiscKey, mirroring the AACS-side
E7022 NoDiscKey it is the analogue of, and joins is_disc_level_no_key.
The per-title raise keeps E7023 and stays skippable. Because the engine's
classifier already tests is_disc_level_no_key before the skippable
branch, this reaches the right outcome downstream with no change there:
such a disc now stops on the first title and reports no-key instead of
returning success with nothing written.

Disc::css_error deliberately still stores CssKeyMissing — autorip matches
that variant on the field to pick the CSS rather than AACS message, and
what consumers classify on is the gate's returned verdict, which is the
only thing that changed.

Two neighbouring CSS raises were examined and deliberately left alone:
the no-key branch in the same function is genuinely unreachable via
ensure_decryptable and documented as defensive, and resolve_dvd_title_key
is per-title on both of its call paths.

Verified by removing the new code from is_disc_level_no_key, which fails
both new tests; each pins both directions so neither can silently flip.
Not proven end to end against a real uncrackable disc — none available.
2026-07-29 22:29:51 -07:00
Matthew Jackson de97c5a607 Round 5: reject a degenerate fixed lace, bound the pending buffer by bytes
Five fixes. Three are real defects with regression tests; two are bounds
that were expressible but not expressed.

A fixed-size lace (RFC 9559 §10.3.4) whose body is empty declared n
frames and carried none. The divisibility check passed, because 0 % n is
0, and `chunks` yields nothing on an empty slice whatever width it is
given — so the clamp that existed to avoid chunks(0) returned zero frames
where the Lacing Head said n. The whole lace vanished with no error
raised and the caller saw a clean short block. A zero-size frame cannot
be a valid frame, so it is now malformed.

A disc read failure while fetching a directory entry's ICB became a file
size of zero rather than an error. Zero is indistinguishable from a
genuinely empty file, so an unreadable ICB on a damaged disc silently
changed which titles a caller saw as present — read_directory already
fails hard on its entry-budget guard, so propagating is also what the
surrounding code does. read_file_size still returns Ok(0) for an ICB
whose tag is neither File Entry nor Extended File Entry, which is a real
zero and not a failure.

The pending-frame buffer was capped at 4096 frames, which does not bound
memory: frames are arbitrarily large and a UHD video frame runs to a few
hundred KB, so the existing cap permitted over a gigabyte. Now bounded by
bytes as well, at 64 MiB.

round_up_grain overflowed for inputs within one grain of u64::MAX —
div_ceil then multiply — and the wrapped product is small, turning the
largest possible estimate into a negligible reserve. It saturates, and
the reserve is clamped to what a `free` box's 32-bit size field can
actually hold, since writing a larger one truncated the size and left
mdat beyond a box claiming to be far shorter. No real title comes close;
a 90 GB UHD title estimates a few MiB.

The AC-3 resync guard now advances the PTS cadence like both of its
sibling branches, so the three paths out of that block cannot disagree.
This one is defensive and has NO test: reaching it needs input that both
parses frames and leaves a megabyte of residue, and the parser's own
carry rules drop pre-sync junk and cap a partial frame at 8192 bytes, so
no such input was found. Stated here rather than covered by a test that
would pass either way.

Two findings from this round were rejected on inspection. A reported
panic in the .mpls suffix check does not exist: the `.get(..)` on the
line above returns None off a char boundary and `filter` never runs its
closure, so the byte index is unreachable. A test written for it passed
against the unfixed code, which is what surfaced the error.
2026-07-29 22:28:43 -07:00
Matthew Jackson 424b0fa32e Stop reporting a corrupt mkv:// source as an empty title
E6008 meant two unrelated things: "this title produced no muxable
frames", which is a benign stub worth skipping, and "the source file is
malformed", which is not. Because a single code carried both,
is_skippable_title_stub answered yes to the second one — so feeding a
truncated or corrupt mkv:// input made the engine classify it
SkippableStub, print a notice saying the title was empty, and exit 0.
Silent data loss reported as success.

Split into E9053 MkvSourceInvalid for the read path (25 raise sites
across mkvstream.rs and ebml.rs's read primitives) and E9054
MkvUnencodable for the four write-side sites, which are the encoder
refusing to emit a body at or above the 56-bit VINT limit — an output
limit with no input involved, so calling it a corrupt source would be
wrong in the other direction. E6008 keeps only the zero-frame guard it
was documented to mean.

Kept one code for the whole read path rather than one per raise site:
nothing a consumer does differs between a bad VINT, a non-UTF-8 string
element, a truncated body and a child overrunning its parent. E9052 is
the model for when a carve-out earns its keep — laced blocks name one
specific RFC 9559 §10.3 feature with its own diagnosis.

Also fixed meta_sink.rs raising MkvInvalid for a serde_json encode
failure in the json:// sink, where no MKV is involved at all; it now
matches the identical guard in mux/meta.rs.

Reverting the split at the single code() arm reproduces the old
classification: 22 tests fail, including both new assertions. The
opposite direction is pinned too — dropping E6008 from the predicate
fails the genuine-stub test, which drives the real muxer end to end.
2026-07-29 22:11:53 -07:00
Matthew Jackson 81596e7735 Round 4: fix 26 defects across crypto, resource use and codec paths
Twenty-six confirmed findings from the fourth audit round, landed as one
cluster because they were found by agents working over disjoint file sets.

The one worth calling out is a pair of AACS tests that could not fail.
Both asserted CBC behaviour against a hand-rolled expectation that
happened to be IV-independent, so replacing AACS_IV with sixteen zero
bytes left them passing — they were pinning the code's own arithmetic,
not the published constant. Replaced with a literal witness of the
published IV plus the NIST SP 800-38A F.2.2 CBC-AES128 vector, and
verified the other way round: zeroing AACS_IV now fails three tests.

The rest are allocation and correctness work on hot paths: the Annex-B
writer in demux_sink allocated and freed a whole-frame Vec per frame,
which for a UHD title is ~200,000 allocations over the mmap threshold
plus the page faults to first-touch each one; it now reuses a buffer on
the writer, and still takes the NAL prefix width from the configuration
record rather than assuming four.

Six findings whose real fix lives in a consumer crate are recorded for
re-filing rather than patched here.
2026-07-29 22:09:52 -07:00
Matthew Jackson dce6d7abdf Parse MKV lacing, route by real TrackNumber, honour NAL length size and edit lists
Four conformance defects in the read paths, two of them silent corruption.

**Lacing was ignored entirely.** RFC 9559 §10.2 defines Xiph, EBML and
fixed-size lacing, where one Block carries several frames; the reader took the
Block payload verbatim, so a laced Block became a single "frame" consisting of a
lacing header followed by concatenated frames — garbage to the codec parser, no
error. Audio tracks from other muxers commonly use lacing, so an ordinary
foreign MKV was silently mangled.

All three modes are now parsed: Xiph 255-run sizes including the trailing-zero
rule for exact multiples of 255, EBML unsigned first size plus SIGNED VINT deltas
with the 2^((7*n)-1)-1 bias of §10.3.3, and fixed-size even division, with the
last frame's size deduced from the remainder. Laced timestamps follow §10.3.5:
the first frame takes the Block timestamp and the rest are spaced by the track's
DefaultDuration, else BlockDuration/count, else shared with a warn.

Parsing was chosen over refusing because refusal would leave freemkv unable to
remux common foreign audio at all, and each mode is about fifteen lines.

A malformed lacing header now raises a NEW code, E_MKV_LACING_INVALID = 9052,
deliberately NOT MkvInvalid — because is_skippable_title_stub classifies
MkvInvalid as a skippable nav stub, so reusing it would have recreated the exact
conflation that is still open as a separate finding. A test asserts the new code
is not skippable.

**TrackNumber was assumed to be 1..N in TrackEntry order.** RFC 9559 §5.1.4.1.1
only requires it to be non-zero and unique, so sparse or unordered numbers are
legal. Block routing and codec_private both computed track + 1. A real
TrackNumber map is now built, recorded only for TrackEntries that yield a stream
so dropped track types no longer shift the mapping.

Verified red here independently, and the failure mode is worse than mis-routing:
with track + 1 restored, a buttons track's payload was attributed to the AUDIO
stream — wrong payload into the wrong codec parser.

**The NAL length prefix was hardcoded to 4 bytes.** lengthSizeMinusOne lives in
avcC byte 4 and hvcC byte 21 (ISO/IEC 14496-15 §5.3.3.1.2, §8.3.3.1.2) and was
never read, so a source declaring 1- or 2-byte prefixes had its raw prefixed
bytes emitted verbatim with no start codes. All four conversion sites now derive
the width from the track's own configuration record.

**Edit lists were ignored.** No edts/elst was parsed, so the presentation
timeline an edit list defines (ISO/IEC 14496-12 §8.6.5/§8.6.6) was dropped —
which is how encoder delay is normally expressed. Leading empty edits and the
first media edit's media_time are now applied to both dts and pts, with the movie
vs media timescale distinction respected. A list needing more than a constant
shift applies the leading edit and warns rather than presenting the result as
faithful.

17 tests. I reproduced the lacing mutant independently: returning the body whole
kills five of them, including the exact-payload and malformed-header cases.

Still open and deliberately untouched: the MkvInvalid / is_skippable_title_stub
conflation across ~20 reader raise sites. It is a cross-cutting error.rs change
and E_MKV_LACING_INVALID is the template for it.
2026-07-29 21:53:19 -07:00
Matthew Jackson 6e9a53e151 Anchor forensic segments to the forensic clip, and stop CPS branching on resolve order
Two correctness defects in AACS 2.1 / FMTS key-map resolution, both order- or
anchor-dependent, and both able to abort a whole disc or silently garble it.

**The single-CPS short-circuit depended on which title resolved first.**
`pool_len` counted the WHOLE unit-key pool, and resolve_fmts_key_map appends the
disc's forensic index keys to that same caller-owned pool. The count was captured
before THIS call's FMTS branch but not before earlier titles', so once any forensic
title resolved, every later title saw a pool larger than one and fell into
multi-CPS sampling — 8 random reads per extent, and a whole-disc DecryptFailed if
no pooled key opened a menu extent's samples. A disc that ripped fine when a
non-forensic playlist sorted first failed when a forensic one did.

Forensic keys are now tagged FMTS_POOL_TAG_BASE = 1 << 24 and the short-circuit
asks single_base_key_slot(), which excludes them. The old 1000 tag was NOT kept,
and the reasoning is worth recording: base CPS ids are Unit_Key_RO.inf position + 1
and that count is a BE16, so 1000 sits inside a genuinely reachable id space. 1<<24
cannot collide. The id field is cosmetic — decrypt.rs indexes the pool by slot and
reads only the key — so widening the tag is safe.

**Forensic segment SPNs were anchored to the wrong clip.** They live in the
forensic feature clip's byte space, but were mapped through
clip_byte_to_lba(&title.extents, ..), which treats byte 0 as the start of the
title's FIRST extent. Any playlist not beginning with the forensic clip mapped
every segment to the wrong LBA: either the anchor probe sampled the wrong clip and
the whole-disc resolve aborted with FmtsKeyMissing, or — worse — a forensic index
key was applied to non-forensic sectors while the real forensic units kept the base
key, giving silently garbled output with no error at all.

The correct anchor turns out to be a DISC fact, not title data: an AACS 2.1 disc
names its forensic feature BDMV/STREAM/<clip>.fmts, and carries one
IndividualSegment.tbl, so the SPNs are in that one clip's byte space. A new
forensic_clip_extents() finds the unique .fmts in the already-walked UDF tree, and
those extents now drive the segment arithmetic, the addressability filter, and the
index probe — whose title parameter is gone, since its reads were mis-anchored too.
"Does this title carry forensic content" is now "does it read the forensic clip's
sectors" rather than "do the segment bytes land somewhere in the concatenation".

Where the clip is NOT identifiable — no .fmts, or several, making the SPN space
ambiguous — on a disc that does carry a non-empty table, the resolve now fails loud
with FmtsKeyMissing rather than guessing an anchor. That is a deliberate behaviour
change: a hypothetical disc with two .fmts clips hard-fails where it previously
produced a possibly-wrong map. Failing loud beats silently garbled output, and
inventing an anchor was not acceptable.

This site had been flagged independently three times — by the agent that added the
FMTS per-disc memo, by the round-4 correctness lens with a concrete scenario, and
by the round-4 conformance pass.

Verified red here independently: reverting single_base_key_slot to count the whole
pool fails both new tests. The agent's own evidence was probe_reads 48 vs 40 (the
8 extra sampling reads) and an E7013 DecryptFailed whole-disc abort, and for the
anchor an E7026 FmtsKeyMissing on a [trailer, forensic] extent list.

All six pre-existing FMTS tests pass unchanged through the new anchor, including
the exact-cost assertions (40 probe reads, one key-service call per disc, one UDF
walk for 60 titles), so the round-3 memoisation wins are intact.
2026-07-29 21:37:50 -07:00
Matthew Jackson 75bcc71f34 Group whole E-AC-3 frame sets, and keep short reads unit-aligned
Two defects in fixes landed the same day, both found by round 4 auditing round 3's
work rather than trusting it.

**The E-AC-3 grouping ignored substreamid, so the timeline still doubled.**
Confirmed independently by the correctness and conformance lenses and verified by
hand: substreamid appeared only in test helpers, never in the production path. Per
ETSI TS 102 366 (A/52) Annex E a frame set is independent substream 0 — mandatory,
always first — with its dependents, then the OPTIONAL additional independent
substreams 1..7 with theirs, all covering the same time period. Treating an
additional independent substream as a new access unit advanced the clock a second
time for the same 32 ms, which is exactly the doubling the grouping fix existed to
prevent. No fixture caught it because every fixture used substreamid 0.

is_dependent_substream becomes substream_role -> Starts | Extends: strmtyp 1
extends; strmtyp 0/2 with substreamid != 0 now extends (this was the bug); strmtyp
0/2 with substreamid == 0, legacy AC-3, and reserved strmtyp 3 start. Reserved 3
starts regardless of its id bits, because its BSI layout is undefined so those bits
cannot be trusted — an unknown frame is neither merged into an unrelated programme
nor silently discarded.

The frame set stays ONE sample rather than being split into a separate track for
the associated service, and the reasoning is in the module doc: a substream
numbered 1..7 with no substream 0 is not conforming, so extracting one would mean
renumbering ids and rebuilding frame sets — a transcode, not a remux. Programme
selection is the player's job.

A stream joined mid-frame-set (first sync is substreamid 3) is skipped with a debug
and resyncs at the next id-0, mirroring the orphan-dependent rule: its mandatory
id-0 substream was never seen, so it is neither decodable alone nor timeable.

MAX_AC3_BUF 128 KiB -> 1 MiB, because an AU is now a whole frame set: worst case 8
independent x 9 substreams x 8192 B = 576 KiB, which the old cap could have dropped
mid-hold.

**The forced probe's two round-3 fixes cancelled each other.** CHUNK_SECTORS = 1023
exists (with a const assert) so every read starts on a 3-sector AACS aligned-unit
boundary; the short-read fix advanced by actual bytes, making the advance a
non-multiple of 3. Every later read was then misaligned, DecryptingSectorSource
refused it before reading, the stop became ReadFailed, and no verdict was asserted
— so content-based forced detection silently fell back to the vendor label on
exactly the encrypted discs the 1023 change was written for.

A partially-satisfied read now advances only by whole aligned units and re-reads
the <=2 residue sectors from the next boundary, feeding only the aligned prefix so
nothing is double-fed and no partial unit reaches the parsers. A read that fully
satisfies its request still advances by all of it. When less than one aligned unit
comes back the bytes are fed and the same LBA is retried twice before stopping, so
a starved source cannot spin — verified by raising the retry limit and watching the
test hang.

Verified red independently here: reverting substream_role to strmtyp-only fails
eac3_additional_independent_substream_stays_in_the_frame_set (6 access units where
3 are correct — the doubling, literally) and the mid-frame-set resync test.

Reported, not fixed: dec3_box still hardcodes num_ind_sub - 1 = 0 and
num_dep_sub = 0, so it under-declares any stream carrying additional independent
or dependent substreams now that frame sets arrive whole. DolbyConfig has no
fields for either; a real fix needs the parser to surface observed substream
counts. That file is another lens's this round.

Unverified: no real multi-programme DD+ stream exists here, so defect 1 rests on
synthetic Annex-E fixtures. The retail DD+ check (No Time to Die, all substreamid
0) confirms single-programme discs are unaffected.
2026-07-29 21:33:18 -07:00
Matthew Jackson de46268785 Make two public-API panics return errors, and drop two shipped citations
**The session panics are reachable, and my earlier triage of them was wrong.**
`DiscSession::scan` and `resolve_keys` both did
`self.drive.as_mut().expect(..)`. I previously downgraded these to LOW on the
grounds that no shipped consumer calls them after the drive has been staged into
the reader slot. That is the wrong test: `stage_drive_as_reader` is a PUBLIC
method that empties the drive slot, so the public surface permits the sequence,
and a library must not panic from public API regardless of what current callers
happen to do. Both now return Error::DeviceNotReady.

**A shipped doc comment cited a third-party source FILE** as the authority for
the CLPI ProgramInfo layout ("Layout per the BD CLPI spec clpi_parse.c"). Now
cites the Blu-ray Disc Read-Only Format Part 3 CLIPINF specification.

**The CHANGELOG justified a muxer decision by naming a commercial competitor**
("MakeMKV's rip of the same disc omits it", "matching MakeMKV"). Reworded to
stand on its own terms: the element is optional in RFC 9559, nothing requires it
for interlaced SD, and the 40 ms DefaultDuration is the frame rate the source
actually carries.

The leak gate is extended for both new classes — third-party `*_parse.c` /
`*_dec.c` / `*_demux.c` style filenames, and a competitor named as authority.

Narrowing that rule took two attempts, which is worth recording. A bare
`\.(c|cpp|cc)` pattern produced eight false positives: this repo has its own C
shim (`macos_shim.c`) that build.rs and the docs legitimately reference, and the
pattern also matched the Rust field access `p.cc`. It now matches only the
suffixes typical of third-party media-library sources. This is the second
false-positive round on this rule — the first flagged ffmpeg INVOCATIONS in the
test harness — so the lesson is that a hygiene pattern needs testing in both
directions before it lands, exactly like any other code.
2026-07-29 21:21:29 -07:00
Matthew Jackson 22f71bb243 Restore the MP4 conformance fixes I clobbered while landing another agent's work
detect_rate's nearest-match fix, the colr HLG/BT.470 fix and their four tests
were silently reverted. Cause: the r3fix-silent worktree was cut BEFORE the
conformance commit landed, and I landed its work by copying whole files into the
main tree. mp4/mod.rs was in both agents' file sets, so silent's copy — built on
the older base — overwrote the conformance changes wholesale. The gate stayed
green throughout, because reverting a fix and its tests together is perfectly
consistent.

Re-applied the conformance commit's diff for that file with a three-way merge;
both agents' changes to mp4/mod.rs now coexist (final_report, UndescribableAudio
and the max(track_id) id fix are all still present alongside RATE_TOLERANCE_FPS
and the colr resolver delegation).

Only mp4/mod.rs was affected. mp4/audio.rs and mkv.rs were in no other agent's
file set and were intact.

Process lesson, recorded because I would otherwise repeat it: NEVER land a
parallel agent's work by copying whole files, when its worktree was cut at an
older base than HEAD. Apply its DIFF (git apply -3), or rebase its worktree
first. Copying files silently discards anything committed to those files in the
interim, and no test can catch it because the tests disappear with the code.
2026-07-29 21:03:26 -07:00
Matthew Jackson c73d6c6291 Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.

The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.

Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.

Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.

The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
2026-07-29 21:00:55 -07:00
Matthew Jackson 41102bf023 Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files
another agent held this round, 0 rejected.

**A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and
1024 % 3 == 1 — verified — so every chunk after the first was misaligned against
the 6144-byte AACS aligned unit and would be REJECTED by
DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle
probe I added last round would have read almost nothing past its first chunk.
Now 1023 sectors (341 aligned units) with a const assertion that fails the build
if it stops dividing, plus set_unit_base per extent so the source's gate is
anchored where the extent actually starts.

**The same probe skipped sectors on a short read**, advancing by the REQUESTED
count rather than the bytes actually returned, so a partial read silently left a
gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n
to the buffer.

**Its cache key omitted the PGS PID set**, so a playlist declaring an extra
subtitle PID got another playlist's verdict for a track that had never been
probed. And the key was the whole extent list, so partial clip sharing missed
entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent
keying was shown SOUND rather than assumed: ForcedTracker is two monotone
booleans, so per-extent evidence composes by field-wise OR, order- and
grouping-independently. Making that honest required per-extent demux state, so an
extent's evidence comes only from its own bytes, and memoising only extents whose
read reached a designed stop.

**A reachable panic in the timeline.** mkvstream::parse_block accepts a
TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the
next passive frame panicked adding the backstep. In release it wrapped negative
instead, firing the straggler clamp for essentially every passive frame — audio
and subtitles rewritten onto the wrong point of the output timeline. All four
sites saturate.

**A public constructor divided by zero**: PrefetchedSectorSource::new_with_events
with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling.

**Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro,
samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as
prevention, because the next tracing::debug! someone adds is the leak. A doc claim
that DiscInputs "contains no secrets" was false and is corrected.

**An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB
like its writeback sibling, with the parse split out so the bound is testable
without touching process env.

**The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification.
Now file_len/16, since only vide/soun tracks are indexed and the shortest legal
AC-3 frame is 128 bytes.

**Two pipeline concurrency defects**: a consumer apply() error was invisible to the
producer, and abandon/finalise had a TOCTOU where a caller could report an
unfinalised output. Both fixed with compare-exchange state rather than a bool.

**Two per-frame copies removed**, both MEASURED rather than reasoned: the AU
assembler now hands its allocation to the frame (same pointer, unchanged capacity,
proven by asserting the pointer) and tsmux reuses one Annex-B buffer across
frames. Both keep capacity deliberately — a naive split_off would have cost more
than it saved.

**A comment pointed at the wrong file** for a mirrored constant; the mirror is now
compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift
fails the build.

Deferred to another agent's files, all confirmed: detect_rate's fractional-twin
snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic
base-key gap fill, and MkvStream's frame cap counting frames rather than bytes.

Every fix verified red by reverting it. Also noted for later:
DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext.
2026-07-29 20:47:31 -07:00
Matthew Jackson 07cb9ec62c Make six silent mux failures observable
All six confirmed against the code. The governing rule this cluster serves: a
lossy or degraded outcome is never silent, because a corrupt rip the user does
not know about is the worst failure available.

**A 3D MKV re-mux silently lost one eye.** The BlockGroup read path had arms for
BLOCK / BLOCK_DURATION / REFERENCE_BLOCK only, so BLOCK_ADDITIONS fell into the
skip arm — while the writer does emit BlockAdditions > BlockMore > BlockAdditional
for the MVC dependent view. Reconstruction was judged out of scope and the
reasoning is recorded: PesFrame has no side-payload field and the header parser
never reads BlockAdditionMapping, so there is no dependent-view track to route the
AU to. Instead the loss is now LOUD — counted in bytes and events, warned once,
and surfaced through MkvStream's errors()/lost_bytes(), which the driver already
samples into MuxOutcome. One detail in the finding was wrong and is corrected: the
re-mux does NOT still advertise the mvcC mapping, because the header parser
ignores that element, so the output is a plain 2D H.264 track.

**An all-titles rip silently skipped real titles.** The header-buffer-cap
overflow returned Error::MkvInvalid, and is_skippable_title_stub matches exactly
E_MKV_INVALID | E_CSS_KEY_MISSING — verified here — so a 512 MiB-of-frames title
was classified as an empty nav/menu PGC stub and dropped. It now has its own
E9051 / MuxHeaderBufferExceeded { bytes }, outside the skippable set.

**The public pre-mux report contradicted the file.** Mp4Sink::finish() drops an
audio track it cannot describe, which I chose last round over failing an export
whose video is fine — but mp4_fit_report still listed that stream as included, so
the application's plan and the actual output disagreed. Fixed at both levels:
Mp4SkipReason is now non_exhaustive with NoSamples and UndescribableAudio,
Mp4Sink::final_report() describes the FILE rather than the plan, and for the
boxed dyn Stream path a defaulted Stream::undelivered_streams() carries the
information out to MuxOutcome::undelivered_streams with a driver-side warn.

**MP4 track ids could collide.** ids were assigned before the retain that drops
sample-less tracks, while next_id came from the post-retain count, so [1,3]
yielded next_id 3. Now max(track_id) + 1, saturating.

**Stream selection silently skipped its codec_privates prune** when the lists were
not the same length — but codec_privates is consumed POSITIONALLY and trailing
extras are documented as benign, so the length-equality guard was itself the bug.
The prune now runs unconditionally by index.

**The m2ts_mux scaffolding armed params_written on both the absent and the
unparseable codec_private arms** — the same defect already fixed in tsmux.rs.
Split into params_attempted (a latch, since retrying identical bytes cannot help)
and params_emitted, with a warn on each failure arm and an accessor so the
eventual wiring and its test can observe it.

Each fix verified red by mutating back to the prior behaviour: errors() 0 vs 1,
E6008 vs E9051, final_report [0,1] vs [0], next_track_id 3 vs [1,3], and the
selection prune resolving index 1 to the wrong track's record.

API surface deliberately widened: MuxOutcome gains a public field and Mp4Sink
becomes public. Nothing in-repo breaks. Note a behaviour change on the mkv://
input path — a 3D re-mux now reports non-zero loss, so a consumer treating
errors > 0 as disc damage will trip on it. That is intended: the outcome IS
degraded.
2026-07-29 20:46:10 -07:00
Matthew Jackson 455763587c Fix four MP4 conformance defects against the standards
**dec3 declared a 0 kbit/s AC-3 substream.** parse_dolby routes bsid < 11 to
parse_ac3, which leaves data_rate_kbps = 0 and keeps the AC-3 bsid, yet
dolby_sample_entry wrapped that config in ec-3/dec3 for any Codec::Ac3Plus track.
ETSI TS 102 366 Annex F.4 assigns ac-3/dac3 to an AC-3 bitstream and F.6 assigns
ec-3/dec3 to an Enhanced AC-3 one, so the entry now follows the SYNCFRAME that was
actually parsed, not the playlist's codec label. Computing an AC-3 data rate and
keeping ec-3 was rejected: it fixes one field while bit_stream_identification and
num_dep_sub keep misdescribing the stream. The bsid threshold is hoisted into one
constant so parser and entry-chooser cannot drift. Adjacent defect fixed in the
same box: data_rate is 13 bits from a u16 source, so push's mask WRAPPED anything
above 8191 (9000 became 808); it now saturates.

**colr tagged HLG as PQ, and PAL as BT.601.** video_colr carried a second,
drifted copy of the ColorSpace-to-CICP map: transfer 16 (PQ) for every BT.2020
stream with no HdrFormat override, and 6 (BT.601) for Bt470bg. Per ITU-T H.273
Table 3, HLG is 18 and BT.470-6 System B/G is 5 — and mkv::cicp_for_video already
got both right. video_colr now delegates to that shared resolver, so the
duplicated table is gone and cannot drift again. It keeps only its own decision
about WHETHER to emit the box, since an absent colr and an all-unspecified colr
mean the same thing per ISO/IEC 14496-12.

**Exact 24.000 / 30.000 / 60.000 fps was declared 23.976 / 29.97 / 59.94.**
detect_rate took the FIRST STD_RATES entry within 0.5 fps, and every 1000/1001
entry precedes its integer twin 0.024 fps away — a 0.1% error across the whole
track's mdhd and stts. Fixed as nearest-wins rather than by reordering the table:
reordering fixes today's table and re-breaks the moment someone appends a rate,
while nearest-wins is order-independent. Verified red here independently by
reverting to first-match, which fails exactly the two timing tests.

**ddts MultiAssetFlag was set from has_extension.** In the DTSSpecificBox
(ETSI TS 102 114) that flag signals more than one audio ASSET. A DTS-HD MA/HRA
track is one asset whose extension substream carries the XLL/XBR component, so
setting it from "an EXSS sync follows the core" sent a parser looking for a second
asset descriptor while StreamConstruction simultaneously said there was no
extension — the box contradicting itself. This module parses the core header only
and never reads the EXSS asset table, so 0 is the only honest declaration. A
StreamConstruction index for core+EXSS was deliberately NOT invented: that field
is a table lookup that could not be confirmed against the standard, and a wrong
index is worse than an under-declaration.

DTS_AMODE_LAYOUT's masks were independently re-derived against ETSI TS 102 114
§5.3.1 and all 16 are CORRECT — only three adjacent comments were wrong (the
AMODE 2/3/4 annotations were rotated by one, and AMODE 9's said "5.1 with LFE"
when 0x0007 is the 5.0 mask and LFE is OR'd in separately). Comments corrected.

Every one of the eight new tests decodes the field back OUT of the emitted bytes —
data_rate from the dec3 body's leading 13 bits, MultiAssetFlag from bit 48 of the
ddts tail, colr from the nclx payload inside a real stsd, and the frame rate from
mdhd.timescale plus stts.sample_delta of a fully muxed MP4 — rather than
restating arithmetic. This audit has already caught one of my own tests doing the
latter.

NOT fixed, root-caused and recorded at the site instead: an A_PCM/INT/BIG track
ships with no BitDepth, which the Matroska Codec Specifications make a MUST. The
width exists on disc (BD LPCM signals it in the ES header byte 3, DVD in the IFO
audio attribute byte 1) but neither source reaches MkvTrack::audio, and the fix
needs a new AudioStream member plus a deferred setter in files another agent held
this round. Guessing 16 was rejected — it would confidently misdecode every
24-bit disc — as was refusing the track, which would regress the 16-bit majority
that currently plays by accident.
2026-07-29 20:29:17 -07:00
Matthew Jackson 4b58d3db98 Resolve the forensic key map once per disc, not once per playlist
resolve_content_key_map loops every title into resolve_mux_key_map, which called
resolve_fmts_key_map FIRST — before the CpsUnitCache — and on an FMTS disc
returned immediately. So every playlist re-derived facts that belong to the DISC:
a full UDF walk plus /AACS/IndividualSegment.tbl, and on an FMTS disc the anchor
probe, the 32-index phase probe, and a fetch.fmts_indexes round trip.

On a 60-playlist disc, measured on a synthetic fixture: 840 -> 14 metadata reads,
2,400 -> 40 probe reads, and 60 -> 1 key-service calls. Worst case before was up
to 256 probe reads and 32 key-service calls per title. The 60 redundant
key-service round trips are a strong candidate for the keyserver storm seen in
the field.

Two memos behind a pub(crate) DiscKeyCache. The table memo (UDF walk + tbl parse)
is disc-invariant outright — nothing in that path mentions the title — and runs on
EVERY disc, so a plain BD benefits too. Only the deterministic negatives are
memoised as "not FMTS"; a DiscRead fault propagates uncached so a later title
retries.

A blind once-per-disc hoist of the PROBES was rejected as unsafe, and this is the
load-bearing reasoning: the title enters through clip_byte_to_lba, which decides
which segments are addressable and which LBA every probed clip byte reads from, so
two titles with different extent lists probe different physical bytes. A hoist
would serve title B an answer derived from title A's media and could silently turn
a per-title FmtsKeyMissing into a success. The extent list is the ONLY per-title
input, so keying on it is exactly sufficient — matching the ForcedProbeCache
precedent.

Result-identity was proved, not assumed: NEITHER probe reads the key pool.
Verified here independently — probe_fmts_index_keys takes no keys parameter at
all; index keys come from `fetch`, and the anchor's reply feeds the phase probe.
So the pool's growth across titles, the one thing that does change between calls,
cannot move a memoised value, and the result is order-independent. A test resolves
three titles through a shared memo and through fresh memos and asserts both the
per-title ranges and the final key pool (keys, slots, order) are identical.

Not memoised, deliberately: fail-loud FmtsKeyMissing, and any run where an index
hit a read fault — that is a property of a transient drive fault, not of the
extents, and caching it would spread one bad read across 59 playlists.

A fully-memoised title now does zero I/O, which made the old in-loop halt polls
unreachable for it, so a check_halt on entry was added with a test that cancels
after warming the memos.

Also corrects my own overstatement from last round: the CpsUnitCache doc now says
plainly that on an FMTS disc it removes NO reads, because this function returns
before the extent loop ever runs.

Five mutations, all verified red. Pre-existing bug flagged but not fixed:
filter_addressable_segments only checks that a segment's START byte maps to some
LBA in the title, so a play-all playlist can pass the filter while mapping segment
bytes into the wrong clip, whose anchor then returns empty and aborts the sweep.
2026-07-29 20:27:50 -07:00
Matthew Jackson d451d00f72 Group E-AC-3 dependent substreams into one access unit
The AC-3 parser's own module doc stated the assumption: "AC3 frames are
self-contained and always start with syncword 0x0B77". True for legacy AC-3,
false for E-AC-3 above 5.1. Per ETSI TS 102 366 (A/52) Annex E, byte 2 of an
E-AC-3 syncframe is strmtyp(2) | substreamid(3) | frmsiz[10:8], and an access
unit is one INDEPENDENT substream plus every DEPENDENT substream that follows it
until the next independent one. The parser emitted one PES frame per syncframe,
so a decoder saw each dependent substream as a standalone frame with no parent —
including the AC-3-core + E-AC-3-dependent form Blu-ray uses for Dolby Digital
Plus. The extra channels were lost and the timeline ran at 2x.

The bit position is cross-checked against code already in the tree: the existing
frmsiz parse takes byte2 & 0x07 as its high bits, which is only consistent with
strmtyp occupying byte2's top two bits. Legacy AC-3 is excluded by bsid < 11,
where byte 2 is crc1 and reading strmtyp there would be nonsense. Reserved
strmtyp 3 is treated as INDEPENDENT so an unknown type starts a fresh AU rather
than merging into an unrelated one.

The AU carries the INDEPENDENT substream's PTS, and only the independent
substream advances the clock — dependents cover the same time period and add zero
duration. That is what removes the doubled timeline.

A trailing AU that can still grow is HELD across the PES boundary, because the
boundary is unknowable until the next independent sync; the whole AU is re-scanned
next call, so there is no shift and no double-count in the loss tally. Plain AC-3
is never held, which keeps DVD/AC-3 latency and behaviour unchanged.

A latent pre-existing bug surfaced while testing this: a new PES's PTS was
re-stamping an AU that began in an earlier PES, a constant one-frame shift. Fixed
with a PtsAnchor so a PES timestamp applies to the first AU that STARTS in that
PES's own bytes, while a genuine PTS jump is still adopted.

Nine tests. Verified red against five mutations, each killing a specific set:
reverting to the pre-fix behaviour kills 8 while
plain_ac3_frames_are_not_grouped_or_delayed SURVIVES as the no-regression guard —
reproduced independently here. Stamping the dependent's PTS kills 6; not holding
across PES kills 4; holding plain AC-3 too kills 15; neutering the PTS anchor
kills exactly the 2 split-across-PES timing tests.

Three sibling defects found and deliberately NOT fixed, all in mp4/audio.rs:
dec3 hardcodes num_dep_sub = 0 (and a nonzero value changes the box LAYOUT, not
just a field, per Annex F/G); parse_eac3 ignores strmtyp/substreamid entirely;
and a 7.1 DD+ track is still labelled 5.1 because the channel count comes from
the independent substream while the extra channels are described by the
dependent's chanmap, which nothing parses.

Not verified: no real E-AC-3-with-dependents sample exists here, so all evidence
is synthetic frames plus the spec layout. The multi-independent-substream case
(num_ind_sub > 1, main + associated audio in one PID) is deliberately treated as
one AU per independent substream and is untested.
2026-07-29 20:26:51 -07:00
Matthew Jackson f6ba4823d1 Fix three defects in last round's own fixes
Round 3 audited the round-1/2 fix commits rather than trusting them, and found
three defects in that new code. This is why the pin moves each round.

1. LICENCE REGRESSION, and it was mine. Reverting the "distinguish a failed key
   source" commit also restored a verbatim reference-decoder table citation in
   src/mux/codec/dts.rs, because both changes were in that one commit. The MIT
   licence cleanup was silently undone at HEAD and nothing caught it.

   The citation is replaced with ETSI TS 102 114 §5.3.1 again, and — more
   importantly — the rule now lives in the leak gate instead of in my memory.
   scan-secrets.sh gains LICENCE_RE, which flags ff_dca*, dcadec, l-smash,
   libav*, and bare ffmpeg/FFmpeg as REF-IMPL-CITATION. `no ffmpeg` is
   explicitly allowed via negative lookbehind: stating what this project does
   NOT depend on carries no risk and is a genuine selling point. Verified by
   re-introducing the citation (gate fails) and removing it (gate clean).

2. TsMuxer armed params_written even when the avcC/hvcC parser returned None, so
   a track whose codec_private exists but will not parse was muxed to BD-TS with
   no VPS/SPS/PPS ever emitted — undecodable video, reported as success, with no
   log line. Last round's fix corrected WHICH parser is used and left this half
   untouched. Arming the flag is still right (retrying identical bytes cannot
   succeed) but it is no longer silent: it now warns with the track, codec and
   codec_private length.

3. test_aes_cbc_roundtrip defined a LOCAL fn aes_cbc_encrypt that SHADOWED the
   production primitive, so it round-tripped a copy of the algorithm against
   itself and never touched crypto::aes_cbc_encrypt — the function this cycle
   added. Any mutation to the shipped code passed it. The shadow is deleted and
   the test now calls the real primitive; verified by mutating
   crypto::aes_cbc_encrypt, which now fails it and previously would not have.
2026-07-29 20:08:09 -07:00
Matthew Jackson 85f2246b0f Pin the toolchain to Rust 1.87
The Windows UI needs winsafe, whose current release requires rustc 1.87. The
alternative was pinning winsafe back to an older release, which would bake a
stale API surface into a brand-new UI permanently to dodge one minor version.

The pin's purpose is to sit BELOW the Mac default so clippy drift is caught
locally before CI, not to stay on 1.86 specifically, so 1.87 preserves the
discipline exactly.

Verified before moving anything, not after: `cargo +1.87 clippy -- -D warnings`
and `cargo +1.87 fmt --check` are clean across all eight repos, and the full
precommit gate (fmt + clippy + tests) passes on libfreemkv, autorip, bdemu,
freemkv-engine and freemkv-keysources. Zero new lints, zero formatting drift.
2026-07-29 20:02:02 -07:00
Matthew Jackson 61f98535d8 Memoise multi-CPS key-map sampling per extent, not per title
resolve_content_key_map calls resolve_mux_key_map once per title, and on a
multi-CPS disc that path issues 8 random single-unit reads per extent. A disc's
playlists overwhelmingly reference the same few clips — main feature, play-all,
per-chapter and seamless-branch variants — so the same physical extents were
re-sampled from the drive once per playlist. On a 60-playlist / 15-clip disc
that is ~2,400 non-sequential 6144-byte reads, roughly 8 minutes of pure seeking
at 200 ms per seek, before the mux starts. Now ~600 reads.

Keyed per EXTENT — (format, start_lba, sector_count) — rather than per title's
whole extent list, which is finer-grained than the forced-subtitle probe's cache
and strictly better here: a play-all playlist sharing 4 of 5 extents with the
main feature still hits on those 4.

Why a cached pool index is provably identical to a recomputed one, verified
rather than assumed:

  * `pick` iterates the pool IN ORDER and returns the FIRST index whose key
    decrypts a sample to clean.
  * The pool is APPEND-ONLY. Checked across the whole crate: only `push`, with no
    insert/remove/clear/retain/sort/dedup/truncate/drain/swap/reverse anywhere.
    So appended keys can only land AFTER a matched index, and the first match for
    the same samples cannot shift.
  * The samples are a pure function of the three values in the key, read from
    read-only optical media.

Two outcomes are deliberately NOT cached, which is what makes this safe rather
than merely faster:

  * the inherited index (`None if samples.is_empty() => last_idx`) is per-TITLE
    state, not a property of the extent — caching it would let one title's
    carry-in index leak into another title's clear extent, i.e. a WRONG key;
  * the fail-loud DecryptFailed verdict, so a retry after a key source banks the
    missing key re-samples instead of inheriting a stale answer.

Halt is still polled before the cache lookup, so cancellation is unchanged.
resolve_mux_key_map keeps its exact signature and delegates with a fresh cache,
so there is no public API change. ContentFormat gains Eq + Hash (additive).

Four tests, and the two mutants that matter both verified red: disabling the
cache short-circuit fails the hit and recompute-equivalence tests, and wrongly
caching the inherited index fails multi_cps_inherited_index_is_not_cached.
2026-07-29 19:49:55 -07:00
Matthew Jackson 36d0801f14 Build BlockGroups in memory so the hot path never seeks
Every BlockGroup frame back-patched its element size via ebml::end_master, which
does two stream_position() calls and two real seeks. BufWriter does not override
Seek::stream_position, so each position query is seek(Current(0)) = flush_buf +
lseek — and each of those flushed the 4 MiB BufWriter while every position-moving
seek reset WritebackPipeline::last_flush_pos. The buffer never got to do its job.

An MPEG-2 title takes this path for EVERY frame (the parser stamps a per-frame
duration, so I, P and B all become BlockGroups) — roughly 350,000 per feature.

A BlockGroup's size is knowable before writing, so there is no need to back-patch
at all. New seek-free twins start_master_buf / end_master_buf patch a placeholder
by buffer INDEX instead of file offset, and build_block_group assembles the whole
element into a persistent buffer that write_block_group and its MVC sibling take,
fill, write once, and hand back — including on the error path — so the allocation
is made once rather than per frame.

Measured with a counting writer that, like BufWriter, does not override
stream_position, over 200 frames:

              seek calls   position-moving
  plain   before 912              451
  plain   after  112               51
  MVC     before 2516            1253
  MVC     after  116               53

Per-frame cost is now zero; the residual is the header, the per-cluster
back-patch and Cues. For 350k BlockGroups that is 1.4M seek calls removed on the
plain path, 4.2M on an MVC title.

end_master is deliberately NOT changed for its other callers. The Cluster master
genuinely streams — frames are appended to an open cluster over time, so its body
cannot be buffered without holding a whole cluster in memory — and the rest
(EBML header, Tracks, Info, Chapters, Cues) run once, not per frame.

Byte-identity is the safety property, and it is structural: end_master always
patches a FIXED-width 8-byte VINT, and the buffered pair writes and patches
exactly that same placeholder, so the encodings cannot differ. Verified by
capture-then-compare over 200 frames (17 keyframes / 183 non-keyframes, multiple
clusters, BlockDuration present and absent, both reference branches) — output
byte-for-byte identical.

Three tests now pin it permanently: buffered vs seeking output byte-for-byte for
empty/tiny/multi-byte bodies, the same for NESTED masters (the MVC path nests
BlockAdditions > BlockMore, where an index-arithmetic slip would surface), and
end_master_buf erroring rather than panicking on a position outside the buffer.
All three verified red against a deliberately divergent placeholder width.

A note on verifying this kind of change: comparing emitted MKV across two
worktrees at different commits shows a spurious 14-byte difference, because
MuxingApp/WritingApp embed the build's git SHA twice. Compare at the same base.
2026-07-29 19:33:31 -07:00
Matthew Jackson 656c6f36d7 Only assert a forced-subtitle verdict the read actually supports
probe_and_set_forced broke out of its read loop on a read error but still
applied whatever partial observation it had accumulated as an authoritative
verdict, overwriting the vendor-label-derived forced flag. A disc that faults
early could have a correct flag replaced by a guess from a fraction of the data.

The file already got the zero-observation case right — it deliberately leaves
the vendor flag alone rather than "assert not-forced from having seen nothing".
The defect was that a PARTIAL observation cut short by a fault was treated as
complete.

The fix rests on the two verdicts not being symmetric evidence.
settled_not_forced() is POSITIVE evidence — a non-forced display set was
actually seen, and no unread data can retract it. is_forced() is an ABSENCE
claim — display sets were seen and none was non-forced — which is only sound if
the read got far enough for the absence to mean something.

So every loop exit now yields a named StopReason, and absence claims are
asserted only for a designed stop:

  * Exhausted / Budget → conclusive. The budget is deliberately conclusive: the
    natural exit is "every track settled not-forced", which a genuinely forced
    track never satisfies, so the budget is the ONLY path by which a real forced
    verdict is ever reached. Treating it as inconclusive would disable forced
    detection entirely.
  * Halted / ReadFailed → inconclusive. A cancelled probe's cut-off point is as
    arbitrary as a faulted one, so its absence claim is worth no more.

Evaluated per track, matching the existing observed() gate: a track that already
saw a non-forced set keeps its sound verdict even on a truncated run, while a
forced-so-far sibling keeps the vendor flag.

An inconclusive run is also NOT memoised. The cache key is the extent list and a
disc's playlists share clips, so caching a truncated run would replay one read
fault onto every playlist referencing those extents and deny any later title the
chance to re-read them.

Four tests; three of them verified red by forcing absence_is_conclusive() back
to always-true (the old semantics), while the budget test correctly stays green
either way — confirming the guard was not over-corrected.

Also moves the function doc comment back onto probe_and_set_forced; the earlier
probe commit left it attached to the ForcedProbeCache type alias.
2026-07-29 19:26:58 -07:00
Matthew Jackson 7d861f5287 Record that the failed-vs-absent key source arm is unreachable
The `Ok(_) | Err(_)` arm in resolve_and_apply_traced conflates "this source
had no entry for the disc" with "this source failed", and reports both as
KeyNode::NoEntry. An operator whose key server is returning 502s is therefore
told their disc is not in the database.

The conflation is real but LATENT, and fixing it here would change nothing an
operator can see, because no shipped KeySource ever returns Err:
KeydbSource::get_unit_keys maps a load/parse failure to Ok(Vec::new()),
OnlineSource::get_unit_keys is Ok(self.query(ctx)) where query returns empty on
transport error, HTTP status, oversize body and bad JSON alike, and MultiSource
discards inner Errs. Only test doubles return Err. FetchOutcome::errored in
drive_unit_keys / drive_fmts_indexes is dead for the same reason — the right
contract, honoured by no source.

autorip already works around the missing signal by re-probing the service over
HTTP (probe_online_reachability / key_service_transient_status), and its own
comment names the incident: "the online keysource swallows every failure
(transport error, 502, timeout)".

So the fix belongs at the source boundary in freemkv-keysources, with
Disc::aacs_error as the channel the operator actually reads — not in this
trace. Documented here so the next reader does not assume the arm works, and
does not "fix" a dead path as I nearly did twice.
2026-07-29 19:24:58 -07:00
Matthew Jackson 032994be40 Revert "Distinguish a failed key source from one with no entry"
This reverts commit e5e3b863fd.
2026-07-29 19:15:21 -07:00
Matthew Jackson a61503502e Remove reference-decoder citations from a public MIT-licensed repo
This crate is MIT licensed. Comments citing another decoder's internal symbols
and reproducing its tables verbatim create licence risk that no engineering
benefit justifies, so every such citation is replaced with the primary source:
ETSI TS 102 114 §5.3.1.

Eleven sites across src/mux/codec/dts.rs and src/mux/mp4/audio.rs. The technical
substance is unchanged in every case — the deficit-sample-count semantics, the
reserved-field skips, the invalid LFF value, and the 16 legal AMODE codes are all
spec facts and are now attributed as such. Four CHANGELOG entries that named a
validator are reworded; the "no a reference decoder" dependency claim stays, since stating
what this project does NOT depend on carries no risk.

I initially argued this was a false positive on the grounds that the project's
hygiene rules name internal infrastructure and reverse-engineering material, not
open-source citations, and that a channel-count table from a standard is fact
rather than expression. That reasoning missed the point: the exposure is MIT
distributing text derived from GPL/LGPL sources, and that is the maintainer's
risk to weigh, not mine. Reversed in full.
2026-07-29 19:09:10 -07:00
Matthew Jackson e5e3b863fd Distinguish a failed key source from one with no entry
resolve_and_apply_traced collapsed `Ok(_) | Err(_)` into a single
KeyNode::NoEntry step, so a key source that FAILED — server unreachable, keydb
unreadable, malformed entry — was recorded identically to one that simply had no
entry for this disc. The front-end renders that trace, so it told the operator
their disc is not in the database when the real cause was a fixable
infrastructure problem. drive_unit_keys and drive_fmts_indexes were refactored
this cycle to preserve exactly this distinction; this path had not been.

KeyNode gains a SourceFailed variant and the two arms are split. freemkv's
trace renderer matches KeyNode exhaustively with no catch-all, so its arm is
added in the same change — otherwise the consumer would not build.

Also made ETSI TS 102 114 the primary authority for DTS_AMODE_COUNT's comment
rather than a reference decoder internal symbol, and pointed it at this
crate's own cross-checked DTS_AMODE_LAYOUT / DTS_AMODE_CH tables.

A round-2 finding asked for every a reference decoder and a reference decoder citation in the DTS parser
to be stripped as a public-repo hygiene violation. Rejected: the project's rules
(scan-secrets.sh, CLAUDE.md) prohibit internal infrastructure references and
reverse-engineering material, and a reference-decoder citation is neither. The
AMODE channel-count table is a factual table from the standard, not expression
copied from an implementation. Citing the spec plus a corroborating
implementation is how a decodability gate should be justified.
2026-07-29 19:07:15 -07:00
Matthew Jackson 36c1469026 Reference keyframes per track, size the DTS reserve, correct two claims
The ReferenceBlock offset was computed for ANY video track, but the keyframe tick
it measures against was recorded in a single global slot gated to the PRIMARY
video track. On a title with two video tracks — an MVC base plus secondary view,
or a multi-angle disc — a secondary track's non-keyframe therefore referenced a
keyframe on a different track, or 0 (a self-reference) when the primary had not
produced one yet. The tick is now recorded per track, so a non-keyframe can only
reference a keyframe on its own track.

The faststart moov-hole estimate modelled every audio track as (E-)AC-3 at 1536
samples per frame. 1.6.0 added DTS to the writer's carried set, and a DTS core AU
is commonly 512 samples — a third of that — so a DTS track's sample table was
under-reserved threefold and the mux fell back to moov-at-end, losing faststart
on exactly the files 1.6.0 newly supports.

mvc_frame_emits_blockgroup_additional_and_reference asserted only that the
non-keyframe's ReferenceBlock was Some(_). Its non-MVC sibling, added in the same
commit, pins the exact offset; this one now does too, so a mutant emitting a
constant or wrong-signed offset no longer passes.

The comment above the mp4 sample budget claimed file_len stops a crafted file
inflating allocations "past the file's own size". Each indexed sample costs ~52
bytes, so the real ceiling is ~52x file_len (still capped by MAX_SAMPLE_COUNT).
The bound is real; the comment overstated how tight it is.
2026-07-29 19:03:59 -07:00
Matthew Jackson 4aff402e9d Validate stream-selection PIDs per class, not across both
StreamSelection::apply validated a listed PID by scanning ALL streams, so a PID
named in the wrong class's filter passed validation — an audio filter listing a
subtitle PID, say. `keeps` then matched it against the audio streams only, so
the requested track was silently absent from the output. That is precisely the
outcome this validation documents itself as preventing: "fail loud rather than
silently emit an MKV missing a requested track".

Each filter is now checked against its own stream class. The `listed_pids`
helper existed only for the cross-class scan and is removed rather than left
behind as dead code.

Test covers both directions plus the sanity case, and asserts a rejected
selection leaves the title unpruned.
2026-07-29 19:00:53 -07:00
Matthew Jackson fae5282eb9 Halve the per-frame allocation and copy on the m2ts NAL video path
The NAL path called length_prefixed_to_annex_b, which allocates a whole-frame
Vec of its own, then copied the result into a second whole-frame Vec — two
full-frame allocations and two full-frame copies per video frame. The crate
already has append_length_prefixed_as_annex_b, which writes the conversion
straight into a destination buffer; it is the same code path with the
intermediate removed.

The destination is also sized once up front instead of starting from Vec::new(),
which re-grew from zero capacity inside every conversion.

On a UHD HEVC title muxed to m2ts:// — ~200k video frames averaging ~310 KB of
ES at 60 Mb/s — that removes roughly 62 GB of allocation and 62 GB of memcpy.

Behaviour is unchanged: the existing tsmux conversion tests, including the
non-NAL passthrough and Annex-B default pair added last round, all still pass.

A first attempt reused a persistent scratch buffer across frames, which does not
work: the buffer is handed out as Cow::Owned and so can never be returned. A
right-sized single allocation gets most of the win without restructuring the
function around the borrow.
2026-07-29 18:58:39 -07:00
Matthew Jackson 2398a57979 Fix ddts numeric truncation and the decrypt pool's poison asymmetry
ddts CoreSize wrapped to zero on a maximum-size core. core_size is FSIZE + 1 and
FSIZE is itself 14 bits, so the maximum is 16384 — one past what the 14-bit
CoreSize field holds — and push()'s mask turned that into 0, declaring an empty
core frame. Clamped to 16383 instead: one byte short beats telling a decoder
there is no core. Proven red first (the field read back as 0).

ddts avg/max bitrate under-declared every non-integral frame rate. It computed
sample_rate / frame_samples first, so a 512-sample core at 48 kHz truncated
93.75 frames/s to 93. Multiplying before dividing, with round-to-nearest, keeps
the precision.

set_decrypt_threads skipped the pool swap on a poisoned lock while
DECRYPT_THREADS had already been updated, so the new thread count was reported
as taking effect while the stale pool kept serving. decrypt_pool() deliberately
recovers from poisoning for exactly this reason; the setter now does the same.
The pool Arc is immutable once stored, so a prior panic cannot have left it
half-written.

The CoreSize test decodes the value back out of the emitted box rather than
restating the clamp — a first draft asserted the clamp arithmetic against
itself, which would have passed against the unfixed writer.
2026-07-29 18:56:48 -07:00
Matthew Jackson 5ef391f029 Make encrypt_unit report a refused slice, and expand its key once
Two defects in the encrypt_unit promoted to public API last round, both found by
round 2 auditing that new code.

It returned silently without encrypting when the slice was shorter than
ALIGNED_UNIT_LEN. Its own contract requires the caller to set the container's
encrypted flag BEFORE calling — the header is the key seed — so a silent no-op
leaves a unit advertised as encrypted while still carrying plaintext, with
nothing for an authoring caller to check. It now returns bool and is
#[must_use], so ignoring the refusal is a compile-time warning; every call site
was updated to assert on it.

bool rather than Result deliberately: a wrong buffer length is a programming
error at a library boundary, not a disc condition, and a new Error variant would
mean a new numeric code plus its rendering in another repo.

It also drove CBC from the single-block aes_ecb_encrypt, rebuilding the AES key
schedule for each of the 383 blocks in a unit — an order of magnitude slower
than its inverse, which expands the key once via aes_cbc_decrypt. The missing
counterpart aes_cbc_encrypt now exists alongside it, and encrypt_unit calls it,
so the two directions are symmetric in structure as well as in result. For an
authoring caller encrypting a 90 GB image that removes ~5.6 billion redundant
key expansions.

New test pins the boundary: ALIGNED_UNIT_LEN - 1 returns false and leaves the
buffer byte-identical, ALIGNED_UNIT_LEN succeeds. The existing round-trip and
padding-asymmetry tests still pass, so the CBC rewrite is provably the same
transform.
2026-07-29 18:54:19 -07:00
Matthew Jackson ef36213331 Correct six stale comments and doc claims
All six describe code that does something different from what they say, which
is the class of defect that gets a maintainer to write a bug on purpose.

docs/clpi.md presented the CLPI stream-PID entry as byte-aligned 2/2/2/4/4-byte
fields with a 32-bit fine-entry count. It is one 80-bit packed block —
reserved(10) + EP_stream_type(4) + num_EP_coarse(16) + num_EP_fine(18) +
EP_map_start_address(32) — and num_EP_fine is 18 bits. Anyone parsing to the
doc's offsets would read garbage. Replaced with the real bit layout.

docs/udf.md said read_directory()'s recursion cap is 3; MAX_DIR_DEPTH is 8.

TROUBLESHOOTING.md called Pass 1 `recovery::copy`. The engine's `sweep` is
documented as "Pass 1 of a multipass rip"; `copy` is the dispatch verb that
chooses between sweep and patch. This inconsistency was mine, introduced in the
1.6.0 doc rewrite. docs/drive-access.md already said `sweep` and was right — a
round-2 finding claimed the opposite on the grounds that `recovery::sweep`
appears nowhere else in this crate, which it cannot, being in another crate.

io/pipeline.rs cited `disc::patch` as WRITE_THROUGH_DEPTH's caller; that moved
to freemkv-engine in 1.6.0 and no `patch` exists here.

truehd.rs's doc on mlp_major_sync_crc_ok said the trailer is compared
big-endian while the body compares u16::from_le_bytes — and a big-endian
compare was the bug the function was fixed for, so the comment described the
defect rather than the code.

sector/decrypting.rs claimed the decorator owns "the only mutable state (its
call-count cap and spent flag)". DecryptingSectorSource has no such fields and
no KeyFetch field at all in this revision.
2026-07-29 18:50:53 -07:00
Matthew Jackson 4b13fc0a5c Apply stream selection on the live path, and treat a halt as a clean stop
Three defects around MuxOptions in the mux driver.

MuxInput::Live never applied MuxOptions.selection, so a caller's audio/subtitle
selection was silently ignored on the live-drive path while the field's own
documentation said it was applied before the demux pipeline is built. The Iso
and Session arms both apply it; Live now does the same, in the same place —
before resolve_inline_base_map, which is keyed on extents and so unaffected by
pruning the stream list.

The header gate returned Error::MkvInvalid whenever headers had not resolved.
On the prefetch-highway path a halt landing while the pump is blocked in a read
can end the stream as Ok(None) rather than Err(Halted), so the loop breaks with
headers unresolved through no fault of the data. Reporting that as MkvInvalid
tells the consumer its disc is malformed and skips the stop-preserves-staging
path that a clean completed=false triggers. The gate now re-checks halt first.

MuxOptions.selection's doc claimed it was applied without naming the one input
it is not applied to. That exception lived only in an internal comment at the
Url match arm, where a caller reading the public field docs would never see it.
It is now on the field, pointing at InputOptions::selection instead.
2026-07-29 18:48:51 -07:00
Matthew Jackson b2abff9792 Fix the DVD MPEG-audio codec mapping and the fabricated AACS docs
parse_audio_attr mapped DVD audio_coding_mode 2 to Codec::Mpeg1 — the MPEG-1
VIDEO variant. Codec::kind() reports Video for it, so a DVD MPEG-audio stream
was classified and handled as video everywhere downstream. Modes 2 and 3 are
both MPEG audio Layer II (3 adds the MPEG-2 multichannel extension), so both
map to Codec::Mp2. A test now walks every coding mode and asserts each result's
kind() is Audio, so no mode can map to a non-audio codec again.

docs/aacs.md documented an entire keydb-resolving API that does not exist:
ScanOptions::with_keydb, Disc::open_title, reader.read_unit(). None of those
symbols appear anywhere in the crate, and ScanOptions has no keydb field — its
own doc comment says "libfreemkv is lookup-free — it resolves no keys". A
reader following that page would conclude the library reads keydb.cfg, which
inverts the actual design: the caller resolves keys out-of-band through a
KeySource and applies them with Disc::decrypt_with.

The section is rewritten against the real API, and the AacsState table's
`key_source` type corrected from KeySource to KeyOrigin.

Worth recording: the first replacement example I wrote was itself wrong. It
used `input("disc://...")`, which resolve.rs explicitly rejects with
Error::DiscUrlNotDirect — live disc must go through Drive::open + Disc::scan +
DiscStream::new. Every symbol and signature in the committed example was
checked against the source rather than assumed.
2026-07-29 18:47:06 -07:00
Matthew Jackson 77b8e520f8 Stop Mp4Sink losing audio frames and writing an empty sample entry
Mp4Sink::write returned Ok(()) without recording the sample whenever an audio
track's frame would not parse into a sample entry. Two consequences, both
silent: leading audio frames were lost until one frame parsed, and a track
whose frames never parsed disappeared from the output entirely — finish()'s
retain() removed the sample-less trak and the run reported success. That
contradicts this crate's stated policy that a skipped track is never silently
dropped.

The drop was never necessary. audio_entry is read in exactly one place,
build_trak, reached only from build_moov inside finish() — nothing on the write
path consumes it. So write() now records every sample and derives the entry
opportunistically from whichever frame parses first.

That makes build_trak's `audio_entry.unwrap_or_default()` reachable, which
would emit an stsd declaring entry_count=1 around an EMPTY sample entry: a
structurally invalid mp4 returned as success. finish() therefore drops any
audio track it cannot describe, with a tracing::warn! naming the codec and
sample count.

Dropping rather than erroring is deliberate. It matches finish()'s existing
treatment of sample-less tracks, keeps an export whose video is fine from
failing outright, and needs no new error code — a new code would mean a new
i18n key across 29 locale files in another repo, which is not this change's
scope. The track's bytes stay unreferenced in mdat: wasted space in a valid
file, which is the cheaper failure.

Test pins both halves — moov describes only the video track, and the
unparseable audio bytes still reach mdat rather than being discarded at write
time.
2026-07-29 18:44:49 -07:00
Matthew Jackson 5a4631a9e1 Stop the mp4 demuxer dropping tracks silently or inventing sample offsets
Three defects in the mp4:// read path, all of the same family: a damaged
source was remuxed minus a track, or with fabricated data, and the run
reported success.

Silent drops. Eight paths dropped a whole track on malformed input with no
report of any kind, so an mp4:// source missing its audio looked like a clean
run. Each now emits a tracing::warn! naming the track and the missing or
inconsistent table (tracing English is permitted in this crate; the numeric
error codes are unchanged). The non-A/V handler case is debug!, since skipping
a timecode or hint track is normal.

Fabricated offsets. sample_offsets ended with a `while offsets.len() <
sizes.len()` loop that packed unplaced samples after the last known offset.
Those samples have no known location, so the invented offsets made the reader
pull frame data from arbitrary file bytes — the exact "emit garbage" outcome
the stco/stsc presence guards refuse. It now returns the short list and the
caller drops the track.

Short stts. `durations.get(i).unwrap_or(0)` gave every sample past the end of a
short stts a duration of 0, collapsing the whole tail onto one timestamp. That
is the same degenerate timing the `durations.is_empty()` guard was written to
refuse, so the guard now refuses both cases.

Two shared test fixtures were internally inconsistent and only passed because
the reader was lenient: stsz declared 3 samples while stsc placed 1, and the
hostile-stsz fixture's stsc/stts covered a single sample. Both are now
consistent. The hostile fixture keeps its lying stsz count — that lie is what
it tests — but its stsc and stts now cover whatever count survives the
file_len bound, so it exercises the allocation bound rather than the
inconsistency guards.

Two new tests pin the new refusals by mutating the consistent fixture: an stsc
that places 1 of 3 samples, and an stts that covers 1 of 3.
2026-07-29 18:42:38 -07:00
Matthew Jackson 69d96e3f02 Bound the forced-subtitle probe, make it cancellable, and stop re-reading clips
The probe's only natural exit was "every PGS track has shown a non-forced
display set". A genuinely FORCED track never satisfies that, so on the common
authoring — a forced-narrative track for foreign dialogue — the loop read the
title's entire extent set at 2 MiB per call with no byte cap, no time cap and
no halt check. It was also invoked once per title rather than once per distinct
clip, and a disc's playlists overwhelmingly reference the same few clips (main
feature, play-all, seamless-branch variants), so the same physical extents were
re-read 30-150 times. The two defects multiplied: tens of GB, times the
playlist count, off an optical drive.

Reached via ScanOptions::probe_forced_subtitles, whose only consumer is
`freemkv info -v` (freemkv/src/disc_info.rs). The rip path leaves it off. So
the symptom is `info -v` never returning on an ordinary UHD, not a corrupt rip.

Three changes:

  * PROBE_BUDGET_SECTORS caps a probe at 256 MiB. A forced track's display sets
    appear throughout the title, so a bounded prefix classifies it; the budget
    only decides how long we keep looking for a non-forced set before accepting
    the forced verdict.
  * ScanOptions::halt is now plumbed in and checked per chunk, so `info -v` is
    cancellable. The probe previously took no halt at all.
  * A ForcedProbeCache memoises verdicts against the title's exact extent list.
    Keying on byte-identical input means a hit cannot change any result — it
    only removes the re-read.

Verdict application is factored into apply_verdicts so the cached and freshly
probed paths cannot diverge.

Three tests pin the behaviour, each against a reader that counts sectors and
never ends: the budget stops at exactly PROBE_BUDGET_SECTORS, a cancelled halt
reads zero sectors, and a second title with identical extents costs no further
reads while a different extent list still misses the cache.

Also worth recording: the CLI acceptance harness never exercised `info -v`
against an optical drive — it reads ISOs from local SSD, where a full-extent
read is fast enough to hide both defects.
2026-07-29 18:38:25 -07:00
Matthew Jackson 230fc0b322 Give every audio and video codec its own registered Matroska CodecID
MkvTrack::audio's catch-all was `_ => CODEC_AC3`, and ebml.rs defined no
A_AAC, A_MPEG/L2, A_MPEG/L3, A_FLAC or A_OPUS constant at all. A CodecID
names the payload, so any of those codecs was written into the MKV declaring
AC-3 while carrying something else — a player either refuses the track or
decodes noise.

Reachable through two ordinary paths, both verified: ifo.rs:577 maps DVD
audio_coding_mode 3 to Codec::Mp2, so a DVD with MPEG audio muxed to mkv://
produced a track declaring A_AC3 over MP2 bytes; and mp4/read.rs:479 maps the
`mp4a` sample entry to Codec::Aac for an mp4:// source. codec/mod.rs already
has working parsers for MP2, MP3, AAC, FLAC and Opus, so the pipeline carried
these codecs end to end and only the container label was wrong.

MkvTrack::video had the same shape: `_ => CODEC_MPEG2` announced Codec::Mpeg1
and Codec::Av1 as MPEG-2 video. V_MPEG1 and V_AV1 added.

Both catch-alls stay, because MkvTrack::audio/video return Self and have no
error channel, but they are now reachable only by a non-audio/non-video or
Unknown codec routed there in error. Two tests enumerate every real codec of
each kind and cross-check each against Codec::kind(), so a codec added to the
enum later cannot silently inherit another codec's ID.

Both proven red first: Aac declared A_AC3 before the fix.
2026-07-29 18:34:53 -07:00
Matthew Jackson d79cb0d6eb Parse H.264 parameter sets with the avcC parser, not the hvcC one
TsMuxer::write_frame handed every NAL video track's codec_private to
hvcc_to_annex_b. An H.264 track carries an avcC record, whose box layout is
different, so the parser returned None and no parameter sets were emitted —
and params_written was set unconditionally, so it never retried. H.264
muxed to m2ts:// reached the player with no SPS/PPS and was undecodable,
silently: frame_count still advanced and the mux reported success. AVC is
the dominant Blu-ray video codec, so this was not an edge case.

The correct dispatch already existed and was already used by
demux_sink::annexb_param_sets. tsmux simply never got the codec: it knew
only PIDs and a NAL-or-not bool, so it could not tell hvcC from avcC.

Rather than add a second setter, set_nal_video(track, bool) becomes
set_video_codec(track, Codec). One fact decides both the ES framing and the
parameter-set parser, so the two can no longer disagree — and that
disagreement is precisely this defect. The default stays Codec::Hevc, which
is the behaviour the bool's `true` default encoded, so a caller that never
calls it is unaffected.

Proven red first, end-to-end through M2tsStream::create with a real avcC
record: before the fix neither the SPS nor the PPS reached the transport
stream.
2026-07-29 18:32:21 -07:00
Matthew Jackson 77d2198499 Stop documenting the recovery API that 1.6.0 deleted
Disc::sweep, Disc::patch, Disc::copy, SweepOptions and PatchOptions have
zero occurrences in src/ — recovery moved to freemkv-engine — but they were
still documented in 30 places across README.md, TROUBLESHOOTING.md, six
files under docs/, seven src/ doc comments and a Cargo.toml comment.
README.md is the crate's GitHub front page and carried a full multi-pass
code example that cannot compile.

Two of the src/ references were intra-doc LINKS to deleted items
([`disc::Disc::copy`], [`disc::Disc::patch`] in scsi/mod.rs). They produced
no warning on a normal `cargo doc` only because they sit on pub(crate)
items; `--document-private-items` reports both, and they are gone now.

The README example is deleted rather than rewritten against the engine's
API: libfreemkv documenting a downstream crate's API on its own front page
is the drift that produced this, and it cannot even depend on it. The src/
references become plain code spans naming freemkv_engine::recovery::* —
deliberately not links, for the same reason.

docs/rip-recovery.md was 202 lines about relocated code. It now documents
only what this crate owns — Drive::read, SenseFamily, DiscStream's adaptive
batch halving — plus the read-path design constraints, which belong with the
code that enforces them, and points at freemkv-engine/src/recovery/ for the
strategy. api-design.md's module tree is regenerated from the real src/disc/
and src/drive/ layouts instead of hand-patched; it had listed sweep.rs,
patch.rs, mapfile.rs and read_error.rs, none of which exist.

Three stale facts surfaced while rewriting and are corrected: the read
timeouts are 10 s / 60 s, not the documented 1.5 s / 30 s; Drive::reset and
SgIoTransport::reset no longer exist at all, so "no SCSI reset from any read
path" is now stated as the stronger fact it has become; and verify_title,
listed as a progress-emitting operation, was removed entirely.

CHANGELOG.md keeps its references — those are the historical record of the
releases that shipped the API.
2026-07-29 18:04:51 -07:00
Matthew Jackson 95c05d97c9 Assert ReferenceBlock per block instead of scanning for the byte 0xFB
Two tests checked only find_id(&data, ebml::REFERENCE_BLOCK).is_some().
ReferenceBlock's EBML ID is the single byte 0xFB, so that asks whether one
exists ANYWHERE in the output — not which blocks carry one. Inside a
BlockGroup, keyframe-ness is signalled by the ABSENCE of ReferenceBlock, so
"which" is the entire question.

Measured rather than assumed. A mutant emitting a ReferenceBlock on every
BlockGroup, keyframes included — which reintroduces the exact 1.6.0 defect
these tests were added for, every frame reading as a non-keyframe — passed
mvc_frame_emits_blockgroup_additional_and_reference untouched. The
duration-bearing roundtrip test did catch it, via its MkvStream read-back
rather than via the presence check. A mutant corrupting the offset VALUE
was invisible to both.

all_block_groups() parses every BlockGroup in emission order with its
Block flags, relative timestamp, duration and decoded signed ReferenceBlock.
first_block_group() now delegates to it, so there is one walker rather than
two. Both tests assert per block: the keyframe carries NO ReferenceBlock,
each non-keyframe carries one, the offset equals the distance back to the
keyframe, and the SimpleBlock-only 0x80 flag is clear throughout.

Both tests previously took muxer.writer.into_inner() WITHOUT calling
finish(), leaving cluster sizes unwritten — an unparseable file, which is
why they could only byte-scan in the first place. They now write through
SharedWriter and finish(), so the assertions run against a well-formed
Matroska file.

Verified: with the fixes in place, the every-block mutant and the
off-by-one-offset mutant both fail.
2026-07-29 18:00:06 -07:00
Matthew Jackson f40e01de4b Cover the non-NAL video path, including the wiring that selects it
set_nal_video had exactly one production caller and zero test callers. The
branch it gates decides whether a video track's ES goes through
length_prefixed_to_annex_b, and MPEG-2 and VC-1 must not: they are already
start-code ES. Getting it wrong is silent — frame_count still increments,
so the mux reports success while emitting a video-less file.

Five tests, each verified against a real mutant:

  * non-NAL ES passes through byte-for-byte, and the default path still
    converts. Both use a deliberately length-prefix-SHAPED payload so a
    wrongly-applied conversion rewrites the leading four bytes into a start
    code — a payload the converter happened to leave alone would let a
    mutant pass.
  * a non-NAL keyframe arms params_written, so following non-keyframes are
    not dropped by the pre-keyframe guard.
  * set_nal_video rejects an out-of-range track instead of panicking.
  * M2tsStream::create wires a VC-1 track to the non-NAL path.

That last one matters more than it looks. The four TsMuxer-level tests set
the flag themselves, so deleting the set_nal_video loop from
M2tsStream::create left all 2366 tests passing — the exact mutant the
finding named went undetected until a test drove the real wiring. It now
also catches the subtler mutant of widening the matches! arm to include
Vc1.
2026-07-29 17:54:27 -07:00
Matthew Jackson 5172803566 Promote AACS unit encryption to real API, and assert decrypt byte-exactly
encrypt_unit becomes public library API rather than a #[cfg(test)] helper.
Authoring an encrypted disc image is a legitimate use of this crate, and
the capability was already written four times over: a pub(crate) test-only
copy in aacs/content.rs plus three hand-rolled duplicates in decrypt.rs,
sector/decrypting.rs and disc/extract.rs. All four now call one function,
removing ~110 lines of duplicated cipher code that could drift from
decrypt_unit independently.

It mirrors decrypt_unit's purity contract: crypto only, no encrypted-flag
handling, because where that flag lives is container-specific (CPI bits in
byte 0 for BD-TS, elsewhere for HD-DVD-PS). Callers set the flag BEFORE
encrypting — bytes 0..16 are the key seed left in plaintext, so touching a
header byte afterwards changes the key a decryptor derives. That footgun is
documented at the function and at every call site.

Two tests pin it: an exact round trip through both directions, and the one
place the pair is deliberately asymmetric — decrypt_unit restores
all-zero-on-disc packets to zero, and the test proves an all-zero plaintext
packet enciphers to non-zero bytes so it is never mistaken for padding.
That asymmetry was previously only prose.

Two decrypt tests were also weaker than their own names:

  * aacs_clear_trailing_partial_passes_through asserted only is_ok(), so a
    mutant corrupting the clear partial while returning Ok passed. It now
    snapshots the buffer and asserts byte equality, matching the
    none_keys_is_noop pattern already in the file.
  * aacs_decorator_decrypts_encrypted_unit_via_map checked only that 0x47
    reappeared at the 192-byte stride, leaving corruption in the other 6112
    bytes undetected. The plaintext is fully known, so it now asserts
    byte-exact recovery against it.

Both were verified red first by mutating the production path.
2026-07-29 17:50:24 -07:00
Matthew Jackson 0c2dbdba6e Make the ddts speaker mask agree with its own channel count
The ddts box declares both a channel count and a 16-bit speaker mask, and
a decoder may trust either. dts_channel_layout ended in a `_ => 0x0007`
catch-all describing five speakers (C + L/R + Ls/Rs), so for AMODE 6, 7,
and 10 through 15 the box contradicted the count DTS_AMODE_CH declared
alongside it — provoking a downmix or an outright decode error.

AMODE 6 (L + R + S) is the reachable case: its S is a single
centre-surround, not the Ls/Rs pair, so it is three channels and 0x0012,
not four and 0x0006. AMODE 13/14/15 do not occur on retail media, which
ships a 5.1 AMODE-9 core plus an extension substream.

Replace the catch-all with the full 16-entry ETSI TS 102 114 mask table.
Every entry is cross-checked against DTS_AMODE_CH by counting the
speakers its bits imply: sixteen independent constraints, all satisfied,
and that table is itself pinned to the per-AMODE channel counts in ETSI TS 102 114.

AMODE is a 6-bit field, so the reserved 16..=63 are reachable from a
malformed stream. The old `unwrap_or(6)` invented a channel count for
them that no mask could match; parse_dts now refuses those frames rather
than guessing a layout it cannot name.

One existing fixture placed the ext-sync pattern at f[4..8], which
incidentally set f[7]=0x25 → AMODE 20. That test is about where the
pattern sits, not about AMODE, so the frame is now spec-legal (AMODE 9,
48 kHz) with the pattern moved into the payload proper.
2026-07-29 17:43:38 -07:00
Matthew Jackson 1994f42ff8 Drop the comment that promised tests this crate no longer has
The bisect ReadAction regression tests moved to freemkv-engine with the
recovery strategy, but their explanatory block stayed behind — seventeen lines
describing a `let _ = handle_read_error(..)` bug and asserting "the tests below
prove the required ReadAction values are produced". There are no tests below it;
handle_read_error is not even resolvable here any more. Anyone auditing whether
that bug is still guarded would read this and conclude yes. The block moved to
the engine alongside the tests it describes.
2026-07-29 16:36:22 -07:00
Matthew Jackson 4993f5df79 Route conduct reports through GitHub instead of a private address
The enforcement contact was an address on a domain used for internal
infrastructure, which does not belong in a public repository. GitHub needs no
mailbox to exist and is reachable for anyone who can read this file.
2026-07-29 16:07:14 -07:00
Matthew Jackson a10f43841f mux: fix keyframe signalling for BlockGroup frames
Inside a Matroska BlockGroup the SimpleBlock 0x80 keyframe bit is
reserved and is always written as 0; keyframe-ness is carried only by
the presence or absence of a ReferenceBlock child. Both halves of the
round-trip got this wrong:

  - the reader skipped past ReferenceBlock and read the reserved bit,
    so every BlockGroup frame came back as a non-keyframe;
  - write_block_group discarded its `keyframe` argument and never
    emitted a ReferenceBlock, on the assumption that only intra frames
    (PGS subtitles) reached that path.

The MPEG-2 parser stamps a per-frame duration on I, P and B pictures
alike, so all MPEG-2 video is written as a BlockGroup. That makes the
assumption false and left no video frame looking like a keyframe on
read-back. Downstream, mkv:// -> m2ts:// dropped every video frame (the
TS muxer discards non-key video until the first keyframe) while still
reporting success, and mkv:// -> mkv:// and the stdio round-trip failed
E6008, because the MKV muxer opens a cluster only on a track-0 video
keyframe and so wrote nothing at all. HEVC was unaffected: it carries no
per-frame duration, so it takes the SimpleBlock path where the flag bit
is authoritative.

Verified on a real CSS DVD: 841 keyframes out of 11440 video packets
survive a re-mux, matching the I-picture count in the source bitstream.

Also stop running non-NAL video through the Annex-B converter. MPEG-2
and VC-1 elementary streams are already start-code framed, so
length-prefix conversion corrupts them. TsMuxer takes a per-track
nal_video flag, defaulting to the previous behaviour, which M2tsStream
sets from each video stream's codec.
2026-07-29 15:29:48 -07:00
Matthew Jackson 98fe02746b mux: document that the Url arm selects via InputOptions.selection
The Url mux path prunes streams inside input() via InputOptions.selection;
MuxOptions.selection only applies on the File/Session arms. A Url-source caller
must set InputOptions.selection — noting it so a future caller does not put the
selection on MuxOptions and silently keep every track (the bug the GUI hit).
2026-07-28 18:40:47 -07:00
Matthew Jackson 99ac4268b1 1.6.0: remove recovery strategy (moved to freemkv-engine) + trim dead surface
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).

- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
  Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
  classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
  three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
  READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
  consumer), and the DriveSpeed enum (its one live use — set max drive
  speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
  decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
2026-07-28 15:35:19 -07:00
Matthew Jackson 8737c5bb65 changelog: 1.6.0 section (layering API, mux fixes, engine relocation, stream selection); date 1.5.2 2026-07-28 13:54:42 -07:00
Matthew Jackson a124be8a76 mux: end-to-end test that stream selection drops excluded-PID frames
A title declaring two audio PIDs, pruned to one via StreamSelection::apply
before build_iso_pipeline, must never surface a frame from the excluded PID.
Proves the declaration-driven seam end-to-end through the full highway (read ->
demux -> codec parse): the demuxer is built from the pruned title.streams, so
the excluded PID is untracked and skipped, and both track headers and frames
follow the pruned list. Builds on the existing synthetic-TS harness.
2026-07-28 13:36:36 -07:00
Matthew Jackson 8cc868ca54 disc: add DiscTitle audio_streams/subtitle_streams/video_streams accessors
Typed iterators over the audio/subtitle/video streams, cleaner than matching on
the Stream enum for the common iterate-the-tracks case (stream selection, the
desktop UI info panel, disc-info). Additive; all lib tests pass on 1.86.
2026-07-28 13:35:12 -07:00
Matthew Jackson e768d1666b mux: StreamSelection primitive + apply sites for per-title stream selection
The demux pipeline is declaration-driven off DiscTitle.streams (build_demux_state,
DiscStream::new, and the MKV writer all key off that list), so 'which streams to
keep' is already a pipeline capability with no public knob. This adds the knob:

- mux/select.rs: StreamSelection { audio, subtitle: PidFilter::All | Only(Vec<u16>) }
  + apply(&mut DiscTitle): keep Video always, keep Audio/Subtitle whose PID the
  filter lists, prune the rest (and the parallel codec_privates in lockstep);
  error SelectionPidUnknown on a listed PID absent from the title (fail loud, not
  a silently-missing track). Pure; 6 unit tests. Re-exported at crate root.
- Error::SelectionPidUnknown (E6014).
- MuxOptions gains  (+ derives Default now) applied in mux_stream's
  Iso/Session arms before the highway/DiscStream builds demux state (and before
  probe_and_remap's DVD AC-3 PID rewrite). InputOptions gains  applied
  in input()'s iso arm right after the title-index bounds check.

PIDs not languages -- language->PID is caller/engine policy. Default All/All is a
no-op (apply gated on !is_all()), so the no-selection path is byte-identical:
nothing below the title-finalization line changes (ts/ps/demux_thread/
pipelined_stream/mkv/disc untouched). All 2488 lib tests pass on 1.86.
2026-07-28 13:14:40 -07:00
Matthew Jackson 1f643cf91a error: add is_disc_level_no_key classifier (re-exported)
Distinguishes a WHOLE-DISC key failure (E_NO_DISC_KEY / E_KEYDB_LOAD /
E_AACS_NO_KEYS -- every title fails identically) from a per-title skippable
stub. The engine's multi-title loop uses it to fail-fast on the first no-key
title instead of iterating all N. Additive.
2026-07-28 12:45:58 -07:00
Matthew Jackson b299e5d8db error: re-export is_halt + is_skippable_title_stub at crate root
The engine's multi-title rip loop classifies per-title mux failures (halt vs
skippable stub vs hard) using these typed classifiers instead of E-code string
matching. Additive; no behavior change.
2026-07-28 12:31:40 -07:00
Matthew Jackson ffc7f93aeb disc: promote locate_ranges to pub (engine multipass reads it)
Small pub promotion + fmt one-lining. The relocated multipass progress
reporting in freemkv-engine needs locate_ranges externally. No behavior change.
2026-07-28 12:09:38 -07:00
Matthew Jackson 3c53eff257 drive: promote extract_scsi_context to pub
Small pure error->(status,sense) introspection helper the relocated
sweep/patch will need externally. Same category as the prior WritebackFile/
resolve_content_key_map promotions -- infra, not policy. No behavior change.
2026-07-28 11:41:49 -07:00
Matthew Jackson f0dca35a91 io: promote WritebackFile to pub
Bounded-cache buffered File replacement used across mux/extract/sweep/patch
-- general I/O infrastructure, not recovery policy. freemkv-engine's
relocated sweep/patch need to construct it directly once those methods
leave this crate. No behavior change.
2026-07-28 11:36:43 -07:00
Matthew Jackson 4e0c0efbc3 scsi: promote SenseFamily to a lib-level SCSI-fact primitive
Moved SenseFamily::from_sense_key + is_wedge_family from disc/read_error.rs
into scsi/mod.rs (with its own tests) and re-exported at the crate root.
This is pure SCSI sense-code classification -- objective hardware fact, zero
recovery-policy opinion -- so it belongs in the library primitives, unlike
the retry-DECISION state machine (ReadCtx/PassSummary/ReadAction/
handle_read_error) built on top of it, which is freemkv's specific recovery
strategy and is moving to freemkv-engine next.

disc/read_error.rs and disc/section_recover.rs now import SenseFamily from
crate::scsi instead of defining/re-exporting their own copy. No behavior
change. Precommit green on Rust 1.86 (fmt+clippy+test).
2026-07-28 11:32:13 -07:00
Matthew Jackson ac51108e95 disc: promote resolve_content_key_map + encrypted_content_ranges to pub
The upcoming freemkv-engine crate needs both to build the multipass
sweep/patch recovery strategy externally over Disc's public API. Everything
else sweep/patch touch on Disc was already pub; these were the only two
gaps. No behavior change -- visibility only.
2026-07-28 11:26:37 -07:00
Matthew Jackson 8c0eb665b3 mux: fix DTS core-header false-drops + close TrueHD/mux gate coverage
DTS core decodability gate (core_header_drop_reason) — full ETSI TS 102 114
spec-conformance sweep against a reference decoder the spec core-header rules and
a reference decoder parse_frame_header:

- deficit_samples: only require ==32 for NORMAL frames (FTYPE==1). A
  TERMINATION frame (FTYPE==0, the last frame of a stream) legitimately
  carries fewer and is fully decodable; the old unconditional check dropped
  it on every stream that ends on one — a guaranteed per-track silence gap.
  Matches a reference decoder (normal_frame && deficit != DCA_PCMBLOCK_SAMPLES) and
  a reference decoder (branches on normal_frame).
- reserved bit (after RATE): both reference decoders SKIP it (a reference decoder
  skip_bits1, a reference decoder bits_skip1 "Reserved field") and never reject on it.
  Rejecting was a false-drop that silenced any real stream whose encoder
  set the bit. Relaxed to read-and-discard; DropReason::ReservedBit removed.

Swept and confirmed spec-correct as-is (no change): npcmblocks multiple-of-8,
frame_size>=96, audio_mode>=16 (a reference decoder-permissive), sample-rate validity
table (matches avpriv_dca_sample_rates incl 96k/192k at 14/15), LFE flag==3
invalid, PCMR bits table (matches a reference decoder sample_res {16,16,20,20,0,24,24,0}).
Bit-read order verified field-by-field against a reference decoder. bit_rate is left
unvalidated (lenient, never-false-drop direction) as before.

Tests: termination frame with small deficit is kept; normal frame with bad
deficit is dropped; reserved-bit-set frame is kept. make_bad_dts_core now
uses an invalid LFE flag (duration-neutral) instead of the relaxed reserved
bit.

TrueHD: add coverage for the EXTENDED major-sync header CRC path (ms[25]&1,
mshdr=28+2+2n) — previously zero-tested, the exact path a shipped endianness
bug once used to silently drop whole 7.1/Atmos tracks. Trailer is an
independently-computed oracle (separate CRC-16/0x2D, anchored to the 0x4FF7
catalogue value, NOT crc16_mlp), stored little-endian; test asserts accept,
body-corruption reject, and big-endian-trailer reject.

mux driver: extract the finish completion mapping into pure mux_run_completed
so the finalize_failed -> completed=false branch (reachable only via real
write-thread wedge timing) is unit-tested; add an out-of-range
MuxInput::Session title_index test asserting a clean Error::MuxTrackRange
(E9011) instead of a panic.
2026-07-24 10:28:21 -07:00
Matthew Jackson 5e33fa4771 Fix read-fault misclassification, DTS AMODE channel table, and untestable guards
- resolve_fmts_key_map: distinguish a genuinely-not-FMTS disc from a
  transient live-drive read fault. read_filesystem now returns the new
  Error::UdfNotFilesystem for a deterministic tag/format mismatch (no AVDP,
  no partition descriptor, no FSD); resolve maps only UdfNotFilesystem (fs)
  and UdfNotFound (.tbl absent) to Ok(None), and PROPAGATES DiscRead / other
  I/O faults so a marginal AACS 2.1 disc fails loud instead of silently
  dropping forensic content under a base-Unit-Key-only map.

- DTS_AMODE_CH (mp4/audio.rs): extend 10→16 entries
  {1,2,2,2,2,3,3,4,4,5,6,6,6,7,8,8} (the spec per-AMODE channel table / ETSI TS 102 114) so
  the spec-legal high AMODEs that now pass the decodability gate declare
  their true channelcount (AMODE 13→7, 14/15→8) instead of a truncated 6.

- session.rs resolve_keys "called before scan" guard is now testable:
  from_parts_for_test takes Option<Disc>; added a test that a disc-less
  session returns a clean DeviceNotReady Err rather than panicking.

- mp4/read.rs: a track with samples but a missing/malformed stts (mandatory
  per ISO/IEC 14496-12) is dropped rather than emitting all-zero timestamps,
  matching the existing stco/stsc guards; all-tracks-dropped → Mp4Invalid.

- Remove the inert MuxInput::Iso.key_map field (the Iso path re-derives its
  map inside build_iso_pipeline); the live path keeps Live.key_map.

All four fixes are mutation-verified.
2026-07-24 09:59:33 -07:00
Matthew Jackson a7b16e3a1e mux: distinguish FMTS phase-probe read fault from wrong key; cover multi-CPS
Fix 1 (correctness): resolve_fmts_key_map's per-index phase probe read a
single representative segment via read_unit (whose read_sectors(...).ok()?
swallows read errors into None) with no fault fallback. A transient live-drive
read fault (e.g. NOT READY 2/04/3E on the BU40N) made every probe read return
None, giving even==0 && odd==0 — indistinguishable from a genuine wrong key —
so resolve_tie_phase returned FmtsKeyMissing and aborted the entire multi-title
rip even though the forensic index keys were valid and in hand.

Extract the probe into probe_index_phase, which returns Phase / WrongKey /
ReadFault. It mirrors the anchor loop's tolerance: it tries multiple same-index
segments and only concludes WrongKey once a read actually succeeded and decrypted
to no clean parity. If every read of every same-index segment faults it returns
ReadFault; the caller then leaves the phase unresolved so the range-builder
defaults to Phase::All (decrypt both parities, demux drops the garbled alternate
half) instead of aborting. A wrong key can never be masked as a read fault:
ReadFault requires that not a single read succeeded, so zero decrypt evidence.

Fix 2 (test coverage): resolve_mux_key_map's multi-CPS branch was only exercised
with an all-zero source, so pick(), the KeyFetch cold path, and the fail-loud
DecryptFailed guard never ran. Add tests over real AACS ciphertext (via
aacs_encrypt_unit_for_test) covering pick() selecting the correct pool index,
the DecryptFailed guard firing when a clean sample matches no held/fetched key,
and the fetch cold path recovering a missing unit key. Mutation-verified.

Fix 3 (doc): move the reader_event_fn EventKind->MuxEvents mapping doc off
session_mux_keys onto reader_event_fn.
2026-07-24 09:28:09 -07:00
Matthew Jackson 298262f6b3 mux: thread halt into live AACS key-map resolution; cover Session arm
Round-2 follow-ups to 6d6e60f (inline base-map resolve on the live
single-pass Session/Live mux arms).

Fix 1 (halt threading) — the inline resolve chain sampled ciphertext off
the LIVE drive with no cancel token, so an operator /api/stop during key
resolution was not honored (the FMTS probe can issue hundreds of reads,
each able to stall to the 60s SCSI recovery timeout — violating the
"don't hammer a struggling live drive" rule). Add an optional
`halt: Option<&Halt>` to `resolve_mux_key_map`, `resolve_fmts_key_map`,
`resolve_inline_base_map`, and `Disc::resolve_content_key_map`, and poll
it at each loop boundary (FMTS anchor + per-index probe loops, multi-CPS
extent loop) — returning Err(Halted) promptly. Live/Session arms pass the
driver's halt; sweep/patch pass their own token (via Halt::from_arc);
file-backed probe/ISO callers pass None. Tested with a pre-cancelled halt
(Err Halted, no extent sampling) and a None-halt no-abort case;
mutation-verified (dropping the extent-loop check → Ok, not Err).

Fix 2 (Session-arm coverage) — the MuxInput::Session arm ran the same
resolve→install→decrypt sequence as Live but had NO end-to-end test
(DiscSession only exposed open(), which needs live hardware). Add a
#[cfg(test)] DiscSession::from_parts_for_test (injected reader + scanned
disc, no Drive), an end-to-end AACS decrypt test through the Session arm
(mutation-verified: dropping with_key_map → mux aborts), and a
missing-reader clean-error (not panic) test.

Fix 3 (cleanups) — io_error_code: remove the unreachable typed-Error
downcast branch (From<Error> for io::Error stringifies; no path builds an
io::Error holding a typed Error), keeping the stringify parse is_halt /
is_skippable_title_stub rely on. Add a resolve_keys_for test covering the
largest-title sampling branch. Document the patch wedge-exit coverage gap
(TODO) in passn_handler_ab.rs.
2026-07-24 09:04:19 -07:00
Matthew Jackson 2999daf1c9 mux: resolve+install AACS key map on live single-pass (Session/Live)
Under the map-only decrypt model an AACS DecryptingSectorSource decrypts
nothing until a key map is installed; with no map the AACS arm fails loud
with DecryptFailed on the first content unit. The two inline live-mux arms
in mux_stream did not install one:

  - MuxInput::Session (freemkv `rip disc://…mkv`) installed NO map at all.
  - MuxInput::Live (autorip non-FMTS single-pass) installed only a
    caller-supplied forensic FMTS map, which is None for a plain AACS disc.

So EVERY plain AACS Blu-ray/UHD ripped via the live single-pass path failed
DecryptFailed on the first content read. This predates the mux_stream
refactor: the bug was introduced with the map-only decrypt model, and the
pre-refactor CLI likewise built DiscStream::new without with_key_map.

Fix: add resolve_inline_base_map, the inline counterpart to what
build_iso_pipeline does for the file highway. Both arms now resolve the
AACS map off the reader (borrow to sample, then move into DiscStream) and
install it via with_key_map before any read. DVD/CSS keeps DecryptKeys::None
(DiscStream's per-title CSS crack owns it); clear/raw resolve to no map.
Session passes session.key_fetch() so a multi-CPS/orphan unit can still be
recovered; a caller-supplied FMTS map (autorip) is used verbatim, never
re-resolved.

Tests: an end-to-end MuxInput::Live mux over a genuinely-AACS-encrypted
synthetic unit now decrypts and finalises (mutation-verified: dropping the
resolve/install makes the mux abort). Adds a pub(crate) test-only AACS
encrypt helper so the mux test can build a real encrypted fixture, and a
gating test for resolve_inline_base_map (AACS→map, CSS/clear/raw→none).
2026-07-24 08:34:51 -07:00
Matthew Jackson 31a87ae994 Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just
  0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements;
  only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are
  decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal
  multichannel core frames as undecodable, silencing recoverable audio. Add a
  regression test (literal 0..16 range) that fails if the bound reverts to 10.

- keysource: only memoize a NEGATIVE (empty) key-fetch result when every source
  genuinely ran and none held the key — never when a source Err'd (network down,
  unreachable). A transient outage was being cached as a permanent "no key" for
  the fingerprint, permanently dropping a unit that could be recovered once the
  source came back. Thread an `errored` flag out of the drivers and gate the
  cache insert on it. Tests cover both the recover-after-outage case and that a
  genuine absence is still memoized.

- pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS
  display sets through the full demux -> parse -> observe -> apply path, both a
  forced verdict landing and a non-forced verdict clearing a vendor flag.

- mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD).

- scan_iso test: add independent fixture expectations (volume id) so the parity
  test is no longer purely tautological against a re-run of the same composition.
2026-07-24 08:32:37 -07:00
Matthew Jackson 6be489a10a mux/driver: map a mid-read halt to completed=false (Stop is not a failure)
The finish stage and the highway path already treat Error::Halted as a clean
operator stop (completed=false), but the header pump (`stream.read()?`) and the
frame pump (`Err(e) => return Err(e)`) propagated it as a hard error. Since slow
recovery reads dominate wall-clock, a Stop almost always lands mid-read, so it
surfaced as a mux failure instead of a resumable incomplete outcome — breaking the
driver's own "a clean operator stop is not an error" contract and diverging from
the ISO/Url highway (which returns Ok(None) on halt).

Both read arms now route Error::Halted to the completed=false path. Adds
error::is_halt(&io::Error) (typed, mirrors is_skippable_title_stub) as the check.
Two regression tests cover a halt landing mid-header-read and mid-frame-read.
2026-07-24 07:52:42 -07:00
Matthew Jackson 922a9d4403 mux: make write-pipeline send deadline per-call via MuxOptions
The 60s per-frame send deadline the mux driver applies in drive_mux was
hardcoded. That is correct for autorip (its hard watchdog + container-restart
model wants a wedged sink surfaced as a bounded per-frame timeout), but wrong
for the CLI's interactive stdout:// / network sinks: the old inline
output.write() had no deadline, so a slow-but-alive downstream (paused pager,
backpressured pipe, slow peer) was spuriously reported as an interrupted mux
after 60s of backpressure.

Add MuxOptions.send_deadline: Option<Duration>. Some(d) keeps a hard d timeout;
None means no backpressure timeout (block on a live-but-slow sink). None is
resolved to an effectively-unbounded-but-Instant-safe NO_SEND_DEADLINE at the
effective_send_deadline seam. Ctrl-C / halt stays responsive under None because
send_with_halt slices its wait at POLL_INTERVAL and re-checks the halt token
every slice regardless of the deadline value.

Unit-test the routing seam (Some(d) -> d, None -> unbounded, never collapses to
60s) and that MuxOptions carries the knob.
2026-07-24 02:38:30 -07:00
Matthew Jackson b70da49f7f mux: restore three drive_mux regressions from the mux hoist
The mux drive loop hoisted out of autorip/CLI into libfreemkv's
drive_mux dropped three guarantees the hand-rolled loops enforced:

A. Header-buffer OOM cap. The header pump buffered every PES frame
   until codec_private resolved with no byte accounting, so a title
   whose video header never resolves (damaged/undecryptable HEVC on
   marginal UHD) buffered the whole 30-90 GB title into RAM until
   OOM-kill. Restore the 512 MiB HEADER_BUFFER_CAP_BYTES cap autorip's
   run_mux enforced; once exceeded, fail fast with the same
   Error::MkvInvalid the headers-never-resolved gate already returns.

B. Watchdog feed during the header-frame drain. After headers resolve
   the buffered header frames are flushed to the sink, but
   on_write_progress (the sole feed for autorip's hard watchdog) fired
   only in the steady-state loop, leaving the watchdog unfed during a
   long/slow drain and able to false-escalate. Feed on_write_progress
   per buffered frame during the drain, matching the steady-state path.

C. Per-title CSS key on MuxInput::Session. The Session arm set keys via
   disc.decrypt_keys() unconditionally, which returns the largest
   title's VTS CSS key; muxing a bonus title in a different VTS
   descrambled with the wrong key (DiscStream's per-title crack only
   fires on DecryptKeys::None). Special-case DVD to None so the pipeline
   cracks the correct per-title key, matching resolve.rs::input(). AACS
   and clear discs still resolve from decrypt_keys().

Adds mutation-verified synthetic tests for each (no live drive).
2026-07-24 02:27:57 -07:00
Matthew Jackson c52240b81f mux: add MuxInput::Live for the inline live-drive single-pass path
STEP 4c-ii (layering refactor): give `mux_stream` a live-drive source so
autorip's live single-pass mux stops hand-rolling `DiscStream::new`.

`MuxInput::Live` is the live analogue of `MuxInput::Iso`: it wraps the
raw `Box<dyn SectorSource>` in the INLINE `DiscStream` (the same
constructor the `Session` arm uses — NOT `build_iso_pipeline`, the
prefetch highway), so a consumer's adaptive batch-retry in
`DiscStream::fill_extents` still fires on a bad live-drive sector. Unlike
`Session`, `Live` carries an optional pre-resolved forensic `AacsKeyMap`
that is applied via `DiscStream::with_key_map` before any read — required
for single-pass FMTS correctness (read only our-phase units; decrypt the
forensic segment with the mapped key). The consumer resolves keys as its
own policy and hands the banked material in; the driver re-resolves
nothing.

Unit test drives `MuxInput::Live` with a recording `SectorSource` and a
phased forensic key map, asserting the alternate-phase (odd) units are
never fetched — proof the arm builds the inline `DiscStream` and applies
the key map. Mutation-verified: dropping the `with_key_map` call reads the
dropped odd-phase LBAs and fails the assertion.
2026-07-24 01:53:36 -07:00
Matthew Jackson 16663841d0 mux/driver: split on_progress; stage live drive as session reader
Split the ambiguous MuxEvents::on_progress(bytes, total) into two
callbacks — on_read_progress(bytes_read, total) and
on_write_progress(bytes_written, total) — so a consumer no longer has
to guess which side of the pipeline a progress figure came from. The
CLI drives its bar from the write side; autorip from the read side.
The reader EventFn's BytesRead now maps to on_read_progress; the
per-frame emit in drive_mux to on_write_progress. Both keep empty
defaults; NoopEvents and the driver tests are updated to match.

Add DiscSession::stage_drive_as_reader so the live single-pass path can
run through MuxInput::Session: the owned Drive (itself a SectorSource)
moves into the reader slot for mux_stream to take. The drive is now
held as Option<Drive> with the device path cached up front, so the
mux driver can still name the device after the drive has been staged;
the driver's Session arm uses the new device_path() accessor.
2026-07-24 01:01:34 -07:00
Matthew Jackson 5670e77def mux/driver: forward reader-side events through Arc<dyn MuxEvents>
mux_stream took events as &dyn MuxEvents and passed None for the
constructors' EventFn, because that hook is Box<dyn Fn(Event)+Send+
'static> and a borrowed handle cannot satisfy 'static. The reader-side
events (BytesRead progress, SectorSkipped, BatchSizeChanged, ReadError)
therefore never fired.

Take events as Arc<dyn MuxEvents> (MuxEvents: Send+Sync+'static) and
clone it into a 'static EventFn (reader_event_fn) that translates each
real EventKind into the matching MuxEvents call. Wire it into
build_iso_pipeline (ISO path) and DiscStream::on_event (live path).
The write-side on_progress in drive_mux is unchanged.

Add a translation unit test (all four variants) and an end-to-end
mux_stream ISO test that asserts the read-side BytesRead reaches the
Arc; passing None regresses the latter.
2026-07-24 00:37:03 -07:00
Matthew Jackson 0b44020d08 mux: add high-level mux_stream driver (layering step 4a)
Add the shared decrypt+mux driver both consumers hand-roll today, as a
library-only API. `mux_stream(input, dest_url, opts, halt, events)` runs
the construct -> headers gate -> open sink -> pump -> finish pipeline:

- MuxInput::{Session, Iso, Url} selects the source. The file/ISO path
  calls the untouched build_iso_pipeline highway (zero added copies —
  the driver reads frames exactly where consumers call stream.read());
  the live path builds a DiscStream; a URL source goes through input().
- chapters:// / json:// metadata sinks short-circuit BEFORE the header
  pump/gate. They write their whole file from the scanned title and need
  no codec headers; the CLI placed this after the gate, so a metadata
  export on a title whose video headers never resolved failed with
  MkvInvalid. Fixed by construction.
- The header gate refuses a stream with no resolved codec_private
  (MkvInvalid); the zero-output gate refuses an empty/undecryptable drain
  (NoStreams); a halt mid-pump yields completed=false, never a success.
- Frames are written through a WRITE_PIPELINE_DEPTH consumer pipeline so
  the latency-bound sink write overlaps the next read.

Add error::is_skippable_title_stub(&io::Error) so consumers can drop the
E7023/E6008 string-match, and DiscSession::take_reader for the live arm.

Driver body unit-tested in isolation via a synthetic Stream against
null:// / chapters:// / json:// sinks (short-circuit, header gate,
zero-output gate, halt, happy path); each gate mutation-verified.
Consumers are NOT migrated yet (steps 4b/4c).
2026-07-24 00:27:47 -07:00
Matthew Jackson 4c73256bdd Add DiscSession::resolve_keys and resolve_keys_for for base AACS keys
Hoist the AACS base-unit-key resolution glue the CLI and autorip hand-rolled
around the existing primitives into one place: sample the largest title's
ciphertext, run the ordered key sources first-valid-wins, bank the winning
unit keys onto the disc, and build the read-time KeyFetch.

- resolve_keys_for(reader, disc, sources): the free-function core, over any
  SectorSource (live drive or a scan_iso file reader). Returns the structured
  ResolutionTrace plus the KeyFetch; a non-AACS disc is a no-op (empty trace,
  no fetch).
- DiscSession::resolve_keys(sources): samples through the session's reader
  (staged file reader if present, else the live drive), banks onto the
  scanned disc, and retains the KeyFetch on the session (key_fetch()
  accessor) for a later mux.
- KeySourceFactory: the Arc source factory the consumer supplies (libfreemkv
  builds no key sources itself).

Sampling is skipped when the factory yields no sources (resolution is a miss
regardless) — no wasted disc read. Tests cover banking, the no-key path
still building a fetch, and the CSS/None non-AACS no-op.
2026-07-24 00:09:28 -07:00
Matthew Jackson 925f4d56f6 Add scan_iso entry point for file-backed ISO scans
Introduce libfreemkv::scan_iso(path, opts) -> (Disc, Box<dyn SectorSource>),
the file-backed counterpart to DiscSession::scan. It is the single place
that opens a FileSectorSource, reads its capacity, and runs Disc::scan_image,
returning the scanned Disc plus a reusable reader over the same image so
consumers stop hand-rolling that triple.

Add an integration test that materialises a minimal synthetic UDF image to a
real file, asserts scan_iso matches the manual open+scan_image composition,
and confirms the returned reader is still usable (capacity + sector read).
Also covers open-failure and scan-failure error propagation.
2026-07-23 23:48:50 -07:00
Matthew Jackson 4ba2c728b2 libfreemkv: add DiscSession (drive open/bring-up hoist, step 1)
New src/session.rs providing DiscSession, DeviceTarget, and KeySpec,
re-exported from lib.rs. DiscSession::open runs the drive open +
advisory wait_ready/init/probe_disc bring-up (owning the Drive by
value); identify()/scan() are split so consumers keep their own
UI/TMDB sequencing. KeySpec carries consumer-built credentials +
key_sources which scan() forwards into ScanOptions without clobbering
caller-set fields (the library derives no certs and reads no keydb).
Unit tests cover the KeySpec->ScanOptions forwarding.
2026-07-23 23:37:20 -07:00
Matthew Jackson 6f2e9a74bd libfreemkv: 10-phase release audit fixes (v1.5.2..HEAD)
Multi-round audit of the decrypt/AACS/mux-codec refactor. Fixes, in
descending severity:

- mux/mp4/read.rs: bound untrusted-input allocations. `sample_budget`
  now also capped by file_len (a fixed-size stsz claiming count=u32::MAX
  can't inflate the Vec<SampleRef> past the file's own size); trak scan
  capped at MAX_TRACKS matches; find_box() takes only the first match
  (cap=1) instead of materializing every match. Removes dead find_boxes
  wrapper.
- disc/mod.rs: merge_content_key_ranges now UNIONS same-key overlapping
  ranges (coverage-preserving) instead of dropping the non-overlapping
  tail, which silently left encrypted LBAs uncovered -> ciphertext
  passthrough in the whole-disc sweep/patch map. Different-key overlap
  (malformed) still dropped to keep the set disjoint.
- sector/decrypting.rs: remove dead unit_key_idx field + with_unit_key_idx
  setter (vestigial from the pre-keymap trial-decrypt design; AACS is
  map-only now). Fix stale docs.
- decrypt.rs / resolve.rs / error.rs / extract.rs: doc/comment drift from
  the refactor (AacsKeyMap positive-map semantics, resolve_mux_key_map doc
  reattachment, decrypt_sectors_in_content legacy-alias, E_MP4_INVALID
  meaning, multi-CPS orphan by-design note).

Test coverage (all mutation-verified real):
- DTS NeedMore force-flush buffer bound; FLAC/MPEG-audio PTS carry-forward;
  mp4 mdhd timescale=0 divide-by-zero guard, MAX_TRACKS cap, sample-count
  file_len bound, MAX_ALLOC_BYTES cap under inflated file_len.
- resolve_fmts_key_map: extracted filter_addressable_segments,
  resolve_tie_phase, fill_base_key_gaps as pure behavior-preserving
  helpers, each unit-tested (segment filter, phase-tie arms, gap-fill
  gaplessness over every extent).
2026-07-23 22:09:04 -07:00
Matthew Jackson 071b65a64e changelog: document 1.5.2 TrueHD/FMTS/extract/trailing-partial fixes 2026-07-23 16:09:03 -07:00
Matthew Jackson b943f7511c Cache AacsKeyMap key indices; extract + test whole-disc range merge
- AacsKeyMap now derives its distinct key-index set once at construction
  (from_ranges_phased) instead of re-allocating/sorting it on every
  decrypt batch; key_indices() returns the cached slice.
- Extract the whole-disc content-map range merge out of
  resolve_content_key_map into merge_content_key_ranges and cover it:
  sort/disjoint, shared-clip dedup, overlap drop, adjacent-kept.
2026-07-23 16:00:00 -07:00
Matthew Jackson 8d730f3d27 Test hygiene: real AACS decrypt coverage, drop dead recovery fixtures
- Repoint three tautological AACS tests (which passed only via the
  unconditional-Err AACS arm of decrypt_sectors) at the real shipping
  path decrypt_sectors_mapped: out-of-range mapped key index, empty pool
  vs a non-empty map, and a scrambled encrypted trailing partial (the new
  guard) — plus a clear-trailing-partial pass-through and an explicit
  fail-loud safety-net test for AACS reaching the unmapped wrapper.
- Add the missing end-to-end AACS round-trip through the decorator +
  key map (aacs_decorator_decrypts_encrypted_unit_via_map) and a
  mapless-AACS-fails-loud decorator test — the exact class of bug the
  TrueHD probe shipped. Fills the // TODO: AACS round-trip test gap.
- Delete the now-dead recovery test fixtures (encrypt_aacs_unit_bad,
  AnyLbaUnit) left after the reactive key-fetch path was removed; keep
  encrypt_aacs_unit + FixedUnit, now used by the round-trip test.
2026-07-23 15:45:17 -07:00
Matthew Jackson 6e15dbaec7 Fix FMTS per-title resolve, extract multi-CPS keying, trailing-partial guard
- resolve_fmts_key_map: filter segments to those addressable within THIS
  title's extents; a title with no forensic content (menu/extras playlist,
  or a different clip) returns Ok(None) and takes the base Unit-Key/CPS
  path instead of hard-failing FmtsKeyMissing. Previously the first
  non-forensic title aborted the entire whole-disc sweep
  (resolve_content_key_map iterates every title) and blocked muxing any
  non-main title.
- FMTS phase probe: an even/odd is_clean tie now only fails loud when
  BOTH halves are 0 (no clean decrypt). A both-clean tie is source-zero
  padding (is_clean is true for any key on all-zero content) — the key is
  valid, default Even, never abort the rip on a padding-heavy sample.
- extract_tree: multi-CPS discs now build the exact per-CPS content map
  (resolve_content_key_map) instead of a blanket key-0 map that silently
  mis-decrypted every secondary-CPS file into garbage. Single-CPS keeps
  the blanket key-0 map (one key opens every unit, incl. orphan clips).
- decrypt_sectors_mapped: a trailing partial unit that is inside a mapped
  range AND flagged encrypted in its clear seed now fails loud (a CBC
  fragment split across a boundary can't be decrypted) instead of being
  emitted as clear. New aacs_unit_seed_encrypted reads the flag on a
  partial.
- Correct the stale decrypt_sectors doc (AACS arm now always errors;
  AACS decrypts only via decrypt_sectors_mapped).
2026-07-23 15:36:15 -07:00
Matthew Jackson 2cf42ec9f3 Fix TrueHD channel-correction probe on AACS discs; delete dead reactive key-fetch path
The mux's TrueHD 7.1/Atmos channel-correction probe built its
DecryptingSectorSource with no key map. Since AACS now decrypts only via
the resolved map, the mapless probe read failed loud on the first unit
and the correction was silently skipped on every AACS disc — 7.1/Atmos
stayed understated as the MPLS-declared 5.1. The probe now resolves the
same up-front key map the mux read installs (non-fatal on failure).

Delete the reactive per-read key-fetch recovery path
(DecryptingSectorSource::with_key_fetch, the recovery field/cipher
scratch, and sector/recovery.rs). It was unreachable: AACS resolves its
full key map up front, so a mapless AACS read is a bug, not a
recover-mid-read case. KeyFetch itself stays — it is the up-front
resolver fetch used by resolve_mux_key_map.
2026-07-23 14:03:08 -07:00
Matthew Jackson 63334dcc8b Decrypt is keymap-only: sweep/patch/extract, no AACS trial-decrypt
Every AACS decrypt now goes through the resolved key map (decrypt_sectors_
mapped): the map keys each content unit up front and a missing key fails at
resolve time. The old trial-decrypt path — try each held key per unit, keep
the first-tried plaintext on a miss — is gone; decrypt_sectors_impl's AACS
arm now fails loud (reaching it means a reader was built without its map,
which would silently apply a wrong key). CSS (self-descramble) and the clear
no-op path are unchanged.

Disc::sweep and Disc::patch resolve a whole-disc key map up front for a
decrypting pass (the fetch secures any missing CPS-unit key, fail-loud) and
decrypt via the map — clear nav/filesystem sectors are in no range and pass
through, so the separate content-range gate and the reactive per-unit
key-fetch recovery are no longer needed. extract_tree keys every unit with
the base Unit Key through the map (its encrypted-flag gate skips clear
files). Multipass sweeps stay --raw.

Removes the obsolete non-mapped-AACS trial/gate/recovery tests (the mapped
path and resolve fail-loud are tested directly).
2026-07-23 13:24:36 -07:00
Matthew Jackson 07c0a5300a AacsKeyMap: a positive map — no key for a sector means pass through
The key map is now purely "these sectors use this key": entry_for returns
Option and an LBA in no range is left untouched (no default-decrypt-
everything fallback). resolve_mux_key_map builds explicit content ranges
for every case — single-CPS keys each content extent, multi-CPS keys each
extent with the key that opens it, and FMTS fills the non-segment content
with the base Unit Key so a whole-disc read decrypts content and passes
nav/filesystem through. Combined with the fail-loud resolve, a map can
never silently apply a wrong key, and clear sectors are never scrambled.

decrypt_sectors_mapped skips a unit with no map entry; read_plan keeps an
unmapped unit (pass-through content) and drops only alternate-phase
forensic units. Tests updated to the Option semantics.
2026-07-23 12:58:35 -07:00
Matthew Jackson 8cac6111a8 Harden mux + decrypt paths; fail-loud on unresolvable keys
mp4 demuxer (untrusted input): bound every allocation sized from a box
field (stsz/stco/stsc counts, stts/ctts run-lengths, per-sample and moov
sizes, plus an absolute cap so a sparse file can't inflate file_len);
guard the parse_stsd slice and a zero mdhd timescale; cap track count so
the per-track PID can't overflow; rewrite read_moov to handle size==0 /
size<8 / 64-bit largesize; parse esds/AudioSpecificConfig for AAC; write
tkhd duration in the movie timescale.

decrypt: resolve_mux_key_map now fails loud on an extent no key can
classify instead of inheriting the previous extent's key, so a keymap
never silently carries a wrong key; the sweep/patch key-fetch recovery
fails loud when a unit is still unresolved after the retry.

AACS: reject inverted forensic segments in both range builders; compare
the forensic index in u16 space so an out-of-range value can't truncate
onto a valid u8 index. RECOVERED_ERROR no longer latches the damage zone,
preserving the 30s wedge cooldown for a following hard error.

audio: AAC/MP2/MP3/FLAC carry the last PTS across a PES with no timestamp;
the DTS-HD extension-sync search is bounded to after the core; the MP4
16.16 sample-rate field saturates. demux_sink records the video reference
before the kind filter so audio:// / sub:// keep multi-clip PTS continuity
and the DELAY tag.

Remove a dead error variant and the AACS-unsupported-video code; codec
comments cite the primary format specs; assorted doc/naming fixes and
regression tests throughout.
2026-07-23 12:02:43 -07:00
Matthew Jackson 9c144d1980 restore freemkv-unlock path dep for local dev (post-v1.5.2) 2026-07-22 22:19:42 -07:00
Matthew Jackson b2ebae8f81 v1.5.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-22 22:19:39 -07:00
Matthew Jackson b2f35a8afc CSS DVD: resolve the per-title key at read time, drop the scan-time crack
Every DVD read path — the file-backed mux highway (build_iso_pipeline) and
the live-drive single-pass DiscStream — now resolves the per-VTS CSS title
key through one shared step, css::resolve_dvd_title_key, cracked keylessly in
playback order from the title's own extents. Removes the earlier design that
reused a single scan-time key (meaningless for a per-VTS scheme) and muxed a
detection-miss disc's scrambled sectors as garbage.

- Disc::scan no longer cracks a key up front; it does only the CSS bus-auth
  read-unlock, hoisted before the UDF prefetch so scrambled small/menu VOBs
  no longer cost a rejected read each (CSS-DVD scan ~25s -> ~6s).
- An uncrackable title hard-fails (E7023) instead of passing ciphertext as
  plaintext; --raw skips the crack entirely; a Stop mid-crack surfaces Halted.
- DiscStream::new is now fallible and threads raw + halt.
- Fix a stale codec-parser doc claim (TrueHD/FLAC/MP2/AAC do gate via DropTally).
2026-07-22 22:15:27 -07:00
Matthew Jackson a6a4a5d7d2 libfreemkv: 1.5.2 changelog reflects the shipped playback-order crack
The prior wording described the round-1 'self-cracking per-sector' approach
that the audit replaced; rewrite it to match what ships (per-title crack in
playback order + scan-key reuse + loud hard-fail on an uncrackable title).
2026-07-22 13:48:56 -07:00
Matthew Jackson 0de6b82a37 libfreemkv: restore freemkv-unlock path dep on the branch tip
The last release left this pinned to the git tag v1.5.1; release.sh expects
a committed path dep on the branch tip (it swaps path -> git tag only in the
tagged commit, matching bdemu). Restores that state so a local full-workspace
build links the sibling freemkv-unlock, and release.sh's path->tag swap works.
2026-07-22 13:43:32 -07:00
Matthew Jackson b9b0a87328 CSS DVD mux: per-title key via scan-key reuse + playback-order crack
A CSS DVD whose main title was mis-detected as unencrypted (the up-front
crack scanned the largest cell first and starved its budget in that cell's
clear prefix) muxed scrambled sectors as plaintext at exit 0. CSS leaves
the pack/PES header clear, so an un-descrambled sector muxes as a
structurally-valid but corrupt PES packet with zero loss reported.

decrypt_keys_for_title resolves a DVD title's CSS key two ways:
- Fast path: reuse the scan's cracked key when its crack_span covers this
  title's VTS (no re-read; on a live drive no second bus-auth).
- Crack: on a detection miss or a different VTS, crack from the title's OWN
  extents in a SINGLE scan in natural PLAYBACK ORDER (never largest-first).
  One scan = one CSS-locked early-bail, so a locked title is not re-hammered
  per cell against a live drive (hard rule #2); the 50k-sector budget is the
  same accepted bound the disc-wide scan uses. Cracked -> key; Unencrypted
  -> clear; ScrambledUncracked -> hard-fail.
ensure_title_decryptable hard-fails an uncrackable DVD title even when
detection missed, and passes a title that resolved its OWN valid key
regardless of the disc-wide css_error. descramble_region is unchanged from
v1.5.1 (validated-key seed).

Also rename the unlocker report's DVD entry CSS -> DVD. Bump 1.5.2.
2026-07-22 13:07:26 -07:00
Matthew Jackson d6cef9f665 v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:04:24 -07:00
Matthew Jackson b9b9185633 restore freemkv-unlock path dep for local dev (post-v1.5.1) 2026-07-20 17:02:14 -07:00
Matthew Jackson 66951a32ed v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:02:11 -07:00
Matthew Jackson 505a78efd1 mux/codec/truehd: fix MLP major-sync checksum endianness (was dropping the whole TrueHD track)
Regression since the previous release, which added an MLP major-sync checksum
gate to drop genuinely-undecodable audio frames. The checksum itself was computed
with mismatched byte order: `crc16_mlp` is the correct crc_2D table (poly 0x2D,
MSB-first) but returns its two bytes in the OPPOSITE order to libavutil's
`av_crc`, and `mlp_major_sync_crc_ok` then folded in the pre-trailer word
little-endian while comparing the trailer big-endian. The net result never
matched a real major sync, so EVERY major sync was judged corrupt. That armed the
drop-forward on the first AU and, since no major sync ever validated to clear it,
collateral-dropped every following AU forever — the entire TrueHD track was
silently dropped. Its AUs then flushed only at mux end, so the track's blocks
landed physically after all the video: a decoder reading video+TrueHD had to
buffer the whole title to reach the first audio block and spiralled into an
unbounded memory runaway ("decoder ran out of memory"). Every TrueHD title
produced after the gate landed was affected; a title from the release before it
is clean. (The header-size parse — a frequent suspect for extended 7.1/Atmos
headers — is NOT the bug; it already matches ffmpeg's `mlp_get_major_sync_size`
byte-for-byte.)

Fix: compute the checksum exactly as ffmpeg's `ff_mlp_checksum16` —
`crc16_mlp(body).swap_bytes() ^ AV_RL16(word) == AV_RL16(trailer)`. Cross-verified
byte-exact against two real discs (a 7.1/Atmos title and a 5.1 title, independent
32-byte headers both validate). With the checksum correct, major syncs validate
and the drop-forward corruption protection works as intended.

Defence in depth: a major-sync checksum that STILL can't be validated (a genuinely
corrupt or as-yet-unparsed header) no longer arms the drop-forward until we hold a
validated baseline (`num_substreams` from a prior clean major sync) — so a single
bad header can never again silently drop an entire track.

Tests: the `finalize_major_sync` fixture now builds the checksum the corrected way;
a synthetic checksum-failed head major sync is kept, not dropped; the existing
baseline-then-corrupt drop-forward tests still pass. Verified end to end against a
real disc: the TrueHD track demuxes to a full, cleanly-decodable 48 kHz 8-channel
stream, interleaved with the video, instead of 0 bytes.
2026-07-20 16:58:06 -07:00
Matthew Jackson 9b80084220 aacs: parse HD DVD VTKF title keys at the spec's 36-byte stride
The HD DVD Title Key File (VTKF*.AACS) stores 64 title-key entries of 36 bytes
each — 1-byte BIFO + 3 reserved + 16-byte encrypted key + 16-byte binding MAC —
per AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, confirmed byte-exact
against real discs (Freedom VTKF090, Dukes VTKF000: every 36-byte slot has
BIFO=0x80, a clean key, and a 0xFF binding MAC).

The parser used a 32-byte stride (a 12-byte pad in place of the 16-byte binding
MAC) with flag-based termination. That aligns entry #1 (key at offset 132, where
both strides agree) but drifts +4 bytes per entry after it and never terminates
(the previous entry's 0xFF MAC reads as a set present-flag), so it recovered a
correct key only for single-CPS-unit discs and garbage for CPS unit >=2. Every
multi-title HD DVD (Freedom, Harry Potter) was affected.

Fix: 36-byte stride, iterate the fixed 64 slots, take slots whose BIFO AV_FLG
(bit 7) is set, key at offset 4, slot index = CPS unit (skip empty slots rather
than terminate so a gap can't renumber later keys), and never read the trailing
16-byte TKF MAC as a key. Tests rebuilt on the real layout, including a full
64-entry file.

Also correct the VTKF-selection TODO in mod.rs: the AACS HD DVD Book gives the
selector explicitly (match the TKF's PLAYLIST_NAME field to the active
playlist), not the "validate against an encrypted unit" placeholder.

Reconciled against the new HD DVD reference (freemkv.org/docs/hddvd/); the spec
source is archived in private-repo/spec/.
2026-07-20 11:57:13 -07:00
Matthew Jackson b246169e5b aacs: discover HD DVD AACS dir + title-key files instead of hardcoding /ANY!/VTKF000
The HD DVD AACS directory name and title-key filename are chosen by the
authoring house, but the resolver hardcoded a single spelling
(/ANY!/VTKF000.AACS, /ANY!/MKBROM.AACS, /ANY!/CONTENT_CERT.AACS). Real discs
diverge: Freedom (Memory-Tech) names its AACS dir AAC! and ships VTKF090.AACS
+ VTKF100.AACS; Harry Potter carries VTKF000/001/002/099. On such a disc the
hardcoded path finds nothing, so no MKB/title-key/cert is read and decryption
silently can't engage.

Replace the fixed HD DVD path constants with structural discovery:
- find_hddvd_aacs_dir() locates the AACS dir as the root child dir ending in
  '!' that contains MKBROM.AACS (so the ..._BAK mirror is skipped; the dozens
  of decoy advanced-content '!' dirs are excluded by the MKBROM.AACS guard).
- role_paths(udf, role) builds the ordered candidate list per role: the static
  BD/UHD /AACS/ paths first, then the discovered HD DVD files — MKBROM.AACS,
  CONTENT_CERT.AACS, and every VTKF*.AACS (sorted), not just VTKF000.
- read_first() is now generic over &str / String so it takes the Vec<String>.

BD/UHD unaffected (no '!' dir → discovery returns None, list is the /AACS/
constants exactly as before). Verified on real Freedom (AAC!/VTKF090+100) and
Dukes (ANY!/VTKF000) ISOs; unit tests cover both shapes.

Open item (TODO(hddvd-encrypted)): when a disc has multiple VTKF variants the
correct one must be chosen by validating its VUK-derived key against a real
encrypted unit rather than first-that-reads. Blocked on obtaining a genuinely
encrypted HD DVD image — all HD DVD ISOs on hand are already-decrypted rips.
2026-07-20 11:29:28 -07:00
Matthew Jackson 2d96a3e87d release: tee up 1.5.1 (marginal-read reporting fix) 2026-07-19 20:47:52 -07:00
Matthew Jackson bae5a9ea07 drive: report marginal reads (PER=1) so a dirty disc can't pass clean
A read was only ever judged "good" by the drive's SCSI GOOD status — with no
integrity check on the bytes. On dirty/marginal media a drive can silently
return best-effort ECC data (occasionally mis-corrected) with GOOD status, so a
rip could "pass clean" yet decode with errors, pushing corruption downstream to
the mux instead of catching it at the read.

Enable recovered-error REPORTING at drive-prep: MODE SELECT the Read-Write
Error Recovery page with PER=1 (TB on / DTE off so the data still comes back),
preserving the drive's own retry count. Marginal reads now surface as
CHECK CONDITION / RECOVERED ERROR. Best-effort — a drive that doesn't honor it
keeps its defaults (no regression), and on a clean disc nothing changes.

A recovered error is distrusted: it marks just that ECC block NonTrimmed for a
Pass N re-read (a clean re-read wins; a persistent marginal becomes an honest
concealed gap), and is counted in the pass summary. Crucially it takes a
per-block SkipBlock, NOT the damage-jump path — a single marginal sector must
not trigger a 64 MB skip that would discard good data on a lightly-smudged disc.
2026-07-19 20:30:21 -07:00
Matthew Jackson 86ed27508d changelog: 1.5.0 release date + trim to app-level detail 2026-07-19 17:34:21 -07:00
Matthew Jackson f28d79b51b restore freemkv-unlock path dep for local dev (post-v1.5.0) 2026-07-19 15:37:27 -07:00
Matthew Jackson 5b27b9f8a5 v1.5.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-19 15:37:25 -07:00
Matthew Jackson 405e9d32c3 changelog: drop-on-undecodable audio + forced PGS detection (1.5.0) 2026-07-19 14:29:57 -07:00
Matthew Jackson 21b362c6e8 disc: detect forced PGS subtitles from stream content for info
Give `info` the same forced-subtitle verdict the muxer derives during a
rip, so the two agree. A shared classifier (mux::codec::pgs::ForcedTracker)
folds a PGS track's display sets — forced iff every one carries the
forced_on_flag — and is used by BOTH the MKV writer and a new scan-time
probe that reads the title's PGS streams (reusing the TS demuxer and PGS
parser). The probe only overrides a track it actually observed content
for, so an undecrypted/unread stream keeps its vendor-derived flag. Gated
behind ScanOptions::probe_forced_subtitles (off for the rip path, which
detects forced while muxing without a second read).
2026-07-19 14:23:04 -07:00
Matthew Jackson 9f8df95513 mux/codec/mpegaudio: keep free-format frames
Free-format MPEG-audio (bitrate_index 0) is a legal, decodable mode — the
decoder derives the frame size from the sync spacing. Dropping it was a
false positive on a clean stream, so it now passes the gate.
2026-07-19 14:23:04 -07:00
Matthew Jackson 2f465f4cfe mux/codec/truehd: fix drop-forward poison + rate/resync validation
Three TrueHD state-machine fixes from an adversarial audit:

- A corrupt access unit drops forward to the next major sync, but only
  the individually-verified corruption now feeds the whole-track poison
  verdict; the resync run is collateral. A couple of transient errors can
  no longer poison and discard an otherwise-good multi-hour track. The
  shared drop tally gains a verified/collateral split for this.
- The per-AU PTS rate is refined only from a CRC-validated major sync, so
  a corrupt major sync whose rate nibble decodes to another rate family
  can no longer shift the resumed audio — a drop stays a silence gap.
- The resync clears only on a CRC-validated major sync, never on a runt
  too short to hold and validate its header.
2026-07-19 14:23:04 -07:00
Matthew Jackson 3fdd463794 release: 1.5.0 2026-07-19 13:42:48 -07:00
Matthew Jackson 6a7392ac8c mux: detect forced PGS subtitles from the stream
Flag a PGS subtitle track FlagForced when it displays subtitles and every
one carries the HDMV forced_on_flag (a dedicated forced/narrative track),
independent of the disc's vendor label metadata. The track header
reserves a FlagForced byte up front and it is promoted at finish() from
the accumulated display-set state. Only ever promotes — a track already
forced from the playlist metadata is never demoted.
2026-07-19 13:42:48 -07:00
Matthew Jackson 28bdecd26b mux/codec: drop undecodable audio frames, keep A/V sync
A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch. Sync is preserved — a drop becomes a silence
gap, never a shift — and every drop is logged. Detection is per-codec,
each mirroring the format's authoritative integrity check:

  DTS          core-header validity gates
  AC-3/E-AC-3  native frame CRC-16 + bitstream-id range
  FLAC         whole-frame CRC-16 residue
  MP2/MP3      header sanity + free-format reject
  AAC-ADTS     header sanity (raw AAC passes through untouched)
  TrueHD/MLP   major-sync CRC-16 + AU parity; corrupt AUs drop forward
               to the next major sync, since decode state carries
               across access units

LPCM and video are excluded by design (no in-frame integrity data;
inter-frame prediction). A shared DropTally handles counting, logging,
and a whole-track fallback for a mostly-undecodable track.
2026-07-19 13:42:48 -07:00
Matthew Jackson d29d3d71b9 dts: emit clean core alone when the extension boundary is garbage
DTS-HD MA access units are a lossy core frame followed by trailing
extension substreams up to the next core sync. On source-damaged discs
(observed on the Bourne UHDs) the bytes where the XLL extension belongs
are neither a core sync nor an extension sync -- pure garbage -- which
desyncs ffmpeg's XLL decoder and cascades into 'Read past end of XLL
band data' / 'DSYNC check failed' across the whole track.

next_core_boundary now distinguishes three boundary states via a new
ext_clean flag on NextCore::Found:
  - precise/recognized extension sync  -> ext_clean=true  (keep full AU)
  - garbage at the boundary byte       -> ext_clean=false (drop the ext)

When ext_clean is false we emit the DTS core alone (drop the smallest
junk piece, keep the frame and its PTS) and drain past the garbage to
the next core. Recognized-but-unsizeable extensions still ride the
heuristic scan and are kept intact, so lossless tracks are unaffected --
only genuinely corrupt extension bytes are dropped.

Bourne s1.dts: 1606 damaged frames / 691620 (0.232%), 93% isolated
single frames, worst run 3 in a row (~32ms lossy blip).
2026-07-19 09:27:28 -07:00
Matthew Jackson 999898a438 changelog: faststart default + DTS/DTS-HD audio (mp4://) 2026-07-18 22:35:37 -07:00
Matthew Jackson b4470ccd81 mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box
(sample rate, channel layout mask, core size, computed bitrate, LFE);
whole access units (core + DTS-HD extension substreams) pass through, so
an HD decoder finds the extension. dtsh when an extension sync follows the
core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR.

Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is
byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1)
and decodes clean (exit 0). Channel layout correct.
2026-07-18 22:18:35 -07:00
Matthew Jackson 668dd95237 mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free
box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample ×
est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes
mdat, sample offsets are fixed from the start — no rewrite, no offset
patch. finish() writes moov into the hole and pads the slack with a free
box; if the estimate is blown it falls back to moov-at-end. Streams over
HTTP without a pre-fetch. Reserve-math + box-order tests added.
2026-07-18 21:43:44 -07:00
Matthew Jackson 777d9a04a7 changelog: mp4:// input demuxer (1.5.0) 2026-07-18 21:22:41 -07:00
Matthew Jackson b717044ae5 mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel
info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/
composition timing, stss → sync), rebuild a DiscTitle, and emit samples
as PesFrames in global decode order. Video NALs are length-prefixed in
MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into
input() so mp4:// flows to every sink (mkv://, audio://, json://, …).
In-memory write→read round-trip test proves symmetry (streams, hvcC,
sample sizes survive). Progressive MP4 only; fragmented (moof) is future.
2026-07-18 21:10:15 -07:00
Matthew Jackson 379767071c mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps
the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow /
NFS staging avoids the dirty-page burst. The mdat backpatch is an
ordinary seek WritebackFile already handles (seek_then_patch_roundtrip).
Tests switched to in-memory Cursor writers.
2026-07-18 20:57:44 -07:00
Matthew Jackson 6a06443dd8 changelog: mp4:// native MP4 muxer (1.5.0) 2026-07-18 20:44:18 -07:00
Matthew Jackson 42df399156 mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio
track. Audio sample entries built from the first frame's bitstream —
AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for
fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio
durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle
(mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3;
TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the
CLI can report exclusions — never a silent drop.

Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3
both frame-exact on every track (video 2650/4270, audio 5567/3454),
duration and colour identical, clean decode. On a pathological 4-clip
title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more
of them) — source-inherent, not a muxer defect.
2026-07-18 20:37:10 -07:00
Matthew Jackson 9ede4b9afd mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into
a 64-bit mdat and building the sample tables in memory, patched at
finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs
(already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts,
CFR-derived decode timeline (pipeline carries presentation PTS only), and
a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the
hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping.

Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and
HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact
(bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land
in M2.
2026-07-18 20:16:19 -07:00
Matthew Jackson 67e3e39499 mux: add video:// sink (video-only per-track elementary streams)
Completes the per-track-class trio with audio:// / sub://. video:// is
the demux path with a TrackKind::Video kind filter — each video track to
its own native elementary stream (.hevc/.h264/.vc1/.m2v/.obu), no audio
or subtitles. Reuses the existing kind_filter machinery and extension
map; scheme/parse/output/write-only-guard arms added symmetrically.
2026-07-18 19:24:59 -07:00
Matthew Jackson 42be09f628 changelog: 1.5.0 — extraction sinks + complete json:// model 2026-07-18 19:13:30 -07:00
Matthew Jackson 8bbbe43724 mux: json:// emits the complete title model (no dropped fields)
The json:// sink previously emitted only codec + language + pid per
stream. Fill it out to the full DiscTitle: video carries resolution
(+ pixel dims, interlaced), frame rate (+ fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channels (+ count),
sample rate (+ Hz), and editorial purpose; subtitles carry the
qualifier. Add the clip list and chapter names. This is everything the
scan resolved, so json:// is a lossless machine-readable view of a
title rather than a summary.
2026-07-18 18:14:11 -07:00
Matthew Jackson 95888f27a4 test: pin audio://sub://chapters://json:// in the scheme round-trip 2026-07-18 17:46:08 -07:00
Matthew Jackson 96dec65040 mux: chapters:// and json:// metadata sinks
Two write-only file sinks that ignore the PES stream and emit the title
metadata at construction (fvi-style, wired through output()):
- chapters:// — chapter markers in the format the extension picks: .xml
  (Matroska, default), .txt/.ogm (OGM simple), .vtt (WebVTT). Reuses the
  existing chapters_xml/chapters_ogm writers; adds a WebVTT writer.
- json:// — one title's model (playlist, duration, size, format, streams,
  chapters) as pretty JSON via serde_json.

New mux/meta_sink.rs; StreamUrl gains Chapters/Json; codec_label +
chapters_xml/ogm promoted to pub(crate) for reuse.

Tests: chapters_format_selected_by_extension, title_json_carries_streams_and_chapters.
2026-07-18 17:45:39 -07:00
Matthew Jackson 378fb226b7 mux: audio:// and sub:// sinks (demux filtered by track class)
Two extraction sinks built on the existing demux machinery: audio:// keeps
only audio tracks (native containers .thd/.dts/.ac3/.eac3/.pcm...), sub://
only subtitle tracks (PGS .sup, VobSub .idx+.sub, text .srt) — one file per
track, no video, no chapters sidecar. DemuxOptions gains a kind_filter; the
DemuxSink writers (already complete) are reused verbatim. StreamUrl gains
Audio/Sub variants; output() builds the filtered DemuxSink.

Test: kind_filter_keeps_only_the_selected_class.
2026-07-18 17:24:28 -07:00
Matthew Jackson 42caa2d90e restore freemkv-unlock path dep for local dev (post-v1.4.5) 2026-07-18 16:08:03 -07:00
Matthew Jackson 41a2493429 v1.4.5: bump version (freemkv-unlock git-pinned for the tag) 2026-07-18 16:08:01 -07:00
Matthew Jackson 0e73650440 changelog: 1.4.5 (FMTS clean single-variant mux + key-Debug redaction + hex fix) 2026-07-18 13:30:36 -07:00
Matthew Jackson bcfd5eb213 mux: FMTS read-plan on the inline live-drive path too (DiscStream::with_key_map)
The read_plan fix landed on the file-backed highway (build_iso_pipeline)
but the inline DiscStream (live single-pass) still passed the alternate
device-group half to the demux — a known FMTS bug on rip_mode="single".

DiscStream::with_key_map installs the proactive map and rewrites the
extent walk to the read plan, so the live single-pass path reads ONLY
our-phase units, exactly like the highway. DecryptingSectorSource gains
set_key_map for the already-constructed decorator. Non-forensic maps
return the extents unchanged, so every non-FMTS disc is byte-identical.

Test: with_key_map_reads_only_our_phase_units — a forensic Even segment
drops exactly its alternate units from the extent walk. Precommit green
on 1.86.
2026-07-18 13:27:48 -07:00
Matthew Jackson 8ce0a66bee mux: FMTS reads only our-phase units (AacsKeyMap::read_plan)
An FMTS forensic segment interleaves our device group's variant with a
foreign group's at the aligned-unit level. The mux was decrypting only
our phase but leaving the alternate (foreign) units in the buffer as
ciphertext, trusting the demux to 'drop untouched ciphertext cleanly'.
It doesn't: random 0x47 bytes at the 192-byte stride hit tracked PIDs,
mis-parse, and trip the demux's concealed-gap keyframe-resync — which
drops GOOD frames of ours around every segment (349 resyncs / ~6391
packets on Stand by Me, visible as playback flaws).

The map already knows which unit each LBA is and, for a forensic
segment, which phase is ours. AacsKeyMap::read_plan turns that into the
title's read plan: every default/CPS unit, plus inside a segment ONLY
our-phase units. The alternate units are never fetched, decrypted, or
handed to the demux — the demux sees one gapless our-variant stream.

- read_plan is general (single-CPS, multi-CPS, FMTS): a map with no
  Even/Odd range returns the extents unchanged, so DVD/CSS, single-CPS
  UHD and multi-CPS Blu-ray read byte-for-byte as before.
- Wired into build_iso_pipeline (the file-backed highway that muxes
  resumed ISOs). Producer re-anchors unit_base per extent, so per-unit
  segment reads stay unit-aligned and decrypt correctly.
- Extent gains PartialEq/Eq for the read_plan tests.

Tests: read_plan non-forensic unchanged; forensic omits exactly the
alternate units, kept units match the decrypt gate unit-for-unit. All
2314 lib tests pass; precommit green on 1.86.

Pending: end-to-end ISO re-mux validation (concealed gaps 349 -> ~0).
2026-07-18 12:06:20 -07:00
Matthew Jackson 7d4d9a3e84 remove user-facing English from the library (KeyOrigin::name, hddvd Title)
Library holds ZERO user-facing English (CLAUDE.md). Removed KeyOrigin::name()'s
English prose — apps map the typed enum (freemkv gets key_origin_label); diag
logs the enum's Debug repr. hddvd unnamed-title fallback 'Title N' -> neutral
TITLE_N identifier (UDF volume-label style).
2026-07-17 21:50:09 -07:00
Matthew Jackson 4dec71a867 narrow leaked-internal pub surface to pub(crate)
Verified zero callers across all consumer crates AND libfreemkv integration tests:
MUX_APP, Disc::{aacs_disc_hash,encrypted_content_ranges,inject_unit_keys},
locate_ranges, mapfile::{MapEntry,entries}, diag::dump_mkv_track,
DiscStream::{errors,lost_bytes} (read via accessors). Removed the unused mux
DemuxSink/FviSink crate-root re-exports (constructed internally by output() via
the direct module path). Staged demux option variants marked allow(dead_code).
2026-07-17 21:45:33 -07:00
Matthew Jackson a9b29c7060 disc: collapse decrypt_keys_for_title twin into one method
Removed the test-only unchecked wrapper and renamed decrypt_keys_for_title_checked
-> decrypt_keys_for_title (one method returning (keys, title_is_clear)). Production
already used the checked form; the _checked suffix had no counterpart. Callers +
docs updated; two tests take .0.
2026-07-17 21:41:10 -07:00
Matthew Jackson 0cb525517b css: remove redundant crack_key_halt wrapper
crack_key_halt had no caller except the crack_key wrapper — a needless middle
layer. crack_key now calls crack_key_scan directly; crack_key (Option) and
crack_key_outcome (full CrackOutcome + halt) remain as the two real entry points.
2026-07-17 21:34:37 -07:00
Matthew Jackson 5b4a3f32dc mux: collapse FviSink::create/create_with_source into one create
create_with_source (full SourceInfo) was only ever called by the create wrapper
with a partial SourceInfo — the extra capability was unused (YAGNI). Inlined the
constructor body into create and deleted the _with_source variant; callers
unchanged.
2026-07-17 21:32:39 -07:00
Matthew Jackson 66b0bc385c mux: collapse MkvStream::create/create_at into one create(.., Option<&Path>)
One method per action (CLAUDE.md), not a foo_with_X pair — create_at was the real
constructor and create was a None wrapper. Callers pass Option<&Path> directly.
2026-07-17 21:31:29 -07:00
Matthew Jackson aaccd83750 hex: canonical hex->integer parsers + public strip_hex_prefix
Adds parse_hex_u16/u32/u8 and exposes strip_hex_prefix so callers stop hand-rolling
from_str_radix(trim_start_matches("0x")) — a case-sensitive strip that this module
exists to prevent. disc::aacs_disc_hash now uses strip_hex_prefix.
2026-07-17 21:25:59 -07:00
Matthew Jackson 40f5b01a12 disc: redact Debug for AacsState/Key/HandshakeResult (test-guarded)
AacsState (public via Disc.aacs) and Key (the decrypt_with key-transport enum)
are crate-root re-exported and carried VUK/unit/read-data keys + volume id on
#[derive(Debug)]; HandshakeResult carried the VID + AACS 2.0 bus key. Manual
Debug impls print shape only, guarded by red->green tests.
2026-07-17 21:10:51 -07:00
Matthew Jackson 82d42b1930 aacs: redact Debug for ResolvedChain/ResolvedKeys/ProcessingKeyMatch (test-guarded)
These carry raw unit-key / VUK / processing-key bytes on their Debug; manual impls
print shape only (unit_keys_len, redacted markers). Each has a red→green test.
2026-07-17 21:03:31 -07:00
Matthew Jackson 913b01b79f css: redact CssState Debug (test-guarded)
CssState is reachable via the public Disc.css field; #[derive(Debug)] leaked the
raw CSS title key on any {:?} of a Disc. Manual Debug prints crack_span only.
2026-07-17 21:00:34 -07:00
Matthew Jackson af7ef59f3e aacs/types: redact Debug for all key-bearing types (test-guarded)
DeviceKey/HostCert/Vid/MediaKey/Vuk/ProcessingKey/UnitKey/DiscEntry carried key
material on #[derive(Debug)] — a stray {:?} would leak device/host-private/media/
volume/unit keys. Manual Debug impls print shape only. redaction_tests asserts no
key bytes appear and a redaction marker is present (fails if a derive returns).
2026-07-17 20:56:38 -07:00
Matthew Jackson ef8e11b6f3 libfreemkv: phase-aware FMTS decode + two-operation KeyFetch/KeySource seam
FMTS (AACS 2.1) now decodes per (LBA, phase): Phase enum + AacsKeyMap::
from_ranges_phased, decrypt only the variant's parity half. resolve_fmts_key_map
does a 2-phase index-1 anchor then per-index phase probe, and sizes the forensic
set to whatever the source returns (no hardcoded 32). KeyFetch is now two explicit
operations (unit_keys / fmts_indexes) and KeySource splits get_uk into
get_unit_keys + get_fmts_indexes. BYPASS_FMTS_KEY gate removed (first-class format).

Teed up for 1.4.5. Local WIP baseline.
2026-07-17 20:51:15 -07:00
Matthew Jackson e1f897fb32 restore freemkv-unlock path dep for local dev (post-v1.4.4) 2026-07-16 21:44:08 -07:00
Matthew Jackson 602e44fde9 v1.4.4: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:44:05 -07:00
Matthew Jackson 622008650c keysource: add DecodeSampleSet — a >=MIN_SAMPLE_UNITS-by-construction sample set so an online request can't be built under-sized 2026-07-16 21:43:02 -07:00
Matthew Jackson c508484451 restore freemkv-unlock path dep for local dev (post-v1.4.3) 2026-07-16 21:06:04 -07:00
Matthew Jackson cf16f66e07 v1.4.3: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:06:01 -07:00
Matthew Jackson adcab75409 aacs: sample only index-1 forensic segments for the online key query; hoist MIN_SAMPLE_UNITS into the base crate 2026-07-16 21:01:16 -07:00
Matthew Jackson 61e4095727 FMTS: resolve the index key map from one forensic keyserver query
The keyserver protocol now returns all 32 index keys as an array for a
forensic content sample (and a single-element array for plain content).
resolve_fmts_key_map sends one forensic batch and maps array element i to
segment index i+1, replacing the per-index blind-probe collection loop
that repeatedly hit the key service. Segment/index parsing and the
aligned-unit content classification are reworked to support this:

- rename variant_select -> index_select (per-index, not per-variant)
- content classification moves to is_clean(buf, ContentFormat) so the
  unit selector emits only units the key service accepts
- segment.rs: parse IndividualSegment.tbl index tags + SPN ranges,
  build contiguous LBA key ranges from the resolved 32-key array
- decrypt/decorator plumbing for the resolved per-index keys

Fail loud (FmtsKeyMissing) when the forensic query returns < 32 keys or
any segment index stays unresolved.
2026-07-16 19:41:44 -07:00
Matthew Jackson 540f84c7d0 restore freemkv-unlock path dep for local dev (post-v1.4.2) 2026-07-15 19:37:54 -07:00
Matthew Jackson 82c4e018d5 v1.4.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-15 19:37:51 -07:00
Matthew Jackson 08e962694f Mux decrypt/verify redesign, HD DVD first-class, MVC 3D
decrypt:
- decrypt_sectors is now a pure decrypt (apply key, leave plaintext, report
  unverified bytes); TS-structure is a separate primitive (is_clean_ts/ps) used
  only for key selection and read-verify. The mux passes decrypted bytes through
  (the demuxer drops non-conforming packets), ending the NULL-TS conceal loop and
  the per-unit key-server refetch storm. Key-proof floor replaces the 75%
  supermajority.

recovery:
- Removed the post-read decrypt-verify gate (verify.rs) that mis-aligned the
  disc-absolute unit grid against clip-anchored AACS units and false-failed good
  clips (e.g. Dunkirk's orphan-CPS clip). Bad sectors are marked by physical read
  result; decryptability is proven at scan + mux time.

HD DVD (first-class AACS):
- Role-based candidate-list file sourcing so an HD DVD's /ANY!/ files
  (MKBROM.AACS, VTKF000.AACS, CONTENT_CERT.AACS) are found with no disc-type
  branch. parse_vtkf parses VTKF000.AACS into the same UnitKeyFile as a BD
  Unit_Key_RO.inf, so the shared VUK unwrap applies unchanged. set_unit_base
  clip-anchoring. Two decrypt-axis assumptions remain UNVERIFIED-HDDVD-DECRYPT
  (no encrypted disc to test).

mux:
- MVC (Blu-ray 3D) track signals unified into one MVCDecoderConfigurationRecord;
  release-safe track_vint (3-byte VINT) and pid_index (i32) guards.

hardening:
- Container-aware is_clean / encryption detection; bytes_bad_in_title fail-safe
  on a corrupt mapfile; CSS crack gated on DiscFormat::Dvd (HD DVD excluded);
  non-vacuous CSS tests; patch NOT_READY/HARDWARE/ILLEGAL_REQUEST/ABORTED
  sense-path tests.
2026-07-15 19:35:12 -07:00
Matthew Jackson 37e563b8ef Mux: pure decrypt, policy at the caller (no null, no key-server storm)
decrypt_sectors is now a pure decrypt — apply the CPS unit key, leave the
plaintext, report how many bytes did not reach clean TS ("unverified"). It
never restores ciphertext, nulls, or re-fetches. "Did a key produce clean TS?"
is a key-selection / read-verify signal, not the verdict "did we decrypt?": a
correct key can decrypt a bad-encoded region, and broken TS is a muxer concern
(the demuxer drops the packet and resyncs).

Callers own the policy:
- mux (read > decrypt > mux): pass the decrypted bytes to the muxer, whatever
  they are; fail loud only on a genuine can't-decrypt (no key / misaligned).
- sweep/patch (reading from a disc): an unverified unit is a bad read — recover
  a fresh key and retry, or fail loud so disc-recovery re-reads it.

Removes three duplicated decisions — the decrypt-time ciphertext restore, the
mux NULL-TS conceal loop, and the per-unit key-server refetch — plus the dead
aacs_unit_still_ciphertext predicate. Key-fetch recovery now samples the on-disc
ciphertext explicitly (a pure decrypt leaves the buffer plaintext) and lives
only on the rip/verify path, never the mux.

Fixes the 30-90s/region mux stalls and key-server storm on bad-encoded UHD runs
that 1.4.1 left behind (it relaxed the gate but not the surrounding machinery).
2026-07-14 21:04:24 -07:00
Matthew Jackson e659613ab8 restore freemkv-unlock path dep for local dev (post-v1.4.1) 2026-07-14 14:45:33 -07:00
Matthew Jackson eaa7ddc683 v1.4.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-14 14:45:30 -07:00
Matthew Jackson 723c7edf45 Fix mux concealing decryptable video over a single defective packet
AACS content decryption rejected a whole 6144-byte aligned unit unless
EVERY content packet was conformant MPEG-TS. One authored-bad packet (a
pressing/encoding defect or an AACS 2.1 forensic-variant frame) made the
mux conceal the entire unit as NULL TS — destroying up to 31/32 good
packets and tallying them as loss, surfacing as false "corruption" on
otherwise-clean discs (observed across two UHD titles).

decrypt_unit now asks only "did a key OPEN this unit?" — a padding-aware
>=75% supermajority of content packets restoring their 0x47 sync, a gate
no wrong key can reach (uniform-AES noise floor) yet one that tolerates a
minority of authored-bad packets. Opened units pass through VERBATIM; a
non-conforming packet is left for the demuxer to drop on sync-loss and
resync past. TS-sync conformance is a muxer concern, never a decryption
verdict. The post-read verify/sweep gate now shares the same primitive so
it can never disagree with the mux decrypt.

Also unify the MVC (Blu-ray 3D) track signals: the mvcC CodecPrivate
extension, the BlockAdditionMapping, and each per-frame BlockAdditional
all derive from one MVCDecoderConfigurationRecord built once per track, so
a malformed dependent-view parameter set can no longer orphan a BlockAddID.
2026-07-14 14:43:06 -07:00
Matthew Jackson d9881c57c0 restore freemkv-unlock path dep for local dev (post-v1.4.0) 2026-07-13 19:28:55 -07:00
Matthew Jackson 61b4486ae9 v1.4.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-13 19:28:52 -07:00
Matthew Jackson 72d39544ac Changelog: 1.4.0 (Blu-ray 3D / MVC) 2026-07-13 12:47:03 -07:00
Matthew Jackson 1e394eaeb8 3D MVC mux: audit round 2 (converged)
Second audit round converged (severity collapsed 6 HIGH -> 1; the one
HIGH was a bounded 32-element scan, not a defect; the sole spec MEDIUM
was the same false-positive re-raised — 0xBF matches ISO/IEC 14496-15
§7.6.2 verbatim). One genuine robustness fix plus coverage:

- extract_mvc_params: skip a zero-length NAL instead of abandoning the
  scan, so a stray length prefix before the subset SPS/PPS no longer
  silently drops 3D signalling. Test proves params after a zero-length
  NAL are still found.
- Tests: parser_for_mvc_dependent routes H.264 to a passthrough parser;
  passthrough with an IDR does not re-assert param sets (the keyframe &&
  !mvc branch).
- Document the per-playlist (not per-clip) is_3d latching as a known
  limitation (real main-feature playlists are uniformly 3D).
2026-07-13 11:39:07 -07:00
Matthew Jackson 4d1aa6bbf7 Harden 3D MVC mux: robustness + tests (audit round 1)
Triage of a 10-lens code audit of the 3D branch. Fixes for real defects;
rejected three spec false-positives that matched the ISO/IEC 14496-15
§7.6.2 record verbatim.

Robustness / correctness:
- Never panic when a title's only video is the MVC dependent view: the
  base is now the first NON-dependent video, so a dependent-only title
  sets up no merge (muxed as an ordinary track) instead of hitting an
  `expect` on the skipped track slot.
- Drop a per-frame BlockAdditional (BlockAddID=2) when the track declared
  no mvcC mapping (dependent params not captured before the header) — a
  plain block keeps the file conforming instead of an orphaned add.
- A non-keyframe MVC base frame always carries a ReferenceBlock (fall back
  to a 0 offset in the pre-first-keyframe corner) so it is never mistaken
  for a seek point.
- Reference the last keyframe on the PRIMARY video track only, so a
  secondary video track's keyframe can't become a cross-track reference.
- dep_by_pts overflow: bound BEFORE inserting so the just-arrived
  dependent survives the drift-clear; count a displaced duplicate-PTS
  dependent as an orphan instead of losing it silently.

API / docs:
- Fold write_frame_with_additional into write_frame(..., Option<&[u8]>)
  per the "no foo_with_X" convention.
- Fix mvc_params doc (StereoMode is intentionally not emitted); remove a
  stale PAT/PMT comment describing an approach that was never taken.

Tests: MVCDecoderConfigurationRecord over-length guards; write_int minimal
two's-complement widths; BlockGroup/BlockAdditions/BlockAdditional +
ReferenceBlock emission; additional dropped without a mapping; h264 MVC
passthrough keeps param sets in-band; extract_mvc_params no-panic on
truncated/empty input; pairing window + dep-overflow edges; no-panic on a
dependent-only title.
2026-07-13 11:24:23 -07:00
Matthew Jackson 9ce03b4d1d Mux Blu-ray 3D (MVC) as a single MVC video track
Fold the MVC dependent (right-eye) view into the base H.264 track as a
per-frame BlockAdditional under an mvcC BlockAdditionMapping, so a 3D
title produces one MVC video track instead of two independent H.264
tracks.

- h264: MVC-passthrough parser mode keeps the dependent view's subset
  SPS/PPS in-band, so each emitted frame is a self-contained dependent
  access unit for a BlockAdditional
- resolve: route the dependent stream through the passthrough parser
- mkvstream: detect the dependent view, pair it to the base frame by
  PTS (bounded FIFO), attach it as a BlockAdditional (BlockAddID=2),
  and skip building its own track; build the mvcC
  MVCDecoderConfigurationRecord from the captured subset SPS/PPS and set
  it on the base track at activation
- mkv: emit the mvcC BlockAdditionMapping and BlockGroup/BlockAdditions,
  with a ReferenceBlock on non-keyframe base frames
- ebml: add BlockAdditions/BlockMore/BlockAdditional/BlockAddID/
  BlockAddIDValue/ReferenceBlock elements and a signed-int writer

Verified against a Blu-ray 3D ISO: ffprobe shows a single MVC track,
the mvcC mapping is present, ~144k BlockAdditionals carry the dependent
view (8.7 GB), and the base view decodes cleanly with no regression.
MVCDecoderConfigurationRecord follows ISO/IEC 14496-15 7.6.2; StereoMode
is intentionally omitted (no enum value describes MVC-in-BlockAdditional;
the mvcC mapping is the primary 3D signal per RFC 9559).
2026-07-13 11:00:04 -07:00
Matthew Jackson f099a4a31a feat(3d): read the SSIF and enumerate the MVC dependent view
Blu-ray 3D as a mux-path variant:
- Detect 3D (a clip has STREAM/SSIF/<clip>.ssif — note the SSIF/ subdir, which
  the old .ssif fallback path got wrong) and use the SSIF extents for the mux:
  one transport stream carrying both the base (left) and MVC dependent (right)
  eyes on distinct PIDs, so muxing it captures the full 3D.
- Add the dependent-view video stream. The base STN table lists only the left
  eye; the on-disc PAT/PMT are AACS-encrypted (unreadable pre-key) and the base
  STN omits the dependent view (it lives in the MPLS STN_table_SS), so use the
  BD-3D PID convention: dependent = base video PID + 1 (0x1011 -> 0x1012),
  coding 0x20 -> H.264. The demux routes its packets from the SSIF.

info now reports two video streams for a 3D title; the mux writes both eyes.
2026-07-12 23:41:47 -07:00
Matthew Jackson e606b5e3a2 feat(3d): enumerate MVC dependent view (stream_type 0x20 -> H.264 video)
from_coding_type maps 0x20 (MVC dependent view) to Codec::H264, so the existing
PAT/PMT scan surfaces the SSIF right-eye substream as a second video stream on
its own PID instead of dropping it as Unknown. No new parser: the dependent eye
rides the same demux path as any other TS video stream. Removes the throwaway
3D-structure probes (findings captured in prior commit messages).
2026-07-12 22:16:13 -07:00
Matthew Jackson de809189ec feat(3d): add MVC dependent-view coding type (0x20)
First step of Blu-ray 3D as a mux-path format variant: name stream_type 0x20
(H.264/MVC dependent view, the SSIF right-eye substream). Recognition in the
scan/demux + dependent-stream enumeration follow.
2026-07-12 22:11:01 -07:00
Matthew Jackson 253b0924e3 wip(3d): prove SSIF de-interleave = ssif−base, dependent decrypts on same UK
Empirical (300, clip 00042): base .m2ts (1567 extents, 25.85 GB) is entirely
inside the SSIF LBA span; dependent = SSIF − base = 1567 ranges, 9.59 GB
(exact vs 35.43−25.85). 200/200 sampled dependent-view units decrypt under the
base UK. => de-interleave needs no SSIF parsing (complement of the base extents
we already resolve), and the dependent eye needs no extra key.
2026-07-12 20:40:58 -07:00
Matthew Jackson 98a7892c34 wip(3d): read-only 3D-structure probe (SSIF/m2ts + MPLS ExtensionData)
Dev tool for the 3D build: uses the library UDF reader (7z cannot parse these
ISOs) to dump base .m2ts vs SSIF sizes and the MPLS ExtensionData entries
(STN_table_SS + SS sub-path). Not product code.
2026-07-12 20:38:37 -07:00
Matthew Jackson b82d21756a fix(udf): follow allocation-descriptor continuation past the AED header
read_icb_extents parsed a continuation block's allocation descriptors from
offset 0, but a continuation block begins with a 24-byte Allocation Extent
Descriptor (ECMA-167 4/14.5): 16-byte descriptor tag + prev-loc(4) +
length_of_allocation_descriptors(4); the real ADs start at offset 24. Reading
from 0 mis-parsed the AED tag as a descriptor -> one garbage extent, then an
unknown-type break, silently truncating any file whose extents spill into a
continuation block.

Few-extent files store their ADs inline and never hit this path, so it stayed
hidden; a heavily fragmented file (~1600 fragments) truncated to 113 extents +
a bogus non-unit-aligned fragment that then aborted the mux
(ExtentNotUnitAligned). Skip the 24-byte AED header, read l_ad bytes of
descriptors from offset 24. Adds a regression test with a non-zero AED tag.
2026-07-12 20:37:55 -07:00
Matthew Jackson 02c584a3d9 restore freemkv-unlock path dep for local dev (post-v1.3.2) 2026-07-10 14:26:11 -07:00
Matthew Jackson 564e66fd24 v1.3.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 14:26:08 -07:00
Matthew Jackson 57d40e81a5 1.3.2: AACS 2.1 FMTS variant-decode foundation
Add UnitKey.variant_number (0 = ordinary, 1..32 = forensic variant) with new/variant constructors, and aacs::variant_select — resolve a disc's single variant and classify each aligned unit (default / variant / drop foreign / conceal keyless). Correct IndividualSegment.tbl: the per-record field is the variant (cycles 1..32 on a retail disc), not a segment number — Segment.number -> Segment.variant.
2026-07-10 14:00:28 -07:00
Matthew Jackson 674819d937 changelog: trim the 1.3.1 relicense note to just the relicense fact 2026-07-10 12:53:17 -07:00
Matthew Jackson c5e8421f9f restore freemkv-unlock path dep for local dev (post-v1.3.1) 2026-07-10 12:35:12 -07:00
Matthew Jackson 7dcf9b5a0d v1.3.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 12:35:09 -07:00
Matthew Jackson f01c097784 1.3.1: relicense to MIT (clean-room CSS + drop copyleft-lib references)
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL). The CSS
content cipher and Stevenson title-key attack are attributed to their published
cryptanalysis (not libdvdcss); all libaacs/libbluray/libdvdread/libdvdnav name
references were dropped from comments while keeping the standard format/spec
descriptions. Also bumps to 1.3.1.
2026-07-10 12:31:19 -07:00
Matthew Jackson c265fa349e hddvd: authoritative title composition from the Advanced-Content playlist
Parse ADV_OBJ/VPLST000.XPL (the real HD-DVD player playlist) with roxmltree
into one DiscTitle per <Title>: its PrimaryAudioVideoClip clips in order (EVO
via the .MAP sidecar), the titleDuration, the displayName, and the ChapterList.
A layer-break split (FEATURE_1+FEATURE_2, feature/feature_Divide) composes into
ONE title with the two parts as clips, each carrying its title-time in/out
points (45kHz ticks) for seamless-join splicing. Falls back to the clip-name
heuristic when no playlist is present.

Fixes unknown runtimes (real durations), poor names (FEATURE -> 'Main Movie'),
and gives authoritative composition + chapters. Validated on ANCHORMAN (97m),
SHAUN (99m), HARRY_POTTER (152m).
2026-07-10 10:28:18 -07:00
Matthew Jackson 6a59dcb7ba ci: drop the dead crates.io publish job (libfreemkv is git-tag-only)
libfreemkv is publish=false (it git-deps the firmware crate freemkv-unlock,
never on crates.io), so the Release workflow's 'cargo publish' step failed
hard on every tag. Consumers git-tag-pin libfreemkv; the git tag is the
release artifact. Removed the publish job.
2026-07-10 08:52:57 -07:00
Matthew Jackson 0eb1817ec7 restore freemkv-unlock path dep for local dev (post-v1.3.0) 2026-07-10 08:43:47 -07:00
Matthew Jackson 21f3b6adbd v1.3.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 08:43:42 -07:00
Matthew Jackson b32d636c64 audit: void empty-timeline duration, cover sniff overlap
Round-11 findings from the 10-phase release audit (no real HIGH):

- When a no-declared-duration source (HD-DVD) muxes a degenerate single
  frame at tick 0 with no per-frame duration, max_block_ticks stays 0 and
  the reserved DURATION placeholder was left as a literal 0.0 (players read
  that as a zero-length file). Void the element instead, so the Segment
  omits DURATION as an unknown-duration source did before the back-patch.
- Add a regression test for the sniff_video_codec overlap fix (a
  picture_start_code whose payload begins 00 00 followed by a real start
  code) so the i+=4 marker skip can't silently regress to i+=3.
2026-07-09 20:40:54 -07:00
Matthew Jackson d877630c45 audit: lock DTS rate table, fix sniff overflow-scan, cover decrypt loss
Round-10 findings from the 10-phase release audit:

- A finder claimed the DTS SFREQ→rate table was wrong at 11/12; verified
  it against ffmpeg's avpriv_dca_sample_rates (12k/24k/48k/96k/192k at
  11-15) — the table is CORRECT. Added a test that locks the full table so
  it can't be mis-"fixed".
- sniff_video_codec advanced 3 bytes after a matched start code, re-reading
  the code byte as an overlapping start code; skip the full 4-byte marker.
- Guard the HD-DVD next_id title counter with saturating_add so a crafted
  disc with >65536 clips can't overflow (panic in debug).
- Add a test that an undecryptable unit (DecryptFailed) is zero-filled and
  counted as loss through ExtractResult (complete=false, bytes_lost>0) —
  the recovery-seam consolidation folded that bucket into bytes_unreadable.
2026-07-09 20:17:12 -07:00
Matthew Jackson 1895095f49 audit: drop dead DTS marks cap, lazy passthrough buf, doc corrections
Round-9 findings from the 10-phase release audit (no HIGH):

- Remove the MAX_PTS_MARKS backstop and its tautological test: an empty
  DTS PES returns before recording a mark, and a non-empty run is already
  bounded by the MAX_AU_BYTES buffer clear (which clears pts_marks) — so
  the deque cannot grow unbounded and the cap was dead code.
- AuAssembler::for_codec no longer reserves 256 KiB for a Passthrough
  stream (audio/subtitle, and every TS/BD stream) whose buf is never
  written; only the reassembling modes reserve.
- Correct the scan comment that claimed region is computed (it is a
  Region-free stub until region detection lands) and drop a public-repo
  reference to internal "private refactor notes" in the mkb module doc.
2026-07-09 19:50:54 -07:00
Matthew Jackson 5f8c049444 audit: clamp BD format fallback, running GOP byte counter, O(1) DTS marks
Round-8 findings from the 10-phase release audit:

- detect_disc_format's BDMV fallback passed detect_format's result
  through unchanged, so an SD bonus/menu title could tag a BD-tree disc as
  DVD (mis-sizing the ECC sweep) — violating its own "never below Blu-ray"
  invariant. Clamp anything but UHD up to Blu-ray.
- Track the MPEG-2 GOP byte total incrementally instead of re-summing the
  whole gop_buf on every pushed picture (was O(pictures²) on any MPEG-2
  disc, not just adversarial input).
- Back the DTS pts_marks deque with a VecDeque so the over-cap prune is an
  O(1) pop_front, not an O(n) Vec::remove(0).
- Add a test exercising parse_stream_id_extension's PTS/DTS skip branches
  (the real AU-opening 0xFD video PES path) — previously untested.
2026-07-09 19:27:01 -07:00
Matthew Jackson 7d427d0b17 audit: bound DTS marks, align disc-format tree order, doc/test cleanups
Round-7 findings from the 10-phase release audit (no HIGH; convergence):

- Cap DtsParser.pts_marks (MAX_PTS_MARKS): a run of zero-length timed PES
  packets grew no buffer bytes, so the drain_front mark-prune never ran —
  the deque could accumulate without bound on hostile PS input.
- detect_disc_format tested HVDVD_TS before BDMV while the title-scan
  dispatch tests BDMV first, so a disc with both trees would be classified
  HD-DVD but enumerated as Blu-ray. Align both to BDMV → HVDVD_TS →
  VIDEO_TS.
- Document why the DTS new-PES re-base can emit a locally-decreasing PTS
  (the muxer's block_ts applies the strictly-monotonic audio nudge, tested
  in mkv.rs) — this is by design, not a mux defect.
- Fix stale aacs/keys.rs comment references (functions moved to
  aacs/inf.rs / aacs::resolve/derive in the module split).
2026-07-09 19:01:56 -07:00
Matthew Jackson 0a47b22db5 audit: byte caps on GOP buffers, opener-scan resume, honest video codec
Round-6 findings from the 10-phase release audit:

- Wire the documented MAX_PENDING_BYTES byte cap into the MPEG-2 GOP
  buffer (it was dead code) and add an equivalent MAX_GOP_BYTES cap to the
  sparse-PTS reorder, so a crafted stream of few-but-huge access units
  cannot over-allocate — both were bounded only by frame count before.
- probe_evo_streams defaulted an unsniffable HD-DVD video stream to H.264,
  which mis-parses a VC-1 (or still-encrypted) clip into a corrupt track.
  Emit the video stream only when the codec is actually identified — the
  honest outcome, matching the audio path (a real clear clip always carries
  its sequence header at the head).
- Resume the AU-opener search from a cursor (like the boundary search), so
  a long unsynced junk run is O(bytes), not O(buffer) per push.
- Mark mpeg2's now-dead MAX_AU_BUFFER test-only; restore #[doc(hidden)] on
  the aacs probe harness module.
- Add regression tests: the 0xFD video-routing guard, the FMTS-is-UHD key
  state, and the GOP byte caps.
2026-07-09 18:31:47 -07:00
Matthew Jackson 2ebcb60059 audit: guard 0xFD video routing, carry frame duration, add cap tests
Round-5 findings from the 10-phase release audit:

- collect_es routed EVERY extended-stream-id (0xFD) PES into the video ES
  buffer, so a 0xFD HD-audio sub-stream (MLP/TrueHD) could pollute the
  video sample and — if it preceded the video PES — stamp the video track
  with the audio PID, losing the video. Only the VC-1 extension (0x55) is
  now treated as video; routing 0xFD audio to its own track is deferred to
  the HD-DVD program-chain follow-up.
- The sparse-PTS reorder now carries its calibrated per-frame duration onto
  each frame, so the muxer emits a BlockDuration and the back-patched
  Segment Duration covers the final frame instead of understating it.
- Add regression tests for the MAX_MARKS and MAX_VTI_HITS caps (promote
  MAX_VTI_HITS to module scope); make the differential-test factory array a
  named type; drop an identity-op in a reorder test.
2026-07-09 17:59:04 -07:00
Matthew Jackson f46d99a348 audit: fix AU mark-field loss, VTI tie determinism, and mark/perf issues
Round-4 findings from the 10-phase release audit (the first fully clean
round; it dug into the new #22/#18 refactor code):

- AuAssembler closed each AU from only the FRONT mark's fields, so when
  one PES fragment carried the source and a later fragment of the same AU
  carried the PTS, the second field was dropped — a regression vs the old
  separate pts/source mark deques. Now merge the first Some of each field
  across all in-range marks.
- parse_vti_clip_order picked the largest residue bucket with
  HashMap::into_values().max_by_key(), nondeterministic on a size tie
  (randomized HashMap iteration) — could select a different clip table
  run-to-run. Break ties by smallest offset.
- Bound the marks/disc_marks deques (MAX_MARKS): the buf-size cap prunes
  marks only when bytes accumulate, so a run of zero-length timed
  fragments could grow them without bound on hostile input.
- Add push_owned so the PS path moves the PES payload into a passthrough
  AU with no copy (MPEG-2 video + all audio), removing a per-PES
  malloc+memcpy the refactor had introduced on the DVD path.
- Back-patch the MKV duration from the block END (start + its own
  duration) so it covers the final frame instead of understating by one.
- Add direct tests for the MKB record-framing walker; drop a stale
  drain_complete_aus doc comment left on process_au.
2026-07-09 17:30:41 -07:00
Matthew Jackson 4d93d90561 docs: drop Renesas from changelog (detection-only stub) 2026-07-09 16:36:47 -07:00
Matthew Jackson 4eeafd3378 docs: soften HD-DVD changelog to partial support (PGC parse deferred) 2026-07-09 16:33:40 -07:00
Matthew Jackson a73ba98f82 mux: resume AU-boundary scans from a cursor (O(n) not O(n²))
AuAssembler::drain rescanned the whole buffered access unit from a fixed
offset on every push, so reassembling one AU split across N program-stream
PES fragments cost O(bytes²/fragment) — amplified on the HD-DVD PS path
where H.264/HEVC/VC-1 frames are large and now flow through this shared
assembler (unlike the TS path, which delivers one AU per PES).

Carry a scan_pos cursor (and, for the stateful VC-1/MPEG-2 rules, a
seen_unit flag) so each push resumes the boundary search where the last
one stopped instead of restarting. Total scan work for one AU is now
O(AU bytes). The from-scratch scanners are retained as a #[cfg(test)]
oracle; a new differential test asserts the resumable path yields
byte-identical AUs at every fragment granularity for all three modes.
2026-07-09 16:22:54 -07:00
Matthew Jackson 7676f3f2f6 audit: bound the VTI clip-table scan; fix stale aacs doc links
Round-2 findings from the 10-phase release audit:

- parse_vti_clip_order bucketed hits by residue with an O(stride*hits)
  rescan and no hit cap, so a crafted HD-DVD VTI packed with millions of
  `.EVO` tokens (up to the 64 MiB UDF read cap) could burn seconds of CPU
  on a routine scan. Bucket in a single O(hits) pass and cap collected
  hits at MAX_VTI_HITS (a real table holds a few dozen).
- Fix the stale `super::keys::…` intra-doc links left by the aacs module
  rename: the referenced fns live in `super::derive`.
2026-07-09 14:41:49 -07:00
Matthew Jackson e1cfe59dc9 audit: cap the sparse-PTS reorder buffer, FMTS key state, zero KCD
Round-1 findings from the 10-phase release audit:

- SparsePtsReorder buffered its current GOP with no bound, draining only on
  a keyframe — an open-GOP or crafted program stream that never signals one
  could hold the whole title in RAM. Force-complete the GOP at
  MAX_GOP_FRAMES, matching the MPEG-2 parser's backstop.
- inject_unit_keys labelled a 2.1 FMTS disc as AACS 1.0 / bus-encryption
  off; FMTS is UHD-family, so synthesize the UHD version + bus encryption.
- The compiled Key Correction Data was a non-zero 16-byte constant fed into
  the Media Key derivation. Per the no-compiled-keys rule it is now all-zero;
  the chain still cannot complete on a real disc (documented), so this is
  behaviour-neutral — all variant tests pass unchanged.
- Fix stale doc references (broken `super::variants` intra-doc links, and
  `aacs::keys` comments) left by the module rename.
2026-07-09 14:14:20 -07:00
Matthew Jackson c43f23a759 mux: back-patch the MKV duration from the timeline when the source has none
A title whose scan yields no duration (HD-DVD — its `.MAP` timemaps are
not parsed, so DiscTitle.duration_secs is 0) produced an MKV with no
Segment Duration element, so players/MediaInfo reported an unknown
runtime.

Reserve a DURATION placeholder when the source declares none, track the
highest block timestamp written, and back-patch the placeholder at
finish() with the real muxed runtime (also enabling the per-track BPS
tags for these titles). Gated on duration_secs == 0, so BD/UHD/DVD —
which carry a real mpls/IFO duration — write it up-front exactly as
before, unchanged.
2026-07-09 12:52:48 -07:00
Matthew Jackson 770e0ab936 docs: changelog for HD-DVD feature composition + MPEG-2 AuAssembler 2026-07-09 09:12:56 -07:00
Matthew Jackson 5557765a86 disc: compose the HD-DVD feature from its VTI clip table
HD-DVD Standard Content splits the main feature across clips at the
layer break (FEATURE_1/FEATURE_2, or feature/feature_Divide). The scanner
enumerated one title per .evo, so main-title selection picked only part 1
(e.g. Shaun's 11 GB FEATURE_1, missing the 6.8 GB FEATURE_2).

Parse the HVA*.VTI navigation file's clip table — a fixed-stride record
list naming every clip in authored order, isolated by residue-mod-stride
rather than the imprecise header pointer — and concatenate the feature
clips (matched by the feature* naming convention) into one title whose
extents run in authored order. Every other clip stays its own title.
Falls back to one-title-per-clip when the VTI is absent or unparseable,
so nothing regresses on a disc with no readable navigation.

Validated on real discs: Shaun 17.8 GB / Anchorman 20.1 GB / Harry Potter
24.4 GB now enumerate as one 2-clip FEATURE title (largest = the movie).
2026-07-09 09:12:13 -07:00
Matthew Jackson 458b04ae63 mux: reassemble MPEG-2 access units via the shared AuAssembler
The MPEG-2 parser hand-rolled its own PES reassembly — a byte buffer plus
parallel PTS / source / discontinuity mark queues keyed by absolute
offset — duplicating what AuAssembler already does for H.264/HEVC/VC-1.

Add a Mode::Mpeg2 to AuAssembler (picture 0x00 with preceding sequence
0xB3 / GOP 0xB8 headers — the same headers-precede-picture shape as the
VC-1 mode) and have the MPEG-2 parser own one via AuAssembler::mpeg2().
parse() now feeds fragments to the assembler and processes each complete
access unit; the buffer, base offset, and three mark queues are gone. The
GOP-buffered temporal_reference reorder and PTS origin-locking are
unchanged. The parser's external contract is unchanged, so all existing
MPEG-2 parser tests pass as-is; new AuAssembler tests cover the MPEG-2
boundary rule directly.
2026-07-09 08:44:01 -07:00
Matthew Jackson c31ac6b66b docs: changelog for HD-DVD VC-1, sparse-PTS, variant layout (1.3.0) 2026-07-09 08:21:09 -07:00
Matthew Jackson 3e84ecfe42 mux: reconstruct display-order PTS for sparse-PTS program streams
HD-DVD EVO (and DVD VOB) program streams timestamp video at GOP
granularity: only one access unit per GOP carries a PES PTS. The H.264 /
HEVC / VC-1 parsers collapsed a missing PTS to 0, so on such a source
every non-anchor frame landed on the same block timestamp and a decoder
reported "non monotonically increasing dts".

Add a shared SparsePtsReorder that rebuilds a display-order PTS per frame
from the coded picture type (I/P/B) plus the sparse anchor PTS, with a
per-frame duration self-calibrated from the spacing between consecutive
GOP anchors (no external frame-rate needed). Display order is derived via
the classic single-anchor-delay rule (an anchor displays only after the
previously-held anchor; a B displays immediately), exact for the
non-hierarchical GOP structures HD-DVD H.264/VC-1 use. It mirrors the
MPEG-2 parser's GOP-buffered origin-locking.

Gated to the program-stream path only: the three parsers enable it via
with_ps_reorder(is_dvd_ps), so the BD/UHD transport path (per-frame PTS)
is byte-identical and untouched.
2026-07-08 21:54:09 -07:00
Matthew Jackson 998f82dc07 aacs(2.1): pin variant record layout against reference MKBs
Correct the variant Media Key chain against two real variant MKBs
(Zombieland v70, Stand By Me v70): C for Kmp is the per-slot block of
the 0x0c cvalue table indexed by the matched subset-difference, not the
0x2d head. The 0x2d record is the VARIANTS table (leading body-16 bytes,
one big-endian u16 per subset-difference) followed by a trailing 16-byte
Nonce, with no leading header. VKD stays at 0x2f.

Route record-type selection through the named REC_* consts in mkb rather
than bare hex, and document that Key Correction Data is per-licensee: no
universal constant exists and none is compiled in, so on a real disc the
chain yields a wrong Media Key that the Verify-Media-Key gate rejects
rather than emitting a bad key. This is a key-acquisition gap, not a
code gap.
2026-07-08 21:01:57 -07:00
Matthew Jackson b80af33775 mux: HD-DVD VC-1 demux via extended stream id 0xFD
VC-1 HD-DVDs (e.g. Shaun of the Dead) carry video on MPEG-PS extended
stream id 0xFD, with the real stream selector in stream_id_extension
inside the PES extension. Parse that field so the video routes to a
distinct track (pid 0xFD00|ext) instead of being dropped.

Reframe VC-1 access units in AuAssembler with a dedicated Mode::Vc1:
an AU is delimited by the next frame BDU (0x0D) once a frame has already
been seen, so the sequence (0x0F) and entry-point (0x0E) headers that
precede an I-frame stay attached to the frame they describe. The old
single-start-code split stranded those headers on the prior AU, which
the decoder reported as bits-overconsumption and hard decode failures.

hddvd probe now tracks the video pid it detects and emits VC-1 on 0xFD.
2026-07-08 21:01:48 -07:00
Matthew Jackson 054902cefe release: prepare 1.3.0
Bump to 1.3.0 and add the 1.3.0 changelog entry (FMTS/HD-DVD formats, AACS 2.1 variant chain, recovery seam, aacs module split, main-title-by-size, and the fixes since v1.2.2). Not tagged or pushed.
2026-07-08 14:48:50 -07:00
Matthew Jackson aa277224a9 fix: assorted correctness fixes and dead-code cleanup
- aacs/resolve: a media-keys-only provider missing the VID classifies as
  VidUnavailable, not NoMaterial (an MK derives the VUK once the VID
  arrives).
- disc/bluray: mark a clip seen only after its .clpi parses, so a
  transient parse failure on the first PlayItem cannot suppress the
  clip's extents for a later PlayItem referencing it that succeeds.
- disc/patch: log rather than swallow mapfile record/flush failures on a
  reverify downgrade, so a failed persist cannot silently mismark a bad
  unit good on resume.
- mux/ts: flag a discontinuity when a partial PES is dropped, matching
  the other partial-drop paths.
- mux/demux_thread: the no-demuxer branch forwards an empty batch for
  early consumer-disconnect detection instead of reading the whole disc.
- io/pipeline: correct the send-timing log (as_secs_f64, not as_micros
  printed as ms).
- aacs/derive, aacs/variant, disc/read_error, keysource: comment/doc
  accuracy. sector/prefetched, udf: remove dead fields/functions.
- mux/disc: assert unit-aligned read counts in the test.
2026-07-08 14:44:15 -07:00
Matthew Jackson 26154b6977 sector: generic recovery seam; FMTS forensic segments as decrypt loss
Replace the AACS-specific inline key-fetch in the decrypt decorator with
a scheme-neutral recovery seam: the input stream (L3) installs a Recover
closure (none / AACS key-fetch) and the decorator (L2) runs it at the
single decrypt-miss point. FMTS (AACS 2.1) forensic-segment units that no
key opens are just undecryptable units, concealed and counted as ordinary
decrypt loss with no FMTS-specific branch ("a loss is a loss"), so the
separate bytes_undecryptable bucket collapses into one loss count.

- sector/recovery.rs: the seam (MissOutcome, none/key_fetch factories),
  naming no encryption scheme in its type.
- FMTS: segment routing primitives + BYPASS_FMTS_KEY, and an upfront
  ensure_forensic_segments_decryptable gate (Error::FmtsKeyMissing) in
  the mux input path, parallel to the unit-key gate.
- CSS descramble/rekey moves from decrypt_sectors into
  css::descramble_region: CSS self-recovers from the data itself, so it
  stays OFF the seam (which is only for external inputs).
- disc/mod.rs also: main-title selection aligned to largest physical
  size; is_regular read from the open file handle, not metadata(path),
  fixing a swallowed sync_all on a fresh-rip ISO. decrypt_threads()
  resolved once via OnceLock off the per-buffer hot path.
2026-07-08 14:44:03 -07:00
Matthew Jackson bb90991e05 labels: reader-backed detection, Criterion fix, menu-language fallback
- DetectFn now takes a SectorSource so a parser can inspect a jar's
  central directory in detect() instead of firing on "any BD-J jar".
  dbp/deluxe do the real com/<vendor>/ prefix check up front, so each
  claims only its own discs (foundational for scaling the registry).
- criterion: treat a stream-map value of 0 as unmapped and synthesize a
  real 1-based number, so a 0 can't shadow or collide with a genuine
  stream 1 (with regression tests).
- png_filenames: new Low-confidence, last-resort parser reading menu
  language from {title}_UHD01_{LANG}_Composite artwork; sits below the
  MPLS floor so a real framework parser always wins.
- vocab: add menu_lang() for 639-2/B to 639-2/T menu-token normalization.
2026-07-08 14:43:52 -07:00
Matthew Jackson 175f131008 aacs: parse FMTS SegmentKey.tbl container (16-bit variant-indexed)
New aacs::segment_key: parses the AACS 2.1 SegmentKeyNNNNN.tbl container, confirmed against a retail disc as an 8-byte header + 65536 records of 536 bytes, indexed by the 16-bit variant selector the Media Key Variant chain produces. This is the confirmed link between the two 2.1 variant layers (selector picks the device's per-segment variant). The per-record 528-byte payload layout is not yet reversed.
2026-07-07 19:18:33 -07:00
Matthew Jackson af2f750629 aacs: parse FMTS forensic segment map (IndividualSegment.tbl)
New aacs::segment: parses the AACS 2.1 IndividualSegment.tbl into the source-packet ranges of the forensic variant segments (validated against a retail disc: 792 segments, 2560 packets each). First piece of the FMTS variant decoder — the segments' variants are encrypted under segment keys, not the unit key, so a unit-key rip corrupts them (broken HEVC refs).
2026-07-07 18:59:09 -07:00
Matthew Jackson f1517f7313 disc: select main title by largest size, not clip count
canonical_title_order keyed on clip-count ascending before duration, so a short 1-clip bonus reel outranked the real feature — which is often chaptered into many clips (one per chapter). Make physical size the primary key: the main feature is the largest non-oversize title; decoy 'play-all' playlists run long but tiny (reused clips) and self-eliminate.

Validated across 23 UHD/BD discs: fixes t1 on F9, Fast Five, Fast & Furious 6, and Furious 7 (feature was ranked #13-36); no regressions on the 19 already correct.
2026-07-07 18:36:36 -07:00
Matthew Jackson 984774b068 disc: first-class FMTS + HD-DVD formats; CPI sample selection
Add DiscFormat::Fmts (AACS 2.1) and DiscFormat::HdDvd as first-class peers. Format derives from the AACS MKB generation (mkb_type().generation(): V10=BD, V20=UHD, V21=FMTS), reusing existing AACS code, and from the on-disc tree for HD-DVD/DVD. One detector (detect_disc_format) shared by the coarse DiscId probe and the full scan — no more 'default BluRay, defer to full scan'.

FMTS is a BD-tree stream variant: parse_playlist resolves the clip stream via CLIP_STREAM_EXTS (.m2ts -> .fmts -> .ssif), so the .fmts main feature yields real extents (previously silently empty). HD-DVD is a tree-level peer with its own enumerator (disc/hddvd.rs): HVDVD_TS/*.evo -> MpegPs titles with real extents (playlist/stream parsing honestly stubbed).

Sample selection for key resolution now uses the authoritative AACS CPI flag (aacs_unit_encrypted, byte-0 & 0xC0) not the ts_sync_destroyed heuristic — container-agnostic (M2TS/FMTS/EVO; TS-sync is meaningless on HD-DVD program streams) and stops the key-service '0 encrypted units' rejection.

Tests live with each format (bluray/hddvd/mod); generic UDF fixture builders extracted to a shared udf::fixture module.
2026-07-07 18:16:04 -07:00
Matthew Jackson ddc44e1105 unlock_bridge: run_features/run_bus dispatch; report LibreDrive vs Renesas
Adopt freemkv-unlock's split Unlocker trait: run_features drives the drive-prep capability, run_bus the content bus removal, each iterating unlockers until one doesn't decline (NotApplicable = try next; Ok or a real error stops).

unlocker_matrix now reports which drive-prep unlocker actually ran — LibreDrive removes the bus at the drive; Renesas unlocks features but leaves the bus to the cert. Wire product_id through to fu::DriveId.

Bump to 1.2.3.
2026-07-07 16:36:05 -07:00
Matthew Jackson b8a51ef1c7 aacs: remove boil veneer; move key newtypes to aacs::types
aacs::boil was a thin newtype veneer over aacs::derive — a duplicate layer.
Delete it: move the shared key newtypes (Vid, MediaKey, Vuk, ProcessingKey,
UnitKey) into aacs::types, and expose resolve_candidate + KeyCandidate from
aacs::derive directly. Downstream (keysource, disc::encrypt) now import from
aacs::{derive,types}. Pure API consolidation, no behaviour change; full test
suite green on Rust 1.86.
2026-07-05 12:09:39 -07:00
Matthew Jackson 08cb80aac1 aacs: keep flat re-exports for the typed key primitives
The module structure is the public API, but the typed key primitives
(MediaKey/UnitKey/Vid/Vuk, mk_from_dk/mk_from_pk/uk_from_vuk/vuk_from_mk),
derive_vuk, the aligned-unit decrypt entry points, and DeviceKey/HostCert
are load-bearing names that downstream key-source crates import through the
flat `aacs::` path. Re-export them here so those crates can track the
module refactor without a lockstep re-pin.
2026-07-04 22:19:39 -07:00
Matthew Jackson 60d0ebc8a1 aacs(2.1): variant Media Key as a clean PK -> Km primitive
Redesign derive_media_key_variant to the minimal derivation surface:

  derive_media_key_variant(mkb_records, pk) -> Km

- PK-input only. Deriving Kp from device keys (DK -> PK) is a separate
  concern (walk_processing_key); a leaked 2.1 key is a PK, and the chain
  starts at Kp. A bare PK arrives without its subset-difference slot, so
  the primitive tries it against every slot and returns the Km for the
  slot that passes the MKB's Verify-Media-Key record -- mirroring the
  classical bare-PK derive_media_key_from_pk, gated by the chain's own
  verify so an unverified key is never returned.
- VID-free: the Media Key is MKB-scoped. VUK stays the separate
  derive_vuk(Km, VID) step.
- KCD is a fixed algorithm constant compiled in, not a caller parameter
  (removes the kcd argument, the placeholder const, and KcdNotProvided).
- Soft-correction / online-challenge slots are treated as non-covering;
  surfaced over the generic miss so a disc needing those modes is
  distinguishable from a non-covering key.

resolve_keys_v21 updated to walk DK -> PK first, then call the primitive
and derive the VUK from Km + VID. Module + helper docs refreshed to the
pinned record layout; tests reworked for the PK-only signature.
2026-07-04 19:16:40 -07:00
Matthew Jackson 5dee947007 aacs(2.1): wire variants_for_uv from the 0x2d record; chain runs end-to-end
variants_for_uv previously returned None (placeholder), dead-stopping the
Media Key Variant chain at VariantsTableUnavailable on any real disc. Layout
now pinned against two real 2.1 variant MKBs (Zombieland v70, Stand By Me v70):
the 0x2d Encrypted-Media-Key-Variant-Data body is sd_count u16 VARIANTS entries
(1:1 with 0x0c cvalues / 0x04 subset-diffs) followed by the 16-byte per-disc
Nonce at the tail. variants_for_uv reads the sd_slot_index-th u16.

With this the chain runs fully: Kmp -> Kpnew(=Kmp^KCD, extracted CyberLink
constant) -> VKD(0x2f) -> Km -> Kvu. The only remaining input is a covering
2.1 Processing Key to validate against a known answer; until then the final
Verify-Media-Key (0x86) gate rejects any wrong layout pick, so a bad key can
never be emitted. Tests updated to reflect the wired lookup; 1.86 precommit
green (fmt+clippy+tests).
2026-07-04 18:26:09 -07:00
Matthew Jackson a1796e8345 aacs: point integration tests at real module paths
The facade removal (5cf4252) left tests/ calling aacs::disc_hash,
aacs::decrypt_unit, aacs::AacsVersion, etc. at the old flat paths,
so the branch did not compile its integration tests. Repoint each
to its real module (inf/derive/content/mkb). No logic change.
2026-07-04 18:25:59 -07:00
Matthew Jackson 5cf4252e44 aacs: remove the mod.rs facade — module paths are the public API
Delete the pub-use re-export facade; consumers now import from the owning
module (aacs::content::decrypt_unit, aacs::mkb::MkbType, aacs::derive::derive_vuk,
aacs::boil::mk_from_dk, aacs::resolve::resolve_keys_v2, ...). Internal callers
repointed accordingly. Path-only change; logic hash identical (95fb9924); 2210
tests green.
2026-07-04 14:19:40 -07:00
Matthew Jackson b5f9c4e426 aacs: merge media_key + volume_key into derive.rs
The volume_key module was only 34 lines and is just the tail of the same
DK/PK -> MK -> VUK -> UK derivation ladder as media_key. Fold both into one
derive module so every aacs module is a substantial, distinct responsibility
(crypto/mkb/derive/inf/content/variant/resolve). Relocation only; logic hash
identical (95fb9924); 2210 tests green.
2026-07-04 14:14:56 -07:00
Matthew Jackson 82dca307ad aacs: split keys.rs god-module into media_key/volume_key/inf/resolve
Break the 2800-line keys.rs into four responsibility-scoped modules:
- media_key.rs: DK/PK -> Media Key subset-difference walk (+ probe harness)
- volume_key.rs: VUK derivation, unit-key unwrap
- inf.rs: Unit_Key_RO.inf parsing, disc_hash, content cert, in-drive MKB read
- resolve.rs: the resolve_keys_* orchestration (keys.rs renamed)

Relocation only; the (white-box) test suite stays in resolve.rs and pulls
the moved items via glob imports. Proven byte-identical to the pre-refactor
state via the logic hash (95fb9924); 2210 tests green.
2026-07-04 14:06:57 -07:00
Matthew Jackson a9a21d8139 aacs: extract mkb.rs (MKB record format, MkbType, finders)
Relocate the shared MKB machinery into a single mkb module: the record
framing walker + MkbRecord view (from variant), the MkbType/AacsVersion
classification, the MKB-file utilities, and the record-body finders (from
keys). Fixes the inversion where the MKB parser lived in the 2.1-only
variant module. variant.rs keeps its local MkbRecord-based mkb_find_mk_dv
(name collision with the raw one; unified in the dedup follow-up).

Relocation only. Proven byte-identical to the pre-refactor state via the
function-body logic hash (imports normalized out); 2210 tests green.
2026-07-04 13:55:06 -07:00
Matthew Jackson 8dc7075356 aacs: extract crypto.rs (shared AES primitives + constants)
Relocate the shared low-level primitives into a single crypto module:
aes_ecb_encrypt/decrypt, aes_cbc_decrypt, aes_g (from content/variant) and
aesg3 + AESG3_SEED (from keys), plus AACS_IV. Fixes the scatter where AES-G
lived in the 2.1 file and AES-G3 in keys. Relocation only — no rename, no
logic change (logic-hash identical to baseline; 277 items; 2210 tests green).
2026-07-04 13:37:53 -07:00
Matthew Jackson 02218a6511 aacs: rename decrypt.rs->content.rs, variants.rs->variant.rs (no logic change)
Pure file+module-path rename. 'content' names the AACS unit-decrypt layer
(distinct from the top-level sector-decrypt driver crate::decrypt), and
'variant' (singular, spec term 'Media Key Variant') names the 2.1 chain.
Logic-hash identical to baseline; 277 items intact; tests green.
2026-07-04 13:24:26 -07:00
Matthew Jackson e32a3af898 aacs: annotate crypto with AACS spec section citations
Add [C]/[PR]/[BD]/[libaacs] §x.y provenance markers across the AACS
crypto so each primitive links to the spec section it implements, with a
source-tag legend in mod.rs. Doc-comments only — no logic, constant, or
signature changes.

Also: correct two stale record-type comments in variants.rs (0x82/0x83 →
the real 0x2d/0x2f) and document the Variant Number width (spec lsb_10 vs
the 2.1 chain's lsb_16, driven by the 65,535-entry VKD table).
2026-07-04 13:15:53 -07:00
Matthew Jackson 1253db037f restore freemkv-unlock path dep for local dev (post-v1.2.2) 2026-07-03 20:50:17 -07:00
Matthew Jackson 4b878dcc06 v1.2.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-03 20:50:14 -07:00
Matthew Jackson 7511b0b808 changelog: 1.2.2 (AACS 2.1 variant MKB, mk_from_dk fix, PK-path perf, autorip online-outage + move-error clear) 2026-07-03 20:48:04 -07:00
Matthew Jackson 707989bd68 aacs: target real AACS 2.1 Variant MKB records (0x2d/0x2f/0x0c)
The Media Key Variant scheme is detected and parsed via the actual MKB
record types found on a real variant disc:
  - 0x2d  Encrypted Media Key Variant Data (C)
  - 0x2f  Variant Key Data table (65,535 x 16)
  - 0x0c  variant cvalues (one per 0x04 subset-difference slot)

Replaces the earlier placeholder 0x82/0x83 record types, which were a
guess and appear on no real MKB. is_variant_mkb, the record finders, and
the subset-difference cvalue source (now 0x0c, falling back to 0x07/0x05)
are updated accordingly, along with the V20->V21 upgrade detection in
resolve_keys_v2 and its fixtures.

The variant chain still halts at variants_for_uv (the VARIANTS[uv] /
Nonce sub-field offsets need a covering key to confirm end-to-end
against the 0x86 verify), so a best-effort offset is never silently
trusted. All 27 variant tests pass on Rust 1.86.
2026-07-03 13:25:01 -07:00
Matthew Jackson 57b2f0c4f9 aacs: make the PK path terminal (no SD-tree walk)
A Processing Key is the key at its Subset-Difference node — one AES-G from
the Media Key — so it is tried directly against the MKB cvalue tables,
matching libaacs _calc_mk_pks (iterate PKs × cvalues). The prior code
treated every PK entry as a device-node label at unknown depth and BFS-walked
the SD tree (depth 3, capped 5), which was both wrong for terminal PKs and
~15x slower on a large UHD MKB (~181k cvalues): PK derivation on UHD dropped
from ~37s to ~2.4s.

The Subset-Difference tree walk now lives solely in the device-key path
(derive_media_key_from_dk), which owns per-node path bits; the PK path never
descends. Removed PK_WALK_MAX_DEPTH / _CAP / _walked / walk_pk_against_tables_impl;
renamed the core scan try_pk_against_tables and its probe test.

Precommit (Rust 1.86): fmt + clippy + tests green.
2026-07-02 23:17:20 -07:00
Matthew Jackson 47406a7e32 aacs: full-ladder resolve_candidate + correct mk_from_dk
resolve_candidate(candidate, mkb, unit_key_ro, vid) -> ResolvedChain: one
composed, PURE-DERIVATION boil-down for a candidate key at any ladder rung
(KeyCandidate::{Uk,Vuk,Mk,Pk,Dk}, each carrying the module's newtype). Walks
DK/PK/MK/VUK -> terminal unit keys, parsing Unit_Key_RO.inf at the version
the disc's MKB declares, and returns every CPS unit key as (cps_unit, key)
(matching ResolvedKeys) plus the intermediate chain. No sampling/validation
(that's the caller's unit_key_validates) and no position recovery (Dk is a
positioned DeviceKey; recover_dk_position first). Adds ProcessingKey newtype.
Consumers stop re-composing the ladder; every client hardens one impl.

Correct mk_from_dk: real Subset-Difference walk (derive_media_key_and_pk_from_dk)
instead of the Media-Key-Variant path, which needed integrator KCD absent
in-tree and Err'd for every real disc -- dead for both consumers. Drops the
now-unused vid arg.
2026-07-01 22:34:52 -07:00
Matthew Jackson 03cbda741a restore freemkv-unlock path dep for local dev (post-v1.2.1) 2026-07-01 19:08:56 -07:00
167 changed files with 73159 additions and 25191 deletions
+152 -6
View File
@@ -2,7 +2,10 @@ name: CI
on:
push:
branches: [main]
# `dev` is where work lands and where CI must be green. `main` only ever
# moves at release time, to the tagged commit, so a push to it is the
# release validation run rather than day-to-day feedback.
branches: [main, dev]
pull_request:
jobs:
@@ -10,42 +13,185 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
- run: cargo clippy -- -D warnings
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
working-directory: libfreemkv
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
check-macos:
runs-on: macos-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo check
working-directory: libfreemkv
check-windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
# Build the tests (not just `cargo check`): catches errors in test
# code and forces full codegen of the Windows-only SPTI transport
# (src/scsi/windows.rs), which never compiles on the Linux/macOS dev
# hosts. We don't `cargo test` here — the suite needs no drive but the
# extra build is the value; running tests is covered by the Linux job.
- run: cargo build --tests
working-directory: libfreemkv
# ── Did this change break anything downstream? ──────────────────────────────
#
# Every job above proves libfreemkv builds. None proved its DEPENDENTS do,
# and that gap is real: an engine signature change broke autorip today and
# went unnoticed because consumer CI only fires on a push to that consumer.
# libfreemkv sits below all five of them, so a break here is worth strictly
# more than a break anywhere else in the project.
#
# `cargo check --all-targets` only — each dependent owns its own behaviour
# and has its own suite. The question here is just "does everything built on
# me still compile against this commit".
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with: { path: libfreemkv }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-unlock, ref: dev, path: freemkv-unlock }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-keysources, ref: dev, path: freemkv-keysources }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-engine, ref: dev, path: freemkv-engine }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-i18n, ref: dev, path: freemkv-i18n }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv, ref: dev, path: freemkv }
- uses: actions/checkout@v5
with: { repository: freemkv/autorip, ref: dev, path: autorip }
- uses: actions/checkout@v5
with: { repository: freemkv/bdemu, ref: dev, path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
- run: cargo check --all-targets
working-directory: freemkv-keysources
- run: cargo check --all-targets
working-directory: freemkv-engine
- run: cargo check --all-targets
working-directory: freemkv
- run: cargo check --all-targets
working-directory: autorip
- run: cargo check --all-targets
working-directory: bdemu
+7 -25
View File
@@ -24,7 +24,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.86, same commit). Binary consumers
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -33,35 +33,17 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
- run: cargo test
# crates.io publish is an INDEPENDENT job: it serves EXTERNAL consumers only.
# The freemkv binaries no longer depend on it (they git-tag-pin libfreemkv via
# a committed [patch.crates-io]), so this publish runs in parallel with their
# release builds rather than gating them. It `needs: [verify, test]` so a
# failing test suite still blocks publication to crates.io — external
# consumers who `cargo add libfreemkv` must never receive a release whose
# tests were failing. (The two upstream jobs run in parallel, so this gate
# does not serialize publish behind test beyond their own completion.)
publish:
needs: [verify, test]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: Swatinem/rust-cache@v2
# --no-verify: CI already compiled this exact commit (in the `test` job
# and on every push via ci.yml). cargo publish's default re-verify does a
# full cold release build of the packaged tarball, which here is pure
# redundant work (~a cold lib build). Skip it.
- name: Publish to crates.io
run: cargo publish --no-verify
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
# (`package.publish = false` — it git-deps the firmware crate freemkv-unlock,
# which never ships to crates.io). Every consumer git-tag-pins libfreemkv via
# a committed [patch.crates-io]; the git tag itself IS the release artifact.
# A `cargo publish` here fails hard on `publish = false`, so it was removed.
release:
# Only needs `verify`; the GitHub Release can be cut as soon as the version
+9
View File
@@ -14,3 +14,12 @@ scratch/
# internal agent context — never publish (path AND dir; leak-guard blocks both)
CLAUDE.md
.claude/
# Nightly harness output. Written into the repo it audits, and it embeds
# absolute paths from the machine that ran it — which must never reach a public
# repo. Ignored rather than relocated so a run from any working copy is safe.
.nightly/
# cargo-mutants working output: large, machine-specific, never committed
mutants.out/
mutants.out.old/
+906 -15
View File
@@ -1,5 +1,895 @@
# Changelog
## [1.6.0] — 2026-08-03
### Fixed
- **A vendor label's forced-subtitle field was read as a boolean when it is an
enumeration, mislabelling full dialogue tracks and discarding the real forced
tracks.** One vendor's `playlists.xml` carries a per-subtitle-slot cell that
looks like a flag; the parser treated the value `1` as "this track is forced"
and every other value as "not forced". Measured across every image in the
corpus that uses this format — seven distinct discs — the cell takes four
values, and `1` is not the forced one: it marks a FULL dialogue track that
additionally contains some forced-narrative signs. Decoding three of those
discs and counting every PGS display set: all nine tracks bearing `1` on one
disc are full tracks of 949-1411 display sets, all seven on another are full
tracks of 1602-1651, and neither disc's `1` tracks are anything but full —
which is how a language ended up presenting as two identical full subtitle
tracks with one of them flagged forced. The values that DO name a dedicated
forced-narrative track, `2` and `3`, were being thrown away: they take their
own trailing stream slots, one per localized language, and measure 7 to 59
display sets against the 1216-2655 of the full tracks they duplicate. So the
reading was wrong in both directions. The cell is now classified as the enumeration it is; only a
dedicated forced slot earns the flag, an unrecognised value never does, and
the "contains forced signs" value is dropped rather than weakened into a
forced label. This is not something content could have corrected afterwards:
clearing a wrong forced label requires a disc whose authoring sets
`forced_on_flag`, and measured discs in this format do not set it — so on
those discs the vendor cell was, and remains, the only evidence there is.
Four of the crate's own tests had been asserting the boolean reading. The
other two parsers that emit a forced qualifier from vendor metadata were
audited and are structurally immune — in both, the forced marker names a slot
of its own rather than hanging off a full track's entry — and are now pinned
by tests saying so.
- **Content-based forced-subtitle detection never observed anything on a
feature-length disc.** The PGS probe spent its entire 256 MiB budget on the
first sectors of a title, where a feature has no subtitles at all — it hit
the budget, observed zero display sets, and contributed nothing to any
verdict, so the vendor label was always the only input. The same budget is
now SPREAD across each extent as ~16 MiB sample windows sized in proportion
to the extent (still on the AACS aligned-unit grid, still bounded by the same
ceiling): a track is disproven by any single non-forced display set anywhere,
and a genuine forced track is small enough to be caught by the spread. Cost
is unchanged; placement is not. The probe also stops spending budget on a
track the moment it is disproven, and skips an extent that owes evidence only
for such tracks.
- **A partially-read extent's evidence was memoised as if the whole extent had
been read.** A budget-cut (and now sampled) read covers a fraction of an
extent, but its result was filed under the extent's full key and replayed to
every other playlist sharing the clip — turning a prefix into an absence
claim about the whole extent. Cache entries now carry the coverage behind
them, and an entry only answers for a run that intended to read no more than
it did. Positive evidence (a non-forced display set was seen) still answers
regardless, being irretractable.
- **A forced verdict could rest on a single display set.** Calling a track
forced is an absence claim — "no display set here was un-flagged" — and a
sampled read sees a fraction of a track. Measured on real discs: tracks exist
that carry `forced_on_flag` on about a quarter of their display sets and not
on the rest, so catching one flagged set and nothing else is exactly what a
wrong promotion looks like. A sampled run now needs at least two display sets
before it may assert forced; a run that read every extent end to end has no
unread gap and may still promote off one (single-sign forced tracks exist).
- **Content could never correct a wrong vendor forced label.** The muxer only
ever promoted `FlagForced` 0 → 1, so a track labelled forced stayed forced in
the output even when the mux had seen every one of its two thousand display
sets and not one was forced. Content may now clear the flag too — in the
muxer and in the scan-time probe — behind a single shared guard: the absence
of `forced_on_flag` means nothing on a disc whose authoring never sets it, so
a demotion requires that some other track demonstrably uses the flag AND that
the track have the shape of a full dialogue track rather than of a
forced-narrative one. Where no track on the disc uses the flag, nothing is
demotable.
- **An `.fvi` index named itself as its own source.** The `fvi://` sink was
handed the DESTINATION path as its `source_path`, so every index reported
`source.path` as the file it was writing. `source.medium` was always `file`
whatever the real source, and `source.title` always `0` whatever title was
muxed — `docs/FVI_FORMAT.md` §6 defines all three as describing the input.
Beyond the wrong data, it made the format non-reproducible: two machines
indexing identical bytes produced different files purely because they wrote
to different paths, and a local filesystem path leaked into a shareable file.
`output()` now takes the source provenance explicitly. One of the crate's own
tests had been asserting the wrong value, which is why the suite never caught
it.
- **A vendor label list was re-numbered from 1 inside every title, so the same
label landed on a different physical stream in each.** A label list describes
ONE playlist's stream table, but it was applied to every title on the disc by
per-type ordinal. Where sibling playlists cover the identical feature clip
and enumerate different subtitle sets — one carrying a full track the other
omits — identical ordinals resolve to different PIDs, and the same stream
came out flagged forced in one title and not in the other. On the title such
a disc actually offers as the rip target, that put `forced` on an
873 MB full-dialogue English subtitle track: the user is shown "English" and
"English (Forced)", picks either, and gets the same subtitles. Labels are now
bound to the stream a PID identifies, not to a position in a list. The title
whose per-type stream-language sequence reproduces the label list is treated
as the table the list describes; the `(clip, PID)` facts it yields bind every
other playlist over that clip, so a flag lands on the same elementary stream
whichever playlist enumerates it, and a stream the list never described is
left alone. Streams no such fact reaches still bind by ordinal, but a label
whose language contradicts the stream it would land on is now dropped rather
than applied — an unlabelled track is a far smaller harm than a mislabelled
one, all the more since the muxer can only undo a wrong `forced` on discs
whose authoring sets `forced_on_flag` at all. Across the disc-image corpus
this cleared every cross-title label conflict, on both affected vendor
formats.
- **Streams borrowed from the playlists were merged into the vendor label list
by a number that meant something else, so a bonus clip could be labelled from
the feature's tracks.** A vendor label's `stream_number` is a slot in the one
stream table its config blob describes. The labels merged in from the
playlists to cover streams the vendor named nothing for carried a different
number entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's slot
numbering at all. The merge matched the two by equality, and the binder then
counted streams against the result. Measured across the 44-image corpus: 22
discs merge such labels, and of the 566 places one lands on a stream, 443
(78%) are a stream it does not describe — the label states which PID it read
itself from, and it is a different one. 142 of those were already stopped by
the language check added alongside the ordinal binding; 301 were applied. The
streams-only labels carry no editorial payload, so the direct damage is
confined to codec text, but the polluted list is also what the anchor gate
reads, and on 11 disc/stream-type pairs it is what decides the anchor — which
is how it reaches the vendor's forced and SDH flags. The same defect ran
through the clip-info orphan streams, numbered from `max + 1` of a list they
share no coordinate system with.
A label now either NAMES the elementary stream it describes — `(clip, PID)`,
read out of the very table the stream itself is built from — or it does not,
and only the ones that do not are ever reached by counting. Playlist- and
clip-info-derived labels bind by that name and by nothing else; the vendor's
bind through the language-sequence anchor as before, over the vendor's own
slots only. A named stream outranks a guessed one, so an editorial flag
reaches a stream only where the disc's own numbering puts it there. The
presence of the name is the provenance marker, which is what the anchor gate
was missing: it can now tell the vendor's slots from the borrowed ones, so a
slot the vendor never named no longer breaks the sequence, and a title with
fewer streams than the list has slots is no longer eligible to hold it. An
orphan stream, being in no playlist, is in no title, and now binds to nothing
rather than to whatever counted its way.
Over the corpus, 41 of 44 images are byte-identical and every one of the
three that move loses a label it should never have had: a dozen featurette
playlists stop reporting a feature subtitle's SDH marking on their own
single, unrelated subtitle; eleven menu and bonus titles stop advertising the
feature's object-audio format on plain stereo tracks; and on a disc whose
every title carries one audio stream, a regional-variant tag that had been
asserted on all seventeen titles is asserted on none — that disc's tables are
too short to anchor anything, so the tag is no longer claimed anywhere, and
in exchange every title now states the codec it actually carries, which none
of them did before. No feature title changes on any image. Six of the crate's
own tests had been asserting the invented numbering, including one pinning
the disc-global counter as a deliberate property.
- **A subtitle the content probe demoted went on calling itself forced.** The
probe writes its verdict to the stream's `forced` flag but left the
qualifier alone, and those are two renderings of one fact for two different
consumers: the muxer writes Matroska `FlagForced` from the flag, the JSON
metadata sidecar writes its qualifier string from the qualifier. A track the
probe cleared therefore shipped with a sidecar calling it forced next to a
header saying it is not. The demotion now clears the qualifier with the flag.
Only a forced claim is cleared — an SDH marking says something the probe
neither confirmed nor refuted, and is left alone.
- **Vendor stream labels were numbered by parsed entry, not by stream slot.**
Label blobs contain entries the parser deliberately does not interpret, but
those entries still occupy a stream-number slot. Counting only the parsed
ones shifted every later label up by the number skipped, so on discs with an
uninterpreted entry early in the list the language, forced, SDH and
commentary flags were attached to the wrong tracks — a subtitle track could
present as both a plain and a forced variant of the same language, with the
forced flag landing on neither or both. Four parsers shared the defect
(`pixelogic`, `paramount`, `mpls_universal`, `deluxe`); the rest are now
pinned by tests proving they are immune. Three of the crate's own tests had
been asserting the shifted numbering.
- **A feature's stream list ran on past its end and picked up menu clips as
streams.** The parser for one vendor's label blob finds the feature
playlist's section by name and ends it at the next named section — but on
most discs of that authoring style the feature playlist IS the last named
section, and the trailing per-language notice, disclaimer and dub-credit
cards carry no name marker at all. The walk therefore swallowed the whole
tail of the blob as more of the feature's own stream list. Those cards are
named per language, in the same shape as a stream token, so each one silently
advanced a stream-number counter, and the ones whose name collided with a
catalogued component were labelled as streams outright — on one disc, five
audio labels for slots 10 to 14 of a playlist that has nine. The same
collisions were being reported as vocabulary gaps and cost that disc's parse
its high-confidence rating. A section now also ends where the next one's
stream list begins, which is at a video slot the section has already listed.
Eight of eleven affected-format discs in the test corpus have no terminating
marker; two of them were producing labels for streams their feature playlist
does not contain. No other label parser walks a flat entry sequence this way
— the rest scope each stream to a structural range or read its number off the
entry itself, and two more now carry tests pinning that.
- **A forced-narrative subtitle marker went uncatalogued.** The token marking
the signs-and-on-screen-text pass that accompanies a dubbed presentation was
not in the vocabulary, so that track lost its forced flag while every other
language in the same run kept theirs. Vocabulary gaps are also no longer
silent: an unrecognized component now produces one aggregated warning per
parse naming the distinct components, rather than a bool nobody could see.
- **A key service that was DOWN was reported as "this disc has no key".** A
failed lookup and a successful lookup that found nothing shared a match arm,
so an unreachable service, a rejected token and a rate limit all arrived as
"no entry". New codes distinguish them: `E7028` service unreachable or 5xx,
`E7029` token rejected, `E7030` rate limited. An operator hunting a missing
VUK during a transient outage was the actual, observed cost.
### Breaking
- **`Resolution::pixels()` now returns `Option<(u32, u32)>`, not a bare tuple.**
The previous sentinel for "unresolved" was `(0, 0)` — a pair every caller
could mistake for a usable value, and one did (see `mp4` under Fixed). Every
caller now has to choose what an unresolved resolution means for it: the
Matroska and metadata sinks take `.unwrap_or((0, 0))` with the reason stated
at each site, the VobSub writer degrades to a palette-only `.idx`, and the
MP4 sink refuses the track.
- **`DiscSession::into_drive()` returns `Result<Drive, Error>`, not a bare
`Drive`.** The empty-slot state it used to panic on is reachable through
ordinary public use (`stage_drive_as_reader` moves the drive out; calling
`into_drive()` twice moves it out again), so the panic was not guarding
caller error — it was guarding a legitimate second call.
- **`DiscSession::drive()` / `drive_mut()` removed.** Dead public API with
zero callers anywhere in the toolchain, and panicking accessors are not
worth preserving the shape of.
- **`clpi`: the unused EP-map → sector-extent path deleted** (`get_extents`,
`resolved_ep_map`, `full_pts`, `full_spn`, `parse_cpi`, `EpCoarse`,
`EpFine`, and the `ep_coarse`/`ep_fine`/`version` fields on `ClipInfo`).
Crate-internal (`pub(crate)`), so not source-breaking for an external
consumer, but listed here because it removes surface: nothing in the
toolchain called it, it carried a truncation bug (fixed in a prior commit,
then removed with the code it fixed), and `ClipInfo` keeps only
`source_packet_count` and `streams`.
- New error codes: **E9055** (`Mp4UnknownResolution`), **E9056**
(`SyncTimeout`), **E9057** (`SyncWorkerLost`). Front-ends rendering error
strings need entries for all three.
### Added
- **High-level orchestration API — a single mux driver and a disc session.**
`mux_stream` drives the whole read → decrypt → demux → write pipeline for any
source (`MuxInput::Url` / `Iso` / `Session` / `Live`), so consumers stop
hand-rolling the frame pump. `DiscSession` hoists drive open + SCSI bring-up +
scan + key resolution behind one type; `scan_iso` does the same for a
file-backed ISO; `resolve_keys` / `resolve_keys_for` resolve base AACS keys.
These let the CLI and autorip shrink to thin front-ends (and back the new
`freemkv-engine` crate).
- **Per-title stream selection (`StreamSelection`).** A pure primitive that
prunes a `DiscTitle`'s audio/subtitle streams to a chosen set of PIDs (video
is always kept) before the mux builds its demux state — so track headers,
`codec_privates`, and frame routing all follow the pruned list, with no
demux-internal filter. Carried on `MuxOptions.selection` /
`InputOptions.selection` (both default to keep-everything, a no-op).
Languages are the caller's concern; the library speaks PIDs.
- `DiscTitle::audio_streams()` / `subtitle_streams()` / `video_streams()` —
typed iterators over each stream class.
- `MuxOptions` gains a per-call write-pipeline `send_deadline` and derives
`Default`.
### Changed
- **The recovery strategy moved to the new `freemkv-engine` crate.** Sweep,
patch, the retry-decision state machine, mapfile bookkeeping, and damage
classification are freemkv's specific recovery *philosophy*, not disc-access
primitives — they now live in `freemkv-engine`, which composes libfreemkv's
public API. The library keeps the raw single-shot read, SCSI-sense-fact
translation (`SenseFamily`, now in `scsi`), decrypt, and the mux highway.
- Small deliberate `pub` promotions to support the engine as an external
consumer: `Disc::resolve_content_key_map` / `encrypted_content_ranges`,
`io::WritebackFile`, `drive::extract_scsi_context`, `disc::locate_ranges`.
- New typed error classifiers re-exported at the crate root — `is_halt`,
`is_skippable_title_stub`, `is_disc_level_no_key` — and a new error variant
`SelectionPidUnknown` (E6014).
### Fixed
- **An undecryptable CSS disc no longer exits successfully.**
`Error::CssKeyMissing` (E7023) carried two conditions needing opposite
responses: one title of a multi-VTS DVD failing its own re-crack — correctly
skippable, the rest of the disc still rips — and the *whole disc* failing its
crack (`Disc::css_error`), where every title fails identically. Both raised
E7023, which `is_skippable_title_stub` classifies, so an uncrackable disc
iterated all N titles logging "title skipped" and exited 0. The disc-wide gate
(`Disc::ensure_decryptable[_keys]`) now raises `Error::CssNoDiscKey`
(**E7027**) — the CSS analogue of `NoDiscKey` (E7022), classified by
`is_disc_level_no_key`, so a rip loop fails fast. The per-title raise keeps
E7023 and stays skippable.
- **A corrupt `mkv://` input is no longer reported as a title worth silently
skipping.** `Error::MkvInvalid` (E6008) carried two unrelated meanings: the
genuine "this title produced no muxable frames" stub — which
`is_skippable_title_stub` classifies as skippable, so an all-titles rip drops
the title and finishes the rest — and *every* malformed-input rejection in the
MKV read path. A truncated file, a bad VINT, a cluster timestamp past
`i64::MAX`, a BlockGroup child overrunning its group: all of them classified as
skippable, so a broken source was passed over by a run that then exited
successfully. The read path now raises `Error::MkvSourceInvalid`
(**E9053**) — the counterpart of `Mp4Invalid` (E9049) — and the writer's
unrepresentable-element-size guards raise `Error::MkvUnencodable` (**E9054**).
Neither is skippable. `E6008` now means only the no-muxable-frames stub (the
mux driver's headers-never-resolved gate and the MKV muxer's zero-frame
`finish()` guard). The `json://` sink's metadata-encoding guard, which also
raised `MkvInvalid`, now raises `NoMetadata` (E9008) like `mux::meta`'s.
Front-ends rendering error strings need entries for E9053 and E9054 (and for
E9051 / E9052, split off `E6008` earlier in this cycle for the same reason).
- **The FMTS (AACS 2.1) forensic key resolution now runs once per disc, not once
per title.** `Disc::resolve_content_key_map` resolves every title, and the FMTS
branch ran ahead of everything else — so each playlist re-walked the UDF
filesystem to re-read `/AACS/IndividualSegment.tbl`, re-ran the forensic anchor
probe and the per-index phase probe, and **re-asked the key service for the
disc's index-key set**. On a 60-playlist AACS 2.1 disc that was 60 identical
key-service round trips (a key-server storm) and tens of thousands of random
6144-byte reads for one disc-wide answer. The UDF walk is now memoised for the
whole disc and the index keys + phases per distinct extent list — the only
per-title input to the probes. A read-faulted phase probe is deliberately never
memoised, so one bad read is not spread across the remaining playlists. The
per-title UDF walk was paid on **every** disc, FMTS or not, so a plain BD sweep
loses ~59 full-stroke seeks too. The multi-CPS extent memo added in 1.6.0 is
also reachable on an FMTS disc for the first time.
- **Mux correctness pass** (the `v1.4.0..HEAD` 10-phase audit): DTS core-header
false-drops that dropped good DTS frames; the TrueHD channel-correction probe
now runs correctly on AACS discs (7.1/Atmos no longer understated as 5.1);
an FMTS phase-probe read fault is distinguished from a wrong key; multi-CPS
and orphan-clip keying; the AACS key map is now a *positive* map (a sector
with no key passes through rather than failing), with fail-loud on genuinely
unresolvable keys; a user Stop mid-read is reported as `completed = false`
(a stop is not a failure), not a spurious error.
- **`udf`: deleted files and directories are no longer read as if they still
existed.** File-characteristics bit 2 (Deleted, ECMA-167 4/14.4.4) was never
decoded, so a deleted File Identifier Descriptor's ICB was followed like any
other. A deleted FID is permitted to point at extent length zero — not at a
File Entry at all — so following it reads whatever descriptor happens to sit
at that LBA: a deleted *directory* landed on the File Set Descriptor and
failed enumeration of the **entire volume**; a deleted *file* read back as a
genuine zero-byte entry.
- **`udf`: the Metadata File is now located from the Metadata Partition Map,
not assumed to sit at block 0.** UDF 2.50 2.2.10 records a
partition-relative Uint32 at offset 40 of the map that is the only
authoritative answer to where the file lives; block 0 is merely where most
authoring tools happen to put it. A conformant volume that recorded it
elsewhere was rejected as not-a-UDF-filesystem, or — on a volume carrying a
decoy descriptor at block 0 — silently mounted the wrong filesystem and
reported success. Block 0 stays in the candidate chain, so a volume with no
map or a wrong one keeps mounting exactly as before; the map is trusted only
when its partition type identifier actually reads `*UDF Metadata Partition`
(a Virtual or Sparable map is also ECMA-167 Type 2 and records unrelated
fields at the same offset).
- **`udf`: a read fault while locating the Metadata File is now a read error,
not "not a UDF disc."** The candidate-loop fix above discarded the
distinction between "read fine and the bytes say no" (structural) and
"could not read" (transient) — a single marginal-sector fault fell through
to the block-0 fallback, mis-tagged, and came back as
`Error::UdfNotFilesystem`. `mux::resolve` **memoises that verdict for the
whole disc**, so one flaky read silently demoted every remaining title to
the base-Unit-Key-only path on an AACS 2.1 forensic disc. The same
read-fault-vs-negative-verdict fix is applied to the Volume Descriptor
Sequence fallback below.
- **`udf`: `file_start_lba` no longer returns an unrecorded extent's LBA.**
A prior change correctly started retaining ECMA-167 4/14.14.1.1 type-1
(allocated-but-not-recorded) extents rather than dropping them — dropping
one slides every later extent down by the hole's length — but
`file_start_lba` still took the *first* extent unconditionally, so it could
hand back a hole's LBA rather than where the file's data actually begins.
`ifo.rs` uses this value as the base for every VTS VOB extent
(`file_start_lba(IFO) + vtstt_vobs + cell.first_sector`), so a DVD whose IFO
opens with a type-1 descriptor read its **entire video title set from the
wrong sectors** — no error anywhere, the reads just landed on unrelated
data.
- **`udf`: the Volume Descriptor Sequence fallback now retries on outcome, not
on the anchor's declared shape.** The Main VDS was selected from the
anchor's declared extent whenever the extent's *shape* was usable (length,
location, no address wrap) and the customary fixed location was tried only
when the shape itself failed. Shape is a property of the field, not of what
is actually there: a stale anchor, a rewritten volume, or deliberate
corruption can pass every shape check and point at nothing, in which case
the sweep finds no Partition Descriptor and the volume is rejected — on
exactly the damaged-disc branch recovery exists for, with no recovery path.
Both locations are now candidates and the fallback fires on whether
following the anchor actually found anything.
- **`css`: a sector whose cached title key is proven stale and whose re-crack
fails is no longer descrambled with the stale key anyway.** That produced
garbage payload behind an intact clear header instead of a hard failure.
Raises `Error::DecryptFailed`, matching AACS's behaviour on the same
condition.
- **`decrypt`: an encrypted unit outside every key-map range no longer passes
through as ciphertext counted as good bytes.** `extract` could report a
scrambled file as complete with exit code 0.
- **`mux`: frames dropped by the resync gate now reach `errors()`.** The gate
zeroes its counter at each resync, so a gap that resolved was previously
invisible to the error count even though frames were genuinely lost.
- **`mux`: a discard by the 8 MiB access-unit backstop now marks the following
unit discontinuous.** Previously the resync gate never armed after corrupt
input recovered via the backstop, so the discontinuity that should have
triggered a resync went undetected.
- **`mp4`: a video track with no resolved resolution is now refused
(`E_MP4_UNKNOWN_RESOLUTION`, E9055) instead of written as a structurally
valid, unrenderable 0x0 track.** `Resolution::pixels()`'s old `(0, 0)`
sentinel for "unknown" was indistinguishable from a real answer, and MP4 has
no optional-element mechanism to omit the field the way Matroska does —
`tkhd` and `VisualSampleEntry` both make width/height mandatory. See
`pixels()` under Breaking.
- **`labels`: Deluxe master-enum selection no longer iterates a `HashMap`.**
Iteration order over a `HashMap` is unspecified, so the same disc could emit
different commentary/SDH labels on different runs. Selection is now
deterministic.
- **`io`: a cancelled rip during the bounded fsync is no longer reported as a
hard I/O failure.** The three bounded-fsync failure modes returned bare
`io::ErrorKind` values with no `E<code>` prefix, so `is_halt()` — which only
recognises that prefix — could not tell a user Stop from a wedged NFS mount
from a lost worker thread, and a genuine cancel surfaced as a hard failure
at the end of an otherwise-complete mux. They now carry distinct codes:
`Error::Halted`, `Error::SyncTimeout` (**E9056**), and
`Error::SyncWorkerLost` (**E9057**) — see New error codes under Breaking.
- **`session`: `into_drive()` is fallible; `drive()` / `drive_mut()` deleted.**
See Breaking.
- **`clpi`: the unused EP-map → sector-extent path deleted**, including a
truncation bug it carried (`out_time` past the last EP entry resolved short
of EOF). See Breaking.
### Tests
- The suite grew from ~2,570 to **2,994** tests over this cycle.
- A long-standing intermittent failure (roughly 1 run in 10 under the full
parallel suite) was diagnosed and removed. It came from asserting on a
`tracing` capture: the capturing subscriber is installed thread-local while
`tracing`'s callsite-interest cache is global, so the two could race
regardless of how carefully the capture was serialised. The predicate is
now a named function tested as a plain value, with no subscriber involved.
Measured clean afterward: 14 consecutive full-suite runs, 2,994 passed, 0
failed.
## [1.5.2] — 2026-07-22
### Fixed
- TrueHD 7.1/Atmos channel correction now works on AACS-encrypted (Blu-ray/UHD)
discs. The channel-correction probe was built without an AACS key map, so on
every AACS disc its first read failed and the correction was silently skipped —
a 7.1/Atmos TrueHD track was muxed with its MPLS-declared channel count (often
understated 5.1). The probe now resolves and installs the same key map the mux
read uses.
- AACS 2.1 (FMTS) discs: a non-forensic title (menu/extras playlist, or any clip
that carries no forensic segments) no longer hard-fails the rip. `resolve_fmts_key_map`
now filters segments to those addressable within the title and falls back to the
base Unit-Key map when none apply — previously the first non-forensic title
aborted the whole-disc decrypt and blocked muxing any non-main title. A
forensic phase probe whose sampled units are all source-zero padding (an
even/odd tie) no longer aborts the rip either.
- Multi-CPS AACS `dir://` extraction now decrypts each clip with its own CPS-unit
key instead of keying the whole disc with unit key 0 (which silently wrote
secondary-CPS files as garbage). A missing key fails loud at resolve. Single-CPS
extraction is unchanged (one key opens every unit, orphan clips included).
- A trailing partial aligned unit that is inside a mapped range AND flagged
encrypted now fails loud (a CBC fragment split across a boundary cannot be
decrypted) instead of being emitted as ciphertext-as-clear.
- CSS DVDs no longer mux to garbage. Every DVD read path — the file-backed mux
highway (`build_iso_pipeline`) and the live-drive single-pass `DiscStream` —
now resolves the per-VTS title key at read time through one shared step
(`resolve_dvd_title_key`), cracked keylessly in playback order from the title's
own extents. An uncrackable title hard-fails (E7023) instead of passing
scrambled sectors through as plaintext; `--raw` skips the crack entirely; a
user Stop mid-crack surfaces as `Halted`.
### Changed
- DVD scan no longer cracks a title key up front (the key is per-VTS, so a single
disc key was meaningless). Scan does only the CSS bus-auth read-unlock — hoisted
before the UDF prefetch so scrambled small/menu VOBs no longer cost a rejected
read each. Cuts a CSS-DVD scan from ~25s to ~6s.
- Unlocker report: the DVD entry is renamed `CSS` → `DVD`.
## [1.5.1] — 2026-07-20
### Fixed
- **TrueHD audio is no longer silently dropped (and no longer sends decoders out
of memory).** The previous release added an MLP major-sync checksum gate to drop
genuinely-undecodable audio frames, but the checksum was computed with mismatched
byte order — the 16-bit CRC result was folded in one endianness and compared in
the other — so it never validated a real major sync. The parser then judged every
major sync corrupt and dropped every audio frame from the first one onward,
flushing the whole TrueHD track at the end of the file: de-interleaved from the
video, which sent players and integrity checkers into an unbounded memory spiral
("decoder ran out of memory"). The checksum now matches the reference
implementation byte-exact (cross-verified against real 7.1/Atmos and 5.1 discs),
so major syncs validate and only genuinely-corrupt frames are dropped; as a
safety net, a major sync the parser still can't validate is kept rather than
allowed to drop an entire track. TrueHD titles produced after the checksum gate
landed need a re-rip.
- **HD DVD AACS key files are now found on every disc, not just the common
layout.** The AACS directory and title-key filename on HD DVD are chosen by
the authoring house, and freemkv previously assumed one fixed spelling
(`/ANY!/VTKF000.AACS`). Discs that name their AACS directory differently (e.g.
`AAC!` instead of `ANY!`) or ship numbered title-key files (`VTKF090.AACS` /
`VTKF100.AACS` rather than `VTKF000.AACS`) are now handled: the AACS directory
is located by its contents and every title-key file in it is picked up. Blu-ray
and UHD are unaffected.
- **HD DVD multi-title decryption reads the right keys.** The HD DVD title-key
file (`VTKF*.AACS`) stores its keys in 36-byte records — per the AACS HD DVD
specification, and confirmed byte-exact on real discs. freemkv had been reading
them at a 32-byte stride, which lands the first key correctly but drifts off
every key after it, so only single-title discs decrypted. Discs with more than
one protected title now recover every title's key instead of only the first.
(Choosing the correct title-key file when a disc carries several playlists
still needs verification against an encrypted HD DVD.)
- **A dirty disc can no longer "rip clean" but decode with errors.** freemkv now
asks the drive to *report* marginal reads instead of silently returning
best-effort data as success — on smudged/scratched media a drive can hand back
subtly-wrong bytes with a clean status, which used to slip through the rip and
surface only as playback/decode errors. A read the drive had to fight for is
now distrusted and re-read in the patch pass: a clean re-read wins, and a spot
that's genuinely unreadable becomes an honest gap rather than silently-wrong
data. Best-effort per drive, and it changes nothing on a clean disc.
## [1.5.0] — 2026-07-19
### Added
- **MP4 as a source (`mp4://`)** — read a progressive `.mp4` back in and send it to
any sink (`mp4:// mkv://`, `mp4:// audio://`, `mp4:// json://`, …). The round-trip
is frame-exact.
- **Native MP4 output (`mp4://`)** — a disc goes straight to a play-everywhere
`.mp4` in one decrypt pass, no external transcoder. Carries HEVC / H.264 video (with HDR10) and
AC-3, E-AC-3, and DTS / DTS-HD audio, and is faststart by default so it plays over
HTTP without downloading the end first. It's a **compatibility export, not
archival**: MP4 can't hold TrueHD, LPCM, or bitmap (PGS / VobSub) subtitles, so
those are **excluded with a loud, itemized report — never a silent drop**
(`mkv://` stays the keep-everything path).
- **Five extraction sinks — dissect a title, don't just rip it.** New destinations
that pull one part of a title out on its own:
- **`video://dir/`** — each video track to its own native elementary-stream file.
- **`audio://dir/`** — each audio track to its own file in its native container
(`.thd`, `.dts`, `.ac3`, `.eac3`, `.aac`, `.flac`; LPCM as `.pcm`).
- **`sub://dir/`** — each subtitle track to its own file (PGS `.sup`, VobSub
`.idx` + `.sub`, text `.srt`).
- **`chapters://file`** — a title's chapter markers as a sidecar (`.xml` / `.txt`
/ `.ogm` / `.vtt`).
- **`json://file`** — a title's complete structure as JSON.
`chapters://` and `json://` read nothing of the elementary streams, so they
return in seconds.
- **Damaged audio frames are dropped instead of shipped as glitches.** When a
source disc has a corrupt audio frame, freemkv drops that frame rather than muxing
it as a decoder-choking glitch — keeping A/V in sync (a drop is a silence gap,
never a shift) and logging every drop. Works across DTS, AC-3 / E-AC-3, FLAC,
MP2 / MP3, AAC, and TrueHD, each using the format's own integrity check; a track
that is mostly undecodable is dropped whole. This catches structurally-broken
frames — corruption of the audio *data* inside an otherwise-valid frame is source
damage that can't be told from good data without decoding.
- **Forced subtitles detected from the stream.** A PGS subtitle track is flagged
forced when its content is entirely forced/narrative subtitles, read from the
stream itself rather than the disc's metadata — so it works on discs that carry
none. `info -v` reports the same, so `info` and a rip agree.
### Changed
- **`json://` emits the complete title model** — every field the scan resolved:
video resolution / frame rate / HDR / colour, audio channel layout / sample rate /
language / purpose, the subtitle forced flag, plus the clip list and chapter names.
### Fixed
- **TrueHD: a couple of transient errors no longer discard a whole track.** A
corrupt access unit is dropped forward to the next clean sync point, but a short
burst of damage no longer trips the whole-track drop, and a corrupt sync point can
no longer shift the audio that follows.
- **Free-format MP2 / MP3** is a legal, decodable mode and is no longer dropped.
## [1.4.5] — 2026-07-18
### Fixed
- **FMTS (AACS 2.1) forensic discs now mux to a clean, single-variant stream.** A
forensic segment interleaves the local device group's variant with a foreign
group's at the aligned-unit level. The mux decrypted only our half but left the
foreign half in the buffer as ciphertext, on the assumption that the demuxer
"drops untouched ciphertext cleanly." It does not — a foreign unit's bytes hit a
tracked PID at the 192-byte stride, mis-parse, and trip the demux's concealed-gap
keyframe resync, which discards good frames of ours around every segment (visible
playback glitches). `AacsKeyMap::read_plan` now turns the map into the title's
read plan: every default / CPS unit, plus inside a forensic segment **only our
phase's units**. The foreign half is never read, decrypted, or handed to the
demux. On a retail 4K UHD title this took concealed-gap resyncs from **349 → 0**
and recovered ~2 GB of previously-dropped frames. Wired into **both** mux paths —
the file-backed highway (`build_iso_pipeline`) and the inline live-drive
`DiscStream` (`with_key_map`) — so single- and multi-pass FMTS rips are both clean.
### Changed
- **Key-bearing types redact their `Debug` output.** Every type that carries key
material (device keys, processing keys, unit keys, media keys, VUKs, resolved
chains, CSS/AACS state, …) now prints a `<redacted>` marker instead of the bytes,
so no key can reach a log or panic message. Each is covered by a test asserting no
key byte appears.
- **Hex parsing is centralized and case-insensitive.** A single set of canonical
`0x`/`0X`-tolerant hex→integer parsers replaces scattered ad-hoc parsing (this is
what silently dropped keydb device keys written with an uppercase `0X` prefix).
- **Internal-only public surface narrowed to `pub(crate)`, and duplicate
`foo_with_X` methods collapsed to one** — no behavioral change, smaller API.
## [1.4.4] — 2026-07-17
### Fixed
- **Online key requests are no longer silently dropped on discs that yield few
sample units.** The online key source refuses any request carrying fewer than
`MIN_SAMPLE_UNITS` (8) encrypted-content samples — too few can match an
incidental unit rather than the one asked about (a false positive, most acute on
AACS 2.1 forensic-variant content). autorip gathered only 4, so every online
lookup was skipped before it ever reached the key service and surfaced to the
user as "key service down." autorip's sample count is now tied to
`MIN_SAMPLE_UNITS` with a **compile-time floor**, so it can never regress below
the minimum again.
### Changed
- **The online request is assembled from a proven-sufficient sample set.** New
`DecodeSampleSet` (`libfreemkv::keysource`) wraps the content-unit samples and
can only be constructed with at least `MIN_SAMPLE_UNITS` of them — so an online
key request cannot be built from too few samples. The minimum is validated once,
at construction, rather than by a runtime check a caller could forget.
## [1.4.3] — 2026-07-17
### Changed
- **`MIN_SAMPLE_UNITS` moved to the base crate.** The minimum sample count an
online key request must carry now has a single definition in
`libfreemkv::keysource`; `freemkv-keysources` re-exports it, so the online source
and libfreemkv's own forensic query size their requests from one shared value.
- **The online unit-key reply is parsed as a list.** A response carries either a
single Unit Key (ordinary disc) or the full ordered set (an AACS 2.1
forensic-variant disc); the client accepts both and maps array position to
forensic index.
### Added
- **Forensic-variant online query samples the anchor segment.** On an AACS 2.1
forensic-variant disc the online key query draws its sample from the first
forensic segment (index 1) — one canonical, deterministic sample — instead of an
arbitrary segment.
## [1.4.2] — 2026-07-15
### Fixed
- **Mux no longer nulls decryptable video or storms the key server on a
bad-encoded region.** 1.4.1 relaxed the decrypt gate but left the surrounding
machinery in place. On a unit whose key *decrypted* but whose plaintext didn't
reassemble to clean MPEG-TS, the read path still restored ciphertext, tallied
loss, and re-asked the online key server (forever returning the same correct key)
while the mux concealed the unit as NULL TS. The root cause: *"did a key produce
clean TS?"* was used as the verdict *"did we decrypt?"* — they are not the same.
A correct key can decrypt content with broken encoding; broken TS is a muxer
concern, never a decrypt verdict.
### Changed
- **One decrypt authority; policy at the caller.** `decrypt_sectors` is now a
pure decrypt: applies the CPS unit key in place, leaves plaintext, and reports
unverified bytes. It never restores ciphertext, nulls, or re-fetches a key.
Clean-TS status is only a key-*selection* hint (multi-CPS) or a read-*verify*
signal (sweep/patch). Callers own the policy: the mux passes decrypted bytes
through unconditionally (the demuxer handles bad TS); sweep/patch treat an
unverified unit as a failed read and re-read it. Removes the decrypt-time
ciphertext restore, the mux NULL-TS conceal loop, and the per-unit key-server
refetch, plus the dead `aacs_unit_still_ciphertext` predicate.
- **Decrypt and TS-structure are now separate primitives.** AACS has no MAC;
the only "did it decrypt?" signal is whether plaintext looks like MPEG-TS —
a data-quality / key-selection question, not a decrypt verdict. The old
`decrypt_unit(...) -> bool` is split into `decrypt_unit_raw` (pure crypto) and
`is_clean_ts` (structural check), composed explicitly only where needed. The
mux calls only `decrypt_unit_raw`.
- **Key-proof floor replaces the 75% supermajority.** The old proportion
(≥75% of content packets synced) conflated *the key worked* with *the content
is well-encoded*. `is_clean_ts` now requires `synced >= min(E, 4)` on
**encrypted** packets (skipping packet 0 whose `0x47` is in the clear seed):
four synced packets ≈ 1-in-4-billion false-positive; `min(E, 4)` scales to
short fragment tails so they're never false-rejected. A unit is "opened" when
a handful of packets prove the key — bad-encoded packets are the muxer's job.
## [1.4.1] — 2026-07-14
### Fixed
- **Mux no longer discards good video over a single defective packet.** AACS
decryption required **every** content packet to be conformant MPEG-TS: one
authored-bad packet (encoding defect, AACS 2.1 forensic-variant frame) made
the mux conceal the **whole** 6144-byte aligned unit as NULL TS (up to 31/32
good packets discarded, tallied as loss). On affected discs this produced
false "corruption" over otherwise-perfect video (~466 MB concealed across two
UHD titles). The gate is now a padding-aware **≥75% supermajority** of content
packets restoring their `0x47` sync — no wrong key reaches this threshold
(uniform-AES noise floor ≈ 256⁻ⁿ), but a minority of authored-bad packets
still passes. Opened units flow through verbatim; the demuxer drops
non-conforming packets on sync-loss. TS-sync conformance is a muxer concern,
never a decrypt verdict. (The supermajority threshold is tightened in 1.4.2.)
- **MVC (Blu-ray 3D) track signals unified and hardened.** The `mvcC`
`CodecPrivate` extension, the `BlockAdditionMapping`, and each frame's
`BlockAdditional` now all derive from a single `MVCDecoderConfigurationRecord`
built once per track, so they can no longer diverge. A track is flagged 3D
only when that record actually builds — a malformed dependent-view parameter
set no longer emits a mapping with no matching record (previously the flag was
taken from `mvc_params.is_some()`, which could orphan a `BlockAddID`). The base
track's `CodecPrivate` now carries the `mvcC` extension block
(`avcC ‖ u32be(size) ‖ "mvcC" ‖ record`, Matroska-spec size = block − 4) so
players and mediainfo detect MVC at the track level.
## [1.4.0] — 2026-07-13
### Added
- **Blu-ray 3D (MVC) support.** A 3D disc now rips to an MKV that preserves
**both eyes** as a single MVC video track — the AVC base (left) view in each
Block, and the MVC dependent (right-eye) view as a per-frame `BlockAdditional`
under an `mvcC` `BlockAdditionMapping` (`MVCDecoderConfigurationRecord` per
ISO/IEC 14496-15 §7.6.2), paired to the base by PTS. Remux only — no
transcode, no side-by-side conversion. The Blu-ray scan reads the interleaved
`STREAM/SSIF/<clip>.ssif`, enumerates the dependent view (stream_type `0x20`)
by the BD-3D PID convention, and parses it in a parameter-set-passthrough mode
so every dependent frame is a self-contained access unit. Verified on
*300: Rise of an Empire*: one MVC track, ~8.7 GB dependent payload carried in
per-frame BlockAdditionals, base view byte-identical to the 2D rip.
## [1.3.2] — 2026-07-10
### Added
- **AACS 2.1 (FMTS) variant-decode foundation.** `UnitKey` gains a
`variant_number` field (`0` = ordinary content, `1..=32` = a forensic
variant) with `UnitKey::new` / `UnitKey::variant` constructors, and a new
`aacs::variant_select` module resolves a disc's single forensic variant and
classifies each aligned unit — decrypt with the default key, decrypt with the
variant key, drop a foreign variant, or conceal a keyless forensic unit. This
is the groundwork for selecting one variant's segments and dropping the other
31; the decrypt-pipeline wiring lands with the variant key source.
### Fixed
- **`IndividualSegment.tbl`: the per-record field is the variant, not a segment
number.** `Segment.number` → `Segment.variant`. Verified against a retail 2.1
disc, the field cycles `1..=32` across the table (a per-variant tag) rather
than counting up, so variant selection routes on the correct value.
## [1.3.1] — 2026-07-10
### Licensing
- **Relicensed to the MIT License, from 1.3.1 onwards** (releases up to and
including 1.3.0 remain under AGPL-3.0).
### Added
- **Authoritative HD-DVD title composition** from the Advanced-Content playlist
(`ADV_OBJ/VPLST000.XPL`): each title's clips, real duration, display name, and
chapters come from the disc's own playlist instead of a clip-name heuristic. A
layer-break split (`FEATURE_1`+`FEATURE_2`, `feature`/`feature_Divide`) composes
into ONE title with the two parts as clips and their title-time offsets. Falls
back to the clip-name heuristic when no playlist is present.
## [1.3.0] — 2026-07-08
### Added
- **AACS 2.1 (FMTS) is a first-class disc format.** FMTS discs are detected,
labeled, and scanned as their own format rather than misread as plain UHD. The
forensic variant segments are located from `IndividualSegment.tbl` and the
`SegmentKey.tbl` container is parsed; the bulk of the title decrypts with the
unit key as usual, and the forensic segments (for which no segment-key source
exists yet) are skipped as expected loss, so a 2.1 disc rips mostly-complete
instead of failing outright.
- **AACS 2.1 variant Media Key chain runs end to end.** The variant media key is
derived as a clean Processing-Key to media-key primitive, with the record
layout pinned against reference variant MKBs — the per-slot `C` block from the
`0x0c` cvalue table, the `VARIANTS` table plus trailing nonce at `0x2d`, and
`VKD` at `0x2f` — so a genuine variant MKB resolves through the ladder.
- **Partial HD-DVD support.** HD-DVD is detected as its own format and its
`HVDVD_TS` `.evo` clips mux through the pipeline: EVO video is demuxed from the
MPEG program stream, including VC-1 titles carried on extended stream id `0xFD`
(real selector in the PES `stream_id_extension`), with the VC-1 access units
reframed so each I-frame keeps its preceding sequence and entry-point headers.
Title composition is heuristic for now (authoritative program-chain parsing is
planned), so a disc that authors two distinct features under the layer-break
naming may present them as one title.
- **Display-order timestamps for program-stream H.264 / VC-1 / HEVC.** A program
stream stamps a PES PTS only once per GOP; the parsers now reconstruct a
display-order PTS per frame from the coded picture type and the sparse anchor
(duration self-calibrated from anchor spacing), so a decoder no longer sees
colliding DTS. Gated to the program-stream path — the BD/UHD transport path
(per-frame PTS) is unchanged.
- **Stream-label parsers: reader-backed detection and a menu-language fallback.**
Label detection can inspect a jar's contents, so vendor parsers claim only
their own discs; a new last-resort parser reads menu-artwork languages.
- **keydb round-trips AACS 2.0 host certs** (the `HC2` line) so a load/save cycle
no longer drops v2 host credentials.
### Changed
- **MPEG-2 reassembles through the shared `AuAssembler`.** The MPEG-2 parser's
hand-rolled PES buffer and offset-keyed mark queues are replaced by the same
access-unit assembler the H.264/HEVC/VC-1 parsers use (in a new MPEG-2 mode);
the GOP-buffered `temporal_reference` reorder and PTS origin-locking are
unchanged, so DVD output is identical.
- **Generic per-scheme recovery seam.** Decrypt-miss handling is now a
scheme-neutral seam the input stream installs (no recovery, or an AACS
fresh-key fetch), with CSS self-recovering separately from the data itself. An
undecryptable unit is counted the same whatever the scheme, so the separate
"undecryptable" loss bucket folds into one loss count.
- **`aacs` module reorganized.** The former god-module is split into
`media_key` / `volume_key` / `inf` / `resolve` / `mkb` / `crypto` / `content`,
the `boil` veneer is removed, and module paths (not a `mod.rs` facade) are the
public API.
### Fixed
- **Main title is chosen by largest physical size, not clip count**, so a
chapter-per-clip disc (e.g. Fast & Furious) is no longer mis-ranked behind a
virtual composite.
- **A fresh-rip ISO `sync_all` failure is no longer swallowed**: `is_regular` is
read from the open file handle instead of a pre-create `metadata(path)` that
always failed on a path that does not exist yet.
- **A transient CLIP-info parse failure no longer suppresses a clip's extents**
for a later playlist item that references the same clip.
- **Reverify downgrades that fail to persist are logged, not swallowed**, so a
bad unit cannot be silently mismarked good on resume.
- **The CLI sanitizes on-disc metadata** (title, volume label, playlist, stream
labels) before printing, so a crafted disc cannot inject terminal escape
sequences.
- **keydb entry validation matches the parser exactly** — a `0x` line counts only
with a ` = ` — so content that parses to zero usable entries can no longer be
saved as valid.
- **autorip** recovers a poisoned config lock in the rip thread instead of
panicking it, and corrects the resume pass count.
- Criterion stream numbering (a map value of 0 no longer shadows stream 1); AACS
resolve classifies a media-keys-only source missing the VID as "VID
unavailable"; a dropped partial PES flags a discontinuity; the no-demuxer path
detects an early consumer disconnect.
### Performance
- **Decrypt thread count is resolved once** and cached off the per-buffer hot
path (the env var and `available_parallelism` are no longer probed per call).
## [1.2.2] — 2026-07-04
### Added
- **AACS 2.1 Media Key Variant support.** The Media Key Variant scheme is now
detected and parsed from the real MKB record types found on variant discs —
`0x2d` (Encrypted Media Key Variant Data), `0x2f` (Variant Key Data table,
65,535 × 16), and `0x0c` (variant cvalues, one per subset-difference slot) —
replacing the earlier placeholder `0x82`/`0x83` types, which were a guess and
appear on no real MKB. The V2.0→V2.1 upgrade detection and fixtures are updated
accordingly, so a genuine AACS 2.1 variant disc now resolves.
- **`resolve_candidate`** — one composed, pure-derivation boil-down for a
candidate key at any ladder rung (DK/PK/MK/VUK → terminal unit keys), parsing
`Unit_Key_RO.inf` at the disc's declared AACS version and returning every CPS
unit key. Consumers stop re-composing the ladder; every client hardens a single
implementation.
### Fixed
- **`mk_from_dk` does the real Subset-Difference walk again.** It previously ran
the Media-Key-Variant path, which needed an integrator KCD absent in-tree and
errored for every real disc — effectively dead for both consumers. It now
performs the genuine device-key SD walk; the Volume ID enters at the VUK step
(where it belongs), not the MK step. This revives the DK→MK fallback across the
toolchain (`freemkv-keysources` adopts the corrected two-argument call).
- **autorip: a down online key service is no longer reported as a missing key.**
When the online key source resolves no key for an encrypted disc, autorip now
runs one bounded reachability probe (SSRF-pinned, ~8 s, no redirects) and
distinguishes a transient outage (transport error / 502·503·504 → down; 429 →
rate-limited) from a genuine no-key (any real HTTP answer → up). A transient
verdict triggers a bounded key-resolution retry (3 attempts, 8/16/32 s backoff)
and, if the service stays down, parks the disc in a distinct retryable state
("Key service unavailable — temporary outage, not a missing key; will retry.")
instead of the permanent "no keys found". Never hammers the drive or service.
### Performance
- **Processing-Key resolution is ~15× faster on UHD.** A Processing Key is the
key at its subset-difference node (one AES-G from the Media Key), so it is now
tried directly against the MKB cvalue tables (direct PK × cvalue iteration)
instead of BFS-walking the SD tree at unknown depth — which was both wrong for
terminal PKs and slow on a large UHD MKB (~181k cvalues). PK derivation on UHD
drops from ~37 s to ~2.4 s; the SD tree walk now lives solely in the device-key
path.
### autorip
- **Clear stuck move errors from the System tab.** Each move-queue error now has
a ✕ to dismiss it, plus Clear all and Refresh — so a resolved or stale error
can be cleared without restarting the container (the mover re-records any that
are still genuinely failing on its next tick).
## [1.2.1] — 2026-07-02
### Fixed
@@ -8,7 +898,7 @@
Program Stream packs several DTS core frames into one PES packet; the parser
stamped every access unit with that single PES timestamp and no per-frame
duration, so consecutive frames collided on one PTS and a strict decode/remux
(ffmpeg) rejected the track — `non monotonically increasing dts to muxer`.
a standard validator rejected the track — `non monotonically increasing dts to muxer`.
The DTS parser now derives each core frame's duration from its header
(`(NBLKS+1)*32` samples ÷ the `SFREQ` sample rate) and re-bases to each PES's
own container timestamp, advancing by a frame duration only *within* a single
@@ -95,7 +985,7 @@ consumers are the in-tree toolchain crates.
frame. The undecryptable aligned unit is concealed as NULL transport-stream
packets (PID 0x1FFF, invisible to every real stream), and the codec layer
**drops forward to the next keyframe** so no frame with a dangling reference
reaches the muxer. An ffmpeg deep scan of the result is clean — no missing
reaches the muxer. A deep validator scan of the result is clean — no missing
references, no partial frames. The loss is tallied and logged, never silently
dropped, and the mux always completes. Audio and subtitle tracks have no
cross-frame references, so only the directly-affected frames are dropped there.
@@ -196,7 +1086,7 @@ consumers are the in-tree toolchain crates.
discontinuity — a continuity-counter break, an adaptation-field
discontinuity_indicator, or a concealed-loss gap — the AC-3 / DTS / TrueHD
parsers held a *truncated* partial access unit and spliced the post-gap bytes
onto it, manufacturing a corrupt frame (ffmpeg "exponent out of range" /
onto it, manufacturing a corrupt frame (a validator reports "exponent out of range" /
"Failed to decode block code(s)" / "Invalid data found") and, for TrueHD, a
non-monotonic timestamp band on multi-segment titles. The video path already
resynced via the keyframe gate; the audio parsers now do too — on a
@@ -204,7 +1094,7 @@ consumers are the in-tree toolchain crates.
syncword, rebasing the timestamp from the post-gap PES. A discontinuity becomes
a clean single-frame gap instead of a corrupt splice. Audio has no inter-frame
references, so dropping the truncated partial is the complete fix; the approach
matches FFmpeg's parser layer and GStreamer's `tsdemux`.
matches how mainstream transport-stream demuxers behave.
- **Drive-prep firmware unlock skipped for DVD discs.** An
`if disc_is_dvd() { return }` guard in `Drive::init()` (present since
1.0.0-rc.1) bypassed the entire drive-prep unlock step for DVDs. That unlock is
@@ -220,7 +1110,7 @@ consumers are the in-tree toolchain crates.
- **Post-read decrypt-verify gate.** Every AACS unit read off the disc is now
buffered, re-aligned to its clip-file 6144-byte unit grid, and verified
(CPI flag → decrypt → strict all-32 TS-sync, matching libaacs `_verify_ts`)
(CPI flag → decrypt → strict all-32 TS-sync)
before it is signed off as good. A unit that no held or freshly-fetched key
decrypts is treated exactly like a bad read — re-read by
the patch pass, terminal loss only if truly unrecoverable — closing the
@@ -241,7 +1131,7 @@ consumers are the in-tree toolchain crates.
- **AACS decrypt acceptance is now standards-strict.** A key is accepted only
when the decrypted unit has the TS sync byte on *all* 32 source packets
(libaacs `_verify_ts`), replacing a majority-vote heuristic where a wrong key
(all-32 TS-sync verify), replacing a majority-vote heuristic where a wrong key
could coincidentally restore enough syncs to pass and silently corrupt a unit.
- keydb download/save moved out of the library into freemkv-keysources;
libfreemkv no longer has any keydb I/O (it already held no keys).
@@ -249,7 +1139,7 @@ consumers are the in-tree toolchain crates.
### Fixed
- **AACS content-certificate bus-encryption flag read from the wrong bit.** The
flag is bit 7 of byte 1 (libaacs `p[1] >> 7`) but was read as bit 0, so a
flag is bit 7 of byte 1 (`p[1] >> 7`) but was read as bit 0, so a
bus-encrypted disc parsed as *not* bus-encrypted — defeating the fail-loud
guard that refuses to decrypt bus-wrapped data to garbage when no bus key was
obtained. Also corrected the cc_id offset (byte 14) and the AACS2 type marker
@@ -347,12 +1237,13 @@ consumers are the in-tree toolchain crates.
SD-DVD.** rc.5.1 added a 20 ms `DefaultDecodedFieldDuration` field element to
the 576i/480i track header on the theory that Windows derives fps from it.
Captured evidence showed that element made Windows Explorer report 12.5 fps
(half) and MediaInfo flip the track to "Frame rate mode: Variable", while
MakeMKV's rip of the same disc omits it. The element is therefore no longer
written (`MkvTrack::video` now passes `field_duration_ns == 0`); the track
keeps `FlagInterlaced=1` + `FieldOrder=TFF` and the full-frame 40 ms
`DefaultDuration` (`1/DefaultDuration` = 25 fps), matching MakeMKV. How a given
player or shell handler chooses to display interlaced fps is not guaranteed.
(half) and MediaInfo flip the track to "Frame rate mode: Variable". The
element is optional in RFC 9559 and nothing requires it for interlaced SD, so
it is no longer written (`MkvTrack::video` now passes `field_duration_ns == 0`);
the track keeps `FlagInterlaced=1` + `FieldOrder=TFF` and the full-frame 40 ms
`DefaultDuration` (`1/DefaultDuration` = 25 fps), which is the frame rate the
source actually carries. How a given player or shell handler chooses to display
interlaced fps is not guaranteed.
- **Correct AC-3 audio track selected on DVDs with non-standard sub-stream
ordering.** freemkv assigned each declared audio stream a physical sub-stream
by ordinal (`0x80+n`), assuming the IFO's first stream lives at `0x80`. On
@@ -372,7 +1263,7 @@ consumers are the in-tree toolchain crates.
The direct-to-MKV path now gives the drive its full ECC recovery budget on a
bad sector (matching the multipass rip) instead of reporting a read failure a
multipass rip would have recovered.
- **4K decode glitches at non-seamless clip joins fixed (Top Gun class).**
- **4K decode glitches at non-seamless clip joins fixed.**
Titles assembled from clips joined at non-seamless boundaries no longer drop
reference frames at the join ("Could not find ref" stutter); the splice
keyframe is rewritten so the decoder discards only the genuinely-dangling
@@ -549,7 +1440,7 @@ hardening.
- **Keyless DVD/CSS title-key recovery.** A CSS-protected DVD decrypts with no
key database — the title key is recovered directly from the scrambled disc
data via the Stevenson known-plaintext attack (ported from libdvdcss) and
data via the Stevenson known-plaintext attack and
validated by descrambling a sector and confirming the known plaintext
reappears, so a wrong key fails cleanly instead of producing silent garbage
(`src/css/stevenson.rs`). `Disc::scan_image` recovers the same title key from
+1 -1
View File
@@ -36,7 +36,7 @@ This Code of Conduct applies within all community spaces, and also applies when
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at owner@example.invalid. All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+1 -1
View File
@@ -24,4 +24,4 @@ cargo test
## License
By contributing, you agree your code will be licensed under AGPL-3.0.
By contributing, you agree your code will be licensed under MIT.
+12 -5
View File
@@ -1,9 +1,9 @@
[package]
name = "libfreemkv"
version = "1.2.1"
version = "1.6.0"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
rust-version = "1.97"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
keywords = ["bluray", "uhd", "optical", "scsi", "disc"]
@@ -28,7 +28,7 @@ cbc = "0.1"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.2.1" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.0" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
@@ -36,7 +36,14 @@ rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
+17 -12
View File
@@ -1,16 +1,21 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
MIT License
Copyright (C) 2026 FreeMKV Contributors
Copyright (c) 2026 Matthew Jackson & Contributors
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, version 3 of the License.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+11 -44
View File
@@ -1,4 +1,4 @@
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
# libfreemkv
@@ -55,48 +55,15 @@ output.finish()?;
### Multi-pass recovery rip
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md).
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the
ddrescue mapfile, damage classification and the multipass loop now live in the
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`.
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
libfreemkv keeps the layers underneath: the raw single-shot read
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's
strategy is built on. The dependency runs engine → libfreemkv, so this crate
cannot call into it; front-ends get recovery from the engine directly. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
## What It Does
@@ -114,7 +81,7 @@ loop {
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
@@ -190,4 +157,4 @@ Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/fre
## License
AGPL-3.0-only
MIT
+3 -3
View File
@@ -141,8 +141,8 @@ If your machine has a free SATA port, use it.
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
@@ -302,7 +302,7 @@ Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not adver
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
2. **Drive model:** from the drive label, or from `freemkv info disc://`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
+15 -8
View File
@@ -22,7 +22,7 @@ fn main() {
&target_arch // x86_64 → x86_64
};
std::process::Command::new("cc")
let cc_status = std::process::Command::new("cc")
.args([
"-arch",
clang_arch,
@@ -38,12 +38,19 @@ fn main() {
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
.expect("failed to spawn cc for macos_shim.c");
// `.status()` succeeding only means the process RAN. A real compile error
// exits non-zero, and ignoring that left no object file, which surfaced
// much later as an unexplained link failure against a missing symbol. The
// shim is macOS-only and is neither linted nor compiled on the other two
// platforms, so a mistake in it has exactly one chance to be noticed.
assert!(cc_status.success(), "cc failed to compile macos_shim.c");
std::process::Command::new("ar")
let ar_status = std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
.expect("failed to spawn ar");
assert!(ar_status.success(), "ar failed to create the static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
@@ -77,10 +84,10 @@ fn emit_git_suffix() {
// Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD") {
if let Some(ref_path) = head.strip_prefix("ref: ") {
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
if let Ok(head) = std::fs::read_to_string(".git/HEAD")
&& let Some(ref_path) = head.strip_prefix("ref: ")
{
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
+27 -16
View File
@@ -65,27 +65,38 @@ if disc.encrypted {
}
}
// Read content -- decryption is automatic
let mut reader = disc.open_title(&mut session, 0).unwrap();
while let Some(unit) = reader.read_unit().unwrap() {
// decrypted content
// Read content -- decryption is applied on read by the DiscStream decorator.
// Live disc does NOT go through the URL resolver: `input("disc://...")` returns
// Error::DiscUrlNotDirect by design.
let keys = disc.decrypt_keys();
let mut stream = DiscStream::new(
Box::new(drive),
disc.titles[0].clone(),
keys,
batch_sectors,
disc.titles[0].content_format,
false, // raw: false → decrypt on read
None, // halt
)?;
while let Ok(Some(frame)) = stream.read() {
// decrypted PES frames
}
```
The application never touches keys, never calls decryption functions, and never
manages handshakes. All of that is internal to `Disc::scan()` and the content
reader.
The application never calls decryption functions and never manages the
drive-level handshake. It DOES own key resolution — see below.
### KEYDB Location
### Key resolution is the caller's job
`ScanOptions` controls where the keydb is loaded from. If no explicit path is
set, the library checks the standard config locations. To specify an explicit
path:
`libfreemkv` is **lookup-free: it resolves no keys and reads no keydb.** There is
no `ScanOptions::with_keydb`, and `ScanOptions` has no keydb field — its only
scan input is the optional drive credentials for the live-drive authenticated
handshake.
```rust
let opts = ScanOptions::with_keydb("/path/to/keydb.cfg");
let disc = Disc::scan(&mut session, &opts).unwrap();
```
The caller resolves a key out-of-band through a key source and applies it with
[`Disc::decrypt_with`]. `freemkv-keysources` is the crate that implements the
keydb and key-server sources; `ScanOptions::key_sources` takes them as
`Box<dyn KeySource>`.
### AacsState
@@ -97,7 +108,7 @@ After a successful scan, `disc.aacs` contains an `AacsState`:
| `bus_encryption` | `bool` | Whether bus encryption is active |
| `mkb_version` | `Option<u32>` | MKB version from disc |
| `disc_hash` | `String` | Identifier for the disc's key-input files |
| `key_source` | `KeySource` | How the disc's key was resolved |
| `key_source` | `KeyOrigin` | How the disc's key was resolved |
## keydb.cfg
+10 -7
View File
@@ -170,17 +170,20 @@ libfreemkv/src/
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── mod.rs Drive struct, init, read (single-shot), eject
│ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
│ ├── mod.rs Disc struct, scan, titles, formats; Disc::copy + Disc::sweep (Pass 1)
│ ├── sweep.rs Pass 1 internal helpers (pub(super))
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── mapfile.rs ddrescue-format mapfile
│ └── read_error.rs ReadCtx / ReadAction state machine
├── disc/ Disc (scan, titles, AACS setup, per-format parsing)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── bluray.rs Blu-ray / UHD scanning (MPLS/CLPI-driven)
│ ├── dvd.rs DVD-Video scanning (IFO-driven)
│ ├── hddvd.rs HD-DVD scanning
│ ├── extract.rs Per-extent content extraction
│ ├── encrypt.rs Encrypted-range mapping for content reads
│ ├── dvd_audio_probe.rs DVD audio-stream probing
│ └── pgs_forced_probe.rs PGS forced-subtitle probing
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── unlock.rs Unlocker trait + registry (pluggable unlock seam)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
+4 -3
View File
@@ -7,7 +7,7 @@ AACS decryption requires an external `keydb.cfg` (default
material is compiled in; DVD CSS player keys are the only compiled-in keys.
**Repository:** <https://github.com/freemkv/libfreemkv>
**License:** AGPL-3.0-only
**License:** MIT
---
@@ -96,12 +96,13 @@ After open:
- `init()` -- routes to the matching registered unlocker (if any); otherwise
a no-op and the cert handshake carries the disc
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (`READ_TIMEOUT_MS` 10 s vs. `READ_RECOVERY_TIMEOUT_MS` 60 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
(`freemkv_engine::recovery::patch`, in the engine crate) handles bad-range
retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
+15 -5
View File
@@ -69,13 +69,23 @@ Each stream PID entry header (14 bytes):
```
Offset Size Field
------ ---- -----
0 2 Stream PID
2 2 Reserved + EP stream type
4 2 Number of coarse entries
6 4 Number of fine entries (note: 32-bit, can be large)
10 4 EP map start offset (relative to EP map start)
2 2 stream_PID (byte-aligned)
4 10 Bit-packed block, 80 bits total (see below)
```
The stream PID entry is **not** byte-aligned past `stream_PID`. Bytes 4..14 are one
80-bit packed field, read as a `u64` plus a trailing `u16`:
Bits Width Field
---- ----- -----
0-9 10 reserved
10-13 4 EP_stream_type
14-29 16 num_EP_coarse
30-47 18 num_EP_fine
48-79 32 EP_map_start_address (relative to the EP map start)
Note `num_EP_fine` is **18 bits**, not 32. See `parse_cpi` in `src/clpi.rs`.
libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking.
### Two-Level Index
+2 -2
View File
@@ -79,7 +79,7 @@ Insert disc
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — Disc::patch re-runs against the mapfile.
│ 1 above this — freemkv_engine::recovery::patch re-runs against the mapfile.
│
▼
PES frames → output stream (MKV, M2TS, network, etc.)
@@ -123,7 +123,7 @@ output.finish()?;
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan (sweep/patch/mapfile moved to freemkv-engine in 1.6.0) |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
+4 -3
View File
@@ -58,15 +58,16 @@ selects the per-CDB timeout:
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
| `false` | 10 s | `freemkv_engine::recovery::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 60 s | `freemkv_engine::recovery::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
- **Layer 1 — `freemkv_engine::recovery::patch`** (in the engine crate, not
here) loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
+64 -167
View File
@@ -1,135 +1,38 @@
# Rip recovery — three-layer architecture
# Rip recovery — what libfreemkv owns
`libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
discs: a fast forward sweep that tolerates read failures, in-loop request-size
adaptation that survives transient drive trouble without bailing, and targeted
retry passes against a persistent bad-range map. The stream pipeline
(`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
the mux stage never touches the drive.
**Recovery strategy moved OUT of this crate in 1.6.0.** The forward sweep, the
targeted retry pass, the ddrescue mapfile, damage classification and the
multipass loop now live in the **`freemkv-engine`** crate as
`freemkv_engine::recovery::{copy, sweep, patch}`. The dependency runs
engine → libfreemkv, so this crate cannot call into the engine; front-ends
(`freemkv` CLI, autorip) get recovery from the engine directly.
Recovery is layered cleanly. Each layer has one responsibility and does not
reach into the others.
What stayed here are the two layers underneath the strategy: the single-shot
read primitive, and the in-stream request-size adaptation that sits in front of
it. This document covers those, plus the design constraints they encode — the
constraints are the reason the strategy above them looks the way it does, so
they belong with the code that enforces them.
For the strategy itself — damage-jump thresholds, pass ordering, mapfile status
state machine, wedge detection — read `freemkv-engine/src/recovery/`.
| Layer | Where it lives | What it does |
|-------|---------------|--------------|
| 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 1 — Bad-range retry | **`freemkv-engine`** (`recovery::patch`) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` in `src/mux/disc.rs` | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
The library exposes flat verbs; the caller drives the multipass loop. Autorip
runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
clean or the configured retry budget is exhausted, then hands the ISO off to
the mux pipeline. The `freemkv` CLI does the same shape with a
terminal-output progress sink. Layer 3 runs inside any consumer of
`DiscStream` (direct PES pipeline, ISO playback, etc.) without caller
involvement.
Layer 2 also translates drive facts: [`SenseFamily`](../src/scsi/mod.rs)
classifies SCSI sense data into the categories the engine's strategy routes on
(marginal vs. hardware vs. not-ready). Getting that classification wrong
silently misroutes recovery, which is why it lives next to the transport rather
than in the strategy.
Three primitives compose the disc-side flow:
Layer 3 runs inside any consumer of `DiscStream` — direct PES pipeline, ISO
playback — without caller involvement, and applies whether or not the engine's
recovery is in play.
| Primitive | What it does |
|---------------------------|-----------------------------------------------------------------------|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
## Data model
### Mapfile
Format: [ddrescue](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)-compatible
plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
so a crashed rip loses at most one block.
```
# Rescue Logfile. Created by libfreemkv v0.13.6
# Current pos / status / pass / pass_time
0x000000000 ? 1 0
# pos size status
0x000000000 0x12a35d000 +
0x12a35d000 0x000003000 -
0x12a360000 0x009c4a000 +
0x12d00a000 0x000064000 *
```
Status characters match ddrescue:
| Char | Meaning |
|------|----------------------------------------------------|
| `?` | Not yet attempted |
| `*` | Fast-pass failed; needs edge-trim |
| `/` | Trimmed; interior needs sector scrape |
| `-` | Unreadable this session |
| `+` | Finished (good) |
Position and size are hex byte offsets into the ISO.
### `SweepOptions` and `PatchOptions`
The library no longer dispatches between sweep and patch internally — the
caller picks the verb explicitly per pass. The two option structs are flat
and have no overlap:
```rust
SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true, // damage-jump + zero-fill on read failure
progress: Some(&reporter),
halt: Some(flag),
}
PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true, // walk bad ranges high → low LBA
wedged_threshold: 50,
progress: Some(&reporter),
halt: Some(flag),
}
```
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
- No mapfile → `sweep` (fresh Pass 1).
- Mapfile with `?` ranges → `sweep` with `resume: true`.
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
- Mapfile clean → done; no further pass needed.
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
lines of `Mapfile::stats()` checks.
## Algorithm
### Pass 1 — fast sweep (`Disc::sweep`)
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
→ **damage-jump**: skip ahead by `1024×batch×multiplier` sectors (64 MB base for
UHD). Double the multiplier on each jump (64→128→256→512 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
## In-stream — adaptive batch halving (`DiscStream::fill_extents`)
When a consumer reads a `DiscStream` directly (no ISO intermediate),
`fill_extents` runs an adaptive sizer in front of `Drive::read`:
@@ -143,59 +46,53 @@ When a consumer reads a `DiscStream` directly (no ISO intermediate),
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
`Err(DiscRead)`.
This is layer 3. It exists so a transient single-sector glitch in a 32-sector
batch can be isolated and read individually without the caller needing to
implement retry logic.
This exists so a transient single-sector glitch inside a 32-sector batch can be
isolated and read individually without the caller implementing retry logic. See
[`src/event.rs`](../src/event.rs) for the emitted events.
## Design choices
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
handles ranges, layer 3 handles request size, neither touches the
wedge-prone reset path.
These are constraints on the read path, enforced here and relied on by the
engine's strategy.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue
nor any consumer ripper does this. Drive firmware has access to raw analog signal, laser
power control, and drive-specific ECC tuning that userspace can't replicate —
disabling it throws away recovery headroom on marginal sectors. We fail fast
via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
in pass 2 / patch.
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset, no
eject cycle. The `recovery` flag controls only the per-CDB timeout (10 s vs.
60 s); on any failure it returns `Err(DiscRead)` immediately. Inline recovery
(5× gentle retry → close + SCSI reset + reopen → 5× more) was removed in
0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale: the inline
reset on the LG BU40N (Initio USB-SATA bridge) wedged drive firmware below the
bridge without ever recovering a sector, and the gentle-retry phase produced
long stretches of 0 KB/s with nothing to show for it. Recovery responsibility
is layered instead: layer 1 handles ranges, layer 3 handles request size,
neither touches the wedge-prone reset path.
**No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
`SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
The macOS reset (which had been a no-op) was removed entirely. The top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
also removed (no callers). The remaining `Drive::reset()` is only invoked
explicitly by callers that need an eject-cycle escape hatch — it is never
reached from a read path.
**No `MODE SELECT` to disable drive retries.** Neither ddrescue nor any
consumer ripper does this. Drive firmware has access to raw analog signal,
laser power control and drive-specific ECC tuning that userspace cannot
replicate — disabling it throws away recovery headroom on marginal sectors. The
fast pass fails quickly via short SG_IO timeouts and lets the firmware work the
long timeout during retry.
**ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
mux stage reads the ISO via `FileSectorSource`. For single-pass (no retries),
this adds ~2-3 min (local disk mux) but gains resumability across crashes,
**No SCSI reset from any read path.** There is no reset escape hatch on
`Drive` at all: the `SG_SCSI_RESET` ioctl and STOP/START UNIT escalation went in
0.13.6, the macOS reset (always a no-op) was removed entirely, and the
top-level `scsi::reset()` wrappers went with their last callers. The only
remaining reset is a Windows-specific device-level helper in
[`src/scsi/windows.rs`](../src/scsi/windows.rs), never reached from a read.
**ISO intermediate, even for single-pass.** The engine's Pass 1 always writes
an ISO, and the mux stage reads it back via `FileSectorSource`. For a
no-retry rip this costs a few minutes but buys resumability across crashes,
re-muxability without re-ripping, and a persistent forensic artifact. Callers
who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
still applies there.
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
coalesce on adjacent same-status ranges so files stay small.
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
ddrescue's but `patch` collapses the distinction — it just tries every
non-finished range with the long timeout. Future work can specialize (trim vs.
scrape vs. retry with direction reversal) if there's measured benefit.
who need pure speed can bypass it with `DiscStream::new(Box::new(drive), …)` —
nothing forbids it, and layer 3 still applies there.
## References
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
- Recovery strategy and mapfile: `freemkv-engine/src/recovery/`
- In this crate: [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`),
[`src/scsi/mod.rs`](../src/scsi/mod.rs) (`SenseFamily`),
[`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`),
[`src/event.rs`](../src/event.rs) (progress events).
+1 -1
View File
@@ -114,7 +114,7 @@ The `read_filesystem()` function in `src/udf.rs` follows the pointer chain above
2. Scans sectors 32-63 for the Partition Descriptor and Logical Volume Descriptor.
3. If two partition maps exist and the second is Type 2, reads the metadata file ICB at partition_start to find metadata_start.
4. Reads the FSD at metadata_start, extracts the root directory ICB LBA.
5. Calls `read_directory()` recursively (max depth 3) to build the full file tree.
5. Calls `read_directory()` recursively (max depth `MAX_DIR_DEPTH` = 8) to build the full file tree.
Each directory read involves two sector reads: one for the ICB, then one or more for the directory data. File sizes are read from info_length in each file's ICB.
-308
View File
@@ -1,308 +0,0 @@
//! AACS derivation "boil-down" — one public home for the key chain.
//!
//! Thin newtypes at the API boundary and three wrapper functions over the
//! existing crypto. Nothing here re-implements a primitive: every function
//! delegates to the already-audited code in [`super::keys`] and
//! [`super::variants`], so the boil-down cannot drift from production math.
//!
//! The newtypes wrap bare `[u8; 16]` ONLY at this boundary — the crypto
//! internals continue to operate on raw arrays. They exist so a caller threads
//! the chain `DK → MK → VUK → UK` without confusing one 16-byte secret for
//! another, not to refactor the resolver.
//!
//! Chain (matches `aacs::keys::resolve_keys_classical` path 1 and
//! `aacs::keys::resolve_keys_v21` path 1 byte-for-byte):
//!
//! ```text
//! mk_from_dk(device_keys, mkb, vid) → MediaKey (Km)
//! mk_from_pk(processing_keys, mkb) → MediaKey (Km)
//! vuk_from_mk(MediaKey, Vid) → Vuk (= AES-G(Km, VID))
//! uk_from_vuk(Vuk, enc_title_keys) → [UnitKey] (decrypt_unit_key each)
//! ```
//!
//! `mk_from_dk` and `mk_from_pk` are two entry points to the SAME Media Key:
//! the device-key path walks the MKB's Media-Key-Variant chain, the
//! processing-key path walks the MKB's Subset-Difference cvalue tables. Neither
//! needs a VID (the VID enters at `vuk_from_mk`).
use super::keys::{decrypt_unit_key, derive_media_key_from_pk, derive_vuk};
use super::types::DeviceKey;
use super::variants::{KEY_CORRECTION_DATA_PLACEHOLDER, derive_media_key_variant, walk_mkb};
/// Volume ID (16 bytes) — read from the disc via the SCSI handshake / OEM path.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Vid(pub [u8; 16]);
/// Media Key (Km, 16 bytes) — the MKB-scoped key derived from device keys.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct MediaKey(pub [u8; 16]);
/// Volume Unique Key (VUK / Kvu, 16 bytes) — derived from `MediaKey` + `Vid`,
/// decrypts the per-disc encrypted title keys in `Unit_Key_RO.inf`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Vuk(pub [u8; 16]);
/// One decrypted per-CPS-unit AACS title key.
///
/// `idx` is the POSITIONAL index of the encrypted title key within the slice
/// handed to [`uk_from_vuk`] (i.e. its order in `Unit_Key_RO.inf`'s key-storage
/// area). The CPS-unit *number* association (the `u32` in
/// `ResolvedKeys::unit_keys`) is a higher-level concern owned by
/// [`super::keys::parse_unit_key_ro`], which pairs each positional key with its
/// declared CPS unit; this primitive only does the AES, so it surfaces position.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct UnitKey {
pub idx: u32,
pub key: [u8; 16],
}
/// Derive the Volume Unique Key from a Media Key and Volume ID.
///
/// Wraps [`derive_vuk`] verbatim: `VUK = AES-128-ECB-DECRYPT(MK, VID) XOR VID`.
/// This is byte-identical to the inline `derive_vuk(&mk, ctx.volume_id)` call in
/// every classical resolver path AND to the `Kvu = AES-G(Km, VID)` step inside
/// [`derive_media_key_variant`] (AES-G and `derive_vuk` are the same math), so
/// `vuk_from_mk(mk_from_dk(..)?, vid)` reproduces the V21 variant VUK exactly.
pub fn vuk_from_mk(mk: MediaKey, vid: Vid) -> Vuk {
Vuk(derive_vuk(&mk.0, &vid.0))
}
/// Decrypt the disc's encrypted title keys with a VUK.
///
/// Wraps [`decrypt_unit_key`] (AES-128-ECB-DECRYPT) per entry, mirroring the
/// `derive_uks` closure in `resolve_keys_classical` / `resolve_keys_v21`. The
/// returned `UnitKey::idx` is the slice position; pair with CPS-unit numbers via
/// [`super::keys::parse_unit_key_ro`] when the numbering matters.
pub fn uk_from_vuk(vuk: Vuk, enc_title_keys: &[[u8; 16]]) -> Vec<UnitKey> {
enc_title_keys
.iter()
.enumerate()
.map(|(i, enc)| UnitKey {
idx: i as u32,
key: decrypt_unit_key(&vuk.0, enc),
})
.collect()
}
/// Derive the Media Key (Km) from device keys via the Media Key Variant chain.
///
/// Wraps [`walk_mkb`] + [`derive_media_key_variant`] with exactly the arguments
/// `resolve_keys_v21` path 1 passes: the placeholder Key Correction Data and the
/// disc Volume ID. Returns the FIRST tuple element `Km` (the Media Key) — the
/// resolver treats `Km` as the media key and derives the VUK from it as
/// `Kvu = AES-G(Km, VID)`, which equals [`vuk_from_mk`]`(MediaKey(km), vid)`. The
/// variant fn's second element is that already-derived `Kvu`; returning `Km`
/// keeps this primitive at the "media key" level so the chain composes.
///
/// Because the integrator KCD is unavailable in-tree (the placeholder is
/// rejected by the variant chain), this returns `Err` for every real disc today
/// — byte-for-byte identical to `resolve_keys_v21` path 1, which the resolver
/// also leaves unreachable in production. All variant-chain failures collapse to
/// [`Error::AacsMkUnavailable`] (E7018): no numeric distinction is load-bearing
/// at this boundary, and the variant error carries no English to preserve.
pub fn mk_from_dk(
device_keys: &[DeviceKey],
mkb: &[u8],
vid: Vid,
) -> Result<MediaKey, crate::error::Error> {
let records = walk_mkb(mkb);
match derive_media_key_variant(
&records,
device_keys,
&KEY_CORRECTION_DATA_PLACEHOLDER,
&vid.0,
) {
Ok((km, _kvu)) => Ok(MediaKey(km)),
Err(_) => Err(crate::error::Error::AacsMkUnavailable),
}
}
/// Derive the Media Key (Km) from one or more Processing Keys and the disc MKB.
///
/// Wraps [`derive_media_key_from_pk`] — the Subset-Difference PK→MK walk: each
/// processing key is validated (and tree-walked) against the MKB's cvalue tables
/// (records `0x04`/`0x05`) until one yields the Media Key whose verify record
/// (`0x81`/`0x86`) matches. Unlike [`mk_from_dk`] this path is reachable for
/// real discs — a leaked/precomputed AACS Processing Key in the keydb resolves
/// the Media Key directly. No VID is involved at this step; the VID enters at
/// [`vuk_from_mk`].
///
/// Returns [`Error::AacsMkUnavailable`] (E7018) when no processing key resolves
/// the MKB — the same terminal error as [`mk_from_dk`]; no numeric distinction
/// is load-bearing at this boundary.
///
/// [`Error::AacsMkUnavailable`]: crate::error::Error::AacsMkUnavailable
pub fn mk_from_pk(
processing_keys: &[[u8; 16]],
mkb: &[u8],
) -> Result<MediaKey, crate::error::Error> {
match derive_media_key_from_pk(mkb, processing_keys) {
Some(km) => Ok(MediaKey(km)),
None => Err(crate::error::Error::AacsMkUnavailable),
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::aacs::decrypt::aes_ecb_encrypt;
use crate::aacs::keys::{decrypt_unit_key, derive_vuk};
/// `vuk_from_mk` must equal the inline `derive_vuk` path bit-for-bit, for
/// several known (MK, VID) vectors.
#[test]
fn vuk_from_mk_matches_inline_derive_vuk() {
let cases: [([u8; 16], [u8; 16]); 3] = [
([0x5A; 16], [0xA5; 16]),
([0x11; 16], [0x22; 16]),
(
[
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C,
0x0D, 0x0E, 0x0F,
],
[
0xF0, 0xE1, 0xD2, 0xC3, 0xB4, 0xA5, 0x96, 0x87, 0x78, 0x69, 0x5A, 0x4B, 0x3C,
0x2D, 0x1E, 0x0F,
],
),
];
for (mk, vid) in cases {
let inline = derive_vuk(&mk, &vid);
let boiled = vuk_from_mk(MediaKey(mk), Vid(vid));
assert_eq!(boiled.0, inline, "vuk_from_mk must equal derive_vuk");
}
}
/// `uk_from_vuk` must equal the inline `decrypt_unit_key` path bit-for-bit
/// and carry positional indices 0..n. Built by encrypting known plaintext
/// title keys under the VUK (the same primitive the resolver inverts).
#[test]
fn uk_from_vuk_matches_inline_decrypt_unit_key() {
let vuk = [0x5Au8; 16];
let plain_keys = [[0x11u8; 16], [0x22u8; 16], [0xCDu8; 16]];
let enc: Vec<[u8; 16]> = plain_keys
.iter()
.map(|k| aes_ecb_encrypt(&vuk, k))
.collect();
let boiled = uk_from_vuk(Vuk(vuk), &enc);
assert_eq!(boiled.len(), enc.len());
for (i, uk) in boiled.iter().enumerate() {
assert_eq!(uk.idx, i as u32, "idx must be the positional index");
// Matches the inline derive_uks closure: decrypt_unit_key(vuk, enc).
assert_eq!(uk.key, decrypt_unit_key(&vuk, &enc[i]));
// And recovers the original plaintext title key.
assert_eq!(
uk.key, plain_keys[i],
"VUK roundtrip recovers the title key"
);
}
}
/// `uk_from_vuk` on an empty slice yields no keys (no panic, no phantom idx).
#[test]
fn uk_from_vuk_empty_is_empty() {
assert!(uk_from_vuk(Vuk([0u8; 16]), &[]).is_empty());
}
/// `mk_from_dk` returns `Err(AacsMkUnavailable)` for the placeholder-KCD
/// path that production also leaves unreachable — never a wrong key, never a
/// panic — on both an empty MKB and a non-variant MKB.
#[test]
fn mk_from_dk_errors_without_integrator_kcd() {
let dk = DeviceKey {
key: [0x11; 16],
node: 1,
uv: 1,
u_mask_shift: 0,
};
// Empty MKB → not a variant MKB → Err.
let e = mk_from_dk(std::slice::from_ref(&dk), &[], Vid([0x09; 16]));
assert!(matches!(e, Err(crate::error::Error::AacsMkUnavailable)));
// A variant-looking MKB (0x82 record) still cannot complete without the
// integrator KCD, so it also errors — never silently yields a key.
let mut mkb: Vec<u8> = Vec::new();
mkb.extend_from_slice(&[0x82, 0x00, 0x00, 0x14]); // variant data record
mkb.extend_from_slice(&[0xAB; 16]);
let e2 = mk_from_dk(&[dk], &mkb, Vid([0x09; 16]));
assert!(matches!(e2, Err(crate::error::Error::AacsMkUnavailable)));
}
/// Build a 4-byte MKB record header (type + 3-byte big-endian total length,
/// header included) and append `body`. Mirrors the MKB record framing the
/// parser expects; no crypto.
fn mkb_record(rec_type: u8, body: &[u8]) -> Vec<u8> {
let total = 4 + body.len();
let mut rec = Vec::with_capacity(total);
rec.push(rec_type);
rec.push(((total >> 16) & 0xFF) as u8);
rec.push(((total >> 8) & 0xFF) as u8);
rec.push((total & 0xFF) as u8);
rec.extend_from_slice(body);
rec
}
/// `mk_from_pk` resolves a planted Processing Key against a synthetic MKB and
/// drives the FULL boil chain PK → MK → VUK → UK. The MKB is built with the
/// same (pk, cv, mk_dv, uv) construction the production SD walk validates, so
/// this proves a PK entry yields real Unit Keys — not just an `Ok`.
#[test]
fn mk_from_pk_drives_full_chain_to_uks() {
let pk: [u8; 16] = [
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE,
0xFF, 0x00,
];
let mk: [u8; 16] = [
0xA0, 0xA1, 0xA2, 0xA3, 0xA4, 0xA5, 0xA6, 0xA7, 0xA8, 0xA9, 0xAA, 0xAB, 0xAC, 0xAD,
0xAE, 0xAF,
];
let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00];
// cv = AES-E(pk, mk_raw), where mk_raw is mk with the last-4-bytes-uv XOR
// pre-undone, so the validate step XORs uv back in and recovers mk.
let mut mk_raw = mk;
for a in 0..4 {
mk_raw[12 + a] ^= uv[a];
}
let cv = aes_ecb_encrypt(&pk, &mk_raw);
// mk_dv = AES-E(mk, magic||pad): AES-D(mk, mk_dv) starts with the AACS
// verify sentinel.
let mut vd = [0x11u8; 16];
vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]);
let mk_dv = aes_ecb_encrypt(&mk, &vd);
// Synthetic MKB: type/version (0x10), verify record (0x86 = mk_dv),
// one-entry SD index (0x04 = [u_mask_shift=0][uv]), one-entry cvalue
// table (0x05 = cv).
let mut sd = vec![0u8];
sd.extend_from_slice(&uv);
let mut mkb = Vec::new();
mkb.extend_from_slice(&mkb_record(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
mkb.extend_from_slice(&mkb_record(0x86, &mk_dv));
mkb.extend_from_slice(&mkb_record(0x04, &sd));
mkb.extend_from_slice(&mkb_record(0x05, &cv));
// PK → MK.
let got_mk = mk_from_pk(std::slice::from_ref(&pk), &mkb).expect("planted PK resolves MK");
assert_eq!(got_mk, MediaKey(mk), "mk_from_pk recovers the planted MK");
// MK → VUK → UK over an encrypted title key.
let vid = Vid([0x42u8; 16]);
let plain_uk = [0x7Eu8; 16];
let vuk = vuk_from_mk(got_mk, vid);
let enc = aes_ecb_encrypt(&vuk.0, &plain_uk);
let uks = uk_from_vuk(vuk, std::slice::from_ref(&enc));
assert_eq!(uks.len(), 1);
assert_eq!(uks[0].key, plain_uk, "PK chain recovers the title key");
// A corrupt PK resolves nothing.
let mut bad = pk;
bad[0] ^= 0xFF;
assert!(matches!(
mk_from_pk(std::slice::from_ref(&bad), &mkb),
Err(crate::error::Error::AacsMkUnavailable)
));
}
}
+1665
View File
File diff suppressed because it is too large Load Diff
+266
View File
@@ -0,0 +1,266 @@
//! AACS common cryptographic primitives — [C] Chapter 2 / §3.2.2.
//!
//! Source: `[C]` = AACS Introduction and Common Cryptographic Elements Book,
//! Rev 0.953. The shared low-level building blocks — AES-128 ECB E/D, AES-G,
//! the AES-G3 Triple Generator, AES-CBC decrypt — and their fixed constants
//! (`iv0`, `s0`). Used by every AACS generation; relocated here so the
//! primitives live in one place instead of being scattered across the
//! content / keys / variant modules.
use aes::Aes128;
use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit, generic_array::GenericArray};
/// Fixed IV used by AACS for all AES-CBC operations. [C] §2.1.2 (default CBC IV, `iv0`).
pub(crate) const AACS_IV: [u8; 16] = [
0x0B, 0xA0, 0xF8, 0xDD, 0xFE, 0xA6, 0x1F, 0xB3, 0xD8, 0xDF, 0x9F, 0x56, 0x6A, 0x05, 0x0F, 0x78,
];
// Per-thread count of AES-128 key schedules built through `new_cipher`.
// Test-only instrumentation: an AES-128 key expansion is 10 round-key
// derivations, and the CBC helpers here run on the per-aligned-unit decrypt hot
// path of a whole disc read, so "how many times was the schedule built for one
// loop-invariant key" is a property worth asserting rather than reasoning about.
// THREAD-LOCAL, not a global atomic: `cargo test` runs tests concurrently, so a
// shared counter would see every other test's expansions. See
// `content::tests::decrypt_bus_expands_the_read_data_key_once_per_unit`.
#[cfg(test)]
thread_local! {
pub(crate) static KEY_EXPANSIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Build an AES-128 key schedule for a caller that will drive
/// [`cbc_decrypt_blocks`] over several regions under one key.
pub(crate) fn new_cipher_for(key: &[u8; 16]) -> Aes128 {
new_cipher(key)
}
/// Build an AES-128 key schedule. The single construction site for the CBC
/// helpers, so [`KEY_EXPANSIONS`] can count them under test.
fn new_cipher(key: &[u8; 16]) -> Aes128 {
#[cfg(test)]
KEY_EXPANSIONS.with(|c| c.set(c.get() + 1));
Aes128::new(GenericArray::from_slice(key))
}
/// AES-128-ECB encrypt a single 16-byte block. [C] §2.1.1 (`AES-128E`).
pub(crate) fn aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
cipher.encrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
out
}
/// AES-128-ECB decrypt a single 16-byte block. [C] §2.1.1 (`AES-128D`).
pub(crate) fn aes_ecb_decrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
cipher.decrypt_block(&mut block);
let mut out = [0u8; 16];
out.copy_from_slice(&block);
out
}
/// AES-128-CBC ENCRYPT in place under the fixed [`AACS_IV`] — the forward
/// direction of [`aes_cbc_decrypt`], and its exact inverse. [C] §2.1.2
/// (`AES-128CBCE`).
///
/// Precondition: `data.len()` is a multiple of 16; the assert
/// documents/enforces that contract.
///
/// Constructs the cipher ONCE for the whole slice. Driving this from the
/// single-block [`aes_ecb_encrypt`] instead rebuilds the AES key schedule per
/// 16-byte block, which for a 6144-byte aligned unit is 383 redundant key
/// expansions.
pub(crate) fn aes_cbc_encrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len().is_multiple_of(16),
"aes_cbc_encrypt requires a block-aligned slice"
);
let cipher = new_cipher(key);
let num_blocks = data.len() / 16;
let mut prev = AACS_IV;
// Forward order: each block is XORed with the PRECEDING ciphertext block.
for i in 0..num_blocks {
let offset = i * 16;
let mut block = [0u8; 16];
for j in 0..16 {
block[j] = data[offset + j] ^ prev[j];
}
let mut ga = GenericArray::clone_from_slice(&block);
cipher.encrypt_block(&mut ga);
data[offset..offset + 16].copy_from_slice(&ga);
prev.copy_from_slice(&ga);
}
}
/// AES-128-CBC DECRYPT in-place with the fixed AACS IV. [C] §2.1.2
/// (`AES-128CBCD`).
///
/// Precondition: `data.len()` is a multiple of 16. Any trailing partial
/// block is silently ignored; all callers pass aligned regions (6128 and
/// 2032 bytes), and the assert documents/enforces that contract.
///
/// (This doc block was orphaned onto `aes_cbc_encrypt` above when that function
/// was inserted directly after it with no separating blank line, so rustdoc
/// rendered the crate's only forward-direction AACS primitive as "decrypt" and
/// cited the spec's DECRYPT clause for it, while this function had no doc at
/// all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in `content.rs` pins
/// the directions behaviourally so a maintainer 'fixing' the contradiction by
/// swapping the two bodies fails the suite instead of shipping a second
/// decryptor behind an already-set encrypted flag.)
pub(crate) fn aes_cbc_decrypt(key: &[u8; 16], data: &mut [u8]) {
debug_assert!(
data.len().is_multiple_of(16),
"aes_cbc_decrypt requires a block-aligned slice"
);
cbc_decrypt_blocks(&new_cipher(key), data);
}
/// AES-128-CBC decrypt in place under the fixed [`AACS_IV`] with an ALREADY
/// EXPANDED key schedule.
///
/// Split out of [`aes_cbc_decrypt`] so a caller that decrypts several regions
/// under one loop-invariant key expands the schedule once. `decrypt_bus`
/// ([`super::content::decrypt_bus`]) is that caller: bus encryption
/// ([C] §4.2 / the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY
/// 2048-byte sector, so a 6144-byte aligned unit is three regions under one
/// `read_data_key` — three key schedules where one suffices, on the per-unit
/// decrypt hot path of a whole 90 GB read.
pub(crate) fn cbc_decrypt_blocks(cipher: &Aes128, data: &mut [u8]) {
let num_blocks = data.len() / 16;
// Process blocks in reverse to avoid clobbering ciphertext needed for XOR
for i in (0..num_blocks).rev() {
let offset = i * 16;
let prev = if i == 0 {
AACS_IV
} else {
let mut p = [0u8; 16];
p.copy_from_slice(&data[(i - 1) * 16..i * 16]);
p
};
let mut block = GenericArray::clone_from_slice(&data[offset..offset + 16]);
cipher.decrypt_block(&mut block);
for j in 0..16 {
data[offset + j] = block[j] ^ prev[j];
}
}
}
/// AES-G(x1, x2) = AES-128D(x1, x2) XOR x2. [C] §2.1.3 (note: uses AES-128**D**).
///
/// The Media Key Variant chain uses AES-G to derive both the variant
/// number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
/// (`Kvu = AES-G(Km, VID)`). See [`super::derive::derive_vuk`] for the
/// classical VUK form — the math is identical, this exposes it as a
/// neutral primitive for the variant chain.
pub(crate) fn aes_g(x1: &[u8; 16], x2: &[u8; 16]) -> [u8; 16] {
let mut out = aes_ecb_decrypt(x1, x2);
for i in 0..16 {
out[i] ^= x2[i];
}
out
}
/// AACS-G3 seed constant (`s0`). [C] §3.2.2.
pub(crate) const AESG3_SEED: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B, 0xD9,
];
/// AACS-G3: derive a subkey from a parent key. [C] §3.2.2 (Triple AES Generator:
/// left=`D(k,s0)⊕s0` inc 0, pk=`D(k,s0+1)⊕(s0+1)` inc 1, right=`D(k,s0+2)⊕(s0+2)` inc 2).
/// seed[15] += inc, then AES-DEC(key, seed) XOR seed.
///
/// Shared with [`super::variant`] (its variant chain runs the same SD
/// tree); a single definition keeps the two walks byte-identical.
pub(crate) fn aesg3(key: &[u8; 16], inc: u8) -> [u8; 16] {
let mut seed = AESG3_SEED;
seed[15] = seed[15].wrapping_add(inc);
let mut out = aes_ecb_decrypt(key, &seed);
for i in 0..16 {
out[i] ^= seed[i];
}
out
}
#[cfg(test)]
mod tests {
use super::*;
/// The AACS-G3 seed `s0`, transcribed independently from [C] §3.2.2 rather
/// than read from [`AESG3_SEED`] — a test that sourced the seed from the
/// production constant would assert that constant against itself and would
/// still pass if it were edited.
const S0: [u8; 16] = [
0x7B, 0x10, 0x3C, 0x5D, 0xCB, 0x08, 0xC4, 0xE5, 0x1A, 0x27, 0xB0, 0x17, 0x99, 0x05, 0x3B,
0xD9,
];
/// An arbitrary non-degenerate key. Nothing about it is secret or special;
/// the AES-G3 relation holds for every key, and a constant-returning body
/// cannot satisfy it for any.
const K: [u8; 16] = [
0x0F, 0x1E, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0x87, 0x96, 0xA5, 0xB4, 0xC3, 0xD2, 0xE1,
0xF0,
];
/// `aesg3` is the node function of the AACS subset-difference tree: every
/// Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every
/// descent step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that
/// returned a fixed block would make every device key in the crate derive
/// the SAME Processing Key, and a `^` that became `|` or `&` would derive a
/// wrong-but-plausible one — in both cases the MKB walk simply stops
/// finding Media Keys, with no error to say why.
///
/// Pinned through the spec relation rather than a re-implementation:
/// [C] §3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for
/// `s = s0 + inc` (added into the last seed byte), so applying the
/// FORWARD primitive [`aes_ecb_encrypt`] — a different function from the
/// one under test — to `aesg3(k, inc) XOR s` must reproduce `s` exactly.
#[test]
fn aesg3_inverts_to_the_spec_seed_under_aes_encrypt() {
for inc in 0u8..=2 {
let mut seed = S0;
seed[15] = seed[15].wrapping_add(inc);
let out = aesg3(&K, inc);
// out == AES-128D(K, seed) XOR seed, so out XOR seed is the raw
// decryption and re-encrypting it must land back on the seed.
let mut pre = [0u8; 16];
for i in 0..16 {
pre[i] = out[i] ^ seed[i];
}
assert_eq!(
aes_ecb_encrypt(&K, &pre),
seed,
"AES-G3 inc={inc} must satisfy out = AES-128D(k, s0+inc) XOR (s0+inc)"
);
}
}
/// The Triple Generator's three outputs ([C] §3.2.2: left = inc 0, the
/// Processing Key = inc 1, right = inc 2) are the two child node keys and
/// the Processing Key of ONE tree node. They must be three different keys —
/// if `inc` were ignored, a descent would revisit its own parent and the
/// walk would derive the same key at every level of the tree.
#[test]
fn aesg3_yields_three_distinct_subkeys_for_the_three_increments() {
let left = aesg3(&K, 0);
let pk = aesg3(&K, 1);
let right = aesg3(&K, 2);
assert_ne!(left, pk, "left child and Processing Key must differ");
assert_ne!(pk, right, "Processing Key and right child must differ");
assert_ne!(left, right, "left and right children must differ");
}
/// Distinct parent keys must yield distinct subkeys — the tree would
/// collapse otherwise.
#[test]
fn aesg3_separates_distinct_parent_keys() {
let mut other = K;
other[0] ^= 0x01;
assert_ne!(aesg3(&K, 1), aesg3(&other, 1));
}
}
-1501
View File
File diff suppressed because it is too large Load Diff
+1769
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -10,8 +10,8 @@
pub fn collect_host_certs(
opts: &crate::disc::ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::HostCert> {
let mut host_certs: Vec<crate::aacs::HostCert> = Vec::new();
) -> Vec<crate::aacs::types::HostCert> {
let mut host_certs: Vec<crate::aacs::types::HostCert> = Vec::new();
if let Some(c) = &opts.credentials {
host_certs.extend(c.host_certs.iter().cloned());
}
+198
View File
@@ -0,0 +1,198 @@
//! FMTS index selection — the pure decode-time decision for a 2.1 disc.
//!
//! A 2.1 disc resolves to exactly one forensic index (1..=32) for a given
//! rip. `IndividualSegment.tbl` tags each forensic segment with an index (see
//! [`super::segment`]); the decode keeps the segments matching our index,
//! drops the other 31, and treats everything outside a segment as ordinary
//! (index-0) content. This module owns that classification and nothing else —
//! no I/O, no keys, no cipher — so it is fully testable in isolation. The
//! decrypt pipeline consumes the [`UnitDisposition`] it returns.
//!
//! Where the resolved index comes from is a separate concern
//! ([`resolve_disc_index`]): today it is read off the index keys the key
//! source handed us; when Processing Keys are available it will come from the
//! VK derivation instead. Either way the disposition logic below is identical.
use super::segment::{Segment, segment_for_unit};
use super::types::UnitKey;
/// What the decode should do with one AACS aligned unit.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UnitDisposition {
/// Outside every forensic segment: ordinary content, decrypt with the
/// default (index-0) unit key.
Default,
/// Inside a forensic segment tagged with OUR resolved index: decrypt with
/// that index's key.
Index(u8),
/// Inside a forensic segment tagged with a DIFFERENT index: not our
/// watermark, so it is not part of our output — drop it.
DropForeignIndex(u8),
/// Inside a forensic segment but no index key is held (the disc's index
/// was never resolved): the segment cannot be decoded, so it is concealed
/// as loss. Carries the segment's index for diagnostics.
ForensicNoKey(u8),
}
/// Resolve the disc's single forensic index from the keys we hold.
///
/// Scans for an index key (`index_number` in `1..=32`) and returns its
/// index. `None` when only default (index-0) keys are held — i.e. no
/// index source answered, so forensic segments are not decodable. A disc has
/// exactly one index, so the first non-zero key decides; if several distinct
/// index keys were somehow supplied the lowest wins (deterministic), which is
/// only a defensive tiebreak — the probe/derivation yields one.
pub fn resolve_disc_index(unit_keys: &[UnitKey]) -> Option<u8> {
unit_keys
.iter()
.map(|k| k.index_number)
.filter(|&v| v != 0)
.min()
}
/// Classify the AACS aligned unit at `unit_offset` (clip-relative bytes) given
/// the forensic segment map and the disc's resolved index (`None` if no
/// index key is held).
pub fn unit_disposition(
unit_offset: u64,
segments: &[Segment],
disc_index: Option<u8>,
) -> UnitDisposition {
match segment_for_unit(segments, unit_offset) {
// Not in any forensic segment → ordinary content.
None => UnitDisposition::Default,
// In a forensic segment → decide by whether it is our index.
Some(seg) => {
// `seg.index` is an untrusted u16 from IndividualSegment.tbl; a real
// forensic index is 1..=32. Compare in u16 space so a corrupt/crafted
// index above 255 can't truncate into a valid u8 and alias our index.
// The disposition carries a u8 for diagnostics (saturated — an
// out-of-range index is never ours anyway).
let seg_index = seg.index;
let diag = seg_index.min(u8::MAX as u16) as u8;
match disc_index {
Some(v) if u16::from(v) == seg_index => UnitDisposition::Index(v),
Some(_) => UnitDisposition::DropForeignIndex(diag),
None => UnitDisposition::ForensicNoKey(diag),
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::aacs::content::ALIGNED_UNIT_LEN;
use crate::aacs::segment::{SOURCE_PACKET_LEN, parse_individual_segments};
/// Build a one-record segment table (index, start_spn, end_spn).
fn tbl(recs: &[(u16, u32, u32)]) -> Vec<Segment> {
let mut v = Vec::new();
v.extend_from_slice(&0x0100_0000u32.to_be_bytes());
v.extend_from_slice(&(recs.len() as u16).to_be_bytes());
v.extend_from_slice(&16u16.to_be_bytes());
for &(n, s, e) in recs {
v.extend_from_slice(&0x0100_0000u32.to_be_bytes());
v.extend_from_slice(&n.to_be_bytes());
v.extend_from_slice(&1u16.to_be_bytes());
v.extend_from_slice(&s.to_be_bytes());
v.extend_from_slice(&e.to_be_bytes());
}
parse_individual_segments(&v).expect("parse")
}
fn uk(idx: u32, index: u8) -> UnitKey {
if index == 0 {
UnitKey::new(idx, [0u8; 16])
} else {
UnitKey::forensic(idx, [index; 16], index)
}
}
#[test]
fn resolve_picks_the_single_index_key() {
// Default keys only → no index resolved.
assert_eq!(resolve_disc_index(&[uk(0, 0)]), None);
assert_eq!(resolve_disc_index(&[]), None);
// One index key among defaults → that index.
assert_eq!(resolve_disc_index(&[uk(0, 0), uk(1, 7)]), Some(7));
// Defensive: lowest of several distinct indexes (deterministic).
assert_eq!(resolve_disc_index(&[uk(0, 9), uk(1, 3)]), Some(3));
}
#[test]
fn unit_outside_segments_is_default() {
let segs = tbl(&[(1, 343680, 346239)]);
let off = 1000u64 * SOURCE_PACKET_LEN; // well before the segment
assert_eq!(
unit_disposition(off, &segs, Some(1)),
UnitDisposition::Default
);
// With no segments at all (1.0 / 2.0), everything is Default.
assert_eq!(
unit_disposition(off, &[], Some(1)),
UnitDisposition::Default
);
}
#[test]
fn unit_in_our_index_decrypts() {
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(7)),
UnitDisposition::Index(7)
);
}
#[test]
fn unit_in_foreign_index_drops() {
// Segment tagged index 7, but our disc index is 3 → drop it.
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(3)),
UnitDisposition::DropForeignIndex(7)
);
}
#[test]
fn forensic_unit_with_no_key_is_concealed() {
// A forensic segment but we never resolved an index → conceal as loss.
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, None),
UnitDisposition::ForensicNoKey(7)
);
}
#[test]
fn out_of_range_index_does_not_truncate_into_ours() {
// A crafted/corrupt segment index of 288 (0x0120) truncates to 32 in a
// u8. With our disc index resolved as 32, the old `seg.index as u8`
// compare would alias it to OUR index and decrypt with the wrong key.
// The u16 compare must instead classify it as foreign.
let segs = tbl(&[(288, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(32)),
UnitDisposition::DropForeignIndex(255)
);
}
#[test]
fn straddling_unit_still_classified_as_its_segment() {
// A unit whose 32-packet span only tails into the segment still routes
// to the segment (matches segment_for_unit's span test).
let segs = tbl(&[(5, 100, 200)]);
let unit_packets = (ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32; // 32
// Start so the unit covers [80, 80+31] = [80, 111]: overlaps at 100.
let off = 80u64 * SOURCE_PACKET_LEN;
assert!(80 + unit_packets > 100, "sanity: unit tails into seg");
assert_eq!(
unit_disposition(off, &segs, Some(5)),
UnitDisposition::Index(5)
);
}
}
+881
View File
@@ -0,0 +1,881 @@
//! AACS on-disc key-input files: `Unit_Key_RO.inf` parsing, the disc-hash
//! keydb lookup key, the Content Certificate, and the in-drive MKB read.
//! These turn raw disc files into the structures the key paths consume.
use super::mkb::*;
/// Parsed Unit_Key_RO.inf file.
pub struct UnitKeyFile {
/// Disc hash (SHA1 of the entire file) — used as KEYDB lookup key
pub disc_hash: [u8; 20],
/// Application type (1 = BD-ROM)
pub app_type: u8,
/// Number of BDMV directories
pub num_bdmv_dir: u8,
/// Whether SKB MKB is used
pub use_skb_mkb: bool,
/// AACS generation this file's stride matches
pub version: AacsVersion,
/// Encrypted unit keys (CPS unit number, encrypted key)
pub encrypted_keys: Vec<(u32, [u8; 16])>,
/// Title → CPS unit index mapping (title_idx → unit_key_idx)
pub title_cps_unit: Vec<u16>,
}
/// Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
/// the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
/// — plus the disc hash they are looked up by. A derived `Debug` printed every key
/// byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
/// message, a future `tracing::debug!` in this module) leaked them. Only
/// non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
impl std::fmt::Debug for UnitKeyFile {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKeyFile")
// The disc hash is the public keydb lookup key, printed as hex the
// same way `DiscEntry` prints its own — never as raw bytes.
.field("disc_hash", &disc_hash_hex(&self.disc_hash))
.field("app_type", &self.app_type)
.field("num_bdmv_dir", &self.num_bdmv_dir)
.field("use_skb_mkb", &self.use_skb_mkb)
.field("version", &self.version)
.field("encrypted_keys", &"<redacted>")
.field("encrypted_keys_len", &self.encrypted_keys.len())
.field("title_cps_unit", &self.title_cps_unit)
.finish()
}
}
/// Compute disc hash (SHA1 of Unit_Key_RO.inf content).
pub fn disc_hash(data: &[u8]) -> [u8; 20] {
use sha1::{Digest, Sha1};
let hash = Sha1::digest(data);
let mut out = [0u8; 20];
out.copy_from_slice(&hash);
out
}
/// Format disc hash as hex string with 0x prefix (for KEYDB lookup).
pub fn disc_hash_hex(hash: &[u8; 20]) -> String {
let mut s = String::with_capacity(42);
s.push_str("0x");
for b in hash {
s.push_str(&format!("{b:02X}"));
}
s
}
/// Parse Unit_Key_RO.inf from raw bytes.
///
/// Format (from AACS spec):
/// [0..4] BE32: offset to key storage area (uk_pos)
/// [16] app_type (1 = BD-ROM)
/// [17] num_bdmv_dir
/// [18] bit 7: use_skb_mkb
/// [20..22] BE16: first_play CPS unit
/// [22..24] BE16: top_menu CPS unit
/// [24..26] BE16: num_titles
/// [26..] title entries: 2 bytes padding + 2 bytes CPS unit, × num_titles
///
/// Key storage at uk_pos:
/// [uk_pos..uk_pos+2] BE16: num_unit_keys
/// [uk_pos+48..] encrypted keys, 16 bytes each
/// AACS 1.0: 48-byte stride
/// AACS 2.0 / 2.1: 64-byte stride (48 + 16 extra)
pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() < 20 {
return None;
}
let hash = disc_hash(data);
// Header
let app_type = data[16];
let num_bdmv_dir = data[17];
let use_skb_mkb = (data[18] >> 7) & 1 == 1;
// Key storage offset
let uk_pos = u32::from_be_bytes([data[0], data[1], data[2], data[3]]) as usize;
if uk_pos + 2 > data.len() {
return None;
}
// Number of unit keys
let num_uk = u16::from_be_bytes([data[uk_pos], data[uk_pos + 1]]) as usize;
if num_uk == 0 {
return Some(UnitKeyFile {
disc_hash: hash,
app_type,
num_bdmv_dir,
use_skb_mkb,
version,
encrypted_keys: Vec::new(),
title_cps_unit: Vec::new(),
});
}
// Stride between keys
let stride = version.unit_key_stride();
// Validate size
let keys_start = uk_pos + 48; // first key at uk_pos + 48
if keys_start + 16 > data.len() {
return None;
}
// Extract encrypted keys
let mut encrypted_keys = Vec::with_capacity(num_uk);
let mut pos = keys_start;
for i in 0..num_uk {
if pos + 16 > data.len() {
break;
}
let mut key = [0u8; 16];
key.copy_from_slice(&data[pos..pos + 16]);
encrypted_keys.push(((i + 1) as u32, key));
pos += stride;
}
// The loop above `break`s if the buffer runs out mid-key. A short list
// means the .inf is malformed/truncated — reject it rather than silently
// accepting fewer keys than the header declared, which would later map
// title CPS units to nonexistent keys.
if encrypted_keys.len() != num_uk {
return None;
}
// Title → CPS unit mapping (AACS Unit_Key_RO format): each on-disc CPS
// value is in `1..=num_uk` (else zeroes it) and converts the 1-based on-disc
// index to a 0-based key index. We mirror that so the stored value is a safe,
// ready-to-use key index rather than a raw 1-based number.
let to_key_idx = |cps: u16| -> u16 {
if cps >= 1 && cps as usize <= num_uk {
cps - 1
} else {
0
}
};
let mut title_cps_unit = Vec::new();
if data.len() >= 26 {
let first_play = u16::from_be_bytes([data[20], data[21]]);
let top_menu = u16::from_be_bytes([data[22], data[23]]);
let num_titles = u16::from_be_bytes([data[24], data[25]]) as usize;
title_cps_unit.push(to_key_idx(first_play));
title_cps_unit.push(to_key_idx(top_menu));
for i in 0..num_titles {
let off = 26 + i * 4 + 2; // 2 bytes padding + 2 bytes CPS unit
if off + 2 <= data.len() {
let cps = u16::from_be_bytes([data[off], data[off + 1]]);
title_cps_unit.push(to_key_idx(cps));
}
}
}
Some(UnitKeyFile {
disc_hash: hash,
app_type,
num_bdmv_dir,
use_skb_mkb,
version,
encrypted_keys,
title_cps_unit,
})
}
/// HD DVD Video Title Key File (`VTKF%%%.AACS`) magic — "DVD_HD_V_TKF".
pub const VTKF_MAGIC: &[u8; 12] = b"DVD_HD_V_TKF";
/// Fixed header length before the first Title Key Entry (AACS HD DVD Book,
/// Table 3-8).
const VTKF_HEADER_LEN: usize = 0x80;
/// Title Key Entry stride (Table 3-8): 1-byte `BIFO` + 3 reserved + 16-byte
/// encrypted title key + 16-byte binding MAC = 36 bytes.
const VTKF_ENTRY_LEN: usize = 0x24;
/// Byte offset of the encrypted title key within an entry (after `BIFO` + 3
/// reserved).
const VTKF_KEY_OFF: usize = 4;
/// Number of Title Key Entry slots in a VTKF (Table 3-8): a fixed 64.
const VTKF_MAX_ENTRIES: usize = 64;
/// `BIFO` bit 7 (`AV_FLG`): set = this slot carries an available title key.
const VTKF_AV_FLG: u8 = 0x80;
/// Parse an HD DVD `VTKF%%%.AACS` into the SAME [`UnitKeyFile`] a BD/UHD
/// `Unit_Key_RO.inf` yields — so the shared AACS crypto (`derive_unit_keys` →
/// `decrypt_unit_key(vuk, …)`) unwraps HD DVD title keys with no change. Only
/// the on-disc CONTAINER differs between BD and HD DVD; the title-key unwrap is
/// the identical AES-128 VUK step (`Kt = AES-128D(Kvu, Kte)`).
///
/// Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
/// 2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
/// Dukes of Hazzard `VTKF000`):
/// ```text
/// [0x00..0x0C] magic "DVD_HD_V_TKF"
/// [0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
/// [0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
/// [0x1C..0x80] reserved
/// [0x80..] 64 entries × 36 bytes:
/// BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
/// BIFO bit 7 (AV_FLG) set = this slot holds a title key
/// (pre-recorded discs fill the binding MAC with 0xFF)
/// [0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
/// ```
/// The slot index (1-based) is the CPS unit number, so an absent slot is
/// SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
/// hand the wrong title key to CPS unit N+1. The title→CPS mapping is
/// playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
/// `title_cps_unit` is left empty here.
///
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
}
// SHA1 of the WHOLE file — the KEYDB lookup key. BackupHDDVD-family key
// databases index an HD DVD disc by SHA1(VTKF000.AACS), the same role the
// BD disc_hash plays for `Unit_Key_RO.inf`.
let hash = disc_hash(data);
let mut encrypted_keys = Vec::new();
for n in 0..VTKF_MAX_ENTRIES {
let pos = VTKF_HEADER_LEN + n * VTKF_ENTRY_LEN;
if pos + VTKF_ENTRY_LEN > data.len() {
break;
}
// AV_FLG clear = empty slot: skip it, but keep the slot index as the CPS
// number (do NOT break — a gap must not renumber the keys that follow).
if data[pos] & VTKF_AV_FLG == 0 {
continue;
}
let mut key = [0u8; 16];
key.copy_from_slice(&data[pos + VTKF_KEY_OFF..pos + VTKF_KEY_OFF + 16]);
encrypted_keys.push((n as u32 + 1, key));
}
if encrypted_keys.is_empty() {
return None;
}
Some(UnitKeyFile {
disc_hash: hash,
app_type: 0, // HD DVD VTKF carries no BD-ROM app_type
num_bdmv_dir: 0, // BD-only concept
use_skb_mkb: false,
version: AacsVersion::V10, // HD DVD is always AACS 1.0
encrypted_keys,
title_cps_unit: Vec::new(),
})
}
/// Parse a disc's title-key file, dispatching on the self-describing magic:
/// an HD DVD `VTKF000.AACS` (`DVD_HD_V_TKF`) → [`parse_vtkf`]; anything else is a
/// BD/UHD `Unit_Key_RO.inf` → [`parse_unit_key_ro`]. Both return the same
/// [`UnitKeyFile`], so every downstream AACS derivation stays container-agnostic
/// — the single seam where BD-vs-HD-DVD key layout is resolved (mirrors the key
/// service, which classifies HD DVD by the very same magic).
pub fn parse_title_keys(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() >= 12 && &data[..12] == VTKF_MAGIC {
parse_vtkf(data)
} else {
parse_unit_key_ro(data, version)
}
}
/// MKB disc structure format code.
const MKB_DISC_STRUCTURE_FORMAT: u8 = 0x83;
/// MKB pack buffer size.
const MKB_PACK_SIZE: usize = 32772;
/// Read MKB from drive via SCSI (REPORT DISC STRUCTURE format 0x83).
/// Returns the concatenated MKB data from all packs.
pub fn read_mkb_from_drive(
session: &mut dyn crate::scsi::ScsiTransport,
) -> crate::error::Result<Vec<u8>> {
use crate::scsi::{DataDirection, SCSI_READ_DISC_STRUCTURE};
let cdb = [
SCSI_READ_DISC_STRUCTURE,
0x01,
0x00,
0x00,
0x00,
0x00,
0x00,
MKB_DISC_STRUCTURE_FORMAT,
(MKB_PACK_SIZE >> 8) as u8,
(MKB_PACK_SIZE & 0xFF) as u8,
0x00,
0x00,
];
let mut buf = vec![0u8; 32772];
session.execute(&cdb, DataDirection::FromDevice, &mut buf, 10_000)?;
let data_len = u16::from_be_bytes([buf[0], buf[1]]) as usize;
if data_len < 2 {
return Ok(Vec::new());
}
let len = data_len - 2;
let num_packs = buf[3] as usize;
let mut mkb = Vec::with_capacity(32768 * num_packs.max(1));
if len > 0 && len <= 32768 {
mkb.extend_from_slice(&buf[4..4 + len]);
}
// Read remaining packs
for pack in 1..num_packs {
let mut cdb = [
SCSI_READ_DISC_STRUCTURE,
0x01,
0x00,
0x00,
0x00,
0x00,
0x00,
MKB_DISC_STRUCTURE_FORMAT,
(MKB_PACK_SIZE >> 8) as u8,
(MKB_PACK_SIZE & 0xFF) as u8,
0x00,
0x00,
];
// Pack number goes in address field
cdb[2] = ((pack >> 24) & 0xFF) as u8;
cdb[3] = ((pack >> 16) & 0xFF) as u8;
cdb[4] = ((pack >> 8) & 0xFF) as u8;
cdb[5] = (pack & 0xFF) as u8;
let mut buf = vec![0u8; 32772];
if session
.execute(&cdb, DataDirection::FromDevice, &mut buf, 10_000)
.is_ok()
{
let len = u16::from_be_bytes([buf[0], buf[1]]) as usize;
if len > 2 && len - 2 <= 32768 {
mkb.extend_from_slice(&buf[4..4 + len - 2]);
}
}
}
Ok(mkb)
}
/// AACS Content Certificate — identifies disc AACS version and features.
#[derive(Debug)]
pub struct ContentCert {
/// Bus encryption enabled flag
pub bus_encryption: bool,
/// Content Certificate ID (6 bytes)
pub cc_id: [u8; 6],
/// AACS generation indicated by the certificate type byte.
///
/// Cert type `0x00` → [`AacsVersion::V10`]; any other value →
/// [`AacsVersion::V20`]. The certificate alone cannot distinguish
/// V20 from V21 — Variant detection happens after the MKB walk.
pub version: AacsVersion,
}
/// Parse a Content Certificate (ContentXXX.cer) file.
pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
if data.len() < 20 {
return None;
}
// Content Certificate layout (per the AACS content-cert format):
// [0] certificate type (0x00 = AACS1, 0x10 = AACS2)
// [1] bit7 bus_encryption_enabled_flag (`p[1] >> 7`)
// [14..20] cc_id (6 bytes) (`p + 14`)
let version = if data[0] == 0x00 {
AacsVersion::V10
} else {
AacsVersion::V20
};
// The flag is bit 7 of byte 1, NOT bit 0. Reading bit 0 (the prior bug) made
// a bus-encrypted cert (byte1=0x80) read as `false`, defeating the
// AacsBusKeyUnavailable fail-loud gate in disc/encrypt.rs.
let bus_encryption = (data[1] >> 7) & 1 == 1;
let mut cc_id = [0u8; 6];
cc_id.copy_from_slice(&data[14..20]);
Some(ContentCert {
bus_encryption,
cc_id,
version,
})
}
#[cfg(test)]
mod vtkf_tests {
use super::*;
/// Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
/// HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
/// `VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
/// entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
/// set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
fn synth_vtkf(keys: &[[u8; 16]]) -> Vec<u8> {
const FILE_LEN: usize = 2480;
let mut v = Vec::new();
v.extend_from_slice(VTKF_MAGIC); // 0x00
v.extend_from_slice(&(FILE_LEN as u32).to_be_bytes()); // 0x0C HD_VTKF_SIZE
v.extend_from_slice(b"VPLST000.XPL"); // 0x10 playlist name
v.resize(VTKF_HEADER_LEN, 0); // reserve to first entry (0x80)
for n in 0..VTKF_MAX_ENTRIES {
if let Some(k) = keys.get(n) {
v.push(VTKF_AV_FLG); // BIFO: AV_FLG set (present)
v.extend_from_slice(&[0, 0, 0]); // reserved
v.extend_from_slice(k); // 16-byte encrypted title key
v.extend_from_slice(&[0xFFu8; 16]); // binding MAC (0xFF, pre-recorded)
} else {
v.extend_from_slice(&[0u8; VTKF_ENTRY_LEN]); // empty slot (AV_FLG clear)
}
}
v.resize(FILE_LEN - 16, 0); // reserved gap before the trailer
v.extend_from_slice(&[0xABu8; 16]); // TKF MAC (must NOT be read as a key)
v
}
#[test]
fn parse_vtkf_reads_present_entries_skips_empty_ignores_mac() {
let k1 = [0x11u8; 16];
let k2 = [0x22u8; 16];
let k3 = [0x33u8; 16];
let data = synth_vtkf(&[k1, k2, k3]);
let ukf = parse_vtkf(&data).expect("valid VTKF must parse");
// Exactly the three present entries — the empty slots and the trailing
// 16-byte TKF MAC are NOT mistaken for keys. Critically, k2/k3 are read
// at the 36-byte stride (offsets 0xA4, 0xC8); the old 32-byte stride
// misread them from inside the previous entry's binding MAC.
assert_eq!(ukf.encrypted_keys.len(), 3);
assert_eq!(
ukf.encrypted_keys[0],
(1, k1),
"CPS units = 1-based slot index"
);
assert_eq!(ukf.encrypted_keys[1], (2, k2));
assert_eq!(ukf.encrypted_keys[2], (3, k3));
assert_eq!(ukf.version, AacsVersion::V10, "HD DVD is AACS 1.0");
// disc_hash is SHA1 of the whole file (the KEYDB lookup key).
assert_eq!(ukf.disc_hash, disc_hash(&data));
}
#[test]
fn parse_vtkf_reads_a_full_64_entry_file() {
// Real discs (Freedom, Dukes) carry all 64 slots present. Every key must
// come back, none dropped and none drifted — the regression the 32-byte
// stride failed.
let keys: Vec<[u8; 16]> = (0..VTKF_MAX_ENTRIES).map(|n| [n as u8; 16]).collect();
let ukf = parse_vtkf(&synth_vtkf(&keys)).expect("64-entry VTKF");
assert_eq!(ukf.encrypted_keys.len(), 64);
assert_eq!(
ukf.encrypted_keys[63],
(64, [63u8; 16]),
"entry 64 at 0x{:x}",
VTKF_HEADER_LEN + 63 * VTKF_ENTRY_LEN
);
}
#[test]
fn parse_vtkf_rejects_non_magic() {
let mut data = synth_vtkf(&[[0x11u8; 16]]);
data[0] = b'X'; // corrupt magic
assert!(
parse_vtkf(&data).is_none(),
"non-VTKF magic must be rejected"
);
assert!(
parse_vtkf(&[0u8; 4]).is_none(),
"too short must be rejected"
);
}
#[test]
fn parse_title_keys_dispatches_by_magic() {
// VTKF magic → parse_vtkf.
let data = synth_vtkf(&[[0x44u8; 16], [0x55u8; 16]]);
let ukf = parse_title_keys(&data, AacsVersion::V10).expect("VTKF dispatch");
assert_eq!(ukf.encrypted_keys.len(), 2);
// Non-VTKF → parse_unit_key_ro (a 2-byte buffer is not a valid inf, so
// this proves it ROUTED to the BD parser rather than parse_vtkf).
assert!(
parse_title_keys(&[0x00, 0x00], AacsVersion::V10).is_none(),
"non-magic input must route to parse_unit_key_ro"
);
}
/// The whole point of the seam: a parsed VTKF feeds the SHARED VUK→title-key
/// crypto (`decrypt_unit_key`) exactly like a BD `Unit_Key_RO.inf` would —
/// no HD-DVD-specific crypto path.
#[test]
fn vtkf_encrypted_keys_feed_shared_vuk_unwrap() {
let enc = [0x9Au8; 16];
let data = synth_vtkf(&[enc]);
let ukf = parse_vtkf(&data).unwrap();
let vuk = [0x5Cu8; 16];
let derived = super::super::derive::decrypt_unit_key(&vuk, &ukf.encrypted_keys[0].1);
// Same as applying the shared unwrap directly to the stored enc key.
assert_eq!(derived, super::super::derive::decrypt_unit_key(&vuk, &enc));
}
/// `UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
/// printed every byte; the hand-written impl must not. Sentinel key byte
/// 0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
/// same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
/// `#[derive(Debug)]` back fails this.
#[test]
fn unit_key_file_debug_is_redacted() {
let f = UnitKeyFile {
disc_hash: [0xD5; 20],
app_type: 1,
num_bdmv_dir: 1,
use_skb_mkb: false,
version: AacsVersion::V20,
encrypted_keys: vec![(0, [0xD5; 16]), (1, [0xD5; 16])],
title_cps_unit: vec![0, 1],
};
let dbg = format!("{f:?}");
assert!(
!dbg.contains("213"),
"UnitKeyFile Debug leaked key bytes (decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"UnitKeyFile Debug missing redaction marker: {dbg}"
);
// Non-secret shape is still useful for diagnostics.
assert!(dbg.contains("encrypted_keys_len: 2"), "{dbg}");
}
}
#[cfg(test)]
mod read_mkb_tests {
use super::*;
use crate::scsi::{DataDirection, SCSI_READ_DISC_STRUCTURE, ScsiResult, ScsiTransport};
/// A drive that answers READ DISC STRUCTURE format 0x83 from a scripted set
/// of packs and records every CDB it was handed.
struct MkbDrive {
/// One entry per pack: the pack's MKB payload bytes.
packs: Vec<Vec<u8>>,
cdbs: Vec<Vec<u8>>,
}
impl ScsiTransport for MkbDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
self.cdbs.push(cdb.to_vec());
// Pack number is carried in the CDB address field (bytes 2..6),
// MMC-6 READ DISC STRUCTURE.
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]) as usize;
let body = self.packs.get(pack).cloned().unwrap_or_default();
// Header: BE16 data length (counts the 2 header bytes that follow
// it plus the payload), reserved byte, pack count, then payload.
let data_len = body.len() + 2;
data[0..2].copy_from_slice(&(data_len as u16).to_be_bytes());
data[2] = 0x00;
data[3] = self.packs.len() as u8;
data[4..4 + body.len()].copy_from_slice(&body);
Ok(ScsiResult {
status: 0,
bytes_transferred: 4 + body.len(),
sense: [0u8; 32],
})
}
}
/// `read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
/// downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
/// Media Key ladder) consumes exactly what it returns. An empty return is
/// not a benign "no MKB" — it is a total read failure reported as success,
/// and every derivation then fails with a key-not-found code that points
/// the operator at their keydb rather than at the drive.
///
/// This pins the CONTENT: the concatenated payload of all packs, in pack
/// order, byte for byte.
#[test]
fn read_mkb_from_drive_returns_the_concatenated_pack_payload() {
let pack0: Vec<u8> = (0..600u32).map(|i| (i % 251) as u8).collect();
let pack1: Vec<u8> = (0..300u32).map(|i| (i % 253) as u8 ^ 0xA5).collect();
let mut drive = MkbDrive {
packs: vec![pack0.clone(), pack1.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
let mut expected = pack0.clone();
expected.extend_from_slice(&pack1);
assert_eq!(
mkb.len(),
expected.len(),
"every pack's payload must be concatenated, none dropped"
);
assert!(
mkb == expected,
"MKB bytes must be the drive's payload in pack order; first \
mismatch at {:?}",
(0..expected.len()).find(|&i| mkb[i] != expected[i])
);
// MMC-6 READ DISC STRUCTURE with the AACS MKB format code, one command
// per pack, pack number in the address field.
assert_eq!(drive.cdbs.len(), 2, "one command per declared pack");
for (i, cdb) in drive.cdbs.iter().enumerate() {
assert_eq!(cdb[0], SCSI_READ_DISC_STRUCTURE, "opcode");
assert_eq!(cdb[7], 0x83, "AACS MKB disc-structure format code");
assert_eq!(
u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]),
i as u32,
"pack {i} must be requested by number"
);
}
}
/// The CDB is what the drive actually acts on, and every byte of it is
/// load-bearing: a wrong format code returns a different disc structure
/// entirely, and a wrong allocation length truncates the pack. The existing
/// test above pins the opcode, the format code and the pack number; this
/// pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
///
/// Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
/// `[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
/// (BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
/// BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
#[test]
fn read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack() {
let mut drive = MkbDrive {
packs: vec![vec![0x11u8; 64], vec![0x22u8; 64], vec![0x33u8; 64]],
cdbs: Vec::new(),
};
read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(drive.cdbs.len(), 3, "one command per declared pack");
for (pack, cdb) in drive.cdbs.iter().enumerate() {
let p = pack as u32;
let expected: [u8; 12] = [
SCSI_READ_DISC_STRUCTURE,
0x01,
(p >> 24) as u8,
(p >> 16) as u8,
(p >> 8) as u8,
p as u8,
0x00,
0x83, // AACS MKB disc-structure format
0x80, // allocation length 32772 = 0x8004, high byte
0x04, // …low byte
0x00,
0x00,
];
assert_eq!(
cdb.as_slice(),
&expected[..],
"CDB for pack {pack} must match the MMC-6 READ DISC STRUCTURE layout"
);
}
}
/// A pack payload filling the FULL 32768-byte window must come back whole.
/// The `len > 0 && len <= 32768` bound is what stands between a maximal
/// pack and a silently dropped one, and the small payloads used elsewhere
/// in this module never reach it.
#[test]
fn read_mkb_from_drive_accepts_a_full_size_pack() {
let full: Vec<u8> = (0..32768u32).map(|i| (i % 251) as u8).collect();
let other: Vec<u8> = (0..32768u32).map(|i| (i % 241) as u8 ^ 0x5A).collect();
// TWO maximal packs: the first-pack read and the per-pack loop carry
// separate bounds, so both must accept a full-window payload.
let mut drive = MkbDrive {
packs: vec![full.clone(), other.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
65536,
"neither maximal pack may be dropped at the size bound"
);
let mut expected = full.clone();
expected.extend_from_slice(&other);
assert!(mkb == expected, "both maximal packs' bytes must be intact");
}
/// A pack that declares only the 2-byte header and NO payload contributes
/// nothing, and must not push a phantom byte into the MKB — an off-by-one
/// at the zero-length boundary corrupts every following pack's alignment.
#[test]
fn read_mkb_from_drive_zero_length_pack_contributes_nothing() {
let mut drive = MkbDrive {
packs: vec![Vec::new(), vec![0xABu8; 32]],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(
mkb.len(),
32,
"an empty pack adds no bytes; only pack 1's payload is present"
);
assert!(mkb == vec![0xABu8; 32], "and the bytes are pack 1's");
}
/// A drive that DECLARES more payload than it returned must not be
/// believed. The BE16 length in the response header is drive-supplied data:
/// a firmware bug, a short transfer, or a hostile device can put a value in
/// it that runs past the 32772-byte buffer. Copying `len` bytes on that word
/// alone panics the rip thread mid-scan.
///
/// Both the first-pack read and the per-pack loop carry the same bound, so
/// both are exercised here: the over-declared pack contributes nothing and
/// the honest pack still comes through.
#[test]
fn read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds() {
/// Pack 0 is honest; pack 1 declares a 60000-byte payload it never sent.
struct LyingDrive {
honest: Vec<u8>,
}
impl ScsiTransport for LyingDrive {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
let pack = u32::from_be_bytes([cdb[2], cdb[3], cdb[4], cdb[5]]);
data[3] = 2; // two packs declared
if pack == 0 {
let dl = self.honest.len() + 2;
data[0..2].copy_from_slice(&(dl as u16).to_be_bytes());
data[4..4 + self.honest.len()].copy_from_slice(&self.honest);
} else {
// A length far beyond the 32772-byte response buffer.
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
}
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let honest = vec![0xC7u8; 256];
let mut drive = LyingDrive {
honest: honest.clone(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("an over-declared pack is not an error");
assert_eq!(
mkb.len(),
honest.len(),
"only the honest pack's bytes may be taken; the over-declared pack \
contributes nothing and must not be read past the buffer"
);
assert!(mkb == honest, "and those bytes are pack 0's");
}
/// The same over-declaration on the FIRST pack, which uses a separate bound
/// from the loop's.
#[test]
fn read_mkb_from_drive_ignores_a_first_pack_declaring_more_than_the_buffer() {
struct LyingFirst;
impl ScsiTransport for LyingFirst {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&60_000u16.to_be_bytes());
data[3] = 1;
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut LyingFirst).expect("not an error");
assert!(
mkb.is_empty(),
"a first pack declaring more than the buffer holds yields no bytes"
);
}
/// A single-pack disc still yields that pack's bytes — the common case, and
/// the one where a body returning an empty vector looks most plausible.
#[test]
fn read_mkb_from_drive_returns_a_single_packs_payload() {
let pack: Vec<u8> = (0..1024u32).map(|i| (i * 7 % 256) as u8).collect();
let mut drive = MkbDrive {
packs: vec![pack.clone()],
cdbs: Vec::new(),
};
let mkb = read_mkb_from_drive(&mut drive).expect("scripted drive answers");
assert_eq!(mkb.len(), pack.len(), "single pack payload length");
assert!(mkb == pack, "single pack payload bytes");
}
/// A drive that reports a header-only response (`data_len < 2`) has no MKB
/// to give. That must be an EMPTY vec, not a partial one — the distinction
/// matters because the AACS paths treat a non-empty MKB as parseable.
#[test]
fn read_mkb_from_drive_empty_response_is_empty() {
struct NoMkb;
impl ScsiTransport for NoMkb {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
data[0..2].copy_from_slice(&0u16.to_be_bytes());
Ok(ScsiResult {
status: 0,
bytes_transferred: 4,
sense: [0u8; 32],
})
}
}
let mkb = read_mkb_from_drive(&mut NoMkb).expect("no-MKB drive still returns Ok");
assert!(
mkb.is_empty(),
"a header-only response carries no MKB bytes"
);
}
/// A transport failure on the FIRST pack must propagate as an error — the
/// MKB is the root of the whole AACS ladder, so an unreadable one cannot be
/// downgraded to "an MKB with no records".
#[test]
fn read_mkb_from_drive_propagates_the_first_pack_failure() {
struct DeadDrive;
impl ScsiTransport for DeadDrive {
fn execute(
&mut self,
_cdb: &[u8],
_direction: DataDirection,
_data: &mut [u8],
_timeout_ms: u32,
) -> crate::error::Result<ScsiResult> {
Err(crate::error::Error::ScsiError {
opcode: SCSI_READ_DISC_STRUCTURE,
status: 0x02,
sense: None,
})
}
}
assert!(
read_mkb_from_drive(&mut DeadDrive).is_err(),
"an unreadable MKB must surface as an error, not an empty MKB"
);
}
}
+540
View File
@@ -0,0 +1,540 @@
//! AACS Media Key Block — [C] Chapter 3.
//!
//! The MKB record format (framing walker, the `MkbRecord` view, record-body
//! finders), the MKBType / AACS-generation classification, and MKB-file
//! utilities (content length, trimming, version). Consolidated here so the one
//! place that understands MKB bytes is `mkb`. Some duplicate record finders
//! still live side by side pending a follow-up that collapses them.
// ── MKB record types ([C] Chapter 3) ──────────────────────────────────────
// The ONE canonical set. Every record-type comparison in the `aacs` module
// references these, so a type byte is never a bare literal scattered across
// files (the `0x0c` variant-data record in particular used to appear in several
// hand-rolled forms).
/// Type-and-Version — carries the 32-bit MKBType / AACS generation.
pub(crate) const REC_TYPE_AND_VERSION: u8 = 0x10;
/// Subset-Difference index — the per-slot `(u_mask_shift, uv)` table.
pub(crate) const REC_SUBSET_DIFFERENCE: u8 = 0x04;
/// Media Key Data — the classical (1.0 / 2.0) per-subset cvalue table.
pub(crate) const REC_MEDIA_KEY_DATA: u8 = 0x05;
/// Explicit Subset-Difference — the smaller cvalue table some MKBs use.
pub(crate) const REC_EXPLICIT_SUBSET_DIFF: u8 = 0x07;
/// Media Key Variant Data (AACS 2.1) — the per-subset-difference `C` table
/// (one 16-byte C per slot); the `Kmp` step reads C from HERE, not `0x2d`.
pub(crate) const REC_MEDIA_KEY_VARIANT_DATA: u8 = 0x0c;
/// Variant Data + Nonce (AACS 2.1) — the `VARIANTS[uv]` table (leading bytes)
/// with the 16-byte `Kvn` Nonce at the tail.
pub(crate) const REC_VARIANT_DATA_AND_NONCE: u8 = 0x2d;
/// Variant Key Data table (AACS 2.1) — 65,535×16, indexed by the resolved VKD index.
pub(crate) const REC_VKD_TABLE: u8 = 0x2f;
/// Verify-Media-Key — AACS 1.0.
pub(crate) const REC_VERIFY_MEDIA_KEY_V1: u8 = 0x81;
/// Verify-Media-Key — AACS 2.x.
pub(crate) const REC_VERIFY_MEDIA_KEY_V2: u8 = 0x86;
/// A single MKB record produced by [`walk_mkb`].
#[derive(Debug, Clone)]
pub struct MkbRecord {
/// Byte offset of the record within the MKB.
pub offset: usize,
/// Record type byte.
pub rec_type: u8,
/// Record length in bytes (includes the 4-byte header).
pub rec_len: usize,
/// Record body (the bytes after the 4-byte header).
pub body: Vec<u8>,
}
/// Walk an MKB into a flat list of records.
///
/// MKB record framing per AACS: 1 byte type, 3 bytes BE length
/// INCLUDING the 4-byte header, followed by payload. The walker stops
/// at the first `(type=0, len=0)` end marker or at end of buffer.
pub fn walk_mkb(mkb: &[u8]) -> Vec<MkbRecord> {
mkb_records(mkb)
.map(|(offset, rec_type, rec_len)| MkbRecord {
offset,
rec_type,
rec_len,
body: mkb[offset + 4..offset + rec_len].to_vec(),
})
.collect()
}
/// THE single MKB record-framing walker: yields `(offset, rec_type, rec_len)`
/// for each record — a 4-byte header (type byte + big-endian 24-bit length)
/// then the body — stopping at the `00 000000` end marker or a
/// malformed/out-of-bounds length. Lazy (no body clone), so a find-one-record
/// caller never materialises the multi-MB cvalue table. [`walk_mkb`] and every
/// MKB record walk in `aacs::resolve`/`aacs::derive` are built on this, so the framing rules — and
/// any future fix to them — live in exactly one place (they had drifted across
/// six hand-rolled copies).
pub(crate) fn mkb_records(mkb: &[u8]) -> impl Iterator<Item = (usize, u8, usize)> + '_ {
let mut pos = 0usize;
std::iter::from_fn(move || {
if pos + 4 > mkb.len() {
return None;
}
let rec_type = mkb[pos];
let rec_len = ((mkb[pos + 1] as usize) << 16)
| ((mkb[pos + 2] as usize) << 8)
| (mkb[pos + 3] as usize);
if rec_type == 0 && rec_len == 0 {
return None;
}
if rec_len < 4 || pos + rec_len > mkb.len() {
return None;
}
let here = pos;
pos += rec_len;
Some((here, rec_type, rec_len))
})
}
pub(crate) fn mkb_find_body(records: &[MkbRecord], rec_type: u8) -> Option<&[u8]> {
records
.iter()
.find(|r| r.rec_type == rec_type && !r.body.is_empty())
.map(|r| r.body.as_slice())
}
/// AACS protection generation a disc carries.
///
/// The content cert byte distinguishes V10 (`0x00`) from V20 (`0x01`). V21
/// cannot be detected from the cert alone — a V21 disc carries a V20 cert
/// and is upgraded to `V21` only after the MKB walk turns up the real Variant
/// records `0x2d` / `0x2f` (Encrypted Media Key Variant Data and the Variant
/// Key Data table).
///
/// Key-storage stride in `Unit_Key_RO.inf` is 48 bytes for V10 and 64
/// bytes for V20 / V21.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AacsVersion {
/// AACS 1.0 — original BD-ROM.
V10,
/// AACS 2.0 — UHD-BD, classical Media Key derivation.
V20,
/// AACS 2.1 — UHD-BD with Media Key Variant chain on top of V20.
V21,
}
/// AACS major version as the small integer threaded through the scan / key
/// paths (`AacsState.version`, `DiscInputs.version`, `DiscInputsCtx::new`):
/// 1 = AACS 1.0 (BD), 2 = AACS 2.x (UHD). Centralised so the bare `1`/`2` — and
/// the V10-vs-else stride choice it drives — lives in exactly one place.
pub const AACS_MAJOR_BD: u8 = 1;
pub const AACS_MAJOR_UHD: u8 = 2;
impl AacsVersion {
/// Stride (in bytes) between successive encrypted unit keys in
/// `Unit_Key_RO.inf`.
pub(crate) fn unit_key_stride(self) -> usize {
match self {
AacsVersion::V10 => 48,
AacsVersion::V20 | AacsVersion::V21 => 64,
}
}
/// This version as the major integer ([`AACS_MAJOR_BD`] / [`AACS_MAJOR_UHD`]).
pub fn major(self) -> u8 {
match self {
AacsVersion::V10 => AACS_MAJOR_BD,
AacsVersion::V20 | AacsVersion::V21 => AACS_MAJOR_UHD,
}
}
/// The version a bare major integer selects for stride purposes: only the
/// BD major is V10; every other value takes the V20/V21 64-byte stride.
pub fn from_major(major: u8) -> Self {
if major == AACS_MAJOR_BD {
AacsVersion::V10
} else {
AacsVersion::V20
}
}
}
/// Find Verify Media Key Record (type 0x81 for AACS 1.0, 0x86 for AACS 2.0/2.1) in MKB.
/// 0x81: [C] §3.2.5.1.4. 0x86 (AACS 2.x): [RE] — not in the public spec (from real 2.x MKBs).
pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
// Verify-Media-Key record (0x81 for AACS 1.0, 0x86 for AACS 2.x): mk_dv is
// the 16 bytes at record offset 4 (body offset 0). Needs rec_len >= 20.
let found = mkb_records(mkb).find(|&(_, rt, len)| {
(rt == REC_VERIFY_MEDIA_KEY_V1 || rt == REC_VERIFY_MEDIA_KEY_V2) && len >= 20
});
match found {
Some((o, rec_type, rec_len)) => {
let mut dv = [0u8; 16];
dv.copy_from_slice(&mkb[o + 4..o + 20]);
tracing::debug!(
target: "freemkv::disc",
phase = "mkb_mk_dv_found",
rec_type,
pos = o,
rec_len,
"mk_dv extracted from MKB"
);
Some(dv)
}
None => {
tracing::warn!(
target: "freemkv::disc",
phase = "mkb_mk_dv_not_found",
"no 0x81/0x86 record with rec_len>=20 found"
);
None
}
}
}
/// Find Subset-Difference records (type 0x04) in MKB. [C] §3.2.5.1.5.
pub(crate) fn mkb_find_subdiff_records(mkb: &[u8]) -> Option<Vec<u8>> {
find_record_body(mkb, 0x04)
}
/// Find the Media Key Data Record (cvalues table) in an MKB. [C] §3.2.4 / §3.2.5.1.7.
///
/// The cvalue table is record type `0x05` (Media Key Data) on BOTH AACS
/// 1.0 and AACS 2.x MKBs — its 16-byte cvalue entries are 1:1 with the
/// 5-byte Subset-Difference index entries in record `0x04` — the standard AACS
/// MKB layout (`0x05` cvalues 1:1 with the `0x04` subset-difference index).
///
/// On AACS 2.x in-drive UHD MKBs the `0x05` table is large (the full
/// subset-difference cvalue set: ~181k entries on a retail MKB, 1:1 with
/// the giant `0x04` index), while record `0x07` (Explicit
/// Subset-Difference Record) is a much smaller structure (~96 entries) and
/// is NOT the cvalue table. An earlier version of this function preferred
/// `0x07`, which under-tested the Subset-Difference walk on UHD discs and
/// prevented the DK→walk path from ever finding the matching uv. The
/// selection MUST therefore be `0x05`-first; `0x07` is only a fallback for
/// malformed/legacy MKBs that somehow lack a `0x05` record.
pub(crate) fn mkb_find_cvalues(mkb: &[u8]) -> Option<Vec<u8>> {
if let Some(body) = find_record_body(mkb, 0x05) {
return Some(body);
}
find_record_body(mkb, 0x07)
}
/// Walk an MKB and return the payload (header stripped) of the first
/// record matching `rec_type`. Returns `None` if no such record exists or
/// the record is empty.
pub(crate) fn find_record_body(mkb: &[u8], rec_type_wanted: u8) -> Option<Vec<u8>> {
mkb_records(mkb)
.find(|&(_, rt, len)| rt == rec_type_wanted && len > 4)
.map(|(o, _, len)| mkb[o + 4..o + len].to_vec())
}
/// Real content length of an MKB: the byte offset where the record stream
/// ends. MKB files (especially `MKB_RW.inf`, but `MKB_RO.inf` too on some
/// discs) are allocated to a fixed size — often ~128 MiB — with the records at
/// the front and the rest zero padding. Walking records (type+len) and stopping
/// at the first padding byte (`type == 0` / zero-length / overrun) gives the
/// actual size so callers can trim off megabytes of zeros before sending or
/// archiving. Returns `mkb.len()` only if the whole buffer parsed as records.
pub fn mkb_content_len(mkb: &[u8]) -> usize {
// End of the last framed record = where the fixed-region zero padding begins.
// (The `00 000000` terminator / overrun stops the walk; real MKBs pad with
// zeros, so this matches the prior "stop at the first padding byte".)
mkb_records(mkb)
.last()
.map(|(o, _, len)| o + len)
.unwrap_or(0)
}
/// Trim an MKB's trailing fixed-region padding to its real content length —
/// but ONLY when [`mkb_content_len`] actually found one. It returns 0 for an
/// MKB whose first record cannot be parsed; truncating to 0 in that case would
/// hand downstream consumers (and the online key service) an EMPTY MKB that can
/// never resolve. So a 0 (or a length that isn't strictly inside the buffer)
/// leaves the MKB untouched. A 0.31.0 regression dropped this guard and
/// `truncate`-d unconditionally, zeroing unrecognised MKBs.
pub fn trim_mkb(mut mkb: Vec<u8>) -> Vec<u8> {
let n = mkb_content_len(&mkb);
if n > 0 && n < mkb.len() {
mkb.truncate(n);
}
mkb
}
/// Get MKB version from Type and Version Record (type 0x10).
/// Layout: 4-byte record header at `pos` (type + BE24 length), then the
/// record body starts at `pos + 4`. The body holds the BE u32 Type field at
/// body offset 0 (`pos + 4`), then the BE u32 version at body offset 4
/// (`pos + 8`).
pub fn mkb_version(mkb: &[u8]) -> Option<u32> {
// Type-and-Version record (0x10): version is the BE u32 at body offset 4
// (record offset 8). Needs rec_len >= 12 (4 header + 4 type + 4 version).
mkb_records(mkb)
.find(|&(_, rt, len)| rt == REC_TYPE_AND_VERSION && len >= 12)
.map(|(o, _, _)| u32::from_be_bytes([mkb[o + 8], mkb[o + 9], mkb[o + 10], mkb[o + 11]]))
}
/// `0x00031003` — recordable media MKB (Class I & II compute Km directly).
pub const MKB_TYPE_3_RECORDABLE: u32 = 0x0003_1003;
/// `0x00041003` — AACS 1.0 pre-recorded content MKB (KCD-based). Standard BD.
pub const MKB_TYPE_4_PRERECORDED: u32 = 0x0004_1003;
/// `0x000A1003` — Class II / Unified MKB (Sequence-Key-Block functionality).
pub const MKB_TYPE_10_CLASS_II: u32 = 0x000A_1003;
/// `0x48141003` — AACS 2.0 Category C (UHD content) MKB type value.
pub const MKB_20_CATEGORY_C: u32 = 0x4814_1003;
/// `0x48151003` — AACS 2.1 Category C (UHD content) MKB type value.
pub const MKB_21_CATEGORY_C: u32 = 0x4815_1003;
/// The AACS MKB Type field, decoded.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MkbType {
/// Type 3 — recordable media.
Recordable,
/// Type 4 — AACS 1.0 pre-recorded content (KCD). Standard Blu-ray.
Prerecorded,
/// Type 10 — Class II / Unified (SKB).
ClassII,
/// AACS 2.0 Category C — UHD content.
CategoryC20,
/// AACS 2.1 Category C — UHD content.
CategoryC21,
/// Unrecognized MKBType value (raw field preserved).
Other(u32),
}
impl MkbType {
pub(crate) fn from_raw(raw: u32) -> Self {
match raw {
MKB_TYPE_3_RECORDABLE => MkbType::Recordable,
MKB_TYPE_4_PRERECORDED => MkbType::Prerecorded,
MKB_TYPE_10_CLASS_II => MkbType::ClassII,
MKB_20_CATEGORY_C => MkbType::CategoryC20,
MKB_21_CATEGORY_C => MkbType::CategoryC21,
other => MkbType::Other(other),
}
}
/// AACS generation this MKB belongs to (Category C → 2.0/2.1, else 1.0).
pub fn generation(self) -> AacsVersion {
match self {
MkbType::CategoryC21 => AacsVersion::V21,
MkbType::CategoryC20 => AacsVersion::V20,
_ => AacsVersion::V10,
}
}
/// `true` for UHD (AACS 2.x Category C); `false` for Blu-ray (AACS 1.x).
pub fn is_uhd(self) -> bool {
matches!(self, MkbType::CategoryC20 | MkbType::CategoryC21)
}
}
/// The raw 32-bit MKBType field from the Type-and-Version record (0x10), bytes
/// 4-7. `None` if no 0x10 record is present. [C] §3.2.5.1.1 Table 3-2.
pub fn mkb_type_raw(mkb: &[u8]) -> Option<u32> {
// Type-and-Version record (0x10): the 32-bit MKBType is bytes 4-7 (body
// offset 0). Needs rec_len >= 8 (4 header + 4 type).
mkb_records(mkb)
.find(|&(_, rt, len)| rt == REC_TYPE_AND_VERSION && len >= 8)
.map(|(o, _, _)| u32::from_be_bytes([mkb[o + 4], mkb[o + 5], mkb[o + 6], mkb[o + 7]]))
}
/// Decode an MKB's Type field. `None` if no Type-and-Version record is present.
pub fn mkb_type(mkb: &[u8]) -> Option<MkbType> {
mkb_type_raw(mkb).map(MkbType::from_raw)
}
/// `Some(true)` if this MKB is a UHD (AACS 2.x Category C) block, `Some(false)`
/// for Blu-ray (AACS 1.x), `None` if the Type record is absent.
pub fn mkb_is_uhd(mkb: &[u8]) -> Option<bool> {
mkb_type(mkb).map(MkbType::is_uhd)
}
#[cfg(test)]
mod tests {
use super::*;
/// One MKB record: 1 type byte + big-endian 24-bit total length + body.
fn rec(rec_type: u8, body: &[u8]) -> Vec<u8> {
let len = 4 + body.len();
let mut v = vec![rec_type, (len >> 16) as u8, (len >> 8) as u8, len as u8];
v.extend_from_slice(body);
v
}
/// Type-and-Version record (0x10): body = 4-byte MKBType + 4-byte version.
fn type_and_version(mkb_type: u32, version: u32) -> Vec<u8> {
let mut body = mkb_type.to_be_bytes().to_vec();
body.extend_from_slice(&version.to_be_bytes());
rec(REC_TYPE_AND_VERSION, &body)
}
#[test]
fn walker_frames_records_and_stops_at_end_marker() {
let mut mkb = type_and_version(MKB_20_CATEGORY_C, 77);
mkb.extend(rec(REC_VKD_TABLE, &[0xAA; 16]));
mkb.extend([0x00, 0x00, 0x00, 0x00]); // end marker
mkb.extend(rec(0x99, &[0xFF; 8])); // must NOT be walked (past the marker)
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "walk stops at the 00 000000 end marker");
assert_eq!(recs[0].rec_type, REC_TYPE_AND_VERSION);
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].body, vec![0xAA; 16]);
}
#[test]
fn walker_stops_on_malformed_or_out_of_bounds_length() {
// A record whose declared length runs past the buffer end must terminate
// the walk rather than panic or read OOB.
let mkb = vec![REC_VKD_TABLE, 0x00, 0xFF, 0xFF, 0x01, 0x02]; // len=0xFFFF, only 6 bytes
assert!(
walk_mkb(&mkb).is_empty(),
"over-long record yields no records"
);
// A sub-4 length (shorter than the header itself) is also rejected.
let short = vec![REC_VKD_TABLE, 0x00, 0x00, 0x02];
assert!(walk_mkb(&short).is_empty(), "sub-4 length is rejected");
// A truncated header (< 4 bytes) yields nothing.
assert!(walk_mkb(&[0x10, 0x00]).is_empty());
}
#[test]
fn mkb_type_and_version_decode_from_the_type_record() {
let mut mkb = type_and_version(MKB_21_CATEGORY_C, 100);
mkb.extend([0x00, 0x00, 0x00, 0x00]);
assert_eq!(mkb_type_raw(&mkb), Some(MKB_21_CATEGORY_C));
assert_eq!(mkb_version(&mkb), Some(100));
assert_eq!(mkb_is_uhd(&mkb), Some(true), "2.1 Category C is UHD");
let bd = type_and_version(MKB_TYPE_4_PRERECORDED, 68);
assert_eq!(
mkb_is_uhd(&bd),
Some(false),
"AACS 1.0 prerecorded is not UHD"
);
// No Type record → None (not a panic, not a fabricated value).
assert_eq!(mkb_version(&rec(REC_VKD_TABLE, &[0; 16])), None);
assert_eq!(mkb_type_raw(&[]), None);
}
#[test]
fn trim_mkb_keeps_only_the_framed_records() {
let mut mkb = type_and_version(MKB_20_CATEGORY_C, 1);
let content_len = mkb.len(); // the single framed record, no end marker
mkb.extend([0x00, 0x00, 0x00, 0x00]); // end marker
mkb.extend([0xDE; 4096]); // trailing padding past the end marker
let trimmed = trim_mkb(mkb);
assert_eq!(
trimmed.len(),
content_len,
"trim keeps the framed records, dropping the end marker and padding"
);
}
// ── BE24 length field: all THREE bytes ────────────────────────────────
/// The record length is a big-endian **24-bit** field, so the high byte
/// carries lengths of 64 KiB and up. The MKB records that matter most are
/// exactly that size — a real UHD cvalue table is `46_101 * 16` bytes and a
/// `0x2d` variant record is ~92 KiB — so a walker that dropped the high
/// byte would mis-frame every record of a real MKB from the first big one
/// onward, and every downstream key lookup would read the wrong bytes.
///
/// (The pre-existing high-byte test used total length `0x0110`, whose high
/// byte is ZERO — it exercised the middle byte only. This one puts a
/// non-zero value in the high byte.)
#[test]
fn mkb_records_honors_the_high_byte_of_the_be24_length() {
const TOTAL: usize = 0x0001_0004; // 65_540 — high byte 0x01
let mut mkb = vec![REC_VKD_TABLE, 0x01, 0x00, 0x04];
mkb.resize(TOTAL, 0xAB);
// A second record follows, so a walker that mis-read the length would
// frame a different number of records rather than merely a short one.
mkb.extend(rec(REC_TYPE_AND_VERSION, &[0x11; 8]));
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the big record must be framed as ONE record");
assert_eq!(
recs[0].rec_len, TOTAL,
"rec_len must include the high BE24 byte"
);
assert_eq!(recs[0].body.len(), TOTAL - 4);
assert_eq!(
recs[1].rec_type, REC_TYPE_AND_VERSION,
"the following record must start where the big one ends"
);
}
// ── Header-only records and the exact end marker ──────────────────────
/// `rec_len == 4` is a well-formed HEADER-ONLY record (the minimum the
/// walker accepts), including one sitting at the very end of the buffer
/// with no bytes after it. Rejecting either — the `pos + 4` bound or the
/// `rec_len < 4` floor being off by one — silently drops the MKB's last
/// record, and "the record isn't there" is indistinguishable from "the disc
/// doesn't carry it".
#[test]
fn mkb_records_yields_a_header_only_record_at_the_buffer_end() {
let mut mkb = rec(REC_TYPE_AND_VERSION, &[0xAA, 0xBB]);
mkb.extend([REC_VKD_TABLE, 0x00, 0x00, 0x04]); // 4-byte, empty body, at EOF
assert_eq!(
mkb.len(),
10,
"sanity: the last record ends at the buffer end"
);
let recs = walk_mkb(&mkb);
assert_eq!(recs.len(), 2, "the trailing header-only record is a record");
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].rec_len, 4);
assert!(recs[1].body.is_empty());
}
/// ONLY the exact `00 00 00 00` marker ends the walk. A record whose TYPE
/// happens to be `0x00` but which declares a real length is a record, not
/// the end of the MKB — stopping there would truncate everything after it,
/// including the cvalue and verify records the key derivation needs.
#[test]
fn mkb_records_stops_only_on_the_all_zero_end_marker() {
// A type-0 record of length 8, then a normal record, then the marker.
let mut mkb = vec![0x00, 0x00, 0x00, 0x08, 1, 2, 3, 4];
mkb.extend(rec(REC_VKD_TABLE, &[0x55; 16]));
mkb.extend([0x00, 0x00, 0x00, 0x00]); // the real end marker
mkb.extend(rec(0x99, &[0xFF; 4])); // past the marker: not walked
let recs = walk_mkb(&mkb);
assert_eq!(
recs.len(),
2,
"a type-0 record with a non-zero length is a record, not the end"
);
assert_eq!(recs[0].rec_type, 0x00);
assert_eq!(recs[0].rec_len, 8);
assert_eq!(recs[1].rec_type, REC_VKD_TABLE);
assert_eq!(recs[1].body, vec![0x55; 16]);
}
/// `mkb_type_raw` reports the 32-bit MKBType field verbatim ([C] §3.2.5.1.1
/// Table 3-2), including a value this build does not recognise — the caller
/// uses it to tell "unknown MKB generation" from "no Type record at all".
/// All four bytes must come from the record body; reading any of them from
/// the wrong offset yields a type that silently classifies as a different
/// AACS generation.
///
/// The recognised constants all share bytes with the `0x10` record-type
/// header byte (e.g. `MKB_21_CATEGORY_C` is `48 15 10 03`), so this uses a
/// value with four distinct bytes, none of them `0x10`.
#[test]
fn mkb_type_raw_reads_all_four_body_bytes() {
const RAW: u32 = 0xDEAD_BEEF;
let mkb = type_and_version(RAW, 7);
assert_eq!(
mkb_type_raw(&mkb),
Some(RAW),
"every byte of the MKBType field must come from the record body"
);
assert_eq!(mkb_version(&mkb), Some(7));
}
}
+345 -60
View File
@@ -13,20 +13,52 @@
//!
//! The VUK decrypts title keys from AACS/Unit_Key_RO.inf on disc.
//! Title keys decrypt m2ts stream content (AES-128-CBC).
//!
//! ## Spec provenance
//!
//! The crypto below carries `[TAG] §x.y` citations back to the published AACS
//! specification (Final Rev 0.953), so each primitive links to the section it
//! implements:
//! - `[C]` — AACS Introduction and Common Cryptographic Elements Book (primitives, MKB/key-management).
//! - `[PR]` — AACS Pre-recorded Video Book (Volume/Title Key layer).
//! - `[BD]` — AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit, Block Key).
//! - `[RE]` — reverse-engineered from real discs, cited only where the public
//! spec is silent (the `0x86` verify record and the Category-C MKB type values).
pub mod boil;
pub mod decrypt;
pub mod content;
pub mod crypto;
pub mod derive;
pub mod host_certs;
pub mod keys;
pub mod index_select;
pub mod inf;
pub mod mkb;
pub mod provider;
pub mod resolve;
pub mod segment;
pub mod segment_key;
pub mod trace;
pub mod types;
pub mod variants;
pub mod variant;
/// On-disc UDF paths to the AACS key-input files (with their fallbacks).
/// Centralised so every reader (`resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, `read_aacs_version`) walks the exact same files — adding
/// or changing a fallback in one place can then never silently diverge the
/// On-disc UDF paths to the AACS key-input files, plus HD DVD AACS-directory
/// discovery.
///
/// BD and UHD keep their key material under a fixed `/AACS/…` tree, so those
/// paths are constants. HD DVD keeps the equivalents in a reserved root
/// directory whose NAME is authoring-house-specific — observed `ANY!` (Dukes
/// of Hazzard) and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK`
/// mirror — and whose title-key file is NOT always `VTKF000.AACS` (Freedom
/// ships `VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED
/// from the parsed UDF tree ([`find_hddvd_aacs_dir`] + [`role_paths`]), never
/// hardcoded.
///
/// Each key ROLE ([`AacsRole`]) resolves to an ordered candidate list — the
/// BD/UHD constants first, then whatever the HD DVD directory actually holds —
/// which every reader walks with [`read_first`], first-that-reads. No reader
/// ever branches on disc type: a BD/UHD disc has the `/AACS/` files so those
/// win; an HD DVD has none of them, so it falls through to the discovered
/// entries. Centralised so `resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, and `read_aacs_version` can never silently diverge the
/// disc_hash / MKB / VID that another reader feeds a key service.
pub const PATH_UNIT_KEY_RO: &str = "/AACS/Unit_Key_RO.inf";
pub const PATH_UNIT_KEY_RO_DUPLICATE: &str = "/AACS/DUPLICATE/Unit_Key_RO.inf";
@@ -35,50 +67,135 @@ pub const PATH_MKB_RW: &str = "/AACS/MKB_RW.inf";
pub const PATH_CONTENT_CERT: &str = "/AACS/Content000.cer";
pub const PATH_CONTENT_CERT_ALT: &str = "/AACS/Content001.cer";
// Boil-down derivation primitives (thin newtypes + wrappers over the crypto).
pub use boil::{MediaKey, UnitKey, Vid, Vuk, mk_from_dk, mk_from_pk, uk_from_vuk, vuk_from_mk};
// Structured, English-free resolution trace.
pub use trace::{KeyNode, KeyOutcome, KeyStep, ResolutionTrace, UnlockOutcome, UnlockStep};
/// An AACS key-input role. [`role_paths`] maps it to an ordered candidate path
/// list (BD/UHD constants, then the discovered HD DVD files).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AacsRole {
/// Title-key file: BD/UHD `Unit_Key_RO.inf`, HD DVD `VTKF*.AACS`
/// (magic `DVD_HD_V_TKF`). The disc_hash is `SHA1` of this file.
UnitKey,
/// Media Key Block: BD/UHD `MKB_RO/RW.inf`, HD DVD `MKBROM.AACS`.
Mkb,
/// Content certificate: BD/UHD `Content000/001.cer`, HD DVD
/// `CONTENT_CERT.AACS` (byte 0 gives the AACS major).
ContentCert,
}
// Explicit re-exports — only items needed by external consumers and sibling crate modules.
// AES primitives (aes_ecb_encrypt, aes_ecb_decrypt, aes_cbc_decrypt) are pub(crate) in decrypt.rs.
pub use decrypt::{
ALIGNED_UNIT_LEN, ALIGNED_UNIT_SECTORS, UnitKeyResult, aacs_unit_encrypted,
aacs_unit_needs_decrypt, aacs_unit_still_ciphertext, decrypt_bus, decrypt_unit,
decrypt_unit_checked, decrypt_unit_full, decrypt_unit_try_keys, fill_null_ts_unit,
is_unit_aligned, ts_packet_total, ts_sync_count, ts_sync_destroyed, unit_is_clean_ps,
unit_is_clean_ts, unit_key_validates,
};
// `probe` is a reproduction-harness helper (see keys.rs), not part of the
// documented 1.0 surface; keep it reachable but off the rendered docs so we
// don't commit semver stability to test primitives.
#[doc(hidden)]
pub use keys::probe;
pub use keys::{
AACS_MAJOR_BD, AACS_MAJOR_UHD, AacsVersion, ContentCert, MKB_20_CATEGORY_C, MKB_21_CATEGORY_C,
MKB_TYPE_3_RECORDABLE, MKB_TYPE_4_PRERECORDED, MKB_TYPE_10_CLASS_II, MkbType, ResolveContext,
ResolveFailure, ResolvedKeys, UnitKeyFile, decrypt_unit_key, derive_media_key_and_pk_from_dk,
derive_media_key_from_dk, derive_media_key_from_pk, derive_vuk, disc_hash, disc_hash_hex,
mkb_content_len, mkb_is_uhd, mkb_type, mkb_type_raw, mkb_version, parse_content_cert,
parse_unit_key_ro, read_mkb_from_drive, recover_dk_position, resolve_keys_v1, resolve_keys_v2,
resolve_keys_v21, resolve_keys_with_reason, trim_mkb,
};
pub use provider::KeyProvider;
pub use types::{DeviceKey, DiscEntry, HostCert};
pub use variants::{
KEY_CORRECTION_DATA_PLACEHOLDER, MediaKeyVariantError, MkbRecord, ProcessingKeyMatch,
derive_media_key_variant, is_variant_mkb, variant_nonce, walk_mkb, walk_processing_key,
};
/// The HD DVD AACS directory in a parsed UDF tree, if present.
///
/// Identified structurally, NOT by a hardcoded name: the root child directory
/// whose name ends in `!` (so the `<name>!_BAK` backup mirror, which also ends
/// in a non-`!` char, is not mistaken for it) and which contains `MKBROM.AACS`.
/// Observed real names: `ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD
/// disc has no such directory → `None`.
pub(crate) fn find_hddvd_aacs_dir(udf: &crate::udf::UdfFs) -> Option<&crate::udf::DirEntry> {
udf.root.entries.iter().find(|e| {
e.is_dir
&& e.name.ends_with('!')
&& e.entries
.iter()
.any(|c| !c.is_dir && c.name.eq_ignore_ascii_case("MKBROM.AACS"))
})
}
/// Ordered candidate paths for an AACS key [`AacsRole`]: the fixed BD/UHD
/// `/AACS/…` paths first, then the actual HD DVD files discovered in the disc's
/// AACS directory (see [`find_hddvd_aacs_dir`]). A disc has only one family, so
/// the other family's entries simply never read.
///
/// For [`AacsRole::UnitKey`] every `VTKF*.AACS` in the directory is appended in
/// sorted name order — a disc may carry more than one variant (Freedom:
/// `VTKF090` + `VTKF100`), not just `VTKF000`.
pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String> {
let mut v: Vec<String> = match role {
AacsRole::UnitKey => vec![PATH_UNIT_KEY_RO, PATH_UNIT_KEY_RO_DUPLICATE],
AacsRole::Mkb => vec![PATH_MKB_RO, PATH_MKB_RW],
AacsRole::ContentCert => vec![PATH_CONTENT_CERT, PATH_CONTENT_CERT_ALT],
}
.into_iter()
.map(String::from)
.collect();
if let Some(dir) = find_hddvd_aacs_dir(udf) {
let d = &dir.name;
match role {
AacsRole::Mkb => v.push(format!("/{d}/MKBROM.AACS")),
AacsRole::ContentCert => v.push(format!("/{d}/CONTENT_CERT.AACS")),
AacsRole::UnitKey => {
// Glob VTKF*.AACS — the title-key filename is not fixed at
// VTKF000 (Freedom ships VTKF090 + VTKF100). Sorted for a
// deterministic try order.
//
// Each VTKF%%%.AACS is bound to ONE playlist (VPLST%%%.XPL): the
// TKF's 12-byte PLAYLIST_NAME field (bytes 0x10..0x1C) names the
// playlist whose Title Keys it carries, and keys from a TKF whose
// name does not match the title's playlist must not be used. The
// caller resolves this by trying candidates in sorted order and
// decrypting with the one whose keys verify — correct for a
// single-playlist disc; a name-matched selection keyed on the
// active playlist is the precise form for multi-playlist discs.
let mut names: Vec<&str> = dir
.entries
.iter()
.filter(|e| !e.is_dir)
.filter(|e| {
let u = e.name.to_ascii_uppercase();
u.starts_with("VTKF") && u.ends_with(".AACS")
})
.map(|e| e.name.as_str())
.collect();
names.sort_unstable();
v.extend(names.into_iter().map(|n| format!("/{d}/{n}")));
}
}
}
v
}
/// Walk an AACS role's candidate paths (from [`role_paths`]) and return the
/// first that reads.
///
/// `read` performs the actual per-path read (full file or bounded prefix), so
/// callers share the same first-present walk regardless of read style. Returns
/// [`Error::AacsNoKeys`] if no candidate is present. Generic over the path
/// element (`&str` or owned `String`) so it accepts the `Vec<String>` that
/// [`role_paths`] builds from the discovered HD DVD directory.
pub(crate) fn read_first<S, F>(candidates: &[S], mut read: F) -> crate::error::Result<Vec<u8>>
where
S: AsRef<str>,
F: FnMut(&str) -> crate::error::Result<Vec<u8>>,
{
for path in candidates {
if let Ok(buf) = read(path.as_ref()) {
return Ok(buf);
}
}
Err(crate::error::Error::AacsNoKeys)
}
// The module structure IS the public API — consumers import from the owning
// module directly (e.g. `aacs::content::decrypt_unit`, `aacs::mkb::MkbType`,
// `aacs::derive::{derive_vuk, resolve_candidate}`, `aacs::resolve::resolve_keys_v2`).
// The `derive::probe` reproduction harness stays reachable via its module path.
//
// A small set of flat re-exports is kept for the typed key primitives and the
// content-decrypt entry points that downstream key-source crates import through
// the `aacs::` path. These are the stable, load-bearing names; keeping them here
// lets those crates track the module refactor without a lockstep re-pin.
pub use content::ALIGNED_UNIT_LEN;
pub use derive::derive_vuk;
pub use types::{DeviceKey, HostCert, MediaKey, ProcessingKey, UnitKey, Vid, Vuk};
#[cfg(test)]
mod tests {
//! Re-export surface guards. The module's public API is the set of
//! `pub use` items above. A regression that drops or renames an export
//! (the class of bug that shipped in 0.31.0 by silently changing a
//! surface) breaks compilation of these references, so they act as a
//! compile-time contract for the crate's AACS surface.
//! Surface guards. The public API is the module tree itself (no facade).
//! Touching one representative item per module keeps these as a
//! compile-time contract that the module paths stay stable.
use super::*;
use super::content::ALIGNED_UNIT_LEN;
use super::inf::{disc_hash, disc_hash_hex};
use super::mkb::{AacsVersion, mkb_content_len, walk_mkb};
use super::variant::is_variant_mkb;
#[test]
fn aligned_unit_len_is_three_2048_byte_sectors() {
@@ -100,21 +217,189 @@ mod tests {
}
#[test]
fn key_correction_data_placeholder_is_all_zero() {
// The variant chain refuses to run against this all-zero placeholder
// KCD; the public constant must therefore be exactly 16 zero bytes.
assert_eq!(KEY_CORRECTION_DATA_PLACEHOLDER, [0u8; 16]);
}
#[test]
fn public_helpers_are_callable_through_the_facade() {
// Touch a representative function from each re-export group so a
// dropped/renamed export fails to compile. These are smoke calls, not
// behavioural assertions (behaviour is covered in each module).
let _ = ts_sync_destroyed(&[0u8; ALIGNED_UNIT_LEN]);
fn public_helpers_are_callable_by_module_path() {
// Touch a representative function from each module so a dropped/renamed
// item fails to compile. Smoke calls, not behavioural assertions.
let _ = !crate::aacs::content::is_clean(
&[0u8; ALIGNED_UNIT_LEN],
crate::disc::ContentFormat::BdTs,
);
let _ = mkb_content_len(&[]);
let _ = is_variant_mkb(&walk_mkb(&[]));
let _ = disc_hash_hex(&disc_hash(b"x"));
let _ = mk_from_pk(&[[0u8; 16]], &[]);
let _ = super::derive::resolve_candidate(
&super::derive::KeyCandidate::Uk(super::types::UnitKey::new(0, [0u8; 16])),
&[],
&[],
None,
);
}
// ── HD DVD AACS directory / filename discovery ────────────────────────
//
// The HD DVD AACS dir name and title-key filename are authoring-specific
// and were previously hardcoded to `/ANY!/VTKF000.AACS`. These verify the
// discovery replacement against both real-disc shapes: Freedom (`AAC!` +
// `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
#[test]
fn role_paths_discovers_hddvd_dir_and_globs_all_vtkf_variants() {
use crate::udf::fixture::*;
// Freedom-shaped: an `AAC!` dir (NOT `ANY!`) holding MKBROM + two VTKF
// variants (090/100, NOT 000) + a VTUF usage file (must be excluded),
// plus the `AAC!_BAK` mirror (must NOT be picked as the AACS dir).
let mut disc = MemDisc::new();
let aacs_files = vec![
file("MKBROM.AACS", 100, 5000, 4096, true),
file("CONTENT_CERT.AACS", 101, 5100, 2048, true),
file("VTKF100.AACS", 102, 5200, 2048, true),
file("VTKF090.AACS", 103, 5300, 2048, true),
file("VTUF090.AACS", 104, 5400, 2048, true),
];
let bak_files = vec![file("MKBROM.AACS", 110, 6000, 4096, true)];
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![
DirSpec {
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: aacs_files,
subdirs: vec![],
},
DirSpec {
name: "AAC!_BAK".to_string(),
icb_lba: 30,
dir_data_lba: 31,
files: bak_files,
subdirs: vec![],
},
],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
// Discovered structurally (ends in '!', holds MKBROM.AACS) — the real
// AACS dir, never the `_BAK` mirror.
let dir = super::find_hddvd_aacs_dir(&udf).expect("aacs dir");
assert_eq!(dir.name, "AAC!");
// UnitKey: BD/UHD paths first, then EVERY VTKF*.AACS in sorted order
// (090 before 100) — NOT hardcoded VTKF000; VTUF (usage) excluded.
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
"/AAC!/VTKF090.AACS".to_string(),
"/AAC!/VTKF100.AACS".to_string(),
]
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::Mkb)
.last()
.unwrap(),
"/AAC!/MKBROM.AACS"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::ContentCert)
.last()
.unwrap(),
"/AAC!/CONTENT_CERT.AACS"
);
}
/// The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required —
/// the discovery is a conjunction, not a disjunction.
///
/// The existing fixtures only ever present a directory that satisfies both
/// (`AAC!` with `MKBROM.AACS`) alongside one that satisfies neither
/// (`AAC!_BAK` — which contains `MKBROM.AACS` but is ALSO reached only after
/// the real dir), so either half of the conjunction could be dropped and the
/// same directory would still be found. Here a directory satisfies the name
/// half and NOT the contents half: it must not be picked.
///
/// If it were, the HD DVD path would resolve `MKBROM.AACS`,
/// `CONTENT_CERT.AACS` and the title-key file under a directory that holds
/// none of them — the disc reports "no AACS key files" and never rips.
#[test]
fn a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory() {
use crate::udf::fixture::*;
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
// Ends in '!' — but carries no MKBROM.AACS, so it is not the
// HD DVD AACS directory.
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTKF090.AACS", 102, 5200, 2048, true),
file("CONTENT_CERT.AACS", 103, 5300, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(
super::find_hddvd_aacs_dir(&udf).is_none(),
"a '!' directory without MKBROM.AACS is not the AACS directory"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
],
"no HD DVD candidates may be appended from a directory that was \
never identified as the AACS directory"
);
}
#[test]
fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() {
use crate::udf::fixture::*;
// A `/AACS/` tree (BD/UHD) has no '!' directory → discovery finds none
// and the candidate list is exactly the static BD/UHD paths.
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "AACS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("Unit_Key_RO.inf", 100, 5000, 2048, true),
file("MKB_RO.inf", 101, 5100, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(super::find_hddvd_aacs_dir(&udf).is_none());
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
]
);
}
}
+48 -1
View File
@@ -42,7 +42,7 @@ use super::types::{DeviceKey, DiscEntry, HostCert};
/// Source of AACS key material.
///
/// Implementors return raw material only — the resolver in
/// `aacs::keys` owns all the crypto (DK→PK walking, PK validation,
/// `aacs::resolve` and `aacs::derive` own the crypto (DK→PK walking, PK validation,
/// MK→VUK→TK derivation). See module docs for method semantics.
pub trait KeyProvider: Send + Sync {
/// Device keys (top-of-tree, walked by the resolver).
@@ -197,6 +197,17 @@ mod tests {
}
}
/// A host cert whose (non-secret) certificate body and private key are both
/// filled with `byte`, so a cert is identifiable in an aggregated list.
fn cert(byte: u8) -> HostCert {
HostCert {
private_key: [byte; 20],
certificate: vec![byte; 92],
private_key_v2: None,
certificate_v2: None,
}
}
fn dk(byte: u8, node: u16) -> DeviceKey {
DeviceKey {
key: [byte; 16],
@@ -357,6 +368,42 @@ mod tests {
assert_eq!(got.disc_hash, "vid-a");
}
/// `Providers::host_certs` is the union across the provider array. It is not
/// wired into the handshake today (see the module docs), so nothing else in
/// the crate would notice a body that dropped every cert on the floor — and
/// the day it IS wired in, a silently-empty cert list means the drive AACS
/// authentication finds no host certificate to present and every disc fails
/// to open, with no indication that the caller's certs were discarded.
///
/// Unlike the bulk key unions this one does NOT dedup (HostCert is not
/// Ord/Hash), so the assertion is on the full concatenation in array order.
#[test]
fn providers_host_certs_unions_every_providers_certs_in_array_order() {
struct Certs(Vec<HostCert>);
impl KeyProvider for Certs {
fn host_certs(&self) -> Vec<HostCert> {
self.0.clone()
}
}
// Distinguish certs by their (non-secret) certificate body, so the
// assertion lands on WHICH certs came back, not merely how many.
let a = Certs(vec![cert(0xA1), cert(0xA2)]);
let b = Certs(vec![cert(0xB1)]);
let arr: &[&dyn KeyProvider] = &[&a, &b];
let got = Providers(arr).host_certs();
let bodies: Vec<Vec<u8>> = got.iter().map(|c| c.certificate.clone()).collect();
assert_eq!(
bodies,
vec![vec![0xA1u8; 92], vec![0xA2u8; 92], vec![0xB1u8; 92]],
"every provider's certs must survive the union, in array order"
);
// The private key travels with the cert — a union that returned default
// certs would still have the right count.
assert_eq!(got[0].private_key, [0xA1u8; 20]);
assert_eq!(got[2].private_key, [0xB1u8; 20]);
}
#[test]
fn providers_empty_array_yields_nothing() {
let arr: &[&dyn KeyProvider] = &[];
File diff suppressed because it is too large Load Diff
+454
View File
@@ -0,0 +1,454 @@
//! AACS 2.1 FMTS forensic segment map — `AACS/IndividualSegment.tbl`.
//!
//! An FMTS main feature interleaves short forensic **segments** — the sequence-key
//! / forensic-watermark mechanism. Each segment carries an **index** (1..32): a
//! tag in `IndividualSegment.tbl` that selects which of the 32 forensic **index
//! keys** decrypts that segment's units, in place of the ordinary CPS Unit Key.
//!
//! Terminology (see the project AACS reference): the **index** here is NOT the
//! AACS 2.1 *Media Key Variant* — that is the 65536-value device selector in the
//! MKB that decides *which set* of index keys a device receives, a layer this
//! module does not deal with. All the index keys belong to one variant, whose
//! number is unknown and irrelevant to the segment map. Decrypting a segment with
//! the Unit Key yields garbage — broken HEVC reference frames (empirically:
//! `Could not find ref with POC …` on a plain unit-key rip).
//!
//! This table says WHERE the segments live and which index each carries, so a
//! decoder can decrypt them with the matching index key instead of muxing
//! unit-key garbage.
//!
//! Format (validated against a retail AACS 2.1 disc):
//! ```text
//! header (8 bytes): u32 type | u16 count | u16 record_size (= 16)
//! record[count] (16 bytes each):
//! u32 marker (= 0x01000000) | u16 index | u16 flag (= 1)
//! u32 start_spn | u32 end_spn (source-packet numbers, inclusive)
//! ```
//! `index` is the 1..32 forensic index tag, NOT a sequential segment id: measured
//! on a retail 2.1 disc it cycles 1,2,…,32,1,2,… across records in
//! file order — 24 full cycles of 32 plus a final partial cycle of 24 = 792
//! records. Source-packet numbers are the 192-byte BDAV packet index: byte offset
//! = `spn * 192`. Each segment is ~2560 packets (~480 KB) = 80 aligned units,
//! spread across the entire 54 GB feature (one roughly every 67 MB). Inside a
//! segment the 80 units interleave in two stride-2 halves: applying the segment's
//! index key decrypts ~40 of them to clean TS and garbles the other ~40 (a second
//! interleaved half, unidentified), which the demux then drops — leaving one
//! coherent stream. Confirmed by decoding a retail disc with a full set of 32
//! index keys.
/// Fixed size of one `IndividualSegment.tbl` record.
pub const SEGMENT_RECORD_LEN: usize = 16;
/// Bytes per BDAV source packet (188-byte TS + 4-byte arrival-time header).
pub const SOURCE_PACKET_LEN: u64 = 192;
/// One forensic segment: the inclusive source-packet range it occupies in the
/// FMTS clip.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Segment {
/// Forensic index tag, 1..=32 (field@4 of the record). Cycles across the
/// table rather than counting up — it selects WHICH of the 32 index keys
/// decrypts this range. (`0` is not used here; the default/non-forensic
/// content carries no segment record at all.)
pub index: u16,
/// First source packet of the segment (inclusive).
pub start_spn: u32,
/// Last source packet of the segment (inclusive).
pub end_spn: u32,
}
impl Segment {
/// Source-packet count in this (inclusive) segment.
pub fn packet_count(&self) -> u32 {
self.end_spn
.saturating_sub(self.start_spn)
.saturating_add(1)
}
/// Byte offset of the segment start within the clip (`start_spn * 192`).
pub fn start_byte(&self) -> u64 {
self.start_spn as u64 * SOURCE_PACKET_LEN
}
/// Byte length of the segment (`packet_count * 192`).
pub fn byte_len(&self) -> u64 {
self.packet_count() as u64 * SOURCE_PACKET_LEN
}
/// True when source packet `spn` falls inside this segment.
pub fn contains_spn(&self, spn: u32) -> bool {
spn >= self.start_spn && spn <= self.end_spn
}
/// True when the inclusive source-packet span `[first, last]` overlaps this
/// segment. Used to decide whether an aligned unit (which spans several
/// packets) touches the segment at all, not just whether one packet does.
pub fn overlaps_spn(&self, first: u32, last: u32) -> bool {
first <= self.end_spn && last >= self.start_spn
}
}
/// Source packets spanned by one AACS aligned unit: `6144 / 192 = 32`.
pub const PACKETS_PER_UNIT: u32 =
(crate::aacs::content::ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32;
/// Byte offset within the clip of a clip-relative 2048-byte sector `lba`. The
/// FMTS decode reads the clip file directly, so `lba` 0 is the clip's first
/// byte and this offset lines up with the source-packet grid the segment map
/// uses.
pub fn lba_byte_offset(lba: u32) -> u64 {
lba as u64 * 2048
}
/// The forensic segment an AACS aligned unit belongs to, if any, given the
/// unit's clip-relative byte offset.
///
/// This is the routing decision behind a 2.1 decrypt-miss: a unit that
/// overlaps a forensic segment must be opened with that segment's **index key**
/// (selected by the segment's `index`), not the CPS Unit Key. Opening it with
/// the Unit Key is exactly what yields the broken-reference-frame garbage a
/// plain unit-key rip produces. A unit outside every segment is ordinary
/// content and a miss on it is a Unit-Key miss, so this returns `None` and the
/// caller falls back to the normal unit-key fetch.
///
/// The unit is tested as a packet *span* (`[off/192, (off+6144-1)/192]`) so a
/// unit that only partly overlaps a segment edge is still classified as
/// forensic; on the observed disc segments are unit-aligned, but the span test
/// does not rely on that.
pub fn segment_for_unit(segments: &[Segment], unit_offset: u64) -> Option<&Segment> {
let unit_len = crate::aacs::content::ALIGNED_UNIT_LEN as u64;
let first = (unit_offset / SOURCE_PACKET_LEN) as u32;
let last = ((unit_offset + unit_len - 1) / SOURCE_PACKET_LEN) as u32;
segments.iter().find(|s| s.overlaps_spn(first, last))
}
/// Parse `IndividualSegment.tbl` into its forensic segments, in table
/// order. Returns `None` when the header is malformed, the record size is not
/// [`SEGMENT_RECORD_LEN`], or the declared record count overruns the buffer —
/// so a truncated / foreign table degrades to "no segment map" rather than
/// yielding bogus ranges.
pub fn parse_individual_segments(tbl: &[u8]) -> Option<Vec<Segment>> {
if tbl.len() < 8 {
return None;
}
let count = u16::from_be_bytes([tbl[4], tbl[5]]) as usize;
let record_size = u16::from_be_bytes([tbl[6], tbl[7]]) as usize;
if record_size != SEGMENT_RECORD_LEN {
return None;
}
if 8usize.checked_add(count.checked_mul(record_size)?)? > tbl.len() {
return None;
}
let mut segments = Vec::with_capacity(count);
for i in 0..count {
let o = 8 + i * record_size;
// o+4..o+8 = index (u16, 1..32) + flag (u16); o+8..o+16 = start/end SPN.
let index = u16::from_be_bytes([tbl[o + 4], tbl[o + 5]]);
let start_spn = u32::from_be_bytes([tbl[o + 8], tbl[o + 9], tbl[o + 10], tbl[o + 11]]);
let end_spn = u32::from_be_bytes([tbl[o + 12], tbl[o + 13], tbl[o + 14], tbl[o + 15]]);
segments.push(Segment {
index,
start_spn,
end_spn,
});
}
Some(segments)
}
/// Map a clip-relative byte offset to the absolute LBA that holds it, by walking
/// the title's extents (the `.fmts` clip's sectors in file order). Segment
/// offsets in [`Segment`] are clip-relative source-packet numbers, so this is how
/// a segment's `spn` range becomes disc LBAs. `None` if the offset is past the
/// clip.
pub fn clip_byte_to_lba(extents: &[crate::disc::Extent], clip_byte: u64) -> Option<u32> {
let mut cum = 0u64;
for e in extents {
let len = e.sector_count as u64 * crate::consts::SECTOR_BYTES as u64;
if clip_byte < cum + len {
let sector_in_ext = ((clip_byte - cum) / crate::consts::SECTOR_BYTES as u64) as u32;
return Some(e.start_lba.saturating_add(sector_in_ext));
}
cum += len;
}
None
}
/// Build the `[start_lba, end_lba) → key_idx` ranges for an FMTS forensic key map.
///
/// Each forensic segment's clip-relative source-packet span becomes an absolute
/// LBA range tagged with the key its `index` selects (via `index_to_key_idx`,
/// e.g. `|i| i as usize` when the pool is `[base, idx1, idx2, …]`). Applying that
/// one key across the whole segment decodes the ~40 units of its interleave half
/// to clean TS and garbles the other ~40 (the second interleaved half), which the
/// demux then drops — yielding one coherent stream. Ranges outside every segment
/// are left for the map's default (the ordinary Unit Key). A segment that straddles
/// a UDF extent boundary is emitted as one range per whole-sector slice it covers.
///
/// The result feeds [`AacsKeyMap::from_ranges`](crate::decrypt::AacsKeyMap::from_ranges)
/// with the Unit-Key index as the default — the same structure the CPS map uses,
/// only finer-grained.
pub fn fmts_key_ranges(
segments: &[Segment],
extents: &[crate::disc::Extent],
index_to_key_idx: &dyn Fn(u16) -> usize,
) -> Vec<(u32, u32, usize)> {
let mut ranges = Vec::new();
for s in segments {
// SPNs are untrusted (from IndividualSegment.tbl); an inverted record
// (start_spn > end_spn) would underflow `end_byte - 1 - start_byte` below.
if s.start_spn > s.end_spn {
continue;
}
let start_byte = s.start_spn as u64 * SOURCE_PACKET_LEN;
let end_byte = (s.end_spn as u64 + 1) * SOURCE_PACKET_LEN; // exclusive
// A segment is unit-aligned and contiguous in clip bytes; map its first
// and last sector to LBAs. Segments are ~480 KB and extents are GB-sized,
// so a segment almost never crosses an extent boundary — but if the two
// ends land in different extents (non-contiguous LBAs), skip rather than
// emit a wrong span; the units there fall to the Unit Key (garble+drop),
// never a mis-decrypt.
let (Some(a), Some(b)) = (
clip_byte_to_lba(extents, start_byte),
clip_byte_to_lba(extents, end_byte - 1),
) else {
continue;
};
if b >= a
&& (b - a) as u64 == (end_byte - 1 - start_byte) / crate::consts::SECTOR_BYTES as u64
{
ranges.push((a, b + 1, index_to_key_idx(s.index)));
}
}
ranges
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a table with the real on-disc layout: 8-byte header + N 16-byte
/// records. `recs` are `(index, start_spn, end_spn)`.
fn build_tbl(recs: &[(u16, u32, u32)]) -> Vec<u8> {
let mut v = Vec::new();
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // type
v.extend_from_slice(&(recs.len() as u16).to_be_bytes()); // count
v.extend_from_slice(&(SEGMENT_RECORD_LEN as u16).to_be_bytes()); // record_size
for &(n, s, e) in recs {
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // marker
v.extend_from_slice(&n.to_be_bytes());
v.extend_from_slice(&1u16.to_be_bytes()); // flag
v.extend_from_slice(&s.to_be_bytes());
v.extend_from_slice(&e.to_be_bytes());
}
v
}
#[test]
fn fmts_key_ranges_maps_segments_to_lba_by_index() {
use crate::disc::Extent;
// One big clip extent starting at LBA 1000. Clip byte B lives at
// LBA 1000 + B/2048.
let extents = vec![Extent {
start_lba: 1000,
sector_count: 1_000_000,
}];
// Two segments, indexes 5 and 7 (spn ranges as on a real disc).
let segs = vec![
Segment {
index: 5,
start_spn: 100,
end_spn: 199,
},
Segment {
index: 7,
start_spn: 10_000,
end_spn: 10_099,
},
];
// Pool layout [base, idx1, idx2, …] → index N uses key slot N.
let ranges = fmts_key_ranges(&segs, &extents, &|v| v as usize);
assert_eq!(ranges.len(), 2, "one LBA range per segment");
// Segment 0: spn 100..=199 → clip bytes [19200, 38400) → sectors 9..=18
// → LBA 1009..1019, key index 5.
assert_eq!(ranges[0], (1009, 1019, 5));
// Segment 1: spn 10000..=10099 → bytes [1_920_000, 1_939_200) →
// sectors 937..=946 → LBA 1937..1947, key index 7.
assert_eq!(ranges[1], (1937, 1947, 7));
// The ranges drive a positive AacsKeyMap: an LBA in no range has no key.
let map = crate::decrypt::AacsKeyMap::from_ranges(ranges);
assert_eq!(map.key_idx_for(500), None, "outside any segment → no key");
assert_eq!(
map.key_idx_for(1012),
Some(5),
"inside index-5 segment → key 5"
);
assert_eq!(
map.key_idx_for(1940),
Some(7),
"inside index-7 segment → key 7"
);
assert_eq!(
map.key_idx_for(1019),
None,
"segment end is exclusive → no key"
);
}
#[test]
fn fmts_key_ranges_skips_inverted_segment_without_underflow() {
use crate::disc::Extent;
let extents = vec![Extent {
start_lba: 1000,
sector_count: 1_000_000,
}];
// start_spn == end_spn + 1: `end_byte - 1 - start_byte` would underflow.
// The record must be skipped rather than panic (debug) / wrap (release).
let segs = vec![Segment {
index: 5,
start_spn: 200,
end_spn: 199,
}];
let ranges = fmts_key_ranges(&segs, &extents, &|v| v as usize);
assert!(ranges.is_empty(), "inverted segment yields no range");
}
#[test]
fn clip_byte_to_lba_walks_extents() {
use crate::disc::Extent;
let extents = vec![
Extent {
start_lba: 100,
sector_count: 10,
}, // clip bytes [0, 20480)
Extent {
start_lba: 500,
sector_count: 10,
}, // clip bytes [20480, 40960)
];
assert_eq!(clip_byte_to_lba(&extents, 0), Some(100));
assert_eq!(clip_byte_to_lba(&extents, 2048), Some(101));
assert_eq!(clip_byte_to_lba(&extents, 20480), Some(500)); // second extent
assert_eq!(clip_byte_to_lba(&extents, 22528), Some(501));
assert_eq!(clip_byte_to_lba(&extents, 40960), None); // past the clip
}
#[test]
fn parses_real_disc_layout() {
// First three records observed on retail 2.1: the variant
// field counts 1,2,3,… (it wraps at 32 further into the table — see
// `index_field_cycles_one_to_thirty_two`), segments are 2560 packets.
let tbl = build_tbl(&[
(1, 343680, 346239),
(2, 695616, 698175),
(3, 1051840, 1054399),
]);
let segs = parse_individual_segments(&tbl).expect("parse");
assert_eq!(segs.len(), 3);
assert_eq!(segs[0].index, 1);
assert_eq!(segs[1].index, 2);
assert_eq!(segs[2].index, 3);
assert_eq!(segs[0].start_spn, 343680);
assert_eq!(segs[0].end_spn, 346239);
assert_eq!(segs[0].packet_count(), 2560);
assert_eq!(segs[0].byte_len(), 2560 * 192);
assert_eq!(segs[0].start_byte(), 343680 * 192);
assert!(segs[0].contains_spn(345000));
assert!(!segs[0].contains_spn(343679));
assert!(!segs[0].contains_spn(346240));
}
#[test]
fn rejects_wrong_record_size() {
let mut tbl = build_tbl(&[(1, 0, 10)]);
tbl[6..8].copy_from_slice(&20u16.to_be_bytes()); // record_size != 16
assert!(parse_individual_segments(&tbl).is_none());
}
#[test]
fn rejects_truncated_and_overrun() {
assert!(parse_individual_segments(&[0u8; 4]).is_none()); // < header
let mut tbl = build_tbl(&[(1, 0, 10)]);
tbl[4..6].copy_from_slice(&99u16.to_be_bytes()); // claims 99 recs, has 1
assert!(parse_individual_segments(&tbl).is_none());
}
#[test]
fn empty_table_is_empty_not_none() {
let tbl = build_tbl(&[]);
assert_eq!(parse_individual_segments(&tbl), Some(Vec::new()));
}
#[test]
fn packets_per_unit_is_thirty_two() {
// 6144-byte aligned unit / 192-byte source packet.
assert_eq!(PACKETS_PER_UNIT, 32);
}
#[test]
fn unit_inside_segment_routes_to_index() {
// A real first-record segment: packets [343680, 346239].
let segs = parse_individual_segments(&build_tbl(&[(1, 343680, 346239)])).unwrap();
// A unit sitting squarely inside: start at packet 344000 → byte 344000*192.
let off = 344000u64 * SOURCE_PACKET_LEN;
let hit = segment_for_unit(&segs, off).expect("inside the segment");
assert_eq!(hit.index, 1);
}
#[test]
fn index_field_cycles_one_to_thirty_two() {
// Reality on a retail 2.1 disc: field@4 is the index, cycling 1..=32 in file
// order (NOT a sequential segment id). Reproduce one-and-a-bit cycles.
let mut recs = Vec::new();
let mut spn = 1000u32;
for row in 0..2 {
for v in 1..=32u16 {
recs.push((v, spn, spn + 2559));
spn += 50_000; // ~one segment every ~67 MB
}
let _ = row;
}
let segs = parse_individual_segments(&build_tbl(&recs)).unwrap();
assert_eq!(segs.len(), 64);
assert_eq!(segs[31].index, 32); // end of first cycle
assert_eq!(segs[32].index, 1); // wraps, does not become 33
assert!(segs.iter().all(|s| (1..=32).contains(&s.index)));
}
#[test]
fn unit_outside_every_segment_is_unit_key_miss() {
let segs = parse_individual_segments(&build_tbl(&[(1, 343680, 346239)])).unwrap();
// A unit well before the segment is ordinary content → None (unit-key path).
let off = 1000u64 * SOURCE_PACKET_LEN;
assert!(segment_for_unit(&segs, off).is_none());
}
#[test]
fn unit_straddling_a_segment_edge_counts_as_forensic() {
// Segment starts at packet 100. A unit that ENDS just inside it (its 32
// packets straddle the boundary) must still route to the index key,
// because part of its ciphertext is forensic-encrypted.
let segs = parse_individual_segments(&build_tbl(&[(7, 100, 200)])).unwrap();
// Unit covering packets [80, 111]: overlaps [100,200] at the tail.
let off = 80u64 * SOURCE_PACKET_LEN;
let hit = segment_for_unit(&segs, off).expect("straddles the start edge");
assert_eq!(hit.index, 7);
// A unit ending exactly at packet 99 (offset s.t. last = 99) does NOT overlap.
let before = 68u64 * SOURCE_PACKET_LEN; // [68, 99]
assert!(segment_for_unit(&segs, before).is_none());
}
#[test]
fn no_segments_never_routes_to_index() {
// The 1.0 / 2.0 case: no forensic map, so every miss is a unit-key miss.
assert!(segment_for_unit(&[], lba_byte_offset(0)).is_none());
assert!(segment_for_unit(&[], lba_byte_offset(9_999_999)).is_none());
}
#[test]
fn lba_maps_to_the_packet_grid() {
// A unit is 3 sectors (6144 bytes) = 32 packets. Clip-relative LBA 3 is
// the second aligned unit, which starts at packet 32.
let off = lba_byte_offset(3);
assert_eq!(off / SOURCE_PACKET_LEN, 32);
}
}
+165
View File
@@ -0,0 +1,165 @@
//! AACS 2.1 FMTS forensic segment keys, `AACS/SegmentKeyNNNNN.tbl`.
//!
//! One file per CPS unit (`SegmentKey00001.tbl`, ...). It is the on-disc key
//! store for the forensic variant segments mapped by [`super::segment`]. A
//! device does not read a segment key directly. It derives a **16-bit variant
//! selector** from the Media Key Variant chain (see [`super::variant`]) and uses
//! that selector to index this table, which is how the device's position in the
//! key tree decides which variant it can decrypt (the traitor-tracing link).
//!
//! Container format (confirmed against a retail AACS 2.1 disc):
//! ```text
//! header (8 bytes): u32 tag | u16 index_space | u16 record_size
//! record[index_space] (record_size bytes each)
//! ```
//! On the reference disc: `index_space` = `0xffff` (the full 16-bit selector
//! space, 65536 records), `record_size` = `0x0218` = 536. Total
//! `8 + 65536 * 536 = 35,127,304` bytes, which matches the file exactly. Each
//! record begins with an 8-byte sub-header, then 528 bytes of encrypted key
//! material.
//!
//! **Not yet reversed:** the internal layout of a record's 528-byte payload, and
//! how it maps onto the segments of [`super::segment`]. One numeric coincidence
//! worth noting for whoever cracks it: the reference disc has 792 segments and
//! `528 = 33 * 16`, with `792 = 24 * 33`, so `33` appears on both sides. Until
//! the mapping and the key derivation are pinned, this module exposes only the
//! confirmed container: locate the record for a given 16-bit selector.
/// Bytes of the fixed file header.
pub const HEADER_LEN: usize = 8;
/// The on-disc segment-key table container. Borrows the file bytes; a record is
/// looked up by the 16-bit variant selector.
#[derive(Debug, Clone, Copy)]
pub struct SegmentKeyTable<'a> {
data: &'a [u8],
/// Number of records (the selector index space, e.g. 65536).
count: usize,
/// Bytes per record (e.g. 536).
record_size: usize,
}
impl<'a> SegmentKeyTable<'a> {
/// Parse and validate the container header against the buffer length.
///
/// Returns `None` when the buffer is too small, or the declared
/// `count * record_size` (plus header) does not match the buffer, so a
/// truncated or foreign table degrades to "no segment keys" rather than
/// handing back bogus records. `index_space` of `0xffff` is read as the full
/// 65536-entry space (a device selector is a full 16-bit value).
pub fn parse(data: &'a [u8]) -> Option<Self> {
if data.len() < HEADER_LEN {
return None;
}
let index_space = u16::from_be_bytes([data[4], data[5]]);
let record_size = u16::from_be_bytes([data[6], data[7]]) as usize;
// 0xffff means the full 16-bit selector space (65536 records).
let count = if index_space == 0xffff {
0x1_0000
} else {
index_space as usize
};
if record_size == 0 {
return None;
}
let body = count.checked_mul(record_size)?;
if HEADER_LEN.checked_add(body)? != data.len() {
return None;
}
Some(Self {
data,
count,
record_size,
})
}
/// Number of records (the selector index space).
pub fn record_count(&self) -> usize {
self.count
}
/// Bytes per record.
pub fn record_size(&self) -> usize {
self.record_size
}
/// The raw record for a 16-bit variant `selector`, including its 8-byte
/// sub-header. `None` if the selector is past the table (only possible when
/// `index_space` was not the full 16-bit space).
pub fn record(&self, selector: u16) -> Option<&'a [u8]> {
let idx = selector as usize;
if idx >= self.count {
return None;
}
let start = HEADER_LEN + idx * self.record_size;
self.data.get(start..start + self.record_size)
}
/// The encrypted key payload for a selector: the record with its 8-byte
/// sub-header stripped. The internal layout of these bytes is not yet
/// reversed (see module docs).
pub fn record_payload(&self, selector: u16) -> Option<&'a [u8]> {
self.record(selector).and_then(|r| r.get(HEADER_LEN..))
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a container with `record_size` and the given `index_space`, filling
/// each record with a distinguishable byte so lookups can be checked.
fn build(index_space: u16, record_size: u16) -> Vec<u8> {
let count = if index_space == 0xffff {
0x1_0000
} else {
index_space as usize
};
let mut v = Vec::with_capacity(HEADER_LEN + count * record_size as usize);
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // tag
v.extend_from_slice(&index_space.to_be_bytes());
v.extend_from_slice(&record_size.to_be_bytes());
for i in 0..count {
let mut rec = vec![(i & 0xff) as u8; record_size as usize];
// sub-header, as seen on disc
rec[..8].copy_from_slice(&[0x01, 0x00, 0x00, 0x00, 0x00, 0x20, 0x01, 0x02]);
v.extend_from_slice(&rec);
}
v
}
#[test]
fn parses_retail_container_geometry() {
// The real disc: 0xffff index space, 536-byte records, 35,127,304 total.
let data = build(0xffff, 536);
assert_eq!(
data.len(),
35_127_304,
"matches the retail file size exactly"
);
let t = SegmentKeyTable::parse(&data).expect("parse");
assert_eq!(t.record_count(), 65_536);
assert_eq!(t.record_size(), 536);
let rec = t.record(0x1234).expect("record");
assert_eq!(rec.len(), 536);
assert_eq!(&rec[..8], &[0x01, 0x00, 0x00, 0x00, 0x00, 0x20, 0x01, 0x02]);
assert_eq!(t.record_payload(0x1234).unwrap().len(), 528);
}
#[test]
fn small_index_space_bounds_lookups() {
let data = build(4, 32);
let t = SegmentKeyTable::parse(&data).expect("parse");
assert_eq!(t.record_count(), 4);
assert!(t.record(3).is_some());
assert!(t.record(4).is_none(), "selector past the table is None");
}
#[test]
fn rejects_size_mismatch_and_truncation() {
assert!(SegmentKeyTable::parse(&[0u8; 4]).is_none());
let mut data = build(4, 32);
data.truncate(data.len() - 1); // body no longer matches header
assert!(SegmentKeyTable::parse(&data).is_none());
}
}
+296 -3
View File
@@ -6,7 +6,7 @@
//! owns only the crypto and these value types that flow through it.
/// A device key for MKB subset-difference tree processing.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DeviceKey {
pub key: [u8; 16],
pub node: u16,
@@ -15,7 +15,7 @@ pub struct DeviceKey {
}
/// Host certificate + private key for AACS SCSI authentication.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct HostCert {
/// AACS 1.0: 20 bytes. AACS 2.0: 32 bytes.
pub private_key: [u8; 20],
@@ -27,8 +27,75 @@ pub struct HostCert {
pub certificate_v2: Option<Vec<u8>>,
}
/// Volume ID (16 bytes) — read from the disc via the SCSI handshake / OEM path.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vid(pub [u8; 16]);
/// Media Key (Km, 16 bytes) — the MKB-scoped key derived from device keys.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct MediaKey(pub [u8; 16]);
/// Volume Unique Key (VUK / Kvu, 16 bytes) — derived from `MediaKey` + `Vid`,
/// decrypts the per-disc encrypted title keys in `Unit_Key_RO.inf`.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vuk(pub [u8; 16]);
/// Processing Key (Kp, 16 bytes) — an MKB Subset-Difference key that yields the
/// Media Key. A leaked/precomputed PK in the keydb, or the intermediate PK a
/// device-key walk derives at its matching SD node.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct ProcessingKey(pub [u8; 16]);
/// One decrypted per-CPS-unit AACS title key.
///
/// `idx` is the POSITIONAL index of the encrypted title key within the slice
/// handed to the VUK→UK step (i.e. its order in `Unit_Key_RO.inf`'s key-storage
/// area). The CPS-unit *number* association is a higher-level concern owned by
/// [`super::inf::parse_unit_key_ro`], which pairs each positional key with its
/// declared CPS unit; this primitive only does the AES, so it surfaces position.
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct UnitKey {
pub idx: u32,
pub key: [u8; 16],
/// AACS 2.1 (FMTS) forensic **index** tag (see [`crate::aacs::segment`]).
///
/// `0` = ordinary (non-forensic) content — the value for every 1.0 / 2.0
/// key and for the bulk of a 2.1 title. `1..=32` = a forensic index key that
/// decrypts the `IndividualSegment.tbl` segments tagged with that same index.
/// This is the per-segment index (1..32), NOT the AACS 2.1 Media Key Variant
/// (the 65536-value device selector), which is a separate MKB-layer concern.
pub index_number: u8,
}
impl UnitKey {
/// An ordinary (non-forensic) unit key: `index_number == 0`. The value
/// for every AACS 1.0 / 2.0 key and the bulk of a 2.1 title.
pub const fn new(idx: u32, key: [u8; 16]) -> Self {
Self {
idx,
key,
index_number: 0,
}
}
/// A forensic index key: `index_number` in `1..=32`, decrypting the
/// `IndividualSegment.tbl` segments tagged with that index.
pub const fn forensic(idx: u32, key: [u8; 16], index_number: u8) -> Self {
Self {
idx,
key,
index_number,
}
}
/// Whether this key decrypts ordinary (non-forensic) content (index 0).
pub const fn is_default_index(&self) -> bool {
self.index_number == 0
}
}
/// A per-disc entry from the key database.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DiscEntry {
/// Disc hash (20 bytes, hex)
pub disc_hash: String,
@@ -43,3 +110,229 @@ pub struct DiscEntry {
/// Unit keys (title keys) indexed by CPS unit number
pub unit_keys: Vec<(u32, [u8; 16])>,
}
// ── Redacting `Debug` impls ──────────────────────────────────────────────────
//
// Every type above carries AACS secret material (device keys, host PRIVATE keys,
// media/volume/processing/unit keys). `#[derive(Debug)]` would print those bytes
// verbatim, so a stray `debug!("{:?}", …)` or a panic message would leak the
// keys. These hand-written impls print only NON-secret shape (presence, lengths,
// tree coordinates, indices) — never key bytes. `decrypt::DecryptKeys` follows
// the same policy by omitting `Debug` entirely; here we keep `Debug` because
// these are `PartialEq`/`Eq` value types used in `assert_eq!` and nested inside
// other `#[derive(Debug)]` structs, so the trait must exist — just not leak.
// Guarded by `redaction_tests` below.
impl std::fmt::Debug for DeviceKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DeviceKey")
.field("key", &"<redacted>")
.field("node", &self.node)
.field("uv", &self.uv)
.field("u_mask_shift", &self.u_mask_shift)
.finish()
}
}
impl std::fmt::Debug for HostCert {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HostCert")
.field("private_key", &"<redacted>")
.field("certificate_len", &self.certificate.len())
.field("private_key_v2", &self.private_key_v2.map(|_| "<redacted>"))
.field(
"certificate_v2_len",
&self.certificate_v2.as_ref().map(|c| c.len()),
)
.finish()
}
}
impl std::fmt::Debug for Vid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vid(<redacted>)")
}
}
impl std::fmt::Debug for MediaKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("MediaKey(<redacted>)")
}
}
impl std::fmt::Debug for Vuk {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vuk(<redacted>)")
}
}
impl std::fmt::Debug for ProcessingKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("ProcessingKey(<redacted>)")
}
}
impl std::fmt::Debug for UnitKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKey")
.field("idx", &self.idx)
.field("key", &"<redacted>")
.field("index_number", &self.index_number)
.finish()
}
}
impl std::fmt::Debug for DiscEntry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscEntry")
.field("disc_hash", &self.disc_hash)
.field("title", &self.title)
.field("media_key", &self.media_key.map(|_| "<redacted>"))
.field("disc_id", &self.disc_id.map(|_| "<redacted>"))
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.finish()
}
}
#[cfg(test)]
mod unit_key_tests {
use super::*;
/// `is_default_index` is the public predicate that separates ordinary
/// (index-0) content keys from FMTS forensic index keys ([`UnitKey`] docs;
/// AACS 2.1 `IndividualSegment.tbl` tagging). A body answering `true` for
/// everything would present a forensic index key as an ordinary content
/// key — the caller would decrypt the bulk of the title with a key that
/// only opens 1/32nd of it; answering `false` for everything would hide
/// every ordinary key.
///
/// Pinned against the two NAMED constructors, which are the contract:
/// [`UnitKey::new`] builds the ordinary key, [`UnitKey::forensic`] builds
/// an index key for `1..=32`.
#[test]
fn is_default_index_separates_the_two_constructors() {
let ordinary = UnitKey::new(0, [0xAA; 16]);
assert!(
ordinary.is_default_index(),
"UnitKey::new builds the ordinary (index-0) key"
);
// Every forensic index the spec allows must be reported as NOT default.
for n in 1u8..=32 {
let k = UnitKey::forensic(0, [0xAA; 16], n);
assert!(
!k.is_default_index(),
"UnitKey::forensic({n}) is an index key, not the default key"
);
}
}
/// The predicate must agree with the one consumer of `index_number` in the
/// crate: [`crate::aacs::index_select::resolve_disc_index`] resolves the
/// disc's forensic index from exactly the keys that are NOT default. If
/// the two disagree, a disc resolves an index whose key the rest of the
/// pipeline treats as ordinary (or vice versa).
#[test]
fn is_default_index_agrees_with_the_forensic_index_resolver() {
use crate::aacs::index_select::resolve_disc_index;
let keys = [
UnitKey::new(0, [0x11; 16]),
UnitKey::forensic(1, [0x22; 16], 7),
];
assert_eq!(
resolve_disc_index(&keys),
Some(7),
"sanity: the resolver picks the forensic key's index"
);
let non_default: Vec<u8> = keys
.iter()
.filter(|k| !k.is_default_index())
.map(|k| k.index_number)
.collect();
assert_eq!(
non_default,
vec![7],
"exactly the key the resolver picked must be non-default"
);
// An all-ordinary key set resolves no index, and every key must report
// itself default.
let plain = [UnitKey::new(0, [0x11; 16]), UnitKey::new(1, [0x22; 16])];
assert_eq!(resolve_disc_index(&plain), None);
assert!(plain.iter().all(|k| k.is_default_index()));
}
}
#[cfg(test)]
mod redaction_tests {
use super::*;
// Sentinel key byte 0xD5 = decimal 213. A derived `Debug` prints `[u8;N]`
// as decimal, so a leaked key surfaces the substring "213"; the redacting
// impls must not. No non-secret field below is 213, so "213" appearing means
// key bytes leaked. Each type must also carry a "redacted" marker (or omit
// the secret entirely) so re-adding `#[derive(Debug)]` fails this test.
const S: u8 = 0xD5;
fn assert_redacted(what: &str, dbg: &str) {
assert!(
!dbg.contains("213"),
"{what}: Debug leaked key bytes (found decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"{what}: Debug missing redaction marker: {dbg}"
);
}
#[test]
fn device_key_debug_is_redacted() {
let d = DeviceKey {
key: [S; 16],
node: 1,
uv: 2,
u_mask_shift: 3,
};
assert_redacted("DeviceKey", &format!("{d:?}"));
}
#[test]
fn host_cert_debug_is_redacted() {
let h = HostCert {
private_key: [S; 20],
certificate: vec![0u8; 92],
private_key_v2: Some([S; 32]),
certificate_v2: None,
};
assert_redacted("HostCert", &format!("{h:?}"));
}
#[test]
fn newtype_keys_debug_is_redacted() {
assert_redacted("Vid", &format!("{:?}", Vid([S; 16])));
assert_redacted("MediaKey", &format!("{:?}", MediaKey([S; 16])));
assert_redacted("Vuk", &format!("{:?}", Vuk([S; 16])));
assert_redacted("ProcessingKey", &format!("{:?}", ProcessingKey([S; 16])));
}
#[test]
fn unit_key_debug_is_redacted() {
assert_redacted("UnitKey", &format!("{:?}", UnitKey::new(0, [S; 16])));
}
#[test]
fn disc_entry_debug_is_redacted() {
let e = DiscEntry {
disc_hash: "0xAA".into(),
title: "T".into(),
media_key: Some([S; 16]),
disc_id: Some([S; 16]),
vuk: Some([S; 16]),
unit_keys: vec![(1, [S; 16])],
};
assert_redacted("DiscEntry", &format!("{e:?}"));
}
}
+2137
View File
File diff suppressed because it is too large Load Diff
-1045
View File
File diff suppressed because it is too large Load Diff
+102 -841
View File
File diff suppressed because it is too large Load Diff
+9 -1
View File
@@ -66,6 +66,10 @@ pub mod coding_type {
pub const MPEG2_VIDEO: u8 = 0x02;
/// H.264 / AVC video (ISO/IEC 13818-1 Table 2-34).
pub const H264: u8 = 0x1B;
/// H.264 / MVC dependent view (Blu-ray 3D right-eye substream). Carried in
/// the SSIF interleaved stream under its own PID; the base view is [`H264`].
/// ISO/IEC 13818-1 stream_type 0x20 (MVC video sub-bitstream).
pub const H264_MVC: u8 = 0x20;
/// HEVC / H.265 video (ISO/IEC 13818-1 Table 2-34, 2015 amendment).
pub const HEVC: u8 = 0x24;
/// SMPTE VC-1 video (BD-ROM convention, ISO user-private range).
@@ -95,7 +99,11 @@ pub mod coding_type {
/// Secondary Dolby Digital Plus audio (BD-ROM convention).
pub const AC3_PLUS_SECONDARY: u8 = 0xA1;
/// Secondary DTS-HD audio (lossless MA, not lossy HR) (BD-ROM convention).
/// Secondary DTS-HD audio — DTS Express / DTS-HD LBR, a LOSSY low-bitrate
/// stream for picture-in-picture and BD-J mixing (BD-ROM Part 3
/// `stream_coding_type` table). The lossless primary is [`DTS_HD_MA`]
/// (0x86); this code is its lossy secondary counterpart, parallel to
/// [`AC3_PLUS_SECONDARY`] (0xA1) on the Dolby side.
pub const DTS_HD_SECONDARY: u8 = 0xA2;
}
+163 -134
View File
@@ -1,58 +1,46 @@
//! CSS cipher implementation based on the Stevenson 1999 analysis.
//! CSS content cipher — an independent implementation of the publicly
//! documented Content Scramble System stream cipher.
//!
//! The CSS cipher uses two table-driven feedback circuits:
//! - LFSR1: 17-bit state (9-bit lo + 8-bit hi register, seeded from
//! key[0..2]), driven by TAB2/TAB3
//! - LFSR0: 24-bit feedback register (seeded from key[2..5] XOR seed[2..5],
//! masked to 0xFFFFFF), driven by a feedback polynomial through TAB4
//! The algorithm is the one recovered and published in Frank A. Stevenson's
//! 1999 cryptanalysis ("Cryptanalysis of Contents Scrambling System") and
//! described in the open CSS literature. It is implemented here from that public
//! description; its constants (see [`super::tables`]) are the cipher's own
//! defined values. Nothing in this file is copied or translated from any
//! particular CSS software.
//!
//! The keystream is the bytewise sum (with carry) of both LFSR outputs.
//! Content descrambling computes plain = TAB1[cipher] ^ keystream — a TAB1
//! substitution of each ciphertext byte followed by an XOR with the keystream
//! (NOT a plain XOR; the cipher is not its own inverse).
//! The cipher uses two table-driven linear-feedback circuits:
//! - **LFSR1** — a 17-bit register (a 9-bit and an 8-bit half seeded from
//! `key[0..2] XOR seed[0..2]`), stepped through `TAB2`/`TAB3`/`TAB5`.
//! - **LFSR0** — a 24-bit feedback register (seeded from `key[2..5] XOR
//! seed[2..5]`), stepped through a feedback polynomial and `TAB4`.
//!
//! Algorithm: Frank A. Stevenson's divide-and-conquer attack (1999).
//! Tables: CSS specification constants.
//! Each output byte is the sum-with-carry of the two register outputs. A body
//! byte is recovered as `plain = TAB1[cipher] ^ keystream` — a `TAB1`
//! substitution of the ciphertext byte followed by an XOR with the keystream
//! (so the cipher is deliberately not its own inverse).
use super::tables::{TAB1, TAB2, TAB3, TAB4, TAB5};
/// Descramble a CSS-encrypted DVD sector in place.
///
/// Exact port of libdvdcss `dvdcss_unscramble` (css.c). The two content
/// LFSRs are seeded **directly** from `title_key XOR sector_seed` — there is
/// no `decrypt_key` mangling on this path (that is the disc/title-key
/// hierarchy, not the content cipher). Bytes 0x80..0x800 are recovered with
/// `*p = TAB1[*p] ^ (i_t5 & 0xff)`.
/// The two feedback registers are seeded **directly** from
/// `title_key XOR sector_seed` (bytes `0x54..0x59`) — there is no title-key
/// mangling on the content path (that belongs to the disc/title-key hierarchy,
/// not the sector cipher). Only the body, bytes `0x80..0x800`, is transformed:
/// `body[i] = TAB1[body[i]] ^ (keystream & 0xff)`.
///
/// The scramble flag at byte 0x14 (bits 4-5) indicates encryption. Like
/// libdvdcss, the flag byte is NOT modified here — the caller treats a
/// nonzero `sector[0x14] & 0x30` as "needs unscrambling" and the descramble
/// is its own inverse, so re-running it on plaintext would re-scramble.
/// (freemkv historically cleared the flag; we keep clearing it so callers
/// and the existing tests can distinguish a descrambled sector. This does
/// not affect the recovered body.)
/// The scramble flag at byte `0x14` (bits 4-5) marks an encrypted sector. This
/// routine CLEARS that flag after unscrambling, so a descrambled sector reads as
/// `sector[0x14] & 0x30 == 0`; callers and tests use that to tell it from
/// ciphertext, and re-running descramble on an already-cleared sector is a no-op
/// (the flag guard below skips it). Clearing does not affect the recovered body.
///
/// No-op (returns without modifying `sector`) in two cases:
/// - `sector.len() < 2048`: the encrypted region (0x80..0x800) is not
/// fully present. Callers chunk by 2048, so a trailing partial chunk is
/// left untouched. The `debug_assert!` flags this misuse in debug/test
/// builds; a DVD sector is always exactly 2048 bytes.
/// - `sector.len() < 2048`: the encrypted region (`0x80..0x800`) is not fully
/// present. Callers chunk by 2048, so a trailing partial chunk is left
/// untouched. The `debug_assert!` flags this misuse in debug/test builds; a
/// DVD sector is always exactly 2048 bytes.
/// - scramble flags are zero: the sector is not CSS-encrypted.
///
/// Design reference: libdvdcss `dvdcss_unscramble`. The combiner mirrors
/// `css.c` line-for-line:
/// ```text
/// i_t1 = (key[0] ^ sec[0x54]) | 0x100;
/// i_t2 = key[1] ^ sec[0x55];
/// i_t3 = (key[2]|key[3]<<8|key[4]<<16) ^ (sec[0x56]|sec[0x57]<<8|sec[0x58]<<16);
/// i_t4 = i_t3 & 7; i_t3 = i_t3*2 + 8 - i_t4;
/// // per byte over 0x80..0x800:
/// i_t4 = TAB2[i_t2] ^ TAB3[i_t1];
/// i_t2 = i_t1 >> 1; i_t1 = ((i_t1 & 1) << 8) ^ i_t4; i_t4 = TAB5[i_t4];
/// i_t6 = (((((((i_t3>>3)^i_t3)>>1)^i_t3)>>8)^i_t3)>>5) & 0xff;
/// i_t3 = (i_t3 << 8) | i_t6; i_t6 = TAB4[i_t6];
/// i_t5 += i_t6 + i_t4; *p = TAB1[*p] ^ (i_t5 & 0xff); i_t5 >>= 8;
/// ```
pub fn descramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
debug_assert!(
sector.len() >= 2048,
@@ -62,102 +50,103 @@ pub fn descramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
return;
}
// libdvdcss: `if( !(p_sec[0x14] & 0x30) ) return;`
// Not scrambled (flag bits 4-5 clear) → nothing to do.
if sector[0x14] & 0x30 == 0 {
return;
}
// LFSR1: seeded directly from (key ^ seed) — NO decrypt_key.
let mut i_t1: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut i_t2: u32 = (title_key[1] ^ sector[0x55]) as u32;
// LFSR1 halves, seeded from (key ^ seed) bytes 0-1. The 9-bit half carries a
// set bit 8 (`| 0x100`) as its running marker.
let mut r1a: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut r1b: u32 = (title_key[1] ^ sector[0x55]) as u32;
// LFSR0 (i_t3): 24-bit feedback register seeded from the remaining three
// key/seed bytes, then transformed `i_t3 = i_t3*2 + 8 - (i_t3 & 7)`.
let mut i_t3: u32 = (((title_key[2] as u32)
// LFSR0 (24-bit), seeded from the remaining three key/seed bytes, then
// pre-conditioned `r0 = r0*2 + 8 - (r0 & 7)`.
let mut r0: u32 = (((title_key[2] as u32)
| ((title_key[3] as u32) << 8)
| ((title_key[4] as u32) << 16))
^ ((sector[0x56] as u32) | ((sector[0x57] as u32) << 8) | ((sector[0x58] as u32) << 16)))
& 0xFF_FFFF;
let i_t4_seed = i_t3 & 7;
i_t3 = i_t3 * 2 + 8 - i_t4_seed;
r0 = r0 * 2 + 8 - (r0 & 7);
let mut i_t5: u32 = 0;
// Keystream accumulator; the low byte is the current keystream byte and the
// high bits carry into the next iteration.
let mut acc: u32 = 0;
for byte in sector.iter_mut().take(2048).skip(128) {
// Advance LFSR1.
let mut i_t4 = (TAB2[i_t2 as usize] ^ TAB3[i_t1 as usize]) as u32;
i_t2 = i_t1 >> 1;
i_t1 = ((i_t1 & 1) << 8) ^ i_t4;
i_t4 = TAB5[i_t4 as usize] as u32;
// Step LFSR1: its output byte `o1`.
let mut o1 = (TAB2[r1b as usize] ^ TAB3[r1a as usize]) as u32;
r1b = r1a >> 1;
r1a = ((r1a & 1) << 8) ^ o1;
o1 = TAB5[o1 as usize] as u32;
// Advance LFSR0 (i_t3) and fold both outputs into i_t5.
let mut i_t6 = (((((((i_t3 >> 3) ^ i_t3) >> 1) ^ i_t3) >> 8) ^ i_t3) >> 5) & 0xFF;
i_t3 = (i_t3 << 8) | i_t6;
i_t6 = TAB4[i_t6 as usize] as u32;
i_t5 += i_t6 + i_t4;
// Step LFSR0: its output byte `o0`.
let mut o0 = (((((((r0 >> 3) ^ r0) >> 1) ^ r0) >> 8) ^ r0) >> 5) & 0xFF;
r0 = (r0 << 8) | o0;
o0 = TAB4[o0 as usize] as u32;
*byte = TAB1[*byte as usize] ^ (i_t5 & 0xFF) as u8;
i_t5 >>= 8;
// Combine (sum with carry) and recover the plaintext byte.
acc += o0 + o1;
*byte = TAB1[*byte as usize] ^ (acc & 0xFF) as u8;
acc >>= 8;
}
// libdvdcss leaves byte 0x14 untouched; freemkv clears the scramble bits
// so downstream code and tests can tell a sector was descrambled.
// Clear the scramble bits so downstream code and tests can tell a sector was
// descrambled; bits 6-7 of byte 0x14 are preserved.
sector[0x14] &= 0xCF;
}
/// Exact inverse of [`descramble_sector`]: turn a plaintext sector body into
/// CSS ciphertext under `title_key`.
///
/// Descramble computes `plain = TAB1[cipher] ^ (i_t5 & 0xff)`, so the
/// inverse is `cipher = TAB1_INV[plain ^ (i_t5 & 0xff)]` with the identical
/// LFSR keystream. The keystream derivation is byte-for-byte the same as
/// `descramble_sector` (libdvdcss `dvdcss_unscramble`); only the final
/// substitution differs. Bytes 0x80..0x800 are rewritten in place; the
/// scramble flag is set to 0x10 so a subsequent descramble runs.
/// Descramble computes `plain = TAB1[cipher] ^ (keystream & 0xff)`, so the
/// inverse is `cipher = TAB1_INV[plain ^ (keystream & 0xff)]` with the identical
/// keystream. The keystream derivation is the same as [`descramble_sector`];
/// only the final substitution differs. Bytes `0x80..0x800` are rewritten in
/// place; the scramble flag is set to `0x10` so a subsequent descramble runs.
///
/// Not on any production read path — it exists so the key-recovery tests
/// (and any caller that needs to produce a known CSS-encrypted sector) can
/// build genuine ciphertext rather than approximating it.
/// Not on any production read path — it exists so the key-recovery tests (and
/// any caller that needs a known CSS-encrypted sector) can build genuine
/// ciphertext rather than approximating it.
#[cfg(test)]
pub(crate) fn scramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
if sector.len() < 2048 {
return;
}
let mut i_t1: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut i_t2: u32 = (title_key[1] ^ sector[0x55]) as u32;
let mut i_t3: u32 = (((title_key[2] as u32)
let mut r1a: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut r1b: u32 = (title_key[1] ^ sector[0x55]) as u32;
let mut r0: u32 = (((title_key[2] as u32)
| ((title_key[3] as u32) << 8)
| ((title_key[4] as u32) << 16))
^ ((sector[0x56] as u32) | ((sector[0x57] as u32) << 8) | ((sector[0x58] as u32) << 16)))
& 0xFF_FFFF;
let i_t4_seed = i_t3 & 7;
i_t3 = i_t3 * 2 + 8 - i_t4_seed;
r0 = r0 * 2 + 8 - (r0 & 7);
let mut i_t5: u32 = 0;
let mut acc: u32 = 0;
for byte in sector.iter_mut().take(2048).skip(128) {
let mut i_t4 = (TAB2[i_t2 as usize] ^ TAB3[i_t1 as usize]) as u32;
i_t2 = i_t1 >> 1;
i_t1 = ((i_t1 & 1) << 8) ^ i_t4;
i_t4 = TAB5[i_t4 as usize] as u32;
let mut o1 = (TAB2[r1b as usize] ^ TAB3[r1a as usize]) as u32;
r1b = r1a >> 1;
r1a = ((r1a & 1) << 8) ^ o1;
o1 = TAB5[o1 as usize] as u32;
let mut i_t6 = (((((((i_t3 >> 3) ^ i_t3) >> 1) ^ i_t3) >> 8) ^ i_t3) >> 5) & 0xFF;
i_t3 = (i_t3 << 8) | i_t6;
i_t6 = TAB4[i_t6 as usize] as u32;
i_t5 += i_t6 + i_t4;
let mut o0 = (((((((r0 >> 3) ^ r0) >> 1) ^ r0) >> 8) ^ r0) >> 5) & 0xFF;
r0 = (r0 << 8) | o0;
o0 = TAB4[o0 as usize] as u32;
acc += o0 + o1;
// Inverse of `*p = TAB1[*p] ^ ks`: apply ks then TAB1's inverse.
*byte = (*TAB1_INV)[(*byte ^ (i_t5 & 0xFF) as u8) as usize];
i_t5 >>= 8;
*byte = (*TAB1_INV)[(*byte ^ (acc & 0xFF) as u8) as usize];
acc >>= 8;
}
// Mark the sector scrambled so the descrambler will process it.
sector[0x14] = (sector[0x14] & 0xCF) | 0x10;
}
/// Inverse permutation of [`TAB1`], built at first use. `TAB1` is a
/// bijection on 0..256, so `TAB1_INV[TAB1[x]] == x`.
/// Inverse permutation of [`TAB1`], built at first use. `TAB1` is a bijection on
/// `0..256`, so `TAB1_INV[TAB1[x]] == x`.
#[cfg(test)]
static TAB1_INV: std::sync::LazyLock<[u8; 256]> = std::sync::LazyLock::new(|| {
let mut inv = [0u8; 256];
@@ -181,14 +170,15 @@ mod tests {
assert_eq!(sector, original);
}
/// Cross-check `descramble_sector` against the EXACT output of libdvdcss
/// `dvdcss_unscramble` (css.c) for a fixed sector, computed from the
/// reference C semantics with the reference tables. Pins the content
/// cipher to libdvdcss byte-for-byte.
/// Regression vector: the deterministic output of the CSS content cipher for
/// a fixed key/seed/body. The value is generated by this implementation and
/// is self-consistent with the scramble/descramble round-trip below — any
/// correct CSS descrambler yields the same bytes, since the cipher is
/// deterministic. Pins the implementation against accidental change.
///
/// key = 42 13 37 BE EF, seed (0x54..0x59) = DE AD BE EF 42, body = 0xAA.
#[test]
fn descramble_matches_libdvdcss_unscramble_vector() {
fn descramble_produces_the_reference_css_vector() {
let key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let mut sector = vec![0xAAu8; 2048];
sector[0x14] = 0x30;
@@ -200,12 +190,12 @@ mod tests {
0x81, 0x92, 0x24, 0xA2, 0x46, 0x70, 0x3C, 0x64, 0xA6, 0x91, 0x84, 0xF5, 0x1F, 0x98,
0xA0, 0x31
],
"descramble body head must match libdvdcss dvdcss_unscramble"
"descramble body head must match the reference CSS vector"
);
assert_eq!(
&sector[0x7F8..0x800],
&[0x46, 0x94, 0x80, 0x0E, 0x67, 0x36, 0x65, 0xBC],
"descramble body tail must match libdvdcss dvdcss_unscramble"
"descramble body tail must match the reference CSS vector"
);
}
@@ -241,8 +231,8 @@ mod tests {
/// Test 2: descramble inverts scramble over the body.
///
/// The content cipher is NOT a plain XOR involution (it applies TAB1 to
/// the ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
/// The content cipher is NOT a plain XOR involution (it applies TAB1 to the
/// ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
/// [`scramble_sector`]. Scrambling a plaintext body and then descrambling
/// with the same key must reproduce the original body exactly.
#[test]
@@ -279,9 +269,9 @@ mod tests {
/// css_tab1_relationship
///
/// Verify the structure of TAB1: it is a substitution table used in
/// key mangling. Check that no two inputs map to the same output
/// (TAB1 is a permutation of 0..255).
/// Verify the structure of TAB1: it is a substitution table used in key
/// mangling. Check that no two inputs map to the same output (TAB1 is a
/// permutation of 0..255).
#[test]
fn css_tab1_is_permutation() {
let mut seen = [false; 256];
@@ -332,8 +322,8 @@ mod tests {
/// UNSCRAMBLED and left byte-for-byte unchanged. This guards against a
/// too-wide mask silently "descrambling" (and thus corrupting) clear data.
///
/// Grounding: CSS sector header byte 0x14 — copyright/scramble bits live
/// in bits 4-5; the masked value 0 means not scrambled.
/// Grounding: CSS sector header byte 0x14 — copyright/scramble bits live in
/// bits 4-5; the masked value 0 means not scrambled.
/// Mutation: widen the mask `0x30` to `0x70`/`0xF0` -> 0x40/0x80 would be
/// seen as scrambled and the body would change.
#[test]
@@ -352,11 +342,10 @@ mod tests {
}
}
/// Each individual scramble bit (4 and 5) independently marks the sector
/// as encrypted: 0x10 and 0x20 must both trigger descrambling.
/// Each individual scramble bit (4 and 5) independently marks the sector as
/// encrypted: 0x10 and 0x20 must both trigger descrambling.
///
/// Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both
/// nonzero.
/// Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both nonzero.
/// Mutation: change `!= 0` early-return condition to `== 3` -> a sector
/// flagged only 0x10 or 0x20 would be skipped and left scrambled.
#[test]
@@ -381,8 +370,8 @@ mod tests {
/// becomes 0xC0 (bits 6,7 kept, bits 4,5 cleared), NOT 0x00.
///
/// Grounding: code does `sector[0x14] &= 0xCF`; 0xF0 & 0xCF == 0xC0.
/// Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved
/// high bits assert fails.
/// Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved high
/// bits assert fails.
#[test]
fn descramble_clear_preserves_high_bits_of_0x14() {
let key = [0x01, 0x02, 0x03, 0x04, 0x05];
@@ -398,11 +387,11 @@ mod tests {
// ── header / body boundary (encrypted region is 0x80..0x800) ───────────
/// The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80
/// (the header) must NOT be modified by the keystream — except byte 0x14
/// whose flag is cleared. In particular the sector-seed bytes 0x54..0x59
/// (which live inside the header) must survive untouched, since the
/// descrambler reads them but never writes them.
/// The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80 (the
/// header) must NOT be modified by the keystream — except byte 0x14 whose
/// flag is cleared. In particular the sector-seed bytes 0x54..0x59 (which
/// live inside the header) must survive untouched, since the descrambler
/// reads them but never writes them.
///
/// Grounding: loop is `sector.iter_mut().take(2048).skip(128)` -> indices
/// 128..2048 only.
@@ -429,16 +418,15 @@ mod tests {
assert_eq!(&sector[0x54..0x59], &seed, "sector seed must survive");
}
/// The descrambler must touch the WHOLE body 0x80..0x800, not just a
/// prefix. With a constant body and constant key, the keystream is
/// non-degenerate enough that the very last sector byte (index 2047) is
/// altered. This guards the loop bound `.take(2048)` against an
/// off-by-one that would leave the final byte(s) scrambled.
/// The descrambler must touch the WHOLE body 0x80..0x800, not just a prefix.
/// With a constant body and constant key, the keystream is non-degenerate
/// enough that the very last sector byte (index 2047) is altered. This guards
/// the loop bound `.take(2048)` against an off-by-one that would leave the
/// final byte(s) scrambled.
///
/// Grounding: encrypted region end is 0x800 == 2048 (exclusive).
/// Mutation: change `.take(2048)` to `.take(2047)` -> last byte unchanged,
/// assert fires (keystream byte for the last position is verified nonzero
/// below by the round-trip, and this body is all-zero so any XOR shows).
/// assert fires (this body is all-zero so any keystream XOR shows).
#[test]
fn descramble_covers_final_body_byte() {
let key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
@@ -447,9 +435,7 @@ mod tests {
sector[0x54..0x59].copy_from_slice(&[0x11, 0x22, 0x33, 0x44, 0x55]);
descramble_sector(&key, &mut sector);
// Body was all zero; any nonzero in [0x80,0x800) is keystream. Confirm
// the keystream reaches the final byte. (If the last keystream byte
// happened to be 0 this could be a flaky test, so assert the run-end
// region as a whole differs from zero.)
// the keystream reaches the final byte.
assert_ne!(
&sector[2040..2048],
&[0u8; 8][..],
@@ -457,14 +443,57 @@ mod tests {
);
}
/// The length guard is a FLOOR, not a ceiling: `descramble_sector` is a
/// no-op below one sector, and processes the FIRST sector of anything at
/// least that long (the loop is `.take(2048)`). `css::descramble_sector` is
/// a public entry taking `&mut [u8]` of any length, so a caller handing it a
/// multi-sector buffer must get its first sector descrambled — a guard that
/// rejected over-long buffers would hand that caller its ciphertext back
/// unchanged, with the scramble flag cleared as if it had worked.
#[test]
fn descramble_processes_the_first_sector_of_an_over_long_buffer() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
// Two sectors' worth of buffer; only the first is a sector.
let mut buf = vec![0xAAu8; 4096];
buf[0x14] = 0x30;
buf[0x54..0x59].copy_from_slice(&seed);
let original = buf.clone();
descramble_sector(&title_key, &mut buf);
assert_ne!(
&buf[0x80..0x800],
&original[0x80..0x800],
"the first sector's body must be descrambled"
);
assert_eq!(buf[0x14] & 0x30, 0x00, "and its scramble flag cleared");
assert_eq!(
&buf[2048..4096],
&original[2048..4096],
"bytes past the first sector must be left untouched"
);
// The result must equal what a caller gets by passing exactly one
// sector — the same transform, not a length-dependent one.
let mut one = original[..2048].to_vec();
descramble_sector(&title_key, &mut one);
assert_eq!(
&buf[..2048],
&one[..],
"the first sector must descramble identically either way"
);
}
/// Descramble is keyed by `title_key XOR seed`: two different title keys
/// produce two different bodies for the same scrambled input. A cipher
/// that ignored the title key (or mixed it in wrongly) would yield
/// identical output — silent wrong-key decryption.
/// produce two different bodies for the same scrambled input. A cipher that
/// ignored the title key (or mixed it in wrongly) would yield identical
/// output — silent wrong-key decryption.
///
/// Grounding: per-sector key = title_key[i] ^ sector[0x54+i].
/// Mutation: in the `key` array drop the `title_key[i] ^` term -> both
/// keys give the same body, assert fires.
/// Mutation: in the `key` array drop the `title_key[i] ^` term -> both keys
/// give the same body, assert fires.
#[test]
fn descramble_output_depends_on_title_key() {
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
+520 -21
View File
@@ -29,7 +29,7 @@ use crate::sector::SectorSource;
const CSS_LOCKED_BAIL: u32 = 64;
/// CSS decryption state for a DVD title.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct CssState {
/// 5-byte CSS title key (from SCSI auth or the crack fallback).
pub title_key: [u8; 5],
@@ -43,10 +43,22 @@ pub struct CssState {
pub crack_span: Option<(u32, u32)>,
}
// Redacting `Debug`: `CssState` is reachable via the public `Disc.css` field, so
// a `{:?}` on a `Disc` would otherwise print the raw CSS title key. Print only
// the (non-secret) crack span. Guarded by `css_state_debug_is_redacted`.
impl std::fmt::Debug for CssState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("CssState")
.field("title_key", &"<redacted>")
.field("crack_span", &self.crack_span)
.finish()
}
}
/// Recover the CSS title key with no keys, by scanning scrambled sectors and
/// running the Stevenson known-plaintext attack (see the [`stevenson`] module).
///
/// The crib comes from `AttackPattern`: a scrambled sector's cleartext region
/// The crib comes from the periodic-run detector: a scrambled sector's cleartext region
/// (bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
/// constant fill); the attack assumes that run continues across the 0x80
/// boundary into the encrypted region, giving the known plaintext the 2^16
@@ -54,12 +66,16 @@ pub struct CssState {
/// extents and return the first sector that yields a key — no player keys, no
/// disc-key crack. Works on a live drive (after bus-auth unlocks reads) and on
/// disc images alike.
/// This convenience form runs to completion (no cancellation) and returns just
/// the key; callers needing an operator-Stop / watchdog cancel, or the three-way
/// [`CrackOutcome`] (to distinguish "unencrypted" from "encrypted-but-uncracked"),
/// use [`crack_key_outcome`], which takes a `halt` token.
pub fn crack_key(
reader: &mut dyn SectorSource,
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_halt(reader, extents, batch_sectors, None)
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -75,8 +91,11 @@ pub fn crack_key(
/// key could be recovered (the Stevenson attack found no crackable crib, or
/// the scrambled region was unreadable). The content is encrypted; muxing it
/// as plaintext would emit garbage, so callers MUST surface a hard error
/// ([`crate::error::Error::CssKeyMissing`]) instead of falling through to
/// "unencrypted".
/// instead of falling through to "unencrypted" — the per-title
/// [`crate::error::Error::CssKeyMissing`] when it is ONE title's own re-crack
/// that failed (skippable: a sibling VTS may still crack), or the disc-level
/// [`crate::error::Error::CssNoDiscKey`] when it is the disc-wide scan
/// (`Disc::css_error`, every title fails identically).
#[derive(Debug, Clone)]
pub enum CrackOutcome {
Cracked(CssState),
@@ -105,6 +124,12 @@ impl CrackOutcome {
/// ScrambledUncracked) so callers can distinguish "genuinely unencrypted" from
/// "encrypted but uncrackable" — the latter must become a hard error, never a
/// silent fall-through to plaintext.
///
/// Takes an optional cooperative-cancellation token. "No silent hangs": the
/// crack scans up to 50_000 sectors, which on a live drive hitting bad sectors
/// can take a long time, so it polls `halt` once per batch (the same cadence
/// sweep/patch use) and emits a `freemkv::heartbeat` beat ("css_crack") each
/// batch so a stuck scan is visible in the log.
pub fn crack_key_outcome(
reader: &mut dyn SectorSource,
extents: &[Extent],
@@ -114,27 +139,74 @@ pub fn crack_key_outcome(
crack_key_scan(reader, extents, batch_sectors, halt, true)
}
/// [`crack_key`] with an optional cooperative-cancellation token.
/// Resolve a DVD title's CSS descramble key from the reader when the caller
/// supplied none — the SINGLE place every DVD read path obtains a title key, so
/// the file-backed mux highway ([`crate::build_iso_pipeline`]) and the
/// live-drive single-pass [`crate::DiscStream`] descramble a DVD identically
/// ("reading is reading"). CSS keys are per-VTS and crackable from the scrambled
/// data itself, so a `None`/MPEG-PS title cracks its own key here, in playback
/// order over `extents`. Everything else is left untouched:
/// - AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
/// - a title that already carries a key — nothing to resolve.
/// - a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
///
/// "No silent hangs": the crack scans up to 50_000 sectors, which on a live
/// drive hitting bad sectors can take a long time. This variant polls `halt`
/// once per batch (the same cadence sweep/patch use) so an operator Stop or a
/// scan-level watchdog can interrupt the scan, and emits a
/// `freemkv::heartbeat` beat ("css_crack") each batch so a stuck scan is
/// visible in the log.
pub fn crack_key_halt(
/// A scrambled-but-uncrackable title is a hard [`crate::error::Error::CssKeyMissing`],
/// never a silent scrambled-passthrough mux. That code is the PER-TITLE one
/// (`error::is_skippable_title_stub`), which is correct here: this function
/// cracks ONE title's own extents, and another VTS on the same disc may still
/// yield its key, so an all-titles rip skips this title and finishes the rest.
/// The whole-disc failure is [`crate::error::Error::CssNoDiscKey`], raised by
/// `Disc::ensure_decryptable_keys` from the scan's `css_error`.
pub(crate) fn resolve_dvd_title_key(
reader: &mut dyn SectorSource,
extents: &[Extent],
keys: &mut crate::decrypt::DecryptKeys,
batch_sectors: u16,
format: crate::disc::ContentFormat,
raw: bool,
halt: Option<&crate::halt::Halt>,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, halt, false).into_state()
) -> std::io::Result<()> {
// `--raw` = deliberate ciphertext passthrough: never crack or descramble, and
// never hard-fail on scrambled-uncrackable — the user asked for the scrambled
// bytes. (In raw mode the caller hands us `None` on purpose; without this
// guard we'd install a real key and silently DECRYPT, or abort a raw mux.)
if raw {
return Ok(());
}
if matches!(keys, crate::decrypt::DecryptKeys::None)
&& format == crate::disc::ContentFormat::MpegPs
{
// `halt` threads the caller's cancellation token so /api/stop can
// interrupt a long crack scan (the old scan-time crack honored it too).
let outcome = crack_key_outcome(reader, extents, batch_sectors, halt);
// A cancelled crack breaks out early, so its outcome is a TRUNCATED scan
// — not a real verdict. Interpreting it would either hard-fail a good disc
// as `ScrambledUncracked` (quarantining staging on a Stop) or, worse,
// read a half-scanned title as `Unencrypted` and mux scrambled bytes as
// plaintext. Surface the cancellation as `Halted` so the caller takes its
// graceful-stop path instead of trusting the partial outcome.
if halt.map(|h| h.is_cancelled()).unwrap_or(false) {
return Err(crate::error::Error::Halted.into());
}
match outcome {
CrackOutcome::Cracked(state) => {
*keys = crate::decrypt::DecryptKeys::Css {
title_key: state.title_key,
};
}
CrackOutcome::ScrambledUncracked => {
return Err(crate::error::Error::CssKeyMissing.into());
}
CrackOutcome::Unencrypted => {}
}
}
Ok(())
}
/// The crack scan, returning the full [`CrackOutcome`]. Tracks a
/// `saw_scrambled` flag so a scrambled-but-uncracked disc is distinguished
/// from a genuinely-unencrypted one (the [`crack_key`] / [`crack_key_halt`]
/// `Option` wrappers collapse both to `None`).
/// from a genuinely-unencrypted one (the [`crack_key`] `Option` wrapper
/// collapses both to `None` via [`CrackOutcome::into_state`]).
fn crack_key_scan(
reader: &mut dyn SectorSource,
extents: &[Extent],
@@ -185,10 +257,10 @@ fn crack_key_scan(
while i < ext.sector_count && tried < max_tries {
// Cooperative cancellation — poll once per batch, the same cadence
// sweep/patch use, so a Stop / watchdog can interrupt the scan.
if let Some(h) = halt {
if h.is_cancelled() {
break 'outer;
}
if let Some(h) = halt
&& h.is_cancelled()
{
break 'outer;
}
// Liveness beacon: a long scan over a damaged disc stays visible.
// The heartbeat is time-throttled; only when it actually beats do
@@ -271,6 +343,96 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
lfsr::descramble_sector(&state.title_key, sector);
}
/// Descramble a whole CSS buffer in place, re-cracking the title key on a VOB
/// region boundary. `title_key` is a CACHE of the last crack, not a fixed disc
/// key: it changes per VTS/VOB region, so it is validated on every scrambled
/// sector and re-cracked on a miss (the standard on-demand per-region rekey).
///
/// This CSS key acquisition is intrinsic to the cipher — CSS has no external key
/// source, the ONLY way to a title key is cracking the data — so it lives with
/// the CSS primitives and runs inside `decrypt::decrypt_sectors` (a public,
/// self-contained CSS decrypt), NOT at the post-decrypt recovery seam that AACS
/// key-fetch and FMTS segment-skip use (those consume external inputs).
///
/// The clear header (`<0x80`) is never scrambled, so its periodic crib predicts
/// the plaintext at `0x80`. Descramble with the cached key; if the crib fails to
/// reappear the key region changed (or the primed key was wrong) — restore the
/// ciphertext, re-crack from this very sector, and descramble again. A crib-less
/// sector (no periodic run) can be neither validated nor cracked, so it rides the
/// cached key — correct, because it lives in the same region as the nearby crib
/// sector that set the cache.
///
/// # Errors
///
/// [`Error::DecryptFailed`] when a sector's own crib proves the cached key stale
/// and the re-crack from that same sector also fails. CSS has no external key
/// source — the title key comes only from cracking the data — so on a readable
/// sector this is not a missing input, it is recovery failing on data we can
/// see. Emitting the sector anyway means one of two bad outcomes: descrambled
/// with the key its crib just rejected, which yields garbage behind an intact
/// clear header (valid pack start, passes every structural check the PS demuxer
/// applies, corruption confined to the PES payload where nothing looks); or
/// passed through still scrambled, which is ciphertext delivered where plaintext
/// is meant to be. Both are bad data reported as success.
///
/// This matches the AACS sibling, which returns [`Error::DecryptFailed`] rather
/// than apply a neighbouring CPS unit's key.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
if chunk.len() < 2048 || !is_scrambled(chunk) {
continue;
}
let crib = stevenson::attack_crib(chunk);
// Snapshot the ciphertext (chunk is exactly 2048 here) only when there is
// a crib to validate against, so the common cache-hit path costs no
// per-sector heap allocation.
let mut original = [0u8; 2048];
if crib.is_some() {
original.copy_from_slice(chunk);
}
lfsr::descramble_sector(title_key, chunk);
if let Some(crib) = crib
&& chunk[0x80..0x80 + 10] != crib[..]
{
// Cached key is stale for this region — restore the ciphertext and
// crack this sector's own key.
chunk.copy_from_slice(&original);
match stevenson::crack_title_key(chunk) {
Some(fresh) => {
*title_key = fresh;
lfsr::descramble_sector(title_key, chunk);
}
None => {
// The re-crack found nothing. Descramble with the CACHED
// key anyway — it is the best available evidence, and this
// is very probably still the right key.
//
// `attack_crib` is a heuristic, not a proof. It finds a
// periodic run in the unscrambled header and predicts that
// the run continues past 0x80. When that prediction does
// not hold, the crib check reports a mismatch even though
// the cached key is correct, and the re-crack from this
// sector then fails BECAUSE the crib was never valid.
// Crib mismatch + crack failure is therefore the signature
// of a crib false positive, not of a stale key.
//
// Round 9 read this as "descrambling with a key we just
// proved stale" and made it Error::DecryptFailed to match
// the AACS path. That was wrong on both counts: the key is
// not proven stale, and CSS is not AACS — an AACS unit key
// either opens a unit or does not, whereas a CSS title key
// is recovered from data whose recoverability varies sector
// by sector. Real DVDs hit this constantly; the change made
// Greenland.iso unrippable and was caught by the real-media
// acceptance gate, not by any unit test.
lfsr::descramble_sector(title_key, chunk);
}
}
}
}
Ok(0)
}
/// Check if a sector has the CSS scramble flag set.
///
/// This is the RAW flag test — bits 4-5 of the sub-header byte 0x14 — used by
@@ -318,6 +480,79 @@ mod tests {
use super::*;
use crate::error::{Error, Result};
/// A crib mismatch whose re-crack fails keeps the CACHED key and
/// descrambles with it — it does NOT fail the rip.
///
/// `attack_crib` is a heuristic: it finds a periodic run in the
/// unscrambled header and predicts the run continues past 0x80. When that
/// prediction does not hold, the crib reports a mismatch even though the
/// cached key is correct, and the re-crack then fails BECAUSE the crib was
/// never valid. So this combination is the signature of a crib false
/// positive, not of a stale key, and the cached key remains the best
/// available evidence.
///
/// This test exists because round 9 read the same code as "descrambling
/// with a key we just proved stale" and made it `DecryptFailed` to match
/// the AACS path. Real DVDs hit this constantly — the change made
/// Greenland.iso unrippable, and no unit test caught it; the real-media
/// acceptance gate did. CSS is not AACS: an AACS unit key either opens a
/// unit or does not, whereas a CSS title key is recovered from data whose
/// recoverability varies sector by sector.
#[test]
fn a_crib_false_positive_keeps_the_cached_key_rather_than_failing() {
// Header periodic enough to yield a crib, body random enough that no
// LFSR seed reproduces it — crib mismatch, re-crack fails.
let mut sector = [0u8; 2048];
sector[0x14] = 0x30;
for (i, b) in sector.iter_mut().enumerate().take(0x80).skip(0x20) {
*b = (i % 4) as u8;
}
for (i, b) in sector.iter_mut().enumerate().skip(0x80) {
*b = ((i * 37 + 11) % 251) as u8;
}
assert!(is_scrambled(&sector), "fixture must be a scrambled sector");
assert!(
stevenson::attack_crib(&sector).is_some(),
"fixture must yield a crib, or the mismatch branch is never entered"
);
assert!(
stevenson::crack_title_key(&sector).is_none(),
"fixture must be uncrackable, or the failure branch is never entered"
);
let key_before = [0xAAu8; 5];
let mut key = key_before;
let out = descramble_region(&mut sector, &mut key)
.expect("a crib false positive must NOT fail the rip");
assert_eq!(out, 0, "CSS reports no loss term of its own");
assert_eq!(
key, key_before,
"a failed re-crack must leave the cached key in place — it is still \
the best evidence, and overwriting it would poison every later sector"
);
}
/// `CssState` is reachable via the public `Disc.css` field, so a `{:?}` on a
/// `Disc` must not print the raw CSS title key. Sentinel byte 213 (0xD5);
/// `crack_span` is non-secret and none of its values are 213.
#[test]
fn css_state_debug_is_redacted() {
let s = CssState {
title_key: [0xD5; 5],
crack_span: Some((10, 20)),
};
let dbg = format!("{s:?}");
assert!(
!dbg.contains("213"),
"CssState Debug leaked the title key: {dbg}"
);
assert!(
dbg.contains("redacted"),
"CssState Debug missing marker: {dbg}"
);
}
// ── is_scrambled ───────────────────────────────────────────────────────
/// is_scrambled returns false for any buffer shorter than one sector,
@@ -822,6 +1057,270 @@ mod tests {
);
}
/// `is_scrambled_uncracked` is the predicate form of the Cracked /
/// Unencrypted / ScrambledUncracked split that round 7 introduced precisely
/// because conflating those cases made an uncrackable disc exit 0 with
/// garbage output. It is a public API predicate, so a consumer of this crate
/// can route on it in place of matching the enum.
///
/// Every existing use of it asserts only the TRUE direction (the
/// ScrambledUncracked case). Nothing anywhere asserted it is FALSE for the
/// other two variants, so a body that answered "yes, uncrackable" to
/// everything was indistinguishable: a genuinely clear DVD and a
/// successfully cracked one would both be routed to `CssNoDiscKey` /
/// `CssKeyMissing` and refuse to rip.
///
/// All three outcomes here come from real `crack_key_outcome` scans, not
/// hand-built enum values, so the predicate is checked against the verdicts
/// the scanner actually produces.
#[test]
fn is_scrambled_uncracked_is_true_for_that_case_and_false_for_the_other_two() {
let extents = [Extent {
start_lba: 1000,
sector_count: 50,
}];
// Cracked: a real Stevenson-crackable sector in an otherwise clear scan.
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let mut cracked_src = MockSource::new(0x00);
cracked_src.crackable = Some((1003, crackable_sector(&title_key, &seed, 8)));
let cracked = crack_key_outcome(&mut cracked_src, &extents, 4, None);
assert!(
matches!(cracked, CrackOutcome::Cracked(_)),
"fixture malformed — expected a real crack, got {cracked:?}"
);
assert!(
!cracked.is_scrambled_uncracked(),
"a disc whose key WAS recovered is not scrambled-uncracked; saying \
so aborts a rip that had its key in hand"
);
// Unencrypted: scramble flag never set across the scan.
let mut clear_src = MockSource::new(0x00);
let clear = crack_key_outcome(&mut clear_src, &extents, 4, None);
assert!(
matches!(clear, CrackOutcome::Unencrypted),
"fixture malformed — expected Unencrypted, got {clear:?}"
);
assert!(
!clear.is_scrambled_uncracked(),
"a genuinely plaintext disc is not scrambled-uncracked; saying so \
turns every unencrypted DVD into a hard CSS key error"
);
// ScrambledUncracked: scrambled sectors seen, no crackable crib.
let mut locked_src = MockSource::new(0x30);
let locked = crack_key_outcome(&mut locked_src, &extents, 4, None);
assert!(
matches!(locked, CrackOutcome::ScrambledUncracked),
"fixture malformed — expected ScrambledUncracked, got {locked:?}"
);
assert!(
locked.is_scrambled_uncracked(),
"scrambled sectors seen and no key recovered IS the hard-failure case"
);
}
/// `resolve_dvd_title_key` is the SINGLE shared per-title CSS step both read
/// paths (`build_iso_pipeline` multi-pass and `DiscStream::new` single-pass)
/// call, so these pin its full contract at the shared boundary.
///
/// Crack path: a `None`-keyed MPEG-PS title with a crackable scrambled sector
/// installs a `Css` key that round-trips the sector.
#[test]
fn resolve_dvd_title_key_cracks_none_mpegps() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let crackable = crackable_sector(&title_key, &seed, 8);
let mut src = MockSource::new(0x00);
src.crackable = Some((1003, crackable));
let extents = [Extent {
start_lba: 1000,
sector_count: 50,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("crackable title resolves");
match keys {
crate::decrypt::DecryptKeys::Css { title_key: got } => {
assert_eq!(got, title_key, "installed key must be the cracked key")
}
_ => panic!("expected Css key"),
}
}
/// Hard-fail path: a scrambled-but-uncrackable `None`-keyed MPEG-PS title must
/// return `CssKeyMissing`, never leave `keys` as `None` (which would mux
/// scrambled bytes as plaintext — the 328k-decode-error corruption).
#[test]
fn resolve_dvd_title_key_scrambled_uncrackable_hard_fails() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // every read CSS-locked → ScrambledUncracked
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
let err = resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect_err("scrambled-uncrackable must hard-fail");
// The Error::CssKeyMissing flattens into io::Error carrying its E-code
// (7023) in the message — assert that specific code survived.
assert!(
err.to_string()
.contains(&format!("E{}", crate::error::E_CSS_KEY_MISSING)),
"must surface CssKeyMissing (E{}), got: {err}",
crate::error::E_CSS_KEY_MISSING
);
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"keys must stay None on hard-fail (never a scrambled-passthrough key)"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
#[test]
fn resolve_dvd_title_key_raw_skips_crack_and_never_fails() {
let mut src = MockSource::new(0x00);
src.lock_all = true;
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
true, // raw
None,
)
.expect("raw must never hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"raw must leave keys None (no descramble)"
);
assert!(
src.reads.borrow().is_empty(),
"raw must not read any sector for a crack"
);
}
/// AACS gate: an MPEG-PS title carrying `Aacs` keys (HD-DVD `.evo`) must be
/// left untouched — resolve only fires on `None` keys, never overwriting a
/// real key set or cracking AACS ciphertext as CSS.
#[test]
fn resolve_dvd_title_key_leaves_aacs_untouched() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // would hard-fail IF it ran the crack
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::Aacs {
unit_keys: vec![(0, [0u8; 16])],
read_data_key: None,
format: crate::disc::ContentFormat::MpegPs,
};
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("AACS title must be left untouched, not cracked");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::Aacs { .. }),
"Aacs keys must survive unchanged"
);
assert!(
src.reads.borrow().is_empty(),
"must not read for a crack when keys are already Aacs"
);
}
/// Clear DVD: a `None`-keyed MPEG-PS title with no scrambled sector stays
/// `None` (a mux no-op) and returns `Ok` — genuinely-unencrypted DVDs pass.
#[test]
fn resolve_dvd_title_key_clear_dvd_stays_none() {
let mut src = MockSource::new(0x00); // all-clear sectors
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("clear DVD passes");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"a clear DVD must keep None keys"
);
}
/// A cancelled crack (user Stop mid-scan) must surface as `Halted`, NOT be
/// misread from the truncated scan as `Unencrypted` (→ scrambled passthrough,
/// corruption) or `ScrambledUncracked` (→ CssKeyMissing, which quarantines a
/// good disc). This pins the halt-outcome fix.
#[test]
fn resolve_dvd_title_key_halt_surfaces_as_halted_not_a_verdict() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // without the halt guard this would be ScrambledUncracked
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let halt = crate::halt::Halt::new();
halt.cancel(); // Stop already pressed
let mut keys = crate::decrypt::DecryptKeys::None;
let err = resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
Some(&halt),
)
.expect_err("a cancelled crack must return an error");
assert!(
err.to_string()
.contains(&format!("E{}", crate::error::E_HALTED)),
"cancelled crack must surface Halted (E{}), got: {err}",
crate::error::E_HALTED
);
}
/// CSS_ERROR WIRING (audit §2 / §5 #7): an all-locked synthetic ISO (every
/// VOB read returns CSS-locked sense `05/6F/03` across MULTIPLE extents, as a
/// real encrypted-but-unauthenticated disc image does) must produce the exact
+608 -61
View File
@@ -1,48 +1,37 @@
//! CSS title-key recovery — Frank A. Stevenson's divide-and-conquer attack
//! (1999), ported exactly from libdvdcss `RecoverTitleKey` + `AttackPattern`
//! (css.c).
//!
//! Recovers the 5-byte CSS title key from a single scrambled DVD sector with
//! no player keys and no disc-key crack, using only known plaintext.
//! (1999), implemented from his published cryptanalysis ("Cryptanalysis of
//! Contents Scrambling System"). It recovers the 5-byte CSS title key from a
//! single scrambled DVD sector with no player keys and no disc-key crack, using
//! only known plaintext. Implemented from that public description; nothing here
//! is copied or translated from any particular CSS software.
//!
//! # The cipher this attacks
//!
//! The content descrambler ([`super::lfsr::descramble_sector`], = libdvdcss
//! `dvdcss_unscramble`) seeds its two LFSRs **directly** from
//! `key = title_key XOR sector_seed` (seed = `sector[0x54..0x59]`):
//!
//! ```text
//! i_t1 = (key[0] ^ sec[0x54]) | 0x100; // LFSR1 low (9-bit)
//! i_t2 = key[1] ^ sec[0x55]; // LFSR1 high
//! i_t3 = (key[2]|key[3]<<8|key[4]<<16) ^ seed3; // LFSR0 (24-bit feedback)
//! i_t3 = i_t3*2 + 8 - (i_t3 & 7);
//! // per byte: *p = TAB1[*p] ^ (i_t5 & 0xff)
//! ```
//!
//! There is NO `decrypt_key` mangling on the content path. So the recovery
//! is a single inversion of `dvdcss_unscramble`, not the multi-stage
//! working-key inversion the previous (non-CSS) implementation used.
//! The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
//! LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
//! `sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
//! bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
//! byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
//! title-key mangling on the content path, so the recovery is a single inversion
//! of the sector cipher.
//!
//! # The attack
//!
//! 1. **Known plaintext → keystream.** Because the descramble applies TAB1
//! to the ciphertext, the per-byte keystream is
//! `buf[i] = TAB1[cipher[i]] ^ plain[i]` (matching libdvdcss
//! `RecoverTitleKey`'s `p_buffer`).
//! 1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
//! ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
//! 2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
//! forward; for the first four steps deduce the LFSR0 output bytes from
//! the keystream (carry-tracked), reconstructing `i_t3`. For the next six
//! steps clock LFSR0 normally and check it reproduces the keystream — a
//! wrong LFSR1 seed fails fast.
//! 3. **Back-clock LFSR0.** Run four backward `i_t3` steps (each a 256-way
//! search for the byte shifted in) to reach the initial state, then undo
//! `i_t3 = i_t3*2 + 8 - (i_t3 & 7)` to recover key[2..5].
//! 4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]` (plain XOR —
//! the descramble seeds directly, so there is no inversion).
//! forward; for the first four steps deduce the LFSR0 output bytes from the
//! keystream (carry-tracked), reconstructing LFSR0's state. For the next six
//! steps clock LFSR0 normally and check it reproduces the keystream — a wrong
//! LFSR1 seed fails fast.
//! 3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
//! the byte shifted in) to reach the initial state, then undo the
//! `r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
//! 4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
//!
//! `AttackPattern` finds known plaintext for step 1: the longest periodic
//! run in the cleartext `sec[0x00..0x80]`, assumed to continue into the
//! encrypted region at 0x80.
//! Known plaintext for step 1 comes from the longest periodic run in the
//! cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
//! 0x80.
use super::lfsr::descramble_sector;
use super::tables::{TAB1, TAB2, TAB3, TAB4, TAB5};
@@ -52,13 +41,11 @@ const ENCRYPTED_START: usize = 0x80; // byte 128
const SEED_OFFSET: usize = 0x54; // sector seed at bytes 0x54-0x58
const FLAG_BYTE: usize = 0x14;
/// RecoverTitleKey: recover the title key from cipher + known plaintext.
///
/// Exact port of libdvdcss `RecoverTitleKey` (css.c). `crypted` is the
/// ciphertext starting at sector byte 0x80; `decrypted` is the matching
/// known plaintext; `seed` is `sector[0x54..0x59]`. On success returns the
/// recovered 5-byte title key; `None` if no LFSR seed reproduces the
/// keystream.
/// Recover the title key from cipher + known plaintext (the core of Stevenson's
/// attack). `crypted` is the ciphertext starting at sector byte 0x80;
/// `decrypted` is the matching known plaintext; `seed` is `sector[0x54..0x59]`.
/// On success returns the recovered 5-byte title key; `None` if no LFSR seed
/// reproduces the keystream.
///
/// At least 10 bytes of `crypted`/`decrypted` are required (the cipher is
/// iterated 10 times: 4 to reconstruct LFSR0, 6 to validate).
@@ -222,16 +209,13 @@ fn descramble_matches(sector: &[u8], title: &[u8; 5], plain: &[u8]) -> bool {
test[ENCRYPTED_START..ENCRYPTED_START + n] == plain[..n]
}
/// AttackPattern: find a repeating pattern just before the encrypted region
/// and assume the plaintext at 0x80 continues it.
///
/// Functionally-equivalent port of libdvdcss `AttackPattern` (css.c) — finds the
/// same periodic cribs on real DVD data, though its byte-comparison anchor
/// differs from the C on phase-misaligned runs. Scans cleartext
/// `sec[0x00..0x80]` for the longest run that repeats with a cycle length in
/// 2..0x2F. If the run is long enough (`plen > 3` and at least two full
/// cycles), the known plaintext at 0x80 is taken to be the periodic run
/// continuing forward, and [`recover_title_key_from_plain`] is applied.
/// Find a repeating pattern just before the encrypted region and assume the
/// plaintext at 0x80 continues it — the known-plaintext step of Stevenson's
/// attack. Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// with a cycle length in 2..0x2F. If the run is long enough (`plen > 3` and at
/// least two full cycles), the known plaintext at 0x80 is taken to be the
/// periodic run continuing forward, and [`recover_title_key_from_plain`] is
/// applied.
pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
if sector.len() < SECTOR_BYTES {
return None;
@@ -260,10 +244,7 @@ pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
result
}
/// Inner body of [`crack_title_key`] — the actual AttackPattern search. Split
/// out so the public entry point can wall-clock the whole attempt for the
/// runaway guard without threading a timer through every return path.
/// AttackPattern crib: the predicted 10-byte plaintext at byte 0x80.
/// Crib: the predicted 10-byte plaintext at byte 0x80.
///
/// Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
/// run that repeats with a cycle length in 2..0x2F. If the run is long enough
@@ -369,7 +350,7 @@ mod tests {
/// Build a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends
/// in a periodic run that continues into the encrypted region — the case
/// `AttackPattern` (crack_title_key) is designed to crack.
/// `crack_title_key` is designed to crack.
fn synth_periodic_sector(
title_key: &[u8; 5],
seed: &[u8; 5],
@@ -382,7 +363,7 @@ mod tests {
// (RUN_START..0x80) and continuing into the encrypted region. This
// mirrors a real VOB: a periodic data run just before the scrambled
// part. The run must NOT overlap the seed bytes (0x54..0x59), or the
// AttackPattern detector would break mid-run. The phase is anchored to
// the crib detector would break mid-run. The phase is anchored to
// offset 0 so the run is consistent across the 0x80 boundary.
// Just above the seed (0x54..0x59); gives a 39-byte run (0x59..0x80)
// — enough for >=2 cycles of every tested period (<=19).
@@ -469,7 +450,65 @@ mod tests {
}
}
/// MANDATORY (Task C.1): the AttackPattern entry point crack_title_key —
/// `descramble_matches` is the ONLY gate between the LFSR search and a key
/// handed back to the caller: both [`recover_title_key`] and the crib-driven
/// `crack_title_key_inner` return a candidate only if this says the key
/// really descrambles the sector to the known plaintext. A body that always
/// answered `true` would let the first spurious LFSR-seed match through as
/// the title key — the ripper would then descramble the whole title with a
/// key that opens nothing, producing garbage rather than a "no key" error.
///
/// Pinned both directions: the genuine key is accepted, and EVERY key one
/// bit away from it is rejected. The one-bit neighbours are the strongest
/// form of wrong key — a gate that only rejects wildly different keys would
/// still pass a near-miss out of the 2^16 seed search.
#[test]
fn descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
assert!(
descramble_matches(&sector, &title_key, &PES),
"the key the sector was scrambled with must be accepted"
);
for byte in 0..5usize {
for bit in 0..8u32 {
let mut wrong = title_key;
wrong[byte] ^= 1u8 << bit;
assert!(
!descramble_matches(&sector, &wrong, &PES),
"key differing only in byte {byte} bit {bit} must be rejected"
);
}
}
}
/// The gate is applied to a COPY: verifying a candidate must not modify the
/// caller's sector. `recover_title_key` runs the gate and then hands the
/// sector on to be descrambled for real — if verification descrambled in
/// place, that second descramble would run over already-transformed bytes
/// (and, worse, a rejected candidate would leave the sector corrupted).
#[test]
fn descramble_matches_does_not_disturb_the_caller_s_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _body) = synth_sector(&title_key, &seed, &PES);
let before = sector.clone();
assert!(descramble_matches(&sector, &title_key, &PES));
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
assert_eq!(
sector, before,
"verification must leave the sector byte-for-byte unchanged"
);
}
/// MANDATORY (Task C.1): the crib-based entry point crack_title_key —
/// no plaintext supplied — recovers a round-tripping key when the
/// cleartext ends in a periodic run that continues into 0x80.
#[test]
@@ -491,7 +530,7 @@ mod tests {
}
}
/// recover_title_key_from_plain inverts dvdcss_unscramble exactly: scramble
/// recover_title_key_from_plain inverts descramble_sector exactly: scramble
/// a known body, hand back the keystream-derived key, and the recovered
/// key (XOR-back included) reproduces the plaintext.
#[test]
@@ -590,4 +629,512 @@ mod tests {
let _ = crack_title_key(&sector);
}
}
// ── entry-point guards on caller- and disc-supplied lengths ────────────
/// A sector buffer that ENDS inside the encrypted region must be refused,
/// not sliced.
///
/// `recover_title_key` slices `sector[0x80..0x8A]` unconditionally after its
/// length guard. The existing short-sector test uses `SECTOR_BYTES - 1`,
/// which is still long enough for that slice to succeed — so the guard was
/// never the thing producing the `None`, and dropping it (or weakening the
/// `||` to `&&`, which a full-length crib satisfies) changed nothing
/// observable. On a real short read this is an out-of-bounds panic on the
/// rip thread.
#[test]
fn recover_rejects_a_sector_that_ends_inside_the_encrypted_region() {
for len in [0x81usize, 0x85, 0x89] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so no other guard fires first
assert!(
recover_title_key(&sector, &PES).is_none(),
"a {len}-byte buffer cannot supply ten ciphertext bytes at 0x80"
);
}
}
/// A buffer LONGER than one sector is still one sector: both entry points
/// read the first `SECTOR_BYTES` and must recover the key from it.
///
/// Callers read DVD data in multi-sector blocks, so an over-long slice is
/// the normal case, not an exotic one. A length guard that rejected it
/// (`len > SECTOR_BYTES` instead of `<`) would make every block-read caller
/// silently unable to crack anything.
#[test]
fn a_buffer_longer_than_one_sector_still_yields_its_key() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let mut padded = sector.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
recover_title_key(&padded, &PES),
Some(title_key),
"a two-and-a-bit-sector buffer must still recover the first sector's key"
);
let (periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
let mut padded = periodic.clone();
padded.extend_from_slice(&[0xA7u8; 512]);
assert_eq!(
crack_title_key(&padded),
crack_title_key(&periodic),
"padding past the sector must not change the crack result"
);
assert!(crack_title_key(&padded).is_some());
}
/// `recover_title_key` accepts MORE than ten bytes of known plaintext, and
/// uses all of it: the extra bytes tighten the `descramble_matches` gate.
/// The ten-byte figure is a MINIMUM (the cipher is iterated ten times), not
/// an exact requirement — a guard reading it as an upper bound would reject
/// every caller that knows a longer crib.
#[test]
fn recover_accepts_more_than_ten_bytes_of_known_plaintext() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let long_plain: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(37).wrapping_add(5))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &long_plain);
assert_eq!(
recover_title_key(&sector, &long_plain),
Some(title_key),
"64 bytes of known plaintext must be accepted, not rejected as \
'more than ten'"
);
}
/// The scramble-flag gate on a sector whose BODY really is ciphertext.
///
/// Both entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
/// unscrambled sector has no title key to recover, and its bytes at 0x80
/// are already plaintext. Every prior test of this gate used an all-zero or
/// all-`0x11` sector, where the recovery would have found nothing anyway —
/// so widening the mask test (`&` to `|`, which makes it true for EVERY
/// flag byte) produced the same `None` and went unseen.
///
/// Here the sector is genuinely scrambled and its key IS recoverable; only
/// the cleared flag stands in the way. If the gate stops working, both
/// functions start returning keys for sectors the disc says are in the
/// clear.
#[test]
fn a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key(&sector, &PES),
Some(title_key),
"fixture check: with the flag set this sector's key IS recoverable"
);
sector[FLAG_BYTE] = 0x00;
assert_eq!(
recover_title_key(&sector, &PES),
None,
"scramble bits clear → no title key, even though one could be found"
);
let (mut periodic, _) = synth_periodic_sector(&title_key, &seed, 5);
assert!(
crack_title_key(&periodic).is_some(),
"fixture check: with the flag set this sector cracks"
);
assert!(
attack_crib(&periodic).is_some(),
"fixture check: with the flag set this sector has a usable crib"
);
periodic[FLAG_BYTE] = 0x00;
assert_eq!(
crack_title_key(&periodic),
None,
"scramble bits clear → no crack, even though one would succeed"
);
// `attack_crib` carries its own copy of the same gate, and it is the one
// that actually stops the crack (`crack_title_key`'s is defensive
// duplication). The crib doubles as the decrypt path's cached-key
// oracle, so a widened mask there would hand that path a "predicted
// plaintext" for sectors that were never scrambled.
assert_eq!(
attack_crib(&periodic),
None,
"an unscrambled sector has no predicted plaintext to offer"
);
}
// ── descramble_matches: the verification gate's own mechanics ──────────
/// The gate must verify a candidate against the sector's CIPHERTEXT
/// regardless of what the sector's own flag byte says.
///
/// `descramble_matches` forces `0x10` on its copy precisely because
/// [`super::lfsr::descramble_sector`] is a no-op when the scramble bits are
/// clear — without that, verifying a scrambled-but-unflagged sector
/// compares raw ciphertext against the crib, and every candidate key is
/// rejected. Nothing exercised it: every fixture already had the flag set,
/// where forcing the bit is a no-op.
#[test]
fn descramble_matches_forces_the_scramble_flag_on_its_own_copy() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (mut sector, _) = synth_sector(&title_key, &seed, &PES);
sector[FLAG_BYTE] = 0x00;
assert!(
descramble_matches(&sector, &title_key, &PES),
"the body is ciphertext and the key is right — the gate must \
descramble it even though the flag byte says otherwise"
);
let mut wrong = title_key;
wrong[0] ^= 0x01;
assert!(!descramble_matches(&sector, &wrong, &PES));
}
/// The gate compares the WHOLE supplied plaintext, clamped to the encrypted
/// region.
///
/// Two properties in one, because they are the two halves of
/// `plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
///
/// - it must compare beyond the first sixteen bytes, or a key that opens
/// only the head of the crib is accepted; and
/// - it must never compare past the end of the sector — a caller that
/// knows more plaintext than the 1920-byte encrypted region holds
/// otherwise indexes off the end of the buffer and panics.
#[test]
fn descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let body: Vec<u8> = (0..64u8)
.map(|k| k.wrapping_mul(29).wrapping_add(3))
.collect();
let (sector, _) = synth_sector(&title_key, &seed, &body);
assert!(descramble_matches(&sector, &title_key, &body));
// A crib agreeing for the first 16 bytes and diverging after must be
// rejected: the comparison window is the crib's length, not a fixed
// prefix.
let mut tail_wrong = body.clone();
tail_wrong[40] ^= 0xFF;
assert!(
!descramble_matches(&sector, &title_key, &tail_wrong),
"a crib that diverges at byte 40 must not match"
);
assert_eq!(
tail_wrong[..16],
body[..16],
"fixture check: the first 16 bytes are identical, so only a \
comparison that runs past them can tell these apart"
);
// A crib LONGER than the encrypted region: the comparison is clamped to
// the sector, not run off the end of it.
let plain_len = SECTOR_BYTES - ENCRYPTED_START;
let mut over_long = vec![0u8; plain_len + 10];
let (full_sector, full_body) = synth_sector(&title_key, &seed, &[0x00u8; 10]);
over_long[..plain_len].copy_from_slice(&full_body[ENCRYPTED_START..]);
assert!(
descramble_matches(&full_sector, &title_key, &over_long),
"a crib longer than the encrypted region must be clamped, not \
compared past the end of the sector"
);
}
// ── attack_crib: known-answer vectors ──────────────────────────────────
//
// `attack_crib` is BOTH the cracker's known plaintext and the decrypt
// path's "did the cached key descramble correctly?" oracle. Until now it
// was only ever exercised end-to-end through `crack_title_key`, on a
// fixture whose periodic run covered 39 bytes (0x59..0x80) — long enough
// that the run start, the cycle count and the `i % best_p` wrap were all
// slack. A crib that silently drifts costs a rip its title key.
/// Build a sector whose clear header ends in a `period`-length repeating
/// run of exactly `run_len` bytes immediately before 0x80.
///
/// The run is anchored to ABSOLUTE sector offset (`sec[x] = pat[x % period]`),
/// which is what makes "the run continues past 0x80" a statement independent
/// of the code under test: the byte at `0x80 + i` of the underlying
/// plaintext is `pat[(0x80 + i) % period]`.
///
/// Everything before the run is `0x00` (the pattern bytes are all >= 0xD0,
/// so the run cannot be extended backwards by accident), and the encrypted
/// region is filled with `0xFF` — so a crib that reads past 0x80 into
/// "ciphertext" is immediately visible.
fn sector_with_trailing_run(period: usize, run_len: usize) -> Vec<u8> {
assert!(
run_len < ENCRYPTED_START,
"the run lives in the clear header"
);
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for x in (ENCRYPTED_START - run_len)..ENCRYPTED_START {
sector[x] = pat[x % period];
}
sector
}
/// The crib the run PREDICTS: the periodic pattern continued past 0x80.
fn expected_crib(period: usize) -> [u8; 10] {
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
let mut out = [0u8; 10];
for (i, o) in out.iter_mut().enumerate() {
*o = pat[(ENCRYPTED_START + i) % period];
}
out
}
/// KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly
/// at 0x80, the crib is the run continued forward — the same ten bytes for
/// every run length, because the prediction depends only on the pattern and
/// the phase, never on how many cycles happened to be visible.
///
/// The short lengths are the load-bearing ones: at `run_len = 11` the crib
/// window starts at 0x76 and is only 10 bytes from the end of the header, so
/// any drift in `plain_start`, in `cycles * best_p`, or in the `i % best_p`
/// wrap reads the 0xFF "ciphertext" instead of the run.
#[test]
fn attack_crib_predicts_the_periodic_run_continuing_past_0x80() {
for &run_len in &[11usize, 12, 13, 14, 15, 16, 20, 31] {
let sector = sector_with_trailing_run(5, run_len);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(5)),
"period-5 run of {run_len} bytes must predict the run continuing"
);
}
}
/// The same known answer across several periods, including a period that
/// does NOT divide 0x80 (so the crib's phase is non-zero and a body that
/// restarted the pattern at index 0 gives a different answer).
#[test]
fn attack_crib_recovers_the_run_period_and_phase() {
// 0x80 % period: 3 for 5, 2 for 6, 2 for 7, 8 for 0x18 — all non-zero,
// so the predicted first byte is NOT pat[0] in any of these cases.
for &period in &[5usize, 6, 7, 0x18] {
let sector = sector_with_trailing_run(period, 3 * period + 1);
let crib =
attack_crib(&sector).unwrap_or_else(|| panic!("no crib for period {period}"));
assert_eq!(crib, expected_crib(period), "period {period}");
assert_ne!(
crib[0], 0xD0,
"period {period} does not divide 0x80, so the crib must not \
start at pattern index 0"
);
assert!(
crib.iter().all(|&b| b != 0xFF),
"period {period}: the crib must never contain a byte read from \
the encrypted region"
);
}
}
/// A run of exactly ONE cycle (plus the trivial tail the detector counts) is
/// not enough to predict forward: [`attack_crib`] requires at least two full
/// cycles. Weakening that guard would let a one-off byte sequence be
/// declared periodic and produce a confidently wrong crib — which the
/// decrypt path uses as its "is my cached key still right?" oracle.
#[test]
fn attack_crib_refuses_a_run_shorter_than_two_cycles() {
// period 8, run of 9 bytes: best_plen = 8, 8 / 8 == 1 cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 9)), None);
// period 0x18, run of 0x19 bytes: one cycle.
assert_eq!(attack_crib(&sector_with_trailing_run(0x18, 0x19)), None);
// ...and one more byte of run does not conjure a second cycle either.
assert_eq!(attack_crib(&sector_with_trailing_run(8, 10)), None);
}
/// A header with no repeating tail at all yields no crib. Asserted on a
/// header whose bytes are pairwise distinct right up to 0x80, so no cycle
/// length in 2..0x2F can match even one byte.
#[test]
fn attack_crib_refuses_a_header_with_no_periodic_tail() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x10;
// 0x00..0x80 strictly increasing: sec[a] == sec[b] iff a == b, so the
// detector's `sec[0x7f - (j % i)] == sec[0x7f - j]` needs j % i == j,
// which the scan's starting `j = i + 1` already excludes.
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = x as u8;
}
assert_eq!(attack_crib(&sector), None);
// And the cracker built on it reports no key rather than guessing.
assert_eq!(crack_title_key(&sector), None);
}
/// `attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
/// own length guard is the only thing between a short buffer and an
/// out-of-bounds read. Nothing reached it: every caller-level test used a
/// full sector, and the entry points' guards fire first.
#[test]
fn attack_crib_refuses_a_buffer_shorter_than_a_sector() {
for len in [0x15usize, 0x40, 0x7F, SECTOR_BYTES - 1] {
let mut sector = vec![0x11u8; len];
sector[FLAG_BYTE] = 0x30; // scrambled, so the flag half cannot fire
assert_eq!(
attack_crib(&sector),
None,
"a {len}-byte buffer is not a sector"
);
}
}
/// A header that is periodic ALL THE WAY to offset 0 must not walk the
/// backward scan off the front of the sector.
///
/// The detector counts backwards from 0x7f while `j < 0x80`. On a fully
/// periodic header the run never breaks, so `j` reaches 0x7f and the bound
/// is the ONLY thing that stops it — one step further and `0x7f - j`
/// underflows a `usize` and panics. A constant or fully-patterned 128-byte
/// header is ordinary DVD data (padding, a run of zeros), not a crafted
/// input, and every existing fixture had a filler/run boundary well before
/// offset 0 that stopped the scan early.
#[test]
fn attack_crib_survives_a_header_that_is_periodic_to_offset_zero() {
let mut sector = vec![0u8; SECTOR_BYTES];
sector[FLAG_BYTE] = 0x30;
let period = 5usize;
let pat: Vec<u8> = (0..period).map(|k| 0xD0u8 + k as u8).collect();
for (x, b) in sector[..ENCRYPTED_START].iter_mut().enumerate() {
*b = pat[x % period];
}
for b in sector[ENCRYPTED_START..].iter_mut() {
*b = 0xFF;
}
// The FLAG byte sits inside the header at 0x14, so it interrupts the
// pattern there; re-lay it and accept that 0x14 breaks the run — the
// scan still reaches offset 0x15 - 1 = 0x14 going backwards, i.e.
// j = 0x7f - 0x14 = 0x6b, well short of the bound. Instead put the
// scramble flag bits into a byte value that IS the pattern's.
sector[FLAG_BYTE] = pat[FLAG_BYTE % period];
assert_ne!(
sector[FLAG_BYTE] & 0x30,
0,
"fixture check: the pattern byte at 0x14 must itself carry \
scramble bits, so the header stays unbroken"
);
assert_eq!(
attack_crib(&sector),
Some(expected_crib(period)),
"a fully periodic header must predict its own continuation, and \
the backward scan must stop at offset 0"
);
}
/// The crib is read from the CLEAR header only. A run that reaches 0x80 must
/// predict from the header bytes, never from the encrypted region — the
/// previously-fixed bug this function's doc comment records. Pinned by
/// rewriting the encrypted region and requiring the crib not to move.
#[test]
fn attack_crib_is_independent_of_the_encrypted_region() {
let base = sector_with_trailing_run(5, 11);
let crib = attack_crib(&base).expect("crib");
for fill in [0x00u8, 0x5A, 0xD1, 0xFF] {
let mut s = base.clone();
for b in s[ENCRYPTED_START..].iter_mut() {
*b = fill;
}
assert_eq!(
attack_crib(&s),
Some(crib),
"the crib must not depend on the encrypted region (fill {fill:#04x})"
);
}
}
// ── recover_title_key_from_plain: input-length guard ───────────────────
/// `recover_title_key_from_plain` unconditionally builds a 10-byte keystream
/// buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard
/// is the only thing standing between a short slice and an
/// index-out-of-bounds PANIC.
///
/// Nothing reached that guard before: `recover_title_key` rejects
/// `plain.len() < 10` at its own door and always hands on exactly ten
/// ciphertext bytes, and `crack_title_key_inner` always passes a fixed
/// `[u8; 10]` crib. The guard is a live contract for any future caller and
/// was executed by no test at either boundary.
#[test]
fn recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input() {
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let full = [0xA5u8; 10];
for n in 0..10usize {
assert_eq!(
recover_title_key_from_plain(&full[..n], &full, &seed),
None,
"{n} ciphertext bytes is fewer than the ten the cipher iterates"
);
assert_eq!(
recover_title_key_from_plain(&full, &full[..n], &seed),
None,
"{n} plaintext bytes is fewer than the ten the cipher iterates"
);
}
// Exactly ten of each is ACCEPTED as far as the search — the boundary is
// `< 10`, not `<= 10`. (Whether this particular keystream has a seed is
// immaterial; what must not happen is an early `None` from the guard.)
// Proven through the round-trip fixture, whose inputs are exactly ten
// bytes and which does recover its key.
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
assert_eq!(
recover_title_key_from_plain(
&sector[ENCRYPTED_START..ENCRYPTED_START + 10],
&PES,
&seed
),
Some(title_key),
"exactly ten bytes of each input must run the search, not trip the guard"
);
}
/// The seed XOR-back ([`recover_title_key_from_plain`]'s last step) is what
/// turns the recovered LFSR key into the TITLE key: `key ^= sector_seed`.
/// Pinned as a known answer across seeds that differ only in one byte — the
/// same ciphertext/plaintext pair therefore must yield title keys differing
/// in exactly that byte.
///
/// Without this, a body that ORed the seed in (or dropped the step) still
/// round-trips on any fixture whose seed is zero, and on the non-zero ones
/// the failure looks like "no key found" rather than a wrong step.
#[test]
fn recover_title_key_from_plain_xors_the_sector_seed_back_out() {
let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let (sector, _) = synth_sector(&title_key, &seed, &PES);
let crypted = &sector[ENCRYPTED_START..ENCRYPTED_START + 10];
// The cipher is seeded from `title_key XOR seed`, so re-running the SAME
// ciphertext/plaintext against a seed differing in one byte must return
// a title key differing in exactly that byte — the XOR is a bijection.
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &seed),
Some(title_key)
);
for byte in 0..5usize {
for bit in [0u32, 3, 7] {
let mut alt_seed = seed;
alt_seed[byte] ^= 1u8 << bit;
let mut expected = title_key;
expected[byte] ^= 1u8 << bit;
assert_eq!(
recover_title_key_from_plain(crypted, &PES, &alt_seed),
Some(expected),
"seed byte {byte} bit {bit} must XOR straight through to the \
title key"
);
}
}
}
}
+11 -12
View File
@@ -24,9 +24,8 @@ pub const TAB1: [u8; 256] = [
0xb7, 0xf7, 0xbf, 0xa2, 0xe7, 0xa7, 0xef, 0xf2, 0xba, 0xfa, 0xb2, 0xaf, 0xea, 0xaa, 0xe2, 0xff,
];
/// Table 2: LFSR1 high-byte feedback permutation.
///
/// Byte-identical to libdvdcss `p_css_tab2` (csstables.h).
/// Table 2: LFSR1 high-byte feedback permutation — a fixed constant of the CSS
/// cipher (per the published algorithm).
pub const TAB2: [u8; 256] = [
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x09, 0x08, 0x0b, 0x0a, 0x0d, 0x0c, 0x0f, 0x0e,
0x12, 0x13, 0x10, 0x11, 0x16, 0x17, 0x14, 0x15, 0x1b, 0x1a, 0x19, 0x18, 0x1f, 0x1e, 0x1d, 0x1c,
@@ -46,12 +45,12 @@ pub const TAB2: [u8; 256] = [
0xed, 0xec, 0xef, 0xee, 0xe9, 0xe8, 0xeb, 0xea, 0xe4, 0xe5, 0xe6, 0xe7, 0xe0, 0xe1, 0xe2, 0xe3,
];
/// Table 3: LFSR1 9-bit low-word feedback table (512 entries).
/// Table 3: LFSR1 9-bit low-word feedback table (512 entries) — a fixed constant
/// of the CSS cipher (per the published algorithm).
///
/// Byte-identical to libdvdcss `p_css_tab3` (csstables.h): the 8-value
/// block `BASE[i & 7]` repeated 64 times. The CSS LFSR1 step indexes this
/// table with the 9-bit low register (0x100..=0x1FF), but only the low 3
/// bits select the output — the high bits are ignored, hence the constant
/// It is the 8-value block `BASE[i & 7]` repeated 64 times. The CSS LFSR1 step
/// indexes this table with the 9-bit low register (0x100..=0x1FF), but only the
/// low 3 bits select the output — the high bits are ignored, hence the constant
/// blocks. The 512-entry width simply lets the 9-bit index be used without
/// masking.
pub const TAB3: [u8; 512] = [
@@ -197,12 +196,12 @@ mod tests {
}
}
/// TAB3 is the libdvdcss `p_css_tab3`: the 8-value feedback block
/// TAB3 is the CSS LFSR1 low-word table: the 8-value feedback block
/// BASE = [0x00,0x24,0x49,0x6d,0x92,0xb6,0xdb,0xff]
/// repeated 64 times — `TAB3[i] == BASE[i & 7]`. The high bits of the
/// 9-bit index do not affect the output (libdvdcss's LFSR1 step indexes
/// with the full 9-bit low register but only `& 7` matters). This pins
/// all 512 entries to the published table.
/// 9-bit index do not affect the output (the LFSR1 step indexes with the
/// full 9-bit low register but only `& 7` matters). This pins all 512
/// entries to the published cipher's table.
///
/// Mutation: flip any single byte in the TAB3 literal -> the formula
/// check fails at that index.
+1266 -783
View File
File diff suppressed because it is too large Load Diff
+104 -47
View File
@@ -22,10 +22,7 @@
//! `Disc`-level dump ([`dump_disc`]) covers everything that survives
//! lowering: titles, streams, the picked main feature, and AACS state.
use crate::disc::{
AudioChannels, ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, SampleRate,
Stream,
};
use crate::disc::{ColorSpace, Disc, DiscTitle, FrameRate, HdrFormat, Resolution, Stream};
use crate::ifo::{CellCategory, DvdTitle};
const DIAG: &str = "freemkv::diag";
@@ -95,36 +92,12 @@ pub fn hdr_str(h: HdrFormat) -> &'static str {
}
}
/// Channel count from an [`AudioChannels`] layout (what lands in the MKV
/// `Channels` element).
pub fn channel_count(ch: AudioChannels) -> u8 {
match ch {
AudioChannels::Mono => 1,
AudioChannels::Stereo => 2,
AudioChannels::Stereo21 => 3,
AudioChannels::Quad => 4,
AudioChannels::Surround50 => 5,
AudioChannels::Surround51 => 6,
AudioChannels::Surround61 => 7,
AudioChannels::Surround71 => 8,
AudioChannels::Unknown => 0,
}
}
/// Sample-rate in Hz for a [`SampleRate`].
pub fn sample_rate_hz(s: SampleRate) -> u32 {
match s {
SampleRate::S44_1 => 44100,
SampleRate::S48 => 48000,
SampleRate::S88_2 => 88200,
SampleRate::S96 => 96000,
SampleRate::S176_4 => 176400,
SampleRate::S192 => 192000,
SampleRate::S48_96 => 96000,
SampleRate::S48_192 => 192000,
SampleRate::Unknown => 0,
}
}
// `channel_count` and `sample_rate_hz` lived here as a third copy of the
// AudioChannels/SampleRate mappings. They were the only HONEST copy — returning
// 0 for Unknown where the canonical accessors fabricated 6 channels at 48 kHz —
// and their only caller was the trace line below, in this same file. The
// canonical accessors are honest now, so the duplicates are gone rather than
// left to drift a fourth time.
// ── DVD cell-category dump (from the IFO scan, pre-lowering) ─────────────────
@@ -336,7 +309,7 @@ fn frame_record(track_idx: usize, pts_ns: i64, keyframe: bool, data: &[u8]) -> V
/// `track_number` is the 1-based MKV track number; `track` is the built
/// [`crate::mux::mkv::MkvTrack`] whose fields map one-to-one onto the emitted
/// elements (see `MkvMuxer::new`). No-op unless the diag target is on.
pub fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
pub(crate) fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
if !diag_enabled() {
return;
}
@@ -499,15 +472,31 @@ pub fn dump_disc(disc: &Disc) {
tracing::debug!(
target: DIAG,
"tag=decision pick=main_feature title_idx=0 playlist={:?} dur={:.1}s \
size={}B clips={} reason=canonical_title_order(fits-disc, fewest-clips, longest, richest-audio)",
size={}B clips={} reason={}",
main.playlist,
main.duration_secs,
main.size_bytes,
main.clips.len(),
main_feature_reason(),
);
}
}
/// The `reason=` token on the main-feature decision row.
///
/// DERIVED from [`Disc::CANONICAL_TITLE_ORDER_KEYS`], which lives beside the
/// comparator that actually implements them — never restated here. The previous
/// hand-written copy drifted (it advertised a `fewest-clips` key the comparator
/// had replaced with largest-physical-size), which made the self-diagnosing log
/// explain the pick with a rule the code does not apply. A diagnostic that
/// disagrees with the decision it documents is worse than no diagnostic.
fn main_feature_reason() -> String {
format!(
"canonical_title_order({})",
Disc::CANONICAL_TITLE_ORDER_KEYS.join(", ")
)
}
fn dump_aacs(disc: &Disc) {
let Some(a) = disc.aacs.as_ref() else {
if disc.css.is_some() {
@@ -530,7 +519,7 @@ fn dump_aacs(disc: &Disc) {
a.bus_encryption,
a.mkb_version,
a.disc_hash,
a.key_source.name(),
a.key_source,
a.vuk.is_some(),
a.unit_keys.len(),
a.uk_ro.len(),
@@ -610,8 +599,8 @@ fn dump_title(ti: usize, title: &DiscTitle) {
a.pid,
a.codec,
a.channels,
channel_count(a.channels),
sample_rate_hz(a.sample_rate),
a.channels.count(),
a.sample_rate.hz(),
a.language,
a.secondary,
),
@@ -631,6 +620,59 @@ fn dump_title(ti: usize, title: &DiscTitle) {
#[cfg(test)]
mod tests {
use super::*;
// Needed only by the tests: the production code in this file no longer names
// these types directly, since the local channel/sample-rate duplicates were
// deleted in favour of the canonical accessors.
use crate::disc::{AudioChannels, SampleRate};
/// The main-feature decision row must NAME `canonical_title_order`'s sort
/// keys, not restate them from memory. The restated copy had drifted: it
/// still advertised a "fewest-clips" key long after the comparator replaced
/// clip-count with largest-physical-size, so a bug report read at
/// `--log-level 3` explained the pick with a rule the code does not apply.
///
/// The behavioural half is asserted first — against the comparator itself,
/// with literals — so the key names are checked against what the code
/// actually does, not against the string that names them.
#[test]
fn main_feature_reason_names_the_comparators_real_keys() {
use crate::disc::{Clip, Disc, DiscTitle};
let sized = |size_bytes: u64, n_clips: usize| DiscTitle {
size_bytes,
clips: (0..n_clips)
.map(|i| Clip {
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 0,
duration_secs: 0.0,
source_packets: 0,
})
.collect(),
..DiscTitle::empty()
};
// A 40-clip 8 GB title beats a 1-clip 1 GB title: the comparator's
// primary key among disc-fitting titles is LARGEST SIZE. "fewest clips"
// would predict the opposite, so the drifted string described a rule
// the comparator does not implement.
let many_clips_big = sized(8_000_000_000, 40);
let one_clip_small = sized(1_000_000_000, 1);
assert_eq!(
Disc::canonical_title_order(&many_clips_big, &one_clip_small, 25_000_000_000),
std::cmp::Ordering::Less,
"largest size wins regardless of clip count"
);
let reason = main_feature_reason();
assert!(
!reason.contains("clips"),
"the reason must not advertise a clip-count key the comparator dropped: {reason}"
);
assert_eq!(
reason, "canonical_title_order(fits-disc, largest-size, longest, richest-audio)",
"the reason must name the comparator's four keys in priority order"
);
}
#[test]
fn res_str_keeps_interlace_marker() {
@@ -656,18 +698,33 @@ mod tests {
assert_eq!(hdr_str(HdrFormat::Sdr), "SDR");
}
/// Moved from the deleted local duplicates onto the canonical accessors,
/// with the Unknown case added — which is the whole point of the change.
#[test]
fn channel_count_matches_layout() {
assert_eq!(channel_count(AudioChannels::Mono), 1);
assert_eq!(channel_count(AudioChannels::Stereo), 2);
assert_eq!(channel_count(AudioChannels::Surround51), 6);
assert_eq!(channel_count(AudioChannels::Surround71), 8);
fn channel_count_matches_layout_and_is_zero_when_unknown() {
assert_eq!(AudioChannels::Mono.count(), 1);
assert_eq!(AudioChannels::Stereo.count(), 2);
assert_eq!(AudioChannels::Surround51.count(), 6);
assert_eq!(AudioChannels::Surround71.count(), 8);
// The one that matters. This used to return 6, which is indistinguishable
// from a real 5.1 track and left every caller responsible for checking
// the variant first.
assert_eq!(
AudioChannels::Unknown.count(),
0,
"an unknown layout must not report a plausible channel count"
);
}
#[test]
fn sample_rate_hz_values() {
assert_eq!(sample_rate_hz(SampleRate::S48), 48000);
assert_eq!(sample_rate_hz(SampleRate::S96), 96000);
fn sample_rate_hz_values_and_zero_when_unknown() {
assert_eq!(SampleRate::S48.hz(), 48000.0);
assert_eq!(SampleRate::S96.hz(), 96000.0);
assert_eq!(
SampleRate::Unknown.hz(),
0.0,
"an unknown sample rate must not report a plausible 48 kHz"
);
}
#[test]
+606 -277
View File
File diff suppressed because it is too large Load Diff
+51 -1
View File
@@ -179,7 +179,10 @@ impl Disc {
// the coded video frame the subpicture was authored against
// (720x480 NTSC / 720x576 PAL) so players place and scale the
// bitmap correctly.
let (vid_w, vid_h) = ts.video.resolution.pixels();
// format_palette guards on (0, 0) and omits its `size:` line,
// so an unresolved resolution degrades to a palette-only .idx
// rather than one claiming a 0x0 frame.
let (vid_w, vid_h) = ts.video.resolution.pixels().unwrap_or((0, 0));
let codec_data = dvd_title
.palette
.as_ref()
@@ -1320,4 +1323,51 @@ mod tests {
// Chapter 0 stays at 0.0 (no shift).
assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01);
}
/// Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio
/// stream has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio,
/// per `ifo::assign_audio_sub_stream_ids` — the PID falls back to
/// `0xBD00 + i` where `i` is the stream's positional index in the IFO
/// audio-attribute table. Two MPEG-audio (coding_mode 2) streams must
/// land on two DISTINCT, correctly-offset PIDs: 0xBD00 and 0xBD01. This
/// pins the `+` (not `-`/`*`) so the second stream doesn't collide with,
/// or wrap under, the first.
#[test]
fn scan_dvd_titles_mp2_audio_pid_fallback_is_additive() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// coding_mode bits are b0>>5 & 0x7; mode 2 = MPEG-1 Layer II (Mp2),
// which `assign_audio_sub_stream_ids` leaves at `sub_stream_id: None`.
// b0 = 0b010_00000 = 0x40. b1 = 0 (mono, sample rate 48k).
let audio = [(0x40u8, 0x00u8, [0u8, 0u8]), (0x40u8, 0x00u8, [0u8, 0u8])];
let vts = build_vts(1000, 0x00, &audio, &[], &[(10, 109)], false);
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts,
},
],
);
let titles = Disc::scan_dvd_titles(&mut disc, &udf);
let t = &titles[0];
let audio_pids: Vec<u16> = t
.streams
.iter()
.filter_map(|s| match s {
Stream::Audio(a) => Some(a.pid),
_ => None,
})
.collect();
assert_eq!(audio_pids, vec![0xBD00u16, 0xBD01u16]);
}
}
+173 -5
View File
@@ -104,10 +104,10 @@ fn max_substream_channels(data: &[u8]) -> Option<u8> {
};
let start = pos + rel;
let frame = &data[start..];
if let Some(ch) = ac3::acmod_channels(frame) {
if ch > 0 {
best = Some(best.map_or(ch, |b| b.max(ch)));
}
if let Some(ch) = ac3::acmod_channels(frame)
&& ch > 0
{
best = Some(best.map_or(ch, |b| b.max(ch)));
}
// Advance past this frame by its declared size when that is mappable;
// otherwise step 2 bytes past the sync and re-scan for the next one.
@@ -242,7 +242,11 @@ pub fn probe_and_remap<S: SectorSource + ?Sized>(
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate};
use crate::disc::{
AudioChannels, AudioStream, Codec, ContentFormat, DiscTitle, Extent, LabelPurpose,
SampleRate,
};
use crate::sector::SectorSource;
/// Build a single, correctly-SIZED AC-3 frame whose `acmod`/`lfeon` encode a
/// known channel count. `byte4` is `fscod=0 | frmsizecod=0`, so
@@ -463,4 +467,168 @@ mod tests {
};
assert_eq!(a.pid, 0xBD80, "no probe data → keep ordinal");
}
/// `max_substream_channels` must locate the sync at its true ABSOLUTE
/// position (`pos + rel`) when it is preceded by non-sync bytes, not just
/// when the sync sits at offset 0. Regression guard for a hand-checked
/// mutation (`+` → `-` at the `pos + rel` offset computation): with `pos`
/// starting at 0 and the first sync found 3 bytes in, `pos - rel` would
/// underflow a `usize` and panic, or (if it somehow didn't) index the
/// wrong start entirely. `pos + rel` is the only computation that is
/// always in-bounds, since `rel` is itself bounded by the length of the
/// slice searched from `pos`.
#[test]
fn max_substream_channels_locates_sync_after_leading_non_sync_bytes() {
let mut data = vec![0xAA, 0xAA, 0xAA]; // no 0x0B77 pattern in here
data.extend(ac3_frame(2, false)); // real 2.0 frame, sync at absolute offset 3
assert_eq!(
max_substream_channels(&data),
Some(2),
"must find and decode the frame whose sync is NOT at offset 0"
);
}
/// When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
/// `fscod == 3`), `max_substream_channels` must fall back to stepping
/// `start + 2` bytes past the sync to re-lock onto the next genuine sync,
/// and must keep making forward progress doing so (never revisit the same
/// sync, which would loop forever, and never jump so far that it skips
/// the very next real frame). This lays a bogus-sized header at absolute
/// offset 4 (so `start == 4`, `start + 2 == 6`) immediately followed, at
/// offset 6, by a real, fully decodable 2.0 frame — the position the
/// `+ 2` fallback must land on exactly.
#[test]
fn max_substream_channels_unmappable_size_steps_forward_by_two() {
let mut real = ac3_frame(2, false);
// Overwrite the (unchecked) CRC bytes of the real frame — these double
// as byte4/byte5 of the bogus header 2 bytes earlier, at absolute
// offset 4: byte4 = 0xC0 (fscod=3 reserved -> ac3_frame_size == 0,
// unmappable), byte5 = 0xF8 (bsid=31 >= 11 -> acmod_channels == None,
// so the bogus header itself never contributes a spurious channel
// count).
real[2] = 0xC0;
real[3] = 0xF8;
let mut data = vec![0xAA, 0xAA, 0xAA, 0xAA]; // offsets 0..4, no sync
data.push(0x0B); // offset 4: bogus header sync byte 0
data.push(0x77); // offset 5: bogus header sync byte 1
data.extend(real); // offset 6..: the real frame (also serves as the
// bogus header's byte4/byte5 at offsets 8/9)
assert_eq!(
max_substream_channels(&data),
Some(2),
"must recover the real frame 2 bytes after the unmappable-size sync, not lose it"
);
}
/// Same fallback as above, but with the unmappable-size sync at absolute
/// offset 0 (`start == 0`) so that stepping backward instead of forward
/// (`start - 2`) would underflow rather than merely land on the wrong
/// byte. Also proves the real frame is still found 6 bytes further in,
/// confirming forward progress past the bogus header.
#[test]
fn max_substream_channels_unmappable_size_at_start_steps_forward_not_back() {
let mut data = vec![0x0B, 0x77, 0x00, 0x00, 0xC0, 0xF8]; // bogus header, offsets 0..6
data.extend(ac3_frame(2, false)); // real 2.0 frame at offset 6
assert_eq!(
max_substream_channels(&data),
Some(2),
"must step forward past the bogus header at offset 0 and find the real frame at offset 6"
);
}
/// `remap_audio_pids` must read a stream's CURRENT physical sub-stream id
/// from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
/// `0x00FF`, both of which force the low byte to `0xFF` regardless of the
/// real PID and so always miss the "already matches" shortcut. That
/// matters observably when TWO physical sub-streams share the same probed
/// channel count: with a correct read, a stream already sitting on a
/// matching sub-stream is left alone (conservative, per the module's
/// documented behaviour); with the low byte forced to `0xFF`,
/// `probed.get(&0xFF)` is always `None`, so the code falls through to the
/// "find any unclaimed match" path and picks the FIRST (lowest-keyed,
/// BTreeMap-ordered) matching physical sub-stream instead — which here is
/// a *different* sub-stream (0x80) than the one the PID already correctly
/// names (0x81), producing a spurious PID change.
#[test]
fn remap_reads_current_substream_via_and_not_or_or_xor() {
let mut probed = BTreeMap::new();
probed.insert(0x80u8, 6u8);
probed.insert(0x81u8, 6u8); // ambiguous: two physical 6ch sub-streams
let mut streams = vec![ac3_stream(0xBD81, AudioChannels::Surround51)];
let changed = remap_audio_pids(&mut streams, &probed);
assert_eq!(
changed, 0,
"already sitting on a matching physical sub-stream (0x81) must be left alone"
);
let Stream::Audio(a) = &streams[0] else {
panic!()
};
assert_eq!(
a.pid, 0xBD81,
"must not be bumped to the other matching sub-stream (0x80)"
);
}
/// A `SectorSource` stub that hands back fixed bytes regardless of the
/// requested LBA/count, for exercising `probe_and_remap`'s end-to-end
/// wiring (format/AC-3/extent/count guards -> read -> probe -> remap).
struct FixedSource {
data: Vec<u8>,
}
impl SectorSource for FixedSource {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let n = self.data.len().min(buf.len());
buf[..n].copy_from_slice(&self.data[..n]);
Ok(n)
}
}
/// End-to-end `probe_and_remap`: a Silence-of-the-Lambs-shaped MpegPs
/// title (one declared 5.1 AC-3 stream ordinally assigned 0x80) whose
/// physical VOB bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on
/// 0x81. This must reach the `remap_audio_pids` call and re-route the
/// stream to 0xBD81. It also, by construction, proves each of the guards
/// along the way lets a real, positive case through: the content-format
/// check must NOT bail on `MpegPs` (only on non-`MpegPs`), the AC-3
/// presence check must NOT bail when AC-3 IS present, and the
/// sector-count check must NOT bail when the count is nonzero — any one
/// of those inverted would skip the probe entirely and leave the PID at
/// its untouched ordinal value (0xBD80), which the assertion below would
/// catch.
#[test]
fn probe_and_remap_reroutes_silence_of_the_lambs_scenario_end_to_end() {
let mut bytes = ps_ac3(0x80, 2, false); // physical 0x80 = 2.0 down-mix
bytes.extend(ps_ac3(0x81, 7, true)); // physical 0x81 = 5.1 main mix
let mut title = DiscTitle {
playlist: "00001.ifo".into(),
playlist_id: 1,
duration_secs: 60.0,
size_bytes: bytes.len() as u64,
clips: Vec::new(),
streams: vec![ac3_stream(0xBD80, AudioChannels::Surround51)],
chapters: Vec::new(),
extents: vec![Extent {
start_lba: 0,
sector_count: 2,
}],
content_format: ContentFormat::MpegPs,
codec_privates: vec![None],
};
let mut source = FixedSource { data: bytes };
probe_and_remap(&mut source, &mut title);
let Stream::Audio(a) = &title.streams[0] else {
panic!("audio")
};
assert_eq!(
a.pid, 0xBD81,
"declared 5.1 stream must be re-routed to the physical 5.1 sub-stream 0x81"
);
}
}
+140 -72
View File
@@ -7,7 +7,6 @@ use crate::udf;
/// Result of SCSI AACS handshake (ECDH authentication).
/// Only available when scanning from a real drive, not ISO images.
#[derive(Debug)]
pub(super) struct HandshakeResult {
pub volume_id: [u8; 16],
pub read_data_key: Option<[u8; 16]>,
@@ -29,6 +28,19 @@ pub(super) struct HandshakeResult {
pub drive_unlocked: bool,
}
// Redacting `Debug`: `volume_id` and `read_data_key` (the AACS 2.0 bus key) are
// secret; print only shape. Guarded by `handshake_result_debug_is_redacted`.
impl std::fmt::Debug for HandshakeResult {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HandshakeResult")
.field("volume_id", &"<redacted>")
.field("read_data_key", &self.read_data_key.map(|_| "<redacted>"))
.field("read_data_key_err", &self.read_data_key_err)
.field("drive_unlocked", &self.drive_unlocked)
.finish()
}
}
/// Single source of truth for "is AACS bus encryption gone for this scan?". The
/// gate asks ONLY this — `if !removed { error }` — never enumerating cases. Bus
/// encryption is gone when ANY of these holds:
@@ -86,9 +98,9 @@ impl AacsCertUnlocker<'_> {
// MKB generation (best-effort) — forwarded to each source's
// `host_certs(mkb)` so a source MAY select a generation-appropriate cert
// (the default impl ignores it). A read failure leaves it `None`.
let mkb_gen = aacs::read_mkb_from_drive(session.scsi_mut())
let mkb_gen = aacs::inf::read_mkb_from_drive(session.scsi_mut())
.ok()
.and_then(|m| aacs::mkb_version(&m));
.and_then(|m| aacs::mkb::mkb_version(&m));
// Host certs are keysource-served, never compiled in — unioned from the
// explicit `DriveCredentials` and the key-source layer. With ZERO certs
@@ -111,13 +123,13 @@ impl AacsCertUnlocker<'_> {
// through method calls, so clone the (cheap) identity first.
let drive_id = session.drive_id.clone();
let fu_certs = crate::unlock_bridge::map_host_certs(&host_certs);
let unlocked = crate::unlock_bridge::run_unlockers(
let (_, unlock_res) = crate::unlock_bridge::run_bus(
session.scsi_mut(),
&drive_id,
freemkv_unlock::DiscKind::Aacs,
&fu_certs,
)
.map_err(CertUnlockFailure::Unlock)?;
);
let unlocked = unlock_res.map_err(CertUnlockFailure::Unlock)?;
// The cert handshake yields a VID on success; its absence is VidUnavailable.
let Some(volume_id) = unlocked.vid else {
return Err(CertUnlockFailure::Unlock(UnlockError::VidUnavailable));
@@ -152,10 +164,10 @@ fn unlock_error_to_error(e: &CertUnlockFailure) -> Error {
}
}
/// Map a [`CertUnlockFailure`] to a structured [`crate::aacs::UnlockOutcome`]
/// Map a [`CertUnlockFailure`] to a structured [`crate::aacs::trace::UnlockOutcome`]
/// for the resolution trace (English-free).
fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::UnlockOutcome {
use crate::aacs::UnlockOutcome;
fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::trace::UnlockOutcome {
use crate::aacs::trace::UnlockOutcome;
use freemkv_unlock::UnlockError;
match e {
CertUnlockFailure::NoHostCert { mkb } => UnlockOutcome::NoUsableHostCert { mkb: *mkb },
@@ -169,6 +181,24 @@ fn cert_unlock_outcome(e: &CertUnlockFailure) -> crate::aacs::UnlockOutcome {
}
}
/// Did the cert handshake actually carry a Volume ID?
///
/// Extracted so it can be tested as a VALUE. It only ever reaches an operator
/// as the `has_volume_id` field of the `bus_key_unavailable` warn, and
/// asserting on a `tracing` field means installing a capturing subscriber —
/// which is thread-local, while `tracing`'s callsite-interest cache is global.
/// Those two facts race: the test failed roughly one run in ten under the full
/// parallel suite while passing every time in isolation, and serialising the
/// captures was not enough because the cache can be re-evaluated against the
/// process default rather than the thread-local dispatch.
///
/// A predicate this small does not need a subscriber to verify. The polarity is
/// the whole point: an `==` here would tell an operator a VID was absent on
/// exactly the discs where one was present.
fn handshake_has_volume_id(h: &HandshakeResult) -> bool {
h.volume_id != [0u8; 16]
}
impl Disc {
/// SCSI handshake — drives the VID-acquisition flow and returns
/// a structured `HandshakeResult` for downstream key resolution.
@@ -230,7 +260,10 @@ impl Disc {
/// `mkb` is the disc's MKB generation when known, forwarded to each source's
/// [`crate::KeySource::host_certs`] so a source MAY return only
/// generation-appropriate certs (the default ignores it).
fn collect_host_certs(opts: &ScanOptions, mkb: Option<u32>) -> Vec<crate::aacs::HostCert> {
fn collect_host_certs(
opts: &ScanOptions,
mkb: Option<u32>,
) -> Vec<crate::aacs::types::HostCert> {
// Delegates to the shared cert primitive (the external freemkv-unlock-aacs
// plugin uses the same one). Kept as a thin Disc method so the existing
// collect_host_certs_* unit tests and call sites are unchanged.
@@ -308,18 +341,19 @@ impl Disc {
) -> Result<AacsState> {
use crate::aacs;
let uk_ro_data = udf_fs
.read_file(reader, crate::aacs::PATH_UNIT_KEY_RO)
.or_else(|_| udf_fs.read_file(reader, crate::aacs::PATH_UNIT_KEY_RO_DUPLICATE))
.map_err(|_| Error::AacsNoKeys)?;
let dh = aacs::disc_hash(&uk_ro_data);
let uk_ro_data =
aacs::read_first(&aacs::role_paths(udf_fs, aacs::AacsRole::UnitKey), |p| {
udf_fs.read_file(reader, p)
})?;
let dh = aacs::inf::disc_hash(&uk_ro_data);
let cc = udf_fs
.read_file(reader, crate::aacs::PATH_CONTENT_CERT)
.or_else(|_| udf_fs.read_file(reader, crate::aacs::PATH_CONTENT_CERT_ALT))
.ok()
.as_deref()
.and_then(aacs::parse_content_cert);
let cc = aacs::read_first(
&aacs::role_paths(udf_fs, aacs::AacsRole::ContentCert),
|p| udf_fs.read_file(reader, p),
)
.ok()
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let bus_encryption = cc.as_ref().map(|c| c.bus_encryption).unwrap_or(false);
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so the
// scanned `AacsState.version` and the out-of-band fetch agree. A wrong
@@ -328,7 +362,7 @@ impl Disc {
let version = cc
.as_ref()
.map(|c| c.version.major())
.unwrap_or(aacs::AACS_MAJOR_UHD);
.unwrap_or(aacs::mkb::AACS_MAJOR_UHD);
// Bus-encryption gate (wrong-keys guard). A bus-encrypted disc (Content
// Certificate bus-encryption bit set) carries bus encryption on its
@@ -353,7 +387,7 @@ impl Disc {
// file/ISO, drive unlock, cert bus key). The gate enumerates nothing.
if !bus_encryption_removed(bus_encryption, handshake) {
let (rdk_err, has_vid) = handshake
.map(|h| (h.read_data_key_err, h.volume_id != [0u8; 16]))
.map(|h| (h.read_data_key_err, handshake_has_volume_id(h)))
.unwrap_or((None, false));
tracing::warn!(
target: "freemkv::disc",
@@ -394,12 +428,12 @@ impl Disc {
Vec::new()
}
};
let mkb_ver = aacs::mkb_version(&mkb_bytes);
let mkb_ver = aacs::mkb::mkb_version(&mkb_bytes);
tracing::debug!(
target: "freemkv::disc",
phase = "scan_aacs_vid_only",
disc_hash = %aacs::disc_hash_hex(&dh),
disc_hash = %aacs::inf::disc_hash_hex(&dh),
version,
bus_encryption,
has_vid = handshake.is_some(),
@@ -410,7 +444,7 @@ impl Disc {
version,
bus_encryption,
mkb_version: mkb_ver,
disc_hash: aacs::disc_hash_hex(&dh),
disc_hash: aacs::inf::disc_hash_hex(&dh),
key_source: KeyOrigin::ExternalUk,
vuk: None,
unit_keys: vec![],
@@ -429,6 +463,27 @@ mod tests {
use crate::sector::SectorSource;
use std::collections::HashMap;
/// `HandshakeResult` carries the Volume ID and the AACS 2.0 bus (read-data)
/// key; `Debug` must redact both. Sentinel 213 (0xD5).
#[test]
fn handshake_result_debug_is_redacted() {
let hs = HandshakeResult {
volume_id: [0xD5; 16],
read_data_key: Some([0xD5; 16]),
read_data_key_err: None,
drive_unlocked: false,
};
let d = format!("{hs:?}");
assert!(
!d.contains("213"),
"HandshakeResult leaked VID/bus key: {d}"
);
assert!(
d.contains("redacted"),
"HandshakeResult missing marker: {d}"
);
}
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical
// partition (metadata_start == partition_start). Offsets cited
@@ -569,7 +624,7 @@ mod tests {
}
/// A content certificate: type byte@0 (0x00 = V10, else V20),
/// bus_encryption bit7@1, cc_id@14..20 (aacs/keys.rs parse_content_cert,
/// bus_encryption bit7@1, cc_id@14..20 (aacs/inf.rs parse_content_cert,
/// which requires ≥20 bytes and reads the bus flag from `data[1] >> 7`).
fn build_content_cert(cert_type: u8, bus_encryption: bool) -> Vec<u8> {
let mut v = vec![0u8; 20];
@@ -581,7 +636,7 @@ mod tests {
/// An MKB with one Type-and-Version record (type 0x10) carrying the
/// version as BE u32 at record offset 8, followed by a recorded EOF
/// record then trailing zero padding. mkb_content_len walks records
/// and stops at the first padding (type 0) byte (aacs/keys.rs).
/// and stops at the first padding (type 0) byte (aacs/inf.rs).
fn build_mkb(version: u32, pad_to: usize) -> Vec<u8> {
let mut v = Vec::new();
// Type 0x10 record, length 16 (>= 12 so version is read).
@@ -687,14 +742,14 @@ mod tests {
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("state");
assert_eq!(
st.version,
aacs::AACS_MAJOR_UHD,
aacs::mkb::AACS_MAJOR_UHD,
"no cert → default UHD (major 2)"
);
assert!(!st.bus_encryption);
}
/// disc_hash is SHA1 of the Unit_Key_RO.inf bytes, hex with 0x prefix
/// and uppercase (aacs::disc_hash + disc_hash_hex). The state's
/// and uppercase (aacs::inf::disc_hash + disc_hash_hex). The state's
/// disc_hash must match independently computing it over the same bytes.
#[test]
fn resolve_vid_only_disc_hash_is_sha1_of_unit_key_ro() {
@@ -710,7 +765,7 @@ mod tests {
}],
);
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("state");
let expected = aacs::disc_hash_hex(&aacs::disc_hash(&uk));
let expected = aacs::inf::disc_hash_hex(&aacs::inf::disc_hash(&uk));
assert_eq!(st.disc_hash, expected);
assert!(st.disc_hash.starts_with("0x"));
// uk_ro must be stashed verbatim for the external resolver.
@@ -746,7 +801,7 @@ mod tests {
// Real record stream is the single 16-byte type-0x10 record.
assert_eq!(
st.mkb.len(),
aacs::mkb_content_len(&mkb),
aacs::mkb::mkb_content_len(&mkb),
"MKB must be trimmed to record-stream length, not the zero-pad"
);
assert_eq!(st.mkb.len(), 16);
@@ -912,41 +967,54 @@ mod tests {
/// Unit_Key_RO.inf is read from /AACS/DUPLICATE when the primary copy
/// is absent (encrypt.rs `.or_else(|_| read_file(DUPLICATE/...))`).
/// This is the damaged-primary recovery path real discs rely on.
#[test]
fn resolve_vid_only_falls_back_to_duplicate_unit_key_ro() {
let mut disc = MemDisc::new();
// Build AACS dir with a DUPLICATE subdir holding Unit_Key_RO.inf.
let uk = vec![0x55u8; 48];
let mut dup_fids = Vec::new();
push_fid(&mut dup_fids, "", 70, true, true);
push_fid(&mut dup_fids, "Unit_Key_RO.inf", 72, false, false);
disc.put(PART_START + 72, build_file_icb(uk.len() as u32, 9000));
disc.put_bytes(PART_START + 9000, &uk);
disc.put(PART_START + 70, build_file_icb(dup_fids.len() as u32, 71));
disc.put_bytes(PART_START + 71, &dup_fids);
// AACS dir: only a DUPLICATE subdir (no primary Unit_Key_RO.inf).
let mut aacs_fids = Vec::new();
push_fid(&mut aacs_fids, "", 50, true, true);
push_fid(&mut aacs_fids, "DUPLICATE", 70, true, false);
disc.put(PART_START + 50, build_file_icb(aacs_fids.len() as u32, 51));
disc.put_bytes(PART_START + 51, &aacs_fids);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "AACS", 50, true, false);
disc.put(PART_START + 10, build_file_icb(root_fids.len() as u32, 11));
disc.put_bytes(PART_START + 11, &root_fids);
build_udf_skeleton(&mut disc, 10);
let udf = udf::read_filesystem(&mut disc).expect("fs");
let st = Disc::resolve_vid_only(&udf, &mut disc, None).expect("DUPLICATE fallback");
// disc_hash must be computed over the DUPLICATE bytes.
assert_eq!(
st.disc_hash,
aacs::disc_hash_hex(&aacs::disc_hash(&uk)),
"fallback must hash the DUPLICATE Unit_Key_RO.inf"
fn handshake_has_volume_id_reports_presence_not_absence() {
let with_vid = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
assert!(
super::handshake_has_volume_id(&with_vid),
"a non-zero Volume ID must report as PRESENT"
);
assert_eq!(st.uk_ro, uk);
let without = HandshakeResult {
volume_id: [0u8; 16],
..with_vid
};
assert!(
!super::handshake_has_volume_id(&without),
"an all-zero Volume ID is the absent case"
);
// One bit of difference is still a VID: the check is != all-zero, not a
// heuristic about how much of it looks populated.
let mut barely = [0u8; 16];
barely[15] = 1;
assert!(
super::handshake_has_volume_id(&HandshakeResult {
volume_id: barely,
..with_vid
}),
"any non-zero byte makes a Volume ID present"
);
}
/// The gate itself still hard-errors — the property the log line annotates.
#[test]
fn resolve_vid_only_bus_key_gate_hard_errors_without_a_read_data_key() {
let (mut disc, udf) = disc_with_cert(0x01, true);
let hs = HandshakeResult {
volume_id: [0x11u8; 16],
read_data_key: None,
read_data_key_err: None,
drive_unlocked: false,
};
let err = Disc::resolve_vid_only(&udf, &mut disc, Some(&hs))
.expect_err("bus-encrypted, no read_data_key must still hard-error");
assert!(matches!(err, Error::AacsBusKeyUnavailable));
}
// ---------------------------------------------------------------
@@ -965,8 +1033,8 @@ mod tests {
// the route fails gracefully (AacsNoHostCert), never panics.
// ---------------------------------------------------------------
fn fake_cert(tag: u8) -> aacs::HostCert {
aacs::HostCert {
fn fake_cert(tag: u8) -> aacs::types::HostCert {
aacs::types::HostCert {
private_key: [tag; 20],
certificate: vec![tag; 92],
private_key_v2: None,
@@ -975,15 +1043,15 @@ mod tests {
}
/// A minimal in-test KeySource that yields no keys but a fixed cert list.
struct CertSource(Vec<aacs::HostCert>);
struct CertSource(Vec<aacs::types::HostCert>);
impl crate::KeySource for CertSource {
fn get_uk(
fn get_unit_keys(
&self,
_ctx: &dyn crate::keysource::ResolveCtx,
) -> Result<Vec<crate::aacs::UnitKey>> {
) -> Result<Vec<crate::aacs::types::UnitKey>> {
Ok(Vec::new())
}
fn host_certs(&self, _mkb: Option<u32>) -> Vec<aacs::HostCert> {
fn host_certs(&self, _mkb: Option<u32>) -> Vec<aacs::types::HostCert> {
self.0.clone()
}
}
@@ -1077,7 +1145,7 @@ mod tests {
#[test]
fn cert_unlock_outcome_maps_to_structured_trace_step() {
use crate::aacs::UnlockOutcome;
use crate::aacs::trace::UnlockOutcome;
use freemkv_unlock::UnlockError;
// The libfreemkv-side no-cert case carries the MKB generation.
assert_eq!(
+631 -150
View File
@@ -1,6 +1,7 @@
//! `Disc::extract_tree` — decrypted file-tree extraction (`dir://`).
//!
//! Sibling of [`Disc::copy`](super::Disc::copy) (disc → ISO sector dump),
//! Sibling of the disc→ISO sector dump (the sweep/patch recovery passes, which
//! now live in the `freemkv-engine` crate),
//! specialized to write **per file** rather than a whole image, applying
//! decryption on the way out, and **without** any multipass / recovery
//! orchestration. 1-shot, decrypt-only.
@@ -22,7 +23,6 @@ use crate::sector::{DecryptingSectorSource, SectorSource};
use crate::udf::{self, DirEntry, UdfFs};
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::atomic::Ordering;
use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// AACS aligned unit = 3 sectors / 6144 bytes. Content reads are issued in
@@ -66,10 +66,9 @@ pub struct FileResult {
pub path: PathBuf,
/// Bytes written that decrypted cleanly.
pub bytes_good: u64,
/// Bytes lost to unreadable sectors (zero-filled holes).
/// Bytes lost — unreadable sectors AND undecryptable units both land here
/// (extract fails a bad decrypt loud, so it is zero-filled like a bad sector).
pub bytes_unreadable: u64,
/// Bytes lost to undecryptable AACS/CSS units (still ciphertext / dropped).
pub bytes_undecryptable: u64,
/// True when the file was fully written (renamed from `.partial`).
pub complete: bool,
}
@@ -81,10 +80,8 @@ pub struct ExtractResult {
pub files: Vec<FileResult>,
/// Aggregate good bytes across all files.
pub bytes_good: u64,
/// Aggregate unreadable (bad-sector) bytes.
/// Aggregate lost bytes — bad sectors AND undecryptable units (one bucket).
pub bytes_unreadable: u64,
/// Aggregate undecryptable (decrypt-loss) bytes.
pub bytes_undecryptable: u64,
/// True when every file completed and no loss was recorded.
pub complete: bool,
/// True when the run stopped early on an interrupt / progress halt.
@@ -92,11 +89,10 @@ pub struct ExtractResult {
}
impl ExtractResult {
/// Total bytes lost (unreadable + undecryptable). A non-zero value means
/// the extraction is holed; the CLI exits non-zero so a script can re-run
/// through the `iso://` multipass path.
/// Total bytes lost. A non-zero value means the extraction is holed; the CLI
/// exits non-zero so a script can re-run through the `iso://` multipass path.
pub fn bytes_lost(&self) -> u64 {
self.bytes_unreadable + self.bytes_undecryptable
self.bytes_unreadable
}
}
@@ -169,13 +165,13 @@ impl Disc {
.iter()
.map(|p| p.size)
.fold(0u64, |a, b| a.saturating_add(b));
if let Some(available) = available_space(dest) {
if available < required {
return Err(Error::DirInsufficientSpace {
required,
available,
});
}
if let Some(available) = available_space(dest)
&& available < required
{
return Err(Error::DirInsufficientSpace {
required,
available,
});
}
// Create directories up-front so a leaf write never races a missing
@@ -190,14 +186,51 @@ impl Disc {
// Per-VTS CSS key map (DVD only): "VTS_xx" -> DecryptKeys. Built lazily
// when a scrambled VOB group needs it. AACS / None discs keep the
// disc-wide keys for every file.
let base_keys = self.decrypt_keys();
let mut base_keys = self.decrypt_keys();
// AACS key map for the extract, chosen by CPS-unit count:
//
// * SINGLE CPS (the overwhelming majority, incl. every single-key UHD): one
// Unit Key opens EVERY encrypted unit on the disc — content in a parsed
// title AND an orphan clip that no playlist references. A blanket key-0
// map over the whole LBA space is exact and covers orphans; clear
// filesystem/nav (encrypted-flag off) passes through untouched.
//
// * MULTI-CPS: each clip is protected by a different Unit Key, so a blanket
// key-0 map would mis-decrypt every secondary-CPS file into garbage
// (silently, since Phase::All is trust-only). Build the EXACT per-CPS
// content map instead (each title's extents → the CPS key that opens a
// real sample from it), up front before the decorator takes the reader. A
// content unit whose key the pool lacks fails loud at resolve (extract has
// no CPS/forensic fetch source), never emits a wrong-key garble.
//
// KNOWN LIMITATION (by design): an orphan encrypted clip on a multi-CPS
// disc — referenced by no playlist, so in no title extent — is in no range
// and passes through as ciphertext. There is no correct key to apply (its
// CPS unit is unknown without a playlist reference), and blind trial-decrypt
// is exactly what this keymap-only model removes. Single-CPS is unaffected
// (the blanket key-0 map above covers orphans).
let key_map =
match &base_keys {
DecryptKeys::Aacs { unit_keys, .. } if unit_keys.len() <= 1 => {
Some(std::sync::Arc::new(
crate::decrypt::AacsKeyMap::from_ranges(vec![(0, u32::MAX, 0)]),
))
}
DecryptKeys::Aacs { .. } => Some(std::sync::Arc::new(
self.resolve_content_key_map(reader, &mut base_keys, None, opts.halt.as_ref())?,
)),
_ => None,
};
// ── Phase 2: stream each file through the decrypting decorator ────
// The decorator owns its inner source for its lifetime. We hand it a
// borrowing wrapper (so the caller keeps `reader`), swap keys per CSS
// VTS group via `set_keys`; AACS/None keep `base_keys` throughout.
let mut dec = DecryptingSectorSource::new(Borrowed(reader), base_keys.clone());
let decrypt_loss = dec.decrypt_loss();
if let Some(map) = key_map {
dec = dec.with_key_map(map);
}
let mut result = ExtractResult::default();
let total_bytes = required;
@@ -232,26 +265,15 @@ impl Disc {
}
}
// Acquire (rather than Relaxed) on these per-file delta loads:
// `extract_tree` drives `dec` single-threaded so there is no race
// today, and Acquire costs nothing on x86. Note this is only half
// the synchronisation: the paired counter store
// (sector/decrypting.rs `fetch_add`) is Relaxed, so an Acquire
// load alone does NOT yet establish a happens-before edge. Before
// file extraction is parallelised, upgrade that store to Release
// (or stronger) so the delta cannot read a stale counter.
let before_loss = decrypt_loss.load(Ordering::Acquire);
let (mut fr, halted) =
// A unit that fails to decrypt fails the read loud (extract runs
// non-tolerate), so extract_one_file already zero-filled it and
// counted it in bytes_unreadable — one 'lost' bucket covers both
// media damage and decrypt failure.
let (fr, halted) =
extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?;
let after_loss = decrypt_loss.load(Ordering::Acquire);
fr.bytes_undecryptable = after_loss.saturating_sub(before_loss);
fr.bytes_good = fr.bytes_good.saturating_sub(fr.bytes_undecryptable);
result.bytes_good = result.bytes_good.saturating_add(fr.bytes_good);
result.bytes_unreadable = result.bytes_unreadable.saturating_add(fr.bytes_unreadable);
result.bytes_undecryptable = result
.bytes_undecryptable
.saturating_add(fr.bytes_undecryptable);
result.files.push(fr);
if halted {
result.halted = true;
@@ -261,7 +283,6 @@ impl Disc {
result.complete = !result.halted
&& result.bytes_unreadable == 0
&& result.bytes_undecryptable == 0
&& result.files.iter().all(|f| f.complete);
Ok(result)
}
@@ -306,63 +327,6 @@ impl Disc {
}
}
/// True for the AACS-encrypted stream files (`.m2ts`, `.ssif`). Every other UDF
/// file is clear (nav / playlists / filesystem) and needs no decrypt verify.
fn is_aacs_clip(name: &str) -> bool {
let lower = name.to_ascii_lowercase();
lower.ends_with(".m2ts") || lower.ends_with(".ssif")
}
/// Enumerate the disc's AACS clip (`.m2ts`/`.ssif`) files as
/// [`crate::disc::verify::ClipLayout`]s for the post-read verify gate: each
/// clip's declared size plus its absolute disc extents in FILE order. Reads the
/// UDF tree through `reader`.
///
/// FAIL-SAFE: any enumeration error (bad UDF read, name collision, …) yields an
/// EMPTY list — the verify gate then covers nothing and the sweep behaves as
/// today. Enumeration must never break a rip, so the error is logged, not
/// propagated.
pub(crate) fn clip_layouts(reader: &mut dyn SectorSource) -> Vec<crate::disc::verify::ClipLayout> {
let result = (|| -> Result<Vec<crate::disc::verify::ClipLayout>> {
let fs = udf::read_filesystem(reader)?;
let mut planned: Vec<PlannedFile> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
let mut seen_hosts: std::collections::HashMap<PathBuf, String> =
std::collections::HashMap::new();
plan_tree(
reader,
&fs,
&fs.root,
Path::new(""),
"",
true,
&mut planned,
&mut dirs,
&mut seen_hosts,
)?;
Ok(planned
.into_iter()
.filter(|pf| pf.inline.is_none() && is_aacs_clip(&pf.disc_name))
.map(|pf| crate::disc::verify::ClipLayout {
size: pf.size,
extents: pf.extents,
// Every AACS clip we enumerate today is BD-TS (`.m2ts`/`.ssif`).
// HD-DVD `.evo` (program stream) maps to `ContainerKind::Ps` here
// once `is_aacs_clip` recognises it — the one-line HD-DVD hook.
container: crate::disc::verify::ContainerKind::Ts,
})
.collect())
})();
result.unwrap_or_else(|e| {
tracing::warn!(
target: "freemkv::verify",
error = %e,
"clip enumeration failed; post-read verify disabled for this pass"
);
Vec::new()
})
}
/// A borrowing `SectorSource` wrapper. Lets the decrypting decorator "own" an
/// inner source for its lifetime while the caller keeps the underlying
/// `&mut dyn SectorSource` (the decorator is a `DecryptingSectorSource<S>`
@@ -424,12 +388,12 @@ fn plan_tree(
let child_rel = host_rel.join(&safe);
let child_disc = format!("{disc_path}/{}", entry.name);
// Collision: two distinct disc paths → same host path.
if let Some(prev) = seen_hosts.insert(child_rel.clone(), child_disc.clone()) {
if prev != child_disc {
return Err(Error::DirNameCollision {
host: child_rel.to_string_lossy().into_owned(),
});
}
if let Some(prev) = seen_hosts.insert(child_rel.clone(), child_disc.clone())
&& prev != child_disc
{
return Err(Error::DirNameCollision {
host: child_rel.to_string_lossy().into_owned(),
});
}
if entry.is_dir {
dirs.push(child_rel.clone());
@@ -492,7 +456,6 @@ fn extract_one_file<S: SectorSource>(
path: pf.host_rel.clone(),
bytes_good: 0,
bytes_unreadable: 0,
bytes_undecryptable: 0,
complete: false,
};
@@ -793,10 +756,11 @@ fn is_windows_reserved(base: &str) -> bool {
}
let up = base.to_ascii_uppercase();
for prefix in ["COM", "LPT"] {
if let Some(rest) = up.strip_prefix(prefix) {
if rest.len() == 1 && matches!(rest.as_bytes()[0], b'1'..=b'9') {
return true;
}
if let Some(rest) = up.strip_prefix(prefix)
&& rest.len() == 1
&& matches!(rest.as_bytes()[0], b'1'..=b'9')
{
return true;
}
}
false
@@ -851,8 +815,11 @@ mod tests {
struct MemDisc {
sectors: HashMap<u32, [u8; 2048]>,
/// Absolute LBAs that fail to read (bad-sector fixture).
/// Absolute LBAs that fail to read (bad-sector fixture → DiscRead).
bad: std::collections::HashSet<u32>,
/// Absolute LBAs whose read fails to DECRYPT (no/wrong key fixture →
/// DecryptFailed), exercising the undecryptable-unit loss path.
decrypt_fail: std::collections::HashSet<u32>,
}
impl MemDisc {
@@ -860,6 +827,7 @@ mod tests {
Self {
sectors: HashMap::new(),
bad: std::collections::HashSet::new(),
decrypt_fail: std::collections::HashSet::new(),
}
}
fn put(&mut self, lba: u32, data: [u8; 2048]) {
@@ -891,6 +859,9 @@ mod tests {
sense: None,
});
}
if self.decrypt_fail.contains(&(lba + i)) {
return Err(Error::DecryptFailed);
}
}
for i in 0..count as u32 {
let off = i as usize * 2048;
@@ -1082,49 +1053,47 @@ mod tests {
s
}
/// Build a clear 6144-byte AACS unit (TS syncs at the 192-byte BD-TS
/// stride) then encrypt it under `unit_key` so `aacs::decrypt_unit`
/// recovers it cleanly (zero decrypt loss). Mirrors the encrypt helper in
/// `sector/decrypting.rs` tests. `tag` distinguishes two units' payloads.
/// Like [`build_two_extent_icb`] but the two extents may have DIFFERENT
/// sector counts — used to exercise the within-extent batch-size
/// arithmetic (`sectors - sector_off`) with a first extent long enough to
/// force a second while-loop iteration.
fn build_two_extent_icb_sized(
sectors_a: u32,
data_lba_a: u32,
sectors_b: u32,
data_lba_b: u32,
) -> [u8; 2048] {
let mut s = [0u8; 2048];
s[0..2].copy_from_slice(&266u16.to_le_bytes()); // Extended File Entry
s[34..36].copy_from_slice(&0u16.to_le_bytes()); // Short AD
let size = (sectors_a as u64 + sectors_b as u64) * SECTOR_BYTES as u64;
s[56..64].copy_from_slice(&size.to_le_bytes()); // info_length
s[208..212].copy_from_slice(&0u32.to_le_bytes()); // l_ea
s[212..216].copy_from_slice(&16u32.to_le_bytes()); // l_ad = 2 Short ADs
let len_a = sectors_a * SECTOR_BYTES as u32;
let len_b = sectors_b * SECTOR_BYTES as u32;
s[216..220].copy_from_slice(&(len_a & 0x3FFF_FFFF).to_le_bytes());
s[220..224].copy_from_slice(&data_lba_a.to_le_bytes());
s[224..228].copy_from_slice(&(len_b & 0x3FFF_FFFF).to_le_bytes());
s[228..232].copy_from_slice(&data_lba_b.to_le_bytes());
s
}
/// Encrypt the clear unit from `clear_aacs_unit(tag)` under `unit_key` so
/// `aacs::content::decrypt_unit` recovers it cleanly (zero decrypt loss).
/// `tag` distinguishes two units' payloads.
fn encrypt_aacs_unit(unit_key: &[u8; 16], tag: u8) -> Vec<u8> {
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
let mut unit = vec![0u8; crate::aacs::ALIGNED_UNIT_LEN];
let mut off = 4;
while off < unit.len() {
unit[off] = 0x47; // TS sync
if off + 1 < unit.len() {
unit[off + 1] = tag; // payload marker so the two extents differ
}
off += 192;
}
// Flag encrypted via CPI bits (byte 0) before key derivation.
unit[0] |= 0xC0;
let header: [u8; 16] = unit[..16].try_into().unwrap();
let derived = crate::aacs::decrypt::aes_ecb_encrypt(unit_key, &header);
let mut k = [0u8; 16];
for i in 0..16 {
k[i] = derived[i] ^ header[i];
}
let cipher = Aes128::new(GenericArray::from_slice(&k));
let mut prev = crate::aacs::decrypt::AACS_IV;
let blocks = (crate::aacs::ALIGNED_UNIT_LEN - 16) / 16;
for i in 0..blocks {
let o = 16 + i * 16;
for j in 0..16 {
unit[o + j] ^= prev[j];
}
let mut blk = GenericArray::clone_from_slice(&unit[o..o + 16]);
cipher.encrypt_block(&mut blk);
unit[o..o + 16].copy_from_slice(&blk);
prev.copy_from_slice(&unit[o..o + 16]);
}
let mut unit = clear_aacs_unit(tag);
assert!(
crate::aacs::content::encrypt_unit(&mut unit, unit_key),
"a full-length unit must encrypt"
);
unit
}
/// The plaintext that `encrypt_aacs_unit(_, tag)` decrypts back to.
fn clear_aacs_unit(tag: u8) -> Vec<u8> {
let mut unit = vec![0u8; crate::aacs::ALIGNED_UNIT_LEN];
let mut unit = vec![0u8; crate::aacs::content::ALIGNED_UNIT_LEN];
let mut off = 4;
while off < unit.len() {
unit[off] = 0x47;
@@ -1412,6 +1381,64 @@ mod tests {
assert_eq!(res.files[0].bytes_unreadable, good.len() as u64);
}
/// An UNDECRYPTABLE unit (DecryptFailed — wrong/missing key) is zero-filled
/// and counted as loss through the public API exactly like a bad sector:
/// the recovery-seam consolidation folded the old bytes_undecryptable bucket
/// into bytes_unreadable, and the run must still report complete == false and
/// bytes_lost() > 0 (this gates the CLI exit code / multipass re-run).
#[test]
fn undecryptable_unit_holes_file_and_accounts_loss() {
let good = vec![0x55u8; 4 * 2048];
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "BDMV".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "STREAM".to_string(),
icb_lba: 22,
dir_data_lba: 23,
files: vec![file("00001.m2ts", 24, 5000, good.clone(), true)],
subdirs: vec![],
}],
}],
};
let mut disc = build_disc(root);
// The whole extent fails to decrypt (no/wrong key) rather than to read.
for i in 0..4u32 {
disc.decrypt_fail.insert(PART_START + 5000 + i);
}
let out = TmpDir::new("decryptfail");
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract does not abort on an undecryptable unit");
let got = read_out(out.path(), "BDMV/STREAM/00001.m2ts").expect("file written");
assert_eq!(
got.len(),
good.len(),
"holed file still sized to declared size"
);
assert!(
got.iter().all(|&b| b == 0),
"undecryptable range zero-filled"
);
assert!(
!res.complete,
"an undecryptable unit makes the rip incomplete"
);
assert!(
res.bytes_lost() > 0,
"decrypt loss counted, not reported clean"
);
assert_eq!(res.bytes_unreadable, good.len() as u64);
assert_eq!(res.files[0].bytes_unreadable, good.len() as u64);
}
/// Path sanitization rejects a host-illegal component in a disc file name.
#[test]
fn sanitize_rejects_illegal_component() {
@@ -1585,10 +1612,6 @@ mod tests {
res.bytes_unreadable, 0,
"per-extent unit base must keep the second extent off the hole path"
);
assert_eq!(
res.bytes_undecryptable, 0,
"clear units decrypt-restore clean"
);
assert!(
res.complete,
"a clean multi-extent AACS file extracts complete"
@@ -1600,10 +1623,10 @@ mod tests {
/// own batch starts are always unit-aligned; anchoring a later extent
/// against the FIRST extent's base mis-aligns whenever the extents' starts
/// differ by a non-multiple of 3 sectors. This is the exact arithmetic the
/// decrypt-on-read gate (`aacs::is_unit_aligned`) performs.
/// decrypt-on-read gate (`aacs::content::is_unit_aligned`) performs.
#[test]
fn per_extent_base_is_aligned_first_extent_base_is_not() {
use crate::aacs::is_unit_aligned;
use crate::aacs::content::is_unit_aligned;
let ext_a_start = 7000u32; // first extent abs LBA
let ext_b_start = 7004u32; // second extent abs LBA (Δ4 — not mult of 3)
@@ -1658,4 +1681,462 @@ mod tests {
assert_eq!(read_out(out.path(), "tiny.inf"), Some(payload));
assert!(res.complete);
}
// ── Mutation-triage additions ───────────────────────────────────────────
/// `cancelled` must stop on EITHER signal alone (a disjunction) — a
/// progress sink asking to stop must cancel even with no halt token, and a
/// cancelled halt token must cancel even when progress says continue.
#[test]
fn cancelled_stops_on_either_signal_alone() {
let opts_no_halt = ExtractOptions::default();
assert!(
opts_no_halt.cancelled(false),
"a progress sink asking to stop must cancel even with no halt token"
);
assert!(
!opts_no_halt.cancelled(true),
"neither signal firing must not cancel"
);
let halt = crate::halt::Halt::new();
halt.cancel();
let opts_halted = ExtractOptions {
halt: Some(halt),
..Default::default()
};
assert!(
opts_halted.cancelled(true),
"a cancelled halt token must cancel even when progress says continue"
);
}
/// The free-space pre-check must fire whenever the disc's declared total
/// exceeds real available space. An absurdly large declared size (an
/// exabyte) trips it regardless of the actual free space on whatever
/// machine runs the test.
#[test]
fn insufficient_space_errors_on_absurdly_large_required() {
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: vec![file("huge.bin", 30, 31, Vec::new(), false)],
subdirs: vec![],
};
let mut disc = build_disc(root);
// Overwrite the declared size (info_length) with an absurd value; the
// extent length stays 0 so no real content is ever read (the space
// gate runs before Phase 2 touches content).
let mut icb = build_file_icb(0, 31, false);
let huge: u64 = 1u64 << 60; // ~1 exabyte -- no real disk has this free
icb[56..64].copy_from_slice(&huge.to_le_bytes());
disc.put(PART_START + 30, icb);
let out = TmpDir::new("insufficient_space");
let err = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect_err("an absurdly large declared size must trip the space gate");
assert!(matches!(err, Error::DirInsufficientSpace { .. }));
}
/// Regression: the per-VTS key crack in `resolve_vts_key` must gather ONLY
/// this VTS's own title-VOB extents. Two VTS groups here carry DISTINCT
/// scrambling keys; if the group filter (`vts_group_of(..) == Some(vts) &&
/// is_title_vob(..)`) is loosened (`!=`, or `&&` -> `||`), one VTS's
/// resolve gathers the OTHER VTS's extents, cracks the wrong key, and that
/// VTS's own content silently fails to descramble to the expected
/// plaintext.
#[test]
fn css_two_vts_groups_do_not_cross_contaminate_keys() {
fn scrambled_vob(title_key: [u8; 5], marker: u8) -> (Vec<u8>, Vec<u8>) {
let seed = [0x11u8, 0x22, 0x33, 0x44, marker];
let mut plain = vec![0u8; 2048];
// The crack scan's hardened `is_scrambled_pack` gate requires the
// MPEG-PS pack-start signature before it will even ATTEMPT a
// Stevenson crack (see `css::is_scrambled_pack`) -- without it,
// `resolve_vts_key` silently falls back to `base_keys` for BOTH
// VTS groups regardless of which extents were gathered, masking
// this exact regression.
plain[0x00..0x04].copy_from_slice(&crate::css::PACK_START);
plain[0x14] = 0x10; // scramble flag
let pat: Vec<u8> = (0..8)
.map(|k| (0xA0u8.wrapping_add(k as u8) ^ marker) ^ 0x5A)
.collect();
for (i, b) in plain.iter_mut().enumerate().skip(0x59) {
*b = pat[i % 8];
}
plain[0x54..0x59].copy_from_slice(&seed);
let mut scrambled = plain.clone();
lfsr::scramble_sector(&title_key, &mut scrambled);
(plain, scrambled)
}
let key_1 = [0x10u8, 0x20, 0x30, 0x40, 0x50];
let key_2 = [0x90u8, 0x80, 0x70, 0x60, 0x51];
let (plain_1, scrambled_1) = scrambled_vob(key_1, 0x01);
let (plain_2, scrambled_2) = scrambled_vob(key_2, 0x02);
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "VIDEO_TS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("VTS_01_1.VOB", 30, 5000, scrambled_1.clone(), false),
file("VTS_02_1.VOB", 32, 6000, scrambled_2.clone(), false),
],
subdirs: vec![],
}],
};
let mut disc = build_disc(root);
let out = TmpDir::new("css_two_vts");
let mut d = clear_disc();
d.content_format = crate::disc::ContentFormat::MpegPs;
// Disc-wide key deliberately matches NEITHER VTS's real key: if the
// per-VTS group filter is broken and a crack attempt fails (or is
// skipped), falling back to this key must still mismatch, so the
// fallback path can never accidentally mask a broken filter.
d.css = Some(crate::css::CssState {
title_key: [0xFFu8; 5],
crack_span: None,
});
let res = d
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract");
let got_1 = read_out(out.path(), "VIDEO_TS/VTS_01_1.VOB").expect("vts01 vob");
let got_2 = read_out(out.path(), "VIDEO_TS/VTS_02_1.VOB").expect("vts02 vob");
let mut expect_1 = plain_1.clone();
expect_1[0x14] = 0x00;
let mut expect_2 = plain_2.clone();
expect_2[0x14] = 0x00;
assert_eq!(
got_1, expect_1,
"VTS_01 must descramble under its OWN cracked key, not VTS_02's"
);
assert_eq!(
got_2, expect_2,
"VTS_02 must descramble under its OWN cracked key, not VTS_01's"
);
assert!(res.complete);
}
/// `Borrowed` is a thin forwarding wrapper the decrypting decorator uses
/// to avoid taking ownership of the caller's reader -- every
/// `SectorSource` method must forward to the wrapped `&mut dyn
/// SectorSource` verbatim. Calling directly on a concrete `Borrowed`
/// value (not through `&mut dyn SectorSource`) exercises the actual
/// forwarding body via static dispatch, not vtable dispatch through a
/// trait object.
#[test]
fn borrowed_forwards_every_sector_source_call() {
struct Recorder {
capacity: u32,
last_speed: Option<u16>,
last_unit_base: Option<u32>,
}
impl SectorSource for Recorder {
fn capacity_sectors(&self) -> u32 {
self.capacity
}
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Ok(0)
}
fn set_speed(&mut self, kbs: u16) {
self.last_speed = Some(kbs);
}
fn set_unit_base(&mut self, lba: u32) {
self.last_unit_base = Some(lba);
}
}
let mut inner = Recorder {
capacity: 42,
last_speed: None,
last_unit_base: None,
};
{
let mut b = Borrowed(&mut inner);
assert_eq!(b.capacity_sectors(), 42, "capacity_sectors must forward");
b.set_speed(7200);
b.set_unit_base(1234);
}
assert_eq!(inner.last_speed, Some(7200), "set_speed must forward");
assert_eq!(
inner.last_unit_base,
Some(1234),
"set_unit_base must forward"
);
}
/// Regression: within ONE extent, a batch's "sectors remaining IN THIS
/// EXTENT" must be computed as `sectors - sector_off`, not `sectors +
/// sector_off`. The latter inflates without bound as the loop advances,
/// letting a later batch's read run PAST this extent's true end into
/// whatever content sits at the following LBAs (an unrelated disc region
/// in this fixture) and get written into the file as if it were this
/// extent's own data -- untrusted-disc content bleed across extent
/// boundaries.
#[test]
fn extent_second_batch_stays_within_its_own_bounds() {
// Extent A: 1600 sectors of pattern 'A' -- just over
// READ_BATCH_SECTORS (1536), forcing a second while-loop iteration
// with sector_off > 0.
const SECTORS_A: u32 = 1600;
const LBA_A: u32 = 5000;
// The disc region immediately following extent A's true end. Must
// NEVER be read as part of extent A: sized to cover a full erroneous
// READ_BATCH_SECTORS second batch starting right after extent A's
// real tail.
const LBA_FILLER: u32 = LBA_A + SECTORS_A;
const SECTORS_FILLER: u32 = 1472;
// Extent B: the file's real second extent, at a completely different
// LBA, same size as the filler region so the two are exact substitutes
// if the arithmetic bug reads the wrong one.
const SECTORS_B: u32 = SECTORS_FILLER;
const LBA_B: u32 = 90_000;
let a = vec![0xAAu8; SECTORS_A as usize * SECTOR_BYTES];
let filler = vec![0xCCu8; SECTORS_FILLER as usize * SECTOR_BYTES];
let b = vec![0xBBu8; SECTORS_B as usize * SECTOR_BYTES];
let mut disc = MemDisc::new();
build_udf_skeleton(&mut disc, 10);
disc.put_bytes(PART_START + LBA_A, &a);
disc.put_bytes(PART_START + LBA_FILLER, &filler);
disc.put_bytes(PART_START + LBA_B, &b);
disc.put(
PART_START + 30,
build_two_extent_icb_sized(SECTORS_A, LBA_A, SECTORS_B, LBA_B),
);
let mut root_fids = Vec::new();
push_fid(&mut root_fids, "", 10, true, true);
push_fid(&mut root_fids, "big.bin", 30, false, false);
disc.put(PART_START + 10, build_dir_icb(11, root_fids.len() as u32));
disc.put_bytes(PART_START + 11, &root_fids);
let out = TmpDir::new("extent_bounds");
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &ExtractOptions::default())
.expect("extract");
let got = read_out(out.path(), "big.bin").expect("file written");
let mut expect = a.clone();
expect.extend_from_slice(&b);
assert_eq!(
got.len(),
expect.len(),
"file size matches the two extents' declared total"
);
assert_eq!(
got, expect,
"extent A's tail batch must not read past its own declared length \
into the following disc region (pattern 'C' must never appear)"
);
assert!(res.complete);
assert_eq!(res.bytes_unreadable, 0);
}
/// Pure-function coverage of `whole_unit_batch`'s cap: a batch that is
/// NOT the extent's final chunk (there is more remaining after it) is
/// capped at `READ_BATCH_SECTORS`, which is itself an exact multiple of
/// `AACS_UNIT_SECTORS` (1536 = 512 * 3) -- so the result is already
/// unit-aligned.
#[test]
fn whole_unit_batch_caps_mid_stream_batches() {
assert_eq!(whole_unit_batch(2000), READ_BATCH_SECTORS);
assert_eq!(whole_unit_batch(READ_BATCH_SECTORS + 1), READ_BATCH_SECTORS);
assert_eq!(whole_unit_batch(READ_BATCH_SECTORS + 2), READ_BATCH_SECTORS);
}
/// Pure-function coverage of `whole_unit_batch`'s tail contract: the
/// FINAL chunk of an extent (`batch == remaining`, i.e. `remaining <=
/// READ_BATCH_SECTORS`) must NOT be rounded down to a unit boundary, even
/// when it is not itself a multiple of 3 -- `decrypt_sectors`'s
/// trailing-partial contract handles a non-multiple tail specially, and
/// rounding it here would silently drop sectors from the read.
#[test]
fn whole_unit_batch_true_tail_is_never_rounded() {
assert_eq!(whole_unit_batch(5), 5);
assert_eq!(whole_unit_batch(1535), 1535);
assert_eq!(whole_unit_batch(2), 2);
assert_eq!(whole_unit_batch(1), 1);
assert_eq!(whole_unit_batch(READ_BATCH_SECTORS), READ_BATCH_SECTORS);
}
/// `read_batch` must retry a non-decrypt read failure up to
/// `READ_RETRIES` times and succeed if a later attempt does -- a
/// transient failure must not be treated as permanent on the very first
/// try.
#[test]
fn read_batch_retries_transient_failures_and_succeeds() {
struct FlakySource {
fail_first: u32,
calls: u32,
}
impl SectorSource for FlakySource {
fn capacity_sectors(&self) -> u32 {
100_000
}
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
self.calls += 1;
if self.calls <= self.fail_first {
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
let need = count as usize * SECTOR_BYTES;
buf[..need].fill(0x11);
Ok(need)
}
}
// Fails exactly READ_RETRIES times (attempts 0..READ_RETRIES all
// error), then succeeds on the FINAL attempt (attempt ==
// READ_RETRIES) -- the last chance the retry budget allows.
let src = FlakySource {
fail_first: READ_RETRIES,
calls: 0,
};
let mut dec = DecryptingSectorSource::new(src, DecryptKeys::None);
let mut buf = vec![0u8; 2 * SECTOR_BYTES];
let ok = read_batch(&mut dec, 0, 2, &mut buf);
assert!(
ok,
"a failure that clears up within the retry budget must succeed, not hole"
);
assert_eq!(dec.inner().calls, READ_RETRIES + 1);
}
/// `report` must reflect the sink's verdict -- a sink asking to stop must
/// actually halt the run mid-file, not be swallowed.
#[test]
fn progress_sink_stop_halts_run_mid_file() {
struct StopImmediately;
impl crate::progress::Progress for StopImmediately {
fn report(&self, _p: &crate::progress::PassProgress) -> bool {
false
}
}
let good = vec![0x66u8; 4 * 2048];
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "BDMV".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "STREAM".to_string(),
icb_lba: 22,
dir_data_lba: 23,
files: vec![file("00001.m2ts", 24, 5000, good, true)],
subdirs: vec![],
}],
}],
};
let mut disc = build_disc(root);
let out = TmpDir::new("progress_stop");
let sink = StopImmediately;
let opts = ExtractOptions {
progress: Some(&sink),
..Default::default()
};
let res = clear_disc()
.extract_tree(&mut disc, out.path(), &opts)
.expect("extract does not error on a progress halt");
assert!(res.halted, "a sink returning false must halt the run");
assert!(
!res.files[0].complete,
"the in-flight file must be left incomplete, not finalized"
);
assert!(
read_out(out.path(), "BDMV/STREAM/00001.m2ts").is_none(),
"an incomplete file must not be renamed to its final name"
);
}
/// `available_space` must return the real free-space figure (`Some`) for
/// an existing directory on a platform that exposes it -- the free-space
/// pre-check in `extract_tree` silently no-ops whenever this returns
/// `None`, so a `statvfs` SUCCESS (`rc == 0`) must never be read as
/// "unavailable".
#[cfg(unix)]
#[test]
fn available_space_reports_free_bytes_on_a_real_dir() {
let out = TmpDir::new("available_space");
std::fs::create_dir_all(out.path()).unwrap();
assert!(
available_space(out.path()).is_some(),
"a real, existing directory must report Some(_) free bytes"
);
}
/// A raw control byte (below 0x20, distinct from the separately-rejected
/// NUL) in a disc-authored name must be rejected, not passed through into
/// the host filename.
#[test]
fn sanitize_rejects_control_bytes() {
assert!(
sanitize_component("a\u{1}b").is_err(),
"0x01 must be rejected"
);
assert!(
sanitize_component("a\nb").is_err(),
"0x0A (newline) must be rejected"
);
assert!(
sanitize_component("a\u{1f}b").is_err(),
"0x1F must be rejected"
);
// 0x20 (space) is NOT a control byte -- allowed mid-name (only
// trimmed if trailing).
assert!(sanitize_component("a b").is_ok());
}
/// The VTS group number must be EXACTLY 2 ASCII digits -- neither a
/// non-numeric group nor a wrong-length one is a valid `VTS_xx` group.
#[test]
fn vts_group_of_requires_exactly_two_digits() {
assert_eq!(
vts_group_of("VTS_AB_1.VOB"),
None,
"letters are not a group number"
);
assert_eq!(
vts_group_of("VTS_123_1.VOB"),
None,
"a 3-digit group is not a valid 2-digit VTS number"
);
assert_eq!(
vts_group_of("VTS_1_1.VOB"),
None,
"a 1-digit group is not valid"
);
assert_eq!(vts_group_of("VTS_01_1.VOB").as_deref(), Some("VTS_01"));
}
}
+1914
View File
File diff suppressed because it is too large Load Diff
-1670
View File
File diff suppressed because it is too large Load Diff
+3508 -2924
View File
File diff suppressed because it is too large Load Diff
-1705
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-232
View File
@@ -1,232 +0,0 @@
//! `Disc::sweep`'s consumer-side `Sink<WorkItem>`.
//!
//! Background: the original sweep loop runs strictly serialised —
//! SCSI read → decrypt → seek + write → mapfile.record → next iter.
//! On a healthy disc the SCSI read costs ~5-12 ms per 64 KB batch and
//! the post-read work (decrypt 1-3 ms + file write + mapfile fsync
//! 5-15 ms) adds another batch's worth of latency. The drive idles
//! during the post-read work; throughput tops out at the *sum* of
//! both costs.
//!
//! A producer/consumer split overlaps the two stages on the generic
//! [`crate::io::Pipeline`] + [`crate::io::Sink`] primitive. This module
//! is the sweep-specific `Sink` impl; the producer-side state machine
//! (read_error context, decrypt, set_speed, halt) stays in
//! `Disc::sweep` in `disc/mod.rs`.
//!
//! Correctness invariants preserved:
//! - Mapfile is single-writer (consumer-only). No locking.
//! - All `read_error::ReadCtx` state stays on the producer thread.
//! - `set_speed` calls happen on the producer thread (same thread that
//! owns the `SectorSource`). No new SCSI concurrency.
//! - Per-iteration ordering of file-write → mapfile-record is kept
//! intact in the consumer (write before record), so the on-disk
//! invariant "mapfile only marks Finished what the file has
//! received" survives a crash mid-pass.
//! - Only one SCSI command is in flight at a time; error-path timing
//! is identical and no new retry logic is introduced.
use std::io::{Seek, SeekFrom, Write};
use std::sync::mpsc::{Receiver, SyncSender, sync_channel};
use crate::error::Error;
use crate::io::{Flow, Sink};
use super::mapfile::{MapStats, Mapfile, SectorStatus};
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. 64 KB
/// matches the existing zero_gap chunk size used by the pre-split
/// sweep loop.
const ZERO_CHUNK: usize = 64 * 1024;
/// Producer → Consumer messages. The consumer applies these in FIFO
/// order; ordering of file writes and mapfile records across items is
/// preserved.
pub(super) enum WorkItem {
/// Successful batch read. Producer has already decrypted `buf` if
/// `opts.decrypt` was set. Consumer writes `buf` at `pos` and
/// records the range as `Finished`.
Good { pos: u64, buf: Vec<u8> },
/// Bisect inner-loop good single sector (already decrypted by the
/// producer). 2048 bytes.
BisectGood { pos: u64, buf: Box<[u8; 2048]> },
/// Bisect inner-loop bad single sector. Consumer writes 2048
/// zeros at `pos` and records the sector as `NonTrimmed`.
BisectBad { pos: u64 },
/// Whole-batch zero-fill (failed batch on `SkipBlock`, or the
/// failed batch portion of `JumpAhead`). Consumer streams zeros
/// across `[pos, pos+len)` and records the range as `NonTrimmed`.
SkipFill { pos: u64, len: u64 },
/// Gap fill following a `JumpAhead`. Same effect as `SkipFill`;
/// distinguished only so future logging / instrumentation can
/// tell them apart without parsing a flag.
GapFill { pos: u64, len: u64 },
/// Post-read verify downgrade. The producer's `UnitVerifier` found that the
/// just-`Finished` clip unit at `[pos, pos+len)` is confidently undecryptable
/// (a silent bad read). The consumer re-records the range as `NonTrimmed` so
/// the patch pass re-reads it — the ISO bytes (ciphertext) already written by
/// the preceding `Good` are left in place for the patch to overwrite. FIFO
/// pipe ordering guarantees this arrives AFTER the `Good` that wrote them.
MarkBad { pos: u64, len: u64 },
/// Producer wants the latest mapfile stats for the progress
/// callback. Consumer responds on `prog_tx` with a fresh
/// [`ProgressSnapshot`]. Best-effort: if the producer hasn't
/// drained the previous snapshot, the new one is silently
/// dropped — the producer's local cache stays current enough.
StatsRequest,
}
/// Snapshot the consumer sends back to the producer for the progress
/// callback.
pub(super) struct ProgressSnapshot {
pub stats: MapStats,
pub bad_ranges: Vec<(u64, u64)>,
}
/// Final summary returned by the consumer thread on shutdown — what
/// `SweepSink::close` produces, surfaced to the producer via
/// `Pipeline::finish`.
pub(super) struct ConsumerSummary {
pub stats: MapStats,
}
/// Drain any pending progress snapshots from the consumer. Returns
/// the most recent one, if any. The producer caches it and uses it
/// for subsequent progress callbacks until a fresh one arrives.
pub(super) fn try_recv_progress(rx: &Receiver<ProgressSnapshot>) -> Option<ProgressSnapshot> {
let mut latest = None;
while let Ok(snap) = rx.try_recv() {
latest = Some(snap);
}
latest
}
/// `Sink<WorkItem>` for sweep. Owns the writeback file + mapfile +
/// progress back-channel. `apply` carries the file-write +
/// mapfile.record per item; `close` drains the writeback pipeline,
/// fsyncs the ISO, and flushes the mapfile.
pub(super) struct SweepSink {
file: crate::io::WritebackFile,
map: Mapfile,
/// `sync_all`-on-failure-is-an-error iff the output is a regular
/// file. `/dev/null` and pipes always fail `sync_all`; that's not
/// a real error.
is_regular: bool,
/// Back-channel for `StatsRequest` responses. The producer caches
/// the latest snapshot and uses it for the progress callback;
/// dropped sends on a full channel are by design.
prog_tx: SyncSender<ProgressSnapshot>,
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. Held
/// in the sink so each apply call doesn't reallocate.
zero: Box<[u8; ZERO_CHUNK]>,
}
impl SweepSink {
/// Construct a new `SweepSink` plus the matching progress
/// receiver. Channel depth on the back-channel is `1` — the
/// producer's cache is the source of truth between snapshots.
pub(super) fn new(
file: crate::io::WritebackFile,
map: Mapfile,
is_regular: bool,
) -> (Self, Receiver<ProgressSnapshot>) {
let (prog_tx, prog_rx) = sync_channel::<ProgressSnapshot>(1);
let sink = SweepSink {
file,
map,
is_regular,
prog_tx,
zero: Box::new([0u8; ZERO_CHUNK]),
};
(sink, prog_rx)
}
}
impl Sink<WorkItem> for SweepSink {
type Output = ConsumerSummary;
fn apply(&mut self, item: WorkItem) -> Result<Flow, Error> {
match item {
WorkItem::Good { pos, buf } => {
// Decrypt is on the producer; consumer assumes plaintext.
let len = buf.len() as u64;
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&buf)?;
self.map.record(pos, len, SectorStatus::Finished)?;
}
WorkItem::BisectGood { pos, buf } => {
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&buf[..])?;
self.map.record(pos, 2048, SectorStatus::Finished)?;
}
WorkItem::BisectBad { pos } => {
self.file.seek(SeekFrom::Start(pos))?;
self.file.write_all(&self.zero[..2048])?;
self.map.record(pos, 2048, SectorStatus::NonTrimmed)?;
}
WorkItem::SkipFill { pos, len } | WorkItem::GapFill { pos, len } => {
self.file.seek(SeekFrom::Start(pos))?;
// Subsequent writes are sequential; `WritebackFile`'s
// seek-elision keeps them on the writeback pipeline path.
let mut filled = 0u64;
while filled < len {
let chunk = (len - filled).min(self.zero.len() as u64) as usize;
self.file.write_all(&self.zero[..chunk])?;
filled += chunk as u64;
}
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::MarkBad { pos, len } => {
// Verify downgrade: the ISO bytes are already written by the
// preceding Good; only the mapfile status changes so patch
// re-reads this range. No file write.
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::StatsRequest => {
let stats = self.map.stats();
// DAMAGE only — NOT NonTried. NonTried is the unread remainder
// ahead of the sweep head, not damage; including it made the live
// located drilldown (at-risk movie time + range count) treat the
// whole unread disc as confirmed damage, so at sweep start it
// showed ~full-movie at-risk and melted to 0 as the sweep
// progressed. Matches the one-shot progress path, which already
// excludes NonTried.
let bad_ranges = self.map.ranges_with(&[
SectorStatus::NonTrimmed,
SectorStatus::Unreadable,
SectorStatus::NonScraped,
]);
// Best-effort: drop on backpressure; producer's cache
// stays current enough.
let _ = self
.prog_tx
.try_send(ProgressSnapshot { stats, bad_ranges });
}
}
Ok(Flow::Continue)
}
fn close(mut self) -> Result<Self::Output, Error> {
// Drain the writeback pipeline + fsync the ISO, then persist
// any pending mapfile state. Same finalisation order as the
// pre-Pipeline consumer loop.
if let Err(e) = self.file.sync_all() {
if self.is_regular {
return Err(Error::IoError { source: e });
}
// Non-regular outputs (/dev/null, pipes) always fail
// sync_all; that's not a real error.
}
self.map.flush()?;
Ok(ConsumerSummary {
stats: self.map.stats(),
})
}
}
-1021
View File
File diff suppressed because it is too large Load Diff
+6 -8
View File
@@ -23,14 +23,12 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
if !std::path::Path::new(&path).exists() {
continue;
}
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path)) {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((path, id));
}
}
if let Ok(mut transport) = crate::scsi::open(std::path::Path::new(&path))
&& let Ok(id) = DriveId::from_drive(transport.as_mut())
&& !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((path, id));
}
}
drives
+35 -6
View File
@@ -25,12 +25,11 @@ pub fn find_drives() -> Vec<(String, DriveId)> {
let path = std::path::Path::new(&info.path);
match crate::scsi::open(path) {
Ok(mut transport) => {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((info.path.clone(), id));
}
if let Ok(id) = DriveId::from_drive(transport.as_mut())
&& !id.raw_inquiry.is_empty()
&& (id.raw_inquiry[0] & 0x1F) == SCSI_PERIPHERAL_TYPE_OPTICAL
{
drives.push((info.path.clone(), id));
}
}
Err(_) => {
@@ -53,3 +52,33 @@ pub fn resolve_device(path: &str) -> Result<(String, DeviceResolution)> {
}
Ok((path.to_string(), DeviceResolution::Direct))
}
#[cfg(test)]
mod resolve_device_tests {
use super::*;
/// An existing path resolves unchanged as `Direct` — macOS has no
/// `sr`->`sg` substitution, so the returned path must be byte-identical
/// to the input, not some canonicalised/mutated form.
#[test]
fn existing_path_resolves_direct_unchanged() {
// Use the test binary's own executable path: guaranteed to exist,
// no fixture file needed.
let exe = std::env::current_exe().unwrap();
let path = exe.to_str().unwrap();
let (resolved, kind) = resolve_device(path).expect("existing path must resolve");
assert_eq!(resolved, path, "path must be returned unchanged");
assert_eq!(kind, DeviceResolution::Direct);
}
/// A path that does not exist must error with `DeviceNotFound` carrying
/// the original path, never silently succeed.
#[test]
fn missing_path_is_device_not_found() {
let path = "/dev/freemkv-definitely-does-not-exist-0xdead";
match resolve_device(path) {
Err(Error::DeviceNotFound { path: p }) => assert_eq!(p, path),
other => panic!("expected DeviceNotFound, got {other:?}"),
}
}
}
+833 -83
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -6,7 +6,7 @@
//!
//! Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against libdvdnav's decoder.
//! verified against real discs.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
+5 -5
View File
@@ -2,7 +2,7 @@
//!
//! An 8-byte navigation command as found in PGC command tables (pre/post/cell)
//! and PCI button info. Decoded per the DVD-Video VM instruction set and
//! verified against libdvdnav's command decoder.
//! verified against real discs.
//!
//! Bit model: the 8 bytes are a big-endian 64-bit word. `byte0` bits 7-5 are the
//! command **type**; for type 1, `byte0` bit 4 selects Link (0) vs Jump (1), and
@@ -133,7 +133,7 @@ const JP_JUMP_SS: u8 = 6;
const JP_CALL_SS: u8 = 8;
// Link (type 1, direct=0) sub-commands. NOTE: sub-op 0 is NOP/no-link and 1 is
// the LinkSub form (libdvdnav `decoder.c` `eval_link_instruction`).
// the LinkSub form (the DVD-Video VM link instruction).
const LK_SUB: u8 = 1;
const LK_PGCN: u8 = 4;
const LK_PTTN: u8 = 5;
@@ -159,7 +159,7 @@ fn be16(b: &[u8; 8], o: usize) -> u16 {
((b[o] as u16) << 8) | b[o + 1] as u16
}
// Compare-operand layouts ("if_version"s) per libdvdnav `decoder.c`. The op
// Compare-operand layouts ("if_version"s) per the DVD-Video VM. The op
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
@@ -205,7 +205,7 @@ pub fn decode(b: &[u8; 8]) -> Command {
let cmd = b[1] & 0x0F;
// Compare predicate, with the operand layout for this command family
// (libdvdnav `decoder.c` `vm_eval_command` type dispatch).
// (the DVD-Video VM command type dispatch).
let compare = match (typ, direct) {
(TYPE_SPECIAL, _) => if_v1(b),
(TYPE_LINK_JUMP, 1) => if_v2(b), // jump
@@ -372,7 +372,7 @@ mod tests {
}
}
// Regression for the libdvdnav cross-check: link sub-op 0 = NOP, 1 = LinkSub.
// Regression for the link sub-op decode: 0 = NOP, 1 = LinkSub.
#[test]
fn link_subop_zero_is_nop_one_is_linksub() {
assert_eq!(decode(&h("2000000000000000")).instr, Instr::Nop);
+498 -1
View File
@@ -59,7 +59,9 @@ pub const E_MKV_INVALID: u16 = 6008;
pub const E_NO_STREAMS: u16 = 6009;
pub const E_HALTED: u16 = 6010;
pub const E_MAPFILE_INVALID: u16 = 6011;
pub const E_SELECTION_PID_UNKNOWN: u16 = 6014;
pub const E_UDF_BUFFER_TOO_SMALL: u16 = 6012;
pub const E_UDF_NOT_FILESYSTEM: u16 = 6013;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -88,6 +90,36 @@ pub const E_NO_DISC_KEY: u16 = 7022;
pub const E_CSS_KEY_MISSING: u16 = 7023;
pub const E_AACS_NO_HOST_CERT: u16 = 7024;
pub const E_AACS_BUS_KEY_UNAVAILABLE: u16 = 7025;
pub const E_FMTS_KEY_MISSING: u16 = 7026;
/// The CSS disc as a WHOLE could not be decrypted — the scan saw scrambled
/// sectors and the known-plaintext crack recovered no title key at all, so every
/// title will fail identically. The CSS analogue of [`E_NO_DISC_KEY`], and
/// deliberately NOT [`E_CSS_KEY_MISSING`], which [`is_skippable_title_stub`]
/// treats as one skippable per-title stub: while both conditions shared
/// `E_CSS_KEY_MISSING`, an uncrackable CSS disc was iterated title by title,
/// each one logged as skipped, and the run exited reporting success — a total
/// failure reported as success. [`is_disc_level_no_key`] classifies this code, so
/// a multi-title rip loop fails fast on it.
pub const E_CSS_NO_DISC_KEY: u16 = 7027;
/// A key SOURCE could not be reached, or failed on its own side — transport
/// error, DNS failure, timeout, TLS failure, an HTTP 5xx, or a reply the client
/// could not read. The source never got as far as answering the question, so
/// nothing at all is known about whether a key for this disc exists.
///
/// Deliberately NOT [`E_NO_DISC_KEY`], which asserts the OPPOSITE — every source
/// answered and none holds a key. A seven-hour run of HTTP 502s reported as
/// `E_NO_DISC_KEY` told operators their disc was not in the key database and sent
/// them hunting for a VUK that was never missing; the correct action was to wait.
/// Transient: retry later.
pub const E_KEY_SERVICE_UNAVAILABLE: u16 = 7028;
/// A key source rejected the configured credentials (HTTP 401/403 from the online
/// key service). NOT transient and NOT an absent key — the operator action is to
/// fix the token, not to wait and not to look for a VUK.
pub const E_KEY_SERVICE_UNAUTHORIZED: u16 = 7029;
/// A key source rate-limited the request (HTTP 429 from the online key service).
/// The operator action is to back off and retry more slowly; the disc's key may
/// well exist.
pub const E_KEY_SERVICE_RATE_LIMITED: u16 = 7030;
// Keydb (8xxx)
pub const E_KEYDB_CONNECT: u16 = 8000;
@@ -148,9 +180,69 @@ pub const E_NETWORK_ADDR_BLOCKED: u16 = 9022;
/// zero-frame mux (undecryptable input, fully-unreadable title, every
/// frame dropped before the first keyframe) cannot report success.
pub const E_MUX_EMPTY: u16 = 9023;
/// The mux driver buffered past its pre-headers cap without every video track's
/// `codec_private` resolving. Deliberately NOT [`E_MKV_INVALID`], which
/// [`is_skippable_title_stub`] treats as a skippable empty nav/menu stub — a
/// cap-overflow is a real title and must never be silently skipped.
pub const E_MUX_HEADER_BUFFER_EXCEEDED: u16 = 9051;
/// An `mkv://` SOURCE Block declared lacing (RFC 9559 §10.3) whose header does
/// not describe its own payload, so the frame boundaries inside the Block are
/// unknowable. Deliberately NOT [`E_MKV_INVALID`], which
/// [`is_skippable_title_stub`] treats as a skippable empty nav/menu stub: a
/// laced Block belongs to a track with real media in it, and mis-reporting the
/// rejection as a stub would drop that media from a run that then exits
/// successfully — the same conflation [`E_MUX_HEADER_BUFFER_EXCEEDED`] exists to
/// avoid.
pub const E_MKV_LACING_INVALID: u16 = 9052;
/// An `mkv://` SOURCE file is malformed or truncated — the EBML/Matroska reader
/// rejected it (bad element ID or VINT size, an unknown-size element where a
/// finite one is required, a child overrunning its parent, a truncated element
/// body, a non-UTF-8 string element, an element size above the parser's
/// allocation caps, an out-of-range TimestampScale / cluster timestamp / track
/// number).
///
/// The read-path counterpart of [`E_MP4_INVALID`], and deliberately NOT
/// [`E_MKV_INVALID`]: [`is_skippable_title_stub`] classifies `E_MKV_INVALID` as
/// a title that yielded no muxable frames, which an all-titles rip may skip
/// while finishing the rest. A corrupt or truncated input file is a FAILURE, not
/// a stub — reporting it as skippable would let a broken source be silently
/// passed over by a run that then exits successfully. The same conflation
/// [`E_MUX_HEADER_BUFFER_EXCEEDED`] and [`E_MKV_LACING_INVALID`] exist to avoid.
pub const E_MKV_SOURCE_INVALID: u16 = 9053;
/// The Matroska WRITER was asked to emit something EBML cannot represent: an
/// element body at or above the 56-bit VINT payload limit (which would encode
/// byte-for-byte as the reserved "unknown size" marker or not fit at all), or a
/// master-element size placeholder that no longer lies inside the buffer being
/// patched. An output-side limit, not a property of any input — so neither
/// [`E_MKV_INVALID`] (a no-frames stub) nor [`E_MKV_SOURCE_INVALID`] (a corrupt
/// source) describes it, and it must not be classified as skippable.
pub const E_MKV_UNENCODABLE: u16 = 9054;
pub const E_EXTENT_NOT_UNIT_ALIGNED: u16 = 9030;
/// `mp4://` output but the title has no (primary) video track to carry.
pub const E_MP4_NO_VIDEO_TRACK: u16 = 9048;
/// `mp4://` SOURCE file is malformed/truncated (bad box structure, sample table,
/// or offsets) — the MP4 demuxer could not parse it.
pub const E_MP4_INVALID: u16 = 9049;
/// `mp4://` video track is missing its codec-configuration record
/// (`hvcC`/`avcC`), without which the sample entry can't be written.
pub const E_MP4_MISSING_CODEC_PRIVATE: u16 = 9050;
/// `mp4://` video track has no resolved frame dimensions. ISO/IEC 14496-12
/// makes width and height mandatory in both `tkhd` (8.3.2) and
/// VisualSampleEntry (12.1.3), so unlike Matroska there is no element to omit:
/// the sink would have to write 0x0, producing a structurally complete file no
/// player can render. Refuse instead.
pub const E_MP4_UNKNOWN_RESOLUTION: u16 = 9055;
/// The bounded durable flush did not complete within its deadline. The data is
/// NOT known to be on stable storage; the kernel will still flush on close, but
/// that is a probability, not a barrier.
pub const E_SYNC_TIMEOUT: u16 = 9056;
/// The bounded durable flush's worker thread was lost before it reported. Same
/// durability consequence as [`E_SYNC_TIMEOUT`], different cause — a caller
/// retrying a timeout should not retry this.
pub const E_SYNC_WORKER_LOST: u16 = 9057;
/// READ CAPACITY returned a short or overflowing transfer.
pub const E_DISC_CAPACITY_MALFORMED: u16 = 9047;
pub const E_DRIVE_INQUIRY_SHORT: u16 = 9058;
// ── Error enum ──────────────────────────────────────────────────────────────
@@ -263,6 +355,14 @@ pub enum Error {
UdfNotFound {
path: String,
},
/// The reader was addressable but the bytes are structurally NOT a UDF
/// filesystem — a deterministic tag/format mismatch (e.g. no Anchor Volume
/// Descriptor Pointer at sector 256, no partition descriptor, no File Set
/// Descriptor). Distinct from [`Error::DiscRead`] (a transient I/O fault):
/// this is a stable property of the media, not something a retry fixes. Lets
/// callers (notably FMTS key resolution) treat "not a UDF/FMTS disc" as a
/// clean negative while still failing loud on a real read fault.
UdfNotFilesystem,
/// A `SectorSource` caller passed a destination buffer smaller than one
/// 2048-byte sector. A contract violation on the public reader API —
/// returned instead of panicking on the slice.
@@ -272,8 +372,36 @@ pub enum Error {
count: usize,
},
IfoParse,
/// A title produced NO muxable frames: the mux driver's pump ended without
/// any video track's `codec_private` resolving, or the MKV muxer reached
/// `finish()` with zero frames written. The canonical case is an empty
/// nav/menu PGC stub, and [`is_skippable_title_stub`] classifies this code as
/// skippable so an all-titles rip drops the stub and finishes the rest.
///
/// This meaning is EXCLUSIVE. Malformed `mkv://` source input is
/// [`Error::MkvSourceInvalid`]; an unrepresentable element on the write side
/// is [`Error::MkvUnencodable`]. Routing either of those here would report a
/// broken file as a title worth silently skipping.
MkvInvalid,
/// An `mkv://` SOURCE file is malformed or truncated — the EBML/Matroska
/// reader rejected it. NOT [`Error::MkvInvalid`]: see
/// [`E_MKV_SOURCE_INVALID`].
MkvSourceInvalid,
/// The Matroska WRITER cannot encode an element size in EBML (body at or
/// above the 56-bit VINT limit, or a stale master-size placeholder). NOT
/// [`Error::MkvInvalid`]: see [`E_MKV_UNENCODABLE`].
MkvUnencodable,
/// An `mkv://` source Block's lacing header does not describe its payload —
/// the frames packed into that Block cannot be separated. NOT
/// [`Error::MkvInvalid`]: see [`E_MKV_LACING_INVALID`].
MkvLacingInvalid,
NoStreams,
/// A [`crate::StreamSelection`] listed a PID that does not exist in the
/// title's declared streams — a caller bug (e.g. a stale scan), reported
/// loudly rather than silently producing an MKV missing a requested track.
SelectionPidUnknown {
pid: u16,
},
/// ddrescue mapfile parse failed. `kind` is a stable, language-neutral
/// identifier (e.g. `"status_char"`, `"hex"`); not a translatable
/// English message.
@@ -335,7 +463,39 @@ pub enum Error {
/// title (e.g. a multi-VTS DVD where the title's VTS could not be
/// re-cracked). Muxing would emit scrambled ciphertext, so the caller
/// fails fast instead. CSS analogue of [`Error::NoDiscKey`].
///
/// PER-TITLE by construction: a sibling title in another VTS may still crack
/// its own key, so [`is_skippable_title_stub`] classifies this code and an
/// all-titles rip skips the title and finishes the rest. The whole-disc
/// counterpart — the main feature's crack failed, so nothing on the disc can
/// be decrypted — is [`Error::CssNoDiscKey`].
CssKeyMissing,
/// The disc is CSS-encrypted and decryption was requested, but the
/// known-plaintext crack recovered NO title key for the disc at all (the scan
/// saw scrambled sectors and stamped `Disc::css_error`). A whole-disc
/// condition: every title would fail the same way, so a multi-title rip loop
/// must stop instead of iterating. The CSS analogue of [`Error::NoDiscKey`]
/// on the disc-wide axis, and classified by [`is_disc_level_no_key`] — NOT by
/// [`is_skippable_title_stub`], which owns the per-title
/// [`Error::CssKeyMissing`]. Raising this as `CssKeyMissing` (as the disc-wide
/// gate once did) makes an undecryptable disc log one "title skipped" notice
/// per title and exit successfully.
CssNoDiscKey,
/// A key source could not be reached, or failed on its own side — transport
/// error, DNS failure, timeout, TLS failure, HTTP 5xx, or an unreadable /
/// unparseable reply. See [`E_KEY_SERVICE_UNAVAILABLE`]: the source never
/// answered the question, so this is emphatically NOT [`Error::NoDiscKey`]
/// (which asserts every source DID answer and none holds a key). Transient.
///
/// Carries no detail by design: the key-service URL and the resolved address
/// are operator-confidential and must not reach a log or a bug report.
KeyServiceUnavailable,
/// A key source rejected the configured credentials (HTTP 401/403). See
/// [`E_KEY_SERVICE_UNAUTHORIZED`]. Not transient: fix the token.
KeyServiceUnauthorized,
/// A key source rate-limited the request (HTTP 429). See
/// [`E_KEY_SERVICE_RATE_LIMITED`]. Back off and retry more slowly.
KeyServiceRateLimited,
/// The live-drive AACS cert-auth handshake (the OEM/AACS baseline route)
/// could not run because NO host certificate was available from any key
/// source. Host certs are keysource-served, never compiled in, so without
@@ -360,6 +520,13 @@ pub enum Error {
/// time and no handshake runs.
AacsBusKeyUnavailable,
/// AACS 2.1 (FMTS) disc carries forensic variant segments, but no segment
/// (variant) key is available to open them. Raised UPFRONT — before the mux —
/// exactly like a missing unit key, so a 2.1 disc that would rip with holes is
/// refused rather than silently producing a forensic-holed output. (The mux
/// resolves the full forensic key set up front; a resolution gap fails here.)
FmtsKeyMissing,
// Keydb (8xxx)
KeydbConnect {
host: String,
@@ -410,6 +577,33 @@ pub enum Error {
/// every frame dropped before the first keyframe — fails loudly. The
/// `m2ts://` analogue of [`Error::MkvInvalid`]'s zero-frame guard.
MuxEmpty,
/// The mux driver's pre-headers frame buffer passed its cap before every
/// video track's `codec_private` resolved: the title keeps yielding real
/// frames but its codec init data never appears, so buffering further would
/// swap the box to death. Carries the buffered byte count.
///
/// DISTINCT from [`Error::MkvInvalid`] on purpose. `MkvInvalid` is what an
/// empty nav/menu PGC stub yields, and [`is_skippable_title_stub`] classifies
/// it as skippable — an all-titles rip drops that title and finishes the
/// rest. Hundreds of megabytes of real frames with unresolvable headers is
/// NOT a stub; reporting it as one silently dropped a main feature from a
/// rip that then exited successfully. This code is not skippable.
MuxHeaderBufferExceeded {
bytes: u64,
},
/// `mp4://` target title has no primary video track to mux.
Mp4NoVideoTrack,
/// `mp4://` source file is malformed/truncated — the MP4 demuxer failed.
Mp4Invalid,
/// `mp4://` video track is missing its `hvcC`/`avcC` configuration record.
Mp4MissingCodecPrivate,
/// `mp4://` video track has no resolved frame dimensions. See
/// [`E_MP4_UNKNOWN_RESOLUTION`].
Mp4UnknownResolution,
/// The bounded durable flush timed out. See [`E_SYNC_TIMEOUT`].
SyncTimeout,
/// The bounded durable flush's worker was lost. See [`E_SYNC_WORKER_LOST`].
SyncWorkerLost,
PesFrameTooLarge {
size: usize,
},
@@ -487,6 +681,11 @@ pub enum Error {
/// last-LBA + 1 overflowed `u32`. Either case means the capacity
/// response is unusable; no English commentary.
DiscCapacityMalformed,
/// INQUIRY returned GOOD status but transferred fewer bytes than the
/// standard 36-byte header, so the identity fields would decode from a
/// zero-filled buffer. A drive reporting an empty INQUIRY would otherwise
/// present as peripheral type 0x00 and be dropped from enumeration.
DriveInquiryShort,
/// `--raw` was given with a `dir://` destination. An encrypted file
/// tree is useless; raw bytes belong in `iso://`.
DirRawRejected,
@@ -542,11 +741,16 @@ impl Error {
Error::MplsParse => E_MPLS_PARSE,
Error::ClpiParse => E_CLPI_PARSE,
Error::UdfNotFound { .. } => E_UDF_NOT_FOUND,
Error::UdfNotFilesystem => E_UDF_NOT_FILESYSTEM,
Error::UdfBufferTooSmall => E_UDF_BUFFER_TOO_SMALL,
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::IfoParse => E_IFO_PARSE,
Error::MkvInvalid => E_MKV_INVALID,
Error::MkvSourceInvalid => E_MKV_SOURCE_INVALID,
Error::MkvUnencodable => E_MKV_UNENCODABLE,
Error::MkvLacingInvalid => E_MKV_LACING_INVALID,
Error::NoStreams => E_NO_STREAMS,
Error::SelectionPidUnknown { .. } => E_SELECTION_PID_UNKNOWN,
Error::MapfileInvalid { .. } => E_MAPFILE_INVALID,
Error::AacsNoKeys => E_AACS_NO_KEYS,
Error::AacsCertShort => E_AACS_CERT_SHORT,
@@ -571,8 +775,13 @@ impl Error {
Error::VidCdbUnavailable => E_VID_CDB_UNAVAILABLE,
Error::NoDiscKey { .. } => E_NO_DISC_KEY,
Error::CssKeyMissing => E_CSS_KEY_MISSING,
Error::CssNoDiscKey => E_CSS_NO_DISC_KEY,
Error::KeyServiceUnavailable => E_KEY_SERVICE_UNAVAILABLE,
Error::KeyServiceUnauthorized => E_KEY_SERVICE_UNAUTHORIZED,
Error::KeyServiceRateLimited => E_KEY_SERVICE_RATE_LIMITED,
Error::AacsNoHostCert { .. } => E_AACS_NO_HOST_CERT,
Error::AacsBusKeyUnavailable => E_AACS_BUS_KEY_UNAVAILABLE,
Error::FmtsKeyMissing => E_FMTS_KEY_MISSING,
Error::KeydbConnect { .. } => E_KEYDB_CONNECT,
Error::KeydbHttp { .. } => E_KEYDB_HTTP,
Error::KeydbInvalid => E_KEYDB_INVALID,
@@ -588,6 +797,13 @@ impl Error {
Error::StreamUrlMissingPort { .. } => E_STREAM_URL_MISSING_PORT,
Error::NetworkAddrBlocked { .. } => E_NETWORK_ADDR_BLOCKED,
Error::MuxEmpty => E_MUX_EMPTY,
Error::MuxHeaderBufferExceeded { .. } => E_MUX_HEADER_BUFFER_EXCEEDED,
Error::Mp4NoVideoTrack => E_MP4_NO_VIDEO_TRACK,
Error::Mp4Invalid => E_MP4_INVALID,
Error::Mp4MissingCodecPrivate => E_MP4_MISSING_CODEC_PRIVATE,
Error::Mp4UnknownResolution => E_MP4_UNKNOWN_RESOLUTION,
Error::SyncTimeout => E_SYNC_TIMEOUT,
Error::SyncWorkerLost => E_SYNC_WORKER_LOST,
Error::PesFrameTooLarge { .. } => E_PES_FRAME_TOO_LARGE,
Error::PesInvalidMagic => E_PES_INVALID_MAGIC,
Error::PesTrackTooLarge { .. } => E_PES_TRACK_TOO_LARGE,
@@ -606,6 +822,7 @@ impl Error {
Error::ExtentNotUnitAligned => E_EXTENT_NOT_UNIT_ALIGNED,
Error::M2tsPacketMalformed => E_M2TS_PACKET_MALFORMED,
Error::DiscCapacityMalformed => E_DISC_CAPACITY_MALFORMED,
Error::DriveInquiryShort => E_DRIVE_INQUIRY_SHORT,
Error::DirRawRejected => E_DIR_RAW_REJECTED,
Error::DirMultipassRejected => E_DIR_MULTIPASS_REJECTED,
Error::DirSourceUnsupported => E_DIR_SOURCE_UNSUPPORTED,
@@ -750,6 +967,12 @@ impl std::fmt::Display for Error {
Error::InvalidCdbLength { len, max } => {
write!(f, "E{}: {}/{}", self.code(), len, max)
}
Error::SelectionPidUnknown { pid } => {
write!(f, "E{}: 0x{:04x}", self.code(), pid)
}
Error::MuxHeaderBufferExceeded { bytes } => {
write!(f, "E{}: {}", self.code(), bytes)
}
_ => write!(f, "E{}", self.code()),
}
}
@@ -792,6 +1015,9 @@ impl From<Error> for std::io::Error {
3000..=3999 => std::io::ErrorKind::PermissionDenied,
4000..=4999 => std::io::ErrorKind::Other,
5000..=5999 => std::io::ErrorKind::Other,
// A stop is an interruption, not invalid data. MUST precede the
// 6000..=6999 arm — E_HALTED is 6010 and match arms are ordered.
E_HALTED => std::io::ErrorKind::Interrupted,
6000..=6999 => std::io::ErrorKind::InvalidData,
7000..=7999 => std::io::ErrorKind::PermissionDenied,
8000..=8999 => std::io::ErrorKind::Other,
@@ -822,6 +1048,29 @@ impl From<Error> for std::io::Error {
// 9023 MuxEmpty: finish() reached with zero frames — the output
// would be a header-only container. Treat as invalid output.
E_MUX_EMPTY => std::io::ErrorKind::InvalidData,
// 9051 MuxHeaderBufferExceeded: the source kept yielding frames but
// never its codec init data — the input is unusable as declared.
E_MUX_HEADER_BUFFER_EXCEEDED => std::io::ErrorKind::InvalidData,
// 9052 MkvLacingInvalid: a source Block's lacing header does not
// describe its own payload — malformed input data.
E_MKV_LACING_INVALID => std::io::ErrorKind::InvalidData,
// 9053 MkvSourceInvalid: the mkv:// source file is malformed or
// truncated — invalid input data.
E_MKV_SOURCE_INVALID => std::io::ErrorKind::InvalidData,
// 9054 MkvUnencodable: the writer was asked for an element size EBML
// cannot represent. An output-side limit, not bad input.
E_MKV_UNENCODABLE => std::io::ErrorKind::InvalidData,
// mp4:// demux errors: a malformed/truncated source file
// (E_MP4_INVALID), or a source whose tracks the mux can't use — no
// video track / missing codec-private config. All are invalid data.
E_MP4_NO_VIDEO_TRACK
| E_MP4_INVALID
| E_MP4_MISSING_CODEC_PRIVATE
| E_MP4_UNKNOWN_RESOLUTION => std::io::ErrorKind::InvalidData,
// Durability, not data validity: the write landed, the flush did
// not. TimedOut keeps the std kind a caller might already branch on
// while the E-code carries the distinction.
E_SYNC_TIMEOUT | E_SYNC_WORKER_LOST => std::io::ErrorKind::TimedOut,
// 9030 ExtentNotUnitAligned: a malformed/non-AACS-aligned
// extent was handed to the prefetch producer.
9030 => std::io::ErrorKind::InvalidInput,
@@ -847,6 +1096,109 @@ impl From<Error> for std::io::Error {
/// Convenience alias for `Result<T, Error>`.
pub type Result<T> = std::result::Result<T, Error>;
/// The numeric error code carried by an [`io::Error`](std::io::Error) that was
/// produced from an [`Error`], or `None` if it carries none.
///
/// Public because consumers need the code itself, not just the yes/no
/// predicates built on it below. `mux_stream` hands back an `io::Error`, and a
/// front-end reporting *why* a title failed had no way to recover the code
/// from it — the typed `Error` is gone by then and only the `E<code>` string
/// prefix survives. Parsing that prefix is this function's job; every consumer
/// re-implementing the parse is how the string-matching this crate spent 1.5.x
/// removing comes back.
///
/// [`From<Error> for io::Error`] is the ONLY path from a typed [`Error`] to an
/// `io::Error` in this crate, and it stringifies (`io::Error::new(kind, msg)`
/// where `msg` is the `Error`'s `E<code>[: …]` [`Display`](std::fmt::Display)
/// string) rather than boxing the typed value — no code path constructs an
/// `io::Error` that still holds a `crate::error::Error` via `get_ref`. So the
/// only recognised shape is the round-tripped `E<code>` message prefix.
pub fn error_code(e: &std::io::Error) -> Option<u16> {
// Round-tripped: `From<Error> for io::Error` stringifies as "E<code>[: …]".
let s = e.to_string();
let digits = s.strip_prefix('E')?;
let end = digits
.find(|c: char| !c.is_ascii_digit())
.unwrap_or(digits.len());
digits.get(..end)?.parse::<u16>().ok()
}
/// Whether a per-title mux failure is a *skippable title stub* — a
/// copy-protected-but-uncrackable title ([`Error::CssKeyMissing`]) or a title
/// that produced no muxable frames ([`Error::MkvInvalid`], an empty nav/menu
/// PGC stub). An all-titles rip skips such a title and finishes the rest;
/// every other error stays fatal.
///
/// This replaces the CLI's `E7023`/`E6008` string-match with a typed check on
/// the [`io::Error`](std::io::Error) `mux_stream` returns.
///
/// # Not skippable
///
/// A BROKEN input is not a stub. [`Error::MkvSourceInvalid`] (malformed or
/// truncated `mkv://` source), [`Error::MkvLacingInvalid`], and
/// [`Error::MkvUnencodable`] all used to be raised as [`Error::MkvInvalid`] and
/// therefore landed in this set, so a corrupt source was reported as a title
/// worth silently passing over by a run that then exited successfully. They now
/// carry their own codes and are fatal here — as is
/// [`Error::MuxHeaderBufferExceeded`].
///
/// Nor is a WHOLE-DISC key failure. [`Error::CssNoDiscKey`] (the disc's CSS
/// crack recovered no key at all) is the same conflation on the decrypt axis:
/// while it too was raised as [`Error::CssKeyMissing`], every title of an
/// undecryptable disc classified as a skippable stub, so the rip loop skipped
/// all of them and exited successfully. It is [`is_disc_level_no_key`]'s, and
/// fatal here.
pub fn is_skippable_title_stub(e: &std::io::Error) -> bool {
matches!(error_code(e), Some(E_MKV_INVALID | E_CSS_KEY_MISSING))
}
/// Whether an [`io::Error`](std::io::Error) is a cooperative user stop
/// ([`Error::Halted`], code [`E_HALTED`]) — vs a structural failure. A stop is
/// resumable, not a rip failure: `mux_stream` maps a mid-run halt to
/// `completed = false`, and consumers preserve staging rather than quarantining.
/// Typed replacement for the consumers' `E<code>`-leading-token string match.
pub fn is_halt(e: &std::io::Error) -> bool {
error_code(e) == Some(E_HALTED)
}
/// Whether an [`io::Error`](std::io::Error) is a **disc-level** key failure —
/// the disc as a whole cannot be decrypted, so EVERY title will fail the same
/// way. Distinct from a per-title skippable stub
/// ([`is_skippable_title_stub`]): `E_NO_DISC_KEY` (keydb present but no entry
/// for this disc), `E_KEYDB_LOAD` (no keydb at all), `E_AACS_NO_KEYS` (no
/// usable AACS key material), and `E_CSS_NO_DISC_KEY` (the CSS crack recovered
/// no title key for the disc at all) are all whole-disc conditions. A
/// multi-title rip loop should stop immediately on this (fail-fast) rather than
/// iterate every title re-printing the same error.
///
/// `E_CSS_NO_DISC_KEY` is the CSS side of exactly that split, and it exists
/// because the disc-wide CSS failure used to be raised as the per-title
/// [`E_CSS_KEY_MISSING`]: an undecryptable CSS disc landed in
/// [`is_skippable_title_stub`], so the rip loop skipped all N titles with an
/// "empty stub" notice and exited successfully.
/// The key-SOURCE failures ([`E_KEY_SERVICE_UNAVAILABLE`],
/// [`E_KEY_SERVICE_UNAUTHORIZED`], [`E_KEY_SERVICE_RATE_LIMITED`]) are here for
/// the same fail-fast reason and NOT because they mean "no key": a service that
/// is down, refusing the token, or throttling is down for every title on the
/// disc, so iterating N titles re-issues N doomed requests (and, on 429, digs the
/// rate-limit hole deeper). They are separate CODES precisely so the front-end
/// can say "retry later" / "fix the token" instead of `E_NO_DISC_KEY`'s "no key
/// source has a key for this disc".
pub fn is_disc_level_no_key(e: &std::io::Error) -> bool {
matches!(
error_code(e),
Some(
E_NO_DISC_KEY
| E_KEYDB_LOAD
| E_AACS_NO_KEYS
| E_CSS_NO_DISC_KEY
| E_KEY_SERVICE_UNAVAILABLE
| E_KEY_SERVICE_UNAUTHORIZED
| E_KEY_SERVICE_RATE_LIMITED
)
)
}
impl Error {
/// Borrow the drive-returned SPC-4 sense triple if this error is a
/// [`Error::ScsiError`] carrying sense data. `None` for any other
@@ -924,7 +1276,7 @@ impl Error {
/// REQUEST, BLANK CHECK, kernel `IoError`, and any non-SCSI variant.
/// Caller-agnostic predicate — describes a property of the *error*,
/// not what one specific call site should do with it. Used by
/// `Disc::copy`'s hysteresis dispatch.
/// `freemkv_engine::recovery::copy`'s hysteresis dispatch.
pub fn is_marginal_read(&self) -> bool {
self.scsi_sense()
.map(crate::scsi::ScsiSense::is_marginal)
@@ -941,6 +1293,103 @@ mod tests {
//! match arms in `code()` / the From impl could silently miscategorize.
use super::*;
#[test]
fn is_skippable_title_stub_excludes_malformed_mkv_input() {
// The two skippable per-title codes, round-tripped through io::Error
// exactly as `mux_stream` returns them. `MkvInvalid` now means ONLY the
// no-muxable-frames stub (the driver's headers-never-resolved gate and the
// MKV muxer's zero-frame `finish()` guard) — the meaning the doc on this
// predicate has always described.
let mkv: std::io::Error = Error::MkvInvalid.into();
let css: std::io::Error = Error::CssKeyMissing.into();
assert!(is_skippable_title_stub(&mkv));
assert!(is_skippable_title_stub(&css));
// A malformed / truncated `mkv://` SOURCE is a FAILURE, not a stub. Every
// read-path rejection in `mux::mkvstream` and `mux::ebml`'s read
// primitives used to be raised as `MkvInvalid`, so a bad VINT, a cluster
// timestamp past i64::MAX or a BlockGroup child overrunning its group all
// landed in the skippable set above: an all-titles rip would pass silently
// over a corrupt input and exit reporting success. Reverting
// `Error::MkvSourceInvalid` back to `Error::MkvInvalid` at those raise
// sites turns this assertion red.
let corrupt: std::io::Error = Error::MkvSourceInvalid.into();
assert!(
!is_skippable_title_stub(&corrupt),
"a malformed mkv:// source must never classify as a skippable stub, got {corrupt}"
);
// Same for the write side: an element size EBML cannot represent is an
// output-side limit, not an empty nav/menu stub.
let unencodable: std::io::Error = Error::MkvUnencodable.into();
assert!(!is_skippable_title_stub(&unencodable));
// And for the lacing rejection carved out in the same spirit.
let lacing: std::io::Error = Error::MkvLacingInvalid.into();
assert!(!is_skippable_title_stub(&lacing));
// A different coded error is NOT skippable (kills a "match anything with
// an E-code" mutant).
let nostreams: std::io::Error = Error::NoStreams.into();
assert!(!is_skippable_title_stub(&nostreams));
// A plain io::Error with no E-code prefix is not skippable.
let plain = std::io::Error::from(std::io::ErrorKind::BrokenPipe);
assert!(!is_skippable_title_stub(&plain));
// A header-buffer cap overflow is a REAL title whose codec init data
// never resolved — hundreds of MiB of frames, not an empty nav/menu
// stub. It used to be reported as `MkvInvalid`, which lands in the
// skippable set above, so an all-titles rip dropped a main feature and
// still exited successfully. It must have its own, non-skippable code.
let cap: std::io::Error = Error::MuxHeaderBufferExceeded {
bytes: 512 * 1024 * 1024 + 1,
}
.into();
assert!(!is_skippable_title_stub(&cap));
}
/// The two CSS no-key conditions must land on OPPOSITE sides of the
/// per-title / whole-disc split, and the AACS pair must keep doing the same.
///
/// [`Error::CssNoDiscKey`] is the disc-wide verdict (the main feature's crack
/// failed, so every title fails identically) and belongs ONLY to
/// [`is_disc_level_no_key`], exactly like its AACS analogue
/// [`Error::NoDiscKey`]. [`Error::CssKeyMissing`] is the per-title verdict
/// (one VTS of a multi-VTS DVD could not be re-cracked) and belongs ONLY to
/// [`is_skippable_title_stub`], so an all-titles rip skips that title and
/// finishes the rest. While the disc-wide raise also used
/// `E_CSS_KEY_MISSING`, an uncrackable CSS disc was iterated title by title,
/// each one "skipped", and the run exited 0.
#[test]
fn css_no_key_codes_split_disc_level_from_skippable() {
let wide: std::io::Error = Error::CssNoDiscKey.into();
assert!(
is_disc_level_no_key(&wide),
"the disc-wide CSS no-key code must be disc-level: {wide}"
);
assert!(
!is_skippable_title_stub(&wide),
"the disc-wide CSS no-key code must not be skippable: {wide}"
);
let per_title: std::io::Error = Error::CssKeyMissing.into();
assert!(
is_skippable_title_stub(&per_title),
"the per-title CSS no-key code must stay skippable: {per_title}"
);
assert!(
!is_disc_level_no_key(&per_title),
"the per-title CSS no-key code must not stop the whole rip: {per_title}"
);
// The AACS side of the same split, unchanged.
let aacs: std::io::Error = Error::NoDiscKey {
disc_hash: String::new(),
}
.into();
assert!(is_disc_level_no_key(&aacs));
assert!(!is_skippable_title_stub(&aacs));
}
#[test]
fn new_variants_have_distinct_codes() {
let codes = [
@@ -1021,6 +1470,15 @@ mod tests {
(Error::MapfileInvalid { kind: "hex" }, E_MAPFILE_INVALID),
(Error::DiscUrlNotDirect, E_DISC_URL_NOT_DIRECT),
(Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED),
// Both CSS no-key verdicts: numeric-only Display, no English.
(Error::CssKeyMissing, E_CSS_KEY_MISSING),
(Error::CssNoDiscKey, E_CSS_NO_DISC_KEY),
// Key-SOURCE failures: bare numeric Display. They must carry NO
// detail — the service URL and its resolved address are
// operator-confidential and must never reach a pasted bug report.
(Error::KeyServiceUnavailable, E_KEY_SERVICE_UNAVAILABLE),
(Error::KeyServiceUnauthorized, E_KEY_SERVICE_UNAUTHORIZED),
(Error::KeyServiceRateLimited, E_KEY_SERVICE_RATE_LIMITED),
];
for (e, want_code) in cases {
let s = e.to_string();
@@ -1093,6 +1551,19 @@ mod tests {
assert_eq!(mapped(Error::M2tsPacketMalformed), ErrorKind::InvalidData);
// 9047 DiscCapacityMalformed → InvalidData
assert_eq!(mapped(Error::DiscCapacityMalformed), ErrorKind::InvalidData);
// 9053/9054: the mkv:// read-path and write-path rejections split off
// `MkvInvalid`. Both are InvalidData, and both must render as a bare code
// with no English (this crate has none).
assert_eq!(mapped(Error::MkvSourceInvalid), ErrorKind::InvalidData);
assert_eq!(mapped(Error::MkvUnencodable), ErrorKind::InvalidData);
assert_eq!(
Error::MkvSourceInvalid.to_string(),
format!("E{}", E_MKV_SOURCE_INVALID)
);
assert_eq!(
Error::MkvUnencodable.to_string(),
format!("E{}", E_MKV_UNENCODABLE)
);
}
/// `Error::IoError` must round-trip back to the *original*
@@ -1184,6 +1655,7 @@ mod tests {
E_HALTED,
E_MAPFILE_INVALID,
E_UDF_BUFFER_TOO_SMALL,
E_UDF_NOT_FILESYSTEM,
E_AACS_NO_KEYS,
E_AACS_CERT_SHORT,
E_AACS_AGID_ALLOC,
@@ -1207,8 +1679,13 @@ mod tests {
E_VID_CDB_UNAVAILABLE,
E_NO_DISC_KEY,
E_CSS_KEY_MISSING,
E_CSS_NO_DISC_KEY,
E_KEY_SERVICE_UNAVAILABLE,
E_KEY_SERVICE_UNAUTHORIZED,
E_KEY_SERVICE_RATE_LIMITED,
E_AACS_NO_HOST_CERT,
E_AACS_BUS_KEY_UNAVAILABLE,
E_FMTS_KEY_MISSING,
E_KEYDB_CONNECT,
E_KEYDB_HTTP,
E_KEYDB_INVALID,
@@ -1224,6 +1701,13 @@ mod tests {
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MUX_HEADER_BUFFER_EXCEEDED,
E_MKV_LACING_INVALID,
E_MKV_SOURCE_INVALID,
E_MKV_UNENCODABLE,
E_MP4_NO_VIDEO_TRACK,
E_MP4_INVALID,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
@@ -1311,6 +1795,19 @@ mod tests {
(Error::PipelineConsumerGone, E_PIPELINE_CONSUMER_GONE),
(Error::DiscCapacityOverflow, E_DISC_CAPACITY_OVERFLOW),
(Error::MuxEmpty, E_MUX_EMPTY),
(
Error::MuxHeaderBufferExceeded { bytes: 0 },
E_MUX_HEADER_BUFFER_EXCEEDED,
),
(Error::MkvLacingInvalid, E_MKV_LACING_INVALID),
(Error::MkvSourceInvalid, E_MKV_SOURCE_INVALID),
(Error::MkvUnencodable, E_MKV_UNENCODABLE),
(Error::Mp4NoVideoTrack, E_MP4_NO_VIDEO_TRACK),
(Error::Mp4Invalid, E_MP4_INVALID),
(Error::Mp4MissingCodecPrivate, E_MP4_MISSING_CODEC_PRIVATE),
(Error::Mp4UnknownResolution, E_MP4_UNKNOWN_RESOLUTION),
(Error::SyncTimeout, E_SYNC_TIMEOUT),
(Error::SyncWorkerLost, E_SYNC_WORKER_LOST),
(Error::M2tsPacketMalformed, E_M2TS_PACKET_MALFORMED),
(Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED),
(Error::DiscCapacityMalformed, E_DISC_CAPACITY_MALFORMED),
+267
View File
@@ -0,0 +1,267 @@
//! Seeded robustness harness for the untrusted-input parsers.
//!
//! Every parser reached from here takes bytes that came off a disc, and this
//! crate's primary boundary is that the disc is untrusted: a malformed, damaged
//! or hostile image must never crash the library. These tests assert exactly
//! that one property — **the parser returns `Ok` or `Err`, and never panics.**
//!
//! # Why this exists rather than `cargo-fuzz`
//!
//! `cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
//! for libFuzzer's coverage feedback) and this project pins stable. So the
//! generator lives here instead. It gives up coverage-guided mutation — the real
//! loss — and keeps everything else: millions of cases, structure-aware input,
//! and a crash corpus. It also gains determinism, which a fuzzer does not have:
//! the same seed replays the same cases on any machine.
//!
//! # Why no `proptest` or `arbitrary`
//!
//! This crate has exactly one dev-dependency. That is a deliberate posture, and
//! a randomness crate is not worth ten transitive dependencies when the parsers
//! take plain `&[u8]` and a good enough generator is forty lines.
//!
//! # Budget
//!
//! `FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
//! enough that the per-commit gate stays under a second). The overnight run sets
//! it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
//! fixed so a failure in CI reproduces locally verbatim.
//!
//! # On failure
//!
//! The panic message carries the seed, generator and case index. Re-run with
//! `FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
//! into `tests/corpus/` as a permanent regression fixture — discovery happens
//! here, defence happens there.
#![cfg(test)]
/// Marsaglia xorshift64. Not cryptographic and does not need to be: the job is
/// a reproducible spread of bytes, and a named algorithm beats an ad-hoc LCG
/// whose period nobody has checked.
struct Rng(u64);
impl Rng {
fn new(seed: u64) -> Self {
// A zero seed is a fixed point of xorshift — it would emit zeros forever
// and every generated case would be identical.
Self(if seed == 0 {
0x2545_F491_4F6C_DD1D
} else {
seed
})
}
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
fn byte(&mut self) -> u8 {
(self.next() >> 24) as u8
}
/// Uniform-ish in `0..n`. The modulo bias is irrelevant at these magnitudes.
fn below(&mut self, n: usize) -> usize {
if n == 0 {
0
} else {
(self.next() % n as u64) as usize
}
}
fn fill(&mut self, len: usize) -> Vec<u8> {
(0..len).map(|_| self.byte()).collect()
}
}
/// Budget per generator per target.
fn cases() -> usize {
std::env::var("FREEMKV_HARNESS_CASES")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(256)
}
fn seed() -> u64 {
std::env::var("FREEMKV_HARNESS_SEED")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(0x5EED_1234_ABCD_0001)
}
/// Largest generated input. Big enough to carry a plausible header plus a body,
/// small enough that millions of cases stay quick.
const MAX_LEN: usize = 4096;
/// Drive `f` over three generators and report which case broke it.
///
/// A panic inside `f` fails the test on its own — nothing is caught here,
/// because catching would risk reporting a pass on an input that aborted. The
/// wrapper exists to make the failing case *identifiable*: the harness prints
/// the seed, generator and index before each call, so the last line before a
/// panic names the exact case to reproduce.
fn sweep<F: FnMut(&[u8])>(target: &str, magic: &[u8], f: F) {
sweep_n(target, magic, cases(), f)
}
/// `sweep` with an explicit budget. The budget is a PARAMETER rather than read
/// from the environment inside the loop: the meta-tests below need a small,
/// fixed count, and `std::env::set_var` is unsound once the test harness runs
/// tests in parallel — two tests setting the same variable race, which is
/// exactly what happened on the first run of this file.
fn sweep_n<F: FnMut(&[u8])>(target: &str, magic: &[u8], n: usize, mut f: F) {
let s = seed();
// 1. Pure random bytes. Cheap, and almost always rejected at the magic
// number — it exercises the entry guards and little else. Kept because
// the entry guards are themselves worth exercising.
let mut rng = Rng::new(s);
for i in 0..n {
let len = rng.below(MAX_LEN);
let buf = rng.fill(len);
run(target, "random", s, i, &buf, &mut f);
}
// 2. Valid magic, random body. THE generator that matters: pure random
// input dies at the magic check and never reaches the parser body, so
// without this the sweep only ever tests the first few lines.
let mut rng = Rng::new(s ^ 0xA5A5_A5A5_A5A5_A5A5);
for i in 0..n {
let mut buf = magic.to_vec();
let tail = rng.below(MAX_LEN.saturating_sub(magic.len()));
buf.extend(rng.fill(tail));
run(target, "magic+noise", s, i, &buf, &mut f);
}
// 3. Structured mutation of a plausible record: a valid magic, then mostly
// zeroes, with a handful of bytes corrupted and a truncation. Length and
// offset fields live in those early bytes, so this is what reaches the
// arithmetic — the offsets, counts and sizes a hostile image would lie
// about.
let mut rng = Rng::new(s ^ 0x1234_5678_9ABC_DEF0);
for i in 0..n {
let mut buf = vec![0u8; 512];
buf[..magic.len().min(512)].copy_from_slice(&magic[..magic.len().min(512)]);
for _ in 0..rng.below(24) + 1 {
let at = rng.below(buf.len());
buf[at] = rng.byte();
}
buf.truncate(rng.below(buf.len()) + 1);
run(target, "mutate", s, i, &buf, &mut f);
}
}
fn run<F: FnMut(&[u8])>(target: &str, generator: &str, seed: u64, i: usize, buf: &[u8], f: &mut F) {
// Printed, not asserted: `cargo test` swallows stdout for passing tests and
// shows it for failing ones, so this line is invisible until it is the last
// thing before a panic — at which point it is exactly what is needed.
println!(
"harness {target}/{generator} seed={seed:#x} case={i} len={} :: \
FREEMKV_HARNESS_SEED={seed} to reproduce",
buf.len()
);
f(buf);
}
#[test]
fn mpls_parse_never_panics() {
sweep("mpls", b"MPLS", |b| {
let _ = crate::mpls::parse(b);
});
}
#[test]
fn clpi_parse_never_panics() {
sweep("clpi", b"HDMV", |b| {
let _ = crate::clpi::parse(b);
});
}
#[test]
fn udf_name_parse_never_panics() {
// No magic: the compression ID is the first byte and every value is legal
// input to reject, so the "magic" is a byte the sweep will mutate anyway.
sweep("udf_name", &[8], |b| {
let _ = crate::udf::parse_udf_name(b);
});
}
#[test]
fn ps_demuxer_feed_never_panics() {
// Stateful, unlike the others: the demuxer carries a buffer across feeds, so
// each case is fed to a FRESH demuxer and then a shared one. The shared pass
// is what exercises cross-feed state — a start code split over a boundary,
// a held PES completed by later bytes, the carry-over cap.
let mut shared = crate::mux::ps::PsDemuxer::new();
sweep("ps_demux", &[0x00, 0x00, 0x01, 0xBA], |b| {
let mut fresh = crate::mux::ps::PsDemuxer::new();
let _ = fresh.feed(b);
let _ = shared.feed(b);
});
}
#[test]
fn mkv_lacing_split_never_panics() {
// All four lacing modes, including the reserved bit pattern. A degenerate
// fixed lace was a real defect found by audit round 5.
sweep("mkv_lacing", &[0x00], |b| {
for lacing in 0u8..=3 {
let _ = crate::mux::mkvstream::split_lacing(lacing, b);
}
});
}
/// The generators must actually differ, or the sweep is one generator run three
/// times and the coverage claim is false.
#[test]
fn the_three_generators_produce_different_inputs() {
let mut seen: Vec<Vec<u8>> = Vec::new();
sweep_n("probe", b"MPLS", 1, |b| seen.push(b.to_vec()));
assert_eq!(seen.len(), 3, "one case per generator");
assert_ne!(seen[0], seen[1], "random and magic+noise must differ");
assert_ne!(seen[1], seen[2], "magic+noise and mutate must differ");
assert!(
seen[1].starts_with(b"MPLS"),
"the magic+noise generator must actually carry the magic, or it never \
reaches the parser body"
);
}
/// The same seed must replay the same bytes, or a reported failure cannot be
/// reproduced and the harness is worthless as a regression tool.
#[test]
fn a_seed_replays_identically() {
let mut a = Vec::new();
let mut b = Vec::new();
sweep_n("probe", b"MPLS", 4, |x| a.push(x.to_vec()));
sweep_n("probe", b"MPLS", 4, |x| b.push(x.to_vec()));
assert_eq!(a, b, "the same seed must produce the same cases");
}
/// The harness is worthless if its cases die at the entry guards, so this
/// MEASURES how deep they actually reach instead of assuming. A generator that
/// never gets past a length or magic check exercises the first ten lines and
/// nothing else — the fuzzing equivalent of a test that cannot fail.
#[test]
fn the_generators_actually_reach_the_parser_bodies() {
// mpls::parse rejects at: len < 40, bad magic, then playlist_start + 10 >
// len. Anything that returns Ok got all the way through the play-item loop.
let mut ok = 0usize;
let mut total = 0usize;
sweep_n("reach", b"MPLS", 20000, |b| {
total += 1;
if crate::mpls::parse(b).is_ok() {
ok += 1;
}
});
assert!(
ok > 0,
"not one of {total} generated cases parsed successfully — the generators \
are all being rejected at the entry guards, so this harness is testing \
the guards and nothing behind them"
);
println!("mpls reach: {ok}/{total} cases parsed to completion");
}
+47 -5
View File
@@ -16,11 +16,11 @@
/// (case-insensitive), then requires an even run of ASCII hex digits. Any
/// non-hex byte, or an odd length, yields `None`.
pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
// Empty → empty Vec (a legitimately-empty variable-length field); odd length
// is malformed. (`parse_hex_fixed` enforces a concrete length separately.)
if bytes.len() % 2 != 0 {
if !bytes.len().is_multiple_of(2) {
return None;
}
let mut out = Vec::with_capacity(bytes.len() / 2);
@@ -34,7 +34,7 @@ pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
/// prefix; requires EXACTLY `2*N` ASCII hex digits after it. `None` on any
/// non-hex byte or a length mismatch.
pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
if bytes.len() != 2 * N {
return None;
@@ -46,8 +46,33 @@ pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
Some(out)
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive).
fn strip_prefix(s: &str) -> &str {
/// Parse a hex string into a `u16`. Accepts an optional `0x`/`0X` prefix
/// (case-insensitive) via the same [`strip_hex_prefix`] the byte parsers use.
/// `None` on any non-hex content or overflow.
///
/// Exists so callers never hand-roll `from_str_radix(s.trim_start_matches("0x"), 16)`
/// — a **case-sensitive** strip that silently dropped an uppercase-`0X` value.
/// (That reintroduced-in-keydb bug is exactly what this module was built to kill;
/// the integer fields now share the one prefix rule.)
pub fn parse_hex_u16(s: &str) -> Option<u16> {
u16::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u32`. See [`parse_hex_u16`].
pub fn parse_hex_u32(s: &str) -> Option<u32> {
u32::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u8`. See [`parse_hex_u16`].
pub fn parse_hex_u8(s: &str) -> Option<u8> {
u8::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive). Public so
/// callers that only need the prefix rule (e.g. normalizing a disc hash) reuse
/// the one definition instead of hand-rolling a case-sensitive
/// `trim_start_matches("0x")`.
pub fn strip_hex_prefix(s: &str) -> &str {
s.strip_prefix("0x")
.or_else(|| s.strip_prefix("0X"))
.unwrap_or(s)
@@ -95,6 +120,23 @@ mod tests {
assert_eq!(parse_hex_fixed::<16>(&s), None);
}
#[test]
fn hex_ints_accept_both_prefix_cases_and_bare() {
// The regression the keydb device-key bug hit: uppercase `0X` must parse
// identically to `0x` and to a bare value.
assert_eq!(parse_hex_u16("0x0001"), Some(1));
assert_eq!(parse_hex_u16("0X0001"), Some(1));
assert_eq!(parse_hex_u16("0001"), Some(1));
assert_eq!(parse_hex_u16(" 0XABCD "), Some(0xABCD));
assert_eq!(parse_hex_u32("0X00000002"), Some(2));
assert_eq!(parse_hex_u32("deadbeef"), Some(0xDEAD_BEEF));
assert_eq!(parse_hex_u8("0X03"), Some(3));
assert_eq!(parse_hex_u8("ff"), Some(0xFF));
// Overflow / non-hex → None.
assert_eq!(parse_hex_u8("0x1FF"), None);
assert_eq!(parse_hex_u16("0xzz"), None);
}
#[test]
fn bytes_variable_length_and_odd_rejected() {
assert_eq!(parse_hex_bytes("0xAABBCC"), Some(vec![0xAA, 0xBB, 0xCC]));
+193 -1
View File
@@ -49,13 +49,30 @@ pub struct DriveId {
pub raw_gc_010c: Vec<u8>,
}
/// SPC-4 standard INQUIRY data: 36 bytes through `product_revision`. Anything
/// shorter cannot populate the identity fields this type promises.
const INQUIRY_STANDARD_LEN: usize = 36;
impl DriveId {
/// Probe a real drive via SCSI and build its identity.
pub fn from_drive(transport: &mut dyn ScsiTransport) -> Result<Self> {
// INQUIRY — SPC-4 §6.4
let mut inquiry = vec![0u8; 96];
let cdb_inq = [0x12, 0x00, 0x00, 0x00, 0x60, 0x00];
transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
let inq = transport.execute(&cdb_inq, DataDirection::FromDevice, &mut inquiry, 5000)?;
// `bytes_transferred` is device-reported and untrusted — the same rule
// the two GET CONFIGURATION calls below already apply. It was ignored
// here, and the buffer is pre-zeroed, so a drive answering GOOD with a
// short or empty data phase (a USB-SATA bridge mid-wedge does exactly
// this) decoded to blank identity strings and a byte 0 of 0x00. Every
// platform enumerator gates on `raw_inquiry[0] & 0x1F == OPTICAL`, so
// 0x00 reads as DIRECT ACCESS and the drive silently disappears from
// the device list instead of reporting a failed probe.
if inq.bytes_transferred < INQUIRY_STANDARD_LEN {
return Err(crate::error::Error::DriveInquiryShort);
}
// Never decode past what the drive actually sent.
inquiry.truncate(inq.bytes_transferred.min(inquiry.len()));
// GET CONFIGURATION Feature 010Ch — MMC-6 §6.6.
// Best-effort: 010Ch (Firmware Information) is an optional feature.
@@ -262,6 +279,54 @@ mod tests {
/// Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer
/// (e.g. a device that reports fewer than 8 bytes) must not panic.
/// Mutation: removing the `data.len() > start` guard makes it panic on short inputs.
/// A drive that answers INQUIRY with GOOD status but a short or empty
/// data phase must fail the probe, not present as a blank drive.
///
/// The buffer is pre-zeroed, so decoding it unconditionally yielded empty
/// vendor/product/revision strings and a byte 0 of 0x00. Every platform
/// enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
/// and 0x00 is DIRECT ACCESS — so the drive silently vanished from the
/// device list rather than reporting that its identity probe failed. A
/// USB-SATA bridge mid-wedge does exactly this.
///
/// The two GET CONFIGURATION calls in the same function already clamped on
/// `bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
/// lines above them, discarded it.
#[test]
fn inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive() {
/// GOOD status, no sense, and only `n` bytes written.
struct ShortInquiry(usize);
impl ScsiTransport for ShortInquiry {
fn execute(
&mut self,
_cdb: &[u8],
_dir: DataDirection,
_buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
Ok(ScsiResult {
status: 0,
sense: [0u8; 32],
bytes_transferred: self.0,
})
}
}
// Empty data phase — the case that made a real drive disappear.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(0)),
Err(crate::error::Error::DriveInquiryShort)
));
// One byte short of the SPC-4 standard 36-byte header.
assert!(matches!(
DriveId::from_drive(&mut ShortInquiry(35)),
Err(crate::error::Error::DriveInquiryShort)
));
// Exactly the standard length is acceptable: the optional
// vendor-specific tail past byte 36 is allowed to be absent.
assert!(DriveId::from_drive(&mut ShortInquiry(36)).is_ok());
}
#[test]
fn ascii_field_short_buffer_returns_empty() {
// Buffer of length 5: start=8 is beyond the end → empty string.
@@ -339,9 +404,136 @@ mod tests {
);
}
/// `ascii_field`'s guard is `data.len() > start` (strictly greater), not
/// `>=`: a buffer whose length is exactly `start` has NO byte at that
/// offset, so it must still yield empty, not attempt to slice.
/// Mutation: `>` -> `>=` would try to slice `data[start..]` when
/// `data.len() == start`, which panics (empty range at the very end is
/// fine, but the guard's job is the `< start` case below it — pinning the
/// exact boundary catches an off-by-one either direction).
#[test]
fn ascii_field_boundary_len_equals_start_is_empty() {
let buf = vec![0u8; 8];
assert_eq!(ascii_field(&buf, 8, 16), "");
}
/// One byte past the boundary: `data.len() == start + 1` must extract
/// that single byte (clamped to `end`), proving the guard is `>` and not
/// off by one in the other direction.
#[test]
fn ascii_field_boundary_len_one_past_start_extracts_one_byte() {
let mut buf = vec![0u8; 9];
buf[8] = b'X';
assert_eq!(ascii_field(&buf, 8, 16), "X");
}
/// `Display` renders the four trimmed identity fields space-separated —
/// the human-readable counterpart of `match_key`'s pipe-separated form.
/// Not exercised anywhere else in this test module.
/// Mutation: replacing the `fmt` body with `Ok(Default::default())`
/// writes nothing at all, so formatting any `DriveId` yields "".
#[test]
fn display_formats_trimmed_fields_space_separated() {
let mut inquiry = vec![0u8; 96];
inquiry[8..16].copy_from_slice(b"PIONEER ");
inquiry[16..32].copy_from_slice(b"BD-RW BDR-S09 ");
inquiry[32..36].copy_from_slice(b"1.34");
inquiry[36..43].copy_from_slice(b" 16/04/");
let id = DriveId::from_inquiry(&inquiry, "201604250000");
assert_eq!(id.to_string(), "PIONEER BD-RW BDR-S09 1.34 16/04/");
}
/// GET CONFIGURATION failure (transport error) must not abort the
/// identity probe — firmware_date is empty, raw_gc_010c is empty.
/// Mutation: propagating the GET_CONFIGURATION error with `?` aborts from_drive.
/// Transport whose GET CONFIGURATION responses report an exact,
/// caller-chosen `bytes_transferred` for each of the two GC features
/// (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
/// boundary guards can be pinned precisely. INQUIRY always succeeds.
struct FixedGcCountTransport {
firmware_bytes: usize,
serial_bytes: usize,
}
impl ScsiTransport for FixedGcCountTransport {
fn execute(
&mut self,
cdb: &[u8],
_dir: DataDirection,
buf: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
for b in buf.iter_mut() {
*b = b'Z';
}
let bytes_transferred = match cdb.first() {
Some(&0x12) => buf.len(),
Some(&0x46) if cdb[3] == 0x0C => self.firmware_bytes,
Some(&0x46) if cdb[3] == 0x08 => self.serial_bytes,
_ => buf.len(),
};
Ok(ScsiResult {
status: 0,
bytes_transferred,
sense: [0u8; 32],
})
}
}
/// `end > 12` in the firmware-date branch (`from_drive`) is a strict
/// inequality: `bytes_transferred == 12` reports the field absent
/// (offset 12 is the first byte of the 12-char date; a count of exactly
/// 12 covers bytes 0..12, none of which is the date), so `firmware_date`
/// must be empty, not the mutant's off-by-one read.
/// Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an
/// empty but non-panicking slice — silently reporting "present" data
/// that is actually all outside the transferred count.
#[test]
fn from_drive_firmware_date_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 12,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "");
}
/// One byte past the boundary (`bytes_transferred == 13`) must extract
/// exactly the one available date byte (offset 12), proving the guard
/// is `>` and the slice end is clamped to `end`, not always to 24.
#[test]
fn from_drive_firmware_date_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 13,
serial_bytes: 0,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.firmware_date, "Z");
}
/// Same `> 12` boundary for the serial-number branch: exactly 12
/// transferred bytes must yield an empty serial.
#[test]
fn from_drive_serial_boundary_exactly_12_is_empty() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 12,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "");
}
/// One byte past the serial boundary extracts exactly that byte.
#[test]
fn from_drive_serial_boundary_13_extracts_one_byte() {
let mut t = FixedGcCountTransport {
firmware_bytes: 0,
serial_bytes: 13,
};
let id = DriveId::from_drive(&mut t).unwrap();
assert_eq!(id.serial_number, "Z");
}
#[test]
fn from_drive_gc_failure_yields_empty_firmware_date() {
struct GcFailTransport;
+598 -36
View File
@@ -58,7 +58,7 @@ pub struct DvdTitle {
pub struct DvdCell {
pub first_sector: u32,
pub last_sector: u32,
/// Raw cell-category byte at `cell_playback + 0` (libdvdread layout).
/// Raw cell-category byte at `cell_playback + 0` (DVD-Video IFO layout).
/// Packs block_mode (bits 7-6), block_type (bits 5-4), seamless_play
/// (bit 3), interleaved (bit 2), stc_discontinuity (bit 1),
/// seamless_angle (bit 0). Carried so the extent builder can recognise
@@ -72,7 +72,7 @@ pub struct DvdCell {
}
/// Decoded view of a cell-category byte (`cell_playback + 0`), per the
/// DVD-Video spec / libdvdread `cell_playback_t` layout. Byte-0 bitfields,
/// DVD-Video IFO cell-playback layout. Byte-0 bitfields,
/// MSB-first: `block_mode`(7-6), `block_type`(5-4), `seamless_play`(3),
/// `interleaved`(2), `stc_discontinuity`(1), `seamless_angle`(0). (The real
/// `cell_type` is a karaoke-only field in byte 1, not used here.)
@@ -93,7 +93,7 @@ pub struct CellCategory {
}
impl CellCategory {
/// Decode the raw `cell_playback + 0` byte (libdvdread `read_cell_playback`).
/// Decode the raw `cell_playback + 0` byte (DVD-Video IFO cell playback).
pub fn decode(raw: u8) -> Self {
CellCategory {
block_mode: (raw >> 6) & 0x03,
@@ -356,32 +356,7 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
return Err(Error::IfoParse);
}
let num_titles = be_u16(&vmg_data, tt_srpt_offset)?;
// Parse title entries — each is 12 bytes, starting at tt_srpt_offset + 8
let entries_start = tt_srpt_offset + 8;
let mut title_set_map: std::collections::BTreeMap<u8, Vec<(u16, u8)>> =
std::collections::BTreeMap::new();
for i in 0..num_titles as usize {
let base = entries_start + i * 12;
if base + 12 > vmg_data.len() {
break; // truncated — parse what we can
}
let num_chapters = be_u16(&vmg_data, base + 2)?;
let vts_number = byte_at(&vmg_data, base + 6)?;
let vts_title_num = byte_at(&vmg_data, base + 7)?;
if vts_number == 0 {
continue; // invalid
}
title_set_map
.entry(vts_number)
.or_default()
.push((num_chapters, vts_title_num));
}
let title_set_map = parse_tt_srpt(&vmg_data, tt_srpt_offset)?;
// Parse each VTS IFO
let mut title_sets = Vec::new();
@@ -398,6 +373,71 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
Ok(DvdInfo { title_sets })
}
/// Maximum TT_SRPT entries honoured. DVD-Video caps a disc at 99 titles
/// (VMGI TT_SRPT `TT_Ns`, and the 99-title / 99-title-set structure the format
/// is built around), but the on-disc count is an untrusted `u16`: a ~800 KB
/// crafted IFO can declare 65535 entries, each re-parsing a PGC into a full
/// `DvdTitle` (~540 MB of `DvdInfo`).
///
/// Headroom: this IS the format maximum, so it clips no conformant disc — a
/// real DVD cannot address a 100th title through TT_SRPT.
const MAX_TT_SRPT_TITLES: usize = 99;
/// Parse the VMG TT_SRPT into a per-title-set map of
/// `(chapter_count, vts_title_number)`.
///
/// Two bounds on untrusted input: the declared entry count is clamped to
/// [`MAX_TT_SRPT_TITLES`], and entries naming a `(vts_number, vts_title_num)`
/// pair already seen are dropped. De-duplication is a correctness fix as well
/// as a bound: two TT_SRPT entries pointing at the same VTS title are the same
/// title, and `parse_pgcit` would otherwise re-parse that one PGC into a
/// separate `DvdTitle` per entry.
fn parse_tt_srpt(
vmg_data: &[u8],
tt_srpt_offset: usize,
) -> Result<std::collections::BTreeMap<u8, Vec<(u16, u8)>>> {
let num_titles = be_u16(vmg_data, tt_srpt_offset)? as usize;
if num_titles > MAX_TT_SRPT_TITLES {
tracing::warn!(
declared = num_titles,
cap = MAX_TT_SRPT_TITLES,
"TT_SRPT title count exceeds the DVD-Video maximum, clamping"
);
}
let num_titles = num_titles.min(MAX_TT_SRPT_TITLES);
// Title entries are 12 bytes each, starting at tt_srpt_offset + 8.
let entries_start = tt_srpt_offset + 8;
let mut title_set_map: std::collections::BTreeMap<u8, Vec<(u16, u8)>> =
std::collections::BTreeMap::new();
let mut seen: std::collections::HashSet<(u8, u8)> = std::collections::HashSet::new();
for i in 0..num_titles {
let base = entries_start + i * 12;
if base + 12 > vmg_data.len() {
break; // truncated — parse what we can
}
let num_chapters = be_u16(vmg_data, base + 2)?;
let vts_number = byte_at(vmg_data, base + 6)?;
let vts_title_num = byte_at(vmg_data, base + 7)?;
if vts_number == 0 {
continue; // invalid
}
if !seen.insert((vts_number, vts_title_num)) {
continue; // duplicate entry for the same VTS title
}
title_set_map
.entry(vts_number)
.or_default()
.push((num_chapters, vts_title_num));
}
Ok(title_set_map)
}
// ── VTS parser ──────────────────────────────────────────────────────────────
/// Parse VTS_XX_0.IFO for one title set.
@@ -423,7 +463,7 @@ fn parse_vts(
}
// VTSI_MAT (VTS_xx_0.IFO header) field offsets — fixed by the DVD-Video
// spec (libdvdread `vtsi_mat_t`). The offsets are constant; the sector
// spec (the VTSI management table). The offsets are constant; the sector
// values they point to are per-disc.
const VTSTT_VOBS_OFFSET: usize = 0xC4; // VTS title VOBS start sector (feature)
const VTS_PGCIT_OFFSET: usize = 0xCC; // VTS_PGCIT sector pointer
@@ -571,16 +611,20 @@ fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
let b1 = byte_at(data, offset + 1)?;
let coding_mode = (b0 >> 5) & 0x07;
// DVD-Video audio_coding_mode. Modes 2 and 3 are both MPEG audio Layer II
// (3 adds the MPEG-2 multichannel extension), so both are Codec::Mp2.
// Mode 2 previously mapped to Codec::Mpeg1 — a VIDEO variant, so
// Codec::kind() reported Video and the audio stream was classified and
// handled as video everywhere downstream.
let codec = match coding_mode {
0 => Codec::Ac3,
2 => Codec::Mpeg1,
3 => Codec::Mp2,
2 | 3 => Codec::Mp2,
4 => Codec::Lpcm,
6 => Codec::Dts,
_ => Codec::Unknown(coding_mode),
};
let sample_rate_flag = (b1 >> 4) & 0x03; // sample_frequency: byte 1 bits 5-4 (libdvdread audio_attr_t)
let sample_rate_flag = (b1 >> 4) & 0x03; // sample_frequency: byte 1 bits 5-4 (DVD-Video audio attributes)
let sample_rate = match sample_rate_flag {
0 => 48000,
1 => 96000,
@@ -854,6 +898,79 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
mod tests {
use super::*;
/// Build a synthetic TT_SRPT at offset 0: a declared `u16` title count
/// followed by `entries` 12-byte records of
/// `(num_chapters, vts_number, vts_title_num)`.
fn tt_srpt_bytes(declared: u16, entries: &[(u16, u8, u8)]) -> Vec<u8> {
let mut data = vec![0u8; 8 + entries.len() * 12];
data[0..2].copy_from_slice(&declared.to_be_bytes());
for (i, &(chapters, vts, title)) in entries.iter().enumerate() {
let base = 8 + i * 12;
data[base + 2..base + 4].copy_from_slice(&chapters.to_be_bytes());
data[base + 6] = vts;
data[base + 7] = title;
}
data
}
#[test]
fn tt_srpt_title_count_is_capped_and_deduplicated() {
// The TT_SRPT count is an untrusted u16. A ~800 KB crafted IFO declares
// 65535 entries, and each one re-parses a PGC into another full DvdTitle.
//
// The entries are DISTINCT, which matters: with 65535 identical entries
// the de-duplication collapses them to one title on its own and the count
// cap is never what bounds the result — the test would then pass with the
// cap removed entirely, proving only that dedup works. Distinct entries
// defeat dedup, so the cap is the only thing left holding the line.
// De-duplication gets its own assertion below, on its own fixture.
// Both title fields are u8, so vary each within its own range to get
// ~65025 distinct pairs out of the 65535 entries.
let entries: Vec<(u16, u8, u8)> = (0..u16::MAX)
.map(|i| (5u16, (i % 255) as u8 + 1, ((i / 255) % 255) as u8 + 1))
.collect();
let data = tt_srpt_bytes(u16::MAX, &entries);
let map = parse_tt_srpt(&data, 0).expect("well-formed TT_SRPT");
let total: usize = map.values().map(|v| v.len()).sum();
// Asserted against the DVD-Video format maximum as a LITERAL, not against
// MAX_TT_SRPT_TITLES. Comparing a result to the very constant under test
// passes vacuously the moment someone raises that constant, which is the
// most likely regression here — and is the tautology class this audit has
// now found six times. 99 is the format's own ceiling, so this pins the
// cap to the spec rather than to whatever the code currently says.
assert!(
total <= 99,
"TT_SRPT produced {total} titles from a crafted 65535-entry table; \
DVD-Video allows at most 99"
);
// De-duplication, on its own fixture: the same VTS title named 65535
// times is one title, not many. Kept separate from the cap assertion
// above so neither guard can mask the other.
let dupes = vec![(5u16, 1u8, 1u8); u16::MAX as usize];
let dup_map =
parse_tt_srpt(&tt_srpt_bytes(u16::MAX, &dupes), 0).expect("well-formed TT_SRPT");
assert_eq!(
dup_map.get(&1).map(Vec::len),
Some(1),
"duplicate (vts_number, vts_title_num) entries must collapse"
);
}
#[test]
fn tt_srpt_admits_a_full_99_title_disc() {
// The DVD-Video maximum must still parse in full: 99 distinct titles
// spread over two title sets.
let entries: Vec<(u16, u8, u8)> = (0..99u8)
.map(|i| (3u16, if i < 50 { 1 } else { 2 }, i + 1))
.collect();
let data = tt_srpt_bytes(99, &entries);
let map = parse_tt_srpt(&data, 0).expect("well-formed TT_SRPT");
let total: usize = map.values().map(|v| v.len()).sum();
assert_eq!(total, 99, "a full 99-title disc must survive the cap");
assert_eq!(map.get(&1).map(Vec::len), Some(50));
assert_eq!(map.get(&2).map(Vec::len), Some(49));
}
#[test]
fn bcd_to_secs_basic() {
// 1 hour, 23 minutes, 45 seconds, 0 frames at 25fps
@@ -1071,7 +1188,7 @@ mod tests {
/// reads with — a co-edit of constant + helper would silently re-introduce
/// the PAL-as-NTSC bug and every test would still pass. This test feeds
/// `parse_video_attr` HARDCODED bytes captured from real DVD-Video layouts
/// (DVD spec / libdvdread `video_attr_t`: mpeg_version[7-6] video_format[5-4]
/// (DVD-Video video attributes: mpeg_version[7-6] video_format[5-4]
/// display_aspect[3-2] permitted_df[1-0]) — no `v_atr_byte`. If the parser's
/// bit positions drift, these fail.
#[test]
@@ -1218,6 +1335,33 @@ mod tests {
assert_eq!(crate::mux::ps::dvd_audio_pid(0x89), Some(0xBD89));
}
#[test]
fn every_audio_coding_mode_maps_to_an_audio_codec() {
// An audio attribute block must never yield a codec whose kind() is Video.
// Mode 2 (MPEG-1 audio Layer II) mapped to Codec::Mpeg1 — the MPEG-1 VIDEO
// variant — so a DVD MPEG-audio stream was classified as video downstream.
for (mode, want) in [
(0u8, Codec::Ac3),
(2, Codec::Mp2),
(3, Codec::Mp2),
(4, Codec::Lpcm),
(6, Codec::Dts),
] {
let mut data = vec![0u8; 16];
data[0] = mode << 5;
data[2] = b'e';
data[3] = b'n';
let attr = parse_audio_attr(&data, 0).unwrap();
assert_eq!(attr.codec, want, "coding_mode {mode} must map to {want:?}");
assert_eq!(
attr.codec.kind(),
crate::disc::CodecKind::Audio,
"coding_mode {mode} produced {:?}, whose kind is not Audio",
attr.codec
);
}
}
#[test]
fn audio_attr_dts() {
let mut data = vec![0u8; 16];
@@ -1238,7 +1382,7 @@ mod tests {
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the DVD-Video IFO spec
// (dvd_udf / libdvdread ifo_types.h; http://dvd.sourceforge.net).
// (DVD-Video IFO format; http://dvd.sourceforge.net).
// ─────────────────────────────────────────────────────────────────────
/// BCD frame-rate flag: bits 7-6 of byte[3]. 0b01 = 25fps (PAL),
@@ -1593,7 +1737,7 @@ mod tests {
}
}
/// CellCategory decodes the libdvdread byte-0 bitfields: block_mode (7-6),
/// CellCategory decodes the DVD-Video cell-category byte-0 bitfields: block_mode (7-6),
/// block_type (5-4), seamless_play (3), interleaved (2),
/// stc_discontinuity (1), seamless_angle (0).
#[test]
@@ -1779,4 +1923,422 @@ mod tests {
title.chapter_times[0]
);
}
// ─────────────────────────────────────────────────────────────────────
// Added: cell-category bit isolation, palette/program-map arithmetic,
// and the malformed-language salvage paths.
// ─────────────────────────────────────────────────────────────────────
/// Each low flag of the cell-category byte comes from its OWN bit and no
/// other: bit3 seamless_play, bit2 interleaved, bit1 stc_discontinuity,
/// bit0 seamless_angle (DVD-Video cell playback `cell_category`).
/// Setting one bit must leave all three siblings clear.
#[test]
fn cell_category_low_flags_are_bit_isolated() {
let c = CellCategory::decode(0x00);
assert_eq!(
[
c.seamless_play,
c.interleaved,
c.stc_discontinuity,
c.seamless_angle
],
[false, false, false, false],
"category 0x00 sets no flag"
);
for bit in 0..4u8 {
let c = CellCategory::decode(1u8 << bit);
assert_eq!(
[
c.seamless_play,
c.interleaved,
c.stc_discontinuity,
c.seamless_angle
],
[bit == 3, bit == 2, bit == 1, bit == 0],
"only bit {bit} may be set"
);
}
}
/// `is_plain_feature` requires BOTH block_mode and block_type to be zero.
/// A cell inside a block is not plain feature content even when the other
/// field happens to be zero.
#[test]
fn is_plain_feature_requires_both_block_fields_zero() {
assert!(CellCategory::decode(0x00).is_plain_feature());
// block_type = 1, block_mode = 0
assert!(!CellCategory::decode(0b0001_0000).is_plain_feature());
// block_mode = 1, block_type = 0
assert!(!CellCategory::decode(0b0100_0000).is_plain_feature());
assert!(!CellCategory::decode(0b0101_0000).is_plain_feature());
}
/// be_u32 reads four CONSECUTIVE big-endian bytes from `offset`.
/// All four values are distinct so a repeated or skipped index shows up.
#[test]
fn be_u32_reads_four_consecutive_be_bytes() {
let data = [0xFFu8, 0x01, 0x02, 0x03, 0x04, 0xFF];
assert_eq!(be_u32(&data, 1).unwrap(), 0x0102_0304);
}
// ── malformed language codes ─────────────────────────────────────────
/// The ISO 639 language code in the audio/subtitle attribute block is
/// two bytes at +2. Only a pair of lowercase a-z bytes is taken verbatim;
/// anything else falls through to the ASCII-alphanumeric salvage, which
/// keeps only the usable characters. A byte outside a-z must never end up
/// in the language string.
#[test]
fn language_code_rejects_non_lowercase_bytes() {
// (byte0, byte1, expected language)
let cases: [(u8, u8, &str); 8] = [
(b'e', b'n', "en"), // both in range → verbatim
(0x21, b'n', "n"), // '!' is below 'a'
(0x7B, b'n', "n"), // '{' is above 'z'
(b'e', 0x21, "e"), // second byte below 'a'
(b'e', 0x7B, "e"), // second byte above 'z'
(b'E', 0x00, "E"), // only the second byte is zero
(0x00, b'E', "E"), // only the first byte is zero
(0x00, 0x00, ""), // both zero → unset
];
for (b0, b1, want) in cases {
let mut audio = vec![0u8; 8];
audio[2] = b0;
audio[3] = b1;
assert_eq!(
parse_audio_attr(&audio, 0).unwrap().language,
want,
"audio language for ({b0:#04x}, {b1:#04x})"
);
let mut sub = vec![0u8; 6];
sub[2] = b0;
sub[3] = b1;
assert_eq!(
parse_subtitle_attr(&sub, 0).unwrap().language,
want,
"subtitle language for ({b0:#04x}, {b1:#04x})"
);
}
}
// ── PGC fixtures ─────────────────────────────────────────────────────
/// Build a standalone PGC starting at offset 0.
///
/// Layout: 0xEA-byte header, then the program map (one byte per program,
/// the 1-based first cell number), then the 24-byte cell playback table.
/// `cells` are `(category, BCD time, first_sector, last_sector)`.
fn build_pgc(
pgc_time: [u8; 4],
cells: &[(u8, [u8; 4], u32, u32)],
programs: &[u8],
palette: Option<[[u8; 4]; 16]>,
) -> Vec<u8> {
let mut d = vec![0u8; 0xEA];
d[0x02] = programs.len() as u8;
d[0x03] = cells.len() as u8;
d[0x04..0x08].copy_from_slice(&pgc_time);
if let Some(p) = palette {
for (i, c) in p.iter().enumerate() {
d[0xA4 + i * 4..0xA4 + i * 4 + 4].copy_from_slice(c);
}
}
let pgm_off = 0xEAusize;
let cell_off = pgm_off + programs.len();
if !programs.is_empty() {
d[0xE6..0xE8].copy_from_slice(&(pgm_off as u16).to_be_bytes());
}
if !cells.is_empty() {
d[0xE8..0xEA].copy_from_slice(&(cell_off as u16).to_be_bytes());
}
d.extend_from_slice(programs);
for &(cat, time, first, last) in cells {
let mut c = vec![0u8; 24];
c[0] = cat;
c[4..8].copy_from_slice(&time);
c[8..12].copy_from_slice(&first.to_be_bytes());
c[20..24].copy_from_slice(&last.to_be_bytes());
d.extend_from_slice(&c);
}
d
}
/// BCD playback time of `secs` seconds (< 60) at the 29.97 fps flag.
fn bcd_secs(secs: u8) -> [u8; 4] {
assert!(secs < 60);
[0, 0, ((secs / 10) << 4) | (secs % 10), 0b11_000000]
}
/// When the PGC-level playback time is zero, the duration is recomputed
/// as the SUM of every cell's own BCD time, each read from its own
/// 24-byte cell playback record at +4.
#[test]
fn pgc_zero_duration_recomputed_as_sum_of_cell_times() {
let pgc = build_pgc(
[0, 0, 0, 0],
&[
(0x00, bcd_secs(10), 100, 199),
(0x00, bcd_secs(20), 200, 299),
(0x00, bcd_secs(31), 300, 399),
],
&[],
None,
);
let title = parse_pgc(&pgc, 0, 3).unwrap();
assert_eq!(title.cells.len(), 3);
assert_eq!(title.cells[0].duration_secs, 10.0);
assert_eq!(title.cells[1].duration_secs, 20.0);
assert_eq!(title.cells[2].duration_secs, 31.0);
assert_eq!(
title.duration_secs, 61.0,
"recomputed duration must be the sum of the distinct cell times"
);
}
/// Chapter times come from the program map: each program's byte is the
/// 1-based number of its first cell, and the chapter time is the sum of
/// the durations of every cell BEFORE it. Distinct cell durations mean a
/// misread program byte or a mis-strided cell table changes the result.
#[test]
fn pgc_chapter_times_sum_preceding_cell_durations() {
let pgc = build_pgc(
bcd_secs(59),
&[
(0x00, bcd_secs(10), 0, 9),
(0x00, bcd_secs(20), 10, 19),
(0x00, bcd_secs(31), 20, 29),
],
&[1, 2, 3],
None,
);
let title = parse_pgc(&pgc, 0, 3).unwrap();
assert_eq!(title.chapter_times, vec![0.0, 10.0, 30.0]);
}
/// A program map offset of 0 means there is no program map, so no chapter
/// times may be produced — the PGC header must not be read as one. Same
/// for a cell playback offset of 0.
#[test]
fn pgc_absent_program_map_or_cell_table_yields_no_chapter_times() {
// programs declared, but pgm_map_offset patched to 0
let mut pgc = build_pgc(
bcd_secs(30),
&[(0x00, bcd_secs(10), 0, 9), (0x00, bcd_secs(20), 10, 19)],
&[1, 2],
None,
);
pgc[0xE6..0xE8].copy_from_slice(&0u16.to_be_bytes());
let title = parse_pgc(&pgc, 0, 2).unwrap();
assert!(
title.chapter_times.is_empty(),
"no program map → no chapter times, got {:?}",
title.chapter_times
);
// program map present, but cell_playback_offset patched to 0
let mut pgc = build_pgc(
bcd_secs(30),
&[(0x00, bcd_secs(10), 0, 9), (0x00, bcd_secs(20), 10, 19)],
&[1, 2],
None,
);
pgc[0xE8..0xEA].copy_from_slice(&0u16.to_be_bytes());
let title = parse_pgc(&pgc, 0, 2).unwrap();
assert!(title.cells.is_empty());
assert!(
title.chapter_times.is_empty(),
"no cell table → no chapter times, got {:?}",
title.chapter_times
);
}
/// A declared cell count larger than the cell table actually holds must
/// not read past the end while collecting durations for the chapter-time
/// calculation; the missing cells contribute zero.
#[test]
fn pgc_cell_count_overshoot_does_not_read_past_end() {
let mut pgc = build_pgc(
bcd_secs(30),
&[(0x00, bcd_secs(10), 0, 9), (0x00, bcd_secs(20), 10, 19)],
&[1, 2],
None,
);
pgc[0x03] = 8; // declare 8 cells; only 2 records exist
let title = parse_pgc(&pgc, 0, 2).unwrap();
assert_eq!(title.cells.len(), 2, "only the readable cells are kept");
// Program 2 starts at cell 2, so its time is cell 0's duration.
assert_eq!(title.chapter_times, vec![0.0, 10.0]);
}
/// A program map that runs past the end of the data must stop at the
/// buffer end rather than reading past it. The fixture's map begins at
/// PGC+0xEA and the buffer holds 50 bytes from there, so a declared
/// count of 255 programs must yield exactly 50 chapter times.
#[test]
fn pgc_program_map_past_end_stops() {
let mut pgc = build_pgc(
bcd_secs(30),
&[(0x00, bcd_secs(10), 0, 9), (0x00, bcd_secs(20), 10, 19)],
&[1, 2],
None,
);
pgc[0x02] = 255; // declare 255 programs
let available = pgc.len() - 0xEA;
assert_eq!(available, 50, "fixture: 2 map bytes + 2 cells of 24");
let title = parse_pgc(&pgc, 0, 2).unwrap();
assert_eq!(
title.chapter_times.len(),
available,
"the program map walk must stop exactly at the buffer end"
);
assert_eq!(title.chapter_times[0], 0.0);
assert_eq!(title.chapter_times[1], 10.0);
}
/// The subtitle palette is 16 entries of 4 bytes at PGC+0xA4, each
/// `[padding, Y, Cb, Cr]`. Every byte of every entry is distinct here, so
/// a wrong stride, a wrong base or a shifted component shows up.
#[test]
// The loop variable is the DOMAIN VALUE being checked (a palette entry number), not a
// cursor into a collection: it is what the assertion message names and
// what the table is keyed by. `.iter().enumerate()` would rename the
// thing under test to `i` and read worse.
#[allow(clippy::needless_range_loop)]
fn pgc_palette_entries_read_at_correct_stride() {
let mut pal = [[0u8; 4]; 16];
for (i, c) in pal.iter_mut().enumerate() {
let b = i as u8;
*c = [0x10 + b, 0x30 + b, 0x50 + b, 0x70 + b];
}
let pgc = build_pgc(bcd_secs(30), &[], &[], Some(pal));
let title = parse_pgc(&pgc, 0, 1).unwrap();
let got = title.palette.expect("palette present");
assert_eq!(got.len(), 16);
for i in 0..16 {
let b = i as u8;
assert_eq!(
got[i],
[0x10 + b, 0x30 + b, 0x50 + b, 0x70 + b],
"palette entry {i}"
);
}
}
/// A palette is "present" when ANY of Y, Cb or Cr is non-zero in ANY
/// entry — a single non-zero chroma component is enough. Only the
/// padding byte [0] is ignored.
#[test]
fn pgc_palette_present_on_any_single_nonzero_component() {
for comp in 1..4usize {
let mut pal = [[0u8; 4]; 16];
pal[7][comp] = 0x40; // exactly one non-zero component, in one entry
let pgc = build_pgc(bcd_secs(30), &[], &[], Some(pal));
let title = parse_pgc(&pgc, 0, 1).unwrap();
assert!(
title.palette.is_some(),
"component {comp} alone must mark the palette present"
);
}
// Only the padding byte set → still empty.
let mut pal = [[0u8; 4]; 16];
for c in pal.iter_mut() {
c[0] = 0xFF;
}
let pgc = build_pgc(bcd_secs(30), &[], &[], Some(pal));
assert!(parse_pgc(&pgc, 0, 1).unwrap().palette.is_none());
}
// ── PGCIT ────────────────────────────────────────────────────────────
/// Build a VTS_PGCIT at offset 0: VTS_PGC_Ns(2) + reserved(2) +
/// VTS_PGCIT_EA(4), then one 8-byte VTS_PGCI_SRP per PGC
/// (VTS_PGC_CAT(4) + VTS_PGCI_SA(4), the PGC's byte offset from the
/// VTS_PGCIT start). `pgcs` are appended after the SRP table.
fn build_pgcit(pgcs: &[Vec<u8>]) -> Vec<u8> {
let mut d = vec![0u8; 8 + pgcs.len() * 8];
d[0..2].copy_from_slice(&(pgcs.len() as u16).to_be_bytes());
let mut off = d.len();
for (i, p) in pgcs.iter().enumerate() {
let e = 8 + i * 8;
d[e + 4..e + 8].copy_from_slice(&(off as u32).to_be_bytes());
off += p.len();
}
for p in pgcs {
d.extend_from_slice(p);
}
d
}
/// Each VTS_PGCI_SRP is 8 bytes and the PGC start address is the second
/// word of its own entry, so title N must resolve to PGC N. Two PGCs
/// with distinct durations catch an entry read at the wrong stride or
/// from the wrong entry.
#[test]
fn pgcit_entry_stride_selects_the_right_pgc() {
let pgc0 = build_pgc(bcd_secs(11), &[(0x00, bcd_secs(11), 0, 9)], &[], None);
let pgc1 = build_pgc(bcd_secs(22), &[(0x00, bcd_secs(22), 50, 59)], &[], None);
let data = build_pgcit(&[pgc0, pgc1]);
// vts_title_num is 1-based: title 2 → PGC index 1.
let titles = parse_pgcit(&data, 0, &[(5, 2)]).unwrap();
assert_eq!(titles.len(), 1);
assert_eq!(titles[0].duration_secs, 22.0);
assert_eq!(titles[0].cells[0].first_sector, 50);
let titles = parse_pgcit(&data, 0, &[(5, 1)]).unwrap();
assert_eq!(titles.len(), 1);
assert_eq!(titles[0].duration_secs, 11.0);
assert_eq!(titles[0].cells[0].first_sector, 0);
// Both titles, in order.
let titles = parse_pgcit(&data, 0, &[(5, 1), (7, 2)]).unwrap();
assert_eq!(titles.len(), 2);
assert_eq!(titles[0].duration_secs, 11.0);
assert_eq!(titles[1].duration_secs, 22.0);
}
/// A VTS_PGCIT whose 8-byte header ends exactly at the end of the data is
/// a complete header with no SRP entries: an empty title list, not an
/// error. Below 8 bytes the header itself is truncated → IfoParse.
#[test]
fn pgcit_header_boundary() {
let data = vec![0u8; 8];
let titles = parse_pgcit(&data, 0, &[(5, 1)]).expect("complete header parses");
assert!(titles.is_empty());
for len in 0..8usize {
assert!(
parse_pgcit(&vec![0u8; len], 0, &[(5, 1)]).is_err(),
"len={len}"
);
}
}
/// An SRP entry that runs past the end of the data is SKIPPED, leaving
/// an empty title list — a truncated entry table must not turn into a
/// parse error for the whole PGCIT.
#[test]
fn pgcit_entry_past_end_is_skipped_not_an_error() {
// 8-byte header declaring 3 PGCs, then only 12 bytes of table: the
// entry for PGC index 2 (offset 24..32) is entirely past the end.
let mut data = vec![0u8; 20];
data[0..2].copy_from_slice(&3u16.to_be_bytes());
let titles = parse_pgcit(&data, 0, &[(5, 3)])
.expect("a truncated SRP entry is skipped, not an error");
assert!(titles.is_empty());
}
/// A TT_SRP entry is 12 bytes: playback_type(1) + angles(1) +
/// number_of_PTTs(2) + parental_mask(2) + VTSN(1) + VTS_TTN(1) +
/// VTS_start_sector(4). The chapter count is the 16-bit field at +2, and
/// each entry's own value must be carried through. Distinct counts per
/// entry catch a read from a neighbouring offset.
#[test]
fn tt_srpt_chapter_count_read_from_entry_offset_2() {
let data = tt_srpt_bytes(3, &[(7, 1, 1), (13, 1, 2), (0x0102, 2, 1)]);
let map = parse_tt_srpt(&data, 0).unwrap();
assert_eq!(map[&1], vec![(7u16, 1u8), (13, 2)]);
assert_eq!(map[&2], vec![(0x0102u16, 1u8)]);
}
}
+4 -4
View File
@@ -133,10 +133,10 @@ where
match rx.recv_timeout(slice) {
Ok(v) => return Ok(v),
Err(RecvTimeoutError::Timeout) => {
if let Some(h) = halt {
if h.is_cancelled() {
return Err(BoundedError::Halted);
}
if let Some(h) = halt
&& h.is_cancelled()
{
return Err(BoundedError::Halted);
}
if Instant::now() >= deadline {
return Err(BoundedError::Timeout);
+21
View File
@@ -249,6 +249,27 @@ impl Drop for BytePrefetcher {
mod tests {
use super::*;
/// `RECYCLE_DEPTH` must be one MORE than `FORWARD_DEPTH` per its own
/// doc comment: the producer needs at least one buffer to fill while
/// the consumer holds the other `FORWARD_DEPTH`-worth in flight. A
/// `+` -> `*`/`-` mutation on `FORWARD_DEPTH + 1` would under-size the
/// recycle channel (e.g. `FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one
/// short), which starves the producer of a spare buffer.
#[test]
fn recycle_depth_is_forward_depth_plus_one() {
assert_eq!(RECYCLE_DEPTH, FORWARD_DEPTH + 1);
assert_eq!(RECYCLE_DEPTH, 3, "FORWARD_DEPTH is 2, so recycle must be 3");
}
/// `DEFAULT_CHUNK_BYTES` is documented as 16 MiB. Pins the literal so a
/// `*` -> `+`/`/` mutation on either factor (16 * 1024 * 1024) is
/// caught by a concrete, spec-derived expected value rather than by
/// recomputing the same expression.
#[test]
fn default_chunk_bytes_is_16_mib() {
assert_eq!(DEFAULT_CHUNK_BYTES, 16_777_216, "documented as 16 MiB");
}
/// Endless reader: every `read` fills the whole buffer and never
/// hits EOF, so the producer keeps trying to push batches forward
/// until the forward channel disconnects. Exactly the shape that
+122 -15
View File
@@ -78,18 +78,34 @@ use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// read side. Mirrors `WRITEBACK_CHUNK_BYTES` so the read-side page
/// cache stays bounded the same way the write side does.
///
/// 32 MiB is the empirically tuned value on the rip1 test bed (single
/// 7200rpm HDD via SATA): smaller windows (8 / 16 MiB) shorten the
/// 32 MiB is the empirically tuned value on a 7200rpm HDD via SATA:
/// smaller windows (8 / 16 MiB) shorten the
/// kernel-readahead overlap and slow the producer; larger windows
/// (64 / 128 MiB) let the page cache pin enough of the ISO to
/// pressure concurrent writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
const READ_DROP_CHUNK_BYTES_DEFAULT: u64 = 32 * 1024 * 1024;
/// Upper bound (in MiB) accepted from `FREEMKV_READ_DROP_CHUNK_MIB`. 64 GiB —
/// generous for any real medium, and small enough that `n * 1024 * 1024` cannot
/// wrap `u64`. Mirrors `WRITEBACK_CHUNK_MIB_MAX`, whose identical multiply is
/// bounded for exactly this reason: without the bound, a value above 2^44
/// overflows — a panic on the first ISO open in an overflow-checked build, and in
/// release a wrap to a near-zero window that fires `drop_window` on every read.
/// Out-of-range values fall back to the default.
const READ_DROP_CHUNK_MIB_MAX: u64 = 64 * 1024;
fn read_drop_chunk_bytes() -> u64 {
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok())
.filter(|&n| n > 0)
resolve_read_drop_chunk(
std::env::var("FREEMKV_READ_DROP_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok()),
)
}
/// The pure part of [`read_drop_chunk_bytes`], split out so the bound is
/// testable without mutating process environment.
fn resolve_read_drop_chunk(mib: Option<u64>) -> u64 {
mib.filter(|&n| n > 0 && n <= READ_DROP_CHUNK_MIB_MAX)
.map(|n| n * 1024 * 1024)
.unwrap_or(READ_DROP_CHUNK_BYTES_DEFAULT)
}
@@ -171,12 +187,20 @@ impl SectorSource for FileSectorSource {
) -> Result<usize> {
let count = count as u32;
let bytes = count as usize * SECTOR_BYTES;
debug_assert!(
out.len() >= bytes,
"FileSectorSource::read_sectors: out len {} < requested {}",
out.len(),
bytes
);
// A real check, not a debug_assert: this is a public `SectorSource` impl,
// so an undersized `out` is caller input, and `out[..bytes]` below would
// panic with 'range end index out of range' in release where the assert is
// compiled away. `Drive::read_fua` already carries exactly this guard, with
// a comment recording the same panic being fixed there — this impl was
// simply never given it, and `PrefetchedSectorSource` has a regression test
// for the case that this one lacked.
if out.len() < bytes {
return Err(Error::DiscRead {
sector: lba as u64,
status: None,
sense: None,
});
}
if count == 0 {
return Ok(0);
}
@@ -219,6 +243,40 @@ mod tests {
use std::io::Write;
use tempfile::tempdir;
/// An undersized output buffer must return an error, never panic. This is a
/// public `SectorSource` impl, so buffer length is caller input, and the guard
/// used to be a `debug_assert!` — compiled out in release, where the
/// `out[..bytes]` slice then panicked with 'range end index out of range'.
///
/// `Drive::read_fua` already carries this exact guard with a comment recording
/// the same panic being fixed there, and `PrefetchedSectorSource` has
/// `direct_read_too_small_buffer_errors` for the same case; this impl had
/// neither.
#[test]
fn read_sectors_with_an_undersized_buffer_errors_rather_than_panicking() {
let dir = tempdir().unwrap();
let iso = dir.path().join("t.iso");
make_iso(&iso, 8);
let mut src = FileSectorSource::open(&iso).expect("iso opens");
// Ask for four sectors but supply room for barely more than one.
let mut out = vec![0u8; SECTOR_BYTES + 1];
let err = src
.read_sectors(0, 4, &mut out, false)
.expect_err("an undersized buffer must be an error, not a panic");
assert!(
matches!(err, Error::DiscRead { .. }),
"expected DiscRead, got {err:?}"
);
// Exactly-sized still works, so the guard is not off by one.
let mut out = vec![0u8; 4 * SECTOR_BYTES];
assert_eq!(
src.read_sectors(0, 4, &mut out, false).unwrap(),
4 * SECTOR_BYTES
);
}
/// Build a deterministic ISO of `sectors` sectors where sector `n`
/// is filled with the byte pattern `((n & 0xff) as u8)`. Lets us
/// verify any sector by content alone.
@@ -376,19 +434,36 @@ mod tests {
// Additional coverage.
// ---------------------------------------------------------------
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or
/// reading, even at an out-of-range LBA — the early-return guard
/// runs before any I/O. Grounding: `if count == 0 { return Ok(0) }`.
/// `count == 0` must short-circuit to Ok(0) WITHOUT seeking or reading,
/// even at an out-of-range LBA. Grounding: `if count == 0 { return Ok(0) }`.
///
/// The `Ok(0)` return alone proves nothing: with the guard deleted, a seek
/// past EOF succeeds (POSIX permits seeking beyond the end of a file) and a
/// zero-length `read_exact` returns `Ok(())` immediately, so the call still
/// returns `Ok(0)`. The observable difference is the file's cursor — the
/// seek MOVES it to `lba * 2048`. Assert on that, so the guard is what the
/// test is actually measuring.
#[test]
fn zero_count_returns_zero_no_io() {
let dir = tempdir().unwrap();
let path = dir.path().join("zc.iso");
make_iso(&path, 4);
let mut src = FileSectorSource::open(&path).unwrap();
let before = src.file.stream_position().expect("cursor readable");
assert_eq!(before, 0, "a freshly opened file starts at offset 0");
// LBA far past EOF — must not matter because count==0 returns early.
let mut buf = [0u8; 1];
let n = src.read_sectors(1_000_000, 0, &mut buf, false).unwrap();
assert_eq!(n, 0);
assert_eq!(
src.file.stream_position().expect("cursor readable"),
before,
"count == 0 must return before the seek — an unmoved cursor is the \
only observable proof that no I/O was issued"
);
// And the drop-window accounting must not have advanced either.
assert_eq!(src.bytes_read_since_drop, 0);
assert_eq!(src.drop_window_start, 0);
}
/// Reading past EOF must ERROR (read_exact's UnexpectedEof), never
@@ -518,4 +593,36 @@ mod tests {
lba += batch as u32;
}
}
/// `FREEMKV_READ_DROP_CHUNK_MIB` must be BOUNDED before the MiB→byte
/// multiply, exactly as its writeback twin bounds the identical multiply.
/// Unbounded, any value above 2^44 overflowed `n * 1024 * 1024`: a panic on
/// the first ISO open in an overflow-checked build, and in release a wrap to
/// a near-zero window that fires `drop_window` on essentially every read.
#[test]
fn read_drop_chunk_env_is_bounded_before_the_multiply() {
// Default when unset / zero / out of range.
assert_eq!(resolve_read_drop_chunk(None), READ_DROP_CHUNK_BYTES_DEFAULT);
assert_eq!(
resolve_read_drop_chunk(Some(0)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// The overflow value: `u64::MAX * 1024 * 1024` panicked here.
assert_eq!(
resolve_read_drop_chunk(Some(u64::MAX)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX + 1)),
READ_DROP_CHUNK_BYTES_DEFAULT
);
// In-range values convert MiB→bytes. Mutation: `* 1024` breaks this.
assert_eq!(resolve_read_drop_chunk(Some(1)), 1024 * 1024);
assert_eq!(
resolve_read_drop_chunk(Some(READ_DROP_CHUNK_MIB_MAX)),
READ_DROP_CHUNK_MIB_MAX * 1024 * 1024
);
// And the bound itself keeps the multiply inside u64.
assert!((READ_DROP_CHUNK_MIB_MAX as u128) * 1024 * 1024 <= u64::MAX as u128);
}
}
+2 -3
View File
@@ -40,9 +40,8 @@ pub(crate) mod platform_macos;
pub mod pipeline;
pub(crate) use writeback_file::WritebackFile;
pub use writeback_file::WritebackFile;
pub use pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, READ_PIPELINE_DEPTH, Sink, WRITE_PIPELINE_DEPTH,
WRITE_THROUGH_DEPTH,
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
+319 -56
View File
@@ -33,7 +33,7 @@
//! consumer lag detection). This is critical for diagnosing stalls.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU8, Ordering};
use std::thread::{self, JoinHandle};
use std::time::{Duration, Instant};
@@ -117,8 +117,32 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
Error::PipelineConsumerPanicked
}
/// Consumer lifecycle state, shared between the caller and the consumer thread.
///
/// A plain `AtomicBool` could not make "the caller abandons" and "the consumer
/// commits to finalising" mutually exclusive: the consumer loaded the flag, the
/// caller stored it, and the consumer then finalised the container anyway — the
/// caller reporting the rip as interrupted while a fully finalised MKV (Cues
/// written, Segment size patched) landed on disk, indistinguishable from a
/// complete one. The two transitions are therefore a single compare-exchange each,
/// out of [`state::RUNNING`]: whoever wins decides, and the loser observes the
/// winner. (`ST_RUNNING` does not exist anywhere in the crate — the constants are
/// `state::RUNNING` / `state::ABANDONED` / `state::CLOSING` below, and both
/// compare-exchange sites that must stay in step with this argument name them.)
mod state {
/// Consumer is running; neither side has committed yet.
pub const RUNNING: u8 = 0;
/// The caller gave up on the consumer and will report failure — the consumer
/// must NOT finalise the output.
pub const ABANDONED: u8 = 1;
/// The consumer has committed to `close()` (finalising the output). The caller
/// can no longer abandon it; it must wait for the result it is about to
/// produce.
pub const CLOSING: u8 = 2;
}
/// After a halt or deadline fires, spin-poll `handle.is_finished()` for
/// [`FINISH_GRACE_SECS`] before accepting the thread leak. This converts
/// `grace` before accepting the thread leak. This converts
/// the common "nearly-done" consumer (whose own bounded_syscall just
/// returned and is about to drop its output file) into a clean join,
/// releasing the file handle without waiting the full grace period.
@@ -132,11 +156,12 @@ fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
/// syscall itself; that still returns on its own (or at process exit).
fn finish_with_grace<R: Send + 'static>(
handle: thread::JoinHandle<Result<R, Error>>,
abandoned: &Arc<AtomicBool>,
state: &Arc<AtomicU8>,
grace: Duration,
leak_err: Error,
) -> Result<R, Error> {
let grace = Instant::now() + Duration::from_secs(FINISH_GRACE_SECS);
while Instant::now() < grace {
let deadline = Instant::now() + grace;
while Instant::now() < deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
@@ -145,16 +170,50 @@ fn finish_with_grace<R: Send + 'static>(
}
thread::sleep(POLL_INTERVAL);
}
// Grace expired. Signal abandonment, then log and leak. Setting the
// flag BEFORE dropping the handle guarantees the leaked consumer
// observes it the moment its wedged syscall returns: it then skips
// any further `apply` and skips `close()`, rather than running on to
// finalise the abandoned output file.
// `Release` here pairs with the `Acquire` loads in the consumer loop so
// the leaked consumer reliably observes the flag the moment its wedged
// syscall returns, even on weak memory models (ARM64/POWER) where
// `Relaxed` gives no cross-thread visibility guarantee.
abandoned.store(true, Ordering::Release);
// Grace expired. CLAIM abandonment, then log and leak. Claiming BEFORE
// dropping the handle guarantees the leaked consumer observes it the moment
// its wedged syscall returns: it then skips any further `apply` and skips
// `close()`, rather than running on to finalise the abandoned output file.
//
// A compare-exchange, not a store, because the consumer may have committed to
// `close()` in the instant between our last `is_finished()` poll and now. It
// then cannot be stopped — the finalise IS happening — so abandoning it would
// report the rip as interrupted while a valid, fully finalised container
// lands on disk. Losing the race means waiting for the result the consumer is
// already producing instead. `AcqRel` pairs with the consumer's own
// compare-exchange and with the `Acquire` loads in its drain loop, so the flag
// is reliably observed even on weak memory models (ARM64/POWER).
if state
.compare_exchange(
state::RUNNING,
state::ABANDONED,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_close_in_flight",
"pipeline consumer had already committed to finalising the output; \
waiting for it rather than reporting an unfinalised output"
);
let close_deadline = Instant::now() + grace;
while Instant::now() < close_deadline {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
Err(payload) => Err(consumer_panicked(payload)),
};
}
thread::sleep(POLL_INTERVAL);
}
// Still finalising after a second grace window: leak and report the wedge.
// The output may end up finalised by the leaked thread — but that is now a
// wedged-`close()` case, not the check-then-finalise race.
drop(handle);
return Err(leak_err);
}
tracing::warn!(
target: "freemkv::pipeline",
phase = "finish_with_halt_grace_expired",
@@ -173,14 +232,9 @@ fn finish_with_grace<R: Send + 'static>(
/// Default channel depth for callers without a specific reason to
/// pick another value. Kept conservative (4) — most callers should
/// use READ_PIPELINE_DEPTH or WRITE_PIPELINE_DEPTH instead.
/// use WRITE_PIPELINE_DEPTH instead.
pub const DEFAULT_PIPELINE_DEPTH: usize = 4;
/// Read pipeline depth. Larger buffer compensates for drive variability
/// and NFS sync_file_range stalls; keeps ISO reader thread fed even when
/// consumer blocks on write.
pub const READ_PIPELINE_DEPTH: usize = 32;
/// Write pipeline depth. Smaller buffer reduces backpressure risk when
/// sync_file_range blocks; prevents producer from accumulating too much
/// work while consumer waits for NFS to drain.
@@ -189,7 +243,8 @@ pub const WRITE_PIPELINE_DEPTH: usize = 16;
/// Channel depth for write-through pipelines. Each `send` fully
/// drains before the next can enqueue. Use this when the producer
/// must observe consumer side-effects (e.g. mapfile state) before
/// emitting the next item. Currently used by `disc::patch`.
/// emitting the next item. Used by `freemkv_engine::recovery::patch` — the
/// recovery strategy moved to that crate in 1.6.0, so there is no `patch` here.
pub const WRITE_THROUGH_DEPTH: usize = 1;
/// Outcome of [`Sink::apply`]: either keep feeding items
@@ -247,7 +302,21 @@ pub struct Pipeline<I: Send + 'static, R: Send + 'static> {
/// a syscall the consumer is currently wedged in, but it does bound
/// the damage to "whatever write is already in flight" once that
/// syscall returns, instead of running on to a clean finalise.
abandoned: Arc<AtomicBool>,
///
/// One of [`state::RUNNING`] / [`state::ABANDONED`] / [`state::CLOSING`];
/// both transitions are compare-exchanges so abandoning and finalising are
/// mutually exclusive rather than racing.
state: Arc<AtomicU8>,
/// Set by the consumer the moment an `apply` returns `Err`. The consumer keeps
/// draining the channel after that (so the producer never blocks on a dead
/// receiver) — which means a producer watching only `send`'s return value
/// cannot tell the difference between "being consumed" and "being discarded
/// after a fatal write error", and would go on reading the whole remaining
/// disc before `finish()` finally surfaced the error. This flag is that
/// missing edge: [`Pipeline::send_with_halt`] fails fast on it, and
/// [`Pipeline::consumer_failed`] exposes it to producers that use plain
/// [`Pipeline::send`].
failed: Arc<AtomicBool>,
}
impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
@@ -281,8 +350,10 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
sink: S,
) -> Result<Self, Error> {
let (tx, rx) = bounded::<I>(depth);
let abandoned = Arc::new(AtomicBool::new(false));
let abandoned_consumer = abandoned.clone();
let state = Arc::new(AtomicU8::new(state::RUNNING));
let state_consumer = state.clone();
let failed = Arc::new(AtomicBool::new(false));
let failed_consumer = failed.clone();
let handle = thread::Builder::new()
.name(name.into())
.spawn(move || -> Result<R, Error> {
@@ -313,7 +384,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// dead receiver, but we touch the output no further. The
// final post-loop abandonment check returns the error
// and skips `close()`.
if abandoned_consumer.load(Ordering::Acquire) {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
continue;
}
@@ -342,6 +413,12 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
}
first_err = Some(e);
// Publish the failure so the producer can stop
// FEEDING a dead write side instead of only learning
// about it at `finish()` — by which time it has read
// the rest of the disc. `Release` pairs with the
// `Acquire` load in `send_with_halt`.
failed_consumer.store(true, Ordering::Release);
}
}
@@ -402,13 +479,38 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
// MKV Cues + patching the segment header) on a file the
// caller already reported as failed is exactly the
// write race we must not run.
if abandoned_consumer.load(Ordering::Acquire) {
return Err(Error::Halted);
}
match first_err {
Some(e) => Err(e),
None => sink.close(),
// No `close()` on this path, so there is nothing to claim —
// just report, unless the caller has already given up on us.
Some(e) => {
if state_consumer.load(Ordering::Acquire) == state::ABANDONED {
Err(Error::Halted)
} else {
Err(e)
}
}
// CLAIM the finalise. A plain load here left a window in which
// the caller stored `abandoned` AFTER we read it as clear, so
// `close()` ran anyway and finalised (Cues + Segment-size
// patch) an output the caller had already reported as
// interrupted — a truncated rip indistinguishable from a
// complete one. The compare-exchange closes that window: if the
// caller got there first we skip `close()`, and if we get there
// first the caller waits for us instead of abandoning.
None => {
if state_consumer
.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire,
)
.is_err()
{
return Err(Error::Halted);
}
sink.close()
}
}
})
.map_err(|e| Error::IoError { source: e })?;
@@ -416,10 +518,24 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Ok(Pipeline {
tx,
handle,
abandoned,
state,
failed,
})
}
/// Whether the consumer's `apply` has already failed fatally.
///
/// The consumer keeps draining the channel after an `apply` error (so the
/// producer never blocks on a dead receiver), which means `send` keeps
/// succeeding and a producer has no other way to tell that everything it feeds
/// is being discarded. A long-running producer — the mux frame pump reading a
/// 60 GB title off an optical drive — should check this and unwind instead of
/// reading the rest of the disc for a write that has already failed.
/// [`Pipeline::send_with_halt`] checks it automatically.
pub fn consumer_failed(&self) -> bool {
self.failed.load(Ordering::Acquire)
}
/// Push one item. Blocks if the channel is full — that's the
/// back-pressure the whole primitive exists to provide. Returns
/// the item back if the consumer thread is gone (panicked or
@@ -449,7 +565,10 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
} else {
// Benign per-item OK: trace-level (L4) only; the
// apply-side rolling summary carries throughput.
tracing::trace!("Pipeline send: OK in {:.3}ms", elapsed.as_micros());
tracing::trace!(
"Pipeline send: OK in {:.3}ms",
elapsed.as_secs_f64() * 1000.0
);
}
}
Ok(())
@@ -464,7 +583,10 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
std::any::type_name::<I>()
);
} else {
tracing::debug!("Pipeline send: failed after {:.3}ms", elapsed.as_micros());
tracing::debug!(
"Pipeline send: failed after {:.3}ms",
elapsed.as_secs_f64() * 1000.0
);
}
}
Err(e.0)
@@ -511,6 +633,21 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let end = Instant::now() + deadline;
let mut pending = item;
loop {
// The consumer's `apply` has failed fatally: everything sent from here
// is drained and discarded, so hand the item back at once. Without this
// the producer saw every send succeed (the channel is always being
// drained) and went on reading the whole remaining title — an hour of
// drive time on a UHD — for a write that died on the first frame, only
// learning about it at `finish()`.
if self.consumer_failed() {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: consumer apply failed, returning item={}",
std::any::type_name::<I>()
);
}
return Err(pending);
}
// Pre-check the cheap exit conditions before parking.
if halt.is_cancelled() {
if debug_enabled() {
@@ -565,7 +702,8 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let Pipeline {
tx,
handle,
abandoned: _,
state: _,
failed: _,
} = self;
// Explicit drop, although the destructure already drops `tx`
// at end-of-scope. Being explicit keeps the intent obvious.
@@ -605,7 +743,8 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let Pipeline {
tx,
handle,
abandoned,
state,
failed: _,
} = self;
drop(tx);
let deadline = Instant::now() + Duration::from_secs(JOIN_TIMEOUT_SECS);
@@ -616,13 +755,23 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
Err(payload) => Err(consumer_panicked(payload)),
};
}
if let Some(h) = halt {
if h.is_cancelled() {
return finish_with_grace(handle, &abandoned, Error::Halted);
}
if let Some(h) = halt
&& h.is_cancelled()
{
return finish_with_grace(
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::Halted,
);
}
if Instant::now() >= deadline {
return finish_with_grace(handle, &abandoned, Error::PipelineJoinTimeout);
return finish_with_grace(
handle,
&state,
Duration::from_secs(FINISH_GRACE_SECS),
Error::PipelineJoinTimeout,
);
}
thread::sleep(POLL_INTERVAL);
}
@@ -1074,20 +1223,6 @@ mod tests {
/// A sink that records the exact order of items it receives, so we
/// can prove the channel is FIFO (no reordering). `close` returns
/// the recorded vector.
struct OrderSink {
seen: Vec<u64>,
}
impl Sink<u64> for OrderSink {
type Output = Vec<u64>;
fn apply(&mut self, item: u64) -> Result<Flow, Error> {
self.seen.push(item);
Ok(Flow::Continue)
}
fn close(self) -> Result<Vec<u64>, Error> {
Ok(self.seen)
}
}
/// Zero items sent: closing the pipeline immediately must still
/// call `close()` exactly once and return its Output. The consumer
/// loop's `while let Ok = rx.recv()` exits on the dropped tx with
@@ -1592,4 +1727,132 @@ mod tests {
let res = pipe.finish_with_halt(None);
assert!(matches!(res, Ok(190)), "expected Ok(190), got {res:?}");
}
/// A fatal `apply` error must become visible to the PRODUCER, not only to
/// `finish()`. The consumer keeps draining after the error (so the producer
/// never blocks on a dead receiver), which meant every `send_with_halt`
/// returned `Ok` for the rest of the run: on a 60 GB mkv:// mux that hit
/// ENOSPC on the first frame, the mux driver read the entire remaining title —
/// an hour of optical-drive time — before learning the write had died.
#[test]
fn send_with_halt_fails_fast_once_apply_has_failed() {
struct FailFirst {
failed: Arc<AtomicUsize>,
}
impl Sink<u64> for FailFirst {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
self.failed.fetch_add(1, Ordering::SeqCst);
Err(Error::DecryptFailed)
}
fn close(self) -> Result<(), Error> {
Ok(())
}
}
let applied = Arc::new(AtomicUsize::new(0));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
FailFirst {
failed: applied.clone(),
},
)
.expect("spawn");
let halt = crate::halt::Halt::new();
let deadline = Duration::from_secs(5);
// Feed one item and wait until the consumer has actually applied (and
// failed on) it, so the check below is deterministic rather than racy.
pipe.send_with_halt(0u64, &halt, deadline)
.expect("the first send lands");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && applied.load(Ordering::SeqCst) == 0 {
std::thread::sleep(Duration::from_millis(5));
}
assert_eq!(applied.load(Ordering::SeqCst), 1, "apply ran and failed");
assert!(pipe.consumer_failed(), "the failure must be observable");
// The very next send must hand the item straight back — the producer's
// signal to stop reading the disc.
assert_eq!(
pipe.send_with_halt(1u64, &halt, deadline),
Err(1u64),
"send_with_halt must fail fast once the consumer's apply has failed"
);
// The halt was never fired, so this is not a cancellation: the real error
// still comes out of finish().
assert!(matches!(pipe.finish(), Err(Error::DecryptFailed)));
assert_eq!(
applied.load(Ordering::SeqCst),
1,
"no further item was applied"
);
}
/// The abandon/finalise race. A consumer that has ALREADY committed to
/// `close()` when the grace period expires cannot be stopped — the finalise is
/// happening — so the caller must wait for its result instead of reporting the
/// output as un-finalised. With a plain flag the consumer read it as clear, the
/// caller then stored it, and the caller returned `Err(Halted)`
/// (`completed = false`) while a fully finalised MKV (Cues written, Segment
/// size patched) landed on disk — a truncated rip indistinguishable from a
/// complete one.
#[test]
fn abandon_loses_to_a_close_already_committed() {
let state = Arc::new(AtomicU8::new(state::RUNNING));
let release = Arc::new(AtomicBool::new(false));
let in_close = Arc::new(AtomicBool::new(false));
let (st, rel, inc) = (state.clone(), release.clone(), in_close.clone());
let handle = thread::Builder::new()
.name("test-consumer".into())
.spawn(move || -> Result<u64, Error> {
// Exactly what the consumer does before finalising: claim the
// right to close.
assert!(
st.compare_exchange(
state::RUNNING,
state::CLOSING,
Ordering::AcqRel,
Ordering::Acquire
)
.is_ok(),
"the consumer claims the finalise first"
);
inc.store(true, Ordering::SeqCst);
// Inside `close()`, finalising the container.
while !rel.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
Ok(42)
})
.expect("spawn");
let until = Instant::now() + Duration::from_secs(2);
while Instant::now() < until && !in_close.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(5));
}
assert!(in_close.load(Ordering::SeqCst), "consumer reached close()");
// Finish the close only AFTER the first grace window has expired, so the
// caller genuinely reaches the abandon decision with a close in flight.
let rel = release.clone();
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
thread::sleep(Duration::from_millis(600));
rel.store(true, Ordering::SeqCst);
});
let grace = Duration::from_millis(300);
let res = finish_with_grace(handle, &state, grace, Error::Halted);
assert!(
matches!(res, Ok(42)),
"a finalise already in flight must be waited for, not abandoned: {res:?}"
);
assert_eq!(
state.load(Ordering::SeqCst),
state::CLOSING,
"the caller must not have overwritten the consumer's claim"
);
}
}
+1 -1
View File
@@ -272,7 +272,7 @@ impl WritebackPipeline {
self.chunk_bytes,
self.skip_wait(),
);
if self.chunk_count % SIZE_LOG_INTERVAL == 0 {
if self.chunk_count.is_multiple_of(SIZE_LOG_INTERVAL) {
tracing::debug!(
target: "mux",
"WritebackPipeline chunk_bytes={} after {} chunks is_nfs={} degraded={}",
+84 -29
View File
@@ -30,14 +30,15 @@ pub(super) fn preallocate(file: &File, size_bytes: u64) {
);
}
/// Run `fsync` on `file` with a 60 s deadline. On timeout — and
/// likewise on halt or a lost worker — we log and return `Ok(())`: the
/// kernel will still flush on close, so the data is best-effort durable.
/// The alternative (trap the thread for the rest of the rip, or return
/// an error that aborts an otherwise-complete mux) is worse, so all
/// three fallbacks return `Ok(())`. `Ok(())` from these paths is NOT a
/// durability barrier — the durable flush did not complete; only the
/// hang is bounded.
/// Run `fsync` on `file` with a 60 s deadline. On timeout, halt or a lost
/// worker we log and return `Err` — matching macOS. POSIX gives `fsync`
/// exactly one way to say "the data is on stable storage" and that is a zero
/// return; a call that never reached the device has not earned it, so `Ok(())`
/// from here means the flush completed and nothing else.
///
/// The kernel will still flush on close, so the data is usually durable
/// anyway — but that is a probability, not a barrier, and a caller that needs
/// crash-consistency has to be able to tell the difference.
///
/// ## fd-reuse safety
///
@@ -78,27 +79,7 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; kernel will flush on close (best-effort)"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::WorkerLost) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(e) => bounded_failure_to_result(e),
}
}
@@ -143,3 +124,77 @@ mod tests {
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `fsync` onto the
/// `io::Error` `durable_sync` returns.
///
/// Every arm means the same thing: **no sync observably ran**. All three used to
/// return `Ok(())`, so `WritebackFile::sync_all` reported success for a
/// durability barrier that never happened. POSIX gives `fsync` one way to say
/// "the data is on stable storage" — a zero return — and a call that never
/// reached the device has not earned it.
///
/// This mirrors the macOS `F_FULLFSYNC` mapping exactly. The two were found
/// carrying the identical defect, and a platform disagreeing with its sibling
/// about whether a failed sync is an error is the "works on my platform" class
/// this crate has been bitten by before — most recently an over-length SCSI CDB
/// that macOS rejected and the other two silently truncated.
///
/// No message text (this crate ships no user-facing English): the kind, and
/// `EIO` for the worker-lost case, are the signal; `tracing` carries the detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
// EIO, matching the macOS sibling: a consumer distinguishing these
// three failures does so on the same value on every platform.
// ErrorKind::Other carries nothing a caller can branch on.
Err(crate::error::Error::SyncWorkerLost.into())
}
}
}
#[cfg(test)]
mod bounded_failure_tests {
use super::*;
use crate::io::bounded::BoundedError;
/// Every bounded-fsync failure must be an error. Asserted per variant rather
/// than as a loop so a new variant defaulting to Ok cannot slip through.
#[test]
fn no_bounded_fsync_failure_maps_to_ok() {
assert_eq!(
bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out fsync must be an error")
.kind(),
io::ErrorKind::TimedOut
);
assert_eq!(
bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted fsync must be an error")
.kind(),
io::ErrorKind::Interrupted
);
assert!(
bounded_failure_to_result(BoundedError::WorkerLost).is_err(),
"a lost fsync worker must be an error"
);
}
}
+107 -5
View File
@@ -105,15 +105,46 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
Err(e) => bounded_failure_to_result(e),
}
}
/// Map a [`crate::io::bounded::BoundedError`] from the bounded `F_FULLFSYNC`
/// onto the `io::Error` `durable_sync` returns.
///
/// Every arm here means the same thing: **no sync observably ran**. All three
/// previously returned `Ok(())`, so `WritebackFile::sync_all` reported success
/// for a durability barrier that never happened — a total failure exiting 0,
/// with only a log line to distinguish it. POSIX gives `fsync` exactly one way
/// to say "the data is on stable storage" and that is a zero return; a call
/// that never reached the device has not earned it.
///
/// The errors carry no message text (this crate ships no user-facing English):
/// the kind, and `EIO` for the worker-lost case, are the whole signal, and the
/// `tracing` lines above/below carry the operator detail.
fn bounded_failure_to_result(e: crate::io::bounded::BoundedError) -> io::Result<()> {
match e {
crate::io::bounded::BoundedError::Timeout => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; kernel will flush on close (best-effort)"
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; data NOT durably flushed, kernel will flush on close"
);
Ok(())
Err(crate::error::Error::SyncTimeout.into())
}
crate::io::bounded::BoundedError::Halted => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC skipped (halt requested); data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::Halted.into())
}
crate::io::bounded::BoundedError::WorkerLost => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC worker lost before completion; data NOT durably flushed, kernel will flush on close"
);
Err(crate::error::Error::SyncWorkerLost.into())
}
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
}
}
@@ -156,4 +187,75 @@ mod tests {
// durable_sync must complete without error on the local tempfile.
durable_sync(f.as_file()).expect("durable_sync must return Ok on a local tempfile");
}
/// Every `BoundedError` arm of the bounded `F_FULLFSYNC` means no sync
/// observably ran. All three returned `Ok(())`, so `sync_all` reported a
/// durability barrier that never happened — the caller could not tell a
/// completed flush from a skipped one by any means except reading a log.
///
/// Asserted on the concrete `ErrorKind` / `errno` each arm must produce,
/// so a future arm that quietly reverts to `Ok(())` fails here.
#[test]
fn every_bounded_failure_is_reported_as_an_error() {
use crate::io::bounded::BoundedError;
let timeout = bounded_failure_to_result(BoundedError::Timeout)
.expect_err("a timed-out F_FULLFSYNC must be an error");
assert_eq!(
timeout.kind(),
io::ErrorKind::TimedOut,
"a timed-out F_FULLFSYNC must not be reported as a completed sync"
);
let halted = bounded_failure_to_result(BoundedError::Halted)
.expect_err("a halted F_FULLFSYNC must be an error");
assert_eq!(
halted.kind(),
io::ErrorKind::Interrupted,
"a halted F_FULLFSYNC must not be reported as a completed sync"
);
// The three arms must be DISTINGUISHABLE, not merely non-Ok. Each
// carries its own numeric code through the "E<code>" prefix that
// `From<Error> for io::Error` mints — the only shape `error_code`
// recognises. A bare `ErrorKind` cannot be classified, which is how a
// user cancel here used to read as a hard I/O failure.
let lost = bounded_failure_to_result(BoundedError::WorkerLost)
.expect_err("a lost F_FULLFSYNC worker must be an error");
assert!(
lost.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_WORKER_LOST)),
"a lost worker must be identifiable, got {lost}"
);
assert!(
timeout
.to_string()
.starts_with(&format!("E{}", crate::error::E_SYNC_TIMEOUT)),
"a timeout must be distinguishable from a lost worker, got {timeout}"
);
assert!(
crate::error::is_halt(&halted),
"a halt must satisfy the crate's own is_halt(), or the CLI reports a \
user cancel as a failure; got {halted}"
);
}
/// The failure path must be reachable through the public surface: a
/// `WritebackFile::sync_all` that hits any of these arms must surface an
/// `Err`, not a silent `Ok`. Pinned at the mapping boundary because the
/// timeout itself is not deterministically inducible in a unit test.
#[test]
fn bounded_failures_are_never_mapped_to_ok() {
use crate::io::bounded::BoundedError;
for e in [
BoundedError::Timeout,
BoundedError::Halted,
BoundedError::WorkerLost,
] {
assert!(
bounded_failure_to_result(e).is_err(),
"a bounded F_FULLFSYNC failure must never map to Ok"
);
}
}
}
+21 -15
View File
@@ -97,7 +97,7 @@ fn writeback_chunk_bytes() -> u64 {
.unwrap_or(WRITEBACK_CHUNK_BYTES_DEFAULT)
}
pub(crate) struct WritebackFile {
pub struct WritebackFile {
file: File,
pipeline: WritebackPipeline,
pos: u64,
@@ -115,7 +115,7 @@ impl WritebackFile {
/// once so the pipeline starts tracking from wherever the file
/// already is (typically 0 for fresh files; non-zero for resumed
/// or appended files).
pub(crate) fn new(mut file: File) -> io::Result<Self> {
pub fn new(mut file: File) -> io::Result<Self> {
let pos = file.stream_position()?;
let pipeline = WritebackPipeline::new(&file, pos, writeback_chunk_bytes());
Ok(Self {
@@ -136,7 +136,7 @@ impl WritebackFile {
/// [`Self::create_with_size_hint`] so the kernel can pre-reserve
/// extents.
#[allow(dead_code)]
pub(crate) fn create(path: &Path) -> io::Result<Self> {
pub fn create(path: &Path) -> io::Result<Self> {
let file = File::create(path)?;
Self::new(file)
}
@@ -153,7 +153,7 @@ impl WritebackFile {
/// On platforms without an extent-preallocation primitive this is
/// equivalent to `create` — the size hint is dropped after a debug
/// log.
pub(crate) fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
pub fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
let file = File::create(path)?;
platform::preallocate(&file, size_bytes);
Self::new(file)
@@ -163,7 +163,7 @@ impl WritebackFile {
/// wrap it. Mirrors `File::open` semantics for the writable case
/// — used by patch / resume paths that mutate an existing ISO in
/// place.
pub(crate) fn open(path: &Path) -> io::Result<Self> {
pub fn open(path: &Path) -> io::Result<Self> {
let file = OpenOptions::new().write(true).open(path)?;
Self::new(file)
}
@@ -178,13 +178,20 @@ impl WritebackFile {
/// is left to the kernel's normal flush-on-close path — best
/// effort, but bounded.
///
/// IMPORTANT: on Linux/macOS a successful `Ok(())` does NOT
/// guarantee the data is durable if the bounded fsync timed out or
/// was halted — only the hang is bounded, the fsync may not have
/// completed. Callers needing crash-consistency (e.g. mux-finish
/// then external commit/DB update) must not treat `Ok(())` as a
/// durability barrier.
pub(crate) fn sync_all(&mut self) -> io::Result<()> {
/// A bounded-fsync failure is returned as an `Err` on BOTH platforms, so
/// `Ok(())` means the flush completed and a caller needing
/// crash-consistency can treat it as a durability barrier.
///
/// The three causes are DISTINGUISHABLE by numeric code, because a caller
/// should not retry a lost worker the way it retries a timeout, and must
/// not report a user cancel as a failure:
///
/// * [`E_SYNC_TIMEOUT`](crate::error::E_SYNC_TIMEOUT) — deadline expired
/// * [`E_HALTED`](crate::error::E_HALTED) — cancelled;
/// [`is_halt`](crate::error::is_halt) recognises it
/// * [`E_SYNC_WORKER_LOST`](crate::error::E_SYNC_WORKER_LOST) — the worker
/// thread died before reporting
pub fn sync_all(&mut self) -> io::Result<()> {
if self.seek_count > 0 {
tracing::debug!(
target: "mux",
@@ -256,9 +263,8 @@ impl super::sink::SequentialSink for WritebackFile {
/// the same work [`Self::sync_all`] does. Implemented explicitly (no
/// blanket impl) so a `dyn SequentialSink` / `dyn RandomAccessSink`
/// `finish()` actually finalises + fsyncs instead of hitting a no-op
/// default. Note the bounded-fsync caveat from [`Self::sync_all`]
/// applies: `Ok(())` is not a durability barrier if the fsync timed
/// out or was halted.
/// default. A bounded-fsync failure surfaces as an `Err` here, on every
/// platform, exactly as it does from [`Self::sync_all`].
fn finish(&mut self) -> io::Result<()> {
self.sync_all()
}
+807 -91
View File
File diff suppressed because it is too large Load Diff
+45 -16
View File
@@ -100,10 +100,10 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<DiscMetadata>
// Disc-set position is disc-global; first one we successfully
// read wins. (All bdmt_*.xml on a given disc carry the same
// value in practice.)
if out.disc_number.is_none() {
if let Some(ds) = disc_set {
out.disc_number = Some(ds);
}
if out.disc_number.is_none()
&& let Some(ds) = disc_set
{
out.disc_number = Some(ds);
}
}
@@ -184,20 +184,20 @@ fn extract_title(xml_text: &str) -> Option<String> {
// xml::text already trims its result, so an empty string after
// extraction means a genuinely empty element.
for tag in ["name", "title"] {
if let Some(s) = xml::text(xml_text, tag) {
if !s.is_empty() {
return Some(s);
}
if let Some(s) = xml::text(xml_text, tag)
&& !s.is_empty()
{
return Some(s);
}
}
// tableOfContents/titleName: search inside the toc block so we
// don't accidentally pick a stray <titleName> from elsewhere.
if let Some((s, e)) = xml::find_element(xml_text, "tableOfContents", 0) {
let block = &xml_text[s..e];
if let Some(t) = xml::text(block, "titleName") {
if !t.is_empty() {
return Some(t);
}
if let Some(t) = xml::text(block, "titleName")
&& !t.is_empty()
{
return Some(t);
}
}
None
@@ -370,10 +370,10 @@ mod tests {
if let Some(d) = desc {
meta.descriptions.insert(lang.to_string(), d);
}
if meta.disc_number.is_none() {
if let Some(d) = ds {
meta.disc_number = Some(d);
}
if meta.disc_number.is_none()
&& let Some(d) = ds
{
meta.disc_number = Some(d);
}
}
@@ -566,4 +566,33 @@ mod tests {
let (title, _, _) = parse_bdmt_xml(xml).unwrap();
assert_eq!(title, "Real Title");
}
/// `is_bdmt_filename` must recognize the `bdmt_<lang>.xml` convention
/// and reject everything else — it drives `detect`'s directory scan.
/// Mutation: stub the return to a constant `true`/`false` → every
/// directory listing (or none) would match regardless of filename.
#[test]
fn is_bdmt_filename_matches_convention_only() {
assert!(is_bdmt_filename("bdmt_eng.xml"));
assert!(is_bdmt_filename("BDMT_FRA.XML"));
assert!(!is_bdmt_filename("bdmt_engl.xml"));
assert!(!is_bdmt_filename("index.bdmv"));
assert!(!is_bdmt_filename("foo.xml"));
}
/// Spec: "Disc 1 of 1" (a single-disc release whose bdmt XML still
/// carries `<di:numSets>1</di:numSets>`) is a valid, non-nonsensical
/// pair — `total < 1` must reject only `total == 0`, not `total == 1`.
/// Mutation: `total < 1` -> `total == 1` or `total <= 1` would reject
/// this legitimate (1, 1) pair as if it were malformed.
#[test]
fn disc_set_allows_single_disc_release() {
let xml = r#"<discInfo xmlns:di="urn:BDA:bdmv;disclibmeta">
<di:name>Film</di:name>
<di:discNumber>1</di:discNumber>
<di:numSets>1</di:numSets>
</discInfo>"#;
let (_, _, set) = parse_bdmt_xml(xml).unwrap();
assert_eq!(set, Some((1, 1)));
}
}
+387 -1
View File
@@ -989,7 +989,18 @@ impl<'a> Reader<'a> {
}
fn slice(&mut self, n: usize, needed: &'static str) -> Result<&'a [u8]> {
if self.pos + n > self.data.len() {
// `n` is attacker-supplied: it comes from a JVMS `u4` attribute_length
// / code_length (§4.7, §4.7.3) or a `u2` Utf8 length (§4.4.7). Unlike
// the fixed-width readers above, whose `self.pos + k` cannot leave the
// buffer's own address range, `self.pos + n` can wrap — on a 32-bit
// target a `u4` length near 0xFFFF_FFFF plus a non-zero `pos` panics
// in debug and in release wraps to a SMALL end offset that passes the
// bounds check, after which the slice index itself panics. Checked, so
// an out-of-range length is the EOF error it always should have been.
let Some(end) = self.pos.checked_add(n) else {
return Err(Error::UnexpectedEof { needed });
};
if end > self.data.len() {
return Err(Error::UnexpectedEof { needed });
}
let s = &self.data[self.pos..self.pos + n];
@@ -1006,6 +1017,46 @@ impl<'a> Reader<'a> {
mod tests {
use super::*;
/// `Reader::slice` takes an attacker-supplied length: a JVMS `u4`
/// `attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8
/// length (§4.4.7). Adding it to `pos` without a wrap check panics on
/// overflow in debug and, in release, wraps to a small end offset that
/// slips past the bounds check and then panics inside the slice index.
/// Both are panics escaping a parser whose whole input is untrusted disc
/// bytes; the contract is an EOF error.
#[test]
fn slice_rejects_a_length_that_would_wrap_pos() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
r.u64("advance pos").expect("8 bytes available");
// pos is now 8; usize::MAX would wrap the end offset to 7.
match r.slice(usize::MAX, "wrapping length") {
Err(Error::UnexpectedEof { .. }) => {}
Err(other) => panic!("expected UnexpectedEof, got {other:?}"),
Ok(s) => panic!("expected UnexpectedEof, got a {}-byte slice", s.len()),
}
// The reader must not have consumed anything.
match r.slice(8, "remaining bytes") {
Ok(s) => assert_eq!(s.len(), 8, "pos moved on the rejected slice"),
Err(e) => panic!("the remaining 8 bytes must still be readable: {e:?}"),
}
}
/// The ordinary out-of-range case (no wrap) must keep returning EOF, and
/// an exactly-fitting length must still succeed — the check is `>`, not
/// `>=`.
#[test]
fn slice_boundary_is_inclusive_of_the_final_byte() {
let data = [0u8; 16];
let mut r = Reader::new(&data);
assert_eq!(r.slice(16, "whole buffer").expect("exact fit").len(), 16);
let mut r = Reader::new(&data);
assert!(matches!(
r.slice(17, "one past"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn rejects_non_class_bytes() {
match ClassFile::parse(b"\x00\x01\x02\x03DEAD") {
@@ -1337,4 +1388,339 @@ mod tests {
let _ = decode_modified_utf8(&buf);
}
}
// -----------------------------------------------------------------
// ConstantPool / ClassFile accessor correctness
//
// These exercise plain data accessors on an already-parsed pool
// (built via the test-only `from_entries` constructor) — not the
// untrusted-bytes parsing path, just "does the right variant map to
// the right Option value."
// -----------------------------------------------------------------
fn sample_pool() -> ConstantPool {
// index: 0=Empty (reserved), 1=Utf8("Hello"), 2=Integer(42),
// 3=String{string_index:1}, 4=Class{name_index:1}, 5=Float(1.5),
// 6=Long(9), 7=Empty (2-slot tail), 8=Double(2.5), 9=Empty (tail).
ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("Hello".to_string()),
CpInfo::Integer(42),
CpInfo::String { string_index: 1 },
CpInfo::Class { name_index: 1 },
CpInfo::Float(1.5),
CpInfo::Long(9),
CpInfo::Empty,
CpInfo::Double(2.5),
CpInfo::Empty,
])
}
#[test]
fn constant_pool_string_resolves_through_string_index() {
let pool = sample_pool();
// index 3 is CpInfo::String{string_index: 1} -> utf8(1) = "Hello".
assert_eq!(pool.string(3), Some("Hello"));
// Wrong variant (Integer at index 2) must not resolve as a string.
assert_eq!(pool.string(2), None);
// Out of range index.
assert_eq!(pool.string(999), None);
}
#[test]
fn constant_pool_integer_resolves_only_integer_entries() {
let pool = sample_pool();
assert_eq!(pool.integer(2), Some(42));
// Wrong variant (Utf8 at index 1) must not resolve as an integer.
assert_eq!(pool.integer(1), None);
assert_eq!(pool.integer(999), None);
}
#[test]
fn constant_pool_load_constant_display_covers_ldc_operand_kinds() {
let pool = sample_pool();
assert_eq!(
pool.load_constant_display(1),
Some("utf8:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(2), Some("int:42".to_string()));
assert_eq!(
pool.load_constant_display(3),
Some("str:\"Hello\"".to_string())
);
assert_eq!(
pool.load_constant_display(4),
Some("class:\"Hello\"".to_string())
);
assert_eq!(pool.load_constant_display(5), Some("float:1.5".to_string()));
assert_eq!(pool.load_constant_display(6), Some("long:9".to_string()));
assert_eq!(
pool.load_constant_display(8),
Some("double:2.5".to_string())
);
// A variant with no display arm (e.g. reserved Empty slot) -> None.
assert_eq!(pool.load_constant_display(0), None);
assert_eq!(pool.load_constant_display(999), None);
}
#[test]
fn constant_pool_len_and_is_empty() {
let pool = sample_pool();
assert_eq!(pool.len(), 10);
assert!(!pool.is_empty());
let empty = ConstantPool::from_entries(vec![]);
assert_eq!(empty.len(), 0);
assert!(empty.is_empty());
}
#[test]
fn constant_pool_iter_yields_index_and_entry_pairs() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("A".to_string()),
CpInfo::Integer(7),
]);
let indices: Vec<u16> = pool.iter().map(|(i, _)| i).collect();
assert_eq!(indices, vec![0, 1, 2]);
// Confirm the entries themselves come through, not an empty iterator.
let utf8_at_1 = pool.iter().find(|(i, _)| *i == 1).map(|(_, e)| match e {
CpInfo::Utf8(s) => s.as_str(),
_ => "?",
});
assert_eq!(utf8_at_1, Some("A"));
}
fn class_file_with(this_class: u16, super_class: u16, pool: ConstantPool) -> ClassFile {
ClassFile {
minor_version: 0,
major_version: 0,
constant_pool: pool,
access_flags: 0,
this_class,
super_class,
interfaces: Vec::new(),
fields: Vec::new(),
methods: Vec::new(),
attributes: Vec::new(),
}
}
#[test]
fn this_class_name_and_super_class_name_resolve_distinct_indices() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("com/example/Foo".to_string()),
CpInfo::Utf8("com/example/Bar".to_string()),
CpInfo::Class { name_index: 1 },
CpInfo::Class { name_index: 2 },
]);
let cf = class_file_with(3, 4, pool);
assert_eq!(cf.this_class_name(), Some("com/example/Foo"));
assert_eq!(cf.super_class_name(), Some("com/example/Bar"));
// this_class index pointing at a non-Class entry must not resolve.
let pool2 = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("not a class ref".to_string()),
]);
let cf2 = class_file_with(1, 1, pool2);
assert_eq!(cf2.this_class_name(), None);
assert_eq!(cf2.super_class_name(), None);
}
#[test]
fn member_descriptor_resolves_the_descriptor_not_the_name() {
let pool = ConstantPool::from_entries(vec![
CpInfo::Empty,
CpInfo::Utf8("doStuff".to_string()), // index 1: name
CpInfo::Utf8("()V".to_string()), // index 2: descriptor
]);
let cf = class_file_with(0, 0, pool);
let m = Member {
access_flags: 0,
name_index: 1,
descriptor_index: 2,
attributes: Vec::new(),
};
assert_eq!(cf.member_descriptor(&m), Some("()V"));
assert_ne!(cf.member_descriptor(&m), Some("doStuff"));
}
// -----------------------------------------------------------------
// Reader::u16/u32/u64 boundary + value correctness
//
// Mirrors `slice_boundary_is_inclusive_of_the_final_byte`: an
// exact-fit read must succeed, one byte short must fail. Plus
// positive-value tests so a scrambled byte assembly (not just an
// out-of-bounds read) would be caught.
// -----------------------------------------------------------------
#[test]
fn u16_boundary_is_inclusive_of_the_final_byte() {
let data = [0xAB, 0xCD];
let mut r = Reader::new(&data);
assert_eq!(r.u16("exact fit").expect("2 bytes available"), 0xABCD);
let data = [0xAB];
let mut r = Reader::new(&data);
assert!(matches!(
r.u16("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u16_decodes_big_endian_value() {
let data = [0x01, 0x02];
let mut r = Reader::new(&data);
assert_eq!(r.u16("value").unwrap(), 0x0102);
}
#[test]
fn u32_boundary_is_inclusive_of_the_final_byte() {
let data = [0x00, 0x00, 0x00, 0x2A];
let mut r = Reader::new(&data);
assert_eq!(r.u32("exact fit").expect("4 bytes available"), 42);
let data = [0x00, 0x00, 0x00];
let mut r = Reader::new(&data);
assert!(matches!(
r.u32("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u32_decodes_big_endian_value() {
let data = [0x00, 0x00, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u32("value").unwrap(), 1337);
}
#[test]
fn u64_boundary_is_inclusive_of_the_final_byte() {
// pos == 0, buffer exactly 8 bytes: must succeed.
let data = [0, 0, 0, 0, 0, 0, 0, 0x7B]; // 123
let mut r = Reader::new(&data);
assert_eq!(r.u64("exact fit").expect("8 bytes available"), 123);
// pos == 0, buffer one byte short of 8: must fail cleanly, not
// panic on the internal self.data[self.pos + 7] index.
let data = [0u8; 7];
let mut r = Reader::new(&data);
assert!(matches!(
r.u64("one byte short"),
Err(Error::UnexpectedEof { .. })
));
}
#[test]
fn u64_decodes_big_endian_value() {
let data = [0, 0, 0, 0, 0, 0, 0x05, 0x39]; // 1337
let mut r = Reader::new(&data);
assert_eq!(r.u64("value").unwrap(), 1337);
}
// -----------------------------------------------------------------
// decode_modified_utf8: 3-byte (BMP) decode path
// -----------------------------------------------------------------
#[test]
fn modified_utf8_three_byte_cjk() {
// U+3042 (hiragana あ) in modified UTF-8: 1110xxxx 10xxxxxx 10xxxxxx
// = 0xE3 0x81 0x82.
let s = decode_modified_utf8(&[0xE3, 0x81, 0x82]).unwrap();
assert_eq!(s, "\u{3042}");
}
#[test]
fn modified_utf8_three_byte_rejects_bad_first_continuation() {
// Valid lead byte (0xE3), but the first continuation byte is not
// 10xxxxxx (0x01 instead) — must be rejected, proving the first
// `& 0xC0 != 0x80` check is live.
assert!(decode_modified_utf8(&[0xE3, 0x01, 0x82]).is_err());
}
#[test]
fn modified_utf8_three_byte_rejects_bad_second_continuation() {
// Valid lead + first continuation, but the second continuation
// byte is not 10xxxxxx — proves the second check is independently
// live (not short-circuited by the first).
assert!(decode_modified_utf8(&[0xE3, 0x81, 0x01]).is_err());
}
// -----------------------------------------------------------------
// read_constant_pool: Long/Double two-slot skip, real byte parsing
// -----------------------------------------------------------------
#[test]
fn constant_pool_long_entry_occupies_two_slots_via_real_parse() {
// Real class-file bytes (not the `from_entries` synthetic ctor):
// magic + minor/major + cp_count=4 + tag=5 (Long, 8-byte payload
// at index 1, reserved slot at index 2) + tag=1 (Utf8 at index 3)
// + empty access_flags/this/super/interfaces/fields/methods/attrs.
let mut buf = vec![
0xCA, 0xFE, 0xBA, 0xBE, // magic
0x00, 0x00, // minor
0x00, 0x34, // major
0x00, 0x04, // cp_count = 4 (0=Empty,1=Long,2=Empty tail,3=Utf8)
5, // Long tag
];
buf.extend_from_slice(&0x1122_3344_5566_7788u64.to_be_bytes()); // 8-byte payload
buf.push(1); // Utf8 tag
let name = b"marker";
buf.extend_from_slice(&(name.len() as u16).to_be_bytes());
buf.extend_from_slice(name);
// access_flags, this_class, super_class, interfaces_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0, 0, 0]);
// fields_count, methods_count, attributes_count
buf.extend_from_slice(&[0, 0, 0, 0, 0, 0]);
let cf = ClassFile::parse(&buf).expect("well-formed synthetic class file");
assert_eq!(cf.constant_pool.len(), 4);
// The Long occupies indices 1 AND 2 (its reserved tail slot).
// The Utf8 must resolve at index 3 = long_index(1) + 2, NOT +1.
assert_eq!(cf.constant_pool.utf8(3), Some("marker"));
// Index 2 is the reserved tail slot: not a Utf8, must not
// resolve as one (guards against the Utf8 landing one slot early).
assert_eq!(cf.constant_pool.utf8(2), None);
match cf.constant_pool.get(1) {
Some(CpInfo::Long(v)) => assert_eq!(*v, 0x1122_3344_5566_7788u64 as i64),
other => panic!("expected Long at index 1, got {:?}", other),
}
}
// -----------------------------------------------------------------
// instruction_size: tableswitch/lookupswitch with non-degenerate
// low/high/npairs (the existing tests only cover low==high==0 and
// npairs==0, which can't distinguish `-` from `+` in the entry-count
// arithmetic).
// -----------------------------------------------------------------
#[test]
fn instruction_size_tableswitch_non_degenerate_range() {
// low=1, high=4 -> 4 entries (high-low+1 = 4). A `-`->`+` mutation
// on that arithmetic would instead compute high+low+1 = 6.
let mut code = vec![TABLESWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default offset
code.extend_from_slice(&1i32.to_be_bytes()); // low = 1
code.extend_from_slice(&4i32.to_be_bytes()); // high = 4
code.extend_from_slice(&[0; 16]); // 4 jump entries * 4 bytes
// total = 1 (opcode) + 3 (pad) + 12 (default/low/high) + 16 (entries) = 32
assert_eq!(instruction_size(&code, 0), Some(32));
}
#[test]
fn instruction_size_lookupswitch_non_degenerate_npairs() {
// npairs = 3 -> 3 * 8 = 24 bytes of pairs.
let mut code = vec![LOOKUPSWITCH];
code.extend_from_slice(&[0, 0, 0]); // padding
code.extend_from_slice(&[0, 0, 0, 0]); // default
code.extend_from_slice(&3i32.to_be_bytes()); // npairs = 3
code.extend_from_slice(&[0; 24]); // 3 pairs
// total = 1 + 3 + 8 (default/npairs) + 24 = 36
assert_eq!(instruction_size(&code, 0), Some(36));
}
}
+242 -56
View File
@@ -16,7 +16,7 @@ use std::collections::HashMap;
/// Cheap signature check: a Criterion disc ships `streamproperties.xml`
/// inside a `/BDMV/JAR/*` archive.
pub fn detect(udf: &UdfFs) -> bool {
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "streamproperties.xml")
}
@@ -36,17 +36,18 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
// Stream number mapping from playbackconfig.xml
let mut stream_map: HashMap<String, u16> = HashMap::new();
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml") {
if let Ok(pc_text) = std::str::from_utf8(&pc_data) {
parse_playback_config(pc_text, &mut stream_map);
}
if let Some(pc_data) = super::read_jar_file(reader, udf, "playbackconfig.xml")
&& let Ok(pc_text) = std::str::from_utf8(&pc_data)
{
parse_playback_config(pc_text, &mut stream_map);
}
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map);
let stream_nums = assign_stream_numbers(&stream_infos, &stream_map)?;
let mut labels = Vec::new();
for (info, &stream_num) in stream_infos.iter().zip(stream_nums.iter()) {
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type: info.stream_type,
language: info.language.clone(),
@@ -76,12 +77,41 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
/// map-assigned one. (Both numbering domains are 1-based per type, and
/// `apply_labels` matches on `(type, stream_number)`, so a collision
/// would mislabel tracks.)
fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>) -> Vec<u16> {
// Numbers already claimed by the map, per type.
///
/// Returns `None` when the 1-based stream-number space is exhausted — every
/// number in `1..=u16::MAX` for that type is either already claimed by the map
/// or already synthesized. That is unreachable on real media: the BD STN_table
/// carries at most 32 primary audio and 32 PG streams per playlist, so the
/// 65535-wide space leaves >2000x headroom. It IS reachable from a crafted
/// `streamproperties.xml` listing >65535 stream entries, and the only correct
/// answers there are "fail the parse" or "emit colliding numbers"; we fail.
///
/// The skip search is bounded by the numbering space itself: a `u16`
/// `saturating_add` here parked the counter at `u16::MAX` forever whenever the
/// map also claimed `u16::MAX`, turning an overflow guard into a hang that
/// `apply()`'s `catch_unwind` cannot interrupt. The counters are therefore
/// widened to `u32` so the skip loop strictly increases toward a fixed ceiling
/// (guaranteeing termination) and exhaustion is reported rather than absorbed.
fn assign_stream_numbers(
infos: &[StreamInfo],
stream_map: &HashMap<String, u16>,
) -> Option<Vec<u16>> {
/// One past the last assignable stream number, as a `u32` so the
/// counters can step off the end of the `u16` domain without wrapping.
const NUMBER_SPACE_END: u32 = u16::MAX as u32 + 1;
// Numbers already claimed by the map, per type. A map value of 0 is NOT a
// claim: apply_labels binds on 1-based stream numbers, so 0 is unmatchable.
// Treat 0 as "unmapped" here (defense in depth — parse_playback_config also
// filters it) so such a stream gets a real synthesized number instead of an
// orphan 0 that collides with / shadows a genuine stream 1.
let mut taken_audio: Vec<u16> = Vec::new();
let mut taken_sub: Vec<u16> = Vec::new();
for info in infos {
if let Some(&n) = stream_map.get(&info.id) {
if n == 0 {
continue;
}
match info.stream_type {
StreamLabelType::Audio => taken_audio.push(n),
StreamLabelType::Subtitle => taken_sub.push(n),
@@ -89,32 +119,42 @@ fn assign_stream_numbers(infos: &[StreamInfo], stream_map: &HashMap<String, u16>
}
}
let mut audio_idx: u16 = 1;
let mut sub_idx: u16 = 1;
let mut audio_idx: u32 = 1;
let mut sub_idx: u32 = 1;
let mut out = Vec::with_capacity(infos.len());
for info in infos {
let n = match stream_map.get(&info.id).copied() {
Some(n) => n,
None => {
Some(n) if n != 0 => n,
_ => {
let (idx, taken) = match info.stream_type {
StreamLabelType::Audio => (&mut audio_idx, &taken_audio),
StreamLabelType::Subtitle => (&mut sub_idx, &taken_sub),
};
// Advance past any number already claimed via the map.
// saturating: a crafted XML with >65k stream entries must
// not overflow (panic in debug, wrap-to-0 in release) on
// untrusted disc bytes.
while taken.contains(idx) {
*idx = idx.saturating_add(1);
// Advance past any number already claimed via the map. The
// counter strictly increases and NUMBER_SPACE_END is fixed, so
// this terminates in at most 65535 steps for any input.
while *idx < NUMBER_SPACE_END && taken.contains(&(*idx as u16)) {
*idx += 1;
}
let n = *idx;
*idx = idx.saturating_add(1);
if *idx >= NUMBER_SPACE_END {
// Numbering space exhausted. Emitting anything here would
// either wrap to 0 (unmatchable) or duplicate a number
// already bound to a different stream, so the parse fails.
tracing::warn!(
streams = infos.len(),
"criterion: 1-based u16 stream-number space exhausted; \
refusing to synthesize a colliding stream number"
);
return None;
}
let n = *idx as u16;
*idx += 1;
n
}
};
out.push(n);
}
out
Some(out)
}
struct StreamInfo {
@@ -182,14 +222,13 @@ fn parse_playback_config(text: &str, map: &mut HashMap<String, u16>) {
if let (Some(stream_id_str), Some(info_id)) = (
xml::text(block, "StreamID"),
xml::text(block, "StreamInfo_ID"),
) {
if let Ok(stream_num) = stream_id_str.parse::<u16>() {
// Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it.
if stream_num != 0 {
map.insert(info_id, stream_num);
}
) && let Ok(stream_num) = stream_id_str.parse::<u16>()
{
// Stream numbers are 1-based per the apply_labels
// contract; a mapped 0 is unmatchable and silently
// drops the label. Skip it rather than store it.
if stream_num != 0 {
map.insert(info_id, stream_num);
}
}
from = end;
@@ -219,11 +258,89 @@ mod tests {
info("a1", StreamLabelType::Audio),
info("s0", StreamLabelType::Subtitle),
];
let nums = assign_stream_numbers(&infos, &HashMap::new());
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Per-type 1-based: audio 1,2 ; subtitle 1.
assert_eq!(nums, vec![1, 2, 1]);
}
/// Immunity pin. `parse_stream_infos` emits one `StreamInfo` per
/// `*StreamInfos` element unconditionally — no filter, no `continue` — so
/// an element whose fields are missing or unrecognized still occupies its
/// position, and `assign_stream_numbers` still spends a number on it.
///
/// That is the property that keeps this parser out of the failure mode
/// where a skipped entry pulls every later label one stream forward. It
/// is load-bearing for the fallback path specifically: with no
/// `playbackconfig.xml` the numbers come purely from position in this
/// list, so dropping an element there would shift the rest.
///
/// Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
/// real audio streams renumber to 1 and 2.
/// Immunity pin, section-boundary half. Each stream here is one closed XML
/// element, and every field is read out of `&text[start..end]` — the range
/// `xml::find_element` returned — so one element can never absorb the next
/// one's fields, however the document is malformed around it. Contrast the
/// flat-string walk in pixelogic, where a section whose end marker is
/// missing keeps consuming entries as STN slots.
///
/// The missing-boundary case fails closed. An element with no close tag of
/// its own ends at the NEXT close tag, so it absorbs the element behind it
/// — the list comes back SHORTER. It cannot come back longer: nothing
/// outside a returned range is ever read as a stream, and `find_element`
/// yields `None` rather than a range running to EOF when no close tag
/// exists at all. A malformed document can cost this parser a slot; it can
/// never invent one.
///
/// Mutation: read fields from the document rather than the element's
/// range, or let a close-less element run to EOF → the trailing elements
/// re-enter the list as extra streams.
#[test]
fn an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it() {
let sp = concat!(
"<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID></AudioStreamInfos>",
// No `</AudioStreamInfos>` for this one.
"<AudioStreamInfos><ID>a1</ID><LangInfoID>FRA</LangInfoID>",
"<AudioStreamInfos><ID>a2</ID><LangInfoID>DEU</LangInfoID></AudioStreamInfos>",
);
let infos = parse_stream_infos(sp);
assert_eq!(
infos.iter().map(|i| i.id.as_str()).collect::<Vec<_>>(),
vec!["a0", "a1"],
"the close-less element absorbs the one behind it — two slots, not \
three, and never four"
);
assert_eq!(infos[1].language, "fra", "and keeps its own leading fields");
// With no close tag anywhere behind it, the element is not returned at
// all and the walk ends — the tail of the document never becomes a
// stream list.
let no_close = "<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID>";
assert!(parse_stream_infos(no_close).is_empty());
}
#[test]
fn unusable_stream_element_still_occupies_its_position() {
let sp = r#"
<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG_US</LangInfoID></AudioStreamInfos>
<AudioStreamInfos></AudioStreamInfos>
<AudioStreamInfos><ID>a2</ID><LangInfoID>FRA</LangInfoID><Content>COMMENTARY</Content></AudioStreamInfos>
<SubtitleStreamInfos><ID>s0</ID><LangInfoID></LangInfoID><Qualifier>WAT</Qualifier></SubtitleStreamInfos>
<SubtitleStreamInfos><ID>s1</ID><LangInfoID>ENG</LangInfoID><Qualifier>SDH</Qualifier></SubtitleStreamInfos>
"#;
let infos = parse_stream_infos(sp);
assert_eq!(infos.len(), 5, "every element yields a StreamInfo");
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
assert_eq!(
nums,
vec![1, 2, 3, 1, 2],
"the blank element owns audio slot 2, so the commentary is slot 3"
);
assert_eq!(infos[2].purpose, LabelPurpose::Commentary);
assert_eq!(infos[4].qualifier, LabelQualifier::Sdh);
}
#[test]
fn fallback_does_not_collide_with_partial_map() {
// Map claims audio "a1" -> 1. The unmapped audio "a0" must NOT
@@ -235,7 +352,7 @@ mod tests {
info("a1", StreamLabelType::Audio), // mapped → 1
info("a2", StreamLabelType::Audio), // unmapped → fallback
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// a0 skips the taken 1 → 2; a1 keeps 1; a2 → 3. All distinct.
assert_eq!(nums, vec![2, 1, 3]);
let mut sorted = nums.clone();
@@ -253,7 +370,10 @@ mod tests {
info("a0", StreamLabelType::Audio),
info("a1", StreamLabelType::Audio),
];
assert_eq!(assign_stream_numbers(&infos, &map), vec![5, 9]);
assert_eq!(
assign_stream_numbers(&infos, &map).expect("numbering space not exhausted"),
vec![5, 9]
);
}
// ── Additional hardening tests ─────────────────────────────────────────
@@ -269,7 +389,8 @@ mod tests {
info("a1", StreamLabelType::Audio),
info("s1", StreamLabelType::Subtitle),
];
let nums = assign_stream_numbers(&infos, &HashMap::new());
let nums =
assign_stream_numbers(&infos, &HashMap::new()).expect("numbering space not exhausted");
// Audio: 1, 2; Subtitle: 1, 2 — each counter resets at 1 per type.
assert_eq!(nums[0], 1); // audio 1
assert_eq!(nums[1], 1); // subtitle 1
@@ -277,27 +398,34 @@ mod tests {
assert_eq!(nums[3], 2); // subtitle 2
}
/// Spec: map stream_num=0 is explicitly rejected (apply_labels uses 1-based).
/// This is documented in parse_playback_config: `if stream_num != 0`.
/// Mutation: remove the `!= 0` guard → zero is stored in map.
/// Spec: a map value of 0 is unmatchable (apply_labels is 1-based), so
/// assign_stream_numbers must treat it as unmapped and synthesize a real
/// 1-based number rather than emit an orphan 0.
/// Mutation: read the map value verbatim → stream_number 0 leaks out.
#[test]
fn map_zero_stream_num_is_skipped() {
// parse_playback_config skips zero; simulate that: the zero shouldn't
// end up in the map. We test assign_stream_numbers with a zero-containing
// map to verify it won't freeze the fallback counter at 1 forever.
fn map_zero_stream_num_is_synthesized_not_emitted() {
let mut map = HashMap::new();
map.insert("a0".to_string(), 0u16); // zero — per spec, was filtered by parse_playback_config
map.insert("a0".to_string(), 0u16); // 0 must not be treated as a claim
let infos = vec![info("a0", StreamLabelType::Audio)];
// If 0 IS in the map and assign_stream_numbers uses it, stream_number=0
// is not matchable (apply_labels is 1-based). The fallback counter
// would assign 1 instead. Test both paths:
let nums = assign_stream_numbers(&infos, &map);
// If the map has 0 for a0, assign_stream_numbers returns 0 (map wins).
// This is a known limitation — the guard lives in parse_playback_config.
// The test documents the ACTUAL behavior so a code change that introduces
// the guard in assign_stream_numbers would be caught.
// Current behavior: map wins → 0.
assert_eq!(nums[0], 0);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// 0 is treated as unmapped → the fallback counter assigns 1.
assert_eq!(nums[0], 1);
}
/// A stream genuinely mapped to 1 plus another stream whose map value is 0
/// must NOT both land on 1: the 0-stream is synthesized past the claimed 1.
#[test]
fn map_zero_does_not_collide_with_a_real_stream_one() {
let mut map = HashMap::new();
map.insert("real".to_string(), 1u16);
map.insert("bad".to_string(), 0u16);
let infos = vec![
info("real", StreamLabelType::Audio),
info("bad", StreamLabelType::Audio),
];
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
assert_eq!(nums[0], 1); // the genuinely-mapped stream keeps 1
assert_eq!(nums[1], 2); // the 0-stream is synthesized to the next free slot
}
/// Spec: collision-avoidance works across audio AND subtitle independently.
@@ -312,16 +440,74 @@ mod tests {
info("a0", StreamLabelType::Audio), // fallback
info("s0", StreamLabelType::Subtitle), // mapped → 2
];
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
// Audio fallback for a0 → 1 (subtitle's taken-2 doesn't block it).
assert_eq!(nums[0], 1);
assert_eq!(nums[1], 2);
}
/// Spec: saturating_add prevents overflow when many streams are listed.
/// Mutation: use wrapping_add → counter wraps to 0 and collides.
/// A crafted `streamproperties.xml` can drive the fallback counter to the
/// top of the 1-based u16 stream-number space and then present one more
/// unmapped stream whose successor number is also claimed by the map.
///
/// This must TERMINATE. The bound is the numbering space itself, so the
/// assertion is on the spec-derived exhaustion behaviour (`None`), not on
/// any tunable constant. Run on a worker thread with a deadline so a
/// non-terminating loop fails the test in 20 s instead of hanging CI.
#[test]
fn assign_stream_numbers_saturation_on_overflow() {
fn exhausted_numbering_terminates_instead_of_looping() {
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
// One mapped audio stream claims the last number in the space.
let mut map = HashMap::new();
map.insert("claims_max".to_string(), u16::MAX);
let mut infos = vec![info("claims_max", StreamLabelType::Audio)];
// Enough unmapped audio streams to walk the counter to the top.
for i in 0..=(u16::MAX as u32) {
infos.push(info(&format!("u{i}"), StreamLabelType::Audio));
}
let _ = tx.send(assign_stream_numbers(&infos, &map));
});
match rx.recv_timeout(std::time::Duration::from_secs(20)) {
Ok(result) => {
worker.join().expect("worker panicked");
assert!(
result.is_none(),
"an exhausted 1-based u16 numbering space must fail the parse, \
not emit colliding or wrapped stream numbers"
);
}
Err(_) => panic!(
"assign_stream_numbers did not terminate within 20s — \
non-terminating skip loop on crafted stream_map"
),
}
}
/// The whole 1-based u16 space must remain usable: 65535 unmapped audio
/// streams get 65535 distinct numbers with no panic and no wrap. The
/// literals here are the JVMS-independent, spec-derived size of a u16
/// 1-based numbering domain, not a tunable cap.
#[test]
fn full_u16_numbering_space_is_usable_and_unique() {
let infos: Vec<StreamInfo> = (0..65_535u32)
.map(|i| info(&format!("a{i}"), StreamLabelType::Audio))
.collect();
let nums = assign_stream_numbers(&infos, &HashMap::new()).expect("space is not exhausted");
assert_eq!(nums.len(), 65_535);
assert_eq!(nums[0], 1);
assert_eq!(nums[65_534], 65_535);
let mut sorted = nums.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), 65_535, "stream numbers must all be distinct");
}
/// Spec: a partially-mapped playlist with many claimed numbers must still
/// synthesize past every claim without panicking or colliding.
/// Mutation: drop the skip loop → the fallback reuses a claimed number.
#[test]
fn fallback_skips_a_dense_block_of_claimed_numbers() {
// Force the counter past u16::MAX by pre-taking all values 1..=u16::MAX.
// Doing that for real would be slow; instead inject u16::MAX into taken.
let mut map = HashMap::new();
@@ -348,7 +534,7 @@ mod tests {
qualifier: LabelQualifier::None,
});
// This must not panic.
let nums = assign_stream_numbers(&infos, &map);
let nums = assign_stream_numbers(&infos, &map).expect("numbering space not exhausted");
assert_eq!(nums.len(), 501);
// The last (unmapped) entry's number must be > 500 (skipped all taken).
assert!(nums[500] > 500);
+277 -120
View File
@@ -11,7 +11,7 @@ use std::collections::HashMap;
/// Cheap signature check: a CTRM disc ships `menu_base.prop` and/or
/// `language_streams.txt` inside a `/BDMV/JAR/*` archive.
pub fn detect(udf: &UdfFs) -> bool {
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "menu_base.prop")
|| super::jar_file_exists(udf, "language_streams.txt")
}
@@ -50,10 +50,10 @@ fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
if let Some(mb_match) = mb
.iter()
.find(|m| m.stream_type == label.stream_type && m.stream_number == label.stream_number)
&& label.name.is_empty()
&& !mb_match.name.is_empty()
{
if label.name.is_empty() && !mb_match.name.is_empty() {
label.name = mb_match.name.clone();
}
label.name = mb_match.name.clone();
}
}
// Append any menu_base-only stream (present in mb but not in ls by
@@ -87,7 +87,21 @@ fn prefix_is_commentary(prefix: &str) -> bool {
fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
let data = super::read_jar_file(reader, udf, "language_streams.txt")?;
let text = std::str::from_utf8(&data).ok()?;
let labels = parse_language_streams_text(text);
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels.
///
/// This is the shipping parser: [`parse_language_streams`] does the UDF read
/// and UTF-8 decode and then delegates here. It is split out — rather than
/// duplicated under `#[cfg(test)]`, which is what it used to be — so the unit
/// tests below exercise production code. A test that re-implements the
/// function it guards cannot fail when the real function breaks.
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
@@ -193,122 +207,7 @@ fn parse_language_streams(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<
}
labels.push(StreamLabel {
stream_number: stream_num,
stream_type,
language,
name: String::new(),
purpose: final_purpose,
qualifier,
codec_hint,
variant: variant_code,
});
}
if labels.is_empty() {
return None;
}
Some(labels)
}
/// Parse the body of a `language_streams.txt` file into stream labels. Split
/// out from [`parse_language_streams`] so unit tests exercise the real parsing
/// logic without needing a SectorSource / UdfFs.
#[cfg(test)]
fn parse_language_streams_text(text: &str) -> Vec<StreamLabel> {
let mut labels = Vec::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let parts: Vec<&str> = line.split(',').map(|s| s.trim()).collect();
if parts.len() < 4 {
continue;
}
let type_str = parts[1];
let stream_num: u16 = match parts[2].parse() {
Ok(n) if n > 0 => n,
_ => continue,
};
let language = parts[3].to_string();
let variant = if parts.len() > 4 {
parts[4].to_string()
} else {
String::new()
};
let (stream_type, purpose, qualifier) = match type_str {
"audio_production" => (
StreamLabelType::Audio,
LabelPurpose::Normal,
LabelQualifier::None,
),
"audio_commentary" => (
StreamLabelType::Audio,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"audio_ime" => (
StreamLabelType::Audio,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_production" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_commentary" => (
StreamLabelType::Subtitle,
LabelPurpose::Commentary,
LabelQualifier::None,
),
"subtitle_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::Forced,
),
"subtitle_dual" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_bonus" => (
StreamLabelType::Subtitle,
LabelPurpose::Normal,
LabelQualifier::None,
),
"subtitle_ime" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::None,
),
"subtitle_ime_narrative" => (
StreamLabelType::Subtitle,
LabelPurpose::Ime,
LabelQualifier::Forced,
),
_ => continue,
};
let mut codec_hint = String::new();
let mut variant_code = String::new();
let mut final_purpose = purpose;
if !variant.is_empty() {
match variant.as_str() {
"eda" => final_purpose = LabelPurpose::Descriptive,
"csp" | "cs" | "lsp" | "ls" | "cf" | "pf" | "bp" | "pp" => {
variant_code = variant.clone();
}
_ => codec_hint = vocab::codec(&variant).to_string(),
}
}
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type,
language,
@@ -426,6 +325,68 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `menu_base.prop` lines are skipped when `is_empty() ||
/// starts_with('#')` — either alone is sufficient. A commented-out
/// key=value line must never be parsed into an entry.
/// Mutation: `||` -> `&&` requires both, which a non-empty comment
/// line can't satisfy, so it falls through to `line.find('=')` and
/// gets parsed as a real property.
#[test]
fn menu_base_comment_line_with_equals_is_still_skipped() {
let labels = parse_props(
"#audio_1.class=AudioButton\n\
#audio_1.streamNumber=9\n\
#audio_1.name=Should Not Appear\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=1\n\
audio_2.name=Real Track\n",
);
assert_eq!(labels.len(), 1, "commented-out entry must not be parsed");
assert_eq!(labels[0].name, "Real Track");
}
/// Spec: `menu_base.prop` streamNumber (or audioStream/subtitleStream)
/// must be strictly positive — `0` means "no STN entry" and must be
/// skipped, matching the `n > 0` guard on the language_streams side.
/// Mutation: `n > 0` -> `n >= 0` (or the guard deleted) would let a
/// stream_num of 0 through, emitting a dead label apply_labels can
/// never match (its counter starts at 1).
#[test]
fn menu_base_zero_stream_number_skipped() {
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_1.name=Disabled Slot\n",
);
assert!(
labels.is_empty(),
"streamNumber=0 must be skipped, got {labels:?}"
);
}
/// Spec: `is_subtitle` is `class.contains("SubtitleButton") ||
/// prefix.starts_with("subtitle_")` — EITHER signal alone is
/// sufficient to classify (and keep) a subtitle entry whose prefix
/// doesn't follow the `subtitle_` naming convention.
/// Mutation: `||` -> `&&` would require BOTH signals; an entry whose
/// class says SubtitleButton but whose prefix is something else
/// (e.g. a vendor-specific button id) would then satisfy neither
/// `is_audio` nor `is_subtitle` and get dropped entirely.
#[test]
fn menu_base_subtitle_class_alone_is_sufficient() {
let labels = parse_props(
"menuBtn7.class=SubtitleButton\n\
menuBtn7.streamNumber=1\n\
menuBtn7.name=English SDH\n",
);
assert_eq!(
labels.len(),
1,
"class=SubtitleButton alone must classify as subtitle, not be dropped"
);
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
}
#[test]
fn prefix_commentary_segment_match_not_substring() {
// Genuine commentary group segments match.
@@ -441,6 +402,7 @@ mod tests {
fn lbl(t: StreamLabelType, n: u16, name: &str) -> StreamLabel {
StreamLabel {
stream_id: None,
stream_number: n,
stream_type: t,
language: String::new(),
@@ -452,6 +414,39 @@ mod tests {
}
}
/// Spec: `merge`'s `mb.iter().find(...)` must match an mb entry by
/// (stream_type AND stream_number) TOGETHER — either alone is not a
/// unique key (there can be an audio #1 and a subtitle #1, or two
/// different audio streams).
/// Mutation: `&&` -> `||` inside the closure would match on type OR
/// number alone, so `.find` (which returns the FIRST match) can pick
/// an mb entry with the right type but the WRONG stream number.
#[test]
fn merge_matches_mb_entry_by_type_and_number_together() {
// ls wants audio #2 (empty name, so it will borrow from mb).
let ls = vec![lbl(StreamLabelType::Audio, 2, "")];
// mb's FIRST audio entry is #1 (wrong number); its #2 entry (the
// real match) comes second.
let mb = vec![
lbl(StreamLabelType::Audio, 1, "Wrong Number Match"),
lbl(StreamLabelType::Audio, 2, "Correct Match"),
];
let merged = merge(ls, mb);
assert_eq!(
merged.len(),
2,
"mb's own audio #1 must also survive as its own entry"
);
let a2 = merged
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio && l.stream_number == 2)
.unwrap();
assert_eq!(
a2.name, "Correct Match",
"must match mb by (type AND number), not type or number alone"
);
}
#[test]
fn merge_preserves_menu_base_only_streams() {
// language_streams covers audio 1; menu_base has audio 1 (name)
@@ -521,6 +516,47 @@ mod tests {
assert_eq!(labels[0].qualifier, LabelQualifier::Forced);
}
/// Immunity pin against the defect measured in the `paramount` parser,
/// where a vendor `forced_sub` cell hung off a FULL dialogue track's own
/// slot to say "this track also contains forced signs", and reading that
/// cell as "this track is forced" flagged 30 MB dialogue tracks forced.
///
/// This format cannot express that. The forced signal is not a flag beside
/// a track's entry — it IS the entry's stream-kind token, drawn from a
/// closed vocabulary in which `subtitle_production` (the full dialogue
/// track) and `subtitle_narrative` (the forced-narrative track) are
/// mutually exclusive alternatives in the same position. A row is one or
/// the other; there is no cell a full track can carry to acquire the
/// qualifier, so the paramount failure mode has no encoding here.
///
/// Mutation: give `subtitle_production` a `Forced` qualifier, or add a
/// forced side-flag that both kinds may carry.
#[test]
fn a_full_subtitle_track_kind_can_never_carry_the_forced_qualifier() {
// Every subtitle kind in the vocabulary, one row each.
let text = "id1,subtitle_production,1,eng\n\
id2,subtitle_commentary,2,eng\n\
id3,subtitle_dual,3,eng\n\
id4,subtitle_bonus,4,eng\n\
id5,subtitle_ime,5,kor\n\
id6,subtitle_narrative,6,eng\n\
id7,subtitle_ime_narrative,7,kor\n";
let labels = parse_language_streams_text(text);
let forced: Vec<&str> = labels
.iter()
.filter(|l| l.qualifier == LabelQualifier::Forced)
.map(|l| l.language.as_str())
.collect();
assert_eq!(
forced.len(),
2,
"only the two narrative kinds are forced, got {forced:?}"
);
// The full dialogue kind specifically.
let production = parse_language_streams_text("id,subtitle_production,1,eng\n");
assert_eq!(production[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_commentary` → Subtitle / Commentary.
/// Mutation: treat as Normal → subtitle commentary not flagged.
#[test]
@@ -564,6 +600,72 @@ mod tests {
assert!(labels.is_empty());
}
/// Immunity pin. `language_streams.txt` states each stream's number in
/// field 3, so a row the parser cannot use is simply dropped — it can
/// never renumber the rows behind it. This is the property that keeps
/// this parser out of the STN-slot-shifting failure mode that bites
/// parsers which count positionally: there, a skipped entry silently
/// pulls every later label one stream forward.
///
/// Mutation: replace `parts[2]` with a running per-type counter → the
/// three unusable rows here collapse the survivors onto 1/2 and 1.
#[test]
fn ls_stream_numbers_come_from_the_row_not_a_counter() {
let labels = parse_language_streams_text(
"id,audio_production,4,eng\n\
id,audio_bonus_extended,5,eng\n\
id,audio_production,0,fra\n\
id,audio_production,7,fra\n\
id,subtitle_production\n\
id,subtitle_narrative,9,deu\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 4),
(StreamLabelType::Audio, 7),
(StreamLabelType::Subtitle, 9),
],
"an unusable row drops out without shifting the numbering"
);
assert_eq!(labels[2].qualifier, LabelQualifier::Forced);
}
/// Immunity pin, `menu_base.prop` side: the number comes from the
/// entry's own `streamNumber` property, so a skipped entry (commented
/// out, `streamNumber=0`, neither audio nor subtitle) leaves the
/// surviving entries on their authored slots.
///
/// Mutation: number by iteration order → the survivors collapse to 1/2.
#[test]
fn menu_base_stream_numbers_come_from_the_entry_not_a_counter() {
let labels = parse_props(
"#audio_0.class=AudioButton\n\
#audio_0.streamNumber=1\n\
audio_1.class=AudioButton\n\
audio_1.streamNumber=0\n\
audio_2.class=AudioButton\n\
audio_2.streamNumber=6\n\
other_1.class=SomeOtherButton\n\
other_1.streamNumber=2\n\
subtitle_1.class=SubtitleButton\n\
subtitle_1.streamNumber=11\n",
);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![(StreamLabelType::Audio, 6), (StreamLabelType::Subtitle, 11),],
"skipped entries must not renumber the ones that survive"
);
}
/// Spec: `eda` variant → `Descriptive` purpose.
/// Mutation: miss the `eda` branch → purpose stays Normal.
#[test]
@@ -608,6 +710,60 @@ mod tests {
assert_eq!(labels[0].language, "eng");
}
/// Spec: the skip test is `is_empty() || starts_with('#')` — EITHER
/// condition alone must skip the line. A commented-out line that
/// happens to look like valid CSV (a real authoring pattern for
/// disabling a stream entry) must never produce a label.
/// Mutation: `||` -> `&&` requires BOTH conditions, which a non-empty
/// comment line can never satisfy, so it would fall through to the
/// CSV parser and (since it has >= 4 comma fields) emit a spurious
/// label instead of being skipped.
#[test]
fn ls_comment_line_with_csv_shape_is_still_skipped() {
let labels =
parse_language_streams_text("#id,audio_production,1,eng\nid2,audio_production,2,fra\n");
assert_eq!(
labels.len(),
1,
"the commented-out CSV-shaped line must not parse"
);
assert_eq!(labels[0].language, "fra");
}
/// Spec: `subtitle_dual` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → falls to the
/// catch-all `_ => continue`, silently dropping the stream.
#[test]
fn ls_subtitle_dual_parsed() {
let labels = parse_language_streams_text("id,subtitle_dual,1,eng\n");
assert_eq!(labels.len(), 1, "subtitle_dual must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: `subtitle_bonus` is a recognized subtitle type (Normal/no
/// qualifier). Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_bonus_parsed() {
let labels = parse_language_streams_text("id,subtitle_bonus,2,eng\n");
assert_eq!(labels.len(), 1, "subtitle_bonus must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
}
/// Spec: `subtitle_ime` maps to Subtitle/Ime (no Forced qualifier,
/// unlike `subtitle_ime_narrative`).
/// Mutation: delete this match arm → dropped as unknown.
#[test]
fn ls_subtitle_ime_parsed() {
let labels = parse_language_streams_text("id,subtitle_ime,3,jpn\n");
assert_eq!(labels.len(), 1, "subtitle_ime must produce a label");
assert_eq!(labels[0].stream_type, StreamLabelType::Subtitle);
assert_eq!(labels[0].purpose, LabelPurpose::Ime);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
/// Spec: multiple valid lines produce multiple labels.
/// Mutation: stop after first label → only 1 label returned.
#[test]
@@ -756,6 +912,7 @@ fn parse_menu_base_text(text: &str) -> Vec<StreamLabel> {
.unwrap_or_default();
labels.push(StreamLabel {
stream_id: None,
stream_number: stream_num,
stream_type,
language,
+275 -16
View File
@@ -35,14 +35,16 @@ use crate::sector::SectorSource;
use crate::udf::UdfFs;
use std::collections::BTreeMap;
/// dbp detect can't peek inside a jar without a SectorSource (the
/// trait function only takes `&UdfFs`), so we trigger on the cheap
/// signal "any top-level .jar in /BDMV/JAR/." That fires on every
/// BD-J disc, but parse() does the real `com/dbp/` check and
/// returns None on a mismatch — so this parser only ever consumes
/// time on discs that fell through every earlier parser.
pub fn detect(udf: &UdfFs) -> bool {
jar::has_any_top_level_jar(udf)
/// The real dbp signal is the `com/dbp/` package prefix inside a top-level
/// jar's central directory. With a reader in `detect`, we check that directly
/// (a cheap central-directory scan, no class decode) so this parser claims
/// only dbp discs instead of firing on every BD-J disc. `parse()` repeats the
/// check as belt-and-suspenders.
pub fn detect(reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
jar::for_each_jar(reader, udf, |_entry, archive| {
jar::has_path_prefix(archive, "com/dbp/").then_some(())
})
.is_some()
}
/// Scan every top-level `/BDMV/JAR/*.jar` for the dbp framework and
@@ -91,6 +93,45 @@ fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
out
}
/// Cap on the bytes retained for one stream label.
///
/// The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
/// whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
/// constant contributes up to 65535 bytes, and the `u16` stream-number
/// keyspace admits 65536 of them per type.
///
/// Headroom: real dbp menu labels are short display names — "English Dolby
/// Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
/// retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
/// bytes. 256 leaves >5x headroom over that, and any string past it is menu
/// geometry or padding, never a language name — `vocab::lang` would not
/// resolve it anyway.
const MAX_LABEL_BYTES: usize = 256;
/// Cap on retained stream slots per type.
///
/// The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
/// type are reachable; paired with [`MAX_LABEL_BYTES`] this bounds the whole
/// scan at 2 x 512 x 256 bytes.
///
/// Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
/// streams per playlist, and dbp emits one menu TextField per stream. 512
/// leaves 16x headroom over the spec maximum.
const MAX_LABELS_PER_TYPE: usize = 512;
/// Record `label` for stream `n`, honouring the retention caps. Existing
/// slots are still overwritten at the cap so the documented last-write-wins
/// behaviour is preserved; only NEW slots are refused.
fn retain_label(map: &mut BTreeMap<u16, String>, n: u16, label: &str) {
if label.len() > MAX_LABEL_BYTES {
return;
}
if map.len() >= MAX_LABELS_PER_TYPE && !map.contains_key(&n) {
return;
}
map.insert(n, label.to_string());
}
fn collect_textfield(
s: &str,
audios: &mut BTreeMap<u16, String>,
@@ -110,15 +151,15 @@ fn collect_textfield(
}
if let Some(rest) = kind_n.strip_prefix("Audio") {
if let Ok(n) = rest.parse::<u16>() {
audios.insert(n, label.to_string());
retain_label(audios, n, label);
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle") {
if let Ok(n) = rest.parse::<u16>() {
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
subs.insert(n, label.to_string());
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle")
&& let Ok(n) = rest.parse::<u16>()
{
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
retain_label(subs, n, label);
}
}
}
@@ -130,6 +171,7 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
let qualifier = vocab::qualifier(&label);
let purpose = vocab::purpose(&label);
StreamLabel {
stream_id: None,
stream_number: num,
stream_type,
language,
@@ -145,6 +187,223 @@ fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLa
mod tests {
use super::super::{LabelPurpose, LabelQualifier};
use super::*;
use std::io::{Cursor, Write as _};
/// Build a minimal, structurally valid `.class` file (JVMS §4.1) whose
/// constant pool holds exactly the given `Utf8` strings (indices 1..=N,
/// no long/double slot padding needed for plain strings). No fields,
/// methods, interfaces, or attributes — `scan_jar`'s only interest is
/// the constant pool.
fn build_class(utf8_entries: &[&str]) -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(&0xCAFEBABEu32.to_be_bytes()); // magic
out.extend_from_slice(&0u16.to_be_bytes()); // minor_version
out.extend_from_slice(&52u16.to_be_bytes()); // major_version (Java 8)
out.extend_from_slice(&((utf8_entries.len() + 1) as u16).to_be_bytes()); // cp_count
for s in utf8_entries {
out.push(1); // CONSTANT_Utf8 tag
out.extend_from_slice(&(s.len() as u16).to_be_bytes());
out.extend_from_slice(s.as_bytes());
}
out.extend_from_slice(&0u16.to_be_bytes()); // access_flags
out.extend_from_slice(&0u16.to_be_bytes()); // this_class
out.extend_from_slice(&0u16.to_be_bytes()); // super_class
out.extend_from_slice(&0u16.to_be_bytes()); // interfaces_count
out.extend_from_slice(&0u16.to_be_bytes()); // fields_count
out.extend_from_slice(&0u16.to_be_bytes()); // methods_count
out.extend_from_slice(&0u16.to_be_bytes()); // attributes_count
out
}
/// Zip `entries` (name -> bytes) into an in-memory, Stored (uncompressed)
/// `jar::Jar` via the `zip` crate's own writer — a real archive, not a
/// hand-rolled central directory.
fn build_jar(entries: &[(&str, Vec<u8>)]) -> jar::Jar {
let mut buf = Vec::new();
{
let mut writer = zip::ZipWriter::new(Cursor::new(&mut buf));
let opts = zip::write::SimpleFileOptions::default()
.compression_method(zip::CompressionMethod::Stored);
for (name, data) in entries {
writer.start_file(*name, opts).expect("start_file");
writer.write_all(&data[..]).expect("write class bytes");
}
writer.finish().expect("finish zip");
}
zip::ZipArchive::new(Cursor::new(buf)).expect("valid zip")
}
/// `scan_jar` wires together `for_each_class`, constant-pool iteration,
/// `collect_textfield`, and `make_label` into the actual per-jar scan
/// used by `parse`. The pure `collect_textfield`/`make_label` unit
/// tests above don't exercise this wiring at all.
///
/// Mutation: replace the whole function body with `vec![]` — every
/// dbp disc would silently lose all its stream labels regardless of
/// what's in the jar.
#[test]
fn scan_jar_extracts_labels_from_real_class_entries() {
let class_bytes = build_class(&[
"com/dbp/Whatever", // unrelated string — must be ignored
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
"HTextField,Subtitle1,English SDH,Fontstrip_Composite,1312,763",
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843", // disable button, skipped
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
assert_eq!(
labels.len(),
2,
"expected one audio + one real subtitle label"
);
let audio = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Audio)
.expect("audio label present");
assert_eq!(audio.stream_number, 1);
assert_eq!(audio.language, "eng");
let sub = labels
.iter()
.find(|l| l.stream_type == StreamLabelType::Subtitle)
.expect("subtitle label present");
assert_eq!(sub.stream_number, 1);
assert_eq!(sub.qualifier, LabelQualifier::Sdh);
}
/// Immunity pin. Every dbp label states its own slot in the `AudioN` /
/// `SubtitleN` token, so the numbering survives gaps and skipped entries
/// intact. Nothing here counts positionally, which is what keeps this
/// parser out of the failure mode where a skipped entry pulls every later
/// label one stream forward.
///
/// Mutation: number by iteration order → `Audio4` becomes 2 and
/// `Subtitle3` becomes 1, silently rebinding both to other streams.
#[test]
fn stream_numbers_come_from_the_token_not_iteration_order() {
let class_bytes = build_class(&[
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763",
// Slots 2 and 3 have no menu TextField authored.
"LTextField,Audio4,French 5.1 Dolby Digital,Fontstrip_Composite,296,803",
// Not a stream: the disable-subtitles button.
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843",
// Unparseable slot token — dropped, and must shift nothing.
"HTextField,SubtitleX,German,Fontstrip_Composite,1312,883",
"HTextField,Subtitle3,English SDH,Fontstrip_Composite,1312,763",
]);
let mut archive = build_jar(&[("com/dbp/Menu.class", class_bytes)]);
let labels = scan_jar(&mut archive);
let nums: Vec<(StreamLabelType, u16)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number))
.collect();
assert_eq!(
nums,
vec![
(StreamLabelType::Audio, 1),
(StreamLabelType::Audio, 4),
(StreamLabelType::Subtitle, 3),
],
"unlabelled and unusable slots leave the authored numbers alone"
);
}
/// A `CONSTANT_Utf8_info` carries a `u16` length (JVMS §4.4.7), so one
/// crafted constant contributes up to 65535 bytes and the `u16` stream
/// keyspace admits 65536 slots per type — ~4 GiB of retained `String` per
/// map from a jar that is orders of magnitude smaller.
///
/// Boundary literals, not the constant: a 256-byte label is kept, 257 and
/// the JVMS maximum 65535 are refused.
#[test]
fn oversized_labels_are_not_retained() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
collect_textfield(
&format!("XTextField,Audio1,{},rest", "A".repeat(256)),
&mut audios,
&mut subs,
);
assert_eq!(
audios.get(&1).map(String::len),
Some(256),
"a 256-byte label must still be retained"
);
collect_textfield(
&format!("XTextField,Audio2,{},rest", "A".repeat(257)),
&mut audios,
&mut subs,
);
assert!(!audios.contains_key(&2), "a 257-byte label must be refused");
collect_textfield(
&format!("XTextField,Subtitle1,{},rest", "B".repeat(65_535)),
&mut audios,
&mut subs,
);
assert!(
!subs.contains_key(&1),
"a JVMS-maximum 65535-byte Utf8 label must be refused"
);
}
/// The stream-slot keyspace is the full `u16` on both maps. Offer 600
/// distinct audio slots; exactly 512 are retained.
#[test]
fn retained_stream_slots_are_capped_per_type() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
assert_eq!(
audios.len(),
512,
"600 audio slots offered, {} retained — the slot count is unbounded",
audios.len()
);
}
/// Reaching the slot cap must not break the documented last-write-wins
/// behaviour for slots already held.
#[test]
fn existing_slot_is_still_overwritten_at_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for n in 1..=600u16 {
collect_textfield(
&format!("XTextField,Audio{n},English,rest"),
&mut audios,
&mut subs,
);
}
collect_textfield("XTextField,Audio1,Spanish,rest", &mut audios, &mut subs);
assert_eq!(audios.get(&1).map(String::as_str), Some("Spanish"));
}
/// Headroom: the longest plausible retail label must survive untouched.
#[test]
fn longest_realistic_label_survives_the_cap() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
let real = "Portuguese (Brazilian) 5.1 Dolby Digital Plus";
assert_eq!(real.len(), 45, "fixture length changed");
collect_textfield(
&format!("XTextField,Audio1,{real},Fontstrip_Composite,296,763"),
&mut audios,
&mut subs,
);
assert_eq!(audios.get(&1).map(String::as_str), Some(real));
}
#[test]
fn collect_extracts_audio_and_subtitle_indices() {
+1633 -36
View File
File diff suppressed because it is too large Load Diff
+21 -15
View File
@@ -26,21 +26,6 @@ const MAX_CLASS_BYTES: u64 = 64 * 1024 * 1024;
/// etc.
pub type Jar = ZipArchive<Cursor<Vec<u8>>>;
/// True if `/BDMV/JAR/` contains at least one top-level `.jar` file
/// (not under a subdir). Used by `detect()` in parsers whose real
/// signal lives inside a jar — they can't open the jar without a
/// `SectorSource`, so they use this cheap pre-check and do the real
/// `com/<vendor>/` discriminator in `parse()`.
pub fn has_any_top_level_jar(udf: &UdfFs) -> bool {
let Some(jar_dir) = udf.find_dir("/BDMV/JAR") else {
return false;
};
jar_dir
.entries
.iter()
.any(|e| !e.is_dir && e.name.to_lowercase().ends_with(".jar"))
}
/// Open every top-level `*.jar` entry in `/BDMV/JAR/` and yield each
/// `(entry_name, Jar)` to `f`. Returns the first `Some(R)` the callback
/// produces, or `None` if every jar was visited without a hit.
@@ -231,6 +216,27 @@ mod tests {
ZipArchive::new(Cursor::new(bytes)).expect("valid zip")
}
/// The doc comment states the cap is 64 MiB. Pin the exact numeric
/// value (not derived from the same `64 * 1024 * 1024` expression
/// under test — a hardcoded literal) so a mutation of the arithmetic
/// (e.g. `*` -> `+`) is caught even though no test builds an actual
/// 64 MiB buffer.
#[test]
fn max_class_bytes_is_64_mebibytes() {
assert_eq!(MAX_CLASS_BYTES, 67_108_864);
}
#[test]
fn has_path_prefix_matches_only_declared_prefix() {
let jar = open(build_stored_zip(
"com/dbp/Loader.class",
MINIMAL_CLASS,
MINIMAL_CLASS.len() as u32,
));
assert!(has_path_prefix(&jar, "com/dbp/"));
assert!(!has_path_prefix(&jar, "com/bydeluxe/"));
}
#[test]
fn try_each_class_reads_minimal_class() {
let mut jar = open(build_stored_zip(
+1792 -239
View File
File diff suppressed because it is too large Load Diff
+246 -120
View File
@@ -29,7 +29,7 @@ use crate::udf::UdfFs;
/// True iff `/BDMV/PLAYLIST/` exists and contains at least one
/// `.mpls` file. Cheap directory walk only — no sector reads.
pub fn detect(udf: &UdfFs) -> bool {
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
let Some(dir) = udf.find_dir("/BDMV/PLAYLIST") else {
return false;
};
@@ -61,22 +61,7 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
return None;
}
let mut labels: Vec<StreamLabel> = Vec::new();
// (stream_type_tag, language, codec_hint, pid) — PID is the
// canonical "same physical stream" key; type+lang+codec round
// out the rare case where two distinct logical streams happen
// to share a PID across playlists with different metadata.
let mut seen: Vec<(StreamLabelType, String, String, u16)> = Vec::new();
// Global 1-based counters keyed by StreamLabelType. Incremented
// only when an entry survives dedup, so stream_numbers are dense
// (1, 2, 3, ...) per type across the whole disc — not reset per
// playlist. A disc with 2 MPLS files that each list the same
// 8 audio streams ends up with audio_1..audio_8, not audio_1..
// audio_16 or audio_1..audio_8 with audio_1 duplicated.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
let mut playlists: Vec<crate::mpls::Playlist> = Vec::new();
for name in &mpls_names {
let path = format!("/BDMV/PLAYLIST/{}", name);
let Ok(data) = udf.read_file(reader, &path) else {
@@ -85,28 +70,66 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
let Ok(playlist) = crate::mpls::parse(&data) else {
continue;
};
playlists.push(playlist);
}
let labels = build_labels(&playlists);
if labels.is_empty() {
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
Some(ParseResult::low(labels))
}
/// Convert every stream entry across `playlists` into one [`StreamLabel`] per
/// physical stream. Factored out of [`parse`] so unit tests can drive the
/// actual conversion logic (stream-type mapping, identity, slot numbering)
/// directly from already-parsed [`crate::mpls::Playlist`] values, without
/// needing a synthetic on-disc UDF image.
///
/// Identity is `(clip, PID)` — what the STN entry states — and it is both the
/// dedup key and the label's [`StreamId`]. A stream twenty playlists list is
/// one label; two clips that both open their first audio at 0x1100 are two.
/// This replaced a disc-global dense counter that numbered surviving entries
/// 1, 2, 3, … in playlist-directory order: that number was not an STN slot in
/// anything, but it was handed to a binder that reads `stream_number` as one.
fn build_labels(playlists: &[crate::mpls::Playlist]) -> Vec<StreamLabel> {
use std::collections::HashSet;
let mut labels: Vec<StreamLabel> = Vec::new();
let mut seen: HashSet<super::StreamId> = HashSet::new();
for playlist in playlists {
// `Playlist::streams` is the FIRST play item's STN table, so every
// entry here is a stream of that play item's clip — the same clip
// `disc::bluray` records as the title's `clips[0]`. That pairing is
// what makes the PID an identity rather than a 16-bit number.
//
// Streams cannot be non-empty without a play item to have read them
// from, so the empty case is unreachable on a real disc; entries we
// cannot identify are skipped rather than emitted as unbindable
// labels.
let Some(clip_id) = playlist.play_items.first().map(|pi| pi.clip_id.clone()) else {
continue;
};
// 1-based STN slot within THIS playlist's table, per type — the
// `stream_number` field's documented meaning, counted the same way
// `disc::bluray` counts the stream list it builds from these entries.
// Nothing binds through it (these labels bind by id); it is stated
// truthfully rather than invented so that a reader of the label list
// sees where on its own playlist each stream sits.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for entry in &playlist.streams {
let label_type = match entry.stream_type {
2 | 5 => StreamLabelType::Audio, // primary + secondary audio
3 => StreamLabelType::Subtitle, // PG subtitle
// 1 = primary video, 6 = secondary video, 7 = DV EL
// → no StreamLabelType variant for video, skip.
// 4 = IG (interactive graphics) — not a user-facing
// stream, skip.
_ => continue,
};
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
let key = (label_type, language.clone(), codec_hint.clone(), entry.pid);
if seen.contains(&key) {
let Some(label_type) = label_type_for(entry) else {
continue;
}
seen.push(key);
};
let stream_number = match label_type {
StreamLabelType::Audio => {
audio_idx += 1;
@@ -118,7 +141,20 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
}
};
let stream_id = super::StreamId {
clip_id: clip_id.clone(),
pid: entry.pid,
};
if !seen.insert(stream_id.clone()) {
continue;
}
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
labels.push(StreamLabel {
stream_id: Some(stream_id),
stream_number,
stream_type: label_type,
language,
@@ -130,17 +166,38 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
});
}
}
labels
}
if labels.is_empty() {
/// Which per-type numbering list an STN entry belongs to, or `None` when it
/// is not a labellable stream at all.
///
/// This MUST agree with the stream list `disc::bluray` builds from the same
/// entries, because that list is what `labels::apply_labels` counts against
/// when it binds `stream_number`. The two counters run over the same STN
/// entries in the same order, so any entry one side keeps and the other drops
/// — or files under a different type — shifts every later label of that type
/// onto the wrong stream. Three rules, all mirroring `disc::bluray`:
///
/// * `coding_type == 0` is the STN table's empty/padding slot. Not a
/// stream on either side.
/// * a PG coding_type in an audio STN slot is a subtitle, not audio.
/// `mpls::parse_stream_entry` has a dedicated arm for this layout, so it
/// is an authored shape rather than a corruption.
/// * video (1 / 6 / 7 = primary, secondary, Dolby Vision EL) and IG (4)
/// have no `StreamLabelType`; they are numbered in their own STN lists
/// and never interleave with the audio or PG lists.
fn label_type_for(entry: &crate::mpls::StreamEntry) -> Option<StreamLabelType> {
use crate::consts::coding_type as c;
if entry.coding_type == 0 {
return None;
}
// MPLS gives language + codec but never editorial info (no
// commentary/SDH/director's cut). Low confidence means framework
// parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always
// win when they match. MPLS only gets chosen as the parser when
// nothing else fired — exactly the universal-fallback role we want.
Some(ParseResult::low(labels))
match entry.stream_type {
2 | 5 if entry.coding_type == c::PG => Some(StreamLabelType::Subtitle),
2 | 5 => Some(StreamLabelType::Audio),
3 => Some(StreamLabelType::Subtitle),
_ => None,
}
}
fn has_mpls_extension(name: &str) -> bool {
@@ -327,69 +384,37 @@ mod tests {
}
}
/// A playlist over clip "00001". `Playlist::streams` is read out of the
/// first play item's STN table, so a playlist that has streams always has
/// a play item to have read them from — the fixture carries one so tests
/// exercise the shape production sees, and so each label gets the
/// `(clip, PID)` identity it is bound by.
fn playlist_with(streams: Vec<StreamEntry>) -> Playlist {
playlist_on("00001", streams)
}
fn playlist_on(clip_id: &str, streams: Vec<StreamEntry>) -> Playlist {
Playlist {
version: "0200".to_string(),
play_items: Vec::new(),
play_items: vec![crate::mpls::PlayItem {
clip_id: clip_id.to_string(),
in_time: 0,
out_time: 0,
connection_condition: 1,
}],
streams,
marks: Vec::new(),
}
}
/// Drive the same conversion logic that `parse()` runs on real
/// disc data, but starting from already-parsed Playlists so we
/// don't have to synthesize valid MPLS bytes.
/// Drive the actual production conversion logic (`build_labels`, the
/// function `parse()` calls) starting from already-parsed Playlists,
/// so tests don't have to synthesize valid on-disc MPLS/UDF bytes.
/// This calls the *real* code under test rather than a hand-written
/// re-implementation, so mutations inside `build_labels` (stream-type
/// mapping, dedup key, counters) are actually caught here.
fn labels_from_playlists(playlists: &[Playlist]) -> Vec<StreamLabel> {
let mut labels: Vec<StreamLabel> = Vec::new();
let mut seen: Vec<(StreamLabelType, String, String, u16)> = Vec::new();
// Global counters hoisted OUT of the playlist loop to match
// production `parse()` (lines 77-78): stream_numbers are dense
// per type across the whole disc, not reset per playlist.
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
for playlist in playlists {
for entry in &playlist.streams {
let label_type = match entry.stream_type {
2 | 5 => StreamLabelType::Audio,
3 => StreamLabelType::Subtitle,
_ => continue,
};
// Dedup BEFORE consuming a counter value, matching prod
// parse() ordering so a deduped duplicate does not burn a
// stream number.
let language = normalize_language(&entry.language);
let name = language_display_name(&language);
let codec_hint = build_codec_hint(label_type, entry);
let key = (label_type, language.clone(), codec_hint.clone(), entry.pid);
if seen.contains(&key) {
continue;
}
seen.push(key);
let stream_number = match label_type {
StreamLabelType::Audio => {
audio_idx += 1;
audio_idx
}
StreamLabelType::Subtitle => {
sub_idx += 1;
sub_idx
}
};
labels.push(StreamLabel {
stream_number,
stream_type: label_type,
language,
name,
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::None,
codec_hint,
variant: String::new(),
});
}
}
labels
build_labels(playlists)
}
#[test]
@@ -445,31 +470,114 @@ mod tests {
assert_eq!(labels[2].language, "fra");
}
/// `stream_number` is bound by `labels::apply_labels` against the title's
/// own stream list, which `disc::bluray` builds from these same STN
/// entries. That builder DROPS an entry whose `coding_type` is 0 — the
/// STN table's empty/padding slot — so it must not be counted here
/// either. Counting it advances the audio counter past a stream that
/// never materializes, and every label behind it binds one stream late.
#[test]
fn dedup_streams_across_playlists() {
// Two playlists, same English TrueHD 7.1 PID 0x1100 in both.
// Expect one Audio label, not two.
let pl1 = playlist_with(vec![
fn padding_stn_entry_does_not_consume_a_label_slot() {
let pl = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
audio_entry(0x1101, 0x81, 6, 1, "fra"),
// coding_type 0: STN padding. Not a stream.
audio_entry(0x1101, 0x00, 0, 0, ""),
audio_entry(0x1102, 0x81, 6, 1, "fra"),
]);
let pl2 = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), // duplicate
audio_entry(0x1102, 0x82, 6, 1, "deu"), // new
]);
let labels = labels_from_playlists(&[pl1, pl2]);
// Expected: eng@0x1100, fra@0x1101, deu@0x1102 — three uniques.
assert_eq!(labels.len(), 3);
// PID isn't stored on StreamLabel, so assert on the surviving
// language set instead.
let mut langs: Vec<String> = labels.iter().map(|l| l.language.clone()).collect();
langs.sort();
assert_eq!(langs, vec!["deu", "eng", "fra"]);
let labels = labels_from_playlists(&[pl]);
assert_eq!(labels.len(), 2, "the padding slot yields no label");
assert_eq!(labels[0].language, "eng");
assert_eq!(labels[0].stream_number, 1);
assert_eq!(labels[1].language, "fra");
assert_eq!(
labels[1].stream_number, 2,
"padding is absent from the title's stream list, so `fra` is \
audio stream 2"
);
}
// Stream numbers must be DENSE and GLOBAL across playlists, not
// reset per playlist. eng (pl1) = 1, fra (pl1) = 2, the duplicate
// eng in pl2 is deduped (no number consumed), and deu (pl2) = 3.
// Regression guard for the per-playlist counter-reset divergence.
/// A PG coding_type sitting in an audio STN slot is a real, documented
/// shape — `mpls::parse_stream_entry` has an explicit arm for it, and
/// `disc::bluray` builds it as a Subtitle stream, not an Audio one. This
/// module must classify it the same way, or the audio counter runs one
/// ahead and the subtitle counter one behind for every later stream.
#[test]
fn pg_coding_type_in_an_audio_slot_counts_as_a_subtitle() {
let mut misplaced = audio_entry(0x1200, 0x90, 0, 0, "spa");
misplaced.stream_type = 2;
let pl = playlist_with(vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
misplaced,
audio_entry(0x1101, 0x81, 6, 1, "fra"),
pg_entry(0x1201, "deu"),
]);
let labels = labels_from_playlists(&[pl]);
let audio: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Audio)
.map(|l| (l.language.as_str(), l.stream_number))
.collect();
assert_eq!(
audio,
vec![("eng", 1), ("fra", 2)],
"the PG entry is not an audio stream and must not number one"
);
let sub: Vec<_> = labels
.iter()
.filter(|l| l.stream_type == StreamLabelType::Subtitle)
.map(|l| (l.language.as_str(), l.stream_number))
.collect();
assert_eq!(
sub,
vec![("spa", 1), ("deu", 2)],
"it is subtitle stream 1, ahead of the PG-slot entry"
);
}
/// Two playlists over the SAME clip that both list PID 0x1100: one
/// physical stream, so one label. Identity is `(clip, PID)`, and each
/// label states the STN slot it holds in its own playlist.
#[test]
fn one_label_per_stream_across_playlists_on_the_same_clip() {
let pl1 = playlist_on(
"00001",
vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"),
audio_entry(0x1101, 0x81, 6, 1, "fra"),
],
);
let pl2 = playlist_on(
"00001",
vec![
audio_entry(0x1100, 0x83, 12, 1, "eng"), // same stream
audio_entry(0x1102, 0x82, 6, 1, "deu"), // new
],
);
let labels = labels_from_playlists(&[pl1, pl2]);
assert_eq!(
labels.len(),
3,
"eng/fra/deu — the duplicate eng is one stream"
);
let id = |lang: &str| {
labels
.iter()
.find(|l| l.language == lang)
.and_then(|l| l.stream_id.clone())
.map(|i| (i.clip_id, i.pid))
};
assert_eq!(id("eng"), Some(("00001".into(), 0x1100)));
assert_eq!(id("fra"), Some(("00001".into(), 0x1101)));
assert_eq!(id("deu"), Some(("00001".into(), 0x1102)));
// `stream_number` is the entry's slot in ITS OWN playlist's STN table
// — deu is pl2's second audio, so 2, not "the third distinct stream
// seen while scanning the disc". It used to be the latter: a dense
// disc-global counter that named no table anyone could count against,
// handed to a binder that reads the field as an STN slot.
let num = |lang: &str| {
labels
.iter()
@@ -478,7 +586,25 @@ mod tests {
};
assert_eq!(num("eng"), Some(1));
assert_eq!(num("fra"), Some(2));
assert_eq!(num("deu"), Some(3));
assert_eq!(num("deu"), Some(2), "pl2's second audio slot");
}
/// The same PID in two DIFFERENT clips is two different streams — a PID is
/// only unique within one clip. Deduping on the PID alone (as the old
/// key's `(type, language, codec_hint, pid)` did across clips) collapses
/// them into one label, and the second clip's stream is then described by
/// the first clip's.
#[test]
fn same_pid_in_two_clips_is_two_streams() {
let pl1 = playlist_on("00001", vec![audio_entry(0x1100, 0x83, 12, 1, "eng")]);
let pl2 = playlist_on("00002", vec![audio_entry(0x1100, 0x83, 12, 1, "eng")]);
let labels = labels_from_playlists(&[pl1, pl2]);
assert_eq!(labels.len(), 2, "different clips: two distinct streams");
let clips: Vec<String> = labels
.iter()
.filter_map(|l| l.stream_id.as_ref().map(|i| i.clip_id.clone()))
.collect();
assert_eq!(clips, vec!["00001", "00002"]);
}
#[test]
+425 -55
View File
@@ -3,20 +3,30 @@
//! Richest structured format. Complete language lists with forced flags
//! and commentary indices per playlist, all in XML attributes.
//!
//! NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this
//! file is one authoring house's internal metadata that happens to ship on
//! the pressing. There is nothing to look up: every field meaning here was
//! derived by measuring real discs and cross-checking against per-display-set
//! content. Treat an unfamiliar value as unknown rather than guessing — the
//! disc's own `forced_on_flag` is the only authoritative forced signal.
//!
//! ```xml
//! <playlist name="Feature" id="00222"
//! aud="eng,deu,spa,spa,fra"
//! sub="eng,eng,zho,ces,dan"
//! forced_sub="0,0,0,1,0"
//! forced_sub="0,0,0,1,3"
//! aud_com1_idx="10"
//! sub_com1_idx="23,24,25" />
//! ```
//!
//! `forced_sub` is an ENUMERATION, not a boolean — see [`ForcedSub`].
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorSource;
use crate::udf::UdfFs;
use std::collections::HashSet;
pub fn detect(udf: &UdfFs) -> bool {
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "playlists.xml")
}
@@ -32,11 +42,70 @@ pub fn parse(reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult>
if labels.is_empty() {
return None;
}
// High confidence: paramount's playlists.xml is fully structured
// and we extract every documented field.
// High confidence: this format is fully structured and we extract
// every field whose meaning the corpus establishes. "Documented" would
// be the wrong word — see the module note; nothing about it is.
Some(ParseResult::high(labels))
}
/// One cell of the `forced_sub` CSV.
///
/// The attribute reads like a boolean and was parsed as one (`cell == "1"` →
/// forced). It is not. Every image in the corpus carrying this vendor's
/// `playlists.xml` — seven distinct discs — uses four values, and decoding
/// three of those discs' feature subtitle tracks and counting every PGS
/// display set separates them into two populations two orders of magnitude
/// apart:
///
/// * `0` — a subtitle track with no forced-narrative content. On the two
/// discs measured that use the flag at all, not one `0` track carried a
/// single `forced_on_flag` display set.
/// * `1` — a FULL DIALOGUE track that additionally contains some
/// forced-narrative signs. On one measured disc, all nine `1` cells are
/// full tracks of 949-1411 display sets, eight of them carrying 5-14
/// flagged sets and the ninth none; that disc has no dedicated forced
/// track at all. On another, all seven `1` cells are full tracks of
/// 1602-1651 display sets carrying 0-31 flagged sets. Reading `1` as
/// forced is what made one language present as two identical full
/// subtitle tracks with one of them flagged forced.
/// * `2` and `3` — a DEDICATED forced-narrative track. These take their own
/// trailing STN slots, one per localized language, duplicating a language
/// that already holds a full track earlier in the list. Measured: the two
/// `2` slots on one disc are 15 and 10 display sets, EVERY one flagged
/// forced, against ~1600 on that disc's full tracks; the four `3` slots on
/// another are 7, 14, 23 and 59 display sets against 1216-2655. What
/// distinguishes `2` from `3` the corpus does not reveal — both sit in the
/// same trailing position, both measure the same shape, and one disc uses
/// each for a different language — so both map alike.
///
/// So the old reading was wrong in BOTH directions: it flagged full dialogue
/// tracks forced, and it discarded the cells that name the real forced tracks.
///
/// The `1` case is deliberately NOT carried through as a weaker "contains
/// forced segments" hint. There is no qualifier for that, and the asymmetry
/// argues against inventing one here: a wrong forced flag on a 30 MB dialogue
/// track is the user-visible defect, while a missing hint costs nothing.
///
/// An unrecognised cell maps to [`ForcedSub::None`] — the conservative
/// direction, since asserting forced is the expensive mistake.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum ForcedSub {
/// No forced-narrative content, or an unrecognised cell.
None,
/// A full dialogue track that also carries forced-narrative segments.
ContainsForcedSegments,
/// A dedicated forced-narrative track.
ForcedNarrative,
}
fn forced_sub_cell(cell: &str) -> ForcedSub {
match cell.trim() {
"1" => ForcedSub::ContainsForcedSegments,
"2" | "3" => ForcedSub::ForcedNarrative,
_ => ForcedSub::None,
}
}
/// Build the stream labels from a single `<playlist .../>` feature
/// element. Split out from `parse` so the per-type numbering and
/// commentary/forced-index logic is unit-testable without a
@@ -49,17 +118,35 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// aud_com1_idx is a trimmed, comma-separated list of CSV positions
// (some authoring tools emit whitespace, and multiple commentary
// tracks are possible) — symmetric with sub_com1_idx below.
let com_indices: Vec<usize> = xml::attr(feature, "aud_com1_idx")
// A HashSet, not a Vec: `com_indices` is parsed straight out of an
// attacker-controlled attribute with no length bound and was scanned
// linearly once per stream, so `aud="..."` and `aud_com1_idx="..."`
// both grown large make this quadratic in the size of one XML file.
// Membership is the only operation performed on it.
let com_indices: HashSet<usize> = xml::attr(feature, "aud_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
// stream_number must match apply_labels' monotonic 1-based
// per-type counter, which increments once per *real* stream — so
// it counts only non-empty slots, not the raw CSV index. The
// commentary index comparison stays on the raw CSV index `i`,
// since aud_com1_idx is positional against the original CSV.
let mut audio_num: u16 = 0;
// The CSV *is* the STN list: one cell per stream, in stream order,
// and `aud_com1_idx` is a 0-based index into those same cells. So
// `stream_number` is the cell's own 1-based position — NOT a counter
// that only advances on cells carrying a language.
//
// A cell with an empty language still occupies its STN slot; it just
// has nothing to label. Renumbering the surviving cells 1..N shifts
// every label behind an empty cell one slot forward, which is how a
// marker authored for one stream ends up written onto the stream in
// front of it (see the subtitle side, where the marker is `forced`).
//
// `u16::try_from` rather than `saturating_add`: past the 1-based u16
// numbering space every cell would collapse onto `u16::MAX`, binding
// several streams to one label. Stop emitting instead. Unreachable on
// real media — the BD STN_table admits at most 32 primary audio
// streams per playlist.
for (i, lang) in aud.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
};
let lang = lang.trim();
if lang.is_empty() {
continue;
@@ -69,9 +156,9 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
} else {
LabelPurpose::Normal
};
audio_num = audio_num.saturating_add(1);
labels.push(StreamLabel {
stream_number: audio_num,
stream_id: None,
stream_number,
stream_type: StreamLabelType::Audio,
language: lang.to_string(),
name: String::new(),
@@ -85,18 +172,25 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
// Parse subtitle streams
if let Some(sub) = xml::attr(feature, "sub") {
let forced: Vec<bool> = xml::attr(feature, "forced_sub")
.map(|s| s.split(',').map(|f| f.trim() == "1").collect())
let forced: Vec<ForcedSub> = xml::attr(feature, "forced_sub")
.map(|s| s.split(',').map(forced_sub_cell).collect())
.unwrap_or_default();
let com_indices: Vec<usize> = xml::attr(feature, "sub_com1_idx")
// HashSet for the same reason as the audio side above: unbounded
// parsed input, membership-only use, linear scan once per stream.
let com_indices: HashSet<usize> = xml::attr(feature, "sub_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
// As with audio: count only non-empty slots for stream_number,
// but keep com/forced lookups on the raw CSV index `i`.
let mut sub_num: u16 = 0;
// As with audio: the cell position IS the STN slot. `forced_sub` and
// `sub_com1_idx` are indexed against those same cells, so an empty
// cell must not renumber the cells behind it — a forced marker
// authored for one PG slot would otherwise be written onto an
// earlier, full-dialogue subtitle track.
for (i, lang) in sub.split(',').enumerate() {
let Ok(stream_number) = u16::try_from(i + 1) else {
break;
};
let lang = lang.trim();
if lang.is_empty() {
continue;
@@ -108,15 +202,17 @@ fn labels_from_feature(feature: &str) -> Vec<StreamLabel> {
LabelPurpose::Normal
};
let qualifier = if forced.get(i).copied().unwrap_or(false) {
LabelQualifier::Forced
} else {
LabelQualifier::None
// Only a DEDICATED forced-narrative slot earns the forced flag.
// A cell marking a full track as merely containing forced segments
// is dropped, not weakened into a forced label (see [`ForcedSub`]).
let qualifier = match forced.get(i).copied().unwrap_or(ForcedSub::None) {
ForcedSub::ForcedNarrative => LabelQualifier::Forced,
ForcedSub::ContainsForcedSegments | ForcedSub::None => LabelQualifier::None,
};
sub_num = sub_num.saturating_add(1);
labels.push(StreamLabel {
stream_number: sub_num,
stream_id: None,
stream_number,
stream_type: StreamLabelType::Subtitle,
language: lang.to_string(),
name: String::new(),
@@ -142,10 +238,10 @@ fn find_feature_playlist(text: &str) -> Option<String> {
let element = &text[start..end];
// Prefer name="Feature" explicitly.
if let Some(name) = xml::attr(element, "name") {
if name.eq_ignore_ascii_case("Feature") {
return Some(element.to_string());
}
if let Some(name) = xml::attr(element, "name")
&& name.eq_ignore_ascii_case("Feature")
{
return Some(element.to_string());
}
// Otherwise pick the one with the most audio streams. Count only
@@ -168,6 +264,134 @@ fn find_feature_playlist(text: &str) -> Option<String> {
mod tests {
use super::*;
/// Immunity pin, section-boundary half. The pixelogic parser walks a flat
/// string sequence and recognises its feature section's END by marker
/// alone, so a section with no marker behind it runs off into whatever
/// follows and counts it as more STN slots. Nothing here can do that: the
/// stream list is one attribute of one XML element, so its length is the
/// CSV's own cell count and its scope is the element's byte range that
/// `xml::find_element` returns. Text after the element — including the
/// next playlist's own `aud` — is not reachable from it.
///
/// And when the boundary is MISSING the failure is closed, not open:
/// `xml::find_element` needs a matching close tag and yields `None`
/// without one, so an unterminated element ends the walk rather than
/// swallowing the rest of the document.
///
/// Mutation: hand `labels_from_feature` the document instead of the
/// element, or let an unterminated element run to EOF → the bonus
/// playlist's languages join the feature's stream list.
#[test]
fn a_playlists_stream_list_cannot_run_into_the_next_playlist() {
let doc = r#"
<playlist name="Feature" aud="eng,fra" sub="eng,spa" forced_sub="0,1"/>
<playlist name="Bonus" aud="deu,ita,jpn" sub="deu,ita,jpn"/>
"#;
let feature = find_feature_playlist(doc).expect("feature playlist found");
let labels = labels_from_feature(&feature);
let got: Vec<(StreamLabelType, u16, &str)> = labels
.iter()
.map(|l| (l.stream_type, l.stream_number, l.language.as_str()))
.collect();
assert_eq!(
got,
vec![
(StreamLabelType::Audio, 1, "eng"),
(StreamLabelType::Audio, 2, "fra"),
(StreamLabelType::Subtitle, 1, "eng"),
(StreamLabelType::Subtitle, 2, "spa"),
],
"the CSV's own cells are the whole stream list"
);
// Same document with the feature element left unterminated.
let unterminated = r#"
<playlist name="Feature" aud="eng,fra">
<playlist name="Bonus" aud="deu,ita,jpn"/>
"#;
assert!(
find_feature_playlist(unterminated).is_none(),
"a missing element boundary truncates the walk, never extends it"
);
}
/// `sub_com1_idx` is an unbounded index list parsed straight out of the
/// disc's `playlists.xml` and was membership-tested with a linear
/// `Vec::contains` once per subtitle stream — quadratic in the size of a
/// single attacker-supplied file.
///
/// Proof is by deadline rather than micro-benchmark. With the linear scan
/// the original fixture (200 000 streams x 1 000 001 indices) measured
/// 31 s in a release build and far longer in debug; with a set it measured
/// 0.03 s release / 0.56 s debug. A 10 s deadline sits ~18x above the
/// slowest passing measurement and ~3x below the fastest failing one, and
/// makes a regression fail fast instead of hanging CI.
///
/// The CSV now stops at the end of the 1-based `u16` stream-numbering
/// space, so only the first 65 535 cells are scanned. `INDICES` is raised
/// to keep the linear-scan work product (`cells x indices`) at or above
/// the original fixture's, preserving that deadline margin.
///
/// Correctness is pinned on fixture-derived literals: indices 0, 2 and 4
/// are the commentary tracks, 1 and 3 are not.
#[test]
fn commentary_index_lookup_is_not_quadratic() {
/// Cells offered. Everything past `u16::MAX` is unnumberable and the
/// parser stops there, so the scanned prefix is 65 535 cells.
const STREAMS: usize = 200_000;
const SCANNED: usize = u16::MAX as usize;
const INDICES: usize = 3_100_000;
let (tx, rx) = std::sync::mpsc::channel();
let worker = std::thread::spawn(move || {
let mut feature = String::from(r#"<playlist name="Feature" sub=""#);
feature.push_str(&"eng,".repeat(STREAMS));
feature.pop();
// Three real commentary indices, then a long run of one
// out-of-range value: nothing here is bounded by the stream count.
feature.push_str(r#"" sub_com1_idx="0,2,4,"#);
feature.push_str(&"9999999,".repeat(INDICES));
feature.pop();
feature.push_str(r#"" />"#);
let _ = tx.send(labels_from_feature(&feature));
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(labels) => {
worker.join().expect("worker panicked");
assert_eq!(labels.len(), SCANNED);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[1].purpose, LabelPurpose::Normal);
assert_eq!(labels[2].purpose, LabelPurpose::Commentary);
assert_eq!(labels[3].purpose, LabelPurpose::Normal);
assert_eq!(labels[4].purpose, LabelPurpose::Commentary);
}
Err(_) => panic!(
"labels_from_feature did not finish {STREAMS} streams x \
{INDICES} commentary indices within 10s — the membership \
test is still a linear scan"
),
}
}
/// Headroom: the BD STN_table admits at most 32 PG streams per playlist,
/// and a real `sub_com1_idx` lists a handful of commentary tracks. The set
/// must behave identically to the old scan on real-shaped input.
#[test]
fn commentary_indices_still_match_on_real_shaped_input() {
let feature = r#"<playlist name="Feature" sub="eng,eng,zho,ces,dan" sub_com1_idx="1,3" />"#;
let labels = labels_from_feature(feature);
let purposes: Vec<LabelPurpose> = labels.iter().map(|l| l.purpose).collect();
assert_eq!(
purposes,
vec![
LabelPurpose::Normal,
LabelPurpose::Commentary,
LabelPurpose::Normal,
LabelPurpose::Commentary,
LabelPurpose::Normal,
]
);
}
fn audio(labels: &[StreamLabel]) -> Vec<&StreamLabel> {
labels
.iter()
@@ -182,11 +406,56 @@ mod tests {
.collect()
}
/// The `aud` / `sub` CSVs are the vendor's STN-ordered stream lists: one
/// slot per stream, and `aud_com1_idx` / `forced_sub` are indexed against
/// those same slot positions. A slot whose language cell is empty carries
/// nothing to label but still OCCUPIES its slot, so it must not renumber
/// the slots behind it.
///
/// Numbering only the slots that carry a language collapsed every later
/// label one position forward per empty cell, which is how a forced
/// marker authored for one STN slot lands on the full-subtitle track in
/// front of it.
#[test]
fn empty_middle_slot_does_not_inflate_stream_number() {
// aud="eng,,fra": the empty middle slot is skipped, and the
// second real stream (fra) must be numbered 2, matching
// apply_labels' monotonic counter — not 3 (its raw CSV index).
fn empty_csv_slot_still_occupies_its_stn_slot() {
// Audio: slot 2 is empty; `fra` is STN slot 3 and is the commentary
// the vendor pointed at with the 0-based CSV index 2.
let feature = r#"<playlist name="Feature" aud="eng,,fra" aud_com1_idx="2" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
assert_eq!(a.len(), 2, "the empty slot carries no label");
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(
a[1].stream_number, 3,
"an empty CSV cell occupies STN slot 2, so `fra` is slot 3"
);
assert_eq!(a[1].purpose, LabelPurpose::Commentary);
// Subtitles: same shape, and the consequence is a misplaced forced
// flag. `forced_sub` index 2 is the forced-narrative track; with the
// empty slot renumbered away it would be written onto STN slot 2.
let feature = r#"<playlist name="Feature" sub="eng,,fra" forced_sub="0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 2);
assert_eq!(s[0].language, "eng");
assert_eq!(s[0].stream_number, 1);
assert_eq!(s[0].qualifier, LabelQualifier::None);
assert_eq!(s[1].language, "fra");
assert_eq!(
s[1].stream_number, 3,
"the forced marker belongs to STN slot 3, not slot 2"
);
assert_eq!(s[1].qualifier, LabelQualifier::Forced);
}
#[test]
fn empty_middle_slot_carries_no_label_but_keeps_its_slot() {
// aud="eng,,fra": the empty middle cell yields no label — there is
// nothing to label — but it still owns STN slot 2, so `fra` is slot
// 3. (This test previously asserted 2, pinning the renumbering bug.)
let feature = r#"<playlist name="Feature" aud="eng,,fra" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -194,7 +463,7 @@ mod tests {
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 2);
assert_eq!(a[1].stream_number, 3);
}
#[test]
@@ -202,7 +471,7 @@ mod tests {
// Whitespace around the index, and a multi-value list, must both
// resolve. com index is positional against the raw CSV, so with
// an empty slot at position 1, " 2 " marks the 'fra' track
// (CSV index 2) as commentary.
// (CSV index 2, STN slot 3) as commentary.
let feature = r#"<playlist aud="eng,,fra" aud_com1_idx=" 2 " />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -214,10 +483,10 @@ mod tests {
#[test]
fn forced_sub_aligns_with_raw_csv_index() {
// sub="eng,eng,zho,ces" forced_sub="0,0,0,1": the forced flag is
// sub="eng,eng,zho,ces" forced_sub="0,0,0,3": the forced marker is
// positional on the raw CSV, so 'ces' (index 3) is forced; its
// stream_number is its non-empty position (4 here, no gaps).
let feature = r#"<playlist sub="eng,eng,zho,ces" forced_sub="0,0,0,1" />"#;
// stream_number is its 1-based cell position, 4.
let feature = r#"<playlist sub="eng,eng,zho,ces" forced_sub="0,0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 4);
@@ -262,10 +531,12 @@ mod tests {
assert!(feature.contains(r#"name="MainMovie""#));
}
/// Spec: stream_number for audio is 1-based and increments only on non-empty slots.
/// Mutation: increment for empty slots too → stream numbers inflate.
/// Spec: stream_number for audio is the cell's own 1-based CSV position,
/// because the CSV is the STN list and empty cells are slots too.
/// Mutation: count only non-empty cells → every label behind an empty
/// cell shifts one slot forward.
#[test]
fn audio_stream_numbering_skips_empty_slots() {
fn audio_stream_numbering_uses_raw_csv_slot_position() {
let feature = r#"<playlist name="Feature" aud="eng,,fra,,spa" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
@@ -273,9 +544,9 @@ mod tests {
assert_eq!(a[0].language, "eng");
assert_eq!(a[0].stream_number, 1);
assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 2);
assert_eq!(a[1].stream_number, 3);
assert_eq!(a[2].language, "spa");
assert_eq!(a[2].stream_number, 3);
assert_eq!(a[2].stream_number, 5);
}
/// Spec: forced subtitle at the last position with gaps in between.
@@ -283,9 +554,9 @@ mod tests {
/// Mutation: use stream_number (dense) instead of raw index → wrong subtitle forced.
#[test]
fn forced_sub_uses_raw_csv_index_with_gaps() {
// sub="eng,,fra,,spa" forced_sub="0,0,0,0,1"
// raw CSV index 4 = "spa"; stream_number for spa = 3 (3rd non-empty).
let feature = r#"<playlist name="Feature" sub="eng,,fra,,spa" forced_sub="0,0,0,0,1" />"#;
// sub="eng,,fra,,spa" forced_sub="0,0,0,0,3"
// raw CSV index 4 = "spa", i.e. STN slot 5.
let feature = r#"<playlist name="Feature" sub="eng,,fra,,spa" forced_sub="0,0,0,0,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 3);
@@ -295,6 +566,7 @@ mod tests {
assert_eq!(s[1].qualifier, LabelQualifier::None);
assert_eq!(s[2].language, "spa");
assert_eq!(s[2].qualifier, LabelQualifier::Forced);
assert_eq!(s[2].stream_number, 5);
}
/// Spec: aud_com1_idx is positional against the raw CSV.
@@ -303,13 +575,14 @@ mod tests {
/// Mutation: use stream_number instead of raw CSV index → wrong stream is commentary.
#[test]
fn audio_commentary_index_raw_csv_position() {
// aud="eng,,fra,spa" aud_com1_idx="2" → CSV index 2 = "fra".
// "fra" is stream_number 2 (second non-empty slot, skipping the empty).
// aud="eng,,fra,spa" aud_com1_idx="2" → CSV index 2 = "fra",
// which is STN slot 3.
let feature = r#"<playlist name="Feature" aud="eng,,fra,spa" aud_com1_idx="2" />"#;
let labels = labels_from_feature(feature);
let a = audio(&labels);
assert_eq!(a.len(), 3);
assert_eq!(a[1].language, "fra");
assert_eq!(a[1].stream_number, 3);
assert_eq!(a[1].purpose, LabelPurpose::Commentary);
assert_eq!(a[0].purpose, LabelPurpose::Normal);
assert_eq!(a[2].purpose, LabelPurpose::Normal);
@@ -352,10 +625,11 @@ mod tests {
assert!(s.is_empty(), "no subtitle labels when sub is absent");
}
/// Spec: audio stream_number uses saturating_add on overflow (per u16 cap).
/// Mutation: use wrapping_add → stream numbers wrap to 0, skipping apply.
/// Spec: audio stream_number is the cell's 1-based position and never
/// wraps; past the u16 space the parser stops emitting.
/// Mutation: cast `i + 1` to u16 → stream numbers wrap to 0, skipping apply.
#[test]
fn audio_stream_number_saturates_not_wraps() {
fn audio_stream_number_never_wraps() {
// 65535 audio tracks is impossible on a real disc but the parser must
// not panic or produce 0. Build a comma-separated list of 65535 "eng"s.
// We only run the number-assignment logic via labels_from_feature.
@@ -375,15 +649,92 @@ mod tests {
assert_eq!(last, 300);
}
/// Spec: forced_sub with whitespace around "1" must still parse as true.
/// Mutation: use `== "1"` instead of `trim() == "1"` → " 1 " fails.
/// Spec: a `forced_sub` cell with surrounding whitespace still classifies.
/// Mutation: drop the `trim()` → " 3 " falls through to the unrecognised
/// arm and the disc's forced-narrative track loses its label.
#[test]
fn forced_sub_whitespace_around_one() {
let feature = r#"<playlist name="Feature" sub="eng,fra" forced_sub="0, 1" />"#;
fn forced_sub_cells_are_trimmed_before_classification() {
let feature = r#"<playlist name="Feature" sub="eng,fra,spa" forced_sub="0, 3 , 1 " />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s[0].qualifier, LabelQualifier::None);
assert_eq!(s[1].qualifier, LabelQualifier::Forced);
assert_eq!(s[2].qualifier, LabelQualifier::None);
}
/// `forced_sub` is an enumeration, and `1` is its "full dialogue track that
/// also carries forced signs" value — NOT "this track is forced".
///
/// Measured on a disc whose feature declares nine `1` cells among 32
/// subtitle slots: all nine are full dialogue tracks of 949-1411 display
/// sets, and the disc has no dedicated forced track at all. Reading `1` as
/// forced is what produced two identical full subtitle tracks for one
/// language with one of them flagged forced.
///
/// Nothing downstream can undo this on the discs that need it most:
/// `mux::codec::pgs::demotable` may only clear a vendor forced label where
/// some track on the disc demonstrably sets `forced_on_flag`, and measured
/// discs using this label format never set it.
///
/// Mutation: `"1" => ForcedNarrative` (the old reading) → red.
#[test]
fn a_contains_forced_segments_cell_is_not_a_forced_track() {
let feature = r#"<playlist name="Feature" sub="eng,ces,deu" forced_sub="0,1,1" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 3);
assert!(
s.iter().all(|l| l.qualifier == LabelQualifier::None),
"a `1` marks a full track containing forced signs, not a forced track"
);
}
/// `2` and `3` are the cells that DO name a dedicated forced-narrative
/// track, and the old boolean reading discarded both.
///
/// Measured: these cells occupy their own trailing STN slots, one per
/// localized language, duplicating a language that already holds a full
/// track earlier in the list. On one measured disc the four `3` slots carry
/// 7, 14, 23 and 59 display sets against 1216-2655 on the full tracks they
/// duplicate — and not one display set anywhere on that disc carries
/// `forced_on_flag`, so neither the scan probe nor the muxer can promote
/// them from content. The vendor cell is the only evidence there is.
///
/// Mutation: drop either arm of the `"2" | "3"` match → red.
#[test]
fn a_dedicated_forced_narrative_cell_is_a_forced_track() {
// The measured shape: full tracks first, their forced companions in
// trailing slots of the same languages.
let feature =
r#"<playlist name="Feature" sub="eng,cat,jpn,cat,jpn" forced_sub="0,0,0,2,3" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 5);
assert_eq!(s[1].qualifier, LabelQualifier::None, "the full cat track");
assert_eq!(s[2].qualifier, LabelQualifier::None, "the full jpn track");
assert_eq!(s[3].qualifier, LabelQualifier::Forced, "cat forced slot");
assert_eq!(s[3].stream_number, 4);
assert_eq!(s[4].qualifier, LabelQualifier::Forced, "jpn forced slot");
assert_eq!(s[4].stream_number, 5);
}
/// An unrecognised cell must fall to NOT forced. Asserting forced is the
/// expensive mistake (a full dialogue track a player then burns on screen),
/// so an unknown value from a future authoring revision must not be able to
/// make that claim.
///
/// Mutation: `_ => ForcedNarrative`, or treating "any non-zero" as forced.
#[test]
fn an_unrecognised_forced_sub_cell_is_not_forced() {
let feature = r#"<playlist name="Feature" sub="eng,fra,spa,ita" forced_sub="4,x,,-1" />"#;
let labels = labels_from_feature(feature);
let s = subs(&labels);
assert_eq!(s.len(), 4);
assert!(s.iter().all(|l| l.qualifier == LabelQualifier::None));
// ...and so must a cell the CSV simply does not reach.
let feature = r#"<playlist name="Feature" sub="eng,fra" forced_sub="0" />"#;
let labels = labels_from_feature(feature);
assert_eq!(subs(&labels)[1].qualifier, LabelQualifier::None);
}
/// Spec: `find_feature_playlist` returns None when XML has no `<playlist>` elements.
@@ -393,4 +744,23 @@ mod tests {
assert!(find_feature_playlist("").is_none());
assert!(find_feature_playlist("<root />").is_none());
}
/// Spec: on a tie in audio-slot count, the FIRST playlist encountered
/// wins (consistent with `select_result`'s first-wins tiebreak
/// elsewhere in the registry) — later playlists only displace the
/// current best on a STRICTLY greater count.
/// Mutation: `count > best_aud_count` -> `count >= best_aud_count`
/// would let a later tied playlist silently displace the first.
#[test]
fn find_feature_first_wins_on_audio_count_tie() {
let xml = r#"
<playlist name="A" aud="eng,fra" />
<playlist name="B" aud="deu,spa" />
"#;
let feature = find_feature_playlist(xml).expect("a feature is found");
assert!(
feature.contains(r#"name="A""#),
"first playlist must win a tie, got: {feature}"
);
}
}
+786 -65
View File
File diff suppressed because it is too large Load Diff
+156
View File
@@ -0,0 +1,156 @@
//! Menu-graphic filename language hints.
//!
//! Some BD-J discs encode per-language menu artwork with the language in the
//! filename, e.g. `Feature_UHD01_Eng_Composite1.png`,
//! `AltFeature_UHD01_FRE_Composite2.png`. The `_UHD01_{LANG}_Composite`
//! marker is authored deliberately, so the set of `{LANG}` tokens is the set
//! of menu languages the disc ships.
//!
//! This is a language-only hint (no per-stream purpose/codec), so it runs at
//! [`Confidence::Low`] — it never displaces a real framework parser, and it
//! sits at the same tier as the MPLS floor. It is here so the pattern is a
//! first-class, testable parser that keeps picking up discs as the corpus
//! grows, rather than lost logic. Detection is precise: it fires only on the
//! `_UHD01_{LANG}_Composite` grammar with a `{LANG}` the vocab recognizes.
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, vocab};
use crate::sector::SectorSource;
use crate::udf::UdfFs;
pub fn detect(_reader: &mut dyn SectorSource, udf: &UdfFs) -> bool {
super::jar_inventory(udf)
.iter()
.any(|f| filename_lang(f).is_some())
}
pub fn parse(_reader: &mut dyn SectorSource, udf: &UdfFs) -> Option<ParseResult> {
let names = super::jar_inventory(udf);
let labels = labels_from_filenames(&names);
if labels.is_empty() {
return None;
}
// Low: language-only, derived from menu-asset filenames. A real framework
// parser (and even the MPLS floor's per-stream data) is preferred; this is
// a hint of which languages the disc menus offer.
Some(ParseResult::low(labels))
}
/// One audio [`StreamLabel`] per distinct menu language found, in first-seen
/// order, numbered 1-based. Split out from `parse` so it is unit-testable
/// without a `UdfFs`.
fn labels_from_filenames(names: &[String]) -> Vec<StreamLabel> {
let mut seen: Vec<&'static str> = Vec::new();
for name in names {
if let Some(code) = filename_lang(name)
&& !seen.contains(&code)
{
seen.push(code);
}
}
seen.into_iter()
.enumerate()
.map(|(i, code)| StreamLabel {
stream_id: None,
stream_number: (i as u16).saturating_add(1),
stream_type: StreamLabelType::Audio,
language: code.to_string(),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
})
.collect()
}
/// Extract the ISO-639-2 language code from a `{title}_UHD01_{LANG}_Composite`
/// menu-graphic filename, or `None` if the name does not match the grammar or
/// carries a `{LANG}` the vocab does not recognize.
///
/// The `_UHD01_` marker plus the `_Composite` suffix keep this from firing on
/// unrelated PNGs (`KeyComposite4.png`, `LoadingComposite1.png` have no
/// `_UHD01_{LANG}_` segment).
fn filename_lang(name: &str) -> Option<&'static str> {
// Case-fold once; the marker/suffix are matched case-insensitively.
let lower = name.to_ascii_lowercase();
let marker = "_uhd01_";
let m = lower.find(marker)?;
let after = m + marker.len();
// The language token runs from `after` up to the next `_`.
let rest = &lower[after..];
let end = rest.find('_')?;
if !rest[end..].starts_with("_composite") {
return None;
}
let token = &name[after..after + end];
vocab::menu_lang(token)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn extracts_confirmed_samples() {
assert_eq!(
filename_lang("Feature_UHD01_Eng_Composite1.png"),
Some("eng")
);
assert_eq!(
filename_lang("Feature_UHD01_Ger_Composite2.png"),
Some("deu")
);
assert_eq!(
filename_lang("AltFeature_UHD01_FRE_Composite2.png"),
Some("fra")
);
}
#[test]
fn ignores_non_language_composites() {
assert_eq!(filename_lang("KeyComposite4.png"), None);
assert_eq!(filename_lang("LoadingComposite1.png"), None);
assert_eq!(
filename_lang("FourKWarningsComposite1_bt2020_HDR.png"),
None
);
assert_eq!(filename_lang("Fast9_UPK75_Composite1.png"), None);
}
#[test]
fn unknown_language_token_is_none() {
// A UHD01 marker but a token the vocab does not recognize must not
// produce a bogus language.
assert_eq!(filename_lang("Movie_UHD01_Zzz_Composite1.png"), None);
}
#[test]
fn dedups_and_numbers_distinct_languages() {
let names = vec![
"Feature_UHD01_Eng_Composite1.png".to_string(),
"Feature_UHD01_Eng_Composite2.png".to_string(),
"Feature_UHD01_Ger_Composite1.png".to_string(),
"LoadingComposite1.png".to_string(),
];
let labels = labels_from_filenames(&names);
assert_eq!(labels.len(), 2);
assert_eq!(labels[0].language, "eng");
assert_eq!(labels[0].stream_number, 1);
assert_eq!(labels[1].language, "deu");
assert_eq!(labels[1].stream_number, 2);
assert!(
labels
.iter()
.all(|l| l.stream_type == StreamLabelType::Audio)
);
}
#[test]
fn no_matching_names_yields_empty() {
let names = vec![
"KeyComposite4.png".to_string(),
"disc.properties".to_string(),
];
assert!(labels_from_filenames(&names).is_empty());
}
}
+1 -1
View File
@@ -169,7 +169,7 @@ mod tests {
/// Mutation: skip the final `if !current.is_empty()` emit → trailing run lost.
#[test]
fn large_buffer_trailing_run_emitted() {
let buf: Vec<u8> = (0..1000u32).map(|i| (0x41u8 + (i % 26) as u8)).collect();
let buf: Vec<u8> = (0..1000u32).map(|i| 0x41u8 + (i % 26) as u8).collect();
let got = extract_ascii_strings(&buf, 1);
// All printable, so one big run at the end.
assert!(!got.is_empty());
+147
View File
@@ -176,6 +176,49 @@ const BARE_LANGS: &[(&str, &str)] = &[
("galician", "glg"),
];
/// Map a short menu-graphic language token (as embedded in authoring
/// filenames like `Feature_UHD01_Eng_Composite1.png`) to an ISO-639-2/T code.
///
/// These filename tokens are compact 2/3-letter abbreviations, NOT the full
/// language names [`lang`] handles, so they get their own certain table.
/// Accepts the ISO-639-2/B spellings some tools emit (`ger`, `fre`, `chi`)
/// and normalizes them to the /T code the rest of the pipeline uses (`deu`,
/// `fra`, `zho`). Case-insensitive. Returns `None` for anything not in the
/// table — never guesses, so an unrecognized token drops rather than
/// mislabels.
pub fn menu_lang(token: &str) -> Option<&'static str> {
let t = token.trim().to_ascii_lowercase();
let code = match t.as_str() {
"eng" | "en" => "eng",
"ger" | "deu" | "de" => "deu",
"fre" | "fra" | "fr" => "fra",
"spa" | "es" => "spa",
"ita" | "it" => "ita",
"por" | "pt" => "por",
"jpn" | "jap" | "ja" => "jpn",
"kor" | "ko" => "kor",
"chi" | "zho" | "zh" => "zho",
"rus" | "ru" => "rus",
"dut" | "nld" | "nl" => "nld",
"pol" | "pl" => "pol",
"cze" | "ces" | "cs" => "ces",
"dan" | "da" => "dan",
"fin" | "fi" => "fin",
"nor" | "no" => "nor",
"swe" | "sv" => "swe",
"hun" | "hu" => "hun",
"gre" | "ell" | "el" => "ell",
"tur" | "tr" => "tur",
"ara" | "ar" => "ara",
"hin" | "hi" => "hin",
"tha" | "th" => "tha",
"ukr" | "uk" => "ukr",
"cat" | "ca" => "cat",
_ => return None,
};
Some(code)
}
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
@@ -642,4 +685,108 @@ mod tests {
fn codec_empty_passes_through() {
assert_eq!(codec(""), "");
}
/// `purpose()`'s multi-word-compound fast path ORs two independent
/// phrase checks ("audio description" / "descriptive service"). Each
/// phrase, when it appears as a *word*-bounded match, is independently
/// caught by the has_word fallback further down — so the OR only
/// matters when a phrase appears as a *substring inside a larger word*
/// (no boundary), which .contains() still catches but has_word() would
/// reject.
///
/// Mutation: replace `||` with `&&` at line 238 → since "audio
/// description" is absent here, the AND fails, the fast path doesn't
/// fire, and the fallback has_word("descriptive") also fails (no word
/// boundary before "descriptive" in "nondescriptive"), so purpose()
/// wrongly returns Normal instead of Descriptive.
#[test]
fn purpose_descriptive_service_substring_without_word_boundary() {
assert_eq!(
purpose("nondescriptive service track"),
LabelPurpose::Descriptive
);
}
/// `menu_lang()` maps every authoring-filename token in its table
/// (ISO-639-2/B and /T spellings, plus ISO-639-1) to the canonical
/// /T code used by the rest of the pipeline. Exhaustive per-arm check:
/// deleting any single match arm makes that arm's tokens return None
/// instead of the documented code.
#[test]
fn menu_lang_covers_every_table_entry() {
let cases: &[(&str, &str)] = &[
("eng", "eng"),
("en", "eng"),
("ger", "deu"),
("deu", "deu"),
("de", "deu"),
("fre", "fra"),
("fra", "fra"),
("fr", "fra"),
("spa", "spa"),
("es", "spa"),
("ita", "ita"),
("it", "ita"),
("por", "por"),
("pt", "por"),
("jpn", "jpn"),
("jap", "jpn"),
("ja", "jpn"),
("kor", "kor"),
("ko", "kor"),
("chi", "zho"),
("zho", "zho"),
("zh", "zho"),
("rus", "rus"),
("ru", "rus"),
("dut", "nld"),
("nld", "nld"),
("nl", "nld"),
("pol", "pol"),
("pl", "pol"),
("cze", "ces"),
("ces", "ces"),
("cs", "ces"),
("dan", "dan"),
("da", "dan"),
("fin", "fin"),
("fi", "fin"),
("nor", "nor"),
("no", "nor"),
("swe", "swe"),
("sv", "swe"),
("hun", "hun"),
("hu", "hun"),
("gre", "ell"),
("ell", "ell"),
("el", "ell"),
("tur", "tur"),
("tr", "tur"),
("ara", "ara"),
("ar", "ara"),
("hin", "hin"),
("hi", "hin"),
("tha", "tha"),
("th", "tha"),
("ukr", "ukr"),
("uk", "ukr"),
("cat", "cat"),
("ca", "cat"),
];
for (token, expected) in cases {
assert_eq!(
menu_lang(token),
Some(*expected),
"menu_lang({:?}) should map to {:?}",
token,
expected
);
}
// Case-insensitive and trimmed.
assert_eq!(menu_lang("ENG"), Some("eng"));
assert_eq!(menu_lang(" Eng "), Some("eng"));
// Unrecognized token -> None, never a guess.
assert_eq!(menu_lang("xyz"), None);
assert_eq!(menu_lang(""), None);
}
}
+132
View File
@@ -634,4 +634,136 @@ mod tests {
let (s, e) = find_element(xml, "name", 0).unwrap();
assert_eq!(&xml[s..e], "<di:name>Title</di:name>");
}
// ── Malformed / truncated input (untrusted on-disc XML) ────────────────
//
// These scrapers run on XML lifted out of BD-J jar entries, which is
// attacker-controllable. Every scan in this module must terminate and
// stay in bounds on truncated or unbalanced input rather than panic.
// XML 1.0 §2.3 defines the Name production these boundary rules model.
/// A quoted attribute value that is never closed must terminate the
/// scan at EOF rather than reading past the end of the buffer.
#[test]
fn attr_unterminated_quoted_value_scan_stops_at_eof() {
// The scanner enters the `y="` value and runs off the end looking
// for the closing quote; `name` is never found.
assert_eq!(attr(r#"<x y="oops"#, "name"), None);
assert_eq!(attr("<x y='oops", "name"), None);
// The truncated attribute itself has no terminated value either.
assert_eq!(attr(r#"<x y="oops"#, "y"), None);
}
/// An attribute name at EOF followed only by whitespace (no `=`) must
/// return None, not read past the buffer while skipping that whitespace.
#[test]
fn attr_name_with_trailing_whitespace_and_no_equals_returns_none() {
assert_eq!(attr("<x name ", "name"), None);
}
/// `name=` followed only by whitespace to EOF has no value to return.
#[test]
fn attr_equals_with_trailing_whitespace_and_no_value_returns_none() {
assert_eq!(attr("<x name= ", "name"), None);
}
/// A quoted attribute value is opaque: a `name="..."` pair that appears
/// *inside* another attribute's value must never be reported, even when
/// it is preceded by whitespace so it would otherwise clear the
/// word-boundary check.
#[test]
fn attr_decoy_name_after_space_inside_quoted_value_is_skipped() {
assert_eq!(attr(r#"<x y=" name='decoy'" />"#, "name"), None);
// The real attribute after the decoy still resolves.
assert_eq!(
attr(r#"<x y=" name='decoy'" name="real" />"#, "name"),
Some("real".into())
);
}
/// XML 1.0 §2.3 NameChar includes `-`, `_` and `.`, so `q-a`, `q_a` and
/// `q.a` are each a single attribute name distinct from `a`. Searching
/// for `a` must not match the tail of any of them.
#[test]
fn attr_name_char_boundary_covers_hyphen_underscore_and_dot() {
assert_eq!(
attr(r#"<x q-a="decoy" a="real" />"#, "a"),
Some("real".into())
);
assert_eq!(
attr(r#"<x q_a="decoy" a="real" />"#, "a"),
Some("real".into())
);
assert_eq!(
attr(r#"<x q.a="decoy" a="real" />"#, "a"),
Some("real".into())
);
}
/// An open tag truncated mid-attribute never terminates, so no element
/// can be returned — and the attribute walk must not read past EOF.
#[test]
fn find_element_unterminated_open_tag_returns_none() {
assert_eq!(find_element("<x attr=", "x", 0), None);
}
/// A `/` as the final byte of the buffer is not a self-closing marker;
/// probing for the `>` that would follow it must stay in bounds.
#[test]
fn find_element_trailing_slash_at_eof_returns_none() {
assert_eq!(find_element("<a /", "a", 0), None);
}
/// `/>` inside a quoted attribute value does not close the element.
#[test]
fn find_element_quoted_self_close_marker_does_not_end_element() {
let xml = r#"<x a="/>"/>"#;
let (s, e) = find_element(xml, "x", 0).unwrap();
assert_eq!(&xml[s..e], r#"<x a="/>"/>"#);
}
/// An attribute value whose quote is never closed leaves the open tag
/// unterminated; the scan must end at EOF and report no element.
#[test]
fn find_element_unterminated_quoted_attr_returns_none() {
assert_eq!(find_element(r#"<x a="oops"#, "x", 0), None);
}
/// A `/` in the middle of an unquoted attribute value is not a
/// self-closing marker — only `/>` is.
#[test]
fn find_element_unquoted_slash_is_not_self_closing() {
let xml = "<a href=x/y>body</a>";
let (s, e) = find_element(xml, "a", 0).unwrap();
assert_eq!(&xml[s..e], "<a href=x/y>body</a>");
}
/// `text` must locate the real end of the open tag: a bare `/` inside
/// an unquoted attribute value must not be treated as `/>`, which would
/// shift the body start and leak tag bytes into the returned text.
#[test]
fn text_unquoted_slash_in_attr_does_not_truncate_body() {
assert_eq!(text("<x a=b/c>hello</x>", "x"), Some("hello".into()));
}
/// A `>` inside a quoted attribute value must not be mistaken for the
/// end of the open tag when `text` computes the body start.
#[test]
fn text_quoted_gt_in_attr_does_not_truncate_body() {
assert_eq!(text(r#"<x a="b>c">hello</x>"#, "x"), Some("hello".into()));
}
/// A close tag truncated mid-name (`</x` with no `>`) is not a close
/// tag; matching it must stay in bounds and report no text.
#[test]
fn text_truncated_close_tag_returns_none() {
assert_eq!(text("<x>body</x", "x"), None);
}
/// A `/` in element content is only a close tag when preceded by `<`.
/// Body text containing `a/x>` must not be mistaken for `</x>`.
#[test]
fn text_slash_in_body_is_not_a_close_tag() {
assert_eq!(text("<x>a/x> </x>", "x"), Some("a/x>".into()));
}
}
+48 -23
View File
@@ -32,7 +32,7 @@
//! let opts = libfreemkv::InputOptions::default();
//! let mut input = libfreemkv::input("iso://disc.iso", &opts)?;
//! let title = input.info().clone();
//! let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
//! let mut output = libfreemkv::output("mkv://Movie.mkv", &title, None)?;
//! // Propagate read errors instead of silently stopping on the first one.
//! while let Some(frame) = input.read()? {
//! output.write(&frame)?;
@@ -98,7 +98,7 @@ pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFI
/// The muxing/writing-application string written into MKV output
/// (`"freemkv <version> (g<hash>)"`).
pub const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub(crate) const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs;
pub(crate) mod clpi;
@@ -112,6 +112,8 @@ pub mod dvdnav;
pub mod error;
pub mod event;
pub mod halt;
#[cfg(test)]
mod harness;
pub mod hex;
pub(crate) mod identity;
pub(crate) mod ifo;
@@ -125,7 +127,7 @@ pub(crate) mod platform;
pub mod progress;
pub mod scsi;
pub mod sector;
pub(crate) mod speed;
pub mod session;
pub(crate) mod udf;
pub(crate) mod unlock_bridge;
@@ -137,38 +139,59 @@ pub(crate) mod unlock_bridge;
pub use drive::capture::{
CapturedFeature, DriveCapture, capture_drive_data, mask_bytes, mask_string,
};
pub use drive::{Drive, DriveStatus, find_drive};
pub use drive::{Drive, DriveStatus, extract_scsi_context, find_drive};
// ─── Disc session (drive open + SCSI bring-up hoist) ─────────────────────────
//
// One entry point that opens a drive and brings the transport up, so consumers
// stop hand-rolling `open → wait_ready → init → probe_disc → identify → scan`.
// Owns the `Drive` by value; forwards consumer-built key material into
// `ScanOptions` (the library derives no certs — see `KeySpec`).
pub use session::{
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_iso,
};
// ─── Errors ─────────────────────────────────────────────────────────────────
//
// All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a
// numeric `code()`; **no English text in the library** — applications map
// codes to localized messages. See `error.rs` for the full taxonomy.
pub use error::{Error, Result};
pub use error::{
Error, Result, error_code, is_disc_level_no_key, is_halt, is_skippable_title_stub,
};
// ─── Cooperative cancellation ───────────────────────────────────────────────
//
// One-bit cooperative cancellation token, shared by every long-running loop
// in libfreemkv (sweep, patch, mux). Clone it cheaply; pass it by value into
// each component; poll `is_cancelled()` inside the loop body.
// One-bit cooperative cancellation token, shared by every long-running loop —
// libfreemkv's mux, and the recovery passes (sweep/patch) that now live in the
// freemkv-engine crate. Clone it cheaply; pass it by value into each component;
// poll `is_cancelled()` inside the loop body.
pub use halt::Halt;
// Generic bounded producer/consumer primitive used by sweep, patch, and
// mux to overlap reads with writes via a dedicated consumer thread.
// Generic bounded producer/consumer primitive used by the mux pipeline (and,
// via this re-export, by the engine's sweep/patch recovery passes) to overlap
// reads with writes via a dedicated consumer thread.
// `Pipeline::spawn(name, depth, sink)` spawns a named consumer; `pipe.send(item)`
// pushes one item with back-pressure; `pipe.finish()` joins the
// consumer and surfaces its `close()` output. Callers implement `Sink`
// to define per-item behaviour and end-of-stream finalisation.
//
// `DEFAULT_PIPELINE_DEPTH` (=4) is for callers without specific needs;
// most should use READ_PIPELINE_DEPTH or WRITE_PIPELINE_DEPTH instead.
// most should use WRITE_PIPELINE_DEPTH instead.
// Patch uses `WRITE_THROUGH_DEPTH` (=1). Returning `Flow::Stop` from
// `apply` ends the consumer cleanly (still calls `close()`).
pub use io::pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, READ_PIPELINE_DEPTH, Sink, WRITE_PIPELINE_DEPTH,
WRITE_THROUGH_DEPTH,
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH, WRITE_THROUGH_DEPTH,
};
// ─── Bounded-cache buffered file writer ─────────────────────────────────────
//
// Drop-in `std::fs::File` replacement used everywhere the lib writes large
// sequential output (mux, extract, sweep, patch) — drains dirty pages
// continuously instead of bursting. General I/O infra, not recovery policy;
// promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too.
pub use io::WritebackFile;
// ─── Drive events (low-level callbacks) ─────────────────────────────────────
pub use event::{BatchSizeReason, Event, EventKind};
pub use identity::DriveId;
@@ -187,7 +210,7 @@ pub use identity::DriveId;
// don't touch `DecryptKeys` directly — `DiscStream::new(reader, title, keys, …)`
// accepts whatever `Disc::decrypt_keys()` returned. `decrypt_sectors()` is
// for callers that operate on raw sector buffers (e.g. ISO patching).
pub use decrypt::{DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_threads};
pub use decrypt::{AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_threads};
// ─── Disc structure ─────────────────────────────────────────────────────────
//
@@ -201,11 +224,10 @@ pub use decrypt::{DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_thr
// different concepts, the same short name; the trait gets the `Pes`
// prefix at the crate root to keep both addressable.
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, DamageSeverity,
Disc, DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult,
FileResult, FrameRate, HdrFormat, Key, KeyOrigin, LabelPurpose, LabelQualifier, PatchOptions,
PatchOutcome, Resolution, SampleRate, ScanOptions, Stream, SubtitleStream, SweepOptions,
VideoStream, classify_damage,
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult,
FileResult, FrameRate, HdrFormat, Key, KeyOrigin, LabelPurpose, LabelQualifier, Resolution,
SampleRate, ScanOptions, Stream, SubtitleStream, VideoStream,
};
pub use keysource::{DiscInputs, KeySource, read_encrypted_units, resolve_and_apply};
@@ -238,6 +260,8 @@ pub use mux::NullStream;
pub use mux::StdioStream;
pub use mux::WriteSeek;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
pub use mux::{Medium, SourceInfo};
pub use mux::{Mp4FitReport, Mp4Sink, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
@@ -248,10 +272,11 @@ pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
// single decrypt-on-read decorator (AACS / CSS / none) — wrap any
// `SectorSource` to get plaintext sectors out.
pub use mux::build_iso_pipeline;
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, drive_has_disc, list_drives};
pub use mux::resolve_mux_key_map;
pub use mux::select::{PidFilter, StreamSelection};
pub use mux::{MuxEvents, MuxInput, MuxOptions, MuxOutcome, mux_stream};
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, SenseFamily, drive_has_disc, list_drives};
pub use sector::{
DecryptingSectorSource, FileSectorSink, FileSectorSource, KeyFetch, PrefetchedSectorSource,
SectorSink, SectorSource,
DecryptingSectorSource, FileSectorSource, KeyFetch, PrefetchedSectorSource, SectorSource,
};
pub use speed::DriveSpeed;
pub use udf::{UdfFs, read_filesystem};
+316 -3
View File
@@ -39,6 +39,18 @@ pub(crate) struct PlaylistMark {
pub timestamp: u32,
}
impl PlaylistMark {
/// Is this mark a chapter entry point?
///
/// Only `mark_type == 1` counts. Type 0 is reserved and type 2 is a link
/// point, and neither is a chapter. Every chapter filter in the crate goes
/// through here: two hand-rolled copies had already drifted, one testing
/// `<= 1` and silently counting reserved marks as chapters.
pub(crate) fn is_chapter_mark(&self) -> bool {
self.mark_type == 1
}
}
/// A play item — one clip reference with in/out times.
#[derive(Debug)]
pub(crate) struct PlayItem {
@@ -432,14 +444,13 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
}
}
}
STREAM_CATEGORY_PG_SUBTITLE => {
STREAM_CATEGORY_PG_SUBTITLE
// PG: coding_type(1) + language(3).
// IG is parsed only to advance spos and is then discarded by the
// caller, so it deliberately has no arm here.
if sa.len() >= 4 {
if sa.len() >= 4 => {
language = String::from_utf8_lossy(&sa[1..4]).to_string();
}
}
_ => {}
}
@@ -1417,4 +1428,306 @@ mod tests {
data[8..12].copy_from_slice(&40u32.to_be_bytes()); // playlist_start = 40 = len
assert!(parse(&data).is_err());
}
// ─────────────────────────────────────────────────────────────────────
// Added: STN-table block alignment and section-boundary hardening.
// ─────────────────────────────────────────────────────────────────────
/// Build an MPLS from raw PlayItem bodies, with no PlayListMark section
/// (mark_start = 0). Lets a test control item_length exactly.
fn build_mpls_raw_items(items: &[Vec<u8>]) -> Vec<u8> {
let playlist_start: u32 = 40;
let mut buf = Vec::new();
buf.extend_from_slice(b"MPLS0200");
buf.extend_from_slice(&playlist_start.to_be_bytes());
buf.extend_from_slice(&[0u8; 28]); // mark_start = 0, then padding
let pl_start = buf.len();
buf.extend_from_slice(&[0u8; 4]); // PlayList length placeholder
buf.extend_from_slice(&[0u8; 2]); // reserved
buf.extend_from_slice(&(items.len() as u16).to_be_bytes());
buf.extend_from_slice(&[0u8; 2]); // num_sub_paths
for it in items {
buf.extend_from_slice(&(it.len() as u16).to_be_bytes());
buf.extend_from_slice(it);
}
let pl_len = (buf.len() - pl_start - 4) as u32;
buf[pl_start..pl_start + 4].copy_from_slice(&pl_len.to_be_bytes());
buf
}
/// The 20 bytes a PlayItem needs for clip_id(5) + codec_id(4) +
/// connection_condition(1) + reserved(2) + IN_time(4) + OUT_time(4).
fn play_item_20(clip: &[u8; 5], cc: u8, in_t: u32, out_t: u32) -> Vec<u8> {
let mut it = Vec::new();
it.extend_from_slice(clip);
it.extend_from_slice(b"M2TS");
it.push(cc);
it.extend_from_slice(&[0u8; 2]);
it.extend_from_slice(&in_t.to_be_bytes());
it.extend_from_slice(&out_t.to_be_bytes());
assert_eq!(it.len(), 20);
it
}
/// A PlayItem body of exactly 20 bytes carries every field the parser
/// reads (the last is OUT_time at [16..20]), so it must be RECORDED,
/// not skipped — and it has no STN table, which starts at byte 32.
#[test]
fn play_item_of_exactly_20_bytes_is_recorded_without_stn() {
let data = build_mpls_raw_items(&[play_item_20(b"00007", 5, 90_000, 180_000)]);
let pl = parse(&data).expect("a 20-byte PlayItem must parse");
assert_eq!(pl.play_items.len(), 1);
assert_eq!(pl.play_items[0].clip_id, "00007");
assert_eq!(pl.play_items[0].in_time, 90_000);
assert_eq!(pl.play_items[0].out_time, 180_000);
assert_eq!(pl.play_items[0].connection_condition, 5);
assert!(pl.streams.is_empty(), "no STN table exists below byte 32");
}
/// A 40-byte MPLS whose PlayList section is exactly its 10-byte header
/// (length(4)+reserved(2)+num_play_items(2)+num_sub_paths(2)) ending at
/// EOF is structurally complete, not truncated: nothing the parser reads
/// lies past the buffer, so it must parse to an empty playlist.
#[test]
fn minimum_size_mpls_with_empty_playlist_header_parses() {
let mut data = vec![0u8; 40];
data[0..4].copy_from_slice(b"MPLS");
data[4..8].copy_from_slice(b"0200");
data[8..12].copy_from_slice(&30u32.to_be_bytes()); // playlist_start + 10 == 40
// mark_start (12..16) stays 0; num_play_items at data[36..38] is 0.
let pl = parse(&data).expect("40-byte MPLS with a complete PlayList header must parse");
assert!(pl.play_items.is_empty());
assert!(pl.streams.is_empty());
assert!(pl.marks.is_empty());
}
/// A mark_start of 0 means "no PlayListMark section". The file header
/// bytes at offset 0 must not be decoded as one — data[4..6] is the
/// version string "02", which as a big-endian num_marks would be 12338.
#[test]
fn mark_start_zero_does_not_parse_header_as_marks() {
let data = build_mpls_raw_items(&[play_item_20(b"00007", 1, 0, 90_000)]);
assert_eq!(
&data[12..16],
&[0, 0, 0, 0],
"fixture must have mark_start 0"
);
let pl = parse(&data).expect("should parse");
assert!(
pl.marks.is_empty(),
"mark_start == 0 must mean absent, got {} marks",
pl.marks.len()
);
}
/// Full STN table walk with every category populated and DISTINCT
/// counts, so no count byte can be read from a neighbour's offset
/// without changing the result.
///
/// Each secondary block is followed by its reference block(s), which
/// per the BD STN table are num_refs(1) + reserved(1) + one byte per
/// ref + one padding byte when the ref count is odd. Every ref count
/// here is 1 — the value that distinguishes `n % 2` (=1) from `n / 2`
/// (=0) — so a wrong skip length misaligns the cursor and every
/// following stream decodes from the wrong offset. IG entries are
/// consumed to keep the cursor aligned but never retained.
#[test]
fn full_stn_table_block_alignment() {
let mut entries: Vec<Vec<u8>> = vec![
build_stream_entry_video(0x1011, 0x1B, 6, 1, None),
build_stream_entry_audio(0x1100, 0x83, 6, 1, b"eng"),
build_stream_entry_audio(0x1101, 0x86, 3, 1, b"fra"),
build_stream_entry_pg(0x1200, 0x90, b"eng"),
build_stream_entry_pg(0x1201, 0x90, b"fra"),
build_stream_entry_pg(0x1202, 0x90, b"deu"),
];
for i in 0..4u16 {
entries.push(build_stream_entry_pg(0x1400 + i, 0x91, b"eng"));
}
// secondary audio + its secondary-audio ref block (1 ref → 1 pad)
let mut sec_audio = build_stream_entry_audio(0x1A00, 0x83, 3, 1, b"spa");
sec_audio.extend_from_slice(&[1, 0, 0x55, 0x00]);
entries.push(sec_audio);
// secondary video + audio-ref block + PiP-PG-ref block
let mut sec_video = build_stream_entry_video(0x1B00, 0x1B, 4, 1, None);
sec_video.extend_from_slice(&[1, 0, 0x55, 0x00]);
sec_video.extend_from_slice(&[1, 0, 0x66, 0x00]);
entries.push(sec_video);
// PiP PG + its ref block
let mut pip_pg = build_stream_entry_pg(0x1C00, 0x90, b"jpn");
pip_pg.extend_from_slice(&[1, 0, 0x77, 0x00]);
entries.push(pip_pg);
// Dolby Vision enhancement layer
entries.push(build_stream_entry_video(0x1015, 0x24, 8, 1, Some(0x12)));
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 2, 3, 4, 1, 1, 1, 1),
&entries,
);
let pl = parse(&data).expect("should parse");
let got: Vec<(u8, u16, bool)> = pl
.streams
.iter()
.map(|s| (s.stream_type, s.pid, s.secondary))
.collect();
assert_eq!(
got,
vec![
(1, 0x1011, false), // primary video
(2, 0x1100, false), // primary audio ×2
(2, 0x1101, false),
(3, 0x1200, false), // PG ×3
(3, 0x1201, false),
(3, 0x1202, false),
// the 4 IG entries are consumed and discarded
(5, 0x1A00, true), // secondary audio
(6, 0x1B00, true), // secondary video
(3, 0x1C00, true), // PiP PG
(7, 0x1015, true), // Dolby Vision EL
]
);
// Languages prove each entry was decoded at its own offset.
assert_eq!(pl.streams[1].language, "eng");
assert_eq!(pl.streams[2].language, "fra");
assert_eq!(pl.streams[6].language, "spa");
assert_eq!(pl.streams[8].language, "jpn");
}
/// A secondary block whose stream entry ends exactly at the end of the
/// PlayItem has no reference block at all; the count byte must not be
/// read from one-past-the-end. Covers all three secondary blocks that
/// carry reference data.
#[test]
fn secondary_ref_block_at_item_end_is_not_read() {
let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None);
// Secondary audio is the last entry, with no ref bytes following.
let sec_audio = build_stream_entry_audio(0x1A00, 0x83, 3, 1, b"eng");
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 1, 0, 0, 0),
&[video.clone(), sec_audio],
);
let pl = parse(&data).expect("secondary audio at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1A00);
// Secondary video is the last entry, with no ref bytes following.
let sec_video = build_stream_entry_video(0x1B00, 0x1B, 4, 1, None);
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 1, 0, 0),
&[video.clone(), sec_video.clone()],
);
let pl = parse(&data).expect("secondary video at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1B00);
// Secondary video whose audio-ref block ends exactly at item end, so
// the PiP-PG ref count byte would sit one past it.
let mut sec_video_arefs = sec_video;
sec_video_arefs.extend_from_slice(&[0, 0]); // n_arefs = 0, reserved
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 1, 0, 0),
&[video.clone(), sec_video_arefs],
);
let pl = parse(&data).expect("secondary video aref block at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1B00);
// PiP PG is the last entry, with no ref bytes following.
let pip_pg = build_stream_entry_pg(0x1C00, 0x90, b"jpn");
let data = build_mpls(
&[(b"00001", 1, 0, 9_000_000)],
(1, 0, 0, 0, 0, 0, 1, 0),
&[video, pip_pg],
);
let pl = parse(&data).expect("PiP PG at item end");
assert_eq!(pl.streams.len(), 2);
assert_eq!(pl.streams[1].pid, 0x1C00);
}
// ─────────────────────────────────────────────────────────────────────
// parse_stream_entry bounds, exercised directly.
// ─────────────────────────────────────────────────────────────────────
/// Fewer than 2 bytes remain for the stream_entry header
/// (length(1) + stream_entry_type(1)) → None, without reading either.
#[test]
fn stream_entry_header_past_end_is_none() {
let item = [0u8; 8];
for pos in 7..12usize {
assert!(
parse_stream_entry(&item, pos, STREAM_CATEGORY_VIDEO).is_none(),
"pos={pos}"
);
}
}
/// The stream_attributes header (length(1) + coding_type(1)) lies past
/// the end of the PlayItem → None, without reading the length byte.
#[test]
fn stream_attributes_header_past_end_is_none() {
// se_len = 3 → se_end = 4 == item.len(); the sa length byte would be
// at item[4] and the coding type at item[5].
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11];
assert!(parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).is_none());
}
/// A declared stream_attributes length of 0 has no coding_type byte and
/// must be rejected — even when the (empty) attribute region is itself
/// in bounds.
#[test]
fn zero_length_attributes_in_bounds_is_none() {
// se_len = 3 → se_end = 4; sa_len = item[4] = 0 → sa_end = 5 ≤ 6.
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11, 0, 0];
assert!(parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).is_none());
}
/// stream_attributes of exactly 1 byte carries only the coding_type.
/// That is the minimum the parser accepts, so the entry is returned
/// with its PID and coding_type and no format-specific fields — for a
/// PG stream the 3-byte language must NOT be read past the attributes.
#[test]
fn one_byte_stream_attributes_yields_bare_entry() {
// se_len = 3 → se_end = 4; sa_len = 1 → sa_end = 6 == item.len().
let item = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x10, 0x11, 1, 0x1B];
let (entry, next) =
parse_stream_entry(&item, 0, STREAM_CATEGORY_VIDEO).expect("1-byte attrs are valid");
assert_eq!(entry.pid, 0x1011);
assert_eq!(entry.coding_type, 0x1B);
assert_eq!(entry.video_format, 0);
assert_eq!(entry.video_rate, 0);
assert_eq!(next, 6);
let pg = [3u8, STREAM_ENTRY_PLAYITEM_CLIP, 0x12, 0x00, 1, 0x90];
let (entry, _) = parse_stream_entry(&pg, 0, STREAM_CATEGORY_PG_SUBTITLE)
.expect("1-byte PG attrs are valid");
assert_eq!(entry.pid, 0x1200);
assert_eq!(entry.coding_type, 0x90);
assert_eq!(entry.language, "");
}
/// Type 0 is reserved and type 2 is a link point; neither is a chapter.
/// `labels::collect_chapter_summary` used to filter on `mark_type <= 1`,
/// which counted reserved marks and inflated the public `chapter_count`
/// (and let a playlist whose only marks are reserved pass the
/// `chapter_count == 0` skip). Both call sites now share this predicate.
#[test]
fn only_entry_marks_count_as_chapters() {
let mk = |mark_type| PlaylistMark {
mark_type,
play_item_ref: 0,
timestamp: 0,
};
assert!(
!mk(0).is_chapter_mark(),
"type 0 is reserved, not a chapter"
);
assert!(mk(1).is_chapter_mark());
assert!(!mk(2).is_chapter_mark(), "type 2 is a link point");
}
}
File diff suppressed because it is too large Load Diff
+921 -118
View File
File diff suppressed because it is too large Load Diff
+275
View File
@@ -0,0 +1,275 @@
//! AAC ADTS decodability gate.
//!
//! Per the ADTS framing defined in ISO/IEC 13818-7 / ISO/IEC 14496-3, a header
//! is structurally invalid in exactly three ways this gate treats as hard
//! rejects: syncword != 0xFFF, a reserved `sampling_frequency_index` (the sample
//! rate table has 13 valid entries, so index ≥ 13 is reserved), and
//! `aac_frame_length < 7` (shorter than the fixed+variable header itself). The
//! optional 16-bit ADTS CRC is not verified here — it is simply skipped. So the
//! gate enforces those three rejects: a packet that begins with the ADTS sync
//! but is otherwise malformed is dropped; a packet with no ADTS sync is raw AAC
//! (e.g. from an MP4 container, which carries no ADTS header) or a continuation
//! and passes through unchanged — never false-dropped. Raw AAC has no per-frame
//! integrity data, so like LPCM it cannot be gated.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// ADTS `sampling_frequency_index` table (ISO/IEC 14496-3) — 13 valid entries;
/// indices 13/14/15 are 0 (reserved) and constitute a hard reject.
const ADTS_SAMPLE_RATE_VALID: [u32; 16] = [
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350, 0, 0,
0,
];
/// ADTS header verdict for the packet head.
enum AdtsVerdict {
/// No 12-bit ADTS sync at the head — not an ADTS frame we can validate.
NoSync,
/// Sync present and the three structural fields are legal.
Valid,
/// Sync present but a reserved sample-rate index or a sub-header
/// frame-length — structurally invalid per the ADTS spec.
Invalid,
}
fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// Need the full 7-byte fixed+variable header to read frame_length.
if data.len() < 7 {
return AdtsVerdict::NoSync;
}
// 12-bit syncword 0xFFF: byte0 == 0xFF and top nibble of byte1 == 0xF.
if data[0] != 0xFF || (data[1] & 0xF0) != 0xF0 {
return AdtsVerdict::NoSync;
}
// sampling_frequency_index: byte2 bits 5..2.
let sr_index = ((data[2] >> 2) & 0x0F) as usize;
if ADTS_SAMPLE_RATE_VALID[sr_index] == 0 {
return AdtsVerdict::Invalid;
}
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
if frame_length < 7 {
return AdtsVerdict::Invalid;
}
AdtsVerdict::Valid
}
pub struct AdtsParser {
tally: DropTally,
/// Last emitted PTS (ns). A PES with no PTS (legal for audio, e.g. a
/// post-discontinuity continuation) carries this forward rather than resetting
/// the timeline to 0 — matching the AC-3/DTS parsers and preserving A/V sync.
last_pts_ns: i64,
}
impl Default for AdtsParser {
fn default() -> Self {
Self::new()
}
}
impl AdtsParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("aac"),
last_pts_ns: 0,
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for AdtsParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes
.pts
.or(pes.dts)
.map(pts_to_ns)
.unwrap_or(self.last_pts_ns);
self.last_pts_ns = pts_ns;
let drop =
self.tally.is_poisoned() || matches!(adts_verdict(&pes.data), AdtsVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with
/// aac_frame_length set to the total size.
fn adts_frame(payload: usize) -> Vec<u8> {
let total = 7 + payload;
let mut f = vec![0u8; total];
f[0] = 0xFF;
f[1] = 0xF1; // sync + MPEG-4 + no CRC (protection_absent=1)
f[2] = 0x50; // profile=AAC-LC, sr_index=4 (44.1 kHz)
f[3] = 0x80; // channel_config low + start of frame_length
// frame_length (13 bits) = total.
let fl = total as u32;
f[3] = (f[3] & 0xFC) | ((fl >> 11) & 0x03) as u8;
f[4] = ((fl >> 3) & 0xFF) as u8;
f[5] = (((fl & 0x07) << 5) as u8) | 0x1F; // low 3 bits of len + buffer-fullness bits
f
}
#[test]
fn valid_adts_is_kept() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(adts_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn pes_without_pts_carries_last_timestamp_not_zero() {
// A PES with no PTS (legal for audio, e.g. after a discontinuity) must
// carry the last known timestamp forward — resetting to 0 would corrupt
// A/V sync.
let mut p = AdtsParser::new();
p.parse(&make_pes(adts_frame(400), Some(90000)));
let f = p.parse(&make_pes(adts_frame(400), None));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(90000),
"carried forward, not reset to 0"
);
}
/// A dropped ADTS frame is dropped BECAUSE its header failed validation, so
/// the very fields a duration would come from (sampling_frequency_index, and
/// the 1024-samples-per-AAC-frame constant applied to it) are the ones known
/// to be untrustworthy. This gate therefore reports the drop's duration as
/// zero rather than deriving a number from a header it has just rejected —
/// the honest answer, and the one the count alongside it must be read with.
/// A nonzero constant here would report silence that was never measured.
#[test]
fn dropped_frames_are_counted_but_their_duration_is_not_invented() {
let mut parser = AdtsParser::new();
// Three frames whose sampling_frequency_index is a reserved value (13),
// so `adts_verdict` rejects each one.
let mut bad = adts_frame(32);
bad[2] = (bad[2] & 0b1100_0011) | (13 << 2);
for i in 0..3 {
let out = parser.parse(&make_pes(bad.clone(), Some(i * 90_000)));
assert!(out.is_empty(), "an invalid ADTS frame is not emitted");
}
assert_eq!(parser.dropped_frames(), 3, "every drop is counted");
assert_eq!(
parser.dropped_duration_ns(),
0,
"the duration comes from the header that just failed validation, so \
it is reported as unmeasured rather than guessed"
);
}
#[test]
fn reserved_sample_rate_index_is_dropped() {
// sr_index = 13 (reserved). byte2 bits5..2 = 1101 → 0x34.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[2] = (f[2] & 0xC3) | (13 << 2); // set sr_index = 13
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn subheader_frame_length_is_dropped() {
// frame_length < 7 (here 0) is a sub-header length → reject.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[3] &= 0xFC; // clear len high bits
f[4] = 0;
f[5] &= 0x1F; // clear len low bits → frame_length = 0
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn raw_aac_without_sync_passes_through() {
// No ADTS sync (e.g. raw AAC from mp4) → cannot validate → keep.
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(
vec![0x21, 0x00, 0x03, 0x40, 0x00, 0x00, 0x00],
Some(0),
));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = AdtsParser::new();
let mut bad = adts_frame(400);
bad[2] = (bad[2] & 0xC3) | (14 << 2); // reserved sr_index
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(adts_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
#[test]
fn short_packet_passes_through() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0)));
assert_eq!(f.len(), 1, "too short to validate → kept");
}
}
+2 -2
View File
@@ -206,8 +206,8 @@ impl PictureInfo {
}
/// Number of field-display periods this picture occupies — the basis for
/// soft-telecine (2:3 pulldown) timing. MPEG-2 (ISO/IEC 13818-2 §6.3.10,
/// ffmpeg `nb_fields = repeat_pict + 2`): a field picture occupies 1 field,
/// soft-telecine (2:3 pulldown) timing. MPEG-2 (ISO/IEC 13818-2 §6.3.10):
/// a field picture occupies 1 field,
/// a normal frame 2, a `repeat_first_field` progressive-frame 3 (or 4/6 in a
/// progressive sequence); an rff bit on a non-progressive interlaced frame is
/// spec-forbidden (§6.3.10) and is treated as 2. Codecs without pulldown

Some files were not shown because too many files have changed in this diff Show More