The keyserver protocol now returns all 32 index keys as an array for a
forensic content sample (and a single-element array for plain content).
resolve_fmts_key_map sends one forensic batch and maps array element i to
segment index i+1, replacing the per-index blind-probe collection loop
that repeatedly hit the key service. Segment/index parsing and the
aligned-unit content classification are reworked to support this:
- rename variant_select -> index_select (per-index, not per-variant)
- content classification moves to is_clean(buf, ContentFormat) so the
unit selector emits only units the key service accepts
- segment.rs: parse IndividualSegment.tbl index tags + SPN ranges,
build contiguous LBA key ranges from the resolved 32-key array
- decrypt/decorator plumbing for the resolved per-index keys
Fail loud (FmtsKeyMissing) when the forensic query returns < 32 keys or
any segment index stays unresolved.
decrypt:
- decrypt_sectors is now a pure decrypt (apply key, leave plaintext, report
unverified bytes); TS-structure is a separate primitive (is_clean_ts/ps) used
only for key selection and read-verify. The mux passes decrypted bytes through
(the demuxer drops non-conforming packets), ending the NULL-TS conceal loop and
the per-unit key-server refetch storm. Key-proof floor replaces the 75%
supermajority.
recovery:
- Removed the post-read decrypt-verify gate (verify.rs) that mis-aligned the
disc-absolute unit grid against clip-anchored AACS units and false-failed good
clips (e.g. Dunkirk's orphan-CPS clip). Bad sectors are marked by physical read
result; decryptability is proven at scan + mux time.
HD DVD (first-class AACS):
- Role-based candidate-list file sourcing so an HD DVD's /ANY!/ files
(MKBROM.AACS, VTKF000.AACS, CONTENT_CERT.AACS) are found with no disc-type
branch. parse_vtkf parses VTKF000.AACS into the same UnitKeyFile as a BD
Unit_Key_RO.inf, so the shared VUK unwrap applies unchanged. set_unit_base
clip-anchoring. Two decrypt-axis assumptions remain UNVERIFIED-HDDVD-DECRYPT
(no encrypted disc to test).
mux:
- MVC (Blu-ray 3D) track signals unified into one MVCDecoderConfigurationRecord;
release-safe track_vint (3-byte VINT) and pid_index (i32) guards.
hardening:
- Container-aware is_clean / encryption detection; bytes_bad_in_title fail-safe
on a corrupt mapfile; CSS crack gated on DiscFormat::Dvd (HD DVD excluded);
non-vacuous CSS tests; patch NOT_READY/HARDWARE/ILLEGAL_REQUEST/ABORTED
sense-path tests.
decrypt_sectors is now a pure decrypt — apply the CPS unit key, leave the
plaintext, report how many bytes did not reach clean TS ("unverified"). It
never restores ciphertext, nulls, or re-fetches. "Did a key produce clean TS?"
is a key-selection / read-verify signal, not the verdict "did we decrypt?": a
correct key can decrypt a bad-encoded region, and broken TS is a muxer concern
(the demuxer drops the packet and resyncs).
Callers own the policy:
- mux (read > decrypt > mux): pass the decrypted bytes to the muxer, whatever
they are; fail loud only on a genuine can't-decrypt (no key / misaligned).
- sweep/patch (reading from a disc): an unverified unit is a bad read — recover
a fresh key and retry, or fail loud so disc-recovery re-reads it.
Removes three duplicated decisions — the decrypt-time ciphertext restore, the
mux NULL-TS conceal loop, and the per-unit key-server refetch — plus the dead
aacs_unit_still_ciphertext predicate. Key-fetch recovery now samples the on-disc
ciphertext explicitly (a pure decrypt leaves the buffer plaintext) and lives
only on the rip/verify path, never the mux.
Fixes the 30-90s/region mux stalls and key-server storm on bad-encoded UHD runs
that 1.4.1 left behind (it relaxed the gate but not the surrounding machinery).
AACS content decryption rejected a whole 6144-byte aligned unit unless
EVERY content packet was conformant MPEG-TS. One authored-bad packet (a
pressing/encoding defect or an AACS 2.1 forensic-variant frame) made the
mux conceal the entire unit as NULL TS — destroying up to 31/32 good
packets and tallying them as loss, surfacing as false "corruption" on
otherwise-clean discs (observed across two UHD titles).
decrypt_unit now asks only "did a key OPEN this unit?" — a padding-aware
>=75% supermajority of content packets restoring their 0x47 sync, a gate
no wrong key can reach (uniform-AES noise floor) yet one that tolerates a
minority of authored-bad packets. Opened units pass through VERBATIM; a
non-conforming packet is left for the demuxer to drop on sync-loss and
resync past. TS-sync conformance is a muxer concern, never a decryption
verdict. The post-read verify/sweep gate now shares the same primitive so
it can never disagree with the mux decrypt.
Also unify the MVC (Blu-ray 3D) track signals: the mvcC CodecPrivate
extension, the BlockAdditionMapping, and each per-frame BlockAdditional
all derive from one MVCDecoderConfigurationRecord built once per track, so
a malformed dependent-view parameter set can no longer orphan a BlockAddID.
Second audit round converged (severity collapsed 6 HIGH -> 1; the one
HIGH was a bounded 32-element scan, not a defect; the sole spec MEDIUM
was the same false-positive re-raised — 0xBF matches ISO/IEC 14496-15
§7.6.2 verbatim). One genuine robustness fix plus coverage:
- extract_mvc_params: skip a zero-length NAL instead of abandoning the
scan, so a stray length prefix before the subset SPS/PPS no longer
silently drops 3D signalling. Test proves params after a zero-length
NAL are still found.
- Tests: parser_for_mvc_dependent routes H.264 to a passthrough parser;
passthrough with an IDR does not re-assert param sets (the keyframe &&
!mvc branch).
- Document the per-playlist (not per-clip) is_3d latching as a known
limitation (real main-feature playlists are uniformly 3D).
Triage of a 10-lens code audit of the 3D branch. Fixes for real defects;
rejected three spec false-positives that matched the ISO/IEC 14496-15
§7.6.2 record verbatim.
Robustness / correctness:
- Never panic when a title's only video is the MVC dependent view: the
base is now the first NON-dependent video, so a dependent-only title
sets up no merge (muxed as an ordinary track) instead of hitting an
`expect` on the skipped track slot.
- Drop a per-frame BlockAdditional (BlockAddID=2) when the track declared
no mvcC mapping (dependent params not captured before the header) — a
plain block keeps the file conforming instead of an orphaned add.
- A non-keyframe MVC base frame always carries a ReferenceBlock (fall back
to a 0 offset in the pre-first-keyframe corner) so it is never mistaken
for a seek point.
- Reference the last keyframe on the PRIMARY video track only, so a
secondary video track's keyframe can't become a cross-track reference.
- dep_by_pts overflow: bound BEFORE inserting so the just-arrived
dependent survives the drift-clear; count a displaced duplicate-PTS
dependent as an orphan instead of losing it silently.
API / docs:
- Fold write_frame_with_additional into write_frame(..., Option<&[u8]>)
per the "no foo_with_X" convention.
- Fix mvc_params doc (StereoMode is intentionally not emitted); remove a
stale PAT/PMT comment describing an approach that was never taken.
Tests: MVCDecoderConfigurationRecord over-length guards; write_int minimal
two's-complement widths; BlockGroup/BlockAdditions/BlockAdditional +
ReferenceBlock emission; additional dropped without a mapping; h264 MVC
passthrough keeps param sets in-band; extract_mvc_params no-panic on
truncated/empty input; pairing window + dep-overflow edges; no-panic on a
dependent-only title.
Fold the MVC dependent (right-eye) view into the base H.264 track as a
per-frame BlockAdditional under an mvcC BlockAdditionMapping, so a 3D
title produces one MVC video track instead of two independent H.264
tracks.
- h264: MVC-passthrough parser mode keeps the dependent view's subset
SPS/PPS in-band, so each emitted frame is a self-contained dependent
access unit for a BlockAdditional
- resolve: route the dependent stream through the passthrough parser
- mkvstream: detect the dependent view, pair it to the base frame by
PTS (bounded FIFO), attach it as a BlockAdditional (BlockAddID=2),
and skip building its own track; build the mvcC
MVCDecoderConfigurationRecord from the captured subset SPS/PPS and set
it on the base track at activation
- mkv: emit the mvcC BlockAdditionMapping and BlockGroup/BlockAdditions,
with a ReferenceBlock on non-keyframe base frames
- ebml: add BlockAdditions/BlockMore/BlockAdditional/BlockAddID/
BlockAddIDValue/ReferenceBlock elements and a signed-int writer
Verified against a Blu-ray 3D ISO: ffprobe shows a single MVC track,
the mvcC mapping is present, ~144k BlockAdditionals carry the dependent
view (8.7 GB), and the base view decodes cleanly with no regression.
MVCDecoderConfigurationRecord follows ISO/IEC 14496-15 7.6.2; StereoMode
is intentionally omitted (no enum value describes MVC-in-BlockAdditional;
the mvcC mapping is the primary 3D signal per RFC 9559).
Blu-ray 3D as a mux-path variant:
- Detect 3D (a clip has STREAM/SSIF/<clip>.ssif — note the SSIF/ subdir, which
the old .ssif fallback path got wrong) and use the SSIF extents for the mux:
one transport stream carrying both the base (left) and MVC dependent (right)
eyes on distinct PIDs, so muxing it captures the full 3D.
- Add the dependent-view video stream. The base STN table lists only the left
eye; the on-disc PAT/PMT are AACS-encrypted (unreadable pre-key) and the base
STN omits the dependent view (it lives in the MPLS STN_table_SS), so use the
BD-3D PID convention: dependent = base video PID + 1 (0x1011 -> 0x1012),
coding 0x20 -> H.264. The demux routes its packets from the SSIF.
info now reports two video streams for a 3D title; the mux writes both eyes.
from_coding_type maps 0x20 (MVC dependent view) to Codec::H264, so the existing
PAT/PMT scan surfaces the SSIF right-eye substream as a second video stream on
its own PID instead of dropping it as Unknown. No new parser: the dependent eye
rides the same demux path as any other TS video stream. Removes the throwaway
3D-structure probes (findings captured in prior commit messages).
First step of Blu-ray 3D as a mux-path format variant: name stream_type 0x20
(H.264/MVC dependent view, the SSIF right-eye substream). Recognition in the
scan/demux + dependent-stream enumeration follow.
Empirical (300, clip 00042): base .m2ts (1567 extents, 25.85 GB) is entirely
inside the SSIF LBA span; dependent = SSIF − base = 1567 ranges, 9.59 GB
(exact vs 35.43−25.85). 200/200 sampled dependent-view units decrypt under the
base UK. => de-interleave needs no SSIF parsing (complement of the base extents
we already resolve), and the dependent eye needs no extra key.
Dev tool for the 3D build: uses the library UDF reader (7z cannot parse these
ISOs) to dump base .m2ts vs SSIF sizes and the MPLS ExtensionData entries
(STN_table_SS + SS sub-path). Not product code.
read_icb_extents parsed a continuation block's allocation descriptors from
offset 0, but a continuation block begins with a 24-byte Allocation Extent
Descriptor (ECMA-167 4/14.5): 16-byte descriptor tag + prev-loc(4) +
length_of_allocation_descriptors(4); the real ADs start at offset 24. Reading
from 0 mis-parsed the AED tag as a descriptor -> one garbage extent, then an
unknown-type break, silently truncating any file whose extents spill into a
continuation block.
Few-extent files store their ADs inline and never hit this path, so it stayed
hidden; a heavily fragmented file (~1600 fragments) truncated to 113 extents +
a bogus non-unit-aligned fragment that then aborted the mux
(ExtentNotUnitAligned). Skip the 24-byte AED header, read l_ad bytes of
descriptors from offset 24. Adds a regression test with a non-zero AED tag.
Add UnitKey.variant_number (0 = ordinary, 1..32 = forensic variant) with new/variant constructors, and aacs::variant_select — resolve a disc's single variant and classify each aligned unit (default / variant / drop foreign / conceal keyless). Correct IndividualSegment.tbl: the per-record field is the variant (cycles 1..32 on a retail disc), not a segment number — Segment.number -> Segment.variant.
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL). The CSS
content cipher and Stevenson title-key attack are attributed to their published
cryptanalysis (not libdvdcss); all libaacs/libbluray/libdvdread/libdvdnav name
references were dropped from comments while keeping the standard format/spec
descriptions. Also bumps to 1.3.1.
Parse ADV_OBJ/VPLST000.XPL (the real HD-DVD player playlist) with roxmltree
into one DiscTitle per <Title>: its PrimaryAudioVideoClip clips in order (EVO
via the .MAP sidecar), the titleDuration, the displayName, and the ChapterList.
A layer-break split (FEATURE_1+FEATURE_2, feature/feature_Divide) composes into
ONE title with the two parts as clips, each carrying its title-time in/out
points (45kHz ticks) for seamless-join splicing. Falls back to the clip-name
heuristic when no playlist is present.
Fixes unknown runtimes (real durations), poor names (FEATURE -> 'Main Movie'),
and gives authoritative composition + chapters. Validated on ANCHORMAN (97m),
SHAUN (99m), HARRY_POTTER (152m).
libfreemkv is publish=false (it git-deps the firmware crate freemkv-unlock,
never on crates.io), so the Release workflow's 'cargo publish' step failed
hard on every tag. Consumers git-tag-pin libfreemkv; the git tag is the
release artifact. Removed the publish job.
Round-11 findings from the 10-phase release audit (no real HIGH):
- When a no-declared-duration source (HD-DVD) muxes a degenerate single
frame at tick 0 with no per-frame duration, max_block_ticks stays 0 and
the reserved DURATION placeholder was left as a literal 0.0 (players read
that as a zero-length file). Void the element instead, so the Segment
omits DURATION as an unknown-duration source did before the back-patch.
- Add a regression test for the sniff_video_codec overlap fix (a
picture_start_code whose payload begins 00 00 followed by a real start
code) so the i+=4 marker skip can't silently regress to i+=3.
Round-10 findings from the 10-phase release audit:
- A finder claimed the DTS SFREQ→rate table was wrong at 11/12; verified
it against ffmpeg's avpriv_dca_sample_rates (12k/24k/48k/96k/192k at
11-15) — the table is CORRECT. Added a test that locks the full table so
it can't be mis-"fixed".
- sniff_video_codec advanced 3 bytes after a matched start code, re-reading
the code byte as an overlapping start code; skip the full 4-byte marker.
- Guard the HD-DVD next_id title counter with saturating_add so a crafted
disc with >65536 clips can't overflow (panic in debug).
- Add a test that an undecryptable unit (DecryptFailed) is zero-filled and
counted as loss through ExtractResult (complete=false, bytes_lost>0) —
the recovery-seam consolidation folded that bucket into bytes_unreadable.
Round-9 findings from the 10-phase release audit (no HIGH):
- Remove the MAX_PTS_MARKS backstop and its tautological test: an empty
DTS PES returns before recording a mark, and a non-empty run is already
bounded by the MAX_AU_BYTES buffer clear (which clears pts_marks) — so
the deque cannot grow unbounded and the cap was dead code.
- AuAssembler::for_codec no longer reserves 256 KiB for a Passthrough
stream (audio/subtitle, and every TS/BD stream) whose buf is never
written; only the reassembling modes reserve.
- Correct the scan comment that claimed region is computed (it is a
Region-free stub until region detection lands) and drop a public-repo
reference to internal "private refactor notes" in the mkb module doc.
Round-8 findings from the 10-phase release audit:
- detect_disc_format's BDMV fallback passed detect_format's result
through unchanged, so an SD bonus/menu title could tag a BD-tree disc as
DVD (mis-sizing the ECC sweep) — violating its own "never below Blu-ray"
invariant. Clamp anything but UHD up to Blu-ray.
- Track the MPEG-2 GOP byte total incrementally instead of re-summing the
whole gop_buf on every pushed picture (was O(pictures²) on any MPEG-2
disc, not just adversarial input).
- Back the DTS pts_marks deque with a VecDeque so the over-cap prune is an
O(1) pop_front, not an O(n) Vec::remove(0).
- Add a test exercising parse_stream_id_extension's PTS/DTS skip branches
(the real AU-opening 0xFD video PES path) — previously untested.
Round-7 findings from the 10-phase release audit (no HIGH; convergence):
- Cap DtsParser.pts_marks (MAX_PTS_MARKS): a run of zero-length timed PES
packets grew no buffer bytes, so the drain_front mark-prune never ran —
the deque could accumulate without bound on hostile PS input.
- detect_disc_format tested HVDVD_TS before BDMV while the title-scan
dispatch tests BDMV first, so a disc with both trees would be classified
HD-DVD but enumerated as Blu-ray. Align both to BDMV → HVDVD_TS →
VIDEO_TS.
- Document why the DTS new-PES re-base can emit a locally-decreasing PTS
(the muxer's block_ts applies the strictly-monotonic audio nudge, tested
in mkv.rs) — this is by design, not a mux defect.
- Fix stale aacs/keys.rs comment references (functions moved to
aacs/inf.rs / aacs::resolve/derive in the module split).
Round-6 findings from the 10-phase release audit:
- Wire the documented MAX_PENDING_BYTES byte cap into the MPEG-2 GOP
buffer (it was dead code) and add an equivalent MAX_GOP_BYTES cap to the
sparse-PTS reorder, so a crafted stream of few-but-huge access units
cannot over-allocate — both were bounded only by frame count before.
- probe_evo_streams defaulted an unsniffable HD-DVD video stream to H.264,
which mis-parses a VC-1 (or still-encrypted) clip into a corrupt track.
Emit the video stream only when the codec is actually identified — the
honest outcome, matching the audio path (a real clear clip always carries
its sequence header at the head).
- Resume the AU-opener search from a cursor (like the boundary search), so
a long unsynced junk run is O(bytes), not O(buffer) per push.
- Mark mpeg2's now-dead MAX_AU_BUFFER test-only; restore #[doc(hidden)] on
the aacs probe harness module.
- Add regression tests: the 0xFD video-routing guard, the FMTS-is-UHD key
state, and the GOP byte caps.
Round-5 findings from the 10-phase release audit:
- collect_es routed EVERY extended-stream-id (0xFD) PES into the video ES
buffer, so a 0xFD HD-audio sub-stream (MLP/TrueHD) could pollute the
video sample and — if it preceded the video PES — stamp the video track
with the audio PID, losing the video. Only the VC-1 extension (0x55) is
now treated as video; routing 0xFD audio to its own track is deferred to
the HD-DVD program-chain follow-up.
- The sparse-PTS reorder now carries its calibrated per-frame duration onto
each frame, so the muxer emits a BlockDuration and the back-patched
Segment Duration covers the final frame instead of understating it.
- Add regression tests for the MAX_MARKS and MAX_VTI_HITS caps (promote
MAX_VTI_HITS to module scope); make the differential-test factory array a
named type; drop an identity-op in a reorder test.
Round-4 findings from the 10-phase release audit (the first fully clean
round; it dug into the new #22/#18 refactor code):
- AuAssembler closed each AU from only the FRONT mark's fields, so when
one PES fragment carried the source and a later fragment of the same AU
carried the PTS, the second field was dropped — a regression vs the old
separate pts/source mark deques. Now merge the first Some of each field
across all in-range marks.
- parse_vti_clip_order picked the largest residue bucket with
HashMap::into_values().max_by_key(), nondeterministic on a size tie
(randomized HashMap iteration) — could select a different clip table
run-to-run. Break ties by smallest offset.
- Bound the marks/disc_marks deques (MAX_MARKS): the buf-size cap prunes
marks only when bytes accumulate, so a run of zero-length timed
fragments could grow them without bound on hostile input.
- Add push_owned so the PS path moves the PES payload into a passthrough
AU with no copy (MPEG-2 video + all audio), removing a per-PES
malloc+memcpy the refactor had introduced on the DVD path.
- Back-patch the MKV duration from the block END (start + its own
duration) so it covers the final frame instead of understating by one.
- Add direct tests for the MKB record-framing walker; drop a stale
drain_complete_aus doc comment left on process_au.
AuAssembler::drain rescanned the whole buffered access unit from a fixed
offset on every push, so reassembling one AU split across N program-stream
PES fragments cost O(bytes²/fragment) — amplified on the HD-DVD PS path
where H.264/HEVC/VC-1 frames are large and now flow through this shared
assembler (unlike the TS path, which delivers one AU per PES).
Carry a scan_pos cursor (and, for the stateful VC-1/MPEG-2 rules, a
seen_unit flag) so each push resumes the boundary search where the last
one stopped instead of restarting. Total scan work for one AU is now
O(AU bytes). The from-scratch scanners are retained as a #[cfg(test)]
oracle; a new differential test asserts the resumable path yields
byte-identical AUs at every fragment granularity for all three modes.
Round-2 findings from the 10-phase release audit:
- parse_vti_clip_order bucketed hits by residue with an O(stride*hits)
rescan and no hit cap, so a crafted HD-DVD VTI packed with millions of
`.EVO` tokens (up to the 64 MiB UDF read cap) could burn seconds of CPU
on a routine scan. Bucket in a single O(hits) pass and cap collected
hits at MAX_VTI_HITS (a real table holds a few dozen).
- Fix the stale `super::keys::…` intra-doc links left by the aacs module
rename: the referenced fns live in `super::derive`.
Round-1 findings from the 10-phase release audit:
- SparsePtsReorder buffered its current GOP with no bound, draining only on
a keyframe — an open-GOP or crafted program stream that never signals one
could hold the whole title in RAM. Force-complete the GOP at
MAX_GOP_FRAMES, matching the MPEG-2 parser's backstop.
- inject_unit_keys labelled a 2.1 FMTS disc as AACS 1.0 / bus-encryption
off; FMTS is UHD-family, so synthesize the UHD version + bus encryption.
- The compiled Key Correction Data was a non-zero 16-byte constant fed into
the Media Key derivation. Per the no-compiled-keys rule it is now all-zero;
the chain still cannot complete on a real disc (documented), so this is
behaviour-neutral — all variant tests pass unchanged.
- Fix stale doc references (broken `super::variants` intra-doc links, and
`aacs::keys` comments) left by the module rename.
A title whose scan yields no duration (HD-DVD — its `.MAP` timemaps are
not parsed, so DiscTitle.duration_secs is 0) produced an MKV with no
Segment Duration element, so players/MediaInfo reported an unknown
runtime.
Reserve a DURATION placeholder when the source declares none, track the
highest block timestamp written, and back-patch the placeholder at
finish() with the real muxed runtime (also enabling the per-track BPS
tags for these titles). Gated on duration_secs == 0, so BD/UHD/DVD —
which carry a real mpls/IFO duration — write it up-front exactly as
before, unchanged.
HD-DVD Standard Content splits the main feature across clips at the
layer break (FEATURE_1/FEATURE_2, or feature/feature_Divide). The scanner
enumerated one title per .evo, so main-title selection picked only part 1
(e.g. Shaun's 11 GB FEATURE_1, missing the 6.8 GB FEATURE_2).
Parse the HVA*.VTI navigation file's clip table — a fixed-stride record
list naming every clip in authored order, isolated by residue-mod-stride
rather than the imprecise header pointer — and concatenate the feature
clips (matched by the feature* naming convention) into one title whose
extents run in authored order. Every other clip stays its own title.
Falls back to one-title-per-clip when the VTI is absent or unparseable,
so nothing regresses on a disc with no readable navigation.
Validated on real discs: Shaun 17.8 GB / Anchorman 20.1 GB / Harry Potter
24.4 GB now enumerate as one 2-clip FEATURE title (largest = the movie).
The MPEG-2 parser hand-rolled its own PES reassembly — a byte buffer plus
parallel PTS / source / discontinuity mark queues keyed by absolute
offset — duplicating what AuAssembler already does for H.264/HEVC/VC-1.
Add a Mode::Mpeg2 to AuAssembler (picture 0x00 with preceding sequence
0xB3 / GOP 0xB8 headers — the same headers-precede-picture shape as the
VC-1 mode) and have the MPEG-2 parser own one via AuAssembler::mpeg2().
parse() now feeds fragments to the assembler and processes each complete
access unit; the buffer, base offset, and three mark queues are gone. The
GOP-buffered temporal_reference reorder and PTS origin-locking are
unchanged. The parser's external contract is unchanged, so all existing
MPEG-2 parser tests pass as-is; new AuAssembler tests cover the MPEG-2
boundary rule directly.
HD-DVD EVO (and DVD VOB) program streams timestamp video at GOP
granularity: only one access unit per GOP carries a PES PTS. The H.264 /
HEVC / VC-1 parsers collapsed a missing PTS to 0, so on such a source
every non-anchor frame landed on the same block timestamp and a decoder
reported "non monotonically increasing dts".
Add a shared SparsePtsReorder that rebuilds a display-order PTS per frame
from the coded picture type (I/P/B) plus the sparse anchor PTS, with a
per-frame duration self-calibrated from the spacing between consecutive
GOP anchors (no external frame-rate needed). Display order is derived via
the classic single-anchor-delay rule (an anchor displays only after the
previously-held anchor; a B displays immediately), exact for the
non-hierarchical GOP structures HD-DVD H.264/VC-1 use. It mirrors the
MPEG-2 parser's GOP-buffered origin-locking.
Gated to the program-stream path only: the three parsers enable it via
with_ps_reorder(is_dvd_ps), so the BD/UHD transport path (per-frame PTS)
is byte-identical and untouched.
Correct the variant Media Key chain against two real variant MKBs
(Zombieland v70, Stand By Me v70): C for Kmp is the per-slot block of
the 0x0c cvalue table indexed by the matched subset-difference, not the
0x2d head. The 0x2d record is the VARIANTS table (leading body-16 bytes,
one big-endian u16 per subset-difference) followed by a trailing 16-byte
Nonce, with no leading header. VKD stays at 0x2f.
Route record-type selection through the named REC_* consts in mkb rather
than bare hex, and document that Key Correction Data is per-licensee: no
universal constant exists and none is compiled in, so on a real disc the
chain yields a wrong Media Key that the Verify-Media-Key gate rejects
rather than emitting a bad key. This is a key-acquisition gap, not a
code gap.
VC-1 HD-DVDs (e.g. Shaun of the Dead) carry video on MPEG-PS extended
stream id 0xFD, with the real stream selector in stream_id_extension
inside the PES extension. Parse that field so the video routes to a
distinct track (pid 0xFD00|ext) instead of being dropped.
Reframe VC-1 access units in AuAssembler with a dedicated Mode::Vc1:
an AU is delimited by the next frame BDU (0x0D) once a frame has already
been seen, so the sequence (0x0F) and entry-point (0x0E) headers that
precede an I-frame stay attached to the frame they describe. The old
single-start-code split stranded those headers on the prior AU, which
the decoder reported as bits-overconsumption and hard decode failures.
hddvd probe now tracks the video pid it detects and emits VC-1 on 0xFD.
Bump to 1.3.0 and add the 1.3.0 changelog entry (FMTS/HD-DVD formats, AACS 2.1 variant chain, recovery seam, aacs module split, main-title-by-size, and the fixes since v1.2.2). Not tagged or pushed.
- aacs/resolve: a media-keys-only provider missing the VID classifies as
VidUnavailable, not NoMaterial (an MK derives the VUK once the VID
arrives).
- disc/bluray: mark a clip seen only after its .clpi parses, so a
transient parse failure on the first PlayItem cannot suppress the
clip's extents for a later PlayItem referencing it that succeeds.
- disc/patch: log rather than swallow mapfile record/flush failures on a
reverify downgrade, so a failed persist cannot silently mismark a bad
unit good on resume.
- mux/ts: flag a discontinuity when a partial PES is dropped, matching
the other partial-drop paths.
- mux/demux_thread: the no-demuxer branch forwards an empty batch for
early consumer-disconnect detection instead of reading the whole disc.
- io/pipeline: correct the send-timing log (as_secs_f64, not as_micros
printed as ms).
- aacs/derive, aacs/variant, disc/read_error, keysource: comment/doc
accuracy. sector/prefetched, udf: remove dead fields/functions.
- mux/disc: assert unit-aligned read counts in the test.
Replace the AACS-specific inline key-fetch in the decrypt decorator with
a scheme-neutral recovery seam: the input stream (L3) installs a Recover
closure (none / AACS key-fetch) and the decorator (L2) runs it at the
single decrypt-miss point. FMTS (AACS 2.1) forensic-segment units that no
key opens are just undecryptable units, concealed and counted as ordinary
decrypt loss with no FMTS-specific branch ("a loss is a loss"), so the
separate bytes_undecryptable bucket collapses into one loss count.
- sector/recovery.rs: the seam (MissOutcome, none/key_fetch factories),
naming no encryption scheme in its type.
- FMTS: segment routing primitives + BYPASS_FMTS_KEY, and an upfront
ensure_forensic_segments_decryptable gate (Error::FmtsKeyMissing) in
the mux input path, parallel to the unit-key gate.
- CSS descramble/rekey moves from decrypt_sectors into
css::descramble_region: CSS self-recovers from the data itself, so it
stays OFF the seam (which is only for external inputs).
- disc/mod.rs also: main-title selection aligned to largest physical
size; is_regular read from the open file handle, not metadata(path),
fixing a swallowed sync_all on a fresh-rip ISO. decrypt_threads()
resolved once via OnceLock off the per-buffer hot path.
- DetectFn now takes a SectorSource so a parser can inspect a jar's
central directory in detect() instead of firing on "any BD-J jar".
dbp/deluxe do the real com/<vendor>/ prefix check up front, so each
claims only its own discs (foundational for scaling the registry).
- criterion: treat a stream-map value of 0 as unmapped and synthesize a
real 1-based number, so a 0 can't shadow or collide with a genuine
stream 1 (with regression tests).
- png_filenames: new Low-confidence, last-resort parser reading menu
language from {title}_UHD01_{LANG}_Composite artwork; sits below the
MPLS floor so a real framework parser always wins.
- vocab: add menu_lang() for 639-2/B to 639-2/T menu-token normalization.
New aacs::segment_key: parses the AACS 2.1 SegmentKeyNNNNN.tbl container, confirmed against a retail disc as an 8-byte header + 65536 records of 536 bytes, indexed by the 16-bit variant selector the Media Key Variant chain produces. This is the confirmed link between the two 2.1 variant layers (selector picks the device's per-segment variant). The per-record 528-byte payload layout is not yet reversed.
New aacs::segment: parses the AACS 2.1 IndividualSegment.tbl into the source-packet ranges of the forensic variant segments (validated against a retail disc: 792 segments, 2560 packets each). First piece of the FMTS variant decoder — the segments' variants are encrypted under segment keys, not the unit key, so a unit-key rip corrupts them (broken HEVC refs).
canonical_title_order keyed on clip-count ascending before duration, so a short 1-clip bonus reel outranked the real feature — which is often chaptered into many clips (one per chapter). Make physical size the primary key: the main feature is the largest non-oversize title; decoy 'play-all' playlists run long but tiny (reused clips) and self-eliminate.
Validated across 23 UHD/BD discs: fixes t1 on F9, Fast Five, Fast & Furious 6, and Furious 7 (feature was ranked #13-36); no regressions on the 19 already correct.
Add DiscFormat::Fmts (AACS 2.1) and DiscFormat::HdDvd as first-class peers. Format derives from the AACS MKB generation (mkb_type().generation(): V10=BD, V20=UHD, V21=FMTS), reusing existing AACS code, and from the on-disc tree for HD-DVD/DVD. One detector (detect_disc_format) shared by the coarse DiscId probe and the full scan — no more 'default BluRay, defer to full scan'.
FMTS is a BD-tree stream variant: parse_playlist resolves the clip stream via CLIP_STREAM_EXTS (.m2ts -> .fmts -> .ssif), so the .fmts main feature yields real extents (previously silently empty). HD-DVD is a tree-level peer with its own enumerator (disc/hddvd.rs): HVDVD_TS/*.evo -> MpegPs titles with real extents (playlist/stream parsing honestly stubbed).
Sample selection for key resolution now uses the authoritative AACS CPI flag (aacs_unit_encrypted, byte-0 & 0xC0) not the ts_sync_destroyed heuristic — container-agnostic (M2TS/FMTS/EVO; TS-sync is meaningless on HD-DVD program streams) and stops the decode-server '0 encrypted units' rejection.
Tests live with each format (bluray/hddvd/mod); generic UDF fixture builders extracted to a shared udf::fixture module.
Adopt freemkv-unlock's split Unlocker trait: run_features drives the drive-prep capability, run_bus the content bus removal, each iterating unlockers until one doesn't decline (NotApplicable = try next; Ok or a real error stops).
unlocker_matrix now reports which drive-prep unlocker actually ran — LibreDrive removes the bus at the drive; Renesas unlocks features but leaves the bus to the cert. Wire product_id through to fu::DriveId.
Bump to 1.2.3.
aacs::boil was a thin newtype veneer over aacs::derive — a duplicate layer.
Delete it: move the shared key newtypes (Vid, MediaKey, Vuk, ProcessingKey,
UnitKey) into aacs::types, and expose resolve_candidate + KeyCandidate from
aacs::derive directly. Downstream (keysource, disc::encrypt) now import from
aacs::{derive,types}. Pure API consolidation, no behaviour change; full test
suite green on Rust 1.86.
The module structure is the public API, but the typed key primitives
(MediaKey/UnitKey/Vid/Vuk, mk_from_dk/mk_from_pk/uk_from_vuk/vuk_from_mk),
derive_vuk, the aligned-unit decrypt entry points, and DeviceKey/HostCert
are load-bearing names that downstream key-source crates import through the
flat `aacs::` path. Re-export them here so those crates can track the
module refactor without a lockstep re-pin.
Redesign derive_media_key_variant to the minimal derivation surface:
derive_media_key_variant(mkb_records, pk) -> Km
- PK-input only. Deriving Kp from device keys (DK -> PK) is a separate
concern (walk_processing_key); a leaked 2.1 key is a PK, and the chain
starts at Kp. A bare PK arrives without its subset-difference slot, so
the primitive tries it against every slot and returns the Km for the
slot that passes the MKB's Verify-Media-Key record -- mirroring the
classical bare-PK derive_media_key_from_pk, gated by the chain's own
verify so an unverified key is never returned.
- VID-free: the Media Key is MKB-scoped. VUK stays the separate
derive_vuk(Km, VID) step.
- KCD is a fixed algorithm constant compiled in, not a caller parameter
(removes the kcd argument, the placeholder const, and KcdNotProvided).
- Soft-correction / online-challenge slots are treated as non-covering;
surfaced over the generic miss so a disc needing those modes is
distinguishable from a non-covering key.
resolve_keys_v21 updated to walk DK -> PK first, then call the primitive
and derive the VUK from Km + VID. Module + helper docs refreshed to the
pinned record layout; tests reworked for the PK-only signature.
variants_for_uv previously returned None (placeholder), dead-stopping the
Media Key Variant chain at VariantsTableUnavailable on any real disc. Layout
now pinned against two real 2.1 variant MKBs (Zombieland v70, Stand By Me v70):
the 0x2d Encrypted-Media-Key-Variant-Data body is sd_count u16 VARIANTS entries
(1:1 with 0x0c cvalues / 0x04 subset-diffs) followed by the 16-byte per-disc
Nonce at the tail. variants_for_uv reads the sd_slot_index-th u16.
With this the chain runs fully: Kmp -> Kpnew(=Kmp^KCD, extracted CyberLink
constant) -> VKD(0x2f) -> Km -> Kvu. The only remaining input is a covering
2.1 Processing Key to validate against a known answer; until then the final
Verify-Media-Key (0x86) gate rejects any wrong layout pick, so a bad key can
never be emitted. Tests updated to reflect the wired lookup; 1.86 precommit
green (fmt+clippy+tests).
The facade removal (5ff0464) left tests/ calling aacs::disc_hash,
aacs::decrypt_unit, aacs::AacsVersion, etc. at the old flat paths,
so the branch did not compile its integration tests. Repoint each
to its real module (inf/derive/content/mkb). No logic change.
The volume_key module was only 34 lines and is just the tail of the same
DK/PK -> MK -> VUK -> UK derivation ladder as media_key. Fold both into one
derive module so every aacs module is a substantial, distinct responsibility
(crypto/mkb/derive/inf/content/variant/resolve). Relocation only; logic hash
identical (95fb9924); 2210 tests green.
Break the 2800-line keys.rs into four responsibility-scoped modules:
- media_key.rs: DK/PK -> Media Key subset-difference walk (+ probe harness)
- volume_key.rs: VUK derivation, unit-key unwrap
- inf.rs: Unit_Key_RO.inf parsing, disc_hash, content cert, in-drive MKB read
- resolve.rs: the resolve_keys_* orchestration (keys.rs renamed)
Relocation only; the (white-box) test suite stays in resolve.rs and pulls
the moved items via glob imports. Proven byte-identical to the pre-refactor
state via the logic hash (95fb9924); 2210 tests green.
Relocate the shared MKB machinery into a single mkb module: the record
framing walker + MkbRecord view (from variant), the MkbType/AacsVersion
classification, the MKB-file utilities, and the record-body finders (from
keys). Fixes the inversion where the MKB parser lived in the 2.1-only
variant module. variant.rs keeps its local MkbRecord-based mkb_find_mk_dv
(name collision with the raw one; unified in the dedup follow-up).
Relocation only. Proven byte-identical to the pre-refactor state via the
function-body logic hash (imports normalized out); 2210 tests green.
Relocate the shared low-level primitives into a single crypto module:
aes_ecb_encrypt/decrypt, aes_cbc_decrypt, aes_g (from content/variant) and
aesg3 + AESG3_SEED (from keys), plus AACS_IV. Fixes the scatter where AES-G
lived in the 2.1 file and AES-G3 in keys. Relocation only — no rename, no
logic change (logic-hash identical to baseline; 277 items; 2210 tests green).
Pure file+module-path rename. 'content' names the AACS unit-decrypt layer
(distinct from the top-level sector-decrypt driver crate::decrypt), and
'variant' (singular, spec term 'Media Key Variant') names the 2.1 chain.
Logic-hash identical to baseline; 277 items intact; tests green.
Add [C]/[PR]/[BD]/[libaacs] §x.y provenance markers across the AACS
crypto so each primitive links to the spec section it implements, with a
source-tag legend in mod.rs. Doc-comments only — no logic, constant, or
signature changes.
Also: correct two stale record-type comments in variants.rs (0x82/0x83 →
the real 0x2d/0x2f) and document the Variant Number width (spec lsb_10 vs
the 2.1 chain's lsb_16, driven by the 65,535-entry VKD table).
The Media Key Variant scheme is detected and parsed via the actual MKB
record types found on a real variant disc:
- 0x2d Encrypted Media Key Variant Data (C)
- 0x2f Variant Key Data table (65,535 x 16)
- 0x0c variant cvalues (one per 0x04 subset-difference slot)
Replaces the earlier placeholder 0x82/0x83 record types, which were a
guess and appear on no real MKB. is_variant_mkb, the record finders, and
the subset-difference cvalue source (now 0x0c, falling back to 0x07/0x05)
are updated accordingly, along with the V20->V21 upgrade detection in
resolve_keys_v2 and its fixtures.
The variant chain still halts at variants_for_uv (the VARIANTS[uv] /
Nonce sub-field offsets need a covering key to confirm end-to-end
against the 0x86 verify), so a best-effort offset is never silently
trusted. All 27 variant tests pass on Rust 1.86.
A Processing Key is the key at its Subset-Difference node — one AES-G from
the Media Key — so it is tried directly against the MKB cvalue tables,
matching libaacs _calc_mk_pks (iterate PKs × cvalues). The prior code
treated every PK entry as a device-node label at unknown depth and BFS-walked
the SD tree (depth 3, capped 5), which was both wrong for terminal PKs and
~15x slower on a large UHD MKB (~181k cvalues): PK derivation on UHD dropped
from ~37s to ~2.4s.
The Subset-Difference tree walk now lives solely in the device-key path
(derive_media_key_from_dk), which owns per-node path bits; the PK path never
descends. Removed PK_WALK_MAX_DEPTH / _CAP / _walked / walk_pk_against_tables_impl;
renamed the core scan try_pk_against_tables and its probe test.
Precommit (Rust 1.86): fmt + clippy + tests green.
The first cut used a global running clock (max(next, own-PES PTS) + advance),
which fixed the same-PES collision but DRIFTED: once accumulated frame
durations exceeded the PES-timestamp spacing, it never re-based, so a
feature-long DVD DTS track ran minutes past its real length (2h44 for a 2h03
film) while AC-3 from the same source stayed exact.
Match the AC-3 path: re-base to each PES's own container timestamp, and advance
by one frame duration ONLY within a run of AUs sharing one PES. Fixes the DVD
multi-frame-per-PES collision without drift; the UHD DTS-HD MA per-PES
attribution (da85f56) is preserved (each AU still takes its own core PES's PTS).
Adds new_pes_rebases_to_its_own_pts_no_drift; full mux suite green (905).
DVD packs several DTS core frames into one PES; the parser stamped every
access unit with that single PES PTS and duration_ns=None, so consecutive
frames collided on one timestamp — ffmpeg rejected the output as 'non
monotonically increasing dts to muxer: X >= X' (deep-decode = corrupt,
e.g. The Punisher). The UHD DTS-HD MA path (one AU per PES, distinct PTS)
was unaffected, which is why this only surfaced on DVD.
Parse the DTS core header for samples ((NBLKS+1)*32) and sample rate
(SFREQ, 48kHz fallback) to derive each AU's duration, and stamp a running
monotonic PTS: max(next_clock, own-core-PES PTS), then advance by the
frame duration. A later PES whose PTS is ahead of the clock still wins
(preserves the UHD per-PES attribution from da85f56/c49a180); frames
sharing one PES advance frame-by-frame instead of colliding.
Tests: the 3 that encoded 'same PES -> same PTS' now assert monotonic
advance; new dvd_many_cores_one_pes_are_strictly_monotonic reproduces the
Punisher bug; duration/SFREQ-fallback unit tests added.
The tier ladder (bucket->mug->thimble) forces tier 0 to attempt ALL
ranges before tier 1, and tier 1 all before tier 2 — so on a hardened
residual (late resume, flood already gone) the marginal specialists are
unreachable for hours: you bail near-empty buckets over hundreds of dead
ranges before a thimble ever touches water.
FREEMKV_PATCH_FLAT collapses the 3 tiers into ONE flat pool of all 16
handler configs. run_handlers already sorts best-first by the live
decayed-yield scorecard, so this is a data-driven bandit: the first
ranges try them all (explore), the ranking floats whatever is actually
landing sectors to the front (exploit), re-measured per range; a handler
that doesn't fit stays last but is never dropped (floor -> can revive).
The new handlers self-limit (yield after 4 dead reads), so the flat
scheduler does NOT reintroduce the old depth-first per-range starvation.
Opt-in; unset keeps the proven tier ladder as the default for 1.2.0.
The tier-2 marginal specialists now additionally probe permanently-bad
residual sectors before leaving them NonTrimmed, so the alternating-bad
profile's finite read count rose past the old tier-0/1 bound. Guard
still catches runaway; behavior asserts unchanged.
Add a third breadth-first tier (PATCH_TIERS 2->3) that runs the marginal
specialists on the hardened residual tiers 0-1 leave: SlowSpin (Linear
fwd+rev @ min), FuaRetry (Linear fwd+rev+Bisect @ FUA), SlowFua (Linear @
min+FUA), CachePrime, Oscillate (@ max and @ min), SpeedSweep. Every read is
a wedge-safe read_span, so they inherit wedge-abort / unproductive-yield /
deadline for free. All are new configs, so the EWMA scorecard calibrates each
once then ranks by decayed rate — a specialist that doesn't fit self-
deprioritises. Tiers 0-1 (fast scouts, slow-deep) are unchanged; this is
purely additive. Also switch the scorecard log sort to sort_by_key.
cargo test -p libfreemkv green (2200 passed).
CachePrime reads the good run immediately preceding a residual island to lock
the servo/PLL, then reads the island warm (boundary sectors the drive can't
cold-seek). Oscillate reads each residual sector by alternating approach —
forward-into then reverse-into — for direction-dependent tracking. Both go
through the wedge-safe read_span (primes included). FakeDisc models a
direction-dependent sector and a servo-primed boundary sector; fixtures prove
a forward/cold Linear misses each while Oscillate/CachePrime recover them.
FuaRetry (Linear fwd+rev+Bisect at {max,fua,deep}) and SlowFua (Linear at
{min,fua,deep}) are Linear/Bisect at FUA params, no new struct. FakeDisc now
models a stochastic sector that lands only on the Nth physical (FUA) read (a
cached re-read keeps missing) and a hardest sector needing BOTH min speed and
FUA. Fixtures prove cached reads keep missing while the FUA group lands the
stochastic sector, and that neither lever alone recovers the hardest sector —
only the min+FUA combination does.
SlowSpin is Linear pinned to min speed (no new struct). SpeedSweep is a new
per-sector handler that tries Max->Min until one reads (speed resonance).
FakeDisc now models a sector that reads ONLY at min speed; fixtures prove a
max-speed Linear misses it while SlowSpin and SpeedSweep recover it (and that
SpeedSweep tries fast-then-slow).
Grade handlers by an exponentially-decayed recovery rate (bytes/second,
alpha=0.5) instead of a cumulative rate. The residual hardens mid-pass, so
the best technique changes: cumulative froze the early winner in the lead
forever; the EWMA re-prices continuously — a handler that stops earning
decays down, a late-starting specialist climbs. Keeps rank()'s untried->top
one-shot calibration and attempted-but-zero-time->bottom.
New flip fixture proves a handler that recovers a lot early then nothing
loses its lead to one that starts recovering later.
Add ReadParams { speed: SpeedPref, fua: bool, timeout: TimeoutPref } and
thread it through read_span so every wedge-safe handler read can request a
spindle speed (SET CD SPEED issued only on change, restored to max when the
handler exits), set the READ(10) FUA bit, and pick the 10s vs 60s timeout.
- SectorSource gains read_sectors_fua (default ignores fua); Drive sets the
CDB bit, DecryptingSectorSource threads fua to its inner read.
- recovery_read gains a fua param.
- Linear becomes { direction, params }; Bisect/Jump take params. Existing
tier-0/1 instances keep identical behavior (max speed, no FUA, fast/deep).
- Scoreboard keys on the full-config String name (linear:fwd:max:fast, ...).
- FakeDisc observes speed + FUA + approach so specialist techniques are
provably exercised in later commits.
cargo test -p libfreemkv green (2193 passed).
Matthew Jackson