Compare commits

..
92 Commits
Author SHA1 Message Date
Matthew Jackson b2ebae8f81 v1.5.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-22 22:19:39 -07:00
Matthew Jackson b2f35a8afc CSS DVD: resolve the per-title key at read time, drop the scan-time crack
Every DVD read path — the file-backed mux highway (build_iso_pipeline) and
the live-drive single-pass DiscStream — now resolves the per-VTS CSS title
key through one shared step, css::resolve_dvd_title_key, cracked keylessly in
playback order from the title's own extents. Removes the earlier design that
reused a single scan-time key (meaningless for a per-VTS scheme) and muxed a
detection-miss disc's scrambled sectors as garbage.

- Disc::scan no longer cracks a key up front; it does only the CSS bus-auth
  read-unlock, hoisted before the UDF prefetch so scrambled small/menu VOBs
  no longer cost a rejected read each (CSS-DVD scan ~25s -> ~6s).
- An uncrackable title hard-fails (E7023) instead of passing ciphertext as
  plaintext; --raw skips the crack entirely; a Stop mid-crack surfaces Halted.
- DiscStream::new is now fallible and threads raw + halt.
- Fix a stale codec-parser doc claim (TrueHD/FLAC/MP2/AAC do gate via DropTally).
2026-07-22 22:15:27 -07:00
Matthew Jackson a6a4a5d7d2 libfreemkv: 1.5.2 changelog reflects the shipped playback-order crack
The prior wording described the round-1 'self-cracking per-sector' approach
that the audit replaced; rewrite it to match what ships (per-title crack in
playback order + scan-key reuse + loud hard-fail on an uncrackable title).
2026-07-22 13:48:56 -07:00
Matthew Jackson 0de6b82a37 libfreemkv: restore freemkv-unlock path dep on the branch tip
The last release left this pinned to the git tag v1.5.1; release.sh expects
a committed path dep on the branch tip (it swaps path -> git tag only in the
tagged commit, matching bdemu). Restores that state so a local full-workspace
build links the sibling freemkv-unlock, and release.sh's path->tag swap works.
2026-07-22 13:43:32 -07:00
Matthew Jackson b9b0a87328 CSS DVD mux: per-title key via scan-key reuse + playback-order crack
A CSS DVD whose main title was mis-detected as unencrypted (the up-front
crack scanned the largest cell first and starved its budget in that cell's
clear prefix) muxed scrambled sectors as plaintext at exit 0. CSS leaves
the pack/PES header clear, so an un-descrambled sector muxes as a
structurally-valid but corrupt PES packet with zero loss reported.

decrypt_keys_for_title resolves a DVD title's CSS key two ways:
- Fast path: reuse the scan's cracked key when its crack_span covers this
  title's VTS (no re-read; on a live drive no second bus-auth).
- Crack: on a detection miss or a different VTS, crack from the title's OWN
  extents in a SINGLE scan in natural PLAYBACK ORDER (never largest-first).
  One scan = one CSS-locked early-bail, so a locked title is not re-hammered
  per cell against a live drive (hard rule #2); the 50k-sector budget is the
  same accepted bound the disc-wide scan uses. Cracked -> key; Unencrypted
  -> clear; ScrambledUncracked -> hard-fail.
ensure_title_decryptable hard-fails an uncrackable DVD title even when
detection missed, and passes a title that resolved its OWN valid key
regardless of the disc-wide css_error. descramble_region is unchanged from
v1.5.1 (validated-key seed).

Also rename the unlocker report's DVD entry CSS -> DVD. Bump 1.5.2.
2026-07-22 13:07:26 -07:00
Matthew Jackson d6cef9f665 v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:04:24 -07:00
Matthew Jackson b9b9185633 restore freemkv-unlock path dep for local dev (post-v1.5.1) 2026-07-20 17:02:14 -07:00
Matthew Jackson 66951a32ed v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:02:11 -07:00
Matthew Jackson 505a78efd1 mux/codec/truehd: fix MLP major-sync checksum endianness (was dropping the whole TrueHD track)
Regression since the previous release, which added an MLP major-sync checksum
gate to drop genuinely-undecodable audio frames. The checksum itself was computed
with mismatched byte order: `crc16_mlp` is the correct crc_2D table (poly 0x2D,
MSB-first) but returns its two bytes in the OPPOSITE order to libavutil's
`av_crc`, and `mlp_major_sync_crc_ok` then folded in the pre-trailer word
little-endian while comparing the trailer big-endian. The net result never
matched a real major sync, so EVERY major sync was judged corrupt. That armed the
drop-forward on the first AU and, since no major sync ever validated to clear it,
collateral-dropped every following AU forever — the entire TrueHD track was
silently dropped. Its AUs then flushed only at mux end, so the track's blocks
landed physically after all the video: a decoder reading video+TrueHD had to
buffer the whole title to reach the first audio block and spiralled into an
unbounded memory runaway ("decoder ran out of memory"). Every TrueHD title
produced after the gate landed was affected; a title from the release before it
is clean. (The header-size parse — a frequent suspect for extended 7.1/Atmos
headers — is NOT the bug; it already matches ffmpeg's `mlp_get_major_sync_size`
byte-for-byte.)

Fix: compute the checksum exactly as ffmpeg's `ff_mlp_checksum16` —
`crc16_mlp(body).swap_bytes() ^ AV_RL16(word) == AV_RL16(trailer)`. Cross-verified
byte-exact against two real discs (a 7.1/Atmos title and a 5.1 title, independent
32-byte headers both validate). With the checksum correct, major syncs validate
and the drop-forward corruption protection works as intended.

Defence in depth: a major-sync checksum that STILL can't be validated (a genuinely
corrupt or as-yet-unparsed header) no longer arms the drop-forward until we hold a
validated baseline (`num_substreams` from a prior clean major sync) — so a single
bad header can never again silently drop an entire track.

Tests: the `finalize_major_sync` fixture now builds the checksum the corrected way;
a synthetic checksum-failed head major sync is kept, not dropped; the existing
baseline-then-corrupt drop-forward tests still pass. Verified end to end against a
real disc: the TrueHD track demuxes to a full, cleanly-decodable 48 kHz 8-channel
stream, interleaved with the video, instead of 0 bytes.
2026-07-20 16:58:06 -07:00
Matthew Jackson 9b80084220 aacs: parse HD DVD VTKF title keys at the spec's 36-byte stride
The HD DVD Title Key File (VTKF*.AACS) stores 64 title-key entries of 36 bytes
each — 1-byte BIFO + 3 reserved + 16-byte encrypted key + 16-byte binding MAC —
per AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, confirmed byte-exact
against real discs (Freedom VTKF090, Dukes VTKF000: every 36-byte slot has
BIFO=0x80, a clean key, and a 0xFF binding MAC).

The parser used a 32-byte stride (a 12-byte pad in place of the 16-byte binding
MAC) with flag-based termination. That aligns entry #1 (key at offset 132, where
both strides agree) but drifts +4 bytes per entry after it and never terminates
(the previous entry's 0xFF MAC reads as a set present-flag), so it recovered a
correct key only for single-CPS-unit discs and garbage for CPS unit >=2. Every
multi-title HD DVD (Freedom, Harry Potter) was affected.

Fix: 36-byte stride, iterate the fixed 64 slots, take slots whose BIFO AV_FLG
(bit 7) is set, key at offset 4, slot index = CPS unit (skip empty slots rather
than terminate so a gap can't renumber later keys), and never read the trailing
16-byte TKF MAC as a key. Tests rebuilt on the real layout, including a full
64-entry file.

Also correct the VTKF-selection TODO in mod.rs: the AACS HD DVD Book gives the
selector explicitly (match the TKF's PLAYLIST_NAME field to the active
playlist), not the "validate against an encrypted unit" placeholder.

Reconciled against the new HD DVD reference (freemkv.org/docs/hddvd/); the spec
source is archived in private-repo/spec/.
2026-07-20 11:57:13 -07:00
Matthew Jackson b246169e5b aacs: discover HD DVD AACS dir + title-key files instead of hardcoding /ANY!/VTKF000
The HD DVD AACS directory name and title-key filename are chosen by the
authoring house, but the resolver hardcoded a single spelling
(/ANY!/VTKF000.AACS, /ANY!/MKBROM.AACS, /ANY!/CONTENT_CERT.AACS). Real discs
diverge: Freedom (Memory-Tech) names its AACS dir AAC! and ships VTKF090.AACS
+ VTKF100.AACS; Harry Potter carries VTKF000/001/002/099. On such a disc the
hardcoded path finds nothing, so no MKB/title-key/cert is read and decryption
silently can't engage.

Replace the fixed HD DVD path constants with structural discovery:
- find_hddvd_aacs_dir() locates the AACS dir as the root child dir ending in
  '!' that contains MKBROM.AACS (so the ..._BAK mirror is skipped; the dozens
  of decoy advanced-content '!' dirs are excluded by the MKBROM.AACS guard).
- role_paths(udf, role) builds the ordered candidate list per role: the static
  BD/UHD /AACS/ paths first, then the discovered HD DVD files — MKBROM.AACS,
  CONTENT_CERT.AACS, and every VTKF*.AACS (sorted), not just VTKF000.
- read_first() is now generic over &str / String so it takes the Vec<String>.

BD/UHD unaffected (no '!' dir → discovery returns None, list is the /AACS/
constants exactly as before). Verified on real Freedom (AAC!/VTKF090+100) and
Dukes (ANY!/VTKF000) ISOs; unit tests cover both shapes.

Open item (TODO(hddvd-encrypted)): when a disc has multiple VTKF variants the
correct one must be chosen by validating its VUK-derived key against a real
encrypted unit rather than first-that-reads. Blocked on obtaining a genuinely
encrypted HD DVD image — all HD DVD ISOs on hand are already-decrypted rips.
2026-07-20 11:29:28 -07:00
Matthew Jackson 2d96a3e87d release: tee up 1.5.1 (marginal-read reporting fix) 2026-07-19 20:47:52 -07:00
Matthew Jackson bae5a9ea07 drive: report marginal reads (PER=1) so a dirty disc can't pass clean
A read was only ever judged "good" by the drive's SCSI GOOD status — with no
integrity check on the bytes. On dirty/marginal media a drive can silently
return best-effort ECC data (occasionally mis-corrected) with GOOD status, so a
rip could "pass clean" yet decode with errors, pushing corruption downstream to
the mux instead of catching it at the read.

Enable recovered-error REPORTING at drive-prep: MODE SELECT the Read-Write
Error Recovery page with PER=1 (TB on / DTE off so the data still comes back),
preserving the drive's own retry count. Marginal reads now surface as
CHECK CONDITION / RECOVERED ERROR. Best-effort — a drive that doesn't honor it
keeps its defaults (no regression), and on a clean disc nothing changes.

A recovered error is distrusted: it marks just that ECC block NonTrimmed for a
Pass N re-read (a clean re-read wins; a persistent marginal becomes an honest
concealed gap), and is counted in the pass summary. Crucially it takes a
per-block SkipBlock, NOT the damage-jump path — a single marginal sector must
not trigger a 64 MB skip that would discard good data on a lightly-smudged disc.
2026-07-19 20:30:21 -07:00
Matthew Jackson 86ed27508d changelog: 1.5.0 release date + trim to app-level detail 2026-07-19 17:34:21 -07:00
Matthew Jackson f28d79b51b restore freemkv-unlock path dep for local dev (post-v1.5.0) 2026-07-19 15:37:27 -07:00
Matthew Jackson 5b27b9f8a5 v1.5.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-19 15:37:25 -07:00
Matthew Jackson 405e9d32c3 changelog: drop-on-undecodable audio + forced PGS detection (1.5.0) 2026-07-19 14:29:57 -07:00
Matthew Jackson 21b362c6e8 disc: detect forced PGS subtitles from stream content for info
Give `info` the same forced-subtitle verdict the muxer derives during a
rip, so the two agree. A shared classifier (mux::codec::pgs::ForcedTracker)
folds a PGS track's display sets — forced iff every one carries the
forced_on_flag — and is used by BOTH the MKV writer and a new scan-time
probe that reads the title's PGS streams (reusing the TS demuxer and PGS
parser). The probe only overrides a track it actually observed content
for, so an undecrypted/unread stream keeps its vendor-derived flag. Gated
behind ScanOptions::probe_forced_subtitles (off for the rip path, which
detects forced while muxing without a second read).
2026-07-19 14:23:04 -07:00
Matthew Jackson 9f8df95513 mux/codec/mpegaudio: keep free-format frames
Free-format MPEG-audio (bitrate_index 0) is a legal, decodable mode — the
decoder derives the frame size from the sync spacing. Dropping it was a
false positive on a clean stream, so it now passes the gate.
2026-07-19 14:23:04 -07:00
Matthew Jackson 2f465f4cfe mux/codec/truehd: fix drop-forward poison + rate/resync validation
Three TrueHD state-machine fixes from an adversarial audit:

- A corrupt access unit drops forward to the next major sync, but only
  the individually-verified corruption now feeds the whole-track poison
  verdict; the resync run is collateral. A couple of transient errors can
  no longer poison and discard an otherwise-good multi-hour track. The
  shared drop tally gains a verified/collateral split for this.
- The per-AU PTS rate is refined only from a CRC-validated major sync, so
  a corrupt major sync whose rate nibble decodes to another rate family
  can no longer shift the resumed audio — a drop stays a silence gap.
- The resync clears only on a CRC-validated major sync, never on a runt
  too short to hold and validate its header.
2026-07-19 14:23:04 -07:00
Matthew Jackson 3fdd463794 release: 1.5.0 2026-07-19 13:42:48 -07:00
Matthew Jackson 6a7392ac8c mux: detect forced PGS subtitles from the stream
Flag a PGS subtitle track FlagForced when it displays subtitles and every
one carries the HDMV forced_on_flag (a dedicated forced/narrative track),
independent of the disc's vendor label metadata. The track header
reserves a FlagForced byte up front and it is promoted at finish() from
the accumulated display-set state. Only ever promotes — a track already
forced from the playlist metadata is never demoted.
2026-07-19 13:42:48 -07:00
Matthew Jackson 28bdecd26b mux/codec: drop undecodable audio frames, keep A/V sync
A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch. Sync is preserved — a drop becomes a silence
gap, never a shift — and every drop is logged. Detection is per-codec,
each mirroring the format's authoritative integrity check:

  DTS          core-header validity gates
  AC-3/E-AC-3  native frame CRC-16 + bitstream-id range
  FLAC         whole-frame CRC-16 residue
  MP2/MP3      header sanity + free-format reject
  AAC-ADTS     header sanity (raw AAC passes through untouched)
  TrueHD/MLP   major-sync CRC-16 + AU parity; corrupt AUs drop forward
               to the next major sync, since decode state carries
               across access units

LPCM and video are excluded by design (no in-frame integrity data;
inter-frame prediction). A shared DropTally handles counting, logging,
and a whole-track fallback for a mostly-undecodable track.
2026-07-19 13:42:48 -07:00
Matthew Jackson d29d3d71b9 dts: emit clean core alone when the extension boundary is garbage
DTS-HD MA access units are a lossy core frame followed by trailing
extension substreams up to the next core sync. On source-damaged discs
(observed on the Bourne UHDs) the bytes where the XLL extension belongs
are neither a core sync nor an extension sync -- pure garbage -- which
desyncs ffmpeg's XLL decoder and cascades into 'Read past end of XLL
band data' / 'DSYNC check failed' across the whole track.

next_core_boundary now distinguishes three boundary states via a new
ext_clean flag on NextCore::Found:
  - precise/recognized extension sync  -> ext_clean=true  (keep full AU)
  - garbage at the boundary byte       -> ext_clean=false (drop the ext)

When ext_clean is false we emit the DTS core alone (drop the smallest
junk piece, keep the frame and its PTS) and drain past the garbage to
the next core. Recognized-but-unsizeable extensions still ride the
heuristic scan and are kept intact, so lossless tracks are unaffected --
only genuinely corrupt extension bytes are dropped.

Bourne s1.dts: 1606 damaged frames / 691620 (0.232%), 93% isolated
single frames, worst run 3 in a row (~32ms lossy blip).
2026-07-19 09:27:28 -07:00
Matthew Jackson 999898a438 changelog: faststart default + DTS/DTS-HD audio (mp4://) 2026-07-18 22:35:37 -07:00
Matthew Jackson b4470ccd81 mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box
(sample rate, channel layout mask, core size, computed bitrate, LFE);
whole access units (core + DTS-HD extension substreams) pass through, so
an HD decoder finds the extension. dtsh when an extension sync follows the
core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR.

Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is
byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1)
and decodes clean (exit 0). Channel layout correct.
2026-07-18 22:18:35 -07:00
Matthew Jackson 668dd95237 mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free
box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample ×
est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes
mdat, sample offsets are fixed from the start — no rewrite, no offset
patch. finish() writes moov into the hole and pads the slack with a free
box; if the estimate is blown it falls back to moov-at-end. Streams over
HTTP without a pre-fetch. Reserve-math + box-order tests added.
2026-07-18 21:43:44 -07:00
Matthew Jackson 777d9a04a7 changelog: mp4:// input demuxer (1.5.0) 2026-07-18 21:22:41 -07:00
Matthew Jackson b717044ae5 mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel
info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/
composition timing, stss → sync), rebuild a DiscTitle, and emit samples
as PesFrames in global decode order. Video NALs are length-prefixed in
MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into
input() so mp4:// flows to every sink (mkv://, audio://, json://, …).
In-memory write→read round-trip test proves symmetry (streams, hvcC,
sample sizes survive). Progressive MP4 only; fragmented (moof) is future.
2026-07-18 21:10:15 -07:00
Matthew Jackson 379767071c mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps
the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow /
NFS staging avoids the dirty-page burst. The mdat backpatch is an
ordinary seek WritebackFile already handles (seek_then_patch_roundtrip).
Tests switched to in-memory Cursor writers.
2026-07-18 20:57:44 -07:00
Matthew Jackson 6a06443dd8 changelog: mp4:// native MP4 muxer (1.5.0) 2026-07-18 20:44:18 -07:00
Matthew Jackson 42df399156 mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio
track. Audio sample entries built from the first frame's bitstream —
AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for
fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio
durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle
(mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3;
TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the
CLI can report exclusions — never a silent drop.

Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3
both frame-exact on every track (video 2650/4270, audio 5567/3454),
duration and colour identical, clean decode. On a pathological 4-clip
title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more
of them) — source-inherent, not a muxer defect.
2026-07-18 20:37:10 -07:00
Matthew Jackson 9ede4b9afd mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into
a 64-bit mdat and building the sample tables in memory, patched at
finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs
(already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts,
CFR-derived decode timeline (pipeline carries presentation PTS only), and
a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the
hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping.

Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and
HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact
(bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land
in M2.
2026-07-18 20:16:19 -07:00
Matthew Jackson 67e3e39499 mux: add video:// sink (video-only per-track elementary streams)
Completes the per-track-class trio with audio:// / sub://. video:// is
the demux path with a TrackKind::Video kind filter — each video track to
its own native elementary stream (.hevc/.h264/.vc1/.m2v/.obu), no audio
or subtitles. Reuses the existing kind_filter machinery and extension
map; scheme/parse/output/write-only-guard arms added symmetrically.
2026-07-18 19:24:59 -07:00
Matthew Jackson 42be09f628 changelog: 1.5.0 — extraction sinks + complete json:// model 2026-07-18 19:13:30 -07:00
Matthew Jackson 8bbbe43724 mux: json:// emits the complete title model (no dropped fields)
The json:// sink previously emitted only codec + language + pid per
stream. Fill it out to the full DiscTitle: video carries resolution
(+ pixel dims, interlaced), frame rate (+ fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channels (+ count),
sample rate (+ Hz), and editorial purpose; subtitles carry the
qualifier. Add the clip list and chapter names. This is everything the
scan resolved, so json:// is a lossless machine-readable view of a
title rather than a summary.
2026-07-18 18:14:11 -07:00
Matthew Jackson 95888f27a4 test: pin audio://sub://chapters://json:// in the scheme round-trip 2026-07-18 17:46:08 -07:00
Matthew Jackson 96dec65040 mux: chapters:// and json:// metadata sinks
Two write-only file sinks that ignore the PES stream and emit the title
metadata at construction (fvi-style, wired through output()):
- chapters:// — chapter markers in the format the extension picks: .xml
  (Matroska, default), .txt/.ogm (OGM simple), .vtt (WebVTT). Reuses the
  existing chapters_xml/chapters_ogm writers; adds a WebVTT writer.
- json:// — one title's model (playlist, duration, size, format, streams,
  chapters) as pretty JSON via serde_json.

New mux/meta_sink.rs; StreamUrl gains Chapters/Json; codec_label +
chapters_xml/ogm promoted to pub(crate) for reuse.

Tests: chapters_format_selected_by_extension, title_json_carries_streams_and_chapters.
2026-07-18 17:45:39 -07:00
Matthew Jackson 378fb226b7 mux: audio:// and sub:// sinks (demux filtered by track class)
Two extraction sinks built on the existing demux machinery: audio:// keeps
only audio tracks (native containers .thd/.dts/.ac3/.eac3/.pcm...), sub://
only subtitle tracks (PGS .sup, VobSub .idx+.sub, text .srt) — one file per
track, no video, no chapters sidecar. DemuxOptions gains a kind_filter; the
DemuxSink writers (already complete) are reused verbatim. StreamUrl gains
Audio/Sub variants; output() builds the filtered DemuxSink.

Test: kind_filter_keeps_only_the_selected_class.
2026-07-18 17:24:28 -07:00
Matthew Jackson 42caa2d90e restore freemkv-unlock path dep for local dev (post-v1.4.5) 2026-07-18 16:08:03 -07:00
Matthew Jackson 41a2493429 v1.4.5: bump version (freemkv-unlock git-pinned for the tag) 2026-07-18 16:08:01 -07:00
Matthew Jackson 0e73650440 changelog: 1.4.5 (FMTS clean single-variant mux + key-Debug redaction + hex fix) 2026-07-18 13:30:36 -07:00
Matthew Jackson bcfd5eb213 mux: FMTS read-plan on the inline live-drive path too (DiscStream::with_key_map)
The read_plan fix landed on the file-backed highway (build_iso_pipeline)
but the inline DiscStream (live single-pass) still passed the alternate
device-group half to the demux — a known FMTS bug on rip_mode="single".

DiscStream::with_key_map installs the proactive map and rewrites the
extent walk to the read plan, so the live single-pass path reads ONLY
our-phase units, exactly like the highway. DecryptingSectorSource gains
set_key_map for the already-constructed decorator. Non-forensic maps
return the extents unchanged, so every non-FMTS disc is byte-identical.

Test: with_key_map_reads_only_our_phase_units — a forensic Even segment
drops exactly its alternate units from the extent walk. Precommit green
on 1.86.
2026-07-18 13:27:48 -07:00
Matthew Jackson 8ce0a66bee mux: FMTS reads only our-phase units (AacsKeyMap::read_plan)
An FMTS forensic segment interleaves our device group's variant with a
foreign group's at the aligned-unit level. The mux was decrypting only
our phase but leaving the alternate (foreign) units in the buffer as
ciphertext, trusting the demux to 'drop untouched ciphertext cleanly'.
It doesn't: random 0x47 bytes at the 192-byte stride hit tracked PIDs,
mis-parse, and trip the demux's concealed-gap keyframe-resync — which
drops GOOD frames of ours around every segment (349 resyncs / ~6391
packets on Stand by Me, visible as playback flaws).

The map already knows which unit each LBA is and, for a forensic
segment, which phase is ours. AacsKeyMap::read_plan turns that into the
title's read plan: every default/CPS unit, plus inside a segment ONLY
our-phase units. The alternate units are never fetched, decrypted, or
handed to the demux — the demux sees one gapless our-variant stream.

- read_plan is general (single-CPS, multi-CPS, FMTS): a map with no
  Even/Odd range returns the extents unchanged, so DVD/CSS, single-CPS
  UHD and multi-CPS Blu-ray read byte-for-byte as before.
- Wired into build_iso_pipeline (the file-backed highway that muxes
  resumed ISOs). Producer re-anchors unit_base per extent, so per-unit
  segment reads stay unit-aligned and decrypt correctly.
- Extent gains PartialEq/Eq for the read_plan tests.

Tests: read_plan non-forensic unchanged; forensic omits exactly the
alternate units, kept units match the decrypt gate unit-for-unit. All
2314 lib tests pass; precommit green on 1.86.

Pending: end-to-end ISO re-mux validation (concealed gaps 349 -> ~0).
2026-07-18 12:06:20 -07:00
Matthew Jackson 7d4d9a3e84 remove user-facing English from the library (KeyOrigin::name, hddvd Title)
Library holds ZERO user-facing English (CLAUDE.md). Removed KeyOrigin::name()'s
English prose — apps map the typed enum (freemkv gets key_origin_label); diag
logs the enum's Debug repr. hddvd unnamed-title fallback 'Title N' -> neutral
TITLE_N identifier (UDF volume-label style).
2026-07-17 21:50:09 -07:00
Matthew Jackson 4dec71a867 narrow leaked-internal pub surface to pub(crate)
Verified zero callers across all consumer crates AND libfreemkv integration tests:
MUX_APP, Disc::{aacs_disc_hash,encrypted_content_ranges,inject_unit_keys},
locate_ranges, mapfile::{MapEntry,entries}, diag::dump_mkv_track,
DiscStream::{errors,lost_bytes} (read via accessors). Removed the unused mux
DemuxSink/FviSink crate-root re-exports (constructed internally by output() via
the direct module path). Staged demux option variants marked allow(dead_code).
2026-07-17 21:45:33 -07:00
Matthew Jackson a9b29c7060 disc: collapse decrypt_keys_for_title twin into one method
Removed the test-only unchecked wrapper and renamed decrypt_keys_for_title_checked
-> decrypt_keys_for_title (one method returning (keys, title_is_clear)). Production
already used the checked form; the _checked suffix had no counterpart. Callers +
docs updated; two tests take .0.
2026-07-17 21:41:10 -07:00
Matthew Jackson 0cb525517b css: remove redundant crack_key_halt wrapper
crack_key_halt had no caller except the crack_key wrapper — a needless middle
layer. crack_key now calls crack_key_scan directly; crack_key (Option) and
crack_key_outcome (full CrackOutcome + halt) remain as the two real entry points.
2026-07-17 21:34:37 -07:00
Matthew Jackson 5b4a3f32dc mux: collapse FviSink::create/create_with_source into one create
create_with_source (full SourceInfo) was only ever called by the create wrapper
with a partial SourceInfo — the extra capability was unused (YAGNI). Inlined the
constructor body into create and deleted the _with_source variant; callers
unchanged.
2026-07-17 21:32:39 -07:00
Matthew Jackson 66b0bc385c mux: collapse MkvStream::create/create_at into one create(.., Option<&Path>)
One method per action (CLAUDE.md), not a foo_with_X pair — create_at was the real
constructor and create was a None wrapper. Callers pass Option<&Path> directly.
2026-07-17 21:31:29 -07:00
Matthew Jackson aaccd83750 hex: canonical hex->integer parsers + public strip_hex_prefix
Adds parse_hex_u16/u32/u8 and exposes strip_hex_prefix so callers stop hand-rolling
from_str_radix(trim_start_matches("0x")) — a case-sensitive strip that this module
exists to prevent. disc::aacs_disc_hash now uses strip_hex_prefix.
2026-07-17 21:25:59 -07:00
Matthew Jackson 40f5b01a12 disc: redact Debug for AacsState/Key/HandshakeResult (test-guarded)
AacsState (public via Disc.aacs) and Key (the decrypt_with key-transport enum)
are crate-root re-exported and carried VUK/unit/read-data keys + volume id on
#[derive(Debug)]; HandshakeResult carried the VID + AACS 2.0 bus key. Manual
Debug impls print shape only, guarded by red->green tests.
2026-07-17 21:10:51 -07:00
Matthew Jackson 82d42b1930 aacs: redact Debug for ResolvedChain/ResolvedKeys/ProcessingKeyMatch (test-guarded)
These carry raw unit-key / VUK / processing-key bytes on their Debug; manual impls
print shape only (unit_keys_len, redacted markers). Each has a red→green test.
2026-07-17 21:03:31 -07:00
Matthew Jackson 913b01b79f css: redact CssState Debug (test-guarded)
CssState is reachable via the public Disc.css field; #[derive(Debug)] leaked the
raw CSS title key on any {:?} of a Disc. Manual Debug prints crack_span only.
2026-07-17 21:00:34 -07:00
Matthew Jackson af7ef59f3e aacs/types: redact Debug for all key-bearing types (test-guarded)
DeviceKey/HostCert/Vid/MediaKey/Vuk/ProcessingKey/UnitKey/DiscEntry carried key
material on #[derive(Debug)] — a stray {:?} would leak device/host-private/media/
volume/unit keys. Manual Debug impls print shape only. redaction_tests asserts no
key bytes appear and a redaction marker is present (fails if a derive returns).
2026-07-17 20:56:38 -07:00
Matthew Jackson ef8e11b6f3 libfreemkv: phase-aware FMTS decode + two-operation KeyFetch/KeySource seam
FMTS (AACS 2.1) now decodes per (LBA, phase): Phase enum + AacsKeyMap::
from_ranges_phased, decrypt only the variant's parity half. resolve_fmts_key_map
does a 2-phase index-1 anchor then per-index phase probe, and sizes the forensic
set to whatever the source returns (no hardcoded 32). KeyFetch is now two explicit
operations (unit_keys / fmts_indexes) and KeySource splits get_uk into
get_unit_keys + get_fmts_indexes. BYPASS_FMTS_KEY gate removed (first-class format).

Teed up for 1.4.5. Local WIP baseline.
2026-07-17 20:51:15 -07:00
Matthew Jackson e1f897fb32 restore freemkv-unlock path dep for local dev (post-v1.4.4) 2026-07-16 21:44:08 -07:00
Matthew Jackson 602e44fde9 v1.4.4: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:44:05 -07:00
Matthew Jackson 622008650c keysource: add DecodeSampleSet — a >=MIN_SAMPLE_UNITS-by-construction sample set so an online request can't be built under-sized 2026-07-16 21:43:02 -07:00
Matthew Jackson c508484451 restore freemkv-unlock path dep for local dev (post-v1.4.3) 2026-07-16 21:06:04 -07:00
Matthew Jackson cf16f66e07 v1.4.3: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:06:01 -07:00
Matthew Jackson adcab75409 aacs: sample only index-1 forensic segments for the online key query; hoist MIN_SAMPLE_UNITS into the base crate 2026-07-16 21:01:16 -07:00
Matthew Jackson 61e4095727 FMTS: resolve the index key map from one forensic keyserver query
The keyserver protocol now returns all 32 index keys as an array for a
forensic content sample (and a single-element array for plain content).
resolve_fmts_key_map sends one forensic batch and maps array element i to
segment index i+1, replacing the per-index blind-probe collection loop
that repeatedly hit the key service. Segment/index parsing and the
aligned-unit content classification are reworked to support this:

- rename variant_select -> index_select (per-index, not per-variant)
- content classification moves to is_clean(buf, ContentFormat) so the
  unit selector emits only units the key service accepts
- segment.rs: parse IndividualSegment.tbl index tags + SPN ranges,
  build contiguous LBA key ranges from the resolved 32-key array
- decrypt/decorator plumbing for the resolved per-index keys

Fail loud (FmtsKeyMissing) when the forensic query returns < 32 keys or
any segment index stays unresolved.
2026-07-16 19:41:44 -07:00
Matthew Jackson 540f84c7d0 restore freemkv-unlock path dep for local dev (post-v1.4.2) 2026-07-15 19:37:54 -07:00
Matthew Jackson 82c4e018d5 v1.4.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-15 19:37:51 -07:00
Matthew Jackson 08e962694f Mux decrypt/verify redesign, HD DVD first-class, MVC 3D
decrypt:
- decrypt_sectors is now a pure decrypt (apply key, leave plaintext, report
  unverified bytes); TS-structure is a separate primitive (is_clean_ts/ps) used
  only for key selection and read-verify. The mux passes decrypted bytes through
  (the demuxer drops non-conforming packets), ending the NULL-TS conceal loop and
  the per-unit key-server refetch storm. Key-proof floor replaces the 75%
  supermajority.

recovery:
- Removed the post-read decrypt-verify gate (verify.rs) that mis-aligned the
  disc-absolute unit grid against clip-anchored AACS units and false-failed good
  clips (e.g. Dunkirk's orphan-CPS clip). Bad sectors are marked by physical read
  result; decryptability is proven at scan + mux time.

HD DVD (first-class AACS):
- Role-based candidate-list file sourcing so an HD DVD's /ANY!/ files
  (MKBROM.AACS, VTKF000.AACS, CONTENT_CERT.AACS) are found with no disc-type
  branch. parse_vtkf parses VTKF000.AACS into the same UnitKeyFile as a BD
  Unit_Key_RO.inf, so the shared VUK unwrap applies unchanged. set_unit_base
  clip-anchoring. Two decrypt-axis assumptions remain UNVERIFIED-HDDVD-DECRYPT
  (no encrypted disc to test).

mux:
- MVC (Blu-ray 3D) track signals unified into one MVCDecoderConfigurationRecord;
  release-safe track_vint (3-byte VINT) and pid_index (i32) guards.

hardening:
- Container-aware is_clean / encryption detection; bytes_bad_in_title fail-safe
  on a corrupt mapfile; CSS crack gated on DiscFormat::Dvd (HD DVD excluded);
  non-vacuous CSS tests; patch NOT_READY/HARDWARE/ILLEGAL_REQUEST/ABORTED
  sense-path tests.
2026-07-15 19:35:12 -07:00
Matthew Jackson 37e563b8ef Mux: pure decrypt, policy at the caller (no null, no key-server storm)
decrypt_sectors is now a pure decrypt — apply the CPS unit key, leave the
plaintext, report how many bytes did not reach clean TS ("unverified"). It
never restores ciphertext, nulls, or re-fetches. "Did a key produce clean TS?"
is a key-selection / read-verify signal, not the verdict "did we decrypt?": a
correct key can decrypt a bad-encoded region, and broken TS is a muxer concern
(the demuxer drops the packet and resyncs).

Callers own the policy:
- mux (read > decrypt > mux): pass the decrypted bytes to the muxer, whatever
  they are; fail loud only on a genuine can't-decrypt (no key / misaligned).
- sweep/patch (reading from a disc): an unverified unit is a bad read — recover
  a fresh key and retry, or fail loud so disc-recovery re-reads it.

Removes three duplicated decisions — the decrypt-time ciphertext restore, the
mux NULL-TS conceal loop, and the per-unit key-server refetch — plus the dead
aacs_unit_still_ciphertext predicate. Key-fetch recovery now samples the on-disc
ciphertext explicitly (a pure decrypt leaves the buffer plaintext) and lives
only on the rip/verify path, never the mux.

Fixes the 30-90s/region mux stalls and key-server storm on bad-encoded UHD runs
that 1.4.1 left behind (it relaxed the gate but not the surrounding machinery).
2026-07-14 21:04:24 -07:00
Matthew Jackson e659613ab8 restore freemkv-unlock path dep for local dev (post-v1.4.1) 2026-07-14 14:45:33 -07:00
Matthew Jackson eaa7ddc683 v1.4.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-14 14:45:30 -07:00
Matthew Jackson 723c7edf45 Fix mux concealing decryptable video over a single defective packet
AACS content decryption rejected a whole 6144-byte aligned unit unless
EVERY content packet was conformant MPEG-TS. One authored-bad packet (a
pressing/encoding defect or an AACS 2.1 forensic-variant frame) made the
mux conceal the entire unit as NULL TS — destroying up to 31/32 good
packets and tallying them as loss, surfacing as false "corruption" on
otherwise-clean discs (observed across two UHD titles).

decrypt_unit now asks only "did a key OPEN this unit?" — a padding-aware
>=75% supermajority of content packets restoring their 0x47 sync, a gate
no wrong key can reach (uniform-AES noise floor) yet one that tolerates a
minority of authored-bad packets. Opened units pass through VERBATIM; a
non-conforming packet is left for the demuxer to drop on sync-loss and
resync past. TS-sync conformance is a muxer concern, never a decryption
verdict. The post-read verify/sweep gate now shares the same primitive so
it can never disagree with the mux decrypt.

Also unify the MVC (Blu-ray 3D) track signals: the mvcC CodecPrivate
extension, the BlockAdditionMapping, and each per-frame BlockAdditional
all derive from one MVCDecoderConfigurationRecord built once per track, so
a malformed dependent-view parameter set can no longer orphan a BlockAddID.
2026-07-14 14:43:06 -07:00
Matthew Jackson d9881c57c0 restore freemkv-unlock path dep for local dev (post-v1.4.0) 2026-07-13 19:28:55 -07:00
Matthew Jackson 61b4486ae9 v1.4.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-13 19:28:52 -07:00
Matthew Jackson 72d39544ac Changelog: 1.4.0 (Blu-ray 3D / MVC) 2026-07-13 12:47:03 -07:00
Matthew Jackson 1e394eaeb8 3D MVC mux: audit round 2 (converged)
Second audit round converged (severity collapsed 6 HIGH -> 1; the one
HIGH was a bounded 32-element scan, not a defect; the sole spec MEDIUM
was the same false-positive re-raised — 0xBF matches ISO/IEC 14496-15
§7.6.2 verbatim). One genuine robustness fix plus coverage:

- extract_mvc_params: skip a zero-length NAL instead of abandoning the
  scan, so a stray length prefix before the subset SPS/PPS no longer
  silently drops 3D signalling. Test proves params after a zero-length
  NAL are still found.
- Tests: parser_for_mvc_dependent routes H.264 to a passthrough parser;
  passthrough with an IDR does not re-assert param sets (the keyframe &&
  !mvc branch).
- Document the per-playlist (not per-clip) is_3d latching as a known
  limitation (real main-feature playlists are uniformly 3D).
2026-07-13 11:39:07 -07:00
Matthew Jackson 4d1aa6bbf7 Harden 3D MVC mux: robustness + tests (audit round 1)
Triage of a 10-lens code audit of the 3D branch. Fixes for real defects;
rejected three spec false-positives that matched the ISO/IEC 14496-15
§7.6.2 record verbatim.

Robustness / correctness:
- Never panic when a title's only video is the MVC dependent view: the
  base is now the first NON-dependent video, so a dependent-only title
  sets up no merge (muxed as an ordinary track) instead of hitting an
  `expect` on the skipped track slot.
- Drop a per-frame BlockAdditional (BlockAddID=2) when the track declared
  no mvcC mapping (dependent params not captured before the header) — a
  plain block keeps the file conforming instead of an orphaned add.
- A non-keyframe MVC base frame always carries a ReferenceBlock (fall back
  to a 0 offset in the pre-first-keyframe corner) so it is never mistaken
  for a seek point.
- Reference the last keyframe on the PRIMARY video track only, so a
  secondary video track's keyframe can't become a cross-track reference.
- dep_by_pts overflow: bound BEFORE inserting so the just-arrived
  dependent survives the drift-clear; count a displaced duplicate-PTS
  dependent as an orphan instead of losing it silently.

API / docs:
- Fold write_frame_with_additional into write_frame(..., Option<&[u8]>)
  per the "no foo_with_X" convention.
- Fix mvc_params doc (StereoMode is intentionally not emitted); remove a
  stale PAT/PMT comment describing an approach that was never taken.

Tests: MVCDecoderConfigurationRecord over-length guards; write_int minimal
two's-complement widths; BlockGroup/BlockAdditions/BlockAdditional +
ReferenceBlock emission; additional dropped without a mapping; h264 MVC
passthrough keeps param sets in-band; extract_mvc_params no-panic on
truncated/empty input; pairing window + dep-overflow edges; no-panic on a
dependent-only title.
2026-07-13 11:24:23 -07:00
Matthew Jackson 9ce03b4d1d Mux Blu-ray 3D (MVC) as a single MVC video track
Fold the MVC dependent (right-eye) view into the base H.264 track as a
per-frame BlockAdditional under an mvcC BlockAdditionMapping, so a 3D
title produces one MVC video track instead of two independent H.264
tracks.

- h264: MVC-passthrough parser mode keeps the dependent view's subset
  SPS/PPS in-band, so each emitted frame is a self-contained dependent
  access unit for a BlockAdditional
- resolve: route the dependent stream through the passthrough parser
- mkvstream: detect the dependent view, pair it to the base frame by
  PTS (bounded FIFO), attach it as a BlockAdditional (BlockAddID=2),
  and skip building its own track; build the mvcC
  MVCDecoderConfigurationRecord from the captured subset SPS/PPS and set
  it on the base track at activation
- mkv: emit the mvcC BlockAdditionMapping and BlockGroup/BlockAdditions,
  with a ReferenceBlock on non-keyframe base frames
- ebml: add BlockAdditions/BlockMore/BlockAdditional/BlockAddID/
  BlockAddIDValue/ReferenceBlock elements and a signed-int writer

Verified against a Blu-ray 3D ISO: ffprobe shows a single MVC track,
the mvcC mapping is present, ~144k BlockAdditionals carry the dependent
view (8.7 GB), and the base view decodes cleanly with no regression.
MVCDecoderConfigurationRecord follows ISO/IEC 14496-15 7.6.2; StereoMode
is intentionally omitted (no enum value describes MVC-in-BlockAdditional;
the mvcC mapping is the primary 3D signal per RFC 9559).
2026-07-13 11:00:04 -07:00
Matthew Jackson f099a4a31a feat(3d): read the SSIF and enumerate the MVC dependent view
Blu-ray 3D as a mux-path variant:
- Detect 3D (a clip has STREAM/SSIF/<clip>.ssif — note the SSIF/ subdir, which
  the old .ssif fallback path got wrong) and use the SSIF extents for the mux:
  one transport stream carrying both the base (left) and MVC dependent (right)
  eyes on distinct PIDs, so muxing it captures the full 3D.
- Add the dependent-view video stream. The base STN table lists only the left
  eye; the on-disc PAT/PMT are AACS-encrypted (unreadable pre-key) and the base
  STN omits the dependent view (it lives in the MPLS STN_table_SS), so use the
  BD-3D PID convention: dependent = base video PID + 1 (0x1011 -> 0x1012),
  coding 0x20 -> H.264. The demux routes its packets from the SSIF.

info now reports two video streams for a 3D title; the mux writes both eyes.
2026-07-12 23:41:47 -07:00
Matthew Jackson e606b5e3a2 feat(3d): enumerate MVC dependent view (stream_type 0x20 -> H.264 video)
from_coding_type maps 0x20 (MVC dependent view) to Codec::H264, so the existing
PAT/PMT scan surfaces the SSIF right-eye substream as a second video stream on
its own PID instead of dropping it as Unknown. No new parser: the dependent eye
rides the same demux path as any other TS video stream. Removes the throwaway
3D-structure probes (findings captured in prior commit messages).
2026-07-12 22:16:13 -07:00
Matthew Jackson de809189ec feat(3d): add MVC dependent-view coding type (0x20)
First step of Blu-ray 3D as a mux-path format variant: name stream_type 0x20
(H.264/MVC dependent view, the SSIF right-eye substream). Recognition in the
scan/demux + dependent-stream enumeration follow.
2026-07-12 22:11:01 -07:00
Matthew Jackson 253b0924e3 wip(3d): prove SSIF de-interleave = ssif−base, dependent decrypts on same UK
Empirical (300, clip 00042): base .m2ts (1567 extents, 25.85 GB) is entirely
inside the SSIF LBA span; dependent = SSIF − base = 1567 ranges, 9.59 GB
(exact vs 35.43−25.85). 200/200 sampled dependent-view units decrypt under the
base UK. => de-interleave needs no SSIF parsing (complement of the base extents
we already resolve), and the dependent eye needs no extra key.
2026-07-12 20:40:58 -07:00
Matthew Jackson 98a7892c34 wip(3d): read-only 3D-structure probe (SSIF/m2ts + MPLS ExtensionData)
Dev tool for the 3D build: uses the library UDF reader (7z cannot parse these
ISOs) to dump base .m2ts vs SSIF sizes and the MPLS ExtensionData entries
(STN_table_SS + SS sub-path). Not product code.
2026-07-12 20:38:37 -07:00
Matthew Jackson b82d21756a fix(udf): follow allocation-descriptor continuation past the AED header
read_icb_extents parsed a continuation block's allocation descriptors from
offset 0, but a continuation block begins with a 24-byte Allocation Extent
Descriptor (ECMA-167 4/14.5): 16-byte descriptor tag + prev-loc(4) +
length_of_allocation_descriptors(4); the real ADs start at offset 24. Reading
from 0 mis-parsed the AED tag as a descriptor -> one garbage extent, then an
unknown-type break, silently truncating any file whose extents spill into a
continuation block.

Few-extent files store their ADs inline and never hit this path, so it stayed
hidden; a heavily fragmented file (~1600 fragments) truncated to 113 extents +
a bogus non-unit-aligned fragment that then aborted the mux
(ExtentNotUnitAligned). Skip the 24-byte AED header, read l_ad bytes of
descriptors from offset 24. Adds a regression test with a non-zero AED tag.
2026-07-12 20:37:55 -07:00
Matthew Jackson 02c584a3d9 restore freemkv-unlock path dep for local dev (post-v1.3.2) 2026-07-10 14:26:11 -07:00
Matthew Jackson 564e66fd24 v1.3.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 14:26:08 -07:00
Matthew Jackson 57d40e81a5 1.3.2: AACS 2.1 FMTS variant-decode foundation
Add UnitKey.variant_number (0 = ordinary, 1..32 = forensic variant) with new/variant constructors, and aacs::variant_select — resolve a disc's single variant and classify each aligned unit (default / variant / drop foreign / conceal keyless). Correct IndividualSegment.tbl: the per-record field is the variant (cycles 1..32 on a retail disc), not a segment number — Segment.number -> Segment.variant.
2026-07-10 14:00:28 -07:00
Matthew Jackson 674819d937 changelog: trim the 1.3.1 relicense note to just the relicense fact 2026-07-10 12:53:17 -07:00
Matthew Jackson c5e8421f9f restore freemkv-unlock path dep for local dev (post-v1.3.1) 2026-07-10 12:35:12 -07:00
Matthew Jackson 7dcf9b5a0d v1.3.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 12:35:09 -07:00
Matthew Jackson f01c097784 1.3.1: relicense to MIT (clean-room CSS + drop copyleft-lib references)
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL). The CSS
content cipher and Stevenson title-key attack are attributed to their published
cryptanalysis (not libdvdcss); all libaacs/libbluray/libdvdread/libdvdnav name
references were dropped from comments while keeping the standard format/spec
descriptions. Also bumps to 1.3.1.
2026-07-10 12:31:19 -07:00
Matthew Jackson c265fa349e hddvd: authoritative title composition from the Advanced-Content playlist
Parse ADV_OBJ/VPLST000.XPL (the real HD-DVD player playlist) with roxmltree
into one DiscTitle per <Title>: its PrimaryAudioVideoClip clips in order (EVO
via the .MAP sidecar), the titleDuration, the displayName, and the ChapterList.
A layer-break split (FEATURE_1+FEATURE_2, feature/feature_Divide) composes into
ONE title with the two parts as clips, each carrying its title-time in/out
points (45kHz ticks) for seamless-join splicing. Falls back to the clip-name
heuristic when no playlist is present.

Fixes unknown runtimes (real durations), poor names (FEATURE -> 'Main Movie'),
and gives authoritative composition + chapters. Validated on ANCHORMAN (97m),
SHAUN (99m), HARRY_POTTER (152m).
2026-07-10 10:28:18 -07:00
Matthew Jackson 6a59dcb7ba ci: drop the dead crates.io publish job (libfreemkv is git-tag-only)
libfreemkv is publish=false (it git-deps the firmware crate freemkv-unlock,
never on crates.io), so the Release workflow's 'cargo publish' step failed
hard on every tag. Consumers git-tag-pin libfreemkv; the git tag is the
release artifact. Removed the publish job.
2026-07-10 08:52:57 -07:00
Matthew Jackson 0eb1817ec7 restore freemkv-unlock path dep for local dev (post-v1.3.0) 2026-07-10 08:43:47 -07:00
84 changed files with 14119 additions and 4334 deletions
+5 -23
View File
@@ -39,29 +39,11 @@ jobs:
# would always fail (no lockfile to lock against on a fresh runner).
- run: cargo test
# crates.io publish is an INDEPENDENT job: it serves EXTERNAL consumers only.
# The freemkv binaries no longer depend on it (they git-tag-pin libfreemkv via
# a committed [patch.crates-io]), so this publish runs in parallel with their
# release builds rather than gating them. It `needs: [verify, test]` so a
# failing test suite still blocks publication to crates.io — external
# consumers who `cargo add libfreemkv` must never receive a release whose
# tests were failing. (The two upstream jobs run in parallel, so this gate
# does not serialize publish behind test beyond their own completion.)
publish:
needs: [verify, test]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: Swatinem/rust-cache@v2
# --no-verify: CI already compiled this exact commit (in the `test` job
# and on every push via ci.yml). cargo publish's default re-verify does a
# full cold release build of the packaged tarball, which here is pure
# redundant work (~a cold lib build). Skip it.
- name: Publish to crates.io
run: cargo publish --no-verify
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
# (`package.publish = false` — it git-deps the firmware crate freemkv-unlock,
# which never ships to crates.io). Every consumer git-tag-pins libfreemkv via
# a committed [patch.crates-io]; the git tag itself IS the release artifact.
# A `cargo publish` here fails hard on `publish = false`, so it was removed.
release:
# Only needs `verify`; the GitHub Release can be cut as soon as the version
+316 -5
View File
@@ -1,5 +1,316 @@
# Changelog
## [1.5.2] — UNRELEASED
### Fixed
- CSS DVDs no longer mux to garbage. Every DVD read path — the file-backed mux
highway (`build_iso_pipeline`) and the live-drive single-pass `DiscStream` —
now resolves the per-VTS title key at read time through one shared step
(`resolve_dvd_title_key`), cracked keylessly in playback order from the title's
own extents. An uncrackable title hard-fails (E7023) instead of passing
scrambled sectors through as plaintext; `--raw` skips the crack entirely; a
user Stop mid-crack surfaces as `Halted`.
### Changed
- DVD scan no longer cracks a title key up front (the key is per-VTS, so a single
disc key was meaningless). Scan does only the CSS bus-auth read-unlock — hoisted
before the UDF prefetch so scrambled small/menu VOBs no longer cost a rejected
read each. Cuts a CSS-DVD scan from ~25s to ~6s.
- Unlocker report: the DVD entry is renamed `CSS` → `DVD`.
## [1.5.1] — 2026-07-20
### Fixed
- **TrueHD audio is no longer silently dropped (and no longer sends decoders out
of memory).** The previous release added an MLP major-sync checksum gate to drop
genuinely-undecodable audio frames, but the checksum was computed with mismatched
byte order — the 16-bit CRC result was folded in one endianness and compared in
the other — so it never validated a real major sync. The parser then judged every
major sync corrupt and dropped every audio frame from the first one onward,
flushing the whole TrueHD track at the end of the file: de-interleaved from the
video, which sent players and integrity checkers into an unbounded memory spiral
("decoder ran out of memory"). The checksum now matches the reference
implementation byte-exact (cross-verified against real 7.1/Atmos and 5.1 discs),
so major syncs validate and only genuinely-corrupt frames are dropped; as a
safety net, a major sync the parser still can't validate is kept rather than
allowed to drop an entire track. TrueHD titles produced after the checksum gate
landed need a re-rip.
- **HD DVD AACS key files are now found on every disc, not just the common
layout.** The AACS directory and title-key filename on HD DVD are chosen by
the authoring house, and freemkv previously assumed one fixed spelling
(`/ANY!/VTKF000.AACS`). Discs that name their AACS directory differently (e.g.
`AAC!` instead of `ANY!`) or ship numbered title-key files (`VTKF090.AACS` /
`VTKF100.AACS` rather than `VTKF000.AACS`) are now handled: the AACS directory
is located by its contents and every title-key file in it is picked up. Blu-ray
and UHD are unaffected.
- **HD DVD multi-title decryption reads the right keys.** The HD DVD title-key
file (`VTKF*.AACS`) stores its keys in 36-byte records — per the AACS HD DVD
specification, and confirmed byte-exact on real discs. freemkv had been reading
them at a 32-byte stride, which lands the first key correctly but drifts off
every key after it, so only single-title discs decrypted. Discs with more than
one protected title now recover every title's key instead of only the first.
(Choosing the correct title-key file when a disc carries several playlists
still needs verification against an encrypted HD DVD.)
- **A dirty disc can no longer "rip clean" but decode with errors.** freemkv now
asks the drive to *report* marginal reads instead of silently returning
best-effort data as success — on smudged/scratched media a drive can hand back
subtly-wrong bytes with a clean status, which used to slip through the rip and
surface only as playback/decode errors. A read the drive had to fight for is
now distrusted and re-read in the patch pass: a clean re-read wins, and a spot
that's genuinely unreadable becomes an honest gap rather than silently-wrong
data. Best-effort per drive, and it changes nothing on a clean disc.
## [1.5.0] — 2026-07-19
### Added
- **MP4 as a source (`mp4://`)** — read a progressive `.mp4` back in and send it to
any sink (`mp4:// mkv://`, `mp4:// audio://`, `mp4:// json://`, …). The round-trip
is frame-exact.
- **Native MP4 output (`mp4://`)** — a disc goes straight to a play-everywhere
`.mp4` in one decrypt pass, no ffmpeg. Carries HEVC / H.264 video (with HDR10) and
AC-3, E-AC-3, and DTS / DTS-HD audio, and is faststart by default so it plays over
HTTP without downloading the end first. It's a **compatibility export, not
archival**: MP4 can't hold TrueHD, LPCM, or bitmap (PGS / VobSub) subtitles, so
those are **excluded with a loud, itemized report — never a silent drop**
(`mkv://` stays the keep-everything path).
- **Five extraction sinks — dissect a title, don't just rip it.** New destinations
that pull one part of a title out on its own:
- **`video://dir/`** — each video track to its own native elementary-stream file.
- **`audio://dir/`** — each audio track to its own file in its native container
(`.thd`, `.dts`, `.ac3`, `.eac3`, `.aac`, `.flac`; LPCM as `.pcm`).
- **`sub://dir/`** — each subtitle track to its own file (PGS `.sup`, VobSub
`.idx` + `.sub`, text `.srt`).
- **`chapters://file`** — a title's chapter markers as a sidecar (`.xml` / `.txt`
/ `.ogm` / `.vtt`).
- **`json://file`** — a title's complete structure as JSON.
`chapters://` and `json://` read nothing of the elementary streams, so they
return in seconds.
- **Damaged audio frames are dropped instead of shipped as glitches.** When a
source disc has a corrupt audio frame, freemkv drops that frame rather than muxing
it as a decoder-choking glitch — keeping A/V in sync (a drop is a silence gap,
never a shift) and logging every drop. Works across DTS, AC-3 / E-AC-3, FLAC,
MP2 / MP3, AAC, and TrueHD, each using the format's own integrity check; a track
that is mostly undecodable is dropped whole. This catches structurally-broken
frames — corruption of the audio *data* inside an otherwise-valid frame is source
damage that can't be told from good data without decoding.
- **Forced subtitles detected from the stream.** A PGS subtitle track is flagged
forced when its content is entirely forced/narrative subtitles, read from the
stream itself rather than the disc's metadata — so it works on discs that carry
none. `info -v` reports the same, so `info` and a rip agree.
### Changed
- **`json://` emits the complete title model** — every field the scan resolved:
video resolution / frame rate / HDR / colour, audio channel layout / sample rate /
language / purpose, the subtitle forced flag, plus the clip list and chapter names.
### Fixed
- **TrueHD: a couple of transient errors no longer discard a whole track.** A
corrupt access unit is dropped forward to the next clean sync point, but a short
burst of damage no longer trips the whole-track drop, and a corrupt sync point can
no longer shift the audio that follows.
- **Free-format MP2 / MP3** is a legal, decodable mode and is no longer dropped.
## [1.4.5] — 2026-07-18
### Fixed
- **FMTS (AACS 2.1) forensic discs now mux to a clean, single-variant stream.** A
forensic segment interleaves the local device group's variant with a foreign
group's at the aligned-unit level. The mux decrypted only our half but left the
foreign half in the buffer as ciphertext, on the assumption that the demuxer
"drops untouched ciphertext cleanly." It does not — a foreign unit's bytes hit a
tracked PID at the 192-byte stride, mis-parse, and trip the demux's concealed-gap
keyframe resync, which discards good frames of ours around every segment (visible
playback glitches). `AacsKeyMap::read_plan` now turns the map into the title's
read plan: every default / CPS unit, plus inside a forensic segment **only our
phase's units**. The foreign half is never read, decrypted, or handed to the
demux. On a retail 4K UHD title this took concealed-gap resyncs from **349 → 0**
and recovered ~2 GB of previously-dropped frames. Wired into **both** mux paths —
the file-backed highway (`build_iso_pipeline`) and the inline live-drive
`DiscStream` (`with_key_map`) — so single- and multi-pass FMTS rips are both clean.
### Changed
- **Key-bearing types redact their `Debug` output.** Every type that carries key
material (device keys, processing keys, unit keys, media keys, VUKs, resolved
chains, CSS/AACS state, …) now prints a `<redacted>` marker instead of the bytes,
so no key can reach a log or panic message. Each is covered by a test asserting no
key byte appears.
- **Hex parsing is centralized and case-insensitive.** A single set of canonical
`0x`/`0X`-tolerant hex→integer parsers replaces scattered ad-hoc parsing (this is
what silently dropped keydb device keys written with an uppercase `0X` prefix).
- **Internal-only public surface narrowed to `pub(crate)`, and duplicate
`foo_with_X` methods collapsed to one** — no behavioral change, smaller API.
## [1.4.4] — 2026-07-17
### Fixed
- **Online key requests are no longer silently dropped on discs that yield few
sample units.** The online key source refuses any request carrying fewer than
`MIN_SAMPLE_UNITS` (8) encrypted-content samples — too few can match an
incidental unit rather than the one asked about (a false positive, most acute on
AACS 2.1 forensic-variant content). autorip gathered only 4, so every online
lookup was skipped before it ever reached the key service and surfaced to the
user as "key service down." autorip's sample count is now tied to
`MIN_SAMPLE_UNITS` with a **compile-time floor**, so it can never regress below
the minimum again.
### Changed
- **The online request is assembled from a proven-sufficient sample set.** New
`DecodeSampleSet` (`libfreemkv::keysource`) wraps the content-unit samples and
can only be constructed with at least `MIN_SAMPLE_UNITS` of them — so an online
key request cannot be built from too few samples. The minimum is validated once,
at construction, rather than by a runtime check a caller could forget.
## [1.4.3] — 2026-07-17
### Changed
- **`MIN_SAMPLE_UNITS` moved to the base crate.** The minimum sample count an
online key request must carry now has a single definition in
`libfreemkv::keysource`; `freemkv-keysources` re-exports it, so the online source
and libfreemkv's own forensic query size their requests from one shared value.
- **The online unit-key reply is parsed as a list.** A response carries either a
single Unit Key (ordinary disc) or the full ordered set (an AACS 2.1
forensic-variant disc); the client accepts both and maps array position to
forensic index.
### Added
- **Forensic-variant online query samples the anchor segment.** On an AACS 2.1
forensic-variant disc the online key query draws its sample from the first
forensic segment (index 1) — one canonical, deterministic sample — instead of an
arbitrary segment.
## [1.4.2] — 2026-07-15
### Fixed
- **Mux no longer nulls decryptable video or storms the key server on a
bad-encoded region.** 1.4.1 relaxed the decrypt gate but left the surrounding
machinery in place. On a unit whose key *decrypted* but whose plaintext didn't
reassemble to clean MPEG-TS, the read path still restored ciphertext, tallied
loss, and re-asked the online key server (forever returning the same correct key)
while the mux concealed the unit as NULL TS. The root cause: *"did a key produce
clean TS?"* was used as the verdict *"did we decrypt?"* — they are not the same.
A correct key can decrypt content with broken encoding; broken TS is a muxer
concern, never a decrypt verdict.
### Changed
- **One decrypt authority; policy at the caller.** `decrypt_sectors` is now a
pure decrypt: applies the CPS unit key in place, leaves plaintext, and reports
unverified bytes. It never restores ciphertext, nulls, or re-fetches a key.
Clean-TS status is only a key-*selection* hint (multi-CPS) or a read-*verify*
signal (sweep/patch). Callers own the policy: the mux passes decrypted bytes
through unconditionally (the demuxer handles bad TS); sweep/patch treat an
unverified unit as a failed read and re-read it. Removes the decrypt-time
ciphertext restore, the mux NULL-TS conceal loop, and the per-unit key-server
refetch, plus the dead `aacs_unit_still_ciphertext` predicate.
- **Decrypt and TS-structure are now separate primitives.** AACS has no MAC;
the only "did it decrypt?" signal is whether plaintext looks like MPEG-TS —
a data-quality / key-selection question, not a decrypt verdict. The old
`decrypt_unit(...) -> bool` is split into `decrypt_unit_raw` (pure crypto) and
`is_clean_ts` (structural check), composed explicitly only where needed. The
mux calls only `decrypt_unit_raw`.
- **Key-proof floor replaces the 75% supermajority.** The old proportion
(≥75% of content packets synced) conflated *the key worked* with *the content
is well-encoded*. `is_clean_ts` now requires `synced >= min(E, 4)` on
**encrypted** packets (skipping packet 0 whose `0x47` is in the clear seed):
four synced packets ≈ 1-in-4-billion false-positive; `min(E, 4)` scales to
short fragment tails so they're never false-rejected. A unit is "opened" when
a handful of packets prove the key — bad-encoded packets are the muxer's job.
## [1.4.1] — 2026-07-14
### Fixed
- **Mux no longer discards good video over a single defective packet.** AACS
decryption required **every** content packet to be conformant MPEG-TS: one
authored-bad packet (encoding defect, AACS 2.1 forensic-variant frame) made
the mux conceal the **whole** 6144-byte aligned unit as NULL TS (up to 31/32
good packets discarded, tallied as loss). On affected discs this produced
false "corruption" over otherwise-perfect video (~466 MB concealed across two
UHD titles). The gate is now a padding-aware **≥75% supermajority** of content
packets restoring their `0x47` sync — no wrong key reaches this threshold
(uniform-AES noise floor ≈ 256⁻ⁿ), but a minority of authored-bad packets
still passes. Opened units flow through verbatim; the demuxer drops
non-conforming packets on sync-loss. TS-sync conformance is a muxer concern,
never a decrypt verdict. (The supermajority threshold is tightened in 1.4.2.)
- **MVC (Blu-ray 3D) track signals unified and hardened.** The `mvcC`
`CodecPrivate` extension, the `BlockAdditionMapping`, and each frame's
`BlockAdditional` now all derive from a single `MVCDecoderConfigurationRecord`
built once per track, so they can no longer diverge. A track is flagged 3D
only when that record actually builds — a malformed dependent-view parameter
set no longer emits a mapping with no matching record (previously the flag was
taken from `mvc_params.is_some()`, which could orphan a `BlockAddID`). The base
track's `CodecPrivate` now carries the `mvcC` extension block
(`avcC ‖ u32be(size) ‖ "mvcC" ‖ record`, Matroska-spec size = block − 4) so
players and mediainfo detect MVC at the track level.
## [1.4.0] — 2026-07-13
### Added
- **Blu-ray 3D (MVC) support.** A 3D disc now rips to an MKV that preserves
**both eyes** as a single MVC video track — the AVC base (left) view in each
Block, and the MVC dependent (right-eye) view as a per-frame `BlockAdditional`
under an `mvcC` `BlockAdditionMapping` (`MVCDecoderConfigurationRecord` per
ISO/IEC 14496-15 §7.6.2), paired to the base by PTS. Remux only — no
transcode, no side-by-side conversion. The Blu-ray scan reads the interleaved
`STREAM/SSIF/<clip>.ssif`, enumerates the dependent view (stream_type `0x20`)
by the BD-3D PID convention, and parses it in a parameter-set-passthrough mode
so every dependent frame is a self-contained access unit. Verified on
*300: Rise of an Empire*: one MVC track, ~8.7 GB dependent payload carried in
per-frame BlockAdditionals, base view byte-identical to the 2D rip.
## [1.3.2] — 2026-07-10
### Added
- **AACS 2.1 (FMTS) variant-decode foundation.** `UnitKey` gains a
`variant_number` field (`0` = ordinary content, `1..=32` = a forensic
variant) with `UnitKey::new` / `UnitKey::variant` constructors, and a new
`aacs::variant_select` module resolves a disc's single forensic variant and
classifies each aligned unit — decrypt with the default key, decrypt with the
variant key, drop a foreign variant, or conceal a keyless forensic unit. This
is the groundwork for selecting one variant's segments and dropping the other
31; the decrypt-pipeline wiring lands with the variant key source.
### Fixed
- **`IndividualSegment.tbl`: the per-record field is the variant, not a segment
number.** `Segment.number` → `Segment.variant`. Verified against a retail 2.1
disc, the field cycles `1..=32` across the table (a per-variant tag) rather
than counting up, so variant selection routes on the correct value.
## [1.3.1] — 2026-07-10
### Licensing
- **Relicensed to the MIT License, from 1.3.1 onwards** (releases up to and
including 1.3.0 remain under AGPL-3.0).
### Added
- **Authoritative HD-DVD title composition** from the Advanced-Content playlist
(`ADV_OBJ/VPLST000.XPL`): each title's clips, real duration, display name, and
chapters come from the disc's own playlist instead of a clip-name heuristic. A
layer-break split (`FEATURE_1`+`FEATURE_2`, `feature`/`feature_Divide`) composes
into ONE title with the two parts as clips and their title-time offsets. Falls
back to the clip-name heuristic when no playlist is present.
## [1.3.0] — 2026-07-08
### Added
@@ -122,7 +433,7 @@
- **Processing-Key resolution is ~15× faster on UHD.** A Processing Key is the
key at its subset-difference node (one AES-G from the Media Key), so it is now
tried directly against the MKB cvalue tables (matching libaacs `_calc_mk_pks`)
tried directly against the MKB cvalue tables (direct PK × cvalue iteration)
instead of BFS-walking the SD tree at unknown depth — which was both wrong for
terminal PKs and slow on a large UHD MKB (~181k cvalues). PK derivation on UHD
drops from ~37 s to ~2.4 s; the SD tree walk now lives solely in the device-key
@@ -355,7 +666,7 @@ consumers are the in-tree toolchain crates.
- **Post-read decrypt-verify gate.** Every AACS unit read off the disc is now
buffered, re-aligned to its clip-file 6144-byte unit grid, and verified
(CPI flag → decrypt → strict all-32 TS-sync, matching libaacs `_verify_ts`)
(CPI flag → decrypt → strict all-32 TS-sync)
before it is signed off as good. A unit that no held or freshly-fetched key
decrypts is treated exactly like a bad read — re-read by
the patch pass, terminal loss only if truly unrecoverable — closing the
@@ -376,7 +687,7 @@ consumers are the in-tree toolchain crates.
- **AACS decrypt acceptance is now standards-strict.** A key is accepted only
when the decrypted unit has the TS sync byte on *all* 32 source packets
(libaacs `_verify_ts`), replacing a majority-vote heuristic where a wrong key
(all-32 TS-sync verify), replacing a majority-vote heuristic where a wrong key
could coincidentally restore enough syncs to pass and silently corrupt a unit.
- keydb download/save moved out of the library into freemkv-keysources;
libfreemkv no longer has any keydb I/O (it already held no keys).
@@ -384,7 +695,7 @@ consumers are the in-tree toolchain crates.
### Fixed
- **AACS content-certificate bus-encryption flag read from the wrong bit.** The
flag is bit 7 of byte 1 (libaacs `p[1] >> 7`) but was read as bit 0, so a
flag is bit 7 of byte 1 (`p[1] >> 7`) but was read as bit 0, so a
bus-encrypted disc parsed as *not* bus-encrypted — defeating the fail-loud
guard that refuses to decrypt bus-wrapped data to garbage when no bus key was
obtained. Also corrected the cc_id offset (byte 14) and the AACS2 type marker
@@ -684,7 +995,7 @@ hardening.
- **Keyless DVD/CSS title-key recovery.** A CSS-protected DVD decrypts with no
key database — the title key is recovered directly from the scrambled disc
data via the Stevenson known-plaintext attack (ported from libdvdcss) and
data via the Stevenson known-plaintext attack and
validated by descrambling a sector and confirming the known plaintext
reappears, so a wrong key fails cleanly instead of producing silent garbage
(`src/css/stevenson.rs`). `Disc::scan_image` recovers the same title key from
+1 -1
View File
@@ -24,4 +24,4 @@ cargo test
## License
By contributing, you agree your code will be licensed under AGPL-3.0.
By contributing, you agree your code will be licensed under MIT.
+9 -3
View File
@@ -1,9 +1,9 @@
[package]
name = "libfreemkv"
version = "1.3.0"
version = "1.5.2"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
keywords = ["bluray", "uhd", "optical", "scsi", "disc"]
@@ -28,7 +28,7 @@ cbc = "0.1"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.3.0" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.5.2" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
@@ -36,6 +36,12 @@ rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
+17 -12
View File
@@ -1,16 +1,21 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
MIT License
Copyright (C) 2026 FreeMKV Contributors
Copyright (c) 2026 Matthew Jackson & Contributors
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, version 3 of the License.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+2 -2
View File
@@ -1,4 +1,4 @@
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
# libfreemkv
@@ -190,4 +190,4 @@ Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/fre
## License
AGPL-3.0-only
MIT
+1 -1
View File
@@ -7,7 +7,7 @@ AACS decryption requires an external `keydb.cfg` (default
material is compiled in; DVD CSS player keys are the only compiled-in keys.
**Repository:** <https://github.com/freemkv/libfreemkv>
**License:** AGPL-3.0-only
**License:** MIT
---
+603 -726
View File
File diff suppressed because it is too large Load Diff
+45 -10
View File
@@ -10,8 +10,8 @@ use super::types::*;
///
/// A Processing Key is **terminal**: it is the key at its Subset-Difference
/// node, one `AES-G` from the Media Key. So this is the fast path — each PK is
/// tried *directly* against the MKB cvalue tables (no tree descent), matching
/// libaacs `_calc_mk_pks` (iterate PKs × cvalues). On a large AACS 2.x UHD MKB
/// tried *directly* against the MKB cvalue tables (no tree descent) — the
/// direct PK × cvalue iteration. On a large AACS 2.x UHD MKB
/// (~181k cvalues) this is ~15x faster than treating a PK as a device-node
/// label and walking the tree.
///
@@ -463,9 +463,9 @@ pub enum KeyCandidate {
/// PURE DERIVATION — no unit sampling, no validation. `unit_keys` holds every
/// CPS-unit key the disc's `Unit_Key_RO.inf` yields from the VUK (paired with
/// its declared CPS-unit number); the caller runs
/// [`super::content::unit_key_validates`] to find which one actually opens the
/// `decrypt_unit` + `is_clean_ts` to find which one actually opens the
/// disc. Rungs above the candidate are `None`.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct ResolvedChain {
pub unit_keys: Vec<(u32, [u8; 16])>,
pub vuk: Option<Vuk>,
@@ -475,6 +475,21 @@ pub struct ResolvedChain {
pub dk: Option<DeviceKey>,
}
// Redacting `Debug`: `unit_keys` holds raw title-key bytes, never printed. The
// other rungs are `types` newtypes that self-redact. Guarded by
// `resolved_chain_debug_is_redacted`.
impl std::fmt::Debug for ResolvedChain {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ResolvedChain")
.field("unit_keys_len", &self.unit_keys.len())
.field("vuk", &self.vuk)
.field("mk", &self.mk)
.field("pk", &self.pk)
.field("dk", &self.dk)
.finish()
}
}
/// Derive the full AACS key chain from a candidate key of ANY ladder rung.
///
/// Runs the deterministic derivation DOWNWARD to the disc's terminal unit keys:
@@ -487,7 +502,7 @@ pub struct ResolvedChain {
///
/// PURE DERIVATION: no sampling, no validation, no position recovery. Validate
/// `unit_keys` against a real encrypted unit with
/// [`super::content::unit_key_validates`] to prove the candidate opens the disc.
/// `decrypt_unit` + `is_clean_ts` to prove the candidate opens the disc.
///
/// Returns `None` only when derivation itself cannot proceed: a PK its MKB
/// rejects, a `Dk` the MKB can't process, a missing VID on a path that needs
@@ -505,7 +520,8 @@ pub fn resolve_candidate(
let version = mkb_type(mkb)
.map(|t| t.generation())
.unwrap_or(AacsVersion::V10);
let ukf = parse_unit_key_ro(unit_key_ro, version)?;
// BD/UHD Unit_Key_RO.inf or HD DVD VTKF000.AACS — dispatched by magic.
let ukf = parse_title_keys(unit_key_ro, version)?;
if ukf.encrypted_keys.is_empty() {
return None;
}
@@ -567,6 +583,28 @@ mod resolve_candidate_tests {
use super::*;
use crate::aacs::crypto::aes_ecb_encrypt;
/// `ResolvedChain.unit_keys` holds raw title-key bytes (the other rungs are
/// self-redacting `types` newtypes). `Debug` must not leak the title keys.
#[test]
fn resolved_chain_debug_is_redacted() {
let c = ResolvedChain {
unit_keys: vec![(1, [0xD5; 16])],
vuk: None,
mk: None,
pk: None,
dk: None,
};
let dbg = format!("{c:?}");
assert!(
!dbg.contains("213"),
"ResolvedChain leaked unit keys: {dbg}"
);
assert!(
dbg.contains("unit_keys_len"),
"ResolvedChain missing redaction: {dbg}"
);
}
/// Minimal AACS-1.0 (48-byte stride) `Unit_Key_RO.inf` with `n` encrypted
/// unit keys — `parse_unit_key_ro` numbers CPS units 1..=n.
fn synth_inf(encs: &[[u8; 16]]) -> Vec<u8> {
@@ -612,10 +650,7 @@ mod resolve_candidate_tests {
/// A bare UK candidate is terminal — it returns itself keyed by its own idx.
#[test]
fn resolve_candidate_uk_is_itself() {
let uk = UnitKey {
idx: 2,
key: [0x9u8; 16],
};
let uk = UnitKey::new(2, [0x9u8; 16]);
let r = resolve_candidate(&KeyCandidate::Uk(uk), &[], &[], None).expect("uk is terminal");
assert_eq!(r.unit_keys, vec![(2, uk.key)]);
assert!(r.vuk.is_none() && r.mk.is_none());
+178
View File
@@ -0,0 +1,178 @@
//! FMTS index selection — the pure decode-time decision for a 2.1 disc.
//!
//! A 2.1 disc resolves to exactly one forensic index (1..=32) for a given
//! rip. `IndividualSegment.tbl` tags each forensic segment with an index (see
//! [`super::segment`]); the decode keeps the segments matching our index,
//! drops the other 31, and treats everything outside a segment as ordinary
//! (index-0) content. This module owns that classification and nothing else —
//! no I/O, no keys, no cipher — so it is fully testable in isolation. The
//! decrypt pipeline consumes the [`UnitDisposition`] it returns.
//!
//! Where the resolved index comes from is a separate concern
//! ([`resolve_disc_index`]): today it is read off the index keys the key
//! source handed us; when Processing Keys are available it will come from the
//! VK derivation instead. Either way the disposition logic below is identical.
use super::segment::{Segment, segment_for_unit};
use super::types::UnitKey;
/// What the decode should do with one AACS aligned unit.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UnitDisposition {
/// Outside every forensic segment: ordinary content, decrypt with the
/// default (index-0) unit key.
Default,
/// Inside a forensic segment tagged with OUR resolved index: decrypt with
/// that index's key.
Index(u8),
/// Inside a forensic segment tagged with a DIFFERENT index: not our
/// watermark, so it is not part of our output — drop it.
DropForeignIndex(u8),
/// Inside a forensic segment but no index key is held (the disc's index
/// was never resolved): the segment cannot be decoded, so it is concealed
/// as loss. Carries the segment's index for diagnostics.
ForensicNoKey(u8),
}
/// Resolve the disc's single forensic index from the keys we hold.
///
/// Scans for an index key (`index_number` in `1..=32`) and returns its
/// index. `None` when only default (index-0) keys are held — i.e. no
/// index source answered, so forensic segments are not decodable. A disc has
/// exactly one index, so the first non-zero key decides; if several distinct
/// index keys were somehow supplied the lowest wins (deterministic), which is
/// only a defensive tiebreak — the probe/derivation yields one.
pub fn resolve_disc_index(unit_keys: &[UnitKey]) -> Option<u8> {
unit_keys
.iter()
.map(|k| k.index_number)
.filter(|&v| v != 0)
.min()
}
/// Classify the AACS aligned unit at `unit_offset` (clip-relative bytes) given
/// the forensic segment map and the disc's resolved index (`None` if no
/// index key is held).
pub fn unit_disposition(
unit_offset: u64,
segments: &[Segment],
disc_index: Option<u8>,
) -> UnitDisposition {
match segment_for_unit(segments, unit_offset) {
// Not in any forensic segment → ordinary content.
None => UnitDisposition::Default,
// In a forensic segment → decide by whether it is our index.
Some(seg) => {
let seg_index = seg.index as u8;
match disc_index {
Some(v) if v == seg_index => UnitDisposition::Index(v),
Some(_) => UnitDisposition::DropForeignIndex(seg_index),
None => UnitDisposition::ForensicNoKey(seg_index),
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::aacs::content::ALIGNED_UNIT_LEN;
use crate::aacs::segment::{SOURCE_PACKET_LEN, parse_individual_segments};
/// Build a one-record segment table (index, start_spn, end_spn).
fn tbl(recs: &[(u16, u32, u32)]) -> Vec<Segment> {
let mut v = Vec::new();
v.extend_from_slice(&0x0100_0000u32.to_be_bytes());
v.extend_from_slice(&(recs.len() as u16).to_be_bytes());
v.extend_from_slice(&16u16.to_be_bytes());
for &(n, s, e) in recs {
v.extend_from_slice(&0x0100_0000u32.to_be_bytes());
v.extend_from_slice(&n.to_be_bytes());
v.extend_from_slice(&1u16.to_be_bytes());
v.extend_from_slice(&s.to_be_bytes());
v.extend_from_slice(&e.to_be_bytes());
}
parse_individual_segments(&v).expect("parse")
}
fn uk(idx: u32, index: u8) -> UnitKey {
if index == 0 {
UnitKey::new(idx, [0u8; 16])
} else {
UnitKey::forensic(idx, [index; 16], index)
}
}
#[test]
fn resolve_picks_the_single_index_key() {
// Default keys only → no index resolved.
assert_eq!(resolve_disc_index(&[uk(0, 0)]), None);
assert_eq!(resolve_disc_index(&[]), None);
// One index key among defaults → that index.
assert_eq!(resolve_disc_index(&[uk(0, 0), uk(1, 7)]), Some(7));
// Defensive: lowest of several distinct indexes (deterministic).
assert_eq!(resolve_disc_index(&[uk(0, 9), uk(1, 3)]), Some(3));
}
#[test]
fn unit_outside_segments_is_default() {
let segs = tbl(&[(1, 343680, 346239)]);
let off = 1000u64 * SOURCE_PACKET_LEN; // well before the segment
assert_eq!(
unit_disposition(off, &segs, Some(1)),
UnitDisposition::Default
);
// With no segments at all (1.0 / 2.0), everything is Default.
assert_eq!(
unit_disposition(off, &[], Some(1)),
UnitDisposition::Default
);
}
#[test]
fn unit_in_our_index_decrypts() {
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(7)),
UnitDisposition::Index(7)
);
}
#[test]
fn unit_in_foreign_index_drops() {
// Segment tagged index 7, but our disc index is 3 → drop it.
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, Some(3)),
UnitDisposition::DropForeignIndex(7)
);
}
#[test]
fn forensic_unit_with_no_key_is_concealed() {
// A forensic segment but we never resolved an index → conceal as loss.
let segs = tbl(&[(7, 100, 200)]);
let off = 120u64 * SOURCE_PACKET_LEN;
assert_eq!(
unit_disposition(off, &segs, None),
UnitDisposition::ForensicNoKey(7)
);
}
#[test]
fn straddling_unit_still_classified_as_its_segment() {
// A unit whose 32-packet span only tails into the segment still routes
// to the segment (matches segment_for_unit's span test).
let segs = tbl(&[(5, 100, 200)]);
let unit_packets = (ALIGNED_UNIT_LEN as u64 / SOURCE_PACKET_LEN) as u32; // 32
// Start so the unit covers [80, 80+31] = [80, 111]: overlaps at 100.
let off = 80u64 * SOURCE_PACKET_LEN;
assert!(80 + unit_packets - 1 >= 100, "sanity: unit tails into seg");
assert_eq!(
unit_disposition(off, &segs, Some(5)),
UnitDisposition::Index(5)
);
}
}
+219 -4
View File
@@ -121,7 +121,7 @@ pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFil
return None;
}
// Title → CPS unit mapping. libaacs (unit_key.c) validates each on-disc CPS
// Title → CPS unit mapping (AACS Unit_Key_RO format): each on-disc CPS
// value is in `1..=num_uk` (else zeroes it) and converts the 1-based on-disc
// index to a 0-based key index. We mirror that so the stored value is a safe,
// ready-to-use key index rather than a raw 1-based number.
@@ -161,6 +161,105 @@ pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFil
})
}
/// HD DVD Video Title Key File (`VTKF%%%.AACS`) magic — "DVD_HD_V_TKF".
pub const VTKF_MAGIC: &[u8; 12] = b"DVD_HD_V_TKF";
/// Fixed header length before the first Title Key Entry (AACS HD DVD Book,
/// Table 3-8).
const VTKF_HEADER_LEN: usize = 0x80;
/// Title Key Entry stride (Table 3-8): 1-byte `BIFO` + 3 reserved + 16-byte
/// encrypted title key + 16-byte binding MAC = 36 bytes.
const VTKF_ENTRY_LEN: usize = 0x24;
/// Byte offset of the encrypted title key within an entry (after `BIFO` + 3
/// reserved).
const VTKF_KEY_OFF: usize = 4;
/// Number of Title Key Entry slots in a VTKF (Table 3-8): a fixed 64.
const VTKF_MAX_ENTRIES: usize = 64;
/// `BIFO` bit 7 (`AV_FLG`): set = this slot carries an available title key.
const VTKF_AV_FLG: u8 = 0x80;
/// Parse an HD DVD `VTKF%%%.AACS` into the SAME [`UnitKeyFile`] a BD/UHD
/// `Unit_Key_RO.inf` yields — so the shared AACS crypto (`derive_unit_keys` →
/// `decrypt_unit_key(vuk, …)`) unwraps HD DVD title keys with no change. Only
/// the on-disc CONTAINER differs between BD and HD DVD; the title-key unwrap is
/// the identical AES-128 VUK step (`Kt = AES-128D(Kvu, Kte)`).
///
/// Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
/// 2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
/// Dukes of Hazzard `VTKF000`):
/// ```text
/// [0x00..0x0C] magic "DVD_HD_V_TKF"
/// [0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
/// [0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
/// [0x1C..0x80] reserved
/// [0x80..] 64 entries × 36 bytes:
/// BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
/// BIFO bit 7 (AV_FLG) set = this slot holds a title key
/// (pre-recorded discs fill the binding MAC with 0xFF)
/// [0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
/// ```
/// The slot index (1-based) is the CPS unit number, so an absent slot is
/// SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
/// hand the wrong title key to CPS unit N+1. The title→CPS mapping is
/// playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
/// `title_cps_unit` is left empty here.
///
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
}
// SHA1 of the WHOLE file — the KEYDB lookup key. BackupHDDVD-family key
// databases index an HD DVD disc by SHA1(VTKF000.AACS), the same role the
// BD disc_hash plays for `Unit_Key_RO.inf`.
let hash = disc_hash(data);
let mut encrypted_keys = Vec::new();
for n in 0..VTKF_MAX_ENTRIES {
let pos = VTKF_HEADER_LEN + n * VTKF_ENTRY_LEN;
if pos + VTKF_ENTRY_LEN > data.len() {
break;
}
// AV_FLG clear = empty slot: skip it, but keep the slot index as the CPS
// number (do NOT break — a gap must not renumber the keys that follow).
if data[pos] & VTKF_AV_FLG == 0 {
continue;
}
let mut key = [0u8; 16];
key.copy_from_slice(&data[pos + VTKF_KEY_OFF..pos + VTKF_KEY_OFF + 16]);
encrypted_keys.push((n as u32 + 1, key));
}
if encrypted_keys.is_empty() {
return None;
}
Some(UnitKeyFile {
disc_hash: hash,
app_type: 0, // HD DVD VTKF carries no BD-ROM app_type
num_bdmv_dir: 0, // BD-only concept
use_skb_mkb: false,
version: AacsVersion::V10, // HD DVD is always AACS 1.0
encrypted_keys,
title_cps_unit: Vec::new(),
})
}
/// Parse a disc's title-key file, dispatching on the self-describing magic:
/// an HD DVD `VTKF000.AACS` (`DVD_HD_V_TKF`) → [`parse_vtkf`]; anything else is a
/// BD/UHD `Unit_Key_RO.inf` → [`parse_unit_key_ro`]. Both return the same
/// [`UnitKeyFile`], so every downstream AACS derivation stays container-agnostic
/// — the single seam where BD-vs-HD-DVD key layout is resolved (mirrors the key
/// service, which classifies HD DVD by the very same magic).
pub fn parse_title_keys(data: &[u8], version: AacsVersion) -> Option<UnitKeyFile> {
if data.len() >= 12 && &data[..12] == VTKF_MAGIC {
parse_vtkf(data)
} else {
parse_unit_key_ro(data, version)
}
}
/// MKB disc structure format code.
const MKB_DISC_STRUCTURE_FORMAT: u8 = 0x83;
@@ -261,10 +360,10 @@ pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
return None;
}
// Content Certificate layout (matches libaacs content_cert.c):
// Content Certificate layout (per the AACS content-cert format):
// [0] certificate type (0x00 = AACS1, 0x10 = AACS2)
// [1] bit7 bus_encryption_enabled_flag (libaacs: `p[1] >> 7`)
// [14..20] cc_id (6 bytes) (libaacs: `p + 14`)
// [1] bit7 bus_encryption_enabled_flag (`p[1] >> 7`)
// [14..20] cc_id (6 bytes) (`p + 14`)
let version = if data[0] == 0x00 {
AacsVersion::V10
} else {
@@ -283,3 +382,119 @@ pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
version,
})
}
#[cfg(test)]
mod vtkf_tests {
use super::*;
/// Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
/// HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
/// `VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
/// entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
/// set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
fn synth_vtkf(keys: &[[u8; 16]]) -> Vec<u8> {
const FILE_LEN: usize = 2480;
let mut v = Vec::new();
v.extend_from_slice(VTKF_MAGIC); // 0x00
v.extend_from_slice(&(FILE_LEN as u32).to_be_bytes()); // 0x0C HD_VTKF_SIZE
v.extend_from_slice(b"VPLST000.XPL"); // 0x10 playlist name
v.resize(VTKF_HEADER_LEN, 0); // reserve to first entry (0x80)
for n in 0..VTKF_MAX_ENTRIES {
if let Some(k) = keys.get(n) {
v.push(VTKF_AV_FLG); // BIFO: AV_FLG set (present)
v.extend_from_slice(&[0, 0, 0]); // reserved
v.extend_from_slice(k); // 16-byte encrypted title key
v.extend_from_slice(&[0xFFu8; 16]); // binding MAC (0xFF, pre-recorded)
} else {
v.extend_from_slice(&[0u8; VTKF_ENTRY_LEN]); // empty slot (AV_FLG clear)
}
}
v.resize(FILE_LEN - 16, 0); // reserved gap before the trailer
v.extend_from_slice(&[0xABu8; 16]); // TKF MAC (must NOT be read as a key)
v
}
#[test]
fn parse_vtkf_reads_present_entries_skips_empty_ignores_mac() {
let k1 = [0x11u8; 16];
let k2 = [0x22u8; 16];
let k3 = [0x33u8; 16];
let data = synth_vtkf(&[k1, k2, k3]);
let ukf = parse_vtkf(&data).expect("valid VTKF must parse");
// Exactly the three present entries — the empty slots and the trailing
// 16-byte TKF MAC are NOT mistaken for keys. Critically, k2/k3 are read
// at the 36-byte stride (offsets 0xA4, 0xC8); the old 32-byte stride
// misread them from inside the previous entry's binding MAC.
assert_eq!(ukf.encrypted_keys.len(), 3);
assert_eq!(
ukf.encrypted_keys[0],
(1, k1),
"CPS units = 1-based slot index"
);
assert_eq!(ukf.encrypted_keys[1], (2, k2));
assert_eq!(ukf.encrypted_keys[2], (3, k3));
assert_eq!(ukf.version, AacsVersion::V10, "HD DVD is AACS 1.0");
// disc_hash is SHA1 of the whole file (the KEYDB lookup key).
assert_eq!(ukf.disc_hash, disc_hash(&data));
}
#[test]
fn parse_vtkf_reads_a_full_64_entry_file() {
// Real discs (Freedom, Dukes) carry all 64 slots present. Every key must
// come back, none dropped and none drifted — the regression the 32-byte
// stride failed.
let keys: Vec<[u8; 16]> = (0..VTKF_MAX_ENTRIES).map(|n| [n as u8; 16]).collect();
let ukf = parse_vtkf(&synth_vtkf(&keys)).expect("64-entry VTKF");
assert_eq!(ukf.encrypted_keys.len(), 64);
assert_eq!(
ukf.encrypted_keys[63],
(64, [63u8; 16]),
"entry 64 at 0x{:x}",
VTKF_HEADER_LEN + 63 * VTKF_ENTRY_LEN
);
}
#[test]
fn parse_vtkf_rejects_non_magic() {
let mut data = synth_vtkf(&[[0x11u8; 16]]);
data[0] = b'X'; // corrupt magic
assert!(
parse_vtkf(&data).is_none(),
"non-VTKF magic must be rejected"
);
assert!(
parse_vtkf(&[0u8; 4]).is_none(),
"too short must be rejected"
);
}
#[test]
fn parse_title_keys_dispatches_by_magic() {
// VTKF magic → parse_vtkf.
let data = synth_vtkf(&[[0x44u8; 16], [0x55u8; 16]]);
let ukf = parse_title_keys(&data, AacsVersion::V10).expect("VTKF dispatch");
assert_eq!(ukf.encrypted_keys.len(), 2);
// Non-VTKF → parse_unit_key_ro (a 2-byte buffer is not a valid inf, so
// this proves it ROUTED to the BD parser rather than parse_vtkf).
assert!(
parse_title_keys(&[0x00, 0x00], AacsVersion::V10).is_none(),
"non-magic input must route to parse_unit_key_ro"
);
}
/// The whole point of the seam: a parsed VTKF feeds the SHARED VUK→title-key
/// crypto (`decrypt_unit_key`) exactly like a BD `Unit_Key_RO.inf` would —
/// no HD-DVD-specific crypto path.
#[test]
fn vtkf_encrypted_keys_feed_shared_vuk_unwrap() {
let enc = [0x9Au8; 16];
let data = synth_vtkf(&[enc]);
let ukf = parse_vtkf(&data).unwrap();
let vuk = [0x5Cu8; 16];
let derived = super::super::derive::decrypt_unit_key(&vuk, &ukf.encrypted_keys[0].1);
// Same as applying the shared unwrap directly to the stored enc key.
assert_eq!(derived, super::super::derive::decrypt_unit_key(&vuk, &enc));
}
}
+5 -6
View File
@@ -157,7 +157,7 @@ impl AacsVersion {
}
/// Find Verify Media Key Record (type 0x81 for AACS 1.0, 0x86 for AACS 2.0/2.1) in MKB.
/// 0x81: [C] §3.2.5.1.4. 0x86 (AACS 2.x): [libaacs] `mkb.c` — not in the public spec.
/// 0x81: [C] §3.2.5.1.4. 0x86 (AACS 2.x): [RE] — not in the public spec (from real 2.x MKBs).
pub(crate) fn mkb_find_mk_dv(mkb: &[u8]) -> Option<[u8; 16]> {
// Verify-Media-Key record (0x81 for AACS 1.0, 0x86 for AACS 2.x): mk_dv is
// the 16 bytes at record offset 4 (body offset 0). Needs rec_len >= 20.
@@ -198,9 +198,8 @@ pub(crate) fn mkb_find_subdiff_records(mkb: &[u8]) -> Option<Vec<u8>> {
///
/// The cvalue table is record type `0x05` (Media Key Data) on BOTH AACS
/// 1.0 and AACS 2.x MKBs — its 16-byte cvalue entries are 1:1 with the
/// 5-byte Subset-Difference index entries in record `0x04`. This matches
/// libaacs, whose `mkb_cvalues()` reads `0x05` and `mkb_subdiff_records()`
/// reads `0x04`.
/// 5-byte Subset-Difference index entries in record `0x04` — the standard AACS
/// MKB layout (`0x05` cvalues 1:1 with the `0x04` subset-difference index).
///
/// On AACS 2.x in-drive UHD MKBs the `0x05` table is large (the full
/// subset-difference cvalue set: ~181k entries on a retail MKB, 1:1 with
@@ -281,10 +280,10 @@ pub const MKB_TYPE_4_PRERECORDED: u32 = 0x0004_1003;
/// `0x000A1003` — Class II / Unified MKB (Sequence-Key-Block functionality).
pub const MKB_TYPE_10_CLASS_II: u32 = 0x000A_1003;
/// `0x48141003` — AACS 2.0 Category C (UHD content). libaacs `MKB_20_CATEGORY_C`.
/// `0x48141003` — AACS 2.0 Category C (UHD content) MKB type value.
pub const MKB_20_CATEGORY_C: u32 = 0x4814_1003;
/// `0x48151003` — AACS 2.1 Category C (UHD content). libaacs `MKB_21_CATEGORY_C`.
/// `0x48151003` — AACS 2.1 Category C (UHD content) MKB type value.
pub const MKB_21_CATEGORY_C: u32 = 0x4815_1003;
/// The AACS MKB Type field, decoded.
+251 -13
View File
@@ -22,13 +22,14 @@
//! - `[C]` — AACS Introduction and Common Cryptographic Elements Book (primitives, MKB/key-management).
//! - `[PR]` — AACS Pre-recorded Video Book (Volume/Title Key layer).
//! - `[BD]` — AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit, Block Key).
//! - `[libaacs]` — the libaacs reference implementation, cited only where the spec
//! is silent (the `0x86` verify record and the Category-C MKBType names).
//! - `[RE]` — reverse-engineered from real discs, cited only where the public
//! spec is silent (the `0x86` verify record and the Category-C MKB type values).
pub mod content;
pub mod crypto;
pub mod derive;
pub mod host_certs;
pub mod index_select;
pub mod inf;
pub mod mkb;
pub mod provider;
@@ -39,10 +40,25 @@ pub mod trace;
pub mod types;
pub mod variant;
/// On-disc UDF paths to the AACS key-input files (with their fallbacks).
/// Centralised so every reader (`resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, `read_aacs_version`) walks the exact same files — adding
/// or changing a fallback in one place can then never silently diverge the
/// On-disc UDF paths to the AACS key-input files, plus HD DVD AACS-directory
/// discovery.
///
/// BD and UHD keep their key material under a fixed `/AACS/…` tree, so those
/// paths are constants. HD DVD keeps the equivalents in a reserved root
/// directory whose NAME is authoring-house-specific — observed `ANY!` (Dukes
/// of Hazzard) and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK`
/// mirror — and whose title-key file is NOT always `VTKF000.AACS` (Freedom
/// ships `VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED
/// from the parsed UDF tree ([`find_hddvd_aacs_dir`] + [`role_paths`]), never
/// hardcoded.
///
/// Each key ROLE ([`AacsRole`]) resolves to an ordered candidate list — the
/// BD/UHD constants first, then whatever the HD DVD directory actually holds —
/// which every reader walks with [`read_first`], first-that-reads. No reader
/// ever branches on disc type: a BD/UHD disc has the `/AACS/` files so those
/// win; an HD DVD has none of them, so it falls through to the discovered
/// entries. Centralised so `resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, and `read_aacs_version` can never silently diverge the
/// disc_hash / MKB / VID that another reader feeds a key service.
pub const PATH_UNIT_KEY_RO: &str = "/AACS/Unit_Key_RO.inf";
pub const PATH_UNIT_KEY_RO_DUPLICATE: &str = "/AACS/DUPLICATE/Unit_Key_RO.inf";
@@ -51,6 +67,114 @@ pub const PATH_MKB_RW: &str = "/AACS/MKB_RW.inf";
pub const PATH_CONTENT_CERT: &str = "/AACS/Content000.cer";
pub const PATH_CONTENT_CERT_ALT: &str = "/AACS/Content001.cer";
/// An AACS key-input role. [`role_paths`] maps it to an ordered candidate path
/// list (BD/UHD constants, then the discovered HD DVD files).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AacsRole {
/// Title-key file: BD/UHD `Unit_Key_RO.inf`, HD DVD `VTKF*.AACS`
/// (magic `DVD_HD_V_TKF`). The disc_hash is `SHA1` of this file.
UnitKey,
/// Media Key Block: BD/UHD `MKB_RO/RW.inf`, HD DVD `MKBROM.AACS`.
Mkb,
/// Content certificate: BD/UHD `Content000/001.cer`, HD DVD
/// `CONTENT_CERT.AACS` (byte 0 gives the AACS major).
ContentCert,
}
/// The HD DVD AACS directory in a parsed UDF tree, if present.
///
/// Identified structurally, NOT by a hardcoded name: the root child directory
/// whose name ends in `!` (so the `<name>!_BAK` backup mirror, which also ends
/// in a non-`!` char, is not mistaken for it) and which contains `MKBROM.AACS`.
/// Observed real names: `ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD
/// disc has no such directory → `None`.
pub(crate) fn find_hddvd_aacs_dir(udf: &crate::udf::UdfFs) -> Option<&crate::udf::DirEntry> {
udf.root.entries.iter().find(|e| {
e.is_dir
&& e.name.ends_with('!')
&& e.entries
.iter()
.any(|c| !c.is_dir && c.name.eq_ignore_ascii_case("MKBROM.AACS"))
})
}
/// Ordered candidate paths for an AACS key [`AacsRole`]: the fixed BD/UHD
/// `/AACS/…` paths first, then the actual HD DVD files discovered in the disc's
/// AACS directory (see [`find_hddvd_aacs_dir`]). A disc has only one family, so
/// the other family's entries simply never read.
///
/// For [`AacsRole::UnitKey`] every `VTKF*.AACS` in the directory is appended in
/// sorted name order — a disc may carry more than one variant (Freedom:
/// `VTKF090` + `VTKF100`), not just `VTKF000`.
pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String> {
let mut v: Vec<String> = match role {
AacsRole::UnitKey => vec![PATH_UNIT_KEY_RO, PATH_UNIT_KEY_RO_DUPLICATE],
AacsRole::Mkb => vec![PATH_MKB_RO, PATH_MKB_RW],
AacsRole::ContentCert => vec![PATH_CONTENT_CERT, PATH_CONTENT_CERT_ALT],
}
.into_iter()
.map(String::from)
.collect();
if let Some(dir) = find_hddvd_aacs_dir(udf) {
let d = &dir.name;
match role {
AacsRole::Mkb => v.push(format!("/{d}/MKBROM.AACS")),
AacsRole::ContentCert => v.push(format!("/{d}/CONTENT_CERT.AACS")),
AacsRole::UnitKey => {
// Glob VTKF*.AACS — the title-key filename is not fixed at
// VTKF000 (Freedom ships VTKF090 + VTKF100). Sorted for a
// deterministic try order.
//
// TODO(hddvd-playlist): each VTKF%%%.AACS is bound to ONE
// playlist (VPLST%%%.XPL) — the AACS HD DVD Book gives the
// selector explicitly: match the TKF's 12-byte PLAYLIST_NAME
// field (bytes 0x10..0x1C) to the playlist of the title being
// decrypted; "unless the names are identical, the Title Keys in
// this TKF must not be used." Today read_first just takes the
// first that reads, which is correct only for a single-playlist
// disc. Thread the active playlist name here (owned by the HD
// DVD enumerator) and pick the name-matched VTKF once a
// multi-playlist encrypted disc is available to validate against.
let mut names: Vec<&str> = dir
.entries
.iter()
.filter(|e| !e.is_dir)
.filter(|e| {
let u = e.name.to_ascii_uppercase();
u.starts_with("VTKF") && u.ends_with(".AACS")
})
.map(|e| e.name.as_str())
.collect();
names.sort_unstable();
v.extend(names.into_iter().map(|n| format!("/{d}/{n}")));
}
}
}
v
}
/// Walk an AACS role's candidate paths (from [`role_paths`]) and return the
/// first that reads.
///
/// `read` performs the actual per-path read (full file or bounded prefix), so
/// callers share the same first-present walk regardless of read style. Returns
/// [`Error::AacsNoKeys`] if no candidate is present. Generic over the path
/// element (`&str` or owned `String`) so it accepts the `Vec<String>` that
/// [`role_paths`] builds from the discovered HD DVD directory.
pub(crate) fn read_first<S, F>(candidates: &[S], mut read: F) -> crate::error::Result<Vec<u8>>
where
S: AsRef<str>,
F: FnMut(&str) -> crate::error::Result<Vec<u8>>,
{
for path in candidates {
if let Ok(buf) = read(path.as_ref()) {
return Ok(buf);
}
}
Err(crate::error::Error::AacsNoKeys)
}
// The module structure IS the public API — consumers import from the owning
// module directly (e.g. `aacs::content::decrypt_unit`, `aacs::mkb::MkbType`,
// `aacs::derive::{derive_vuk, resolve_candidate}`, `aacs::resolve::resolve_keys_v2`).
@@ -60,7 +184,7 @@ pub const PATH_CONTENT_CERT_ALT: &str = "/AACS/Content001.cer";
// content-decrypt entry points that downstream key-source crates import through
// the `aacs::` path. These are the stable, load-bearing names; keeping them here
// lets those crates track the module refactor without a lockstep re-pin.
pub use content::{ALIGNED_UNIT_LEN, decrypt_unit_try_keys};
pub use content::ALIGNED_UNIT_LEN;
pub use derive::derive_vuk;
pub use types::{DeviceKey, HostCert, MediaKey, ProcessingKey, UnitKey, Vid, Vuk};
@@ -70,7 +194,7 @@ mod tests {
//! Touching one representative item per module keeps these as a
//! compile-time contract that the module paths stay stable.
use super::content::{ALIGNED_UNIT_LEN, ts_sync_destroyed};
use super::content::ALIGNED_UNIT_LEN;
use super::inf::{disc_hash, disc_hash_hex};
use super::mkb::{AacsVersion, mkb_content_len, walk_mkb};
use super::variant::is_variant_mkb;
@@ -98,18 +222,132 @@ mod tests {
fn public_helpers_are_callable_by_module_path() {
// Touch a representative function from each module so a dropped/renamed
// item fails to compile. Smoke calls, not behavioural assertions.
let _ = ts_sync_destroyed(&[0u8; ALIGNED_UNIT_LEN]);
let _ = !crate::aacs::content::is_clean(
&[0u8; ALIGNED_UNIT_LEN],
crate::disc::ContentFormat::BdTs,
);
let _ = mkb_content_len(&[]);
let _ = is_variant_mkb(&walk_mkb(&[]));
let _ = disc_hash_hex(&disc_hash(b"x"));
let _ = super::derive::resolve_candidate(
&super::derive::KeyCandidate::Uk(super::types::UnitKey {
idx: 0,
key: [0u8; 16],
}),
&super::derive::KeyCandidate::Uk(super::types::UnitKey::new(0, [0u8; 16])),
&[],
&[],
None,
);
}
// ── HD DVD AACS directory / filename discovery ────────────────────────
//
// The HD DVD AACS dir name and title-key filename are authoring-specific
// and were previously hardcoded to `/ANY!/VTKF000.AACS`. These verify the
// discovery replacement against both real-disc shapes: Freedom (`AAC!` +
// `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
#[test]
fn role_paths_discovers_hddvd_dir_and_globs_all_vtkf_variants() {
use crate::udf::fixture::*;
// Freedom-shaped: an `AAC!` dir (NOT `ANY!`) holding MKBROM + two VTKF
// variants (090/100, NOT 000) + a VTUF usage file (must be excluded),
// plus the `AAC!_BAK` mirror (must NOT be picked as the AACS dir).
let mut disc = MemDisc::new();
let aacs_files = vec![
file("MKBROM.AACS", 100, 5000, 4096, true),
file("CONTENT_CERT.AACS", 101, 5100, 2048, true),
file("VTKF100.AACS", 102, 5200, 2048, true),
file("VTKF090.AACS", 103, 5300, 2048, true),
file("VTUF090.AACS", 104, 5400, 2048, true),
];
let bak_files = vec![file("MKBROM.AACS", 110, 6000, 4096, true)];
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![
DirSpec {
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: aacs_files,
subdirs: vec![],
},
DirSpec {
name: "AAC!_BAK".to_string(),
icb_lba: 30,
dir_data_lba: 31,
files: bak_files,
subdirs: vec![],
},
],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
// Discovered structurally (ends in '!', holds MKBROM.AACS) — the real
// AACS dir, never the `_BAK` mirror.
let dir = super::find_hddvd_aacs_dir(&udf).expect("aacs dir");
assert_eq!(dir.name, "AAC!");
// UnitKey: BD/UHD paths first, then EVERY VTKF*.AACS in sorted order
// (090 before 100) — NOT hardcoded VTKF000; VTUF (usage) excluded.
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
"/AAC!/VTKF090.AACS".to_string(),
"/AAC!/VTKF100.AACS".to_string(),
]
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::Mkb)
.last()
.unwrap(),
"/AAC!/MKBROM.AACS"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::ContentCert)
.last()
.unwrap(),
"/AAC!/CONTENT_CERT.AACS"
);
}
#[test]
fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() {
use crate::udf::fixture::*;
// A `/AACS/` tree (BD/UHD) has no '!' directory → discovery finds none
// and the candidate list is exactly the static BD/UHD paths.
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "AACS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("Unit_Key_RO.inf", 100, 5000, 2048, true),
file("MKB_RO.inf", 101, 5100, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(super::find_hddvd_aacs_dir(&udf).is_none());
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
]
);
}
}
+50 -12
View File
@@ -8,12 +8,11 @@ use super::mkb::*;
//
// Canonical form is `<category>1003` (low 16 bits `0x1003` is a fixed marker).
// Types 3/4/10 are from the AACS Common Cryptographic Elements spec (0.953,
// §3.2.5.1.1); the Category-C 2.0/2.1 values match libaacs `mkb.h` constants.
// §3.2.5.1.1); the Category-C 2.0/2.1 values are the standard MKB type constants.
// ── Full VUK resolution chain ───────────────────────────────────────────────
/// Result of resolving a disc's VUK.
#[derive(Debug)]
pub struct ResolvedKeys {
/// Disc hash (SHA1 of Unit_Key_RO.inf)
pub disc_hash: [u8; 20],
@@ -34,6 +33,23 @@ pub struct ResolvedKeys {
pub key_source: u8,
}
// Redacting `Debug`: `vuk` and `unit_keys` are raw key bytes, never printed.
// `disc_hash` is the public per-disc identifier (SHA-1 of the .inf), not secret.
// Guarded by `resolved_keys_debug_is_redacted`.
impl std::fmt::Debug for ResolvedKeys {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ResolvedKeys")
.field("disc_hash", &self.disc_hash)
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.field("title_cps_unit", &self.title_cps_unit)
.field("version", &self.version)
.field("bus_encryption", &self.bus_encryption)
.field("key_source", &self.key_source)
.finish()
}
}
/// Inputs shared by every classical-path resolver. References only —
/// callers retain ownership of all buffers.
pub struct ResolveContext<'a> {
@@ -156,7 +172,7 @@ pub fn resolve_keys_v2(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
/// equivalent of path 2 — there's no host-side PK derivation against a
/// Variant MKB.)
pub fn resolve_keys_v21(ctx: &ResolveContext<'_>) -> Option<ResolvedKeys> {
let uk_file = parse_unit_key_ro(ctx.unit_key_ro, AacsVersion::V20)?;
let uk_file = parse_title_keys(ctx.unit_key_ro, AacsVersion::V20)?;
let hash_hex = disc_hash_hex(&uk_file.disc_hash);
let bus_encryption = ctx
.content_cert
@@ -280,8 +296,9 @@ fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Opt
.map(|cc| cc.bus_encryption)
.unwrap_or(false);
// Parse Unit_Key_RO.inf at the version-appropriate stride.
let uk_file = parse_unit_key_ro(ctx.unit_key_ro, version)?;
// Parse the disc's title-key file (BD/UHD Unit_Key_RO.inf at the
// version-appropriate stride, or HD DVD VTKF000.AACS) → common UnitKeyFile.
let uk_file = parse_title_keys(ctx.unit_key_ro, version)?;
let hash_hex = disc_hash_hex(&uk_file.disc_hash);
let has_vid = *ctx.volume_id != [0u8; 16];
@@ -466,6 +483,27 @@ mod tests {
use super::super::types::*;
use super::*;
/// `ResolvedKeys` carries the disc's VUK and unit keys raw; `Debug` must not
/// leak them. Sentinel 213 (0xD5); non-secret fields are not 213.
#[test]
fn resolved_keys_debug_is_redacted() {
let rk = ResolvedKeys {
disc_hash: [0u8; 20],
vuk: Some([0xD5; 16]),
unit_keys: vec![(1, [0xD5; 16])],
title_cps_unit: vec![0],
version: AacsVersion::V21,
bus_encryption: true,
key_source: 1,
};
let dbg = format!("{rk:?}");
assert!(!dbg.contains("213"), "ResolvedKeys leaked keys: {dbg}");
assert!(
dbg.contains("redacted"),
"ResolvedKeys missing marker: {dbg}"
);
}
/// Audit #5: the `major` / `from_major` mapping is load-bearing for the
/// Unit_Key_RO stride, so pin it as a table. V10 ↔ BD; V20/V21 → UHD; any
/// non-BD major selects the V20/V21 64-byte stride (V10 is the only 48-byte).
@@ -790,7 +828,7 @@ mod tests {
#[test]
fn validate_processing_key_round_trip_with_nonzero_uv() {
// Synthesise a (pk, uv, mk, cvalue, mk_dv) tuple that satisfies the
// libaacs _validate_pk relation, then confirm validate_processing_key
// AACS PK-validation relation, then confirm validate_processing_key
// recovers mk. Catches the bugs that landed pre-fix:
// * uv XOR step was missing → mk wrong whenever uv != 0
// * AES-128E + 12-zero check instead of AES-128D + magic
@@ -844,8 +882,8 @@ mod tests {
// ── MKB cvalue-record selection (issue #259 / #281) ─────────────────
//
// The cvalue (Media Key Data) table is record 0x05; the
// Subset-Difference index is record 0x04. This matches libaacs
// (`mkb_cvalues` → 0x05, `mkb_subdiff_records` → 0x04). Record 0x07
// Subset-Difference index is record 0x04 (the standard AACS MKB layout:
// 0x05 = cvalues, 0x04 = subset-difference index). Record 0x07
// (Explicit Subset-Difference Record) is NOT the cvalue table. On real
// in-drive AACS 2.x UHD MKBs 0x07 is small (~96 entries) while the 0x05
// table is large (181270 entries, 1:1 with 0x04). An earlier
@@ -1264,7 +1302,7 @@ mod tests {
}
#[test]
fn test_content_cert_parse() {
// AACS 1.0 cert, bus encryption OFF. Layout matches libaacs: flag in
// AACS 1.0 cert, bus encryption OFF. Content-cert layout: flag in
// BIT 7 of byte 1, cc_id at bytes 14..20.
let mut data = vec![0u8; 20];
data[0] = 0x00; // AACS 1.0
@@ -1460,7 +1498,7 @@ mod tests {
// [20..22] first_play, [22..24] top_menu, [24..26] num_titles, then
// per-title 2-byte pad + 2-byte CPS unit at 26 + i*4 + 2. Each on-disc
// 1-based CPS number in `1..=num_uk` is validated and converted to a
// 0-based key index (libaacs unit_key.c); an out-of-range number → 0.
// 0-based key index (per the AACS Unit_Key_RO format); an out-of-range number → 0.
let mut data = build_unit_key_ro(4, 64); // num_uk = 4 → CPS 1..=4 valid
data[20..22].copy_from_slice(&1u16.to_be_bytes()); // first_play CPS 1
data[22..24].copy_from_slice(&2u16.to_be_bytes()); // top_menu CPS 2
@@ -1580,7 +1618,7 @@ mod tests {
}
#[test]
fn parse_content_cert_extracts_cc_id_and_nonzero_type_is_v20() {
// libaacs layout: [0]=type, [1] bit7=bus-enc, [14..20]=cc_id. Any
// Content-cert layout: [0]=type, [1] bit7=bus-enc, [14..20]=cc_id. Any
// non-0x00 type → V20.
let mut data = vec![0u8; 20];
data[0] = 0x10; // AACS2 type marker → V20
@@ -1593,7 +1631,7 @@ mod tests {
}
#[test]
fn parse_content_cert_bus_encryption_reads_bit7() {
// bus_encryption = (data[1] >> 7) & 1 (libaacs). Low bits set with bit7
// bus_encryption = (data[1] >> 7) & 1. Low bits set with bit7
// clear → false; bit7 set → true. Pins the bit, not a truthiness of the byte.
let mut data = vec![0u8; 20];
data[1] = 0x7F; // bits 0..6 set, bit 7 clear
+207 -57
View File
@@ -1,55 +1,55 @@
//! AACS 2.1 FMTS forensic segment map — `AACS/IndividualSegment.tbl`.
//!
//! An FMTS main feature interleaves N "variant" segments — the sequence-key /
//! forensic-watermark mechanism. The same frames are authored as several
//! slightly different variants; each variant is encrypted under its own SEGMENT
//! key (from `SegmentKeyNNNNN.tbl`), NOT the CPS Unit Key. A player with the
//! right device keys can decrypt exactly one variant per segment, and which one
//! silently identifies the player (traitor tracing). Decrypting a variant
//! segment with the Unit Key yields garbage — broken HEVC reference frames
//! (empirically: `Could not find ref with POC …` on a plain unit-key rip).
//! An FMTS main feature interleaves short forensic **segments** — the sequence-key
//! / forensic-watermark mechanism. Each segment carries an **index** (1..32): a
//! tag in `IndividualSegment.tbl` that selects which of the 32 forensic **index
//! keys** decrypts that segment's units, in place of the ordinary CPS Unit Key.
//!
//! This table says WHERE the variant segments live so a decoder can decrypt
//! them with segment keys and select one coherent variant instead of muxing
//! Terminology (see the project AACS reference): the **index** here is NOT the
//! AACS 2.1 *Media Key Variant* — that is the 65536-value device selector in the
//! MKB that decides *which set* of index keys a device receives, a layer this
//! module does not deal with. All the index keys belong to one variant, whose
//! number is unknown and irrelevant to the segment map. Decrypting a segment with
//! the Unit Key yields garbage — broken HEVC reference frames (empirically:
//! `Could not find ref with POC …` on a plain unit-key rip).
//!
//! This table says WHERE the segments live and which index each carries, so a
//! decoder can decrypt them with the matching index key instead of muxing
//! unit-key garbage.
//!
//! Format (validated against a retail AACS 2.1 disc):
//! ```text
//! header (8 bytes): u32 type | u16 count | u16 record_size (= 16)
//! record[count] (16 bytes each):
//! u32 marker (= 0x01000000) | u16 segment_number | u16 flag (= 1)
//! u32 marker (= 0x01000000) | u16 index | u16 flag (= 1)
//! u32 start_spn | u32 end_spn (source-packet numbers, inclusive)
//! ```
//! Source-packet numbers are the 192-byte BDAV packet index: byte offset =
//! `spn * 192`. Observed on one disc: 792 segments, each ~2560 packets
//! (~480 KB), spread across the entire 54 GB feature (one roughly every 67 MB).
//! `index` is the 1..32 forensic index tag, NOT a sequential segment id: measured
//! on a retail 2.1 disc (Zombieland) it cycles 1,2,…,32,1,2,… across records in
//! file order — 24 full cycles of 32 plus a final partial cycle of 24 = 792
//! records. Source-packet numbers are the 192-byte BDAV packet index: byte offset
//! = `spn * 192`. Each segment is ~2560 packets (~480 KB) = 80 aligned units,
//! spread across the entire 54 GB feature (one roughly every 67 MB). Inside a
//! segment the 80 units interleave in two stride-2 halves: applying the segment's
//! index key decrypts ~40 of them to clean TS and garbles the other ~40 (a second
//! interleaved half, unidentified), which the demux then drops — leaving one
//! coherent stream. Confirmed by decoding a retail disc with a full set of 32
//! index keys.
/// Fixed size of one `IndividualSegment.tbl` record.
pub const SEGMENT_RECORD_LEN: usize = 16;
/// Bytes per BDAV source packet (188-byte TS + 4-byte arrival-time header).
pub const SOURCE_PACKET_LEN: u64 = 192;
/// Whether a 2.1 (FMTS) disc may rip WITHOUT segment (variant) keys.
///
/// `true` (today): the forensic variant segments are skipped as expected loss
/// and the bulk of the title decodes with the unit key, so a 2.1 disc rips
/// mostly-complete. A unit key (VUK) is still required, exactly as for any AACS
/// disc. `false`: the absence of a segment-key source is a hard, UPFRONT failure
/// ([`Error::FmtsKeyMissing`]) — the same policy as a missing unit key, so a
/// forensic-holed rip is refused rather than produced. No segment-key source
/// exists yet, so `true` is the only value under which a 2.1 disc rips at all;
/// flip to `false` once segment keys can be sourced and a partial rip should be
/// refused. Hardcoded on purpose — not a user setting.
///
/// [`Error::FmtsKeyMissing`]: crate::error::Error::FmtsKeyMissing
pub const BYPASS_FMTS_KEY: bool = true;
/// One forensic variant segment: the inclusive source-packet range it occupies
/// in the FMTS clip.
/// One forensic segment: the inclusive source-packet range it occupies in the
/// FMTS clip.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Segment {
/// 1-based segment number (table order).
pub number: u16,
/// Forensic index tag, 1..=32 (field@4 of the record). Cycles across the
/// table rather than counting up — it selects WHICH of the 32 index keys
/// decrypts this range. (`0` is not used here; the default/non-forensic
/// content carries no segment record at all.)
pub index: u16,
/// First source packet of the segment (inclusive).
pub start_spn: u32,
/// Last source packet of the segment (inclusive).
@@ -103,8 +103,8 @@ pub fn lba_byte_offset(lba: u32) -> u64 {
/// unit's clip-relative byte offset.
///
/// This is the routing decision behind a 2.1 decrypt-miss: a unit that
/// overlaps a forensic segment must be opened with that segment's **variant
/// key** (from `SegmentKeyNNNNN.tbl`), not the CPS Unit Key. Opening it with
/// overlaps a forensic segment must be opened with that segment's **index key**
/// (selected by the segment's `index`), not the CPS Unit Key. Opening it with
/// the Unit Key is exactly what yields the broken-reference-frame garbage a
/// plain unit-key rip produces. A unit outside every segment is ordinary
/// content and a miss on it is a Unit-Key miss, so this returns `None` and the
@@ -112,16 +112,16 @@ pub fn lba_byte_offset(lba: u32) -> u64 {
///
/// The unit is tested as a packet *span* (`[off/192, (off+6144-1)/192]`) so a
/// unit that only partly overlaps a segment edge is still classified as
/// variant; on the observed disc segments are unit-aligned, but the span test
/// forensic; on the observed disc segments are unit-aligned, but the span test
/// does not rely on that.
pub fn variant_segment_for_unit(segments: &[Segment], unit_offset: u64) -> Option<&Segment> {
pub fn segment_for_unit(segments: &[Segment], unit_offset: u64) -> Option<&Segment> {
let unit_len = crate::aacs::content::ALIGNED_UNIT_LEN as u64;
let first = (unit_offset / SOURCE_PACKET_LEN) as u32;
let last = ((unit_offset + unit_len - 1) / SOURCE_PACKET_LEN) as u32;
segments.iter().find(|s| s.overlaps_spn(first, last))
}
/// Parse `IndividualSegment.tbl` into its forensic variant segments, in table
/// Parse `IndividualSegment.tbl` into its forensic segments, in table
/// order. Returns `None` when the header is malformed, the record size is not
/// [`SEGMENT_RECORD_LEN`], or the declared record count overruns the buffer —
/// so a truncated / foreign table degrades to "no segment map" rather than
@@ -141,12 +141,12 @@ pub fn parse_individual_segments(tbl: &[u8]) -> Option<Vec<Segment>> {
let mut segments = Vec::with_capacity(count);
for i in 0..count {
let o = 8 + i * record_size;
// o+4..o+8 = segment_number (u16) + flag (u16); o+8..o+16 = start/end SPN.
let number = u16::from_be_bytes([tbl[o + 4], tbl[o + 5]]);
// o+4..o+8 = index (u16, 1..32) + flag (u16); o+8..o+16 = start/end SPN.
let index = u16::from_be_bytes([tbl[o + 4], tbl[o + 5]]);
let start_spn = u32::from_be_bytes([tbl[o + 8], tbl[o + 9], tbl[o + 10], tbl[o + 11]]);
let end_spn = u32::from_be_bytes([tbl[o + 12], tbl[o + 13], tbl[o + 14], tbl[o + 15]]);
segments.push(Segment {
number,
index,
start_spn,
end_spn,
});
@@ -154,12 +154,74 @@ pub fn parse_individual_segments(tbl: &[u8]) -> Option<Vec<Segment>> {
Some(segments)
}
/// Map a clip-relative byte offset to the absolute LBA that holds it, by walking
/// the title's extents (the `.fmts` clip's sectors in file order). Segment
/// offsets in [`Segment`] are clip-relative source-packet numbers, so this is how
/// a segment's `spn` range becomes disc LBAs. `None` if the offset is past the
/// clip.
pub fn clip_byte_to_lba(extents: &[crate::disc::Extent], clip_byte: u64) -> Option<u32> {
let mut cum = 0u64;
for e in extents {
let len = e.sector_count as u64 * crate::consts::SECTOR_BYTES as u64;
if clip_byte < cum + len {
let sector_in_ext = ((clip_byte - cum) / crate::consts::SECTOR_BYTES as u64) as u32;
return Some(e.start_lba.saturating_add(sector_in_ext));
}
cum += len;
}
None
}
/// Build the `[start_lba, end_lba) → key_idx` ranges for an FMTS forensic key map.
///
/// Each forensic segment's clip-relative source-packet span becomes an absolute
/// LBA range tagged with the key its `index` selects (via `index_to_key_idx`,
/// e.g. `|i| i as usize` when the pool is `[base, idx1, idx2, …]`). Applying that
/// one key across the whole segment decodes the ~40 units of its interleave half
/// to clean TS and garbles the other ~40 (the second interleaved half), which the
/// demux then drops — yielding one coherent stream. Ranges outside every segment
/// are left for the map's default (the ordinary Unit Key). A segment that straddles
/// a UDF extent boundary is emitted as one range per whole-sector slice it covers.
///
/// The result feeds [`AacsKeyMap::from_ranges`](crate::decrypt::AacsKeyMap::from_ranges)
/// with the Unit-Key index as the default — the same structure the CPS map uses,
/// only finer-grained.
pub fn fmts_key_ranges(
segments: &[Segment],
extents: &[crate::disc::Extent],
index_to_key_idx: &dyn Fn(u16) -> usize,
) -> Vec<(u32, u32, usize)> {
let mut ranges = Vec::new();
for s in segments {
let start_byte = s.start_spn as u64 * SOURCE_PACKET_LEN;
let end_byte = (s.end_spn as u64 + 1) * SOURCE_PACKET_LEN; // exclusive
// A segment is unit-aligned and contiguous in clip bytes; map its first
// and last sector to LBAs. Segments are ~480 KB and extents are GB-sized,
// so a segment almost never crosses an extent boundary — but if the two
// ends land in different extents (non-contiguous LBAs), skip rather than
// emit a wrong span; the units there fall to the Unit Key (garble+drop),
// never a mis-decrypt.
let (Some(a), Some(b)) = (
clip_byte_to_lba(extents, start_byte),
clip_byte_to_lba(extents, end_byte - 1),
) else {
continue;
};
if b >= a
&& (b - a) as u64 == (end_byte - 1 - start_byte) / crate::consts::SECTOR_BYTES as u64
{
ranges.push((a, b + 1, index_to_key_idx(s.index)));
}
}
ranges
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a table with the real on-disc layout: 8-byte header + N 16-byte
/// records. `recs` are `(segment_number, start_spn, end_spn)`.
/// records. `recs` are `(index, start_spn, end_spn)`.
fn build_tbl(recs: &[(u16, u32, u32)]) -> Vec<u8> {
let mut v = Vec::new();
v.extend_from_slice(&0x0100_0000u32.to_be_bytes()); // type
@@ -175,9 +237,75 @@ mod tests {
v
}
#[test]
fn fmts_key_ranges_maps_segments_to_lba_by_index() {
use crate::disc::Extent;
// One big clip extent starting at LBA 1000. Clip byte B lives at
// LBA 1000 + B/2048.
let extents = vec![Extent {
start_lba: 1000,
sector_count: 1_000_000,
}];
// Two segments, indexes 5 and 7 (spn ranges as on a real disc).
let segs = vec![
Segment {
index: 5,
start_spn: 100,
end_spn: 199,
},
Segment {
index: 7,
start_spn: 10_000,
end_spn: 10_099,
},
];
// Pool layout [base, idx1, idx2, …] → index N uses key slot N.
let ranges = fmts_key_ranges(&segs, &extents, &|v| v as usize);
assert_eq!(ranges.len(), 2, "one LBA range per segment");
// Segment 0: spn 100..=199 → clip bytes [19200, 38400) → sectors 9..=18
// → LBA 1009..1019, key index 5.
assert_eq!(ranges[0], (1009, 1019, 5));
// Segment 1: spn 10000..=10099 → bytes [1_920_000, 1_939_200) →
// sectors 937..=946 → LBA 1937..1947, key index 7.
assert_eq!(ranges[1], (1937, 1947, 7));
// The ranges drive an AacsKeyMap with the Unit Key (index 0) as default.
let map = crate::decrypt::AacsKeyMap::from_ranges(ranges, 0);
assert_eq!(map.key_idx_for(500), 0, "outside any segment → Unit Key");
assert_eq!(map.key_idx_for(1012), 5, "inside index-5 segment → key 5");
assert_eq!(map.key_idx_for(1940), 7, "inside index-7 segment → key 7");
assert_eq!(
map.key_idx_for(1019),
0,
"segment end is exclusive → Unit Key"
);
}
#[test]
fn clip_byte_to_lba_walks_extents() {
use crate::disc::Extent;
let extents = vec![
Extent {
start_lba: 100,
sector_count: 10,
}, // clip bytes [0, 20480)
Extent {
start_lba: 500,
sector_count: 10,
}, // clip bytes [20480, 40960)
];
assert_eq!(clip_byte_to_lba(&extents, 0), Some(100));
assert_eq!(clip_byte_to_lba(&extents, 2048), Some(101));
assert_eq!(clip_byte_to_lba(&extents, 20480), Some(500)); // second extent
assert_eq!(clip_byte_to_lba(&extents, 22528), Some(501));
assert_eq!(clip_byte_to_lba(&extents, 40960), None); // past the clip
}
#[test]
fn parses_real_disc_layout() {
// First three records observed on a retail 2.1 disc: 2560-packet segments.
// First three records observed on retail 2.1 (Zombieland): the variant
// field counts 1,2,3,… (it wraps at 32 further into the table — see
// `index_field_cycles_one_to_thirty_two`), segments are 2560 packets.
let tbl = build_tbl(&[
(1, 343680, 346239),
(2, 695616, 698175),
@@ -185,7 +313,9 @@ mod tests {
]);
let segs = parse_individual_segments(&tbl).expect("parse");
assert_eq!(segs.len(), 3);
assert_eq!(segs[0].number, 1);
assert_eq!(segs[0].index, 1);
assert_eq!(segs[1].index, 2);
assert_eq!(segs[2].index, 3);
assert_eq!(segs[0].start_spn, 343680);
assert_eq!(segs[0].end_spn, 346239);
assert_eq!(segs[0].packet_count(), 2560);
@@ -224,13 +354,33 @@ mod tests {
}
#[test]
fn unit_inside_segment_routes_to_variant() {
fn unit_inside_segment_routes_to_index() {
// A real first-record segment: packets [343680, 346239].
let segs = parse_individual_segments(&build_tbl(&[(1, 343680, 346239)])).unwrap();
// A unit sitting squarely inside: start at packet 344000 → byte 344000*192.
let off = 344000u64 * SOURCE_PACKET_LEN;
let hit = variant_segment_for_unit(&segs, off).expect("inside the segment");
assert_eq!(hit.number, 1);
let hit = segment_for_unit(&segs, off).expect("inside the segment");
assert_eq!(hit.index, 1);
}
#[test]
fn index_field_cycles_one_to_thirty_two() {
// Reality on Zombieland: field@4 is the index, cycling 1..=32 in file
// order (NOT a sequential segment id). Reproduce one-and-a-bit cycles.
let mut recs = Vec::new();
let mut spn = 1000u32;
for row in 0..2 {
for v in 1..=32u16 {
recs.push((v, spn, spn + 2559));
spn += 50_000; // ~one segment every ~67 MB
}
let _ = row;
}
let segs = parse_individual_segments(&build_tbl(&recs)).unwrap();
assert_eq!(segs.len(), 64);
assert_eq!(segs[31].index, 32); // end of first cycle
assert_eq!(segs[32].index, 1); // wraps, does not become 33
assert!(segs.iter().all(|s| (1..=32).contains(&s.index)));
}
#[test]
@@ -238,29 +388,29 @@ mod tests {
let segs = parse_individual_segments(&build_tbl(&[(1, 343680, 346239)])).unwrap();
// A unit well before the segment is ordinary content → None (unit-key path).
let off = 1000u64 * SOURCE_PACKET_LEN;
assert!(variant_segment_for_unit(&segs, off).is_none());
assert!(segment_for_unit(&segs, off).is_none());
}
#[test]
fn unit_straddling_a_segment_edge_counts_as_variant() {
fn unit_straddling_a_segment_edge_counts_as_forensic() {
// Segment starts at packet 100. A unit that ENDS just inside it (its 32
// packets straddle the boundary) must still route to the variant key,
// because part of its ciphertext is variant-encrypted.
// packets straddle the boundary) must still route to the index key,
// because part of its ciphertext is forensic-encrypted.
let segs = parse_individual_segments(&build_tbl(&[(7, 100, 200)])).unwrap();
// Unit covering packets [80, 111]: overlaps [100,200] at the tail.
let off = 80u64 * SOURCE_PACKET_LEN;
let hit = variant_segment_for_unit(&segs, off).expect("straddles the start edge");
assert_eq!(hit.number, 7);
let hit = segment_for_unit(&segs, off).expect("straddles the start edge");
assert_eq!(hit.index, 7);
// A unit ending exactly at packet 99 (offset s.t. last = 99) does NOT overlap.
let before = 68u64 * SOURCE_PACKET_LEN; // [68, 99]
assert!(variant_segment_for_unit(&segs, before).is_none());
assert!(segment_for_unit(&segs, before).is_none());
}
#[test]
fn no_segments_never_routes_to_variant() {
fn no_segments_never_routes_to_index() {
// The 1.0 / 2.0 case: no forensic map, so every miss is a unit-key miss.
assert!(variant_segment_for_unit(&[], lba_byte_offset(0)).is_none());
assert!(variant_segment_for_unit(&[], lba_byte_offset(9_999_999)).is_none());
assert!(segment_for_unit(&[], lba_byte_offset(0)).is_none());
assert!(segment_for_unit(&[], lba_byte_offset(9_999_999)).is_none());
}
#[test]
+198 -8
View File
@@ -6,7 +6,7 @@
//! owns only the crypto and these value types that flow through it.
/// A device key for MKB subset-difference tree processing.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DeviceKey {
pub key: [u8; 16],
pub node: u16,
@@ -15,7 +15,7 @@ pub struct DeviceKey {
}
/// Host certificate + private key for AACS SCSI authentication.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct HostCert {
/// AACS 1.0: 20 bytes. AACS 2.0: 32 bytes.
pub private_key: [u8; 20],
@@ -28,22 +28,22 @@ pub struct HostCert {
}
/// Volume ID (16 bytes) — read from the disc via the SCSI handshake / OEM path.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vid(pub [u8; 16]);
/// Media Key (Km, 16 bytes) — the MKB-scoped key derived from device keys.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct MediaKey(pub [u8; 16]);
/// Volume Unique Key (VUK / Kvu, 16 bytes) — derived from `MediaKey` + `Vid`,
/// decrypts the per-disc encrypted title keys in `Unit_Key_RO.inf`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vuk(pub [u8; 16]);
/// Processing Key (Kp, 16 bytes) — an MKB Subset-Difference key that yields the
/// Media Key. A leaked/precomputed PK in the keydb, or the intermediate PK a
/// device-key walk derives at its matching SD node.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct ProcessingKey(pub [u8; 16]);
/// One decrypted per-CPS-unit AACS title key.
@@ -53,14 +53,49 @@ pub struct ProcessingKey(pub [u8; 16]);
/// area). The CPS-unit *number* association is a higher-level concern owned by
/// [`super::inf::parse_unit_key_ro`], which pairs each positional key with its
/// declared CPS unit; this primitive only does the AES, so it surfaces position.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct UnitKey {
pub idx: u32,
pub key: [u8; 16],
/// AACS 2.1 (FMTS) forensic **index** tag (see [`crate::aacs::segment`]).
///
/// `0` = ordinary (non-forensic) content — the value for every 1.0 / 2.0
/// key and for the bulk of a 2.1 title. `1..=32` = a forensic index key that
/// decrypts the `IndividualSegment.tbl` segments tagged with that same index.
/// This is the per-segment index (1..32), NOT the AACS 2.1 Media Key Variant
/// (the 65536-value device selector), which is a separate MKB-layer concern.
pub index_number: u8,
}
impl UnitKey {
/// An ordinary (non-forensic) unit key: `index_number == 0`. The value
/// for every AACS 1.0 / 2.0 key and the bulk of a 2.1 title.
pub const fn new(idx: u32, key: [u8; 16]) -> Self {
Self {
idx,
key,
index_number: 0,
}
}
/// A forensic index key: `index_number` in `1..=32`, decrypting the
/// `IndividualSegment.tbl` segments tagged with that index.
pub const fn forensic(idx: u32, key: [u8; 16], index_number: u8) -> Self {
Self {
idx,
key,
index_number,
}
}
/// Whether this key decrypts ordinary (non-forensic) content (index 0).
pub const fn is_default_index(&self) -> bool {
self.index_number == 0
}
}
/// A per-disc entry from the key database.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DiscEntry {
/// Disc hash (20 bytes, hex)
pub disc_hash: String,
@@ -75,3 +110,158 @@ pub struct DiscEntry {
/// Unit keys (title keys) indexed by CPS unit number
pub unit_keys: Vec<(u32, [u8; 16])>,
}
// ── Redacting `Debug` impls ──────────────────────────────────────────────────
//
// Every type above carries AACS secret material (device keys, host PRIVATE keys,
// media/volume/processing/unit keys). `#[derive(Debug)]` would print those bytes
// verbatim, so a stray `debug!("{:?}", …)` or a panic message would leak the
// keys. These hand-written impls print only NON-secret shape (presence, lengths,
// tree coordinates, indices) — never key bytes. `decrypt::DecryptKeys` follows
// the same policy by omitting `Debug` entirely; here we keep `Debug` because
// these are `PartialEq`/`Eq` value types used in `assert_eq!` and nested inside
// other `#[derive(Debug)]` structs, so the trait must exist — just not leak.
// Guarded by `redaction_tests` below.
impl std::fmt::Debug for DeviceKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DeviceKey")
.field("key", &"<redacted>")
.field("node", &self.node)
.field("uv", &self.uv)
.field("u_mask_shift", &self.u_mask_shift)
.finish()
}
}
impl std::fmt::Debug for HostCert {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HostCert")
.field("private_key", &"<redacted>")
.field("certificate_len", &self.certificate.len())
.field("private_key_v2", &self.private_key_v2.map(|_| "<redacted>"))
.field(
"certificate_v2_len",
&self.certificate_v2.as_ref().map(|c| c.len()),
)
.finish()
}
}
impl std::fmt::Debug for Vid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vid(<redacted>)")
}
}
impl std::fmt::Debug for MediaKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("MediaKey(<redacted>)")
}
}
impl std::fmt::Debug for Vuk {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vuk(<redacted>)")
}
}
impl std::fmt::Debug for ProcessingKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("ProcessingKey(<redacted>)")
}
}
impl std::fmt::Debug for UnitKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKey")
.field("idx", &self.idx)
.field("key", &"<redacted>")
.field("index_number", &self.index_number)
.finish()
}
}
impl std::fmt::Debug for DiscEntry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscEntry")
.field("disc_hash", &self.disc_hash)
.field("title", &self.title)
.field("media_key", &self.media_key.map(|_| "<redacted>"))
.field("disc_id", &self.disc_id.map(|_| "<redacted>"))
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.finish()
}
}
#[cfg(test)]
mod redaction_tests {
use super::*;
// Sentinel key byte 0xD5 = decimal 213. A derived `Debug` prints `[u8;N]`
// as decimal, so a leaked key surfaces the substring "213"; the redacting
// impls must not. No non-secret field below is 213, so "213" appearing means
// key bytes leaked. Each type must also carry a "redacted" marker (or omit
// the secret entirely) so re-adding `#[derive(Debug)]` fails this test.
const S: u8 = 0xD5;
fn assert_redacted(what: &str, dbg: &str) {
assert!(
!dbg.contains("213"),
"{what}: Debug leaked key bytes (found decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"{what}: Debug missing redaction marker: {dbg}"
);
}
#[test]
fn device_key_debug_is_redacted() {
let d = DeviceKey {
key: [S; 16],
node: 1,
uv: 2,
u_mask_shift: 3,
};
assert_redacted("DeviceKey", &format!("{d:?}"));
}
#[test]
fn host_cert_debug_is_redacted() {
let h = HostCert {
private_key: [S; 20],
certificate: vec![0u8; 92],
private_key_v2: Some([S; 32]),
certificate_v2: None,
};
assert_redacted("HostCert", &format!("{h:?}"));
}
#[test]
fn newtype_keys_debug_is_redacted() {
assert_redacted("Vid", &format!("{:?}", Vid([S; 16])));
assert_redacted("MediaKey", &format!("{:?}", MediaKey([S; 16])));
assert_redacted("Vuk", &format!("{:?}", Vuk([S; 16])));
assert_redacted("ProcessingKey", &format!("{:?}", ProcessingKey([S; 16])));
}
#[test]
fn unit_key_debug_is_redacted() {
assert_redacted("UnitKey", &format!("{:?}", UnitKey::new(0, [S; 16])));
}
#[test]
fn disc_entry_debug_is_redacted() {
let e = DiscEntry {
disc_hash: "0xAA".into(),
title: "T".into(),
media_key: Some([S; 16]),
disc_id: Some([S; 16]),
vuk: Some([S; 16]),
unit_keys: vec![(1, [S; 16])],
};
assert_redacted("DiscEntry", &format!("{e:?}"));
}
}
+36 -1
View File
@@ -146,7 +146,7 @@ use super::derive::{calc_pk_from_dk, calc_v_mask};
/// Outcome of a subset-difference walk against an MKB. Carries the
/// processing key and the matching `uv` slot — both needed as inputs
/// to the variant chain.
#[derive(Debug, Clone, Copy)]
#[derive(Clone, Copy)]
pub struct ProcessingKeyMatch {
/// Processing Key.
pub kp: [u8; 16],
@@ -158,6 +158,20 @@ pub struct ProcessingKeyMatch {
pub cvalue_index: usize,
}
// Redacting `Debug`: `kp` (a Processing Key) and `cvalue` are secret, never
// printed. `uv` / `cvalue_index` are non-secret coordinates. Guarded by
// `processing_key_match_debug_is_redacted`.
impl std::fmt::Debug for ProcessingKeyMatch {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ProcessingKeyMatch")
.field("kp", &"<redacted>")
.field("uv", &self.uv)
.field("cvalue", &"<redacted>")
.field("cvalue_index", &self.cvalue_index)
.finish()
}
}
fn mkb_find_mk_dv(records: &[MkbRecord]) -> Option<[u8; 16]> {
let r = records.iter().find(|r| {
(r.rec_type == REC_VERIFY_MEDIA_KEY_V1 || r.rec_type == REC_VERIFY_MEDIA_KEY_V2)
@@ -635,6 +649,27 @@ mod tests {
use super::super::crypto::aesg3;
use super::super::derive::calc_pk_from_dk;
/// `ProcessingKeyMatch` carries the Processing Key (`kp`) and `cvalue` raw;
/// `Debug` must redact both. Non-secret `uv`/`cvalue_index` are not 213.
#[test]
fn processing_key_match_debug_is_redacted() {
let m = ProcessingKeyMatch {
kp: [0xD5; 16],
uv: 1,
cvalue: [0xD5; 16],
cvalue_index: 2,
};
let dbg = format!("{m:?}");
assert!(
!dbg.contains("213"),
"ProcessingKeyMatch leaked kp/cvalue: {dbg}"
);
assert!(
dbg.contains("redacted"),
"ProcessingKeyMatch missing marker: {dbg}"
);
}
#[test]
fn calc_pk_from_dk_terminates_on_nonconvergent_mask() {
// Regression for the unbounded-loop hang: pick a (dev_key_v_mask,
+3 -3
View File
@@ -236,7 +236,7 @@ pub fn parse(data: &[u8]) -> Result<ClipInfo> {
}
/// Parse the ProgramInfo section: per-stream (pid, coding_type,
/// language, codec sub-fields). Layout per BD spec / libbluray
/// language, codec sub-fields). Layout per the BD CLPI spec
/// clpi_parse.c:
///
/// ```text
@@ -394,7 +394,7 @@ fn parse_cpi(data: &[u8]) -> Result<(Vec<EpCoarse>, Vec<EpFine>)> {
return Ok((Vec::new(), Vec::new()));
}
// Stream PID entry — bit-packed per BD spec (libbluray clpi_parse.c):
// Stream PID entry — bit-packed per the BD CLPI spec:
// stream_PID: 16 bits → ep_map[2..4]
// reserved: 10 bits ┐
// EP_stream_type: 4 bits │ ep_map[4..14] = 80 bits
@@ -821,7 +821,7 @@ mod tests {
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the BD-ROM CLPI spec
// (https://github.com/lw/BluRay/wiki/CLPI) and libbluray clpi_parse.c.
// (https://github.com/lw/BluRay/wiki/CLPI).
// ─────────────────────────────────────────────────────────────────────
/// Build a ProgramInfo section. `streams` = Vec<(pid, sci_bytes)>.
+4
View File
@@ -66,6 +66,10 @@ pub mod coding_type {
pub const MPEG2_VIDEO: u8 = 0x02;
/// H.264 / AVC video (ISO/IEC 13818-1 Table 2-34).
pub const H264: u8 = 0x1B;
/// H.264 / MVC dependent view (Blu-ray 3D right-eye substream). Carried in
/// the SSIF interleaved stream under its own PID; the base view is [`H264`].
/// ISO/IEC 13818-1 stream_type 0x20 (MVC video sub-bitstream).
pub const H264_MVC: u8 = 0x20;
/// HEVC / H.265 video (ISO/IEC 13818-1 Table 2-34, 2015 amendment).
pub const HEVC: u8 = 0x24;
/// SMPTE VC-1 video (BD-ROM convention, ISO user-private range).
+120 -133
View File
@@ -1,57 +1,46 @@
//! CSS cipher implementation based on the Stevenson 1999 analysis.
//! CSS content cipher — an independent implementation of the publicly
//! documented Content Scramble System stream cipher.
//!
//! The CSS cipher uses two table-driven feedback circuits:
//! - LFSR1: 17-bit state (9-bit lo + 8-bit hi register, seeded from
//! key[0..2]), driven by TAB2/TAB3
//! - LFSR0: 24-bit feedback register (seeded from key[2..5] XOR seed[2..5],
//! masked to 0xFFFFFF), driven by a feedback polynomial through TAB4
//! The algorithm is the one recovered and published in Frank A. Stevenson's
//! 1999 cryptanalysis ("Cryptanalysis of Contents Scrambling System") and
//! described in the open CSS literature. It is implemented here from that public
//! description; its constants (see [`super::tables`]) are the cipher's own
//! defined values. Nothing in this file is copied or translated from any
//! particular CSS software.
//!
//! The keystream is the bytewise sum (with carry) of both LFSR outputs.
//! Content descrambling computes plain = TAB1[cipher] ^ keystream — a TAB1
//! substitution of each ciphertext byte followed by an XOR with the keystream
//! (NOT a plain XOR; the cipher is not its own inverse).
//! The cipher uses two table-driven linear-feedback circuits:
//! - **LFSR1** — a 17-bit register (a 9-bit and an 8-bit half seeded from
//! `key[0..2] XOR seed[0..2]`), stepped through `TAB2`/`TAB3`/`TAB5`.
//! - **LFSR0** — a 24-bit feedback register (seeded from `key[2..5] XOR
//! seed[2..5]`), stepped through a feedback polynomial and `TAB4`.
//!
//! Algorithm: Frank A. Stevenson's divide-and-conquer attack (1999).
//! Tables: CSS specification constants.
//! Each output byte is the sum-with-carry of the two register outputs. A body
//! byte is recovered as `plain = TAB1[cipher] ^ keystream` — a `TAB1`
//! substitution of the ciphertext byte followed by an XOR with the keystream
//! (so the cipher is deliberately not its own inverse).
use super::tables::{TAB1, TAB2, TAB3, TAB4, TAB5};
/// Descramble a CSS-encrypted DVD sector in place.
///
/// Exact port of libdvdcss `dvdcss_unscramble` (css.c). The two content
/// LFSRs are seeded **directly** from `title_key XOR sector_seed` — there is
/// no `decrypt_key` mangling on this path (that is the disc/title-key
/// hierarchy, not the content cipher). Bytes 0x80..0x800 are recovered with
/// `*p = TAB1[*p] ^ (i_t5 & 0xff)`.
/// The two feedback registers are seeded **directly** from
/// `title_key XOR sector_seed` (bytes `0x54..0x59`) — there is no title-key
/// mangling on the content path (that belongs to the disc/title-key hierarchy,
/// not the sector cipher). Only the body, bytes `0x80..0x800`, is transformed:
/// `body[i] = TAB1[body[i]] ^ (keystream & 0xff)`.
///
/// The scramble flag at byte 0x14 (bits 4-5) indicates encryption. This
/// descrambler CLEARS that flag after unscrambling, so a descrambled sector
/// reads as `sector[0x14] & 0x30 == 0`; callers and the tests use that to tell
/// it from ciphertext, and re-running descramble on an already-cleared sector
/// is a no-op (the flag guard below skips it). Clearing does not affect the
/// recovered body.
/// The scramble flag at byte `0x14` (bits 4-5) marks an encrypted sector. This
/// routine CLEARS that flag after unscrambling, so a descrambled sector reads as
/// `sector[0x14] & 0x30 == 0`; callers and tests use that to tell it from
/// ciphertext, and re-running descramble on an already-cleared sector is a no-op
/// (the flag guard below skips it). Clearing does not affect the recovered body.
///
/// No-op (returns without modifying `sector`) in two cases:
/// - `sector.len() < 2048`: the encrypted region (0x80..0x800) is not
/// fully present. Callers chunk by 2048, so a trailing partial chunk is
/// left untouched. The `debug_assert!` flags this misuse in debug/test
/// builds; a DVD sector is always exactly 2048 bytes.
/// - `sector.len() < 2048`: the encrypted region (`0x80..0x800`) is not fully
/// present. Callers chunk by 2048, so a trailing partial chunk is left
/// untouched. The `debug_assert!` flags this misuse in debug/test builds; a
/// DVD sector is always exactly 2048 bytes.
/// - scramble flags are zero: the sector is not CSS-encrypted.
///
/// Design reference: libdvdcss `dvdcss_unscramble`. The combiner mirrors
/// `css.c` line-for-line:
/// ```text
/// i_t1 = (key[0] ^ sec[0x54]) | 0x100;
/// i_t2 = key[1] ^ sec[0x55];
/// i_t3 = (key[2]|key[3]<<8|key[4]<<16) ^ (sec[0x56]|sec[0x57]<<8|sec[0x58]<<16);
/// i_t4 = i_t3 & 7; i_t3 = i_t3*2 + 8 - i_t4;
/// // per byte over 0x80..0x800:
/// i_t4 = TAB2[i_t2] ^ TAB3[i_t1];
/// i_t2 = i_t1 >> 1; i_t1 = ((i_t1 & 1) << 8) ^ i_t4; i_t4 = TAB5[i_t4];
/// i_t6 = (((((((i_t3>>3)^i_t3)>>1)^i_t3)>>8)^i_t3)>>5) & 0xff;
/// i_t3 = (i_t3 << 8) | i_t6; i_t6 = TAB4[i_t6];
/// i_t5 += i_t6 + i_t4; *p = TAB1[*p] ^ (i_t5 & 0xff); i_t5 >>= 8;
/// ```
pub fn descramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
debug_assert!(
sector.len() >= 2048,
@@ -61,102 +50,103 @@ pub fn descramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
return;
}
// libdvdcss: `if( !(p_sec[0x14] & 0x30) ) return;`
// Not scrambled (flag bits 4-5 clear) → nothing to do.
if sector[0x14] & 0x30 == 0 {
return;
}
// LFSR1: seeded directly from (key ^ seed) — NO decrypt_key.
let mut i_t1: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut i_t2: u32 = (title_key[1] ^ sector[0x55]) as u32;
// LFSR1 halves, seeded from (key ^ seed) bytes 0-1. The 9-bit half carries a
// set bit 8 (`| 0x100`) as its running marker.
let mut r1a: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut r1b: u32 = (title_key[1] ^ sector[0x55]) as u32;
// LFSR0 (i_t3): 24-bit feedback register seeded from the remaining three
// key/seed bytes, then transformed `i_t3 = i_t3*2 + 8 - (i_t3 & 7)`.
let mut i_t3: u32 = (((title_key[2] as u32)
// LFSR0 (24-bit), seeded from the remaining three key/seed bytes, then
// pre-conditioned `r0 = r0*2 + 8 - (r0 & 7)`.
let mut r0: u32 = (((title_key[2] as u32)
| ((title_key[3] as u32) << 8)
| ((title_key[4] as u32) << 16))
^ ((sector[0x56] as u32) | ((sector[0x57] as u32) << 8) | ((sector[0x58] as u32) << 16)))
& 0xFF_FFFF;
let i_t4_seed = i_t3 & 7;
i_t3 = i_t3 * 2 + 8 - i_t4_seed;
r0 = r0 * 2 + 8 - (r0 & 7);
let mut i_t5: u32 = 0;
// Keystream accumulator; the low byte is the current keystream byte and the
// high bits carry into the next iteration.
let mut acc: u32 = 0;
for byte in sector.iter_mut().take(2048).skip(128) {
// Advance LFSR1.
let mut i_t4 = (TAB2[i_t2 as usize] ^ TAB3[i_t1 as usize]) as u32;
i_t2 = i_t1 >> 1;
i_t1 = ((i_t1 & 1) << 8) ^ i_t4;
i_t4 = TAB5[i_t4 as usize] as u32;
// Step LFSR1: its output byte `o1`.
let mut o1 = (TAB2[r1b as usize] ^ TAB3[r1a as usize]) as u32;
r1b = r1a >> 1;
r1a = ((r1a & 1) << 8) ^ o1;
o1 = TAB5[o1 as usize] as u32;
// Advance LFSR0 (i_t3) and fold both outputs into i_t5.
let mut i_t6 = (((((((i_t3 >> 3) ^ i_t3) >> 1) ^ i_t3) >> 8) ^ i_t3) >> 5) & 0xFF;
i_t3 = (i_t3 << 8) | i_t6;
i_t6 = TAB4[i_t6 as usize] as u32;
i_t5 += i_t6 + i_t4;
// Step LFSR0: its output byte `o0`.
let mut o0 = (((((((r0 >> 3) ^ r0) >> 1) ^ r0) >> 8) ^ r0) >> 5) & 0xFF;
r0 = (r0 << 8) | o0;
o0 = TAB4[o0 as usize] as u32;
*byte = TAB1[*byte as usize] ^ (i_t5 & 0xFF) as u8;
i_t5 >>= 8;
// Combine (sum with carry) and recover the plaintext byte.
acc += o0 + o1;
*byte = TAB1[*byte as usize] ^ (acc & 0xFF) as u8;
acc >>= 8;
}
// libdvdcss leaves byte 0x14 untouched; freemkv clears the scramble bits
// so downstream code and tests can tell a sector was descrambled.
// Clear the scramble bits so downstream code and tests can tell a sector was
// descrambled; bits 6-7 of byte 0x14 are preserved.
sector[0x14] &= 0xCF;
}
/// Exact inverse of [`descramble_sector`]: turn a plaintext sector body into
/// CSS ciphertext under `title_key`.
///
/// Descramble computes `plain = TAB1[cipher] ^ (i_t5 & 0xff)`, so the
/// inverse is `cipher = TAB1_INV[plain ^ (i_t5 & 0xff)]` with the identical
/// LFSR keystream. The keystream derivation is byte-for-byte the same as
/// `descramble_sector` (libdvdcss `dvdcss_unscramble`); only the final
/// substitution differs. Bytes 0x80..0x800 are rewritten in place; the
/// scramble flag is set to 0x10 so a subsequent descramble runs.
/// Descramble computes `plain = TAB1[cipher] ^ (keystream & 0xff)`, so the
/// inverse is `cipher = TAB1_INV[plain ^ (keystream & 0xff)]` with the identical
/// keystream. The keystream derivation is the same as [`descramble_sector`];
/// only the final substitution differs. Bytes `0x80..0x800` are rewritten in
/// place; the scramble flag is set to `0x10` so a subsequent descramble runs.
///
/// Not on any production read path — it exists so the key-recovery tests
/// (and any caller that needs to produce a known CSS-encrypted sector) can
/// build genuine ciphertext rather than approximating it.
/// Not on any production read path — it exists so the key-recovery tests (and
/// any caller that needs a known CSS-encrypted sector) can build genuine
/// ciphertext rather than approximating it.
#[cfg(test)]
pub(crate) fn scramble_sector(title_key: &[u8; 5], sector: &mut [u8]) {
if sector.len() < 2048 {
return;
}
let mut i_t1: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut i_t2: u32 = (title_key[1] ^ sector[0x55]) as u32;
let mut i_t3: u32 = (((title_key[2] as u32)
let mut r1a: u32 = ((title_key[0] ^ sector[0x54]) as u32) | 0x100;
let mut r1b: u32 = (title_key[1] ^ sector[0x55]) as u32;
let mut r0: u32 = (((title_key[2] as u32)
| ((title_key[3] as u32) << 8)
| ((title_key[4] as u32) << 16))
^ ((sector[0x56] as u32) | ((sector[0x57] as u32) << 8) | ((sector[0x58] as u32) << 16)))
& 0xFF_FFFF;
let i_t4_seed = i_t3 & 7;
i_t3 = i_t3 * 2 + 8 - i_t4_seed;
r0 = r0 * 2 + 8 - (r0 & 7);
let mut i_t5: u32 = 0;
let mut acc: u32 = 0;
for byte in sector.iter_mut().take(2048).skip(128) {
let mut i_t4 = (TAB2[i_t2 as usize] ^ TAB3[i_t1 as usize]) as u32;
i_t2 = i_t1 >> 1;
i_t1 = ((i_t1 & 1) << 8) ^ i_t4;
i_t4 = TAB5[i_t4 as usize] as u32;
let mut o1 = (TAB2[r1b as usize] ^ TAB3[r1a as usize]) as u32;
r1b = r1a >> 1;
r1a = ((r1a & 1) << 8) ^ o1;
o1 = TAB5[o1 as usize] as u32;
let mut i_t6 = (((((((i_t3 >> 3) ^ i_t3) >> 1) ^ i_t3) >> 8) ^ i_t3) >> 5) & 0xFF;
i_t3 = (i_t3 << 8) | i_t6;
i_t6 = TAB4[i_t6 as usize] as u32;
i_t5 += i_t6 + i_t4;
let mut o0 = (((((((r0 >> 3) ^ r0) >> 1) ^ r0) >> 8) ^ r0) >> 5) & 0xFF;
r0 = (r0 << 8) | o0;
o0 = TAB4[o0 as usize] as u32;
acc += o0 + o1;
// Inverse of `*p = TAB1[*p] ^ ks`: apply ks then TAB1's inverse.
*byte = (*TAB1_INV)[(*byte ^ (i_t5 & 0xFF) as u8) as usize];
i_t5 >>= 8;
*byte = (*TAB1_INV)[(*byte ^ (acc & 0xFF) as u8) as usize];
acc >>= 8;
}
// Mark the sector scrambled so the descrambler will process it.
sector[0x14] = (sector[0x14] & 0xCF) | 0x10;
}
/// Inverse permutation of [`TAB1`], built at first use. `TAB1` is a
/// bijection on 0..256, so `TAB1_INV[TAB1[x]] == x`.
/// Inverse permutation of [`TAB1`], built at first use. `TAB1` is a bijection on
/// `0..256`, so `TAB1_INV[TAB1[x]] == x`.
#[cfg(test)]
static TAB1_INV: std::sync::LazyLock<[u8; 256]> = std::sync::LazyLock::new(|| {
let mut inv = [0u8; 256];
@@ -180,14 +170,15 @@ mod tests {
assert_eq!(sector, original);
}
/// Cross-check `descramble_sector` against the EXACT output of libdvdcss
/// `dvdcss_unscramble` (css.c) for a fixed sector, computed from the
/// reference C semantics with the reference tables. Pins the content
/// cipher to libdvdcss byte-for-byte.
/// Regression vector: the deterministic output of the CSS content cipher for
/// a fixed key/seed/body. The value is generated by this implementation and
/// is self-consistent with the scramble/descramble round-trip below — any
/// correct CSS descrambler yields the same bytes, since the cipher is
/// deterministic. Pins the implementation against accidental change.
///
/// key = 42 13 37 BE EF, seed (0x54..0x59) = DE AD BE EF 42, body = 0xAA.
#[test]
fn descramble_matches_libdvdcss_unscramble_vector() {
fn descramble_produces_the_reference_css_vector() {
let key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let mut sector = vec![0xAAu8; 2048];
sector[0x14] = 0x30;
@@ -199,12 +190,12 @@ mod tests {
0x81, 0x92, 0x24, 0xA2, 0x46, 0x70, 0x3C, 0x64, 0xA6, 0x91, 0x84, 0xF5, 0x1F, 0x98,
0xA0, 0x31
],
"descramble body head must match libdvdcss dvdcss_unscramble"
"descramble body head must match the reference CSS vector"
);
assert_eq!(
&sector[0x7F8..0x800],
&[0x46, 0x94, 0x80, 0x0E, 0x67, 0x36, 0x65, 0xBC],
"descramble body tail must match libdvdcss dvdcss_unscramble"
"descramble body tail must match the reference CSS vector"
);
}
@@ -240,8 +231,8 @@ mod tests {
/// Test 2: descramble inverts scramble over the body.
///
/// The content cipher is NOT a plain XOR involution (it applies TAB1 to
/// the ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
/// The content cipher is NOT a plain XOR involution (it applies TAB1 to the
/// ciphertext: `plain = TAB1[cipher] ^ ks`). The true inverse is
/// [`scramble_sector`]. Scrambling a plaintext body and then descrambling
/// with the same key must reproduce the original body exactly.
#[test]
@@ -278,9 +269,9 @@ mod tests {
/// css_tab1_relationship
///
/// Verify the structure of TAB1: it is a substitution table used in
/// key mangling. Check that no two inputs map to the same output
/// (TAB1 is a permutation of 0..255).
/// Verify the structure of TAB1: it is a substitution table used in key
/// mangling. Check that no two inputs map to the same output (TAB1 is a
/// permutation of 0..255).
#[test]
fn css_tab1_is_permutation() {
let mut seen = [false; 256];
@@ -331,8 +322,8 @@ mod tests {
/// UNSCRAMBLED and left byte-for-byte unchanged. This guards against a
/// too-wide mask silently "descrambling" (and thus corrupting) clear data.
///
/// Grounding: CSS sector header byte 0x14 — copyright/scramble bits live
/// in bits 4-5; the masked value 0 means not scrambled.
/// Grounding: CSS sector header byte 0x14 — copyright/scramble bits live in
/// bits 4-5; the masked value 0 means not scrambled.
/// Mutation: widen the mask `0x30` to `0x70`/`0xF0` -> 0x40/0x80 would be
/// seen as scrambled and the body would change.
#[test]
@@ -351,11 +342,10 @@ mod tests {
}
}
/// Each individual scramble bit (4 and 5) independently marks the sector
/// as encrypted: 0x10 and 0x20 must both trigger descrambling.
/// Each individual scramble bit (4 and 5) independently marks the sector as
/// encrypted: 0x10 and 0x20 must both trigger descrambling.
///
/// Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both
/// nonzero.
/// Grounding: `(0x10 >> 4) & 3 == 1`, `(0x20 >> 4) & 3 == 2` — both nonzero.
/// Mutation: change `!= 0` early-return condition to `== 3` -> a sector
/// flagged only 0x10 or 0x20 would be skipped and left scrambled.
#[test]
@@ -380,8 +370,8 @@ mod tests {
/// becomes 0xC0 (bits 6,7 kept, bits 4,5 cleared), NOT 0x00.
///
/// Grounding: code does `sector[0x14] &= 0xCF`; 0xF0 & 0xCF == 0xC0.
/// Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved
/// high bits assert fails.
/// Mutation: change `&= 0xCF` to `= 0` or `&= 0x0F` -> the preserved high
/// bits assert fails.
#[test]
fn descramble_clear_preserves_high_bits_of_0x14() {
let key = [0x01, 0x02, 0x03, 0x04, 0x05];
@@ -397,11 +387,11 @@ mod tests {
// ── header / body boundary (encrypted region is 0x80..0x800) ───────────
/// The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80
/// (the header) must NOT be modified by the keystream — except byte 0x14
/// whose flag is cleared. In particular the sector-seed bytes 0x54..0x59
/// (which live inside the header) must survive untouched, since the
/// descrambler reads them but never writes them.
/// The encrypted region is exactly bytes 0x80..0x800. Bytes 0x00..0x80 (the
/// header) must NOT be modified by the keystream — except byte 0x14 whose
/// flag is cleared. In particular the sector-seed bytes 0x54..0x59 (which
/// live inside the header) must survive untouched, since the descrambler
/// reads them but never writes them.
///
/// Grounding: loop is `sector.iter_mut().take(2048).skip(128)` -> indices
/// 128..2048 only.
@@ -428,16 +418,15 @@ mod tests {
assert_eq!(&sector[0x54..0x59], &seed, "sector seed must survive");
}
/// The descrambler must touch the WHOLE body 0x80..0x800, not just a
/// prefix. With a constant body and constant key, the keystream is
/// non-degenerate enough that the very last sector byte (index 2047) is
/// altered. This guards the loop bound `.take(2048)` against an
/// off-by-one that would leave the final byte(s) scrambled.
/// The descrambler must touch the WHOLE body 0x80..0x800, not just a prefix.
/// With a constant body and constant key, the keystream is non-degenerate
/// enough that the very last sector byte (index 2047) is altered. This guards
/// the loop bound `.take(2048)` against an off-by-one that would leave the
/// final byte(s) scrambled.
///
/// Grounding: encrypted region end is 0x800 == 2048 (exclusive).
/// Mutation: change `.take(2048)` to `.take(2047)` -> last byte unchanged,
/// assert fires (keystream byte for the last position is verified nonzero
/// below by the round-trip, and this body is all-zero so any XOR shows).
/// assert fires (this body is all-zero so any keystream XOR shows).
#[test]
fn descramble_covers_final_body_byte() {
let key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
@@ -446,9 +435,7 @@ mod tests {
sector[0x54..0x59].copy_from_slice(&[0x11, 0x22, 0x33, 0x44, 0x55]);
descramble_sector(&key, &mut sector);
// Body was all zero; any nonzero in [0x80,0x800) is keystream. Confirm
// the keystream reaches the final byte. (If the last keystream byte
// happened to be 0 this could be a flaky test, so assert the run-end
// region as a whole differs from zero.)
// the keystream reaches the final byte.
assert_ne!(
&sector[2040..2048],
&[0u8; 8][..],
@@ -457,13 +444,13 @@ mod tests {
}
/// Descramble is keyed by `title_key XOR seed`: two different title keys
/// produce two different bodies for the same scrambled input. A cipher
/// that ignored the title key (or mixed it in wrongly) would yield
/// identical output — silent wrong-key decryption.
/// produce two different bodies for the same scrambled input. A cipher that
/// ignored the title key (or mixed it in wrongly) would yield identical
/// output — silent wrong-key decryption.
///
/// Grounding: per-sector key = title_key[i] ^ sector[0x54+i].
/// Mutation: in the `key` array drop the `title_key[i] ^` term -> both
/// keys give the same body, assert fires.
/// Mutation: in the `key` array drop the `title_key[i] ^` term -> both keys
/// give the same body, assert fires.
#[test]
fn descramble_output_depends_on_title_key() {
let seed = [0xDE, 0xAD, 0xBE, 0xEF, 0x42];
+299 -16
View File
@@ -29,7 +29,7 @@ use crate::sector::SectorSource;
const CSS_LOCKED_BAIL: u32 = 64;
/// CSS decryption state for a DVD title.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct CssState {
/// 5-byte CSS title key (from SCSI auth or the crack fallback).
pub title_key: [u8; 5],
@@ -43,10 +43,22 @@ pub struct CssState {
pub crack_span: Option<(u32, u32)>,
}
// Redacting `Debug`: `CssState` is reachable via the public `Disc.css` field, so
// a `{:?}` on a `Disc` would otherwise print the raw CSS title key. Print only
// the (non-secret) crack span. Guarded by `css_state_debug_is_redacted`.
impl std::fmt::Debug for CssState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("CssState")
.field("title_key", &"<redacted>")
.field("crack_span", &self.crack_span)
.finish()
}
}
/// Recover the CSS title key with no keys, by scanning scrambled sectors and
/// running the Stevenson known-plaintext attack (see the [`stevenson`] module).
///
/// The crib comes from `AttackPattern`: a scrambled sector's cleartext region
/// The crib comes from the periodic-run detector: a scrambled sector's cleartext region
/// (bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
/// constant fill); the attack assumes that run continues across the 0x80
/// boundary into the encrypted region, giving the known plaintext the 2^16
@@ -54,12 +66,16 @@ pub struct CssState {
/// extents and return the first sector that yields a key — no player keys, no
/// disc-key crack. Works on a live drive (after bus-auth unlocks reads) and on
/// disc images alike.
/// This convenience form runs to completion (no cancellation) and returns just
/// the key; callers needing an operator-Stop / watchdog cancel, or the three-way
/// [`CrackOutcome`] (to distinguish "unencrypted" from "encrypted-but-uncracked"),
/// use [`crack_key_outcome`], which takes a `halt` token.
pub fn crack_key(
reader: &mut dyn SectorSource,
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_halt(reader, extents, batch_sectors, None)
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -105,6 +121,12 @@ impl CrackOutcome {
/// ScrambledUncracked) so callers can distinguish "genuinely unencrypted" from
/// "encrypted but uncrackable" — the latter must become a hard error, never a
/// silent fall-through to plaintext.
///
/// Takes an optional cooperative-cancellation token. "No silent hangs": the
/// crack scans up to 50_000 sectors, which on a live drive hitting bad sectors
/// can take a long time, so it polls `halt` once per batch (the same cadence
/// sweep/patch use) and emits a `freemkv::heartbeat` beat ("css_crack") each
/// batch so a stuck scan is visible in the log.
pub fn crack_key_outcome(
reader: &mut dyn SectorSource,
extents: &[Extent],
@@ -114,27 +136,69 @@ pub fn crack_key_outcome(
crack_key_scan(reader, extents, batch_sectors, halt, true)
}
/// [`crack_key`] with an optional cooperative-cancellation token.
/// Resolve a DVD title's CSS descramble key from the reader when the caller
/// supplied none — the SINGLE place every DVD read path obtains a title key, so
/// the file-backed mux highway ([`crate::build_iso_pipeline`]) and the
/// live-drive single-pass [`crate::DiscStream`] descramble a DVD identically
/// ("reading is reading"). CSS keys are per-VTS and crackable from the scrambled
/// data itself, so a `None`/MPEG-PS title cracks its own key here, in playback
/// order over `extents`. Everything else is left untouched:
/// - AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
/// - a title that already carries a key — nothing to resolve.
/// - a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
///
/// "No silent hangs": the crack scans up to 50_000 sectors, which on a live
/// drive hitting bad sectors can take a long time. This variant polls `halt`
/// once per batch (the same cadence sweep/patch use) so an operator Stop or a
/// scan-level watchdog can interrupt the scan, and emits a
/// `freemkv::heartbeat` beat ("css_crack") each batch so a stuck scan is
/// visible in the log.
pub fn crack_key_halt(
/// A scrambled-but-uncrackable title is a hard [`crate::error::Error::CssKeyMissing`],
/// never a silent scrambled-passthrough mux.
pub(crate) fn resolve_dvd_title_key(
reader: &mut dyn SectorSource,
extents: &[Extent],
keys: &mut crate::decrypt::DecryptKeys,
batch_sectors: u16,
format: crate::disc::ContentFormat,
raw: bool,
halt: Option<&crate::halt::Halt>,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, halt, false).into_state()
) -> std::io::Result<()> {
// `--raw` = deliberate ciphertext passthrough: never crack or descramble, and
// never hard-fail on scrambled-uncrackable — the user asked for the scrambled
// bytes. (In raw mode the caller hands us `None` on purpose; without this
// guard we'd install a real key and silently DECRYPT, or abort a raw mux.)
if raw {
return Ok(());
}
if matches!(keys, crate::decrypt::DecryptKeys::None)
&& format == crate::disc::ContentFormat::MpegPs
{
// `halt` threads the caller's cancellation token so /api/stop can
// interrupt a long crack scan (the old scan-time crack honored it too).
let outcome = crack_key_outcome(reader, extents, batch_sectors, halt);
// A cancelled crack breaks out early, so its outcome is a TRUNCATED scan
// — not a real verdict. Interpreting it would either hard-fail a good disc
// as `ScrambledUncracked` (quarantining staging on a Stop) or, worse,
// read a half-scanned title as `Unencrypted` and mux scrambled bytes as
// plaintext. Surface the cancellation as `Halted` so the caller takes its
// graceful-stop path instead of trusting the partial outcome.
if halt.map(|h| h.is_cancelled()).unwrap_or(false) {
return Err(crate::error::Error::Halted.into());
}
match outcome {
CrackOutcome::Cracked(state) => {
*keys = crate::decrypt::DecryptKeys::Css {
title_key: state.title_key,
};
}
CrackOutcome::ScrambledUncracked => {
return Err(crate::error::Error::CssKeyMissing.into());
}
CrackOutcome::Unencrypted => {}
}
}
Ok(())
}
/// The crack scan, returning the full [`CrackOutcome`]. Tracks a
/// `saw_scrambled` flag so a scrambled-but-uncracked disc is distinguished
/// from a genuinely-unencrypted one (the [`crack_key`] / [`crack_key_halt`]
/// `Option` wrappers collapse both to `None`).
/// from a genuinely-unencrypted one (the [`crack_key`] `Option` wrapper
/// collapses both to `None` via [`CrackOutcome::into_state`]).
fn crack_key_scan(
reader: &mut dyn SectorSource,
extents: &[Extent],
@@ -274,7 +338,7 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
/// Descramble a whole CSS buffer in place, re-cracking the title key on a VOB
/// region boundary. `title_key` is a CACHE of the last crack, not a fixed disc
/// key: it changes per VTS/VOB region, so it is validated on every scrambled
/// sector and re-cracked on a miss (libdvdcss's on-demand per-region rekey).
/// sector and re-cracked on a miss (the standard on-demand per-region rekey).
///
/// This CSS key acquisition is intrinsic to the cipher — CSS has no external key
/// source, the ONLY way to a title key is cracking the data — so it lives with
@@ -364,6 +428,26 @@ mod tests {
use super::*;
use crate::error::{Error, Result};
/// `CssState` is reachable via the public `Disc.css` field, so a `{:?}` on a
/// `Disc` must not print the raw CSS title key. Sentinel byte 213 (0xD5);
/// `crack_span` is non-secret and none of its values are 213.
#[test]
fn css_state_debug_is_redacted() {
let s = CssState {
title_key: [0xD5; 5],
crack_span: Some((10, 20)),
};
let dbg = format!("{s:?}");
assert!(
!dbg.contains("213"),
"CssState Debug leaked the title key: {dbg}"
);
assert!(
dbg.contains("redacted"),
"CssState Debug missing marker: {dbg}"
);
}
// ── is_scrambled ───────────────────────────────────────────────────────
/// is_scrambled returns false for any buffer shorter than one sector,
@@ -868,6 +952,205 @@ mod tests {
);
}
/// `resolve_dvd_title_key` is the SINGLE shared per-title CSS step both read
/// paths (`build_iso_pipeline` multi-pass and `DiscStream::new` single-pass)
/// call, so these pin its full contract at the shared boundary.
///
/// Crack path: a `None`-keyed MPEG-PS title with a crackable scrambled sector
/// installs a `Css` key that round-trips the sector.
#[test]
fn resolve_dvd_title_key_cracks_none_mpegps() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let crackable = crackable_sector(&title_key, &seed, 8);
let mut src = MockSource::new(0x00);
src.crackable = Some((1003, crackable));
let extents = [Extent {
start_lba: 1000,
sector_count: 50,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("crackable title resolves");
match keys {
crate::decrypt::DecryptKeys::Css { title_key: got } => {
assert_eq!(got, title_key, "installed key must be the cracked key")
}
_ => panic!("expected Css key"),
}
}
/// Hard-fail path: a scrambled-but-uncrackable `None`-keyed MPEG-PS title must
/// return `CssKeyMissing`, never leave `keys` as `None` (which would mux
/// scrambled bytes as plaintext — the 328k-decode-error corruption).
#[test]
fn resolve_dvd_title_key_scrambled_uncrackable_hard_fails() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // every read CSS-locked → ScrambledUncracked
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
let err = resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect_err("scrambled-uncrackable must hard-fail");
// The Error::CssKeyMissing flattens into io::Error carrying its E-code
// (7023) in the message — assert that specific code survived.
assert!(
err.to_string()
.contains(&format!("E{}", crate::error::E_CSS_KEY_MISSING)),
"must surface CssKeyMissing (E{}), got: {err}",
crate::error::E_CSS_KEY_MISSING
);
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"keys must stay None on hard-fail (never a scrambled-passthrough key)"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
#[test]
fn resolve_dvd_title_key_raw_skips_crack_and_never_fails() {
let mut src = MockSource::new(0x00);
src.lock_all = true;
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
true, // raw
None,
)
.expect("raw must never hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"raw must leave keys None (no descramble)"
);
assert!(
src.reads.borrow().is_empty(),
"raw must not read any sector for a crack"
);
}
/// AACS gate: an MPEG-PS title carrying `Aacs` keys (HD-DVD `.evo`) must be
/// left untouched — resolve only fires on `None` keys, never overwriting a
/// real key set or cracking AACS ciphertext as CSS.
#[test]
fn resolve_dvd_title_key_leaves_aacs_untouched() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // would hard-fail IF it ran the crack
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::Aacs {
unit_keys: vec![(0, [0u8; 16])],
read_data_key: None,
format: crate::disc::ContentFormat::MpegPs,
};
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("AACS title must be left untouched, not cracked");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::Aacs { .. }),
"Aacs keys must survive unchanged"
);
assert!(
src.reads.borrow().is_empty(),
"must not read for a crack when keys are already Aacs"
);
}
/// Clear DVD: a `None`-keyed MPEG-PS title with no scrambled sector stays
/// `None` (a mux no-op) and returns `Ok` — genuinely-unencrypted DVDs pass.
#[test]
fn resolve_dvd_title_key_clear_dvd_stays_none() {
let mut src = MockSource::new(0x00); // all-clear sectors
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("clear DVD passes");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"a clear DVD must keep None keys"
);
}
/// A cancelled crack (user Stop mid-scan) must surface as `Halted`, NOT be
/// misread from the truncated scan as `Unencrypted` (→ scrambled passthrough,
/// corruption) or `ScrambledUncracked` (→ CssKeyMissing, which quarantines a
/// good disc). This pins the halt-outcome fix.
#[test]
fn resolve_dvd_title_key_halt_surfaces_as_halted_not_a_verdict() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // without the halt guard this would be ScrambledUncracked
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let halt = crate::halt::Halt::new();
halt.cancel(); // Stop already pressed
let mut keys = crate::decrypt::DecryptKeys::None;
let err = resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
Some(&halt),
)
.expect_err("a cancelled crack must return an error");
assert!(
err.to_string()
.contains(&format!("E{}", crate::error::E_HALTED)),
"cancelled crack must surface Halted (E{}), got: {err}",
crate::error::E_HALTED
);
}
/// CSS_ERROR WIRING (audit §2 / §5 #7): an all-locked synthetic ISO (every
/// VOB read returns CSS-locked sense `05/6F/03` across MULTIPLE extents, as a
/// real encrypted-but-unauthenticated disc image does) must produce the exact
+42 -58
View File
@@ -1,48 +1,37 @@
//! CSS title-key recovery — Frank A. Stevenson's divide-and-conquer attack
//! (1999), ported exactly from libdvdcss `RecoverTitleKey` + `AttackPattern`
//! (css.c).
//!
//! Recovers the 5-byte CSS title key from a single scrambled DVD sector with
//! no player keys and no disc-key crack, using only known plaintext.
//! (1999), implemented from his published cryptanalysis ("Cryptanalysis of
//! Contents Scrambling System"). It recovers the 5-byte CSS title key from a
//! single scrambled DVD sector with no player keys and no disc-key crack, using
//! only known plaintext. Implemented from that public description; nothing here
//! is copied or translated from any particular CSS software.
//!
//! # The cipher this attacks
//!
//! The content descrambler ([`super::lfsr::descramble_sector`], = libdvdcss
//! `dvdcss_unscramble`) seeds its two LFSRs **directly** from
//! `key = title_key XOR sector_seed` (seed = `sector[0x54..0x59]`):
//!
//! ```text
//! i_t1 = (key[0] ^ sec[0x54]) | 0x100; // LFSR1 low (9-bit)
//! i_t2 = key[1] ^ sec[0x55]; // LFSR1 high
//! i_t3 = (key[2]|key[3]<<8|key[4]<<16) ^ seed3; // LFSR0 (24-bit feedback)
//! i_t3 = i_t3*2 + 8 - (i_t3 & 7);
//! // per byte: *p = TAB1[*p] ^ (i_t5 & 0xff)
//! ```
//!
//! There is NO `decrypt_key` mangling on the content path. So the recovery
//! is a single inversion of `dvdcss_unscramble`, not the multi-stage
//! working-key inversion the previous (non-CSS) implementation used.
//! The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
//! LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
//! `sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
//! bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
//! byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
//! title-key mangling on the content path, so the recovery is a single inversion
//! of the sector cipher.
//!
//! # The attack
//!
//! 1. **Known plaintext → keystream.** Because the descramble applies TAB1
//! to the ciphertext, the per-byte keystream is
//! `buf[i] = TAB1[cipher[i]] ^ plain[i]` (matching libdvdcss
//! `RecoverTitleKey`'s `p_buffer`).
//! 1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
//! ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
//! 2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
//! forward; for the first four steps deduce the LFSR0 output bytes from
//! the keystream (carry-tracked), reconstructing `i_t3`. For the next six
//! steps clock LFSR0 normally and check it reproduces the keystream — a
//! wrong LFSR1 seed fails fast.
//! 3. **Back-clock LFSR0.** Run four backward `i_t3` steps (each a 256-way
//! search for the byte shifted in) to reach the initial state, then undo
//! `i_t3 = i_t3*2 + 8 - (i_t3 & 7)` to recover key[2..5].
//! 4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]` (plain XOR —
//! the descramble seeds directly, so there is no inversion).
//! forward; for the first four steps deduce the LFSR0 output bytes from the
//! keystream (carry-tracked), reconstructing LFSR0's state. For the next six
//! steps clock LFSR0 normally and check it reproduces the keystream — a wrong
//! LFSR1 seed fails fast.
//! 3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
//! the byte shifted in) to reach the initial state, then undo the
//! `r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
//! 4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
//!
//! `AttackPattern` finds known plaintext for step 1: the longest periodic
//! run in the cleartext `sec[0x00..0x80]`, assumed to continue into the
//! encrypted region at 0x80.
//! Known plaintext for step 1 comes from the longest periodic run in the
//! cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
//! 0x80.
use super::lfsr::descramble_sector;
use super::tables::{TAB1, TAB2, TAB3, TAB4, TAB5};
@@ -52,13 +41,11 @@ const ENCRYPTED_START: usize = 0x80; // byte 128
const SEED_OFFSET: usize = 0x54; // sector seed at bytes 0x54-0x58
const FLAG_BYTE: usize = 0x14;
/// RecoverTitleKey: recover the title key from cipher + known plaintext.
///
/// Exact port of libdvdcss `RecoverTitleKey` (css.c). `crypted` is the
/// ciphertext starting at sector byte 0x80; `decrypted` is the matching
/// known plaintext; `seed` is `sector[0x54..0x59]`. On success returns the
/// recovered 5-byte title key; `None` if no LFSR seed reproduces the
/// keystream.
/// Recover the title key from cipher + known plaintext (the core of Stevenson's
/// attack). `crypted` is the ciphertext starting at sector byte 0x80;
/// `decrypted` is the matching known plaintext; `seed` is `sector[0x54..0x59]`.
/// On success returns the recovered 5-byte title key; `None` if no LFSR seed
/// reproduces the keystream.
///
/// At least 10 bytes of `crypted`/`decrypted` are required (the cipher is
/// iterated 10 times: 4 to reconstruct LFSR0, 6 to validate).
@@ -222,16 +209,13 @@ fn descramble_matches(sector: &[u8], title: &[u8; 5], plain: &[u8]) -> bool {
test[ENCRYPTED_START..ENCRYPTED_START + n] == plain[..n]
}
/// AttackPattern: find a repeating pattern just before the encrypted region
/// and assume the plaintext at 0x80 continues it.
///
/// Functionally-equivalent port of libdvdcss `AttackPattern` (css.c) — finds the
/// same periodic cribs on real DVD data, though its byte-comparison anchor
/// differs from the C on phase-misaligned runs. Scans cleartext
/// `sec[0x00..0x80]` for the longest run that repeats with a cycle length in
/// 2..0x2F. If the run is long enough (`plen > 3` and at least two full
/// cycles), the known plaintext at 0x80 is taken to be the periodic run
/// continuing forward, and [`recover_title_key_from_plain`] is applied.
/// Find a repeating pattern just before the encrypted region and assume the
/// plaintext at 0x80 continues it — the known-plaintext step of Stevenson's
/// attack. Scans cleartext `sec[0x00..0x80]` for the longest run that repeats
/// with a cycle length in 2..0x2F. If the run is long enough (`plen > 3` and at
/// least two full cycles), the known plaintext at 0x80 is taken to be the
/// periodic run continuing forward, and [`recover_title_key_from_plain`] is
/// applied.
pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
if sector.len() < SECTOR_BYTES {
return None;
@@ -260,7 +244,7 @@ pub fn crack_title_key(sector: &[u8]) -> Option<[u8; 5]> {
result
}
/// AttackPattern crib: the predicted 10-byte plaintext at byte 0x80.
/// Crib: the predicted 10-byte plaintext at byte 0x80.
///
/// Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
/// run that repeats with a cycle length in 2..0x2F. If the run is long enough
@@ -366,7 +350,7 @@ mod tests {
/// Build a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends
/// in a periodic run that continues into the encrypted region — the case
/// `AttackPattern` (crack_title_key) is designed to crack.
/// `crack_title_key` is designed to crack.
fn synth_periodic_sector(
title_key: &[u8; 5],
seed: &[u8; 5],
@@ -379,7 +363,7 @@ mod tests {
// (RUN_START..0x80) and continuing into the encrypted region. This
// mirrors a real VOB: a periodic data run just before the scrambled
// part. The run must NOT overlap the seed bytes (0x54..0x59), or the
// AttackPattern detector would break mid-run. The phase is anchored to
// the crib detector would break mid-run. The phase is anchored to
// offset 0 so the run is consistent across the 0x80 boundary.
// Just above the seed (0x54..0x59); gives a 39-byte run (0x59..0x80)
// — enough for >=2 cycles of every tested period (<=19).
@@ -466,7 +450,7 @@ mod tests {
}
}
/// MANDATORY (Task C.1): the AttackPattern entry point crack_title_key —
/// MANDATORY (Task C.1): the crib-based entry point crack_title_key —
/// no plaintext supplied — recovers a round-tripping key when the
/// cleartext ends in a periodic run that continues into 0x80.
#[test]
@@ -488,7 +472,7 @@ mod tests {
}
}
/// recover_title_key_from_plain inverts dvdcss_unscramble exactly: scramble
/// recover_title_key_from_plain inverts descramble_sector exactly: scramble
/// a known body, hand back the keystream-derived key, and the recovered
/// key (XOR-back included) reproduces the plaintext.
#[test]
+11 -12
View File
@@ -24,9 +24,8 @@ pub const TAB1: [u8; 256] = [
0xb7, 0xf7, 0xbf, 0xa2, 0xe7, 0xa7, 0xef, 0xf2, 0xba, 0xfa, 0xb2, 0xaf, 0xea, 0xaa, 0xe2, 0xff,
];
/// Table 2: LFSR1 high-byte feedback permutation.
///
/// Byte-identical to libdvdcss `p_css_tab2` (csstables.h).
/// Table 2: LFSR1 high-byte feedback permutation — a fixed constant of the CSS
/// cipher (per the published algorithm).
pub const TAB2: [u8; 256] = [
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x09, 0x08, 0x0b, 0x0a, 0x0d, 0x0c, 0x0f, 0x0e,
0x12, 0x13, 0x10, 0x11, 0x16, 0x17, 0x14, 0x15, 0x1b, 0x1a, 0x19, 0x18, 0x1f, 0x1e, 0x1d, 0x1c,
@@ -46,12 +45,12 @@ pub const TAB2: [u8; 256] = [
0xed, 0xec, 0xef, 0xee, 0xe9, 0xe8, 0xeb, 0xea, 0xe4, 0xe5, 0xe6, 0xe7, 0xe0, 0xe1, 0xe2, 0xe3,
];
/// Table 3: LFSR1 9-bit low-word feedback table (512 entries).
/// Table 3: LFSR1 9-bit low-word feedback table (512 entries) — a fixed constant
/// of the CSS cipher (per the published algorithm).
///
/// Byte-identical to libdvdcss `p_css_tab3` (csstables.h): the 8-value
/// block `BASE[i & 7]` repeated 64 times. The CSS LFSR1 step indexes this
/// table with the 9-bit low register (0x100..=0x1FF), but only the low 3
/// bits select the output — the high bits are ignored, hence the constant
/// It is the 8-value block `BASE[i & 7]` repeated 64 times. The CSS LFSR1 step
/// indexes this table with the 9-bit low register (0x100..=0x1FF), but only the
/// low 3 bits select the output — the high bits are ignored, hence the constant
/// blocks. The 512-entry width simply lets the 9-bit index be used without
/// masking.
pub const TAB3: [u8; 512] = [
@@ -197,12 +196,12 @@ mod tests {
}
}
/// TAB3 is the libdvdcss `p_css_tab3`: the 8-value feedback block
/// TAB3 is the CSS LFSR1 low-word table: the 8-value feedback block
/// BASE = [0x00,0x24,0x49,0x6d,0x92,0xb6,0xdb,0xff]
/// repeated 64 times — `TAB3[i] == BASE[i & 7]`. The high bits of the
/// 9-bit index do not affect the output (libdvdcss's LFSR1 step indexes
/// with the full 9-bit low register but only `& 7` matters). This pins
/// all 512 entries to the published table.
/// 9-bit index do not affect the output (the LFSR1 step indexes with the
/// full 9-bit low register but only `& 7` matters). This pins all 512
/// entries to the published cipher's table.
///
/// Mutation: flip any single byte in the TAB3 literal -> the formula
/// check fails at that index.
+698 -80
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -336,7 +336,7 @@ fn frame_record(track_idx: usize, pts_ns: i64, keyframe: bool, data: &[u8]) -> V
/// `track_number` is the 1-based MKV track number; `track` is the built
/// [`crate::mux::mkv::MkvTrack`] whose fields map one-to-one onto the emitted
/// elements (see `MkvMuxer::new`). No-op unless the diag target is on.
pub fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
pub(crate) fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
if !diag_enabled() {
return;
}
@@ -530,7 +530,7 @@ fn dump_aacs(disc: &Disc) {
a.bus_encryption,
a.mkb_version,
a.disc_hash,
a.key_source.name(),
a.key_source,
a.vuk.is_some(),
a.unit_keys.len(),
a.uk_ro.len(),
+59 -5
View File
@@ -72,6 +72,11 @@ impl Disc {
// Parse each clip for size, duration, and sector extents
let mut extents = Vec::new();
let mut total_size: u64 = 0;
// Set when any clip resolves to a STREAM/SSIF/<clip>.ssif — a Blu-ray 3D
// interleaved stream carrying both the base (left) and MVC dependent
// (right) views. Drives reading the SSIF for both eyes and adding the
// dependent-view stream below.
let mut is_3d = false;
let mut clips = Vec::with_capacity(parsed.play_items.len());
// BD playlists legally reference the same .m2ts clip_id from
// multiple PlayItems (multi-angle, seamless splits, looped
@@ -110,10 +115,23 @@ impl Disc {
// use `.ssif` (see [`CLIP_STREAM_EXTS`]). A normal `.m2ts`
// clip is unchanged — the fallback only runs when `.m2ts`
// is absent, which is exactly when `file_extents` errors.
let file_exts = CLIP_STREAM_EXTS.iter().find_map(|ext| {
let path = format!("/BDMV/STREAM/{}.{}", play_item.clip_id, ext);
udf_fs.file_extents(reader, &path).ok()
});
// 3D discs interleave the left (base) and right (MVC
// dependent) views in STREAM/SSIF/<clip>.ssif — note the
// SSIF/ subdir. Prefer it when present: the SSIF is one
// transport stream carrying BOTH eyes on distinct PIDs,
// so muxing it captures the full 3D. 2D clips fall back to
// the base .m2ts / .fmts as before.
let ssif = format!("/BDMV/STREAM/SSIF/{}.ssif", play_item.clip_id);
let file_exts = match udf_fs.file_extents(reader, &ssif) {
Ok(exts) => {
is_3d = true;
Some(exts)
}
Err(_) => CLIP_STREAM_EXTS.iter().find_map(|ext| {
let path = format!("/BDMV/STREAM/{}.{}", play_item.clip_id, ext);
udf_fs.file_extents(reader, &path).ok()
}),
};
if let Some(file_exts) = file_exts {
for (lba, sectors) in file_exts {
if sectors > 0 && lba > 0 {
@@ -138,7 +156,7 @@ impl Disc {
}
// Build streams from STN table
let streams: Vec<Stream> = parsed
let mut streams: Vec<Stream> = parsed
.streams
.iter()
.filter_map(|s| {
@@ -232,6 +250,42 @@ impl Disc {
})
.collect();
// 3D: add the MVC dependent (right-eye) video stream. The base STN table
// lists only the left-eye video; the dependent view is a second video
// PID (stream_type 0x20) carried in the SSIF. The on-disc PAT/PMT are
// AACS-encrypted (unreadable pre-key) and the base STN omits the
// dependent view (it lives in the MPLS STN_table_SS), so we use the
// BD-3D PID convention: dependent = base-view video PID + 1
// (e.g. 0x1011 -> 0x1012). Reading the SSIF (above) provides its packets.
//
// Limitation: `is_3d` latches per PLAYLIST, not per clip. A playlist that
// mixed a 3D clip (has an SSIF) with a 2D clip (no SSIF) would tag the
// whole title 3D; the 2D clip's frames then mux as plain Blocks (no
// dependent PID → no BlockAdditional) under a track that still advertises
// the mvcC mapping. That output is valid (per-frame BlockAdditional is
// optional) but over-claims 3D for those frames. Real 3D main-feature
// playlists are single-clip or uniformly 3D, so this is not exercised;
// per-clip 3D would need per-clip stream sets (a larger change).
if is_3d {
if let Some(base) = streams.iter().find_map(|s| match s {
Stream::Video(v) => Some(v.clone()),
_ => None,
}) {
let dep_pid = base.pid.wrapping_add(1);
let have_dep = streams
.iter()
.any(|s| matches!(s, Stream::Video(v) if v.pid == dep_pid));
if !have_dep {
streams.push(Stream::Video(VideoStream {
pid: dep_pid,
secondary: true,
label: crate::disc::MVC_DEPENDENT_LABEL.to_string(),
..base
}));
}
}
}
// Convert marks to chapters. mark_type == 1 is an entry-mark
// (chapter); type 2 is a link point and type 0 is reserved, so
// neither is a chapter.
+46 -12
View File
@@ -7,7 +7,6 @@ use crate::udf;
/// Result of SCSI AACS handshake (ECDH authentication).
/// Only available when scanning from a real drive, not ISO images.
#[derive(Debug)]
pub(super) struct HandshakeResult {
pub volume_id: [u8; 16],
pub read_data_key: Option<[u8; 16]>,
@@ -29,6 +28,19 @@ pub(super) struct HandshakeResult {
pub drive_unlocked: bool,
}
// Redacting `Debug`: `volume_id` and `read_data_key` (the AACS 2.0 bus key) are
// secret; print only shape. Guarded by `handshake_result_debug_is_redacted`.
impl std::fmt::Debug for HandshakeResult {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HandshakeResult")
.field("volume_id", &"<redacted>")
.field("read_data_key", &self.read_data_key.map(|_| "<redacted>"))
.field("read_data_key_err", &self.read_data_key_err)
.field("drive_unlocked", &self.drive_unlocked)
.finish()
}
}
/// Single source of truth for "is AACS bus encryption gone for this scan?". The
/// gate asks ONLY this — `if !removed { error }` — never enumerating cases. Bus
/// encryption is gone when ANY of these holds:
@@ -311,18 +323,19 @@ impl Disc {
) -> Result<AacsState> {
use crate::aacs;
let uk_ro_data = udf_fs
.read_file(reader, crate::aacs::PATH_UNIT_KEY_RO)
.or_else(|_| udf_fs.read_file(reader, crate::aacs::PATH_UNIT_KEY_RO_DUPLICATE))
.map_err(|_| Error::AacsNoKeys)?;
let uk_ro_data =
aacs::read_first(&aacs::role_paths(udf_fs, aacs::AacsRole::UnitKey), |p| {
udf_fs.read_file(reader, p)
})?;
let dh = aacs::inf::disc_hash(&uk_ro_data);
let cc = udf_fs
.read_file(reader, crate::aacs::PATH_CONTENT_CERT)
.or_else(|_| udf_fs.read_file(reader, crate::aacs::PATH_CONTENT_CERT_ALT))
.ok()
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let cc = aacs::read_first(
&aacs::role_paths(udf_fs, aacs::AacsRole::ContentCert),
|p| udf_fs.read_file(reader, p),
)
.ok()
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let bus_encryption = cc.as_ref().map(|c| c.bus_encryption).unwrap_or(false);
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so the
// scanned `AacsState.version` and the out-of-band fetch agree. A wrong
@@ -432,6 +445,27 @@ mod tests {
use crate::sector::SectorSource;
use std::collections::HashMap;
/// `HandshakeResult` carries the Volume ID and the AACS 2.0 bus (read-data)
/// key; `Debug` must redact both. Sentinel 213 (0xD5).
#[test]
fn handshake_result_debug_is_redacted() {
let hs = HandshakeResult {
volume_id: [0xD5; 16],
read_data_key: Some([0xD5; 16]),
read_data_key_err: None,
drive_unlocked: false,
};
let d = format!("{hs:?}");
assert!(
!d.contains("213"),
"HandshakeResult leaked VID/bus key: {d}"
);
assert!(
d.contains("redacted"),
"HandshakeResult missing marker: {d}"
);
}
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical
// partition (metadata_start == partition_start). Offsets cited
@@ -980,7 +1014,7 @@ mod tests {
/// A minimal in-test KeySource that yields no keys but a fixed cert list.
struct CertSource(Vec<aacs::types::HostCert>);
impl crate::KeySource for CertSource {
fn get_uk(
fn get_unit_keys(
&self,
_ctx: &dyn crate::keysource::ResolveCtx,
) -> Result<Vec<crate::aacs::types::UnitKey>> {
-57
View File
@@ -288,63 +288,6 @@ impl Disc {
}
}
/// True for the AACS-encrypted stream files (`.m2ts`, `.ssif`). Every other UDF
/// file is clear (nav / playlists / filesystem) and needs no decrypt verify.
fn is_aacs_clip(name: &str) -> bool {
let lower = name.to_ascii_lowercase();
lower.ends_with(".m2ts") || lower.ends_with(".ssif")
}
/// Enumerate the disc's AACS clip (`.m2ts`/`.ssif`) files as
/// [`crate::disc::verify::ClipLayout`]s for the post-read verify gate: each
/// clip's declared size plus its absolute disc extents in FILE order. Reads the
/// UDF tree through `reader`.
///
/// FAIL-SAFE: any enumeration error (bad UDF read, name collision, …) yields an
/// EMPTY list — the verify gate then covers nothing and the sweep behaves as
/// today. Enumeration must never break a rip, so the error is logged, not
/// propagated.
pub(crate) fn clip_layouts(reader: &mut dyn SectorSource) -> Vec<crate::disc::verify::ClipLayout> {
let result = (|| -> Result<Vec<crate::disc::verify::ClipLayout>> {
let fs = udf::read_filesystem(reader)?;
let mut planned: Vec<PlannedFile> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
let mut seen_hosts: std::collections::HashMap<PathBuf, String> =
std::collections::HashMap::new();
plan_tree(
reader,
&fs,
&fs.root,
Path::new(""),
"",
true,
&mut planned,
&mut dirs,
&mut seen_hosts,
)?;
Ok(planned
.into_iter()
.filter(|pf| pf.inline.is_none() && is_aacs_clip(&pf.disc_name))
.map(|pf| crate::disc::verify::ClipLayout {
size: pf.size,
extents: pf.extents,
// Every AACS clip we enumerate today is BD-TS (`.m2ts`/`.ssif`).
// HD-DVD `.evo` (program stream) maps to `ContainerKind::Ps` here
// once `is_aacs_clip` recognises it — the one-line HD-DVD hook.
container: crate::disc::verify::ContainerKind::Ts,
})
.collect())
})();
result.unwrap_or_else(|e| {
tracing::warn!(
target: "freemkv::verify",
error = %e,
"clip enumeration failed; post-read verify disabled for this pass"
);
Vec::new()
})
}
/// A borrowing `SectorSource` wrapper. Lets the decrypting decorator "own" an
/// inner source for its lifetime while the caller keeps the underlying
/// `&mut dyn SectorSource` (the decorator is a `DecryptingSectorSource<S>`
+446 -18
View File
@@ -9,26 +9,28 @@
//! rule: a genuinely different format is a new enumerator, not an extension
//! bolted into the BD path.
//!
//! Title composition: the `HVA*.VTI` navigation file (the DVD-IFO analogue) names
//! every clip in authored order. Standard Content splits the main feature across
//! clips at the layer break (`FEATURE_1`/`FEATURE_2`, or `feature`/`feature_Divide`);
//! the scanner parses the VTI clip table and concatenates those parts into ONE
//! [`DiscTitle`] (so the largest-title pick is the whole movie, not just part 1),
//! emitting every other clip as its own title. Container is
//! [`ContentFormat::MpegPs`], so the existing PS mux path handles it. Per-clip
//! streams ARE enumerated: the clip head is demuxed through the PS demuxer and one
//! [`Stream`] is built per distinct elementary stream (video + DD+ audio
//! sub-streams), with the codec sniffed from the ES bytes.
//! Title composition — authoritative, from the Advanced-Content playlist. HD-DVD
//! ships a real player playlist at `ADV_OBJ/VPLST000.XPL` (DVD-Forum
//! `HDDVDVideo/Playlist` XML). The scanner parses it (with a real XML parser,
//! `roxmltree`) into one [`DiscTitle`] per `<Title>`: its `<PrimaryAudioVideoClip>`
//! clips in playback order (each an EVO, referenced via its `.MAP` sidecar), the
//! `titleDuration`, the `displayName`, and the `<ChapterList>`. A layer-break
//! split (`FEATURE_1` + `FEATURE_2`, or `feature`/`feature_Divide`) is composed
//! into ONE title with the two parts as clips, carrying each clip's title-time
//! in/out points (45 kHz ticks) so a seamless join can be spliced onto one
//! timeline. Container is [`ContentFormat::MpegPs`], so the existing PS mux path
//! handles it. Per-clip streams are enumerated by demuxing the clip head and
//! building one [`Stream`] per distinct elementary stream (video + DD+ audio),
//! codec sniffed from the ES bytes.
//!
//! What is NOT parsed yet — and is honestly stubbed, not faked:
//! * full VTI program-chain parsing (chapters, non-feature title grouping) —
//! the feature join uses the clip table + the `feature*` naming convention,
//! not the authoritative PGC,
//! * `.map` timemap → real durations,
//! * subtitles (8-bit RLC on `0xBD` sub `0x20..=0x3F`).
//! When no playlist is present (or it fails to parse), the scanner falls back to
//! the older clip-name heuristic: parse the `HVA*.VTI` clip table, join the
//! `feature*`-named clips into one title, and emit every other clip on its own.
//!
//! Extents and size ARE real (the ripper needs those to image a clip); durations
//! and chapters are left empty rather than guessed.
//! Not parsed yet: subtitles (8-bit RLC on `0xBD` sub `0x20..=0x3F`) and per-track
//! audio languages (the XPL carries `<Audio description=...>` but they are not yet
//! wired onto the streams). Extents and size are real (the ripper images the
//! clips).
use super::*;
use crate::mux::ps::{PsDemuxer, dvd_audio_pid};
@@ -316,6 +318,231 @@ fn collect_es(
}
}
// ─────────────────────── Advanced-Content playlist (XPL) ──────────────────
//
// HD-DVD Advanced Content ships an authoritative playlist at
// `ADV_OBJ/VPLST000.XPL` (DVD-Forum `HDDVDVideo/Playlist` XML). It is the real
// player playlist: a `<TitleSet>` of `<Title>`s, each naming its
// `<PrimaryAudioVideoClip>` clips (an EVO, via its `.MAP` sidecar) in playback
// order with title-time in/out points, a `titleDuration`, a `displayName`, and a
// `<ChapterList>`. Parsing it gives authoritative title composition — clips,
// duration, name, chapters — instead of the `feature*` clip-name heuristic, plus
// the per-clip title-time offsets needed to splice a layer-break split
// (`FEATURE_1` + `FEATURE_2`) onto one continuous timeline. Parsed with a real
// XML parser (`roxmltree`), not a hand-rolled scanner — the XPL is genuine XML.
/// One clip reference inside an XPL `<Title>`: the resolved `.evo` name (lower
/// case) and the clip's placement on the title timeline, in seconds.
struct XplClip {
evo: String,
begin_secs: f64,
end_secs: f64,
}
/// One `<Title>` from the XPL: number, display name, total duration, its clips
/// in playback order, and chapter start times (seconds).
struct XplTitle {
number: u16,
name: String,
duration_secs: f64,
clips: Vec<XplClip>,
chapters: Vec<f64>,
}
/// Parse an `HH:MM:SS:FF` (or `MM:SS:FF`) timecode at `tick_base` frames/sec into
/// seconds. `None` on a malformed field.
fn parse_timecode(s: &str, tick_base: u32) -> Option<f64> {
let n: Vec<u32> = s
.split(':')
.map(|p| p.trim().parse::<u32>())
.collect::<std::result::Result<Vec<u32>, _>>()
.ok()?;
let tb = tick_base.max(1) as f64;
let (h, m, sec, f) = match n.as_slice() {
[h, m, s, f] => (*h, *m, *s, *f),
[m, s, f] => (0, *m, *s, *f),
_ => return None,
};
Some(h as f64 * 3600.0 + m as f64 * 60.0 + sec as f64 + f as f64 / tb)
}
/// `tickBase="60fps"` → 60. Defaults to 60 when absent/unparseable.
fn parse_tick_base(s: &str) -> u32 {
let digits: String = s.chars().take_while(|c| c.is_ascii_digit()).collect();
digits.parse().unwrap_or(60)
}
/// `<PrimaryAudioVideoClip src="file:///.../FEATURE_1.MAP">` → `feature_1.evo`:
/// take the basename, drop the extension, normalise to a lower-case `.evo` name
/// (the playlist references the `.MAP` sidecar; the A/V is the same-stem `.EVO`).
fn evo_from_src(src: &str) -> Option<String> {
let base = src.rsplit(['/', '\\']).next().unwrap_or(src);
let stem = base.rsplit_once('.').map(|(s, _)| s).unwrap_or(base);
if stem.is_empty() {
return None;
}
Some(format!("{}.evo", stem.to_ascii_lowercase()))
}
/// Parse the Advanced-Content playlist into its titles. Elements are matched by
/// LOCAL name (the document is in the `HDDVDVideo/Playlist` default namespace).
/// Returns empty for a non-XML / non-playlist blob so the caller falls back to
/// the clip-name heuristic.
fn parse_xpl_titles(xpl: &[u8]) -> Vec<XplTitle> {
let text = String::from_utf8_lossy(xpl);
let Ok(doc) = roxmltree::Document::parse(&text) else {
return Vec::new();
};
let local = |n: &roxmltree::Node, name: &str| n.tag_name().name() == name;
// tickBase lives on <TitleSet> (default 60fps).
let tick_base = doc
.descendants()
.find(|n| local(n, "TitleSet"))
.and_then(|n| n.attribute("tickBase"))
.map(parse_tick_base)
.unwrap_or(60);
let mut titles = Vec::new();
for tnode in doc.descendants().filter(|n| local(n, "Title")) {
let number = tnode
.attribute("titleNumber")
.and_then(|s| s.trim().parse::<u16>().ok())
.unwrap_or(0);
let name = tnode
.attribute("displayName")
.or_else(|| tnode.attribute("id"))
.unwrap_or("")
.to_string();
let duration_secs = tnode
.attribute("titleDuration")
.and_then(|s| parse_timecode(s, tick_base))
.unwrap_or(0.0);
let mut clips = Vec::new();
for c in tnode
.descendants()
.filter(|n| local(n, "PrimaryAudioVideoClip"))
{
let Some(evo) = c.attribute("src").and_then(evo_from_src) else {
continue;
};
let begin_secs = c
.attribute("titleTimeBegin")
.and_then(|s| parse_timecode(s, tick_base))
.unwrap_or(0.0);
let end_secs = c
.attribute("titleTimeEnd")
.and_then(|s| parse_timecode(s, tick_base))
.unwrap_or(begin_secs);
clips.push(XplClip {
evo,
begin_secs,
end_secs,
});
}
if clips.is_empty() {
continue;
}
let chapters = tnode
.descendants()
.filter(|n| local(n, "Chapter"))
.filter_map(|ch| {
ch.attribute("titleTimeBegin")
.and_then(|s| parse_timecode(s, tick_base))
})
.collect();
titles.push(XplTitle {
number,
name,
duration_secs,
clips,
chapters,
});
}
titles
}
/// Read the Advanced-Content playlist `ADV_OBJ/VPLST*.XPL`, if present.
fn read_adv_obj_xpl(reader: &mut dyn SectorSource, udf_fs: &udf::UdfFs) -> Option<Vec<u8>> {
let dir = udf_fs.find_dir("/ADV_OBJ")?;
let name = dir.entries.iter().find_map(|e| {
let lower = e.name.to_ascii_lowercase();
(!e.is_dir && lower.starts_with("vplst") && lower.ends_with(".xpl")).then(|| e.name.clone())
})?;
udf_fs.read_file(reader, &format!("/ADV_OBJ/{name}")).ok()
}
/// Compose [`DiscTitle`]s from parsed XPL titles: resolve each title's clips to
/// physical extents, concatenate them in playback order, carry the title-time
/// in/out points onto each [`Clip`] (45 kHz ticks — the offset that splices a
/// layer-break split onto one timeline), and attach the duration, name, and
/// chapters. A title whose clips resolve to no on-disc extents is skipped.
fn compose_xpl_titles(
reader: &mut dyn SectorSource,
xpl_titles: &[XplTitle],
clip_extents: &BTreeMap<String, (String, u64, Vec<Extent>)>,
) -> Vec<DiscTitle> {
let mut titles = Vec::new();
for t in xpl_titles {
let mut extents = Vec::new();
let mut size_bytes = 0u64;
let mut parts = Vec::new();
for c in &t.clips {
let Some((orig, size, exts)) = clip_extents.get(&c.evo) else {
continue;
};
extents.extend_from_slice(exts);
size_bytes += *size;
parts.push(Clip {
clip_id: orig
.rsplit_once('.')
.map(|(b, _)| b)
.unwrap_or(orig)
.to_string(),
in_time: (c.begin_secs * 45000.0).clamp(0.0, u32::MAX as f64) as u32,
out_time: (c.end_secs * 45000.0).clamp(0.0, u32::MAX as f64) as u32,
duration_secs: (c.end_secs - c.begin_secs).max(0.0),
source_packets: 0,
});
}
if parts.is_empty() {
continue;
}
let streams = probe_evo_streams(reader, &extents);
let chapters = t
.chapters
.iter()
.enumerate()
.map(|(i, &ts)| Chapter {
time_secs: ts.max(0.0),
name: super::chapter_name(i),
})
.collect();
titles.push(DiscTitle {
// Language-neutral identifier (no user-facing English in the library):
// matches the UDF `TITLE_*` volume-label style. Apps localize display.
playlist: if t.name.is_empty() {
format!("TITLE_{}", t.number)
} else {
t.name.clone()
},
playlist_id: t.number,
duration_secs: t.duration_secs,
size_bytes,
clips: parts,
streams,
chapters,
extents,
content_format: ContentFormat::MpegPs,
codec_privates: Vec::new(),
});
}
titles
}
impl Disc {
/// Scan HD-DVD titles from the `HVDVD_TS/` `.evo` clips.
///
@@ -375,6 +602,18 @@ impl Disc {
}
}
// Authoritative composition from the Advanced-Content playlist
// (`ADV_OBJ/VPLST*.XPL`): real per-title clip lists, durations, names,
// chapters, and title-time offsets. This is the primary path; the
// clip-name heuristic below is the fallback when the playlist is absent
// or unparseable (or resolves to no on-disc clips).
if let Some(xpl) = read_adv_obj_xpl(reader, udf_fs) {
let composed = compose_xpl_titles(reader, &parse_xpl_titles(&xpl), &clip_extents);
if !composed.is_empty() {
return composed;
}
}
// Feature clips, in authored order, that actually resolved to extents.
let feature: Vec<String> = order
.iter()
@@ -966,4 +1205,193 @@ mod tests {
"VC-1 routes to the extended-stream-id PID 0xFD55"
);
}
// ─────────────────────── Advanced-Content playlist (XPL) ──────────────
/// A minimal but faithful VPLST000.XPL: the DVD-Forum default namespace, an
/// XML comment, a `<TitleSet tickBase="60fps">`, a MainMovie title whose main
/// feature is a two-clip layer-break split (FEATURE_1 + FEATURE_2, seamless)
/// with two chapters, and a separate deleted-scene title. Attribute order
/// varies (as it does across real discs).
const SYNTH_XPL: &str = r#"<?xml version="1.0" encoding="UTF-8"?>
<Playlist majorVersion="1" minorVersion="0" xmlns="http://www.dvdforum.org/2005/HDDVDVideo/Playlist">
<!-- Authored with TOSHIBA AdvMain -->
<TitleSet timeBase="60fps" tickBase="60fps" defaultLanguage="en">
<Title titleNumber="2" titleDuration="01:37:20:00" id="MainMovie" displayName="Main Movie">
<PrimaryAudioVideoClip titleTimeBegin="00:00:00:00" titleTimeEnd="00:48:29:50" src="file:///dvddisc/HVDVD_TS/FEATURE_1.MAP" dataSource="Disc">
<Video track="1" mediaAttr="2"/>
<Audio track="1" streamNumber="1" description="English DD+"/>
</PrimaryAudioVideoClip>
<PrimaryAudioVideoClip titleTimeBegin="00:48:29:50" clipTimeBegin="00:00:00:00" titleTimeEnd="01:37:20:00" src="file:///dvddisc/HVDVD_TS/FEATURE_2.MAP" seamless="true">
<Video track="1" mediaAttr="2"/>
</PrimaryAudioVideoClip>
<ChapterList>
<Chapter displayName="Chapter 1" titleTimeBegin="00:00:00:00" />
<Chapter displayName="Chapter 2" titleTimeBegin="00:03:40:30" />
</ChapterList>
</Title>
<Title titleNumber="7" titleDuration="00:00:44:29" id="Deleted5" displayName="Deleted Scenes - Veronica Past">
<PrimaryAudioVideoClip titleTimeBegin="00:00:00:00" titleTimeEnd="00:00:44:29" src="file:///dvddisc/HVDVD_TS/DEL5_VERONICAPAST.MAP" />
</Title>
</TitleSet>
</Playlist>"#;
#[test]
fn parse_timecode_hhmmssff_at_60fps() {
// 01:37:20:00 = 5840 s exactly.
assert!((parse_timecode("01:37:20:00", 60).unwrap() - 5840.0).abs() < 1e-6);
// 00:48:29:50 = 48m29s + 50/60 frames.
let want = 48.0 * 60.0 + 29.0 + 50.0 / 60.0;
assert!((parse_timecode("00:48:29:50", 60).unwrap() - want).abs() < 1e-6);
// MM:SS:FF short form (hours omitted).
assert!((parse_timecode("02:05:15", 60).unwrap() - (125.0 + 15.0 / 60.0)).abs() < 1e-6);
assert_eq!(parse_timecode("garbage", 60), None);
assert_eq!(parse_timecode("", 60), None);
}
#[test]
fn evo_from_src_maps_map_sidecar_to_evo() {
assert_eq!(
evo_from_src("file:///dvddisc/HVDVD_TS/FEATURE_1.MAP").as_deref(),
Some("feature_1.evo")
);
// Already an EVO, or lowercase feature — normalise to lower `.evo`.
assert_eq!(evo_from_src("feature.EVO").as_deref(), Some("feature.evo"));
assert_eq!(
evo_from_src("file:///x/feature_Divide.MAP").as_deref(),
Some("feature_divide.evo")
);
// Empty stem → None (defensive against a malformed src).
assert_eq!(evo_from_src("file:///x/").as_deref(), None);
}
#[test]
fn parse_xpl_titles_reads_titles_clips_chapters_durations() {
let titles = parse_xpl_titles(SYNTH_XPL.as_bytes());
assert_eq!(titles.len(), 2, "MainMovie + one deleted-scene title");
let mm = &titles[0];
assert_eq!(mm.number, 2);
assert_eq!(mm.name, "Main Movie");
assert!(
(mm.duration_secs - 5840.0).abs() < 1e-6,
"97:20 from titleDuration"
);
// The layer-break split is ONE title with TWO clips, contiguous timeline.
assert_eq!(mm.clips.len(), 2);
assert_eq!(mm.clips[0].evo, "feature_1.evo");
assert_eq!(mm.clips[1].evo, "feature_2.evo");
assert!(
(mm.clips[0].end_secs - mm.clips[1].begin_secs).abs() < 1e-6,
"FEATURE_2 begins exactly where FEATURE_1 ends (seamless join)"
);
assert!(
mm.clips[1].begin_secs > 0.0,
"second clip carries a title-time offset"
);
assert_eq!(mm.chapters.len(), 2);
assert!((mm.chapters[1] - (3.0 * 60.0 + 40.0 + 30.0 / 60.0)).abs() < 1e-6);
let del = &titles[1];
assert_eq!(del.name, "Deleted Scenes - Veronica Past");
assert_eq!(del.clips.len(), 1);
assert_eq!(del.clips[0].evo, "del5_veronicapast.evo");
}
#[test]
fn parse_xpl_titles_returns_empty_on_non_xml() {
assert!(parse_xpl_titles(b"not xml at all").is_empty());
assert!(parse_xpl_titles(&[0xFF, 0x00, 0x01, 0x02]).is_empty());
assert!(parse_xpl_titles(b"<Playlist></Playlist>").is_empty());
}
/// Build a UDF with `HVDVD_TS/` `.evo` clips plus an `ADV_OBJ/VPLST000.XPL`
/// carrying `xpl`, so `scan_hddvd_titles` takes the playlist path.
fn make_hddvd_fs_xpl(
disc: &mut MemDisc,
evos: &[(&str, u32, u32)],
xpl: &[u8],
) -> crate::udf::UdfFs {
let mut hv_files = Vec::new();
let mut icb = 100u32;
for (name, sectors, data_lba) in evos {
hv_files.push(file(name, icb, *data_lba, sectors * 2048, true));
icb += 1;
}
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![
DirSpec {
name: "HVDVD_TS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: hv_files,
subdirs: vec![],
},
DirSpec {
name: "ADV_OBJ".to_string(),
icb_lba: 30,
dir_data_lba: 31,
files: vec![file_with("VPLST000.XPL", 40, 4000, xpl.to_vec(), true)],
subdirs: vec![],
},
],
};
build_udf_skeleton(disc, 10);
lay_dir(disc, &root);
crate::udf::read_filesystem(disc).expect("fs")
}
/// The authoritative path: when a VPLST000.XPL is present, titles come from
/// the playlist — the layer-break split (FEATURE_1 + FEATURE_2) is composed
/// into ONE title with the real duration, name, chapters, and per-clip
/// title-time offsets — not the clip-name heuristic.
#[test]
fn scan_hddvd_composes_titles_from_xpl_playlist() {
let mut disc = MemDisc::new();
let udf = make_hddvd_fs_xpl(
&mut disc,
&[
("FEATURE_1.EVO", 2000, 5000),
("FEATURE_2.EVO", 1800, 9000),
("DEL5_VERONICAPAST.EVO", 100, 12000),
],
SYNTH_XPL.as_bytes(),
);
let titles = Disc::scan_hddvd_titles(&mut disc, &udf);
assert_eq!(
titles.len(),
2,
"the two playlist titles that resolve to clips"
);
let mm = titles
.iter()
.find(|t| t.playlist == "Main Movie")
.expect("MainMovie composed from the playlist");
assert_eq!(mm.playlist_id, 2);
assert!(
(mm.duration_secs - 5840.0).abs() < 1.0,
"97:20 duration from titleDuration, not 0/unknown"
);
assert_eq!(
mm.clips.len(),
2,
"layer-break split kept as ONE title, two clips"
);
// FEATURE_2's clip carries the 48:29 offset (45 kHz ticks) — the datum
// that splices it onto FEATURE_1's timeline instead of restarting at 0.
assert!(
mm.clips[1].in_time > 100_000_000,
"FEATURE_2 offset onto the title timeline (48:29 * 45000), got {}",
mm.clips[1].in_time
);
assert_eq!(mm.chapters.len(), 2);
assert_eq!(mm.chapters[0].name, "1", "bare ordinal chapter name");
// Both feature halves are in ONE title's extents.
assert!(!mm.extents.is_empty());
}
}
+2 -2
View File
@@ -77,7 +77,7 @@ impl SectorStatus {
/// One contiguous range of bytes with a status.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MapEntry {
pub(crate) struct MapEntry {
pub pos: u64,
pub size: u64,
pub status: SectorStatus,
@@ -498,7 +498,7 @@ impl Mapfile {
/// All map entries, sorted ascending by `pos` and (after load)
/// guaranteed disjoint and non-overflowing.
pub fn entries(&self) -> &[MapEntry] {
pub(crate) fn entries(&self) -> &[MapEntry] {
&self.entries
}
+695 -408
View File
File diff suppressed because it is too large Load Diff
+32 -86
View File
@@ -405,9 +405,15 @@ pub(super) fn compute_initial_state(
bad_ranges.reverse();
}
let work_total: u64 = bad_ranges.iter().map(|(_, sz)| *sz).sum();
// Fail SAFE when metadata is indeterminate: assume a regular file so a
// real `sync_all` failure is surfaced, not swallowed. `/dev/null` and pipes
// report success-with-non-file here (so they still correctly map to
// `false`); only a genuine metadata error (e.g. transient NFS ESTALE) hits
// the default, and for a data-integrity guard "surface the error" is the
// right side to err on.
let is_regular = std::fs::metadata(path)
.map(|m| m.file_type().is_file())
.unwrap_or(false);
.unwrap_or(true);
Ok((
map,
initial_stats,
@@ -1247,7 +1253,24 @@ impl Disc {
pub fn bytes_bad_in_title(&self, mapfile_path: &std::path::Path, title: &DiscTitle) -> u64 {
let map = match mapfile::Mapfile::load(mapfile_path) {
Ok(m) => m,
Err(_) => return 0,
// A MISSING mapfile is legitimate (no damage was ever tracked — e.g. a
// clean single-pass rip): 0 bad bytes is correct. Any OTHER load error
// (corrupt / unreadable mapfile) means we CANNOT know the damage — and
// a returned 0 reads to the caller as "clean." Logging alone is not
// fail-safe: the RETURN VALUE drives the loss/abort accounting, not the
// log. So fail safe by reporting the ENTIRE title as bad (its full
// in-extent byte count) — a corrupt damage record must surface as
// maximal loss, never as a clean rip.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return 0,
Err(e) => {
tracing::warn!(
target: "freemkv::disc",
path = %mapfile_path.display(),
error = %e,
"bytes_bad_in_title: mapfile load failed; reporting whole title bad (fail-safe: cannot confirm clean)"
);
return bytes_bad_in_title(title, &[(0, u64::MAX)]);
}
};
let bad_ranges = map.ranges_with(&[
mapfile::SectorStatus::NonTrimmed,
@@ -1299,33 +1322,13 @@ impl Disc {
let bytes_good_before = initial_stats.bytes_good;
let bytes_good_start = bytes_good_before;
// Post-read verify gate for the patch pass (ciphertext multipass only,
// `!opts.decrypt`). Built here from the raw reader's UDF enumeration;
// reused AFTER the recovery loop (`reverify_iso`) to re-check the units
// this pass touched by reading them WHOLE back from the patched ISO —
// patch re-reads only the bad sectors of a unit, so per-unit verify
// can't run live. Fail-safe `None` when disabled / non-AACS / no keys.
let mut verifier = if opts.decrypt {
None
} else {
let verify_keys = self.decrypt_keys();
let layouts = crate::disc::extract::clip_layouts(&mut *reader);
crate::disc::verify::UnitVerifier::new(&layouts, &verify_keys, opts.key_fetch.clone())
};
// Decrypt-aware read — symmetric with `Disc::sweep`. A decrypting patch
// (`opts.decrypt`) decrypts in place (plaintext ISO). A NON-decrypting
// patch (the multipass / `--raw --multipass` path) resolves the keys and
// VERIFIES each unit on a scratch copy: a re-read that STILL won't decrypt
// fails the read (`DECRYPT_VERIFY_READ`) and stays NonTrimmed, so the
// retry loop keeps re-reading it "until it decrypts or retries exhaust"
// exactly as for a SCSI read error — and a unit that DOES decrypt on a
// fresh read (the drive returned different bytes) is recovered for free.
// With no usable AACS keys this degrades to a plain pass-through.
// Symmetric with `Disc::sweep`: the patch COPIES ciphertext (multipass /
// `--raw`) or decrypts IN PLACE (`opts.decrypt`). It does NOT decrypt-
// VERIFY — the disc-absolute read can't anchor to a clip's file-relative
// unit grid (see `Disc::sweep` + `Disc::verify_clips`). Re-reads recover
// bad sectors; the clip-anchored verify pass re-checks them afterward.
// (`opts.decrypt`) decrypts in place (plaintext ISO); a NON-decrypting
// patch (the multipass / `--raw --multipass` path) copies ciphertext
// verbatim (keys = `None` → pass-through). Bad sectors are found by
// PHYSICAL read success, not by decrypt structure: a re-read that returns
// good bytes recovers the range; a read that errors leaves it NonTrimmed
// for the next pass. (The old decrypt-VERIFY read gate was removed.)
let keys = if opts.decrypt {
self.decrypt_keys()
} else {
@@ -1435,64 +1438,7 @@ impl Disc {
// sink's summary. `close` failing on a regular-file sync_all is
// surfaced here as `Error::IoError`, matching pre-split
// behaviour.
let mut summary = pipe.finish()?;
// Scoped post-read re-verify (decrypt-fail == bad read). The consumer
// has flushed the ISO + mapfile; re-read each clip unit this pass touched
// WHOLE from the patched ISO and downgrade any that still won't decrypt
// to NonTrimmed, so the orchestrator's end-of-recovery promotion
// terminalizes it. Reuses the same verifier as the sweep. Fail-safe:
// disabled gate / unreadable ISO / load failure all leave the pass as-is.
if let Some(mut v) = verifier.take() {
if let Ok(mut m) = mapfile::Mapfile::load(&mapfile_path) {
// Only units whose every backing sector was actually READ
// (Finished) may be re-verified — we can't verify what wasn't read
// (a non-Finished sector is zero-filled because the read failed),
// and must not waste a key lookup on a known-bad block.
let finished = m.ranges_with(&[mapfile::SectorStatus::Finished]);
let is_finished = |lba: u32| -> bool {
let p = lba as u64 * 2048;
finished.iter().any(|&(s, sz)| p >= s && p < s + sz)
};
if let Ok(mut iso) = crate::io::file_sector_source::FileSectorSource::open(path) {
let bad = v.reverify_iso(&mut iso, &bad_ranges, &is_finished);
if !bad.is_empty() {
let n: usize = bad.len();
for (lba, cnt) in bad {
if let Err(e) = m.record(
lba as u64 * 2048,
cnt as u64 * 2048,
mapfile::SectorStatus::NonTrimmed,
) {
tracing::warn!(
lba,
"reverify downgrade: mapfile record failed ({e}) — unit may stay mismarked as good"
);
}
}
if let Err(e) = m.flush() {
tracing::warn!(
"reverify downgrade: mapfile flush failed ({e}) — downgrade not persisted; a resume could mismark it good"
);
}
// The re-verify ran AFTER `pipe.finish()` snapshotted
// `summary.stats`, so those stats still count the just-
// downgraded units as good. Refresh from the mapfile so
// `build_outcome` reports the true post-downgrade picture
// (bytes_good ↓, bytes_pending ↑) — otherwise the caller
// over-reports recovery and can call an imperfect rip
// "complete".
summary.stats = m.stats();
tracing::info!(
target: "freemkv::verify",
phase = "patch.reverify",
downgraded_ranges = n,
"post-read re-verify downgraded undecryptable units to NonTrimmed"
);
}
}
}
}
let summary = pipe.finish()?;
let outcome = build_outcome(
&state,
+196
View File
@@ -0,0 +1,196 @@
//! Content-based forced-subtitle detection for Blu-ray/UHD PGS tracks.
//!
//! `freemkv info` and the muxer must agree on which subtitle tracks are forced.
//! The muxer derives it from the PGS `forced_on_flag` while muxing a rip; this
//! module gives `info` the SAME verdict up front by reading the title's PGS
//! streams and feeding them through the one shared classifier
//! ([`crate::mux::codec::pgs::ForcedTracker`]) — so the two never diverge.
//!
//! Cost: a track is only confirmed forced once EVERY display set is seen to be
//! forced, so a disc that has a forced track is read through — the
//! accuracy-over-speed tradeoff `info` opts into. Full tracks early-exit as soon
//! as they show a single non-forced subtitle, and a whole run stops early once
//! every track has settled.
//!
//! Encrypted content: the probe reuses whatever [`SectorSource`] the scan holds.
//! With a decrypting source it sees real PGS; without keys it reads ciphertext
//! and observes no display sets, in which case it leaves each track's existing
//! (vendor-label-derived) forced flag untouched rather than asserting anything.
use crate::disc::{Codec, DiscTitle, Stream};
use crate::mux::codec::CodecParser;
use crate::mux::codec::pgs::{ForcedTracker, PgsParser};
use crate::mux::ts::TsDemuxer;
use crate::sector::SectorSource;
use std::collections::HashMap;
const SECTOR_BYTES: usize = 2048;
/// Read the clip in 2 MiB chunks.
const CHUNK_SECTORS: u16 = 1024;
/// Read the title's PGS streams and set `SubtitleStream::forced` from their
/// content. Best-effort: any read error ends the probe with whatever verdicts
/// have accumulated. Only PGS tracks are touched (DVD VobSub forced comes from
/// the IFO/vendor path).
pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
reader: &mut S,
title: &mut DiscTitle,
) {
let pg_pids: Vec<u16> = title
.streams
.iter()
.filter_map(|s| match s {
Stream::Subtitle(sub) if sub.codec == Codec::Pgs => Some(sub.pid),
_ => None,
})
.collect();
if pg_pids.is_empty() {
return;
}
let mut demux = TsDemuxer::new(&pg_pids);
let mut parsers: HashMap<u16, PgsParser> =
pg_pids.iter().map(|&p| (p, PgsParser::new())).collect();
let mut trackers: HashMap<u16, ForcedTracker> =
pg_pids.iter().map(|&p| (p, ForcedTracker::new())).collect();
let extents = title.extents.clone();
let mut buf = vec![0u8; CHUNK_SECTORS as usize * SECTOR_BYTES];
'outer: for ext in &extents {
let mut lba = ext.start_lba;
let mut remaining = ext.sector_count;
while remaining > 0 {
let count = remaining.min(CHUNK_SECTORS as u32) as u16;
let want = count as usize * SECTOR_BYTES;
let n = match reader.read_sectors(lba, count, &mut buf[..want], false) {
Ok(n) => n,
Err(_) => break 'outer, // best-effort — stop, keep what we have
};
if n == 0 {
break 'outer;
}
for pes in demux.feed(&buf[..n]) {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
{
for frame in parser.parse(&pes) {
tracker.observe(&frame.data);
}
}
}
// Every track has already shown a non-forced set → nothing left to
// learn; stop reading the (huge) clip.
if trackers.values().all(ForcedTracker::settled_not_forced) {
break 'outer;
}
lba += count as u32;
remaining -= count as u32;
}
}
// Drain any buffered final display set.
for (pid, parser) in parsers.iter_mut() {
if let Some(tracker) = trackers.get_mut(pid) {
for frame in parser.flush() {
tracker.observe(&frame.data);
}
}
}
// Apply verdicts. Only override a track we actually saw content for — an
// undecrypted/unread track keeps its vendor-derived flag.
for s in &mut title.streams {
if let Stream::Subtitle(sub) = s {
if sub.codec == Codec::Pgs {
if let Some(t) = trackers.get(&sub.pid) {
if t.observed() {
sub.forced = t.is_forced();
}
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{ContentFormat, Extent, LabelQualifier, SubtitleStream};
/// A reader that yields all-zeros (an encrypted / unreadable clip) for a
/// bounded span, then EOF.
struct ZeroReader {
served: u32,
cap: u32,
}
impl SectorSource for ZeroReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
if self.served >= self.cap {
return Ok(0);
}
self.served += count as u32;
buf.fill(0);
Ok(buf.len())
}
fn capacity_sectors(&self) -> u32 {
self.cap
}
}
fn pgs_title(pid: u16, vendor_forced: bool) -> DiscTitle {
DiscTitle {
playlist: String::new(),
playlist_id: 0,
duration_secs: 0.0,
size_bytes: 0,
clips: vec![],
streams: vec![Stream::Subtitle(SubtitleStream {
pid,
codec: Codec::Pgs,
language: "eng".into(),
forced: vendor_forced,
qualifier: LabelQualifier::None,
codec_data: None,
})],
chapters: vec![],
extents: vec![Extent {
start_lba: 0,
sector_count: 4,
}],
content_format: ContentFormat::BdTs,
codec_privates: vec![None],
}
}
#[test]
fn no_observed_content_preserves_vendor_forced() {
// An unreadable/encrypted clip yields no PGS display sets — the probe must
// leave the existing vendor-derived forced flag untouched, never assert
// "not forced" from having seen nothing.
let mut reader = ZeroReader { served: 0, cap: 4 };
let mut title = pgs_title(0x1200, true);
probe_and_set_forced(&mut reader, &mut title);
let Stream::Subtitle(s) = &title.streams[0] else {
panic!()
};
assert!(s.forced, "no content observed → vendor forced preserved");
}
#[test]
fn no_pgs_streams_is_noop() {
// A title with no PGS subtitle streams is a no-op (the reader is never
// touched — a DVD/VobSub or audio-only title).
let mut reader = ZeroReader { served: 0, cap: 0 };
let mut title = pgs_title(0x1200, false);
// Swap the PGS sub for an audio stream so there are no PGS PIDs.
title.streams.clear();
probe_and_set_forced(&mut reader, &mut title);
assert_eq!(reader.served, 0, "no PGS PIDs → no reads");
}
}
+97
View File
@@ -114,6 +114,12 @@ pub struct ReadCtx {
/// good reads after the last error in the cluster." Used to count
/// zone entries and to bound zone_reads accurately.
pub in_damage_zone: bool,
/// Count of RECOVERED ERROR (marginal) reads the drive reported this pass
/// (surfaced by the PER=1 mode-select at drive-prep). Each is distrusted and
/// marked NonTrimmed for a Pass N re-read; the count is reported in the
/// pass summary so an operator can see how much of a "clean" rip was actually
/// marginal.
pub marginal_recovered: u64,
}
/// Coarse classification of a SCSI sense key for diagnostic logging.
@@ -187,6 +193,7 @@ impl ReadCtx {
zones_entered: 0,
jumps_taken: 0,
in_damage_zone: false,
marginal_recovered: 0,
}
}
@@ -226,6 +233,7 @@ impl ReadCtx {
zones_entered: 0,
jumps_taken: 0,
in_damage_zone: false,
marginal_recovered: 0,
}
}
@@ -283,6 +291,7 @@ impl ReadCtx {
total_errors: self.total_errors,
zones_entered: self.zones_entered,
jumps_taken: self.jumps_taken,
marginal_recovered: self.marginal_recovered,
}
}
}
@@ -296,6 +305,8 @@ pub struct PassSummary {
pub total_errors: u64,
pub zones_entered: u64,
pub jumps_taken: u64,
/// RECOVERED ERROR (marginal) reads distrusted and re-queued for Pass N.
pub marginal_recovered: u64,
}
/// What the caller should do after a read failure. The caller owns the
@@ -639,6 +650,36 @@ pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
};
}
// 4b. RECOVERED ERROR — the drive returned this sector but had to fight for
// it (ECC worked hard / retried). We enable reporting of these via MODE
// SELECT PER=1 at drive-prep precisely so they surface: on marginal/dirty
// media the drive's best-effort correction can be silently WRONG (a rip
// that "passed clean" but decoded with errors — the Bourne case). We
// distrust it: mark THIS block NonTrimmed and let Pass N re-read it with
// proper recovery (FUA cache-bypass) — a clean re-read wins, a persistent
// marginal becomes an honest concealed gap.
//
// Critically this is a per-block SkipBlock, NOT the damage-jump path: a
// recovered error is a single marginal sector, not a clustered hard-damage
// zone, so it must NOT trigger the 64 MB JumpAhead (which would nuke huge
// swaths of good data on a lightly-smudged disc). It also does NOT touch
// the damage window / multiplier — a recovered read is not damage-zone
// signal. Returns before the fast-jump logic below.
if sense_key == scsi::SENSE_KEY_RECOVERED_ERROR {
ctx.marginal_recovered += 1;
tracing::warn!(
target: "freemkv::disc",
phase = "recovered_error",
marginal_recovered = ctx.marginal_recovered,
asc = err.scsi_sense().map(|s| s.asc),
ascq = err.scsi_sense().map(|s| s.ascq),
"drive reported a recovered (marginal) read — distrusting; marking NonTrimmed for Pass N re-read"
);
return ReadAction::SkipBlock {
pause_secs: FAIL_PAUSE_SECS,
};
}
// 5. Marginal media (MEDIUM_ERROR / ABORTED_COMMAND) on a multi-
// sector batch: the drive can often read the same sectors
// individually. Bisect into single-sector reads (gentler on the
@@ -803,6 +844,62 @@ mod tests {
}
}
fn recovered_err() -> Error {
Error::DiscRead {
sector: 100,
status: Some(2),
sense: Some(ScsiSense {
sense_key: scsi::SENSE_KEY_RECOVERED_ERROR,
asc: 0x17,
ascq: 0x01,
}),
}
}
#[test]
fn recovered_error_skips_block_not_jump_pass_1() {
// A recovered (marginal) read on Pass 1 must NOT trigger the 64 MB
// damage-jump — that would nuke huge good regions on a lightly-smudged
// disc. It marks just this block NonTrimmed (SkipBlock) for a Pass N
// re-read, and is counted as marginal_recovered.
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&recovered_err(), &mut ctx);
assert!(
matches!(action, ReadAction::SkipBlock { .. }),
"recovered error must SkipBlock, not JumpAhead; got {action:?}"
);
assert_eq!(ctx.marginal_recovered, 1);
// It must not have inflated the damage-jump multiplier (not damage signal).
assert_eq!(ctx.jump_multiplier, 1);
assert_eq!(ctx.jumps_taken, 0);
}
#[test]
fn recovered_error_skips_block_pass_n_too() {
// Pass N sees the same: a recovered read is distrusted → SkipBlock (the
// outer patch loop re-reads the range with FUA).
let mut ctx = ReadCtx::for_patch(1);
let action = handle_read_error(&recovered_err(), &mut ctx);
assert!(
matches!(action, ReadAction::SkipBlock { .. }),
"got {action:?}"
);
assert_eq!(ctx.marginal_recovered, 1);
}
#[test]
fn many_recovered_errors_never_jump() {
// Even a run of recovered errors must never escalate to a damage-jump —
// they're marginal reads, not a hard-damage cluster.
let mut ctx = ReadCtx::for_sweep(32);
for _ in 0..40 {
let a = handle_read_error(&recovered_err(), &mut ctx);
assert!(matches!(a, ReadAction::SkipBlock { .. }), "got {a:?}");
}
assert_eq!(ctx.jumps_taken, 0, "recovered errors never jump");
assert_eq!(ctx.marginal_recovered, 40);
}
#[test]
fn pass_n_marginal_with_batch_gt_1_bisects() {
let mut ctx = ReadCtx::for_patch(32);
-14
View File
@@ -66,14 +66,6 @@ pub(super) enum WorkItem {
/// tell them apart without parsing a flag.
GapFill { pos: u64, len: u64 },
/// Post-read verify downgrade. The producer's `UnitVerifier` found that the
/// just-`Finished` clip unit at `[pos, pos+len)` is confidently undecryptable
/// (a silent bad read). The consumer re-records the range as `NonTrimmed` so
/// the patch pass re-reads it — the ISO bytes (ciphertext) already written by
/// the preceding `Good` are left in place for the patch to overwrite. FIFO
/// pipe ordering guarantees this arrives AFTER the `Good` that wrote them.
MarkBad { pos: u64, len: u64 },
/// Producer wants the latest mapfile stats for the progress
/// callback. Consumer responds on `prog_tx` with a fresh
/// [`ProgressSnapshot`]. Best-effort: if the producer hasn't
@@ -182,12 +174,6 @@ impl Sink<WorkItem> for SweepSink {
}
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::MarkBad { pos, len } => {
// Verify downgrade: the ISO bytes are already written by the
// preceding Good; only the mapfile status changes so patch
// re-reads this range. No file write.
self.map.record(pos, len, SectorStatus::NonTrimmed)?;
}
WorkItem::StatsRequest => {
let stats = self.map.stats();
// DAMAGE only — NOT NonTried. NonTried is the unread remainder
-1022
View File
File diff suppressed because it is too large Load Diff
+191
View File
@@ -61,8 +61,28 @@ const SPIN_UP_SETTLE_SECS: u64 = 10;
const SCSI_PREVENT_ALLOW_MEDIUM_REMOVAL: u8 = 0x1E;
const SCSI_GET_EVENT_STATUS: u8 = 0x4A;
const SCSI_MODE_SENSE: u8 = 0x5A;
const SCSI_MODE_SELECT: u8 = 0x55;
const SCSI_REPORT_KEY: u8 = 0xA4;
/// SBC/MMC Read-Write Error Recovery mode page (page code 0x01). We flip the
/// `PER` bit to make the drive REPORT a recovered read (via CHECK CONDITION +
/// sense key RECOVERED ERROR) instead of silently returning best-effort data as
/// GOOD status. On marginal/dirty media that silent-GOOD data can be
/// mis-corrected — a rip that "passed clean" but decoded with errors. With PER
/// on, freemkv sees the marginal read and re-reads it in Pass N (a loud miss,
/// never a silent commit). See `build_error_recovery_select_payload`.
const MODE_PAGE_ERROR_RECOVERY: u8 = 0x01;
/// Bit masks in the Read-Write Error Recovery flags byte (page byte 2).
const ERP_FLAG_TB: u8 = 0x20; // Transfer Block: still deliver the recovered data
const ERP_FLAG_PER: u8 = 0x04; // Post Error: report recovered errors
const ERP_FLAG_DTE: u8 = 0x02; // Data Terminate on Error: MUST be off (we want the data)
/// `Parameters Saveable` bit in a mode page's byte 0 — valid only on MODE SENSE;
/// must be cleared before echoing the page back in a MODE SELECT.
const MODE_PAGE_PS_BIT: u8 = 0x80;
/// MODE SENSE(10) parameter header length (bytes), preceding any block
/// descriptors and the mode pages.
const MODE10_HEADER_LEN: usize = 8;
/// Optical disc drive session -- open, identify, unlock, and read.
pub struct Drive {
scsi: Box<dyn ScsiTransport>,
@@ -464,6 +484,12 @@ impl Drive {
// wants max speed. Best-effort: a failure here must NOT fail the rip.
if r.is_ok() {
self.set_speed(crate::speed::DriveSpeed::Max.to_kbps());
// Ask the drive to REPORT recovered/marginal reads rather than
// silently commit best-effort data as GOOD (the dirty-disc
// "passed-clean-but-decodes-with-errors" trap). Best-effort: a drive
// that doesn't honor it just keeps its defaults — no regression, and
// on a clean disc it changes nothing.
self.enable_recovered_error_reporting();
}
tracing::info!(
target: "freemkv::drive",
@@ -599,6 +625,53 @@ impl Drive {
}
}
/// Ask the drive to REPORT recovered/marginal reads instead of silently
/// returning best-effort data as GOOD status. MODE SENSE the Read-Write
/// Error Recovery page, flip `PER` (and `TB` on / `DTE` off so we still get
/// the data), and MODE SELECT it back — preserving the drive's own retry
/// count and other bits.
///
/// Best-effort: a drive that doesn't support the page, or rejects the SELECT,
/// simply keeps its default behaviour — no regression, the rip proceeds. On a
/// clean disc this changes nothing (no recovered errors fire); it only
/// surfaces the marginal reads that a dirty disc would otherwise commit
/// silently. Returns whether the page was successfully written.
pub fn enable_recovered_error_reporting(&mut self) -> bool {
let Some(sense) = self.mode_sense_page(MODE_PAGE_ERROR_RECOVERY) else {
tracing::debug!(target: "freemkv::drive", "MODE SENSE error-recovery page unavailable; leaving drive defaults");
return false;
};
let Some(payload) = build_error_recovery_select_payload(&sense) else {
tracing::debug!(target: "freemkv::drive", "error-recovery page malformed/short; leaving drive defaults");
return false;
};
// MODE SELECT(10): PF=1 (page format), parameter list length = payload.
let len = payload.len() as u16;
let cdb = [
SCSI_MODE_SELECT,
0x10, // PF=1, SP=0 (don't persist across power cycles)
0x00,
0x00,
0x00,
0x00,
0x00,
(len >> 8) as u8,
len as u8,
0x00,
];
let mut buf = payload;
match self.checked_exec(&cdb, crate::scsi::DataDirection::ToDevice, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(target: "freemkv::drive", phase = "error_recovery", "recovered-error reporting enabled (PER=1) — marginal reads will surface instead of committing silently");
true
}
Err(e) => {
tracing::debug!(target: "freemkv::drive", error = %e, "MODE SELECT error-recovery page rejected; leaving drive defaults");
false
}
}
}
/// Read vendor-specific READ BUFFER data.
pub fn read_buffer(&mut self, mode: u8, buffer_id: u8, length: u16) -> Option<Vec<u8>> {
let cdb = crate::scsi::build_read_buffer(mode, buffer_id, 0, length as u32);
@@ -1101,6 +1174,46 @@ fn select_drive_with_media(drives: impl Iterator<Item = Drive>) -> Option<Drive>
fallback
}
/// Turn a MODE SENSE(10) Read-Write Error Recovery page response into the
/// payload for a MODE SELECT(10) that enables recovered-error REPORTING —
/// preserving every other bit (notably the drive's own read-retry count).
///
/// Pure so the bit-twiddling is unit-tested without a drive. Steps:
/// - locate the page after the 8-byte header + block descriptors (bytes 6-7);
/// - verify it is page 0x01 with a flags byte present;
/// - in the flags byte: set `PER` (report) and `TB` (still deliver the data),
/// clear `DTE` (don't terminate the transfer on the recovered error);
/// - clear the page's `PS` bit (valid only on SENSE) and zero the header's
/// mode-data-length field (reserved on SELECT).
///
/// Returns `None` (caller leaves the drive at its defaults) when the response is
/// too short or isn't the error-recovery page — never panics on adversarial
/// bytes.
fn build_error_recovery_select_payload(sense: &[u8]) -> Option<Vec<u8>> {
if sense.len() < MODE10_HEADER_LEN {
return None;
}
let block_desc_len = u16::from_be_bytes([sense[6], sense[7]]) as usize;
let page_off = MODE10_HEADER_LEN.checked_add(block_desc_len)?;
// Need page byte 0 (code), byte 1 (length), byte 2 (flags).
if page_off.checked_add(3)? > sense.len() {
return None;
}
if sense[page_off] & 0x3F != MODE_PAGE_ERROR_RECOVERY {
return None;
}
let mut payload = sense.to_vec();
// Header: mode-data-length is reserved on SELECT — zero it.
payload[0] = 0;
payload[1] = 0;
// Page byte 0: clear PS (SENSE-only).
payload[page_off] &= !MODE_PAGE_PS_BIT;
// Flags byte: PER on, TB on, DTE off. Retry count (next byte) untouched.
payload[page_off + 2] |= ERP_FLAG_PER | ERP_FLAG_TB;
payload[page_off + 2] &= !ERP_FLAG_DTE;
Some(payload)
}
/// Decode a READ CAPACITY (10) response into a sector count.
///
/// A short transfer (`bytes_transferred < 4`, which would leave the high
@@ -1276,6 +1389,84 @@ mod command_tests {
use super::*;
use crate::scsi::{DataDirection, ScsiResult, ScsiTransport};
/// A minimal MODE SENSE(10) response carrying the Read-Write Error Recovery
/// page (0x01) with the given flags byte and retry count, no block
/// descriptors. `ps` sets the page's PS bit (SENSE-only), which the SELECT
/// payload must clear.
fn mode_sense_error_recovery(flags: u8, retry: u8, ps: bool) -> Vec<u8> {
let mut v = vec![0u8; MODE10_HEADER_LEN + 12];
// Header: nonzero mode-data-length (must be zeroed on SELECT); no block
// descriptors.
v[0] = 0x00;
v[1] = 0x22;
v[6] = 0x00;
v[7] = 0x00; // block descriptor length = 0
let po = MODE10_HEADER_LEN;
v[po] = MODE_PAGE_ERROR_RECOVERY | if ps { MODE_PAGE_PS_BIT } else { 0 };
v[po + 1] = 0x0A; // page length
v[po + 2] = flags; // error-recovery flags
v[po + 3] = retry; // read retry count
v
}
#[test]
fn error_recovery_payload_sets_per_tb_clears_dte_ps_preserves_retry() {
// Start with PER off, DTE on, PS on, a specific retry count. The SELECT
// payload must flip PER on, TB on, DTE off, clear PS, zero the header
// mode-data-length, and leave the retry count untouched.
let sense = mode_sense_error_recovery(ERP_FLAG_DTE, 0x2C, true);
let out = build_error_recovery_select_payload(&sense).expect("valid page");
let po = MODE10_HEADER_LEN;
assert_eq!(out[0], 0, "header mode-data-length zeroed for SELECT");
assert_eq!(out[1], 0);
assert_eq!(out[po] & MODE_PAGE_PS_BIT, 0, "PS cleared for SELECT");
assert_eq!(out[po] & 0x3F, MODE_PAGE_ERROR_RECOVERY, "still page 0x01");
assert_eq!(out[po + 2] & ERP_FLAG_PER, ERP_FLAG_PER, "PER set");
assert_eq!(
out[po + 2] & ERP_FLAG_TB,
ERP_FLAG_TB,
"TB set (still get data)"
);
assert_eq!(
out[po + 2] & ERP_FLAG_DTE,
0,
"DTE cleared (don't terminate)"
);
assert_eq!(out[po + 3], 0x2C, "read retry count preserved");
}
#[test]
fn error_recovery_payload_honors_block_descriptor_offset() {
// With an 8-byte block descriptor between header and page, the function
// must locate the page at header+desc, not a fixed offset.
let mut sense = vec![0u8; MODE10_HEADER_LEN + 8 + 12];
sense[7] = 8; // block descriptor length
let po = MODE10_HEADER_LEN + 8;
sense[po] = MODE_PAGE_ERROR_RECOVERY;
sense[po + 1] = 0x0A;
sense[po + 2] = 0x00;
let out = build_error_recovery_select_payload(&sense).expect("valid");
assert_eq!(
out[po + 2] & ERP_FLAG_PER,
ERP_FLAG_PER,
"PER set at the descriptor-offset page"
);
}
#[test]
fn error_recovery_payload_rejects_wrong_or_short_page() {
// Wrong page code → None (leave drive at defaults).
let mut wrong = mode_sense_error_recovery(0, 0, false);
wrong[MODE10_HEADER_LEN] = 0x08; // page 0x08 (caching), not 0x01
assert!(build_error_recovery_select_payload(&wrong).is_none());
// Too short to hold the header → None, no panic.
assert!(build_error_recovery_select_payload(&[0u8; 4]).is_none());
// Header claims a block descriptor that runs off the buffer → None.
let mut bad = mode_sense_error_recovery(0, 0, false);
bad[7] = 0xF0; // descriptor length way past the buffer
assert!(build_error_recovery_select_payload(&bad).is_none());
}
/// Mock transport: returns a fixed data payload (copied into the
/// caller's buffer, truncated to fit) on every `execute()`.
struct FixedTransport {
+1 -1
View File
@@ -6,7 +6,7 @@
//!
//! Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against libdvdnav's decoder.
//! verified against real discs.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
+5 -5
View File
@@ -2,7 +2,7 @@
//!
//! An 8-byte navigation command as found in PGC command tables (pre/post/cell)
//! and PCI button info. Decoded per the DVD-Video VM instruction set and
//! verified against libdvdnav's command decoder.
//! verified against real discs.
//!
//! Bit model: the 8 bytes are a big-endian 64-bit word. `byte0` bits 7-5 are the
//! command **type**; for type 1, `byte0` bit 4 selects Link (0) vs Jump (1), and
@@ -133,7 +133,7 @@ const JP_JUMP_SS: u8 = 6;
const JP_CALL_SS: u8 = 8;
// Link (type 1, direct=0) sub-commands. NOTE: sub-op 0 is NOP/no-link and 1 is
// the LinkSub form (libdvdnav `decoder.c` `eval_link_instruction`).
// the LinkSub form (the DVD-Video VM link instruction).
const LK_SUB: u8 = 1;
const LK_PGCN: u8 = 4;
const LK_PTTN: u8 = 5;
@@ -159,7 +159,7 @@ fn be16(b: &[u8; 8], o: usize) -> u16 {
((b[o] as u16) << 8) | b[o + 1] as u16
}
// Compare-operand layouts ("if_version"s) per libdvdnav `decoder.c`. The op
// Compare-operand layouts ("if_version"s) per the DVD-Video VM. The op
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
@@ -205,7 +205,7 @@ pub fn decode(b: &[u8; 8]) -> Command {
let cmd = b[1] & 0x0F;
// Compare predicate, with the operand layout for this command family
// (libdvdnav `decoder.c` `vm_eval_command` type dispatch).
// (the DVD-Video VM command type dispatch).
let compare = match (typ, direct) {
(TYPE_SPECIAL, _) => if_v1(b),
(TYPE_LINK_JUMP, 1) => if_v2(b), // jump
@@ -372,7 +372,7 @@ mod tests {
}
}
// Regression for the libdvdnav cross-check: link sub-op 0 = NOP, 1 = LinkSub.
// Regression for the link sub-op decode: 0 = NOP, 1 = LinkSub.
#[test]
fn link_subop_zero_is_nop_one_is_linksub() {
assert_eq!(decode(&h("2000000000000000")).instr, Instr::Nop);
+31
View File
@@ -150,6 +150,14 @@ pub const E_NETWORK_ADDR_BLOCKED: u16 = 9022;
/// frame dropped before the first keyframe) cannot report success.
pub const E_MUX_EMPTY: u16 = 9023;
pub const E_EXTENT_NOT_UNIT_ALIGNED: u16 = 9030;
/// `mp4://` output but the title has no (primary) video track to carry.
pub const E_MP4_NO_VIDEO_TRACK: u16 = 9048;
/// `mp4://` video track uses a codec with no MP4 mapping / no config record
/// available here (e.g. VC-1, or MPEG-2 without an avcC/hvcC-style record).
pub const E_MP4_UNSUPPORTED_VIDEO_CODEC: u16 = 9049;
/// `mp4://` video track is missing its codec-configuration record
/// (`hvcC`/`avcC`), without which the sample entry can't be written.
pub const E_MP4_MISSING_CODEC_PRIVATE: u16 = 9050;
/// READ CAPACITY returned a short or overflowing transfer.
pub const E_DISC_CAPACITY_MALFORMED: u16 = 9047;
@@ -420,6 +428,12 @@ pub enum Error {
/// every frame dropped before the first keyframe — fails loudly. The
/// `m2ts://` analogue of [`Error::MkvInvalid`]'s zero-frame guard.
MuxEmpty,
/// `mp4://` target title has no primary video track to mux.
MuxNoVideoTrack,
/// `mp4://` video codec has no MP4 sample-entry mapping available here.
Mp4UnsupportedVideoCodec,
/// `mp4://` video track is missing its `hvcC`/`avcC` configuration record.
MuxMissingCodecPrivate,
PesFrameTooLarge {
size: usize,
},
@@ -599,6 +613,9 @@ impl Error {
Error::StreamUrlMissingPort { .. } => E_STREAM_URL_MISSING_PORT,
Error::NetworkAddrBlocked { .. } => E_NETWORK_ADDR_BLOCKED,
Error::MuxEmpty => E_MUX_EMPTY,
Error::MuxNoVideoTrack => E_MP4_NO_VIDEO_TRACK,
Error::Mp4UnsupportedVideoCodec => E_MP4_UNSUPPORTED_VIDEO_CODEC,
Error::MuxMissingCodecPrivate => E_MP4_MISSING_CODEC_PRIVATE,
Error::PesFrameTooLarge { .. } => E_PES_FRAME_TOO_LARGE,
Error::PesInvalidMagic => E_PES_INVALID_MAGIC,
Error::PesTrackTooLarge { .. } => E_PES_TRACK_TOO_LARGE,
@@ -833,6 +850,11 @@ impl From<Error> for std::io::Error {
// 9023 MuxEmpty: finish() reached with zero frames — the output
// would be a header-only container. Treat as invalid output.
E_MUX_EMPTY => std::io::ErrorKind::InvalidData,
// 9048-9050 mp4:// track/codec/config mismatches: the requested
// output can't hold this title's video — invalid output request.
E_MP4_NO_VIDEO_TRACK | E_MP4_UNSUPPORTED_VIDEO_CODEC | E_MP4_MISSING_CODEC_PRIVATE => {
std::io::ErrorKind::InvalidData
}
// 9030 ExtentNotUnitAligned: a malformed/non-AACS-aligned
// extent was handed to the prefetch producer.
9030 => std::io::ErrorKind::InvalidInput,
@@ -1236,6 +1258,9 @@ mod tests {
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MP4_NO_VIDEO_TRACK,
E_MP4_UNSUPPORTED_VIDEO_CODEC,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
@@ -1323,6 +1348,12 @@ mod tests {
(Error::PipelineConsumerGone, E_PIPELINE_CONSUMER_GONE),
(Error::DiscCapacityOverflow, E_DISC_CAPACITY_OVERFLOW),
(Error::MuxEmpty, E_MUX_EMPTY),
(Error::MuxNoVideoTrack, E_MP4_NO_VIDEO_TRACK),
(
Error::Mp4UnsupportedVideoCodec,
E_MP4_UNSUPPORTED_VIDEO_CODEC,
),
(Error::MuxMissingCodecPrivate, E_MP4_MISSING_CODEC_PRIVATE),
(Error::M2tsPacketMalformed, E_M2TS_PACKET_MALFORMED),
(Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED),
(Error::DiscCapacityMalformed, E_DISC_CAPACITY_MALFORMED),
+46 -4
View File
@@ -16,7 +16,7 @@
/// (case-insensitive), then requires an even run of ASCII hex digits. Any
/// non-hex byte, or an odd length, yields `None`.
pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
// Empty → empty Vec (a legitimately-empty variable-length field); odd length
// is malformed. (`parse_hex_fixed` enforces a concrete length separately.)
@@ -34,7 +34,7 @@ pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
/// prefix; requires EXACTLY `2*N` ASCII hex digits after it. `None` on any
/// non-hex byte or a length mismatch.
pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
if bytes.len() != 2 * N {
return None;
@@ -46,8 +46,33 @@ pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
Some(out)
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive).
fn strip_prefix(s: &str) -> &str {
/// Parse a hex string into a `u16`. Accepts an optional `0x`/`0X` prefix
/// (case-insensitive) via the same [`strip_hex_prefix`] the byte parsers use.
/// `None` on any non-hex content or overflow.
///
/// Exists so callers never hand-roll `from_str_radix(s.trim_start_matches("0x"), 16)`
/// — a **case-sensitive** strip that silently dropped an uppercase-`0X` value.
/// (That reintroduced-in-keydb bug is exactly what this module was built to kill;
/// the integer fields now share the one prefix rule.)
pub fn parse_hex_u16(s: &str) -> Option<u16> {
u16::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u32`. See [`parse_hex_u16`].
pub fn parse_hex_u32(s: &str) -> Option<u32> {
u32::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u8`. See [`parse_hex_u16`].
pub fn parse_hex_u8(s: &str) -> Option<u8> {
u8::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive). Public so
/// callers that only need the prefix rule (e.g. normalizing a disc hash) reuse
/// the one definition instead of hand-rolling a case-sensitive
/// `trim_start_matches("0x")`.
pub fn strip_hex_prefix(s: &str) -> &str {
s.strip_prefix("0x")
.or_else(|| s.strip_prefix("0X"))
.unwrap_or(s)
@@ -95,6 +120,23 @@ mod tests {
assert_eq!(parse_hex_fixed::<16>(&s), None);
}
#[test]
fn hex_ints_accept_both_prefix_cases_and_bare() {
// The regression the keydb device-key bug hit: uppercase `0X` must parse
// identically to `0x` and to a bare value.
assert_eq!(parse_hex_u16("0x0001"), Some(1));
assert_eq!(parse_hex_u16("0X0001"), Some(1));
assert_eq!(parse_hex_u16("0001"), Some(1));
assert_eq!(parse_hex_u16(" 0XABCD "), Some(0xABCD));
assert_eq!(parse_hex_u32("0X00000002"), Some(2));
assert_eq!(parse_hex_u32("deadbeef"), Some(0xDEAD_BEEF));
assert_eq!(parse_hex_u8("0X03"), Some(3));
assert_eq!(parse_hex_u8("ff"), Some(0xFF));
// Overflow / non-hex → None.
assert_eq!(parse_hex_u8("0x1FF"), None);
assert_eq!(parse_hex_u16("0xzz"), None);
}
#[test]
fn bytes_variable_length_and_odd_rejected() {
assert_eq!(parse_hex_bytes("0xAABBCC"), Some(vec![0xAA, 0xBB, 0xCC]));
+8 -8
View File
@@ -58,7 +58,7 @@ pub struct DvdTitle {
pub struct DvdCell {
pub first_sector: u32,
pub last_sector: u32,
/// Raw cell-category byte at `cell_playback + 0` (libdvdread layout).
/// Raw cell-category byte at `cell_playback + 0` (DVD-Video IFO layout).
/// Packs block_mode (bits 7-6), block_type (bits 5-4), seamless_play
/// (bit 3), interleaved (bit 2), stc_discontinuity (bit 1),
/// seamless_angle (bit 0). Carried so the extent builder can recognise
@@ -72,7 +72,7 @@ pub struct DvdCell {
}
/// Decoded view of a cell-category byte (`cell_playback + 0`), per the
/// DVD-Video spec / libdvdread `cell_playback_t` layout. Byte-0 bitfields,
/// DVD-Video IFO cell-playback layout. Byte-0 bitfields,
/// MSB-first: `block_mode`(7-6), `block_type`(5-4), `seamless_play`(3),
/// `interleaved`(2), `stc_discontinuity`(1), `seamless_angle`(0). (The real
/// `cell_type` is a karaoke-only field in byte 1, not used here.)
@@ -93,7 +93,7 @@ pub struct CellCategory {
}
impl CellCategory {
/// Decode the raw `cell_playback + 0` byte (libdvdread `read_cell_playback`).
/// Decode the raw `cell_playback + 0` byte (DVD-Video IFO cell playback).
pub fn decode(raw: u8) -> Self {
CellCategory {
block_mode: (raw >> 6) & 0x03,
@@ -423,7 +423,7 @@ fn parse_vts(
}
// VTSI_MAT (VTS_xx_0.IFO header) field offsets — fixed by the DVD-Video
// spec (libdvdread `vtsi_mat_t`). The offsets are constant; the sector
// spec (the VTSI management table). The offsets are constant; the sector
// values they point to are per-disc.
const VTSTT_VOBS_OFFSET: usize = 0xC4; // VTS title VOBS start sector (feature)
const VTS_PGCIT_OFFSET: usize = 0xCC; // VTS_PGCIT sector pointer
@@ -580,7 +580,7 @@ fn parse_audio_attr(data: &[u8], offset: usize) -> Result<DvdAudioAttr> {
_ => Codec::Unknown(coding_mode),
};
let sample_rate_flag = (b1 >> 4) & 0x03; // sample_frequency: byte 1 bits 5-4 (libdvdread audio_attr_t)
let sample_rate_flag = (b1 >> 4) & 0x03; // sample_frequency: byte 1 bits 5-4 (DVD-Video audio attributes)
let sample_rate = match sample_rate_flag {
0 => 48000,
1 => 96000,
@@ -1071,7 +1071,7 @@ mod tests {
/// reads with — a co-edit of constant + helper would silently re-introduce
/// the PAL-as-NTSC bug and every test would still pass. This test feeds
/// `parse_video_attr` HARDCODED bytes captured from real DVD-Video layouts
/// (DVD spec / libdvdread `video_attr_t`: mpeg_version[7-6] video_format[5-4]
/// (DVD-Video video attributes: mpeg_version[7-6] video_format[5-4]
/// display_aspect[3-2] permitted_df[1-0]) — no `v_atr_byte`. If the parser's
/// bit positions drift, these fail.
#[test]
@@ -1238,7 +1238,7 @@ mod tests {
// ─────────────────────────────────────────────────────────────────────
// Added hardening tests. Grounded in the DVD-Video IFO spec
// (dvd_udf / libdvdread ifo_types.h; http://dvd.sourceforge.net).
// (DVD-Video IFO format; http://dvd.sourceforge.net).
// ─────────────────────────────────────────────────────────────────────
/// BCD frame-rate flag: bits 7-6 of byte[3]. 0b01 = 25fps (PAL),
@@ -1593,7 +1593,7 @@ mod tests {
}
}
/// CellCategory decodes the libdvdread byte-0 bitfields: block_mode (7-6),
/// CellCategory decodes the DVD-Video cell-category byte-0 bitfields: block_mode (7-6),
/// block_type (5-4), seamless_play (3), interleaved (2),
/// stc_discontinuity (1), seamless_angle (0).
#[test]
+2 -2
View File
@@ -78,8 +78,8 @@ use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// read side. Mirrors `WRITEBACK_CHUNK_BYTES` so the read-side page
/// cache stays bounded the same way the write side does.
///
/// 32 MiB is the empirically tuned value on the rip1 test bed (single
/// 7200rpm HDD via SATA): smaller windows (8 / 16 MiB) shorten the
/// 32 MiB is the empirically tuned value on a 7200rpm HDD via SATA:
/// smaller windows (8 / 16 MiB) shorten the
/// kernel-readahead overlap and slow the producer; larger windows
/// (64 / 128 MiB) let the page cache pin enough of the ISO to
/// pressure concurrent writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
+301 -49
View File
@@ -23,6 +23,64 @@ use crate::aacs::types::{UnitKey, Vid};
use crate::disc::Key;
use crate::error::Error;
/// Minimum encrypted-content unit samples a single online key request must carry.
///
/// The key service identifies a key by which of the submitted units it decrypts,
/// so too few samples — especially on FMTS, where a segment interleaves several
/// variants at the unit level — can return a key that matches an incidental unit
/// rather than the one asked about (a false positive). This many distinct units
/// make the request unambiguous.
///
/// Canonical here (the base crate) so BOTH consumers agree on one value: the
/// online source in `freemkv-keysources` (which refuses to send an under-sampled
/// request) re-exports it, and libfreemkv's own FMTS forensic query
/// ([`crate::mux`]) sizes its per-segment batch by it. Layering forbids the
/// reverse import (keysources depends on libfreemkv, not vice versa), so the
/// value lives at the lower layer both share.
pub const MIN_SAMPLE_UNITS: usize = 8;
/// A set of encrypted content-unit samples PROVEN to carry at least
/// [`MIN_SAMPLE_UNITS`] units — the online `/decode` request's proof-of-ownership.
///
/// "Parse, don't validate": the only constructor, [`DecodeSampleSet::new`], returns
/// `None` for an under-sized slice, so an online key request simply *cannot be built*
/// from too few samples. The runtime `len() < MIN_SAMPLE_UNITS` check that used to
/// live at the request site (and was silently forgotten by an under-sampling caller,
/// reading as "key service down") becomes a compile-time obligation: a request builder
/// that takes `&DecodeSampleSet` can never receive an unchecked `Vec`.
///
/// The *count* enforced here is a runtime property of the disc (how many encrypted
/// units it yields); the *requested* count is a caller-side compile-time constant that
/// callers pin to `MIN_SAMPLE_UNITS` (see e.g. autorip's `SAMPLE_UNITS`). Together the
/// two make under-sampling unrepresentable at the request boundary.
#[derive(Debug, Clone)]
pub struct DecodeSampleSet(Vec<Vec<u8>>);
impl DecodeSampleSet {
/// Wrap `units` iff it carries at least [`MIN_SAMPLE_UNITS`] samples; `None`
/// otherwise (the caller then skips the online source rather than sending an
/// ambiguous request). This is the sole way to obtain a `DecodeSampleSet`.
pub fn new(units: Vec<Vec<u8>>) -> Option<Self> {
(units.len() >= MIN_SAMPLE_UNITS).then_some(Self(units))
}
/// The proven-sufficient samples. Guaranteed `>= MIN_SAMPLE_UNITS` in length.
pub fn units(&self) -> &[Vec<u8>] {
&self.0
}
/// Number of samples — always `>= MIN_SAMPLE_UNITS`.
pub fn len(&self) -> usize {
self.0.len()
}
/// Always `false` (a `DecodeSampleSet` never holds fewer than `MIN_SAMPLE_UNITS`);
/// provided so the type satisfies the usual `len`/`is_empty` pairing.
pub fn is_empty(&self) -> bool {
false
}
}
/// The public AACS inputs a key source needs to look a disc up. Captured at
/// scan; contains no secrets — only the disc identity and the on-disc AACS
/// structures a source or key server may key on.
@@ -57,7 +115,7 @@ pub struct DiscInputs {
pub volume_label: Option<String>,
}
/// A lazy view of a disc's AACS material, handed to [`KeySource::get_uk`] so a
/// A lazy view of a disc's AACS material, handed to [`KeySource::get_unit_keys`] so a
/// source can drive the derivation chain without holding the disc reader.
///
/// "Lazy" by contract: each accessor returns only what the source asks for, so a
@@ -175,10 +233,34 @@ impl ResolveCtx for DiscInputsCtx<'_> {
/// ([`resolve_and_apply`]) tries each source in order and validates the returned
/// keys against real ciphertext before committing them, so a wrong key from one
/// source transparently falls through to the next.
///
/// Two explicit resolve operations, one per key kind — never one overloaded call
/// whose meaning depends on how many keys came back:
/// * [`get_unit_keys`](Self::get_unit_keys) — the disc's base per-CPS-unit Unit
/// Keys (index space = CPS-unit number). The common path for every disc.
/// * [`get_fmts_indexes`](Self::get_fmts_indexes) — the AACS 2.1 forensic index
/// keys (index space = forensic index 1..N). Defaults to empty: a source with
/// no forensic material opts out, and only an FMTS disc ever asks.
///
/// What each source must do to answer is the source's own business: a keydb keys
/// on `disc_hash` and reads no samples; the online source submits the ctx's
/// content samples (a base batch for `get_unit_keys`, an index-1 anchor batch for
/// `get_fmts_indexes`) to the key service.
pub trait KeySource {
/// Resolve this disc's terminal Unit Keys from this source. An empty `Vec`
/// is a genuine "no key here"; `Err` is a source failure.
fn get_uk(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error>;
/// Resolve this disc's base per-CPS-unit Unit Keys from this source. An empty
/// `Vec` is a genuine "no key here"; `Err` is a source failure.
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error>;
/// Resolve this disc's AACS 2.1 forensic index keys — the per-index keys the
/// base Unit Key cannot open (see [`crate::aacs::segment`]) — ordered by
/// forensic index (element `i` carries `UnitKey.idx == i`, forensic index
/// `i + 1`). The source hands back the COMPLETE set it holds; the caller
/// trusts any non-empty result as all of them and never assumes a fixed count.
/// Defaults to empty: a source with no forensic material (a plain keydb, the
/// mapfile) opts out, and only an FMTS disc's mux ever calls this.
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
/// The AACS host certificate(s) this source can supply for the live-drive
/// SCSI mutual-auth handshake (the OEM/AACS baseline route). `mkb` is the
@@ -215,7 +297,7 @@ pub fn resolve_and_apply(
/// [`crate::aacs::trace::ResolutionTrace`] recording, per source, what happened — for
/// applications to render. ZERO English; the trace is typed enums only.
///
/// One-shot per source: each source's [`KeySource::get_uk`] is called exactly
/// One-shot per source: each source's [`KeySource::get_unit_keys`] is called exactly
/// once with a [`DiscInputsCtx`] over `inputs`. Non-empty Unit Keys are mapped
/// to terminal [`Key::Unit`]s and applied via [`crate::Disc::decrypt_with`],
/// which validates them against `inputs.samples` and only mutates the disc on
@@ -243,7 +325,7 @@ pub fn resolve_and_apply_traced(
for source in sources {
// `who` is the source's own stable identifier — no enum to map back to.
let who = source.label().to_string();
match source.get_uk(&ctx) {
match source.get_unit_keys(&ctx) {
Ok(uks) if !uks.is_empty() => {
// Positional index → canonical CPS-unit number (position + 1).
let unit_keys: Vec<(u32, [u8; 16])> = uks
@@ -294,7 +376,7 @@ pub fn resolve_and_apply_traced(
/// decorator re-decrypts with the returned keys, which is the validation.
pub fn fetch_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
for source in sources {
if let Ok(uks) = source.get_uk(ctx) {
if let Ok(uks) = source.get_unit_keys(ctx) {
if !uks.is_empty() {
return uks;
}
@@ -303,35 +385,92 @@ pub fn fetch_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) ->
Vec::new()
}
/// Build the read-time key-fetch closure from the disc's public AACS inputs and
/// a way to (re)build the application's key sources. The decorator calls it with
/// the still-scrambled unit ciphertext when no held key opens that unit; it runs
/// [`fetch_unit_keys`] with those bytes as `samples` and returns any keys.
/// The forensic counterpart to [`fetch_unit_keys`]: drive `sources` in order and
/// return the first source's non-empty AACS 2.1 forensic index set. `ctx` carries
/// the index-1 anchor batch (the mux, which owns disc geometry, gathers it and
/// injects it as the ctx's samples); a source that needs no samples (a keydb
/// keying on `disc_hash`) ignores them. Whatever the winning source returns —
/// ≥ 1 key — is trusted as the COMPLETE ordered set; no fixed count is assumed.
pub fn fetch_fmts_indexes(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
for source in sources {
if let Ok(uks) = source.get_fmts_indexes(ctx) {
if !uks.is_empty() {
return uks;
}
}
}
Vec::new()
}
/// Build the read-time [`crate::sector::KeyFetch`] from the disc's public AACS
/// inputs and a way to (re)build the application's key sources. The returned
/// resolver has the two explicit operations the mux and recovery decorator call:
/// [`unit_keys`](crate::sector::KeyFetch::unit_keys) drives [`fetch_unit_keys`]
/// (base per-CPS-unit keys), [`fmts_indexes`](crate::sector::KeyFetch::fmts_indexes)
/// drives [`fetch_fmts_indexes`] (the AACS 2.1 forensic set). Each is handed the
/// caller's sample batch as the ctx's `samples`, so a source pulls whatever
/// material it needs.
///
/// One builder, used by every read path (sweep / patch / mux) and by every
/// consumer (CLI, autorip) — neither application contains the fetch logic, only
/// its key-source config. Returns a **shared, stateless** [`crate::sector::KeyFetch`]
/// (`Arc<Fn>`): build it once, clone it into each read path. `make_sources` is
/// invoked per fetch (the cold path, ~once per CPS unit) so the closure stays
/// One builder, used by every read path (sweep / patch / mux) and every consumer
/// (CLI, autorip) — neither application contains the fetch logic, only its
/// key-source config. Cheap to clone; build once, clone into each read path.
/// `make_sources` is invoked per fetch (the cold path) so the resolver stays
/// `Send + Sync` without requiring `KeySource: Send`.
pub fn key_fetch(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
) -> crate::sector::KeyFetch {
std::sync::Arc::new(move |samples: &[Vec<u8>]| -> Vec<[u8; 16]> {
let sources = make_sources();
let mut di = inputs.clone();
di.samples = samples.to_vec();
// Parse Unit_Key_RO.inf at the disc's OWN stride (carried on `inputs`):
// an online /decode reply that returns a VUK (not a terminal UK) then
// derives unit keys from `enc_title_keys`, which a V10 disc parses at the
// 48-byte stride — hardcoding the V20 stride here corrupted them.
let ctx = DiscInputsCtx::new(&di);
fetch_unit_keys(&sources, &ctx)
.into_iter()
.map(|u| u.key)
.collect()
})
// One driver behind both operations: rebuild the sources, inject `samples`
// as the ctx's content samples, run `drive` (the per-kind fetch), map the
// resolved UnitKeys to raw keys. Memoized by the fingerprint of the sample
// batch: the resolved keys are disc-level (a clip's index / CPS keys are
// identical for every title that references it), so the first batch resolves
// over the network and every repeat is answered from the cache with no
// request. Empty replies are cached too — a key the service lacks for a batch
// won't appear on a re-ask, so re-hitting the network buys nothing. Each
// operation gets its OWN cache: a base batch and a forensic anchor never
// collide, and the same bytes could legitimately resolve differently per op.
// The per-kind driver: `fetch_unit_keys` or `fetch_fmts_indexes`.
type FetchDriver = fn(&[Box<dyn KeySource>], &dyn ResolveCtx) -> Vec<UnitKey>;
fn make_op(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
drive: FetchDriver,
) -> crate::sector::KeyFetchFn {
let cache: std::sync::Arc<std::sync::Mutex<std::collections::HashMap<u64, Vec<[u8; 16]>>>> =
std::sync::Arc::new(std::sync::Mutex::new(std::collections::HashMap::new()));
std::sync::Arc::new(move |samples: &[Vec<u8>]| -> Vec<[u8; 16]> {
let fp = {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
samples.len().hash(&mut h);
for s in samples {
s.hash(&mut h);
}
h.finish()
};
if let Some(hit) = cache.lock().unwrap_or_else(|e| e.into_inner()).get(&fp) {
return hit.clone();
}
let sources = make_sources();
let mut di = inputs.clone();
di.samples = samples.to_vec();
// Parse Unit_Key_RO.inf at the disc's OWN stride (carried on `inputs`):
// an online /decode reply that returns a VUK (not a terminal UK) then
// derives unit keys from `enc_title_keys`, which a V10 disc parses at
// the 48-byte stride — hardcoding the V20 stride here corrupted them.
let ctx = DiscInputsCtx::new(&di);
let keys: Vec<[u8; 16]> = drive(&sources, &ctx).into_iter().map(|u| u.key).collect();
cache
.lock()
.unwrap_or_else(|e| e.into_inner())
.insert(fp, keys.clone());
keys
})
}
let unit = make_op(inputs.clone(), make_sources.clone(), fetch_unit_keys);
let fmts = make_op(inputs, make_sources, fetch_fmts_indexes);
crate::sector::KeyFetch::new(unit, fmts)
}
/// Read up to `n` ENCRYPTED 6144-byte aligned units from `title`'s body, raw (no
@@ -346,7 +485,7 @@ pub fn key_fetch(
///
/// "Encrypted" is decided by [`crate::aacs::content::aacs_unit_encrypted`] — the
/// AACS Copy Permission Indicator (CPI) in the top 2 bits of byte 0, the
/// spec-correct signal (libaacs' `buf[0] & 0xc0`). NOT the `ts_sync_destroyed`
/// spec-correct signal (`buf[0] & 0xc0`). NOT the `ts_sync_destroyed`
/// sync heuristic: destroyed TS syncs do not imply encryption (an FMTS variant
/// frame or an odd clear unit can lack syncs yet be unencrypted), and a clear
/// unit sent to a key server yields nothing to validate against — the "0
@@ -407,7 +546,7 @@ pub fn read_encrypted_units(
break;
}
let u = &buf[o..o + ALIGNED_UNIT_LEN];
if aacs_unit_encrypted(u) {
if aacs_unit_encrypted(u, title.content_format) {
out.push(u.to_vec());
if out.len() >= n {
return out;
@@ -425,6 +564,46 @@ mod tests {
use crate::aacs::types::UnitKey;
use std::sync::{Arc, Mutex};
fn units(n: usize) -> Vec<Vec<u8>> {
(0..n).map(|i| vec![i as u8; 4]).collect()
}
// ── DecodeSampleSet: the online request can't be built under-sized ─────────
/// Fewer than MIN_SAMPLE_UNITS → no set. Mutation: accepting a short slice
/// resurrects the exact autorip bug (a 4-sample request silently skipped /
/// read as "service down").
#[test]
fn decode_sample_set_rejects_under_min() {
for n in 0..MIN_SAMPLE_UNITS {
assert!(
DecodeSampleSet::new(units(n)).is_none(),
"{n} samples (< {MIN_SAMPLE_UNITS}) must not build a DecodeSampleSet"
);
}
}
/// Exactly the minimum, and above it, construct — and expose all samples.
#[test]
fn decode_sample_set_accepts_min_and_above() {
let exact = DecodeSampleSet::new(units(MIN_SAMPLE_UNITS)).expect("min builds");
assert_eq!(exact.len(), MIN_SAMPLE_UNITS);
assert_eq!(exact.units().len(), MIN_SAMPLE_UNITS);
assert!(!exact.is_empty());
let more = DecodeSampleSet::new(units(MIN_SAMPLE_UNITS + 5)).expect("above min builds");
assert_eq!(more.len(), MIN_SAMPLE_UNITS + 5);
}
/// The wrapped units round-trip byte-for-byte (the request carries exactly what
/// was gathered — no reordering/truncation).
#[test]
fn decode_sample_set_preserves_units() {
let raw = units(MIN_SAMPLE_UNITS);
let set = DecodeSampleSet::new(raw.clone()).unwrap();
assert_eq!(set.units(), raw.as_slice());
}
// ── KeySource default-method behaviour ────────────────────────────────────
/// KeySource::host_certs() defaults to empty regardless of the MKB argument.
@@ -436,7 +615,7 @@ mod tests {
fn key_source_host_certs_defaults_to_empty() {
struct MinimalSource;
impl KeySource for MinimalSource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
}
@@ -493,7 +672,7 @@ mod tests {
fn trace_who_is_the_source_label_verbatim() {
struct LabeledSource(&'static str);
impl KeySource for LabeledSource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
fn label(&self) -> &'static str {
@@ -550,23 +729,20 @@ mod tests {
struct EmptySource;
impl KeySource for EmptySource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
}
struct ErroringSource;
impl KeySource for ErroringSource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Err(Error::AacsNoKeys)
}
}
struct HasKey([u8; 16]);
impl KeySource for HasKey {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey {
idx: 0,
key: self.0,
}])
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, self.0)])
}
}
@@ -608,14 +784,11 @@ mod tests {
seen: Arc<Mutex<Vec<Vec<u8>>>>,
}
impl KeySource for Probe {
fn get_uk(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
if let Ok(s) = ctx.samples(8) {
self.seen.lock().unwrap().extend(s);
}
Ok(vec![UnitKey {
idx: 0,
key: self.key,
}])
Ok(vec![UnitKey::new(0, self.key)])
}
}
@@ -631,7 +804,7 @@ mod tests {
let cb = key_fetch(empty_inputs(), make);
let samples = vec![vec![0xEEu8; crate::aacs::content::ALIGNED_UNIT_LEN]];
let got = cb(&samples);
let got = cb.unit_keys(&samples);
assert_eq!(
got,
vec![key],
@@ -645,6 +818,85 @@ mod tests {
assert_eq!(*builds.lock().unwrap(), 1, "make_sources invoked per fetch");
}
/// The two `KeyFetch` operations route to the two DISTINCT trait methods:
/// `unit_keys` drives `get_unit_keys`, `fmts_indexes` drives
/// `get_fmts_indexes`. A source that returns different keys per method proves
/// the seam no longer collapses "1 base key" and "the forensic set" into one
/// overloaded call — the operation, not the return length, decides which.
#[test]
fn key_fetch_routes_unit_and_fmts_to_distinct_source_methods() {
const BASE: [u8; 16] = [0xB0; 16];
const F1: [u8; 16] = [0xF1; 16];
const F2: [u8; 16] = [0xF2; 16];
struct TwoOp;
impl KeySource for TwoOp {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, BASE)])
}
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, F1), UnitKey::new(1, F2)])
}
}
let make: Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync> =
Arc::new(|| vec![Box::new(TwoOp) as Box<dyn KeySource>]);
let cb = key_fetch(empty_inputs(), make);
let samples = vec![vec![0x01u8; 4]];
assert_eq!(
cb.unit_keys(&samples),
vec![BASE],
"unit_keys resolves the base Unit Key via get_unit_keys"
);
assert_eq!(
cb.fmts_indexes(&samples),
vec![F1, F2],
"fmts_indexes resolves the forensic set (any length) via get_fmts_indexes"
);
}
/// `KeyFetch::unit_only` serves base keys but NEVER a forensic set — the
/// contract the sweep/patch recovery decorator relies on (it resolves CPS
/// units only). Its `fmts_indexes` is unconditionally empty.
#[test]
fn key_fetch_unit_only_never_serves_forensic() {
let f = crate::sector::KeyFetch::unit_only(std::sync::Arc::new(|_| vec![[0xAA; 16]]));
assert_eq!(f.unit_keys(&[vec![0u8; 4]]), vec![[0xAA; 16]]);
assert!(
f.fmts_indexes(&[vec![0u8; 4]]).is_empty(),
"unit_only resolver yields no forensic keys"
);
}
/// `get_fmts_indexes` defaults to empty, so a base-only source (a keydb) opts
/// out of the forensic path without implementing it. `fetch_fmts_indexes` then
/// falls through to the next source, exactly like the unit-key driver.
#[test]
fn fetch_fmts_indexes_skips_default_optout_source() {
struct BaseOnly; // uses the default (empty) get_fmts_indexes
impl KeySource for BaseOnly {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, [0x11; 16])])
}
}
struct Forensic;
impl KeySource for Forensic {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, [0x77; 16])])
}
}
let inputs = empty_inputs();
let ctx = DiscInputsCtx::new(&inputs);
let sources: Vec<Box<dyn KeySource>> = vec![Box::new(BaseOnly), Box::new(Forensic)];
let got = fetch_fmts_indexes(&sources, &ctx);
assert_eq!(got.len(), 1);
assert_eq!(got[0].key, [0x77; 16], "the base-only source is skipped");
}
/// #4 regression: encrypted content NOT at the extent midpoint (a late-
/// starting feature, or a midpoint landing in clear nav) must still be
/// sampled — empty samples make `decrypt_with` skip wrong-key validation.
@@ -723,7 +975,7 @@ mod tests {
);
for s in &samples {
assert!(
aacs_unit_encrypted(s),
aacs_unit_encrypted(s, crate::disc::ContentFormat::BdTs),
"every sample is a CPI-flagged encrypted unit (byte0 & 0xC0 != 0)"
);
}
@@ -806,7 +1058,7 @@ mod tests {
);
for s in &samples {
assert!(
aacs_unit_encrypted(s),
aacs_unit_encrypted(s, crate::disc::ContentFormat::BdTs),
"only CPI-flagged units are selected"
);
assert_eq!(
+7 -2
View File
@@ -98,7 +98,7 @@ pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFI
/// The muxing/writing-application string written into MKV output
/// (`"freemkv <version> (g<hash>)"`).
pub const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub(crate) const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs;
pub(crate) mod clpi;
@@ -187,7 +187,10 @@ pub use identity::DriveId;
// don't touch `DecryptKeys` directly — `DiscStream::new(reader, title, keys, …)`
// accepts whatever `Disc::decrypt_keys()` returned. `decrypt_sectors()` is
// for callers that operate on raw sector buffers (e.g. ISO patching).
pub use decrypt::{DecryptKeys, decrypt_sectors, decrypt_threads, set_decrypt_threads};
pub use decrypt::{
AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_sectors_mapped, decrypt_threads,
set_decrypt_threads,
};
// ─── Disc structure ─────────────────────────────────────────────────────────
//
@@ -238,6 +241,7 @@ pub use mux::NullStream;
pub use mux::StdioStream;
pub use mux::WriteSeek;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
pub use mux::{Mp4FitReport, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
@@ -248,6 +252,7 @@ pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
// single decrypt-on-read decorator (AACS / CSS / none) — wrap any
// `SectorSource` to get plaintext sectors out.
pub use mux::build_iso_pipeline;
pub use mux::resolve_mux_key_map;
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, drive_has_disc, list_drives};
pub use sector::{
DecryptingSectorSource, FileSectorSink, FileSectorSource, KeyFetch, PrefetchedSectorSource,
+246 -53
View File
@@ -41,6 +41,11 @@ pub struct Ac3Parser {
/// the running per-frame PTS at the point the partial tail was retained.
/// Used by `flush()` to time the final buffered frame at EOS.
flush_pts_ns: i64,
/// Keep/drop bookkeeping for the CRC decodability gate. A frame that fails
/// its native CRC is dropped rather than shipped as a decoder-choking glitch;
/// the running PTS is advanced across it (see the emit loop) so the drop is a
/// silence gap, never a shift of the following audio.
tally: super::dropgate::DropTally,
}
impl Default for Ac3Parser {
@@ -54,8 +59,99 @@ impl Ac3Parser {
Self {
buf: Vec::with_capacity(4096),
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
}
}
/// Access units dropped as undecodable so far — surfaced to the CLI/mux.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Emit the final buffered frame at EOS, through the decodability gate.
/// During streaming a final frame may sit in `buf` with no following PES to
/// complete it; without this drain the last ~32 ms of audio is lost. Only a
/// fully-sized frame at a syncword is considered; a partial/garbage tail is
/// discarded, and a corrupt (CRC-failing) final frame is dropped.
fn flush_tail(&mut self) -> Vec<Frame> {
let buf = std::mem::take(&mut self.buf);
let Some(off) = find_ac3_sync(&buf) else {
return Vec::new();
};
let frame_all = &buf[off..];
if frame_all.len() < 6 {
return Vec::new();
}
let bsid = get_bsid(frame_all);
let frame_size = if bsid >= 11 {
eac3_frame_size(frame_all)
} else {
ac3_frame_size(frame_all)
};
if !(MIN_FRAME_BYTES..=8192).contains(&frame_size) || off + frame_size > buf.len() {
return Vec::new();
}
let frame = &buf[off..off + frame_size];
let duration_ns = frame_duration_ns(frame, bsid);
if let Some(reason) = ac3_drop_reason(&self.tally, frame, bsid) {
self.tally
.record_drop(self.flush_pts_ns, duration_ns as i64, frame.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.flush_pts_ns,
keyframe: true,
data: frame.to_vec(),
duration_ns: Some(duration_ns),
}]
}
}
use super::crc::crc16_ansi;
/// Whether a fully-buffered (E-)AC-3 frame passes its native CRC. ffmpeg's
/// decoder checks exactly this — `av_crc(AV_CRC_16_ANSI, 0, &buf[2],
/// frame_size - 2) == 0` (ac3dec.c) — over the frame after the 2-byte syncword;
/// the trailing crc word makes a clean frame's residue zero. A nonzero residue
/// is a ~1-in-65536-certain sign of payload corruption, so we drop the frame
/// (silence gap) rather than ship a glitch. `frame` must be exactly the frame
/// bytes (syncword .. frame_size).
fn frame_crc_ok(frame: &[u8]) -> bool {
// Need the syncword (2) plus at least one covered byte; the caller only
// invokes this on a fully-sized frame, so this is defensive.
if frame.len() < 4 {
return true;
}
crc16_ansi(&frame[2..]) == 0
}
/// Decodability verdict for a fully-sized (E-)AC-3 frame: `Some(reason)` when it
/// must be dropped, `None` when it decodes. Drops (in order): a poisoned track
/// (mostly-undecodable → drop the rest), a bitstream id ffmpeg's parser rejects
/// (`bsid > 16` → `AC3_PARSE_ERROR_BSID`), or a failed native frame CRC.
fn ac3_drop_reason(
tally: &super::dropgate::DropTally,
frame: &[u8],
bsid: u8,
) -> Option<&'static str> {
if tally.is_poisoned() {
Some("track-poisoned")
} else if bsid > 16 {
Some("bsid")
} else if !frame_crc_ok(frame) {
Some("crc")
} else {
None
}
}
impl CodecParser for Ac3Parser {
@@ -91,10 +187,12 @@ impl CodecParser for Ac3Parser {
// in practice, so this is defense-in-depth.
let base_pts_ns = pes.pts.map(pts_to_ns).unwrap_or(self.flush_pts_ns);
// Prepend leftover from previous PES
// Prepend leftover from previous PES, then take the whole buffer into a
// local so the emit loop can call `self.tally` (the bytes are no longer
// borrowed from `self`). The unconsumed tail is written back at the end.
self.buf.extend_from_slice(&pes.data);
let data = &self.buf;
let buf = std::mem::take(&mut self.buf);
let data = &buf;
let mut frames = Vec::new();
let mut pos = 0;
// Running PTS for the next frame to emit in this call.
@@ -134,15 +232,26 @@ impl CodecParser for Ac3Parser {
}
let duration_ns = frame_duration_ns(remaining, bsid);
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: frame_pts_ns,
keyframe: true,
data: data[start..start + frame_size].to_vec(),
duration_ns: Some(duration_ns),
});
let frame = &data[start..start + frame_size];
// Decodability gate: drop a frame ffmpeg's parser rejects (bsid > 16)
// or whose native CRC fails (payload corruption). `frame_pts_ns` is
// advanced BELOW whether or not the frame survives, so a drop is a
// silence gap and the following frames keep their true PTS.
if let Some(reason) = ac3_drop_reason(&self.tally, frame, bsid) {
self.tally
.record_drop(frame_pts_ns, duration_ns as i64, frame.len(), reason);
} else {
self.tally.record_kept();
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: frame_pts_ns,
keyframe: true,
data: frame.to_vec(),
duration_ns: Some(duration_ns),
});
}
frame_pts_ns += duration_ns as i64;
pos = start + frame_size;
}
@@ -198,38 +307,10 @@ impl CodecParser for Ac3Parser {
}
fn flush(&mut self) -> Vec<Frame> {
// End of stream: emit a complete final frame still buffered. During
// streaming a final frame may sit in `buf` with no following PES to
// complete/confirm it; without this drain the last ~32 ms of audio is
// dropped at EOS (mirrors dts.rs::flush). Only a fully-sized frame at a
// syncword is emitted; a partial/garbage tail is discarded.
let buf = std::mem::take(&mut self.buf);
let Some(off) = find_ac3_sync(&buf) else {
return Vec::new();
};
let frame = &buf[off..];
if frame.len() < 6 {
return Vec::new();
}
let bsid = get_bsid(frame);
let frame_size = if bsid >= 11 {
eac3_frame_size(frame)
} else {
ac3_frame_size(frame)
};
if !(MIN_FRAME_BYTES..=8192).contains(&frame_size) || off + frame_size > buf.len() {
return Vec::new();
}
let duration_ns = frame_duration_ns(frame, bsid);
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.flush_pts_ns,
keyframe: true,
data: buf[off..off + frame_size].to_vec(),
duration_ns: Some(duration_ns),
}]
let out = self.flush_tail();
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
}
fn codec_private(&self) -> Option<Vec<u8>> {
@@ -460,9 +541,24 @@ mod tests {
frame[1] = 0x77;
frame[4] = (fscod << 6) | frmsizecod;
frame[5] = 0x08 << 3; // bsid = 8 (AC-3)
finalize_ac3_crc(&mut frame);
frame
}
/// Set the trailing CRC word so the whole-frame residue over `[2..]` is zero
/// — i.e. the frame passes the decodability gate. Relies on the CRC-16/ANSI
/// residue property: appending `crc16([2..n-2])` (big-endian) zeroes the
/// register over `[2..n]`. Leaves the crc1 field (bytes 2-3) untouched.
fn finalize_ac3_crc(frame: &mut [u8]) {
let n = frame.len();
if n < 4 {
return;
}
let c = crc16_ansi(&frame[2..n - 2]);
frame[n - 2] = (c >> 8) as u8;
frame[n - 1] = (c & 0xFF) as u8;
}
#[test]
fn parse_empty_pes() {
let mut parser = Ac3Parser::new();
@@ -1090,27 +1186,33 @@ mod tests {
// --- frame acceptance / rejection at the size boundaries ---
#[test]
fn eac3_frame_at_min_frame_bytes_is_accepted() {
// The smallest acceptable (E-)AC-3 frame is MIN_FRAME_BYTES = 6.
// Build an E-AC-3 frame whose frmsiz sizes it to exactly 6 bytes
// (frmsiz=2). bsid >= 11 selects E-AC-3 sizing. The parser must emit it.
fn eac3_frame_at_min_frame_bytes_passes_sizing_then_crc_gate() {
// The smallest frame the SIZING layer accepts is MIN_FRAME_BYTES = 6
// (frmsiz=2). A synthetic all-zero 6-byte frame passes sizing (so it
// reaches the decodability gate — proven by it being COUNTED as a drop,
// not silently size-skipped) but fails the CRC gate and is dropped; the
// following real AC-3 frame (valid CRC) is emitted.
let mut parser = Ac3Parser::new();
// 0x0B 0x77 | byte2=0 byte3=2 (frmsiz=2 → 6 bytes) | byte4=0 | byte5 bsid
let mut data = vec![0x0B, 0x77, 0x00, 0x02, 0x00, 16 << 3];
// pad to exactly 6 bytes (already 6). Then a trailing real AC-3 frame so
// the 6-byte frame isn't a tail that needs more data.
data.truncate(6);
data.extend_from_slice(&make_ac3_frame(0, 2));
let f = parser.parse(&make_eac3_pes(data));
assert_eq!(f.len(), 2, "6-byte E-AC-3 frame accepted + following AC-3");
assert_eq!(f[0].data.len(), 6);
assert_eq!(f.len(), 1, "6-byte frame dropped (CRC), real AC-3 emitted");
assert_eq!(f[0].data.len(), 160, "the surviving frame is the real AC-3");
assert_eq!(
parser.dropped_frames(),
1,
"the 6-byte frame reached the gate"
);
}
#[test]
fn eac3_max_frmsiz_frame_within_window_accepted() {
// E-AC-3 frmsiz is an 11-bit field (3 bits of byte2 + 8 bits of byte3),
// so its maximum value is 0x7FF = 2047 → (2048)*2 = 4096 bytes, which is
// inside the MIN_FRAME_BYTES..=8192 accept window and must be emitted.
// inside the MIN_FRAME_BYTES..=8192 accept window and, with a valid CRC,
// must be emitted.
let mut parser = Ac3Parser::new();
let mut frame = vec![0u8; 4096];
frame[0] = 0x0B;
@@ -1118,6 +1220,7 @@ mod tests {
frame[2] = 0x07; // frmsiz high
frame[3] = 0xFF; // frmsiz low → 0x7FF = 2047 → 4096 bytes
frame[5] = 16 << 3; // bsid 16 (E-AC-3)
finalize_ac3_crc(&mut frame); // pass the decodability gate
let f = parser.parse(&make_eac3_pes(frame));
assert_eq!(f.len(), 1, "4096-byte E-AC-3 frame within window accepted");
assert_eq!(f[0].data.len(), 4096);
@@ -1296,6 +1399,96 @@ mod tests {
assert_eq!(acmod_channels(&frame), Some(2));
}
// --- decodability (CRC) gate: keep clean frames, drop corrupt ones ---
/// A structurally-valid AC-3 frame with one payload byte corrupted so its
/// native CRC fails (header/size intact, so the framer delimits it normally).
fn make_corrupt_ac3_frame(fscod: u8, frmsizecod: u8) -> Vec<u8> {
let mut f = make_ac3_frame(fscod, frmsizecod);
f[20] ^= 0xFF; // flip a payload byte → CRC no longer zero
assert!(!frame_crc_ok(&f), "corruption must break the CRC");
f
}
#[test]
fn crc16_residue_zero_after_finalize_nonzero_after_corruption() {
// The CRC-16/ANSI residue property the gate relies on: a finalized frame
// has residue 0 over [2..]; flipping any covered byte makes it nonzero.
let good = make_ac3_frame(0, 2);
assert!(frame_crc_ok(&good));
let bad = make_corrupt_ac3_frame(0, 2);
assert!(!frame_crc_ok(&bad));
}
#[test]
fn crc_fail_frame_is_dropped_survivors_kept() {
// good / corrupt / good in one PES: the corrupt middle frame is dropped
// (CRC), the two clean frames are emitted, and the drop is counted.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2);
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2));
data.extend_from_slice(&make_ac3_frame(0, 2));
let f = parser.parse(&make_eac3_pes(data));
// Only two of three survive; flush has nothing (all closed in-call).
assert_eq!(f.len(), 2, "corrupt frame dropped, two clean survive");
assert_eq!(parser.dropped_frames(), 1);
assert_eq!(
parser.dropped_duration_ns(),
32_000_000,
"one 32ms frame of silence"
);
}
#[test]
fn crc_drop_preserves_pts_sync_no_shift() {
// THE INVARIANT: dropping a corrupt frame must not shift the audio after
// it. good / corrupt / good in one PES — the corrupt frame is dropped but
// the trailing clean frame keeps the EXACT PTS it would have had with no
// drop (base + 2 frame durations): a silence gap, not a shift.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2); // f0
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2)); // dropped
data.extend_from_slice(&make_ac3_frame(0, 2)); // f2
let f = parser.parse(&make_eac3_pes(data));
assert_eq!(f.len(), 2);
let base = pts_to_ns(90000);
let frame_dur = 32_000_000i64; // 1536 @ 48k
assert_eq!(f[0].pts_ns, base, "f0 at PES base");
assert_eq!(
f[1].pts_ns,
base + 2 * frame_dur,
"surviving frame keeps its true timeline (base + 2 frames) — gap, not shift"
);
}
#[test]
fn bsid_over_16_is_dropped() {
// ffmpeg's parser rejects bsid > 16 (AC3_PARSE_ERROR_BSID). A frame with
// bsid = 17 that still sizes must be dropped, not emitted.
let mut frame = vec![0u8; 128];
frame[0] = 0x0B;
frame[1] = 0x77;
frame[3] = 63; // frmsiz = 63 → (63+1)*2 = 128 bytes (E-AC-3 sizing)
frame[5] = 17 << 3; // bsid = 17 (> 16)
assert_eq!(get_bsid(&frame), 17);
let tally = super::super::dropgate::DropTally::new("ac3");
assert_eq!(ac3_drop_reason(&tally, &frame, 17), Some("bsid"));
}
#[test]
fn clean_stream_drops_nothing() {
// A stream of valid frames passes untouched — zero false positives.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..5 {
data.extend_from_slice(&make_ac3_frame(0, 2));
}
let mut f = parser.parse(&make_eac3_pes(data));
f.extend(parser.flush());
assert_eq!(f.len(), 5);
assert_eq!(parser.dropped_frames(), 0);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
+220
View File
@@ -0,0 +1,220 @@
//! AAC ADTS decodability gate.
//!
//! ffmpeg's `ff_adts_header_parse` (adts_header.c) has exactly three hard
//! rejects: syncword != 0xFFF, a reserved `sampling_frequency_index`
//! (`ff_mpeg4audio_sample_rates[sr] == 0`, i.e. index ≥ 13), and
//! `aac_frame_length < 7`. It does NOT verify the optional ADTS CRC (it only
//! `skip_bits(16)` past it). So the gate mirrors those three rejects: a packet
//! that begins with the ADTS sync but is otherwise malformed is dropped; a
//! packet with no ADTS sync is raw AAC (e.g. from mp4, which carries no ADTS
//! header) or a continuation and passes through unchanged — never false-dropped.
//! Raw AAC has no per-frame integrity data, so like LPCM it cannot be gated.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// `ff_mpeg4audio_sample_rates` — 13 valid entries; indices 13/14/15 are 0
/// (reserved), which is exactly what ffmpeg rejects.
const ADTS_SAMPLE_RATE_VALID: [u32; 16] = [
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350, 0, 0,
0,
];
/// ADTS header verdict for the packet head.
enum AdtsVerdict {
/// No 12-bit ADTS sync at the head — not an ADTS frame we can validate.
NoSync,
/// Sync present and the three ffmpeg-checked fields are legal.
Valid,
/// Sync present but a reserved sample-rate index or a sub-header
/// frame-length — ffmpeg's parser rejects this.
Invalid,
}
fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// Need the full 7-byte fixed+variable header to read frame_length.
if data.len() < 7 {
return AdtsVerdict::NoSync;
}
// 12-bit syncword 0xFFF: byte0 == 0xFF and top nibble of byte1 == 0xF.
if data[0] != 0xFF || (data[1] & 0xF0) != 0xF0 {
return AdtsVerdict::NoSync;
}
// sampling_frequency_index: byte2 bits 5..2.
let sr_index = ((data[2] >> 2) & 0x0F) as usize;
if ADTS_SAMPLE_RATE_VALID[sr_index] == 0 {
return AdtsVerdict::Invalid;
}
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
if frame_length < 7 {
return AdtsVerdict::Invalid;
}
AdtsVerdict::Valid
}
pub struct AdtsParser {
tally: DropTally,
}
impl Default for AdtsParser {
fn default() -> Self {
Self::new()
}
}
impl AdtsParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("aac"),
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for AdtsParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
let drop =
self.tally.is_poisoned() || matches!(adts_verdict(&pes.data), AdtsVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with
/// aac_frame_length set to the total size.
fn adts_frame(payload: usize) -> Vec<u8> {
let total = 7 + payload;
let mut f = vec![0u8; total];
f[0] = 0xFF;
f[1] = 0xF1; // sync + MPEG-4 + no CRC (protection_absent=1)
f[2] = 0x50; // profile=AAC-LC, sr_index=4 (44.1 kHz)
f[3] = 0x80; // channel_config low + start of frame_length
// frame_length (13 bits) = total.
let fl = total as u32;
f[3] = (f[3] & 0xFC) | ((fl >> 11) & 0x03) as u8;
f[4] = ((fl >> 3) & 0xFF) as u8;
f[5] = (((fl & 0x07) << 5) as u8) | 0x1F; // low 3 bits of len + buffer-fullness bits
f
}
#[test]
fn valid_adts_is_kept() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(adts_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn reserved_sample_rate_index_is_dropped() {
// sr_index = 13 (reserved). byte2 bits5..2 = 1101 → 0x34.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[2] = (f[2] & 0xC3) | (13 << 2); // set sr_index = 13
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn subheader_frame_length_is_dropped() {
// frame_length < 7 (here 0) is a sub-header length → reject.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[3] &= 0xFC; // clear len high bits
f[4] = 0;
f[5] &= 0x1F; // clear len low bits → frame_length = 0
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn raw_aac_without_sync_passes_through() {
// No ADTS sync (e.g. raw AAC from mp4) → cannot validate → keep.
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(
vec![0x21, 0x00, 0x03, 0x40, 0x00, 0x00, 0x00],
Some(0),
));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = AdtsParser::new();
let mut bad = adts_frame(400);
bad[2] = (bad[2] & 0xC3) | (14 << 2); // reserved sr_index
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(adts_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
#[test]
fn short_packet_passes_through() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0)));
assert_eq!(f.len(), 1, "too short to validate → kept");
}
}
+110
View File
@@ -0,0 +1,110 @@
//! Bit-exact CRC helpers shared by the audio codec decodability gates.
//!
//! Both match ffmpeg's `av_crc` tables so a frame that ffmpeg's decoder would
//! flag as a CRC mismatch is flagged identically here. All are MSB-first
//! (non-reflected), init 0, no final XOR — the ffmpeg `AV_CRC_*` (big-endian)
//! variants. Each format transmits its CRC so that the residue over
//! `data + transmitted_crc` is zero, which is exactly how these are used:
//! compute over the whole frame (including its trailing CRC) and check `== 0`.
/// CRC-16/ANSI (a.k.a. CRC-16/BUYPASS): polynomial 0x8005, init 0x0000,
/// MSB-first, no reflection, no final XOR — ffmpeg `AV_CRC_16_ANSI`.
/// Used by AC-3/E-AC-3 (frame CRC), FLAC (frame footer), MPEG-audio and
/// AAC-ADTS (header CRC).
pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x8005
} else {
crc << 1
};
}
}
crc
}
/// CRC-16 with polynomial 0x002D, init 0, MSB-first — ffmpeg's `crc_2D` table
/// (`av_crc_init(crc_2D, 0, 16, 0x002D)`), used by the MLP/TrueHD major-sync
/// header checksum. NOTE: MLP's checksum is the "reversed" scheme — ffmpeg
/// computes `av_crc(...) ^ AV_RL16(trailer)` and compares against `AV_RL16` of
/// the stored word; equivalently, this standard CRC compared against the stored
/// bytes read big-endian. The caller handles that comparison
/// (see `truehd::mlp_major_sync_ok`). Verified against real ffmpeg TrueHD
/// output (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x002D
} else {
crc << 1
};
}
}
crc
}
/// CRC-8/ATM (a.k.a. CRC-8/ITU without the final XOR): polynomial 0x07, init 0,
/// MSB-first, no reflection — ffmpeg `AV_CRC_8_ATM`. Used by the FLAC frame
/// header.
pub(crate) fn crc8_atm(data: &[u8]) -> u8 {
let mut crc: u8 = 0;
for &b in data {
crc ^= b;
for _ in 0..8 {
crc = if crc & 0x80 != 0 {
(crc << 1) ^ 0x07
} else {
crc << 1
};
}
}
crc
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn crc16_residue_property_holds() {
// Appending the big-endian CRC-16 of a message zeroes the residue over
// message+crc — the property every frame gate relies on.
let msg = [0x12u8, 0x34, 0x56, 0x78, 0x9A];
let c = crc16_ansi(&msg);
let mut framed = msg.to_vec();
framed.push((c >> 8) as u8);
framed.push((c & 0xFF) as u8);
assert_eq!(crc16_ansi(&framed), 0);
}
#[test]
fn crc16_known_vector_check_bytes() {
// CRC-16/BUYPASS check value for the ASCII string "123456789" is 0xFEE8
// (the standard catalogue check value for poly 0x8005, init 0).
assert_eq!(crc16_ansi(b"123456789"), 0xFEE8);
}
#[test]
fn crc8_residue_property_holds() {
// Appending the CRC-8 of a message zeroes the residue over message+crc —
// how FLAC's header CRC-8 is verified.
let msg = [0xDEu8, 0xAD, 0xBE, 0xEF];
let c = crc8_atm(&msg);
let mut framed = msg.to_vec();
framed.push(c);
assert_eq!(crc8_atm(&framed), 0);
}
#[test]
fn crc8_known_vector_check_byte() {
// CRC-8/SMBUS (poly 0x07, init 0, no reflection) check value for
// "123456789" is 0xF4 — the catalogue check value.
assert_eq!(crc8_atm(b"123456789"), 0xF4);
}
}
+201
View File
@@ -0,0 +1,201 @@
//! Shared "keep what decodes, drop what doesn't" bookkeeping for the audio
//! codec parsers.
//!
//! The user's rule: a clean mux keeps every frame it can and drops the ones it
//! can't — video always survives (it's inter-frame predicted; a per-frame drop
//! would cascade, so video resyncs/conceals instead), audio keeps every
//! decodable access unit, and a damaged audio AU is dropped rather than shipped
//! as a decoder-choking glitch.
//!
//! The DETECTION is inherently per-codec — each format carries its own
//! authoritative corruption check (DTS: ffmpeg's core-header parse; AC-3: the
//! header CRC; FLAC: the frame CRC-16; …). This type only carries the UNIFORM
//! response so every audio parser behaves identically:
//!
//! 1. **Count** kept vs dropped AUs and the dropped duration.
//! 2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
//! once-per-track aggregate at `warn` so it surfaces without debug logging.
//! 3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
//! a poison flag so the remainder is dropped too (a track that damaged isn't
//! worth muxing).
//!
//! **Sync preservation is the caller's responsibility**, not this type's: the
//! parser must advance its PTS clock across a dropped AU exactly as it would for
//! an emitted one, so a drop becomes a silence gap and never a shift of the
//! following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
/// Minimum access units observed before the whole-track drop verdict can fire.
/// Below this, a short damaged burst can't poison an otherwise-good track.
const TRACK_VERDICT_MIN_AUS: u64 = 200;
/// Per-track drop bookkeeping shared by the audio codec parsers.
pub(crate) struct DropTally {
/// Static codec label for log lines (e.g. `"dts"`, `"ac3"`).
codec: &'static str,
kept: u64,
dropped: u64,
/// AUs dropped because they were INDIVIDUALLY verified undecodable (a failed
/// CRC/header/parity check). Only these feed the whole-track poison verdict.
/// Distinct from `dropped`, which also counts *collateral* drops — AUs
/// discarded as a consequence of one corruption (TrueHD's resync-forward run,
/// or a poisoned track), which must NOT amplify a few real errors into a
/// false whole-track loss.
verified_dropped: u64,
dropped_dur_ns: u64,
poisoned: bool,
}
impl DropTally {
pub(crate) fn new(codec: &'static str) -> Self {
Self {
codec,
kept: 0,
dropped: 0,
verified_dropped: 0,
dropped_dur_ns: 0,
poisoned: false,
}
}
/// Whether the track has been judged too damaged to mux. Once `true`, the
/// caller should drop every remaining AU (passing them to [`record_drop`]
/// with a poison reason) rather than emit them.
pub(crate) fn is_poisoned(&self) -> bool {
self.poisoned
}
/// Access units dropped as undecodable so far — surfaced to the CLI/mux.
pub(crate) fn dropped_frames(&self) -> u64 {
self.dropped
}
/// Total decoded duration (ns) of dropped AUs — the audio silence introduced.
pub(crate) fn dropped_duration_ns(&self) -> u64 {
self.dropped_dur_ns
}
/// Record an emitted (decodable) access unit.
pub(crate) fn record_kept(&mut self) {
self.kept += 1;
}
/// Record a dropped access unit that was INDIVIDUALLY verified undecodable
/// (a failed CRC/header/parity check). Counts toward the whole-track poison
/// verdict. `reason` is a short static label for the check that failed.
pub(crate) fn record_drop(&mut self, pts_ns: i64, dur_ns: i64, bytes: usize, reason: &str) {
self.verified_dropped += 1;
self.record_drop_common(pts_ns, dur_ns, bytes, reason);
self.maybe_poison();
}
/// Record a COLLATERAL drop — an AU discarded as a consequence of another
/// corruption rather than being individually undecodable (TrueHD's
/// resync-forward run to the next major sync, or an already-poisoned track).
/// Counted and logged for the drop report, but deliberately does NOT feed the
/// poison verdict, so one corruption event can't amplify into a false
/// whole-track loss.
pub(crate) fn record_collateral_drop(
&mut self,
pts_ns: i64,
dur_ns: i64,
bytes: usize,
reason: &str,
) {
self.record_drop_common(pts_ns, dur_ns, bytes, reason);
}
fn record_drop_common(&mut self, pts_ns: i64, dur_ns: i64, bytes: usize, reason: &str) {
self.dropped += 1;
self.dropped_dur_ns += dur_ns.max(0) as u64;
tracing::debug!(
target: "mux",
"{}: dropped undecodable AU #{} pts_ns={} dur_ns={} bytes={} reason={}",
self.codec,
self.dropped,
pts_ns,
dur_ns,
bytes,
reason
);
}
/// Whole-track fallback: after enough AUs to judge, if more than half were
/// dropped the track is too damaged to be worth muxing — latch `poisoned`
/// and log it loudly once. The minimum-sample gate keeps a short damaged
/// burst from poisoning an otherwise-good track.
fn maybe_poison(&mut self) {
if self.poisoned {
return;
}
// Judge on VERIFIED drops vs all AUs seen: a track is only poisoned when
// a majority of its access units are individually undecodable — not when
// a couple of corruption events forced long collateral resync runs.
let total = self.kept + self.dropped;
if total >= TRACK_VERDICT_MIN_AUS && self.verified_dropped * 2 > total {
self.poisoned = true;
tracing::warn!(
target: "mux",
"{}: track too damaged to mux — {}/{} AUs individually undecodable (>50%); dropping the whole track",
self.codec,
self.verified_dropped,
total
);
}
}
/// End-of-stream aggregate report, logged at `warn` so a track's dropped
/// audio is never hidden even without debug logging. No-op if nothing was
/// dropped.
pub(crate) fn log_summary(&self) {
if self.dropped > 0 {
tracing::warn!(
target: "mux",
"{}: dropped {} undecodable AU(s) totaling {} ns of audio ({} kept)",
self.codec,
self.dropped,
self.dropped_dur_ns,
self.kept
);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn counts_kept_and_dropped() {
let mut t = DropTally::new("test");
t.record_kept();
t.record_drop(0, 1000, 512, "bad");
t.record_kept();
assert_eq!(t.dropped_frames(), 1);
assert_eq!(t.dropped_duration_ns(), 1000);
assert!(!t.is_poisoned());
}
#[test]
fn poisons_after_min_aus_over_half_dropped() {
let mut t = DropTally::new("test");
// 199 AUs, all dropped: below the min-AU gate, must NOT poison yet.
for _ in 0..199 {
t.record_drop(0, 1000, 512, "bad");
}
assert!(!t.is_poisoned(), "below the 200-AU minimum, no verdict");
// The 200th drop reaches the minimum with >50% dropped → poison.
t.record_drop(0, 1000, 512, "bad");
assert!(t.is_poisoned());
}
#[test]
fn does_not_poison_a_mostly_good_track() {
let mut t = DropTally::new("test");
// 400 AUs, 1 dropped: nowhere near 50%.
t.record_drop(0, 1000, 512, "bad");
for _ in 0..399 {
t.record_kept();
}
assert!(!t.is_poisoned());
}
}
+535 -41
View File
@@ -49,6 +49,12 @@ pub struct DtsParser {
/// running cursor: previous emit + its duration). Only consulted when
/// `front_pts` is unchanged from `last_front_pts`. `PTS_UNSET` = no base yet.
next_pts_ns: i64,
/// Keep/drop bookkeeping for the decodability gate: counts, per-drop and
/// aggregate logging, and the whole-track poison fallback. A dropped AU is
/// NEVER emitted, but the PTS clock is still advanced across it (see
/// [`stamp_pts`] usage) so every SURVIVING AU keeps the exact timestamp it
/// would have had — a drop becomes a silence gap, never a shift.
tally: super::dropgate::DropTally,
}
impl Default for DtsParser {
@@ -65,6 +71,50 @@ impl DtsParser {
pts_marks: std::collections::VecDeque::new(),
last_front_pts: PTS_UNSET,
next_pts_ns: PTS_UNSET,
tally: super::dropgate::DropTally::new("dts"),
}
}
/// Number of access units dropped as undecodable so far. The mux/CLI reads
/// this to surface the count ("dropped N damaged DTS frames").
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of all dropped access units — the length of
/// audio silence introduced by dropping undecodable frames.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Gate an assembled access unit through the decodability check and either
/// push it or drop it. `au_pts`/`dur_ns` are already stamped on the shared
/// PTS clock (which the caller advances whether or not the AU survives), so
/// a drop leaves the following audio on its true timeline — a gap, not a
/// shift. Every drop is logged (fail-loud, never silent).
fn emit_or_drop(&mut self, au: Vec<u8>, au_pts: i64, dur_ns: i64, out: &mut Vec<Frame>) {
let verdict = if self.tally.is_poisoned() {
Err(DropReason::TrackPoisoned)
} else {
core_header_drop_reason(&au).map_or(Ok(()), Err)
};
match verdict {
Ok(()) => {
self.tally.record_kept();
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: au_pts,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
});
}
Err(reason) => {
self.tally
.record_drop(au_pts, dur_ns, au.len(), reason.as_str());
}
}
}
@@ -299,8 +349,8 @@ impl CodecParser for DtsParser {
// flush is an extension-substream PES, carrying its own later
// timestamp) must NOT become the next unit's PTS base.
let mut forced = false;
let au_end = match next_core_boundary(&self.buf, core_size) {
NextCore::Found(end) => end,
let (au_end, ext_clean) = match next_core_boundary(&self.buf, core_size) {
NextCore::Found { end, ext_clean } => (end, ext_clean),
NextCore::NeedMore => break, // candidate sync needs more header
NextCore::None => {
// No next core sync buffered yet. The trailing extension
@@ -312,27 +362,32 @@ impl CodecParser for DtsParser {
break;
}
forced = true;
self.buf.len()
(self.buf.len(), true)
}
};
let au: Vec<u8> = self.buf[..au_end].to_vec();
// Damaged source encoding: when the extension boundary was GARBAGE
// (not any DTS sync — `ext_clean == false`), the extension bytes for
// this AU are corrupt and would make the decoder cascade "DSYNC check
// failed" / "Read past end of XLL band data". Emit the clean DTS core
// ALONE (a decodable, lossy frame) and still drain past the garbage to
// the next core — a perfect mux drops the bad frame's corrupt part
// rather than shipping it. A recognized-but-unsizeable extension
// (`ext_clean == true`) is preserved in full (lossless).
let emit_end = if ext_clean { au_end } else { core_size };
let au: Vec<u8> = self.buf[..emit_end].to_vec();
// The AU's own core PES PTS (the PES covering its first byte, even if
// that PES preceded the one(s) carrying its extensions or the next
// core), stamped monotonically: honored when it advances past the
// running clock (UHD one-AU-per-PES), but never allowed to collide
// with the previous AU when several cores share ONE PES (DVD).
let dur_ns = dts_core_duration_ns(&au) as i64;
// Advance the PTS clock for this AU BEFORE the decodability gate, so
// a dropped AU still advances the timeline exactly as an emitted one
// would: the following AU keeps its true PTS and the drop is a gap,
// never a shift. `emit_or_drop` decides whether to actually push it.
let au_pts = self.stamp_pts(self.front_pts(), dur_ns);
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: au_pts,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
});
self.emit_or_drop(au, au_pts, dur_ns, &mut frames);
self.drain_front(au_end);
// After draining, the marker covering the new front (if any) carries
// the next AU's PTS; `pending_pts` is only the fallback when no
@@ -359,10 +414,23 @@ impl CodecParser for DtsParser {
}
fn flush(&mut self) -> Vec<Frame> {
// End of stream: emit the final access unit still buffered (the last
// core + its extension substreams, which had no following core sync to
// close it during streaming). Require a complete core frame; drop a
// bare partial sync tail.
let out = self.flush_tail();
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
impl DtsParser {
/// Emit the final buffered access unit (the last core + its extension
/// substreams, which had no following core sync to close it during
/// streaming), gated through the decodability check. Require a complete core
/// frame; drop a bare partial sync tail.
fn flush_tail(&mut self) -> Vec<Frame> {
if find_sync(&self.buf, &DTS_CORE_SYNC) != Some(0) || self.buf.len() < CORE_HEADER_MIN_BYTES
{
self.buf.clear();
@@ -381,19 +449,9 @@ impl CodecParser for DtsParser {
let dur_ns = dts_core_duration_ns(&au) as i64;
let pts_ns = self.stamp_pts(self.front_pts(), dur_ns);
self.pts_marks.clear();
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
}]
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
let mut out = Vec::new();
self.emit_or_drop(au, pts_ns, dur_ns, &mut out);
out
}
}
@@ -407,7 +465,14 @@ fn find_sync(data: &[u8], pattern: &[u8; 4]) -> Option<usize> {
/// Result of scanning for the next valid core sync that closes an access unit.
enum NextCore {
/// A valid next core sync was found; the access unit ends at this offset.
Found(usize),
/// `ext_clean` is `false` only when the byte at the extension boundary was
/// GARBAGE — neither a core sync nor a DTS-HD extension sync — meaning the
/// extension region is corrupt (damaged source encoding). The caller then
/// emits the clean DTS core alone and drops the garbage, instead of shipping
/// a corrupt AU that makes the decoder cascade DSYNC / "Read past end of XLL".
/// It stays `true` when the region is a real (if unsizeable) extension sync —
/// that path is load-bearing for valid streams and must NOT be dropped.
Found { end: usize, ext_clean: bool },
/// A candidate core sync was found but its header isn't fully buffered yet,
/// so its validity can't be decided — wait for more data.
NeedMore,
@@ -492,8 +557,11 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
pos += sz; // skip the whole extension substream precisely
}
// Couldn't size it (truncated/garbage header) — heuristic fallback.
_ => return scan_for_next_core(buf, pos),
// A real extension sync we couldn't size (truncated header /
// unsupported sub-form) — heuristic fallback, but the region IS a
// recognized extension, so keep it (ext_clean = true). This path
// is load-bearing for valid streams.
_ => return scan_for_next_core(buf, pos, true),
}
} else if buf[pos..].starts_with(&DTS_CORE_SYNC) {
// The bytes right after the precisely-skipped extensions are the next
@@ -503,13 +571,18 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
return NextCore::Found {
end: pos,
ext_clean: true,
};
}
return scan_for_next_core(buf, pos); // implausible core here — fall back
return scan_for_next_core(buf, pos, true); // implausible core — recognized sync, keep
} else {
// Neither a known extension nor a core sync at the precise boundary
// (padding / junk) — fall back to the heuristic scan.
return scan_for_next_core(buf, pos);
// GARBAGE at the extension boundary — neither a core sync nor a
// DTS-HD extension sync. This is damaged source encoding: the
// extension region is corrupt. Mark ext_clean = false so the caller
// emits the clean core alone and drops the garbage.
return scan_for_next_core(buf, pos, false);
}
}
}
@@ -518,7 +591,7 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
/// syncword whose decoded size is plausible. Used only when precise extension
/// skipping can't proceed; a chance core syncword in extension payload usually
/// decodes to an implausible size and is skipped.
fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
fn scan_for_next_core(buf: &[u8], from: usize, ext_clean: bool) -> NextCore {
let mut from = from;
while let Some(rel) = find_sync(&buf[from..], &DTS_CORE_SYNC) {
let pos = from + rel;
@@ -527,7 +600,10 @@ fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
return NextCore::Found {
end: pos,
ext_clean,
};
}
from = pos + SYNCWORD_BYTES;
}
@@ -597,6 +673,128 @@ fn dts_core_duration_ns(data: &[u8]) -> u64 {
(samples * 1_000_000_000 + rate / 2) / rate
}
/// DCA core-header constants, mirrored from ffmpeg `libavcodec/dca_core.h`.
/// `deficit_samples` must equal this (`DCA_PCMBLOCK_SAMPLES`); `npcmblocks`
/// must be a multiple of `DCA_SUBBAND_SAMPLES`; `audio_mode` must be below
/// `DCA_AMODE_COUNT`; `lfe_present == DCA_LFE_FLAG_INVALID` is rejected.
const DTS_PCMBLOCK_SAMPLES: u32 = 32;
const DTS_SUBBAND_SAMPLES: u32 = 8;
const DTS_AMODE_COUNT: u32 = 10;
const DTS_LFE_FLAG_INVALID: u32 = 3;
/// `ff_dca_sample_rates[16]` — sample rate (Hz) per core `SFREQ` code; a `0`
/// entry marks a reserved code that ffmpeg's parser rejects
/// (`DCA_PARSE_ERROR_SAMPLE_RATE`). Valid entries are locked to the spec by
/// `dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
/// {0, 4, 5, 9, 10}.
const DTS_CORE_SR_VALID: [u32; 16] = [
0, 8_000, 16_000, 32_000, 0, 0, 11_025, 22_050, 44_100, 0, 0, 12_000, 24_000, 48_000, 96_000,
192_000,
];
/// `ff_dca_bits_per_sample[8]` — a `0` entry marks a reserved `PCMR` code that
/// ffmpeg's parser rejects (`DCA_PARSE_ERROR_PCM_RES`); reserved codes are
/// {4, 7}.
const DTS_CORE_PCMR_BITS: [u8; 8] = [16, 16, 20, 20, 0, 24, 24, 0];
/// Why an access unit was judged undecodable. Each core-header variant is the
/// exact condition under which ffmpeg's `ff_dca_parse_core_frame_header` returns
/// the matching `DCA_PARSE_ERROR_*`; `TrackPoisoned` is our whole-track drop.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum DropReason {
DeficitSamples,
PcmBlocks,
FrameSize,
Amode,
SampleRate,
ReservedBit,
LfeFlag,
PcmRes,
TrackPoisoned,
}
impl DropReason {
/// Short static label for the drop log (the shared tally logs `&str`).
fn as_str(&self) -> &'static str {
match self {
DropReason::DeficitSamples => "deficit-samples",
DropReason::PcmBlocks => "pcm-blocks",
DropReason::FrameSize => "frame-size",
DropReason::Amode => "audio-mode",
DropReason::SampleRate => "sample-rate",
DropReason::ReservedBit => "reserved-bit",
DropReason::LfeFlag => "lfe-flag",
DropReason::PcmRes => "pcm-resolution",
DropReason::TrackPoisoned => "track-poisoned",
}
}
}
/// Decodability gate: a faithful port of ffmpeg's `ff_dca_parse_core_frame_header`
/// validity checks (libavcodec/dca.c). Returns `Some(reason)` when ffmpeg's own
/// parser would reject this core frame's header — in which case the packet is
/// undecodable ("Invalid data found") and dropping it loses nothing a decoder
/// could have used. Returns `None` (keep) for a decodable header OR if the
/// header can't be fully read (never false-drop on our own buffer underrun; the
/// framer only emits AUs whose core is fully buffered and ≥ 96 bytes).
///
/// The 4-byte core sync is already validated by the framer, so this reads the
/// header fields that follow it. ffmpeg's parser does NOT verify the CPF header
/// CRC (it `skip_bits(16)` past it — dca.c) and the core decoder likewise skips
/// the audio-header/side-info CRCs (dca_core.c), so no CRC check is mirrored
/// here: doing so would drop frames ffmpeg decodes fine (false positives).
fn core_header_drop_reason(au: &[u8]) -> Option<DropReason> {
let mut r = BitReader::new(au.get(SYNCWORD_BYTES..)?);
let _ftype = r.read_bit()?;
let deficit_samples = r.read_bits(5)? + 1;
if deficit_samples != DTS_PCMBLOCK_SAMPLES {
return Some(DropReason::DeficitSamples);
}
let crc_present = r.read_bit()? == 1;
let npcmblocks = r.read_bits(7)? + 1;
if npcmblocks & (DTS_SUBBAND_SAMPLES - 1) != 0 {
return Some(DropReason::PcmBlocks);
}
let frame_size = r.read_bits(14)? + 1;
if frame_size < MIN_CORE_FRAME_BYTES as u32 {
return Some(DropReason::FrameSize);
}
let audio_mode = r.read_bits(6)?;
if audio_mode >= DTS_AMODE_COUNT {
return Some(DropReason::Amode);
}
let sr_code = r.read_bits(4)? as usize;
if DTS_CORE_SR_VALID[sr_code] == 0 {
return Some(DropReason::SampleRate);
}
let _br_code = r.read_bits(5)?;
if r.read_bit()? != 0 {
return Some(DropReason::ReservedBit);
}
// drc, ts, aux, hdcd (1 each) → ext_audio_type (3) → ext_present, aspf (1 each).
r.skip_bits(4)?;
r.skip_bits(3)?;
r.skip_bits(2)?;
let lfe_present = r.read_bits(2)?;
if lfe_present == DTS_LFE_FLAG_INVALID {
return Some(DropReason::LfeFlag);
}
let _predictor_history = r.read_bit()?;
if crc_present {
// ffmpeg only skips the 16-bit header CRC here — it is not verified.
r.skip_bits(16)?;
}
let _filter_perfect = r.read_bit()?;
let _encoder_rev = r.read_bits(4)?;
let _copy_hist = r.read_bits(2)?;
let pcmr_code = r.read_bits(3)? as usize;
if DTS_CORE_PCMR_BITS[pcmr_code] == 0 {
return Some(DropReason::PcmRes);
}
None
}
#[cfg(test)]
mod tests {
use super::*;
@@ -617,6 +815,11 @@ mod tests {
let fsize = size - 1;
let mut data = vec![0u8; size];
data[0..4].copy_from_slice(&DTS_CORE_SYNC);
// byte4: FTYPE(0) SHORT(5) CPF(0) NBLKS-high(0). SHORT = 31 makes
// deficit_samples = 32 = DCA_PCMBLOCK_SAMPLES, which ffmpeg's parser
// (and our decodability gate) require of a real core frame. NBLKS high
// bit (byte4 bit0) stays 0 for NBLKS = 15.
data[4] = 31u8 << 2;
// NBLKS = 15 → (15+1)*32 = 512 samples/frame (the DVD/UHD DTS-core norm).
// NBLKS is byte4 bit0 + byte5 bits7-2; here byte4 bit0 = 0, byte5 = 15<<2.
data[5] = (15u8 << 2) | ((fsize >> 12) & 0x03) as u8;
@@ -676,12 +879,60 @@ mod tests {
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found(end) if end == core.len() + exss.len()
NextCore::Found { end, .. } if end == core.len() + exss.len()
),
"AU must end at the REAL next core (after the full EXSS), not the false sync inside it"
);
}
#[test]
fn garbage_extension_emits_core_only_but_valid_ext_is_kept() {
// Damaged source: a valid core, then GARBAGE (no core sync, no extension
// sync) where the extension belongs, then the next core. The framer must
// mark this boundary ext_clean=false and the parser must emit the clean
// 512-byte CORE alone (dropping the garbage), draining to the next core.
let core = make_dts_core(512);
let garbage = vec![0xE4, 0x3F, 0xE3, 0x90, 0xCC, 0x6C]; // real Bourne head bytes
let mut garbage = garbage;
garbage.extend(std::iter::repeat(0xAB).take(300));
let next = make_dts_core(512);
let mut buf = core.clone();
buf.extend_from_slice(&garbage);
buf.extend_from_slice(&next);
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found { end, ext_clean: false } if end == core.len() + garbage.len()
),
"garbage boundary must be flagged unclean"
);
let mut parser = DtsParser::new();
let mut frames = parser.parse(&make_pes(buf, Some(90000)));
frames.extend(parser.flush());
assert!(!frames.is_empty());
for f in &frames {
assert_eq!(f.data.len(), 512, "garbage-extension AU emits core only");
assert_eq!(&f.data[0..4], &DTS_CORE_SYNC);
}
// Contrast: a REAL extension sync (even if unsizeable) must be KEPT in
// full — ext_clean stays true, never downgraded to core-only.
let mut buf2 = make_dts_core(512);
buf2.extend_from_slice(&make_dts_ext(256));
buf2.extend_from_slice(&make_dts_core(512));
assert!(
matches!(
next_core_boundary(&buf2, 512),
NextCore::Found {
ext_clean: true,
..
}
),
"a recognized extension sync is preserved, not dropped"
);
}
#[test]
fn parse_empty_pes() {
let mut parser = DtsParser::new();
@@ -1478,4 +1729,247 @@ mod tests {
"sub-floor sync skipped, real 512 core is AU1"
);
}
/// A structurally-framed but UNDECODABLE core: a valid `make_dts_core`
/// whose reserved header bit is set. It still sizes and syncs correctly (so
/// the framer delimits it normally), but ffmpeg's `ff_dca_parse_core_frame_header`
/// — and our port — reject it (`DCA_PARSE_ERROR_RESERVED_BIT`). The reserved
/// bit is byte9 bit4 in the core header (after SYNC..RATE).
fn make_bad_dts_core(size: usize) -> Vec<u8> {
let mut d = make_dts_core(size);
assert!(
core_header_drop_reason(&d).is_none(),
"base core is decodable"
);
d[9] |= 0x10; // set the reserved bit
assert_eq!(
core_header_drop_reason(&d),
Some(DropReason::ReservedBit),
"reserved-bit core must be judged undecodable"
);
d
}
#[test]
fn valid_stream_drops_nothing() {
// A clean stream of decodable cores must pass the gate untouched — the
// detector is an exact ffmpeg-parity port, so zero false positives.
let mut parser = DtsParser::new();
let mut stream = Vec::new();
for _ in 0..5 {
stream.extend_from_slice(&make_dts_core(512));
}
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 5, "all five cores emitted");
assert_eq!(parser.dropped_frames(), 0, "nothing dropped");
assert_eq!(parser.dropped_duration_ns(), 0);
}
#[test]
fn undecodable_core_is_dropped_and_counted() {
// A single undecodable core between good ones is dropped; the survivors
// are emitted and the drop is counted.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512);
stream.extend_from_slice(&make_bad_dts_core(512));
stream.extend_from_slice(&make_dts_core(640));
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "the bad core is dropped, two survive");
assert_eq!(frames[0].data.len(), 512);
assert_eq!(frames[1].data.len(), 640);
assert_eq!(parser.dropped_frames(), 1);
assert_eq!(
parser.dropped_duration_ns(),
DTS_CORE_DUR_NS as u64,
"one frame's worth of audio silence introduced"
);
}
#[test]
fn drop_preserves_av_sync_no_shift() {
// THE INVARIANT: dropping an undecodable AU must never shift the audio
// that follows. A good/bad/good run in ONE PES — the bad middle core is
// dropped, but the trailing good core must keep the EXACT PTS it would
// have had with no drop (base + 2 frame durations), so the drop is a
// silence gap, not a shift.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512); // c1: good
stream.extend_from_slice(&make_bad_dts_core(512)); // c2: undecodable
stream.extend_from_slice(&make_dts_core(640)); // c3: good
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "c2 dropped; c1 and c3 survive");
let base = pts_to_ns(90000);
assert_eq!(frames[0].pts_ns, base, "c1 keeps the PES base PTS");
assert_eq!(
frames[1].pts_ns,
base + 2 * DTS_CORE_DUR_NS,
"c3 keeps its TRUE timeline (base + 2 frames) — the drop is a gap, not a shift"
);
// The gap between the survivors is exactly the dropped frame's duration
// beyond the normal one-frame spacing.
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
2 * DTS_CORE_DUR_NS,
"surviving AUs are spaced by the real timeline including the dropped frame's slot"
);
assert_eq!(parser.dropped_frames(), 1);
}
#[test]
fn whole_track_poison_drops_remainder() {
// A track dominated by undecodable frames is judged too damaged to mux:
// once the >50% verdict fires (after the minimum sample count), the
// whole track — including any later good frames — is dropped.
let mut parser = DtsParser::new();
let mut stream = Vec::new();
// 300 AUs, ~2/3 undecodable → well over the 50% threshold and the
// 200-AU minimum.
for i in 0..300 {
if i % 3 == 0 {
stream.extend_from_slice(&make_dts_core(512));
} else {
stream.extend_from_slice(&make_bad_dts_core(512));
}
}
// A trailing burst of GOOD cores that must be dropped once poisoned.
for _ in 0..20 {
stream.extend_from_slice(&make_dts_core(512));
}
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert!(
parser.tally.is_poisoned(),
"track poisoned by >50% drop rate"
);
// Once poisoned, later good cores are dropped too, so the kept count
// (kept = emitted survivors) is far below the ~120 good cores present.
let kept = frames.len() as u64;
assert!(
kept < 120,
"post-poison good frames also dropped (kept={kept})"
);
assert!(parser.dropped_frames() > 150, "majority dropped");
}
#[test]
fn sr_validity_table_marks_reserved_codes() {
// The core-header sample-rate validity table must have ZERO (reject) at
// exactly the reserved SFREQ codes {0,4,5,9,10} and a real rate
// elsewhere — this is what mirrors ffmpeg's DCA_PARSE_ERROR_SAMPLE_RATE.
for code in 0..16usize {
let reserved = matches!(code, 0 | 4 | 5 | 9 | 10);
assert_eq!(
DTS_CORE_SR_VALID[code] == 0,
reserved,
"SFREQ code {code} reserved={reserved}"
);
}
}
#[test]
fn every_core_header_error_class_is_detected() {
// Exercise each ffmpeg-parity rejection so the port stays faithful.
// Start from a decodable core and corrupt one field at a time.
let good = make_dts_core(512);
assert_eq!(core_header_drop_reason(&good), None);
// deficit_samples != 32: clear SHORT (byte4 bits6-2) → deficit = 1.
let mut d = good.clone();
d[4] &= !0x7C;
assert_eq!(
core_header_drop_reason(&d),
Some(DropReason::DeficitSamples)
);
// npcmblocks not a multiple of 8: NBLKS low bits (byte5 bits7-2) → 14
// (npcmblocks=15, 15 & 7 = 7 ≠ 0).
let mut d = good.clone();
d[5] = (d[5] & 0x03) | (14u8 << 2);
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmBlocks));
// audio_mode >= 10: AMODE = byte7 bits3-0 (high 4) + byte8 bits7-6. Set
// AMODE high nibble to 0xF → audio_mode >= 60.
let mut d = good.clone();
d[7] |= 0x0F;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::Amode));
// sample_rate reserved: SFREQ (byte8 bits5-2) = 0.
let mut d = good.clone();
d[8] &= !0x3C;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::SampleRate));
// reserved bit set: byte9 bit4.
let mut d = good.clone();
d[9] |= 0x10;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::ReservedBit));
// lfe_present == 3: LFE is byte10 bits2-1.
let mut d = good.clone();
d[10] |= 0x06;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::LfeFlag));
// pcmr_code reserved (7): pcmr is byte11 bit0 + byte12 bits7-6 → set all
// three to 1 (code 7 → ff_dca_bits_per_sample[7] = 0).
let mut d = good.clone();
d[11] |= 0x01;
d[12] |= 0xC0;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmRes));
}
/// Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
/// through `DtsParser` and writes the emitted access units back out, so the
/// garbage-extension → core-only drop can be validated against an actual
/// damaged stream (e.g. the extracted Bourne DTS-HD MA track) end-to-end
/// with ffmpeg. Env: `DTS_IN` (input), `DTS_OUT` (output).
/// cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
#[test]
#[ignore]
fn reparse_real_dts_file() {
use std::io::Write;
let inp = std::env::var("DTS_IN").expect("DTS_IN");
let outp = std::env::var("DTS_OUT").expect("DTS_OUT");
let bytes = std::fs::read(&inp).expect("read DTS_IN");
let mut parser = DtsParser::new();
let mut out =
std::io::BufWriter::new(std::fs::File::create(&outp).expect("create DTS_OUT"));
let mut au_count = 0usize;
let mut out_bytes = 0usize;
// 90 kHz PTS advancing per chunk; arbitrary chunking is faithful because
// the framer resyncs on core sync and buffers across PES boundaries.
let mut pts: i64 = 90_000;
const CHUNK: usize = 64 * 1024;
for chunk in bytes.chunks(CHUNK) {
let pes = PesPacket {
source: None,
pid: 0x1100,
pts: Some(pts),
dts: None,
data: chunk.to_vec(),
discontinuity: false,
};
pts += 2_100; // ~one AU worth; value irrelevant to AU framing/bytes
for f in parser.parse(&pes) {
au_count += 1;
out_bytes += f.data.len();
out.write_all(&f.data).expect("write AU");
}
}
for f in parser.flush() {
au_count += 1;
out_bytes += f.data.len();
out.write_all(&f.data).expect("write AU");
}
out.flush().expect("flush");
eprintln!(
"REPARSE in={} bytes -> out={} bytes across {} AUs ({} bytes dropped)",
bytes.len(),
out_bytes,
au_count,
bytes.len().saturating_sub(out_bytes)
);
}
}
+218
View File
@@ -0,0 +1,218 @@
//! FLAC elementary-stream decodability gate.
//!
//! FLAC frames carry no length field, so a raw stream is delimited only by
//! sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
//! it comes from mp4/mkv, where each packet is exactly one container-delimited
//! FLAC frame (the `PARSER_FLAG_COMPLETE_FRAMES` case in ffmpeg). So this parser
//! is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
//! (poly 0x8005) computed so the residue over the whole frame is zero
//! (ffmpeg `flac_decode_frame`, `av_crc(AV_CRC_16_ANSI, 0, buf, len) == 0`). A
//! nonzero residue is definitive corruption → drop the frame (a silence gap,
//! never a shift — each packet keeps its own PTS), logged via the shared tally.
//!
//! A packet that does not begin with the FLAC frame sync is not a delimited
//! frame we can validate, so it is passed through unchanged (never false-dropped).
use super::crc::crc16_ansi;
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// FLAC frame sync: 14-bit code `0x3FFE` + a mandatory-0 reserved bit; the next
/// bit (blocking strategy) is masked off. ffmpeg tests `(AV_RB16 & 0xFFFE) ==
/// 0xFFF8` (flac_parser.c).
fn has_flac_sync(data: &[u8]) -> bool {
data.len() >= 2 && ((u16::from(data[0]) << 8 | u16::from(data[1])) & 0xFFFE) == 0xFFF8
}
/// Block-size code → samples, `ff_flac_blocksize_table` (0 = reserved/explicit).
const FLAC_BLOCKSIZE_TABLE: [u32; 16] = [
0, 192, 576, 1152, 2304, 4608, 0, 0, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768,
];
/// Sample-rate code → Hz, `ff_flac_sample_rate_table` (0 = STREAMINFO/explicit).
const FLAC_SAMPLE_RATE_TABLE: [u32; 16] = [
0, 88_200, 176_400, 192_000, 8_000, 16_000, 22_050, 24_000, 32_000, 44_100, 48_000, 96_000, 0,
0, 0, 0,
];
/// Best-effort duration (ns) of a FLAC frame from its header block-size and
/// sample-rate codes (byte 2). Only the table-coded cases are resolved; the
/// explicit-in-trailing-bytes codes (block 6/7, rate 12/13/14) and
/// STREAMINFO-derived (code 0) return `None`. Used only for the dropped-audio
/// accounting, so a `None` (→ 0) is harmless.
fn flac_frame_duration_ns(frame: &[u8]) -> Option<i64> {
if frame.len() < 3 {
return None;
}
let bs_code = (frame[2] >> 4) & 0x0F;
let sr_code = frame[2] & 0x0F;
let blocksize = FLAC_BLOCKSIZE_TABLE[bs_code as usize];
let rate = FLAC_SAMPLE_RATE_TABLE[sr_code as usize];
if blocksize == 0 || rate == 0 {
return None;
}
Some((blocksize as i64 * 1_000_000_000 + rate as i64 / 2) / rate as i64)
}
pub struct FlacParser {
tally: DropTally,
}
impl Default for FlacParser {
fn default() -> Self {
Self::new()
}
}
impl FlacParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("flac"),
}
}
/// Access units dropped as undecodable so far.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for FlacParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
// Gate: a packet that begins with a FLAC frame sync but whose whole-frame
// CRC-16 residue is nonzero is corrupt → drop. Anything else passes
// through (a non-sync packet is not a frame we can validate; a poisoned
// track drops everything).
let corrupt = has_flac_sync(&pes.data) && crc16_ansi(&pes.data) != 0;
if self.tally.is_poisoned() || corrupt {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"crc"
};
let dur = flac_frame_duration_ns(&pes.data).unwrap_or(0);
self.tally.record_drop(pts_ns, dur, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A minimal FLAC-frame-shaped buffer: sync `0xFFF8`, a plausible header
/// (block code 1 = 192 samples, rate code 9 = 44.1 kHz), some payload, and a
/// trailing CRC-16 so the whole-frame residue is zero (a valid frame).
fn make_flac_frame(payload_len: usize) -> Vec<u8> {
let mut f = vec![0u8; 6 + payload_len + 2];
f[0] = 0xFF;
f[1] = 0xF8; // sync + fixed blocksize
f[2] = (1 << 4) | 9; // bs_code=1 (192), sr_code=9 (44100)
// bytes 3..end-2 arbitrary; last two bytes carry the CRC-16.
let n = f.len();
let c = crc16_ansi(&f[..n - 2]);
f[n - 2] = (c >> 8) as u8;
f[n - 1] = (c & 0xFF) as u8;
assert_eq!(crc16_ansi(&f), 0, "finalized frame has zero residue");
f
}
#[test]
fn valid_frame_is_kept() {
let mut p = FlacParser::new();
let f = p.parse(&make_pes(make_flac_frame(100), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn corrupt_frame_is_dropped() {
let mut p = FlacParser::new();
let mut frame = make_flac_frame(100);
frame[20] ^= 0xFF; // corrupt a payload byte → CRC residue nonzero
assert!(crc16_ansi(&frame) != 0);
let f = p.parse(&make_pes(frame, Some(90000)));
assert!(f.is_empty(), "corrupt FLAC frame dropped");
assert_eq!(p.dropped_frames(), 1);
// 192 samples @ 44.1 kHz ≈ 4.354 ms of silence accounted.
assert_eq!(
p.dropped_duration_ns(),
(192u64 * 1_000_000_000 + 44_100 / 2) / 44_100
);
}
#[test]
fn corrupt_drop_preserves_sync_via_own_pts() {
// Each packet carries its own PTS, so dropping one leaves the next frame
// on its true timeline — a gap, not a shift.
let mut p = FlacParser::new();
let mut bad = make_flac_frame(100);
bad[20] ^= 0xFF;
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(make_flac_frame(100), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"surviving frame keeps its own container PTS — the drop is a gap"
);
}
#[test]
fn non_flac_packet_passes_through() {
// A packet without the FLAC sync isn't a frame we can validate — never
// false-drop it.
let mut p = FlacParser::new();
let f = p.parse(&make_pes(vec![0x00, 0x01, 0x02, 0x03], Some(0)));
assert_eq!(f.len(), 1, "unrecognized packet passed through");
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn empty_pes_emits_nothing() {
let mut p = FlacParser::new();
assert!(p.parse(&make_pes(Vec::new(), Some(0))).is_empty());
}
}
+114 -3
View File
@@ -56,6 +56,13 @@ pub struct H264Parser {
/// (HD-DVD EVO) path where the source stamps a PTS once per GOP. `None` on
/// the BD/UHD transport path, which carries a per-frame PTS.
reorder: Option<super::reorder::SparsePtsReorder>,
/// MVC dependent-view (Blu-ray 3D right-eye) passthrough mode. When set, the
/// parser does NOT strip SPS/PPS (nor re-assert at keyframes): every NAL —
/// subset SPS (type 15), prefix (14), coded-slice-extension (20), PPS (8) —
/// is length-prefixed in-band, so each emitted frame is a self-contained
/// dependent access unit suitable for a Matroska `BlockAdditional`. The base
/// view's avcC/param-set stripping is unchanged (separate parser instance).
mvc_passthrough: bool,
}
impl Default for H264Parser {
@@ -73,6 +80,7 @@ impl H264Parser {
cur_sps: None,
cur_pps: None,
reorder: None,
mvc_passthrough: false,
}
}
@@ -85,6 +93,15 @@ impl H264Parser {
self
}
/// Enable MVC dependent-view passthrough (see the `mvc_passthrough` field):
/// keep every parameter set in-band so each frame is a self-contained
/// dependent access unit for a Matroska `BlockAdditional`. Used only for the
/// Blu-ray 3D dependent (right-eye) stream.
pub(crate) fn with_mvc_passthrough(mut self, enabled: bool) -> Self {
self.mvc_passthrough = enabled;
self
}
/// Route a finished frame through the PTS reorderer when enabled, else emit
/// it directly (unchanged transport-stream behaviour).
fn finish(&mut self, explicit: Option<i64>, frame: Frame) -> Vec<Frame> {
@@ -192,17 +209,23 @@ impl CodecParser for H264Parser {
// frame in the mux hot path (mirrors the HEVC parser).
let mut frame_data = Vec::with_capacity(pes.data.len() + 64);
// MVC dependent-view passthrough: keep ALL param sets in-band (the frame
// is a self-contained BlockAdditional access unit), never strip/re-assert.
let mvc = self.mvc_passthrough;
for nal in NalIterator::new(&pes.data) {
let nal_type = nal[0] & 0x1F;
match nal_type {
// Param sets: seed avcC, strip if unchanged vs the active set,
// emit in-band on any change (incl. reverting to the avcC copy).
NAL_SPS => {
// In MVC passthrough these fall through to the default arm so the
// subset SPS / PPS stay in-band (self-contained dependent AU).
NAL_SPS if !mvc => {
emitted_sps |=
handle_param_set(&mut self.sps, &mut self.cur_sps, nal, &mut frame_data)
}
NAL_PPS => {
NAL_PPS if !mvc => {
emitted_pps |=
handle_param_set(&mut self.pps, &mut self.cur_pps, nal, &mut frame_data)
}
@@ -250,7 +273,7 @@ impl CodecParser for H264Parser {
// ahead of the slices (even when unchanged vs codecPrivate) so a decoder
// that dropped the set at a reset recovers, and a stale avcC re-apply
// can't revert it. Skipped per-type only when this AU already carried it.
if keyframe {
if keyframe && !mvc {
let mut prefix = Vec::new();
reassert_active(&mut prefix, &self.cur_sps, emitted_sps);
reassert_active(&mut prefix, &self.cur_pps, emitted_pps);
@@ -651,6 +674,94 @@ mod tests {
v
}
#[test]
fn mvc_passthrough_keeps_param_sets_inband() {
// A dependent-view access unit: subset SPS (NAL 15) + PPS (NAL 8) +
// coded-slice-extension (NAL 20). No IDR (type 5), so keyframe stays
// false and there is no keyframe re-assertion.
let au = || {
let mut d = Vec::new();
d.extend_from_slice(&h264_nal(0x6F, &[0x80, 0x00, 0x33, 0xAA])); // subset SPS (15)
d.extend_from_slice(&h264_nal(0x68, &[0xCE, 0x01])); // PPS (8)
d.extend_from_slice(&h264_nal(0x74, &[0x11, 0x22])); // slice-ext (20)
d
};
let nal_types =
|f: &Frame| -> Vec<u8> { h264_nals_in(&f.data).iter().map(|n| n[0] & 0x1F).collect() };
// Normal parser strips the PPS from a non-keyframe AU (it is captured for
// the avcC and, without an IDR, never re-asserted in-band).
let mut normal = H264Parser::new();
let f = normal.parse(&make_pes(au(), Some(90000)));
assert_eq!(f.len(), 1);
assert!(
!nal_types(&f[0]).contains(&8),
"normal parser strips PPS from a non-keyframe AU: {:?}",
nal_types(&f[0])
);
// Passthrough keeps EVERY parameter set in-band, so each dependent frame
// is a self-contained access unit for a BlockAdditional.
let mut pt = H264Parser::new().with_mvc_passthrough(true);
let f = pt.parse(&make_pes(au(), Some(90000)));
assert_eq!(f.len(), 1);
let types = nal_types(&f[0]);
assert!(types.contains(&15), "subset SPS kept in-band: {types:?}");
assert!(
types.contains(&8),
"PPS kept in-band under passthrough: {types:?}"
);
assert!(types.contains(&20), "slice kept: {types:?}");
}
#[test]
fn parser_for_mvc_dependent_h264_is_passthrough() {
// The dependent-view stream must get a passthrough parser: a PPS in a
// non-keyframe AU is kept in-band, not stripped like the base parser.
let mut p = crate::mux::codec::parser_for_mvc_dependent(crate::disc::Codec::H264, false);
let mut d = Vec::new();
d.extend_from_slice(&h264_nal(0x68, &[0xCE, 0x01])); // PPS (8)
d.extend_from_slice(&h264_nal(0x74, &[0x11, 0x22])); // slice-ext (20)
let f = p.parse(&make_pes(d, Some(90000)));
assert_eq!(f.len(), 1);
let types: Vec<u8> = h264_nals_in(&f[0].data)
.iter()
.map(|n| n[0] & 0x1F)
.collect();
assert!(
types.contains(&8),
"dependent parser keeps PPS in-band (passthrough): {types:?}"
);
}
#[test]
fn mvc_passthrough_with_idr_does_not_reassert_param_sets() {
// With an IDR present (keyframe=true), passthrough must NOT re-assert the
// param sets (the `keyframe && !mvc` guard), so SPS/PPS appear exactly
// once — a duplicate would corrupt the dependent BlockAdditional.
let mut p = H264Parser::new().with_mvc_passthrough(true);
let mut d = Vec::new();
d.extend_from_slice(&h264_nal(0x67, &[0x42, 0x00, 0x1E, 0x01])); // SPS (7)
d.extend_from_slice(&h264_nal(0x68, &[0xCE, 0x01])); // PPS (8)
d.extend_from_slice(&h264_nal(0x65, &[0x88, 0x00])); // IDR slice (5)
let f = p.parse(&make_pes(d, Some(90000)));
assert_eq!(f.len(), 1);
let types: Vec<u8> = h264_nals_in(&f[0].data)
.iter()
.map(|n| n[0] & 0x1F)
.collect();
assert_eq!(
types.iter().filter(|&&t| t == 7).count(),
1,
"exactly one SPS, no keyframe re-assert under passthrough: {types:?}"
);
assert_eq!(
types.iter().filter(|&&t| t == 8).count(),
1,
"exactly one PPS, no keyframe re-assert under passthrough: {types:?}"
);
}
#[test]
fn h264_populates_measured_coding_type_and_source() {
use super::super::coding::CodingType;
+48 -3
View File
@@ -9,12 +9,19 @@
/// AC-3 / E-AC-3 (Dolby Digital / Digital Plus) elementary-stream parser.
pub mod ac3;
pub mod adts;
/// Codec-agnostic per-picture coding carrier (`PictureInfo` + accessors).
pub mod coding;
/// DTS / DTS-HD elementary-stream parser.
pub(crate) mod crc;
pub(crate) mod dropgate;
pub mod dts;
/// DVD bitmap subtitle (VobSub) parser.
pub mod dvdsub;
pub mod flac;
/// H.264 (AVC) Annex-B elementary-stream parser.
pub mod h264;
/// HEVC (H.265) Annex-B elementary-stream parser.
@@ -23,6 +30,8 @@ pub mod hevc;
pub mod lpcm;
/// MPEG-2 Video elementary-stream parser.
pub mod mpeg2;
pub mod mpegaudio;
/// HDMV PGS (Presentation Graphics Stream) subtitle parser.
pub mod pgs;
/// Display-order PTS reconstruction for sparse-PTS program-stream video.
@@ -154,6 +163,25 @@ impl CodecParser for PassthroughParser {
}
}
/// Drop-on-undecodable policy across codecs ("clean muxes always"):
///
/// - **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
/// through a per-codec corruption check and drops the ones that fail, keeping
/// A/V sync (a drop is a silence gap, never a shift) and logging every drop
/// via the shared [`dropgate::DropTally`]. DTS uses ffmpeg's core-header parse;
/// AC-3 uses its native frame CRC.
/// - **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
/// data, so a corrupt sample is indistinguishable from a quiet one — there is
/// nothing to detect, so nothing can be honestly dropped.
/// - **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
/// predicted, so dropping one frame corrupts every frame that references it
/// until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
/// ResyncGate) and lets the decoder conceal — a fundamentally different model
/// than per-frame audio dropping.
/// - TrueHD/MLP, FLAC, MP2/MP3 and AAC-ADTS also gate undecodable frames via a
/// `DropTally` (poison/drop-forward for MLP's inter-AU restart state on a
/// major-sync boundary; CRC/sync-verdict drops for the passthrough codecs).
///
/// Create the appropriate parser for a codec, with optional codec private data.
///
/// For DvdSub, `codec_data` should be the pre-formatted VobSub .idx palette header.
@@ -177,6 +205,9 @@ pub fn parser_for_codec(
Codec::Mpeg2 => Box::new(mpeg2::Mpeg2Parser::new()),
Codec::Vc1 => Box::new(vc1::Vc1Parser::new().with_ps_reorder(is_dvd_ps)),
Codec::Ac3 | Codec::Ac3Plus => Box::new(ac3::Ac3Parser::new()),
Codec::Flac => Box::new(flac::FlacParser::new()),
Codec::Mp2 | Codec::Mp3 => Box::new(mpegaudio::MpegAudioParser::new()),
Codec::Aac => Box::new(adts::AdtsParser::new()),
Codec::DtsHdMa | Codec::DtsHdHr | Codec::Dts => Box::new(dts::DtsParser::new()),
Codec::TrueHd => Box::new(truehd::TrueHdParser::new()),
Codec::Pgs => Box::new(pgs::PgsParser::new()),
@@ -200,13 +231,27 @@ pub fn parser_for_codec(
// Remaining audio-only codecs (Aac, Mp2, Mp3, Flac, Opus) where PES =
// frame: all-keyframe passthrough is correct. Subtitle/Unknown also land
// here; keyframe flag is irrelevant for them.
Codec::Aac | Codec::Mp2 | Codec::Mp3 | Codec::Flac | Codec::Opus => {
Box::new(PassthroughParser::new(true))
}
Codec::Opus => Box::new(PassthroughParser::new(true)),
Codec::Srt | Codec::Ssa | Codec::Unknown(_) => Box::new(PassthroughParser::new(true)),
}
}
/// Build the codec parser for a Blu-ray 3D **MVC dependent (right-eye)** video
/// stream. Same codec space as the base view (H.264), but in param-set
/// passthrough mode so each emitted frame is a self-contained dependent access
/// unit for a Matroska `BlockAdditional`. Non-H.264 (unexpected) falls back to
/// the ordinary parser.
pub fn parser_for_mvc_dependent(codec: Codec, is_dvd_ps: bool) -> Box<dyn CodecParser> {
match codec {
Codec::H264 => Box::new(
h264::H264Parser::new()
.with_ps_reorder(is_dvd_ps)
.with_mvc_passthrough(true),
),
_ => parser_for_codec(codec, None, is_dvd_ps),
}
}
#[cfg(test)]
mod tests {
use super::*;
+224
View File
@@ -0,0 +1,224 @@
//! MPEG-1/2/2.5 audio (MP1/MP2/MP3) decodability gate.
//!
//! ffmpeg validates MPEG-audio frames by header sanity + framing resync, not a
//! payload CRC (`mpegaudiodecheader.c` `ff_mpa_check_header`; the optional CRC
//! covers only side-info and is off by default). Its `ff_mpa_decode_header`
//! additionally rejects free-format (`bitrate_index == 0`). So the gate mirrors
//! exactly those header rejects: a packet that begins with the 11-bit MPEG-audio
//! sync but whose version / layer / bitrate-index / sample-rate fields are the
//! reserved/invalid values is undecodable → drop it (a silence gap; each packet
//! keeps its own PTS). A packet with no leading sync is not a frame we can
//! validate (raw payload / continuation), so it passes through unchanged —
//! never false-dropped.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// Decoded validity of a candidate MPEG-audio header.
enum MpaVerdict {
/// No 11-bit sync at the packet head — not a frame we can validate.
NoSync,
/// Sync present and every field is legal — decodable.
Valid,
/// Sync present but a field is reserved/invalid (or free-format) — ffmpeg's
/// parser rejects this exactly.
Invalid,
}
/// Mirror ffmpeg's `ff_mpa_check_header` + the `ff_mpa_decode_header`
/// free-format reject. A dropped MPEG-audio frame has a corrupt header, so no
/// duration is computed (the fields it would come from are the invalid ones).
fn mpa_verdict(data: &[u8]) -> MpaVerdict {
if data.len() < 4 {
return MpaVerdict::NoSync;
}
let h = u32::from_be_bytes([data[0], data[1], data[2], data[3]]);
// 11-bit sync (0x7FF at the top).
if (h & 0xffe0_0000) != 0xffe0_0000 {
return MpaVerdict::NoSync;
}
// ff_mpa_check_header rejects: version field 01, layer field 00,
// bitrate_index 15, sample-rate field 3.
if (h & (3 << 19)) == (1 << 19)
|| (h & (3 << 17)) == 0
|| (h & (0xf << 12)) == (0xf << 12)
|| (h & (3 << 10)) == (3 << 10)
{
return MpaVerdict::Invalid;
}
// NOTE: bitrate_index == 0 (free format) is NOT rejected. It is a legal,
// decodable MPEG-audio mode (ffmpeg's ff_mpa_check_header accepts it and the
// decoder derives the frame size from the sync spacing). Dropping it would be
// a false positive on a clean stream, so it passes the gate.
MpaVerdict::Valid
}
pub struct MpegAudioParser {
tally: DropTally,
}
impl Default for MpegAudioParser {
fn default() -> Self {
Self::new()
}
}
impl MpegAudioParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("mpegaudio"),
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for MpegAudioParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
let drop =
self.tally.is_poisoned() || matches!(mpa_verdict(&pes.data), MpaVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid MPEG-1 Layer III header: sync 0xFFF, version MPEG-1 (11), layer
/// III (01), bitrate_index 9, sample-rate 0 (44.1 kHz), no CRC. Bytes:
/// 0xFF 0xFB 0x90 0x00 — the canonical MP3 frame header.
fn mp3_frame(payload: usize) -> Vec<u8> {
let mut f = vec![0xFF, 0xFB, 0x90, 0x00];
f.extend(std::iter::repeat(0xAA).take(payload));
f
}
#[test]
fn valid_header_is_kept() {
let mut p = MpegAudioParser::new();
let f = p.parse(&make_pes(mp3_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn reserved_sample_rate_is_dropped() {
// Sync present but sample-rate field = 3 (reserved) → ffmpeg rejects.
// 0xFF 0xFB then byte2 with bits 11..10 = 11: 0x9C.
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0x9C; // freq field = 3
let f = p.parse(&make_pes(frame, Some(90000)));
assert!(f.is_empty(), "reserved sample rate dropped");
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn reserved_layer_is_dropped() {
// Layer field 00 (reserved). byte1 bits 2..1 = 00 → 0xF9 keeps sync
// (0xFFF needs byte1 top 3 bits set) and sets layer=00.
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[1] = 0xF9; // 1111_1001: sync ok (top 3 =111), version 11, layer 00
let f = p.parse(&make_pes(frame, Some(0)));
assert!(f.is_empty(), "reserved layer dropped");
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn bad_bitrate_index_15_is_dropped() {
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0xF0; // bitrate_index = 1111
assert!(p.parse(&make_pes(frame, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn free_format_bitrate_zero_is_kept() {
// Free format (bitrate_index == 0) is legal and decodable — it must NOT
// be dropped (that would be a false positive on a clean stream).
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0x00; // bitrate_index = 0000 (free format); sync/layer/rate ok
let f = p.parse(&make_pes(frame, Some(0)));
assert_eq!(f.len(), 1, "free-format frame kept");
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn non_sync_packet_passes_through() {
// No 11-bit sync → not a validatable frame → keep (conservative).
let mut p = MpegAudioParser::new();
let f = p.parse(&make_pes(vec![0x00, 0x11, 0x22, 0x33, 0x44], Some(0)));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = MpegAudioParser::new();
let mut bad = mp3_frame(400);
bad[2] = 0x9C; // reserved sample rate
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(mp3_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
}
+119
View File
@@ -30,6 +30,90 @@ const MAX_PGS_PENDING_BYTES: usize = 4 * 1024 * 1024;
// (video_w/h, frame_rate, comp_num, comp_state, palette_update,
// palette_id_ref) = 13.
const PCS_NUM_OBJECTS_OFFSET: usize = 13;
// Offset of the first composition_object's flags byte within a PCS PES payload:
// PCS header(13) + number_of_composition_objects(1) + object_id_ref(2) +
// window_id_ref(1) = 17. `forced_on_flag` is bit 0x40 of that byte (HDMV PCS).
const PCS_FIRST_OBJECT_FLAGS_OFFSET: usize = 17;
const PCS_FORCED_ON_FLAG: u8 = 0x40;
/// Whether an emitted PGS display-set frame is a FORCED subtitle — the
/// `forced_on_flag` (0x40) on its first composition object. The frame data an
/// emitted PGS block carries begins with the display PCS (segment type 0x16), so
/// the flag is read directly from it. Returns `None` when the block is not a
/// display PCS with a composition object (nothing to classify — a clear PCS, a
/// non-PCS segment, or a truncated header).
///
/// The mux uses this to detect a *forced-narrative track* (every displayed
/// subtitle forced) without relying on the disc's vendor label metadata, so
/// forced subs are flagged `FlagForced` even on discs that carry no such blob.
pub fn display_set_is_forced(frame_data: &[u8]) -> Option<bool> {
if frame_data.first() != Some(&SEGMENT_PCS) {
return None;
}
if *frame_data.get(PCS_NUM_OBJECTS_OFFSET)? == 0 {
return None; // clear PCS — no composition to classify
}
let flags = *frame_data.get(PCS_FIRST_OBJECT_FLAGS_OFFSET)?;
Some(flags & PCS_FORCED_ON_FLAG != 0)
}
/// Accumulates the "is this PGS subtitle track a forced-narrative track?" verdict
/// from its display sets. A track is forced iff it displayed at least one subtitle
/// and EVERY display set carried the forced_on_flag — a dedicated forced track,
/// as opposed to a full track that merely has occasional forced signs.
///
/// This is the SINGLE classification used by both the MKV muxer (accumulating a
/// track's frames during a rip) and the `info`-time forced probe (feeding the
/// demuxed display sets), so both reach the identical verdict.
#[derive(Debug, Clone)]
pub struct ForcedTracker {
has_display: bool,
all_forced: bool,
}
impl Default for ForcedTracker {
fn default() -> Self {
Self {
has_display: false,
all_forced: true,
}
}
}
impl ForcedTracker {
pub fn new() -> Self {
Self::default()
}
/// Fold one emitted PGS block into the verdict. Non-display blocks (clear
/// PCS, other segments) are ignored.
pub fn observe(&mut self, frame_data: &[u8]) {
if let Some(forced) = display_set_is_forced(frame_data) {
self.has_display = true;
self.all_forced &= forced;
}
}
/// Whether the track has already shown a NON-forced subtitle — i.e. its
/// verdict is settled at "not forced" and further observation can be skipped
/// (the early-exit the probe uses to avoid reading the whole clip).
pub fn settled_not_forced(&self) -> bool {
self.has_display && !self.all_forced
}
/// Whether ANY display set was observed. When false the track's forced state
/// is unknown (no PGS content seen — e.g. an undecrypted/unread stream), so a
/// probe should leave any existing (vendor-derived) flag untouched rather
/// than assert "not forced".
pub fn observed(&self) -> bool {
self.has_display
}
/// Final verdict: forced iff it displayed subtitles and every one was forced.
pub fn is_forced(&self) -> bool {
self.has_display && self.all_forced
}
}
/// Stateful parser that collapses PGS display/clear PCS pairs into
/// duration-bearing Matroska frames. Implements [`CodecParser`].
@@ -222,6 +306,41 @@ mod tests {
use super::*;
use crate::mux::ts::PesPacket;
/// A PCS display-set block with one composition object; `forced` sets the
/// forced_on_flag (0x40) in its flags byte at offset 17.
fn pcs_display(forced: bool) -> Vec<u8> {
let mut d = vec![0u8; 18];
d[0] = SEGMENT_PCS;
d[PCS_NUM_OBJECTS_OFFSET] = 1;
d[PCS_FIRST_OBJECT_FLAGS_OFFSET] = if forced { PCS_FORCED_ON_FLAG } else { 0 };
d
}
#[test]
fn display_set_forced_flag_detection() {
assert_eq!(display_set_is_forced(&pcs_display(true)), Some(true));
assert_eq!(display_set_is_forced(&pcs_display(false)), Some(false));
// Other flag bits set but not forced_on_flag → still not forced.
let mut cropped = pcs_display(false);
cropped[PCS_FIRST_OBJECT_FLAGS_OFFSET] = 0x80; // object_cropped_flag only
assert_eq!(display_set_is_forced(&cropped), Some(false));
}
#[test]
fn display_set_forced_none_for_non_display() {
// Clear PCS (0 objects) → None.
let mut clear = pcs_display(false);
clear[PCS_NUM_OBJECTS_OFFSET] = 0;
assert_eq!(display_set_is_forced(&clear), None);
// Non-PCS segment → None.
let mut ods = pcs_display(true);
ods[0] = 0x15; // ODS
assert_eq!(display_set_is_forced(&ods), None);
// Truncated (no flags byte) → None, no panic.
assert_eq!(display_set_is_forced(&pcs_display(true)[..15]), None);
assert_eq!(display_set_is_forced(&[]), None);
}
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
+562 -20
View File
@@ -12,6 +12,8 @@
//! AC-3 frames (interleaved, same PID): start with sync word 0x0B77.
//! We skip AC-3 frames and only emit TrueHD access units.
use super::crc::crc16_mlp;
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
use crate::mux::timeline::DISCONTINUITY_BACKSTEP_NS;
@@ -43,6 +45,18 @@ pub struct TrueHdParser {
/// yet seen (head of stream) — preserving byte-identical timing for the
/// common 48 kHz case.
au_duration_ns: i64,
/// Keep/drop bookkeeping for the decodability gate.
tally: DropTally,
/// `num_substreams` from the most recent major sync — needed to size the
/// substream directory for the per-AU parity check. `None` until the first
/// major sync is seen (before which no AU can be parity-checked).
num_substreams: Option<u8>,
/// True while dropping forward to the next clean resync point. MLP/TrueHD
/// carries filter/predictor + restart state ACROSS access units, so a corrupt
/// AU cannot be excised in place — it poisons decoding until the next major
/// sync re-initialises state. On corruption we set this and drop every AU
/// until a major sync whose header CRC validates, which we then emit.
resync_pending: bool,
}
impl Default for TrueHdParser {
@@ -57,6 +71,82 @@ impl TrueHdParser {
buf: Vec::with_capacity(32768),
next_pts_ns: 0,
au_duration_ns: AU_DURATION_NS,
tally: DropTally::new("truehd"),
num_substreams: None,
resync_pending: false,
}
}
/// Access units dropped as undecodable so far.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Decide whether an access unit is corrupt, updating `num_substreams` from a
/// valid major sync. Mirrors ffmpeg `read_access_unit`: a major sync with a
/// bad header CRC, or any AU whose header parity fails, is undecodable.
/// Returns `false` (not corrupt) when the AU is too short to judge or no
/// major sync has established `num_substreams` yet — we never drop what we
/// cannot verify. Verified against real ffmpeg TrueHD (3600/3600 AUs).
fn au_check(&mut self, au: &[u8], is_major_sync: bool) -> AuCheck {
let mut header_size = 4;
let mut format_info = None;
if is_major_sync {
let ms = &au[4..];
let Some(mshdr) = mlp_major_sync_header_size(ms) else {
// A major sync too short to hold its header can't be CRC-validated
// — NOT a safe resync/re-init point. Treat as unverifiable, not a
// clean major sync.
return AuCheck::Unverifiable;
};
if !mlp_major_sync_crc_ok(ms, mshdr) {
// A failing major-sync checksum is only a TRUSTWORTHY corruption
// signal once we already hold a validated baseline (num_substreams
// captured from a PRIOR checksum-clean major sync). Before that — at
// stream head, when no major sync has validated yet — a checksum
// mismatch is far more likely a limitation of our own major-sync
// header-size parse than real corruption. Arming the drop-forward
// there is catastrophic: every following AU is collateral-dropped
// waiting for a "validated" major sync that (if our parse is the
// problem) NEVER comes, so the WHOLE TrueHD track is silently dropped
// — the AUs are physically emitted late (empty until the mux end),
// de-interleaving the track and sending decoders into an unbounded
// memory spiral. So until we have a baseline to protect, treat a
// checksum-failed major sync as UNVERIFIABLE (keep it) rather than
// Corrupt. Once a clean major sync HAS established the baseline, a
// later failure is a real re-sync trigger and still drops forward.
if self.num_substreams.is_some() {
return AuCheck::Corrupt; // real corruption vs a proven baseline
}
return AuCheck::Unverifiable; // no baseline yet — keep, don't nuke the track
}
self.num_substreams = mlp_num_substreams(ms);
header_size += mshdr;
// The rate nibble is only trustworthy once the major sync's CRC has
// validated (above), so capture format_info here and refine the PTS
// cadence from it ONLY on this validated path.
if au.len() >= 12 {
format_info = Some(u32::from_be_bytes([au[8], au[9], au[10], au[11]]));
}
}
let Some(nss) = self.num_substreams else {
return AuCheck::Unverifiable; // no major sync seen yet — can't check parity
};
let Some(shs) = mlp_substr_header_size(au, header_size, nss) else {
return AuCheck::Unverifiable; // directory runs off the AU — can't judge
};
if !mlp_parity_ok(au, header_size, shs) {
return AuCheck::Corrupt;
}
if is_major_sync {
AuCheck::ValidMajorSync { format_info }
} else {
AuCheck::Ok
}
}
@@ -114,6 +204,23 @@ fn ac3_boundary_corroborated(buf: &[u8], frame_bytes: usize) -> bool {
next_words != 0 && next_words * 2 <= 32768
}
/// Decodability verdict for one TrueHD/MLP access unit.
enum AuCheck {
/// Verified undecodable: a major-sync header whose CRC failed, or any AU
/// whose substream-directory parity failed. Feeds the poison verdict.
Corrupt,
/// A CRC-validated major sync — a safe re-init / resync point. `format_info`
/// (AU bytes 8..12, present when the AU is long enough) is trustworthy here,
/// so the caller refines the PTS cadence ONLY from this validated path.
ValidMajorSync { format_info: Option<u32> },
/// A valid (parity-OK) non-major-sync access unit.
Ok,
/// Cannot be judged — a major sync too short to hold/CRC its header, or a
/// stream head before any major sync established `num_substreams`. Never
/// dropped on its own, and never treated as a clean resync point.
Unverifiable,
}
/// Outcome of sizing the AC-3 frame at the TrueHD buffer head.
enum Ac3Size {
/// fscod/frmsizecod don't map to a real frame size — resync, don't wait.
@@ -124,6 +231,92 @@ enum Ac3Size {
Frame(usize),
}
// --- MLP/TrueHD access-unit integrity (mirrors ffmpeg mlpdec.c / mlp_parse.c) ---
/// Major-sync header size in bytes: base 28, plus `2 + extensions*2` when the
/// extension flag (major-sync byte 25, bit 0) is set (`extensions` = byte 26
/// high nibble). `ms` is the major-sync header, i.e. AU bytes `[4..]`. `None`
/// when the AU is too short to contain the full header.
fn mlp_major_sync_header_size(ms: &[u8]) -> Option<usize> {
if ms.len() < 28 {
return None;
}
let mut size = 28;
if ms[25] & 1 != 0 {
size += 2 + ((ms[26] >> 4) as usize) * 2;
}
if ms.len() < size {
return None;
}
Some(size)
}
/// Validate the MLP/TrueHD major-sync header checksum (ffmpeg `ff_mlp_checksum16`,
/// CRC-16 poly 0x002D). The stored trailer is the last 2 header bytes; because
/// MLP's checksum is byte-reversed relative to a standard CRC, a standard CRC of
/// the header body XOR the little-endian word before the trailer must equal the
/// trailer read big-endian.
fn mlp_major_sync_crc_ok(ms: &[u8], mshdr: usize) -> bool {
if mshdr < 4 || ms.len() < mshdr {
return false;
}
// ffmpeg `ff_mlp_checksum16(buf, buf_size)` (libavcodec/mlp.c):
// av_crc(crc_2D, 0, buf, buf_size - 2) ^ AV_RL16(buf + buf_size - 2)
// is called with `buf_size = mshdr - 2` and its result compared to
// `AV_RL16(buf + mshdr - 2)` — i.e. the 16-bit checksum over `ms[..mshdr-4]`,
// XORed with the LITTLE-ENDIAN word just before the trailer, must equal the
// LITTLE-ENDIAN trailer word. `crc16_mlp` is the same crc_2D table (poly 0x2D,
// MSB-first) but yields its two bytes in the OPPOSITE order to libavutil's
// `av_crc`, so swap them back to match. (The previous code mixed endianness —
// little-endian XOR word but big-endian compare — so the checksum could never
// validate any real extended major sync, silently dropping the whole track;
// cross-verified byte-exact against real 7.1/Atmos and 5.1 discs.)
let checksum = crc16_mlp(&ms[..mshdr - 4]).swap_bytes()
^ u16::from_le_bytes([ms[mshdr - 4], ms[mshdr - 3]]);
checksum == u16::from_le_bytes([ms[mshdr - 2], ms[mshdr - 1]])
}
/// `num_substreams` from a major-sync header: it sits at bit 128 (byte 16, top
/// nibble) for both MLP (0xbb) and TrueHD (0xba) — the fields before it total
/// the same 128 bits in either layout.
fn mlp_num_substreams(ms: &[u8]) -> Option<u8> {
ms.get(16).map(|&b| b >> 4)
}
/// Size in bytes of the substream directory that follows the AU header: each of
/// the `num_substreams` entries is 2 bytes, plus 2 more when its extraword flag
/// (entry's top bit) is set. `None` if the directory runs past the AU.
fn mlp_substr_header_size(au: &[u8], header_size: usize, num_substreams: u8) -> Option<usize> {
let mut off = header_size;
let mut shs = 0;
for _ in 0..num_substreams {
if off + 2 > au.len() {
return None;
}
let extraword = au[off] & 0x80 != 0;
shs += 2;
off += 2;
if extraword {
shs += 2;
off += 2;
}
}
Some(shs)
}
/// MLP/TrueHD AU-header parity check (ffmpeg `ff_mlp_calculate_parity`): the XOR
/// of the 4-byte AU header with the substream directory, folded, must have its
/// two nibbles XOR to 0xF.
fn mlp_parity_ok(au: &[u8], header_size: usize, substr_header_size: usize) -> bool {
let end = header_size + substr_header_size;
if end > au.len() {
return false;
}
let xor_fold = |d: &[u8]| d.iter().fold(0u8, |a, &b| a ^ b);
let p = xor_fold(&au[0..4]) ^ xor_fold(&au[header_size..end]);
((p >> 4) ^ p) & 0xF == 0xF
}
impl CodecParser for TrueHdParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
// B1: a concealed/lost gap means the buffered TrueHD AU is TRUNCATED.
@@ -256,26 +449,93 @@ impl CodecParser for TrueHdParser {
& 0xFFFF_FFFE)
== 0xF872_6FBA;
// On a major sync the 32-bit `format_info` word (immediately after
// the 4-byte sync, i.e. AU bytes 8..12) carries the rate nibble.
// Refine the per-AU PTS increment to the actual rate family. The
// 48 kHz family resolves to the unchanged 833_333 default, so the
// common case stays byte-identical; only the 44.1 kHz family shifts.
if is_major_sync && unit_bytes >= 12 {
let format_info =
u32::from_be_bytes([self.buf[8], self.buf[9], self.buf[10], self.buf[11]]);
self.au_duration_ns = truehd_au_duration_ns(format_info);
// Decodability gate. MLP/TrueHD decode state persists across access
// units, so a corrupt AU is dropped FORWARD to the next VALIDATED
// major sync (the clean re-init point) rather than excised in place.
// The PTS clock advances across every dropped AU so a drop is a
// silence gap, never a shift.
let au = self.buf[..unit_bytes].to_vec();
let pts = self.next_pts_ns;
let mut emit_keyframe: Option<bool> = None; // Some(is_keyframe) => emit
let mut drop_reason: Option<(&'static str, bool)> = None; // (reason, verified)
if self.tally.is_poisoned() {
// Whole track already judged dead — collateral drop (does not
// re-feed the poison verdict).
drop_reason = Some(("track-poisoned", false));
} else {
match self.au_check(&au, is_major_sync) {
AuCheck::ValidMajorSync { format_info } => {
// The rate nibble is trustworthy only now that the major
// sync's CRC has validated. Refine the per-AU PTS
// increment (48 kHz family stays the 833_333 default).
if let Some(fi) = format_info {
self.au_duration_ns = truehd_au_duration_ns(fi);
}
// A validated major sync is the ONLY clean resync point.
self.resync_pending = false;
emit_keyframe = Some(true);
}
AuCheck::Corrupt => {
if self.resync_pending {
// Part of the current drop-forward run — collateral.
drop_reason = Some(("resync", false));
} else {
// The trigger: one verified corruption that starts the
// drop-forward. Only this counts toward poison.
let r = if is_major_sync {
"major-sync-crc"
} else {
"parity"
};
drop_reason = Some((r, true));
self.resync_pending = true;
}
}
AuCheck::Ok => {
if self.resync_pending {
// Decode state is invalid until the next validated
// major sync, so even a parity-OK AU is undecodable
// here — collateral drop.
drop_reason = Some(("resync", false));
} else {
emit_keyframe = Some(false);
}
}
AuCheck::Unverifiable => {
if self.resync_pending {
// Not a validated major sync — do NOT clear the resync
// on it; keep dropping forward.
drop_reason = Some(("resync", false));
} else {
// Head of stream / too-short AU: keep (never drop what
// we cannot verify).
emit_keyframe = Some(is_major_sync);
}
}
}
}
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.next_pts_ns,
keyframe: is_major_sync,
data: self.buf[..unit_bytes].to_vec(),
duration_ns: None,
});
if let Some(keyframe) = emit_keyframe {
self.tally.record_kept();
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: pts,
keyframe,
data: au,
duration_ns: None,
});
} else if let Some((reason, verified)) = drop_reason {
if verified {
self.tally
.record_drop(pts, self.au_duration_ns, au.len(), reason);
} else {
self.tally
.record_collateral_drop(pts, self.au_duration_ns, au.len(), reason);
}
}
self.buf.drain(..unit_bytes);
self.next_pts_ns += self.au_duration_ns;
}
@@ -289,6 +549,11 @@ impl CodecParser for TrueHdParser {
frames
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
@@ -451,6 +716,275 @@ mod tests {
data
}
/// Turn a synthetic major-sync AU (sync bytes already set at offset 4, any
/// `format_info` set) into one that passes the decodability gate: 1 substream,
/// a clean substream directory, a valid major-sync CRC-16, and a valid header
/// parity nibble. Mirrors what a real encoder writes (verified against real
/// ffmpeg TrueHD). The AU must be ≥ 36 bytes (4 AU header + 28 major-sync
/// header + 2 directory + slack), which every `make_truehd_unit(≥200)` is.
fn finalize_major_sync(au: &mut [u8]) {
const MSHDR: usize = 28; // no extension (byte 25 clear)
// num_substreams = 1 → major-sync byte 16 (AU[20]) top nibble.
au[20] = (au[20] & 0x0F) | 0x10;
// Substream directory entry at AU[4+MSHDR] = AU[32]: extraword flag clear.
au[32] &= 0x7F;
// Major-sync checksum, built EXACTLY as `mlp_major_sync_crc_ok` verifies it
// (ffmpeg `ff_mlp_checksum16`): swap_bytes(crc16_mlp(body)) ^ LE word before
// the trailer, stored little-endian in the trailer.
let body_end = 4 + MSHDR - 4; // AU[4..28]
let crc = super::crc16_mlp(&au[4..body_end]).swap_bytes()
^ u16::from_le_bytes([au[body_end], au[body_end + 1]]);
au[4 + MSHDR - 2] = (crc & 0xFF) as u8;
au[4 + MSHDR - 1] = (crc >> 8) as u8;
// Parity: choose the AU check nibble (AU[0] high bits) so the header +
// directory fold to 0xF. The length low nibble (AU[0] low bits) is kept.
let hi = au[0] & 0x0F;
let p0 = (hi ^ au[1] ^ au[2] ^ au[3]) ^ (au[32] ^ au[33]);
let c = ((p0 >> 4) ^ (p0 & 0x0F) ^ 0x0F) & 0x0F;
au[0] = (c << 4) | hi;
}
/// Give a synthetic NON-major-sync AU a valid header parity nibble (1
/// substream, directory at AU[4..6]), so it passes the gate once a preceding
/// major sync has established `num_substreams`.
fn finalize_normal_parity(au: &mut [u8]) {
au[4] &= 0x7F; // no extraword
let hi = au[0] & 0x0F;
let p0 = (hi ^ au[1] ^ au[2] ^ au[3]) ^ (au[4] ^ au[5]);
let c = ((p0 >> 4) ^ (p0 & 0x0F) ^ 0x0F) & 0x0F;
au[0] = (c << 4) | hi;
}
fn valid_major_sync() -> Vec<u8> {
let mut u = make_truehd_unit(200);
u[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
finalize_major_sync(&mut u);
u
}
fn valid_normal_au() -> Vec<u8> {
let mut u = make_truehd_unit(200);
finalize_normal_parity(&mut u);
u
}
#[test]
fn corrupt_major_sync_drops_forward_to_next_valid() {
// MLP state carries across AUs, so a corrupt AU is dropped FORWARD to the
// next valid major sync (the clean re-init point). Sequence: valid MS,
// corrupt MS (bad CRC), a normal AU, then a valid MS. Only the two valid
// major syncs survive; the corrupt MS and the intervening normal AU are
// dropped (the latter because decode state is poisoned until re-init).
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF; // corrupt a CRC-covered header byte
let normal = valid_normal_au(); // clean parity, but arrives mid-resync
let ms2 = valid_major_sync();
let mut data = ms1.clone();
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&normal);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "only the two valid major syncs survive");
assert!(frames[0].keyframe && frames[1].keyframe);
assert_eq!(
parser.dropped_frames(),
2,
"corrupt MS + poisoned normal AU"
);
}
#[test]
fn crc_failed_head_major_sync_is_kept_not_track_killed() {
// REGRESSION (every TrueHD title muxed after the checksum gate landed): a
// real-world major sync whose checksum our own header-size parse can't
// validate must NOT, at stream head — before ANY major sync has validated —
// arm the drop-forward. Doing so latched `resync_pending` on the very first
// AU and collateral-dropped EVERY following AU forever (no "validated" major
// sync ever came), silently dropping the entire TrueHD track: its AUs were
// emitted only at mux end, de-interleaving the track and spiralling decoders
// into unbounded memory ("decoder ran out of memory"). With no baseline to
// protect, a checksum-failed major sync is KEPT and the audio flows.
let mut parser = TrueHdParser::new();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF; // break a checksum-covered header byte → checksum fails
let mut data = ms_bad;
for _ in 0..6 {
data.extend_from_slice(&valid_normal_au());
}
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(
frames.len(),
7,
"no baseline yet: the CRC-failed head major sync + all following AUs are \
kept, not dropped (got {})",
frames.len()
);
assert_eq!(
parser.dropped_frames(),
0,
"nothing dropped without a validated baseline to protect"
);
// And the invariant still holds ONCE a baseline exists: after a genuinely
// valid major sync, a later corrupt one IS dropped (see
// `corrupt_major_sync_drops_forward_to_next_valid`).
}
#[test]
fn parity_failure_is_dropped() {
// A normal AU whose header parity is broken (after a major sync sets
// num_substreams) is undecodable → dropped.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut bad = valid_normal_au();
// A single-nibble flip: MLP's nibble-fold parity (like ffmpeg's) is blind
// to a full-byte flip, which changes both nibbles equally and cancels.
bad[2] ^= 0x01;
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&bad);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "the parity-broken AU is dropped");
assert_eq!(parser.dropped_frames(), 1);
}
#[test]
fn drop_forward_preserves_av_sync_no_shift() {
// THE INVARIANT: the resumed major sync keeps the exact PTS it would have
// had with no drop — base + 3 AU durations (MS1, corrupt-MS, normal, MS2)
// — so the drop is a silence gap, never a shift.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF;
let normal = valid_normal_au();
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&normal);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2);
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
3 * AU_DURATION_NS,
"resumed major sync keeps its true timeline (gap, not shift)"
);
}
#[test]
fn transient_corruptions_do_not_poison_whole_track() {
// Regression (audit HIGH): TrueHD drop-forward must NOT amplify a couple
// of transient errors into a false whole-track poison. Two corruptions,
// each forcing a long collateral resync run past 200 total AUs, must
// leave the track un-poisoned and keep the good audio that follows.
let mut parser = TrueHdParser::new();
let mut data = valid_major_sync();
// Corruption #1 then a long run of normal AUs (all collateral-dropped
// while resyncing — no major sync to re-init on).
let mut bad1 = valid_normal_au();
bad1[2] ^= 0x01; // single-nibble parity break
data.extend_from_slice(&bad1);
for _ in 0..210 {
data.extend_from_slice(&valid_normal_au());
}
// A valid major sync resumes; the good AUs after it MUST be kept.
data.extend_from_slice(&valid_major_sync());
for _ in 0..5 {
data.extend_from_slice(&valid_normal_au());
}
let frames = parser.parse(&make_pes(data, Some(90000)));
assert!(
!parser.tally.is_poisoned(),
"two transient errors must not poison the track"
);
// MS1 + resumed MS2 + the 5 good AUs after it survive.
assert_eq!(
frames.len(),
7,
"post-resync good audio is kept, not poisoned away"
);
assert!(
parser.dropped_frames() > 200,
"the resync run was still counted for reporting"
);
}
#[test]
fn corrupt_major_sync_rate_nibble_does_not_shift_pts() {
// Regression (audit MED): a corrupt major sync whose rate nibble decodes
// to the 44.1 kHz family must NOT refine au_duration_ns — the rate is only
// trustworthy after the CRC validates. Otherwise the resumed 48 kHz audio
// is shifted (not gapped).
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync(); // 48 kHz
let mut ms_bad = valid_major_sync();
// Set the rate nibble (top nibble of format_info = au[8]) to 0x8 (44.1k).
// au[8] is CRC-covered, so this also breaks the major-sync CRC → corrupt.
ms_bad[8] = (ms_bad[8] & 0x0F) | 0x80;
let ms2 = valid_major_sync(); // 48 kHz
let mut data = ms1;
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&ms2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2, "corrupt MS dropped; MS1 and MS2 survive");
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
2 * AU_DURATION_NS,
"resumed audio keeps the 48 kHz cadence — the corrupt MS's 44.1k rate was ignored"
);
}
#[test]
fn too_short_major_sync_does_not_clear_resync() {
// Regression (audit LOW): while resyncing, a major sync too short to hold
// (and CRC-validate) its header must NOT be treated as a clean resync
// point — the runt is dropped and only a real validated major sync resumes.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut bad = valid_normal_au();
bad[2] ^= 0x01; // parity break → triggers resync
// An 8-byte "major sync": length=4 words, sync at bytes 4..8, too short
// to hold the 28-byte major-sync header.
let runt = vec![0x00, 0x04, 0x00, 0x00, 0xF8, 0x72, 0x6F, 0xBA];
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&bad);
data.extend_from_slice(&runt);
data.extend_from_slice(&ms2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2, "the runt major sync did not resume decode");
for f in &frames {
assert_eq!(
f.data.len(),
200,
"only the real 200-byte major syncs survive"
);
}
}
#[test]
fn clean_truehd_stream_drops_nothing() {
// A run of valid AUs passes untouched — zero false positives (the CRC and
// parity are verified against real ffmpeg TrueHD output).
let mut parser = TrueHdParser::new();
let mut data = valid_major_sync();
for _ in 0..5 {
data.extend_from_slice(&valid_normal_au());
}
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 6);
assert_eq!(parser.dropped_frames(), 0);
}
fn make_ac3_frame() -> Vec<u8> {
// Minimal AC-3 frame: sync 0x0B77, fscod=0 (48kHz), frmsizecod=0 (64 words = 128 bytes)
let mut data = vec![0u8; 128];
@@ -878,6 +1412,7 @@ mod tests {
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
finalize_major_sync(&mut unit);
let f = parser.parse(&make_pes(unit, Some(90000)));
assert_eq!(f.len(), 1);
assert!(f[0].keyframe, "major-sync AU must be flagged keyframe");
@@ -899,6 +1434,7 @@ mod tests {
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBBu32.to_be_bytes());
finalize_major_sync(&mut unit);
let f = parser.parse(&make_pes(unit, Some(90000)));
assert_eq!(f.len(), 1);
assert!(f[0].keyframe, "major-sync variant 0xFB also a keyframe");
@@ -1101,8 +1637,11 @@ mod tests {
let mut a1 = make_truehd_unit(200);
a1[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes()); // major sync
a1[8..12].copy_from_slice(&format_info_with(0x8).to_be_bytes()); // 44.1 k
finalize_major_sync(&mut a1);
let mut a2 = make_truehd_unit(200);
finalize_normal_parity(&mut a2);
let mut data = a1;
data.extend_from_slice(&make_truehd_unit(200));
data.extend_from_slice(&a2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2);
assert_eq!(
@@ -1120,8 +1659,11 @@ mod tests {
let mut a1 = make_truehd_unit(200);
a1[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
a1[8..12].copy_from_slice(&format_info_with(0x0).to_be_bytes()); // 48 k
finalize_major_sync(&mut a1);
let mut a2 = make_truehd_unit(200);
finalize_normal_parity(&mut a2);
let mut data = a1;
data.extend_from_slice(&make_truehd_unit(200));
data.extend_from_slice(&a2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2);
assert_eq!(frames[1].pts_ns - frames[0].pts_ns, 833_333);
+86 -10
View File
@@ -31,6 +31,10 @@ use std::io::{self, BufWriter, Write};
use std::path::{Path, PathBuf};
/// Filename-naming strategy for the per-track files.
// `allow(dead_code)`: the sink honours all variants, but only the `#[default]` is
// constructed today (`output()` builds `DemuxOptions::default()`). The alternates
// are a staged option surface awaiting the CLI `--naming` flag (not yet wired).
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum Naming {
/// `<base> <track> <lang> <codec> [DELAY <n>ms].<ext>` — human-readable.
@@ -55,6 +59,9 @@ pub enum DelayMode {
}
/// Chapter export format.
// `allow(dead_code)`: only the `#[default]` XML variant is constructed today (via
// `DemuxOptions::default()`); OGM/Both await the CLI `--chapters` flag.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ChaptersFmt {
/// mkvmerge chapter XML.
@@ -81,6 +88,10 @@ pub struct DemuxOptions {
pub export_chapters: bool,
/// Selected track indices. `None` = all tracks.
pub selection: Option<Vec<usize>>,
/// Restrict output to one track class. `None` = every class (plain
/// `demux://`). `Some(Audio)` is the `audio://` sink; `Some(Subtitle)` is
/// `sub://`. Filtered tracks are skipped entirely (no file written).
pub kind_filter: Option<TrackKind>,
}
impl Default for DemuxOptions {
@@ -92,13 +103,16 @@ impl Default for DemuxOptions {
chapters_fmt: ChaptersFmt::default(),
export_chapters: true,
selection: None,
kind_filter: None,
}
}
}
/// Track class, used for delay attribution and naming.
/// Track class, used for delay attribution and naming — and, via
/// [`DemuxOptions::kind_filter`], to restrict a demux to one class (the
/// `audio://` / `sub://` sinks are a `demux://` filtered to Audio / Subtitle).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum TrackKind {
pub enum TrackKind {
Video,
Audio,
Subtitle,
@@ -136,7 +150,7 @@ fn extension_for(codec: Codec) -> &'static str {
}
/// Short codec label for friendly filenames.
fn codec_label(codec: Codec) -> &'static str {
pub(crate) fn codec_label(codec: Codec) -> &'static str {
match codec {
Codec::Hevc => "HEVC",
Codec::H264 => "AVC",
@@ -241,11 +255,23 @@ impl EsWriter for AnnexBWriter {
/// Delegates to the canonical hvcC/avcC → Annex-B converters in
/// [`crate::mux::hevc`] — the single source of truth across all muxers.
fn annexb_param_sets(codec: Codec, record: &[u8]) -> Vec<u8> {
match codec {
Codec::Hevc => hvcc_to_annex_b(record).unwrap_or_default(),
Codec::H264 => avcc_to_annex_b(record).unwrap_or_default(),
_ => Vec::new(),
}
let converted = match codec {
Codec::Hevc => hvcc_to_annex_b(record),
Codec::H264 => avcc_to_annex_b(record),
_ => return Vec::new(),
};
converted.unwrap_or_else(|| {
// A malformed hvcC/avcC record yields no parameter sets. Returning empty
// means keyframes ship WITHOUT in-band SPS/PPS — playable from the first
// keyframe but broken for seek-to-arbitrary-point and hardware decoders.
// Surface it rather than silently degrading the output.
tracing::warn!(
target: "mux",
?codec,
"codec-private (hvcC/avcC) parse failed; keyframes will lack in-band SPS/PPS"
);
Vec::new()
})
}
/// PGS `.sup` writer: rebuilds the HDMV segment framing the parser stripped.
@@ -522,7 +548,7 @@ fn fmt_chapter_time_ns(time_secs: f64) -> String {
}
/// Serialize chapters as mkvmerge chapter XML.
fn chapters_xml(chapters: &[Chapter]) -> String {
pub(crate) fn chapters_xml(chapters: &[Chapter]) -> String {
let mut s = String::new();
s.push_str("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
s.push_str("<!DOCTYPE Chapters SYSTEM \"matroskachapters.dtd\">\n");
@@ -552,7 +578,7 @@ fn chapters_xml(chapters: &[Chapter]) -> String {
}
/// Serialize chapters as OGM/simple chapter text.
fn chapters_ogm(chapters: &[Chapter]) -> String {
pub(crate) fn chapters_ogm(chapters: &[Chapter]) -> String {
let mut s = String::new();
for (i, c) in chapters.iter().enumerate() {
let n = i + 1;
@@ -639,6 +665,11 @@ impl DemuxSink {
(TrackKind::Subtitle, s.codec, s.pid, s.language.clone())
}
};
// Kind filter: `audio://` / `sub://` keep only their class.
if opts.kind_filter.is_some_and(|k| k != kind) {
tracks.push(None);
continue;
}
if kind == TrackKind::Video && ref_video_track.is_none() {
ref_video_track = Some(idx);
}
@@ -855,6 +886,17 @@ mod tests {
})
}
fn subtitle_stream(codec: Codec, lang: &str) -> DiscStream {
DiscStream::Subtitle(crate::disc::SubtitleStream {
pid: 0x1200,
codec,
language: lang.to_string(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
})
}
fn title_with(streams: Vec<DiscStream>, privates: Vec<Option<Vec<u8>>>) -> DiscTitle {
let mut t = DiscTitle::empty();
t.streams = streams;
@@ -863,6 +905,40 @@ mod tests {
t
}
/// `audio://` and `sub://` are `demux://` with a kind filter: only tracks of
/// the selected class get a file; every other track is skipped entirely.
#[test]
fn kind_filter_keeps_only_the_selected_class() {
let title = title_with(
vec![
video_stream(Codec::H264),
audio_stream(Codec::Ac3, "eng"),
subtitle_stream(Codec::Pgs, "eng"),
],
vec![None, None, None],
);
let sub_opts = DemuxOptions {
kind_filter: Some(TrackKind::Subtitle),
export_chapters: false,
..Default::default()
};
let sub = DemuxSink::create(&tempdir(), &title, &sub_opts).unwrap();
assert!(
sub.tracks[0].is_none() && sub.tracks[1].is_none() && sub.tracks[2].is_some(),
"sub:// keeps only the subtitle track"
);
let audio_opts = DemuxOptions {
kind_filter: Some(TrackKind::Audio),
export_chapters: false,
..Default::default()
};
let audio = DemuxSink::create(&tempdir(), &title, &audio_opts).unwrap();
assert!(
audio.tracks[0].is_none() && audio.tracks[1].is_some() && audio.tracks[2].is_none(),
"audio:// keeps only the audio track"
);
}
// ── Annex-B reframing ────────────────────────────────────────────────────
//
// The length-prefixed → Annex-B conversion and the hvcC/avcC param-set
+1 -1
View File
@@ -7,7 +7,7 @@
//! With [`crate::sector::PrefetchedSectorSource`] alone, read+decrypt
//! already runs on a producer thread; the *consumer* (main) thread
//! still serialises `ts_demuxer.feed` (M2TS parsing) with the codec
//! parsers. Profiling on the rip1 testbed showed feed at ~37 % and
//! parsers. Profiling showed feed at ~37 % and
//! codec parse at ~44 % of consumer wall time — i.e. feed is heavy
//! enough that pipelining it with parse pays for itself.
//!
+211 -36
View File
@@ -108,11 +108,6 @@ pub struct DiscStream {
/// inline `decrypt::decrypt_sectors` step. `DecryptKeys::None`
/// (raw / unencrypted disc) makes the decorator a pass-through.
reader: DecryptingSectorSource<Box<dyn SectorSource>>,
/// Shared decrypt-loss counter, cloned once at construction from
/// `reader.decrypt_loss()`. `lost_bytes()` loads it directly so the
/// per-frame hot path performs no per-call `Arc::clone` (matching the
/// `PipelinedPesStream` pattern).
decrypt_loss: std::sync::Arc<std::sync::atomic::AtomicU64>,
title: DiscTitle,
/// Mirror of the keys handed in at construction. The decorator
/// owns the cryptographic state; this field is kept for
@@ -144,14 +139,14 @@ pub struct DiscStream {
// Adaptive batch sizer — preferred comes from the caller
// (detect_max_batch_sectors), shrinks/grows based on read outcomes.
adaptive: AdaptiveBatch,
pub errors: u64,
errors: u64,
/// Cumulative bytes actually skipped (zero-filled) on read error.
/// Distinct from `errors`, which counts skip *events*: one event can
/// cover a whole AACS unit (`unit_align` sectors = 6144 bytes), so
/// `errors * 2048` understates real loss by the alignment factor.
/// Consumers estimating lost video time must scale by this, not by
/// the event count.
pub lost_bytes: u64,
lost_bytes: u64,
pub skip_errors: bool,
/// When set and the token is cancelled, fill_extents returns Err(Halted)
/// at the next retry boundary. Unlike skip_errors, this propagates the
@@ -214,14 +209,34 @@ impl DiscStream {
/// The caller opens the source, scans for titles/keys, and passes them in.
/// The stream handles demuxing, decryption, and codec parsing internally.
pub fn new(
reader: Box<dyn SectorSource>,
mut reader: Box<dyn SectorSource>,
title: DiscTitle,
decrypt_keys: crate::decrypt::DecryptKeys,
mut decrypt_keys: crate::decrypt::DecryptKeys,
batch_sectors: u16,
content_format: crate::disc::ContentFormat,
) -> Self {
raw: bool,
halt: Option<Halt>,
) -> std::io::Result<Self> {
let mut title = title;
let extents = title.extents.clone();
// Resolve this title's CSS key from the reader if the caller supplied
// none — the SAME shared step the file-backed mux highway
// (`build_iso_pipeline`) uses, so single-pass and multi-pass descramble a
// DVD identically. No-op for AACS / already-keyed / genuinely-clear input
// or `raw`; a scrambled-but-uncrackable DVD is a hard `CssKeyMissing`.
// `halt` is passed here (not deferred to `with_halt`) so a Stop during the
// crack scan is honored — the scan runs at construction, before the caller
// can attach a token.
crate::css::resolve_dvd_title_key(
&mut *reader,
&extents,
&mut decrypt_keys,
batch_sectors,
content_format,
raw,
halt.as_ref(),
)?;
let bytes_total_extents: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
// Debug log reader type at construction — critical for diagnosing mux
@@ -237,12 +252,12 @@ impl DiscStream {
// CSS/unencrypted content needs a decrypting wrapper to yield plaintext
// VOB bytes before the AC-3 sub-stream probe can read real `acmod`s.
// MUX path: tolerate decrypt loss — conceal an undecryptable unit (NULL TS
// fill) + tally + log rather than abort the stream (P3). DiscStream is a
// decode/mux stream (live-drive single-pass / direct), never the
// ciphertext-preserving sweep, so concealment is always correct here.
let mut reader =
DecryptingSectorSource::new(reader, decrypt_keys.clone()).tolerate_decrypt_loss();
// MUX path (read > decrypt > mux): decrypt every unit in place and pass the
// bytes to the muxer; a unit that decrypts to broken TS is the muxer's
// concern, never conceal / re-fetch / count as loss (fail loud only on a
// genuine can't-decrypt). DiscStream is a decode/mux stream (live-drive
// single-pass / direct), never the ciphertext-preserving sweep.
let mut reader = DecryptingSectorSource::new(reader, decrypt_keys.clone());
// Wrong-substream fix (Silence-of-the-Lambs): re-route the title's
// declared AC-3 audio onto the physically-correct `0x8x` sub-streams by
@@ -293,9 +308,6 @@ impl DiscStream {
// the decorator is a pass-through). Reset the unit base the probe read
// advanced so the first fill_extents read starts cleanly.
reader.set_unit_base(0);
// Clone the shared loss counter once here so `lost_bytes()` never
// clones an Arc per frame on the mux hot path.
let decrypt_loss = reader.decrypt_loss();
// B1 resync gates: one per stream, video flagged so the gate only
// drop-to-keyframes video (audio/subtitle always admit). Computed before
@@ -309,9 +321,8 @@ impl DiscStream {
.map(|_| super::resync::ResyncGate::new())
.collect();
Self {
Ok(Self {
reader,
decrypt_loss,
title,
decrypt_keys,
unit_align,
@@ -324,7 +335,7 @@ impl DiscStream {
errors: 0,
lost_bytes: 0,
skip_errors: false,
halt: None,
halt,
event_fn: None,
eof: false,
dropped_nav_packets: 0,
@@ -339,7 +350,7 @@ impl DiscStream {
profiling: std::env::var_os("FREEMKV_PROFILE").is_some(),
resync,
is_video,
}
})
}
/// Set event handler for sector-level events (binary search, skip, recover).
@@ -360,6 +371,27 @@ impl DiscStream {
self
}
/// Install a proactive [`AacsKeyMap`](crate::decrypt::AacsKeyMap) on the inline
/// live-drive path — the counterpart to what
/// [`build_iso_pipeline`](crate::mux::resolve::build_iso_pipeline) does for the
/// file-backed highway. The map is the title's read plan: it decides which unit
/// each LBA is and, for an FMTS forensic segment, which phase is ours. The
/// extent walk is rewritten to the read plan so **only our-phase units are read
/// off the drive** (the alternate device-group units are never fetched,
/// decrypted, or muxed), and the map is installed so each unit decrypts with its
/// mapped key. A non-forensic map returns the extents unchanged, so a plain
/// single/multi-CPS disc reads exactly as before.
pub fn with_key_map(mut self, map: std::sync::Arc<crate::decrypt::AacsKeyMap>) -> Self {
self.extents = map.read_plan(&self.extents, self.unit_align.max(1) as u32);
self.bytes_total_extents = self
.extents
.iter()
.map(|e| e.sector_count as u64 * 2048)
.sum();
self.reader.set_key_map(map);
self
}
fn is_halted(&self) -> bool {
self.halt
.as_ref()
@@ -1003,14 +1035,12 @@ impl crate::pes::Stream for DiscStream {
}
fn lost_bytes(&self) -> u64 {
// Read-error zero-fill loss (counted in fill_extents) PLUS decrypt-time
// loss — bytes of scrambled AACS units the decorator could not decrypt
// and passed through still encrypted (the TS assembler silently drops
// them). Both are real missing content the abort gate must see; without
// the decrypt term a partial key failure reports lost_bytes=0 and a rip
// missing segments passes even under abort_on_lost_secs=0.
// Read-error zero-fill loss (counted in fill_extents) — real missing
// content the abort gate must see. There is no decrypt-loss term: the
// decrypt path passes bad-encoded/undecryptable units through (a broken-TS
// unit is the muxer's concern, and a missing key is indistinguishable from
// bad authoring here), so only physical read loss is reported.
self.lost_bytes
.saturating_add(self.decrypt_loss.load(std::sync::atomic::Ordering::Relaxed))
}
}
@@ -1084,7 +1114,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
let mut src: Box<dyn Stream> = Box::new(stream);
@@ -1119,7 +1152,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
crate::disc::ContentFormat::BdTs,
false,
None,
)
.unwrap()
.with_halt(halt.clone());
assert!(!stream.is_halted());
halt.cancel();
@@ -1129,6 +1165,47 @@ mod tests {
);
}
/// `with_key_map` on the inline live-drive path applies the same FMTS read plan
/// the file-backed highway uses: within a forensic segment only our-phase units
/// survive the extent walk, so the alternate device-group units are never read.
#[test]
fn with_key_map_reads_only_our_phase_units() {
use crate::decrypt::{AacsKeyMap, DecryptKeys, Phase};
// AACS keys → unit_align = 3, so a unit is 3 sectors and the phase filter
// engages. Key contents are irrelevant to the read plan.
let aacs = DecryptKeys::Aacs {
unit_keys: vec![(0, [0u8; 16]), (1, [1u8; 16])],
read_data_key: None,
format: ContentFormat::BdTs,
};
// 100 units (300 sectors). A 10-unit Even forensic segment at LBA [30,60):
// even units (30,36,42,48,54) are ours; odd (33,39,45,51,57) are dropped.
let map = AacsKeyMap::from_ranges_phased(vec![(30, 60, 1, Phase::Even)], 0);
let stream = DiscStream::new(
Box::new(ZeroReader { capacity: 300 }),
synthetic_title(300),
aacs,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap()
.with_key_map(std::sync::Arc::new(map));
let total: u32 = stream.extents.iter().map(|e| e.sector_count).sum();
assert_eq!(
total,
300 - 5 * 3,
"exactly the 5 alternate-phase units (15 sectors) are dropped from the read walk"
);
assert!(
stream.extents.len() > 1,
"the forensic segment split the single extent into our-phase-only runs"
);
// The progress denominator tracks the reduced read set.
assert_eq!(stream.bytes_total_extents, total as u64 * 2048);
}
/// Recording `SectorSource`: logs every `(lba, count)` request and
/// returns `Err` whenever the requested range covers `bad_sector`.
/// Successful reads return zeroed sectors (which are NOT
@@ -1272,7 +1349,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
// skip_errors=false: if the recovery read did NOT succeed, fill_extents
// would return Err — so reaching EOF cleanly proves recovery worked.
stream.skip_errors = false;
@@ -1385,7 +1465,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
stream.skip_errors = true;
// Drive fill_extents across batches: the good leading sectors mux fine,
@@ -1445,7 +1528,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
stream.skip_errors = true;
let res = stream.fill_extents();
@@ -1489,8 +1575,18 @@ mod tests {
let keys = crate::decrypt::DecryptKeys::Aacs {
unit_keys: vec![(0, [0u8; 16])],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
let mut stream = DiscStream::new(Box::new(reader), title, keys, 8, ContentFormat::BdTs);
let mut stream = DiscStream::new(
Box::new(reader),
title,
keys,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
stream.skip_errors = true;
assert_eq!(
stream.unit_align, ALIGN as u16,
@@ -1586,7 +1682,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
stream.skip_errors = true;
assert_eq!(stream.unit_align, 1, "None keys must leave unit_align=1");
@@ -1627,7 +1726,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
crate::disc::ContentFormat::BdTs,
false,
None,
)
.unwrap()
.with_halt(Halt::from_arc(arc.clone()));
assert!(!stream.is_halted());
arc.store(true, std::sync::atomic::Ordering::Relaxed);
@@ -1636,4 +1738,77 @@ mod tests {
"with_halt(Halt::from_arc) must observe Arc-side flips"
);
}
/// Every read fails CSS-locked (`05/6F/03`) — a scrambled DVD whose title key
/// can't be cracked. Drives `resolve_dvd_title_key` to `ScrambledUncracked`.
struct LockedReader;
impl crate::sector::SectorSource for LockedReader {
fn read_sectors(
&mut self,
lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(2),
sense: Some(crate::scsi::ScsiSense {
sense_key: 0x05,
asc: 0x6F,
ascq: 0x03,
}),
})
}
fn capacity_sectors(&self) -> u32 {
64
}
}
fn mpegps_title(sector_count: u32) -> DiscTitle {
let mut t = synthetic_title(sector_count);
t.content_format = ContentFormat::MpegPs;
t
}
/// PARITY with `build_iso_pipeline_dvd_none_keys_scrambled_hard_fails`: the
/// live-drive single-pass constructor must ALSO hard-fail (not build a
/// scrambled-passthrough stream) for a `None`-keyed scrambled MPEG-PS DVD —
/// the exact 328k-decode-error corruption path, on the single-pass side.
#[test]
fn disc_stream_new_dvd_none_scrambled_hard_fails() {
let res = DiscStream::new(
Box::new(LockedReader),
mpegps_title(8),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::MpegPs,
false,
None,
);
assert!(
res.is_err(),
"single-pass DiscStream must hard-fail on a scrambled, keyless CSS DVD"
);
}
/// `raw` must bypass the CSS crack at the DiscStream boundary too: the same
/// scrambled-uncrackable input that hard-fails above must CONSTRUCT in raw
/// mode (ciphertext passthrough), never hard-fail.
#[test]
fn disc_stream_new_raw_bypasses_css_crack() {
let res = DiscStream::new(
Box::new(LockedReader),
mpegps_title(8),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::MpegPs,
true, // raw
None,
);
assert!(
res.is_ok(),
"raw single-pass must construct without cracking, even on scrambled-uncrackable input"
);
}
}
+69
View File
@@ -94,6 +94,31 @@ pub fn write_uint(w: &mut impl Write, id: u32, val: u64) -> io::Result<()> {
}
}
/// Write a complete EBML signed-integer element (two's-complement, big-endian,
/// minimal width). Used for `ReferenceBlock` (0xFB), whose value is a signed
/// tick offset relative to the current block's timestamp.
pub fn write_int(w: &mut impl Write, id: u32, val: i64) -> io::Result<()> {
write_id(w, id)?;
// Minimal two's-complement width: shrink while the top byte is pure sign
// extension of the next byte's MSB.
let be = val.to_be_bytes();
let mut start = 0usize;
while start < 7 {
let sign_ext = if be[start + 1] & 0x80 != 0 {
0xFF
} else {
0x00
};
if be[start] != sign_ext {
break;
}
start += 1;
}
let bytes = &be[start..];
write_size(w, bytes.len() as u64)?;
w.write_all(bytes)
}
/// Write a complete EBML float element (8-byte double).
pub fn write_float(w: &mut impl Write, id: u32, val: f64) -> io::Result<()> {
write_id(w, id)?;
@@ -481,6 +506,25 @@ pub const LUMINANCE_MIN: u32 = 0x55DA;
pub const BLOCK_ADDITION_MAPPING: u32 = 0x41E4;
pub const BLOCK_ADD_ID_TYPE: u32 = 0x41E7;
pub const BLOCK_ADD_ID_EXTRA_DATA: u32 = 0x41ED;
/// BlockAddIDValue (RFC 9559) — the value a per-frame `BlockAddID` references
/// to select this BlockAdditionMapping. Values ≥ 2 (1 is the default plain
/// BlockAdditional). Used by the MVC (`mvcC`) mapping for Blu-ray 3D.
pub const BLOCK_ADD_ID_VALUE: u32 = 0x41F0;
// Block additions carried inside a BlockGroup — per-frame side data. For
// Blu-ray 3D (MVC) the dependent (right-eye) view NAL units for an access unit
// ride here as a BlockAdditional under the track's `mvcC` mapping (RFC 9559
// §5.1.4.1.4; Matroska Codec Specifications §4.1.5).
pub const BLOCK_ADDITIONS: u32 = 0x75A1;
pub const BLOCK_MORE: u32 = 0xA6;
pub const BLOCK_ADDITIONAL: u32 = 0xA5;
pub const BLOCK_ADD_ID: u32 = 0xEE;
/// ReferenceBlock (RFC 9559 element 0xFB, child of BlockGroup) — signed
/// timestamp (in TimestampScale ticks) of a block this one references, relative
/// to this block's own timestamp. Its PRESENCE marks the Block as non-keyframe
/// (a keyframe Block in a BlockGroup carries none). Written for non-keyframe
/// video frames that must live in a BlockGroup to carry an MVC BlockAdditional.
pub const REFERENCE_BLOCK: u32 = 0xFB;
// Audio
pub const AUDIO: u32 = 0xE1;
@@ -960,6 +1004,31 @@ mod tests {
assert_eq!(buf, [0x42, 0x86, 0x81, 0x00]);
}
#[test]
fn write_int_minimal_two_complement_width() {
// ReferenceBlock (0xFB) signed offsets, minimal two's-complement width.
let enc = |v: i64| {
let mut b = Vec::new();
write_int(&mut b, REFERENCE_BLOCK, v).unwrap();
b
};
assert_eq!(enc(0), [0xFB, 0x81, 0x00], "0 -> 1 byte 0x00");
assert_eq!(enc(-1), [0xFB, 0x81, 0xFF], "-1 -> 1 byte 0xFF");
assert_eq!(enc(127), [0xFB, 0x81, 0x7F], "127 -> 1 byte");
assert_eq!(
enc(128),
[0xFB, 0x82, 0x00, 0x80],
"128 needs 2 bytes (0x80 alone is -128)"
);
assert_eq!(enc(-128), [0xFB, 0x81, 0x80], "-128 -> 1 byte 0x80");
assert_eq!(enc(-129), [0xFB, 0x82, 0xFF, 0x7F], "-129 needs 2 bytes");
// i64::MIN is the widest: 8 bytes, size 0x88.
let mn = enc(i64::MIN);
assert_eq!(mn[0], 0xFB);
assert_eq!(mn[1], 0x88);
assert_eq!(&mn[2..], &i64::MIN.to_be_bytes());
}
// ============================================================
// write_float — EBML floats here are always 8-byte IEEE-754 doubles,
// big-endian (Matroska SamplingFrequency/Duration). size byte = 0x88.
+7 -19
View File
@@ -157,31 +157,19 @@ impl FviSink {
///
/// `source_path` / `source_title` record where the index was built from
/// (the input URL path + the 0-based title index); they are carried into the
/// header's `source` object. The medium defaults to `file` — callers with a
/// known medium / playlist / volume use [`FviSink::create_with_source`].
/// header's `source` object. The remaining provenance (medium, playlist,
/// volume) takes its `SourceInfo` defaults — no caller needs to override them.
pub fn create(
path: &Path,
title: &DiscTitle,
source_path: String,
source_title: usize,
) -> io::Result<Self> {
Self::create_with_source(
path,
title,
SourceInfo {
path: source_path,
title: source_title,
..SourceInfo::default()
},
)
}
/// Create the sink with a fully-specified [`SourceInfo`] provenance root.
pub fn create_with_source(
path: &Path,
title: &DiscTitle,
source: SourceInfo,
) -> io::Result<Self> {
let source = SourceInfo {
path: source_path,
title: source_title,
..SourceInfo::default()
};
let file = File::create(path)?;
let video_track = title
+357
View File
@@ -0,0 +1,357 @@
//! `chapters://` and `json://` metadata sinks.
//!
//! Both ignore the PES stream entirely: everything they emit is already known
//! from the [`DiscTitle`] at construction, so each writes its whole file at
//! `create()` and treats every `write()` frame as a no-op. They are wired
//! through [`super::resolve::output`] like the other write-only sinks; the
//! ISO/disc scan that builds the title is all they need.
use crate::disc::{Chapter, DiscTitle, Stream as DiscStream};
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Write};
use std::path::Path;
// ── chapters:// ──────────────────────────────────────────────────────────────
/// `HH:MM:SS.mmm` for a WebVTT cue timestamp.
fn vtt_time(secs: f64) -> String {
let total_ms = (secs.max(0.0) * 1000.0).round() as u64;
let ms = total_ms % 1000;
let total_s = total_ms / 1000;
format!(
"{:02}:{:02}:{:02}.{:03}",
total_s / 3600,
(total_s / 60) % 60,
total_s % 60,
ms
)
}
/// WebVTT chapter cues (`.vtt`). Each chapter spans until the next one starts
/// (the last runs to its own start — length is unknown without the title tail).
fn chapters_vtt(chapters: &[Chapter]) -> String {
let mut s = String::from("WEBVTT\n\n");
for (i, c) in chapters.iter().enumerate() {
let start = c.time_secs.max(0.0);
let end = chapters
.get(i + 1)
.map(|n| n.time_secs.max(0.0))
.unwrap_or(start);
s.push_str(&format!(
"{}\n{} --> {}\nChapter {}\n\n",
i + 1,
vtt_time(start),
vtt_time(end),
c.name
));
}
s
}
/// Chapter content in the format the output extension selects: `.txt`/`.ogm`
/// (OGM simple), `.vtt` (WebVTT), else Matroska XML (`.xml` / default).
pub(crate) fn chapters_content(chapters: &[Chapter], ext: Option<&str>) -> String {
match ext.map(|e| e.to_ascii_lowercase()).as_deref() {
Some("txt") | Some("ogm") => super::demux_sink::chapters_ogm(chapters),
Some("vtt") => chapters_vtt(chapters),
_ => super::demux_sink::chapters_xml(chapters),
}
}
/// `chapters://` sink: writes the title's chapter markers at construction; the
/// PES stream is ignored.
pub struct ChaptersSink {
title: DiscTitle,
}
impl ChaptersSink {
pub fn create(path: &Path, title: &DiscTitle) -> io::Result<Self> {
let ext = path.extension().and_then(|e| e.to_str());
let content = chapters_content(&title.chapters, ext);
File::create(path)?.write_all(content.as_bytes())?;
Ok(Self {
title: title.clone(),
})
}
}
impl Stream for ChaptersSink {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Ok(()) // whole file written at create()
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
// ── json:// ──────────────────────────────────────────────────────────────────
/// Serialization id for an audio stream's editorial purpose.
fn purpose_id(p: crate::labels::LabelPurpose) -> &'static str {
use crate::labels::LabelPurpose::*;
match p {
Normal => "normal",
Commentary => "commentary",
Descriptive => "descriptive",
Score => "score",
Ime => "ime",
}
}
/// Serialization id for a subtitle stream's qualifier.
fn qualifier_id(q: crate::labels::LabelQualifier) -> &'static str {
use crate::labels::LabelQualifier::*;
match q {
None => "none",
Sdh => "sdh",
DescriptiveService => "descriptive_service",
Forced => "forced",
}
}
/// One stream as JSON — every field the scan resolved, nothing dropped. This is
/// the complete per-stream model (`disc::Stream`), not a summary: consumers get
/// resolution/HDR/aspect for video, channels/sample-rate/purpose for audio, and
/// the qualifier for subtitles, all in machine-readable form.
fn stream_json(s: &DiscStream) -> serde_json::Value {
use super::demux_sink::codec_label;
use serde_json::json;
match s {
DiscStream::Video(v) => {
let (w, h) = v.resolution.pixels();
let (fps_num, fps_den) = v.frame_rate.as_fraction();
let mut o = json!({
"kind": "video",
"codec": codec_label(v.codec),
"pid": v.pid,
"resolution": v.resolution.to_string(),
"width": w,
"height": h,
"interlaced": v.resolution.is_interlaced(),
"frame_rate": v.frame_rate.to_string(),
"frame_rate_num": fps_num,
"frame_rate_den": fps_den,
"hdr": v.hdr.id(),
"color_space": v.color_space.id(),
"secondary": v.secondary,
"mvc_dependent": v.is_mvc_dependent(),
});
if let Some((num, den)) = v.display_aspect {
o["display_aspect"] = json!(format!("{num}:{den}"));
}
if let Some(c) = v.measured_cicp {
o["measured_cicp"] = json!({
"matrix": c.matrix,
"transfer": c.transfer,
"primaries": c.primaries,
"range": c.range,
});
}
if !v.label.is_empty() {
o["label"] = json!(v.label);
}
o
}
DiscStream::Audio(a) => {
let mut o = json!({
"kind": "audio",
"codec": codec_label(a.codec),
"pid": a.pid,
"language": a.language,
"channels": a.channels.to_string(),
"channel_count": a.channels.count(),
"sample_rate": a.sample_rate.to_string(),
"sample_rate_hz": a.sample_rate.hz(),
"secondary": a.secondary,
"purpose": purpose_id(a.purpose),
});
if !a.label.is_empty() {
o["label"] = json!(a.label);
}
o
}
DiscStream::Subtitle(t) => json!({
"kind": "subtitle",
"codec": codec_label(t.codec),
"pid": t.pid,
"language": t.language,
"forced": t.forced,
"qualifier": qualifier_id(t.qualifier),
}),
}
}
/// The `json://` document for one title: identity, duration/size, its clips,
/// its complete stream models, and its chapter points. A stable, machine-
/// readable view of one title — the same information the scan resolved, no loss.
pub(crate) fn title_json(title: &DiscTitle) -> serde_json::Value {
use serde_json::json;
let streams: Vec<_> = title.streams.iter().map(stream_json).collect();
let clips: Vec<_> = title
.clips
.iter()
.map(|c| {
json!({
"clip_id": c.clip_id,
"duration_secs": c.duration_secs,
"source_packets": c.source_packets,
})
})
.collect();
let chapters: Vec<_> = title
.chapters
.iter()
.enumerate()
.map(|(i, c)| json!({ "n": i + 1, "start_secs": c.time_secs, "name": c.name }))
.collect();
json!({
"playlist": title.playlist,
"playlist_id": title.playlist_id,
"duration_secs": title.duration_secs,
"size_bytes": title.size_bytes,
"format": format!("{:?}", title.content_format),
"clips": clips,
"streams": streams,
"chapters": chapters,
})
}
/// `json://` sink: writes the title's structured metadata at construction; the
/// PES stream is ignored.
pub struct JsonSink {
title: DiscTitle,
}
impl JsonSink {
pub fn create(path: &Path, title: &DiscTitle) -> io::Result<Self> {
let doc =
serde_json::to_string_pretty(&title_json(title)).unwrap_or_else(|_| "{}".to_string());
let mut f = File::create(path)?;
f.write_all(doc.as_bytes())?;
f.write_all(b"\n")?;
Ok(Self {
title: title.clone(),
})
}
}
impl Stream for JsonSink {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Ok(())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::Chapter;
fn chaps() -> Vec<Chapter> {
vec![
Chapter {
time_secs: 0.0,
name: "1".into(),
},
Chapter {
time_secs: 62.5,
name: "2".into(),
},
]
}
#[test]
fn chapters_format_selected_by_extension() {
let xml = chapters_content(&chaps(), Some("xml"));
assert!(xml.contains("<Chapters>"), "xml chosen for .xml");
let ogm = chapters_content(&chaps(), Some("txt"));
assert!(ogm.contains("CHAPTER01="), "ogm chosen for .txt");
let vtt = chapters_content(&chaps(), Some("vtt"));
assert!(
vtt.starts_with("WEBVTT") && vtt.contains("00:01:02.500"),
"vtt chosen for .vtt, with cue timing"
);
// Unknown / missing extension defaults to XML.
assert!(chapters_content(&chaps(), None).contains("<Chapters>"));
}
#[test]
fn title_json_carries_streams_and_chapters() {
use crate::disc::{AudioChannels, AudioStream, Codec, DiscTitle};
use crate::disc::{LabelPurpose, SampleRate, Stream as DiscStream};
let mut t = DiscTitle::empty();
t.playlist = "MAIN".into();
t.chapters = chaps();
t.streams = vec![DiscStream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Stereo,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
})];
let v = title_json(&t);
assert_eq!(v["playlist"], "MAIN");
let a = &v["streams"][0];
assert_eq!(a["kind"], "audio");
assert_eq!(a["codec"], "TrueHD");
assert_eq!(a["language"], "eng");
// Completeness: audio carries channels + sample rate + purpose, not just codec.
assert_eq!(a["channels"], "stereo");
assert_eq!(a["channel_count"], 2);
assert_eq!(a["sample_rate"], "48kHz");
assert_eq!(a["sample_rate_hz"], 48000.0);
assert_eq!(a["purpose"], "normal");
assert_eq!(v["chapters"][1]["n"], 2);
assert_eq!(v["chapters"][1]["start_secs"], 62.5);
assert_eq!(v["chapters"][1]["name"], "2");
}
#[test]
fn video_json_carries_resolution_and_hdr() {
use crate::disc::Codec;
use crate::disc::{
ColorSpace, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStream,
VideoStream,
};
let mut t = DiscTitle::empty();
t.streams = vec![DiscStream::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R2160p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
})];
let vid = &title_json(&t)["streams"][0];
assert_eq!(vid["kind"], "video");
assert_eq!(vid["resolution"], "2160p");
assert_eq!(vid["width"], 3840);
assert_eq!(vid["height"], 2160);
assert_eq!(vid["frame_rate"], "23.976");
assert_eq!(vid["frame_rate_num"], 24000);
assert_eq!(vid["hdr"], "hdr10");
assert_eq!(vid["color_space"], "bt2020");
}
}
+715 -46
View File
File diff suppressed because it is too large Load Diff
+562 -64
View File
@@ -120,36 +120,253 @@ struct PendingMux {
video_track: Option<usize>,
/// `--log-level 3` opening-capture side-file path (if any).
opening_capture_path: Option<std::path::PathBuf>,
/// Frames received before activation, replayed in order once built.
buffered: Vec<crate::pes::PesFrame>,
/// Frames received before activation, replayed in order once built. Each
/// carries an optional MVC dependent-view `BlockAdditional` (present only
/// for a 3D base-view frame that was already paired before activation).
buffered: Vec<(crate::pes::PesFrame, Option<Vec<u8>>)>,
}
/// Matroska container stream.
pub struct MkvStream {
disc_title: DiscTitle,
mode: Mode,
/// Blu-ray 3D (MVC) merge state — present iff the title carries an MVC
/// dependent (right-eye) view. Folds the dependent stream's frames into the
/// base video track as per-frame `BlockAdditional`, paired by PTS, so the
/// output is a single MVC track instead of two independent H.264 tracks.
mvc: Option<MvcMerge>,
}
/// Largest number of base frames held awaiting their PTS-matching dependent AU
/// before the oldest is flushed unpaired (a plain Block). The SSIF interleaves
/// base and dependent access units per unit, so a base's dependent normally
/// arrives within one or two frames; this window only bounds memory/latency for
/// a stream where the pairing drifts.
const MVC_PAIR_WINDOW: usize = 32;
/// A base-view frame (track already remapped to the muxer's base track index)
/// awaiting — or already carrying — its dependent-view `BlockAdditional`.
struct PendingBase {
frame: crate::pes::PesFrame,
additional: Option<Vec<u8>>,
}
/// State for folding the MVC dependent (right-eye) view into the base track.
struct MvcMerge {
/// `title.streams` index of the base (left-eye) video stream.
base_stream_idx: usize,
/// `title.streams` index of the dependent (right-eye) video stream.
dep_stream_idx: usize,
/// Muxer track index of the base view — where the dependent AU is attached
/// as a `BlockAdditional` and where `mvc_params` (the `mvcC` mapping) lives.
base_track_idx: usize,
/// `title.streams` index → muxer track index. The dependent maps to `None`
/// (it becomes a BlockAdditional, not a track); every other stream shifts
/// down by one if it followed the dependent in stream order.
stream_to_track: Vec<Option<usize>>,
/// Base frames (decode order) awaiting their dependent or a window flush.
pending_base: std::collections::VecDeque<PendingBase>,
/// Dependent AU data keyed by PTS, waiting for the matching base.
dep_by_pts: std::collections::HashMap<i64, Vec<u8>>,
/// `(subset_sps, pps)` from the first dependent AU — builds the `mvcC`
/// MVCDecoderConfigurationRecord for the base track's BlockAdditionMapping.
captured_params: Option<(Vec<u8>, Vec<u8>)>,
/// Count of dependent AUs dropped with no matching base (diagnostic).
orphan_deps: u64,
}
impl MvcMerge {
/// Ingest one incoming frame; returns `(frame, additional)` pairs ready to
/// hand to the muxer, in emit order. Base frames buffer briefly to pair with
/// their dependent by PTS; the dependent stream produces no frames of its own
/// (it becomes `BlockAdditional`); all other streams pass straight through
/// with their track index remapped.
fn ingest(
&mut self,
frame: &crate::pes::PesFrame,
) -> Vec<(crate::pes::PesFrame, Option<Vec<u8>>)> {
let mut out = Vec::new();
if frame.track == self.dep_stream_idx {
if self.captured_params.is_none() {
self.captured_params = extract_mvc_params(&frame.data);
}
// Attach to a waiting base of the same PTS, else stash by PTS.
if let Some(pb) = self
.pending_base
.iter_mut()
.find(|pb| pb.frame.pts == frame.pts && pb.additional.is_none())
{
pb.additional = Some(frame.data.clone());
} else {
// Bound the orphan map BEFORE inserting: if dependents pile up
// unpaired (pairing badly drifted), drop the drifted buffer so it
// stays bounded — but keep THIS just-arrived dependent, whose base
// frame commonly arrives next. Clearing after the insert would
// discard it and overcount orphans by one.
if self.dep_by_pts.len() >= MVC_PAIR_WINDOW * 4 {
self.orphan_deps += self.dep_by_pts.len() as u64;
self.dep_by_pts.clear();
}
// A duplicate-PTS dependent (e.g. a stale repeat after a stream
// discontinuity) displaces the prior one — count it as an orphan
// rather than losing it silently.
if self
.dep_by_pts
.insert(frame.pts, frame.data.clone())
.is_some()
{
self.orphan_deps += 1;
}
}
} else if frame.track == self.base_stream_idx {
let additional = self.dep_by_pts.remove(&frame.pts);
let mut remapped = frame.clone();
remapped.track = self.base_track_idx;
self.pending_base.push_back(PendingBase {
frame: remapped,
additional,
});
} else {
// Audio / subtitle / other video: remap the track index and forward.
let mut remapped = frame.clone();
if let Some(Some(t)) = self.stream_to_track.get(frame.track) {
remapped.track = *t;
out.push((remapped, None));
}
}
self.drain_ready(&mut out);
out
}
/// Emit base frames from the FIFO front once each has its dependent attached,
/// or flush the oldest unpaired base as a plain Block when the window is full.
fn drain_ready(&mut self, out: &mut Vec<(crate::pes::PesFrame, Option<Vec<u8>>)>) {
loop {
let front_ready = self
.pending_base
.front()
.map(|pb| pb.additional.is_some())
.unwrap_or(false);
if front_ready || self.pending_base.len() > MVC_PAIR_WINDOW {
if let Some(pb) = self.pending_base.pop_front() {
out.push((pb.frame, pb.additional));
continue;
}
}
break;
}
}
/// Flush every remaining buffered base frame (unpaired → plain Block) at EOF.
fn flush(&mut self) -> Vec<(crate::pes::PesFrame, Option<Vec<u8>>)> {
let mut out = Vec::new();
for pb in self.pending_base.drain(..) {
out.push((pb.frame, pb.additional));
}
self.orphan_deps += self.dep_by_pts.len() as u64;
self.dep_by_pts.clear();
out
}
}
/// Hand a frame to the muxer, attaching the MVC dependent view as a
/// `BlockAdditional` when `additional` is `Some` (a 3D base frame), else a
/// plain block.
fn emit_to_muxer(
m: &mut MkvMuxer<Box<dyn WriteSeek + Send>>,
frame: &crate::pes::PesFrame,
additional: Option<&[u8]>,
) -> io::Result<()> {
m.write_frame(
frame.track,
frame.pts,
frame.keyframe,
&frame.data,
frame.duration_ns,
additional,
)
}
/// Scan a length-prefixed (4-byte big-endian) H.264 NAL stream for the first
/// subset SPS (NAL type 15) and first PPS (NAL type 8) — the two parameter sets
/// that populate the `mvcC` MVCDecoderConfigurationRecord. Returns
/// `Some((subset_sps, pps))` only when BOTH are found; `None` otherwise (the
/// serializer then emits no mvcC mapping and logs it).
fn extract_mvc_params(data: &[u8]) -> Option<(Vec<u8>, Vec<u8>)> {
let mut subset_sps: Option<Vec<u8>> = None;
let mut pps: Option<Vec<u8>> = None;
let mut i = 0usize;
while i + 4 <= data.len() {
let len = u32::from_be_bytes([data[i], data[i + 1], data[i + 2], data[i + 3]]) as usize;
i += 4;
// A zero-length NAL (a stray length prefix) is skipped, not fatal — the
// subset SPS / PPS may still follow. A length that runs past the buffer
// end IS unrecoverable (the NAL can't be read), so stop there.
if len == 0 {
continue;
}
if i + len > data.len() {
break;
}
let nal = &data[i..i + len];
i += len;
match nal[0] & 0x1F {
15 if subset_sps.is_none() => subset_sps = Some(nal.to_vec()),
8 if pps.is_none() => pps = Some(nal.to_vec()),
_ => {}
}
if subset_sps.is_some() && pps.is_some() {
break;
}
}
Some((subset_sps?, pps?))
}
impl MkvStream {
/// Create for writing PES frames → MKV container.
/// Codec privates come from title.codec_privates (populated by input stream).
pub fn create(writer: Box<dyn WriteSeek + Send>, title: &DiscTitle) -> io::Result<Self> {
Self::create_at(writer, title, None)
}
/// As [`create`](Self::create), but `output_path` (when known) enables the
/// `--log-level 3` opening-frame capture to `<output>.opening.bin`. A `None`
/// path (e.g. an in-memory / stdio sink) silently skips the side-file
/// capture; the per-track TrackEntry dump still fires.
pub fn create_at(
/// Create for writing PES frames → MKV container. Codec privates come from
/// `title.codec_privates` (populated by the input stream).
///
/// `output_path` (when known) enables the `--log-level 3` opening-frame
/// capture to `<output>.opening.bin`; `None` (e.g. an in-memory / stdio sink)
/// silently skips the side-file capture — the per-track TrackEntry dump still
/// fires either way.
pub fn create(
writer: Box<dyn WriteSeek + Send>,
title: &DiscTitle,
output_path: Option<&std::path::Path>,
) -> io::Result<Self> {
// Blu-ray 3D (MVC): a dependent (right-eye) view stream is NOT emitted as
// its own track — it is folded into the base track as per-frame
// BlockAdditional. Detect it so we skip building a track for it and set up
// the merge. `base_stream_idx` is the first video stream.
let dep_stream_idx = title
.streams
.iter()
.position(|s| matches!(s, crate::disc::Stream::Video(v) if v.is_mvc_dependent()));
// The base is the first NON-dependent video. Excluding the dependent here
// means a (malformed / hand-built) title whose only video IS the dependent
// yields `base_stream_idx == None` → no merge (the dependent is muxed as an
// ordinary track) instead of `base == dep` and a panic on the skipped slot.
let base_stream_idx = title
.streams
.iter()
.position(|s| matches!(s, crate::disc::Stream::Video(v) if !v.is_mvc_dependent()));
// The merge is only active when BOTH a dependent and a distinct base exist;
// only then is the dependent's track skipped/folded.
let mvc_active = dep_stream_idx.is_some() && base_stream_idx.is_some();
let skip_stream_idx = if mvc_active { dep_stream_idx } else { None };
let mut tracks = Vec::new();
let mut has_default_video = false;
let mut has_default_audio = false;
// `title.streams` index → muxer track index (`None` = the dependent view,
// which has no track). Streams after the dependent shift down by one.
let mut stream_to_track: Vec<Option<usize>> = Vec::with_capacity(title.streams.len());
for (idx, s) in title.streams.iter().enumerate() {
if Some(idx) == skip_stream_idx {
stream_to_track.push(None);
continue;
}
let mut track = match s {
crate::disc::Stream::Video(v) => MkvTrack::video(v),
crate::disc::Stream::Audio(a) => MkvTrack::audio(a),
@@ -166,9 +383,31 @@ impl MkvStream {
if let Some(cp) = title.codec_privates.get(idx).and_then(|c| c.as_ref()) {
track.codec_private = Some(cp.clone());
}
stream_to_track.push(Some(tracks.len()));
tracks.push(track);
}
// Assemble the MVC merge only when active — i.e. a dependent AND a
// distinct base video both exist (established above). `base_stream_idx`
// then always has a built track, so its remap is `Some` (no panic path).
let mvc = match (mvc_active, dep_stream_idx, base_stream_idx) {
(true, Some(dep_stream_idx), Some(base_stream_idx)) => stream_to_track
.get(base_stream_idx)
.copied()
.flatten()
.map(|base_track_idx| MvcMerge {
base_stream_idx,
dep_stream_idx,
base_track_idx,
stream_to_track,
pending_base: std::collections::VecDeque::new(),
dep_by_pts: std::collections::HashMap::new(),
captured_params: None,
orphan_deps: 0,
}),
_ => None,
};
// Defer muxer construction (and the TrackEntry dump) until the first
// coded picture arrives, so the primary video track's FieldOrder is set
// from the parser's MEASURED value before the header is written — never
@@ -177,6 +416,7 @@ impl MkvStream {
Ok(Self {
disc_title: title.clone(),
mvc,
mode: Mode::Write(WriteMode::Pending(Box::new(PendingMux {
writer,
tracks,
@@ -209,6 +449,24 @@ impl MkvStream {
if let Some(vt) = pending.video_track {
apply_coding_to_track(&mut pending.tracks[vt], coding, video_picture_seen);
}
// Blu-ray 3D: set the base video track's `mvc_params` from the dependent
// view's captured subset-SPS/PPS BEFORE the header is written, so the
// TrackEntry carries the `mvcC` BlockAdditionMapping. Captured from the
// first dependent AU (which arrives right after the first base AU in the
// SSIF), so it is available by the time the first base frame activates.
if let Some(mvc) = &self.mvc {
if let Some(params) = &mvc.captured_params {
if let Some(t) = pending.tracks.get_mut(mvc.base_track_idx) {
t.mvc_params = Some(params.clone());
}
} else {
tracing::warn!(
target: "mux",
"MVC: no dependent-view subset-SPS/PPS captured before activation; \
the base track will carry no mvcC mapping (3D not signalled)."
);
}
}
// --log-level 3: dump the FINAL TrackEntry metadata (field order set).
for (i, track) in pending.tracks.iter().enumerate() {
crate::diag::dump_mkv_track((i + 1) as u64, track);
@@ -223,18 +481,72 @@ impl MkvStream {
if let Some(path) = &pending.opening_capture_path {
muxer.set_opening_capture(crate::diag::OpeningCapture::new(path, pending.tracks.len()));
}
for f in pending.buffered.drain(..) {
muxer.write_frame(f.track, f.pts, f.keyframe, &f.data, f.duration_ns)?;
for (f, additional) in pending.buffered.drain(..) {
muxer.write_frame(
f.track,
f.pts,
f.keyframe,
&f.data,
f.duration_ns,
additional.as_deref(),
)?;
}
self.mode = Mode::Write(WriteMode::Active(Box::new(muxer)));
Ok(())
}
/// Emit one frame (track index already muxer-relative) with an optional MVC
/// dependent-view `BlockAdditional`, honouring the deferred-activation
/// machinery: the first video frame triggers muxer construction (its coding
/// sets FieldOrder); earlier frames buffer. `additional` is `None` for every
/// non-3D frame and for the 3D base frames that had no paired dependent.
fn emit(&mut self, frame: &crate::pes::PesFrame, additional: Option<&[u8]>) -> io::Result<()> {
match &mut self.mode {
Mode::Read(_) => return Err(crate::error::Error::StreamReadOnly.into()),
Mode::Write(WriteMode::Active(m)) => {
return emit_to_muxer(m, frame, additional);
}
Mode::Write(WriteMode::Building) => return Ok(()),
Mode::Write(WriteMode::Pending(_)) => {}
}
// Pending: the first video frame (or the safety cap) triggers muxer
// construction; that frame's coding sets the field order. Other frames
// buffer until then.
let (activate_now, use_coding) = match &self.mode {
Mode::Write(WriteMode::Pending(p)) => {
let is_video = match p.video_track {
Some(vt) => frame.track == vt,
// No video track: nothing to wait for — build on frame one.
None => true,
};
(is_video || p.buffered.len() >= MAX_PENDING_FRAMES, is_video)
}
_ => unreachable!("guarded above"),
};
if activate_now {
// Pass the trigger frame's coding only when it IS the video frame; a
// cap-triggered build never saw the video frame, so nothing measured
// is passed (apply_coding_to_track then logs + leaves UNDETERMINED).
self.activate(if use_coding { frame.coding } else { None }, use_coding)?;
if let Mode::Write(WriteMode::Active(m)) = &mut self.mode {
return emit_to_muxer(m, frame, additional);
}
Ok(())
} else {
if let Mode::Write(WriteMode::Pending(p)) = &mut self.mode {
p.buffered
.push((frame.clone(), additional.map(|a| a.to_vec())));
}
Ok(())
}
}
/// Open an MKV file for reading → PES frames.
pub fn open(mut reader: impl Read + Send + 'static) -> io::Result<Self> {
let (disc_title, codec_privates, ts_scale_ns) = parse_mkv_header(&mut reader)?;
Ok(Self {
disc_title,
mvc: None,
mode: Mode::Read(ReadState {
reader: Box::new(reader),
cluster_ts_ticks: 0,
@@ -425,59 +737,40 @@ impl crate::pes::Stream for MkvStream {
}
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
// Fast paths.
match &mut self.mode {
Mode::Read(_) => return Err(crate::error::Error::StreamReadOnly.into()),
Mode::Write(WriteMode::Active(m)) => {
return m.write_frame(
frame.track,
frame.pts,
frame.keyframe,
&frame.data,
frame.duration_ns,
);
}
Mode::Write(WriteMode::Building) => return Ok(()),
Mode::Write(WriteMode::Pending(_)) => {}
if matches!(self.mode, Mode::Read(_)) {
return Err(crate::error::Error::StreamReadOnly.into());
}
// Pending: the first video frame (or the safety cap) triggers muxer
// construction; that frame's coding sets the field order. Other frames
// buffer until then.
let (activate_now, use_coding) = match &self.mode {
Mode::Write(WriteMode::Pending(p)) => {
let is_video = match p.video_track {
Some(vt) => frame.track == vt,
// No video track: nothing to wait for — build on frame one.
None => true,
};
(is_video || p.buffered.len() >= MAX_PENDING_FRAMES, is_video)
}
_ => unreachable!("guarded above"),
};
if activate_now {
// Pass the trigger frame's coding only when it IS the video frame; a
// cap-triggered build never saw the video frame, so nothing measured
// is passed (apply_coding_to_track then logs + leaves UNDETERMINED).
self.activate(if use_coding { frame.coding } else { None }, use_coding)?;
if let Mode::Write(WriteMode::Active(m)) = &mut self.mode {
return m.write_frame(
frame.track,
frame.pts,
frame.keyframe,
&frame.data,
frame.duration_ns,
);
}
Ok(())
} else {
if let Mode::Write(WriteMode::Pending(p)) = &mut self.mode {
p.buffered.push(frame.clone());
}
Ok(())
// Non-3D fast path: emit the frame directly, no clone, no buffering.
if self.mvc.is_none() {
return self.emit(frame, None);
}
// Blu-ray 3D: run the frame through the MVC merge, which remaps track
// indices, folds the dependent view into the base as BlockAdditional
// (paired by PTS), and yields 0+ frames ready to emit. `ingest` returns
// owned pairs so the `self.mvc` borrow is released before `emit`.
let emits = self.mvc.as_mut().unwrap().ingest(frame);
for (f, additional) in emits {
self.emit(&f, additional.as_deref())?;
}
Ok(())
}
fn finish(&mut self) -> io::Result<()> {
// Blu-ray 3D: flush any base frames still awaiting a dependent (emitted
// unpaired as plain Blocks) before finalizing.
if let Some(mvc) = self.mvc.as_mut() {
let tail = mvc.flush();
let orphans = mvc.orphan_deps;
if orphans > 0 {
tracing::debug!(
target: "mux",
"MVC: {orphans} dependent-view access units had no matching base frame (dropped)"
);
}
for (f, additional) in tail {
self.emit(&f, additional.as_deref())?;
}
}
// A title that produced no frames (or only buffered ones) is still
// finalized into a valid MKV: activate now with no measured coding.
if matches!(self.mode, Mode::Write(WriteMode::Pending(_))) {
@@ -720,7 +1013,11 @@ fn parse_track(
arem = arem.saturating_sub(ahlen as u64 + as_);
match aid {
ebml::SAMPLING_FREQUENCY => sr = ebml::read_float_val(r, as_ as usize)?,
ebml::CHANNELS => ch = read_uint_bounded(r, as_)? as u8,
// Clamp instead of `as u8`: a foreign/corrupt MKV with a
// CHANNELS value that is a multiple of 256 would truncate to
// 0 (an invalid channel count) on a bare cast. Saturate to
// u8::MAX so an absurd count degrades to "many", never to 0.
ebml::CHANNELS => ch = read_uint_bounded(r, as_)?.min(u8::MAX as u64) as u8,
_ => {
skip_bytes(r, as_)?;
}
@@ -916,6 +1213,207 @@ mod tests {
use crate::pes::Stream as _;
use std::io::Cursor;
/// Length-prefix (4-byte big-endian) each NAL, as the H.264 parser emits.
fn lp(nals: &[&[u8]]) -> Vec<u8> {
let mut v = Vec::new();
for n in nals {
v.extend_from_slice(&(n.len() as u32).to_be_bytes());
v.extend_from_slice(n);
}
v
}
fn mvc_frame(track: usize, pts: i64, keyframe: bool, data: Vec<u8>) -> crate::pes::PesFrame {
crate::pes::PesFrame {
track,
pts,
keyframe,
data,
duration_ns: None,
source: None,
coding: None,
}
}
// A subset SPS (NAL type 15), a PPS (type 8), and a coded-slice-extension
// (type 20) — the shape of a dependent-view access unit.
const SUBSET_SPS: [u8; 5] = [0x6F, 0x80, 0x00, 0x33, 0xAA]; // 0x6F & 0x1F = 15
const DEP_PPS: [u8; 3] = [0x68, 0xEE, 0x3C]; // 0x68 & 0x1F = 8
const DEP_SLICE: [u8; 3] = [0x74, 0x11, 0x22]; // 0x74 & 0x1F = 20
#[test]
fn extract_mvc_params_finds_subset_sps_and_pps() {
let data = lp(&[&SUBSET_SPS, &DEP_PPS, &DEP_SLICE]);
let (s, p) = extract_mvc_params(&data).expect("both param sets present");
assert_eq!(s, SUBSET_SPS, "subset SPS (NAL 15) captured verbatim");
assert_eq!(p, DEP_PPS, "PPS (NAL 8) captured verbatim");
// Missing PPS → None (the serializer then emits no mvcC mapping).
assert!(extract_mvc_params(&lp(&[&SUBSET_SPS, &DEP_SLICE])).is_none());
// Missing subset SPS → None.
assert!(extract_mvc_params(&lp(&[&DEP_PPS, &DEP_SLICE])).is_none());
}
fn empty_merge() -> MvcMerge {
MvcMerge {
base_stream_idx: 0,
dep_stream_idx: 2,
base_track_idx: 0,
stream_to_track: vec![Some(0), Some(1), None],
pending_base: std::collections::VecDeque::new(),
dep_by_pts: std::collections::HashMap::new(),
captured_params: None,
orphan_deps: 0,
}
}
#[test]
fn mvc_merge_pairs_base_and_dependent_by_pts() {
let mut m = empty_merge();
let dep = lp(&[&SUBSET_SPS, &DEP_PPS, &DEP_SLICE]);
// Base arrives first (SSIF order): buffered, nothing emitted yet.
let e = m.ingest(&mvc_frame(0, 100, true, lp(&[&[0x65, 1, 2]])));
assert!(e.is_empty(), "base held until its dependent arrives");
// Dependent arrives → base is emitted, remapped to the base track, with
// the dependent AU as its BlockAdditional; params are captured.
let e = m.ingest(&mvc_frame(2, 100, false, dep.clone()));
assert_eq!(e.len(), 1, "the paired base frame is emitted");
assert_eq!(e[0].0.track, 0, "remapped to the base muxer track");
assert_eq!(
e[0].1.as_deref(),
Some(dep.as_slice()),
"dependent attached"
);
assert!(m.captured_params.is_some(), "mvcC params captured");
// Audio passes straight through (remapped, no additional).
let e = m.ingest(&mvc_frame(1, 100, true, vec![0xAA]));
assert_eq!(e.len(), 1);
assert_eq!(e[0].0.track, 1);
assert!(e[0].1.is_none());
// Dependent-before-base (reordered) also pairs.
let dep2 = lp(&[&DEP_SLICE]);
assert!(m.ingest(&mvc_frame(2, 200, false, dep2.clone())).is_empty());
let e = m.ingest(&mvc_frame(0, 200, false, lp(&[&[0x61, 3, 4]])));
assert_eq!(e.len(), 1);
assert_eq!(e[0].1.as_deref(), Some(dep2.as_slice()));
}
#[test]
fn mvc_merge_flushes_unpaired_base_at_eof() {
let mut m = empty_merge();
// Base with no dependent ever → held, then flushed unpaired at EOF.
assert!(
m.ingest(&mvc_frame(0, 10, true, vec![0, 0, 0, 1]))
.is_empty()
);
let tail = m.flush();
assert_eq!(tail.len(), 1, "unpaired base still emitted");
assert!(tail[0].1.is_none(), "no BlockAdditional when unpaired");
}
#[test]
fn extract_mvc_params_no_panic_on_truncated_or_empty() {
// Empty, sub-header, zero-length NAL, and a length prefix claiming more
// than is present must all return None without panicking (untrusted AU).
assert!(extract_mvc_params(&[]).is_none());
assert!(extract_mvc_params(&[0, 0, 0]).is_none());
assert!(
extract_mvc_params(&[0, 0, 0, 0]).is_none(),
"lone zero-length NAL yields no params"
);
assert!(
extract_mvc_params(&[0, 0, 0, 10, 0x6F]).is_none(),
"length prefix past end breaks, no slice panic"
);
// A zero-length NAL is SKIPPED, not fatal: valid param sets that follow
// are still found (a stray length prefix must not abandon the whole AU).
let mut d = vec![0, 0, 0, 0];
d.extend_from_slice(&lp(&[&SUBSET_SPS, &DEP_PPS]));
let (s, p) = extract_mvc_params(&d).expect("params found past the zero-length NAL");
assert_eq!(s, SUBSET_SPS);
assert_eq!(p, DEP_PPS);
}
#[test]
fn mvc_merge_flushes_oldest_base_once_past_window() {
let mut m = empty_merge();
// Push more unpaired base frames than the window; the excess flush as
// plain (unpaired) blocks in FIFO order once len exceeds MVC_PAIR_WINDOW.
let n = MVC_PAIR_WINDOW + 8;
let mut emitted = 0usize;
for pts in 0..n {
emitted += m
.ingest(&mvc_frame(0, pts as i64, false, vec![0, 0, 0, 1]))
.len();
}
assert_eq!(emitted, 8, "the {n} bases beyond the window flush unpaired");
assert_eq!(m.pending_base.len(), MVC_PAIR_WINDOW, "window still held");
assert!(m.flush().iter().all(|(_, add)| add.is_none()));
}
#[test]
fn mvc_merge_dep_overflow_drops_old_keeps_newest() {
let mut m = empty_merge();
// Fill dep_by_pts to the bound with unpaired dependents (unique PTS).
for pts in 0..(MVC_PAIR_WINDOW * 4) {
assert!(
m.ingest(&mvc_frame(2, pts as i64, false, lp(&[&DEP_SLICE])))
.is_empty()
);
}
assert_eq!(m.dep_by_pts.len(), MVC_PAIR_WINDOW * 4);
// One more overflows: the drifted buffer is cleared BUT the newest survives
// so its (soon-to-arrive) base can still pair.
let dep_new = lp(&[&DEP_SLICE]);
m.ingest(&mvc_frame(2, 9_999, false, dep_new.clone()));
assert_eq!(m.dep_by_pts.len(), 1, "old cleared, newest kept");
assert!(m.dep_by_pts.contains_key(&9_999));
assert_eq!(
m.orphan_deps,
(MVC_PAIR_WINDOW * 4) as u64,
"old buffer counted once"
);
// The surviving dependent pairs with its base.
let e = m.ingest(&mvc_frame(0, 9_999, false, vec![0x61, 1]));
assert_eq!(e.len(), 1);
assert_eq!(e[0].1.as_deref(), Some(dep_new.as_slice()));
}
#[test]
fn create_does_not_panic_when_only_video_is_mvc_dependent() {
// A (malformed / hand-built) title whose single video IS the dependent
// must NOT panic: base_stream_idx is None, so no merge is set up and the
// dependent is muxed as an ordinary track.
use crate::disc::{
Codec, ColorSpace, DiscTitle, FrameRate, HdrFormat, Resolution, Stream, VideoStream,
};
let dep = VideoStream {
pid: 0x1012,
codec: Codec::H264,
resolution: Resolution::R1080p,
frame_rate: FrameRate::F24,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Bt709,
display_aspect: None,
secondary: true,
label: crate::disc::MVC_DEPENDENT_LABEL.to_string(),
measured_cicp: None,
};
let title = DiscTitle {
streams: vec![Stream::Video(dep)],
..DiscTitle::empty()
};
let s = MkvStream::create(Box::new(Cursor::new(Vec::new())), &title, None)
.expect("create must succeed, not panic");
assert!(
s.mvc.is_none(),
"no merge when there is no distinct base view"
);
}
#[test]
fn apply_coding_to_track_sets_measured_field_order_never_guesses() {
use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream};
+13 -2
View File
@@ -62,6 +62,7 @@ pub(crate) mod m2ts;
/// to round-trip codec_privates that don't fit inside the underlying format).
/// Exposed for integration tests that exercise the wire format directly.
pub mod meta;
pub(crate) mod meta_sink;
// ── Sequential-sink muxers ──────────────────────────────────────────────────
//
@@ -89,6 +90,7 @@ pub(crate) mod hevc;
pub(crate) mod m2ts_mux;
pub(crate) mod mkv;
pub(crate) mod mkvstream;
pub(crate) mod mp4;
pub(crate) mod network;
pub(crate) mod null;
pub(crate) mod ps;
@@ -108,15 +110,18 @@ pub(crate) mod tsmux;
#[allow(dead_code)]
pub(crate) mod videomap;
pub use demux_sink::{ChaptersFmt, DelayMode, DemuxOptions, DemuxSink, Naming};
// `demux://` and `fvi://` sinks are constructed internally by `output()` via the
// direct `super::demux_sink::` / `super::fvi_sink::` paths — no re-export needed,
// and no consumer names these types, so they are not public API.
pub use disc::DiscStream;
pub use fvi_sink::FviSink;
pub use m2ts::M2tsStream;
pub use mkvstream::MkvStream;
pub use mp4::{Mp4FitReport, Mp4SkipReason, fit_report as mp4_fit_report};
pub use network::NetworkStream;
pub use null::NullStream;
pub use pipelined_stream::PipelinedPesStream;
pub use resolve::build_iso_pipeline;
pub use resolve::resolve_mux_key_map;
pub use resolve::{InputOptions, StreamUrl, input, output, parse_url};
pub use stdio::StdioStream;
@@ -151,11 +156,17 @@ mod tests {
assert_eq!(parse_url("disc://").scheme(), "disc");
assert_eq!(parse_url("m2ts://f").scheme(), "m2ts");
assert_eq!(parse_url("mkv://f").scheme(), "mkv");
assert_eq!(parse_url("mp4://f").scheme(), "mp4");
assert_eq!(parse_url("network://h:1").scheme(), "network");
assert_eq!(parse_url("stdio://").scheme(), "stdio");
assert_eq!(parse_url("iso://f").scheme(), "iso");
assert_eq!(parse_url("null://").scheme(), "null");
assert_eq!(parse_url("demux://out/").scheme(), "demux");
assert_eq!(parse_url("video://out/").scheme(), "video");
assert_eq!(parse_url("audio://out/").scheme(), "audio");
assert_eq!(parse_url("sub://out/").scheme(), "sub");
assert_eq!(parse_url("chapters://c.xml").scheme(), "chapters");
assert_eq!(parse_url("json://t.json").scheme(), "json");
assert_eq!(parse_url("bogus://x").scheme(), "unknown");
}
+526
View File
@@ -0,0 +1,526 @@
//! MP4 audio sample entries and codec-config boxes for the `mp4://` muxer.
//!
//! Covers the codecs that map cleanly into MP4 and play widely: **AC-3**
//! (`ac-3` + `dac3`), **E-AC-3 / Dolby Digital Plus** (`ec-3` + `dec3`, incl.
//! Atmos-in-DD+ JOC), and **DTS / DTS-HD** (`dtsc`/`dtsh` + `ddts`, describing
//! the core with whole access units passed through so an HD decoder finds the
//! extension). Config boxes are derived from the first audio frame's bitstream
//! (ISO/IEC 14496-12 amendments; ETSI TS 102 366 / 102 114). Codecs with no
//! clean MP4 mapping (TrueHD, LPCM, bitmap subtitles) are excluded by the fit
//! oracle in the sink.
use super::boxes::bx;
use crate::disc::Codec;
/// AC-3 / E-AC-3 sample rates indexed by `fscod` (byte-4 bits 7-6).
const FSCOD_RATES: [u32; 3] = [48_000, 44_100, 32_000];
/// E-AC-3 reduced rates indexed by `fscod2` (byte-4 bits 5-4) when `fscod == 3`.
const EAC3_REDUCED_RATES: [u32; 4] = [24_000, 22_050, 16_000, 48_000];
/// Base channel count per `acmod` (A/52 Table 5.8), before the LFE.
const ACMOD_CHANNELS: [u8; 8] = [2, 1, 2, 3, 3, 4, 4, 5];
/// A big-endian MSB-first bit reader over a byte slice.
struct BitReader<'a> {
data: &'a [u8],
bit: usize,
}
impl<'a> BitReader<'a> {
fn new(data: &'a [u8]) -> Self {
Self { data, bit: 0 }
}
fn skip(&mut self, n: usize) {
self.bit += n;
}
/// Read `n` bits (n ≤ 32). Returns 0 past end of data (callers pre-check len).
fn read(&mut self, n: usize) -> u32 {
let mut v = 0u32;
for _ in 0..n {
let byte = self.data.get(self.bit / 8).copied().unwrap_or(0);
let shift = 7 - (self.bit % 8);
v = (v << 1) | ((byte >> shift) & 1) as u32;
self.bit += 1;
}
v
}
}
/// Decoded (E-)AC-3 stream parameters needed for the `dac3`/`dec3` config box
/// and the audio sample entry.
pub(super) struct DolbyConfig {
pub fscod: u8,
pub bsid: u8,
pub bsmod: u8,
pub acmod: u8,
pub lfeon: bool,
/// AC-3 only: `bit_rate_code` (= `frmsizecod >> 1`). Unused for E-AC-3.
pub bit_rate_code: u8,
/// E-AC-3 only: nominal data rate in kbps (for `dec3`). 0 for AC-3.
pub data_rate_kbps: u16,
pub sample_rate: u32,
pub channels: u16,
}
impl DolbyConfig {
fn channel_count(acmod: u8, lfeon: bool) -> u16 {
ACMOD_CHANNELS[acmod as usize] as u16 + lfeon as u16
}
}
/// Parse the first (E-)AC-3 frame starting at the 0x0B77 syncword. Returns
/// `None` if the frame is too short or the syncword is absent.
pub(super) fn parse_dolby(frame: &[u8]) -> Option<DolbyConfig> {
let start =
(0..frame.len().saturating_sub(1)).find(|&i| frame[i] == 0x0B && frame[i + 1] == 0x77)?;
let f = &frame[start..];
if f.len() < 6 {
return None;
}
// bsid lives in byte 5 bits 7-3 for both AC-3 and E-AC-3.
let bsid = (f[5] >> 3) & 0x1F;
if bsid >= 11 {
parse_eac3(f)
} else {
parse_ac3(f)
}
}
/// Legacy AC-3 (A/52 §5.3.2): syncword | crc(16) | fscod(2) frmsizecod(6) |
/// bsid(5) bsmod(3) | acmod(3) …optional… lfeon.
fn parse_ac3(f: &[u8]) -> Option<DolbyConfig> {
if f.len() < 8 {
return None;
}
let fscod = (f[4] >> 6) & 0x03;
let frmsizecod = f[4] & 0x3F;
let bsid = (f[5] >> 3) & 0x1F;
let bsmod = f[5] & 0x07;
// acmod + trailing optional 2-bit fields, then lfeon (byte 6 onward).
let mut r = BitReader::new(f);
r.bit = 6 * 8;
let acmod = r.read(3) as u8;
if (acmod & 0x1) != 0 && acmod != 0x1 {
r.skip(2); // cmixlev
}
if (acmod & 0x4) != 0 {
r.skip(2); // surmixlev
}
if acmod == 0x2 {
r.skip(2); // dsurmod
}
let lfeon = r.read(1) == 1;
Some(DolbyConfig {
fscod,
bsid,
bsmod,
acmod,
lfeon,
bit_rate_code: frmsizecod >> 1,
data_rate_kbps: 0,
sample_rate: FSCOD_RATES.get(fscod as usize).copied().unwrap_or(48_000),
channels: DolbyConfig::channel_count(acmod, lfeon),
})
}
/// E-AC-3 (A/52 Annex E BSI): syncword | strmtyp(2) substreamid(3) frmsiz(11) |
/// fscod(2) numblkscod(2) acmod(3) lfeon(1) | bsid(5) …
fn parse_eac3(f: &[u8]) -> Option<DolbyConfig> {
if f.len() < 6 {
return None;
}
let frmsiz = (((f[2] & 0x07) as u32) << 8) | f[3] as u32; // words minus one
let fscod = (f[4] >> 6) & 0x03;
let numblkscod = (f[4] >> 4) & 0x03;
let acmod = (f[4] >> 1) & 0x07;
let lfeon = (f[4] & 0x01) == 1;
let bsid = (f[5] >> 3) & 0x1F;
let (sample_rate, blocks) = if fscod == 0x03 {
let fscod2 = (f[4] >> 4) & 0x03; // shares bits with numblkscod when fscod==3
(EAC3_REDUCED_RATES[fscod2 as usize], 6u32)
} else {
let blocks = [1u32, 2, 3, 6][numblkscod as usize];
(FSCOD_RATES[fscod as usize], blocks)
};
// Nominal data rate (kbps): frame is (frmsiz+1) 16-bit words per (blocks·256)
// samples at sample_rate. rate = bytes·8·sr / samples / 1000.
let frame_bytes = (frmsiz as u64 + 1) * 2;
let samples = blocks as u64 * 256;
let data_rate_kbps = if samples > 0 {
((frame_bytes * 8 * sample_rate as u64) / samples / 1000) as u16
} else {
0
};
Some(DolbyConfig {
fscod,
bsid,
bsmod: 0, // not in the E-AC-3 main header; dec3 default
acmod,
lfeon,
bit_rate_code: 0,
data_rate_kbps,
sample_rate,
channels: DolbyConfig::channel_count(acmod, lfeon),
})
}
/// The `dac3` config box (ETSI TS 102 366 Annex F.4): 24 bits —
/// fscod(2) bsid(5) bsmod(3) acmod(3) lfeon(1) bit_rate_code(5) reserved(5).
pub(super) fn dac3_box(c: &DolbyConfig) -> Vec<u8> {
let mut v: u32 = 0;
let mut push = |val: u32, bits: u32| v = (v << bits) | (val & ((1 << bits) - 1));
push(c.fscod as u32, 2);
push(c.bsid as u32, 5);
push(c.bsmod as u32, 3);
push(c.acmod as u32, 3);
push(c.lfeon as u32, 1);
push(c.bit_rate_code as u32, 5);
push(0, 5); // reserved
// 24 bits → the top 3 bytes of the big-endian u32.
let b = v.to_be_bytes();
bx(b"dac3", &[b[1], b[2], b[3]])
}
/// The `dec3` config box (ETSI TS 102 366 Annex G.3) for a single independent
/// substream, no dependent substreams: data_rate(13) num_ind_sub(3) then
/// fscod(2) bsid(5) reserved(1) asvc(1) bsmod(3) acmod(3) lfeon(1) reserved(3)
/// num_dep_sub(4) reserved(1).
pub(super) fn dec3_box(c: &DolbyConfig) -> Vec<u8> {
let mut v: u64 = 0;
let mut push = |val: u64, bits: u32| v = (v << bits) | (val & ((1u64 << bits) - 1));
push(c.data_rate_kbps as u64, 13);
push(0, 3); // num_ind_sub - 1 = 0 (one substream)
push(c.fscod as u64, 2);
push(c.bsid as u64, 5);
push(0, 1); // reserved
push(0, 1); // asvc
push(c.bsmod as u64, 3);
push(c.acmod as u64, 3);
push(c.lfeon as u64, 1);
push(0, 3); // reserved
push(0, 4); // num_dep_sub = 0
push(0, 1); // reserved (chan_loc absent when num_dep_sub == 0)
// 40 bits → the low 5 bytes of the big-endian u64.
let b = v.to_be_bytes();
bx(b"dec3", &[b[3], b[4], b[5], b[6], b[7]])
}
/// Build an audio sample entry (`ac-3` / `ec-3`) with the given config box.
/// `AudioSampleEntry` per ISO/IEC 14496-12 §12.2.3.
pub(super) fn audio_sample_entry(
fourcc: &[u8; 4],
channels: u16,
sample_rate: u32,
config: &[u8],
) -> Vec<u8> {
let mut e = Vec::new();
e.extend_from_slice(&[0u8; 6]); // reserved
e.extend_from_slice(&1u16.to_be_bytes()); // data_reference_index
e.extend_from_slice(&[0u8; 8]); // reserved (version 0)
e.extend_from_slice(&channels.to_be_bytes());
e.extend_from_slice(&16u16.to_be_bytes()); // samplesize
e.extend_from_slice(&0u16.to_be_bytes()); // pre_defined
e.extend_from_slice(&0u16.to_be_bytes()); // reserved
// samplerate is 16.16 fixed point; the integer rate in the high 16 bits.
e.extend_from_slice(&(sample_rate << 16).to_be_bytes());
e.extend_from_slice(config);
bx(fourcc, &e)
}
// ── DTS (dtsc/dtsh + ddts) ───────────────────────────────────────────────────
/// DTS core `SFREQ` (4-bit) → sample rate (Hz). Reserved indices → 48 kHz.
const DTS_SFREQ: [u32; 16] = [
48_000, 8_000, 16_000, 32_000, 48_000, 48_000, 11_025, 22_050, 44_100, 48_000, 48_000, 12_000,
24_000, 48_000, 96_000, 192_000,
];
/// DTS core base channel count per `AMODE` (0..=9); higher AMODEs are rare on disc.
const DTS_AMODE_CH: [u8; 10] = [1, 2, 2, 2, 2, 3, 3, 4, 4, 5];
/// Decoded DTS core parameters needed for the `ddts` box.
struct DtsConfig {
sample_rate: u32,
channels: u16,
amode: u8,
lfe: bool,
core_size: u32,
/// Samples per frame ((NBLKS+1)·32).
frame_samples: u32,
/// Whether a DTS-HD extension substream follows the core.
has_extension: bool,
channel_layout: u16,
}
/// Parse the DTS core header (ETSI TS 102 114 §5.3.1), starting at the
/// 0x7FFE8001 big-endian core sync. Returns `None` if too short / no sync.
fn parse_dts(frame: &[u8]) -> Option<DtsConfig> {
let start = (0..frame.len().saturating_sub(3)).find(|&i| {
frame[i] == 0x7F && frame[i + 1] == 0xFE && frame[i + 2] == 0x80 && frame[i + 3] == 0x01
})?;
let f = &frame[start..];
if f.len() < 11 {
return None;
}
// Bit fields after the 32-bit sync (MSB-first):
// FTYPE1 SHORT5 CPF1 NBLKS7 FSIZE14 AMODE6 SFREQ4 RATE5 ...
let nblks = (((f[4] & 0x01) as u32) << 6) | ((f[5] >> 2) as u32 & 0x3F);
let fsize = (((f[5] & 0x03) as u32) << 12) | ((f[6] as u32) << 4) | ((f[7] >> 4) as u32 & 0x0F);
let amode = (((f[7] & 0x0F) << 2) | ((f[8] >> 6) & 0x03)) as usize;
let sfreq = ((f[8] >> 2) & 0x0F) as usize;
// LFF is 2 bits at bit offset 85 → byte10 bits 2-1.
let lff = (f[10] >> 1) & 0x03;
let lfe = lff == 1 || lff == 2;
let sample_rate = DTS_SFREQ[sfreq];
let base_ch = DTS_AMODE_CH.get(amode).copied().unwrap_or(6);
let channels = base_ch as u16 + lfe as u16;
let channel_layout = dts_channel_layout(amode, lfe);
// DTS-HD extension substream sync (0x64582025) after the core frame.
let ext_sync = [0x64, 0x58, 0x20, 0x25];
let has_extension = f
.windows(4)
.skip((fsize as usize + 1).min(f.len()).saturating_sub(4))
.any(|w| w == ext_sync)
|| f.windows(4).any(|w| w == ext_sync);
Some(DtsConfig {
sample_rate,
channels,
amode: amode as u8,
lfe,
core_size: fsize + 1,
frame_samples: (nblks + 1) * 32,
has_extension,
channel_layout,
})
}
/// `ddts` ChannelLayout (16-bit speaker mask) for the common core layouts.
/// bit0=C, bit1=L/R, bit2=Ls/Rs, bit3=LFE.
fn dts_channel_layout(amode: usize, lfe: bool) -> u16 {
let mut m = match amode {
0 => 0x0001, // C (mono)
1..=4 => 0x0002, // L/R
5 => 0x0003, // C + L/R
6 | 8 => 0x0006, // L/R + Ls/Rs (no centre)
_ => 0x0007, // C + L/R + surround (amode 7, 9, …)
};
if lfe {
m |= 0x0008;
}
m
}
/// The `ddts` config box (ETSI TS 102 114 Annex; DTS-in-ISO registration).
/// Describes the DTS core; whole access units (core + any extension) are passed
/// through as samples, so a DTS-HD-aware decoder still finds the extension.
fn ddts_box(c: &DtsConfig) -> Vec<u8> {
// avg/max bitrate: computed from the core frame size × frame rate (the core
// RATE field reads "open/variable" for lossless, so it's not usable directly).
let frames_per_sec = if c.frame_samples > 0 {
c.sample_rate as u64 / c.frame_samples as u64
} else {
0
};
let bitrate = (c.core_size as u64 * 8 * frames_per_sec) as u32;
let mut out = Vec::new();
out.extend_from_slice(&c.sample_rate.to_be_bytes()); // DTSSamplingFrequency
out.extend_from_slice(&bitrate.to_be_bytes()); // maxBitrate
out.extend_from_slice(&bitrate.to_be_bytes()); // avgBitrate
out.push(if c.has_extension { 24 } else { 16 }); // pcmSampleDepth
// Bit-packed tail (56 bits):
// FrameDuration2 StreamConstruction5 CoreLFEPresent1 CoreLayout6 CoreSize14
// StereoDownmix1 RepresentationType3 ChannelLayout16 MultiAssetFlag1
// LBRDurationMod1 ReservedBoxPresent1 Reserved5
let frame_duration = match c.frame_samples {
0..=512 => 0,
513..=1024 => 1,
1025..=2048 => 2,
_ => 3,
};
// StreamConstruction: 1 = DTS core present. Whole-AU passthrough means an
// HD decoder still parses the extension substreams from the stream itself.
let stream_construction = 1u128;
let mut v: u128 = 0;
let mut push = |val: u128, bits: u32| v = (v << bits) | (val & ((1u128 << bits) - 1));
push(frame_duration as u128, 2);
push(stream_construction, 5);
push(c.lfe as u128, 1);
push(c.amode as u128, 6);
push(c.core_size as u128, 14);
push(0, 1); // StereoDownmix
push(0, 3); // RepresentationType
push(c.channel_layout as u128, 16);
push(c.has_extension as u128, 1); // MultiAssetFlag
push(0, 1); // LBRDurationMod
push(0, 1); // ReservedBoxPresent
push(0, 5); // Reserved
// 56 bits → the low 7 bytes of the big-endian u128.
let b = v.to_be_bytes();
out.extend_from_slice(&b[9..16]);
bx(b"ddts", &out)
}
/// The MP4 fourcc + config box for an audio frame, or `None` if the codec has no
/// MP4 mapping here. Together with [`audio_fits`] this is the fit oracle for
/// audio: only what returns `Some` is muxable.
pub(super) fn dolby_sample_entry(codec: Codec, first_frame: &[u8]) -> Option<Vec<u8>> {
match codec {
Codec::Ac3 => {
let c = parse_dolby(first_frame)?;
Some(audio_sample_entry(
b"ac-3",
c.channels,
c.sample_rate,
&dac3_box(&c),
))
}
Codec::Ac3Plus => {
let c = parse_dolby(first_frame)?;
Some(audio_sample_entry(
b"ec-3",
c.channels,
c.sample_rate,
&dec3_box(&c),
))
}
Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr => {
let c = parse_dts(first_frame)?;
// `dtsc` = DTS core; `dtsh` = DTS-HD (core + extension substreams).
let fourcc: &[u8; 4] = if c.has_extension { b"dtsh" } else { b"dtsc" };
Some(audio_sample_entry(
fourcc,
c.channels,
c.sample_rate,
&ddts_box(&c),
))
}
_ => None,
}
}
/// Fit oracle for an audio codec: does `mp4://` currently carry it? Covers the
/// Dolby family (AC-3 / E-AC-3) and DTS (core / DTS-HD HRA / DTS-HD MA — the core
/// is described, whole access units pass through). TrueHD, LPCM, AAC are not yet
/// mapped and are skipped with a loud report (never silently dropped).
pub(super) fn audio_fits(codec: Codec) -> bool {
matches!(
codec,
Codec::Ac3 | Codec::Ac3Plus | Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr
)
}
#[cfg(test)]
mod tests {
use super::*;
/// A synthetic legacy AC-3 header: syncword, crc, fscod=0 (48k),
/// frmsizecod, bsid=8, bsmod=0, acmod=7 (3/2), lfeon=1 → 5.1.
fn ac3_frame_5_1() -> Vec<u8> {
let mut f = vec![0x0B, 0x77, 0x00, 0x00];
// byte4: fscod(2)=0 | frmsizecod(6)=0b010110 (22)
f.push(0b00_010110);
// byte5: bsid(5)=8 (0b01000) | bsmod(3)=0
f.push(0b01000_000);
// byte6: acmod(3)=7 (0b111) | cmixlev(2) | surmixlev(2) | lfeon(1)...
// acmod=7 has centre (needs cmixlev) and surround (needs surmixlev):
// 111 | 00 | 00 | 1(lfeon) = 0b111_00_00_1
f.push(0b111_00_00_1);
f.push(0x00);
f
}
#[test]
fn ac3_bsi_and_dac3() {
let c = parse_dolby(&ac3_frame_5_1()).expect("parsed");
assert!(c.bsid < 11, "legacy AC-3");
assert_eq!(c.fscod, 0);
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.bsid, 8);
assert_eq!(c.acmod, 7);
assert!(c.lfeon);
assert_eq!(c.channels, 6, "3/2 + LFE = 5.1");
assert_eq!(c.bit_rate_code, 22 >> 1);
let dac3 = dac3_box(&c);
// [size:4]["dac3"][3-byte payload] = 11 bytes.
assert_eq!(dac3.len(), 11);
assert_eq!(&dac3[4..8], b"dac3");
}
#[test]
fn eac3_bsi_and_dec3() {
// E-AC-3: syncword | strmtyp/substreamid/frmsiz | fscod/numblks/acmod/lfeon | bsid
let mut f = vec![0x0B, 0x77];
f.push(0x00); // strmtyp=0, substreamid=0, frmsiz high=0
f.push(0x3F); // frmsiz low = 63 → frame 128 bytes
// byte4: fscod(2)=0 | numblkscod(2)=3 (6 blocks) | acmod(3)=7 | lfeon(1)=1
f.push(0b00_11_111_1);
// byte5: bsid(5)=16 (E-AC-3) | dialnorm high(3)
f.push(0b10000_000);
f.push(0x00);
let c = parse_dolby(&f).expect("parsed");
assert!(c.bsid >= 11, "E-AC-3");
assert_eq!(c.bsid, 16);
assert_eq!(c.fscod, 0);
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.acmod, 7);
assert!(c.lfeon);
assert_eq!(c.channels, 6);
let dec3 = dec3_box(&c);
// [size:4]["dec3"][5-byte payload] = 13 bytes.
assert_eq!(dec3.len(), 13);
assert_eq!(&dec3[4..8], b"dec3");
}
#[test]
fn sample_entry_shape() {
let c = parse_dolby(&ac3_frame_5_1()).unwrap();
let e = audio_sample_entry(b"ac-3", c.channels, c.sample_rate, &dac3_box(&c));
assert_eq!(&e[4..8], b"ac-3");
// channelcount at entry-body offset 16 (after 6 reserved + 2 dri + 8 reserved).
let ch = u16::from_be_bytes([e[8 + 16], e[8 + 17]]);
assert_eq!(ch, 6);
}
#[test]
fn fit_oracle_covers_dolby_and_dts() {
assert!(audio_fits(Codec::Ac3));
assert!(audio_fits(Codec::Ac3Plus));
assert!(audio_fits(Codec::Dts));
assert!(audio_fits(Codec::DtsHdMa));
assert!(!audio_fits(Codec::TrueHd));
assert!(!audio_fits(Codec::Lpcm));
}
#[test]
fn dts_core_5_1_and_ddts() {
// Synthetic DTS core: SFREQ=13 (48k), AMODE=9 (5ch), LFF=1 (LFE) → 5.1.
let f = vec![
0x7F, 0xFE, 0x80, 0x01, 0x00, 0x3C, 0x05, 0xF2, 0x77, 0x00, 0x02, 0x00,
];
let c = parse_dts(&f).expect("dts core parsed");
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.amode, 9);
assert!(c.lfe);
assert_eq!(c.channels, 6, "5 core + LFE = 5.1");
assert_eq!(c.channel_layout, 0x000F, "C + L/R + Ls/Rs + LFE");
assert_eq!(c.core_size, 96);
assert_eq!(c.frame_samples, 512);
let ddts = ddts_box(&c);
assert_eq!(&ddts[4..8], b"ddts");
// DTSSamplingFrequency (first field) = 48000.
assert_eq!(
u32::from_be_bytes([ddts[8], ddts[9], ddts[10], ddts[11]]),
48_000
);
// Sample entry uses dtsc (no extension in this synthetic frame).
let e = dolby_sample_entry(Codec::DtsHdMa, &f).unwrap();
assert_eq!(&e[4..8], b"dtsc");
}
}
+27
View File
@@ -0,0 +1,27 @@
//! ISO-BMFF box primitives: `[size:u32-BE][type:4][body]` (ISO/IEC 14496-12
//! §4.2), and the FullBox variant that prefixes a 1-byte version + 3-byte flags.
/// Wrap a body in a plain box `[size][type][body]`. `size` counts the 8-byte
/// header. All `moov`-tree boxes are small (the large `mdat` is written directly
/// with a 64-bit size, not through here), so a `u32` size never overflows.
pub(super) fn bx(box_type: &[u8; 4], body: &[u8]) -> Vec<u8> {
let total = body.len() + 8;
debug_assert!(
total <= u32::MAX as usize,
"mp4 box {box_type:?} exceeds u32"
);
let mut out = Vec::with_capacity(total);
out.extend_from_slice(&(total as u32).to_be_bytes());
out.extend_from_slice(box_type);
out.extend_from_slice(body);
out
}
/// Wrap a body in a FullBox: `[size][type][version:1][flags:3][body]`.
pub(super) fn fullbox(box_type: &[u8; 4], version: u8, flags: u32, body: &[u8]) -> Vec<u8> {
let mut full = Vec::with_capacity(body.len() + 4);
full.push(version);
full.extend_from_slice(&flags.to_be_bytes()[1..]); // low 3 bytes
full.extend_from_slice(body);
bx(box_type, &full)
}
+1099
View File
File diff suppressed because it is too large Load Diff
+705
View File
@@ -0,0 +1,705 @@
//! Progressive MP4 (ISO-BMFF) demuxer — the read side of `mp4://`.
//!
//! The inverse of the writer in [`super`]: parse `moov`/`trak`/`stbl`, rebuild a
//! [`DiscTitle`] and a per-sample index (offset/size/timing/sync from
//! `stsc`+`stco`/`co64`+`stsz`, `stts`+`ctts`, `stss`), then stream each sample
//! out as a [`PesFrame`] in decode order. Video NALs are length-prefixed in MP4 —
//! exactly the framing the MKV muxer consumes — so `mp4://` → `mkv://` (and
//! `mp4://` → any sink) needs no reframing.
//!
//! Scope: progressive MP4 (`moov` + `mdat`). Fragmented MP4 (`moof`) is out of
//! scope for now — its samples live in `traf`/`trun`, not `stbl`.
use crate::disc::{
AudioChannels, AudioStream, Codec, DiscTitle, Resolution, SampleRate, Stream as DiscStream,
VideoStream,
};
use crate::labels::LabelPurpose;
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Read, Seek, SeekFrom};
use std::path::Path;
const NS: i128 = 1_000_000_000;
/// One sample's location + timing in the emission plan.
struct SampleRef {
track: usize,
offset: u64,
size: u32,
/// Composition (presentation) time in nanoseconds.
pts_ns: i64,
/// Decode time in nanoseconds — the key the global emission order sorts on.
dts_ns: i64,
keyframe: bool,
}
/// MP4 reader: a `Stream` source that emits a file's samples as PES frames.
/// Generic over the backing reader so it works over a `File` (the `mp4://`
/// source) or an in-memory `Cursor` (round-trip tests).
pub struct Mp4Reader<R: Read + Seek> {
file: R,
title: DiscTitle,
samples: Vec<SampleRef>,
cursor: usize,
}
impl Mp4Reader<File> {
/// Open and index an MP4 file by path.
pub fn open(path: &Path) -> io::Result<Self> {
let name = path
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("mp4")
.to_string();
Self::from_reader(File::open(path)?, name)
}
}
impl<R: Read + Seek> Mp4Reader<R> {
/// Index an already-opened seekable MP4 reader.
pub fn from_reader(mut file: R, name: String) -> io::Result<Self> {
let moov = read_moov(&mut file)?;
let mut title = DiscTitle::empty();
title.playlist = name;
let mut samples: Vec<SampleRef> = Vec::new();
let mut codec_privates: Vec<Option<Vec<u8>>> = Vec::new();
let mut track_idx = 0usize;
for trak in find_boxes(&moov, b"trak") {
let Some(mdia) = find_box(trak, b"mdia") else {
continue;
};
let timescale = find_box(mdia, b"mdhd")
.and_then(mdhd_timescale)
.unwrap_or(90_000);
let language = find_box(mdia, b"mdhd").and_then(mdhd_language);
let handler = find_box(mdia, b"hdlr").and_then(hdlr_type);
let Some(minf) = find_box(mdia, b"minf") else {
continue;
};
let Some(stbl) = find_box(minf, b"stbl") else {
continue;
};
let Some(stsd) = find_box(stbl, b"stsd") else {
continue;
};
let Some(StsdInfo {
codec,
height,
config,
channels,
}) = parse_stsd(stsd)
else {
continue; // unrecognised sample entry — skip the track
};
// Build the stream model for this track.
let stream = match handler {
Some(h) if &h == b"vide" => DiscStream::Video(VideoStream {
pid: 0x1011 + track_idx as u16,
codec,
resolution: Resolution::from_height(height as u32),
frame_rate: crate::disc::FrameRate::Unknown,
hdr: crate::disc::HdrFormat::Sdr,
color_space: crate::disc::ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
}),
Some(h) if &h == b"soun" => DiscStream::Audio(AudioStream {
pid: 0x1100 + track_idx as u16,
codec,
channels: AudioChannels::from_count(channels as u8),
language: language.clone().unwrap_or_else(|| "und".into()),
sample_rate: SampleRate::from_hz(timescale),
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
}),
_ => continue, // non-A/V handler
};
// Per-sample tables.
let sizes = find_box(stbl, b"stsz").map(parse_stsz).unwrap_or_default();
let n = sizes.len();
if n == 0 {
track_idx += 1;
title.streams.push(stream);
codec_privates.push(config);
continue;
}
let chunk_offsets = find_box(stbl, b"stco")
.map(|b| parse_stco(b, false))
.or_else(|| find_box(stbl, b"co64").map(|b| parse_stco(b, true)))
.unwrap_or_default();
let stsc = find_box(stbl, b"stsc").map(parse_stsc).unwrap_or_default();
let offsets = sample_offsets(&sizes, &chunk_offsets, &stsc);
let durations = find_box(stbl, b"stts").map(parse_stts).unwrap_or_default();
let ctts = find_box(stbl, b"ctts").map(parse_ctts).unwrap_or_default();
let sync = find_box(stbl, b"stss").map(parse_stss);
let mut decode_ticks: i64 = 0;
for (i, &size) in sizes.iter().enumerate() {
let dur = durations.get(i).copied().unwrap_or(0);
let comp = ctts.get(i).copied().unwrap_or(0);
let dts_ns = (decode_ticks as i128 * NS / timescale as i128) as i64;
let pts_ticks = decode_ticks + comp as i64;
let pts_ns = (pts_ticks as i128 * NS / timescale as i128) as i64;
decode_ticks += dur as i64;
let keyframe = match &sync {
Some(set) => set.contains(&(i as u32 + 1)),
None => true, // no stss → every sample is a sync sample
};
samples.push(SampleRef {
track: track_idx,
offset: offsets.get(i).copied().unwrap_or(0),
size,
pts_ns,
dts_ns,
keyframe,
});
}
title.streams.push(stream);
codec_privates.push(config);
track_idx += 1;
}
if title.streams.is_empty() {
return Err(crate::error::Error::MkvInvalid.into());
}
title.codec_privates = codec_privates;
// Emit in global decode order so the consumer sees interleaved,
// monotonic-DTS frames (a stable sort keeps per-track order on ties).
samples.sort_by_key(|s| s.dts_ns);
Ok(Self {
file,
title,
samples,
cursor: 0,
})
}
}
impl<R: Read + Seek + Send> Stream for Mp4Reader<R> {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
let Some(s) = self.samples.get(self.cursor) else {
return Ok(None);
};
self.cursor += 1;
self.file.seek(SeekFrom::Start(s.offset))?;
let mut data = vec![0u8; s.size as usize];
self.file.read_exact(&mut data)?;
Ok(Some(PesFrame {
track: s.track,
pts: s.pts_ns,
keyframe: s.keyframe,
data,
duration_ns: None,
source: None,
coding: None,
}))
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Err(crate::error::Error::StreamReadOnly.into())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
self.title.codec_privates.get(track).and_then(|c| c.clone())
}
}
// ── box tree navigation ──────────────────────────────────────────────────────
/// Read top-level boxes until `moov`, returning its payload (after the header).
/// Skips over `ftyp`/`mdat`/etc. via seek; samples are read later by offset.
fn read_moov<R: Read + Seek>(file: &mut R) -> io::Result<Vec<u8>> {
loop {
let mut hdr = [0u8; 8];
if file.read_exact(&mut hdr).is_err() {
return Err(crate::error::Error::MkvInvalid.into());
}
let size32 = u32::from_be_bytes([hdr[0], hdr[1], hdr[2], hdr[3]]);
let btype = [hdr[4], hdr[5], hdr[6], hdr[7]];
// 64-bit largesize (size==1): the real size is the next 8 bytes; a
// 16-byte header precedes the payload. size==0 means "to EOF".
let payload_len: u64 = if size32 == 1 {
let mut ext = [0u8; 8];
file.read_exact(&mut ext)?;
u64::from_be_bytes(ext).saturating_sub(16)
} else {
(size32 as u64).saturating_sub(8)
};
if &btype == b"moov" {
let mut buf = vec![0u8; payload_len as usize];
file.read_exact(&mut buf)?;
return Ok(buf);
}
file.seek(SeekFrom::Current(payload_len as i64))?;
}
}
/// The first child box of `payload` with the given type — returns its payload
/// (bytes after the 8-byte header). One level.
fn find_box<'a>(payload: &'a [u8], want: &[u8; 4]) -> Option<&'a [u8]> {
find_boxes(payload, want).into_iter().next()
}
/// All child boxes of `payload` with the given type — each as its payload slice.
fn find_boxes<'a>(payload: &'a [u8], want: &[u8; 4]) -> Vec<&'a [u8]> {
let mut out = Vec::new();
let mut pos = 0;
while pos + 8 <= payload.len() {
let size = u32::from_be_bytes([
payload[pos],
payload[pos + 1],
payload[pos + 2],
payload[pos + 3],
]) as usize;
let bt = [
payload[pos + 4],
payload[pos + 5],
payload[pos + 6],
payload[pos + 7],
];
if size < 8 || pos + size > payload.len() {
break;
}
if &bt == want {
out.push(&payload[pos + 8..pos + size]);
}
pos += size;
}
out
}
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn be16(b: &[u8], o: usize) -> u16 {
u16::from_be_bytes([b[o], b[o + 1]])
}
/// mdhd (version 0/1) → media timescale.
fn mdhd_timescale(b: &[u8]) -> Option<u32> {
let version = b.first().copied()?;
if version == 1 {
// version(1)+flags(3) creation(8) modification(8) timescale(4) ...
(b.len() >= 24).then(|| be32(b, 20))
} else {
// creation(4) modification(4) timescale(4) ...
(b.len() >= 16).then(|| be32(b, 12))
}
}
/// mdhd language (5-bit packed ISO 639-2) → lowercase 3-letter code.
fn mdhd_language(b: &[u8]) -> Option<String> {
let version = b.first().copied()?;
let off = if version == 1 { 28 } else { 20 };
if b.len() < off + 2 {
return None;
}
let packed = be16(b, off);
let c0 = ((packed >> 10) & 0x1F) as u8 + 0x60;
let c1 = ((packed >> 5) & 0x1F) as u8 + 0x60;
let c2 = (packed & 0x1F) as u8 + 0x60;
let s: String = [c0, c1, c2].iter().map(|&c| c as char).collect();
if s.chars().all(|c| c.is_ascii_lowercase()) {
Some(s)
} else {
None
}
}
/// hdlr → handler_type fourcc ('vide' / 'soun').
fn hdlr_type(b: &[u8]) -> Option<[u8; 4]> {
// version+flags(4) pre_defined(4) handler_type(4) ...
(b.len() >= 12).then(|| [b[8], b[9], b[10], b[11]])
}
/// Decoded first sample entry of an `stsd` box.
struct StsdInfo {
codec: Codec,
height: u16,
config: Option<Vec<u8>>,
channels: u16,
}
/// stsd → codec + dimensions + codec_private + channel count (first entry).
fn parse_stsd(b: &[u8]) -> Option<StsdInfo> {
// version+flags(4) entry_count(4) then the first sample entry box.
if b.len() < 8 {
return None;
}
let entry = &b[8..];
if entry.len() < 8 {
return None;
}
let size = be32(entry, 0) as usize;
let fourcc = [entry[4], entry[5], entry[6], entry[7]];
let body = &entry[8..size.min(entry.len())];
let codec = match &fourcc {
b"hvc1" | b"hev1" => Codec::Hevc,
b"avc1" | b"avc3" => Codec::H264,
b"ac-3" => Codec::Ac3,
b"ec-3" => Codec::Ac3Plus,
b"mp4a" => Codec::Aac,
b"dtsc" | b"dtse" | b"dtsh" | b"dtsl" => Codec::Dts,
_ => return None,
};
if matches!(codec, Codec::Hevc | Codec::H264) {
// VisualSampleEntry: 6 reserved + 2 dri + 16 pre/reserved + width(2)
// height(2) + 14 + 32 compressorname + 2 depth + 2 pre = 78 bytes, then
// child boxes (hvcC/avcC, colr, …).
if body.len() < 78 {
return None;
}
let height = be16(body, 26);
let children = &body[78..];
let config = find_box(children, b"hvcC")
.or_else(|| find_box(children, b"avcC"))
.map(|c| c.to_vec());
Some(StsdInfo {
codec,
height,
config,
channels: 0,
})
} else {
// AudioSampleEntry: 6 reserved + 2 dri + 8 reserved + channelcount(2)
// samplesize(2) + 2 pre + 2 reserved + samplerate(4) = 28 bytes.
let channels = if body.len() >= 28 { be16(body, 16) } else { 2 };
Some(StsdInfo {
codec,
height: 0,
config: None,
channels,
})
}
}
/// stsz → per-sample sizes.
fn parse_stsz(b: &[u8]) -> Vec<u32> {
if b.len() < 12 {
return Vec::new();
}
let sample_size = be32(b, 4);
let count = be32(b, 8) as usize;
if sample_size != 0 {
return vec![sample_size; count];
}
let mut out = Vec::with_capacity(count);
for i in 0..count {
let o = 12 + i * 4;
if o + 4 > b.len() {
break;
}
out.push(be32(b, o));
}
out
}
/// stco (32-bit) / co64 (64-bit) → chunk offsets.
fn parse_stco(b: &[u8], is64: bool) -> Vec<u64> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::with_capacity(count);
let stride = if is64 { 8 } else { 4 };
for i in 0..count {
let o = 8 + i * stride;
if o + stride > b.len() {
break;
}
if is64 {
out.push(u64::from_be_bytes([
b[o],
b[o + 1],
b[o + 2],
b[o + 3],
b[o + 4],
b[o + 5],
b[o + 6],
b[o + 7],
]));
} else {
out.push(be32(b, o) as u64);
}
}
out
}
/// stsc → (first_chunk, samples_per_chunk) entries (1-based first_chunk).
fn parse_stsc(b: &[u8]) -> Vec<(u32, u32)> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::with_capacity(count);
for i in 0..count {
let o = 8 + i * 12;
if o + 12 > b.len() {
break;
}
out.push((be32(b, o), be32(b, o + 4)));
}
out
}
/// Reconstruct per-sample file offsets from sizes + chunk offsets + stsc.
fn sample_offsets(sizes: &[u32], chunk_offsets: &[u64], stsc: &[(u32, u32)]) -> Vec<u64> {
let n_chunks = chunk_offsets.len();
// Expand stsc → samples_per_chunk for every chunk.
let mut spc = vec![0u32; n_chunks];
for (idx, &(first, per)) in stsc.iter().enumerate() {
let start = (first.saturating_sub(1)) as usize;
let end = stsc
.get(idx + 1)
.map(|&(nf, _)| (nf.saturating_sub(1)) as usize)
.unwrap_or(n_chunks);
let end = end.min(n_chunks);
if start < end {
spc[start..end].fill(per);
}
}
let mut offsets = Vec::with_capacity(sizes.len());
let mut sidx = 0usize;
for (ci, &choff) in chunk_offsets.iter().enumerate() {
let mut off = choff;
for _ in 0..spc[ci] {
if sidx >= sizes.len() {
break;
}
offsets.push(off);
off += sizes[sidx] as u64;
sidx += 1;
}
}
// Any trailing samples with no chunk mapping: pack after the last offset.
while offsets.len() < sizes.len() {
let last = offsets.last().copied().unwrap_or(0);
let last_sz = sizes
.get(offsets.len().saturating_sub(1))
.copied()
.unwrap_or(0);
offsets.push(last + last_sz as u64);
}
offsets
}
/// stts → per-sample decode durations (expanded from run-length entries).
fn parse_stts(b: &[u8]) -> Vec<u32> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::new();
for i in 0..count {
let o = 8 + i * 8;
if o + 8 > b.len() {
break;
}
let n = be32(b, o);
let delta = be32(b, o + 4);
for _ in 0..n {
out.push(delta);
}
}
out
}
/// ctts → per-sample composition offsets (version 0 unsigned / version 1 signed).
fn parse_ctts(b: &[u8]) -> Vec<i32> {
if b.len() < 8 {
return Vec::new();
}
// version 0 = unsigned, version 1 = signed offsets; the u32→i32 bit-cast
// reads both correctly (real composition offsets fit in i32 either way).
let count = be32(b, 4) as usize;
let mut out = Vec::new();
for i in 0..count {
let o = 8 + i * 8;
if o + 8 > b.len() {
break;
}
let n = be32(b, o);
let offset = be32(b, o + 4) as i32;
for _ in 0..n {
out.push(offset);
}
}
out
}
/// stss → set of 1-based sync sample numbers.
fn parse_stss(b: &[u8]) -> std::collections::HashSet<u32> {
let mut set = std::collections::HashSet::new();
if b.len() < 8 {
return set;
}
let count = be32(b, 4) as usize;
for i in 0..count {
let o = 8 + i * 4;
if o + 4 > b.len() {
break;
}
set.insert(be32(b, o));
}
set
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn stsc_offsets_one_sample_per_chunk() {
// Our writer's layout: 1 sample/chunk, co64 lists every offset.
let sizes = vec![100u32, 200, 300];
let chunks = vec![1000u64, 1100, 1300];
let stsc = vec![(1u32, 1u32)];
assert_eq!(
sample_offsets(&sizes, &chunks, &stsc),
vec![1000, 1100, 1300]
);
}
#[test]
fn stsc_offsets_multi_sample_chunks() {
// 2 samples in chunk 1, 1 in chunk 2.
let sizes = vec![10u32, 20, 30];
let chunks = vec![500u64, 900];
let stsc = vec![(1u32, 2u32), (2u32, 1u32)];
// chunk1@500: s0@500, s1@510; chunk2@900: s2@900.
assert_eq!(sample_offsets(&sizes, &chunks, &stsc), vec![500, 510, 900]);
}
#[test]
fn write_then_read_round_trip() {
// Mux a small A/V title to an in-memory MP4, then demux it back and
// check the streams, codec_private, and sample payloads survive.
use crate::disc::{
AudioChannels, AudioStream, Codec, DiscTitle, FrameRate, HdrFormat, LabelPurpose,
SampleRate, Stream as DiscStreamE, VideoStream,
};
use crate::mux::mp4::Mp4Sink;
use crate::pes::{PesFrame, Stream as _};
use std::io::Cursor;
let mut t = DiscTitle::empty();
t.streams = vec![
DiscStreamE::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R1080p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Sdr,
color_space: crate::disc::ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
}),
DiscStreamE::Audio(AudioStream {
pid: 0x1100,
codec: Codec::Ac3,
channels: AudioChannels::Surround51,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
}),
];
let hvcc = vec![0xAA, 0xBB, 0xCC, 0xDD, 0xEE];
t.codec_privates = vec![Some(hvcc.clone()), None];
let ac3 = vec![
0x0B,
0x77,
0x00,
0x00,
0b00_010110,
0b01000_000,
0b111_00_00_1,
0x00,
0xFF,
0xFF,
];
let d = 41_708_333i64;
let vid0 = vec![0x11u8; 700];
let vid1 = vec![0x22u8; 350];
let mut buf = Vec::new();
{
let mk = |track, pts, key, data: Vec<u8>| PesFrame {
track,
pts,
keyframe: key,
data,
duration_ns: None,
source: None,
coding: None,
};
let mut sink = Mp4Sink::create(Cursor::new(&mut buf), &t).unwrap();
sink.write(&mk(0, 0, true, vid0.clone())).unwrap();
sink.write(&mk(1, 0, true, ac3.clone())).unwrap();
sink.write(&mk(0, d, false, vid1.clone())).unwrap();
sink.write(&mk(1, 32_000_000, true, ac3.clone())).unwrap();
sink.finish().unwrap();
}
let mut rd = Mp4Reader::from_reader(Cursor::new(buf), "rt".into()).unwrap();
// Two streams: HEVC video + AC-3 audio, and the hvcC round-trips.
assert_eq!(rd.info().streams.len(), 2);
assert!(
matches!(rd.info().streams[0], DiscStreamE::Video(ref v) if v.codec == Codec::Hevc)
);
assert!(matches!(rd.info().streams[1], DiscStreamE::Audio(ref a) if a.codec == Codec::Ac3));
assert_eq!(rd.codec_private(0), Some(hvcc));
// Read all frames back; match them to what we wrote by (track, size).
let mut got = Vec::new();
while let Some(f) = rd.read().unwrap() {
got.push((f.track, f.data.len(), f.keyframe));
}
assert_eq!(got.len(), 4, "4 samples round-trip");
let vids: Vec<_> = got.iter().filter(|(t, _, _)| *t == 0).collect();
let auds: Vec<_> = got.iter().filter(|(t, _, _)| *t == 1).collect();
assert_eq!(vids.len(), 2);
assert_eq!(auds.len(), 2);
assert_eq!(vids[0].1, 700, "first video sample size");
assert_eq!(vids[1].1, 350, "second video sample size");
assert!(vids[0].2, "first video frame is a keyframe");
assert_eq!(auds[0].1, ac3.len());
}
#[test]
fn stts_and_ctts_expand() {
// stts: 3 samples × 1001 ticks.
let mut stts = Vec::new();
stts.extend_from_slice(&[0, 0, 0, 0]); // version+flags
stts.extend_from_slice(&1u32.to_be_bytes()); // entry_count
stts.extend_from_slice(&3u32.to_be_bytes());
stts.extend_from_slice(&1001u32.to_be_bytes());
assert_eq!(parse_stts(&stts), vec![1001, 1001, 1001]);
}
}
+3 -34
View File
@@ -56,14 +56,6 @@ pub struct PipelinedPesStream {
/// `std::env::var_os` takes a process-wide lock, so the per-batch /
/// per-poll reads it replaces were needless hot-path overhead.
skip_parse: bool,
/// Cumulative bytes of scrambled AACS units the producer's decrypt step
/// could not decrypt — silent decrypt loss the demux drops without a sync.
/// Shared with the producer thread's [`DecryptingSectorSource`]
/// (`crate::sector::DecryptingSectorSource::decrypt_loss`). Surfaced through
/// [`Stream::lost_bytes`] so the file-backed mux abort gate sees a partial
/// decrypt failure instead of reporting a perfect rip. `None` for pipelines
/// with no AACS decrypt step (e.g. the M2TS byte-stream path).
decrypt_loss: Option<std::sync::Arc<std::sync::atomic::AtomicU64>>,
/// Count of dropped DVD navigation packets (private_stream_2, 0xBF). These
/// are expected on every disc; instead of a per-packet WARN they're tallied
/// and summarised once at EOF.
@@ -134,7 +126,6 @@ impl PipelinedPesStream {
pending_frames: std::collections::VecDeque::new(),
eof: false,
skip_parse: std::env::var_os("FREEMKV_SKIP_PARSE").is_some(),
decrypt_loss: None,
dropped_nav_packets: 0,
resync,
is_video,
@@ -142,20 +133,6 @@ impl PipelinedPesStream {
}
}
/// Attach the producer's decrypt-loss counter so [`Stream::lost_bytes`]
/// reports bytes of scrambled AACS units that could not be decrypted (and
/// were therefore silently dropped downstream). Obtained from the
/// producer's `DecryptingSectorSource::decrypt_loss()` before it is moved
/// into the prefetch thread. The M2TS / no-decrypt pipelines leave this
/// unset.
pub(crate) fn with_decrypt_loss(
mut self,
loss: std::sync::Arc<std::sync::atomic::AtomicU64>,
) -> Self {
self.decrypt_loss = Some(loss);
self
}
/// Pull one batch of `PesPacket`s from the demux thread, run
/// codec parse on each, enqueue resulting `PesFrame`s on
/// `pending_frames`. Returns Ok(true) on success, Ok(false) on
@@ -464,17 +441,9 @@ impl Stream for PipelinedPesStream {
.and_then(|(_, parser)| parser.codec_private())
}
fn lost_bytes(&self) -> u64 {
// The file-backed highway has no read-error zero-fill term (resolve
// tracks read loss separately), but the producer's decrypt step can
// pass scrambled units through undecrypted — silent loss the demux
// drops. Surface that so the mux abort gate sees a partial AACS/CSS
// decrypt failure rather than reporting a perfect rip.
self.decrypt_loss
.as_ref()
.map(|c| c.load(std::sync::atomic::Ordering::Relaxed))
.unwrap_or(0)
}
// `lost_bytes` uses the trait default (0): the file-backed highway has no
// read-error zero-fill term (resolve/mapfile tracks physical read loss
// separately) and the decrypt path no longer reports a decrypt-loss term.
}
#[cfg(test)]
+674 -70
View File
@@ -45,6 +45,10 @@ pub enum StreamUrl {
M2ts { path: PathBuf },
/// Matroska container file.
Mkv { path: PathBuf },
/// Progressive MP4 (ISO-BMFF) mux output (`mp4://`). Like `mkv://` but writes
/// a single self-contained `.mp4` (ftyp+mdat+moov). Compatibility export —
/// carries only MP4-mappable codecs; see `mux::mp4`.
Mp4 { path: PathBuf },
/// Network stream (host:port).
Network { addr: String },
/// Standard I/O (stdin/stdout).
@@ -62,10 +66,31 @@ pub enum StreamUrl {
/// chapters + delay metadata). Like `dir://` it targets a directory; the
/// CLI constructs the `DemuxSink` with full options before the mux loop.
Demux { dir: PathBuf },
/// Video-only per-track output directory (`video://`) — a `demux://`
/// restricted to video tracks (native elementary streams: `.hevc`, `.h264`,
/// `.vc1`, `.m2v`, …). One file per video track; no audio/subtitles.
Video { dir: PathBuf },
/// Audio-only per-track output directory (`audio://`) — a `demux://`
/// restricted to audio tracks (native containers: `.thd`, `.dts`, `.ac3`,
/// `.eac3`, `.pcm`, …). One file per audio track; no video/subtitles.
Audio { dir: PathBuf },
/// Subtitle-only per-track output directory (`sub://`) — a `demux://`
/// restricted to subtitle tracks (PGS `.sup`, VobSub `.idx`+`.sub`, text
/// `.srt`). One file per subtitle track.
Sub { dir: PathBuf },
/// freemkv native per-picture video index (`fvi://`). A write-only PES sink
/// that emits one JSON-Lines record per coded picture of the title's primary
/// video track to a `.fvi` file (normative spec `docs/FVI_FORMAT.md`).
Fvi { path: PathBuf },
/// Chapter-marker export (`chapters://`). A write-only sink that ignores the
/// PES stream and writes the title's chapter points to a single file, format
/// chosen by the output extension: `.xml` (Matroska, default), `.txt` (OGM),
/// `.vtt` (WebVTT).
Chapters { path: PathBuf },
/// Structured title/stream/chapter metadata (`json://`). A write-only sink
/// that ignores the PES stream and writes the selected title's model as one
/// JSON document — machine-readable `info` for one title.
Json { path: PathBuf },
/// Unrecognized URL.
Unknown { raw: String },
}
@@ -77,13 +102,19 @@ impl StreamUrl {
StreamUrl::Disc { .. } => "disc",
StreamUrl::M2ts { .. } => "m2ts",
StreamUrl::Mkv { .. } => "mkv",
StreamUrl::Mp4 { .. } => "mp4",
StreamUrl::Network { .. } => "network",
StreamUrl::Stdio => "stdio",
StreamUrl::Iso { .. } => "iso",
StreamUrl::Dir { .. } => "dir",
StreamUrl::Null => "null",
StreamUrl::Demux { .. } => "demux",
StreamUrl::Video { .. } => "video",
StreamUrl::Audio { .. } => "audio",
StreamUrl::Sub { .. } => "sub",
StreamUrl::Fvi { .. } => "fvi",
StreamUrl::Chapters { .. } => "chapters",
StreamUrl::Json { .. } => "json",
StreamUrl::Unknown { .. } => "unknown",
}
}
@@ -95,10 +126,16 @@ impl StreamUrl {
StreamUrl::Disc { device: None } => "",
StreamUrl::M2ts { path }
| StreamUrl::Mkv { path }
| StreamUrl::Mp4 { path }
| StreamUrl::Iso { path }
| StreamUrl::Dir { path }
| StreamUrl::Demux { dir: path }
| StreamUrl::Fvi { path } => path.to_str().unwrap_or(""),
| StreamUrl::Video { dir: path }
| StreamUrl::Audio { dir: path }
| StreamUrl::Sub { dir: path }
| StreamUrl::Fvi { path }
| StreamUrl::Chapters { path }
| StreamUrl::Json { path } => path.to_str().unwrap_or(""),
StreamUrl::Network { addr } => addr,
StreamUrl::Stdio | StreamUrl::Null => "",
StreamUrl::Unknown { raw } => raw,
@@ -138,6 +175,11 @@ pub fn parse_url(url: &str) -> StreamUrl {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("mp4://") {
return StreamUrl::Mp4 {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("network://") {
return StreamUrl::Network {
addr: rest.to_string(),
@@ -171,6 +213,31 @@ pub fn parse_url(url: &str) -> StreamUrl {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("video://") {
return StreamUrl::Video {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("audio://") {
return StreamUrl::Audio {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("sub://") {
return StreamUrl::Sub {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("chapters://") {
return StreamUrl::Chapters {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("json://") {
return StreamUrl::Json {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("fvi://") {
return StreamUrl::Fvi {
path: PathBuf::from(rest),
@@ -336,34 +403,34 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
}
.into());
}
// Per-title key resolution. For a multi-VTS CSS DVD the scan's
// single cracked key only descrambles its own VTS; re-crack from
// the chosen title's extents if it lives elsewhere. A fresh reader
// avoids disturbing the mux reader below. 64 sectors is a
// file-safe batch for an ISO. AACS / single-VTS paths are
// unchanged (decrypt_keys_for_title short-circuits to decrypt_keys).
let (keys, title_is_clear) =
match crate::io::file_sector_source::FileSectorSource::open(path) {
Ok(mut crack_reader) => {
disc.decrypt_keys_for_title_checked(idx, &mut crack_reader, 64)
}
Err(_) => (disc.decrypt_keys(), false),
};
// Per-title decrypt gate (parallel to the disc-wide gate above): on
// a multi-VTS CSS disc, the per-title re-crack may return `None` when
// the chosen title's VTS could not be re-cracked. Muxing that would
// emit scrambled ciphertext verbatim, so fail loudly here — EXCEPT
// when the title proved genuinely clear (`title_is_clear`), an
// unencrypted stub on an otherwise-CSS disc that needs no key. That
// case must NOT raise a false E7023.
disc.ensure_title_decryptable(opts.raw, &keys, title_is_clear)
.map_err(|e| -> io::Error { e.into() })?;
// Upfront FMTS gate, parallel to the unit-key gate above. With
// BYPASS_FMTS_KEY this is a no-op and a 2.1 disc's forensic units are
// concealed as ordinary decrypt loss below; without it, a 2.1 disc
// lacking segment keys fails here rather than emitting a holed mux.
disc.ensure_forensic_segments_decryptable(opts.raw)
.map_err(|e| -> io::Error { e.into() })?;
// Per-title key resolution. DVD CSS is resolved at exactly ONE site —
// `build_iso_pipeline`'s per-title crack (below), which decrypts a
// crackable title, passes a genuinely-clear one through, and
// hard-fails an uncrackable one with CssKeyMissing. So for a DVD we do
// NOT pre-crack here: pass `None` and let the pipeline own it.
// Pre-cracking would re-open the ISO and re-scan every clear title
// (`decrypt_keys_for_title` → None → the pipeline re-cracks anyway).
// AACS / unencrypted resolve from `decrypt_keys()` with NO read; `--raw`
// (any format) is deliberate ciphertext passthrough — also `None`.
let is_dvd = disc.format == crate::disc::DiscFormat::Dvd;
let (keys, title_is_clear) = if opts.raw || is_dvd {
(crate::decrypt::DecryptKeys::None, false)
} else {
(disc.decrypt_keys(), false)
};
// Decrypt gate for the AACS / non-DVD path: a None key means no usable
// disc key, which would mux scrambled ciphertext verbatim — fail loudly
// (NoDiscKey). The DVD path is gated inside `build_iso_pipeline` (its
// CSS hard-fail), and `--raw` passes.
if !is_dvd {
disc.ensure_title_decryptable(opts.raw, &keys, title_is_clear)
.map_err(|e| -> io::Error { e.into() })?;
}
// FMTS (AACS 2.1) forensic segments are sourced + fail-loud-checked
// downstream by `resolve_mux_key_map`/`resolve_fmts_key_map`, which hold
// the key-fetch closure and can actually attempt resolution. (An older
// upfront blanket-reject gate lived here; it predated the resolver and
// rejected every 2.1 disc before a source could be tried.)
// Correct TrueHD channel counts (MPLS understates 7.1/Atmos as 5.1)
// by probing the first DECRYPTED access units of the chosen title.
// A fresh reader avoids disturbing the mux reader below. Skipped in
@@ -391,11 +458,11 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
let title = disc.titles[idx].clone();
let format = disc.content_format;
// ISO file: 8192-sector batch (16 MiB at 2048 B/sector) —
// sequential read from fast storage, no bad sectors. Measured
// optimum on the rip1 testbed; bumping to 16384 sectors (32 MiB)
// regressed (more cache pressure, longer per-batch latency starves
// the consumer between iterations). Physical drives keep smaller
// batches for adaptive error handling.
// sequential read from fast storage, no bad sectors. Empirically
// optimal; bumping to 16384 sectors (32 MiB) regressed (more cache
// pressure, longer per-batch latency starves the consumer between
// iterations). Physical drives keep smaller batches for adaptive
// error handling.
const ISO_MUX_BATCH_SECTORS: u16 = 8192;
// Pass `DecryptKeys::None` to the decrypt decorator when
@@ -421,6 +488,7 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
effective_keys,
ISO_MUX_BATCH_SECTORS,
format,
opts.raw,
None,
None,
fetch,
@@ -449,8 +517,16 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
// PES source. Mirror `null://` → write-only.
StreamUrl::Dir { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Null => Err(crate::error::Error::StreamWriteOnly.into()),
// `mp4://` as a source: demux a progressive MP4 back into PES frames, so
// `mp4://` flows to every sink (mkv://, audio://, json://, …).
StreamUrl::Mp4 { ref path } => Ok(Box::new(super::mp4::Mp4Reader::open(path)?)),
// `demux://` is an output-only sink (per-track ES files); never a source.
StreamUrl::Demux { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Demux { .. }
| StreamUrl::Video { .. }
| StreamUrl::Audio { .. }
| StreamUrl::Sub { .. }
| StreamUrl::Chapters { .. }
| StreamUrl::Json { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
// `fvi://` is an output-only sink (per-picture video index); never a source.
StreamUrl::Fvi { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Unknown { ref raw } => {
@@ -480,7 +556,19 @@ pub fn output(
IO_BUF_SIZE,
crate::io::WritebackFile::create_with_size_hint(path, title.size_bytes)?,
));
Ok(Box::new(MkvStream::create_at(writer, title, Some(path))?))
Ok(Box::new(MkvStream::create(writer, title, Some(path))?))
}
StreamUrl::Mp4 { ref path } => {
validate_file_path(path, "mp4")?;
// Bounded-cache writeback (like mkv://) so a UHD-scale mux to slow /
// network-attached staging doesn't hit the dirty-page burst
// pathology; the mdat backpatch is an ordinary seek WritebackFile
// handles. BufWriter coalesces the many small moov box-header writes.
let writer = std::io::BufWriter::with_capacity(
IO_BUF_SIZE,
crate::io::WritebackFile::create_with_size_hint(path, title.size_bytes)?,
);
Ok(Box::new(super::mp4::Mp4Sink::create(writer, title)?))
}
StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?;
@@ -529,11 +617,35 @@ pub fn output(
dir, title, &opts,
)?))
}
// `video://`, `audio://`, and `sub://` are `demux://` restricted to one
// track class — video as native elementary streams, audio in native
// containers, or subtitles as `.sup`/`.idx+.sub`/`.srt`. No chapters
// sidecar (that's a `demux://` / `chapters://` concern).
StreamUrl::Video { ref dir }
| StreamUrl::Audio { ref dir }
| StreamUrl::Sub { ref dir } => {
let (scheme, kind) = match parsed {
StreamUrl::Video { .. } => ("video", super::demux_sink::TrackKind::Video),
StreamUrl::Audio { .. } => ("audio", super::demux_sink::TrackKind::Audio),
_ => ("sub", super::demux_sink::TrackKind::Subtitle),
};
validate_file_path(dir, scheme)?;
let mut opts = super::demux_sink::DemuxOptions {
kind_filter: Some(kind),
export_chapters: false,
..Default::default()
};
if !title.playlist.is_empty() {
opts.base = title.playlist.clone();
}
Ok(Box::new(super::demux_sink::DemuxSink::create(
dir, title, &opts,
)?))
}
// `fvi://` writes the per-picture video index (`docs/FVI_FORMAT.md`).
// The bare `output()` arm records the resolver path as the provenance
// `source.path` and defaults the title index to 0 (the resolver carries
// no title-index context — the CLI follow-up passes the real medium /
// title via `FviSink::create_with_source`).
// no title-index context).
StreamUrl::Fvi { ref path } => {
validate_file_path(path, "fvi")?;
Ok(Box::new(super::fvi_sink::FviSink::create(
@@ -543,6 +655,18 @@ pub fn output(
0,
)?))
}
// `chapters://` and `json://` write the title metadata at construction and
// ignore the PES stream (see `meta_sink`).
StreamUrl::Chapters { ref path } => {
validate_file_path(path, "chapters")?;
Ok(Box::new(super::meta_sink::ChaptersSink::create(
path, title,
)?))
}
StreamUrl::Json { ref path } => {
validate_file_path(path, "json")?;
Ok(Box::new(super::meta_sink::JsonSink::create(path, title)?))
}
StreamUrl::Unknown { ref raw } => {
Err(crate::error::Error::StreamUrlInvalid { url: raw.clone() }.into())
}
@@ -575,7 +699,17 @@ fn build_demux_state(title: &DiscTitle, format: ContentFormat) -> DemuxState {
pids.push(pid);
pid_to_track.push((pid, idx));
let is_dvd_ps = matches!(format, ContentFormat::MpegPs);
parsers.push((pid, super::codec::parser_for_codec(codec, None, is_dvd_ps)));
// The Blu-ray 3D MVC dependent (right-eye) view uses a param-set-
// passthrough H.264 parser so each frame is a self-contained dependent
// access unit for a BlockAdditional; every other stream uses the
// ordinary parser for its codec.
let parser = match s {
crate::disc::Stream::Video(v) if v.is_mvc_dependent() => {
super::codec::parser_for_mvc_dependent(codec, is_dvd_ps)
}
_ => super::codec::parser_for_codec(codec, None, is_dvd_ps),
};
parsers.push((pid, parser));
}
let (ts, ps) = match format {
ContentFormat::MpegPs => (None, Some(super::ps::PsDemuxer::new())),
@@ -590,6 +724,391 @@ fn build_demux_state(title: &DiscTitle, format: ContentFormat) -> DemuxState {
(parsers, pid_to_track, ts, ps)
}
/// Resolve the proactive [`AacsKeyMap`](crate::decrypt::AacsKeyMap) for a title
/// before muxing. It decides which held unit key decrypts each of the title's
/// LBA ranges and secures any key the pool is missing through the app's
/// configured source (`fetch`) up front, never reactively per unit at mux time.
///
/// This is what ends the key-server storm. The old mux decrypted a unit, checked
/// whether the plaintext looked like clean MPEG-TS, and — because authored-bad
/// content never reaches that bar — re-asked the key service for a key it already
/// held. There is no per-unit byte pattern that separates "correctly decrypted
/// but authored-bad" from "still encrypted", so that check is unanswerable. Here
/// we answer the answerable question instead: which CPS unit does each LBA range
/// belong to, decided by the disc's key structure (validated once against real
/// ciphertext samples, where the `is_clean` proof IS sound). The mux then just
/// decrypts each unit with its mapped key and trusts it.
///
/// Single-CPS (the overwhelming majority, incl. every single-key UHD) is the
/// trivial map: one key everywhere, no sampling. Multi-CPS assigns each extent to
/// the key that opens a real sample from it; a bad-content extent no sample can
/// classify inherits its predecessor's key (contiguity). FMTS segment mapping
/// layers onto the same structure.
/// FMTS (AACS 2.1) branch of [`resolve_mux_key_map`]. Returns `Some(map)` when the
/// disc carries `IndividualSegment.tbl` AND a key source is configured; `None`
/// otherwise (not FMTS, or no source — the caller's base-Unit-Key path then
/// applies, and the forensic units garble and are dropped by the demux).
///
/// The forensic segments each carry an **index** tag (1..32) selecting one of 32
/// **index keys** the base Unit Key cannot open (see [`crate::aacs::segment`]).
/// This resolves those keys up front from the configured source — sending, per
/// index, a batch of same-index units the service maps to that index's key — adds
/// them to the pool, and builds a per-segment LBA→key map. Applying a segment's
/// key over its whole range decodes the ~40 units of that index's interleave half
/// to clean TS and garbles the other ~40 (the alternate half), which the demux
/// then drops, yielding one coherent stream. The base Unit Key covers everything
/// outside a segment.
fn resolve_fmts_key_map(
reader: &mut dyn SectorSource,
title: &DiscTitle,
keys: &mut crate::decrypt::DecryptKeys,
fetch: Option<&crate::sector::KeyFetch>,
format: ContentFormat,
) -> io::Result<Option<crate::decrypt::AacsKeyMap>> {
use crate::aacs::content::{ALIGNED_UNIT_LEN, aacs_unit_encrypted, decrypt_unit, is_clean};
use crate::aacs::segment::{clip_byte_to_lba, parse_individual_segments};
// Load the segment map; absent → not an FMTS disc.
let Ok(udf) = crate::udf::read_filesystem(reader) else {
return Ok(None);
};
let Ok(tbl) = udf.read_file(reader, "/AACS/IndividualSegment.tbl") else {
return Ok(None);
};
let Some(segments) = parse_individual_segments(&tbl) else {
return Ok(None);
};
if segments.is_empty() {
return Ok(None);
}
// This IS an FMTS disc, so the forensic index keys are REQUIRED — exactly like
// a Unit Key. Without a configured key source we cannot obtain them, so we
// cannot produce a complete rip: fail loud rather than silently drop the
// forensic segments. (The caller may still choose `--raw`, which never reaches
// this path.)
let Some(fetch) = fetch else {
return Err(crate::error::Error::FmtsKeyMissing.into());
};
tracing::info!(target: "freemkv::keysource", segments = segments.len(), extents = title.extents.len(), "fmts: begin index-key resolution");
// Read aligned unit `index` of `seg`: clip byte `start_spn*192 + index*6144`.
let read_unit =
|reader: &mut dyn SectorSource, seg: &crate::aacs::segment::Segment, index: usize| {
let clip_byte = seg.start_spn as u64 * 192 + index as u64 * ALIGNED_UNIT_LEN as u64;
let lba = clip_byte_to_lba(&title.extents, clip_byte)?;
let mut c = vec![0u8; ALIGNED_UNIT_LEN];
reader.read_sectors(lba, 3, &mut c, false).ok()?;
Some(c)
};
// ── ANCHOR — fetch the whole 32-key set from ONE index-1 batch. The key
// service returns ALL forensic index keys ordered (element i = index i+1)
// only for a canonical INDEX-1 sample that decrypts under the index-1 key.
// A forensic segment interleaves TWO variants at the aligned-unit level, so
// index-1's real content is one PHASE (even or odd units) and the alternate
// is a different variant that won't decrypt. We don't know the phase a
// priori, so try PHASE A (even) then PHASE B (odd): whichever is index-1's
// content comes back with the full set. Both phases failing (across the
// read-fault fallback over index-1 segments) ⇒ this disc has no FMTS keys.
//
// The set's SIZE is whatever the source returns (≥ 1) — never assumed. 32
// is all we have seen, but a disc with a different forensic index count is
// not ruled out, so the map is sized to the returned `len()`, not a const.
// ─────────────────────────────────────────────────────────────────────────
// Batch size = the key service's minimum-samples floor (same as the online
// source), drawn from ONE phase to land a clean single-variant half.
const BATCH_UNITS: usize = crate::keysource::MIN_SAMPLE_UNITS;
// Read-fault fallback: how many index-1 segments to attempt if the leading one
// is unreadable. The 2 phase requests happen per readable segment.
const MAX_ANCHOR_ATTEMPTS: usize = 16;
// Even units = p*2; odd units = p*2 + 1.
let read_phase_batch = |reader: &mut dyn SectorSource,
seg: &crate::aacs::segment::Segment,
phase_off: usize|
-> Option<Vec<Vec<u8>>> {
let mut batch: Vec<Vec<u8>> = Vec::with_capacity(BATCH_UNITS);
for p in 0..BATCH_UNITS {
batch.push(read_unit(reader, seg, p * 2 + phase_off)?);
}
Some(batch)
};
let mut index_keys: Vec<[u8; 16]> = Vec::new();
'anchor: for seg in segments
.iter()
.filter(|s| s.index == 1)
.take(MAX_ANCHOR_ATTEMPTS)
{
for phase_off in [0usize, 1usize] {
let Some(batch) = read_phase_batch(reader, seg, phase_off) else {
continue; // read fault on this phase — try the other / next segment
};
let fresh = fetch.fmts_indexes(&batch);
// Any non-empty reply is the source's COMPLETE ordered forensic set;
// trust it and stop. An empty reply = this phase/segment did not anchor.
if !fresh.is_empty() {
index_keys = fresh;
break 'anchor;
}
}
}
// The count is whatever the source returned — not a fixed 32. Sized here, used
// everywhere below.
let n_index = index_keys.len();
tracing::info!(target: "freemkv::keysource", held = n_index, "fmts: collection done");
// At least one forensic index key is required. None ⇒ no FMTS key for this
// disc from any source — fail loud like a missing Unit Key rather than emit
// forensic-holed output.
if index_keys.is_empty() {
return Err(crate::error::Error::FmtsKeyMissing.into());
}
// Map array position → forensic index (element i = index i+1); add each key to
// the pool and remember its slot by tag. `base_idx` is the Unit Key (slot 0).
let base_idx = 0usize;
let mut tag_slot: std::collections::HashMap<u16, usize> = std::collections::HashMap::new();
if let crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } = keys {
for (i, k) in index_keys.iter().enumerate() {
let tag = (i + 1) as u16;
let slot = match unit_keys.iter().position(|(_, h)| h == k) {
Some(s) => s,
None => {
let s = unit_keys.len();
// CPS-unit id is cosmetic for the mapped decrypt (it indexes by
// slot); use a high, distinct number for the forensic keys.
unit_keys.push((1000 + s as u32, *k));
s
}
};
tag_slot.insert(tag, slot);
}
}
// ── PROBE each index's phase. A forensic segment interleaves two variants at
// the aligned-unit level; only ONE parity is this index's real content (the
// other is the alternate variant — a different key, garbles under ours). For
// each index, read a representative tagged segment and count clean decrypts
// of its EVEN vs ODD units under that index's key: the clean half is the
// index's phase. This is the ONE place `is_clean` runs — "the map must be
// right", verified here, once — so the mux decrypt can then trust the map.
// Phase is per-index and shared by every segment carrying that index. ──────
let mut phase_of_index: std::collections::HashMap<u16, crate::decrypt::Phase> =
std::collections::HashMap::new();
for (i, k) in index_keys.iter().enumerate() {
let tag = (i + 1) as u16;
let Some(seg) = segments.iter().find(|s| s.index == tag) else {
continue; // no segment carries this index on this feature — skip
};
let (mut even, mut odd) = (0usize, 0usize);
for p in 0..BATCH_UNITS {
for (phase_off, counter) in [(0usize, &mut even), (1usize, &mut odd)] {
if let Some(mut c) = read_unit(reader, seg, p * 2 + phase_off) {
if aacs_unit_encrypted(&c, format) {
decrypt_unit(&mut c, k);
if is_clean(&c, format) {
*counter += 1;
}
}
}
}
}
let phase = match even.cmp(&odd) {
std::cmp::Ordering::Greater => crate::decrypt::Phase::Even,
std::cmp::Ordering::Less => crate::decrypt::Phase::Odd,
std::cmp::Ordering::Equal => {
// Neither half decrypts clean under this index's key: the map would
// be wrong. Fail loud rather than emit a broken segment map.
tracing::warn!(target: "freemkv::keysource", index = tag, even, odd, "fmts: no clean phase under index key — refusing broken map");
return Err(crate::error::Error::FmtsKeyMissing.into());
}
};
phase_of_index.insert(tag, phase);
}
// ── Build the per-segment LBA ranges: each forensic segment → its tag's key
// AND its index's phase. The mapped decrypt opens only that half and leaves
// the alternate as ciphertext (the muxer drops untouched ciphertext) —
// clean by construction, no garble. A segment straddling an extent boundary
// is left unmapped and tallied (a hard failure below). ────────────────────
let mut ranges: Vec<(u32, u32, usize, crate::decrypt::Phase)> =
Vec::with_capacity(segments.len());
let mut unresolved = 0usize;
for seg in &segments {
let Some(&slot) = tag_slot.get(&seg.index) else {
unresolved += 1;
continue;
};
let phase = phase_of_index
.get(&seg.index)
.copied()
.unwrap_or(crate::decrypt::Phase::All);
let start_byte = seg.start_spn as u64 * 192;
let end_byte = (seg.end_spn as u64 + 1) * 192;
let (Some(a), Some(b)) = (
clip_byte_to_lba(&title.extents, start_byte),
clip_byte_to_lba(&title.extents, end_byte - 1),
) else {
unresolved += 1;
continue;
};
// Only emit a contiguous within-extent range (segments are ~480 KB; a rare
// extent-straddle is left unresolved rather than given a wrong span).
if b >= a && (b - a) as u64 == (end_byte - 1 - start_byte) / 2048 {
ranges.push((a, b + 1, slot, phase));
} else {
unresolved += 1;
}
}
// Every forensic segment must map to an index key. Any that did not is a hole
// in the rip — with the full 32-key set in hand this should never happen, so
// treat it as a hard failure rather than silently emitting a garbled segment.
if unresolved != 0 {
return Err(crate::error::Error::FmtsKeyMissing.into());
}
Ok(Some(crate::decrypt::AacsKeyMap::from_ranges_phased(
ranges, base_idx,
)))
}
pub fn resolve_mux_key_map(
reader: &mut dyn SectorSource,
title: &DiscTitle,
keys: &mut crate::decrypt::DecryptKeys,
fetch: Option<&crate::sector::KeyFetch>,
format: ContentFormat,
) -> io::Result<crate::decrypt::AacsKeyMap> {
use crate::aacs::content::{
ALIGNED_UNIT_LEN, ALIGNED_UNIT_SECTORS, aacs_unit_encrypted, decrypt_unit, is_clean,
};
let pool_len = match keys {
crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } => unit_keys.len(),
// CSS / clear: no AACS map (the decorator's map path is AACS-only).
_ => return Ok(crate::decrypt::AacsKeyMap::single(0)),
};
// Secure the disc's key up front from the configured source when the pool is
// empty (a genuine "no key yet" — e.g. keydb miss, online-only disc).
if pool_len == 0 {
if let Some(f) = fetch {
let samples = crate::keysource::read_encrypted_units(reader, title, 8);
if !samples.is_empty() {
let fresh = f.unit_keys(&samples);
if let crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } = keys {
for k in fresh {
if !unit_keys.iter().any(|(_, h)| *h == k) {
let i = unit_keys.len() as u32;
unit_keys.push((i, k));
}
}
}
}
}
// If the pool is STILL empty, this AACS-encrypted title needs a Unit Key we
// could not obtain from any source. That is the same situation as any known
// key we don't hold — fail loud at resolve time rather than deferring an
// opaque decrypt error (or, worse, emitting ciphertext) at mux time.
let empty = matches!(
keys,
crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } if unit_keys.is_empty()
);
if empty {
return Err(crate::error::Error::DecryptFailed.into());
}
return Ok(crate::decrypt::AacsKeyMap::single(0));
}
// FMTS (AACS 2.1): if the disc carries `IndividualSegment.tbl`, the forensic
// segments need per-index keys the base Unit Key can't open. Resolve them up
// front from the configured source and build a per-segment map. Returns `None`
// when the disc is not FMTS, or no key source is configured (then the base UK
// path below applies and the forensic units garble → demux drops them).
if let Some(map) = resolve_fmts_key_map(reader, title, keys, fetch, format)? {
return Ok(map);
}
if pool_len == 1 {
// One CPS unit → one key everywhere. No structural walk, no sampling.
return Ok(crate::decrypt::AacsKeyMap::single(0));
}
// Multi-CPS: read a spread of real encrypted units from each extent and pick
// the held key that opens one (the `is_clean` proof is sound HERE — samples
// are guaranteed real content, not the authored-bad units that trip the mux).
let sample_units = |reader: &mut dyn SectorSource, start: u32, sectors: u32| -> Vec<Vec<u8>> {
let total_units = sectors / ALIGNED_UNIT_SECTORS;
let mut out = Vec::new();
if total_units == 0 {
return out;
}
const PROBES: u32 = 8;
for p in 1..=PROBES {
let unit = ((total_units as u64 * p as u64) / (PROBES as u64 + 1)) as u32;
if unit >= total_units {
continue;
}
let lba = start.saturating_add(unit.saturating_mul(ALIGNED_UNIT_SECTORS));
let mut buf = vec![0u8; ALIGNED_UNIT_LEN];
if reader
.read_sectors(lba, ALIGNED_UNIT_SECTORS as u16, &mut buf, false)
.is_ok()
&& aacs_unit_encrypted(&buf, format)
{
out.push(buf);
}
}
out
};
let pick = |samples: &[Vec<u8>], pool: &[(u32, [u8; 16])]| -> Option<usize> {
for (i, (_, k)) in pool.iter().enumerate() {
if samples.iter().any(|s| {
let mut u = s.clone();
decrypt_unit(&mut u, k);
is_clean(&u, format)
}) {
return Some(i);
}
}
None
};
let mut ranges: Vec<(u32, u32, usize)> = Vec::with_capacity(title.extents.len());
let mut last_idx = 0usize;
for ext in &title.extents {
let samples = sample_units(reader, ext.start_lba, ext.sector_count);
// Snapshot the current pool for the pure `pick` closure.
let pool: Vec<(u32, [u8; 16])> = match keys {
crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } => unit_keys.clone(),
_ => Vec::new(),
};
let mut idx = pick(&samples, &pool);
if idx.is_none() {
if let Some(f) = fetch {
if !samples.is_empty() {
let fresh = f.unit_keys(&samples);
if let crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } = keys {
for k in fresh {
if !unit_keys.iter().any(|(_, h)| *h == k) {
let i = unit_keys.len() as u32;
unit_keys.push((i, k));
}
}
idx = pick(&samples, unit_keys);
}
}
}
}
// A bad-content extent no sample can classify inherits its predecessor's
// key (CPS boundaries are contiguous, so the neighbour is almost always
// right); this never storms and never mis-fails a decryptable disc.
let idx = idx.unwrap_or(last_idx);
last_idx = idx;
ranges.push((
ext.start_lba,
ext.start_lba.saturating_add(ext.sector_count),
idx,
));
}
Ok(crate::decrypt::AacsKeyMap::from_ranges(ranges, 0))
}
/// Assemble the ISO mux pipeline (read+decrypt → demux → parse) for
/// a `FileSectorSource`-backed reader. Returns the resulting
/// `PipelinedPesStream`.
@@ -605,27 +1124,46 @@ fn build_demux_state(title: &DiscTitle, format: ContentFormat) -> DemuxState {
/// - `batch_sectors`: read batch size in logical (2048-byte) sectors — a
/// throughput/latency tuning knob, not a correctness parameter.
/// - `format`: container format (`BdTs` → TS demuxer, `MpegPs` → PS demuxer).
/// - `raw`: ciphertext passthrough. When `true`, the per-title CSS crack
/// (`resolve_dvd_title_key`) is skipped entirely — no key is resolved and a
/// scrambled title is neither descrambled nor hard-failed.
/// - `halt`: cooperative cancel token (not a timeout); when cancelled the
/// pipeline stops at the next boundary. `None` disables cancellation.
/// pipeline stops at the next boundary (and the CSS crack surfaces `Halted`).
/// `None` disables cancellation.
/// - `event_fn`: optional progress/event callback invoked by the prefetcher.
/// - `fetch`: optional fresh-key-on-failure callback (see
/// [`crate::sector::KeyFetch`]). When a unit no held key decrypts, the
/// decrypt decorator hands that ciphertext to `fetch` and adds any key it
/// returns. `None` keeps the prior behaviour (the unit is counted as loss).
// Eight reader/title/keys/tuning/callback params is inherent to the mux entry
/// - `fetch`: optional key source used UP FRONT by [`resolve_mux_key_map`] to
/// secure any CPS-unit key the pool is missing. Not a per-unit mux-time
/// callback: the map decides the key for every LBA before the read loop starts.
// Nine reader/title/keys/tuning/callback params is inherent to the mux entry
// point; grouping them into a struct would only move the same fields around.
#[allow(clippy::too_many_arguments)]
pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
reader: S,
mut reader: S,
title: DiscTitle,
keys: crate::decrypt::DecryptKeys,
mut keys: crate::decrypt::DecryptKeys,
batch_sectors: u16,
format: ContentFormat,
raw: bool,
halt: Option<crate::halt::Halt>,
event_fn: Option<crate::sector::prefetched::EventFn>,
fetch: Option<crate::sector::KeyFetch>,
) -> io::Result<PipelinedPesStream> {
let extents = title.extents.clone();
// CSS (DVD) key resolution — the shared per-title step (also used by the
// live-drive single-pass `DiscStream`). A `None`/MPEG-PS title cracks its own
// key from the reader in playback order; AACS `.evo` (also MPEG-PS) arrives as
// `Aacs` and is untouched; a clear DVD stays `None`; `raw` skips it entirely.
// Without this a detection-miss CSS DVD would mux scrambled sectors as corrupt
// video. `halt` lets /api/stop interrupt the crack scan.
crate::css::resolve_dvd_title_key(
&mut reader,
&extents,
&mut keys,
batch_sectors,
format,
raw,
halt.as_ref(),
)?;
// Unit alignment is an AACS concept: AACS decrypts whole 6144-byte (3-sector)
// units, so the producer must hand the decrypt step 3-sector-aligned batches.
// CSS (DVD) and unencrypted content decrypt per 2048-byte sector — forcing
@@ -635,27 +1173,42 @@ pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
crate::decrypt::DecryptKeys::Aacs { .. } => 3,
_ => 1,
};
// MUX path: tolerate decrypt loss. An undecryptable content unit is concealed
// (NULL TS fill) + tallied + logged, never an abort — decrypt-verify is a RIP
// gate, not a mux gate (P3). The rip's own read paths keep their fail-loud
// decorator; only this mux pipeline opts in.
// MUX path: read > decrypt > mux. Resolve the proactive AACS key map UP FRONT
// — one key per CPS unit / segment, secured from the configured source and
// recorded against the LBA ranges it covers. The mux then decrypts each unit
// with its KNOWN key and trusts it: no per-unit `is_clean` verdict, no reactive
// key-fetch, no key-server storm. A unit that decrypts to broken TS is the
// muxer's problem, exactly as before. AACS-only; CSS self-cracks per region.
let key_map =
match &keys {
crate::decrypt::DecryptKeys::Aacs { .. } => Some(std::sync::Arc::new(
resolve_mux_key_map(&mut reader, &title, &mut keys, fetch.as_ref(), format)?,
)),
_ => None,
};
// The map IS the title's read plan: it says which CPS unit / forensic segment
// each LBA belongs to. Walk ONLY the units it marks as ours — every default /
// CPS unit, and inside an FMTS forensic segment only our-phase units. The
// alternate-phase units are a different device group's variant; a licensed
// player never reads them, and neither do we — they are never fetched,
// decrypted, or handed to the demux, so the demux sees one gapless our-variant
// stream (no ciphertext to trip a concealed-gap resync). A non-forensic map
// returns the extents unchanged, so the common disc reads exactly as before.
let extents = match &key_map {
Some(map) => map.read_plan(&extents, unit_align as u32),
None => extents,
};
let mut decrypting =
crate::sector::DecryptingSectorSource::new(Box::new(reader) as Box<dyn SectorSource>, keys)
.tolerate_decrypt_loss();
// Install the fresh-key-on-failure callback (if any) so a unit no held key
// decrypts is re-tried via the application's key source before being counted
// as loss. An AACS 2.1 forensic-segment unit that no key opens is just an
// undecryptable unit like any other: concealed and counted as decrypt loss —
// a loss is a loss, no FMTS special casing.
if let Some(cb) = fetch {
decrypting = decrypting.with_key_fetch(cb);
crate::sector::DecryptingSectorSource::new(Box::new(reader) as Box<dyn SectorSource>, keys);
if let Some(map) = key_map {
decrypting = decrypting.with_key_map(map);
}
// Grab the loss counters before the decorator is moved into the producer
// thread. It tracks bytes of scrambled AACS units no key could decrypt —
// silent loss the demux drops; the consuming stream surfaces it through
// `lost_bytes()` so the mux abort gate sees a partial decrypt failure rather
// than a clean rip. Forensic (2.1) undecryptable units land here too.
let decrypt_loss = decrypting.decrypt_loss();
// Loss-counter handle. The mux does NOT tally decrypt-quality misses: a
// broken-TS unit is the muxer's concern, and a missing key is an up-front
// resolve failure — indistinguishable from bad authoring at this seam, so
// counting it would false-abort a bad-encoded-but-decryptable disc. A genuine
// can't-decrypt surfaces as `Err`; `lost_bytes()` reflects physical read loss
// only (there is no decrypt-loss term to fold in).
// Wrong-substream fix (Silence-of-the-Lambs): before the prefetcher takes
// the reader, probe the feature head through the (plaintext) decrypting
@@ -681,10 +1234,13 @@ pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
let (demux_thread, demux_rx) =
super::demux_thread::DemuxThread::spawn_zero_copy(rx, recycle_tx, shell, halt, ts, ps)
.map_err(|e| -> io::Error { e.into() })?;
Ok(
PipelinedPesStream::new(demux_thread, demux_rx, title, parsers, pid_to_track)
.with_decrypt_loss(decrypt_loss),
)
Ok(PipelinedPesStream::new(
demux_thread,
demux_rx,
title,
parsers,
pid_to_track,
))
}
/// Assemble the M2TS file mux pipeline (read → demux → parse) for a
@@ -1170,6 +1726,7 @@ mod tests {
DecryptKeys::None,
8192,
ContentFormat::BdTs,
false,
None,
None,
None,
@@ -1211,6 +1768,7 @@ mod tests {
DecryptKeys::None,
8192,
ContentFormat::BdTs,
false,
None,
None,
None,
@@ -1263,10 +1821,56 @@ mod tests {
DecryptKeys::None,
0,
ContentFormat::BdTs,
false,
None,
None,
None,
);
assert!(res.is_err(), "zero batch_sectors must be rejected");
}
/// REGRESSION (autorip production corruption): `build_iso_pipeline` for a DVD
/// (MPEG-PS) with `None` keys — what autorip's mux passes on a detection-miss
/// DVD (`disc.decrypt_keys()` == None) — must resolve the CSS key from the
/// reader itself. A scrambled-but-uncrackable title must HARD-FAIL, never
/// build a passthrough pipeline that muxes the scrambled sectors as corrupt
/// video. Before this fix, autorip handed None straight through and the mux
/// wrote garbage at exit 0.
#[test]
fn build_iso_pipeline_dvd_none_keys_scrambled_hard_fails() {
// One CSS-scrambled, crib-less (uncrackable) MPEG-PS sector.
let key = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let mut sec = vec![0u8; 2048];
sec[0..4].copy_from_slice(&crate::css::PACK_START);
for (i, b) in sec.iter_mut().enumerate().take(0x80).skip(4) {
*b = (i as u8).wrapping_mul(7).wrapping_add(1); // non-repeating → no crib
}
sec[0x14] = 0x10; // scramble flag
for (i, b) in sec.iter_mut().enumerate().skip(0x80) {
*b = (i as u8) ^ 0x3C;
}
crate::css::lfsr::scramble_sector(&key, &mut sec);
let mut title = aac_audio_title(0x1100);
title.extents = vec![Extent {
start_lba: 0,
sector_count: 1,
}];
let res = build_iso_pipeline(
MemSource { data: sec },
title,
DecryptKeys::None,
8192,
ContentFormat::MpegPs,
false,
None,
None,
None,
);
assert!(
res.is_err(),
"a scrambled DVD title with no key must hard-fail, not build a scrambled-passthrough pipeline"
);
}
}
+25 -21
View File
@@ -12,9 +12,11 @@ use crate::consts::TS_PACKET_BYTES;
/// TS sync byte.
const SYNC_BYTE: u8 = 0x47;
/// MPEG-TS null-packet PID (0x1FFF). Carries no elementary stream; the P3
/// concealment fill emits null packets on this PID, tagged with an
/// adaptation-field discontinuity_indicator to signal a concealed gap.
/// MPEG-TS null-packet PID (0x1FFF). Carries no elementary stream. The demuxer
/// still recognises a `0x1FFF` packet with an adaptation-field
/// discontinuity_indicator as a concealed-gap loss signal, but the in-tree WRITER
/// that emitted these (the removed NULL-TS concealment fill) is gone — the mux no
/// longer conceals; only externally-authored markers reach this path now.
const NULL_PID: u16 = 0x1FFF;
/// A reassembled PES packet with timestamp info.
@@ -34,8 +36,10 @@ pub struct PesPacket {
pub source: Option<crate::pes::SourcePos>,
/// True when one or more packets for this stream were lost before this PES —
/// a continuity break (CC gap or adaptation-field discontinuity_indicator) on
/// a tracked PID, or the CC-independent concealment marker the mux emits when
/// it replaces an undecryptable unit with NULL-TS packets (P3/A2). This PES is
/// a tracked PID, or a CC-independent NULL-TS concealment marker (P3/B1). NOTE:
/// the mux no longer emits such markers (the concealment writer was removed);
/// this now flags only real discontinuities and externally-authored markers.
/// This PES is
/// the FIRST whose data is entirely after the gap: a mid-frame loss drops the
/// truncated partial and flags the next complete PES; a loss landing on a PES
/// boundary flags the PES STARTING after it (never the one just flushed). So
@@ -200,7 +204,7 @@ impl PesAssembler {
/// BD Transport Stream demuxer.
pub struct TsDemuxer {
assemblers: Vec<PesAssembler>,
pid_index: Vec<i16>, // PID → index into assemblers, -1 = not tracked
pid_index: Vec<i32>, // PID → index into assemblers, -1 = not tracked
remainder: Vec<u8>, // leftover bytes from previous feed() call
/// Absolute source byte offset of the NEXT byte to be fed — the running
/// base that turns an in-buffer packet offset into a source position.
@@ -224,20 +228,17 @@ impl TsDemuxer {
/// limits. Empty `pids` yields max_pid 0; the floor still produces a
/// valid (wholly-unused) table.
pub fn new(pids: &[u16]) -> Self {
// The PID→assembler index is stored as i16 (-1 = untracked), so a
// 32768th+ tracked PID would truncate to a negative value and be
// silently treated as untracked. Callers pass a handful of PIDs
// (BD-TS has at most ~8192), so this is a programmer-error guard.
debug_assert!(
pids.len() <= i16::MAX as usize,
"TsDemuxer: too many PIDs for an i16 index table"
);
// The PID→assembler index is stored as i32 (-1 = untracked). PIDs are
// u16 (≤ 65535) and the assembler index `i` is bounded by the number of
// distinct PIDs (≤ 65536), both far below i32::MAX, so `i as i32` can
// never truncate to a negative value and be mis-read as untracked —
// unlike an i16 table, this is safe in RELEASE, not just under debug.
let max_pid = pids.iter().copied().max().unwrap_or(0) as usize;
let table_size = (max_pid + 1).max(8192);
let mut pid_index = vec![-1i16; table_size];
let mut pid_index = vec![-1i32; table_size];
let mut assemblers = Vec::with_capacity(pids.len());
for (i, &pid) in pids.iter().enumerate() {
pid_index[pid as usize] = i as i16;
pid_index[pid as usize] = i as i32;
assemblers.push(PesAssembler::new(pid));
}
Self {
@@ -368,10 +369,13 @@ impl TsDemuxer {
let pusi = ts[1] & 0x40 != 0; // Payload Unit Start Indicator
let adaptation = (ts[3] >> 4) & 0x03;
// P3/B1 CONCEALMENT MARKER. The decrypt layer fills an undecryptable
// aligned unit with NULL-TS packets (PID 0x1FFF) that carry an
// adaptation-field discontinuity_indicator (see `aacs::content::fill_null_ts_unit`).
// This is the authoritative loss signal — unlike a tracked PID's 4-bit
// P3/B1 CONCEALMENT MARKER: a NULL-TS packet (PID 0x1FFF) carrying an
// adaptation-field discontinuity_indicator. NOTE: the in-tree writer that
// laid these down on an undecryptable unit was removed with the pure-decrypt
// passthrough change (the mux no longer conceals), so this recognition now
// only fires on externally-authored markers — a candidate for removal with
// the rest of the retired concealment path.
// As a loss signal it is CC-INDEPENDENT — unlike a tracked PID's 4-bit
// continuity_counter it is CC-INDEPENDENT, so it survives a loss that is
// an exact multiple of 16 packets and a loss at the very start of a PID
// (no prior CC to diff against). The decrypt layer cannot know which
@@ -1057,7 +1061,7 @@ mod tests {
/// One 192-byte BD source packet that is a B1 concealment marker: a PID-0x1FFF
/// null packet carrying the adaptation-field discontinuity_indicator (the byte
/// shape `fill_null_ts_unit` writes for every packet of a concealed unit).
/// shape of a concealed-unit packet).
fn null_marker_packet() -> Vec<u8> {
let mut pkt = vec![0u8; BD_SOURCE_PACKET_BYTES];
pkt[4] = SYNC_BYTE; // 0x47
+391 -859
View File
File diff suppressed because it is too large Load Diff
+62 -12
View File
@@ -129,6 +129,10 @@ impl SectorSource for Box<dyn SectorSource> {
fn set_speed(&mut self, kbs: u16) {
(**self).set_speed(kbs)
}
fn set_unit_base(&mut self, lba: u32) {
(**self).set_unit_base(lba)
}
}
impl SectorSource for &mut (dyn SectorSource + '_) {
@@ -160,6 +164,10 @@ impl SectorSource for &mut (dyn SectorSource + '_) {
fn set_speed(&mut self, kbs: u16) {
(**self).set_speed(kbs)
}
fn set_unit_base(&mut self, lba: u32) {
(**self).set_unit_base(lba)
}
}
/// Write 2048-byte sectors to a disc image or composed sink.
@@ -181,7 +189,7 @@ pub trait SectorSink: Send {
}
pub use crate::io::file_sector_source::FileSectorSource;
pub use decrypting::{DECRYPT_VERIFY_READ, DecryptingSectorSource, KeyFetch};
pub use decrypting::{DecryptingSectorSource, KeyFetch, KeyFetchFn};
pub use file::FileSectorSink;
pub use prefetched::PrefetchedSectorSource;
@@ -198,23 +206,33 @@ mod tests {
capacity: u32,
reads: Arc<Mutex<Vec<(u32, u16, bool)>>>,
speeds: Arc<Mutex<Vec<u16>>>,
unit_bases: Arc<Mutex<Vec<u32>>>,
}
/// A `Spy` under test plus the handles recording its reads and speed sets.
type SpyHarness = (Spy, Arc<Mutex<Vec<(u32, u16, bool)>>>, Arc<Mutex<Vec<u16>>>);
/// A `Spy` under test plus the handles recording its reads, speed sets,
/// and unit-base sets.
type SpyHarness = (
Spy,
Arc<Mutex<Vec<(u32, u16, bool)>>>,
Arc<Mutex<Vec<u16>>>,
Arc<Mutex<Vec<u32>>>,
);
impl Spy {
fn new(capacity: u32) -> SpyHarness {
let reads = Arc::new(Mutex::new(Vec::new()));
let speeds = Arc::new(Mutex::new(Vec::new()));
let unit_bases = Arc::new(Mutex::new(Vec::new()));
(
Self {
capacity,
reads: reads.clone(),
speeds: speeds.clone(),
unit_bases: unit_bases.clone(),
},
reads,
speeds,
unit_bases,
)
}
}
@@ -238,6 +256,16 @@ mod tests {
fn set_speed(&mut self, kbs: u16) {
self.speeds.lock().unwrap().push(kbs);
}
fn set_unit_base(&mut self, lba: u32) {
self.unit_bases.lock().unwrap().push(lba);
}
}
/// Call `set_unit_base` through a generic `S: SectorSource` bound — this is
/// the path that actually exercises the `Box<dyn>` / `&mut dyn` FORWARDING
/// impls (a direct call on a `dyn` value dispatches via the vtable instead).
fn set_unit_base_generic<S: SectorSource>(mut s: S, base: u32) {
s.set_unit_base(base);
}
/// The default `capacity_sectors` is 0 (unknown). Grounding: trait
@@ -286,7 +314,7 @@ mod tests {
/// Box<dyn SectorSource>` forwarding bodies.
#[test]
fn boxed_dyn_forwards_all_methods() {
let (spy, reads, speeds) = Spy::new(777);
let (spy, reads, speeds, unit_bases) = Spy::new(777);
let mut boxed: Box<dyn SectorSource> = Box::new(spy);
assert_eq!(boxed.capacity_sectors(), 777, "capacity must forward");
@@ -308,24 +336,46 @@ mod tests {
vec![5400],
"set_speed must forward"
);
// set_unit_base through the generic bound exercises the forwarding impl
// (a direct `boxed.set_unit_base()` would vtable-dispatch instead). A
// missing forwarding body would silently no-op and record nothing.
set_unit_base_generic(boxed, 64);
assert_eq!(
*unit_bases.lock().unwrap(),
vec![64],
"set_unit_base must forward through Box<dyn>"
);
}
/// `&mut dyn SectorSource` must likewise forward all three methods.
/// `&mut dyn SectorSource` must likewise forward every method.
/// Grounding: `impl SectorSource for &mut (dyn SectorSource + '_)`.
#[test]
fn mut_ref_dyn_forwards_all_methods() {
let (mut spy, reads, speeds) = Spy::new(123);
let r: &mut dyn SectorSource = &mut spy;
let (mut spy, reads, speeds, unit_bases) = Spy::new(123);
assert_eq!(r.capacity_sectors(), 123);
{
let r: &mut dyn SectorSource = &mut spy;
assert_eq!(r.capacity_sectors(), 123);
let mut buf = vec![0u8; 2 * 2048];
let n = r.read_sectors(7, 2, &mut buf, false).unwrap();
assert_eq!(n, 2 * 2048);
let mut buf = vec![0u8; 2 * 2048];
let n = r.read_sectors(7, 2, &mut buf, false).unwrap();
assert_eq!(n, 2 * 2048);
r.set_speed(8800);
r.set_speed(8800);
}
// Pass `&mut dyn` as a generic S so the forwarding impl's set_unit_base
// is the one under test, not the vtable path.
let r2: &mut dyn SectorSource = &mut spy;
set_unit_base_generic(r2, 128);
assert_eq!(*reads.lock().unwrap(), vec![(7, 2, false)]);
assert_eq!(*speeds.lock().unwrap(), vec![8800]);
assert_eq!(
*unit_bases.lock().unwrap(),
vec![128],
"set_unit_base must forward through &mut dyn"
);
}
}
+8 -5
View File
@@ -252,11 +252,14 @@ impl PrefetchedSectorSource {
};
if bytes <= buf.capacity() {
// Re-expose `bytes` without zero-filling pages that
// `read_sectors` is about to overwrite. The enclosing
// capacity guard makes the `set_len` provably sound even
// if a recycled buffer ever comes back smaller than the
// `vec![0u8; batch_bytes]` it was born with.
debug_assert!(bytes <= buf.capacity(), "set_len exceeds capacity");
// `read_sectors` is about to overwrite. Sound because the
// enclosing `bytes <= capacity` guard bounds the length,
// and every byte below `capacity` is physically
// initialised: buffers are born `vec![0u8; batch_bytes]`
// and only ever grown via `resize(_, 0)`, so a recycled
// buffer that came back shorter (consumer `truncate`)
// still has initialised backing storage under `set_len`,
// which `read_sectors` then overwrites before any read.
unsafe { buf.set_len(bytes) };
} else {
buf.resize(bytes, 0);
+65 -37
View File
@@ -103,25 +103,30 @@ pub struct RecoverCtx {
pub prev_dropped: usize,
}
/// A recovery: given a read's still-scrambled `buf` and the **generic**
/// [`DecryptKeys`], make units decrypt (crack or fetch a key into `keys`) and/or
/// classify the loss (see [`MissOutcome`]). The type names NO encryption scheme
/// — the installed recovery decides what to do with the generic keys, so any
/// scheme (an AACS key-fetch, a future CSS re-crack) is just a different
/// [`Recover`] the input stream installs. `FnMut` so per-recovery
/// state (the AACS dedup set / call budget) lives in the closure's captures with
/// no lock; `Send` so it can ride the mux highway's producer thread.
pub type Recover = Box<dyn FnMut(&mut [u8], &mut DecryptKeys, &RecoverCtx) -> MissOutcome + Send>;
/// A recovery: given a read's post-decrypt `target` (pure decrypt leaves the
/// applied-key plaintext), the matching on-disc `ciphertext`, and the **generic**
/// [`DecryptKeys`], make units decrypt (fetch a key into `keys` and retry) and/or
/// classify the loss (see [`MissOutcome`]). Decryption itself lives in ONE place
/// (`decrypt_sectors`); a recovery only supplies the missing KEY and re-runs it.
/// `ciphertext` is separate from `target` because a pure decrypt overwrites the
/// target with plaintext — the key server still needs the original on-disc bytes,
/// and the retry re-decrypts from them. The type names NO encryption scheme; any
/// scheme is just a different [`Recover`] the input stream installs. `FnMut` so
/// per-recovery state (dedup set / call budget) lives in the closure's captures;
/// `Send` so it can ride the mux highway's producer thread.
pub type Recover =
Box<dyn FnMut(&mut [u8], &[u8], &mut DecryptKeys, &RecoverCtx) -> MissOutcome + Send>;
/// The AACS key-fetch step used by [`key_fetch`]: gather the
/// still-scrambled units, ask `fetch` for keys, add any new ones to the pool and
/// re-decrypt. `dry` / `calls` are the caller-owned dedup set and call budget.
/// Returns the post-retry dropped-byte count.
/// The AACS key-fetch step used by [`key_fetch`]: gather the units the pool did
/// NOT open, ask `fetch` for keys, add any new ones to the pool and re-decrypt.
/// `dry` / `calls` are the caller-owned dedup set and call budget. Returns the
/// post-retry unverified-byte count.
fn aacs_fetch_step(
dry: &mut HashSet<u64>,
calls: &mut usize,
fetch: &KeyFetch,
buf: &mut [u8],
target: &mut [u8],
ciphertext: &[u8],
keys: &mut DecryptKeys,
ctx: &RecoverCtx,
) -> usize {
@@ -130,14 +135,25 @@ fn aacs_fetch_step(
return prev_dropped;
}
let unit_len = crate::aacs::content::ALIGNED_UNIT_LEN;
// Gather up to MAX_FETCH_SAMPLES still-scrambled aligned units — the exact
// on-disc ciphertext no held key could open. A trailing partial unit
// Container of this disc's content — travels with the keys; drives the
// encrypted-flag / structure check below (TS vs PS).
let format = match &*keys {
DecryptKeys::Aacs { format, .. } => *format,
_ => crate::disc::ContentFormat::BdTs,
};
// Gather up to MAX_FETCH_SAMPLES units the current pool did NOT open. Detect
// them on the post-decrypt TARGET (a failed unit stays TS-destroyed; an opened
// one is now clean TS and is skipped), but SAMPLE the matching on-disc
// `ciphertext` — the exact bytes the key server needs. A trailing partial unit
// (chunks_exact remainder) can't be a whole scrambled unit, so skipping it is
// correct.
let mut samples: Vec<Vec<u8>> = Vec::new();
for chunk in buf.chunks_exact(unit_len) {
if crate::aacs::content::aacs_unit_needs_decrypt(chunk) {
samples.push(chunk.to_vec());
for (t, c) in target
.chunks_exact(unit_len)
.zip(ciphertext.chunks_exact(unit_len))
{
if crate::aacs::content::aacs_unit_needs_decrypt(t, format) {
samples.push(c.to_vec());
if samples.len() >= MAX_FETCH_SAMPLES {
break;
}
@@ -154,7 +170,7 @@ fn aacs_fetch_step(
return prev_dropped;
}
*calls += 1;
let fresh = (fetch)(&samples);
let fresh = fetch.unit_keys(&samples);
// Add only keys we don't already hold (dedup by value).
let mut added = 0usize;
if let DecryptKeys::Aacs { unit_keys, .. } = keys {
@@ -172,10 +188,13 @@ fn aacs_fetch_step(
dry.extend(fps);
return prev_dropped;
}
// Retry now that the pool has grown; a unit that still won't decrypt is
// genuine loss. A retry error must not mask the original count.
// Retry now that the pool has grown. Reset the target to the on-disc
// ciphertext first (a pure decrypt already overwrote it with the failed
// plaintext), then re-run the ONE decrypt. A unit that still won't reach clean
// TS stays unverified; a retry error must not mask the original count.
target.copy_from_slice(ciphertext);
redecrypt(
buf,
target,
keys,
ctx.unit_key_idx,
ctx.lba,
@@ -187,7 +206,7 @@ fn aacs_fetch_step(
/// No recovery: a miss is loss. Equivalent to installing nothing — provided so a
/// caller that wants an explicit "give up" recovery has one.
pub fn none() -> Recover {
Box::new(|_buf, _keys, ctx| MissOutcome::loss(ctx.prev_dropped))
Box::new(|_target, _ciphertext, _keys, ctx| MissOutcome::loss(ctx.prev_dropped))
}
/// AACS key-fetch recovery (BD / UHD): on a miss, ask the application's key
@@ -195,9 +214,9 @@ pub fn none() -> Recover {
pub fn key_fetch(fetch: KeyFetch) -> Recover {
let mut dry: HashSet<u64> = HashSet::new();
let mut calls: usize = 0;
Box::new(move |buf, keys, ctx| {
Box::new(move |target, ciphertext, keys, ctx| {
MissOutcome::loss(aacs_fetch_step(
&mut dry, &mut calls, &fetch, buf, keys, ctx,
&mut dry, &mut calls, &fetch, target, ciphertext, keys, ctx,
))
})
}
@@ -243,8 +262,10 @@ mod tests {
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
let out = r(&mut buf, &mut keys, &ctx(0, 6144));
let cipher = buf.clone();
let out = r(&mut buf, &cipher, &mut keys, &ctx(0, 6144));
assert_eq!(out.dropped, 6144);
}
@@ -255,18 +276,20 @@ mod tests {
// decorator's integration tests); here we pin the seam's key-plumbing.
let calls = Arc::new(AtomicUsize::new(0));
let c2 = Arc::clone(&calls);
let fetch: KeyFetch = Arc::new(move |samples: &[Vec<u8>]| {
let fetch: KeyFetch = KeyFetch::unit_only(Arc::new(move |samples: &[Vec<u8>]| {
c2.fetch_add(1, Ordering::SeqCst);
assert!(!samples.is_empty(), "failing ciphertext is forwarded");
vec![[0xAB; 16]]
});
}));
let mut r = key_fetch(fetch);
let mut buf = scrambled_unit(0x33);
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
r(&mut buf, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
let cipher = buf.clone();
r(&mut buf, &cipher, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
assert_eq!(calls.load(Ordering::SeqCst), 1, "fetch called once");
let DecryptKeys::Aacs { unit_keys, .. } = &keys else {
unreachable!()
@@ -281,19 +304,22 @@ mod tests {
// the SAME ciphertext must not call the fetch again.
let calls = Arc::new(AtomicUsize::new(0));
let c2 = Arc::clone(&calls);
let fetch: KeyFetch = Arc::new(move |_: &[Vec<u8>]| {
let fetch: KeyFetch = KeyFetch::unit_only(Arc::new(move |_: &[Vec<u8>]| {
c2.fetch_add(1, Ordering::SeqCst);
Vec::new() // never helps
});
}));
let mut r = key_fetch(fetch);
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
let mut buf = scrambled_unit(0x44);
r(&mut buf, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
let cipher = buf.clone();
r(&mut buf, &cipher, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
let mut buf2 = scrambled_unit(0x44); // identical ciphertext
r(&mut buf2, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
let cipher2 = buf2.clone();
r(&mut buf2, &cipher2, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
assert_eq!(
calls.load(Ordering::SeqCst),
1,
@@ -305,21 +331,23 @@ mod tests {
fn key_fetch_call_budget_bounds_fetches() {
let calls = Arc::new(AtomicUsize::new(0));
let c2 = Arc::clone(&calls);
let fetch: KeyFetch = Arc::new(move |_: &[Vec<u8>]| {
let fetch: KeyFetch = KeyFetch::unit_only(Arc::new(move |_: &[Vec<u8>]| {
c2.fetch_add(1, Ordering::SeqCst);
Vec::new()
});
}));
let mut r = key_fetch(fetch);
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![],
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
// Distinct ciphertext each time so the dry-set never short-circuits; only
// the internal call budget should stop the fetch. The closure self-limits,
// so the decorator can call it unconditionally.
for i in 0..(MAX_FETCH_CALLS as u8 + 5) {
let mut buf = scrambled_unit(i);
r(&mut buf, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
let cipher = buf.clone();
r(&mut buf, &cipher, &mut keys, &ctx(0, ALIGNED_UNIT_LEN));
}
assert_eq!(
calls.load(Ordering::SeqCst),
+109 -5
View File
@@ -593,10 +593,27 @@ impl UdfFs {
match next_block {
Some(cont_lba) => {
read_sector(reader, self.meta_to_abs(cont_lba)?, &mut block)?;
// A continuation block is a list of Short ADs from byte 0,
// spanning the whole sector.
ad_start = 0;
ad_bytes = block.len();
// A continuation block does NOT begin with allocation
// descriptors — it begins with a 24-byte Allocation Extent
// Descriptor (ECMA-167 4/14.5, tag id 258): the 16-byte
// descriptor tag, then prev_allocation_extent_location
// (Uint32 @16) and length_of_allocation_descriptors
// (Uint32 @20). The real ADs (same Short/Long/Extended type
// as the file) start at offset 24, and their total byte
// length is that @20 field.
//
// Reading from offset 0 parses the AED's own tag header as
// allocation descriptors → one garbage extent, then an
// unknown extent_type break — silently truncating every
// file whose ADs spill into a continuation block (a heavily
// fragmented file, e.g. a Blu-ray 3D interleaved base-view
// .m2ts with ~1600 fragments). A normal few-extent .m2ts
// fits inline and never reaches here, which is why this
// stayed hidden.
let aed_l_ad =
u32::from_le_bytes([block[20], block[21], block[22], block[23]]) as usize;
ad_start = 24;
ad_bytes = aed_l_ad.min(block.len().saturating_sub(24));
}
None => break,
}
@@ -1535,8 +1552,15 @@ mod tests {
/// A continuation block: a bare list of short ADs from byte 0.
fn build_cont_block(ads: &[(u32, u32, u32)]) -> [u8; 2048] {
// A continuation block is an Allocation Extent Descriptor (ECMA-167
// 4/14.5): a 16-byte descriptor tag, then prev_allocation_extent_location
// (Uint32 @16) and length_of_allocation_descriptors (Uint32 @20). The
// actual allocation descriptors begin at offset 24. The parser skips the
// 24-byte header and reads `l_ad` bytes of ADs from there.
let mut s = [0u8; 2048];
let mut off = 0usize;
let l_ad = (ads.len() * 8) as u32;
s[20..24].copy_from_slice(&l_ad.to_le_bytes());
let mut off = 24usize;
for &(etype, dlen, dlba) in ads {
let raw_len = (etype << 30) | (dlen & 0x3FFF_FFFF);
s[off..off + 4].copy_from_slice(&raw_len.to_le_bytes());
@@ -1590,6 +1614,35 @@ mod tests {
assert_eq!(extents, vec![(10, 4096), (20, 2048)]);
}
#[test]
fn icb_extents_continuation_skips_aed_header_not_read_as_extent() {
// Regression (Blu-ray 3D): a continuation block begins with a 24-byte
// Allocation Extent Descriptor whose 16-byte descriptor tag holds
// NON-zero bytes (tag id 258, CRC, location, ...). Reading allocation
// descriptors from offset 0 parses that tag header as an AD → a garbage
// extent, then an unknown extent_type break → every fragment past the
// first continuation is lost. On a 3D disc that truncated the 25.8 GB
// interleaved base-view feature to ~1.8 GB (113 of ~1600 fragments) and
// then choked the mux on the bogus, non-unit-aligned extent. The parser
// MUST skip the 24-byte AED header and read the real ADs from offset 24.
let icb = build_efe(6144, &[(0, 4096, 10), (3, 2048, 50)]);
let mut cont = build_cont_block(&[(0, 2048, 20)]);
// Stamp a realistic AED descriptor tag (id 258) into the header so a
// regression that reads from offset 0 mis-parses it as a bogus extent
// instead of finding the real (20, 2048) at offset 24.
cont[0..2].copy_from_slice(&258u16.to_le_bytes());
let mut reader = MapReader::new();
reader.put(5, icb);
reader.put(50, cont);
let fs = fs_with(0, 0, file_entry("3D", 5, 6144));
let extents = fs.read_icb_extents(&mut reader, 5).expect("extents");
assert_eq!(
extents,
vec![(10, 4096), (20, 2048)],
"continuation ADs must be read past the 24-byte AED header, not from offset 0"
);
}
#[test]
fn icb_extents_long_ad_returns_all_extents_not_just_first() {
// Regression: BD-ROM large .m2ts files use Long ADs (16-byte
@@ -1737,6 +1790,57 @@ mod tests {
assert!(inf[2048..].iter().all(|&b| b == 0xBB));
}
#[test]
fn read_aacs_inputs_falls_through_to_hddvd_any_dir() {
// HD DVD keeps its AACS material under /ANY!/ (VTKF000.AACS title-key
// file + MKBROM.AACS), NOT /AACS/Unit_Key_RO.inf + /AACS/MKB_RO.inf. The
// role-based candidate lists must fall through to the /ANY!/ files with
// NO disc-type branch, so the online keyserver POST carries the HD DVD
// title-key file (magic "DVD_HD_V_TKF") as inf_b64 + MKBROM as mkb_b64 —
// the server then classifies the disc as HD DVD by that magic.
let any = DirEntry {
name: "ANY!".to_string(),
is_dir: true,
meta_lba: 0,
size: 0,
entries: vec![
file_entry("VTKF000.AACS", 5, 2048),
file_entry("MKBROM.AACS", 7, 2048),
],
};
let root = DirEntry {
name: String::new(),
is_dir: true,
meta_lba: 0,
size: 0,
entries: vec![any], // deliberately NO /AACS/ dir
};
let mut reader = MapReader::new();
// VTKF000.AACS: one extent whose content opens with the HD DVD magic.
let mut vtkf = [0u8; 2048];
vtkf[..12].copy_from_slice(b"DVD_HD_V_TKF");
reader.put(5, build_efe_long(2048, &[(0, 2048, 10)]));
reader.put(10, vtkf);
// MKBROM.AACS: one extent with a type-0x10 AACS-1.0 (HD DVD) version record.
let mut mkb = [0u8; 2048];
mkb[..12].copy_from_slice(&[
0x10, 0x00, 0x00, 0x0C, 0x00, 0x04, 0x10, 0x03, 0x00, 0x00, 0x00, 0x03,
]);
reader.put(7, build_efe_long(2048, &[(0, 2048, 50)]));
reader.put(50, mkb);
let fs = fs_with(0, 0, root);
let (inf, _mkb, _version) =
crate::disc::Disc::read_aacs_inputs_from_reader(&mut reader, &fs)
.expect("read_aacs_inputs must source the HD DVD /ANY!/ files");
assert_eq!(
&inf[..12],
b"DVD_HD_V_TKF",
"inf must be the HD DVD VTKF (its magic), sourced from /ANY!/ via the \
candidate fall-through — not /AACS/Unit_Key_RO.inf"
);
}
#[test]
fn merge_ranges_saturates_near_u32_max() {
// Adjacent ranges near u32::MAX must not panic (debug) or wrap.
+67 -61
View File
@@ -131,16 +131,15 @@ fn aacs_decrypt_unit_roundtrip() {
}
// Verify it looks encrypted (body TS syncs scrambled)
assert!(aacs::content::ts_sync_destroyed(&plain));
assert!(!aacs::content::is_clean(
&plain,
libfreemkv::disc::ContentFormat::BdTs
));
// Now decrypt
let result = aacs::content::decrypt_unit(&mut plain, &unit_key);
aacs::content::decrypt_unit(&mut plain, &unit_key);
assert!(
result,
"decrypt_unit should return true on valid encrypted unit"
);
assert!(
!aacs::content::ts_sync_destroyed(&plain),
aacs::content::is_clean(&plain, libfreemkv::disc::ContentFormat::BdTs),
"decrypted unit should read as clear (TS syncs restored)"
);
@@ -258,9 +257,9 @@ fn aacs_vuk_derivation_roundtrip() {
assert_eq!(vuk, vuk2, "derive_vuk not deterministic");
}
/// Test: aacs_ts_sync_destroyed detects scrambled units via the raw TS syncs.
/// Test: `is_clean` distinguishes clean vs scrambled units via the TS proof floor.
#[test]
fn aacs_ts_sync_destroyed_detection() {
fn aacs_is_clean_detection() {
// A clear unit: TS sync (0x47) intact at every 192-byte packet → not
// scrambled. (Flag bits play no role.)
let mut clear = vec![0u8; aacs::content::ALIGNED_UNIT_LEN];
@@ -270,7 +269,7 @@ fn aacs_ts_sync_destroyed_detection() {
off += 192;
}
assert!(
!aacs::content::ts_sync_destroyed(&clear),
aacs::content::is_clean(&clear, libfreemkv::disc::ContentFormat::BdTs),
"clear unit (syncs intact) must not be scrambled"
);
@@ -279,32 +278,35 @@ fn aacs_ts_sync_destroyed_detection() {
flagged[0] = 0xC0; // copy-control bits
flagged[7] = 0xC0; // TSC bits
assert!(
!aacs::content::ts_sync_destroyed(&flagged),
aacs::content::is_clean(&flagged, libfreemkv::disc::ContentFormat::BdTs),
"flag bits must not be read as encryption"
);
// A scrambled body (syncs destroyed) → scrambled.
let scrambled = vec![0x99u8; aacs::content::ALIGNED_UNIT_LEN];
assert!(
aacs::content::ts_sync_destroyed(&scrambled),
!aacs::content::is_clean(&scrambled, libfreemkv::disc::ContentFormat::BdTs),
"unit with no intact TS syncs must read as scrambled"
);
// Too short
let short = vec![0xFFu8; 100];
assert!(
!aacs::content::ts_sync_destroyed(&short),
aacs::content::is_clean(&short, libfreemkv::disc::ContentFormat::BdTs),
"short buffer should not be detected"
);
}
/// Test: aacs_decrypt_unit_unencrypted_passthrough
/// Test: aacs_clear_unit_reports_not_encrypted
///
/// A clear unit (TS syncs intact) should pass through decrypt_unit unchanged.
/// `decrypt_unit` is now PURE (applies the key unconditionally). The "leave a
/// clear unit untouched" policy lives at the caller's gate `aacs_unit_encrypted`:
/// a CPI-clear unit reports not-encrypted, so the caller never hands it to
/// decrypt_unit.
#[test]
fn aacs_decrypt_unit_unencrypted_passthrough() {
fn aacs_clear_unit_reports_not_encrypted() {
let mut unit = vec![0x42u8; aacs::content::ALIGNED_UNIT_LEN];
// Intact TS syncs every 192 bytes → not scrambled → passthrough.
// Intact TS syncs every 192 bytes → not scrambled.
let mut off = 4;
while off < aacs::content::ALIGNED_UNIT_LEN {
unit[off] = 0x47;
@@ -312,13 +314,15 @@ fn aacs_decrypt_unit_unencrypted_passthrough() {
}
// CPI bits (byte 0) CLEAR → the authoritative gate reads this as plaintext.
unit[0] &= 0x3F;
let original = unit.clone();
let key = [0xAA; 16];
assert!(!aacs::content::ts_sync_destroyed(&unit));
let result = aacs::content::decrypt_unit(&mut unit, &key);
assert!(result, "clear unit should return true");
assert_eq!(unit, original, "clear unit should be unchanged");
assert!(aacs::content::is_clean(
&unit,
libfreemkv::disc::ContentFormat::BdTs
));
assert!(
!aacs::content::aacs_unit_encrypted(&unit, libfreemkv::disc::ContentFormat::BdTs),
"CPI-clear unit reports not-encrypted; the caller never decrypts it"
);
}
// ── AACS cross-validation with independent AES implementation ──────────────
@@ -413,11 +417,7 @@ fn aacs_cross_validation_encrypt_then_decrypt() {
);
// -- Decrypt with the library --
let ok = aacs::content::decrypt_unit(&mut plaintext, &unit_key);
assert!(
ok,
"decrypt_unit returned false (TS sync verification failed)"
);
aacs::content::decrypt_unit(&mut plaintext, &unit_key);
// Decryption clears no flag, so the unit round-trips byte-for-byte.
assert_eq!(
@@ -458,44 +458,14 @@ fn aacs_cross_validation_alternate_key() {
&mut plaintext[16..aacs::content::ALIGNED_UNIT_LEN],
);
assert!(aacs::content::decrypt_unit(&mut plaintext, &unit_key));
aacs::content::decrypt_unit(&mut plaintext, &unit_key);
// Decryption clears no flag, so the unit round-trips byte-for-byte.
assert_eq!(&plaintext[..], &expected[..]);
}
/// Verify that `decrypt_bus` correctly reverses AES-CBC encryption applied
/// per-sector to bytes 16..2048 (bus encryption layer).
#[test]
fn aacs_bus_decrypt_cross_validation() {
let read_data_key: [u8; 16] = [
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF,
0x00,
];
let mut plaintext = vec![0u8; aacs::content::ALIGNED_UNIT_LEN];
#[allow(clippy::needless_range_loop)]
for i in 0..aacs::content::ALIGNED_UNIT_LEN {
plaintext[i] = ((i * 3 + 17) & 0xFF) as u8;
}
let expected = plaintext.clone();
// Encrypt per-sector: AES-CBC encrypt bytes 16..2048 of each 2048-byte sector
for sector_start in (0..aacs::content::ALIGNED_UNIT_LEN).step_by(2048) {
ref_aes_cbc_encrypt(
&read_data_key,
&CROSS_AACS_IV,
&mut plaintext[sector_start + 16..sector_start + 2048],
);
}
assert_ne!(&plaintext[16..32], &expected[16..32]);
aacs::content::decrypt_bus(&mut plaintext, &read_data_key);
assert_eq!(
plaintext, expected,
"bus decrypt did not recover original plaintext"
);
}
// (`decrypt_bus` is a crate-internal layer — its cross-validation lives in-crate
// in `aacs::content`'s unit tests, not here.)
// ── CSS roundtrip test vectors ─────────────────────────────────────────────
@@ -663,6 +633,28 @@ fn css_stevenson_attack_validates_cracked_key() {
This is expected: synthetic sectors lack the TAB1 output encoding \
present in real CSS-encrypted DVD sectors."
);
// Never let this test pass vacuously: when the attack can't converge on
// synthetic data, still assert always-true properties of the CSS keystream
// so a real regression is caught on every run — descramble_sector is
// DETERMINISTIC (same key/seed/data → same output) and NON-TRIVIAL (it
// actually transforms the payload, not a silent no-op).
for (key, seed) in candidates {
let mut base = vec![0x00u8; 2048];
base[0x14] = 0x30;
base[0x54..0x59].copy_from_slice(seed);
base[0x80..0x8A]
.copy_from_slice(&[0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x80, 0x05, 0x21]);
let mut a = base.clone();
let mut b = base.clone();
css::lfsr::descramble_sector(key, &mut a);
css::lfsr::descramble_sector(key, &mut b);
assert_eq!(a, b, "descramble must be deterministic for key={key:02X?}");
assert_ne!(
&a[0x80..2048],
&base[0x80..2048],
"descramble must transform the payload for key={key:02X?}"
);
}
}
}
@@ -707,6 +699,20 @@ fn css_recover_title_key_with_exact_plaintext() {
The LFSR0 recovery phase may not converge for this combination.",
title_key, seed
);
// Never pass vacuously: when LFSR0 recovery can't converge on this
// synthetic sector, still assert always-true properties of the cipher so
// a real regression is caught on every run — descramble_sector is
// DETERMINISTIC and NON-TRIVIAL (actually transforms the payload).
let mut a = original.clone();
let mut b = original.clone();
css::lfsr::descramble_sector(&title_key, &mut a);
css::lfsr::descramble_sector(&title_key, &mut b);
assert_eq!(a, b, "descramble must be deterministic");
assert_ne!(
&a[0x80..2048],
&original[0x80..2048],
"descramble must transform the payload"
);
}
}
+1
View File
@@ -175,6 +175,7 @@ fn run_to_fvi(image: Vec<u8>, title: DiscTitle, path: &std::path::Path) {
DecryptKeys::None,
3, // 3-sector (one AACS unit) batches → one source stamp per GOP region
ContentFormat::MpegPs,
false,
None,
None,
None,
+20 -2
View File
@@ -136,7 +136,16 @@ fn test_bytes_read_emitted_during_disc_copy() {
let title = synthetic_title(64);
let keys = libfreemkv::DecryptKeys::None;
let mut stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
let mut stream = DiscStream::new(
Box::new(reader),
title,
keys,
60,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
let count = Arc::new(AtomicU64::new(0));
let count_cb = count.clone();
@@ -286,7 +295,16 @@ fn test_drop_impls_do_not_panic_or_block() {
let reader = ZeroSectorReader::new(64);
let title = synthetic_title(64);
let keys = libfreemkv::DecryptKeys::None;
let stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
let stream = DiscStream::new(
Box::new(reader),
title,
keys,
60,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
// Drop on a worker thread; main thread enforces the timeout.
let handle = std::thread::spawn(move || {
+40 -12
View File
@@ -27,21 +27,39 @@ fn decrypt_sectors_with_aacs_keys_works() {
let unit_key: [u8; 16] = [0xAAu8; 16];
// Encrypt the unit using AACS algorithm
aacs::content::decrypt_unit(&mut unit, &unit_key); // decrypt_unit is idempotent on already-encrypted data
// Apply the key to the pattern to produce ciphertext-shaped bytes for the
// call below. (decrypt_unit is now PURE — it applies the key unconditionally,
// so it is NOT idempotent; never call it twice on the same unit.)
aacs::content::decrypt_unit(&mut unit, &unit_key);
// (byte 0 keeps its CPI bits set from above, so `decrypt_sectors` recognises
// this as encrypted content and actually applies the key.)
// Now we have encrypted data - create DecryptKeys with actual keys
let mut keys = DecryptKeys::Aacs {
let mut aacs_keys = DecryptKeys::Aacs {
unit_keys: vec![(0u32, unit_key)],
read_data_key: None,
format: libfreemkv::disc::ContentFormat::BdTs,
};
let mut none_keys = DecryptKeys::None;
// decrypt_sectors should handle this without error
let result = libfreemkv::decrypt::decrypt_sectors(&mut unit, &mut keys, 0);
// The regression this guards is passing `DecryptKeys::None` where AACS keys
// were meant. Prove the two DIVERGE: AACS applies the key (bytes change), None
// leaves the unit byte-for-byte untouched. is_ok alone can't catch that —
// both variants return Ok.
let mut with_aacs = unit.clone();
let mut with_none = unit.clone();
libfreemkv::decrypt::decrypt_sectors(&mut with_aacs, &mut aacs_keys, 0)
.expect("AACS decrypt must not error");
libfreemkv::decrypt::decrypt_sectors(&mut with_none, &mut none_keys, 0)
.expect("None decrypt must not error");
assert!(
result.is_ok(),
"decrypt_sectors with AACS keys should not error"
assert_ne!(
with_aacs, unit,
"AACS keys must actually transform the unit"
);
assert_eq!(with_none, unit, "None keys must leave the unit untouched");
assert_ne!(
with_aacs, with_none,
"AACS decrypt must differ from the None no-op (the None-vs-Aacs regression)"
);
}
@@ -90,16 +108,25 @@ fn aacs_encryption_flag_detection() {
off += 192;
}
// Encryption is the scrambled body (TS syncs destroyed), NOT a flag bit.
assert!(!aacs::content::ts_sync_destroyed(&unit));
assert!(aacs::content::is_clean(
&unit,
libfreemkv::disc::ContentFormat::BdTs
));
// Flag bits on a synced unit do not make it look encrypted.
unit[0] = 0xC0;
unit[7] = 0xC0;
assert!(!aacs::content::ts_sync_destroyed(&unit));
assert!(aacs::content::is_clean(
&unit,
libfreemkv::disc::ContentFormat::BdTs
));
// Scrambled body (syncs gone) → encrypted.
let scrambled = vec![0x99u8; aacs::content::ALIGNED_UNIT_LEN];
assert!(aacs::content::ts_sync_destroyed(&scrambled));
assert!(!aacs::content::is_clean(
&scrambled,
libfreemkv::disc::ContentFormat::BdTs
));
}
/// Test: DecryptKeys::is_encrypted() correctly identifies encrypted state.
@@ -111,6 +138,7 @@ fn decrypt_keys_is_encrypted_variants() {
let aacs = DecryptKeys::Aacs {
unit_keys: vec![],
read_data_key: None,
format: libfreemkv::disc::ContentFormat::BdTs,
};
assert!(aacs.is_encrypted());
-1
View File
@@ -275,7 +275,6 @@ fn patch_block_sectors_zero_does_not_busy_spin() {
progress: None,
halt: Some(halt.clone()),
key_fetch: None,
fast_capture: false,
};
let outcome = disc.patch(&mut reader, &iso_path, &opts);
+187 -29
View File
@@ -232,8 +232,6 @@ struct Golden {
bytes_unreadable: u64,
/// `bytes_pending` (NonTrimmed) at end.
bytes_pending: u64,
/// Did the pass exit via wedge-detection?
wedged_exit: bool,
/// Sanity bound on trace length — patch makes a finite number of
/// reads bounded by `MAX_SKIPS_PER_RANGE * range_sectors` plus
/// retries. Asserted as an UPPER bound only (so any reduction in
@@ -319,7 +317,6 @@ fn profile_01_clean_all_recoverable() {
bytes_good: capacity_sectors as u64 * 2048,
bytes_unreadable: 0,
bytes_pending: 0,
wedged_exit: false,
max_reads: 8, // adaptive batch=32 reads finishes 16 sectors in 1 read; allow up to 8.
};
assert_eq!(stats.bytes_good, expected.bytes_good, "01_clean bytes_good");
@@ -796,29 +793,190 @@ fn profile_08_batch_fail_singles_ok() {
// ─────────────────────────────────────────────────────────────────────────
//
// Suppressed for now: NOT_READY-then-recover, HARDWARE_ERROR (wedge),
// ILLEGAL_REQUEST (wedge), and ABORTED_COMMAND profiles. Each would
// trigger long real-time sleeps inside `handle_read_failure`:
// Sense-family error paths in `Disc::patch`: NOT_READY-then-recover,
// HARDWARE_ERROR, ILLEGAL_REQUEST, and ABORTED_COMMAND.
//
// - NOT_READY (sense_key=0x02, asc=0x02/0x03/0x04): 15 s pause per
// occurrence (`patch_not_ready_pause`), and retries the same LBA
// in-place. Even one NOT_READY costs the test 15 s wall-time.
// These drive `disc.patch(...)` DIRECTLY rather than through `run_profile`
// (which drives `Disc::copy`, whose SWEEP path really sleeps on NOT_READY /
// wedge cooldowns via `sleep_secs_or_halt`). The patch handler chain itself
// uses an injectable deadline clock (`Instant::now` in production) and never
// `thread::sleep`s, so these paths run at full speed with no wall-time cost —
// the earlier "sleeps aren't injectable" suppression only ever applied to the
// copy/sweep driver, not to patch.
//
// - HARDWARE_ERROR / ILLEGAL_REQUEST: 30 s per occurrence
// (`WEDGE_FAMILY_COOLDOWN_SECS`), bounded by
// `WEDGE_ABORT_THRESHOLD=16` before wedged-exit. Worst case ~8
// minutes per profile.
//
// The sleeps are not injectable. Adding them would require either a
// `now()` / `sleep()` trait injection (out of scope for the unification
// task) or a "test mode" compile-time flag (architectural smell). The
// behavioural contracts for those paths are captured in
// `read_error.rs`'s in-module tests instead — they exercise the
// classifier without invoking the patch loop's sleep side-effects.
//
// If the unification ever proceeds, the next step is to add a clock
// injection point in `handle_read_failure` and extend this fixture
// with the wedge/NOT_READY profiles too.
// The load-bearing invariant asserted across every PERSISTENT failure sense is
// the recovery contract: a patch pass NEVER promotes a sector to Unreadable
// (the orchestrator does that only after the final pass) and NEVER silently
// drops bytes — a still-bad sector stays NonTrimmed (pending), so
// good + pending always conserves the total. Exact good/pending splits are
// left loose so wedge-skip tuning can't spuriously fail these.
/// Run a single-always-bad-sector (LBA 130, inside a NonTrimmed [128,192)
/// range) patch pass with the given failure step and return the final map
/// stats. 256-sector synthetic disc; everything outside the range is Finished.
fn single_dead_sector_patch_stats(step: ScriptStep) -> libfreemkv::disc::mapfile::MapStats {
let capacity_sectors: u32 = 256;
let (mut reader, _trace) = ScriptedSectorReader::new(capacity_sectors);
reader.always(130, step);
let total_bytes = capacity_sectors as u64 * SECTOR_SIZE as u64;
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().unwrap();
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let nontrimmed = [(128 * 2048, 64 * 2048)];
let finished = [
(0, 128 * 2048),
(192 * 2048, (capacity_sectors as u64 - 192) * 2048),
];
prep_iso_and_mapfile(&iso_path, total_bytes, &finished, &nontrimmed);
let opts = libfreemkv::disc::PatchOptions {
decrypt: false,
block_sectors: Some(32),
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
key_fetch: None,
};
disc.patch(&mut reader, &iso_path, &opts)
.expect("patch must not error on a per-sector failure sense");
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let stats = Mapfile::load(&map_path).unwrap().stats();
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(&map_path);
stats
}
/// A persistent sense that never clears must obey the pass contract: nothing
/// Unreadable, nothing lost (good + pending == total), and at least the dead
/// sector left pending.
fn assert_persistent_sense_contract(step: ScriptStep, label: &str) {
let stats = single_dead_sector_patch_stats(step);
let total = 256u64 * 2048;
assert_eq!(
stats.bytes_unreadable, 0,
"{label}: a patch pass must NEVER mark Unreadable"
);
assert_eq!(
stats.bytes_good + stats.bytes_pending,
total,
"{label}: conservation — no byte may be silently dropped"
);
assert!(
stats.bytes_pending >= 2048,
"{label}: the always-dead sector must remain pending (NonTrimmed)"
);
}
#[test]
fn patch_persistent_hardware_error_conserves_and_never_unreadable() {
// HARDWARE_ERROR (sense_key=0x04) — wedge family.
assert_persistent_sense_contract(
ScriptStep::Err {
sense_key: 0x04,
asc: 0x11,
ascq: 0x00,
},
"HARDWARE_ERROR",
);
}
#[test]
fn patch_persistent_illegal_request_conserves_and_never_unreadable() {
// ILLEGAL_REQUEST (sense_key=0x05) — wedge family.
assert_persistent_sense_contract(
ScriptStep::Err {
sense_key: 0x05,
asc: 0x21,
ascq: 0x00,
},
"ILLEGAL_REQUEST",
);
}
#[test]
fn patch_persistent_aborted_command_conserves_and_never_unreadable() {
// ABORTED_COMMAND (sense_key=0x0B).
assert_persistent_sense_contract(
ScriptStep::Err {
sense_key: 0x0B,
asc: 0x00,
ascq: 0x00,
},
"ABORTED_COMMAND",
);
}
#[test]
fn patch_not_ready_then_recovers_fully() {
// NOT_READY (sense_key=0x02, asc=0x04) that clears after two attempts must
// recover the sector in-pass — no residual loss, no Unreadable, no hang.
let capacity_sectors: u32 = 256;
let (mut reader, _trace) = ScriptedSectorReader::new(capacity_sectors);
reader.sequence(
130,
vec![
ScriptStep::Err {
sense_key: 0x02,
asc: 0x04,
ascq: 0x00,
},
ScriptStep::Err {
sense_key: 0x02,
asc: 0x04,
ascq: 0x00,
},
ScriptStep::Ok,
],
);
let total_bytes = capacity_sectors as u64 * SECTOR_SIZE as u64;
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().unwrap();
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let nontrimmed = [(128 * 2048, 64 * 2048)];
let finished = [
(0, 128 * 2048),
(192 * 2048, (capacity_sectors as u64 - 192) * 2048),
];
prep_iso_and_mapfile(&iso_path, total_bytes, &finished, &nontrimmed);
let opts = libfreemkv::disc::PatchOptions {
decrypt: false,
block_sectors: Some(32),
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
key_fetch: None,
};
disc.patch(&mut reader, &iso_path, &opts)
.expect("patch must not error on a transient NOT_READY");
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let stats = Mapfile::load(&map_path).unwrap().stats();
assert_eq!(
stats.bytes_unreadable, 0,
"NOT_READY recovery must not mark Unreadable"
);
assert_eq!(
stats.bytes_pending, 0,
"a NOT_READY that clears must leave nothing pending"
);
assert_eq!(
stats.bytes_good,
capacity_sectors as u64 * 2048,
"every sector recovers once NOT_READY clears"
);
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(&map_path);
}
// ──────── Handler chain recovers re-readable sectors inside a bad block ────────
//
@@ -826,9 +984,10 @@ fn profile_08_batch_fail_singles_ok() {
// handler chain's linear pass narrows a failed batch to per-sector reads, so it
// recovers EVERY re-readable sector and leaves ONLY the dead sector NonTrimmed —
// strictly better than the old fast-capture path, which left the whole failed
// 32-block untouched. (`fast_capture` is now inert: the chain supersedes it. The
// breadth-first "fast on all ranges, then escalate" ORDERING it once provided is
// a scheduling concern for the handler scheduler, tracked separately.)
// 32-block untouched. (The old `fast_capture` knob was removed: the handler
// chain supersedes it. The breadth-first "fast on all ranges, then escalate"
// ORDERING it once provided is a scheduling concern for the handler scheduler,
// tracked separately.)
//
// The load-bearing invariant is unchanged: NO data is dropped. A still-bad
// sector becomes NonTrimmed (pending, retried by a later pass), NEVER Unreadable.
@@ -869,10 +1028,9 @@ fn handler_chain_recovers_readable_sectors_leaving_only_dead_pending() {
progress: None,
halt: None,
key_fetch: None,
fast_capture: true,
};
disc.patch(&mut reader, &iso_path, &opts)
.expect("fast-capture patch must not error");
.expect("handler-chain patch must not error");
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let stats = Mapfile::load(&map_path).unwrap().stats();
+5 -5
View File
@@ -591,7 +591,7 @@ fn meta_all_stream_types() {
fn mkvstream_write_finish() {
let dt = sample_disc_title();
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let mut stream = MkvStream::create(writer, &dt).unwrap();
let mut stream = MkvStream::create(writer, &dt, None).unwrap();
// Write some fake PES frames (they won't produce valid MKV content
// since there is no real codec data, but it should not panic)
@@ -616,7 +616,7 @@ fn mkvstream_write_finish() {
fn mkvstream_meta_sets_title() {
let dt = sample_disc_title();
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let stream = MkvStream::create(writer, &dt).unwrap();
let stream = MkvStream::create(writer, &dt, None).unwrap();
let info = PesStream::info(&stream);
assert_eq!(info.playlist, "Test Movie");
@@ -656,7 +656,7 @@ fn mkvstream_roundtrip_bdts() {
};
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let mut stream = MkvStream::create(writer, &dt).unwrap();
let mut stream = MkvStream::create(writer, &dt, None).unwrap();
// Write PES frames targeting the audio track
for i in 0..10u8 {
@@ -745,7 +745,7 @@ fn mkvstream_meta_preserves_all_streams() {
};
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let stream = MkvStream::create(writer, &dt).unwrap();
let stream = MkvStream::create(writer, &dt, None).unwrap();
let info = PesStream::info(&stream);
assert_eq!(info.streams.len(), 5, "all 5 streams should be preserved");
@@ -856,7 +856,7 @@ fn mkvstream_e2e_h264_produces_valid_mkv() {
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> =
Box::new(SharedWriter(output2.clone()));
let mut stream2 = MkvStream::create(writer, &dt).unwrap();
let mut stream2 = MkvStream::create(writer, &dt, None).unwrap();
// Write the ES data (SPS+PPS+IDR) as a keyframe PES frame.
let frame1 = libfreemkv::pes::PesFrame {