Compare commits

...
42 Commits
Author SHA1 Message Date
Matthew Jackson 5b27b9f8a5 v1.5.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-19 15:37:25 -07:00
Matthew Jackson 405e9d32c3 changelog: drop-on-undecodable audio + forced PGS detection (1.5.0) 2026-07-19 14:29:57 -07:00
Matthew Jackson 21b362c6e8 disc: detect forced PGS subtitles from stream content for info
Give `info` the same forced-subtitle verdict the muxer derives during a
rip, so the two agree. A shared classifier (mux::codec::pgs::ForcedTracker)
folds a PGS track's display sets — forced iff every one carries the
forced_on_flag — and is used by BOTH the MKV writer and a new scan-time
probe that reads the title's PGS streams (reusing the TS demuxer and PGS
parser). The probe only overrides a track it actually observed content
for, so an undecrypted/unread stream keeps its vendor-derived flag. Gated
behind ScanOptions::probe_forced_subtitles (off for the rip path, which
detects forced while muxing without a second read).
2026-07-19 14:23:04 -07:00
Matthew Jackson 9f8df95513 mux/codec/mpegaudio: keep free-format frames
Free-format MPEG-audio (bitrate_index 0) is a legal, decodable mode — the
decoder derives the frame size from the sync spacing. Dropping it was a
false positive on a clean stream, so it now passes the gate.
2026-07-19 14:23:04 -07:00
Matthew Jackson 2f465f4cfe mux/codec/truehd: fix drop-forward poison + rate/resync validation
Three TrueHD state-machine fixes from an adversarial audit:

- A corrupt access unit drops forward to the next major sync, but only
  the individually-verified corruption now feeds the whole-track poison
  verdict; the resync run is collateral. A couple of transient errors can
  no longer poison and discard an otherwise-good multi-hour track. The
  shared drop tally gains a verified/collateral split for this.
- The per-AU PTS rate is refined only from a CRC-validated major sync, so
  a corrupt major sync whose rate nibble decodes to another rate family
  can no longer shift the resumed audio — a drop stays a silence gap.
- The resync clears only on a CRC-validated major sync, never on a runt
  too short to hold and validate its header.
2026-07-19 14:23:04 -07:00
Matthew Jackson 3fdd463794 release: 1.5.0 2026-07-19 13:42:48 -07:00
Matthew Jackson 6a7392ac8c mux: detect forced PGS subtitles from the stream
Flag a PGS subtitle track FlagForced when it displays subtitles and every
one carries the HDMV forced_on_flag (a dedicated forced/narrative track),
independent of the disc's vendor label metadata. The track header
reserves a FlagForced byte up front and it is promoted at finish() from
the accumulated display-set state. Only ever promotes — a track already
forced from the playlist metadata is never demoted.
2026-07-19 13:42:48 -07:00
Matthew Jackson 28bdecd26b mux/codec: drop undecodable audio frames, keep A/V sync
A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch. Sync is preserved — a drop becomes a silence
gap, never a shift — and every drop is logged. Detection is per-codec,
each mirroring the format's authoritative integrity check:

  DTS          core-header validity gates
  AC-3/E-AC-3  native frame CRC-16 + bitstream-id range
  FLAC         whole-frame CRC-16 residue
  MP2/MP3      header sanity + free-format reject
  AAC-ADTS     header sanity (raw AAC passes through untouched)
  TrueHD/MLP   major-sync CRC-16 + AU parity; corrupt AUs drop forward
               to the next major sync, since decode state carries
               across access units

LPCM and video are excluded by design (no in-frame integrity data;
inter-frame prediction). A shared DropTally handles counting, logging,
and a whole-track fallback for a mostly-undecodable track.
2026-07-19 13:42:48 -07:00
Matthew Jackson d29d3d71b9 dts: emit clean core alone when the extension boundary is garbage
DTS-HD MA access units are a lossy core frame followed by trailing
extension substreams up to the next core sync. On source-damaged discs
(observed on the Bourne UHDs) the bytes where the XLL extension belongs
are neither a core sync nor an extension sync -- pure garbage -- which
desyncs ffmpeg's XLL decoder and cascades into 'Read past end of XLL
band data' / 'DSYNC check failed' across the whole track.

next_core_boundary now distinguishes three boundary states via a new
ext_clean flag on NextCore::Found:
  - precise/recognized extension sync  -> ext_clean=true  (keep full AU)
  - garbage at the boundary byte       -> ext_clean=false (drop the ext)

When ext_clean is false we emit the DTS core alone (drop the smallest
junk piece, keep the frame and its PTS) and drain past the garbage to
the next core. Recognized-but-unsizeable extensions still ride the
heuristic scan and are kept intact, so lossless tracks are unaffected --
only genuinely corrupt extension bytes are dropped.

Bourne s1.dts: 1606 damaged frames / 691620 (0.232%), 93% isolated
single frames, worst run 3 in a row (~32ms lossy blip).
2026-07-19 09:27:28 -07:00
Matthew Jackson 999898a438 changelog: faststart default + DTS/DTS-HD audio (mp4://) 2026-07-18 22:35:37 -07:00
Matthew Jackson b4470ccd81 mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box
(sample rate, channel layout mask, core size, computed bitrate, LFE);
whole access units (core + DTS-HD extension substreams) pass through, so
an HD decoder finds the extension. dtsh when an extension sync follows the
core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR.

Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is
byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1)
and decodes clean (exit 0). Channel layout correct.
2026-07-18 22:18:35 -07:00
Matthew Jackson 668dd95237 mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free
box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample ×
est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes
mdat, sample offsets are fixed from the start — no rewrite, no offset
patch. finish() writes moov into the hole and pads the slack with a free
box; if the estimate is blown it falls back to moov-at-end. Streams over
HTTP without a pre-fetch. Reserve-math + box-order tests added.
2026-07-18 21:43:44 -07:00
Matthew Jackson 777d9a04a7 changelog: mp4:// input demuxer (1.5.0) 2026-07-18 21:22:41 -07:00
Matthew Jackson b717044ae5 mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel
info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/
composition timing, stss → sync), rebuild a DiscTitle, and emit samples
as PesFrames in global decode order. Video NALs are length-prefixed in
MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into
input() so mp4:// flows to every sink (mkv://, audio://, json://, …).
In-memory write→read round-trip test proves symmetry (streams, hvcC,
sample sizes survive). Progressive MP4 only; fragmented (moof) is future.
2026-07-18 21:10:15 -07:00
Matthew Jackson 379767071c mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps
the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow /
NFS staging avoids the dirty-page burst. The mdat backpatch is an
ordinary seek WritebackFile already handles (seek_then_patch_roundtrip).
Tests switched to in-memory Cursor writers.
2026-07-18 20:57:44 -07:00
Matthew Jackson 6a06443dd8 changelog: mp4:// native MP4 muxer (1.5.0) 2026-07-18 20:44:18 -07:00
Matthew Jackson 42df399156 mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio
track. Audio sample entries built from the first frame's bitstream —
AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for
fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio
durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle
(mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3;
TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the
CLI can report exclusions — never a silent drop.

Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3
both frame-exact on every track (video 2650/4270, audio 5567/3454),
duration and colour identical, clean decode. On a pathological 4-clip
title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more
of them) — source-inherent, not a muxer defect.
2026-07-18 20:37:10 -07:00
Matthew Jackson 9ede4b9afd mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into
a 64-bit mdat and building the sample tables in memory, patched at
finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs
(already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts,
CFR-derived decode timeline (pipeline carries presentation PTS only), and
a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the
hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping.

Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and
HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact
(bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land
in M2.
2026-07-18 20:16:19 -07:00
Matthew Jackson 67e3e39499 mux: add video:// sink (video-only per-track elementary streams)
Completes the per-track-class trio with audio:// / sub://. video:// is
the demux path with a TrackKind::Video kind filter — each video track to
its own native elementary stream (.hevc/.h264/.vc1/.m2v/.obu), no audio
or subtitles. Reuses the existing kind_filter machinery and extension
map; scheme/parse/output/write-only-guard arms added symmetrically.
2026-07-18 19:24:59 -07:00
Matthew Jackson 42be09f628 changelog: 1.5.0 — extraction sinks + complete json:// model 2026-07-18 19:13:30 -07:00
Matthew Jackson 8bbbe43724 mux: json:// emits the complete title model (no dropped fields)
The json:// sink previously emitted only codec + language + pid per
stream. Fill it out to the full DiscTitle: video carries resolution
(+ pixel dims, interlaced), frame rate (+ fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channels (+ count),
sample rate (+ Hz), and editorial purpose; subtitles carry the
qualifier. Add the clip list and chapter names. This is everything the
scan resolved, so json:// is a lossless machine-readable view of a
title rather than a summary.
2026-07-18 18:14:11 -07:00
Matthew Jackson 95888f27a4 test: pin audio://sub://chapters://json:// in the scheme round-trip 2026-07-18 17:46:08 -07:00
Matthew Jackson 96dec65040 mux: chapters:// and json:// metadata sinks
Two write-only file sinks that ignore the PES stream and emit the title
metadata at construction (fvi-style, wired through output()):
- chapters:// — chapter markers in the format the extension picks: .xml
  (Matroska, default), .txt/.ogm (OGM simple), .vtt (WebVTT). Reuses the
  existing chapters_xml/chapters_ogm writers; adds a WebVTT writer.
- json:// — one title's model (playlist, duration, size, format, streams,
  chapters) as pretty JSON via serde_json.

New mux/meta_sink.rs; StreamUrl gains Chapters/Json; codec_label +
chapters_xml/ogm promoted to pub(crate) for reuse.

Tests: chapters_format_selected_by_extension, title_json_carries_streams_and_chapters.
2026-07-18 17:45:39 -07:00
Matthew Jackson 378fb226b7 mux: audio:// and sub:// sinks (demux filtered by track class)
Two extraction sinks built on the existing demux machinery: audio:// keeps
only audio tracks (native containers .thd/.dts/.ac3/.eac3/.pcm...), sub://
only subtitle tracks (PGS .sup, VobSub .idx+.sub, text .srt) — one file per
track, no video, no chapters sidecar. DemuxOptions gains a kind_filter; the
DemuxSink writers (already complete) are reused verbatim. StreamUrl gains
Audio/Sub variants; output() builds the filtered DemuxSink.

Test: kind_filter_keeps_only_the_selected_class.
2026-07-18 17:24:28 -07:00
Matthew Jackson 42caa2d90e restore freemkv-unlock path dep for local dev (post-v1.4.5) 2026-07-18 16:08:03 -07:00
Matthew Jackson 41a2493429 v1.4.5: bump version (freemkv-unlock git-pinned for the tag) 2026-07-18 16:08:01 -07:00
Matthew Jackson 0e73650440 changelog: 1.4.5 (FMTS clean single-variant mux + key-Debug redaction + hex fix) 2026-07-18 13:30:36 -07:00
Matthew Jackson bcfd5eb213 mux: FMTS read-plan on the inline live-drive path too (DiscStream::with_key_map)
The read_plan fix landed on the file-backed highway (build_iso_pipeline)
but the inline DiscStream (live single-pass) still passed the alternate
device-group half to the demux — a known FMTS bug on rip_mode="single".

DiscStream::with_key_map installs the proactive map and rewrites the
extent walk to the read plan, so the live single-pass path reads ONLY
our-phase units, exactly like the highway. DecryptingSectorSource gains
set_key_map for the already-constructed decorator. Non-forensic maps
return the extents unchanged, so every non-FMTS disc is byte-identical.

Test: with_key_map_reads_only_our_phase_units — a forensic Even segment
drops exactly its alternate units from the extent walk. Precommit green
on 1.86.
2026-07-18 13:27:48 -07:00
Matthew Jackson 8ce0a66bee mux: FMTS reads only our-phase units (AacsKeyMap::read_plan)
An FMTS forensic segment interleaves our device group's variant with a
foreign group's at the aligned-unit level. The mux was decrypting only
our phase but leaving the alternate (foreign) units in the buffer as
ciphertext, trusting the demux to 'drop untouched ciphertext cleanly'.
It doesn't: random 0x47 bytes at the 192-byte stride hit tracked PIDs,
mis-parse, and trip the demux's concealed-gap keyframe-resync — which
drops GOOD frames of ours around every segment (349 resyncs / ~6391
packets on Stand by Me, visible as playback flaws).

The map already knows which unit each LBA is and, for a forensic
segment, which phase is ours. AacsKeyMap::read_plan turns that into the
title's read plan: every default/CPS unit, plus inside a segment ONLY
our-phase units. The alternate units are never fetched, decrypted, or
handed to the demux — the demux sees one gapless our-variant stream.

- read_plan is general (single-CPS, multi-CPS, FMTS): a map with no
  Even/Odd range returns the extents unchanged, so DVD/CSS, single-CPS
  UHD and multi-CPS Blu-ray read byte-for-byte as before.
- Wired into build_iso_pipeline (the file-backed highway that muxes
  resumed ISOs). Producer re-anchors unit_base per extent, so per-unit
  segment reads stay unit-aligned and decrypt correctly.
- Extent gains PartialEq/Eq for the read_plan tests.

Tests: read_plan non-forensic unchanged; forensic omits exactly the
alternate units, kept units match the decrypt gate unit-for-unit. All
2314 lib tests pass; precommit green on 1.86.

Pending: end-to-end ISO re-mux validation (concealed gaps 349 -> ~0).
2026-07-18 12:06:20 -07:00
Matthew Jackson 7d4d9a3e84 remove user-facing English from the library (KeyOrigin::name, hddvd Title)
Library holds ZERO user-facing English (CLAUDE.md). Removed KeyOrigin::name()'s
English prose — apps map the typed enum (freemkv gets key_origin_label); diag
logs the enum's Debug repr. hddvd unnamed-title fallback 'Title N' -> neutral
TITLE_N identifier (UDF volume-label style).
2026-07-17 21:50:09 -07:00
Matthew Jackson 4dec71a867 narrow leaked-internal pub surface to pub(crate)
Verified zero callers across all consumer crates AND libfreemkv integration tests:
MUX_APP, Disc::{aacs_disc_hash,encrypted_content_ranges,inject_unit_keys},
locate_ranges, mapfile::{MapEntry,entries}, diag::dump_mkv_track,
DiscStream::{errors,lost_bytes} (read via accessors). Removed the unused mux
DemuxSink/FviSink crate-root re-exports (constructed internally by output() via
the direct module path). Staged demux option variants marked allow(dead_code).
2026-07-17 21:45:33 -07:00
Matthew Jackson a9b29c7060 disc: collapse decrypt_keys_for_title twin into one method
Removed the test-only unchecked wrapper and renamed decrypt_keys_for_title_checked
-> decrypt_keys_for_title (one method returning (keys, title_is_clear)). Production
already used the checked form; the _checked suffix had no counterpart. Callers +
docs updated; two tests take .0.
2026-07-17 21:41:10 -07:00
Matthew Jackson 0cb525517b css: remove redundant crack_key_halt wrapper
crack_key_halt had no caller except the crack_key wrapper — a needless middle
layer. crack_key now calls crack_key_scan directly; crack_key (Option) and
crack_key_outcome (full CrackOutcome + halt) remain as the two real entry points.
2026-07-17 21:34:37 -07:00
Matthew Jackson 5b4a3f32dc mux: collapse FviSink::create/create_with_source into one create
create_with_source (full SourceInfo) was only ever called by the create wrapper
with a partial SourceInfo — the extra capability was unused (YAGNI). Inlined the
constructor body into create and deleted the _with_source variant; callers
unchanged.
2026-07-17 21:32:39 -07:00
Matthew Jackson 66b0bc385c mux: collapse MkvStream::create/create_at into one create(.., Option<&Path>)
One method per action (CLAUDE.md), not a foo_with_X pair — create_at was the real
constructor and create was a None wrapper. Callers pass Option<&Path> directly.
2026-07-17 21:31:29 -07:00
Matthew Jackson aaccd83750 hex: canonical hex->integer parsers + public strip_hex_prefix
Adds parse_hex_u16/u32/u8 and exposes strip_hex_prefix so callers stop hand-rolling
from_str_radix(trim_start_matches("0x")) — a case-sensitive strip that this module
exists to prevent. disc::aacs_disc_hash now uses strip_hex_prefix.
2026-07-17 21:25:59 -07:00
Matthew Jackson 40f5b01a12 disc: redact Debug for AacsState/Key/HandshakeResult (test-guarded)
AacsState (public via Disc.aacs) and Key (the decrypt_with key-transport enum)
are crate-root re-exported and carried VUK/unit/read-data keys + volume id on
#[derive(Debug)]; HandshakeResult carried the VID + AACS 2.0 bus key. Manual
Debug impls print shape only, guarded by red->green tests.
2026-07-17 21:10:51 -07:00
Matthew Jackson 82d42b1930 aacs: redact Debug for ResolvedChain/ResolvedKeys/ProcessingKeyMatch (test-guarded)
These carry raw unit-key / VUK / processing-key bytes on their Debug; manual impls
print shape only (unit_keys_len, redacted markers). Each has a red→green test.
2026-07-17 21:03:31 -07:00
Matthew Jackson 913b01b79f css: redact CssState Debug (test-guarded)
CssState is reachable via the public Disc.css field; #[derive(Debug)] leaked the
raw CSS title key on any {:?} of a Disc. Manual Debug prints crack_span only.
2026-07-17 21:00:34 -07:00
Matthew Jackson af7ef59f3e aacs/types: redact Debug for all key-bearing types (test-guarded)
DeviceKey/HostCert/Vid/MediaKey/Vuk/ProcessingKey/UnitKey/DiscEntry carried key
material on #[derive(Debug)] — a stray {:?} would leak device/host-private/media/
volume/unit keys. Manual Debug impls print shape only. redaction_tests asserts no
key bytes appear and a redaction marker is present (fails if a derive returns).
2026-07-17 20:56:38 -07:00
Matthew Jackson ef8e11b6f3 libfreemkv: phase-aware FMTS decode + two-operation KeyFetch/KeySource seam
FMTS (AACS 2.1) now decodes per (LBA, phase): Phase enum + AacsKeyMap::
from_ranges_phased, decrypt only the variant's parity half. resolve_fmts_key_map
does a 2-phase index-1 anchor then per-index phase probe, and sizes the forensic
set to whatever the source returns (no hardcoded 32). KeyFetch is now two explicit
operations (unit_keys / fmts_indexes) and KeySource splits get_uk into
get_unit_keys + get_fmts_indexes. BYPASS_FMTS_KEY gate removed (first-class format).

Teed up for 1.4.5. Local WIP baseline.
2026-07-17 20:51:15 -07:00
Matthew Jackson e1f897fb32 restore freemkv-unlock path dep for local dev (post-v1.4.4) 2026-07-16 21:44:08 -07:00
45 changed files with 7307 additions and 474 deletions
+168
View File
@@ -1,5 +1,173 @@
# Changelog
## [1.5.0] — UNRELEASED
### Added
- **MP4 input (`mp4://` as a source)** — a progressive-MP4 demuxer, the inverse
of the muxer: it rebuilds the title + per-sample index from `moov`/`stbl` and
streams samples back as PES frames, so `mp4://` now flows to **any** sink
(`mp4:// mkv://`, `mp4:// audio://`, `mp4:// json://`, …). Round-trip is
frame-exact (verified `iso:// mp4://` → `mp4:// mkv://` preserves every video
and audio packet). Progressive MP4 only; fragmented (`moof`) is future work.
- **MP4 output is faststart by default** — `moov` is written *before* `mdat`, so
the file plays over HTTP without pre-fetching the end. Done with zero extra I/O:
a `moov`-sized hole (`round_up_4MB(16 B/sample × est_samples) + 4 MB`) is
reserved up front, so sample offsets are fixed and `moov` is dropped into the
hole at finish with a `free` box for the slack (falls back to moov-at-end only
if the estimate is blown).
- **Native MP4 output (`mp4://`)** — a progressive ISO-BMFF muxer
(`ftyp`+`mdat`+`moov`), so a disc goes straight to a play-everywhere `.mp4` in
one decrypt pass, no ffmpeg round-trip. Carries HEVC / H.264 video (HDR10 colour
via `colr`) and the audio MP4 supports — AC-3 (`dac3`), E-AC-3 (`dec3`), and
DTS / DTS-HD (`dtsc`/`dtsh` + `ddts`) — with correct sample tables and B-frame
composition offsets. It's a **compatibility export, not archival**: MP4 can't
hold TrueHD, LPCM, or bitmap (PGS/VobSub) subtitles, so those are **excluded
with a loud, itemized report — never a silent drop** (`mkv://` remains the
keep-everything path). Verified byte-for-byte equivalent to `ffmpeg -c copy`:
frame-exact video and audio (incl. a real DTS-HD MA 7.1 track, layout-exact),
identical colour and duration, clean decode.
- **Five extraction sinks — dissect a title, don't just rip it.** New write-only
destinations that pull one facet of a title out on its own:
- **`video://dir/`** — every video track to its own file, as a raw elementary
stream in the codec's native form (`.hevc`, `.h264`, `.vc1`, `.m2v`, `.obu`).
No audio, no subtitles. Completes the per-track-class trio with `audio://` /
`sub://`.
- **`audio://dir/`** — every audio track to its own file in a directory, each in
its native container (`.thd`, `.dts` / `.dtshd`, `.ac3`, `.eac3`, `.aac`,
`.flac`). No video, no subtitles. BD/DVD LPCM has no container of its own and is
written as headerless big-endian `.pcm`.
- **`sub://dir/`** — every subtitle track to its own file: PGS as `.sup`, VobSub
as a paired `.idx` + `.sub`, text subtitles as `.srt`. No video, no audio.
- **`chapters://file`** — a title's chapter markers as a single sidecar, in the
format the output extension selects: `.xml` (Matroska), `.txt` / `.ogm` (OGM
simple), or `.vtt` (WebVTT).
- **`json://file`** — one title's complete structure as JSON.
`audio://` and `sub://` are the demux path with a **kind filter** (only their
track class is opened and written). `chapters://` and `json://` are **scan-only**
— they read nothing of the elementary streams, so they return in seconds.
- **Drop-on-undecodable audio — keep everything decodable, drop the frames that
aren't.** A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch, and A/V sync is preserved: a drop becomes a silence
gap, never a shift of the following audio, and every drop is logged. Detection
is per-codec, each mirroring the format's own authoritative integrity check:
**DTS** (ffmpeg's core-header validity parse), **AC-3 / E-AC-3** (native frame
CRC-16), **FLAC** (whole-frame CRC-16), **MP2 / MP3** and **AAC-ADTS** (header
sanity — raw AAC passes through), and **TrueHD / MLP** (major-sync CRC-16 + AU
parity, verified against real ffmpeg output; because MLP state carries across
access units, a corrupt AU is dropped *forward* to the next major sync). LPCM
and video are excluded by design (no in-frame integrity data; inter-frame
prediction). A shared tally counts and logs drops and, for a track that is
mostly undecodable, drops the whole track. Detects only frames that are
*structurally* undecodable — corruption of the sample payload inside an
otherwise-valid frame (bad bit-allocation / XLL data) is source damage a
container remux cannot distinguish from good data.
- **Forced subtitle detection from PGS content.** A PGS subtitle track is flagged
`FlagForced` when it displays subtitles and every one carries the HDMV
`forced_on_flag` — a dedicated forced/narrative track — read from the stream
itself rather than the disc's vendor label metadata, so it works on discs that
carry no such blob. `info -v` reports the same verdict (one shared classifier
drives both the muxer and the scan-time probe), so `info` and a rip agree.
### Changed
- **`json://` emits the complete title model — lossless, not a summary.** Every
field the scan resolved is serialized: video carries resolution (+ pixel
dimensions, interlaced flag), frame rate (+ exact fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channel layout (+ count), sample
rate (+ Hz), language, and editorial purpose; subtitles carry the forced flag and
qualifier. The clip list and chapter names are included too.
### Fixed
- **TrueHD: a couple of transient errors can no longer poison a whole track.**
A corrupt access unit drops forward to the next major sync, but only the
individually-verified corruption feeds the whole-track drop verdict now; the
resync run is collateral — so two scattered errors no longer discard an
otherwise-good multi-hour track. The per-AU PTS rate is refined only from a
CRC-validated major sync (a corrupt one can't shift the resumed audio), and the
resync clears only on a validated major sync.
- **Free-format MPEG-audio** (`bitrate_index == 0`) is a legal, decodable mode and
is no longer dropped.
## [1.4.5] — 2026-07-18
### Fixed
- **FMTS (AACS 2.1) forensic discs now mux to a clean, single-variant stream.** A
forensic segment interleaves the local device group's variant with a foreign
group's at the aligned-unit level. The mux decrypted only our half but left the
foreign half in the buffer as ciphertext, on the assumption that the demuxer
"drops untouched ciphertext cleanly." It does not — a foreign unit's bytes hit a
tracked PID at the 192-byte stride, mis-parse, and trip the demux's concealed-gap
keyframe resync, which discards good frames of ours around every segment (visible
playback glitches). `AacsKeyMap::read_plan` now turns the map into the title's
read plan: every default / CPS unit, plus inside a forensic segment **only our
phase's units**. The foreign half is never read, decrypted, or handed to the
demux. On a retail 4K UHD title this took concealed-gap resyncs from **349 → 0**
and recovered ~2 GB of previously-dropped frames. Wired into **both** mux paths —
the file-backed highway (`build_iso_pipeline`) and the inline live-drive
`DiscStream` (`with_key_map`) — so single- and multi-pass FMTS rips are both clean.
### Changed
- **Key-bearing types redact their `Debug` output.** Every type that carries key
material (device keys, processing keys, unit keys, media keys, VUKs, resolved
chains, CSS/AACS state, …) now prints a `<redacted>` marker instead of the bytes,
so no key can reach a log or panic message. Each is covered by a test asserting no
key byte appears.
- **Hex parsing is centralized and case-insensitive.** A single set of canonical
`0x`/`0X`-tolerant hex→integer parsers replaces scattered ad-hoc parsing (this is
what silently dropped keydb device keys written with an uppercase `0X` prefix).
- **Internal-only public surface narrowed to `pub(crate)`, and duplicate
`foo_with_X` methods collapsed to one** — no behavioral change, smaller API.
## [1.4.4] — 2026-07-17
### Fixed
- **Online key requests are no longer silently dropped on discs that yield few
sample units.** The online key source refuses any request carrying fewer than
`MIN_SAMPLE_UNITS` (8) encrypted-content samples — too few can match an
incidental unit rather than the one asked about (a false positive, most acute on
AACS 2.1 forensic-variant content). autorip gathered only 4, so every online
lookup was skipped before it ever reached the key service and surfaced to the
user as "key service down." autorip's sample count is now tied to
`MIN_SAMPLE_UNITS` with a **compile-time floor**, so it can never regress below
the minimum again.
### Changed
- **The online request is assembled from a proven-sufficient sample set.** New
`DecodeSampleSet` (`libfreemkv::keysource`) wraps the content-unit samples and
can only be constructed with at least `MIN_SAMPLE_UNITS` of them — so an online
key request cannot be built from too few samples. The minimum is validated once,
at construction, rather than by a runtime check a caller could forget.
## [1.4.3] — 2026-07-17
### Changed
- **`MIN_SAMPLE_UNITS` moved to the base crate.** The minimum sample count an
online key request must carry now has a single definition in
`libfreemkv::keysource`; `freemkv-keysources` re-exports it, so the online source
and libfreemkv's own forensic query size their requests from one shared value.
- **The online unit-key reply is parsed as a list.** A response carries either a
single Unit Key (ordinary disc) or the full ordered set (an AACS 2.1
forensic-variant disc); the client accepts both and maps array position to
forensic index.
### Added
- **Forensic-variant online query samples the anchor segment.** On an AACS 2.1
forensic-variant disc the online key query draws its sample from the first
forensic segment (index 1) — one canonical, deterministic sample — instead of an
arbitrary segment.
## [1.4.2] — 2026-07-15
### Fixed
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "1.4.4"
version = "1.5.0"
edition = "2024"
rust-version = "1.86"
license = "MIT"
@@ -28,7 +28,7 @@ cbc = "0.1"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.4.4" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.5.0" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
+38 -1
View File
@@ -465,7 +465,7 @@ pub enum KeyCandidate {
/// its declared CPS-unit number); the caller runs
/// `decrypt_unit` + `is_clean_ts` to find which one actually opens the
/// disc. Rungs above the candidate are `None`.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct ResolvedChain {
pub unit_keys: Vec<(u32, [u8; 16])>,
pub vuk: Option<Vuk>,
@@ -475,6 +475,21 @@ pub struct ResolvedChain {
pub dk: Option<DeviceKey>,
}
// Redacting `Debug`: `unit_keys` holds raw title-key bytes, never printed. The
// other rungs are `types` newtypes that self-redact. Guarded by
// `resolved_chain_debug_is_redacted`.
impl std::fmt::Debug for ResolvedChain {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ResolvedChain")
.field("unit_keys_len", &self.unit_keys.len())
.field("vuk", &self.vuk)
.field("mk", &self.mk)
.field("pk", &self.pk)
.field("dk", &self.dk)
.finish()
}
}
/// Derive the full AACS key chain from a candidate key of ANY ladder rung.
///
/// Runs the deterministic derivation DOWNWARD to the disc's terminal unit keys:
@@ -568,6 +583,28 @@ mod resolve_candidate_tests {
use super::*;
use crate::aacs::crypto::aes_ecb_encrypt;
/// `ResolvedChain.unit_keys` holds raw title-key bytes (the other rungs are
/// self-redacting `types` newtypes). `Debug` must not leak the title keys.
#[test]
fn resolved_chain_debug_is_redacted() {
let c = ResolvedChain {
unit_keys: vec![(1, [0xD5; 16])],
vuk: None,
mk: None,
pk: None,
dk: None,
};
let dbg = format!("{c:?}");
assert!(
!dbg.contains("213"),
"ResolvedChain leaked unit keys: {dbg}"
);
assert!(
dbg.contains("unit_keys_len"),
"ResolvedChain missing redaction: {dbg}"
);
}
/// Minimal AACS-1.0 (48-byte stride) `Unit_Key_RO.inf` with `n` encrypted
/// unit keys — `parse_unit_key_ro` numbers CPS units 1..=n.
fn synth_inf(encs: &[[u8; 16]]) -> Vec<u8> {
+38 -1
View File
@@ -13,7 +13,6 @@ use super::mkb::*;
// ── Full VUK resolution chain ───────────────────────────────────────────────
/// Result of resolving a disc's VUK.
#[derive(Debug)]
pub struct ResolvedKeys {
/// Disc hash (SHA1 of Unit_Key_RO.inf)
pub disc_hash: [u8; 20],
@@ -34,6 +33,23 @@ pub struct ResolvedKeys {
pub key_source: u8,
}
// Redacting `Debug`: `vuk` and `unit_keys` are raw key bytes, never printed.
// `disc_hash` is the public per-disc identifier (SHA-1 of the .inf), not secret.
// Guarded by `resolved_keys_debug_is_redacted`.
impl std::fmt::Debug for ResolvedKeys {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ResolvedKeys")
.field("disc_hash", &self.disc_hash)
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.field("title_cps_unit", &self.title_cps_unit)
.field("version", &self.version)
.field("bus_encryption", &self.bus_encryption)
.field("key_source", &self.key_source)
.finish()
}
}
/// Inputs shared by every classical-path resolver. References only —
/// callers retain ownership of all buffers.
pub struct ResolveContext<'a> {
@@ -467,6 +483,27 @@ mod tests {
use super::super::types::*;
use super::*;
/// `ResolvedKeys` carries the disc's VUK and unit keys raw; `Debug` must not
/// leak them. Sentinel 213 (0xD5); non-secret fields are not 213.
#[test]
fn resolved_keys_debug_is_redacted() {
let rk = ResolvedKeys {
disc_hash: [0u8; 20],
vuk: Some([0xD5; 16]),
unit_keys: vec![(1, [0xD5; 16])],
title_cps_unit: vec![0],
version: AacsVersion::V21,
bus_encryption: true,
key_source: 1,
};
let dbg = format!("{rk:?}");
assert!(!dbg.contains("213"), "ResolvedKeys leaked keys: {dbg}");
assert!(
dbg.contains("redacted"),
"ResolvedKeys missing marker: {dbg}"
);
}
/// Audit #5: the `major` / `from_major` mapping is load-bearing for the
/// Unit_Key_RO stride, so pin it as a table. V10 ↔ BD; V20/V21 → UHD; any
/// non-BD major selects the V20/V21 64-byte stride (V10 is the only 48-byte).
-15
View File
@@ -41,21 +41,6 @@ pub const SEGMENT_RECORD_LEN: usize = 16;
/// Bytes per BDAV source packet (188-byte TS + 4-byte arrival-time header).
pub const SOURCE_PACKET_LEN: u64 = 192;
/// Whether a 2.1 (FMTS) disc may rip WITHOUT the forensic index keys.
///
/// `true` (today): the forensic segments are skipped as expected loss
/// and the bulk of the title decodes with the unit key, so a 2.1 disc rips
/// mostly-complete. A unit key (VUK) is still required, exactly as for any AACS
/// disc. `false`: the absence of a segment-key source is a hard, UPFRONT failure
/// ([`Error::FmtsKeyMissing`]) — the same policy as a missing unit key, so a
/// forensic-holed rip is refused rather than produced. No segment-key source
/// exists yet, so `true` is the only value under which a 2.1 disc rips at all;
/// flip to `false` once segment keys can be sourced and a partial rip should be
/// refused. Hardcoded on purpose — not a user setting.
///
/// [`Error::FmtsKeyMissing`]: crate::error::Error::FmtsKeyMissing
pub const BYPASS_FMTS_KEY: bool = false;
/// One forensic segment: the inclusive source-packet range it occupies in the
/// FMTS clip.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+163 -8
View File
@@ -6,7 +6,7 @@
//! owns only the crypto and these value types that flow through it.
/// A device key for MKB subset-difference tree processing.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DeviceKey {
pub key: [u8; 16],
pub node: u16,
@@ -15,7 +15,7 @@ pub struct DeviceKey {
}
/// Host certificate + private key for AACS SCSI authentication.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct HostCert {
/// AACS 1.0: 20 bytes. AACS 2.0: 32 bytes.
pub private_key: [u8; 20],
@@ -28,22 +28,22 @@ pub struct HostCert {
}
/// Volume ID (16 bytes) — read from the disc via the SCSI handshake / OEM path.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vid(pub [u8; 16]);
/// Media Key (Km, 16 bytes) — the MKB-scoped key derived from device keys.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct MediaKey(pub [u8; 16]);
/// Volume Unique Key (VUK / Kvu, 16 bytes) — derived from `MediaKey` + `Vid`,
/// decrypts the per-disc encrypted title keys in `Unit_Key_RO.inf`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Vuk(pub [u8; 16]);
/// Processing Key (Kp, 16 bytes) — an MKB Subset-Difference key that yields the
/// Media Key. A leaked/precomputed PK in the keydb, or the intermediate PK a
/// device-key walk derives at its matching SD node.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct ProcessingKey(pub [u8; 16]);
/// One decrypted per-CPS-unit AACS title key.
@@ -53,7 +53,7 @@ pub struct ProcessingKey(pub [u8; 16]);
/// area). The CPS-unit *number* association is a higher-level concern owned by
/// [`super::inf::parse_unit_key_ro`], which pairs each positional key with its
/// declared CPS unit; this primitive only does the AES, so it surfaces position.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct UnitKey {
pub idx: u32,
pub key: [u8; 16],
@@ -95,7 +95,7 @@ impl UnitKey {
}
/// A per-disc entry from the key database.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct DiscEntry {
/// Disc hash (20 bytes, hex)
pub disc_hash: String,
@@ -110,3 +110,158 @@ pub struct DiscEntry {
/// Unit keys (title keys) indexed by CPS unit number
pub unit_keys: Vec<(u32, [u8; 16])>,
}
// ── Redacting `Debug` impls ──────────────────────────────────────────────────
//
// Every type above carries AACS secret material (device keys, host PRIVATE keys,
// media/volume/processing/unit keys). `#[derive(Debug)]` would print those bytes
// verbatim, so a stray `debug!("{:?}", …)` or a panic message would leak the
// keys. These hand-written impls print only NON-secret shape (presence, lengths,
// tree coordinates, indices) — never key bytes. `decrypt::DecryptKeys` follows
// the same policy by omitting `Debug` entirely; here we keep `Debug` because
// these are `PartialEq`/`Eq` value types used in `assert_eq!` and nested inside
// other `#[derive(Debug)]` structs, so the trait must exist — just not leak.
// Guarded by `redaction_tests` below.
impl std::fmt::Debug for DeviceKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DeviceKey")
.field("key", &"<redacted>")
.field("node", &self.node)
.field("uv", &self.uv)
.field("u_mask_shift", &self.u_mask_shift)
.finish()
}
}
impl std::fmt::Debug for HostCert {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HostCert")
.field("private_key", &"<redacted>")
.field("certificate_len", &self.certificate.len())
.field("private_key_v2", &self.private_key_v2.map(|_| "<redacted>"))
.field(
"certificate_v2_len",
&self.certificate_v2.as_ref().map(|c| c.len()),
)
.finish()
}
}
impl std::fmt::Debug for Vid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vid(<redacted>)")
}
}
impl std::fmt::Debug for MediaKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("MediaKey(<redacted>)")
}
}
impl std::fmt::Debug for Vuk {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Vuk(<redacted>)")
}
}
impl std::fmt::Debug for ProcessingKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("ProcessingKey(<redacted>)")
}
}
impl std::fmt::Debug for UnitKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("UnitKey")
.field("idx", &self.idx)
.field("key", &"<redacted>")
.field("index_number", &self.index_number)
.finish()
}
}
impl std::fmt::Debug for DiscEntry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscEntry")
.field("disc_hash", &self.disc_hash)
.field("title", &self.title)
.field("media_key", &self.media_key.map(|_| "<redacted>"))
.field("disc_id", &self.disc_id.map(|_| "<redacted>"))
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.finish()
}
}
#[cfg(test)]
mod redaction_tests {
use super::*;
// Sentinel key byte 0xD5 = decimal 213. A derived `Debug` prints `[u8;N]`
// as decimal, so a leaked key surfaces the substring "213"; the redacting
// impls must not. No non-secret field below is 213, so "213" appearing means
// key bytes leaked. Each type must also carry a "redacted" marker (or omit
// the secret entirely) so re-adding `#[derive(Debug)]` fails this test.
const S: u8 = 0xD5;
fn assert_redacted(what: &str, dbg: &str) {
assert!(
!dbg.contains("213"),
"{what}: Debug leaked key bytes (found decimal 213): {dbg}"
);
assert!(
dbg.contains("redacted"),
"{what}: Debug missing redaction marker: {dbg}"
);
}
#[test]
fn device_key_debug_is_redacted() {
let d = DeviceKey {
key: [S; 16],
node: 1,
uv: 2,
u_mask_shift: 3,
};
assert_redacted("DeviceKey", &format!("{d:?}"));
}
#[test]
fn host_cert_debug_is_redacted() {
let h = HostCert {
private_key: [S; 20],
certificate: vec![0u8; 92],
private_key_v2: Some([S; 32]),
certificate_v2: None,
};
assert_redacted("HostCert", &format!("{h:?}"));
}
#[test]
fn newtype_keys_debug_is_redacted() {
assert_redacted("Vid", &format!("{:?}", Vid([S; 16])));
assert_redacted("MediaKey", &format!("{:?}", MediaKey([S; 16])));
assert_redacted("Vuk", &format!("{:?}", Vuk([S; 16])));
assert_redacted("ProcessingKey", &format!("{:?}", ProcessingKey([S; 16])));
}
#[test]
fn unit_key_debug_is_redacted() {
assert_redacted("UnitKey", &format!("{:?}", UnitKey::new(0, [S; 16])));
}
#[test]
fn disc_entry_debug_is_redacted() {
let e = DiscEntry {
disc_hash: "0xAA".into(),
title: "T".into(),
media_key: Some([S; 16]),
disc_id: Some([S; 16]),
vuk: Some([S; 16]),
unit_keys: vec![(1, [S; 16])],
};
assert_redacted("DiscEntry", &format!("{e:?}"));
}
}
+36 -1
View File
@@ -146,7 +146,7 @@ use super::derive::{calc_pk_from_dk, calc_v_mask};
/// Outcome of a subset-difference walk against an MKB. Carries the
/// processing key and the matching `uv` slot — both needed as inputs
/// to the variant chain.
#[derive(Debug, Clone, Copy)]
#[derive(Clone, Copy)]
pub struct ProcessingKeyMatch {
/// Processing Key.
pub kp: [u8; 16],
@@ -158,6 +158,20 @@ pub struct ProcessingKeyMatch {
pub cvalue_index: usize,
}
// Redacting `Debug`: `kp` (a Processing Key) and `cvalue` are secret, never
// printed. `uv` / `cvalue_index` are non-secret coordinates. Guarded by
// `processing_key_match_debug_is_redacted`.
impl std::fmt::Debug for ProcessingKeyMatch {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ProcessingKeyMatch")
.field("kp", &"<redacted>")
.field("uv", &self.uv)
.field("cvalue", &"<redacted>")
.field("cvalue_index", &self.cvalue_index)
.finish()
}
}
fn mkb_find_mk_dv(records: &[MkbRecord]) -> Option<[u8; 16]> {
let r = records.iter().find(|r| {
(r.rec_type == REC_VERIFY_MEDIA_KEY_V1 || r.rec_type == REC_VERIFY_MEDIA_KEY_V2)
@@ -635,6 +649,27 @@ mod tests {
use super::super::crypto::aesg3;
use super::super::derive::calc_pk_from_dk;
/// `ProcessingKeyMatch` carries the Processing Key (`kp`) and `cvalue` raw;
/// `Debug` must redact both. Non-secret `uv`/`cvalue_index` are not 213.
#[test]
fn processing_key_match_debug_is_redacted() {
let m = ProcessingKeyMatch {
kp: [0xD5; 16],
uv: 1,
cvalue: [0xD5; 16],
cvalue_index: 2,
};
let dbg = format!("{m:?}");
assert!(
!dbg.contains("213"),
"ProcessingKeyMatch leaked kp/cvalue: {dbg}"
);
assert!(
dbg.contains("redacted"),
"ProcessingKeyMatch missing marker: {dbg}"
);
}
#[test]
fn calc_pk_from_dk_terminates_on_nonconvergent_mask() {
// Regression for the unbounded-loop hang: pick a (dev_key_v_mask,
+46 -21
View File
@@ -29,7 +29,7 @@ use crate::sector::SectorSource;
const CSS_LOCKED_BAIL: u32 = 64;
/// CSS decryption state for a DVD title.
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct CssState {
/// 5-byte CSS title key (from SCSI auth or the crack fallback).
pub title_key: [u8; 5],
@@ -43,6 +43,18 @@ pub struct CssState {
pub crack_span: Option<(u32, u32)>,
}
// Redacting `Debug`: `CssState` is reachable via the public `Disc.css` field, so
// a `{:?}` on a `Disc` would otherwise print the raw CSS title key. Print only
// the (non-secret) crack span. Guarded by `css_state_debug_is_redacted`.
impl std::fmt::Debug for CssState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("CssState")
.field("title_key", &"<redacted>")
.field("crack_span", &self.crack_span)
.finish()
}
}
/// Recover the CSS title key with no keys, by scanning scrambled sectors and
/// running the Stevenson known-plaintext attack (see the [`stevenson`] module).
///
@@ -54,12 +66,16 @@ pub struct CssState {
/// extents and return the first sector that yields a key — no player keys, no
/// disc-key crack. Works on a live drive (after bus-auth unlocks reads) and on
/// disc images alike.
/// This convenience form runs to completion (no cancellation) and returns just
/// the key; callers needing an operator-Stop / watchdog cancel, or the three-way
/// [`CrackOutcome`] (to distinguish "unencrypted" from "encrypted-but-uncracked"),
/// use [`crack_key_outcome`], which takes a `halt` token.
pub fn crack_key(
reader: &mut dyn SectorSource,
extents: &[Extent],
batch_sectors: u16,
) -> Option<CssState> {
crack_key_halt(reader, extents, batch_sectors, None)
crack_key_scan(reader, extents, batch_sectors, None, false).into_state()
}
/// Outcome of a CSS crack scan that distinguishes the THREE cases the bare
@@ -105,6 +121,12 @@ impl CrackOutcome {
/// ScrambledUncracked) so callers can distinguish "genuinely unencrypted" from
/// "encrypted but uncrackable" — the latter must become a hard error, never a
/// silent fall-through to plaintext.
///
/// Takes an optional cooperative-cancellation token. "No silent hangs": the
/// crack scans up to 50_000 sectors, which on a live drive hitting bad sectors
/// can take a long time, so it polls `halt` once per batch (the same cadence
/// sweep/patch use) and emits a `freemkv::heartbeat` beat ("css_crack") each
/// batch so a stuck scan is visible in the log.
pub fn crack_key_outcome(
reader: &mut dyn SectorSource,
extents: &[Extent],
@@ -114,27 +136,10 @@ pub fn crack_key_outcome(
crack_key_scan(reader, extents, batch_sectors, halt, true)
}
/// [`crack_key`] with an optional cooperative-cancellation token.
///
/// "No silent hangs": the crack scans up to 50_000 sectors, which on a live
/// drive hitting bad sectors can take a long time. This variant polls `halt`
/// once per batch (the same cadence sweep/patch use) so an operator Stop or a
/// scan-level watchdog can interrupt the scan, and emits a
/// `freemkv::heartbeat` beat ("css_crack") each batch so a stuck scan is
/// visible in the log.
pub fn crack_key_halt(
reader: &mut dyn SectorSource,
extents: &[Extent],
batch_sectors: u16,
halt: Option<&crate::halt::Halt>,
) -> Option<CssState> {
crack_key_scan(reader, extents, batch_sectors, halt, false).into_state()
}
/// The crack scan, returning the full [`CrackOutcome`]. Tracks a
/// `saw_scrambled` flag so a scrambled-but-uncracked disc is distinguished
/// from a genuinely-unencrypted one (the [`crack_key`] / [`crack_key_halt`]
/// `Option` wrappers collapse both to `None`).
/// from a genuinely-unencrypted one (the [`crack_key`] `Option` wrapper
/// collapses both to `None` via [`CrackOutcome::into_state`]).
fn crack_key_scan(
reader: &mut dyn SectorSource,
extents: &[Extent],
@@ -364,6 +369,26 @@ mod tests {
use super::*;
use crate::error::{Error, Result};
/// `CssState` is reachable via the public `Disc.css` field, so a `{:?}` on a
/// `Disc` must not print the raw CSS title key. Sentinel byte 213 (0xD5);
/// `crack_span` is non-secret and none of its values are 213.
#[test]
fn css_state_debug_is_redacted() {
let s = CssState {
title_key: [0xD5; 5],
crack_span: Some((10, 20)),
};
let dbg = format!("{s:?}");
assert!(
!dbg.contains("213"),
"CssState Debug leaked the title key: {dbg}"
);
assert!(
dbg.contains("redacted"),
"CssState Debug missing marker: {dbg}"
);
}
// ── is_scrambled ───────────────────────────────────────────────────────
/// is_scrambled returns false for any buffer shorter than one sector,
+374 -25
View File
@@ -191,9 +191,23 @@ impl DecryptKeys {
/// sorted and disjoint. `default_idx` covers any LBA no range claims — the
/// single-CPS case is just an empty range list with `default_idx = 0`, so the
/// common disc pays zero lookup cost and needs no structural walk.
/// Which aligned units of a range a key decrypts. AACS 2.1 FMTS forensic segments
/// interleave TWO variants at the unit level; `Even`/`Odd` selects the variant's
/// half (parity of the unit's index within the segment) and the ALTERNATE half is
/// left untouched (ciphertext) for the muxer to drop. Every non-forensic range —
/// the base Unit Key, a multi-CPS unit — is `All` (decrypt every unit), so the
/// common disc is byte-for-byte unchanged.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Phase {
All,
Even,
Odd,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct AacsKeyMap {
ranges: Vec<(u32, u32, usize)>,
// (start_lba, end_lba, key_idx, phase)
ranges: Vec<(u32, u32, usize, Phase)>,
default_idx: usize,
}
@@ -207,43 +221,68 @@ impl AacsKeyMap {
}
}
/// Build from explicit `[start_lba, end_lba) → key_idx` ranges (multi-CPS /
/// FMTS). Ranges are sorted; `default_idx` answers any uncovered LBA.
pub fn from_ranges(mut ranges: Vec<(u32, u32, usize)>, default_idx: usize) -> Self {
ranges.sort_by_key(|&(start, _, _)| start);
/// Build from `[start_lba, end_lba) → key_idx` ranges that decrypt EVERY unit
/// (multi-CPS): each range is [`Phase::All`]. `default_idx` answers any
/// uncovered LBA. Byte-for-byte identical behaviour to before phases existed.
pub fn from_ranges(ranges: Vec<(u32, u32, usize)>, default_idx: usize) -> Self {
let phased = ranges
.into_iter()
.map(|(s, e, i)| (s, e, i, Phase::All))
.collect();
Self::from_ranges_phased(phased, default_idx)
}
/// Build a PHASE-AWARE map (FMTS): each range carries which unit-parity its key
/// opens ([`Phase::Even`]/[`Phase::Odd`] for a forensic segment, [`Phase::All`]
/// for base/CPS). Ranges are sorted; `default_idx` answers any uncovered LBA.
pub fn from_ranges_phased(
mut ranges: Vec<(u32, u32, usize, Phase)>,
default_idx: usize,
) -> Self {
ranges.sort_by_key(|&(start, _, _, _)| start);
Self {
ranges,
default_idx,
}
}
/// The unit-key index to decrypt the aligned unit at `lba` with. O(log n) —
/// the last range whose start is `<= lba` and whose end is `> lba`, else the
/// default. Cheap enough to call per aligned unit on the mux hot path.
pub fn key_idx_for(&self, lba: u32) -> usize {
/// The `(key_idx, phase, range_start_lba)` for the aligned unit at `lba`.
/// O(log n) — the last range whose start is `<= lba` and whose end is `> lba`,
/// else `(default_idx, All, 0)`. `range_start_lba` lets the mapped decrypt
/// compute a unit's parity WITHIN a forensic segment (for `Even`/`Odd`).
pub fn entry_for(&self, lba: u32) -> (usize, Phase, u32) {
if self.ranges.is_empty() {
return self.default_idx;
return (self.default_idx, Phase::All, 0);
}
match self
.ranges
.binary_search_by(|&(start, _, _)| start.cmp(&lba))
.binary_search_by(|&(start, _, _, _)| start.cmp(&lba))
{
Ok(i) => self.ranges[i].2,
Err(0) => self.default_idx,
Ok(i) => {
let (start, _, idx, ph) = self.ranges[i];
(idx, ph, start)
}
Err(0) => (self.default_idx, Phase::All, 0),
Err(i) => {
let (start, end, idx) = self.ranges[i - 1];
let (start, end, idx, ph) = self.ranges[i - 1];
if lba >= start && lba < end {
idx
(idx, ph, start)
} else {
self.default_idx
(self.default_idx, Phase::All, 0)
}
}
}
}
/// The `[start_lba, end_lba) → key_idx` ranges (sorted, disjoint). Empty for a
/// single-CPS map (everything uses [`default_idx`](Self::default_idx)).
pub fn ranges(&self) -> &[(u32, u32, usize)] {
/// The unit-key index for the aligned unit at `lba` (phase-agnostic; see
/// [`entry_for`](Self::entry_for) for the phase). Cheap per-unit hot-path call.
pub fn key_idx_for(&self, lba: u32) -> usize {
self.entry_for(lba).0
}
/// The `[start_lba, end_lba) → (key_idx, phase)` ranges (sorted, disjoint).
/// Empty for a single-CPS map (everything uses [`default_idx`](Self::default_idx)).
pub fn ranges(&self) -> &[(u32, u32, usize, Phase)] {
&self.ranges
}
@@ -256,12 +295,95 @@ impl AacsKeyMap {
/// title actually reaches. Used by the resolver to know which keys to secure
/// up front.
pub fn key_indices(&self) -> Vec<usize> {
let mut v: Vec<usize> = self.ranges.iter().map(|&(_, _, i)| i).collect();
let mut v: Vec<usize> = self.ranges.iter().map(|&(_, _, i, _)| i).collect();
v.push(self.default_idx);
v.sort_unstable();
v.dedup();
v
}
/// Build the FMTS **read plan**: the title's aligned units filtered down to
/// only the units this rip must actually read — every default / CPS unit,
/// plus, inside each forensic segment, ONLY our-phase ([`Phase::Even`] /
/// [`Phase::Odd`]) units. The alternate-phase units are a different device
/// group's variant: a licensed player never reads them, and neither do we.
/// They are omitted from the plan entirely, so they are never fetched,
/// decrypted, or handed to the demux — the demux therefore sees one gapless
/// our-variant stream, with no ciphertext to trip a concealed-gap resync (the
/// old behaviour that dropped good frames around every segment).
///
/// `extents` are the title's clip extents (unit-aligned in the interior;
/// a shorter tail is ordinary content and always kept). `unit_sectors` is the
/// AACS aligned-unit size in sectors (3). Contiguous kept units coalesce into
/// as few extents as possible so the producer still issues large sequential
/// reads across default content; only inside a ~480 KB forensic segment do
/// reads become unit-granular (every other unit). A map with no forensic
/// (Even/Odd) range returns `extents` unchanged — the common disc is not
/// touched.
///
/// The parity test is byte-identical to the decrypt hot loop
/// (`(unit_lba - range_start) / unit_sectors`), so a unit kept here is exactly
/// a unit [`decrypt_sectors_mapped`] would open, and vice-versa.
pub fn read_plan(
&self,
extents: &[crate::disc::Extent],
unit_sectors: u32,
) -> Vec<crate::disc::Extent> {
// No forensic segment → read everything, unchanged (byte-for-byte).
if !self
.ranges
.iter()
.any(|&(_, _, _, p)| matches!(p, Phase::Even | Phase::Odd))
{
return extents.to_vec();
}
let us = unit_sectors.max(1);
let mut plan: Vec<crate::disc::Extent> = Vec::new();
// Append `sectors` at `lba`, coalescing with the previous extent when they
// are physically contiguous so default runs stay one big sequential read.
let mut push = |lba: u32, sectors: u32| {
if sectors == 0 {
return;
}
if let Some(last) = plan.last_mut() {
if last.start_lba.saturating_add(last.sector_count) == lba {
last.sector_count += sectors;
return;
}
}
plan.push(crate::disc::Extent {
start_lba: lba,
sector_count: sectors,
});
};
for e in extents {
let mut off = 0u32;
while off < e.sector_count {
let lba = e.start_lba.saturating_add(off);
let remaining = e.sector_count - off;
if remaining < us {
// Extent tail shorter than a whole unit: ordinary content
// (nothing follows to desync), always read.
push(lba, remaining);
break;
}
let (_, phase, range_start) = self.entry_for(lba);
let keep = match phase {
Phase::All => true,
Phase::Even | Phase::Odd => {
let unit_ix = (lba - range_start) / us;
let is_odd = unit_ix % 2 == 1;
is_odd == matches!(phase, Phase::Odd)
}
};
if keep {
push(lba, us);
}
off += us;
}
}
plan
}
}
/// Decrypt a buffer of sectors in-place using a resolved [`AacsKeyMap`] — the
@@ -305,24 +427,46 @@ pub fn decrypt_sectors_mapped(
}
}
// Cheap safety net for the "map must be right" model: with a correct
// phase-aware map, every CORRECT-PHASE forensic unit decrypts to clean TS, so
// this never fires in the happy path — but a map bug (wrong phase/key for a
// segment) surfaces as a loud DecryptFailed instead of silent corruption. Only
// forensic (Even/Odd) ranges are verified; base / multi-CPS (All) stays
// trust-only, so the common disc is byte-for-byte unchanged.
let verify_failed = std::sync::atomic::AtomicBool::new(false);
let decrypt_one = |idx_in_buf: usize, chunk: &mut [u8]| {
if chunk.len() != unit_len {
return; // trailing partial unit: clear tail on disc, leave as-is
}
// Gate on the authoritative encrypted flag ONLY (the CPI bits in the clear
// seed) — no `is_clean`. A clear unit (flag unset) is left untouched; an
// encrypted unit is decrypted with its MAPPED key and trusted.
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
let (key_idx, phase, range_start) = map.entry_for(unit_lba);
// PHASE GATE (FMTS forensic segment): the segment interleaves two variants
// at the unit level. Decrypt ONLY our parity; leave the alternate half as
// ciphertext (the muxer drops untouched ciphertext cleanly — no garble).
if matches!(phase, Phase::Even | Phase::Odd) {
let unit_ix = (unit_lba - range_start) / unit_sectors;
let is_odd = unit_ix % 2 == 1;
if is_odd != matches!(phase, Phase::Odd) {
return; // alternate half — leave as-is
}
}
// Gate on the authoritative encrypted flag ONLY (CPI bits in the clear
// seed): a clear unit is left untouched; an encrypted unit is decrypted
// with its MAPPED key and trusted.
if !aacs::content::aacs_unit_encrypted(chunk, format) {
return;
}
let unit_lba = base_lba.saturating_add((idx_in_buf as u32) * unit_sectors);
let key_idx = map.key_idx_for(unit_lba);
// Bounds already proven above; index directly.
let key = &unit_keys[key_idx].1;
if let Some(ref rdk_key) = rdk {
aacs::content::decrypt_bus(chunk, rdk_key);
}
aacs::content::decrypt_unit(chunk, key);
// Correct-phase forensic verify (silent unless the map is wrong).
if matches!(phase, Phase::Even | Phase::Odd) && !aacs::content::is_clean(chunk, format) {
verify_failed.store(true, std::sync::atomic::Ordering::Relaxed);
}
};
let nthreads = decrypt_threads();
@@ -345,6 +489,9 @@ pub fn decrypt_sectors_mapped(
}
}
}
if verify_failed.load(std::sync::atomic::Ordering::Relaxed) {
return Err(crate::error::Error::DecryptFailed);
}
Ok(())
}
@@ -1476,6 +1623,208 @@ mod tests {
unit
}
// ── FMTS phase-aware map ──────────────────────────────────────────────────
/// `entry_for` returns (idx, phase, range_start); `from_ranges` is All,
/// `from_ranges_phased` carries the phase; uncovered → (default, All, 0).
#[test]
fn aacskeymap_phase_entry_for() {
let all = AacsKeyMap::from_ranges(vec![(100, 200, 3)], 0);
assert_eq!(all.entry_for(150), (3, Phase::All, 100));
assert_eq!(all.entry_for(50), (0, Phase::All, 0));
let phased = AacsKeyMap::from_ranges_phased(vec![(100, 200, 3, Phase::Odd)], 7);
assert_eq!(phased.entry_for(150), (3, Phase::Odd, 100));
assert_eq!(phased.entry_for(250), (7, Phase::All, 0));
assert_eq!(phased.key_idx_for(150), 3);
}
/// A map with no forensic (Even/Odd) range is the common disc: `read_plan`
/// returns the extents unchanged, so nothing but FMTS is affected.
#[test]
fn read_plan_non_forensic_is_unchanged() {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
let ext = vec![
Extent {
start_lba: 1000,
sector_count: 300,
},
Extent {
start_lba: 5000,
sector_count: 60,
},
];
// Single-CPS and multi-CPS (All) maps both leave the plan untouched.
assert_eq!(AacsKeyMap::single(0).read_plan(&ext, us), ext);
let multi = AacsKeyMap::from_ranges(vec![(1000, 1150, 2)], 0);
assert_eq!(multi.read_plan(&ext, us), ext);
}
/// FMTS: a forensic Even segment drops exactly its alternate (odd) units from
/// the read plan — they are never fetched — while default content on either
/// side stays one coalesced sequential run. The kept units are byte-identical
/// to the ones the decrypt hot loop opens.
#[test]
fn read_plan_forensic_reads_only_our_phase_units() {
use crate::disc::Extent;
let us = (aacs::content::ALIGNED_UNIT_LEN / 2048) as u32; // 3
// One extent, 100 units [1000, 1300). A 10-unit Even forensic segment at
// LBA [1030, 1060): kept even units are ix 0,2,4,6,8 → LBA 1030,1036,1042,
// 1048,1054; dropped odd units → 1033,1039,1045,1051,1057.
let ext = vec![Extent {
start_lba: 1000,
sector_count: 300,
}];
let map = AacsKeyMap::from_ranges_phased(vec![(1030, 1060, 5, Phase::Even)], 0);
let plan = map.read_plan(&ext, us);
let expected = vec![
Extent {
start_lba: 1000,
sector_count: 33,
}, // 1000..1030 default + the ix-0 even unit at 1030
Extent {
start_lba: 1036,
sector_count: 3,
},
Extent {
start_lba: 1042,
sector_count: 3,
},
Extent {
start_lba: 1048,
sector_count: 3,
},
Extent {
start_lba: 1054,
sector_count: 3,
},
Extent {
start_lba: 1060,
sector_count: 240,
}, // default resumes, coalesced to the extent end
];
assert_eq!(plan, expected);
// Exactly the 5 odd units (15 sectors) are omitted; nothing else.
let kept: u32 = plan.iter().map(|e| e.sector_count).sum();
assert_eq!(
kept,
300 - 5 * us,
"only the alternate-phase units are dropped"
);
// Every kept LBA is one the decrypt loop would decrypt (All or our parity),
// and no dropped LBA is: the plan and the decrypt gate agree unit-for-unit.
for e in &plan {
let mut off = 0;
while off < e.sector_count {
let lba = e.start_lba + off;
let (_, phase, rs) = map.entry_for(lba);
if let Phase::Even | Phase::Odd = phase {
let is_odd = ((lba - rs) / us) % 2 == 1;
assert!(
is_odd == matches!(phase, Phase::Odd),
"plan kept an alternate-phase unit at LBA {lba}"
);
}
off += us;
}
}
}
/// Phase::Even → only even-index units in the range are decrypted; the odd
/// (alternate variant) half is left BYTE-FOR-BYTE as ciphertext for the muxer.
#[test]
fn mapped_phase_even_decrypts_even_leaves_odd_ciphertext() {
use crate::disc::ContentFormat;
let key_a = [0xAAu8; 16];
let key_b = [0xBBu8; 16];
let ul = aacs::content::ALIGNED_UNIT_LEN;
let usz = (ul / 2048) as u32;
let mut buf = vec![0u8; 8 * ul];
let mut odd_cipher = Vec::new();
for i in 0..8 {
let mut u = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u, if i % 2 == 0 { &key_a } else { &key_b });
if i % 2 == 1 {
odd_cipher.push(u.clone());
}
buf[i * ul..(i + 1) * ul].copy_from_slice(&u);
}
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, key_a)],
read_data_key: None,
format: ContentFormat::BdTs,
};
let map = AacsKeyMap::from_ranges_phased(vec![(0, 8 * usz, 0, Phase::Even)], 0);
decrypt_sectors_mapped(&mut buf, &keys, 0, &map).expect("even phase decrypts clean");
for i in 0..8 {
let u = &buf[i * ul..(i + 1) * ul];
if i % 2 == 0 {
assert!(
aacs::content::is_clean(u, ContentFormat::BdTs),
"even unit {i} decrypted to clean TS"
);
} else {
assert_eq!(
u,
odd_cipher[i / 2].as_slice(),
"odd unit {i} left as ciphertext"
);
}
}
}
/// The correct-phase safety `is_clean` fires loud: an even unit whose mapped
/// key is wrong does NOT come clean → `DecryptFailed` (not silent corruption).
#[test]
fn mapped_phase_verify_fails_loud_on_wrong_key() {
use crate::disc::ContentFormat;
let ul = aacs::content::ALIGNED_UNIT_LEN;
let usz = (ul / 2048) as u32;
let mut buf = vec![0u8; 2 * ul];
let mut u0 = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u0, &[0xAAu8; 16]); // encrypted under A
buf[..ul].copy_from_slice(&u0);
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, [0xCCu8; 16])], // map slot points at the WRONG key
read_data_key: None,
format: ContentFormat::BdTs,
};
let map = AacsKeyMap::from_ranges_phased(vec![(0, 2 * usz, 0, Phase::Even)], 0);
assert!(matches!(
decrypt_sectors_mapped(&mut buf, &keys, 0, &map),
Err(crate::error::Error::DecryptFailed)
));
}
/// Phase::All (multi-CPS / base) decrypts EVERY unit and never runs the verify
/// — the common-disc path is byte-for-byte unchanged.
#[test]
fn mapped_all_phase_decrypts_every_unit() {
use crate::disc::ContentFormat;
let key = [0x11u8; 16];
let ul = aacs::content::ALIGNED_UNIT_LEN;
let mut buf = vec![0u8; 4 * ul];
for i in 0..4 {
let mut u = clear_ts_unit();
aacs_encrypt_unit_for_test(&mut u, &key);
buf[i * ul..(i + 1) * ul].copy_from_slice(&u);
}
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, key)],
read_data_key: None,
format: ContentFormat::BdTs,
};
decrypt_sectors_mapped(&mut buf, &keys, 0, &AacsKeyMap::single(0))
.expect("all-phase decrypts");
for i in 0..4 {
assert!(
aacs::content::is_clean(&buf[i * ul..(i + 1) * ul], ContentFormat::BdTs),
"unit {i} decrypted (All)"
);
}
}
/// A unit encrypted under unit_keys[1] (the second CPS unit) on a
/// two-key disc must be correctly decrypted — not left as garbage —
/// when `decrypt_sectors` is called with unit_key_idx=0 (the default).
+2 -2
View File
@@ -336,7 +336,7 @@ fn frame_record(track_idx: usize, pts_ns: i64, keyframe: bool, data: &[u8]) -> V
/// `track_number` is the 1-based MKV track number; `track` is the built
/// [`crate::mux::mkv::MkvTrack`] whose fields map one-to-one onto the emitted
/// elements (see `MkvMuxer::new`). No-op unless the diag target is on.
pub fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
pub(crate) fn dump_mkv_track(track_number: u64, track: &crate::mux::mkv::MkvTrack) {
if !diag_enabled() {
return;
}
@@ -530,7 +530,7 @@ fn dump_aacs(disc: &Disc) {
a.bus_encryption,
a.mkb_version,
a.disc_hash,
a.key_source.name(),
a.key_source,
a.vuk.is_some(),
a.unit_keys.len(),
a.uk_ro.len(),
+35 -2
View File
@@ -7,7 +7,6 @@ use crate::udf;
/// Result of SCSI AACS handshake (ECDH authentication).
/// Only available when scanning from a real drive, not ISO images.
#[derive(Debug)]
pub(super) struct HandshakeResult {
pub volume_id: [u8; 16],
pub read_data_key: Option<[u8; 16]>,
@@ -29,6 +28,19 @@ pub(super) struct HandshakeResult {
pub drive_unlocked: bool,
}
// Redacting `Debug`: `volume_id` and `read_data_key` (the AACS 2.0 bus key) are
// secret; print only shape. Guarded by `handshake_result_debug_is_redacted`.
impl std::fmt::Debug for HandshakeResult {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("HandshakeResult")
.field("volume_id", &"<redacted>")
.field("read_data_key", &self.read_data_key.map(|_| "<redacted>"))
.field("read_data_key_err", &self.read_data_key_err)
.field("drive_unlocked", &self.drive_unlocked)
.finish()
}
}
/// Single source of truth for "is AACS bus encryption gone for this scan?". The
/// gate asks ONLY this — `if !removed { error }` — never enumerating cases. Bus
/// encryption is gone when ANY of these holds:
@@ -428,6 +440,27 @@ mod tests {
use crate::sector::SectorSource;
use std::collections::HashMap;
/// `HandshakeResult` carries the Volume ID and the AACS 2.0 bus (read-data)
/// key; `Debug` must redact both. Sentinel 213 (0xD5).
#[test]
fn handshake_result_debug_is_redacted() {
let hs = HandshakeResult {
volume_id: [0xD5; 16],
read_data_key: Some([0xD5; 16]),
read_data_key_err: None,
drive_unlocked: false,
};
let d = format!("{hs:?}");
assert!(
!d.contains("213"),
"HandshakeResult leaked VID/bus key: {d}"
);
assert!(
d.contains("redacted"),
"HandshakeResult missing marker: {d}"
);
}
// ---------------------------------------------------------------
// In-memory disc + minimal UDF image with a single physical
// partition (metadata_start == partition_start). Offsets cited
@@ -976,7 +1009,7 @@ mod tests {
/// A minimal in-test KeySource that yields no keys but a fixed cert list.
struct CertSource(Vec<aacs::types::HostCert>);
impl crate::KeySource for CertSource {
fn get_uk(
fn get_unit_keys(
&self,
_ctx: &dyn crate::keysource::ResolveCtx,
) -> Result<Vec<crate::aacs::types::UnitKey>> {
+3 -1
View File
@@ -522,8 +522,10 @@ fn compose_xpl_titles(
})
.collect();
titles.push(DiscTitle {
// Language-neutral identifier (no user-facing English in the library):
// matches the UDF `TITLE_*` volume-label style. Apps localize display.
playlist: if t.name.is_empty() {
format!("Title {}", t.number)
format!("TITLE_{}", t.number)
} else {
t.name.clone()
},
+2 -2
View File
@@ -77,7 +77,7 @@ impl SectorStatus {
/// One contiguous range of bytes with a status.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MapEntry {
pub(crate) struct MapEntry {
pub pos: u64,
pub size: u64,
pub status: SectorStatus,
@@ -498,7 +498,7 @@ impl Mapfile {
/// All map entries, sorted ascending by `pos` and (after load)
/// guaranteed disjoint and non-overflowing.
pub fn entries(&self) -> &[MapEntry] {
pub(crate) fn entries(&self) -> &[MapEntry] {
&self.entries
}
+118 -43
View File
@@ -16,6 +16,7 @@ mod extract;
mod hddvd;
pub mod mapfile;
mod patch;
pub(crate) mod pgs_forced_probe;
pub mod read_error;
mod section_recover;
mod sweep;
@@ -457,7 +458,7 @@ pub(crate) fn chapter_name(i: usize) -> String {
}
/// A contiguous range of sectors on disc.
#[derive(Debug, Clone, Copy)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Extent {
pub start_lba: u32,
pub sector_count: u32,
@@ -683,7 +684,10 @@ fn range_chapter(lba: u32, title: &DiscTitle) -> (Option<u32>, Option<f64>) {
/// This is the single place range→chapter/time annotation happens; autorip used
/// to own it and read the mapfile to do so. Now the library computes it from
/// its in-memory mapfile + title, and the client renders the result verbatim.
pub fn locate_ranges(raw: &[(u64, u64)], title: &DiscTitle) -> crate::progress::LocatedProgress {
pub(crate) fn locate_ranges(
raw: &[(u64, u64)],
title: &DiscTitle,
) -> crate::progress::LocatedProgress {
use crate::consts::{MILLIS_PER_SEC, SECTOR_BYTES_U64};
use crate::progress::{LocatedProgress, LocatedRange};
const MAX_LOCATED: usize = 50;
@@ -1363,7 +1367,6 @@ impl DiscTitle {
// ─── Encryption ─────────────────────────────────────────────────────────────
/// AACS decryption state for a disc.
#[derive(Debug)]
pub struct AacsState {
/// AACS version (1 or 2)
pub version: u8,
@@ -1394,6 +1397,28 @@ pub struct AacsState {
pub mkb: Vec<u8>,
}
// Redacting `Debug`: `AacsState` is crate-root re-exported and reachable via the
// public `Disc.aacs` field; it carries VUK / unit keys / read-data (bus) key /
// volume id / raw .inf + MKB. Print only non-secret shape; redact every
// key/secret field. Guarded by `aacs_state_and_key_debug_are_redacted`.
impl std::fmt::Debug for AacsState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("AacsState")
.field("version", &self.version)
.field("bus_encryption", &self.bus_encryption)
.field("mkb_version", &self.mkb_version)
.field("disc_hash", &self.disc_hash)
.field("key_source", &self.key_source)
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.field("read_data_key", &self.read_data_key.map(|_| "<redacted>"))
.field("volume_id", &"<redacted>")
.field("uk_ro_len", &self.uk_ro.len())
.field("mkb_len", &self.mkb.len())
.finish()
}
}
/// How AACS keys were resolved. Variants are ordered root-of-trust →
/// per-disc-leaf, matching the resolver's path-try order: the resolver
/// attempts derivation from the strongest input it has first and falls
@@ -1416,18 +1441,9 @@ pub enum KeyOrigin {
ExternalUk,
}
impl KeyOrigin {
pub fn name(&self) -> &'static str {
match self {
KeyOrigin::DeviceKey => "MKB + device key",
KeyOrigin::ProcessingKey => "MKB + processing key",
KeyOrigin::KeyDbDerived => "KEYDB (derived)",
KeyOrigin::KeyDb => "KEYDB",
KeyOrigin::KeyDbUnitKeys => "KEYDB (unit keys)",
KeyOrigin::ExternalUk => "external UK",
}
}
}
// No `KeyOrigin::name()`: the library holds ZERO user-facing English (CLAUDE.md).
// `KeyOrigin` is a typed enum; applications map its variants to display text
// (see freemkv's `disc_info::key_origin_label`).
// ─── Disc scanning ──────────────────────────────────────────────────────────
@@ -1476,6 +1492,13 @@ pub struct ScanOptions {
/// 50_000 sectors on a live DVD) poll it and bail out cleanly so a
/// scan-phase watchdog or operator Stop is never stuck behind a hang.
pub halt: Option<crate::halt::Halt>,
/// Read the PGS subtitle streams during the scan to detect forced-narrative
/// tracks from their content (the `forced_on_flag`), matching what the mux
/// derives during a rip. OFF by default — it reads the clip's PGS content,
/// which is slow, so only callers that want authoritative forced flags in the
/// scanned title (e.g. `freemkv info`) opt in. The rip path leaves it off:
/// the muxer detects forced during muxing without a second read.
pub probe_forced_subtitles: bool,
}
/// Quick disc identification — name, format, capacity. No title/stream parsing.
@@ -1968,6 +1991,18 @@ impl Disc {
// 4. Metadata + labels
let meta_title = Self::read_meta_title(reader, &udf_fs);
crate::labels::apply(reader, &udf_fs, &mut titles);
// Optional content-based forced-subtitle detection. `info` opts in so its
// forced flags match what the muxer derives during a rip (both use the
// one shared PGS classifier); the rip path leaves it off — the muxer
// detects forced while muxing, without a second read of the clip.
if _opts.probe_forced_subtitles {
for title in &mut titles {
if title.content_format == ContentFormat::BdTs {
pgs_forced_probe::probe_and_set_forced(reader, title);
}
}
}
crate::labels::fill_defaults(&mut titles);
// 5. Format (AACS MKB generation → BD/UHD/FMTS; tree → HD-DVD/DVD) and
@@ -2244,7 +2279,7 @@ impl Disc {
/// point at one level of that chain; [`Disc::decrypt_with`] derives down from
/// it to the per-CPS unit keys. New levels can be added without breaking
/// callers.
#[derive(Debug, Clone)]
#[derive(Clone)]
#[non_exhaustive]
pub enum Key {
/// Device key(s) (AACS DK, positioned). libfreemkv walks the MKB
@@ -2273,6 +2308,22 @@ pub enum Key {
Unit(Vec<(u32, [u8; 16])>),
}
// Redacting `Debug`: `Key` is crate-root re-exported and is the key-transport
// type crossing `Disc::decrypt_with`; every variant carries raw key material.
// Print only the variant name and count — never bytes. Guarded by
// `aacs_state_and_key_debug_are_redacted`.
impl std::fmt::Debug for Key {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Key::Device(v) => write!(f, "Key::Device(<{} redacted>)", v.len()),
Key::Processing(v) => write!(f, "Key::Processing(<{} redacted>)", v.len()),
Key::Media(v) => write!(f, "Key::Media(<{} redacted>)", v.len()),
Key::Volume(_) => f.write_str("Key::Volume(<redacted>)"),
Key::Unit(v) => write!(f, "Key::Unit(<{} redacted>)", v.len()),
}
}
}
/// True if `unit_keys` covers EVERY supplied scrambled content `sample` — the
/// validation gate for [`Disc::decrypt_with`]. Conservative: a sample that is
/// not AACS-scrambled proves nothing, and with no scrambled sample at all there
@@ -2387,7 +2438,7 @@ impl Disc {
///
/// Empty when the disc has no parsed titles (CSS / unencrypted / unscanned);
/// callers treat an empty map as "no content gate" and fall back accordingly.
pub fn encrypted_content_ranges(&self) -> Vec<(u32, u32)> {
pub(crate) fn encrypted_content_ranges(&self) -> Vec<(u32, u32)> {
merged_extents(self.titles.iter().flat_map(|t| &t.extents))
}
@@ -2395,10 +2446,10 @@ impl Disc {
/// empty when this disc has no captured AACS state. Used to name the disc in
/// a [`Error::NoDiscKey`] so the application can tell the user which disc to
/// add to the keydb.
pub fn aacs_disc_hash(&self) -> String {
pub(crate) fn aacs_disc_hash(&self) -> String {
self.aacs
.as_ref()
.map(|a| a.disc_hash.trim_start_matches("0x").to_string())
.map(|a| crate::hex::strip_hex_prefix(&a.disc_hash).to_string())
.unwrap_or_default()
}
@@ -2535,35 +2586,25 @@ impl Disc {
///
/// `batch_sectors` sizes the crack's batched reads (file-safe value for
/// an ISO; `detect_max_batch_sectors` for a live drive).
pub fn decrypt_keys_for_title(
&self,
idx: usize,
reader: &mut dyn SectorSource,
batch_sectors: u16,
) -> crate::decrypt::DecryptKeys {
self.decrypt_keys_for_title_checked(idx, reader, batch_sectors)
.0
}
/// [`Self::decrypt_keys_for_title`] plus the per-title encryption verdict the
/// gate needs to AVOID A FALSE ERROR on a genuinely-clear extra title.
/// Also returns the per-title encryption verdict the gate needs to AVOID A
/// FALSE ERROR on a genuinely-clear extra title.
///
/// On a multi-VTS CSS DVD that ALSO carries a clear, unencrypted stub title
/// (a 0.5 s menu loop, an FBI-warning nav title) living in its own VTS, the
/// re-crack over that stub's extents finds NO scrambled sector and recovers
/// no key. The bare `decrypt_keys_for_title` collapses that to
/// no key. Returning ONLY the keys would collapse that to
/// `DecryptKeys::None`, indistinguishable from "scrambled but uncrackable",
/// so [`Self::ensure_decryptable_keys`] (which fails whenever `css.is_some()`
/// and the key is `None`) wrongly raised `E7023` for a title that needs no
/// key at all. That is the false error the multi-title mux must never emit.
/// and the key is `None`) would wrongly raise `E7023` for a title that needs
/// no key at all — the false error the multi-title mux must never emit.
///
/// This variant runs the re-crack via [`crate::css::crack_key_outcome`] and
/// So the re-crack runs via [`crate::css::crack_key_outcome`] and ALSO
/// returns `title_is_clear == true` when the title's own extents showed NO
/// scrambling (`CrackOutcome::Unencrypted`) — the gate then treats that title
/// as needing no key and passes it cleanly. A title that genuinely IS
/// scrambled but uncrackable returns `(None, false)` and still hard-fails.
/// The returned bool pairs with [`Self::ensure_title_decryptable`].
pub fn decrypt_keys_for_title_checked(
pub fn decrypt_keys_for_title(
&self,
idx: usize,
reader: &mut dyn SectorSource,
@@ -2627,7 +2668,7 @@ impl Disc {
}
/// Per-title decrypt gate that honours the `title_is_clear` verdict from
/// [`Self::decrypt_keys_for_title_checked`].
/// [`Self::decrypt_keys_for_title`].
///
/// Identical to [`Self::ensure_decryptable_keys`] EXCEPT it does not raise
/// `E7023` when the chosen title proved genuinely clear (`title_is_clear`):
@@ -2672,7 +2713,7 @@ impl Disc {
/// file-backed mux. Without this, a keyed disc swept without a keydb would
/// recover its UK yet still report E8005 (no usable `decrypt_keys`) at
/// remux. No-op for an unencrypted or CSS (DVD) disc.
pub fn inject_unit_keys(&mut self, keys: Vec<(u32, [u8; 16])>) {
pub(crate) fn inject_unit_keys(&mut self, keys: Vec<(u32, [u8; 16])>) {
if let Some(aacs) = self.aacs.as_mut() {
aacs.unit_keys = keys;
aacs.key_source = KeyOrigin::ExternalUk;
@@ -4188,6 +4229,40 @@ pub fn detect_max_batch_sectors(device_path: &str) -> u16 {
mod tests {
use super::*;
/// `AacsState` (public via `Disc.aacs`) and `Key` (the key-transport enum)
/// must never print raw key bytes on `{:?}`. Sentinel 213 (0xD5); non-secret
/// fields below are not 213.
#[test]
fn aacs_state_and_key_debug_are_redacted() {
let st = AacsState {
version: 2,
bus_encryption: true,
mkb_version: Some(77),
disc_hash: "0xAA".into(),
key_source: KeyOrigin::ExternalUk,
vuk: Some([0xD5; 16]),
unit_keys: vec![(1, [0xD5; 16])],
read_data_key: Some([0xD5; 16]),
volume_id: [0xD5; 16],
uk_ro: vec![1, 2, 3],
mkb: vec![4, 5, 6],
};
let d = format!("{st:?}");
assert!(!d.contains("213"), "AacsState leaked key bytes: {d}");
assert!(d.contains("redacted"), "AacsState missing marker: {d}");
for k in [
Key::Unit(vec![(1, [0xD5; 16])]),
Key::Volume([0xD5; 16]),
Key::Processing(vec![[0xD5; 16]]),
Key::Media(vec![[0xD5; 16]]),
] {
let d = format!("{k:?}");
assert!(!d.contains("213"), "Key leaked bytes: {d}");
assert!(d.contains("redacted"), "Key missing marker: {d}");
}
}
// ── encrypted-content map (`merged_extents` core) ────────────────────────
fn ext(start_lba: u32, sector_count: u32) -> Extent {
@@ -5063,8 +5138,8 @@ mod tests {
// ── Fix 2/3: a genuinely-clear extra title on a CSS disc never E7023s ──────
/// Reader that serves clear (unscrambled) sectors for one extent range and
/// CSS-locked errors elsewhere — enough to drive `decrypt_keys_for_title_
/// checked`'s per-title re-crack to `Unencrypted` for a clear stub.
/// CSS-locked errors elsewhere — enough to drive `decrypt_keys_for_title`'s
/// per-title re-crack to `Unencrypted` for a clear stub.
struct ClearStubReader {
clear_range: (u32, u32),
}
@@ -5124,7 +5199,7 @@ mod tests {
let mut reader = ClearStubReader {
clear_range: (0, 100_000),
};
let (keys, title_is_clear) = disc.decrypt_keys_for_title_checked(stub_idx, &mut reader, 8);
let (keys, title_is_clear) = disc.decrypt_keys_for_title(stub_idx, &mut reader, 8);
assert!(
!keys.is_encrypted(),
"a clear stub needs no key (got encrypted keys)"
@@ -5546,7 +5621,7 @@ mod tests {
let mut src = RecordingSource {
reads: std::cell::RefCell::new(Vec::new()),
};
match disc.decrypt_keys_for_title(0, &mut src, 16) {
match disc.decrypt_keys_for_title(0, &mut src, 16).0 {
crate::decrypt::DecryptKeys::Css { title_key } => {
assert_eq!(title_key, [0xAB; 5], "same-VTS title reuses cracked key");
}
@@ -5571,7 +5646,7 @@ mod tests {
let mut src = RecordingSource {
reads: std::cell::RefCell::new(Vec::new()),
};
let keys = disc.decrypt_keys_for_title(1, &mut src, 16);
let keys = disc.decrypt_keys_for_title(1, &mut src, 16).0;
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"a re-crack miss in a provably-different VTS must be a hard failure (None), \
+196
View File
@@ -0,0 +1,196 @@
//! Content-based forced-subtitle detection for Blu-ray/UHD PGS tracks.
//!
//! `freemkv info` and the muxer must agree on which subtitle tracks are forced.
//! The muxer derives it from the PGS `forced_on_flag` while muxing a rip; this
//! module gives `info` the SAME verdict up front by reading the title's PGS
//! streams and feeding them through the one shared classifier
//! ([`crate::mux::codec::pgs::ForcedTracker`]) — so the two never diverge.
//!
//! Cost: a track is only confirmed forced once EVERY display set is seen to be
//! forced, so a disc that has a forced track is read through — the
//! accuracy-over-speed tradeoff `info` opts into. Full tracks early-exit as soon
//! as they show a single non-forced subtitle, and a whole run stops early once
//! every track has settled.
//!
//! Encrypted content: the probe reuses whatever [`SectorSource`] the scan holds.
//! With a decrypting source it sees real PGS; without keys it reads ciphertext
//! and observes no display sets, in which case it leaves each track's existing
//! (vendor-label-derived) forced flag untouched rather than asserting anything.
use crate::disc::{Codec, DiscTitle, Stream};
use crate::mux::codec::CodecParser;
use crate::mux::codec::pgs::{ForcedTracker, PgsParser};
use crate::mux::ts::TsDemuxer;
use crate::sector::SectorSource;
use std::collections::HashMap;
const SECTOR_BYTES: usize = 2048;
/// Read the clip in 2 MiB chunks.
const CHUNK_SECTORS: u16 = 1024;
/// Read the title's PGS streams and set `SubtitleStream::forced` from their
/// content. Best-effort: any read error ends the probe with whatever verdicts
/// have accumulated. Only PGS tracks are touched (DVD VobSub forced comes from
/// the IFO/vendor path).
pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
reader: &mut S,
title: &mut DiscTitle,
) {
let pg_pids: Vec<u16> = title
.streams
.iter()
.filter_map(|s| match s {
Stream::Subtitle(sub) if sub.codec == Codec::Pgs => Some(sub.pid),
_ => None,
})
.collect();
if pg_pids.is_empty() {
return;
}
let mut demux = TsDemuxer::new(&pg_pids);
let mut parsers: HashMap<u16, PgsParser> =
pg_pids.iter().map(|&p| (p, PgsParser::new())).collect();
let mut trackers: HashMap<u16, ForcedTracker> =
pg_pids.iter().map(|&p| (p, ForcedTracker::new())).collect();
let extents = title.extents.clone();
let mut buf = vec![0u8; CHUNK_SECTORS as usize * SECTOR_BYTES];
'outer: for ext in &extents {
let mut lba = ext.start_lba;
let mut remaining = ext.sector_count;
while remaining > 0 {
let count = remaining.min(CHUNK_SECTORS as u32) as u16;
let want = count as usize * SECTOR_BYTES;
let n = match reader.read_sectors(lba, count, &mut buf[..want], false) {
Ok(n) => n,
Err(_) => break 'outer, // best-effort — stop, keep what we have
};
if n == 0 {
break 'outer;
}
for pes in demux.feed(&buf[..n]) {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
{
for frame in parser.parse(&pes) {
tracker.observe(&frame.data);
}
}
}
// Every track has already shown a non-forced set → nothing left to
// learn; stop reading the (huge) clip.
if trackers.values().all(ForcedTracker::settled_not_forced) {
break 'outer;
}
lba += count as u32;
remaining -= count as u32;
}
}
// Drain any buffered final display set.
for (pid, parser) in parsers.iter_mut() {
if let Some(tracker) = trackers.get_mut(pid) {
for frame in parser.flush() {
tracker.observe(&frame.data);
}
}
}
// Apply verdicts. Only override a track we actually saw content for — an
// undecrypted/unread track keeps its vendor-derived flag.
for s in &mut title.streams {
if let Stream::Subtitle(sub) = s {
if sub.codec == Codec::Pgs {
if let Some(t) = trackers.get(&sub.pid) {
if t.observed() {
sub.forced = t.is_forced();
}
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{ContentFormat, Extent, LabelQualifier, SubtitleStream};
/// A reader that yields all-zeros (an encrypted / unreadable clip) for a
/// bounded span, then EOF.
struct ZeroReader {
served: u32,
cap: u32,
}
impl SectorSource for ZeroReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
if self.served >= self.cap {
return Ok(0);
}
self.served += count as u32;
buf.fill(0);
Ok(buf.len())
}
fn capacity_sectors(&self) -> u32 {
self.cap
}
}
fn pgs_title(pid: u16, vendor_forced: bool) -> DiscTitle {
DiscTitle {
playlist: String::new(),
playlist_id: 0,
duration_secs: 0.0,
size_bytes: 0,
clips: vec![],
streams: vec![Stream::Subtitle(SubtitleStream {
pid,
codec: Codec::Pgs,
language: "eng".into(),
forced: vendor_forced,
qualifier: LabelQualifier::None,
codec_data: None,
})],
chapters: vec![],
extents: vec![Extent {
start_lba: 0,
sector_count: 4,
}],
content_format: ContentFormat::BdTs,
codec_privates: vec![None],
}
}
#[test]
fn no_observed_content_preserves_vendor_forced() {
// An unreadable/encrypted clip yields no PGS display sets — the probe must
// leave the existing vendor-derived forced flag untouched, never assert
// "not forced" from having seen nothing.
let mut reader = ZeroReader { served: 0, cap: 4 };
let mut title = pgs_title(0x1200, true);
probe_and_set_forced(&mut reader, &mut title);
let Stream::Subtitle(s) = &title.streams[0] else {
panic!()
};
assert!(s.forced, "no content observed → vendor forced preserved");
}
#[test]
fn no_pgs_streams_is_noop() {
// A title with no PGS subtitle streams is a no-op (the reader is never
// touched — a DVD/VobSub or audio-only title).
let mut reader = ZeroReader { served: 0, cap: 0 };
let mut title = pgs_title(0x1200, false);
// Swap the PGS sub for an audio stream so there are no PGS PIDs.
title.streams.clear();
probe_and_set_forced(&mut reader, &mut title);
assert_eq!(reader.served, 0, "no PGS PIDs → no reads");
}
}
+31
View File
@@ -150,6 +150,14 @@ pub const E_NETWORK_ADDR_BLOCKED: u16 = 9022;
/// frame dropped before the first keyframe) cannot report success.
pub const E_MUX_EMPTY: u16 = 9023;
pub const E_EXTENT_NOT_UNIT_ALIGNED: u16 = 9030;
/// `mp4://` output but the title has no (primary) video track to carry.
pub const E_MP4_NO_VIDEO_TRACK: u16 = 9048;
/// `mp4://` video track uses a codec with no MP4 mapping / no config record
/// available here (e.g. VC-1, or MPEG-2 without an avcC/hvcC-style record).
pub const E_MP4_UNSUPPORTED_VIDEO_CODEC: u16 = 9049;
/// `mp4://` video track is missing its codec-configuration record
/// (`hvcC`/`avcC`), without which the sample entry can't be written.
pub const E_MP4_MISSING_CODEC_PRIVATE: u16 = 9050;
/// READ CAPACITY returned a short or overflowing transfer.
pub const E_DISC_CAPACITY_MALFORMED: u16 = 9047;
@@ -420,6 +428,12 @@ pub enum Error {
/// every frame dropped before the first keyframe — fails loudly. The
/// `m2ts://` analogue of [`Error::MkvInvalid`]'s zero-frame guard.
MuxEmpty,
/// `mp4://` target title has no primary video track to mux.
MuxNoVideoTrack,
/// `mp4://` video codec has no MP4 sample-entry mapping available here.
Mp4UnsupportedVideoCodec,
/// `mp4://` video track is missing its `hvcC`/`avcC` configuration record.
MuxMissingCodecPrivate,
PesFrameTooLarge {
size: usize,
},
@@ -599,6 +613,9 @@ impl Error {
Error::StreamUrlMissingPort { .. } => E_STREAM_URL_MISSING_PORT,
Error::NetworkAddrBlocked { .. } => E_NETWORK_ADDR_BLOCKED,
Error::MuxEmpty => E_MUX_EMPTY,
Error::MuxNoVideoTrack => E_MP4_NO_VIDEO_TRACK,
Error::Mp4UnsupportedVideoCodec => E_MP4_UNSUPPORTED_VIDEO_CODEC,
Error::MuxMissingCodecPrivate => E_MP4_MISSING_CODEC_PRIVATE,
Error::PesFrameTooLarge { .. } => E_PES_FRAME_TOO_LARGE,
Error::PesInvalidMagic => E_PES_INVALID_MAGIC,
Error::PesTrackTooLarge { .. } => E_PES_TRACK_TOO_LARGE,
@@ -833,6 +850,11 @@ impl From<Error> for std::io::Error {
// 9023 MuxEmpty: finish() reached with zero frames — the output
// would be a header-only container. Treat as invalid output.
E_MUX_EMPTY => std::io::ErrorKind::InvalidData,
// 9048-9050 mp4:// track/codec/config mismatches: the requested
// output can't hold this title's video — invalid output request.
E_MP4_NO_VIDEO_TRACK | E_MP4_UNSUPPORTED_VIDEO_CODEC | E_MP4_MISSING_CODEC_PRIVATE => {
std::io::ErrorKind::InvalidData
}
// 9030 ExtentNotUnitAligned: a malformed/non-AACS-aligned
// extent was handed to the prefetch producer.
9030 => std::io::ErrorKind::InvalidInput,
@@ -1236,6 +1258,9 @@ mod tests {
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MP4_NO_VIDEO_TRACK,
E_MP4_UNSUPPORTED_VIDEO_CODEC,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
@@ -1323,6 +1348,12 @@ mod tests {
(Error::PipelineConsumerGone, E_PIPELINE_CONSUMER_GONE),
(Error::DiscCapacityOverflow, E_DISC_CAPACITY_OVERFLOW),
(Error::MuxEmpty, E_MUX_EMPTY),
(Error::MuxNoVideoTrack, E_MP4_NO_VIDEO_TRACK),
(
Error::Mp4UnsupportedVideoCodec,
E_MP4_UNSUPPORTED_VIDEO_CODEC,
),
(Error::MuxMissingCodecPrivate, E_MP4_MISSING_CODEC_PRIVATE),
(Error::M2tsPacketMalformed, E_M2TS_PACKET_MALFORMED),
(Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED),
(Error::DiscCapacityMalformed, E_DISC_CAPACITY_MALFORMED),
+46 -4
View File
@@ -16,7 +16,7 @@
/// (case-insensitive), then requires an even run of ASCII hex digits. Any
/// non-hex byte, or an odd length, yields `None`.
pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
// Empty → empty Vec (a legitimately-empty variable-length field); odd length
// is malformed. (`parse_hex_fixed` enforces a concrete length separately.)
@@ -34,7 +34,7 @@ pub fn parse_hex_bytes(s: &str) -> Option<Vec<u8>> {
/// prefix; requires EXACTLY `2*N` ASCII hex digits after it. `None` on any
/// non-hex byte or a length mismatch.
pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
let body = strip_prefix(s.trim());
let body = strip_hex_prefix(s.trim());
let bytes = body.as_bytes();
if bytes.len() != 2 * N {
return None;
@@ -46,8 +46,33 @@ pub fn parse_hex_fixed<const N: usize>(s: &str) -> Option<[u8; N]> {
Some(out)
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive).
fn strip_prefix(s: &str) -> &str {
/// Parse a hex string into a `u16`. Accepts an optional `0x`/`0X` prefix
/// (case-insensitive) via the same [`strip_hex_prefix`] the byte parsers use.
/// `None` on any non-hex content or overflow.
///
/// Exists so callers never hand-roll `from_str_radix(s.trim_start_matches("0x"), 16)`
/// — a **case-sensitive** strip that silently dropped an uppercase-`0X` value.
/// (That reintroduced-in-keydb bug is exactly what this module was built to kill;
/// the integer fields now share the one prefix rule.)
pub fn parse_hex_u16(s: &str) -> Option<u16> {
u16::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u32`. See [`parse_hex_u16`].
pub fn parse_hex_u32(s: &str) -> Option<u32> {
u32::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Parse a hex string into a `u8`. See [`parse_hex_u16`].
pub fn parse_hex_u8(s: &str) -> Option<u8> {
u8::from_str_radix(strip_hex_prefix(s.trim()), 16).ok()
}
/// Strip a single leading `0x` / `0X` if present (case-insensitive). Public so
/// callers that only need the prefix rule (e.g. normalizing a disc hash) reuse
/// the one definition instead of hand-rolling a case-sensitive
/// `trim_start_matches("0x")`.
pub fn strip_hex_prefix(s: &str) -> &str {
s.strip_prefix("0x")
.or_else(|| s.strip_prefix("0X"))
.unwrap_or(s)
@@ -95,6 +120,23 @@ mod tests {
assert_eq!(parse_hex_fixed::<16>(&s), None);
}
#[test]
fn hex_ints_accept_both_prefix_cases_and_bare() {
// The regression the keydb device-key bug hit: uppercase `0X` must parse
// identically to `0x` and to a bare value.
assert_eq!(parse_hex_u16("0x0001"), Some(1));
assert_eq!(parse_hex_u16("0X0001"), Some(1));
assert_eq!(parse_hex_u16("0001"), Some(1));
assert_eq!(parse_hex_u16(" 0XABCD "), Some(0xABCD));
assert_eq!(parse_hex_u32("0X00000002"), Some(2));
assert_eq!(parse_hex_u32("deadbeef"), Some(0xDEAD_BEEF));
assert_eq!(parse_hex_u8("0X03"), Some(3));
assert_eq!(parse_hex_u8("ff"), Some(0xFF));
// Overflow / non-hex → None.
assert_eq!(parse_hex_u8("0x1FF"), None);
assert_eq!(parse_hex_u16("0xzz"), None);
}
#[test]
fn bytes_variable_length_and_odd_rejected() {
assert_eq!(parse_hex_bytes("0xAABBCC"), Some(vec![0xAA, 0xBB, 0xCC]));
+196 -62
View File
@@ -115,7 +115,7 @@ pub struct DiscInputs {
pub volume_label: Option<String>,
}
/// A lazy view of a disc's AACS material, handed to [`KeySource::get_uk`] so a
/// A lazy view of a disc's AACS material, handed to [`KeySource::get_unit_keys`] so a
/// source can drive the derivation chain without holding the disc reader.
///
/// "Lazy" by contract: each accessor returns only what the source asks for, so a
@@ -233,10 +233,34 @@ impl ResolveCtx for DiscInputsCtx<'_> {
/// ([`resolve_and_apply`]) tries each source in order and validates the returned
/// keys against real ciphertext before committing them, so a wrong key from one
/// source transparently falls through to the next.
///
/// Two explicit resolve operations, one per key kind — never one overloaded call
/// whose meaning depends on how many keys came back:
/// * [`get_unit_keys`](Self::get_unit_keys) — the disc's base per-CPS-unit Unit
/// Keys (index space = CPS-unit number). The common path for every disc.
/// * [`get_fmts_indexes`](Self::get_fmts_indexes) — the AACS 2.1 forensic index
/// keys (index space = forensic index 1..N). Defaults to empty: a source with
/// no forensic material opts out, and only an FMTS disc ever asks.
///
/// What each source must do to answer is the source's own business: a keydb keys
/// on `disc_hash` and reads no samples; the online source submits the ctx's
/// content samples (a base batch for `get_unit_keys`, an index-1 anchor batch for
/// `get_fmts_indexes`) to the key service.
pub trait KeySource {
/// Resolve this disc's terminal Unit Keys from this source. An empty `Vec`
/// is a genuine "no key here"; `Err` is a source failure.
fn get_uk(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error>;
/// Resolve this disc's base per-CPS-unit Unit Keys from this source. An empty
/// `Vec` is a genuine "no key here"; `Err` is a source failure.
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error>;
/// Resolve this disc's AACS 2.1 forensic index keys — the per-index keys the
/// base Unit Key cannot open (see [`crate::aacs::segment`]) — ordered by
/// forensic index (element `i` carries `UnitKey.idx == i`, forensic index
/// `i + 1`). The source hands back the COMPLETE set it holds; the caller
/// trusts any non-empty result as all of them and never assumes a fixed count.
/// Defaults to empty: a source with no forensic material (a plain keydb, the
/// mapfile) opts out, and only an FMTS disc's mux ever calls this.
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
/// The AACS host certificate(s) this source can supply for the live-drive
/// SCSI mutual-auth handshake (the OEM/AACS baseline route). `mkb` is the
@@ -273,7 +297,7 @@ pub fn resolve_and_apply(
/// [`crate::aacs::trace::ResolutionTrace`] recording, per source, what happened — for
/// applications to render. ZERO English; the trace is typed enums only.
///
/// One-shot per source: each source's [`KeySource::get_uk`] is called exactly
/// One-shot per source: each source's [`KeySource::get_unit_keys`] is called exactly
/// once with a [`DiscInputsCtx`] over `inputs`. Non-empty Unit Keys are mapped
/// to terminal [`Key::Unit`]s and applied via [`crate::Disc::decrypt_with`],
/// which validates them against `inputs.samples` and only mutates the disc on
@@ -301,7 +325,7 @@ pub fn resolve_and_apply_traced(
for source in sources {
// `who` is the source's own stable identifier — no enum to map back to.
let who = source.label().to_string();
match source.get_uk(&ctx) {
match source.get_unit_keys(&ctx) {
Ok(uks) if !uks.is_empty() => {
// Positional index → canonical CPS-unit number (position + 1).
let unit_keys: Vec<(u32, [u8; 16])> = uks
@@ -352,7 +376,7 @@ pub fn resolve_and_apply_traced(
/// decorator re-decrypts with the returned keys, which is the validation.
pub fn fetch_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
for source in sources {
if let Ok(uks) = source.get_uk(ctx) {
if let Ok(uks) = source.get_unit_keys(ctx) {
if !uks.is_empty() {
return uks;
}
@@ -361,61 +385,92 @@ pub fn fetch_unit_keys(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) ->
Vec::new()
}
/// Build the read-time key-fetch closure from the disc's public AACS inputs and
/// a way to (re)build the application's key sources. The decorator calls it with
/// the still-scrambled unit ciphertext when no held key opens that unit; it runs
/// [`fetch_unit_keys`] with those bytes as `samples` and returns any keys.
/// The forensic counterpart to [`fetch_unit_keys`]: drive `sources` in order and
/// return the first source's non-empty AACS 2.1 forensic index set. `ctx` carries
/// the index-1 anchor batch (the mux, which owns disc geometry, gathers it and
/// injects it as the ctx's samples); a source that needs no samples (a keydb
/// keying on `disc_hash`) ignores them. Whatever the winning source returns —
/// ≥ 1 key — is trusted as the COMPLETE ordered set; no fixed count is assumed.
pub fn fetch_fmts_indexes(sources: &[Box<dyn KeySource>], ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
for source in sources {
if let Ok(uks) = source.get_fmts_indexes(ctx) {
if !uks.is_empty() {
return uks;
}
}
}
Vec::new()
}
/// Build the read-time [`crate::sector::KeyFetch`] from the disc's public AACS
/// inputs and a way to (re)build the application's key sources. The returned
/// resolver has the two explicit operations the mux and recovery decorator call:
/// [`unit_keys`](crate::sector::KeyFetch::unit_keys) drives [`fetch_unit_keys`]
/// (base per-CPS-unit keys), [`fmts_indexes`](crate::sector::KeyFetch::fmts_indexes)
/// drives [`fetch_fmts_indexes`] (the AACS 2.1 forensic set). Each is handed the
/// caller's sample batch as the ctx's `samples`, so a source pulls whatever
/// material it needs.
///
/// One builder, used by every read path (sweep / patch / mux) and by every
/// consumer (CLI, autorip) — neither application contains the fetch logic, only
/// its key-source config. Returns a **shared, stateless** [`crate::sector::KeyFetch`]
/// (`Arc<Fn>`): build it once, clone it into each read path. `make_sources` is
/// invoked per fetch (the cold path, ~once per CPS unit) so the closure stays
/// One builder, used by every read path (sweep / patch / mux) and every consumer
/// (CLI, autorip) — neither application contains the fetch logic, only its
/// key-source config. Cheap to clone; build once, clone into each read path.
/// `make_sources` is invoked per fetch (the cold path) so the resolver stays
/// `Send + Sync` without requiring `KeySource: Send`.
pub fn key_fetch(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
) -> crate::sector::KeyFetch {
// Memoize by the fingerprint of the sample batch. The resolved keys are
// disc-level (the same clip's index / CPS keys are identical for every title
// that references it), and this one closure is shared across every title's mux
// — so the first title resolves a given batch over the network and every later
// title (or repeated batch) is answered from the cache with no request. Empty
// replies are cached too: a key the service does not have for a batch will not
// appear on a re-ask, so re-hitting the network buys nothing.
let cache: std::sync::Arc<std::sync::Mutex<std::collections::HashMap<u64, Vec<[u8; 16]>>>> =
std::sync::Arc::new(std::sync::Mutex::new(std::collections::HashMap::new()));
std::sync::Arc::new(move |samples: &[Vec<u8>]| -> Vec<[u8; 16]> {
let fp = {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
samples.len().hash(&mut h);
for s in samples {
s.hash(&mut h);
// One driver behind both operations: rebuild the sources, inject `samples`
// as the ctx's content samples, run `drive` (the per-kind fetch), map the
// resolved UnitKeys to raw keys. Memoized by the fingerprint of the sample
// batch: the resolved keys are disc-level (a clip's index / CPS keys are
// identical for every title that references it), so the first batch resolves
// over the network and every repeat is answered from the cache with no
// request. Empty replies are cached too — a key the service lacks for a batch
// won't appear on a re-ask, so re-hitting the network buys nothing. Each
// operation gets its OWN cache: a base batch and a forensic anchor never
// collide, and the same bytes could legitimately resolve differently per op.
// The per-kind driver: `fetch_unit_keys` or `fetch_fmts_indexes`.
type FetchDriver = fn(&[Box<dyn KeySource>], &dyn ResolveCtx) -> Vec<UnitKey>;
fn make_op(
inputs: DiscInputs,
make_sources: std::sync::Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync>,
drive: FetchDriver,
) -> crate::sector::KeyFetchFn {
let cache: std::sync::Arc<std::sync::Mutex<std::collections::HashMap<u64, Vec<[u8; 16]>>>> =
std::sync::Arc::new(std::sync::Mutex::new(std::collections::HashMap::new()));
std::sync::Arc::new(move |samples: &[Vec<u8>]| -> Vec<[u8; 16]> {
let fp = {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
samples.len().hash(&mut h);
for s in samples {
s.hash(&mut h);
}
h.finish()
};
if let Some(hit) = cache.lock().unwrap_or_else(|e| e.into_inner()).get(&fp) {
return hit.clone();
}
h.finish()
};
if let Some(hit) = cache.lock().unwrap_or_else(|e| e.into_inner()).get(&fp) {
return hit.clone();
}
let sources = make_sources();
let mut di = inputs.clone();
di.samples = samples.to_vec();
// Parse Unit_Key_RO.inf at the disc's OWN stride (carried on `inputs`):
// an online /decode reply that returns a VUK (not a terminal UK) then
// derives unit keys from `enc_title_keys`, which a V10 disc parses at the
// 48-byte stride — hardcoding the V20 stride here corrupted them.
let ctx = DiscInputsCtx::new(&di);
let keys: Vec<[u8; 16]> = fetch_unit_keys(&sources, &ctx)
.into_iter()
.map(|u| u.key)
.collect();
cache
.lock()
.unwrap_or_else(|e| e.into_inner())
.insert(fp, keys.clone());
keys
})
let sources = make_sources();
let mut di = inputs.clone();
di.samples = samples.to_vec();
// Parse Unit_Key_RO.inf at the disc's OWN stride (carried on `inputs`):
// an online /decode reply that returns a VUK (not a terminal UK) then
// derives unit keys from `enc_title_keys`, which a V10 disc parses at
// the 48-byte stride — hardcoding the V20 stride here corrupted them.
let ctx = DiscInputsCtx::new(&di);
let keys: Vec<[u8; 16]> = drive(&sources, &ctx).into_iter().map(|u| u.key).collect();
cache
.lock()
.unwrap_or_else(|e| e.into_inner())
.insert(fp, keys.clone());
keys
})
}
let unit = make_op(inputs.clone(), make_sources.clone(), fetch_unit_keys);
let fmts = make_op(inputs, make_sources, fetch_fmts_indexes);
crate::sector::KeyFetch::new(unit, fmts)
}
/// Read up to `n` ENCRYPTED 6144-byte aligned units from `title`'s body, raw (no
@@ -560,7 +615,7 @@ mod tests {
fn key_source_host_certs_defaults_to_empty() {
struct MinimalSource;
impl KeySource for MinimalSource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
}
@@ -617,7 +672,7 @@ mod tests {
fn trace_who_is_the_source_label_verbatim() {
struct LabeledSource(&'static str);
impl KeySource for LabeledSource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
fn label(&self) -> &'static str {
@@ -674,19 +729,19 @@ mod tests {
struct EmptySource;
impl KeySource for EmptySource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
}
struct ErroringSource;
impl KeySource for ErroringSource {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Err(Error::AacsNoKeys)
}
}
struct HasKey([u8; 16]);
impl KeySource for HasKey {
fn get_uk(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, self.0)])
}
}
@@ -729,7 +784,7 @@ mod tests {
seen: Arc<Mutex<Vec<Vec<u8>>>>,
}
impl KeySource for Probe {
fn get_uk(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
if let Ok(s) = ctx.samples(8) {
self.seen.lock().unwrap().extend(s);
}
@@ -749,7 +804,7 @@ mod tests {
let cb = key_fetch(empty_inputs(), make);
let samples = vec![vec![0xEEu8; crate::aacs::content::ALIGNED_UNIT_LEN]];
let got = cb(&samples);
let got = cb.unit_keys(&samples);
assert_eq!(
got,
vec![key],
@@ -763,6 +818,85 @@ mod tests {
assert_eq!(*builds.lock().unwrap(), 1, "make_sources invoked per fetch");
}
/// The two `KeyFetch` operations route to the two DISTINCT trait methods:
/// `unit_keys` drives `get_unit_keys`, `fmts_indexes` drives
/// `get_fmts_indexes`. A source that returns different keys per method proves
/// the seam no longer collapses "1 base key" and "the forensic set" into one
/// overloaded call — the operation, not the return length, decides which.
#[test]
fn key_fetch_routes_unit_and_fmts_to_distinct_source_methods() {
const BASE: [u8; 16] = [0xB0; 16];
const F1: [u8; 16] = [0xF1; 16];
const F2: [u8; 16] = [0xF2; 16];
struct TwoOp;
impl KeySource for TwoOp {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, BASE)])
}
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, F1), UnitKey::new(1, F2)])
}
}
let make: Arc<dyn Fn() -> Vec<Box<dyn KeySource>> + Send + Sync> =
Arc::new(|| vec![Box::new(TwoOp) as Box<dyn KeySource>]);
let cb = key_fetch(empty_inputs(), make);
let samples = vec![vec![0x01u8; 4]];
assert_eq!(
cb.unit_keys(&samples),
vec![BASE],
"unit_keys resolves the base Unit Key via get_unit_keys"
);
assert_eq!(
cb.fmts_indexes(&samples),
vec![F1, F2],
"fmts_indexes resolves the forensic set (any length) via get_fmts_indexes"
);
}
/// `KeyFetch::unit_only` serves base keys but NEVER a forensic set — the
/// contract the sweep/patch recovery decorator relies on (it resolves CPS
/// units only). Its `fmts_indexes` is unconditionally empty.
#[test]
fn key_fetch_unit_only_never_serves_forensic() {
let f = crate::sector::KeyFetch::unit_only(std::sync::Arc::new(|_| vec![[0xAA; 16]]));
assert_eq!(f.unit_keys(&[vec![0u8; 4]]), vec![[0xAA; 16]]);
assert!(
f.fmts_indexes(&[vec![0u8; 4]]).is_empty(),
"unit_only resolver yields no forensic keys"
);
}
/// `get_fmts_indexes` defaults to empty, so a base-only source (a keydb) opts
/// out of the forensic path without implementing it. `fetch_fmts_indexes` then
/// falls through to the next source, exactly like the unit-key driver.
#[test]
fn fetch_fmts_indexes_skips_default_optout_source() {
struct BaseOnly; // uses the default (empty) get_fmts_indexes
impl KeySource for BaseOnly {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, [0x11; 16])])
}
}
struct Forensic;
impl KeySource for Forensic {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(Vec::new())
}
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
Ok(vec![UnitKey::new(0, [0x77; 16])])
}
}
let inputs = empty_inputs();
let ctx = DiscInputsCtx::new(&inputs);
let sources: Vec<Box<dyn KeySource>> = vec![Box::new(BaseOnly), Box::new(Forensic)];
let got = fetch_fmts_indexes(&sources, &ctx);
assert_eq!(got.len(), 1);
assert_eq!(got[0].key, [0x77; 16], "the base-only source is skipped");
}
/// #4 regression: encrypted content NOT at the extent midpoint (a late-
/// starting feature, or a midpoint landing in clear nav) must still be
/// sampled — empty samples make `decrypt_with` skip wrong-key validation.
+2 -1
View File
@@ -98,7 +98,7 @@ pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFI
/// The muxing/writing-application string written into MKV output
/// (`"freemkv <version> (g<hash>)"`).
pub const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub(crate) const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs;
pub(crate) mod clpi;
@@ -241,6 +241,7 @@ pub use mux::NullStream;
pub use mux::StdioStream;
pub use mux::WriteSeek;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
pub use mux::{Mp4FitReport, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
+246 -53
View File
@@ -41,6 +41,11 @@ pub struct Ac3Parser {
/// the running per-frame PTS at the point the partial tail was retained.
/// Used by `flush()` to time the final buffered frame at EOS.
flush_pts_ns: i64,
/// Keep/drop bookkeeping for the CRC decodability gate. A frame that fails
/// its native CRC is dropped rather than shipped as a decoder-choking glitch;
/// the running PTS is advanced across it (see the emit loop) so the drop is a
/// silence gap, never a shift of the following audio.
tally: super::dropgate::DropTally,
}
impl Default for Ac3Parser {
@@ -54,8 +59,99 @@ impl Ac3Parser {
Self {
buf: Vec::with_capacity(4096),
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
}
}
/// Access units dropped as undecodable so far — surfaced to the CLI/mux.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Emit the final buffered frame at EOS, through the decodability gate.
/// During streaming a final frame may sit in `buf` with no following PES to
/// complete it; without this drain the last ~32 ms of audio is lost. Only a
/// fully-sized frame at a syncword is considered; a partial/garbage tail is
/// discarded, and a corrupt (CRC-failing) final frame is dropped.
fn flush_tail(&mut self) -> Vec<Frame> {
let buf = std::mem::take(&mut self.buf);
let Some(off) = find_ac3_sync(&buf) else {
return Vec::new();
};
let frame_all = &buf[off..];
if frame_all.len() < 6 {
return Vec::new();
}
let bsid = get_bsid(frame_all);
let frame_size = if bsid >= 11 {
eac3_frame_size(frame_all)
} else {
ac3_frame_size(frame_all)
};
if !(MIN_FRAME_BYTES..=8192).contains(&frame_size) || off + frame_size > buf.len() {
return Vec::new();
}
let frame = &buf[off..off + frame_size];
let duration_ns = frame_duration_ns(frame, bsid);
if let Some(reason) = ac3_drop_reason(&self.tally, frame, bsid) {
self.tally
.record_drop(self.flush_pts_ns, duration_ns as i64, frame.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.flush_pts_ns,
keyframe: true,
data: frame.to_vec(),
duration_ns: Some(duration_ns),
}]
}
}
use super::crc::crc16_ansi;
/// Whether a fully-buffered (E-)AC-3 frame passes its native CRC. ffmpeg's
/// decoder checks exactly this — `av_crc(AV_CRC_16_ANSI, 0, &buf[2],
/// frame_size - 2) == 0` (ac3dec.c) — over the frame after the 2-byte syncword;
/// the trailing crc word makes a clean frame's residue zero. A nonzero residue
/// is a ~1-in-65536-certain sign of payload corruption, so we drop the frame
/// (silence gap) rather than ship a glitch. `frame` must be exactly the frame
/// bytes (syncword .. frame_size).
fn frame_crc_ok(frame: &[u8]) -> bool {
// Need the syncword (2) plus at least one covered byte; the caller only
// invokes this on a fully-sized frame, so this is defensive.
if frame.len() < 4 {
return true;
}
crc16_ansi(&frame[2..]) == 0
}
/// Decodability verdict for a fully-sized (E-)AC-3 frame: `Some(reason)` when it
/// must be dropped, `None` when it decodes. Drops (in order): a poisoned track
/// (mostly-undecodable → drop the rest), a bitstream id ffmpeg's parser rejects
/// (`bsid > 16` → `AC3_PARSE_ERROR_BSID`), or a failed native frame CRC.
fn ac3_drop_reason(
tally: &super::dropgate::DropTally,
frame: &[u8],
bsid: u8,
) -> Option<&'static str> {
if tally.is_poisoned() {
Some("track-poisoned")
} else if bsid > 16 {
Some("bsid")
} else if !frame_crc_ok(frame) {
Some("crc")
} else {
None
}
}
impl CodecParser for Ac3Parser {
@@ -91,10 +187,12 @@ impl CodecParser for Ac3Parser {
// in practice, so this is defense-in-depth.
let base_pts_ns = pes.pts.map(pts_to_ns).unwrap_or(self.flush_pts_ns);
// Prepend leftover from previous PES
// Prepend leftover from previous PES, then take the whole buffer into a
// local so the emit loop can call `self.tally` (the bytes are no longer
// borrowed from `self`). The unconsumed tail is written back at the end.
self.buf.extend_from_slice(&pes.data);
let data = &self.buf;
let buf = std::mem::take(&mut self.buf);
let data = &buf;
let mut frames = Vec::new();
let mut pos = 0;
// Running PTS for the next frame to emit in this call.
@@ -134,15 +232,26 @@ impl CodecParser for Ac3Parser {
}
let duration_ns = frame_duration_ns(remaining, bsid);
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: frame_pts_ns,
keyframe: true,
data: data[start..start + frame_size].to_vec(),
duration_ns: Some(duration_ns),
});
let frame = &data[start..start + frame_size];
// Decodability gate: drop a frame ffmpeg's parser rejects (bsid > 16)
// or whose native CRC fails (payload corruption). `frame_pts_ns` is
// advanced BELOW whether or not the frame survives, so a drop is a
// silence gap and the following frames keep their true PTS.
if let Some(reason) = ac3_drop_reason(&self.tally, frame, bsid) {
self.tally
.record_drop(frame_pts_ns, duration_ns as i64, frame.len(), reason);
} else {
self.tally.record_kept();
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: frame_pts_ns,
keyframe: true,
data: frame.to_vec(),
duration_ns: Some(duration_ns),
});
}
frame_pts_ns += duration_ns as i64;
pos = start + frame_size;
}
@@ -198,38 +307,10 @@ impl CodecParser for Ac3Parser {
}
fn flush(&mut self) -> Vec<Frame> {
// End of stream: emit a complete final frame still buffered. During
// streaming a final frame may sit in `buf` with no following PES to
// complete/confirm it; without this drain the last ~32 ms of audio is
// dropped at EOS (mirrors dts.rs::flush). Only a fully-sized frame at a
// syncword is emitted; a partial/garbage tail is discarded.
let buf = std::mem::take(&mut self.buf);
let Some(off) = find_ac3_sync(&buf) else {
return Vec::new();
};
let frame = &buf[off..];
if frame.len() < 6 {
return Vec::new();
}
let bsid = get_bsid(frame);
let frame_size = if bsid >= 11 {
eac3_frame_size(frame)
} else {
ac3_frame_size(frame)
};
if !(MIN_FRAME_BYTES..=8192).contains(&frame_size) || off + frame_size > buf.len() {
return Vec::new();
}
let duration_ns = frame_duration_ns(frame, bsid);
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.flush_pts_ns,
keyframe: true,
data: buf[off..off + frame_size].to_vec(),
duration_ns: Some(duration_ns),
}]
let out = self.flush_tail();
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
}
fn codec_private(&self) -> Option<Vec<u8>> {
@@ -460,9 +541,24 @@ mod tests {
frame[1] = 0x77;
frame[4] = (fscod << 6) | frmsizecod;
frame[5] = 0x08 << 3; // bsid = 8 (AC-3)
finalize_ac3_crc(&mut frame);
frame
}
/// Set the trailing CRC word so the whole-frame residue over `[2..]` is zero
/// — i.e. the frame passes the decodability gate. Relies on the CRC-16/ANSI
/// residue property: appending `crc16([2..n-2])` (big-endian) zeroes the
/// register over `[2..n]`. Leaves the crc1 field (bytes 2-3) untouched.
fn finalize_ac3_crc(frame: &mut [u8]) {
let n = frame.len();
if n < 4 {
return;
}
let c = crc16_ansi(&frame[2..n - 2]);
frame[n - 2] = (c >> 8) as u8;
frame[n - 1] = (c & 0xFF) as u8;
}
#[test]
fn parse_empty_pes() {
let mut parser = Ac3Parser::new();
@@ -1090,27 +1186,33 @@ mod tests {
// --- frame acceptance / rejection at the size boundaries ---
#[test]
fn eac3_frame_at_min_frame_bytes_is_accepted() {
// The smallest acceptable (E-)AC-3 frame is MIN_FRAME_BYTES = 6.
// Build an E-AC-3 frame whose frmsiz sizes it to exactly 6 bytes
// (frmsiz=2). bsid >= 11 selects E-AC-3 sizing. The parser must emit it.
fn eac3_frame_at_min_frame_bytes_passes_sizing_then_crc_gate() {
// The smallest frame the SIZING layer accepts is MIN_FRAME_BYTES = 6
// (frmsiz=2). A synthetic all-zero 6-byte frame passes sizing (so it
// reaches the decodability gate — proven by it being COUNTED as a drop,
// not silently size-skipped) but fails the CRC gate and is dropped; the
// following real AC-3 frame (valid CRC) is emitted.
let mut parser = Ac3Parser::new();
// 0x0B 0x77 | byte2=0 byte3=2 (frmsiz=2 → 6 bytes) | byte4=0 | byte5 bsid
let mut data = vec![0x0B, 0x77, 0x00, 0x02, 0x00, 16 << 3];
// pad to exactly 6 bytes (already 6). Then a trailing real AC-3 frame so
// the 6-byte frame isn't a tail that needs more data.
data.truncate(6);
data.extend_from_slice(&make_ac3_frame(0, 2));
let f = parser.parse(&make_eac3_pes(data));
assert_eq!(f.len(), 2, "6-byte E-AC-3 frame accepted + following AC-3");
assert_eq!(f[0].data.len(), 6);
assert_eq!(f.len(), 1, "6-byte frame dropped (CRC), real AC-3 emitted");
assert_eq!(f[0].data.len(), 160, "the surviving frame is the real AC-3");
assert_eq!(
parser.dropped_frames(),
1,
"the 6-byte frame reached the gate"
);
}
#[test]
fn eac3_max_frmsiz_frame_within_window_accepted() {
// E-AC-3 frmsiz is an 11-bit field (3 bits of byte2 + 8 bits of byte3),
// so its maximum value is 0x7FF = 2047 → (2048)*2 = 4096 bytes, which is
// inside the MIN_FRAME_BYTES..=8192 accept window and must be emitted.
// inside the MIN_FRAME_BYTES..=8192 accept window and, with a valid CRC,
// must be emitted.
let mut parser = Ac3Parser::new();
let mut frame = vec![0u8; 4096];
frame[0] = 0x0B;
@@ -1118,6 +1220,7 @@ mod tests {
frame[2] = 0x07; // frmsiz high
frame[3] = 0xFF; // frmsiz low → 0x7FF = 2047 → 4096 bytes
frame[5] = 16 << 3; // bsid 16 (E-AC-3)
finalize_ac3_crc(&mut frame); // pass the decodability gate
let f = parser.parse(&make_eac3_pes(frame));
assert_eq!(f.len(), 1, "4096-byte E-AC-3 frame within window accepted");
assert_eq!(f[0].data.len(), 4096);
@@ -1296,6 +1399,96 @@ mod tests {
assert_eq!(acmod_channels(&frame), Some(2));
}
// --- decodability (CRC) gate: keep clean frames, drop corrupt ones ---
/// A structurally-valid AC-3 frame with one payload byte corrupted so its
/// native CRC fails (header/size intact, so the framer delimits it normally).
fn make_corrupt_ac3_frame(fscod: u8, frmsizecod: u8) -> Vec<u8> {
let mut f = make_ac3_frame(fscod, frmsizecod);
f[20] ^= 0xFF; // flip a payload byte → CRC no longer zero
assert!(!frame_crc_ok(&f), "corruption must break the CRC");
f
}
#[test]
fn crc16_residue_zero_after_finalize_nonzero_after_corruption() {
// The CRC-16/ANSI residue property the gate relies on: a finalized frame
// has residue 0 over [2..]; flipping any covered byte makes it nonzero.
let good = make_ac3_frame(0, 2);
assert!(frame_crc_ok(&good));
let bad = make_corrupt_ac3_frame(0, 2);
assert!(!frame_crc_ok(&bad));
}
#[test]
fn crc_fail_frame_is_dropped_survivors_kept() {
// good / corrupt / good in one PES: the corrupt middle frame is dropped
// (CRC), the two clean frames are emitted, and the drop is counted.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2);
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2));
data.extend_from_slice(&make_ac3_frame(0, 2));
let f = parser.parse(&make_eac3_pes(data));
// Only two of three survive; flush has nothing (all closed in-call).
assert_eq!(f.len(), 2, "corrupt frame dropped, two clean survive");
assert_eq!(parser.dropped_frames(), 1);
assert_eq!(
parser.dropped_duration_ns(),
32_000_000,
"one 32ms frame of silence"
);
}
#[test]
fn crc_drop_preserves_pts_sync_no_shift() {
// THE INVARIANT: dropping a corrupt frame must not shift the audio after
// it. good / corrupt / good in one PES — the corrupt frame is dropped but
// the trailing clean frame keeps the EXACT PTS it would have had with no
// drop (base + 2 frame durations): a silence gap, not a shift.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2); // f0
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2)); // dropped
data.extend_from_slice(&make_ac3_frame(0, 2)); // f2
let f = parser.parse(&make_eac3_pes(data));
assert_eq!(f.len(), 2);
let base = pts_to_ns(90000);
let frame_dur = 32_000_000i64; // 1536 @ 48k
assert_eq!(f[0].pts_ns, base, "f0 at PES base");
assert_eq!(
f[1].pts_ns,
base + 2 * frame_dur,
"surviving frame keeps its true timeline (base + 2 frames) — gap, not shift"
);
}
#[test]
fn bsid_over_16_is_dropped() {
// ffmpeg's parser rejects bsid > 16 (AC3_PARSE_ERROR_BSID). A frame with
// bsid = 17 that still sizes must be dropped, not emitted.
let mut frame = vec![0u8; 128];
frame[0] = 0x0B;
frame[1] = 0x77;
frame[3] = 63; // frmsiz = 63 → (63+1)*2 = 128 bytes (E-AC-3 sizing)
frame[5] = 17 << 3; // bsid = 17 (> 16)
assert_eq!(get_bsid(&frame), 17);
let tally = super::super::dropgate::DropTally::new("ac3");
assert_eq!(ac3_drop_reason(&tally, &frame, 17), Some("bsid"));
}
#[test]
fn clean_stream_drops_nothing() {
// A stream of valid frames passes untouched — zero false positives.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..5 {
data.extend_from_slice(&make_ac3_frame(0, 2));
}
let mut f = parser.parse(&make_eac3_pes(data));
f.extend(parser.flush());
assert_eq!(f.len(), 5);
assert_eq!(parser.dropped_frames(), 0);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
+220
View File
@@ -0,0 +1,220 @@
//! AAC ADTS decodability gate.
//!
//! ffmpeg's `ff_adts_header_parse` (adts_header.c) has exactly three hard
//! rejects: syncword != 0xFFF, a reserved `sampling_frequency_index`
//! (`ff_mpeg4audio_sample_rates[sr] == 0`, i.e. index ≥ 13), and
//! `aac_frame_length < 7`. It does NOT verify the optional ADTS CRC (it only
//! `skip_bits(16)` past it). So the gate mirrors those three rejects: a packet
//! that begins with the ADTS sync but is otherwise malformed is dropped; a
//! packet with no ADTS sync is raw AAC (e.g. from mp4, which carries no ADTS
//! header) or a continuation and passes through unchanged — never false-dropped.
//! Raw AAC has no per-frame integrity data, so like LPCM it cannot be gated.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// `ff_mpeg4audio_sample_rates` — 13 valid entries; indices 13/14/15 are 0
/// (reserved), which is exactly what ffmpeg rejects.
const ADTS_SAMPLE_RATE_VALID: [u32; 16] = [
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350, 0, 0,
0,
];
/// ADTS header verdict for the packet head.
enum AdtsVerdict {
/// No 12-bit ADTS sync at the head — not an ADTS frame we can validate.
NoSync,
/// Sync present and the three ffmpeg-checked fields are legal.
Valid,
/// Sync present but a reserved sample-rate index or a sub-header
/// frame-length — ffmpeg's parser rejects this.
Invalid,
}
fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// Need the full 7-byte fixed+variable header to read frame_length.
if data.len() < 7 {
return AdtsVerdict::NoSync;
}
// 12-bit syncword 0xFFF: byte0 == 0xFF and top nibble of byte1 == 0xF.
if data[0] != 0xFF || (data[1] & 0xF0) != 0xF0 {
return AdtsVerdict::NoSync;
}
// sampling_frequency_index: byte2 bits 5..2.
let sr_index = ((data[2] >> 2) & 0x0F) as usize;
if ADTS_SAMPLE_RATE_VALID[sr_index] == 0 {
return AdtsVerdict::Invalid;
}
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
if frame_length < 7 {
return AdtsVerdict::Invalid;
}
AdtsVerdict::Valid
}
pub struct AdtsParser {
tally: DropTally,
}
impl Default for AdtsParser {
fn default() -> Self {
Self::new()
}
}
impl AdtsParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("aac"),
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for AdtsParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
let drop =
self.tally.is_poisoned() || matches!(adts_verdict(&pes.data), AdtsVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with
/// aac_frame_length set to the total size.
fn adts_frame(payload: usize) -> Vec<u8> {
let total = 7 + payload;
let mut f = vec![0u8; total];
f[0] = 0xFF;
f[1] = 0xF1; // sync + MPEG-4 + no CRC (protection_absent=1)
f[2] = 0x50; // profile=AAC-LC, sr_index=4 (44.1 kHz)
f[3] = 0x80; // channel_config low + start of frame_length
// frame_length (13 bits) = total.
let fl = total as u32;
f[3] = (f[3] & 0xFC) | ((fl >> 11) & 0x03) as u8;
f[4] = ((fl >> 3) & 0xFF) as u8;
f[5] = (((fl & 0x07) << 5) as u8) | 0x1F; // low 3 bits of len + buffer-fullness bits
f
}
#[test]
fn valid_adts_is_kept() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(adts_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn reserved_sample_rate_index_is_dropped() {
// sr_index = 13 (reserved). byte2 bits5..2 = 1101 → 0x34.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[2] = (f[2] & 0xC3) | (13 << 2); // set sr_index = 13
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn subheader_frame_length_is_dropped() {
// frame_length < 7 (here 0) is a sub-header length → reject.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[3] &= 0xFC; // clear len high bits
f[4] = 0;
f[5] &= 0x1F; // clear len low bits → frame_length = 0
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn raw_aac_without_sync_passes_through() {
// No ADTS sync (e.g. raw AAC from mp4) → cannot validate → keep.
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(
vec![0x21, 0x00, 0x03, 0x40, 0x00, 0x00, 0x00],
Some(0),
));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = AdtsParser::new();
let mut bad = adts_frame(400);
bad[2] = (bad[2] & 0xC3) | (14 << 2); // reserved sr_index
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(adts_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
#[test]
fn short_packet_passes_through() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0)));
assert_eq!(f.len(), 1, "too short to validate → kept");
}
}
+110
View File
@@ -0,0 +1,110 @@
//! Bit-exact CRC helpers shared by the audio codec decodability gates.
//!
//! Both match ffmpeg's `av_crc` tables so a frame that ffmpeg's decoder would
//! flag as a CRC mismatch is flagged identically here. All are MSB-first
//! (non-reflected), init 0, no final XOR — the ffmpeg `AV_CRC_*` (big-endian)
//! variants. Each format transmits its CRC so that the residue over
//! `data + transmitted_crc` is zero, which is exactly how these are used:
//! compute over the whole frame (including its trailing CRC) and check `== 0`.
/// CRC-16/ANSI (a.k.a. CRC-16/BUYPASS): polynomial 0x8005, init 0x0000,
/// MSB-first, no reflection, no final XOR — ffmpeg `AV_CRC_16_ANSI`.
/// Used by AC-3/E-AC-3 (frame CRC), FLAC (frame footer), MPEG-audio and
/// AAC-ADTS (header CRC).
pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x8005
} else {
crc << 1
};
}
}
crc
}
/// CRC-16 with polynomial 0x002D, init 0, MSB-first — ffmpeg's `crc_2D` table
/// (`av_crc_init(crc_2D, 0, 16, 0x002D)`), used by the MLP/TrueHD major-sync
/// header checksum. NOTE: MLP's checksum is the "reversed" scheme — ffmpeg
/// computes `av_crc(...) ^ AV_RL16(trailer)` and compares against `AV_RL16` of
/// the stored word; equivalently, this standard CRC compared against the stored
/// bytes read big-endian. The caller handles that comparison
/// (see `truehd::mlp_major_sync_ok`). Verified against real ffmpeg TrueHD
/// output (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x002D
} else {
crc << 1
};
}
}
crc
}
/// CRC-8/ATM (a.k.a. CRC-8/ITU without the final XOR): polynomial 0x07, init 0,
/// MSB-first, no reflection — ffmpeg `AV_CRC_8_ATM`. Used by the FLAC frame
/// header.
pub(crate) fn crc8_atm(data: &[u8]) -> u8 {
let mut crc: u8 = 0;
for &b in data {
crc ^= b;
for _ in 0..8 {
crc = if crc & 0x80 != 0 {
(crc << 1) ^ 0x07
} else {
crc << 1
};
}
}
crc
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn crc16_residue_property_holds() {
// Appending the big-endian CRC-16 of a message zeroes the residue over
// message+crc — the property every frame gate relies on.
let msg = [0x12u8, 0x34, 0x56, 0x78, 0x9A];
let c = crc16_ansi(&msg);
let mut framed = msg.to_vec();
framed.push((c >> 8) as u8);
framed.push((c & 0xFF) as u8);
assert_eq!(crc16_ansi(&framed), 0);
}
#[test]
fn crc16_known_vector_check_bytes() {
// CRC-16/BUYPASS check value for the ASCII string "123456789" is 0xFEE8
// (the standard catalogue check value for poly 0x8005, init 0).
assert_eq!(crc16_ansi(b"123456789"), 0xFEE8);
}
#[test]
fn crc8_residue_property_holds() {
// Appending the CRC-8 of a message zeroes the residue over message+crc —
// how FLAC's header CRC-8 is verified.
let msg = [0xDEu8, 0xAD, 0xBE, 0xEF];
let c = crc8_atm(&msg);
let mut framed = msg.to_vec();
framed.push(c);
assert_eq!(crc8_atm(&framed), 0);
}
#[test]
fn crc8_known_vector_check_byte() {
// CRC-8/SMBUS (poly 0x07, init 0, no reflection) check value for
// "123456789" is 0xF4 — the catalogue check value.
assert_eq!(crc8_atm(b"123456789"), 0xF4);
}
}
+201
View File
@@ -0,0 +1,201 @@
//! Shared "keep what decodes, drop what doesn't" bookkeeping for the audio
//! codec parsers.
//!
//! The user's rule: a clean mux keeps every frame it can and drops the ones it
//! can't — video always survives (it's inter-frame predicted; a per-frame drop
//! would cascade, so video resyncs/conceals instead), audio keeps every
//! decodable access unit, and a damaged audio AU is dropped rather than shipped
//! as a decoder-choking glitch.
//!
//! The DETECTION is inherently per-codec — each format carries its own
//! authoritative corruption check (DTS: ffmpeg's core-header parse; AC-3: the
//! header CRC; FLAC: the frame CRC-16; …). This type only carries the UNIFORM
//! response so every audio parser behaves identically:
//!
//! 1. **Count** kept vs dropped AUs and the dropped duration.
//! 2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
//! once-per-track aggregate at `warn` so it surfaces without debug logging.
//! 3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
//! a poison flag so the remainder is dropped too (a track that damaged isn't
//! worth muxing).
//!
//! **Sync preservation is the caller's responsibility**, not this type's: the
//! parser must advance its PTS clock across a dropped AU exactly as it would for
//! an emitted one, so a drop becomes a silence gap and never a shift of the
//! following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
/// Minimum access units observed before the whole-track drop verdict can fire.
/// Below this, a short damaged burst can't poison an otherwise-good track.
const TRACK_VERDICT_MIN_AUS: u64 = 200;
/// Per-track drop bookkeeping shared by the audio codec parsers.
pub(crate) struct DropTally {
/// Static codec label for log lines (e.g. `"dts"`, `"ac3"`).
codec: &'static str,
kept: u64,
dropped: u64,
/// AUs dropped because they were INDIVIDUALLY verified undecodable (a failed
/// CRC/header/parity check). Only these feed the whole-track poison verdict.
/// Distinct from `dropped`, which also counts *collateral* drops — AUs
/// discarded as a consequence of one corruption (TrueHD's resync-forward run,
/// or a poisoned track), which must NOT amplify a few real errors into a
/// false whole-track loss.
verified_dropped: u64,
dropped_dur_ns: u64,
poisoned: bool,
}
impl DropTally {
pub(crate) fn new(codec: &'static str) -> Self {
Self {
codec,
kept: 0,
dropped: 0,
verified_dropped: 0,
dropped_dur_ns: 0,
poisoned: false,
}
}
/// Whether the track has been judged too damaged to mux. Once `true`, the
/// caller should drop every remaining AU (passing them to [`record_drop`]
/// with a poison reason) rather than emit them.
pub(crate) fn is_poisoned(&self) -> bool {
self.poisoned
}
/// Access units dropped as undecodable so far — surfaced to the CLI/mux.
pub(crate) fn dropped_frames(&self) -> u64 {
self.dropped
}
/// Total decoded duration (ns) of dropped AUs — the audio silence introduced.
pub(crate) fn dropped_duration_ns(&self) -> u64 {
self.dropped_dur_ns
}
/// Record an emitted (decodable) access unit.
pub(crate) fn record_kept(&mut self) {
self.kept += 1;
}
/// Record a dropped access unit that was INDIVIDUALLY verified undecodable
/// (a failed CRC/header/parity check). Counts toward the whole-track poison
/// verdict. `reason` is a short static label for the check that failed.
pub(crate) fn record_drop(&mut self, pts_ns: i64, dur_ns: i64, bytes: usize, reason: &str) {
self.verified_dropped += 1;
self.record_drop_common(pts_ns, dur_ns, bytes, reason);
self.maybe_poison();
}
/// Record a COLLATERAL drop — an AU discarded as a consequence of another
/// corruption rather than being individually undecodable (TrueHD's
/// resync-forward run to the next major sync, or an already-poisoned track).
/// Counted and logged for the drop report, but deliberately does NOT feed the
/// poison verdict, so one corruption event can't amplify into a false
/// whole-track loss.
pub(crate) fn record_collateral_drop(
&mut self,
pts_ns: i64,
dur_ns: i64,
bytes: usize,
reason: &str,
) {
self.record_drop_common(pts_ns, dur_ns, bytes, reason);
}
fn record_drop_common(&mut self, pts_ns: i64, dur_ns: i64, bytes: usize, reason: &str) {
self.dropped += 1;
self.dropped_dur_ns += dur_ns.max(0) as u64;
tracing::debug!(
target: "mux",
"{}: dropped undecodable AU #{} pts_ns={} dur_ns={} bytes={} reason={}",
self.codec,
self.dropped,
pts_ns,
dur_ns,
bytes,
reason
);
}
/// Whole-track fallback: after enough AUs to judge, if more than half were
/// dropped the track is too damaged to be worth muxing — latch `poisoned`
/// and log it loudly once. The minimum-sample gate keeps a short damaged
/// burst from poisoning an otherwise-good track.
fn maybe_poison(&mut self) {
if self.poisoned {
return;
}
// Judge on VERIFIED drops vs all AUs seen: a track is only poisoned when
// a majority of its access units are individually undecodable — not when
// a couple of corruption events forced long collateral resync runs.
let total = self.kept + self.dropped;
if total >= TRACK_VERDICT_MIN_AUS && self.verified_dropped * 2 > total {
self.poisoned = true;
tracing::warn!(
target: "mux",
"{}: track too damaged to mux — {}/{} AUs individually undecodable (>50%); dropping the whole track",
self.codec,
self.verified_dropped,
total
);
}
}
/// End-of-stream aggregate report, logged at `warn` so a track's dropped
/// audio is never hidden even without debug logging. No-op if nothing was
/// dropped.
pub(crate) fn log_summary(&self) {
if self.dropped > 0 {
tracing::warn!(
target: "mux",
"{}: dropped {} undecodable AU(s) totaling {} ns of audio ({} kept)",
self.codec,
self.dropped,
self.dropped_dur_ns,
self.kept
);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn counts_kept_and_dropped() {
let mut t = DropTally::new("test");
t.record_kept();
t.record_drop(0, 1000, 512, "bad");
t.record_kept();
assert_eq!(t.dropped_frames(), 1);
assert_eq!(t.dropped_duration_ns(), 1000);
assert!(!t.is_poisoned());
}
#[test]
fn poisons_after_min_aus_over_half_dropped() {
let mut t = DropTally::new("test");
// 199 AUs, all dropped: below the min-AU gate, must NOT poison yet.
for _ in 0..199 {
t.record_drop(0, 1000, 512, "bad");
}
assert!(!t.is_poisoned(), "below the 200-AU minimum, no verdict");
// The 200th drop reaches the minimum with >50% dropped → poison.
t.record_drop(0, 1000, 512, "bad");
assert!(t.is_poisoned());
}
#[test]
fn does_not_poison_a_mostly_good_track() {
let mut t = DropTally::new("test");
// 400 AUs, 1 dropped: nowhere near 50%.
t.record_drop(0, 1000, 512, "bad");
for _ in 0..399 {
t.record_kept();
}
assert!(!t.is_poisoned());
}
}
+535 -41
View File
@@ -49,6 +49,12 @@ pub struct DtsParser {
/// running cursor: previous emit + its duration). Only consulted when
/// `front_pts` is unchanged from `last_front_pts`. `PTS_UNSET` = no base yet.
next_pts_ns: i64,
/// Keep/drop bookkeeping for the decodability gate: counts, per-drop and
/// aggregate logging, and the whole-track poison fallback. A dropped AU is
/// NEVER emitted, but the PTS clock is still advanced across it (see
/// [`stamp_pts`] usage) so every SURVIVING AU keeps the exact timestamp it
/// would have had — a drop becomes a silence gap, never a shift.
tally: super::dropgate::DropTally,
}
impl Default for DtsParser {
@@ -65,6 +71,50 @@ impl DtsParser {
pts_marks: std::collections::VecDeque::new(),
last_front_pts: PTS_UNSET,
next_pts_ns: PTS_UNSET,
tally: super::dropgate::DropTally::new("dts"),
}
}
/// Number of access units dropped as undecodable so far. The mux/CLI reads
/// this to surface the count ("dropped N damaged DTS frames").
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of all dropped access units — the length of
/// audio silence introduced by dropping undecodable frames.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Gate an assembled access unit through the decodability check and either
/// push it or drop it. `au_pts`/`dur_ns` are already stamped on the shared
/// PTS clock (which the caller advances whether or not the AU survives), so
/// a drop leaves the following audio on its true timeline — a gap, not a
/// shift. Every drop is logged (fail-loud, never silent).
fn emit_or_drop(&mut self, au: Vec<u8>, au_pts: i64, dur_ns: i64, out: &mut Vec<Frame>) {
let verdict = if self.tally.is_poisoned() {
Err(DropReason::TrackPoisoned)
} else {
core_header_drop_reason(&au).map_or(Ok(()), Err)
};
match verdict {
Ok(()) => {
self.tally.record_kept();
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: au_pts,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
});
}
Err(reason) => {
self.tally
.record_drop(au_pts, dur_ns, au.len(), reason.as_str());
}
}
}
@@ -299,8 +349,8 @@ impl CodecParser for DtsParser {
// flush is an extension-substream PES, carrying its own later
// timestamp) must NOT become the next unit's PTS base.
let mut forced = false;
let au_end = match next_core_boundary(&self.buf, core_size) {
NextCore::Found(end) => end,
let (au_end, ext_clean) = match next_core_boundary(&self.buf, core_size) {
NextCore::Found { end, ext_clean } => (end, ext_clean),
NextCore::NeedMore => break, // candidate sync needs more header
NextCore::None => {
// No next core sync buffered yet. The trailing extension
@@ -312,27 +362,32 @@ impl CodecParser for DtsParser {
break;
}
forced = true;
self.buf.len()
(self.buf.len(), true)
}
};
let au: Vec<u8> = self.buf[..au_end].to_vec();
// Damaged source encoding: when the extension boundary was GARBAGE
// (not any DTS sync — `ext_clean == false`), the extension bytes for
// this AU are corrupt and would make the decoder cascade "DSYNC check
// failed" / "Read past end of XLL band data". Emit the clean DTS core
// ALONE (a decodable, lossy frame) and still drain past the garbage to
// the next core — a perfect mux drops the bad frame's corrupt part
// rather than shipping it. A recognized-but-unsizeable extension
// (`ext_clean == true`) is preserved in full (lossless).
let emit_end = if ext_clean { au_end } else { core_size };
let au: Vec<u8> = self.buf[..emit_end].to_vec();
// The AU's own core PES PTS (the PES covering its first byte, even if
// that PES preceded the one(s) carrying its extensions or the next
// core), stamped monotonically: honored when it advances past the
// running clock (UHD one-AU-per-PES), but never allowed to collide
// with the previous AU when several cores share ONE PES (DVD).
let dur_ns = dts_core_duration_ns(&au) as i64;
// Advance the PTS clock for this AU BEFORE the decodability gate, so
// a dropped AU still advances the timeline exactly as an emitted one
// would: the following AU keeps its true PTS and the drop is a gap,
// never a shift. `emit_or_drop` decides whether to actually push it.
let au_pts = self.stamp_pts(self.front_pts(), dur_ns);
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: au_pts,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
});
self.emit_or_drop(au, au_pts, dur_ns, &mut frames);
self.drain_front(au_end);
// After draining, the marker covering the new front (if any) carries
// the next AU's PTS; `pending_pts` is only the fallback when no
@@ -359,10 +414,23 @@ impl CodecParser for DtsParser {
}
fn flush(&mut self) -> Vec<Frame> {
// End of stream: emit the final access unit still buffered (the last
// core + its extension substreams, which had no following core sync to
// close it during streaming). Require a complete core frame; drop a
// bare partial sync tail.
let out = self.flush_tail();
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
impl DtsParser {
/// Emit the final buffered access unit (the last core + its extension
/// substreams, which had no following core sync to close it during
/// streaming), gated through the decodability check. Require a complete core
/// frame; drop a bare partial sync tail.
fn flush_tail(&mut self) -> Vec<Frame> {
if find_sync(&self.buf, &DTS_CORE_SYNC) != Some(0) || self.buf.len() < CORE_HEADER_MIN_BYTES
{
self.buf.clear();
@@ -381,19 +449,9 @@ impl CodecParser for DtsParser {
let dur_ns = dts_core_duration_ns(&au) as i64;
let pts_ns = self.stamp_pts(self.front_pts(), dur_ns);
self.pts_marks.clear();
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
}]
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
let mut out = Vec::new();
self.emit_or_drop(au, pts_ns, dur_ns, &mut out);
out
}
}
@@ -407,7 +465,14 @@ fn find_sync(data: &[u8], pattern: &[u8; 4]) -> Option<usize> {
/// Result of scanning for the next valid core sync that closes an access unit.
enum NextCore {
/// A valid next core sync was found; the access unit ends at this offset.
Found(usize),
/// `ext_clean` is `false` only when the byte at the extension boundary was
/// GARBAGE — neither a core sync nor a DTS-HD extension sync — meaning the
/// extension region is corrupt (damaged source encoding). The caller then
/// emits the clean DTS core alone and drops the garbage, instead of shipping
/// a corrupt AU that makes the decoder cascade DSYNC / "Read past end of XLL".
/// It stays `true` when the region is a real (if unsizeable) extension sync —
/// that path is load-bearing for valid streams and must NOT be dropped.
Found { end: usize, ext_clean: bool },
/// A candidate core sync was found but its header isn't fully buffered yet,
/// so its validity can't be decided — wait for more data.
NeedMore,
@@ -492,8 +557,11 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
pos += sz; // skip the whole extension substream precisely
}
// Couldn't size it (truncated/garbage header) — heuristic fallback.
_ => return scan_for_next_core(buf, pos),
// A real extension sync we couldn't size (truncated header /
// unsupported sub-form) — heuristic fallback, but the region IS a
// recognized extension, so keep it (ext_clean = true). This path
// is load-bearing for valid streams.
_ => return scan_for_next_core(buf, pos, true),
}
} else if buf[pos..].starts_with(&DTS_CORE_SYNC) {
// The bytes right after the precisely-skipped extensions are the next
@@ -503,13 +571,18 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
return NextCore::Found {
end: pos,
ext_clean: true,
};
}
return scan_for_next_core(buf, pos); // implausible core here — fall back
return scan_for_next_core(buf, pos, true); // implausible core — recognized sync, keep
} else {
// Neither a known extension nor a core sync at the precise boundary
// (padding / junk) — fall back to the heuristic scan.
return scan_for_next_core(buf, pos);
// GARBAGE at the extension boundary — neither a core sync nor a
// DTS-HD extension sync. This is damaged source encoding: the
// extension region is corrupt. Mark ext_clean = false so the caller
// emits the clean core alone and drops the garbage.
return scan_for_next_core(buf, pos, false);
}
}
}
@@ -518,7 +591,7 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
/// syncword whose decoded size is plausible. Used only when precise extension
/// skipping can't proceed; a chance core syncword in extension payload usually
/// decodes to an implausible size and is skipped.
fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
fn scan_for_next_core(buf: &[u8], from: usize, ext_clean: bool) -> NextCore {
let mut from = from;
while let Some(rel) = find_sync(&buf[from..], &DTS_CORE_SYNC) {
let pos = from + rel;
@@ -527,7 +600,10 @@ fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
return NextCore::Found {
end: pos,
ext_clean,
};
}
from = pos + SYNCWORD_BYTES;
}
@@ -597,6 +673,128 @@ fn dts_core_duration_ns(data: &[u8]) -> u64 {
(samples * 1_000_000_000 + rate / 2) / rate
}
/// DCA core-header constants, mirrored from ffmpeg `libavcodec/dca_core.h`.
/// `deficit_samples` must equal this (`DCA_PCMBLOCK_SAMPLES`); `npcmblocks`
/// must be a multiple of `DCA_SUBBAND_SAMPLES`; `audio_mode` must be below
/// `DCA_AMODE_COUNT`; `lfe_present == DCA_LFE_FLAG_INVALID` is rejected.
const DTS_PCMBLOCK_SAMPLES: u32 = 32;
const DTS_SUBBAND_SAMPLES: u32 = 8;
const DTS_AMODE_COUNT: u32 = 10;
const DTS_LFE_FLAG_INVALID: u32 = 3;
/// `ff_dca_sample_rates[16]` — sample rate (Hz) per core `SFREQ` code; a `0`
/// entry marks a reserved code that ffmpeg's parser rejects
/// (`DCA_PARSE_ERROR_SAMPLE_RATE`). Valid entries are locked to the spec by
/// `dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
/// {0, 4, 5, 9, 10}.
const DTS_CORE_SR_VALID: [u32; 16] = [
0, 8_000, 16_000, 32_000, 0, 0, 11_025, 22_050, 44_100, 0, 0, 12_000, 24_000, 48_000, 96_000,
192_000,
];
/// `ff_dca_bits_per_sample[8]` — a `0` entry marks a reserved `PCMR` code that
/// ffmpeg's parser rejects (`DCA_PARSE_ERROR_PCM_RES`); reserved codes are
/// {4, 7}.
const DTS_CORE_PCMR_BITS: [u8; 8] = [16, 16, 20, 20, 0, 24, 24, 0];
/// Why an access unit was judged undecodable. Each core-header variant is the
/// exact condition under which ffmpeg's `ff_dca_parse_core_frame_header` returns
/// the matching `DCA_PARSE_ERROR_*`; `TrackPoisoned` is our whole-track drop.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum DropReason {
DeficitSamples,
PcmBlocks,
FrameSize,
Amode,
SampleRate,
ReservedBit,
LfeFlag,
PcmRes,
TrackPoisoned,
}
impl DropReason {
/// Short static label for the drop log (the shared tally logs `&str`).
fn as_str(&self) -> &'static str {
match self {
DropReason::DeficitSamples => "deficit-samples",
DropReason::PcmBlocks => "pcm-blocks",
DropReason::FrameSize => "frame-size",
DropReason::Amode => "audio-mode",
DropReason::SampleRate => "sample-rate",
DropReason::ReservedBit => "reserved-bit",
DropReason::LfeFlag => "lfe-flag",
DropReason::PcmRes => "pcm-resolution",
DropReason::TrackPoisoned => "track-poisoned",
}
}
}
/// Decodability gate: a faithful port of ffmpeg's `ff_dca_parse_core_frame_header`
/// validity checks (libavcodec/dca.c). Returns `Some(reason)` when ffmpeg's own
/// parser would reject this core frame's header — in which case the packet is
/// undecodable ("Invalid data found") and dropping it loses nothing a decoder
/// could have used. Returns `None` (keep) for a decodable header OR if the
/// header can't be fully read (never false-drop on our own buffer underrun; the
/// framer only emits AUs whose core is fully buffered and ≥ 96 bytes).
///
/// The 4-byte core sync is already validated by the framer, so this reads the
/// header fields that follow it. ffmpeg's parser does NOT verify the CPF header
/// CRC (it `skip_bits(16)` past it — dca.c) and the core decoder likewise skips
/// the audio-header/side-info CRCs (dca_core.c), so no CRC check is mirrored
/// here: doing so would drop frames ffmpeg decodes fine (false positives).
fn core_header_drop_reason(au: &[u8]) -> Option<DropReason> {
let mut r = BitReader::new(au.get(SYNCWORD_BYTES..)?);
let _ftype = r.read_bit()?;
let deficit_samples = r.read_bits(5)? + 1;
if deficit_samples != DTS_PCMBLOCK_SAMPLES {
return Some(DropReason::DeficitSamples);
}
let crc_present = r.read_bit()? == 1;
let npcmblocks = r.read_bits(7)? + 1;
if npcmblocks & (DTS_SUBBAND_SAMPLES - 1) != 0 {
return Some(DropReason::PcmBlocks);
}
let frame_size = r.read_bits(14)? + 1;
if frame_size < MIN_CORE_FRAME_BYTES as u32 {
return Some(DropReason::FrameSize);
}
let audio_mode = r.read_bits(6)?;
if audio_mode >= DTS_AMODE_COUNT {
return Some(DropReason::Amode);
}
let sr_code = r.read_bits(4)? as usize;
if DTS_CORE_SR_VALID[sr_code] == 0 {
return Some(DropReason::SampleRate);
}
let _br_code = r.read_bits(5)?;
if r.read_bit()? != 0 {
return Some(DropReason::ReservedBit);
}
// drc, ts, aux, hdcd (1 each) → ext_audio_type (3) → ext_present, aspf (1 each).
r.skip_bits(4)?;
r.skip_bits(3)?;
r.skip_bits(2)?;
let lfe_present = r.read_bits(2)?;
if lfe_present == DTS_LFE_FLAG_INVALID {
return Some(DropReason::LfeFlag);
}
let _predictor_history = r.read_bit()?;
if crc_present {
// ffmpeg only skips the 16-bit header CRC here — it is not verified.
r.skip_bits(16)?;
}
let _filter_perfect = r.read_bit()?;
let _encoder_rev = r.read_bits(4)?;
let _copy_hist = r.read_bits(2)?;
let pcmr_code = r.read_bits(3)? as usize;
if DTS_CORE_PCMR_BITS[pcmr_code] == 0 {
return Some(DropReason::PcmRes);
}
None
}
#[cfg(test)]
mod tests {
use super::*;
@@ -617,6 +815,11 @@ mod tests {
let fsize = size - 1;
let mut data = vec![0u8; size];
data[0..4].copy_from_slice(&DTS_CORE_SYNC);
// byte4: FTYPE(0) SHORT(5) CPF(0) NBLKS-high(0). SHORT = 31 makes
// deficit_samples = 32 = DCA_PCMBLOCK_SAMPLES, which ffmpeg's parser
// (and our decodability gate) require of a real core frame. NBLKS high
// bit (byte4 bit0) stays 0 for NBLKS = 15.
data[4] = 31u8 << 2;
// NBLKS = 15 → (15+1)*32 = 512 samples/frame (the DVD/UHD DTS-core norm).
// NBLKS is byte4 bit0 + byte5 bits7-2; here byte4 bit0 = 0, byte5 = 15<<2.
data[5] = (15u8 << 2) | ((fsize >> 12) & 0x03) as u8;
@@ -676,12 +879,60 @@ mod tests {
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found(end) if end == core.len() + exss.len()
NextCore::Found { end, .. } if end == core.len() + exss.len()
),
"AU must end at the REAL next core (after the full EXSS), not the false sync inside it"
);
}
#[test]
fn garbage_extension_emits_core_only_but_valid_ext_is_kept() {
// Damaged source: a valid core, then GARBAGE (no core sync, no extension
// sync) where the extension belongs, then the next core. The framer must
// mark this boundary ext_clean=false and the parser must emit the clean
// 512-byte CORE alone (dropping the garbage), draining to the next core.
let core = make_dts_core(512);
let garbage = vec![0xE4, 0x3F, 0xE3, 0x90, 0xCC, 0x6C]; // real Bourne head bytes
let mut garbage = garbage;
garbage.extend(std::iter::repeat(0xAB).take(300));
let next = make_dts_core(512);
let mut buf = core.clone();
buf.extend_from_slice(&garbage);
buf.extend_from_slice(&next);
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found { end, ext_clean: false } if end == core.len() + garbage.len()
),
"garbage boundary must be flagged unclean"
);
let mut parser = DtsParser::new();
let mut frames = parser.parse(&make_pes(buf, Some(90000)));
frames.extend(parser.flush());
assert!(!frames.is_empty());
for f in &frames {
assert_eq!(f.data.len(), 512, "garbage-extension AU emits core only");
assert_eq!(&f.data[0..4], &DTS_CORE_SYNC);
}
// Contrast: a REAL extension sync (even if unsizeable) must be KEPT in
// full — ext_clean stays true, never downgraded to core-only.
let mut buf2 = make_dts_core(512);
buf2.extend_from_slice(&make_dts_ext(256));
buf2.extend_from_slice(&make_dts_core(512));
assert!(
matches!(
next_core_boundary(&buf2, 512),
NextCore::Found {
ext_clean: true,
..
}
),
"a recognized extension sync is preserved, not dropped"
);
}
#[test]
fn parse_empty_pes() {
let mut parser = DtsParser::new();
@@ -1478,4 +1729,247 @@ mod tests {
"sub-floor sync skipped, real 512 core is AU1"
);
}
/// A structurally-framed but UNDECODABLE core: a valid `make_dts_core`
/// whose reserved header bit is set. It still sizes and syncs correctly (so
/// the framer delimits it normally), but ffmpeg's `ff_dca_parse_core_frame_header`
/// — and our port — reject it (`DCA_PARSE_ERROR_RESERVED_BIT`). The reserved
/// bit is byte9 bit4 in the core header (after SYNC..RATE).
fn make_bad_dts_core(size: usize) -> Vec<u8> {
let mut d = make_dts_core(size);
assert!(
core_header_drop_reason(&d).is_none(),
"base core is decodable"
);
d[9] |= 0x10; // set the reserved bit
assert_eq!(
core_header_drop_reason(&d),
Some(DropReason::ReservedBit),
"reserved-bit core must be judged undecodable"
);
d
}
#[test]
fn valid_stream_drops_nothing() {
// A clean stream of decodable cores must pass the gate untouched — the
// detector is an exact ffmpeg-parity port, so zero false positives.
let mut parser = DtsParser::new();
let mut stream = Vec::new();
for _ in 0..5 {
stream.extend_from_slice(&make_dts_core(512));
}
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 5, "all five cores emitted");
assert_eq!(parser.dropped_frames(), 0, "nothing dropped");
assert_eq!(parser.dropped_duration_ns(), 0);
}
#[test]
fn undecodable_core_is_dropped_and_counted() {
// A single undecodable core between good ones is dropped; the survivors
// are emitted and the drop is counted.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512);
stream.extend_from_slice(&make_bad_dts_core(512));
stream.extend_from_slice(&make_dts_core(640));
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "the bad core is dropped, two survive");
assert_eq!(frames[0].data.len(), 512);
assert_eq!(frames[1].data.len(), 640);
assert_eq!(parser.dropped_frames(), 1);
assert_eq!(
parser.dropped_duration_ns(),
DTS_CORE_DUR_NS as u64,
"one frame's worth of audio silence introduced"
);
}
#[test]
fn drop_preserves_av_sync_no_shift() {
// THE INVARIANT: dropping an undecodable AU must never shift the audio
// that follows. A good/bad/good run in ONE PES — the bad middle core is
// dropped, but the trailing good core must keep the EXACT PTS it would
// have had with no drop (base + 2 frame durations), so the drop is a
// silence gap, not a shift.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512); // c1: good
stream.extend_from_slice(&make_bad_dts_core(512)); // c2: undecodable
stream.extend_from_slice(&make_dts_core(640)); // c3: good
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "c2 dropped; c1 and c3 survive");
let base = pts_to_ns(90000);
assert_eq!(frames[0].pts_ns, base, "c1 keeps the PES base PTS");
assert_eq!(
frames[1].pts_ns,
base + 2 * DTS_CORE_DUR_NS,
"c3 keeps its TRUE timeline (base + 2 frames) — the drop is a gap, not a shift"
);
// The gap between the survivors is exactly the dropped frame's duration
// beyond the normal one-frame spacing.
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
2 * DTS_CORE_DUR_NS,
"surviving AUs are spaced by the real timeline including the dropped frame's slot"
);
assert_eq!(parser.dropped_frames(), 1);
}
#[test]
fn whole_track_poison_drops_remainder() {
// A track dominated by undecodable frames is judged too damaged to mux:
// once the >50% verdict fires (after the minimum sample count), the
// whole track — including any later good frames — is dropped.
let mut parser = DtsParser::new();
let mut stream = Vec::new();
// 300 AUs, ~2/3 undecodable → well over the 50% threshold and the
// 200-AU minimum.
for i in 0..300 {
if i % 3 == 0 {
stream.extend_from_slice(&make_dts_core(512));
} else {
stream.extend_from_slice(&make_bad_dts_core(512));
}
}
// A trailing burst of GOOD cores that must be dropped once poisoned.
for _ in 0..20 {
stream.extend_from_slice(&make_dts_core(512));
}
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert!(
parser.tally.is_poisoned(),
"track poisoned by >50% drop rate"
);
// Once poisoned, later good cores are dropped too, so the kept count
// (kept = emitted survivors) is far below the ~120 good cores present.
let kept = frames.len() as u64;
assert!(
kept < 120,
"post-poison good frames also dropped (kept={kept})"
);
assert!(parser.dropped_frames() > 150, "majority dropped");
}
#[test]
fn sr_validity_table_marks_reserved_codes() {
// The core-header sample-rate validity table must have ZERO (reject) at
// exactly the reserved SFREQ codes {0,4,5,9,10} and a real rate
// elsewhere — this is what mirrors ffmpeg's DCA_PARSE_ERROR_SAMPLE_RATE.
for code in 0..16usize {
let reserved = matches!(code, 0 | 4 | 5 | 9 | 10);
assert_eq!(
DTS_CORE_SR_VALID[code] == 0,
reserved,
"SFREQ code {code} reserved={reserved}"
);
}
}
#[test]
fn every_core_header_error_class_is_detected() {
// Exercise each ffmpeg-parity rejection so the port stays faithful.
// Start from a decodable core and corrupt one field at a time.
let good = make_dts_core(512);
assert_eq!(core_header_drop_reason(&good), None);
// deficit_samples != 32: clear SHORT (byte4 bits6-2) → deficit = 1.
let mut d = good.clone();
d[4] &= !0x7C;
assert_eq!(
core_header_drop_reason(&d),
Some(DropReason::DeficitSamples)
);
// npcmblocks not a multiple of 8: NBLKS low bits (byte5 bits7-2) → 14
// (npcmblocks=15, 15 & 7 = 7 ≠ 0).
let mut d = good.clone();
d[5] = (d[5] & 0x03) | (14u8 << 2);
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmBlocks));
// audio_mode >= 10: AMODE = byte7 bits3-0 (high 4) + byte8 bits7-6. Set
// AMODE high nibble to 0xF → audio_mode >= 60.
let mut d = good.clone();
d[7] |= 0x0F;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::Amode));
// sample_rate reserved: SFREQ (byte8 bits5-2) = 0.
let mut d = good.clone();
d[8] &= !0x3C;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::SampleRate));
// reserved bit set: byte9 bit4.
let mut d = good.clone();
d[9] |= 0x10;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::ReservedBit));
// lfe_present == 3: LFE is byte10 bits2-1.
let mut d = good.clone();
d[10] |= 0x06;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::LfeFlag));
// pcmr_code reserved (7): pcmr is byte11 bit0 + byte12 bits7-6 → set all
// three to 1 (code 7 → ff_dca_bits_per_sample[7] = 0).
let mut d = good.clone();
d[11] |= 0x01;
d[12] |= 0xC0;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmRes));
}
/// Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
/// through `DtsParser` and writes the emitted access units back out, so the
/// garbage-extension → core-only drop can be validated against an actual
/// damaged stream (e.g. the extracted Bourne DTS-HD MA track) end-to-end
/// with ffmpeg. Env: `DTS_IN` (input), `DTS_OUT` (output).
/// cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
#[test]
#[ignore]
fn reparse_real_dts_file() {
use std::io::Write;
let inp = std::env::var("DTS_IN").expect("DTS_IN");
let outp = std::env::var("DTS_OUT").expect("DTS_OUT");
let bytes = std::fs::read(&inp).expect("read DTS_IN");
let mut parser = DtsParser::new();
let mut out =
std::io::BufWriter::new(std::fs::File::create(&outp).expect("create DTS_OUT"));
let mut au_count = 0usize;
let mut out_bytes = 0usize;
// 90 kHz PTS advancing per chunk; arbitrary chunking is faithful because
// the framer resyncs on core sync and buffers across PES boundaries.
let mut pts: i64 = 90_000;
const CHUNK: usize = 64 * 1024;
for chunk in bytes.chunks(CHUNK) {
let pes = PesPacket {
source: None,
pid: 0x1100,
pts: Some(pts),
dts: None,
data: chunk.to_vec(),
discontinuity: false,
};
pts += 2_100; // ~one AU worth; value irrelevant to AU framing/bytes
for f in parser.parse(&pes) {
au_count += 1;
out_bytes += f.data.len();
out.write_all(&f.data).expect("write AU");
}
}
for f in parser.flush() {
au_count += 1;
out_bytes += f.data.len();
out.write_all(&f.data).expect("write AU");
}
out.flush().expect("flush");
eprintln!(
"REPARSE in={} bytes -> out={} bytes across {} AUs ({} bytes dropped)",
bytes.len(),
out_bytes,
au_count,
bytes.len().saturating_sub(out_bytes)
);
}
}
+218
View File
@@ -0,0 +1,218 @@
//! FLAC elementary-stream decodability gate.
//!
//! FLAC frames carry no length field, so a raw stream is delimited only by
//! sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
//! it comes from mp4/mkv, where each packet is exactly one container-delimited
//! FLAC frame (the `PARSER_FLAG_COMPLETE_FRAMES` case in ffmpeg). So this parser
//! is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
//! (poly 0x8005) computed so the residue over the whole frame is zero
//! (ffmpeg `flac_decode_frame`, `av_crc(AV_CRC_16_ANSI, 0, buf, len) == 0`). A
//! nonzero residue is definitive corruption → drop the frame (a silence gap,
//! never a shift — each packet keeps its own PTS), logged via the shared tally.
//!
//! A packet that does not begin with the FLAC frame sync is not a delimited
//! frame we can validate, so it is passed through unchanged (never false-dropped).
use super::crc::crc16_ansi;
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// FLAC frame sync: 14-bit code `0x3FFE` + a mandatory-0 reserved bit; the next
/// bit (blocking strategy) is masked off. ffmpeg tests `(AV_RB16 & 0xFFFE) ==
/// 0xFFF8` (flac_parser.c).
fn has_flac_sync(data: &[u8]) -> bool {
data.len() >= 2 && ((u16::from(data[0]) << 8 | u16::from(data[1])) & 0xFFFE) == 0xFFF8
}
/// Block-size code → samples, `ff_flac_blocksize_table` (0 = reserved/explicit).
const FLAC_BLOCKSIZE_TABLE: [u32; 16] = [
0, 192, 576, 1152, 2304, 4608, 0, 0, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768,
];
/// Sample-rate code → Hz, `ff_flac_sample_rate_table` (0 = STREAMINFO/explicit).
const FLAC_SAMPLE_RATE_TABLE: [u32; 16] = [
0, 88_200, 176_400, 192_000, 8_000, 16_000, 22_050, 24_000, 32_000, 44_100, 48_000, 96_000, 0,
0, 0, 0,
];
/// Best-effort duration (ns) of a FLAC frame from its header block-size and
/// sample-rate codes (byte 2). Only the table-coded cases are resolved; the
/// explicit-in-trailing-bytes codes (block 6/7, rate 12/13/14) and
/// STREAMINFO-derived (code 0) return `None`. Used only for the dropped-audio
/// accounting, so a `None` (→ 0) is harmless.
fn flac_frame_duration_ns(frame: &[u8]) -> Option<i64> {
if frame.len() < 3 {
return None;
}
let bs_code = (frame[2] >> 4) & 0x0F;
let sr_code = frame[2] & 0x0F;
let blocksize = FLAC_BLOCKSIZE_TABLE[bs_code as usize];
let rate = FLAC_SAMPLE_RATE_TABLE[sr_code as usize];
if blocksize == 0 || rate == 0 {
return None;
}
Some((blocksize as i64 * 1_000_000_000 + rate as i64 / 2) / rate as i64)
}
pub struct FlacParser {
tally: DropTally,
}
impl Default for FlacParser {
fn default() -> Self {
Self::new()
}
}
impl FlacParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("flac"),
}
}
/// Access units dropped as undecodable so far.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for FlacParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
// Gate: a packet that begins with a FLAC frame sync but whose whole-frame
// CRC-16 residue is nonzero is corrupt → drop. Anything else passes
// through (a non-sync packet is not a frame we can validate; a poisoned
// track drops everything).
let corrupt = has_flac_sync(&pes.data) && crc16_ansi(&pes.data) != 0;
if self.tally.is_poisoned() || corrupt {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"crc"
};
let dur = flac_frame_duration_ns(&pes.data).unwrap_or(0);
self.tally.record_drop(pts_ns, dur, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A minimal FLAC-frame-shaped buffer: sync `0xFFF8`, a plausible header
/// (block code 1 = 192 samples, rate code 9 = 44.1 kHz), some payload, and a
/// trailing CRC-16 so the whole-frame residue is zero (a valid frame).
fn make_flac_frame(payload_len: usize) -> Vec<u8> {
let mut f = vec![0u8; 6 + payload_len + 2];
f[0] = 0xFF;
f[1] = 0xF8; // sync + fixed blocksize
f[2] = (1 << 4) | 9; // bs_code=1 (192), sr_code=9 (44100)
// bytes 3..end-2 arbitrary; last two bytes carry the CRC-16.
let n = f.len();
let c = crc16_ansi(&f[..n - 2]);
f[n - 2] = (c >> 8) as u8;
f[n - 1] = (c & 0xFF) as u8;
assert_eq!(crc16_ansi(&f), 0, "finalized frame has zero residue");
f
}
#[test]
fn valid_frame_is_kept() {
let mut p = FlacParser::new();
let f = p.parse(&make_pes(make_flac_frame(100), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn corrupt_frame_is_dropped() {
let mut p = FlacParser::new();
let mut frame = make_flac_frame(100);
frame[20] ^= 0xFF; // corrupt a payload byte → CRC residue nonzero
assert!(crc16_ansi(&frame) != 0);
let f = p.parse(&make_pes(frame, Some(90000)));
assert!(f.is_empty(), "corrupt FLAC frame dropped");
assert_eq!(p.dropped_frames(), 1);
// 192 samples @ 44.1 kHz ≈ 4.354 ms of silence accounted.
assert_eq!(
p.dropped_duration_ns(),
(192u64 * 1_000_000_000 + 44_100 / 2) / 44_100
);
}
#[test]
fn corrupt_drop_preserves_sync_via_own_pts() {
// Each packet carries its own PTS, so dropping one leaves the next frame
// on its true timeline — a gap, not a shift.
let mut p = FlacParser::new();
let mut bad = make_flac_frame(100);
bad[20] ^= 0xFF;
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(make_flac_frame(100), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"surviving frame keeps its own container PTS — the drop is a gap"
);
}
#[test]
fn non_flac_packet_passes_through() {
// A packet without the FLAC sync isn't a frame we can validate — never
// false-drop it.
let mut p = FlacParser::new();
let f = p.parse(&make_pes(vec![0x00, 0x01, 0x02, 0x03], Some(0)));
assert_eq!(f.len(), 1, "unrecognized packet passed through");
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn empty_pes_emits_nothing() {
let mut p = FlacParser::new();
assert!(p.parse(&make_pes(Vec::new(), Some(0))).is_empty());
}
}
+33 -3
View File
@@ -9,12 +9,19 @@
/// AC-3 / E-AC-3 (Dolby Digital / Digital Plus) elementary-stream parser.
pub mod ac3;
pub mod adts;
/// Codec-agnostic per-picture coding carrier (`PictureInfo` + accessors).
pub mod coding;
/// DTS / DTS-HD elementary-stream parser.
pub(crate) mod crc;
pub(crate) mod dropgate;
pub mod dts;
/// DVD bitmap subtitle (VobSub) parser.
pub mod dvdsub;
pub mod flac;
/// H.264 (AVC) Annex-B elementary-stream parser.
pub mod h264;
/// HEVC (H.265) Annex-B elementary-stream parser.
@@ -23,6 +30,8 @@ pub mod hevc;
pub mod lpcm;
/// MPEG-2 Video elementary-stream parser.
pub mod mpeg2;
pub mod mpegaudio;
/// HDMV PGS (Presentation Graphics Stream) subtitle parser.
pub mod pgs;
/// Display-order PTS reconstruction for sparse-PTS program-stream video.
@@ -154,6 +163,26 @@ impl CodecParser for PassthroughParser {
}
}
/// Drop-on-undecodable policy across codecs ("clean muxes always"):
///
/// - **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
/// through a per-codec corruption check and drops the ones that fail, keeping
/// A/V sync (a drop is a silence gap, never a shift) and logging every drop
/// via the shared [`dropgate::DropTally`]. DTS uses ffmpeg's core-header parse;
/// AC-3 uses its native frame CRC.
/// - **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
/// data, so a corrupt sample is indistinguishable from a quiet one — there is
/// nothing to detect, so nothing can be honestly dropped.
/// - **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
/// predicted, so dropping one frame corrupts every frame that references it
/// until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
/// ResyncGate) and lets the decoder conceal — a fundamentally different model
/// than per-frame audio dropping.
/// - TrueHD/MLP and the rare passthrough audio codecs (FLAC/MP2/AAC) do not yet
/// gate: MLP carries inter-AU restart state so a safe drop must land on a
/// major-sync boundary, and the passthrough codecs are essentially never seen
/// on optical media.
///
/// Create the appropriate parser for a codec, with optional codec private data.
///
/// For DvdSub, `codec_data` should be the pre-formatted VobSub .idx palette header.
@@ -177,6 +206,9 @@ pub fn parser_for_codec(
Codec::Mpeg2 => Box::new(mpeg2::Mpeg2Parser::new()),
Codec::Vc1 => Box::new(vc1::Vc1Parser::new().with_ps_reorder(is_dvd_ps)),
Codec::Ac3 | Codec::Ac3Plus => Box::new(ac3::Ac3Parser::new()),
Codec::Flac => Box::new(flac::FlacParser::new()),
Codec::Mp2 | Codec::Mp3 => Box::new(mpegaudio::MpegAudioParser::new()),
Codec::Aac => Box::new(adts::AdtsParser::new()),
Codec::DtsHdMa | Codec::DtsHdHr | Codec::Dts => Box::new(dts::DtsParser::new()),
Codec::TrueHd => Box::new(truehd::TrueHdParser::new()),
Codec::Pgs => Box::new(pgs::PgsParser::new()),
@@ -200,9 +232,7 @@ pub fn parser_for_codec(
// Remaining audio-only codecs (Aac, Mp2, Mp3, Flac, Opus) where PES =
// frame: all-keyframe passthrough is correct. Subtitle/Unknown also land
// here; keyframe flag is irrelevant for them.
Codec::Aac | Codec::Mp2 | Codec::Mp3 | Codec::Flac | Codec::Opus => {
Box::new(PassthroughParser::new(true))
}
Codec::Opus => Box::new(PassthroughParser::new(true)),
Codec::Srt | Codec::Ssa | Codec::Unknown(_) => Box::new(PassthroughParser::new(true)),
}
}
+224
View File
@@ -0,0 +1,224 @@
//! MPEG-1/2/2.5 audio (MP1/MP2/MP3) decodability gate.
//!
//! ffmpeg validates MPEG-audio frames by header sanity + framing resync, not a
//! payload CRC (`mpegaudiodecheader.c` `ff_mpa_check_header`; the optional CRC
//! covers only side-info and is off by default). Its `ff_mpa_decode_header`
//! additionally rejects free-format (`bitrate_index == 0`). So the gate mirrors
//! exactly those header rejects: a packet that begins with the 11-bit MPEG-audio
//! sync but whose version / layer / bitrate-index / sample-rate fields are the
//! reserved/invalid values is undecodable → drop it (a silence gap; each packet
//! keeps its own PTS). A packet with no leading sync is not a frame we can
//! validate (raw payload / continuation), so it passes through unchanged —
//! never false-dropped.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// Decoded validity of a candidate MPEG-audio header.
enum MpaVerdict {
/// No 11-bit sync at the packet head — not a frame we can validate.
NoSync,
/// Sync present and every field is legal — decodable.
Valid,
/// Sync present but a field is reserved/invalid (or free-format) — ffmpeg's
/// parser rejects this exactly.
Invalid,
}
/// Mirror ffmpeg's `ff_mpa_check_header` + the `ff_mpa_decode_header`
/// free-format reject. A dropped MPEG-audio frame has a corrupt header, so no
/// duration is computed (the fields it would come from are the invalid ones).
fn mpa_verdict(data: &[u8]) -> MpaVerdict {
if data.len() < 4 {
return MpaVerdict::NoSync;
}
let h = u32::from_be_bytes([data[0], data[1], data[2], data[3]]);
// 11-bit sync (0x7FF at the top).
if (h & 0xffe0_0000) != 0xffe0_0000 {
return MpaVerdict::NoSync;
}
// ff_mpa_check_header rejects: version field 01, layer field 00,
// bitrate_index 15, sample-rate field 3.
if (h & (3 << 19)) == (1 << 19)
|| (h & (3 << 17)) == 0
|| (h & (0xf << 12)) == (0xf << 12)
|| (h & (3 << 10)) == (3 << 10)
{
return MpaVerdict::Invalid;
}
// NOTE: bitrate_index == 0 (free format) is NOT rejected. It is a legal,
// decodable MPEG-audio mode (ffmpeg's ff_mpa_check_header accepts it and the
// decoder derives the frame size from the sync spacing). Dropping it would be
// a false positive on a clean stream, so it passes the gate.
MpaVerdict::Valid
}
pub struct MpegAudioParser {
tally: DropTally,
}
impl Default for MpegAudioParser {
fn default() -> Self {
Self::new()
}
}
impl MpegAudioParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("mpegaudio"),
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for MpegAudioParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
let drop =
self.tally.is_poisoned() || matches!(mpa_verdict(&pes.data), MpaVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid MPEG-1 Layer III header: sync 0xFFF, version MPEG-1 (11), layer
/// III (01), bitrate_index 9, sample-rate 0 (44.1 kHz), no CRC. Bytes:
/// 0xFF 0xFB 0x90 0x00 — the canonical MP3 frame header.
fn mp3_frame(payload: usize) -> Vec<u8> {
let mut f = vec![0xFF, 0xFB, 0x90, 0x00];
f.extend(std::iter::repeat(0xAA).take(payload));
f
}
#[test]
fn valid_header_is_kept() {
let mut p = MpegAudioParser::new();
let f = p.parse(&make_pes(mp3_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn reserved_sample_rate_is_dropped() {
// Sync present but sample-rate field = 3 (reserved) → ffmpeg rejects.
// 0xFF 0xFB then byte2 with bits 11..10 = 11: 0x9C.
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0x9C; // freq field = 3
let f = p.parse(&make_pes(frame, Some(90000)));
assert!(f.is_empty(), "reserved sample rate dropped");
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn reserved_layer_is_dropped() {
// Layer field 00 (reserved). byte1 bits 2..1 = 00 → 0xF9 keeps sync
// (0xFFF needs byte1 top 3 bits set) and sets layer=00.
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[1] = 0xF9; // 1111_1001: sync ok (top 3 =111), version 11, layer 00
let f = p.parse(&make_pes(frame, Some(0)));
assert!(f.is_empty(), "reserved layer dropped");
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn bad_bitrate_index_15_is_dropped() {
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0xF0; // bitrate_index = 1111
assert!(p.parse(&make_pes(frame, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn free_format_bitrate_zero_is_kept() {
// Free format (bitrate_index == 0) is legal and decodable — it must NOT
// be dropped (that would be a false positive on a clean stream).
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0x00; // bitrate_index = 0000 (free format); sync/layer/rate ok
let f = p.parse(&make_pes(frame, Some(0)));
assert_eq!(f.len(), 1, "free-format frame kept");
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn non_sync_packet_passes_through() {
// No 11-bit sync → not a validatable frame → keep (conservative).
let mut p = MpegAudioParser::new();
let f = p.parse(&make_pes(vec![0x00, 0x11, 0x22, 0x33, 0x44], Some(0)));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = MpegAudioParser::new();
let mut bad = mp3_frame(400);
bad[2] = 0x9C; // reserved sample rate
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(mp3_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
}
+119
View File
@@ -30,6 +30,90 @@ const MAX_PGS_PENDING_BYTES: usize = 4 * 1024 * 1024;
// (video_w/h, frame_rate, comp_num, comp_state, palette_update,
// palette_id_ref) = 13.
const PCS_NUM_OBJECTS_OFFSET: usize = 13;
// Offset of the first composition_object's flags byte within a PCS PES payload:
// PCS header(13) + number_of_composition_objects(1) + object_id_ref(2) +
// window_id_ref(1) = 17. `forced_on_flag` is bit 0x40 of that byte (HDMV PCS).
const PCS_FIRST_OBJECT_FLAGS_OFFSET: usize = 17;
const PCS_FORCED_ON_FLAG: u8 = 0x40;
/// Whether an emitted PGS display-set frame is a FORCED subtitle — the
/// `forced_on_flag` (0x40) on its first composition object. The frame data an
/// emitted PGS block carries begins with the display PCS (segment type 0x16), so
/// the flag is read directly from it. Returns `None` when the block is not a
/// display PCS with a composition object (nothing to classify — a clear PCS, a
/// non-PCS segment, or a truncated header).
///
/// The mux uses this to detect a *forced-narrative track* (every displayed
/// subtitle forced) without relying on the disc's vendor label metadata, so
/// forced subs are flagged `FlagForced` even on discs that carry no such blob.
pub fn display_set_is_forced(frame_data: &[u8]) -> Option<bool> {
if frame_data.first() != Some(&SEGMENT_PCS) {
return None;
}
if *frame_data.get(PCS_NUM_OBJECTS_OFFSET)? == 0 {
return None; // clear PCS — no composition to classify
}
let flags = *frame_data.get(PCS_FIRST_OBJECT_FLAGS_OFFSET)?;
Some(flags & PCS_FORCED_ON_FLAG != 0)
}
/// Accumulates the "is this PGS subtitle track a forced-narrative track?" verdict
/// from its display sets. A track is forced iff it displayed at least one subtitle
/// and EVERY display set carried the forced_on_flag — a dedicated forced track,
/// as opposed to a full track that merely has occasional forced signs.
///
/// This is the SINGLE classification used by both the MKV muxer (accumulating a
/// track's frames during a rip) and the `info`-time forced probe (feeding the
/// demuxed display sets), so both reach the identical verdict.
#[derive(Debug, Clone)]
pub struct ForcedTracker {
has_display: bool,
all_forced: bool,
}
impl Default for ForcedTracker {
fn default() -> Self {
Self {
has_display: false,
all_forced: true,
}
}
}
impl ForcedTracker {
pub fn new() -> Self {
Self::default()
}
/// Fold one emitted PGS block into the verdict. Non-display blocks (clear
/// PCS, other segments) are ignored.
pub fn observe(&mut self, frame_data: &[u8]) {
if let Some(forced) = display_set_is_forced(frame_data) {
self.has_display = true;
self.all_forced &= forced;
}
}
/// Whether the track has already shown a NON-forced subtitle — i.e. its
/// verdict is settled at "not forced" and further observation can be skipped
/// (the early-exit the probe uses to avoid reading the whole clip).
pub fn settled_not_forced(&self) -> bool {
self.has_display && !self.all_forced
}
/// Whether ANY display set was observed. When false the track's forced state
/// is unknown (no PGS content seen — e.g. an undecrypted/unread stream), so a
/// probe should leave any existing (vendor-derived) flag untouched rather
/// than assert "not forced".
pub fn observed(&self) -> bool {
self.has_display
}
/// Final verdict: forced iff it displayed subtitles and every one was forced.
pub fn is_forced(&self) -> bool {
self.has_display && self.all_forced
}
}
/// Stateful parser that collapses PGS display/clear PCS pairs into
/// duration-bearing Matroska frames. Implements [`CodecParser`].
@@ -222,6 +306,41 @@ mod tests {
use super::*;
use crate::mux::ts::PesPacket;
/// A PCS display-set block with one composition object; `forced` sets the
/// forced_on_flag (0x40) in its flags byte at offset 17.
fn pcs_display(forced: bool) -> Vec<u8> {
let mut d = vec![0u8; 18];
d[0] = SEGMENT_PCS;
d[PCS_NUM_OBJECTS_OFFSET] = 1;
d[PCS_FIRST_OBJECT_FLAGS_OFFSET] = if forced { PCS_FORCED_ON_FLAG } else { 0 };
d
}
#[test]
fn display_set_forced_flag_detection() {
assert_eq!(display_set_is_forced(&pcs_display(true)), Some(true));
assert_eq!(display_set_is_forced(&pcs_display(false)), Some(false));
// Other flag bits set but not forced_on_flag → still not forced.
let mut cropped = pcs_display(false);
cropped[PCS_FIRST_OBJECT_FLAGS_OFFSET] = 0x80; // object_cropped_flag only
assert_eq!(display_set_is_forced(&cropped), Some(false));
}
#[test]
fn display_set_forced_none_for_non_display() {
// Clear PCS (0 objects) → None.
let mut clear = pcs_display(false);
clear[PCS_NUM_OBJECTS_OFFSET] = 0;
assert_eq!(display_set_is_forced(&clear), None);
// Non-PCS segment → None.
let mut ods = pcs_display(true);
ods[0] = 0x15; // ODS
assert_eq!(display_set_is_forced(&ods), None);
// Truncated (no flags byte) → None, no panic.
assert_eq!(display_set_is_forced(&pcs_display(true)[..15]), None);
assert_eq!(display_set_is_forced(&[]), None);
}
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
+493 -20
View File
@@ -12,6 +12,8 @@
//! AC-3 frames (interleaved, same PID): start with sync word 0x0B77.
//! We skip AC-3 frames and only emit TrueHD access units.
use super::crc::crc16_mlp;
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
use crate::mux::timeline::DISCONTINUITY_BACKSTEP_NS;
@@ -43,6 +45,18 @@ pub struct TrueHdParser {
/// yet seen (head of stream) — preserving byte-identical timing for the
/// common 48 kHz case.
au_duration_ns: i64,
/// Keep/drop bookkeeping for the decodability gate.
tally: DropTally,
/// `num_substreams` from the most recent major sync — needed to size the
/// substream directory for the per-AU parity check. `None` until the first
/// major sync is seen (before which no AU can be parity-checked).
num_substreams: Option<u8>,
/// True while dropping forward to the next clean resync point. MLP/TrueHD
/// carries filter/predictor + restart state ACROSS access units, so a corrupt
/// AU cannot be excised in place — it poisons decoding until the next major
/// sync re-initialises state. On corruption we set this and drop every AU
/// until a major sync whose header CRC validates, which we then emit.
resync_pending: bool,
}
impl Default for TrueHdParser {
@@ -57,6 +71,64 @@ impl TrueHdParser {
buf: Vec::with_capacity(32768),
next_pts_ns: 0,
au_duration_ns: AU_DURATION_NS,
tally: DropTally::new("truehd"),
num_substreams: None,
resync_pending: false,
}
}
/// Access units dropped as undecodable so far.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Decide whether an access unit is corrupt, updating `num_substreams` from a
/// valid major sync. Mirrors ffmpeg `read_access_unit`: a major sync with a
/// bad header CRC, or any AU whose header parity fails, is undecodable.
/// Returns `false` (not corrupt) when the AU is too short to judge or no
/// major sync has established `num_substreams` yet — we never drop what we
/// cannot verify. Verified against real ffmpeg TrueHD (3600/3600 AUs).
fn au_check(&mut self, au: &[u8], is_major_sync: bool) -> AuCheck {
let mut header_size = 4;
let mut format_info = None;
if is_major_sync {
let ms = &au[4..];
let Some(mshdr) = mlp_major_sync_header_size(ms) else {
// A major sync too short to hold its header can't be CRC-validated
// — NOT a safe resync/re-init point. Treat as unverifiable, not a
// clean major sync.
return AuCheck::Unverifiable;
};
if !mlp_major_sync_crc_ok(ms, mshdr) {
return AuCheck::Corrupt; // corrupt major-sync header
}
self.num_substreams = mlp_num_substreams(ms);
header_size += mshdr;
// The rate nibble is only trustworthy once the major sync's CRC has
// validated (above), so capture format_info here and refine the PTS
// cadence from it ONLY on this validated path.
if au.len() >= 12 {
format_info = Some(u32::from_be_bytes([au[8], au[9], au[10], au[11]]));
}
}
let Some(nss) = self.num_substreams else {
return AuCheck::Unverifiable; // no major sync seen yet — can't check parity
};
let Some(shs) = mlp_substr_header_size(au, header_size, nss) else {
return AuCheck::Unverifiable; // directory runs off the AU — can't judge
};
if !mlp_parity_ok(au, header_size, shs) {
return AuCheck::Corrupt;
}
if is_major_sync {
AuCheck::ValidMajorSync { format_info }
} else {
AuCheck::Ok
}
}
@@ -114,6 +186,23 @@ fn ac3_boundary_corroborated(buf: &[u8], frame_bytes: usize) -> bool {
next_words != 0 && next_words * 2 <= 32768
}
/// Decodability verdict for one TrueHD/MLP access unit.
enum AuCheck {
/// Verified undecodable: a major-sync header whose CRC failed, or any AU
/// whose substream-directory parity failed. Feeds the poison verdict.
Corrupt,
/// A CRC-validated major sync — a safe re-init / resync point. `format_info`
/// (AU bytes 8..12, present when the AU is long enough) is trustworthy here,
/// so the caller refines the PTS cadence ONLY from this validated path.
ValidMajorSync { format_info: Option<u32> },
/// A valid (parity-OK) non-major-sync access unit.
Ok,
/// Cannot be judged — a major sync too short to hold/CRC its header, or a
/// stream head before any major sync established `num_substreams`. Never
/// dropped on its own, and never treated as a clean resync point.
Unverifiable,
}
/// Outcome of sizing the AC-3 frame at the TrueHD buffer head.
enum Ac3Size {
/// fscod/frmsizecod don't map to a real frame size — resync, don't wait.
@@ -124,6 +213,80 @@ enum Ac3Size {
Frame(usize),
}
// --- MLP/TrueHD access-unit integrity (mirrors ffmpeg mlpdec.c / mlp_parse.c) ---
/// Major-sync header size in bytes: base 28, plus `2 + extensions*2` when the
/// extension flag (major-sync byte 25, bit 0) is set (`extensions` = byte 26
/// high nibble). `ms` is the major-sync header, i.e. AU bytes `[4..]`. `None`
/// when the AU is too short to contain the full header.
fn mlp_major_sync_header_size(ms: &[u8]) -> Option<usize> {
if ms.len() < 28 {
return None;
}
let mut size = 28;
if ms[25] & 1 != 0 {
size += 2 + ((ms[26] >> 4) as usize) * 2;
}
if ms.len() < size {
return None;
}
Some(size)
}
/// Validate the MLP/TrueHD major-sync header checksum (ffmpeg `ff_mlp_checksum16`,
/// CRC-16 poly 0x002D). The stored trailer is the last 2 header bytes; because
/// MLP's checksum is byte-reversed relative to a standard CRC, a standard CRC of
/// the header body XOR the little-endian word before the trailer must equal the
/// trailer read big-endian.
fn mlp_major_sync_crc_ok(ms: &[u8], mshdr: usize) -> bool {
if mshdr < 4 || ms.len() < mshdr {
return false;
}
let crc = crc16_mlp(&ms[..mshdr - 4]) ^ u16::from_le_bytes([ms[mshdr - 4], ms[mshdr - 3]]);
crc == u16::from_be_bytes([ms[mshdr - 2], ms[mshdr - 1]])
}
/// `num_substreams` from a major-sync header: it sits at bit 128 (byte 16, top
/// nibble) for both MLP (0xbb) and TrueHD (0xba) — the fields before it total
/// the same 128 bits in either layout.
fn mlp_num_substreams(ms: &[u8]) -> Option<u8> {
ms.get(16).map(|&b| b >> 4)
}
/// Size in bytes of the substream directory that follows the AU header: each of
/// the `num_substreams` entries is 2 bytes, plus 2 more when its extraword flag
/// (entry's top bit) is set. `None` if the directory runs past the AU.
fn mlp_substr_header_size(au: &[u8], header_size: usize, num_substreams: u8) -> Option<usize> {
let mut off = header_size;
let mut shs = 0;
for _ in 0..num_substreams {
if off + 2 > au.len() {
return None;
}
let extraword = au[off] & 0x80 != 0;
shs += 2;
off += 2;
if extraword {
shs += 2;
off += 2;
}
}
Some(shs)
}
/// MLP/TrueHD AU-header parity check (ffmpeg `ff_mlp_calculate_parity`): the XOR
/// of the 4-byte AU header with the substream directory, folded, must have its
/// two nibbles XOR to 0xF.
fn mlp_parity_ok(au: &[u8], header_size: usize, substr_header_size: usize) -> bool {
let end = header_size + substr_header_size;
if end > au.len() {
return false;
}
let xor_fold = |d: &[u8]| d.iter().fold(0u8, |a, &b| a ^ b);
let p = xor_fold(&au[0..4]) ^ xor_fold(&au[header_size..end]);
((p >> 4) ^ p) & 0xF == 0xF
}
impl CodecParser for TrueHdParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
// B1: a concealed/lost gap means the buffered TrueHD AU is TRUNCATED.
@@ -256,26 +419,93 @@ impl CodecParser for TrueHdParser {
& 0xFFFF_FFFE)
== 0xF872_6FBA;
// On a major sync the 32-bit `format_info` word (immediately after
// the 4-byte sync, i.e. AU bytes 8..12) carries the rate nibble.
// Refine the per-AU PTS increment to the actual rate family. The
// 48 kHz family resolves to the unchanged 833_333 default, so the
// common case stays byte-identical; only the 44.1 kHz family shifts.
if is_major_sync && unit_bytes >= 12 {
let format_info =
u32::from_be_bytes([self.buf[8], self.buf[9], self.buf[10], self.buf[11]]);
self.au_duration_ns = truehd_au_duration_ns(format_info);
// Decodability gate. MLP/TrueHD decode state persists across access
// units, so a corrupt AU is dropped FORWARD to the next VALIDATED
// major sync (the clean re-init point) rather than excised in place.
// The PTS clock advances across every dropped AU so a drop is a
// silence gap, never a shift.
let au = self.buf[..unit_bytes].to_vec();
let pts = self.next_pts_ns;
let mut emit_keyframe: Option<bool> = None; // Some(is_keyframe) => emit
let mut drop_reason: Option<(&'static str, bool)> = None; // (reason, verified)
if self.tally.is_poisoned() {
// Whole track already judged dead — collateral drop (does not
// re-feed the poison verdict).
drop_reason = Some(("track-poisoned", false));
} else {
match self.au_check(&au, is_major_sync) {
AuCheck::ValidMajorSync { format_info } => {
// The rate nibble is trustworthy only now that the major
// sync's CRC has validated. Refine the per-AU PTS
// increment (48 kHz family stays the 833_333 default).
if let Some(fi) = format_info {
self.au_duration_ns = truehd_au_duration_ns(fi);
}
// A validated major sync is the ONLY clean resync point.
self.resync_pending = false;
emit_keyframe = Some(true);
}
AuCheck::Corrupt => {
if self.resync_pending {
// Part of the current drop-forward run — collateral.
drop_reason = Some(("resync", false));
} else {
// The trigger: one verified corruption that starts the
// drop-forward. Only this counts toward poison.
let r = if is_major_sync {
"major-sync-crc"
} else {
"parity"
};
drop_reason = Some((r, true));
self.resync_pending = true;
}
}
AuCheck::Ok => {
if self.resync_pending {
// Decode state is invalid until the next validated
// major sync, so even a parity-OK AU is undecodable
// here — collateral drop.
drop_reason = Some(("resync", false));
} else {
emit_keyframe = Some(false);
}
}
AuCheck::Unverifiable => {
if self.resync_pending {
// Not a validated major sync — do NOT clear the resync
// on it; keep dropping forward.
drop_reason = Some(("resync", false));
} else {
// Head of stream / too-short AU: keep (never drop what
// we cannot verify).
emit_keyframe = Some(is_major_sync);
}
}
}
}
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.next_pts_ns,
keyframe: is_major_sync,
data: self.buf[..unit_bytes].to_vec(),
duration_ns: None,
});
if let Some(keyframe) = emit_keyframe {
self.tally.record_kept();
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: pts,
keyframe,
data: au,
duration_ns: None,
});
} else if let Some((reason, verified)) = drop_reason {
if verified {
self.tally
.record_drop(pts, self.au_duration_ns, au.len(), reason);
} else {
self.tally
.record_collateral_drop(pts, self.au_duration_ns, au.len(), reason);
}
}
self.buf.drain(..unit_bytes);
self.next_pts_ns += self.au_duration_ns;
}
@@ -289,6 +519,11 @@ impl CodecParser for TrueHdParser {
frames
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
@@ -451,6 +686,236 @@ mod tests {
data
}
/// Turn a synthetic major-sync AU (sync bytes already set at offset 4, any
/// `format_info` set) into one that passes the decodability gate: 1 substream,
/// a clean substream directory, a valid major-sync CRC-16, and a valid header
/// parity nibble. Mirrors what a real encoder writes (verified against real
/// ffmpeg TrueHD). The AU must be ≥ 36 bytes (4 AU header + 28 major-sync
/// header + 2 directory + slack), which every `make_truehd_unit(≥200)` is.
fn finalize_major_sync(au: &mut [u8]) {
const MSHDR: usize = 28; // no extension (byte 25 clear)
// num_substreams = 1 → major-sync byte 16 (AU[20]) top nibble.
au[20] = (au[20] & 0x0F) | 0x10;
// Substream directory entry at AU[4+MSHDR] = AU[32]: extraword flag clear.
au[32] &= 0x7F;
// Major-sync CRC over the header body, stored big-endian in the trailer.
let body_end = 4 + MSHDR - 4; // AU[4..28]
let crc = super::crc16_mlp(&au[4..body_end])
^ u16::from_le_bytes([au[body_end], au[body_end + 1]]);
au[4 + MSHDR - 2] = (crc >> 8) as u8;
au[4 + MSHDR - 1] = (crc & 0xFF) as u8;
// Parity: choose the AU check nibble (AU[0] high bits) so the header +
// directory fold to 0xF. The length low nibble (AU[0] low bits) is kept.
let hi = au[0] & 0x0F;
let p0 = (hi ^ au[1] ^ au[2] ^ au[3]) ^ (au[32] ^ au[33]);
let c = ((p0 >> 4) ^ (p0 & 0x0F) ^ 0x0F) & 0x0F;
au[0] = (c << 4) | hi;
}
/// Give a synthetic NON-major-sync AU a valid header parity nibble (1
/// substream, directory at AU[4..6]), so it passes the gate once a preceding
/// major sync has established `num_substreams`.
fn finalize_normal_parity(au: &mut [u8]) {
au[4] &= 0x7F; // no extraword
let hi = au[0] & 0x0F;
let p0 = (hi ^ au[1] ^ au[2] ^ au[3]) ^ (au[4] ^ au[5]);
let c = ((p0 >> 4) ^ (p0 & 0x0F) ^ 0x0F) & 0x0F;
au[0] = (c << 4) | hi;
}
fn valid_major_sync() -> Vec<u8> {
let mut u = make_truehd_unit(200);
u[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
finalize_major_sync(&mut u);
u
}
fn valid_normal_au() -> Vec<u8> {
let mut u = make_truehd_unit(200);
finalize_normal_parity(&mut u);
u
}
#[test]
fn corrupt_major_sync_drops_forward_to_next_valid() {
// MLP state carries across AUs, so a corrupt AU is dropped FORWARD to the
// next valid major sync (the clean re-init point). Sequence: valid MS,
// corrupt MS (bad CRC), a normal AU, then a valid MS. Only the two valid
// major syncs survive; the corrupt MS and the intervening normal AU are
// dropped (the latter because decode state is poisoned until re-init).
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF; // corrupt a CRC-covered header byte
let normal = valid_normal_au(); // clean parity, but arrives mid-resync
let ms2 = valid_major_sync();
let mut data = ms1.clone();
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&normal);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "only the two valid major syncs survive");
assert!(frames[0].keyframe && frames[1].keyframe);
assert_eq!(
parser.dropped_frames(),
2,
"corrupt MS + poisoned normal AU"
);
}
#[test]
fn parity_failure_is_dropped() {
// A normal AU whose header parity is broken (after a major sync sets
// num_substreams) is undecodable → dropped.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut bad = valid_normal_au();
// A single-nibble flip: MLP's nibble-fold parity (like ffmpeg's) is blind
// to a full-byte flip, which changes both nibbles equally and cancels.
bad[2] ^= 0x01;
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&bad);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "the parity-broken AU is dropped");
assert_eq!(parser.dropped_frames(), 1);
}
#[test]
fn drop_forward_preserves_av_sync_no_shift() {
// THE INVARIANT: the resumed major sync keeps the exact PTS it would have
// had with no drop — base + 3 AU durations (MS1, corrupt-MS, normal, MS2)
// — so the drop is a silence gap, never a shift.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF;
let normal = valid_normal_au();
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&normal);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2);
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
3 * AU_DURATION_NS,
"resumed major sync keeps its true timeline (gap, not shift)"
);
}
#[test]
fn transient_corruptions_do_not_poison_whole_track() {
// Regression (audit HIGH): TrueHD drop-forward must NOT amplify a couple
// of transient errors into a false whole-track poison. Two corruptions,
// each forcing a long collateral resync run past 200 total AUs, must
// leave the track un-poisoned and keep the good audio that follows.
let mut parser = TrueHdParser::new();
let mut data = valid_major_sync();
// Corruption #1 then a long run of normal AUs (all collateral-dropped
// while resyncing — no major sync to re-init on).
let mut bad1 = valid_normal_au();
bad1[2] ^= 0x01; // single-nibble parity break
data.extend_from_slice(&bad1);
for _ in 0..210 {
data.extend_from_slice(&valid_normal_au());
}
// A valid major sync resumes; the good AUs after it MUST be kept.
data.extend_from_slice(&valid_major_sync());
for _ in 0..5 {
data.extend_from_slice(&valid_normal_au());
}
let frames = parser.parse(&make_pes(data, Some(90000)));
assert!(
!parser.tally.is_poisoned(),
"two transient errors must not poison the track"
);
// MS1 + resumed MS2 + the 5 good AUs after it survive.
assert_eq!(
frames.len(),
7,
"post-resync good audio is kept, not poisoned away"
);
assert!(
parser.dropped_frames() > 200,
"the resync run was still counted for reporting"
);
}
#[test]
fn corrupt_major_sync_rate_nibble_does_not_shift_pts() {
// Regression (audit MED): a corrupt major sync whose rate nibble decodes
// to the 44.1 kHz family must NOT refine au_duration_ns — the rate is only
// trustworthy after the CRC validates. Otherwise the resumed 48 kHz audio
// is shifted (not gapped).
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync(); // 48 kHz
let mut ms_bad = valid_major_sync();
// Set the rate nibble (top nibble of format_info = au[8]) to 0x8 (44.1k).
// au[8] is CRC-covered, so this also breaks the major-sync CRC → corrupt.
ms_bad[8] = (ms_bad[8] & 0x0F) | 0x80;
let ms2 = valid_major_sync(); // 48 kHz
let mut data = ms1;
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&ms2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2, "corrupt MS dropped; MS1 and MS2 survive");
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
2 * AU_DURATION_NS,
"resumed audio keeps the 48 kHz cadence — the corrupt MS's 44.1k rate was ignored"
);
}
#[test]
fn too_short_major_sync_does_not_clear_resync() {
// Regression (audit LOW): while resyncing, a major sync too short to hold
// (and CRC-validate) its header must NOT be treated as a clean resync
// point — the runt is dropped and only a real validated major sync resumes.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut bad = valid_normal_au();
bad[2] ^= 0x01; // parity break → triggers resync
// An 8-byte "major sync": length=4 words, sync at bytes 4..8, too short
// to hold the 28-byte major-sync header.
let runt = vec![0x00, 0x04, 0x00, 0x00, 0xF8, 0x72, 0x6F, 0xBA];
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&bad);
data.extend_from_slice(&runt);
data.extend_from_slice(&ms2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2, "the runt major sync did not resume decode");
for f in &frames {
assert_eq!(
f.data.len(),
200,
"only the real 200-byte major syncs survive"
);
}
}
#[test]
fn clean_truehd_stream_drops_nothing() {
// A run of valid AUs passes untouched — zero false positives (the CRC and
// parity are verified against real ffmpeg TrueHD output).
let mut parser = TrueHdParser::new();
let mut data = valid_major_sync();
for _ in 0..5 {
data.extend_from_slice(&valid_normal_au());
}
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 6);
assert_eq!(parser.dropped_frames(), 0);
}
fn make_ac3_frame() -> Vec<u8> {
// Minimal AC-3 frame: sync 0x0B77, fscod=0 (48kHz), frmsizecod=0 (64 words = 128 bytes)
let mut data = vec![0u8; 128];
@@ -878,6 +1343,7 @@ mod tests {
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
finalize_major_sync(&mut unit);
let f = parser.parse(&make_pes(unit, Some(90000)));
assert_eq!(f.len(), 1);
assert!(f[0].keyframe, "major-sync AU must be flagged keyframe");
@@ -899,6 +1365,7 @@ mod tests {
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBBu32.to_be_bytes());
finalize_major_sync(&mut unit);
let f = parser.parse(&make_pes(unit, Some(90000)));
assert_eq!(f.len(), 1);
assert!(f[0].keyframe, "major-sync variant 0xFB also a keyframe");
@@ -1101,8 +1568,11 @@ mod tests {
let mut a1 = make_truehd_unit(200);
a1[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes()); // major sync
a1[8..12].copy_from_slice(&format_info_with(0x8).to_be_bytes()); // 44.1 k
finalize_major_sync(&mut a1);
let mut a2 = make_truehd_unit(200);
finalize_normal_parity(&mut a2);
let mut data = a1;
data.extend_from_slice(&make_truehd_unit(200));
data.extend_from_slice(&a2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2);
assert_eq!(
@@ -1120,8 +1590,11 @@ mod tests {
let mut a1 = make_truehd_unit(200);
a1[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
a1[8..12].copy_from_slice(&format_info_with(0x0).to_be_bytes()); // 48 k
finalize_major_sync(&mut a1);
let mut a2 = make_truehd_unit(200);
finalize_normal_parity(&mut a2);
let mut data = a1;
data.extend_from_slice(&make_truehd_unit(200));
data.extend_from_slice(&a2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2);
assert_eq!(frames[1].pts_ns - frames[0].pts_ns, 833_333);
+69 -5
View File
@@ -31,6 +31,10 @@ use std::io::{self, BufWriter, Write};
use std::path::{Path, PathBuf};
/// Filename-naming strategy for the per-track files.
// `allow(dead_code)`: the sink honours all variants, but only the `#[default]` is
// constructed today (`output()` builds `DemuxOptions::default()`). The alternates
// are a staged option surface awaiting the CLI `--naming` flag (not yet wired).
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum Naming {
/// `<base> <track> <lang> <codec> [DELAY <n>ms].<ext>` — human-readable.
@@ -55,6 +59,9 @@ pub enum DelayMode {
}
/// Chapter export format.
// `allow(dead_code)`: only the `#[default]` XML variant is constructed today (via
// `DemuxOptions::default()`); OGM/Both await the CLI `--chapters` flag.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ChaptersFmt {
/// mkvmerge chapter XML.
@@ -81,6 +88,10 @@ pub struct DemuxOptions {
pub export_chapters: bool,
/// Selected track indices. `None` = all tracks.
pub selection: Option<Vec<usize>>,
/// Restrict output to one track class. `None` = every class (plain
/// `demux://`). `Some(Audio)` is the `audio://` sink; `Some(Subtitle)` is
/// `sub://`. Filtered tracks are skipped entirely (no file written).
pub kind_filter: Option<TrackKind>,
}
impl Default for DemuxOptions {
@@ -92,13 +103,16 @@ impl Default for DemuxOptions {
chapters_fmt: ChaptersFmt::default(),
export_chapters: true,
selection: None,
kind_filter: None,
}
}
}
/// Track class, used for delay attribution and naming.
/// Track class, used for delay attribution and naming — and, via
/// [`DemuxOptions::kind_filter`], to restrict a demux to one class (the
/// `audio://` / `sub://` sinks are a `demux://` filtered to Audio / Subtitle).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum TrackKind {
pub enum TrackKind {
Video,
Audio,
Subtitle,
@@ -136,7 +150,7 @@ fn extension_for(codec: Codec) -> &'static str {
}
/// Short codec label for friendly filenames.
fn codec_label(codec: Codec) -> &'static str {
pub(crate) fn codec_label(codec: Codec) -> &'static str {
match codec {
Codec::Hevc => "HEVC",
Codec::H264 => "AVC",
@@ -534,7 +548,7 @@ fn fmt_chapter_time_ns(time_secs: f64) -> String {
}
/// Serialize chapters as mkvmerge chapter XML.
fn chapters_xml(chapters: &[Chapter]) -> String {
pub(crate) fn chapters_xml(chapters: &[Chapter]) -> String {
let mut s = String::new();
s.push_str("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
s.push_str("<!DOCTYPE Chapters SYSTEM \"matroskachapters.dtd\">\n");
@@ -564,7 +578,7 @@ fn chapters_xml(chapters: &[Chapter]) -> String {
}
/// Serialize chapters as OGM/simple chapter text.
fn chapters_ogm(chapters: &[Chapter]) -> String {
pub(crate) fn chapters_ogm(chapters: &[Chapter]) -> String {
let mut s = String::new();
for (i, c) in chapters.iter().enumerate() {
let n = i + 1;
@@ -651,6 +665,11 @@ impl DemuxSink {
(TrackKind::Subtitle, s.codec, s.pid, s.language.clone())
}
};
// Kind filter: `audio://` / `sub://` keep only their class.
if opts.kind_filter.is_some_and(|k| k != kind) {
tracks.push(None);
continue;
}
if kind == TrackKind::Video && ref_video_track.is_none() {
ref_video_track = Some(idx);
}
@@ -867,6 +886,17 @@ mod tests {
})
}
fn subtitle_stream(codec: Codec, lang: &str) -> DiscStream {
DiscStream::Subtitle(crate::disc::SubtitleStream {
pid: 0x1200,
codec,
language: lang.to_string(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
})
}
fn title_with(streams: Vec<DiscStream>, privates: Vec<Option<Vec<u8>>>) -> DiscTitle {
let mut t = DiscTitle::empty();
t.streams = streams;
@@ -875,6 +905,40 @@ mod tests {
t
}
/// `audio://` and `sub://` are `demux://` with a kind filter: only tracks of
/// the selected class get a file; every other track is skipped entirely.
#[test]
fn kind_filter_keeps_only_the_selected_class() {
let title = title_with(
vec![
video_stream(Codec::H264),
audio_stream(Codec::Ac3, "eng"),
subtitle_stream(Codec::Pgs, "eng"),
],
vec![None, None, None],
);
let sub_opts = DemuxOptions {
kind_filter: Some(TrackKind::Subtitle),
export_chapters: false,
..Default::default()
};
let sub = DemuxSink::create(&tempdir(), &title, &sub_opts).unwrap();
assert!(
sub.tracks[0].is_none() && sub.tracks[1].is_none() && sub.tracks[2].is_some(),
"sub:// keeps only the subtitle track"
);
let audio_opts = DemuxOptions {
kind_filter: Some(TrackKind::Audio),
export_chapters: false,
..Default::default()
};
let audio = DemuxSink::create(&tempdir(), &title, &audio_opts).unwrap();
assert!(
audio.tracks[0].is_none() && audio.tracks[1].is_some() && audio.tracks[2].is_none(),
"audio:// keeps only the audio track"
);
}
// ── Annex-B reframing ────────────────────────────────────────────────────
//
// The length-prefixed → Annex-B conversion and the hvcC/avcC param-set
+61 -2
View File
@@ -139,14 +139,14 @@ pub struct DiscStream {
// Adaptive batch sizer — preferred comes from the caller
// (detect_max_batch_sectors), shrinks/grows based on read outcomes.
adaptive: AdaptiveBatch,
pub errors: u64,
errors: u64,
/// Cumulative bytes actually skipped (zero-filled) on read error.
/// Distinct from `errors`, which counts skip *events*: one event can
/// cover a whole AACS unit (`unit_align` sectors = 6144 bytes), so
/// `errors * 2048` understates real loss by the alignment factor.
/// Consumers estimating lost video time must scale by this, not by
/// the event count.
pub lost_bytes: u64,
lost_bytes: u64,
pub skip_errors: bool,
/// When set and the token is cancelled, fill_extents returns Err(Halted)
/// at the next retry boundary. Unlike skip_errors, this propagates the
@@ -351,6 +351,27 @@ impl DiscStream {
self
}
/// Install a proactive [`AacsKeyMap`](crate::decrypt::AacsKeyMap) on the inline
/// live-drive path — the counterpart to what
/// [`build_iso_pipeline`](crate::mux::resolve::build_iso_pipeline) does for the
/// file-backed highway. The map is the title's read plan: it decides which unit
/// each LBA is and, for an FMTS forensic segment, which phase is ours. The
/// extent walk is rewritten to the read plan so **only our-phase units are read
/// off the drive** (the alternate device-group units are never fetched,
/// decrypted, or muxed), and the map is installed so each unit decrypts with its
/// mapped key. A non-forensic map returns the extents unchanged, so a plain
/// single/multi-CPS disc reads exactly as before.
pub fn with_key_map(mut self, map: std::sync::Arc<crate::decrypt::AacsKeyMap>) -> Self {
self.extents = map.read_plan(&self.extents, self.unit_align.max(1) as u32);
self.bytes_total_extents = self
.extents
.iter()
.map(|e| e.sector_count as u64 * 2048)
.sum();
self.reader.set_key_map(map);
self
}
fn is_halted(&self) -> bool {
self.halt
.as_ref()
@@ -1118,6 +1139,44 @@ mod tests {
);
}
/// `with_key_map` on the inline live-drive path applies the same FMTS read plan
/// the file-backed highway uses: within a forensic segment only our-phase units
/// survive the extent walk, so the alternate device-group units are never read.
#[test]
fn with_key_map_reads_only_our_phase_units() {
use crate::decrypt::{AacsKeyMap, DecryptKeys, Phase};
// AACS keys → unit_align = 3, so a unit is 3 sectors and the phase filter
// engages. Key contents are irrelevant to the read plan.
let aacs = DecryptKeys::Aacs {
unit_keys: vec![(0, [0u8; 16]), (1, [1u8; 16])],
read_data_key: None,
format: ContentFormat::BdTs,
};
// 100 units (300 sectors). A 10-unit Even forensic segment at LBA [30,60):
// even units (30,36,42,48,54) are ours; odd (33,39,45,51,57) are dropped.
let map = AacsKeyMap::from_ranges_phased(vec![(30, 60, 1, Phase::Even)], 0);
let stream = DiscStream::new(
Box::new(ZeroReader { capacity: 300 }),
synthetic_title(300),
aacs,
8,
ContentFormat::BdTs,
)
.with_key_map(std::sync::Arc::new(map));
let total: u32 = stream.extents.iter().map(|e| e.sector_count).sum();
assert_eq!(
total,
300 - 5 * 3,
"exactly the 5 alternate-phase units (15 sectors) are dropped from the read walk"
);
assert!(
stream.extents.len() > 1,
"the forensic segment split the single extent into our-phase-only runs"
);
// The progress denominator tracks the reduced read set.
assert_eq!(stream.bytes_total_extents, total as u64 * 2048);
}
/// Recording `SectorSource`: logs every `(lba, count)` request and
/// returns `Err` whenever the requested range covers `bad_sector`.
/// Successful reads return zeroed sectors (which are NOT
+7 -19
View File
@@ -157,31 +157,19 @@ impl FviSink {
///
/// `source_path` / `source_title` record where the index was built from
/// (the input URL path + the 0-based title index); they are carried into the
/// header's `source` object. The medium defaults to `file` — callers with a
/// known medium / playlist / volume use [`FviSink::create_with_source`].
/// header's `source` object. The remaining provenance (medium, playlist,
/// volume) takes its `SourceInfo` defaults — no caller needs to override them.
pub fn create(
path: &Path,
title: &DiscTitle,
source_path: String,
source_title: usize,
) -> io::Result<Self> {
Self::create_with_source(
path,
title,
SourceInfo {
path: source_path,
title: source_title,
..SourceInfo::default()
},
)
}
/// Create the sink with a fully-specified [`SourceInfo`] provenance root.
pub fn create_with_source(
path: &Path,
title: &DiscTitle,
source: SourceInfo,
) -> io::Result<Self> {
let source = SourceInfo {
path: source_path,
title: source_title,
..SourceInfo::default()
};
let file = File::create(path)?;
let video_track = title
+357
View File
@@ -0,0 +1,357 @@
//! `chapters://` and `json://` metadata sinks.
//!
//! Both ignore the PES stream entirely: everything they emit is already known
//! from the [`DiscTitle`] at construction, so each writes its whole file at
//! `create()` and treats every `write()` frame as a no-op. They are wired
//! through [`super::resolve::output`] like the other write-only sinks; the
//! ISO/disc scan that builds the title is all they need.
use crate::disc::{Chapter, DiscTitle, Stream as DiscStream};
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Write};
use std::path::Path;
// ── chapters:// ──────────────────────────────────────────────────────────────
/// `HH:MM:SS.mmm` for a WebVTT cue timestamp.
fn vtt_time(secs: f64) -> String {
let total_ms = (secs.max(0.0) * 1000.0).round() as u64;
let ms = total_ms % 1000;
let total_s = total_ms / 1000;
format!(
"{:02}:{:02}:{:02}.{:03}",
total_s / 3600,
(total_s / 60) % 60,
total_s % 60,
ms
)
}
/// WebVTT chapter cues (`.vtt`). Each chapter spans until the next one starts
/// (the last runs to its own start — length is unknown without the title tail).
fn chapters_vtt(chapters: &[Chapter]) -> String {
let mut s = String::from("WEBVTT\n\n");
for (i, c) in chapters.iter().enumerate() {
let start = c.time_secs.max(0.0);
let end = chapters
.get(i + 1)
.map(|n| n.time_secs.max(0.0))
.unwrap_or(start);
s.push_str(&format!(
"{}\n{} --> {}\nChapter {}\n\n",
i + 1,
vtt_time(start),
vtt_time(end),
c.name
));
}
s
}
/// Chapter content in the format the output extension selects: `.txt`/`.ogm`
/// (OGM simple), `.vtt` (WebVTT), else Matroska XML (`.xml` / default).
pub(crate) fn chapters_content(chapters: &[Chapter], ext: Option<&str>) -> String {
match ext.map(|e| e.to_ascii_lowercase()).as_deref() {
Some("txt") | Some("ogm") => super::demux_sink::chapters_ogm(chapters),
Some("vtt") => chapters_vtt(chapters),
_ => super::demux_sink::chapters_xml(chapters),
}
}
/// `chapters://` sink: writes the title's chapter markers at construction; the
/// PES stream is ignored.
pub struct ChaptersSink {
title: DiscTitle,
}
impl ChaptersSink {
pub fn create(path: &Path, title: &DiscTitle) -> io::Result<Self> {
let ext = path.extension().and_then(|e| e.to_str());
let content = chapters_content(&title.chapters, ext);
File::create(path)?.write_all(content.as_bytes())?;
Ok(Self {
title: title.clone(),
})
}
}
impl Stream for ChaptersSink {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Ok(()) // whole file written at create()
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
// ── json:// ──────────────────────────────────────────────────────────────────
/// Serialization id for an audio stream's editorial purpose.
fn purpose_id(p: crate::labels::LabelPurpose) -> &'static str {
use crate::labels::LabelPurpose::*;
match p {
Normal => "normal",
Commentary => "commentary",
Descriptive => "descriptive",
Score => "score",
Ime => "ime",
}
}
/// Serialization id for a subtitle stream's qualifier.
fn qualifier_id(q: crate::labels::LabelQualifier) -> &'static str {
use crate::labels::LabelQualifier::*;
match q {
None => "none",
Sdh => "sdh",
DescriptiveService => "descriptive_service",
Forced => "forced",
}
}
/// One stream as JSON — every field the scan resolved, nothing dropped. This is
/// the complete per-stream model (`disc::Stream`), not a summary: consumers get
/// resolution/HDR/aspect for video, channels/sample-rate/purpose for audio, and
/// the qualifier for subtitles, all in machine-readable form.
fn stream_json(s: &DiscStream) -> serde_json::Value {
use super::demux_sink::codec_label;
use serde_json::json;
match s {
DiscStream::Video(v) => {
let (w, h) = v.resolution.pixels();
let (fps_num, fps_den) = v.frame_rate.as_fraction();
let mut o = json!({
"kind": "video",
"codec": codec_label(v.codec),
"pid": v.pid,
"resolution": v.resolution.to_string(),
"width": w,
"height": h,
"interlaced": v.resolution.is_interlaced(),
"frame_rate": v.frame_rate.to_string(),
"frame_rate_num": fps_num,
"frame_rate_den": fps_den,
"hdr": v.hdr.id(),
"color_space": v.color_space.id(),
"secondary": v.secondary,
"mvc_dependent": v.is_mvc_dependent(),
});
if let Some((num, den)) = v.display_aspect {
o["display_aspect"] = json!(format!("{num}:{den}"));
}
if let Some(c) = v.measured_cicp {
o["measured_cicp"] = json!({
"matrix": c.matrix,
"transfer": c.transfer,
"primaries": c.primaries,
"range": c.range,
});
}
if !v.label.is_empty() {
o["label"] = json!(v.label);
}
o
}
DiscStream::Audio(a) => {
let mut o = json!({
"kind": "audio",
"codec": codec_label(a.codec),
"pid": a.pid,
"language": a.language,
"channels": a.channels.to_string(),
"channel_count": a.channels.count(),
"sample_rate": a.sample_rate.to_string(),
"sample_rate_hz": a.sample_rate.hz(),
"secondary": a.secondary,
"purpose": purpose_id(a.purpose),
});
if !a.label.is_empty() {
o["label"] = json!(a.label);
}
o
}
DiscStream::Subtitle(t) => json!({
"kind": "subtitle",
"codec": codec_label(t.codec),
"pid": t.pid,
"language": t.language,
"forced": t.forced,
"qualifier": qualifier_id(t.qualifier),
}),
}
}
/// The `json://` document for one title: identity, duration/size, its clips,
/// its complete stream models, and its chapter points. A stable, machine-
/// readable view of one title — the same information the scan resolved, no loss.
pub(crate) fn title_json(title: &DiscTitle) -> serde_json::Value {
use serde_json::json;
let streams: Vec<_> = title.streams.iter().map(stream_json).collect();
let clips: Vec<_> = title
.clips
.iter()
.map(|c| {
json!({
"clip_id": c.clip_id,
"duration_secs": c.duration_secs,
"source_packets": c.source_packets,
})
})
.collect();
let chapters: Vec<_> = title
.chapters
.iter()
.enumerate()
.map(|(i, c)| json!({ "n": i + 1, "start_secs": c.time_secs, "name": c.name }))
.collect();
json!({
"playlist": title.playlist,
"playlist_id": title.playlist_id,
"duration_secs": title.duration_secs,
"size_bytes": title.size_bytes,
"format": format!("{:?}", title.content_format),
"clips": clips,
"streams": streams,
"chapters": chapters,
})
}
/// `json://` sink: writes the title's structured metadata at construction; the
/// PES stream is ignored.
pub struct JsonSink {
title: DiscTitle,
}
impl JsonSink {
pub fn create(path: &Path, title: &DiscTitle) -> io::Result<Self> {
let doc =
serde_json::to_string_pretty(&title_json(title)).unwrap_or_else(|_| "{}".to_string());
let mut f = File::create(path)?;
f.write_all(doc.as_bytes())?;
f.write_all(b"\n")?;
Ok(Self {
title: title.clone(),
})
}
}
impl Stream for JsonSink {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Ok(())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::Chapter;
fn chaps() -> Vec<Chapter> {
vec![
Chapter {
time_secs: 0.0,
name: "1".into(),
},
Chapter {
time_secs: 62.5,
name: "2".into(),
},
]
}
#[test]
fn chapters_format_selected_by_extension() {
let xml = chapters_content(&chaps(), Some("xml"));
assert!(xml.contains("<Chapters>"), "xml chosen for .xml");
let ogm = chapters_content(&chaps(), Some("txt"));
assert!(ogm.contains("CHAPTER01="), "ogm chosen for .txt");
let vtt = chapters_content(&chaps(), Some("vtt"));
assert!(
vtt.starts_with("WEBVTT") && vtt.contains("00:01:02.500"),
"vtt chosen for .vtt, with cue timing"
);
// Unknown / missing extension defaults to XML.
assert!(chapters_content(&chaps(), None).contains("<Chapters>"));
}
#[test]
fn title_json_carries_streams_and_chapters() {
use crate::disc::{AudioChannels, AudioStream, Codec, DiscTitle};
use crate::disc::{LabelPurpose, SampleRate, Stream as DiscStream};
let mut t = DiscTitle::empty();
t.playlist = "MAIN".into();
t.chapters = chaps();
t.streams = vec![DiscStream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Stereo,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
})];
let v = title_json(&t);
assert_eq!(v["playlist"], "MAIN");
let a = &v["streams"][0];
assert_eq!(a["kind"], "audio");
assert_eq!(a["codec"], "TrueHD");
assert_eq!(a["language"], "eng");
// Completeness: audio carries channels + sample rate + purpose, not just codec.
assert_eq!(a["channels"], "stereo");
assert_eq!(a["channel_count"], 2);
assert_eq!(a["sample_rate"], "48kHz");
assert_eq!(a["sample_rate_hz"], 48000.0);
assert_eq!(a["purpose"], "normal");
assert_eq!(v["chapters"][1]["n"], 2);
assert_eq!(v["chapters"][1]["start_secs"], 62.5);
assert_eq!(v["chapters"][1]["name"], "2");
}
#[test]
fn video_json_carries_resolution_and_hdr() {
use crate::disc::Codec;
use crate::disc::{
ColorSpace, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStream,
VideoStream,
};
let mut t = DiscTitle::empty();
t.streams = vec![DiscStream::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R2160p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
})];
let vid = &title_json(&t)["streams"][0];
assert_eq!(vid["kind"], "video");
assert_eq!(vid["resolution"], "2160p");
assert_eq!(vid["width"], 3840);
assert_eq!(vid["height"], 2160);
assert_eq!(vid["frame_rate"], "23.976");
assert_eq!(vid["frame_rate_num"], 24000);
assert_eq!(vid["hdr"], "hdr10");
assert_eq!(vid["color_space"], "bt2020");
}
}
+179 -5
View File
@@ -672,6 +672,14 @@ pub struct MkvMuxer<W: Write + Seek> {
/// (to patch in place) and the IFO-claimed count (to warn on disagreement);
/// `corrected` flips once patched so we only act on the first frame.
ac3_channel_fixups: std::collections::HashMap<usize, Ac3ChannelFixup>,
/// Deferred PGS forced-subtitle detection. A PGS subtitle track reserves a
/// 1-byte `FlagForced` value up-front; as its display sets are written, the
/// track is judged forced iff it displayed at least one subtitle and EVERY
/// display set carried the HDMV `forced_on_flag` (a dedicated forced/narrative
/// track). At `finish()` the reserved byte is promoted to 1 for such tracks —
/// so forced subs are flagged even on discs without vendor label metadata.
/// Only ever promotes (0→1); a scan/vendor forced flag is never demoted.
pgs_forced_fixups: std::collections::HashMap<usize, PgsForcedFixup>,
/// `--log-level 3` opening-frame capture: the first ~100 coded frames per
/// track are written (raw) to a `<output>.opening.bin` side file with a
/// per-frame summary logged, so an opening-GOP / menu / mid-GOP-open issue is
@@ -692,6 +700,15 @@ struct Ac3ChannelFixup {
corrected: bool,
}
/// Deferred PGS forced-subtitle detection state for one PGS subtitle track.
struct PgsForcedFixup {
/// Absolute file offset of the 1-byte `FlagForced` value in the Tracks element.
value_offset: u64,
/// Shared forced-narrative classifier fed the track's display sets. The same
/// type drives the `info`-time forced probe, so both classify identically.
tracker: super::codec::pgs::ForcedTracker,
}
/// TimestampScale: nanoseconds per Matroska timestamp tick. 0.1 ms (100_000 ns).
///
/// The classic 1 ms scale truncates two distinct cadences onto the same tick:
@@ -917,6 +934,8 @@ impl<W: Write + Seek> MkvMuxer<W> {
let mut track_uids: Vec<u64> = Vec::with_capacity(tracks.len());
let mut ac3_channel_fixups: std::collections::HashMap<usize, Ac3ChannelFixup> =
std::collections::HashMap::new();
let mut pgs_forced_fixups: std::collections::HashMap<usize, PgsForcedFixup> =
std::collections::HashMap::new();
// Per track: whether it emitted a conforming `mvcC` BlockAdditionMapping.
// Filled below from the SAME built record that drives the CodecPrivate
// mvcC extension, so the three MVC signals never diverge.
@@ -948,7 +967,24 @@ impl<W: Write + Seek> MkvMuxer<W> {
if !track.is_default {
ebml::write_uint(&mut writer, ebml::FLAG_DEFAULT, 0)?;
}
if track.is_forced {
if track.track_type == ebml::TRACK_TYPE_SUBTITLE && track.codec_id == ebml::CODEC_PGS {
// Reserve a 1-byte FlagForced (initial value = the scan/vendor
// flag) and record its offset, so PGS content can promote it to 1
// at finish() if the track proves to be forced narrative. Written
// explicitly (ID + size + value) so the value is a single,
// in-place-patchable byte — same idiom as the Channels fixup.
ebml::write_id(&mut writer, ebml::FLAG_FORCED)?;
ebml::write_size(&mut writer, 1)?;
let value_offset = writer.stream_position()?;
writer.write_all(&[track.is_forced as u8])?;
pgs_forced_fixups.insert(
i,
PgsForcedFixup {
value_offset,
tracker: super::codec::pgs::ForcedTracker::new(),
},
);
} else if track.is_forced {
ebml::write_uint(&mut writer, ebml::FLAG_FORCED, 1)?;
}
@@ -1203,6 +1239,7 @@ impl<W: Write + Seek> MkvMuxer<W> {
max_block_ticks: 0,
last_video_keyframe_ticks: None,
ac3_channel_fixups,
pgs_forced_fixups,
opening_capture: None,
})
}
@@ -1514,6 +1551,14 @@ impl<W: Write + Seek> MkvMuxer<W> {
}
}
// Accumulate PGS forced-subtitle state: a display set marks the track as
// having shown a subtitle, and clears `all_forced` the moment a
// non-forced set appears. `finish()` promotes FlagForced only for a track
// that displayed subtitles and had every one forced.
if let Some(fixup) = self.pgs_forced_fixups.get_mut(&track_idx) {
fixup.tracker.observe(data);
}
Ok(())
}
@@ -1546,6 +1591,26 @@ impl<W: Write + Seek> MkvMuxer<W> {
// Close final cluster
self.end_cluster()?;
// Promote FlagForced for PGS subtitle tracks that proved to be forced
// narrative (displayed subtitles, every one forced). In-place single-byte
// rewrite of the reserved value, then restore the append position for the
// Cues that follow. Only promotes (0→1); a track already forced from the
// scan/vendor flag stays forced.
let forced_offsets: Vec<u64> = self
.pgs_forced_fixups
.values()
.filter(|f| f.tracker.is_forced())
.map(|f| f.value_offset)
.collect();
if !forced_offsets.is_empty() {
let here = self.writer.stream_position()?;
for off in forced_offsets {
self.writer.seek(std::io::SeekFrom::Start(off))?;
self.writer.write_all(&[1u8])?;
}
self.writer.seek(std::io::SeekFrom::Start(here))?;
}
// Write Cues
let cues_start = self.writer.stream_position()?;
let cues_offset = cues_start - self.segment_start;
@@ -2600,13 +2665,122 @@ mod tests {
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// FlagForced should NOT be written for non-forced tracks
assert!(
find_id(&data, ebml::FLAG_FORCED).is_none(),
"FlagForced element should not be present for non-forced subtitle"
// A PGS subtitle track now RESERVES a FlagForced element (so PGS content
// can promote it later), but for a non-forced track its value is 0. The
// value byte sits after the 2-byte ID (0x55AA) + 1-byte size.
let pos =
find_id(&data, ebml::FLAG_FORCED).expect("PGS subtitle reserves a FlagForced element");
assert_eq!(
data[pos + 3],
0,
"reserved FlagForced value must be 0 for a non-forced subtitle"
);
}
/// A PGS display-set PES block (PCS) with one composition object, whose
/// forced_on_flag is set per `forced`.
#[cfg(test)]
fn pgs_display_set(forced: bool) -> Vec<u8> {
let mut pcs = vec![0u8; 18];
pcs[0] = 0x16; // segment type PCS
pcs[13] = 1; // number_of_composition_objects
pcs[17] = if forced { 0x40 } else { 0x00 }; // first object flags
pcs
}
#[test]
fn mkv_pgs_forced_promoted_when_all_display_sets_forced() {
// End-to-end: a PGS subtitle track whose every display set is forced is
// promoted to FlagForced=1 at finish(), even though the scan flag was
// false (no vendor metadata).
use crate::disc::SubtitleStream;
use std::sync::{Arc, Mutex};
let shared = Arc::new(Mutex::new(Cursor::new(Vec::new())));
let sub = MkvTrack::subtitle(&SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
});
let tracks = [make_video_track(), sub];
let mut muxer =
MkvMuxer::new(SharedWriter(shared.clone()), &tracks, None, 60.0, &[]).unwrap();
muxer
.write_frame(0, 0, true, &[0u8; 16], Some(40_000_000), None)
.unwrap(); // video keyframe opens a cluster
muxer
.write_frame(
1,
1_000_000,
true,
&pgs_display_set(true),
Some(2_000_000),
None,
)
.unwrap();
muxer.finish().unwrap();
let data = shared.lock().unwrap().clone().into_inner();
let pos = find_id(&data, ebml::FLAG_FORCED).expect("reserved FlagForced");
assert_eq!(
data[pos + 3],
1,
"all-forced PGS track promoted to FlagForced=1"
);
}
#[test]
fn mkv_pgs_not_promoted_when_a_display_set_is_not_forced() {
// A track with ANY non-forced display set is a full track, not forced
// narrative — FlagForced stays 0.
use crate::disc::SubtitleStream;
use std::sync::{Arc, Mutex};
let shared = Arc::new(Mutex::new(Cursor::new(Vec::new())));
let sub = MkvTrack::subtitle(&SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
});
let tracks = [make_video_track(), sub];
let mut muxer =
MkvMuxer::new(SharedWriter(shared.clone()), &tracks, None, 60.0, &[]).unwrap();
muxer
.write_frame(0, 0, true, &[0u8; 16], Some(40_000_000), None)
.unwrap();
muxer
.write_frame(
1,
1_000_000,
true,
&pgs_display_set(true),
Some(2_000_000),
None,
)
.unwrap();
muxer
.write_frame(
1,
3_000_000,
true,
&pgs_display_set(false),
Some(4_000_000),
None,
)
.unwrap(); // a normal (non-forced) subtitle → not a forced track
muxer.finish().unwrap();
let data = shared.lock().unwrap().clone().into_inner();
let pos = find_id(&data, ebml::FLAG_FORCED).expect("reserved FlagForced");
assert_eq!(data[pos + 3], 0, "mixed PGS track stays FlagForced=0");
}
// ============================================================
// Seekability tests: SeekHead, keyframe-aligned clusters, Cues
// ============================================================
+9 -12
View File
@@ -323,17 +323,14 @@ fn extract_mvc_params(data: &[u8]) -> Option<(Vec<u8>, Vec<u8>)> {
}
impl MkvStream {
/// Create for writing PES frames → MKV container.
/// Codec privates come from title.codec_privates (populated by input stream).
pub fn create(writer: Box<dyn WriteSeek + Send>, title: &DiscTitle) -> io::Result<Self> {
Self::create_at(writer, title, None)
}
/// As [`create`](Self::create), but `output_path` (when known) enables the
/// `--log-level 3` opening-frame capture to `<output>.opening.bin`. A `None`
/// path (e.g. an in-memory / stdio sink) silently skips the side-file
/// capture; the per-track TrackEntry dump still fires.
pub fn create_at(
/// Create for writing PES frames → MKV container. Codec privates come from
/// `title.codec_privates` (populated by the input stream).
///
/// `output_path` (when known) enables the `--log-level 3` opening-frame
/// capture to `<output>.opening.bin`; `None` (e.g. an in-memory / stdio sink)
/// silently skips the side-file capture — the per-track TrackEntry dump still
/// fires either way.
pub fn create(
writer: Box<dyn WriteSeek + Send>,
title: &DiscTitle,
output_path: Option<&std::path::Path>,
@@ -1409,7 +1406,7 @@ mod tests {
streams: vec![Stream::Video(dep)],
..DiscTitle::empty()
};
let s = MkvStream::create(Box::new(Cursor::new(Vec::new())), &title)
let s = MkvStream::create(Box::new(Cursor::new(Vec::new())), &title, None)
.expect("create must succeed, not panic");
assert!(
s.mvc.is_none(),
+12 -2
View File
@@ -62,6 +62,7 @@ pub(crate) mod m2ts;
/// to round-trip codec_privates that don't fit inside the underlying format).
/// Exposed for integration tests that exercise the wire format directly.
pub mod meta;
pub(crate) mod meta_sink;
// ── Sequential-sink muxers ──────────────────────────────────────────────────
//
@@ -89,6 +90,7 @@ pub(crate) mod hevc;
pub(crate) mod m2ts_mux;
pub(crate) mod mkv;
pub(crate) mod mkvstream;
pub(crate) mod mp4;
pub(crate) mod network;
pub(crate) mod null;
pub(crate) mod ps;
@@ -108,11 +110,13 @@ pub(crate) mod tsmux;
#[allow(dead_code)]
pub(crate) mod videomap;
pub use demux_sink::{ChaptersFmt, DelayMode, DemuxOptions, DemuxSink, Naming};
// `demux://` and `fvi://` sinks are constructed internally by `output()` via the
// direct `super::demux_sink::` / `super::fvi_sink::` paths — no re-export needed,
// and no consumer names these types, so they are not public API.
pub use disc::DiscStream;
pub use fvi_sink::FviSink;
pub use m2ts::M2tsStream;
pub use mkvstream::MkvStream;
pub use mp4::{Mp4FitReport, Mp4SkipReason, fit_report as mp4_fit_report};
pub use network::NetworkStream;
pub use null::NullStream;
pub use pipelined_stream::PipelinedPesStream;
@@ -152,11 +156,17 @@ mod tests {
assert_eq!(parse_url("disc://").scheme(), "disc");
assert_eq!(parse_url("m2ts://f").scheme(), "m2ts");
assert_eq!(parse_url("mkv://f").scheme(), "mkv");
assert_eq!(parse_url("mp4://f").scheme(), "mp4");
assert_eq!(parse_url("network://h:1").scheme(), "network");
assert_eq!(parse_url("stdio://").scheme(), "stdio");
assert_eq!(parse_url("iso://f").scheme(), "iso");
assert_eq!(parse_url("null://").scheme(), "null");
assert_eq!(parse_url("demux://out/").scheme(), "demux");
assert_eq!(parse_url("video://out/").scheme(), "video");
assert_eq!(parse_url("audio://out/").scheme(), "audio");
assert_eq!(parse_url("sub://out/").scheme(), "sub");
assert_eq!(parse_url("chapters://c.xml").scheme(), "chapters");
assert_eq!(parse_url("json://t.json").scheme(), "json");
assert_eq!(parse_url("bogus://x").scheme(), "unknown");
}
+526
View File
@@ -0,0 +1,526 @@
//! MP4 audio sample entries and codec-config boxes for the `mp4://` muxer.
//!
//! Covers the codecs that map cleanly into MP4 and play widely: **AC-3**
//! (`ac-3` + `dac3`), **E-AC-3 / Dolby Digital Plus** (`ec-3` + `dec3`, incl.
//! Atmos-in-DD+ JOC), and **DTS / DTS-HD** (`dtsc`/`dtsh` + `ddts`, describing
//! the core with whole access units passed through so an HD decoder finds the
//! extension). Config boxes are derived from the first audio frame's bitstream
//! (ISO/IEC 14496-12 amendments; ETSI TS 102 366 / 102 114). Codecs with no
//! clean MP4 mapping (TrueHD, LPCM, bitmap subtitles) are excluded by the fit
//! oracle in the sink.
use super::boxes::bx;
use crate::disc::Codec;
/// AC-3 / E-AC-3 sample rates indexed by `fscod` (byte-4 bits 7-6).
const FSCOD_RATES: [u32; 3] = [48_000, 44_100, 32_000];
/// E-AC-3 reduced rates indexed by `fscod2` (byte-4 bits 5-4) when `fscod == 3`.
const EAC3_REDUCED_RATES: [u32; 4] = [24_000, 22_050, 16_000, 48_000];
/// Base channel count per `acmod` (A/52 Table 5.8), before the LFE.
const ACMOD_CHANNELS: [u8; 8] = [2, 1, 2, 3, 3, 4, 4, 5];
/// A big-endian MSB-first bit reader over a byte slice.
struct BitReader<'a> {
data: &'a [u8],
bit: usize,
}
impl<'a> BitReader<'a> {
fn new(data: &'a [u8]) -> Self {
Self { data, bit: 0 }
}
fn skip(&mut self, n: usize) {
self.bit += n;
}
/// Read `n` bits (n ≤ 32). Returns 0 past end of data (callers pre-check len).
fn read(&mut self, n: usize) -> u32 {
let mut v = 0u32;
for _ in 0..n {
let byte = self.data.get(self.bit / 8).copied().unwrap_or(0);
let shift = 7 - (self.bit % 8);
v = (v << 1) | ((byte >> shift) & 1) as u32;
self.bit += 1;
}
v
}
}
/// Decoded (E-)AC-3 stream parameters needed for the `dac3`/`dec3` config box
/// and the audio sample entry.
pub(super) struct DolbyConfig {
pub fscod: u8,
pub bsid: u8,
pub bsmod: u8,
pub acmod: u8,
pub lfeon: bool,
/// AC-3 only: `bit_rate_code` (= `frmsizecod >> 1`). Unused for E-AC-3.
pub bit_rate_code: u8,
/// E-AC-3 only: nominal data rate in kbps (for `dec3`). 0 for AC-3.
pub data_rate_kbps: u16,
pub sample_rate: u32,
pub channels: u16,
}
impl DolbyConfig {
fn channel_count(acmod: u8, lfeon: bool) -> u16 {
ACMOD_CHANNELS[acmod as usize] as u16 + lfeon as u16
}
}
/// Parse the first (E-)AC-3 frame starting at the 0x0B77 syncword. Returns
/// `None` if the frame is too short or the syncword is absent.
pub(super) fn parse_dolby(frame: &[u8]) -> Option<DolbyConfig> {
let start =
(0..frame.len().saturating_sub(1)).find(|&i| frame[i] == 0x0B && frame[i + 1] == 0x77)?;
let f = &frame[start..];
if f.len() < 6 {
return None;
}
// bsid lives in byte 5 bits 7-3 for both AC-3 and E-AC-3.
let bsid = (f[5] >> 3) & 0x1F;
if bsid >= 11 {
parse_eac3(f)
} else {
parse_ac3(f)
}
}
/// Legacy AC-3 (A/52 §5.3.2): syncword | crc(16) | fscod(2) frmsizecod(6) |
/// bsid(5) bsmod(3) | acmod(3) …optional… lfeon.
fn parse_ac3(f: &[u8]) -> Option<DolbyConfig> {
if f.len() < 8 {
return None;
}
let fscod = (f[4] >> 6) & 0x03;
let frmsizecod = f[4] & 0x3F;
let bsid = (f[5] >> 3) & 0x1F;
let bsmod = f[5] & 0x07;
// acmod + trailing optional 2-bit fields, then lfeon (byte 6 onward).
let mut r = BitReader::new(f);
r.bit = 6 * 8;
let acmod = r.read(3) as u8;
if (acmod & 0x1) != 0 && acmod != 0x1 {
r.skip(2); // cmixlev
}
if (acmod & 0x4) != 0 {
r.skip(2); // surmixlev
}
if acmod == 0x2 {
r.skip(2); // dsurmod
}
let lfeon = r.read(1) == 1;
Some(DolbyConfig {
fscod,
bsid,
bsmod,
acmod,
lfeon,
bit_rate_code: frmsizecod >> 1,
data_rate_kbps: 0,
sample_rate: FSCOD_RATES.get(fscod as usize).copied().unwrap_or(48_000),
channels: DolbyConfig::channel_count(acmod, lfeon),
})
}
/// E-AC-3 (A/52 Annex E BSI): syncword | strmtyp(2) substreamid(3) frmsiz(11) |
/// fscod(2) numblkscod(2) acmod(3) lfeon(1) | bsid(5) …
fn parse_eac3(f: &[u8]) -> Option<DolbyConfig> {
if f.len() < 6 {
return None;
}
let frmsiz = (((f[2] & 0x07) as u32) << 8) | f[3] as u32; // words minus one
let fscod = (f[4] >> 6) & 0x03;
let numblkscod = (f[4] >> 4) & 0x03;
let acmod = (f[4] >> 1) & 0x07;
let lfeon = (f[4] & 0x01) == 1;
let bsid = (f[5] >> 3) & 0x1F;
let (sample_rate, blocks) = if fscod == 0x03 {
let fscod2 = (f[4] >> 4) & 0x03; // shares bits with numblkscod when fscod==3
(EAC3_REDUCED_RATES[fscod2 as usize], 6u32)
} else {
let blocks = [1u32, 2, 3, 6][numblkscod as usize];
(FSCOD_RATES[fscod as usize], blocks)
};
// Nominal data rate (kbps): frame is (frmsiz+1) 16-bit words per (blocks·256)
// samples at sample_rate. rate = bytes·8·sr / samples / 1000.
let frame_bytes = (frmsiz as u64 + 1) * 2;
let samples = blocks as u64 * 256;
let data_rate_kbps = if samples > 0 {
((frame_bytes * 8 * sample_rate as u64) / samples / 1000) as u16
} else {
0
};
Some(DolbyConfig {
fscod,
bsid,
bsmod: 0, // not in the E-AC-3 main header; dec3 default
acmod,
lfeon,
bit_rate_code: 0,
data_rate_kbps,
sample_rate,
channels: DolbyConfig::channel_count(acmod, lfeon),
})
}
/// The `dac3` config box (ETSI TS 102 366 Annex F.4): 24 bits —
/// fscod(2) bsid(5) bsmod(3) acmod(3) lfeon(1) bit_rate_code(5) reserved(5).
pub(super) fn dac3_box(c: &DolbyConfig) -> Vec<u8> {
let mut v: u32 = 0;
let mut push = |val: u32, bits: u32| v = (v << bits) | (val & ((1 << bits) - 1));
push(c.fscod as u32, 2);
push(c.bsid as u32, 5);
push(c.bsmod as u32, 3);
push(c.acmod as u32, 3);
push(c.lfeon as u32, 1);
push(c.bit_rate_code as u32, 5);
push(0, 5); // reserved
// 24 bits → the top 3 bytes of the big-endian u32.
let b = v.to_be_bytes();
bx(b"dac3", &[b[1], b[2], b[3]])
}
/// The `dec3` config box (ETSI TS 102 366 Annex G.3) for a single independent
/// substream, no dependent substreams: data_rate(13) num_ind_sub(3) then
/// fscod(2) bsid(5) reserved(1) asvc(1) bsmod(3) acmod(3) lfeon(1) reserved(3)
/// num_dep_sub(4) reserved(1).
pub(super) fn dec3_box(c: &DolbyConfig) -> Vec<u8> {
let mut v: u64 = 0;
let mut push = |val: u64, bits: u32| v = (v << bits) | (val & ((1u64 << bits) - 1));
push(c.data_rate_kbps as u64, 13);
push(0, 3); // num_ind_sub - 1 = 0 (one substream)
push(c.fscod as u64, 2);
push(c.bsid as u64, 5);
push(0, 1); // reserved
push(0, 1); // asvc
push(c.bsmod as u64, 3);
push(c.acmod as u64, 3);
push(c.lfeon as u64, 1);
push(0, 3); // reserved
push(0, 4); // num_dep_sub = 0
push(0, 1); // reserved (chan_loc absent when num_dep_sub == 0)
// 40 bits → the low 5 bytes of the big-endian u64.
let b = v.to_be_bytes();
bx(b"dec3", &[b[3], b[4], b[5], b[6], b[7]])
}
/// Build an audio sample entry (`ac-3` / `ec-3`) with the given config box.
/// `AudioSampleEntry` per ISO/IEC 14496-12 §12.2.3.
pub(super) fn audio_sample_entry(
fourcc: &[u8; 4],
channels: u16,
sample_rate: u32,
config: &[u8],
) -> Vec<u8> {
let mut e = Vec::new();
e.extend_from_slice(&[0u8; 6]); // reserved
e.extend_from_slice(&1u16.to_be_bytes()); // data_reference_index
e.extend_from_slice(&[0u8; 8]); // reserved (version 0)
e.extend_from_slice(&channels.to_be_bytes());
e.extend_from_slice(&16u16.to_be_bytes()); // samplesize
e.extend_from_slice(&0u16.to_be_bytes()); // pre_defined
e.extend_from_slice(&0u16.to_be_bytes()); // reserved
// samplerate is 16.16 fixed point; the integer rate in the high 16 bits.
e.extend_from_slice(&(sample_rate << 16).to_be_bytes());
e.extend_from_slice(config);
bx(fourcc, &e)
}
// ── DTS (dtsc/dtsh + ddts) ───────────────────────────────────────────────────
/// DTS core `SFREQ` (4-bit) → sample rate (Hz). Reserved indices → 48 kHz.
const DTS_SFREQ: [u32; 16] = [
48_000, 8_000, 16_000, 32_000, 48_000, 48_000, 11_025, 22_050, 44_100, 48_000, 48_000, 12_000,
24_000, 48_000, 96_000, 192_000,
];
/// DTS core base channel count per `AMODE` (0..=9); higher AMODEs are rare on disc.
const DTS_AMODE_CH: [u8; 10] = [1, 2, 2, 2, 2, 3, 3, 4, 4, 5];
/// Decoded DTS core parameters needed for the `ddts` box.
struct DtsConfig {
sample_rate: u32,
channels: u16,
amode: u8,
lfe: bool,
core_size: u32,
/// Samples per frame ((NBLKS+1)·32).
frame_samples: u32,
/// Whether a DTS-HD extension substream follows the core.
has_extension: bool,
channel_layout: u16,
}
/// Parse the DTS core header (ETSI TS 102 114 §5.3.1), starting at the
/// 0x7FFE8001 big-endian core sync. Returns `None` if too short / no sync.
fn parse_dts(frame: &[u8]) -> Option<DtsConfig> {
let start = (0..frame.len().saturating_sub(3)).find(|&i| {
frame[i] == 0x7F && frame[i + 1] == 0xFE && frame[i + 2] == 0x80 && frame[i + 3] == 0x01
})?;
let f = &frame[start..];
if f.len() < 11 {
return None;
}
// Bit fields after the 32-bit sync (MSB-first):
// FTYPE1 SHORT5 CPF1 NBLKS7 FSIZE14 AMODE6 SFREQ4 RATE5 ...
let nblks = (((f[4] & 0x01) as u32) << 6) | ((f[5] >> 2) as u32 & 0x3F);
let fsize = (((f[5] & 0x03) as u32) << 12) | ((f[6] as u32) << 4) | ((f[7] >> 4) as u32 & 0x0F);
let amode = (((f[7] & 0x0F) << 2) | ((f[8] >> 6) & 0x03)) as usize;
let sfreq = ((f[8] >> 2) & 0x0F) as usize;
// LFF is 2 bits at bit offset 85 → byte10 bits 2-1.
let lff = (f[10] >> 1) & 0x03;
let lfe = lff == 1 || lff == 2;
let sample_rate = DTS_SFREQ[sfreq];
let base_ch = DTS_AMODE_CH.get(amode).copied().unwrap_or(6);
let channels = base_ch as u16 + lfe as u16;
let channel_layout = dts_channel_layout(amode, lfe);
// DTS-HD extension substream sync (0x64582025) after the core frame.
let ext_sync = [0x64, 0x58, 0x20, 0x25];
let has_extension = f
.windows(4)
.skip((fsize as usize + 1).min(f.len()).saturating_sub(4))
.any(|w| w == ext_sync)
|| f.windows(4).any(|w| w == ext_sync);
Some(DtsConfig {
sample_rate,
channels,
amode: amode as u8,
lfe,
core_size: fsize + 1,
frame_samples: (nblks + 1) * 32,
has_extension,
channel_layout,
})
}
/// `ddts` ChannelLayout (16-bit speaker mask) for the common core layouts.
/// bit0=C, bit1=L/R, bit2=Ls/Rs, bit3=LFE.
fn dts_channel_layout(amode: usize, lfe: bool) -> u16 {
let mut m = match amode {
0 => 0x0001, // C (mono)
1..=4 => 0x0002, // L/R
5 => 0x0003, // C + L/R
6 | 8 => 0x0006, // L/R + Ls/Rs (no centre)
_ => 0x0007, // C + L/R + surround (amode 7, 9, …)
};
if lfe {
m |= 0x0008;
}
m
}
/// The `ddts` config box (ETSI TS 102 114 Annex; DTS-in-ISO registration).
/// Describes the DTS core; whole access units (core + any extension) are passed
/// through as samples, so a DTS-HD-aware decoder still finds the extension.
fn ddts_box(c: &DtsConfig) -> Vec<u8> {
// avg/max bitrate: computed from the core frame size × frame rate (the core
// RATE field reads "open/variable" for lossless, so it's not usable directly).
let frames_per_sec = if c.frame_samples > 0 {
c.sample_rate as u64 / c.frame_samples as u64
} else {
0
};
let bitrate = (c.core_size as u64 * 8 * frames_per_sec) as u32;
let mut out = Vec::new();
out.extend_from_slice(&c.sample_rate.to_be_bytes()); // DTSSamplingFrequency
out.extend_from_slice(&bitrate.to_be_bytes()); // maxBitrate
out.extend_from_slice(&bitrate.to_be_bytes()); // avgBitrate
out.push(if c.has_extension { 24 } else { 16 }); // pcmSampleDepth
// Bit-packed tail (56 bits):
// FrameDuration2 StreamConstruction5 CoreLFEPresent1 CoreLayout6 CoreSize14
// StereoDownmix1 RepresentationType3 ChannelLayout16 MultiAssetFlag1
// LBRDurationMod1 ReservedBoxPresent1 Reserved5
let frame_duration = match c.frame_samples {
0..=512 => 0,
513..=1024 => 1,
1025..=2048 => 2,
_ => 3,
};
// StreamConstruction: 1 = DTS core present. Whole-AU passthrough means an
// HD decoder still parses the extension substreams from the stream itself.
let stream_construction = 1u128;
let mut v: u128 = 0;
let mut push = |val: u128, bits: u32| v = (v << bits) | (val & ((1u128 << bits) - 1));
push(frame_duration as u128, 2);
push(stream_construction, 5);
push(c.lfe as u128, 1);
push(c.amode as u128, 6);
push(c.core_size as u128, 14);
push(0, 1); // StereoDownmix
push(0, 3); // RepresentationType
push(c.channel_layout as u128, 16);
push(c.has_extension as u128, 1); // MultiAssetFlag
push(0, 1); // LBRDurationMod
push(0, 1); // ReservedBoxPresent
push(0, 5); // Reserved
// 56 bits → the low 7 bytes of the big-endian u128.
let b = v.to_be_bytes();
out.extend_from_slice(&b[9..16]);
bx(b"ddts", &out)
}
/// The MP4 fourcc + config box for an audio frame, or `None` if the codec has no
/// MP4 mapping here. Together with [`audio_fits`] this is the fit oracle for
/// audio: only what returns `Some` is muxable.
pub(super) fn dolby_sample_entry(codec: Codec, first_frame: &[u8]) -> Option<Vec<u8>> {
match codec {
Codec::Ac3 => {
let c = parse_dolby(first_frame)?;
Some(audio_sample_entry(
b"ac-3",
c.channels,
c.sample_rate,
&dac3_box(&c),
))
}
Codec::Ac3Plus => {
let c = parse_dolby(first_frame)?;
Some(audio_sample_entry(
b"ec-3",
c.channels,
c.sample_rate,
&dec3_box(&c),
))
}
Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr => {
let c = parse_dts(first_frame)?;
// `dtsc` = DTS core; `dtsh` = DTS-HD (core + extension substreams).
let fourcc: &[u8; 4] = if c.has_extension { b"dtsh" } else { b"dtsc" };
Some(audio_sample_entry(
fourcc,
c.channels,
c.sample_rate,
&ddts_box(&c),
))
}
_ => None,
}
}
/// Fit oracle for an audio codec: does `mp4://` currently carry it? Covers the
/// Dolby family (AC-3 / E-AC-3) and DTS (core / DTS-HD HRA / DTS-HD MA — the core
/// is described, whole access units pass through). TrueHD, LPCM, AAC are not yet
/// mapped and are skipped with a loud report (never silently dropped).
pub(super) fn audio_fits(codec: Codec) -> bool {
matches!(
codec,
Codec::Ac3 | Codec::Ac3Plus | Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr
)
}
#[cfg(test)]
mod tests {
use super::*;
/// A synthetic legacy AC-3 header: syncword, crc, fscod=0 (48k),
/// frmsizecod, bsid=8, bsmod=0, acmod=7 (3/2), lfeon=1 → 5.1.
fn ac3_frame_5_1() -> Vec<u8> {
let mut f = vec![0x0B, 0x77, 0x00, 0x00];
// byte4: fscod(2)=0 | frmsizecod(6)=0b010110 (22)
f.push(0b00_010110);
// byte5: bsid(5)=8 (0b01000) | bsmod(3)=0
f.push(0b01000_000);
// byte6: acmod(3)=7 (0b111) | cmixlev(2) | surmixlev(2) | lfeon(1)...
// acmod=7 has centre (needs cmixlev) and surround (needs surmixlev):
// 111 | 00 | 00 | 1(lfeon) = 0b111_00_00_1
f.push(0b111_00_00_1);
f.push(0x00);
f
}
#[test]
fn ac3_bsi_and_dac3() {
let c = parse_dolby(&ac3_frame_5_1()).expect("parsed");
assert!(c.bsid < 11, "legacy AC-3");
assert_eq!(c.fscod, 0);
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.bsid, 8);
assert_eq!(c.acmod, 7);
assert!(c.lfeon);
assert_eq!(c.channels, 6, "3/2 + LFE = 5.1");
assert_eq!(c.bit_rate_code, 22 >> 1);
let dac3 = dac3_box(&c);
// [size:4]["dac3"][3-byte payload] = 11 bytes.
assert_eq!(dac3.len(), 11);
assert_eq!(&dac3[4..8], b"dac3");
}
#[test]
fn eac3_bsi_and_dec3() {
// E-AC-3: syncword | strmtyp/substreamid/frmsiz | fscod/numblks/acmod/lfeon | bsid
let mut f = vec![0x0B, 0x77];
f.push(0x00); // strmtyp=0, substreamid=0, frmsiz high=0
f.push(0x3F); // frmsiz low = 63 → frame 128 bytes
// byte4: fscod(2)=0 | numblkscod(2)=3 (6 blocks) | acmod(3)=7 | lfeon(1)=1
f.push(0b00_11_111_1);
// byte5: bsid(5)=16 (E-AC-3) | dialnorm high(3)
f.push(0b10000_000);
f.push(0x00);
let c = parse_dolby(&f).expect("parsed");
assert!(c.bsid >= 11, "E-AC-3");
assert_eq!(c.bsid, 16);
assert_eq!(c.fscod, 0);
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.acmod, 7);
assert!(c.lfeon);
assert_eq!(c.channels, 6);
let dec3 = dec3_box(&c);
// [size:4]["dec3"][5-byte payload] = 13 bytes.
assert_eq!(dec3.len(), 13);
assert_eq!(&dec3[4..8], b"dec3");
}
#[test]
fn sample_entry_shape() {
let c = parse_dolby(&ac3_frame_5_1()).unwrap();
let e = audio_sample_entry(b"ac-3", c.channels, c.sample_rate, &dac3_box(&c));
assert_eq!(&e[4..8], b"ac-3");
// channelcount at entry-body offset 16 (after 6 reserved + 2 dri + 8 reserved).
let ch = u16::from_be_bytes([e[8 + 16], e[8 + 17]]);
assert_eq!(ch, 6);
}
#[test]
fn fit_oracle_covers_dolby_and_dts() {
assert!(audio_fits(Codec::Ac3));
assert!(audio_fits(Codec::Ac3Plus));
assert!(audio_fits(Codec::Dts));
assert!(audio_fits(Codec::DtsHdMa));
assert!(!audio_fits(Codec::TrueHd));
assert!(!audio_fits(Codec::Lpcm));
}
#[test]
fn dts_core_5_1_and_ddts() {
// Synthetic DTS core: SFREQ=13 (48k), AMODE=9 (5ch), LFF=1 (LFE) → 5.1.
let f = vec![
0x7F, 0xFE, 0x80, 0x01, 0x00, 0x3C, 0x05, 0xF2, 0x77, 0x00, 0x02, 0x00,
];
let c = parse_dts(&f).expect("dts core parsed");
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.amode, 9);
assert!(c.lfe);
assert_eq!(c.channels, 6, "5 core + LFE = 5.1");
assert_eq!(c.channel_layout, 0x000F, "C + L/R + Ls/Rs + LFE");
assert_eq!(c.core_size, 96);
assert_eq!(c.frame_samples, 512);
let ddts = ddts_box(&c);
assert_eq!(&ddts[4..8], b"ddts");
// DTSSamplingFrequency (first field) = 48000.
assert_eq!(
u32::from_be_bytes([ddts[8], ddts[9], ddts[10], ddts[11]]),
48_000
);
// Sample entry uses dtsc (no extension in this synthetic frame).
let e = dolby_sample_entry(Codec::DtsHdMa, &f).unwrap();
assert_eq!(&e[4..8], b"dtsc");
}
}
+27
View File
@@ -0,0 +1,27 @@
//! ISO-BMFF box primitives: `[size:u32-BE][type:4][body]` (ISO/IEC 14496-12
//! §4.2), and the FullBox variant that prefixes a 1-byte version + 3-byte flags.
/// Wrap a body in a plain box `[size][type][body]`. `size` counts the 8-byte
/// header. All `moov`-tree boxes are small (the large `mdat` is written directly
/// with a 64-bit size, not through here), so a `u32` size never overflows.
pub(super) fn bx(box_type: &[u8; 4], body: &[u8]) -> Vec<u8> {
let total = body.len() + 8;
debug_assert!(
total <= u32::MAX as usize,
"mp4 box {box_type:?} exceeds u32"
);
let mut out = Vec::with_capacity(total);
out.extend_from_slice(&(total as u32).to_be_bytes());
out.extend_from_slice(box_type);
out.extend_from_slice(body);
out
}
/// Wrap a body in a FullBox: `[size][type][version:1][flags:3][body]`.
pub(super) fn fullbox(box_type: &[u8; 4], version: u8, flags: u32, body: &[u8]) -> Vec<u8> {
let mut full = Vec::with_capacity(body.len() + 4);
full.push(version);
full.extend_from_slice(&flags.to_be_bytes()[1..]); // low 3 bytes
full.extend_from_slice(body);
bx(box_type, &full)
}
+1099
View File
File diff suppressed because it is too large Load Diff
+705
View File
@@ -0,0 +1,705 @@
//! Progressive MP4 (ISO-BMFF) demuxer — the read side of `mp4://`.
//!
//! The inverse of the writer in [`super`]: parse `moov`/`trak`/`stbl`, rebuild a
//! [`DiscTitle`] and a per-sample index (offset/size/timing/sync from
//! `stsc`+`stco`/`co64`+`stsz`, `stts`+`ctts`, `stss`), then stream each sample
//! out as a [`PesFrame`] in decode order. Video NALs are length-prefixed in MP4 —
//! exactly the framing the MKV muxer consumes — so `mp4://` → `mkv://` (and
//! `mp4://` → any sink) needs no reframing.
//!
//! Scope: progressive MP4 (`moov` + `mdat`). Fragmented MP4 (`moof`) is out of
//! scope for now — its samples live in `traf`/`trun`, not `stbl`.
use crate::disc::{
AudioChannels, AudioStream, Codec, DiscTitle, Resolution, SampleRate, Stream as DiscStream,
VideoStream,
};
use crate::labels::LabelPurpose;
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Read, Seek, SeekFrom};
use std::path::Path;
const NS: i128 = 1_000_000_000;
/// One sample's location + timing in the emission plan.
struct SampleRef {
track: usize,
offset: u64,
size: u32,
/// Composition (presentation) time in nanoseconds.
pts_ns: i64,
/// Decode time in nanoseconds — the key the global emission order sorts on.
dts_ns: i64,
keyframe: bool,
}
/// MP4 reader: a `Stream` source that emits a file's samples as PES frames.
/// Generic over the backing reader so it works over a `File` (the `mp4://`
/// source) or an in-memory `Cursor` (round-trip tests).
pub struct Mp4Reader<R: Read + Seek> {
file: R,
title: DiscTitle,
samples: Vec<SampleRef>,
cursor: usize,
}
impl Mp4Reader<File> {
/// Open and index an MP4 file by path.
pub fn open(path: &Path) -> io::Result<Self> {
let name = path
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("mp4")
.to_string();
Self::from_reader(File::open(path)?, name)
}
}
impl<R: Read + Seek> Mp4Reader<R> {
/// Index an already-opened seekable MP4 reader.
pub fn from_reader(mut file: R, name: String) -> io::Result<Self> {
let moov = read_moov(&mut file)?;
let mut title = DiscTitle::empty();
title.playlist = name;
let mut samples: Vec<SampleRef> = Vec::new();
let mut codec_privates: Vec<Option<Vec<u8>>> = Vec::new();
let mut track_idx = 0usize;
for trak in find_boxes(&moov, b"trak") {
let Some(mdia) = find_box(trak, b"mdia") else {
continue;
};
let timescale = find_box(mdia, b"mdhd")
.and_then(mdhd_timescale)
.unwrap_or(90_000);
let language = find_box(mdia, b"mdhd").and_then(mdhd_language);
let handler = find_box(mdia, b"hdlr").and_then(hdlr_type);
let Some(minf) = find_box(mdia, b"minf") else {
continue;
};
let Some(stbl) = find_box(minf, b"stbl") else {
continue;
};
let Some(stsd) = find_box(stbl, b"stsd") else {
continue;
};
let Some(StsdInfo {
codec,
height,
config,
channels,
}) = parse_stsd(stsd)
else {
continue; // unrecognised sample entry — skip the track
};
// Build the stream model for this track.
let stream = match handler {
Some(h) if &h == b"vide" => DiscStream::Video(VideoStream {
pid: 0x1011 + track_idx as u16,
codec,
resolution: Resolution::from_height(height as u32),
frame_rate: crate::disc::FrameRate::Unknown,
hdr: crate::disc::HdrFormat::Sdr,
color_space: crate::disc::ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
}),
Some(h) if &h == b"soun" => DiscStream::Audio(AudioStream {
pid: 0x1100 + track_idx as u16,
codec,
channels: AudioChannels::from_count(channels as u8),
language: language.clone().unwrap_or_else(|| "und".into()),
sample_rate: SampleRate::from_hz(timescale),
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
}),
_ => continue, // non-A/V handler
};
// Per-sample tables.
let sizes = find_box(stbl, b"stsz").map(parse_stsz).unwrap_or_default();
let n = sizes.len();
if n == 0 {
track_idx += 1;
title.streams.push(stream);
codec_privates.push(config);
continue;
}
let chunk_offsets = find_box(stbl, b"stco")
.map(|b| parse_stco(b, false))
.or_else(|| find_box(stbl, b"co64").map(|b| parse_stco(b, true)))
.unwrap_or_default();
let stsc = find_box(stbl, b"stsc").map(parse_stsc).unwrap_or_default();
let offsets = sample_offsets(&sizes, &chunk_offsets, &stsc);
let durations = find_box(stbl, b"stts").map(parse_stts).unwrap_or_default();
let ctts = find_box(stbl, b"ctts").map(parse_ctts).unwrap_or_default();
let sync = find_box(stbl, b"stss").map(parse_stss);
let mut decode_ticks: i64 = 0;
for (i, &size) in sizes.iter().enumerate() {
let dur = durations.get(i).copied().unwrap_or(0);
let comp = ctts.get(i).copied().unwrap_or(0);
let dts_ns = (decode_ticks as i128 * NS / timescale as i128) as i64;
let pts_ticks = decode_ticks + comp as i64;
let pts_ns = (pts_ticks as i128 * NS / timescale as i128) as i64;
decode_ticks += dur as i64;
let keyframe = match &sync {
Some(set) => set.contains(&(i as u32 + 1)),
None => true, // no stss → every sample is a sync sample
};
samples.push(SampleRef {
track: track_idx,
offset: offsets.get(i).copied().unwrap_or(0),
size,
pts_ns,
dts_ns,
keyframe,
});
}
title.streams.push(stream);
codec_privates.push(config);
track_idx += 1;
}
if title.streams.is_empty() {
return Err(crate::error::Error::MkvInvalid.into());
}
title.codec_privates = codec_privates;
// Emit in global decode order so the consumer sees interleaved,
// monotonic-DTS frames (a stable sort keeps per-track order on ties).
samples.sort_by_key(|s| s.dts_ns);
Ok(Self {
file,
title,
samples,
cursor: 0,
})
}
}
impl<R: Read + Seek + Send> Stream for Mp4Reader<R> {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
let Some(s) = self.samples.get(self.cursor) else {
return Ok(None);
};
self.cursor += 1;
self.file.seek(SeekFrom::Start(s.offset))?;
let mut data = vec![0u8; s.size as usize];
self.file.read_exact(&mut data)?;
Ok(Some(PesFrame {
track: s.track,
pts: s.pts_ns,
keyframe: s.keyframe,
data,
duration_ns: None,
source: None,
coding: None,
}))
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Err(crate::error::Error::StreamReadOnly.into())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
self.title.codec_privates.get(track).and_then(|c| c.clone())
}
}
// ── box tree navigation ──────────────────────────────────────────────────────
/// Read top-level boxes until `moov`, returning its payload (after the header).
/// Skips over `ftyp`/`mdat`/etc. via seek; samples are read later by offset.
fn read_moov<R: Read + Seek>(file: &mut R) -> io::Result<Vec<u8>> {
loop {
let mut hdr = [0u8; 8];
if file.read_exact(&mut hdr).is_err() {
return Err(crate::error::Error::MkvInvalid.into());
}
let size32 = u32::from_be_bytes([hdr[0], hdr[1], hdr[2], hdr[3]]);
let btype = [hdr[4], hdr[5], hdr[6], hdr[7]];
// 64-bit largesize (size==1): the real size is the next 8 bytes; a
// 16-byte header precedes the payload. size==0 means "to EOF".
let payload_len: u64 = if size32 == 1 {
let mut ext = [0u8; 8];
file.read_exact(&mut ext)?;
u64::from_be_bytes(ext).saturating_sub(16)
} else {
(size32 as u64).saturating_sub(8)
};
if &btype == b"moov" {
let mut buf = vec![0u8; payload_len as usize];
file.read_exact(&mut buf)?;
return Ok(buf);
}
file.seek(SeekFrom::Current(payload_len as i64))?;
}
}
/// The first child box of `payload` with the given type — returns its payload
/// (bytes after the 8-byte header). One level.
fn find_box<'a>(payload: &'a [u8], want: &[u8; 4]) -> Option<&'a [u8]> {
find_boxes(payload, want).into_iter().next()
}
/// All child boxes of `payload` with the given type — each as its payload slice.
fn find_boxes<'a>(payload: &'a [u8], want: &[u8; 4]) -> Vec<&'a [u8]> {
let mut out = Vec::new();
let mut pos = 0;
while pos + 8 <= payload.len() {
let size = u32::from_be_bytes([
payload[pos],
payload[pos + 1],
payload[pos + 2],
payload[pos + 3],
]) as usize;
let bt = [
payload[pos + 4],
payload[pos + 5],
payload[pos + 6],
payload[pos + 7],
];
if size < 8 || pos + size > payload.len() {
break;
}
if &bt == want {
out.push(&payload[pos + 8..pos + size]);
}
pos += size;
}
out
}
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn be16(b: &[u8], o: usize) -> u16 {
u16::from_be_bytes([b[o], b[o + 1]])
}
/// mdhd (version 0/1) → media timescale.
fn mdhd_timescale(b: &[u8]) -> Option<u32> {
let version = b.first().copied()?;
if version == 1 {
// version(1)+flags(3) creation(8) modification(8) timescale(4) ...
(b.len() >= 24).then(|| be32(b, 20))
} else {
// creation(4) modification(4) timescale(4) ...
(b.len() >= 16).then(|| be32(b, 12))
}
}
/// mdhd language (5-bit packed ISO 639-2) → lowercase 3-letter code.
fn mdhd_language(b: &[u8]) -> Option<String> {
let version = b.first().copied()?;
let off = if version == 1 { 28 } else { 20 };
if b.len() < off + 2 {
return None;
}
let packed = be16(b, off);
let c0 = ((packed >> 10) & 0x1F) as u8 + 0x60;
let c1 = ((packed >> 5) & 0x1F) as u8 + 0x60;
let c2 = (packed & 0x1F) as u8 + 0x60;
let s: String = [c0, c1, c2].iter().map(|&c| c as char).collect();
if s.chars().all(|c| c.is_ascii_lowercase()) {
Some(s)
} else {
None
}
}
/// hdlr → handler_type fourcc ('vide' / 'soun').
fn hdlr_type(b: &[u8]) -> Option<[u8; 4]> {
// version+flags(4) pre_defined(4) handler_type(4) ...
(b.len() >= 12).then(|| [b[8], b[9], b[10], b[11]])
}
/// Decoded first sample entry of an `stsd` box.
struct StsdInfo {
codec: Codec,
height: u16,
config: Option<Vec<u8>>,
channels: u16,
}
/// stsd → codec + dimensions + codec_private + channel count (first entry).
fn parse_stsd(b: &[u8]) -> Option<StsdInfo> {
// version+flags(4) entry_count(4) then the first sample entry box.
if b.len() < 8 {
return None;
}
let entry = &b[8..];
if entry.len() < 8 {
return None;
}
let size = be32(entry, 0) as usize;
let fourcc = [entry[4], entry[5], entry[6], entry[7]];
let body = &entry[8..size.min(entry.len())];
let codec = match &fourcc {
b"hvc1" | b"hev1" => Codec::Hevc,
b"avc1" | b"avc3" => Codec::H264,
b"ac-3" => Codec::Ac3,
b"ec-3" => Codec::Ac3Plus,
b"mp4a" => Codec::Aac,
b"dtsc" | b"dtse" | b"dtsh" | b"dtsl" => Codec::Dts,
_ => return None,
};
if matches!(codec, Codec::Hevc | Codec::H264) {
// VisualSampleEntry: 6 reserved + 2 dri + 16 pre/reserved + width(2)
// height(2) + 14 + 32 compressorname + 2 depth + 2 pre = 78 bytes, then
// child boxes (hvcC/avcC, colr, …).
if body.len() < 78 {
return None;
}
let height = be16(body, 26);
let children = &body[78..];
let config = find_box(children, b"hvcC")
.or_else(|| find_box(children, b"avcC"))
.map(|c| c.to_vec());
Some(StsdInfo {
codec,
height,
config,
channels: 0,
})
} else {
// AudioSampleEntry: 6 reserved + 2 dri + 8 reserved + channelcount(2)
// samplesize(2) + 2 pre + 2 reserved + samplerate(4) = 28 bytes.
let channels = if body.len() >= 28 { be16(body, 16) } else { 2 };
Some(StsdInfo {
codec,
height: 0,
config: None,
channels,
})
}
}
/// stsz → per-sample sizes.
fn parse_stsz(b: &[u8]) -> Vec<u32> {
if b.len() < 12 {
return Vec::new();
}
let sample_size = be32(b, 4);
let count = be32(b, 8) as usize;
if sample_size != 0 {
return vec![sample_size; count];
}
let mut out = Vec::with_capacity(count);
for i in 0..count {
let o = 12 + i * 4;
if o + 4 > b.len() {
break;
}
out.push(be32(b, o));
}
out
}
/// stco (32-bit) / co64 (64-bit) → chunk offsets.
fn parse_stco(b: &[u8], is64: bool) -> Vec<u64> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::with_capacity(count);
let stride = if is64 { 8 } else { 4 };
for i in 0..count {
let o = 8 + i * stride;
if o + stride > b.len() {
break;
}
if is64 {
out.push(u64::from_be_bytes([
b[o],
b[o + 1],
b[o + 2],
b[o + 3],
b[o + 4],
b[o + 5],
b[o + 6],
b[o + 7],
]));
} else {
out.push(be32(b, o) as u64);
}
}
out
}
/// stsc → (first_chunk, samples_per_chunk) entries (1-based first_chunk).
fn parse_stsc(b: &[u8]) -> Vec<(u32, u32)> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::with_capacity(count);
for i in 0..count {
let o = 8 + i * 12;
if o + 12 > b.len() {
break;
}
out.push((be32(b, o), be32(b, o + 4)));
}
out
}
/// Reconstruct per-sample file offsets from sizes + chunk offsets + stsc.
fn sample_offsets(sizes: &[u32], chunk_offsets: &[u64], stsc: &[(u32, u32)]) -> Vec<u64> {
let n_chunks = chunk_offsets.len();
// Expand stsc → samples_per_chunk for every chunk.
let mut spc = vec![0u32; n_chunks];
for (idx, &(first, per)) in stsc.iter().enumerate() {
let start = (first.saturating_sub(1)) as usize;
let end = stsc
.get(idx + 1)
.map(|&(nf, _)| (nf.saturating_sub(1)) as usize)
.unwrap_or(n_chunks);
let end = end.min(n_chunks);
if start < end {
spc[start..end].fill(per);
}
}
let mut offsets = Vec::with_capacity(sizes.len());
let mut sidx = 0usize;
for (ci, &choff) in chunk_offsets.iter().enumerate() {
let mut off = choff;
for _ in 0..spc[ci] {
if sidx >= sizes.len() {
break;
}
offsets.push(off);
off += sizes[sidx] as u64;
sidx += 1;
}
}
// Any trailing samples with no chunk mapping: pack after the last offset.
while offsets.len() < sizes.len() {
let last = offsets.last().copied().unwrap_or(0);
let last_sz = sizes
.get(offsets.len().saturating_sub(1))
.copied()
.unwrap_or(0);
offsets.push(last + last_sz as u64);
}
offsets
}
/// stts → per-sample decode durations (expanded from run-length entries).
fn parse_stts(b: &[u8]) -> Vec<u32> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::new();
for i in 0..count {
let o = 8 + i * 8;
if o + 8 > b.len() {
break;
}
let n = be32(b, o);
let delta = be32(b, o + 4);
for _ in 0..n {
out.push(delta);
}
}
out
}
/// ctts → per-sample composition offsets (version 0 unsigned / version 1 signed).
fn parse_ctts(b: &[u8]) -> Vec<i32> {
if b.len() < 8 {
return Vec::new();
}
// version 0 = unsigned, version 1 = signed offsets; the u32→i32 bit-cast
// reads both correctly (real composition offsets fit in i32 either way).
let count = be32(b, 4) as usize;
let mut out = Vec::new();
for i in 0..count {
let o = 8 + i * 8;
if o + 8 > b.len() {
break;
}
let n = be32(b, o);
let offset = be32(b, o + 4) as i32;
for _ in 0..n {
out.push(offset);
}
}
out
}
/// stss → set of 1-based sync sample numbers.
fn parse_stss(b: &[u8]) -> std::collections::HashSet<u32> {
let mut set = std::collections::HashSet::new();
if b.len() < 8 {
return set;
}
let count = be32(b, 4) as usize;
for i in 0..count {
let o = 8 + i * 4;
if o + 4 > b.len() {
break;
}
set.insert(be32(b, o));
}
set
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn stsc_offsets_one_sample_per_chunk() {
// Our writer's layout: 1 sample/chunk, co64 lists every offset.
let sizes = vec![100u32, 200, 300];
let chunks = vec![1000u64, 1100, 1300];
let stsc = vec![(1u32, 1u32)];
assert_eq!(
sample_offsets(&sizes, &chunks, &stsc),
vec![1000, 1100, 1300]
);
}
#[test]
fn stsc_offsets_multi_sample_chunks() {
// 2 samples in chunk 1, 1 in chunk 2.
let sizes = vec![10u32, 20, 30];
let chunks = vec![500u64, 900];
let stsc = vec![(1u32, 2u32), (2u32, 1u32)];
// chunk1@500: s0@500, s1@510; chunk2@900: s2@900.
assert_eq!(sample_offsets(&sizes, &chunks, &stsc), vec![500, 510, 900]);
}
#[test]
fn write_then_read_round_trip() {
// Mux a small A/V title to an in-memory MP4, then demux it back and
// check the streams, codec_private, and sample payloads survive.
use crate::disc::{
AudioChannels, AudioStream, Codec, DiscTitle, FrameRate, HdrFormat, LabelPurpose,
SampleRate, Stream as DiscStreamE, VideoStream,
};
use crate::mux::mp4::Mp4Sink;
use crate::pes::{PesFrame, Stream as _};
use std::io::Cursor;
let mut t = DiscTitle::empty();
t.streams = vec![
DiscStreamE::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R1080p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Sdr,
color_space: crate::disc::ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
}),
DiscStreamE::Audio(AudioStream {
pid: 0x1100,
codec: Codec::Ac3,
channels: AudioChannels::Surround51,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
}),
];
let hvcc = vec![0xAA, 0xBB, 0xCC, 0xDD, 0xEE];
t.codec_privates = vec![Some(hvcc.clone()), None];
let ac3 = vec![
0x0B,
0x77,
0x00,
0x00,
0b00_010110,
0b01000_000,
0b111_00_00_1,
0x00,
0xFF,
0xFF,
];
let d = 41_708_333i64;
let vid0 = vec![0x11u8; 700];
let vid1 = vec![0x22u8; 350];
let mut buf = Vec::new();
{
let mk = |track, pts, key, data: Vec<u8>| PesFrame {
track,
pts,
keyframe: key,
data,
duration_ns: None,
source: None,
coding: None,
};
let mut sink = Mp4Sink::create(Cursor::new(&mut buf), &t).unwrap();
sink.write(&mk(0, 0, true, vid0.clone())).unwrap();
sink.write(&mk(1, 0, true, ac3.clone())).unwrap();
sink.write(&mk(0, d, false, vid1.clone())).unwrap();
sink.write(&mk(1, 32_000_000, true, ac3.clone())).unwrap();
sink.finish().unwrap();
}
let mut rd = Mp4Reader::from_reader(Cursor::new(buf), "rt".into()).unwrap();
// Two streams: HEVC video + AC-3 audio, and the hvcC round-trips.
assert_eq!(rd.info().streams.len(), 2);
assert!(
matches!(rd.info().streams[0], DiscStreamE::Video(ref v) if v.codec == Codec::Hevc)
);
assert!(matches!(rd.info().streams[1], DiscStreamE::Audio(ref a) if a.codec == Codec::Ac3));
assert_eq!(rd.codec_private(0), Some(hvcc));
// Read all frames back; match them to what we wrote by (track, size).
let mut got = Vec::new();
while let Some(f) = rd.read().unwrap() {
got.push((f.track, f.data.len(), f.keyframe));
}
assert_eq!(got.len(), 4, "4 samples round-trip");
let vids: Vec<_> = got.iter().filter(|(t, _, _)| *t == 0).collect();
let auds: Vec<_> = got.iter().filter(|(t, _, _)| *t == 1).collect();
assert_eq!(vids.len(), 2);
assert_eq!(auds.len(), 2);
assert_eq!(vids[0].1, 700, "first video sample size");
assert_eq!(vids[1].1, 350, "second video sample size");
assert!(vids[0].2, "first video frame is a keyframe");
assert_eq!(auds[0].1, ac3.len());
}
#[test]
fn stts_and_ctts_expand() {
// stts: 3 samples × 1001 ticks.
let mut stts = Vec::new();
stts.extend_from_slice(&[0, 0, 0, 0]); // version+flags
stts.extend_from_slice(&1u32.to_be_bytes()); // entry_count
stts.extend_from_slice(&3u32.to_be_bytes());
stts.extend_from_slice(&1001u32.to_be_bytes());
assert_eq!(parse_stts(&stts), vec![1001, 1001, 1001]);
}
}
+248 -74
View File
@@ -45,6 +45,10 @@ pub enum StreamUrl {
M2ts { path: PathBuf },
/// Matroska container file.
Mkv { path: PathBuf },
/// Progressive MP4 (ISO-BMFF) mux output (`mp4://`). Like `mkv://` but writes
/// a single self-contained `.mp4` (ftyp+mdat+moov). Compatibility export —
/// carries only MP4-mappable codecs; see `mux::mp4`.
Mp4 { path: PathBuf },
/// Network stream (host:port).
Network { addr: String },
/// Standard I/O (stdin/stdout).
@@ -62,10 +66,31 @@ pub enum StreamUrl {
/// chapters + delay metadata). Like `dir://` it targets a directory; the
/// CLI constructs the `DemuxSink` with full options before the mux loop.
Demux { dir: PathBuf },
/// Video-only per-track output directory (`video://`) — a `demux://`
/// restricted to video tracks (native elementary streams: `.hevc`, `.h264`,
/// `.vc1`, `.m2v`, …). One file per video track; no audio/subtitles.
Video { dir: PathBuf },
/// Audio-only per-track output directory (`audio://`) — a `demux://`
/// restricted to audio tracks (native containers: `.thd`, `.dts`, `.ac3`,
/// `.eac3`, `.pcm`, …). One file per audio track; no video/subtitles.
Audio { dir: PathBuf },
/// Subtitle-only per-track output directory (`sub://`) — a `demux://`
/// restricted to subtitle tracks (PGS `.sup`, VobSub `.idx`+`.sub`, text
/// `.srt`). One file per subtitle track.
Sub { dir: PathBuf },
/// freemkv native per-picture video index (`fvi://`). A write-only PES sink
/// that emits one JSON-Lines record per coded picture of the title's primary
/// video track to a `.fvi` file (normative spec `docs/FVI_FORMAT.md`).
Fvi { path: PathBuf },
/// Chapter-marker export (`chapters://`). A write-only sink that ignores the
/// PES stream and writes the title's chapter points to a single file, format
/// chosen by the output extension: `.xml` (Matroska, default), `.txt` (OGM),
/// `.vtt` (WebVTT).
Chapters { path: PathBuf },
/// Structured title/stream/chapter metadata (`json://`). A write-only sink
/// that ignores the PES stream and writes the selected title's model as one
/// JSON document — machine-readable `info` for one title.
Json { path: PathBuf },
/// Unrecognized URL.
Unknown { raw: String },
}
@@ -77,13 +102,19 @@ impl StreamUrl {
StreamUrl::Disc { .. } => "disc",
StreamUrl::M2ts { .. } => "m2ts",
StreamUrl::Mkv { .. } => "mkv",
StreamUrl::Mp4 { .. } => "mp4",
StreamUrl::Network { .. } => "network",
StreamUrl::Stdio => "stdio",
StreamUrl::Iso { .. } => "iso",
StreamUrl::Dir { .. } => "dir",
StreamUrl::Null => "null",
StreamUrl::Demux { .. } => "demux",
StreamUrl::Video { .. } => "video",
StreamUrl::Audio { .. } => "audio",
StreamUrl::Sub { .. } => "sub",
StreamUrl::Fvi { .. } => "fvi",
StreamUrl::Chapters { .. } => "chapters",
StreamUrl::Json { .. } => "json",
StreamUrl::Unknown { .. } => "unknown",
}
}
@@ -95,10 +126,16 @@ impl StreamUrl {
StreamUrl::Disc { device: None } => "",
StreamUrl::M2ts { path }
| StreamUrl::Mkv { path }
| StreamUrl::Mp4 { path }
| StreamUrl::Iso { path }
| StreamUrl::Dir { path }
| StreamUrl::Demux { dir: path }
| StreamUrl::Fvi { path } => path.to_str().unwrap_or(""),
| StreamUrl::Video { dir: path }
| StreamUrl::Audio { dir: path }
| StreamUrl::Sub { dir: path }
| StreamUrl::Fvi { path }
| StreamUrl::Chapters { path }
| StreamUrl::Json { path } => path.to_str().unwrap_or(""),
StreamUrl::Network { addr } => addr,
StreamUrl::Stdio | StreamUrl::Null => "",
StreamUrl::Unknown { raw } => raw,
@@ -138,6 +175,11 @@ pub fn parse_url(url: &str) -> StreamUrl {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("mp4://") {
return StreamUrl::Mp4 {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("network://") {
return StreamUrl::Network {
addr: rest.to_string(),
@@ -171,6 +213,31 @@ pub fn parse_url(url: &str) -> StreamUrl {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("video://") {
return StreamUrl::Video {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("audio://") {
return StreamUrl::Audio {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("sub://") {
return StreamUrl::Sub {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("chapters://") {
return StreamUrl::Chapters {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("json://") {
return StreamUrl::Json {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("fvi://") {
return StreamUrl::Fvi {
path: PathBuf::from(rest),
@@ -344,9 +411,7 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
// unchanged (decrypt_keys_for_title short-circuits to decrypt_keys).
let (keys, title_is_clear) =
match crate::io::file_sector_source::FileSectorSource::open(path) {
Ok(mut crack_reader) => {
disc.decrypt_keys_for_title_checked(idx, &mut crack_reader, 64)
}
Ok(mut crack_reader) => disc.decrypt_keys_for_title(idx, &mut crack_reader, 64),
Err(_) => (disc.decrypt_keys(), false),
};
// Per-title decrypt gate (parallel to the disc-wide gate above): on
@@ -448,8 +513,16 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
// PES source. Mirror `null://` → write-only.
StreamUrl::Dir { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Null => Err(crate::error::Error::StreamWriteOnly.into()),
// `mp4://` as a source: demux a progressive MP4 back into PES frames, so
// `mp4://` flows to every sink (mkv://, audio://, json://, …).
StreamUrl::Mp4 { ref path } => Ok(Box::new(super::mp4::Mp4Reader::open(path)?)),
// `demux://` is an output-only sink (per-track ES files); never a source.
StreamUrl::Demux { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Demux { .. }
| StreamUrl::Video { .. }
| StreamUrl::Audio { .. }
| StreamUrl::Sub { .. }
| StreamUrl::Chapters { .. }
| StreamUrl::Json { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
// `fvi://` is an output-only sink (per-picture video index); never a source.
StreamUrl::Fvi { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Unknown { ref raw } => {
@@ -479,7 +552,19 @@ pub fn output(
IO_BUF_SIZE,
crate::io::WritebackFile::create_with_size_hint(path, title.size_bytes)?,
));
Ok(Box::new(MkvStream::create_at(writer, title, Some(path))?))
Ok(Box::new(MkvStream::create(writer, title, Some(path))?))
}
StreamUrl::Mp4 { ref path } => {
validate_file_path(path, "mp4")?;
// Bounded-cache writeback (like mkv://) so a UHD-scale mux to slow /
// network-attached staging doesn't hit the dirty-page burst
// pathology; the mdat backpatch is an ordinary seek WritebackFile
// handles. BufWriter coalesces the many small moov box-header writes.
let writer = std::io::BufWriter::with_capacity(
IO_BUF_SIZE,
crate::io::WritebackFile::create_with_size_hint(path, title.size_bytes)?,
);
Ok(Box::new(super::mp4::Mp4Sink::create(writer, title)?))
}
StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?;
@@ -528,11 +613,35 @@ pub fn output(
dir, title, &opts,
)?))
}
// `video://`, `audio://`, and `sub://` are `demux://` restricted to one
// track class — video as native elementary streams, audio in native
// containers, or subtitles as `.sup`/`.idx+.sub`/`.srt`. No chapters
// sidecar (that's a `demux://` / `chapters://` concern).
StreamUrl::Video { ref dir }
| StreamUrl::Audio { ref dir }
| StreamUrl::Sub { ref dir } => {
let (scheme, kind) = match parsed {
StreamUrl::Video { .. } => ("video", super::demux_sink::TrackKind::Video),
StreamUrl::Audio { .. } => ("audio", super::demux_sink::TrackKind::Audio),
_ => ("sub", super::demux_sink::TrackKind::Subtitle),
};
validate_file_path(dir, scheme)?;
let mut opts = super::demux_sink::DemuxOptions {
kind_filter: Some(kind),
export_chapters: false,
..Default::default()
};
if !title.playlist.is_empty() {
opts.base = title.playlist.clone();
}
Ok(Box::new(super::demux_sink::DemuxSink::create(
dir, title, &opts,
)?))
}
// `fvi://` writes the per-picture video index (`docs/FVI_FORMAT.md`).
// The bare `output()` arm records the resolver path as the provenance
// `source.path` and defaults the title index to 0 (the resolver carries
// no title-index context — the CLI follow-up passes the real medium /
// title via `FviSink::create_with_source`).
// no title-index context).
StreamUrl::Fvi { ref path } => {
validate_file_path(path, "fvi")?;
Ok(Box::new(super::fvi_sink::FviSink::create(
@@ -542,6 +651,18 @@ pub fn output(
0,
)?))
}
// `chapters://` and `json://` write the title metadata at construction and
// ignore the PES stream (see `meta_sink`).
StreamUrl::Chapters { ref path } => {
validate_file_path(path, "chapters")?;
Ok(Box::new(super::meta_sink::ChaptersSink::create(
path, title,
)?))
}
StreamUrl::Json { ref path } => {
validate_file_path(path, "json")?;
Ok(Box::new(super::meta_sink::JsonSink::create(path, title)?))
}
StreamUrl::Unknown { ref raw } => {
Err(crate::error::Error::StreamUrlInvalid { url: raw.clone() }.into())
}
@@ -638,20 +759,11 @@ fn resolve_fmts_key_map(
title: &DiscTitle,
keys: &mut crate::decrypt::DecryptKeys,
fetch: Option<&crate::sector::KeyFetch>,
_format: ContentFormat,
format: ContentFormat,
) -> io::Result<Option<crate::decrypt::AacsKeyMap>> {
use crate::aacs::content::ALIGNED_UNIT_LEN;
use crate::aacs::content::{ALIGNED_UNIT_LEN, aacs_unit_encrypted, decrypt_unit, is_clean};
use crate::aacs::segment::{clip_byte_to_lba, parse_individual_segments};
// Off by default: while `BYPASS_FMTS_KEY` is set, forensic decode is disabled —
// no segment table read, no key-service traffic — and the caller's base-Unit-Key
// path applies (the forensic units garble and the demux drops them, the shipped
// behaviour). Flip `BYPASS_FMTS_KEY` to false to activate forensic decode once
// the index-key resolution is validated end to end.
if crate::aacs::segment::BYPASS_FMTS_KEY {
return Ok(None);
}
// Load the segment map; absent → not an FMTS disc.
let Ok(udf) = crate::udf::read_filesystem(reader) else {
return Ok(None);
@@ -684,56 +796,64 @@ fn resolve_fmts_key_map(
reader.read_sectors(lba, 3, &mut c, false).ok()?;
Some(c)
};
// ── ONE forensic query. The key service returns ALL forensic index keys for the
// disc in a single response, ORDERED by index (array element i = index i+1).
// So send one clean single-variant batch (a segment's even-phase units) and
// read the whole set back — no per-index probing, no phase measurement, no
// decrypt-and-check: the array position IS the index. The first readable
// segment whose batch yields the full set wins; a short (e.g. 1-key,
// base-UK-shaped) response means that batch wasn't forensic (a wrong
// feature-title mapping), so try the next segment.
// ── ANCHOR — fetch the whole 32-key set from ONE index-1 batch. The key
// service returns ALL forensic index keys ordered (element i = index i+1)
// only for a canonical INDEX-1 sample that decrypts under the index-1 key.
// A forensic segment interleaves TWO variants at the aligned-unit level, so
// index-1's real content is one PHASE (even or odd units) and the alternate
// is a different variant that won't decrypt. We don't know the phase a
// priori, so try PHASE A (even) then PHASE B (odd): whichever is index-1's
// content comes back with the full set. Both phases failing (across the
// read-fault fallback over index-1 segments) ⇒ this disc has no FMTS keys.
//
// ANCHOR RULE: the query MUST sample an INDEX-1 segment. The key service only
// returns the full set for the canonical anchor sample (a unit that decrypts
// under the index-1 key); a batch from any other forensic index is rejected
// (a base-UK-shaped miss). The `index == 1` filter guarantees every batch we
// send is an anchor — and the forensic tag cycles 1..32 in file order, so
// ~1-in-32 segments qualify (~25 across the feature), leaving ample read-fault
// fallback within the `MAX_ANCHOR_ATTEMPTS` budget. ─────────────────────────
const N_INDEX: usize = 32;
// Each forensic batch carries the server's minimum-samples count (the same
// disambiguation floor the online source enforces), drawn as even-phase units
// to land one clean variant half.
// The set's SIZE is whatever the source returns (≥ 1) — never assumed. 32
// is all we have seen, but a disc with a different forensic index count is
// not ruled out, so the map is sized to the returned `len()`, not a const.
// ─────────────────────────────────────────────────────────────────────────
// Batch size = the key service's minimum-samples floor (same as the online
// source), drawn from ONE phase to land a clean single-variant half.
const BATCH_UNITS: usize = crate::keysource::MIN_SAMPLE_UNITS;
// Read-fault fallback budget: how many INDEX-1 (anchor) segments to attempt
// before giving up. Only matters when the leading anchor segments are
// unreadable; each attempt is one server round-trip, so it is bounded.
// Read-fault fallback: how many index-1 segments to attempt if the leading one
// is unreadable. The 2 phase requests happen per readable segment.
const MAX_ANCHOR_ATTEMPTS: usize = 16;
// Even units = p*2; odd units = p*2 + 1.
let read_phase_batch = |reader: &mut dyn SectorSource,
seg: &crate::aacs::segment::Segment,
phase_off: usize|
-> Option<Vec<Vec<u8>>> {
let mut batch: Vec<Vec<u8>> = Vec::with_capacity(BATCH_UNITS);
for p in 0..BATCH_UNITS {
batch.push(read_unit(reader, seg, p * 2 + phase_off)?);
}
Some(batch)
};
let mut index_keys: Vec<[u8; 16]> = Vec::new();
for seg in segments
'anchor: for seg in segments
.iter()
.filter(|s| s.index == 1)
.take(MAX_ANCHOR_ATTEMPTS)
{
let mut batch: Vec<Vec<u8>> = Vec::new();
for p in 0..BATCH_UNITS {
if let Some(c) = read_unit(reader, seg, p * 2) {
batch.push(c);
for phase_off in [0usize, 1usize] {
let Some(batch) = read_phase_batch(reader, seg, phase_off) else {
continue; // read fault on this phase — try the other / next segment
};
let fresh = fetch.fmts_indexes(&batch);
// Any non-empty reply is the source's COMPLETE ordered forensic set;
// trust it and stop. An empty reply = this phase/segment did not anchor.
if !fresh.is_empty() {
index_keys = fresh;
break 'anchor;
}
}
if batch.len() < BATCH_UNITS {
continue; // read fault / short tail
}
let fresh = fetch(&batch);
if fresh.len() >= N_INDEX {
index_keys = fresh;
break;
}
}
tracing::info!(target: "freemkv::keysource", held = index_keys.len(), need = N_INDEX, "fmts: collection done");
// The full set is required. Anything short holes the rip — fail loud like a
// missing Unit Key rather than emit forensic-holed output.
if index_keys.len() < N_INDEX {
// The count is whatever the source returned — not a fixed 32. Sized here, used
// everywhere below.
let n_index = index_keys.len();
tracing::info!(target: "freemkv::keysource", held = n_index, "fmts: collection done");
// At least one forensic index key is required. None ⇒ no FMTS key for this
// disc from any source — fail loud like a missing Unit Key rather than emit
// forensic-holed output.
if index_keys.is_empty() {
return Err(crate::error::Error::FmtsKeyMissing.into());
}
@@ -742,7 +862,7 @@ fn resolve_fmts_key_map(
let base_idx = 0usize;
let mut tag_slot: std::collections::HashMap<u16, usize> = std::collections::HashMap::new();
if let crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } = keys {
for (i, k) in index_keys.iter().take(N_INDEX).enumerate() {
for (i, k) in index_keys.iter().enumerate() {
let tag = (i + 1) as u16;
let slot = match unit_keys.iter().position(|(_, h)| h == k) {
Some(s) => s,
@@ -758,21 +878,64 @@ fn resolve_fmts_key_map(
}
}
// ── Build the per-segment LBA ranges directly from the tag. Each segment is
// decoded from its TAG half: the map routes the segment's whole span to its
// tag's key; the tag key opens the tag half wherever it interleaves, and the
// un-served version-B half — decrypted with that (for it, wrong) key —
// garbles and the demux drops it, leaving one clean variant per span. No
// re-read and no phase needed here: byte-5 `seg.index` selects the key. A
// segment whose tag is somehow absent (cannot happen with all 32 held) or
// that straddles an extent boundary is left unmapped and tallied. ─────────
let mut ranges: Vec<(u32, u32, usize)> = Vec::with_capacity(segments.len());
// ── PROBE each index's phase. A forensic segment interleaves two variants at
// the aligned-unit level; only ONE parity is this index's real content (the
// other is the alternate variant — a different key, garbles under ours). For
// each index, read a representative tagged segment and count clean decrypts
// of its EVEN vs ODD units under that index's key: the clean half is the
// index's phase. This is the ONE place `is_clean` runs — "the map must be
// right", verified here, once — so the mux decrypt can then trust the map.
// Phase is per-index and shared by every segment carrying that index. ──────
let mut phase_of_index: std::collections::HashMap<u16, crate::decrypt::Phase> =
std::collections::HashMap::new();
for (i, k) in index_keys.iter().enumerate() {
let tag = (i + 1) as u16;
let Some(seg) = segments.iter().find(|s| s.index == tag) else {
continue; // no segment carries this index on this feature — skip
};
let (mut even, mut odd) = (0usize, 0usize);
for p in 0..BATCH_UNITS {
for (phase_off, counter) in [(0usize, &mut even), (1usize, &mut odd)] {
if let Some(mut c) = read_unit(reader, seg, p * 2 + phase_off) {
if aacs_unit_encrypted(&c, format) {
decrypt_unit(&mut c, k);
if is_clean(&c, format) {
*counter += 1;
}
}
}
}
}
let phase = match even.cmp(&odd) {
std::cmp::Ordering::Greater => crate::decrypt::Phase::Even,
std::cmp::Ordering::Less => crate::decrypt::Phase::Odd,
std::cmp::Ordering::Equal => {
// Neither half decrypts clean under this index's key: the map would
// be wrong. Fail loud rather than emit a broken segment map.
tracing::warn!(target: "freemkv::keysource", index = tag, even, odd, "fmts: no clean phase under index key — refusing broken map");
return Err(crate::error::Error::FmtsKeyMissing.into());
}
};
phase_of_index.insert(tag, phase);
}
// ── Build the per-segment LBA ranges: each forensic segment → its tag's key
// AND its index's phase. The mapped decrypt opens only that half and leaves
// the alternate as ciphertext (the muxer drops untouched ciphertext) —
// clean by construction, no garble. A segment straddling an extent boundary
// is left unmapped and tallied (a hard failure below). ────────────────────
let mut ranges: Vec<(u32, u32, usize, crate::decrypt::Phase)> =
Vec::with_capacity(segments.len());
let mut unresolved = 0usize;
for seg in &segments {
let Some(&slot) = tag_slot.get(&seg.index) else {
unresolved += 1;
continue;
};
let phase = phase_of_index
.get(&seg.index)
.copied()
.unwrap_or(crate::decrypt::Phase::All);
let start_byte = seg.start_spn as u64 * 192;
let end_byte = (seg.end_spn as u64 + 1) * 192;
let (Some(a), Some(b)) = (
@@ -785,20 +948,19 @@ fn resolve_fmts_key_map(
// Only emit a contiguous within-extent range (segments are ~480 KB; a rare
// extent-straddle is left unresolved rather than given a wrong span).
if b >= a && (b - a) as u64 == (end_byte - 1 - start_byte) / 2048 {
ranges.push((a, b + 1, slot));
ranges.push((a, b + 1, slot, phase));
} else {
unresolved += 1;
}
}
// Every forensic segment must map to an index key. Any that did not is a hole
// in the rip — with the full 32-key set in hand this should never happen, so
// treat it as a hard failure (a read fault or an unexpected on-disc layout)
// rather than silently emitting a segment the base Unit Key only garbles.
// treat it as a hard failure rather than silently emitting a garbled segment.
if unresolved != 0 {
return Err(crate::error::Error::FmtsKeyMissing.into());
}
Ok(Some(crate::decrypt::AacsKeyMap::from_ranges(
Ok(Some(crate::decrypt::AacsKeyMap::from_ranges_phased(
ranges, base_idx,
)))
}
@@ -826,7 +988,7 @@ pub fn resolve_mux_key_map(
if let Some(f) = fetch {
let samples = crate::keysource::read_encrypted_units(reader, title, 8);
if !samples.is_empty() {
let fresh = f(&samples);
let fresh = f.unit_keys(&samples);
if let crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } = keys {
for k in fresh {
if !unit_keys.iter().any(|(_, h)| *h == k) {
@@ -916,7 +1078,7 @@ pub fn resolve_mux_key_map(
if idx.is_none() {
if let Some(f) = fetch {
if !samples.is_empty() {
let fresh = f(&samples);
let fresh = f.unit_keys(&samples);
if let crate::decrypt::DecryptKeys::Aacs { unit_keys, .. } = keys {
for k in fresh {
if !unit_keys.iter().any(|(_, h)| *h == k) {
@@ -1000,6 +1162,18 @@ pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
)),
_ => None,
};
// The map IS the title's read plan: it says which CPS unit / forensic segment
// each LBA belongs to. Walk ONLY the units it marks as ours — every default /
// CPS unit, and inside an FMTS forensic segment only our-phase units. The
// alternate-phase units are a different device group's variant; a licensed
// player never reads them, and neither do we — they are never fetched,
// decrypted, or handed to the demux, so the demux sees one gapless our-variant
// stream (no ciphertext to trip a concealed-gap resync). A non-forensic map
// returns the extents unchanged, so the common disc reads exactly as before.
let extents = match &key_map {
Some(map) => map.read_plan(&extents, unit_align as u32),
None => extents,
};
let mut decrypting =
crate::sector::DecryptingSectorSource::new(Box::new(reader) as Box<dyn SectorSource>, keys);
if let Some(map) = key_map {
+100 -34
View File
@@ -28,14 +28,67 @@ use super::SectorSource;
/// empty if the source can't help. Mirrors the DVD model (try the held key,
/// then ask the key source for the failing data) generalised to AACS.
///
/// The library performs NO key lookup or network I/O itself; this closure is
/// the seam an application uses to call its key source (e.g. an online key
/// service) with the exact ciphertext that failed. A **stateless, shared**
/// `Arc<Fn>` — the decorator owns the only mutable state (its call-count cap and
/// spent flag), so one closure is built once and cloned cheaply into every read
/// path (sweep / patch / mux); no per-decorator factory is needed. `Send + Sync`
/// The library performs NO key lookup or network I/O itself; this is the seam an
/// application uses to call its key source (e.g. an online key service) with the
/// exact ciphertext that failed.
///
/// TWO explicit operations, so the "one base key vs a whole forensic set"
/// contract lives in the type instead of a caller guessing at the return length:
///
/// * [`unit_keys`](Self::unit_keys) — the base Unit Key(s) for a CPS unit, from
/// real encrypted samples drawn from it. The non-forensic path: one key per CPS
/// unit (the pool grows by whatever it returns). Used by the mux's base /
/// multi-CPS map resolution AND by the sweep/patch recovery decorator.
/// * [`fmts_indexes`](Self::fmts_indexes) — the disc's AACS 2.1 forensic index
/// keys, from an index-1 single-phase anchor batch. The source hands back the
/// COMPLETE set (ordered index 1..N); the caller sizes the forensic map to
/// `len()` and never assumes a fixed N (32 is all we've seen, but the contract
/// is "whatever the source returns, ≥ 1, is all of them").
///
/// A **stateless, shared** pair of `Arc<Fn>` — the decorator owns the only
/// mutable state (its call-count cap and spent flag), so one `KeyFetch` is built
/// once and cloned cheaply (two `Arc` bumps) into every read path. `Send + Sync`
/// so it can ride the mux highway's producer thread.
pub type KeyFetch = std::sync::Arc<dyn Fn(&[Vec<u8>]) -> Vec<[u8; 16]> + Send + Sync>;
/// A closure resolving keys from encrypted-content samples — the shape of both
/// [`KeyFetch`] operations. Named so the two constructors (and the struct fields)
/// read clearly.
pub type KeyFetchFn = std::sync::Arc<dyn Fn(&[Vec<u8>]) -> Vec<[u8; 16]> + Send + Sync>;
#[derive(Clone)]
pub struct KeyFetch {
unit: KeyFetchFn,
fmts: KeyFetchFn,
}
impl KeyFetch {
/// Build a resolver from its two operations: `unit` resolves base Unit Keys
/// from a CPS unit's samples; `fmts` resolves the forensic index set from an
/// index-1 anchor batch.
pub fn new(unit: KeyFetchFn, fmts: KeyFetchFn) -> Self {
Self { unit, fmts }
}
/// A resolver that serves ONLY base Unit Keys; [`fmts_indexes`](Self::fmts_indexes)
/// is always empty. For read paths that never resolve forensic keys — the
/// sweep/patch recovery decorator, which handles CPS units only.
pub fn unit_only(unit: KeyFetchFn) -> Self {
Self::new(unit, std::sync::Arc::new(|_| Vec::new()))
}
/// Resolve the base Unit Key(s) for a CPS unit from `samples` (real encrypted
/// units drawn from it). Normally one key; the caller adds whatever it returns
/// to the pool.
pub fn unit_keys(&self, samples: &[Vec<u8>]) -> Vec<[u8; 16]> {
(self.unit)(samples)
}
/// Resolve the disc's AACS 2.1 forensic index keys from an index-1 single-
/// phase `anchor` batch. The source returns the COMPLETE ordered set (index i
/// = element i); the caller trusts any non-empty result as all of them.
pub fn fmts_indexes(&self, anchor: &[Vec<u8>]) -> Vec<[u8; 16]> {
(self.fmts)(anchor)
}
}
/// Decorator: read from `inner`, then run the configured
/// AACS / CSS decrypt over the bytes that landed in `buf`.
@@ -122,6 +175,14 @@ impl<S: SectorSource> DecryptingSectorSource<S> {
self
}
/// `&mut` counterpart of [`with_key_map`](Self::with_key_map): install the
/// proactive map on an already-constructed source (the inline live-drive
/// [`DiscStream`](crate::mux::DiscStream) builds the decorator first, then
/// installs the map via its own `with_key_map`).
pub fn set_key_map(&mut self, map: Arc<crate::decrypt::AacsKeyMap>) {
self.key_map = Some(map);
}
/// Restrict decrypt to the disc's encrypted-content extents
/// (sorted/merged `(start_lba, sector_count)` — see
/// [`Disc::encrypted_content_ranges`](crate::Disc::encrypted_content_ranges)).
@@ -1167,10 +1228,11 @@ mod tests {
// Capture what the callback was handed, and how many times it fired.
let seen: Arc<Mutex<Vec<Vec<u8>>>> = Arc::new(Mutex::new(Vec::new()));
let seen_cb = Arc::clone(&seen);
let fetch: super::KeyFetch = std::sync::Arc::new(move |samples: &[Vec<u8>]| {
seen_cb.lock().unwrap().extend_from_slice(samples);
vec![real_key]
});
let fetch: super::KeyFetch =
super::KeyFetch::unit_only(std::sync::Arc::new(move |samples: &[Vec<u8>]| {
seen_cb.lock().unwrap().extend_from_slice(samples);
vec![real_key]
}));
let mut wrapped = DecryptingSectorSource::new(
EncUnitSource { unit: unit.clone() },
@@ -1251,10 +1313,11 @@ mod tests {
let seen: Arc<Mutex<Vec<Vec<u8>>>> = Arc::new(Mutex::new(Vec::new()));
let seen_cb = Arc::clone(&seen);
let fetch: super::KeyFetch = std::sync::Arc::new(move |samples: &[Vec<u8>]| {
seen_cb.lock().unwrap().extend_from_slice(samples);
Vec::new() // service has nothing for the orphan — forces the sampling path
});
let fetch: super::KeyFetch =
super::KeyFetch::unit_only(std::sync::Arc::new(move |samples: &[Vec<u8>]| {
seen_cb.lock().unwrap().extend_from_slice(samples);
Vec::new() // service has nothing for the orphan — forces the sampling path
}));
let ranges: Arc<[(u32, u32)]> = Arc::from(vec![(0u32, 6u32)]);
let mut dec = DecryptingSectorSource::new(
@@ -1337,14 +1400,15 @@ mod tests {
let unit_b_cb = unit_b.clone();
let calls = Arc::new(Mutex::new(0usize));
let calls_cb = Arc::clone(&calls);
let fetch: super::KeyFetch = std::sync::Arc::new(move |samples: &[Vec<u8>]| {
*calls_cb.lock().unwrap() += 1;
if samples.iter().any(|s| *s == unit_b_cb) {
vec![key_b]
} else {
vec![]
}
});
let fetch: super::KeyFetch =
super::KeyFetch::unit_only(std::sync::Arc::new(move |samples: &[Vec<u8>]| {
*calls_cb.lock().unwrap() += 1;
if samples.iter().any(|s| *s == unit_b_cb) {
vec![key_b]
} else {
vec![]
}
}));
let mut wrapped = DecryptingSectorSource::new(
AltSource {
@@ -1478,13 +1542,14 @@ mod tests {
let calls = Arc::new(Mutex::new(0usize));
let calls_cb = Arc::clone(&calls);
let fetch: super::KeyFetch = std::sync::Arc::new(move |samples: &[Vec<u8>]| {
*calls_cb.lock().unwrap() += 1;
// The closure is handed the still-scrambled on-disc ciphertext.
assert!(!samples.is_empty(), "fetch receives the failing units");
assert_eq!(samples[0].len(), crate::aacs::content::ALIGNED_UNIT_LEN);
vec![real_key]
});
let fetch: super::KeyFetch =
super::KeyFetch::unit_only(std::sync::Arc::new(move |samples: &[Vec<u8>]| {
*calls_cb.lock().unwrap() += 1;
// The closure is handed the still-scrambled on-disc ciphertext.
assert!(!samples.is_empty(), "fetch receives the failing units");
assert_eq!(samples[0].len(), crate::aacs::content::ALIGNED_UNIT_LEN);
vec![real_key]
}));
let ranges: Arc<[(u32, u32)]> = Arc::from(vec![(0u32, 6u32)]); // LBA 0..6 content
let mut dec = DecryptingSectorSource::new(
@@ -1563,10 +1628,11 @@ mod tests {
let unit = encrypt_aacs_unit(&real_key);
let calls = Arc::new(Mutex::new(0usize));
let calls_cb = Arc::clone(&calls);
let fetch: super::KeyFetch = std::sync::Arc::new(move |_: &[Vec<u8>]| {
*calls_cb.lock().unwrap() += 1;
vec![real_key]
});
let fetch: super::KeyFetch =
super::KeyFetch::unit_only(std::sync::Arc::new(move |_: &[Vec<u8>]| {
*calls_cb.lock().unwrap() += 1;
vec![real_key]
}));
// Content lives far away; LBA 0 is "filesystem".
let ranges: Arc<[(u32, u32)]> = Arc::from(vec![(1002u32, 99u32)]);
let mut dec = DecryptingSectorSource::new(
+1 -1
View File
@@ -189,7 +189,7 @@ pub trait SectorSink: Send {
}
pub use crate::io::file_sector_source::FileSectorSource;
pub use decrypting::{DecryptingSectorSource, KeyFetch};
pub use decrypting::{DecryptingSectorSource, KeyFetch, KeyFetchFn};
pub use file::FileSectorSink;
pub use prefetched::PrefetchedSectorSource;
+7 -7
View File
@@ -170,7 +170,7 @@ fn aacs_fetch_step(
return prev_dropped;
}
*calls += 1;
let fresh = (fetch)(&samples);
let fresh = fetch.unit_keys(&samples);
// Add only keys we don't already hold (dedup by value).
let mut added = 0usize;
if let DecryptKeys::Aacs { unit_keys, .. } = keys {
@@ -276,11 +276,11 @@ mod tests {
// decorator's integration tests); here we pin the seam's key-plumbing.
let calls = Arc::new(AtomicUsize::new(0));
let c2 = Arc::clone(&calls);
let fetch: KeyFetch = Arc::new(move |samples: &[Vec<u8>]| {
let fetch: KeyFetch = KeyFetch::unit_only(Arc::new(move |samples: &[Vec<u8>]| {
c2.fetch_add(1, Ordering::SeqCst);
assert!(!samples.is_empty(), "failing ciphertext is forwarded");
vec![[0xAB; 16]]
});
}));
let mut r = key_fetch(fetch);
let mut buf = scrambled_unit(0x33);
let mut keys = DecryptKeys::Aacs {
@@ -304,10 +304,10 @@ mod tests {
// the SAME ciphertext must not call the fetch again.
let calls = Arc::new(AtomicUsize::new(0));
let c2 = Arc::clone(&calls);
let fetch: KeyFetch = Arc::new(move |_: &[Vec<u8>]| {
let fetch: KeyFetch = KeyFetch::unit_only(Arc::new(move |_: &[Vec<u8>]| {
c2.fetch_add(1, Ordering::SeqCst);
Vec::new() // never helps
});
}));
let mut r = key_fetch(fetch);
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![],
@@ -331,10 +331,10 @@ mod tests {
fn key_fetch_call_budget_bounds_fetches() {
let calls = Arc::new(AtomicUsize::new(0));
let c2 = Arc::clone(&calls);
let fetch: KeyFetch = Arc::new(move |_: &[Vec<u8>]| {
let fetch: KeyFetch = KeyFetch::unit_only(Arc::new(move |_: &[Vec<u8>]| {
c2.fetch_add(1, Ordering::SeqCst);
Vec::new()
});
}));
let mut r = key_fetch(fetch);
let mut keys = DecryptKeys::Aacs {
unit_keys: vec![],
+5 -5
View File
@@ -591,7 +591,7 @@ fn meta_all_stream_types() {
fn mkvstream_write_finish() {
let dt = sample_disc_title();
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let mut stream = MkvStream::create(writer, &dt).unwrap();
let mut stream = MkvStream::create(writer, &dt, None).unwrap();
// Write some fake PES frames (they won't produce valid MKV content
// since there is no real codec data, but it should not panic)
@@ -616,7 +616,7 @@ fn mkvstream_write_finish() {
fn mkvstream_meta_sets_title() {
let dt = sample_disc_title();
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let stream = MkvStream::create(writer, &dt).unwrap();
let stream = MkvStream::create(writer, &dt, None).unwrap();
let info = PesStream::info(&stream);
assert_eq!(info.playlist, "Test Movie");
@@ -656,7 +656,7 @@ fn mkvstream_roundtrip_bdts() {
};
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let mut stream = MkvStream::create(writer, &dt).unwrap();
let mut stream = MkvStream::create(writer, &dt, None).unwrap();
// Write PES frames targeting the audio track
for i in 0..10u8 {
@@ -745,7 +745,7 @@ fn mkvstream_meta_preserves_all_streams() {
};
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let stream = MkvStream::create(writer, &dt).unwrap();
let stream = MkvStream::create(writer, &dt, None).unwrap();
let info = PesStream::info(&stream);
assert_eq!(info.streams.len(), 5, "all 5 streams should be preserved");
@@ -856,7 +856,7 @@ fn mkvstream_e2e_h264_produces_valid_mkv() {
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> =
Box::new(SharedWriter(output2.clone()));
let mut stream2 = MkvStream::create(writer, &dt).unwrap();
let mut stream2 = MkvStream::create(writer, &dt, None).unwrap();
// Write the ES data (SPS+PPS+IDR) as a keyframe PES frame.
let frame1 = libfreemkv::pes::PesFrame {