Compare commits

..
40 Commits
Author SHA1 Message Date
Matthew Jackson b2ebae8f81 v1.5.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-22 22:19:39 -07:00
Matthew Jackson b2f35a8afc CSS DVD: resolve the per-title key at read time, drop the scan-time crack
Every DVD read path — the file-backed mux highway (build_iso_pipeline) and
the live-drive single-pass DiscStream — now resolves the per-VTS CSS title
key through one shared step, css::resolve_dvd_title_key, cracked keylessly in
playback order from the title's own extents. Removes the earlier design that
reused a single scan-time key (meaningless for a per-VTS scheme) and muxed a
detection-miss disc's scrambled sectors as garbage.

- Disc::scan no longer cracks a key up front; it does only the CSS bus-auth
  read-unlock, hoisted before the UDF prefetch so scrambled small/menu VOBs
  no longer cost a rejected read each (CSS-DVD scan ~25s -> ~6s).
- An uncrackable title hard-fails (E7023) instead of passing ciphertext as
  plaintext; --raw skips the crack entirely; a Stop mid-crack surfaces Halted.
- DiscStream::new is now fallible and threads raw + halt.
- Fix a stale codec-parser doc claim (TrueHD/FLAC/MP2/AAC do gate via DropTally).
2026-07-22 22:15:27 -07:00
Matthew Jackson a6a4a5d7d2 libfreemkv: 1.5.2 changelog reflects the shipped playback-order crack
The prior wording described the round-1 'self-cracking per-sector' approach
that the audit replaced; rewrite it to match what ships (per-title crack in
playback order + scan-key reuse + loud hard-fail on an uncrackable title).
2026-07-22 13:48:56 -07:00
Matthew Jackson 0de6b82a37 libfreemkv: restore freemkv-unlock path dep on the branch tip
The last release left this pinned to the git tag v1.5.1; release.sh expects
a committed path dep on the branch tip (it swaps path -> git tag only in the
tagged commit, matching bdemu). Restores that state so a local full-workspace
build links the sibling freemkv-unlock, and release.sh's path->tag swap works.
2026-07-22 13:43:32 -07:00
Matthew Jackson b9b0a87328 CSS DVD mux: per-title key via scan-key reuse + playback-order crack
A CSS DVD whose main title was mis-detected as unencrypted (the up-front
crack scanned the largest cell first and starved its budget in that cell's
clear prefix) muxed scrambled sectors as plaintext at exit 0. CSS leaves
the pack/PES header clear, so an un-descrambled sector muxes as a
structurally-valid but corrupt PES packet with zero loss reported.

decrypt_keys_for_title resolves a DVD title's CSS key two ways:
- Fast path: reuse the scan's cracked key when its crack_span covers this
  title's VTS (no re-read; on a live drive no second bus-auth).
- Crack: on a detection miss or a different VTS, crack from the title's OWN
  extents in a SINGLE scan in natural PLAYBACK ORDER (never largest-first).
  One scan = one CSS-locked early-bail, so a locked title is not re-hammered
  per cell against a live drive (hard rule #2); the 50k-sector budget is the
  same accepted bound the disc-wide scan uses. Cracked -> key; Unencrypted
  -> clear; ScrambledUncracked -> hard-fail.
ensure_title_decryptable hard-fails an uncrackable DVD title even when
detection missed, and passes a title that resolved its OWN valid key
regardless of the disc-wide css_error. descramble_region is unchanged from
v1.5.1 (validated-key seed).

Also rename the unlocker report's DVD entry CSS -> DVD. Bump 1.5.2.
2026-07-22 13:07:26 -07:00
Matthew Jackson d6cef9f665 v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:04:24 -07:00
Matthew Jackson b9b9185633 restore freemkv-unlock path dep for local dev (post-v1.5.1) 2026-07-20 17:02:14 -07:00
Matthew Jackson 66951a32ed v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:02:11 -07:00
Matthew Jackson 505a78efd1 mux/codec/truehd: fix MLP major-sync checksum endianness (was dropping the whole TrueHD track)
Regression since the previous release, which added an MLP major-sync checksum
gate to drop genuinely-undecodable audio frames. The checksum itself was computed
with mismatched byte order: `crc16_mlp` is the correct crc_2D table (poly 0x2D,
MSB-first) but returns its two bytes in the OPPOSITE order to libavutil's
`av_crc`, and `mlp_major_sync_crc_ok` then folded in the pre-trailer word
little-endian while comparing the trailer big-endian. The net result never
matched a real major sync, so EVERY major sync was judged corrupt. That armed the
drop-forward on the first AU and, since no major sync ever validated to clear it,
collateral-dropped every following AU forever — the entire TrueHD track was
silently dropped. Its AUs then flushed only at mux end, so the track's blocks
landed physically after all the video: a decoder reading video+TrueHD had to
buffer the whole title to reach the first audio block and spiralled into an
unbounded memory runaway ("decoder ran out of memory"). Every TrueHD title
produced after the gate landed was affected; a title from the release before it
is clean. (The header-size parse — a frequent suspect for extended 7.1/Atmos
headers — is NOT the bug; it already matches ffmpeg's `mlp_get_major_sync_size`
byte-for-byte.)

Fix: compute the checksum exactly as ffmpeg's `ff_mlp_checksum16` —
`crc16_mlp(body).swap_bytes() ^ AV_RL16(word) == AV_RL16(trailer)`. Cross-verified
byte-exact against two real discs (a 7.1/Atmos title and a 5.1 title, independent
32-byte headers both validate). With the checksum correct, major syncs validate
and the drop-forward corruption protection works as intended.

Defence in depth: a major-sync checksum that STILL can't be validated (a genuinely
corrupt or as-yet-unparsed header) no longer arms the drop-forward until we hold a
validated baseline (`num_substreams` from a prior clean major sync) — so a single
bad header can never again silently drop an entire track.

Tests: the `finalize_major_sync` fixture now builds the checksum the corrected way;
a synthetic checksum-failed head major sync is kept, not dropped; the existing
baseline-then-corrupt drop-forward tests still pass. Verified end to end against a
real disc: the TrueHD track demuxes to a full, cleanly-decodable 48 kHz 8-channel
stream, interleaved with the video, instead of 0 bytes.
2026-07-20 16:58:06 -07:00
Matthew Jackson 9b80084220 aacs: parse HD DVD VTKF title keys at the spec's 36-byte stride
The HD DVD Title Key File (VTKF*.AACS) stores 64 title-key entries of 36 bytes
each — 1-byte BIFO + 3 reserved + 16-byte encrypted key + 16-byte binding MAC —
per AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, confirmed byte-exact
against real discs (Freedom VTKF090, Dukes VTKF000: every 36-byte slot has
BIFO=0x80, a clean key, and a 0xFF binding MAC).

The parser used a 32-byte stride (a 12-byte pad in place of the 16-byte binding
MAC) with flag-based termination. That aligns entry #1 (key at offset 132, where
both strides agree) but drifts +4 bytes per entry after it and never terminates
(the previous entry's 0xFF MAC reads as a set present-flag), so it recovered a
correct key only for single-CPS-unit discs and garbage for CPS unit >=2. Every
multi-title HD DVD (Freedom, Harry Potter) was affected.

Fix: 36-byte stride, iterate the fixed 64 slots, take slots whose BIFO AV_FLG
(bit 7) is set, key at offset 4, slot index = CPS unit (skip empty slots rather
than terminate so a gap can't renumber later keys), and never read the trailing
16-byte TKF MAC as a key. Tests rebuilt on the real layout, including a full
64-entry file.

Also correct the VTKF-selection TODO in mod.rs: the AACS HD DVD Book gives the
selector explicitly (match the TKF's PLAYLIST_NAME field to the active
playlist), not the "validate against an encrypted unit" placeholder.

Reconciled against the new HD DVD reference (freemkv.org/docs/hddvd/); the spec
source is archived in private-repo/spec/.
2026-07-20 11:57:13 -07:00
Matthew Jackson b246169e5b aacs: discover HD DVD AACS dir + title-key files instead of hardcoding /ANY!/VTKF000
The HD DVD AACS directory name and title-key filename are chosen by the
authoring house, but the resolver hardcoded a single spelling
(/ANY!/VTKF000.AACS, /ANY!/MKBROM.AACS, /ANY!/CONTENT_CERT.AACS). Real discs
diverge: Freedom (Memory-Tech) names its AACS dir AAC! and ships VTKF090.AACS
+ VTKF100.AACS; Harry Potter carries VTKF000/001/002/099. On such a disc the
hardcoded path finds nothing, so no MKB/title-key/cert is read and decryption
silently can't engage.

Replace the fixed HD DVD path constants with structural discovery:
- find_hddvd_aacs_dir() locates the AACS dir as the root child dir ending in
  '!' that contains MKBROM.AACS (so the ..._BAK mirror is skipped; the dozens
  of decoy advanced-content '!' dirs are excluded by the MKBROM.AACS guard).
- role_paths(udf, role) builds the ordered candidate list per role: the static
  BD/UHD /AACS/ paths first, then the discovered HD DVD files — MKBROM.AACS,
  CONTENT_CERT.AACS, and every VTKF*.AACS (sorted), not just VTKF000.
- read_first() is now generic over &str / String so it takes the Vec<String>.

BD/UHD unaffected (no '!' dir → discovery returns None, list is the /AACS/
constants exactly as before). Verified on real Freedom (AAC!/VTKF090+100) and
Dukes (ANY!/VTKF000) ISOs; unit tests cover both shapes.

Open item (TODO(hddvd-encrypted)): when a disc has multiple VTKF variants the
correct one must be chosen by validating its VUK-derived key against a real
encrypted unit rather than first-that-reads. Blocked on obtaining a genuinely
encrypted HD DVD image — all HD DVD ISOs on hand are already-decrypted rips.
2026-07-20 11:29:28 -07:00
Matthew Jackson 2d96a3e87d release: tee up 1.5.1 (marginal-read reporting fix) 2026-07-19 20:47:52 -07:00
Matthew Jackson bae5a9ea07 drive: report marginal reads (PER=1) so a dirty disc can't pass clean
A read was only ever judged "good" by the drive's SCSI GOOD status — with no
integrity check on the bytes. On dirty/marginal media a drive can silently
return best-effort ECC data (occasionally mis-corrected) with GOOD status, so a
rip could "pass clean" yet decode with errors, pushing corruption downstream to
the mux instead of catching it at the read.

Enable recovered-error REPORTING at drive-prep: MODE SELECT the Read-Write
Error Recovery page with PER=1 (TB on / DTE off so the data still comes back),
preserving the drive's own retry count. Marginal reads now surface as
CHECK CONDITION / RECOVERED ERROR. Best-effort — a drive that doesn't honor it
keeps its defaults (no regression), and on a clean disc nothing changes.

A recovered error is distrusted: it marks just that ECC block NonTrimmed for a
Pass N re-read (a clean re-read wins; a persistent marginal becomes an honest
concealed gap), and is counted in the pass summary. Crucially it takes a
per-block SkipBlock, NOT the damage-jump path — a single marginal sector must
not trigger a 64 MB skip that would discard good data on a lightly-smudged disc.
2026-07-19 20:30:21 -07:00
Matthew Jackson 86ed27508d changelog: 1.5.0 release date + trim to app-level detail 2026-07-19 17:34:21 -07:00
Matthew Jackson f28d79b51b restore freemkv-unlock path dep for local dev (post-v1.5.0) 2026-07-19 15:37:27 -07:00
Matthew Jackson 5b27b9f8a5 v1.5.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-19 15:37:25 -07:00
Matthew Jackson 405e9d32c3 changelog: drop-on-undecodable audio + forced PGS detection (1.5.0) 2026-07-19 14:29:57 -07:00
Matthew Jackson 21b362c6e8 disc: detect forced PGS subtitles from stream content for info
Give `info` the same forced-subtitle verdict the muxer derives during a
rip, so the two agree. A shared classifier (mux::codec::pgs::ForcedTracker)
folds a PGS track's display sets — forced iff every one carries the
forced_on_flag — and is used by BOTH the MKV writer and a new scan-time
probe that reads the title's PGS streams (reusing the TS demuxer and PGS
parser). The probe only overrides a track it actually observed content
for, so an undecrypted/unread stream keeps its vendor-derived flag. Gated
behind ScanOptions::probe_forced_subtitles (off for the rip path, which
detects forced while muxing without a second read).
2026-07-19 14:23:04 -07:00
Matthew Jackson 9f8df95513 mux/codec/mpegaudio: keep free-format frames
Free-format MPEG-audio (bitrate_index 0) is a legal, decodable mode — the
decoder derives the frame size from the sync spacing. Dropping it was a
false positive on a clean stream, so it now passes the gate.
2026-07-19 14:23:04 -07:00
Matthew Jackson 2f465f4cfe mux/codec/truehd: fix drop-forward poison + rate/resync validation
Three TrueHD state-machine fixes from an adversarial audit:

- A corrupt access unit drops forward to the next major sync, but only
  the individually-verified corruption now feeds the whole-track poison
  verdict; the resync run is collateral. A couple of transient errors can
  no longer poison and discard an otherwise-good multi-hour track. The
  shared drop tally gains a verified/collateral split for this.
- The per-AU PTS rate is refined only from a CRC-validated major sync, so
  a corrupt major sync whose rate nibble decodes to another rate family
  can no longer shift the resumed audio — a drop stays a silence gap.
- The resync clears only on a CRC-validated major sync, never on a runt
  too short to hold and validate its header.
2026-07-19 14:23:04 -07:00
Matthew Jackson 3fdd463794 release: 1.5.0 2026-07-19 13:42:48 -07:00
Matthew Jackson 6a7392ac8c mux: detect forced PGS subtitles from the stream
Flag a PGS subtitle track FlagForced when it displays subtitles and every
one carries the HDMV forced_on_flag (a dedicated forced/narrative track),
independent of the disc's vendor label metadata. The track header
reserves a FlagForced byte up front and it is promoted at finish() from
the accumulated display-set state. Only ever promotes — a track already
forced from the playlist metadata is never demoted.
2026-07-19 13:42:48 -07:00
Matthew Jackson 28bdecd26b mux/codec: drop undecodable audio frames, keep A/V sync
A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch. Sync is preserved — a drop becomes a silence
gap, never a shift — and every drop is logged. Detection is per-codec,
each mirroring the format's authoritative integrity check:

  DTS          core-header validity gates
  AC-3/E-AC-3  native frame CRC-16 + bitstream-id range
  FLAC         whole-frame CRC-16 residue
  MP2/MP3      header sanity + free-format reject
  AAC-ADTS     header sanity (raw AAC passes through untouched)
  TrueHD/MLP   major-sync CRC-16 + AU parity; corrupt AUs drop forward
               to the next major sync, since decode state carries
               across access units

LPCM and video are excluded by design (no in-frame integrity data;
inter-frame prediction). A shared DropTally handles counting, logging,
and a whole-track fallback for a mostly-undecodable track.
2026-07-19 13:42:48 -07:00
Matthew Jackson d29d3d71b9 dts: emit clean core alone when the extension boundary is garbage
DTS-HD MA access units are a lossy core frame followed by trailing
extension substreams up to the next core sync. On source-damaged discs
(observed on the Bourne UHDs) the bytes where the XLL extension belongs
are neither a core sync nor an extension sync -- pure garbage -- which
desyncs ffmpeg's XLL decoder and cascades into 'Read past end of XLL
band data' / 'DSYNC check failed' across the whole track.

next_core_boundary now distinguishes three boundary states via a new
ext_clean flag on NextCore::Found:
  - precise/recognized extension sync  -> ext_clean=true  (keep full AU)
  - garbage at the boundary byte       -> ext_clean=false (drop the ext)

When ext_clean is false we emit the DTS core alone (drop the smallest
junk piece, keep the frame and its PTS) and drain past the garbage to
the next core. Recognized-but-unsizeable extensions still ride the
heuristic scan and are kept intact, so lossless tracks are unaffected --
only genuinely corrupt extension bytes are dropped.

Bourne s1.dts: 1606 damaged frames / 691620 (0.232%), 93% isolated
single frames, worst run 3 in a row (~32ms lossy blip).
2026-07-19 09:27:28 -07:00
Matthew Jackson 999898a438 changelog: faststart default + DTS/DTS-HD audio (mp4://) 2026-07-18 22:35:37 -07:00
Matthew Jackson b4470ccd81 mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box
(sample rate, channel layout mask, core size, computed bitrate, LFE);
whole access units (core + DTS-HD extension substreams) pass through, so
an HD decoder finds the extension. dtsh when an extension sync follows the
core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR.

Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is
byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1)
and decodes clean (exit 0). Channel layout correct.
2026-07-18 22:18:35 -07:00
Matthew Jackson 668dd95237 mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free
box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample ×
est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes
mdat, sample offsets are fixed from the start — no rewrite, no offset
patch. finish() writes moov into the hole and pads the slack with a free
box; if the estimate is blown it falls back to moov-at-end. Streams over
HTTP without a pre-fetch. Reserve-math + box-order tests added.
2026-07-18 21:43:44 -07:00
Matthew Jackson 777d9a04a7 changelog: mp4:// input demuxer (1.5.0) 2026-07-18 21:22:41 -07:00
Matthew Jackson b717044ae5 mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel
info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/
composition timing, stss → sync), rebuild a DiscTitle, and emit samples
as PesFrames in global decode order. Video NALs are length-prefixed in
MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into
input() so mp4:// flows to every sink (mkv://, audio://, json://, …).
In-memory write→read round-trip test proves symmetry (streams, hvcC,
sample sizes survive). Progressive MP4 only; fragmented (moof) is future.
2026-07-18 21:10:15 -07:00
Matthew Jackson 379767071c mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps
the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow /
NFS staging avoids the dirty-page burst. The mdat backpatch is an
ordinary seek WritebackFile already handles (seek_then_patch_roundtrip).
Tests switched to in-memory Cursor writers.
2026-07-18 20:57:44 -07:00
Matthew Jackson 6a06443dd8 changelog: mp4:// native MP4 muxer (1.5.0) 2026-07-18 20:44:18 -07:00
Matthew Jackson 42df399156 mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio
track. Audio sample entries built from the first frame's bitstream —
AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for
fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio
durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle
(mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3;
TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the
CLI can report exclusions — never a silent drop.

Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3
both frame-exact on every track (video 2650/4270, audio 5567/3454),
duration and colour identical, clean decode. On a pathological 4-clip
title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more
of them) — source-inherent, not a muxer defect.
2026-07-18 20:37:10 -07:00
Matthew Jackson 9ede4b9afd mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into
a 64-bit mdat and building the sample tables in memory, patched at
finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs
(already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts,
CFR-derived decode timeline (pipeline carries presentation PTS only), and
a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the
hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping.

Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and
HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact
(bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land
in M2.
2026-07-18 20:16:19 -07:00
Matthew Jackson 67e3e39499 mux: add video:// sink (video-only per-track elementary streams)
Completes the per-track-class trio with audio:// / sub://. video:// is
the demux path with a TrackKind::Video kind filter — each video track to
its own native elementary stream (.hevc/.h264/.vc1/.m2v/.obu), no audio
or subtitles. Reuses the existing kind_filter machinery and extension
map; scheme/parse/output/write-only-guard arms added symmetrically.
2026-07-18 19:24:59 -07:00
Matthew Jackson 42be09f628 changelog: 1.5.0 — extraction sinks + complete json:// model 2026-07-18 19:13:30 -07:00
Matthew Jackson 8bbbe43724 mux: json:// emits the complete title model (no dropped fields)
The json:// sink previously emitted only codec + language + pid per
stream. Fill it out to the full DiscTitle: video carries resolution
(+ pixel dims, interlaced), frame rate (+ fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channels (+ count),
sample rate (+ Hz), and editorial purpose; subtitles carry the
qualifier. Add the clip list and chapter names. This is everything the
scan resolved, so json:// is a lossless machine-readable view of a
title rather than a summary.
2026-07-18 18:14:11 -07:00
Matthew Jackson 95888f27a4 test: pin audio://sub://chapters://json:// in the scheme round-trip 2026-07-18 17:46:08 -07:00
Matthew Jackson 96dec65040 mux: chapters:// and json:// metadata sinks
Two write-only file sinks that ignore the PES stream and emit the title
metadata at construction (fvi-style, wired through output()):
- chapters:// — chapter markers in the format the extension picks: .xml
  (Matroska, default), .txt/.ogm (OGM simple), .vtt (WebVTT). Reuses the
  existing chapters_xml/chapters_ogm writers; adds a WebVTT writer.
- json:// — one title's model (playlist, duration, size, format, streams,
  chapters) as pretty JSON via serde_json.

New mux/meta_sink.rs; StreamUrl gains Chapters/Json; codec_label +
chapters_xml/ogm promoted to pub(crate) for reuse.

Tests: chapters_format_selected_by_extension, title_json_carries_streams_and_chapters.
2026-07-18 17:45:39 -07:00
Matthew Jackson 378fb226b7 mux: audio:// and sub:// sinks (demux filtered by track class)
Two extraction sinks built on the existing demux machinery: audio:// keeps
only audio tracks (native containers .thd/.dts/.ac3/.eac3/.pcm...), sub://
only subtitle tracks (PGS .sup, VobSub .idx+.sub, text .srt) — one file per
track, no video, no chapters sidecar. DemuxOptions gains a kind_filter; the
DemuxSink writers (already complete) are reused verbatim. StreamUrl gains
Audio/Sub variants; output() builds the filtered DemuxSink.

Test: kind_filter_keeps_only_the_selected_class.
2026-07-18 17:24:28 -07:00
Matthew Jackson 42caa2d90e restore freemkv-unlock path dep for local dev (post-v1.4.5) 2026-07-18 16:08:03 -07:00
34 changed files with 7537 additions and 491 deletions
+117
View File
@@ -1,5 +1,122 @@
# Changelog
## [1.5.2] — UNRELEASED
### Fixed
- CSS DVDs no longer mux to garbage. Every DVD read path — the file-backed mux
highway (`build_iso_pipeline`) and the live-drive single-pass `DiscStream` —
now resolves the per-VTS title key at read time through one shared step
(`resolve_dvd_title_key`), cracked keylessly in playback order from the title's
own extents. An uncrackable title hard-fails (E7023) instead of passing
scrambled sectors through as plaintext; `--raw` skips the crack entirely; a
user Stop mid-crack surfaces as `Halted`.
### Changed
- DVD scan no longer cracks a title key up front (the key is per-VTS, so a single
disc key was meaningless). Scan does only the CSS bus-auth read-unlock — hoisted
before the UDF prefetch so scrambled small/menu VOBs no longer cost a rejected
read each. Cuts a CSS-DVD scan from ~25s to ~6s.
- Unlocker report: the DVD entry is renamed `CSS` → `DVD`.
## [1.5.1] — 2026-07-20
### Fixed
- **TrueHD audio is no longer silently dropped (and no longer sends decoders out
of memory).** The previous release added an MLP major-sync checksum gate to drop
genuinely-undecodable audio frames, but the checksum was computed with mismatched
byte order — the 16-bit CRC result was folded in one endianness and compared in
the other — so it never validated a real major sync. The parser then judged every
major sync corrupt and dropped every audio frame from the first one onward,
flushing the whole TrueHD track at the end of the file: de-interleaved from the
video, which sent players and integrity checkers into an unbounded memory spiral
("decoder ran out of memory"). The checksum now matches the reference
implementation byte-exact (cross-verified against real 7.1/Atmos and 5.1 discs),
so major syncs validate and only genuinely-corrupt frames are dropped; as a
safety net, a major sync the parser still can't validate is kept rather than
allowed to drop an entire track. TrueHD titles produced after the checksum gate
landed need a re-rip.
- **HD DVD AACS key files are now found on every disc, not just the common
layout.** The AACS directory and title-key filename on HD DVD are chosen by
the authoring house, and freemkv previously assumed one fixed spelling
(`/ANY!/VTKF000.AACS`). Discs that name their AACS directory differently (e.g.
`AAC!` instead of `ANY!`) or ship numbered title-key files (`VTKF090.AACS` /
`VTKF100.AACS` rather than `VTKF000.AACS`) are now handled: the AACS directory
is located by its contents and every title-key file in it is picked up. Blu-ray
and UHD are unaffected.
- **HD DVD multi-title decryption reads the right keys.** The HD DVD title-key
file (`VTKF*.AACS`) stores its keys in 36-byte records — per the AACS HD DVD
specification, and confirmed byte-exact on real discs. freemkv had been reading
them at a 32-byte stride, which lands the first key correctly but drifts off
every key after it, so only single-title discs decrypted. Discs with more than
one protected title now recover every title's key instead of only the first.
(Choosing the correct title-key file when a disc carries several playlists
still needs verification against an encrypted HD DVD.)
- **A dirty disc can no longer "rip clean" but decode with errors.** freemkv now
asks the drive to *report* marginal reads instead of silently returning
best-effort data as success — on smudged/scratched media a drive can hand back
subtly-wrong bytes with a clean status, which used to slip through the rip and
surface only as playback/decode errors. A read the drive had to fight for is
now distrusted and re-read in the patch pass: a clean re-read wins, and a spot
that's genuinely unreadable becomes an honest gap rather than silently-wrong
data. Best-effort per drive, and it changes nothing on a clean disc.
## [1.5.0] — 2026-07-19
### Added
- **MP4 as a source (`mp4://`)** — read a progressive `.mp4` back in and send it to
any sink (`mp4:// mkv://`, `mp4:// audio://`, `mp4:// json://`, …). The round-trip
is frame-exact.
- **Native MP4 output (`mp4://`)** — a disc goes straight to a play-everywhere
`.mp4` in one decrypt pass, no ffmpeg. Carries HEVC / H.264 video (with HDR10) and
AC-3, E-AC-3, and DTS / DTS-HD audio, and is faststart by default so it plays over
HTTP without downloading the end first. It's a **compatibility export, not
archival**: MP4 can't hold TrueHD, LPCM, or bitmap (PGS / VobSub) subtitles, so
those are **excluded with a loud, itemized report — never a silent drop**
(`mkv://` stays the keep-everything path).
- **Five extraction sinks — dissect a title, don't just rip it.** New destinations
that pull one part of a title out on its own:
- **`video://dir/`** — each video track to its own native elementary-stream file.
- **`audio://dir/`** — each audio track to its own file in its native container
(`.thd`, `.dts`, `.ac3`, `.eac3`, `.aac`, `.flac`; LPCM as `.pcm`).
- **`sub://dir/`** — each subtitle track to its own file (PGS `.sup`, VobSub
`.idx` + `.sub`, text `.srt`).
- **`chapters://file`** — a title's chapter markers as a sidecar (`.xml` / `.txt`
/ `.ogm` / `.vtt`).
- **`json://file`** — a title's complete structure as JSON.
`chapters://` and `json://` read nothing of the elementary streams, so they
return in seconds.
- **Damaged audio frames are dropped instead of shipped as glitches.** When a
source disc has a corrupt audio frame, freemkv drops that frame rather than muxing
it as a decoder-choking glitch — keeping A/V in sync (a drop is a silence gap,
never a shift) and logging every drop. Works across DTS, AC-3 / E-AC-3, FLAC,
MP2 / MP3, AAC, and TrueHD, each using the format's own integrity check; a track
that is mostly undecodable is dropped whole. This catches structurally-broken
frames — corruption of the audio *data* inside an otherwise-valid frame is source
damage that can't be told from good data without decoding.
- **Forced subtitles detected from the stream.** A PGS subtitle track is flagged
forced when its content is entirely forced/narrative subtitles, read from the
stream itself rather than the disc's metadata — so it works on discs that carry
none. `info -v` reports the same, so `info` and a rip agree.
### Changed
- **`json://` emits the complete title model** — every field the scan resolved:
video resolution / frame rate / HDR / colour, audio channel layout / sample rate /
language / purpose, the subtitle forced flag, plus the clip list and chapter names.
### Fixed
- **TrueHD: a couple of transient errors no longer discard a whole track.** A
corrupt access unit is dropped forward to the next clean sync point, but a short
burst of damage no longer trips the whole-track drop, and a corrupt sync point can
no longer shift the audio that follows.
- **Free-format MP2 / MP3** is a legal, decodable mode and is no longer dropped.
## [1.4.5] — 2026-07-18
### Fixed
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "1.4.5"
version = "1.5.2"
edition = "2024"
rust-version = "1.86"
license = "MIT"
@@ -28,7 +28,7 @@ cbc = "0.1"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.4.5" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.5.2" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
+92 -49
View File
@@ -161,33 +161,52 @@ pub fn parse_unit_key_ro(data: &[u8], version: AacsVersion) -> Option<UnitKeyFil
})
}
/// HD DVD Video Title Key File (`VTKF000.AACS`) magic — "DVD HD Video TKF".
/// HD DVD Video Title Key File (`VTKF%%%.AACS`) magic — "DVD_HD_V_TKF".
pub const VTKF_MAGIC: &[u8; 12] = b"DVD_HD_V_TKF";
/// Fixed header length before the first title-key entry.
/// Fixed header length before the first Title Key Entry (AACS HD DVD Book,
/// Table 3-8).
const VTKF_HEADER_LEN: usize = 0x80;
/// Each title-key entry: BE32 flag + 16-byte encrypted key + 12-byte 0xFF pad.
const VTKF_ENTRY_LEN: usize = 0x20;
/// Title Key Entry stride (Table 3-8): 1-byte `BIFO` + 3 reserved + 16-byte
/// encrypted title key + 16-byte binding MAC = 36 bytes.
const VTKF_ENTRY_LEN: usize = 0x24;
/// Byte offset of the encrypted title key within an entry (after `BIFO` + 3
/// reserved).
const VTKF_KEY_OFF: usize = 4;
/// Number of Title Key Entry slots in a VTKF (Table 3-8): a fixed 64.
const VTKF_MAX_ENTRIES: usize = 64;
/// `BIFO` bit 7 (`AV_FLG`): set = this slot carries an available title key.
const VTKF_AV_FLG: u8 = 0x80;
/// Parse an HD DVD `VTKF000.AACS` into the SAME [`UnitKeyFile`] a BD/UHD
/// Parse an HD DVD `VTKF%%%.AACS` into the SAME [`UnitKeyFile`] a BD/UHD
/// `Unit_Key_RO.inf` yields — so the shared AACS crypto (`derive_unit_keys` →
/// `decrypt_unit_key(vuk, …)`) unwraps HD DVD title keys with no change. Only
/// the on-disc CONTAINER differs between BD and HD DVD; the title-key unwrap is
/// the identical AES-128 VUK step (`Kt = AES-128D(Kvu, Kte)`).
///
/// Layout (grounded in real discs — Shaun of the Dead, Anchorman, Harry Potter):
/// Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
/// 2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
/// Dukes of Hazzard `VTKF000`):
/// ```text
/// [0x00..0x0C] magic "DVD_HD_V_TKF"
/// [0x0C..0x10] BE32 total file length
/// [0x10..0x1C] associated playlist name ("VPLST000.XPL")
/// [0x1C..0x80] reserved (zero)
/// [0x80..] 32-byte entries: BE32 flag | 16-byte ENCRYPTED title key | 12-byte 0xFF pad
/// flag bit 31 (0x8000_0000) set = present; a cleared flag ends the table
/// [tail] 16-byte signature/MAC (never a key — the cleared-flag stop guards it)
/// [0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
/// [0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
/// [0x1C..0x80] reserved
/// [0x80..] 64 entries × 36 bytes:
/// BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
/// BIFO bit 7 (AV_FLG) set = this slot holds a title key
/// (pre-recorded discs fill the binding MAC with 0xFF)
/// [0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
/// ```
/// Entries number 1..=N as CPS units, matching `Unit_Key_RO`'s 1-based CPS
/// numbering, so a title's CPS unit indexes this list identically. The
/// title→CPS mapping itself is playlist-driven (`VPLST000.XPL`) and owned by the
/// HD DVD enumerator, so `title_cps_unit` is left empty here.
/// The slot index (1-based) is the CPS unit number, so an absent slot is
/// SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
/// hand the wrong title key to CPS unit N+1. The title→CPS mapping is
/// playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
/// `title_cps_unit` is left empty here.
///
/// The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
/// binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
/// after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
/// (Freedom, Harry Potter) yielded garbage keys for CPS unit ≥2.
pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
if data.len() < VTKF_HEADER_LEN || &data[..12] != VTKF_MAGIC {
return None;
@@ -198,20 +217,19 @@ pub fn parse_vtkf(data: &[u8]) -> Option<UnitKeyFile> {
let hash = disc_hash(data);
let mut encrypted_keys = Vec::new();
let mut pos = VTKF_HEADER_LEN;
let mut cps: u32 = 1;
while pos + VTKF_ENTRY_LEN <= data.len() {
let flag = u32::from_be_bytes([data[pos], data[pos + 1], data[pos + 2], data[pos + 3]]);
// A cleared present-bit terminates the key table. The file's trailing
// 16-byte signature then follows and must NOT be read as a key.
if flag & 0x8000_0000 == 0 {
for n in 0..VTKF_MAX_ENTRIES {
let pos = VTKF_HEADER_LEN + n * VTKF_ENTRY_LEN;
if pos + VTKF_ENTRY_LEN > data.len() {
break;
}
// AV_FLG clear = empty slot: skip it, but keep the slot index as the CPS
// number (do NOT break — a gap must not renumber the keys that follow).
if data[pos] & VTKF_AV_FLG == 0 {
continue;
}
let mut key = [0u8; 16];
key.copy_from_slice(&data[pos + 4..pos + 20]);
encrypted_keys.push((cps, key));
cps += 1;
pos += VTKF_ENTRY_LEN;
key.copy_from_slice(&data[pos + VTKF_KEY_OFF..pos + VTKF_KEY_OFF + 16]);
encrypted_keys.push((n as u32 + 1, key));
}
if encrypted_keys.is_empty() {
return None;
@@ -369,42 +387,51 @@ pub fn parse_content_cert(data: &[u8]) -> Option<ContentCert> {
mod vtkf_tests {
use super::*;
/// Build a synthetic `VTKF000.AACS` matching the real on-disc layout
/// (Shaun of the Dead / Anchorman): magic, BE32 size, playlist name,
/// reserved to 0x80, then 32-byte present-flagged entries, a cleared-flag
/// terminator, and a 16-byte trailer.
/// Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
/// HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
/// `VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
/// entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
/// set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
fn synth_vtkf(keys: &[[u8; 16]]) -> Vec<u8> {
const FILE_LEN: usize = 2480;
let mut v = Vec::new();
v.extend_from_slice(VTKF_MAGIC); // 0x00
v.extend_from_slice(&0u32.to_be_bytes()); // 0x0C size (patched below)
v.extend_from_slice(b"VPLST000.XPL"); // 0x10
v.resize(0x80, 0); // reserve to first entry
for k in keys {
v.extend_from_slice(&0x8000_0000u32.to_be_bytes()); // present flag
v.extend_from_slice(k); // 16-byte encrypted title key
v.extend_from_slice(&[0xFFu8; 12]); // 0xFF pad → 32-byte entry
v.extend_from_slice(&(FILE_LEN as u32).to_be_bytes()); // 0x0C HD_VTKF_SIZE
v.extend_from_slice(b"VPLST000.XPL"); // 0x10 playlist name
v.resize(VTKF_HEADER_LEN, 0); // reserve to first entry (0x80)
for n in 0..VTKF_MAX_ENTRIES {
if let Some(k) = keys.get(n) {
v.push(VTKF_AV_FLG); // BIFO: AV_FLG set (present)
v.extend_from_slice(&[0, 0, 0]); // reserved
v.extend_from_slice(k); // 16-byte encrypted title key
v.extend_from_slice(&[0xFFu8; 16]); // binding MAC (0xFF, pre-recorded)
} else {
v.extend_from_slice(&[0u8; VTKF_ENTRY_LEN]); // empty slot (AV_FLG clear)
}
}
// Cleared-flag terminator entry (must NOT be read as a key).
v.extend_from_slice(&[0u8; VTKF_ENTRY_LEN]);
// 16-byte trailing signature (must NOT be read as a key).
v.extend_from_slice(&[0xABu8; 16]);
let len = v.len() as u32;
v[0x0C..0x10].copy_from_slice(&len.to_be_bytes());
v.resize(FILE_LEN - 16, 0); // reserved gap before the trailer
v.extend_from_slice(&[0xABu8; 16]); // TKF MAC (must NOT be read as a key)
v
}
#[test]
fn parse_vtkf_extracts_present_entries_and_stops_at_terminator() {
fn parse_vtkf_reads_present_entries_skips_empty_ignores_mac() {
let k1 = [0x11u8; 16];
let k2 = [0x22u8; 16];
let k3 = [0x33u8; 16];
let data = synth_vtkf(&[k1, k2, k3]);
let ukf = parse_vtkf(&data).expect("valid VTKF must parse");
// Exactly the three present entries — the cleared-flag terminator and
// the 16-byte trailer are NOT mistaken for keys.
assert_eq!(ukf.encrypted_keys.len(), 3, "must stop at the cleared flag");
assert_eq!(ukf.encrypted_keys[0], (1, k1), "CPS units number 1..=N");
// Exactly the three present entries — the empty slots and the trailing
// 16-byte TKF MAC are NOT mistaken for keys. Critically, k2/k3 are read
// at the 36-byte stride (offsets 0xA4, 0xC8); the old 32-byte stride
// misread them from inside the previous entry's binding MAC.
assert_eq!(ukf.encrypted_keys.len(), 3);
assert_eq!(
ukf.encrypted_keys[0],
(1, k1),
"CPS units = 1-based slot index"
);
assert_eq!(ukf.encrypted_keys[1], (2, k2));
assert_eq!(ukf.encrypted_keys[2], (3, k3));
assert_eq!(ukf.version, AacsVersion::V10, "HD DVD is AACS 1.0");
@@ -412,6 +439,22 @@ mod vtkf_tests {
assert_eq!(ukf.disc_hash, disc_hash(&data));
}
#[test]
fn parse_vtkf_reads_a_full_64_entry_file() {
// Real discs (Freedom, Dukes) carry all 64 slots present. Every key must
// come back, none dropped and none drifted — the regression the 32-byte
// stride failed.
let keys: Vec<[u8; 16]> = (0..VTKF_MAX_ENTRIES).map(|n| [n as u8; 16]).collect();
let ukf = parse_vtkf(&synth_vtkf(&keys)).expect("64-entry VTKF");
assert_eq!(ukf.encrypted_keys.len(), 64);
assert_eq!(
ukf.encrypted_keys[63],
(64, [63u8; 16]),
"entry 64 at 0x{:x}",
VTKF_HEADER_LEN + 63 * VTKF_ENTRY_LEN
);
}
#[test]
fn parse_vtkf_rejects_non_magic() {
let mut data = synth_vtkf(&[[0x11u8; 16]]);
+225 -36
View File
@@ -40,17 +40,24 @@ pub mod trace;
pub mod types;
pub mod variant;
/// On-disc UDF paths to the AACS key-input files.
/// On-disc UDF paths to the AACS key-input files, plus HD DVD AACS-directory
/// discovery.
///
/// BD and UHD keep their key material under `/AACS/…`; HD DVD keeps the
/// equivalents under `/ANY!/…` with different names (`VTKF000.AACS` is the
/// title-key file — magic `DVD_HD_V_TKF`; `MKBROM.AACS` is the MKB). The
/// container difference is expressed here purely as DATA: each ROLE
/// ([`UNIT_KEY_RO_PATHS`], [`MKB_PATHS`], [`CONTENT_CERT_PATHS`]) is an ordered
/// candidate list, and every reader walks it with [`read_first`] taking the
/// first that reads. No reader ever branches on disc type — a BD/UHD disc has
/// the `/AACS/` files so those win; an HD DVD has neither, so it falls through
/// to the `/ANY!/` entry. Centralised so `resolve_vid_only`, `read_aacs_inputs`,
/// BD and UHD keep their key material under a fixed `/AACS/…` tree, so those
/// paths are constants. HD DVD keeps the equivalents in a reserved root
/// directory whose NAME is authoring-house-specific — observed `ANY!` (Dukes
/// of Hazzard) and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK`
/// mirror — and whose title-key file is NOT always `VTKF000.AACS` (Freedom
/// ships `VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED
/// from the parsed UDF tree ([`find_hddvd_aacs_dir`] + [`role_paths`]), never
/// hardcoded.
///
/// Each key ROLE ([`AacsRole`]) resolves to an ordered candidate list — the
/// BD/UHD constants first, then whatever the HD DVD directory actually holds —
/// which every reader walks with [`read_first`], first-that-reads. No reader
/// ever branches on disc type: a BD/UHD disc has the `/AACS/` files so those
/// win; an HD DVD has none of them, so it falls through to the discovered
/// entries. Centralised so `resolve_vid_only`, `read_aacs_inputs`,
/// `read_mkb_content`, and `read_aacs_version` can never silently diverge the
/// disc_hash / MKB / VID that another reader feeds a key service.
pub const PATH_UNIT_KEY_RO: &str = "/AACS/Unit_Key_RO.inf";
@@ -59,41 +66,109 @@ pub const PATH_MKB_RO: &str = "/AACS/MKB_RO.inf";
pub const PATH_MKB_RW: &str = "/AACS/MKB_RW.inf";
pub const PATH_CONTENT_CERT: &str = "/AACS/Content000.cer";
pub const PATH_CONTENT_CERT_ALT: &str = "/AACS/Content001.cer";
/// HD DVD title-key file (`/ANY!/`), forwarded as `inf_b64`; the key service
/// recognises it by its `DVD_HD_V_TKF` magic.
pub const PATH_VTKF_HDDVD: &str = "/ANY!/VTKF000.AACS";
/// HD DVD Media Key Block (`/ANY!/`), forwarded as `mkb_b64`.
pub const PATH_MKBROM_HDDVD: &str = "/ANY!/MKBROM.AACS";
/// HD DVD content certificate (`/ANY!/`); byte 0 gives the AACS major (0x00 → V10).
pub const PATH_CONTENT_CERT_HDDVD: &str = "/ANY!/CONTENT_CERT.AACS";
/// Title-key / `Unit_Key_RO.inf` role, in resolution order (BD/UHD, then HD DVD).
pub const UNIT_KEY_RO_PATHS: &[&str] = &[
PATH_UNIT_KEY_RO,
PATH_UNIT_KEY_RO_DUPLICATE,
PATH_VTKF_HDDVD,
];
/// MKB role, in resolution order (BD/UHD RO then RW, then HD DVD).
pub const MKB_PATHS: &[&str] = &[PATH_MKB_RO, PATH_MKB_RW, PATH_MKBROM_HDDVD];
/// Content-certificate role, in resolution order (BD/UHD, then HD DVD).
pub const CONTENT_CERT_PATHS: &[&str] = &[
PATH_CONTENT_CERT,
PATH_CONTENT_CERT_ALT,
PATH_CONTENT_CERT_HDDVD,
];
/// An AACS key-input role. [`role_paths`] maps it to an ordered candidate path
/// list (BD/UHD constants, then the discovered HD DVD files).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AacsRole {
/// Title-key file: BD/UHD `Unit_Key_RO.inf`, HD DVD `VTKF*.AACS`
/// (magic `DVD_HD_V_TKF`). The disc_hash is `SHA1` of this file.
UnitKey,
/// Media Key Block: BD/UHD `MKB_RO/RW.inf`, HD DVD `MKBROM.AACS`.
Mkb,
/// Content certificate: BD/UHD `Content000/001.cer`, HD DVD
/// `CONTENT_CERT.AACS` (byte 0 gives the AACS major).
ContentCert,
}
/// Walk an AACS role's candidate paths and return the first that reads.
/// The HD DVD AACS directory in a parsed UDF tree, if present.
///
/// Identified structurally, NOT by a hardcoded name: the root child directory
/// whose name ends in `!` (so the `<name>!_BAK` backup mirror, which also ends
/// in a non-`!` char, is not mistaken for it) and which contains `MKBROM.AACS`.
/// Observed real names: `ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD
/// disc has no such directory → `None`.
pub(crate) fn find_hddvd_aacs_dir(udf: &crate::udf::UdfFs) -> Option<&crate::udf::DirEntry> {
udf.root.entries.iter().find(|e| {
e.is_dir
&& e.name.ends_with('!')
&& e.entries
.iter()
.any(|c| !c.is_dir && c.name.eq_ignore_ascii_case("MKBROM.AACS"))
})
}
/// Ordered candidate paths for an AACS key [`AacsRole`]: the fixed BD/UHD
/// `/AACS/…` paths first, then the actual HD DVD files discovered in the disc's
/// AACS directory (see [`find_hddvd_aacs_dir`]). A disc has only one family, so
/// the other family's entries simply never read.
///
/// For [`AacsRole::UnitKey`] every `VTKF*.AACS` in the directory is appended in
/// sorted name order — a disc may carry more than one variant (Freedom:
/// `VTKF090` + `VTKF100`), not just `VTKF000`.
pub(crate) fn role_paths(udf: &crate::udf::UdfFs, role: AacsRole) -> Vec<String> {
let mut v: Vec<String> = match role {
AacsRole::UnitKey => vec![PATH_UNIT_KEY_RO, PATH_UNIT_KEY_RO_DUPLICATE],
AacsRole::Mkb => vec![PATH_MKB_RO, PATH_MKB_RW],
AacsRole::ContentCert => vec![PATH_CONTENT_CERT, PATH_CONTENT_CERT_ALT],
}
.into_iter()
.map(String::from)
.collect();
if let Some(dir) = find_hddvd_aacs_dir(udf) {
let d = &dir.name;
match role {
AacsRole::Mkb => v.push(format!("/{d}/MKBROM.AACS")),
AacsRole::ContentCert => v.push(format!("/{d}/CONTENT_CERT.AACS")),
AacsRole::UnitKey => {
// Glob VTKF*.AACS — the title-key filename is not fixed at
// VTKF000 (Freedom ships VTKF090 + VTKF100). Sorted for a
// deterministic try order.
//
// TODO(hddvd-playlist): each VTKF%%%.AACS is bound to ONE
// playlist (VPLST%%%.XPL) — the AACS HD DVD Book gives the
// selector explicitly: match the TKF's 12-byte PLAYLIST_NAME
// field (bytes 0x10..0x1C) to the playlist of the title being
// decrypted; "unless the names are identical, the Title Keys in
// this TKF must not be used." Today read_first just takes the
// first that reads, which is correct only for a single-playlist
// disc. Thread the active playlist name here (owned by the HD
// DVD enumerator) and pick the name-matched VTKF once a
// multi-playlist encrypted disc is available to validate against.
let mut names: Vec<&str> = dir
.entries
.iter()
.filter(|e| !e.is_dir)
.filter(|e| {
let u = e.name.to_ascii_uppercase();
u.starts_with("VTKF") && u.ends_with(".AACS")
})
.map(|e| e.name.as_str())
.collect();
names.sort_unstable();
v.extend(names.into_iter().map(|n| format!("/{d}/{n}")));
}
}
}
v
}
/// Walk an AACS role's candidate paths (from [`role_paths`]) and return the
/// first that reads.
///
/// `read` performs the actual per-path read (full file or bounded prefix), so
/// callers share the same first-present walk regardless of read style. Returns
/// [`Error::AacsNoKeys`] if no candidate is present. This is the single place
/// the `/AACS/` (BD/UHD) vs `/ANY!/` (HD DVD) layout difference is resolved.
pub(crate) fn read_first<F>(candidates: &[&str], mut read: F) -> crate::error::Result<Vec<u8>>
/// [`Error::AacsNoKeys`] if no candidate is present. Generic over the path
/// element (`&str` or owned `String`) so it accepts the `Vec<String>` that
/// [`role_paths`] builds from the discovered HD DVD directory.
pub(crate) fn read_first<S, F>(candidates: &[S], mut read: F) -> crate::error::Result<Vec<u8>>
where
S: AsRef<str>,
F: FnMut(&str) -> crate::error::Result<Vec<u8>>,
{
for path in candidates {
if let Ok(buf) = read(path) {
if let Ok(buf) = read(path.as_ref()) {
return Ok(buf);
}
}
@@ -161,4 +236,118 @@ mod tests {
None,
);
}
// ── HD DVD AACS directory / filename discovery ────────────────────────
//
// The HD DVD AACS dir name and title-key filename are authoring-specific
// and were previously hardcoded to `/ANY!/VTKF000.AACS`. These verify the
// discovery replacement against both real-disc shapes: Freedom (`AAC!` +
// `VTKF090`/`VTKF100`) and a BD/UHD disc (no HD DVD dir).
#[test]
fn role_paths_discovers_hddvd_dir_and_globs_all_vtkf_variants() {
use crate::udf::fixture::*;
// Freedom-shaped: an `AAC!` dir (NOT `ANY!`) holding MKBROM + two VTKF
// variants (090/100, NOT 000) + a VTUF usage file (must be excluded),
// plus the `AAC!_BAK` mirror (must NOT be picked as the AACS dir).
let mut disc = MemDisc::new();
let aacs_files = vec![
file("MKBROM.AACS", 100, 5000, 4096, true),
file("CONTENT_CERT.AACS", 101, 5100, 2048, true),
file("VTKF100.AACS", 102, 5200, 2048, true),
file("VTKF090.AACS", 103, 5300, 2048, true),
file("VTUF090.AACS", 104, 5400, 2048, true),
];
let bak_files = vec![file("MKBROM.AACS", 110, 6000, 4096, true)];
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![
DirSpec {
name: "AAC!".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: aacs_files,
subdirs: vec![],
},
DirSpec {
name: "AAC!_BAK".to_string(),
icb_lba: 30,
dir_data_lba: 31,
files: bak_files,
subdirs: vec![],
},
],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
// Discovered structurally (ends in '!', holds MKBROM.AACS) — the real
// AACS dir, never the `_BAK` mirror.
let dir = super::find_hddvd_aacs_dir(&udf).expect("aacs dir");
assert_eq!(dir.name, "AAC!");
// UnitKey: BD/UHD paths first, then EVERY VTKF*.AACS in sorted order
// (090 before 100) — NOT hardcoded VTKF000; VTUF (usage) excluded.
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
"/AAC!/VTKF090.AACS".to_string(),
"/AAC!/VTKF100.AACS".to_string(),
]
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::Mkb)
.last()
.unwrap(),
"/AAC!/MKBROM.AACS"
);
assert_eq!(
super::role_paths(&udf, super::AacsRole::ContentCert)
.last()
.unwrap(),
"/AAC!/CONTENT_CERT.AACS"
);
}
#[test]
fn role_paths_bd_uhd_disc_yields_no_hddvd_candidates() {
use crate::udf::fixture::*;
// A `/AACS/` tree (BD/UHD) has no '!' directory → discovery finds none
// and the candidate list is exactly the static BD/UHD paths.
let mut disc = MemDisc::new();
let root = DirSpec {
name: String::new(),
icb_lba: 10,
dir_data_lba: 11,
files: Vec::new(),
subdirs: vec![DirSpec {
name: "AACS".to_string(),
icb_lba: 20,
dir_data_lba: 21,
files: vec![
file("Unit_Key_RO.inf", 100, 5000, 2048, true),
file("MKB_RO.inf", 101, 5100, 2048, true),
],
subdirs: vec![],
}],
};
build_udf_skeleton(&mut disc, 10);
lay_dir(&mut disc, &root);
let udf = crate::udf::read_filesystem(&mut disc).expect("fs");
assert!(super::find_hddvd_aacs_dir(&udf).is_none());
assert_eq!(
super::role_paths(&udf, super::AacsRole::UnitKey),
vec![
super::PATH_UNIT_KEY_RO.to_string(),
super::PATH_UNIT_KEY_RO_DUPLICATE.to_string(),
]
);
}
}
+258
View File
@@ -136,6 +136,65 @@ pub fn crack_key_outcome(
crack_key_scan(reader, extents, batch_sectors, halt, true)
}
/// Resolve a DVD title's CSS descramble key from the reader when the caller
/// supplied none — the SINGLE place every DVD read path obtains a title key, so
/// the file-backed mux highway ([`crate::build_iso_pipeline`]) and the
/// live-drive single-pass [`crate::DiscStream`] descramble a DVD identically
/// ("reading is reading"). CSS keys are per-VTS and crackable from the scrambled
/// data itself, so a `None`/MPEG-PS title cracks its own key here, in playback
/// order over `extents`. Everything else is left untouched:
/// - AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
/// - a title that already carries a key — nothing to resolve.
/// - a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
///
/// A scrambled-but-uncrackable title is a hard [`crate::error::Error::CssKeyMissing`],
/// never a silent scrambled-passthrough mux.
pub(crate) fn resolve_dvd_title_key(
reader: &mut dyn SectorSource,
extents: &[Extent],
keys: &mut crate::decrypt::DecryptKeys,
batch_sectors: u16,
format: crate::disc::ContentFormat,
raw: bool,
halt: Option<&crate::halt::Halt>,
) -> std::io::Result<()> {
// `--raw` = deliberate ciphertext passthrough: never crack or descramble, and
// never hard-fail on scrambled-uncrackable — the user asked for the scrambled
// bytes. (In raw mode the caller hands us `None` on purpose; without this
// guard we'd install a real key and silently DECRYPT, or abort a raw mux.)
if raw {
return Ok(());
}
if matches!(keys, crate::decrypt::DecryptKeys::None)
&& format == crate::disc::ContentFormat::MpegPs
{
// `halt` threads the caller's cancellation token so /api/stop can
// interrupt a long crack scan (the old scan-time crack honored it too).
let outcome = crack_key_outcome(reader, extents, batch_sectors, halt);
// A cancelled crack breaks out early, so its outcome is a TRUNCATED scan
// — not a real verdict. Interpreting it would either hard-fail a good disc
// as `ScrambledUncracked` (quarantining staging on a Stop) or, worse,
// read a half-scanned title as `Unencrypted` and mux scrambled bytes as
// plaintext. Surface the cancellation as `Halted` so the caller takes its
// graceful-stop path instead of trusting the partial outcome.
if halt.map(|h| h.is_cancelled()).unwrap_or(false) {
return Err(crate::error::Error::Halted.into());
}
match outcome {
CrackOutcome::Cracked(state) => {
*keys = crate::decrypt::DecryptKeys::Css {
title_key: state.title_key,
};
}
CrackOutcome::ScrambledUncracked => {
return Err(crate::error::Error::CssKeyMissing.into());
}
CrackOutcome::Unencrypted => {}
}
}
Ok(())
}
/// The crack scan, returning the full [`CrackOutcome`]. Tracks a
/// `saw_scrambled` flag so a scrambled-but-uncracked disc is distinguished
/// from a genuinely-unencrypted one (the [`crack_key`] `Option` wrapper
@@ -893,6 +952,205 @@ mod tests {
);
}
/// `resolve_dvd_title_key` is the SINGLE shared per-title CSS step both read
/// paths (`build_iso_pipeline` multi-pass and `DiscStream::new` single-pass)
/// call, so these pin its full contract at the shared boundary.
///
/// Crack path: a `None`-keyed MPEG-PS title with a crackable scrambled sector
/// installs a `Css` key that round-trips the sector.
#[test]
fn resolve_dvd_title_key_cracks_none_mpegps() {
let title_key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
let seed = [0x11, 0x22, 0x33, 0x44, 0x55];
let crackable = crackable_sector(&title_key, &seed, 8);
let mut src = MockSource::new(0x00);
src.crackable = Some((1003, crackable));
let extents = [Extent {
start_lba: 1000,
sector_count: 50,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("crackable title resolves");
match keys {
crate::decrypt::DecryptKeys::Css { title_key: got } => {
assert_eq!(got, title_key, "installed key must be the cracked key")
}
_ => panic!("expected Css key"),
}
}
/// Hard-fail path: a scrambled-but-uncrackable `None`-keyed MPEG-PS title must
/// return `CssKeyMissing`, never leave `keys` as `None` (which would mux
/// scrambled bytes as plaintext — the 328k-decode-error corruption).
#[test]
fn resolve_dvd_title_key_scrambled_uncrackable_hard_fails() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // every read CSS-locked → ScrambledUncracked
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
let err = resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect_err("scrambled-uncrackable must hard-fail");
// The Error::CssKeyMissing flattens into io::Error carrying its E-code
// (7023) in the message — assert that specific code survived.
assert!(
err.to_string()
.contains(&format!("E{}", crate::error::E_CSS_KEY_MISSING)),
"must surface CssKeyMissing (E{}), got: {err}",
crate::error::E_CSS_KEY_MISSING
);
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"keys must stay None on hard-fail (never a scrambled-passthrough key)"
);
}
/// `raw` is deliberate ciphertext passthrough: even a scrambled-uncrackable
/// title must return `Ok` and leave `keys` untouched (`None`) — no crack, no
/// hard-fail. This is the `--raw` guarantee.
#[test]
fn resolve_dvd_title_key_raw_skips_crack_and_never_fails() {
let mut src = MockSource::new(0x00);
src.lock_all = true;
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
true, // raw
None,
)
.expect("raw must never hard-fail");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"raw must leave keys None (no descramble)"
);
assert!(
src.reads.borrow().is_empty(),
"raw must not read any sector for a crack"
);
}
/// AACS gate: an MPEG-PS title carrying `Aacs` keys (HD-DVD `.evo`) must be
/// left untouched — resolve only fires on `None` keys, never overwriting a
/// real key set or cracking AACS ciphertext as CSS.
#[test]
fn resolve_dvd_title_key_leaves_aacs_untouched() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // would hard-fail IF it ran the crack
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::Aacs {
unit_keys: vec![(0, [0u8; 16])],
read_data_key: None,
format: crate::disc::ContentFormat::MpegPs,
};
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("AACS title must be left untouched, not cracked");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::Aacs { .. }),
"Aacs keys must survive unchanged"
);
assert!(
src.reads.borrow().is_empty(),
"must not read for a crack when keys are already Aacs"
);
}
/// Clear DVD: a `None`-keyed MPEG-PS title with no scrambled sector stays
/// `None` (a mux no-op) and returns `Ok` — genuinely-unencrypted DVDs pass.
#[test]
fn resolve_dvd_title_key_clear_dvd_stays_none() {
let mut src = MockSource::new(0x00); // all-clear sectors
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let mut keys = crate::decrypt::DecryptKeys::None;
resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
None,
)
.expect("clear DVD passes");
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"a clear DVD must keep None keys"
);
}
/// A cancelled crack (user Stop mid-scan) must surface as `Halted`, NOT be
/// misread from the truncated scan as `Unencrypted` (→ scrambled passthrough,
/// corruption) or `ScrambledUncracked` (→ CssKeyMissing, which quarantines a
/// good disc). This pins the halt-outcome fix.
#[test]
fn resolve_dvd_title_key_halt_surfaces_as_halted_not_a_verdict() {
let mut src = MockSource::new(0x00);
src.lock_all = true; // without the halt guard this would be ScrambledUncracked
let extents = [Extent {
start_lba: 0,
sector_count: 4,
}];
let halt = crate::halt::Halt::new();
halt.cancel(); // Stop already pressed
let mut keys = crate::decrypt::DecryptKeys::None;
let err = resolve_dvd_title_key(
&mut src,
&extents,
&mut keys,
4,
crate::disc::ContentFormat::MpegPs,
false,
Some(&halt),
)
.expect_err("a cancelled crack must return an error");
assert!(
err.to_string()
.contains(&format!("E{}", crate::error::E_HALTED)),
"cancelled crack must surface Halted (E{}), got: {err}",
crate::error::E_HALTED
);
}
/// CSS_ERROR WIRING (audit §2 / §5 #7): an all-locked synthetic ISO (every
/// VOB read returns CSS-locked sense `05/6F/03` across MULTIPLE extents, as a
/// real encrypted-but-unauthenticated disc image does) must produce the exact
+10 -5
View File
@@ -324,13 +324,18 @@ impl Disc {
use crate::aacs;
let uk_ro_data =
aacs::read_first(aacs::UNIT_KEY_RO_PATHS, |p| udf_fs.read_file(reader, p))?;
aacs::read_first(&aacs::role_paths(udf_fs, aacs::AacsRole::UnitKey), |p| {
udf_fs.read_file(reader, p)
})?;
let dh = aacs::inf::disc_hash(&uk_ro_data);
let cc = aacs::read_first(aacs::CONTENT_CERT_PATHS, |p| udf_fs.read_file(reader, p))
.ok()
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let cc = aacs::read_first(
&aacs::role_paths(udf_fs, aacs::AacsRole::ContentCert),
|p| udf_fs.read_file(reader, p),
)
.ok()
.as_deref()
.and_then(aacs::inf::parse_content_cert);
let bus_encryption = cc.as_ref().map(|c| c.bus_encryption).unwrap_or(false);
// No-cert default = UHD (V20 stride), matching `read_aacs_version` so the
// scanned `AacsState.version` and the out-of-band fetch agree. A wrong
+507 -234
View File
@@ -16,6 +16,7 @@ mod extract;
mod hddvd;
pub mod mapfile;
mod patch;
pub(crate) mod pgs_forced_probe;
pub mod read_error;
mod section_recover;
mod sweep;
@@ -1491,6 +1492,13 @@ pub struct ScanOptions {
/// 50_000 sectors on a live DVD) poll it and bail out cleanly so a
/// scan-phase watchdog or operator Stop is never stuck behind a hang.
pub halt: Option<crate::halt::Halt>,
/// Read the PGS subtitle streams during the scan to detect forced-narrative
/// tracks from their content (the `forced_on_flag`), matching what the mux
/// derives during a rip. OFF by default — it reads the clip's PGS content,
/// which is slow, so only callers that want authoritative forced flags in the
/// scanned title (e.g. `freemkv info`) opt in. The rip path leaves it off:
/// the muxer detects forced during muxing without a second read.
pub probe_forced_subtitles: bool,
}
/// Quick disc identification — name, format, capacity. No title/stream parsing.
@@ -1603,6 +1611,36 @@ impl Disc {
// (BD/UHD speed is set by drive unlock/init, but DVD needs explicit SET CD SPEED)
session.set_speed(0xFFFF);
// CSS bus-auth unlock — run BEFORE any scrambled-sector read.
// On a CSS-enforcing drive (e.g. the BU40N) the UDF metadata prefetch
// below reaches small/menu VOB extents that are themselves CSS-scrambled;
// without the bus-auth handshake first, each of those reads is rejected
// with sense 05/6F/03 ("read of scrambled sector without authentication")
// after a full drive round-trip — ~13s of pure waste on a real disc, and
// the prefetch caches nothing. The handshake needs no title info (it takes
// no extents), is self-guarding to DVD media, and is non-fatal on failure,
// so run it as soon as the drive has classified the disc as a DVD. The
// per-VTS title key is NOT recovered here (or anywhere in scan) — it is
// cracked keylessly at read/mux time via `css::resolve_dvd_title_key`;
// this only unlocks the drive's read gating so those reads can happen.
if session.disc_is_dvd() {
tracing::info!(target: "freemkv::scan", "phase: CSS — bus-auth unlock (pre-scan)");
let drive_id = session.drive_id.clone();
let (_, css_unlock_res) = crate::unlock_bridge::run_bus(
session.scsi_mut(),
&drive_id,
freemkv_unlock::DiscKind::Css,
&[],
);
if let Err(e) = css_unlock_res {
tracing::warn!(
target: "freemkv::scan",
outcome = ?e,
"CSS bus-auth unlock did not apply; scrambled sectors may be unavailable"
);
}
}
// Read UDF filesystem with buffered sector reader
tracing::info!(target: "freemkv::scan", "phase: reading UDF filesystem");
let (capacity, mut buffered, udf_fs) = Self::read_udf(session)?;
@@ -1615,7 +1653,7 @@ impl Disc {
}
tracing::info!(target: "freemkv::scan", "phase: parsing titles/streams");
let mut disc = Self::scan_with(
let disc = Self::scan_with(
&mut buffered,
capacity,
handshake,
@@ -1625,119 +1663,21 @@ impl Disc {
)?;
tracing::info!(target: "freemkv::scan", titles = disc.titles.len(), format = ?disc.content_format, "phase: titles parsed");
// CSS key extraction for DVDs (bus auth → disc key → title key).
// Must be a single auth session — can't call authenticate() separately.
// Route through the DRM dispatcher: probe a title sector, detect
// CSS if scrambled, then load via the SCSI auth path.
// We already know this is a DVD (MPEG-PS program stream), so drive the
// CSS handshake DIRECTLY off the main title's first content sector. We
// must NOT first read a scrambled sector to "detect" CSS: a drive that
// enforces CSS (e.g. the BU40N) rejects an UNauthenticated read of a
// scrambled sector with sense 05/6F/03 ("read of scrambled sector
// without authentication"), so a detect-then-auth ordering dead-locks —
// detection needs the read, the read needs auth, auth needs detection.
// The handshake is itself the detector: on a non-CSS (unencrypted) DVD
// the disc-key read fails, `resolve` returns None, and the disc is left
// in the clear. This block is DVD-only: gate on `DiscFormat::Dvd`, NOT
// `content_format == MpegPs` — HD-DVD `.evo` is ALSO MPEG-PS but is AACS,
// not CSS, so it must never enter the CSS/REPORT-KEY handshake (it goes
// through the AACS path above). BD/UHD are MPEG-TS and never reach here.
if disc.css.is_none() && disc.format == DiscFormat::Dvd && !disc.titles.is_empty() {
// CSS title keys are per-VTS, and ONLY the scrambled movie content
// carries a non-zero key. Menu / VMG / logo cells (often the
// low-LBA first extent) return a ZERO title key over REPORT KEY —
// accepting that would leave the whole feature un-descrambled
// (raw scrambled bytes passed through as "clear"). So build
// candidate LBAs from the MAIN feature (largest title), LARGEST
// extent first (the movie body is the biggest scrambled chunk),
// and accept the first auth that yields a NON-ZERO title key. A
// genuinely unencrypted DVD returns zero for every candidate →
// disc stays in the clear. This block is DVD-only (MPEG-PS);
// BD/UHD (MPEG-TS) used the AACS handshake above and never reach here.
// Main feature = the largest title; its extents, largest (the movie
// body) first — that's where the scrambled content with a recoverable
// title key lives.
let main_extents = match disc
.titles
.iter()
.filter(|t| !t.extents.is_empty())
.max_by_key(|t| t.extents.iter().map(|e| e.sector_count as u64).sum::<u64>())
{
Some(t) => {
let mut v = t.extents.clone();
v.sort_by(|a, b| b.sector_count.cmp(&a.sector_count));
v
}
None => Vec::new(),
};
tracing::info!(target: "freemkv::scan", extents = main_extents.len(), "phase: CSS — main feature located");
if let Some(unlock_lba) = main_extents.first().map(|e| e.start_lba) {
tracing::info!(target: "freemkv::scan", unlock_lba, "phase: CSS — bus-auth unlock");
// Unlock the drive's CSS read gating through the uniform
// unlocker dispatch: the CSS unlocker matches DiscKind::Css and
// runs the bus-auth handshake (self-guarding to DVD media). A
// CSS-enforcing drive (the BU40N) refuses to return scrambled
// sectors until that handshake has run; we run it purely for that
// unlock and IGNORE any key (the descramble key is recovered
// keylessly from the scrambled movie data via the known-plaintext
// attack — no player keys, no disc-key crack, no REPORT-KEY title
// key). Any failure is non-fatal: continue to the crack, which
// simply finds nothing if the drive kept the sectors gated.
let drive_id = session.drive_id.clone();
let (_, css_unlock_res) = crate::unlock_bridge::run_bus(
session.scsi_mut(),
&drive_id,
freemkv_unlock::DiscKind::Css,
&[],
);
if let Err(e) = css_unlock_res {
tracing::warn!(
target: "freemkv::scan",
outcome = ?e,
"CSS bus-auth unlock did not apply; scrambled sectors may be unavailable"
);
}
// Size the crack's batch reads to THIS drive's per-command max
// (DVD ≈ 16; the USB bridge may be lower) — an over-large
// READ(10) fails outright and would scan nothing.
let crack_batch = detect_max_batch_sectors(session.device_path());
tracing::info!(target: "freemkv::scan", crack_batch, "phase: CSS — known-plaintext crack");
let crack_t0 = std::time::Instant::now();
let crack_result = crate::css::crack_key_outcome(
session,
&main_extents,
crack_batch,
opts.halt.as_ref(),
);
tracing::info!(
target: "freemkv::scan",
elapsed_ms = crack_t0.elapsed().as_millis() as u64,
outcome = ?crack_result,
"phase: CSS — crack done"
);
match crack_result {
crate::css::CrackOutcome::Cracked(state) => {
tracing::debug!(target: "freemkv::disc", "dvd css: title key recovered via known-plaintext crack");
disc.css = Some(state);
disc.encrypted = true;
}
crate::css::CrackOutcome::ScrambledUncracked => {
// Scrambled sectors WERE seen but no key could be
// recovered — the content is encrypted-but-uncrackable.
// Record a hard error so callers fail loudly instead of
// muxing scrambled MPEG as plaintext garbage at exit 0.
tracing::warn!(target: "freemkv::disc", "dvd css: scrambled sectors seen but no title key cracked");
disc.encrypted = true;
disc.css_error = Some(crate::error::Error::CssKeyMissing);
}
crate::css::CrackOutcome::Unencrypted => {
tracing::debug!(target: "freemkv::disc", "dvd css: no scrambled sector seen (genuinely unencrypted)");
}
}
}
}
// No CSS key recovery at scan time. DVD CSS keys are per-VTS/per-title,
// and the descramble path re-cracks each title's key keylessly from that
// title's own extents at the moment its sectors are read/decrypted — so a
// single up-front "disc key" is meaningless (it's not valid for the other
// titles and every title re-derives its key at read time anyway). The
// scan's only CSS responsibility is the bus-auth unlock above, which opens
// the drive's read gating so the sweep can read scrambled sectors at all.
// Detection is likewise moot: encrypted or not, a DVD muxes identically —
// the read-time descrambler cracks a title key if the sectors are
// scrambled and is a no-op if they are clear.
tracing::info!(target: "freemkv::scan", css = disc.css.is_some(), "phase: scan complete");
// `disc.css` is intentionally never set at scan time now (per-title CSS
// keys are cracked at read/mux time), so log the format the scan actually
// determined rather than a key state that is always `None` here.
tracing::info!(target: "freemkv::scan", format = ?disc.format, titles = disc.titles.len(), "phase: scan complete");
Ok(disc)
}
@@ -1816,9 +1756,10 @@ impl Disc {
reader: &mut dyn SectorSource,
udf_fs: &udf::UdfFs,
) -> Result<(Vec<u8>, Vec<u8>, u8)> {
let inf = crate::aacs::read_first(crate::aacs::UNIT_KEY_RO_PATHS, |p| {
udf_fs.read_file(reader, p)
})?;
let inf = crate::aacs::read_first(
&crate::aacs::role_paths(udf_fs, crate::aacs::AacsRole::UnitKey),
|p| udf_fs.read_file(reader, p),
)?;
let mkb = Self::read_mkb_content(reader, udf_fs)?;
let version = Self::read_aacs_version(reader, udf_fs);
Ok((inf, mkb, version))
@@ -1836,9 +1777,10 @@ impl Disc {
/// mis-strided title keys (silent wrong unit keys), so a missing cert must
/// not quietly pick the V10 stride for a UHD disc.
fn read_aacs_version(reader: &mut dyn SectorSource, udf_fs: &udf::UdfFs) -> u8 {
match crate::aacs::read_first(crate::aacs::CONTENT_CERT_PATHS, |p| {
udf_fs.read_file(reader, p)
})
match crate::aacs::read_first(
&crate::aacs::role_paths(udf_fs, crate::aacs::AacsRole::ContentCert),
|p| udf_fs.read_file(reader, p),
)
.ok()
.as_deref()
.and_then(crate::aacs::inf::parse_content_cert)
@@ -1872,9 +1814,10 @@ impl Disc {
const MAX_BYTES: usize = 64 * 1024 * 1024;
let mut want = START_BYTES;
loop {
let buf = crate::aacs::read_first(crate::aacs::MKB_PATHS, |p| {
udf_fs.read_file_prefix(reader, p, want)
})?;
let buf = crate::aacs::read_first(
&crate::aacs::role_paths(udf_fs, crate::aacs::AacsRole::Mkb),
|p| udf_fs.read_file_prefix(reader, p, want),
)?;
let n = crate::aacs::mkb::mkb_content_len(&buf);
// `n` strictly inside `buf` => the record walk reached the padding
// boundary (full content captured). `buf` shorter than `want` =>
@@ -1983,6 +1926,18 @@ impl Disc {
// 4. Metadata + labels
let meta_title = Self::read_meta_title(reader, &udf_fs);
crate::labels::apply(reader, &udf_fs, &mut titles);
// Optional content-based forced-subtitle detection. `info` opts in so its
// forced flags match what the muxer derives during a rip (both use the
// one shared PGS classifier); the rip path leaves it off — the muxer
// detects forced while muxing, without a second read of the clip.
if _opts.probe_forced_subtitles {
for title in &mut titles {
if title.content_format == ContentFormat::BdTs {
pgs_forced_probe::probe_and_set_forced(reader, title);
}
}
}
crate::labels::fill_defaults(&mut titles);
// 5. Format (AACS MKB generation → BD/UHD/FMTS; tree → HD-DVD/DVD) and
@@ -2463,8 +2418,19 @@ impl Disc {
// firmware didn't (stock or Renesas drive) AND AACS state was
// actually obtained.
"AACS" => self.aacs.is_some() && !ld_removed_bus,
// The CSS handshake/crack succeeded → title keys recovered.
"CSS" => self.css.is_some(),
// The DVD read-unlock (CSS bus-auth) is issued during the scan
// for EVERY DVD to clear the drive's scrambled-read barrier;
// unlike the firmware/AACS arms above (whose success is already
// captured in disc state), the bus-auth outcome is not tracked
// separately, so this arm reports the MEDIUM — "the DVD
// read-unlock path engaged" — rather than a per-rip success
// bit. That is intentional: the CSS descramble is keyless and
// handled at mux time, and a genuine bus-auth failure surfaces
// downstream as a read/crack error, not here. (Formerly this
// reported `self.css.is_some()` — "a crack recovered a key" —
// which under-reported: an encrypted DVD that reads and muxes
// fine showed "no".)
"DVD" => self.format == DiscFormat::Dvd,
// A newly-registered unlocker with no runtime signal wired
// here yet: report `no` rather than guess.
_ => false,
@@ -2556,94 +2522,97 @@ impl Disc {
/// Resolve decryption keys for muxing a *specific* title.
///
/// CSS title keys are per-VTS. The scan cracks one key (from the main
/// feature, title 0 for autorip). Applying it to a title that lives in
/// a *different* VTS would silently descramble with the wrong key
/// (garbage output). When the requested title's extents don't overlap
/// the span the cracked key came from, re-crack the key from this
/// title's own extents using `reader`. AACS / unencrypted / single-VTS
/// paths are identical to [`Self::decrypt_keys`].
/// For a **DVD** the CSS title key MUST be recovered before descrambling: a
/// scrambled sector without a Stevenson crib cannot self-crack, and CSS leaves
/// the pack/PES header clear, so a sector left un-descrambled would mux as a
/// structurally-valid but corrupt PES packet with no loss reported. Two ways
/// to get it:
///
/// `batch_sectors` sizes the crack's batched reads (file-safe value for
/// an ISO; `detect_max_batch_sectors` for a live drive).
/// Also returns the per-title encryption verdict the gate needs to AVOID A
/// FALSE ERROR on a genuinely-clear extra title.
/// - **Fast path** — the scan already cracked a key whose LBA span
/// ([`crate::css::CssState::crack_span`]) covers this title's VTS: reuse it.
/// No re-read, and on a live drive no second CSS bus-auth round-trip. CSS
/// title keys are per-VTS, so an overlapping span is the same key.
/// - **Crack** — when up-front detection missed (`self.css == None`) or the
/// title lives in a different VTS: crack the key from this title's OWN
/// extents in a SINGLE scan, in natural PLAYBACK ORDER (never largest-cell-
/// first, which starved the crack in a big cell's clear prefix — the 1.5.1
/// bug). Playback order reaches the scrambled feature body after only the
/// small clear front matter (logo / rating card) that precedes it. One
/// scan = one CSS-locked early-bail, so a locked title is not re-hammered
/// per cell against a live drive (hard rule #2); its 50k-sector budget is
/// the same accepted bound the disc-wide scan uses.
///
/// On a multi-VTS CSS DVD that ALSO carries a clear, unencrypted stub title
/// (a 0.5 s menu loop, an FBI-warning nav title) living in its own VTS, the
/// re-crack over that stub's extents finds NO scrambled sector and recovers
/// no key. Returning ONLY the keys would collapse that to
/// `DecryptKeys::None`, indistinguishable from "scrambled but uncrackable",
/// so [`Self::ensure_decryptable_keys`] (which fails whenever `css.is_some()`
/// and the key is `None`) would wrongly raise `E7023` for a title that needs
/// no key at all — the false error the multi-title mux must never emit.
/// Crucially the crack path does NOT gate on `self.css`: a detection miss can
/// never route the mux into raw passthrough of scrambled sectors.
///
/// So the re-crack runs via [`crate::css::crack_key_outcome`] and ALSO
/// returns `title_is_clear == true` when the title's own extents showed NO
/// scrambling (`CrackOutcome::Unencrypted`) — the gate then treats that title
/// as needing no key and passes it cleanly. A title that genuinely IS
/// scrambled but uncrackable returns `(None, false)` and still hard-fails.
/// The returned bool pairs with [`Self::ensure_title_decryptable`].
/// Outcomes (`batch_sectors` sizes the crack's batched reads):
/// - `(Css{title_key}, false)` — descramble with the recovered/reused key.
/// - `(None, true)` — a genuinely-clear title needs no key; the gate passes it.
/// - `(None, false)` — scrambled but no key recoverable → hard failure via
/// [`Self::ensure_title_decryptable`], never a silent garbage mux.
///
/// Non-DVD schemes (AACS / FMTS / genuinely unencrypted) return
/// [`Self::decrypt_keys`] unchanged.
pub fn decrypt_keys_for_title(
&self,
idx: usize,
reader: &mut dyn SectorSource,
batch_sectors: u16,
) -> (crate::decrypt::DecryptKeys, bool) {
let css = match self.css {
Some(ref c) => c,
None => return (self.decrypt_keys(), false),
};
let title = match self.titles.get(idx) {
Some(t) if !t.extents.is_empty() => t,
// No extents to crack from — fall back to the disc-wide key.
_ => return (self.decrypt_keys(), false),
};
// If the title overlaps the span the existing key was cracked from,
// it's the same VTS — the cracked key applies. `crack_span: None`
// (unknown provenance) is also treated as "applies".
let overlaps = match css.crack_span {
None => true,
Some((cs, ce)) => title.extents.iter().any(|e| {
let ts = e.start_lba;
let te = e.start_lba.saturating_add(e.sector_count);
ts < ce && cs < te
}),
};
if overlaps {
// Non-DVD (AACS / FMTS / genuinely unencrypted): disc-wide keys, unchanged.
if self.format != DiscFormat::Dvd {
return (self.decrypt_keys(), false);
}
// Different VTS: re-crack from this title's extents, largest first
// (the movie body is the biggest scrambled chunk — same heuristic
// the scan uses). The disc-wide key provably does NOT apply here
// (crack_span is Some and this title doesn't overlap it). Use
// `crack_key_outcome` (not the bare `crack_key`) so we can tell a
// genuinely-clear title (`Unencrypted` — no scrambled sector in its
// own extents) apart from a scrambled-but-uncrackable one:
// - Cracked → the title's own key.
// - Unencrypted → (None, title_is_clear=true): this extra title
// needs no key; the gate must NOT raise E7023.
// - ScrambledUncracked → (None, false): genuinely encrypted but no key
// → a real hard failure, still surfaced.
// The disc-wide fallback is reserved for the unknown-provenance case
// (crack_span == None), already handled above via overlaps == true.
let mut extents = title.extents.clone();
extents.sort_by(|a, b| b.sector_count.cmp(&a.sector_count));
match crate::css::crack_key_outcome(reader, &extents, batch_sectors, None) {
let title = match self.titles.get(idx) {
Some(t) if !t.extents.is_empty() => t,
// No extents to crack from: nothing scrambled to worry about, so mark
// it clear (`true`). Returning `false` here would let the gate's DVD
// "None keys + not clear = scrambled-uncracked" rule wrongly hard-fail
// a genuinely-unencrypted DVD title that has no extents.
_ => return (self.decrypt_keys(), true),
};
// Fast path: reuse the scan's cracked key if its span covers this title's
// VTS. `crack_span: None` (unknown provenance) is treated as covering.
if let Some(css) = self.css.as_ref() {
let covers = match css.crack_span {
None => true,
Some((cs, ce)) => title
.extents
.iter()
.any(|e| e.start_lba < ce && cs < e.start_lba.saturating_add(e.sector_count)),
};
if covers {
return (
crate::decrypt::DecryptKeys::Css {
title_key: css.title_key,
},
false,
);
}
}
// Detection miss or a different VTS: crack this title's key from its OWN
// extents in a SINGLE scan, in natural PLAYBACK ORDER (never largest-cell-
// first — that was the 1.5.1 garbage bug, where a big cell's clear prefix
// starved the crack). Playback order reaches the scrambled feature body
// after only the (small) clear front matter that precedes it on a real
// disc. This is ONE crack_key_outcome call, exactly like the disc-wide
// scan: its single CSS-locked early-bail runs at most once, so a locked/
// uncrackable title is NOT re-hammered per cell against the live drive
// (hard rule #2). The crack's 50k-sector budget bounds a fully-clear title,
// the same accepted bound the disc-wide scan uses.
match crate::css::crack_key_outcome(reader, &title.extents, batch_sectors, None) {
crate::css::CrackOutcome::Cracked(state) => (
crate::decrypt::DecryptKeys::Css {
title_key: state.title_key,
},
false,
),
// No scrambled sector in THIS title's extents: it is genuinely clear.
// Signal `title_is_clear` so the per-title gate passes it without a
// key — NO FALSE E7023 for an unencrypted extra title.
crate::css::CrackOutcome::Unencrypted => (crate::decrypt::DecryptKeys::None, true),
// Scrambled sectors seen but no key recovered: a genuine hard failure.
// Scrambled but no key recoverable → hard failure.
crate::css::CrackOutcome::ScrambledUncracked => {
(crate::decrypt::DecryptKeys::None, false)
}
// No scrambled sector anywhere in the whole title → genuinely clear.
crate::css::CrackOutcome::Unencrypted => (crate::decrypt::DecryptKeys::None, true),
}
}
@@ -2673,6 +2642,25 @@ impl Disc {
if title_is_clear && !keys.is_encrypted() {
return Ok(());
}
// A DVD title that cracked to no key and is NOT clear is scrambled-but-
// uncrackable (`decrypt_keys_for_title` → `ScrambledUncracked`). Hard-fail
// here directly: `ensure_decryptable_keys` gates CSS on `self.css.is_some()`
// (the scan's disc-wide detection), which can be `None` when that up-front
// detection missed — exactly the case the per-title crack exists to catch.
// Without this, an uncrackable DVD title would fall through to `Ok` and mux
// scrambled sectors as corrupt PES at exit 0.
if self.format == DiscFormat::Dvd && !title_is_clear && !keys.is_encrypted() {
return Err(Error::CssKeyMissing);
}
// A usable per-title key was resolved (a freshly-cracked CSS key, or AACS
// unit keys) — the title IS decryptable, so pass it WITHOUT consulting the
// disc-wide gate. `ensure_decryptable_keys` hard-fails on `self.css_error`
// unconditionally, which reflects the MAIN feature's crack: a bonus title
// in a different VTS that just cracked its own key must not be blocked by
// the main title having failed.
if keys.is_encrypted() {
return Ok(());
}
self.ensure_decryptable_keys(raw, keys)
}
@@ -3873,6 +3861,7 @@ impl Disc {
total_errors = pass_sum.total_errors,
zones_entered = pass_sum.zones_entered,
jumps_taken = pass_sum.jumps_taken,
marginal_recovered = pass_sum.marginal_recovered,
bytes_good = stats.bytes_good,
bytes_pending = stats.bytes_pending,
copy_elapsed_ms = copy_t0.elapsed().as_millis() as u64,
@@ -5145,6 +5134,8 @@ mod tests {
/// `[main_lba, main_end)`, plus a clear stub title living in a DISJOINT VTS.
fn css_disc_with_clear_stub() -> (Disc, usize) {
let mut disc = make_test_disc(100_000, "DVD");
disc.format = DiscFormat::Dvd; // make_test_disc defaults to Uhd
disc.content_format = ContentFormat::MpegPs;
disc.encrypted = true;
disc.css = Some(crate::css::CssState {
title_key: [0u8; 5],
@@ -5166,29 +5157,31 @@ mod tests {
(disc, 1) // stub is title index 1
}
/// THE Fix 2/3 regression: on a multi-VTS CSS DVD, a genuinely-clear extra
/// title (an unencrypted menu stub in its own VTS) must resolve to
/// `title_is_clear = true` with `None` keys, and `ensure_title_decryptable`
/// must PASS it — no false E7023. The old `decrypt_keys_for_title` +
/// `ensure_decryptable_keys` pair raised CssKeyMissing here because the
/// re-crack of the clear stub returned `None`, indistinguishable from a
/// scrambled-uncracked title.
/// A genuinely-clear extra title (an unencrypted menu stub in its own VTS)
/// on a CSS DVD must mux without a false E7023. The stub lives in a DISJOINT
/// VTS (its extents don't overlap the scan's `crack_span`), so
/// `decrypt_keys_for_title` takes the crack path over the stub's own extents;
/// the reader serves only clear sectors, so the crack returns `Unencrypted`
/// → `(None, title_is_clear=true)`. The gate must then PASS the title with no
/// key — no false E7023.
#[test]
fn clear_stub_title_on_css_disc_is_not_a_key_failure() {
let (disc, stub_idx) = css_disc_with_clear_stub();
assert_eq!(
disc.format,
DiscFormat::Dvd,
"fixture must exercise the DVD path"
);
let mut reader = ClearStubReader {
clear_range: (0, 100_000),
};
let (keys, title_is_clear) = disc.decrypt_keys_for_title(stub_idx, &mut reader, 8);
assert!(
!keys.is_encrypted(),
"a clear stub needs no key (got encrypted keys)"
matches!(keys, crate::decrypt::DecryptKeys::None),
"a clear stub in a disjoint VTS cracks to no key"
);
assert!(
title_is_clear,
"the stub's own extents show no scrambling → title_is_clear must be true"
);
// The gate must PASS the clear stub — NO false E7023.
assert!(title_is_clear, "the stub's own extents show no scrambling");
// The gate must PASS a clear title — NO false E7023.
assert!(
disc.ensure_title_decryptable(false, &keys, title_is_clear)
.is_ok(),
@@ -5593,53 +5586,333 @@ mod tests {
disc
}
/// Regression (multi-VTS CSS): a title that OVERLAPS the cracked span is
/// the same VTS — the existing key is reused and the reader is NOT touched.
/// Build a Stevenson-crackable scrambled CSS sector (a periodic run in the
/// clear header continuing past 0x80), mirroring the css-module fixture.
fn crackable_css_sector(title_key: &[u8; 5]) -> [u8; 2048] {
const RUN_START: usize = 0x59;
const PERIOD: usize = 8;
let mut sec = [0u8; 2048];
sec[0x00..0x04].copy_from_slice(&crate::css::PACK_START);
sec[0x14] = 0x10; // scramble flag
for (i, b) in sec.iter_mut().enumerate().skip(RUN_START) {
*b = (0xA0u8.wrapping_add((i % PERIOD) as u8)) ^ 0x5A;
}
crate::css::lfsr::scramble_sector(title_key, &mut sec);
sec
}
/// A reader that serves crackable CSS sectors for LBAs in `scrambled`
/// (half-open), all-zero (clear) elsewhere — records every LBA read.
struct CssMapReader {
key: [u8; 5],
scrambled: (u32, u32),
reads: std::cell::RefCell<Vec<u32>>,
}
impl SectorSource for CssMapReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
self.reads.borrow_mut().push(lba);
let n = (count as usize * 2048).min(buf.len());
for s in 0..(n / 2048) {
let this = lba + s as u32;
let dst = &mut buf[s * 2048..(s + 1) * 2048];
if this >= self.scrambled.0 && this < self.scrambled.1 {
dst.copy_from_slice(&crackable_css_sector(&self.key));
} else {
dst.fill(0);
}
}
Ok(n)
}
}
fn css_dvd_with_extents(extents: Vec<Extent>) -> Disc {
let mut disc = make_test_disc(200_000, "DVD");
disc.format = DiscFormat::Dvd;
disc.content_format = ContentFormat::MpegPs;
disc.encrypted = true;
let mut t = title_with_video(Codec::Mpeg2, Resolution::R480p);
t.extents = extents;
disc.titles = vec![t];
disc
}
/// `decrypt_keys_for_title` cracks a scrambled DVD title's key from the
/// title's OWN extents and hands the mux the validated key — the seed the
/// descramble needs, since a crib-less sector cannot self-crack and CSS leaves
/// the pack/PES header clear (an un-seeded mux would emit corrupt PES).
#[test]
fn decrypt_keys_for_title_reuses_key_for_same_vts() {
let disc = css_disc_with_two_vts();
let mut src = RecordingSource {
fn decrypt_keys_for_title_cracks_the_titles_key() {
let key = [0x11, 0x22, 0x33, 0x44, 0x55];
let disc = css_dvd_with_extents(vec![Extent {
start_lba: 100,
sector_count: 64,
}]);
let mut src = CssMapReader {
key,
scrambled: (100, 164),
reads: std::cell::RefCell::new(Vec::new()),
};
match disc.decrypt_keys_for_title(0, &mut src, 16).0 {
let (keys, title_is_clear) = disc.decrypt_keys_for_title(0, &mut src, 16);
assert!(!title_is_clear, "a scrambled title is not clear");
match keys {
crate::decrypt::DecryptKeys::Css { title_key } => {
assert_eq!(title_key, [0xAB; 5], "same-VTS title reuses cracked key");
assert_eq!(title_key, key, "must crack the title's own key")
}
_ => panic!("expected Css keys for same-VTS title"),
_ => panic!("expected Css{{key}} for a scrambled DVD title"),
}
}
/// REGRESSION (the 1.5.1 garbage bug): the crack scans extents in PLAYBACK
/// ORDER, never largest-cell-first. A title whose LARGEST cell opens with a
/// long unscrambled run must still crack its key from the smaller,
/// scrambled-early cell that plays first — largest-first would exhaust the
/// crack budget in the clear giant and wrongly report the title unencrypted,
/// which the mux would pass through as scrambled garbage.
#[test]
fn decrypt_keys_for_title_scans_playback_order_not_largest_first() {
let key = [0xDE, 0xAD, 0xBE, 0xEF, 0x01];
let disc = css_dvd_with_extents(vec![
// Plays FIRST: small, scrambled from its start.
Extent {
start_lba: 100,
sector_count: 32,
},
// A CLEAR cell far larger than the crack budget (would starve a
// largest-first scan before it reached the scrambled cell above).
Extent {
start_lba: 10_000,
sector_count: 100_000,
},
]);
let mut src = CssMapReader {
key,
scrambled: (100, 132),
reads: std::cell::RefCell::new(Vec::new()),
};
let (keys, _) = disc.decrypt_keys_for_title(0, &mut src, 16);
match keys {
crate::decrypt::DecryptKeys::Css { title_key } => assert_eq!(
title_key, key,
"must crack from the scrambled cell that plays first, not miss it behind the clear giant"
),
_ => panic!("largest-first regression: the title was read as unencrypted"),
}
assert!(
src.reads.borrow().is_empty(),
"an overlapping title must not trigger a re-crack read"
src.reads.borrow().iter().all(|&l| l < 10_000),
"the key is found in the first (scrambled) cell — the clear giant must never be scanned: {:?}",
src.reads.borrow()
);
}
/// Regression (multi-VTS CSS): a title in a DIFFERENT VTS (no overlap with
/// the cracked span) must re-crack from its OWN extents — verified by the
/// reader being driven over that title's LBA range (5000..). The fixture
/// yields unscrambled sectors so the re-crack finds NO key; the fix
/// requires this to be a HARD failure (`DecryptKeys::None`), NOT a silent
/// fall-back to the known-wrong-VTS disc-wide key (which would descramble
/// to garbage). Both the read-attempt and the None result are asserted.
/// A reader whose every read is CSS-locked (`05/6F/03`) — a genuinely
/// encrypted DVD whose sectors can't be authenticated/cracked.
struct LockedReader;
impl SectorSource for LockedReader {
fn read_sectors(
&mut self,
lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Err(Error::DiscRead {
sector: lba as u64,
status: Some(2),
sense: Some(crate::scsi::ScsiSense {
sense_key: 0x05,
asc: 0x6F,
ascq: 0x03,
}),
})
}
}
/// End-to-end: a scrambled-but-uncrackable DVD title with NO up-front
/// detection (`self.css == None`) drives `decrypt_keys_for_title` to
/// `(None, false)`, and the gate MUST hard-fail (CssKeyMissing) rather than
/// pass it to the muxer — the silent-garbage case the per-title crack catches.
#[test]
fn decrypt_keys_for_title_recracks_for_other_vts() {
let disc = css_disc_with_two_vts();
fn decrypt_keys_for_title_scrambled_uncracked_dvd_hard_fails_even_without_detection() {
let disc = css_dvd_with_extents(vec![Extent {
start_lba: 100,
sector_count: 8,
}]);
assert!(disc.css.is_none(), "fixture: no up-front detection");
let mut reader = LockedReader;
let (keys, title_is_clear) = disc.decrypt_keys_for_title(0, &mut reader, 8);
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None) && !title_is_clear,
"a locked/uncrackable scrambled title resolves to (None, false)"
);
let err = disc
.ensure_title_decryptable(false, &keys, title_is_clear)
.expect_err("scrambled-uncracked DVD title must hard-fail without detection");
assert_eq!(err.code(), crate::error::Error::CssKeyMissing.code());
}
/// Fast path: when the scan already cracked a key whose `crack_span` COVERS
/// this title's VTS, `decrypt_keys_for_title` reuses it and never touches the
/// reader (no redundant crack, no second bus-auth on a live drive).
#[test]
fn decrypt_keys_for_title_reuses_covered_scan_key_without_reading() {
let disc = css_disc_with_two_vts(); // css=[0xAB;5], crack_span=(100,200)
let mut src = RecordingSource {
reads: std::cell::RefCell::new(Vec::new()),
};
let keys = disc.decrypt_keys_for_title(1, &mut src, 16).0;
// Title 0's extents (100..200) overlap the cracked span → reuse.
let (keys, clear) = disc.decrypt_keys_for_title(0, &mut src, 16);
assert!(!clear);
match keys {
crate::decrypt::DecryptKeys::Css { title_key } => {
assert_eq!(title_key, [0xAB; 5], "reuse the scan's cracked key")
}
_ => panic!("expected the reused Css key"),
}
assert!(
matches!(keys, crate::decrypt::DecryptKeys::None),
"a re-crack miss in a provably-different VTS must be a hard failure (None), \
not the wrong-VTS disc-wide key"
src.reads.borrow().is_empty(),
"a covered title must NOT re-read/re-crack: {:?}",
src.reads.borrow()
);
let reads = src.reads.borrow();
}
/// A title in a DIFFERENT VTS (extents disjoint from `crack_span`) does NOT
/// reuse the scan key — it cracks its own key from its own extents.
#[test]
fn decrypt_keys_for_title_cracks_other_vts_on_no_overlap() {
let key = [0x77, 0x66, 0x55, 0x44, 0x33];
let disc = css_disc_with_two_vts(); // title 1 lives at 5000.., span=(100,200)
let mut src = CssMapReader {
key,
scrambled: (5000, 5100),
reads: std::cell::RefCell::new(Vec::new()),
};
let (keys, _) = disc.decrypt_keys_for_title(1, &mut src, 16);
match keys {
crate::decrypt::DecryptKeys::Css { title_key } => assert_eq!(
title_key, key,
"a disjoint-VTS title cracks its OWN key, not the reused scan key"
),
_ => panic!("expected a freshly-cracked Css key for the other VTS"),
}
assert!(
!reads.is_empty(),
"a non-overlapping title must trigger a re-crack read"
src.reads.borrow().iter().all(|&l| l >= 5000),
"must crack from title 1's own extents (>=5000): {:?}",
src.reads.borrow()
);
}
/// A title whose (realistic) clear front matter — studio logo / rating card —
/// plays FIRST, then the scrambled feature, still cracks: the single
/// playback-order scan reads through the small clear prefix and reaches the
/// scrambled body within its budget. (A clear prefix LARGER than the ~100 MB
/// crack budget would starve — the accepted bounded-budget limit, identical to
/// the disc-wide scan; not producible by real DVD front matter.)
#[test]
fn decrypt_keys_for_title_cracks_feature_after_clear_front_matter() {
let key = [0xCA, 0xFE, 0xBA, 0xBE, 0x02];
// css=None so the crack path runs. ~10 MB of clear front matter plays
// first (well under the crack budget), then the scrambled feature.
let disc = css_dvd_with_extents(vec![
Extent {
start_lba: 10_000,
sector_count: 5_000,
}, // clear front matter (~10 MB), plays first
Extent {
start_lba: 100,
sector_count: 2_000,
}, // scrambled feature body
]);
let mut src = CssMapReader {
key,
scrambled: (100, 2_100),
reads: std::cell::RefCell::new(Vec::new()),
};
let (keys, _) = disc.decrypt_keys_for_title(0, &mut src, 16);
match keys {
crate::decrypt::DecryptKeys::Css { title_key } => assert_eq!(
title_key, key,
"must crack the scrambled feature after reading through clear front matter"
),
_ => panic!("clear front matter wrongly starved the crack"),
}
}
/// Scrambling that begins well INTO a cell (after a clear prefix), not at its
/// start, must still be cracked: the single playback-order scan reads through
/// the clear prefix and reaches the scrambled body within its budget — never a
/// silent "clear" verdict that would mux the scrambled tail as corrupt PES.
#[test]
fn decrypt_keys_for_title_cracks_scrambling_after_a_clear_prefix_in_one_cell() {
let key = [0x0D, 0xEE, 0x40, 0x00, 0x05];
// One cell: clear for the first 9000 sectors, then scrambled (well within
// the crack budget). css=None so the crack path runs.
let disc = css_dvd_with_extents(vec![Extent {
start_lba: 100,
sector_count: 20_000,
}]);
let mut src = CssMapReader {
key,
scrambled: (100 + 9_000, 100 + 20_000),
reads: std::cell::RefCell::new(Vec::new()),
};
let (keys, _) = disc.decrypt_keys_for_title(0, &mut src, 16);
match keys {
crate::decrypt::DecryptKeys::Css { title_key } => assert_eq!(
title_key, key,
"the scan must crack scrambling that starts past a clear prefix"
),
_ => panic!("in-cell-deep scrambling was misread as clear (silent-garbage direction)"),
}
}
/// A DVD title with EMPTY extents (an angle/PGC placeholder with no cells)
/// resolves to `(decrypt_keys(), true)` — clear, no key needed — and the gate
/// must PASS it. Returning `false` here would trip the DVD scrambled-uncracked
/// rule and wrongly hard-fail a genuinely-clear empty title.
#[test]
fn decrypt_keys_for_title_empty_extents_is_clear_not_hard_fail() {
let mut disc = css_dvd_with_extents(vec![Extent {
start_lba: 100,
sector_count: 8,
}]);
disc.titles
.push(title_with_video(Codec::Mpeg2, Resolution::R480p)); // idx 1: no extents
let mut reader = LockedReader;
let (keys, title_is_clear) = disc.decrypt_keys_for_title(1, &mut reader, 8);
assert!(
title_is_clear,
"an empty-extents title is clear (nothing to descramble)"
);
assert!(
reads.iter().all(|&lba| lba >= 5000),
"re-crack must read title 1's own extents (>=5000), got {reads:?}"
disc.ensure_title_decryptable(false, &keys, title_is_clear)
.is_ok(),
"an empty-extents DVD title must not hard-fail"
);
}
/// A bonus title that cracked its OWN valid key must NOT be blocked by the
/// disc-wide `css_error` set when the MAIN feature's scan failed. A usable
/// per-title key means the title is decryptable regardless of another title's
/// failure. (Regression for the audit r5 css_error-over-valid-key finding.)
#[test]
fn ensure_title_decryptable_valid_key_ignores_disc_wide_css_error() {
let mut disc = css_dvd_with_extents(vec![Extent {
start_lba: 100,
sector_count: 8,
}]);
disc.css_error = Some(crate::error::Error::CssKeyMissing); // main feature failed
let keys = crate::decrypt::DecryptKeys::Css {
title_key: [0x42; 5], // this bonus title cracked its own key
};
assert!(
disc.ensure_title_decryptable(false, &keys, false).is_ok(),
"a title with its own valid CSS key must pass despite disc-wide css_error"
);
}
+196
View File
@@ -0,0 +1,196 @@
//! Content-based forced-subtitle detection for Blu-ray/UHD PGS tracks.
//!
//! `freemkv info` and the muxer must agree on which subtitle tracks are forced.
//! The muxer derives it from the PGS `forced_on_flag` while muxing a rip; this
//! module gives `info` the SAME verdict up front by reading the title's PGS
//! streams and feeding them through the one shared classifier
//! ([`crate::mux::codec::pgs::ForcedTracker`]) — so the two never diverge.
//!
//! Cost: a track is only confirmed forced once EVERY display set is seen to be
//! forced, so a disc that has a forced track is read through — the
//! accuracy-over-speed tradeoff `info` opts into. Full tracks early-exit as soon
//! as they show a single non-forced subtitle, and a whole run stops early once
//! every track has settled.
//!
//! Encrypted content: the probe reuses whatever [`SectorSource`] the scan holds.
//! With a decrypting source it sees real PGS; without keys it reads ciphertext
//! and observes no display sets, in which case it leaves each track's existing
//! (vendor-label-derived) forced flag untouched rather than asserting anything.
use crate::disc::{Codec, DiscTitle, Stream};
use crate::mux::codec::CodecParser;
use crate::mux::codec::pgs::{ForcedTracker, PgsParser};
use crate::mux::ts::TsDemuxer;
use crate::sector::SectorSource;
use std::collections::HashMap;
const SECTOR_BYTES: usize = 2048;
/// Read the clip in 2 MiB chunks.
const CHUNK_SECTORS: u16 = 1024;
/// Read the title's PGS streams and set `SubtitleStream::forced` from their
/// content. Best-effort: any read error ends the probe with whatever verdicts
/// have accumulated. Only PGS tracks are touched (DVD VobSub forced comes from
/// the IFO/vendor path).
pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
reader: &mut S,
title: &mut DiscTitle,
) {
let pg_pids: Vec<u16> = title
.streams
.iter()
.filter_map(|s| match s {
Stream::Subtitle(sub) if sub.codec == Codec::Pgs => Some(sub.pid),
_ => None,
})
.collect();
if pg_pids.is_empty() {
return;
}
let mut demux = TsDemuxer::new(&pg_pids);
let mut parsers: HashMap<u16, PgsParser> =
pg_pids.iter().map(|&p| (p, PgsParser::new())).collect();
let mut trackers: HashMap<u16, ForcedTracker> =
pg_pids.iter().map(|&p| (p, ForcedTracker::new())).collect();
let extents = title.extents.clone();
let mut buf = vec![0u8; CHUNK_SECTORS as usize * SECTOR_BYTES];
'outer: for ext in &extents {
let mut lba = ext.start_lba;
let mut remaining = ext.sector_count;
while remaining > 0 {
let count = remaining.min(CHUNK_SECTORS as u32) as u16;
let want = count as usize * SECTOR_BYTES;
let n = match reader.read_sectors(lba, count, &mut buf[..want], false) {
Ok(n) => n,
Err(_) => break 'outer, // best-effort — stop, keep what we have
};
if n == 0 {
break 'outer;
}
for pes in demux.feed(&buf[..n]) {
if let (Some(parser), Some(tracker)) =
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
{
for frame in parser.parse(&pes) {
tracker.observe(&frame.data);
}
}
}
// Every track has already shown a non-forced set → nothing left to
// learn; stop reading the (huge) clip.
if trackers.values().all(ForcedTracker::settled_not_forced) {
break 'outer;
}
lba += count as u32;
remaining -= count as u32;
}
}
// Drain any buffered final display set.
for (pid, parser) in parsers.iter_mut() {
if let Some(tracker) = trackers.get_mut(pid) {
for frame in parser.flush() {
tracker.observe(&frame.data);
}
}
}
// Apply verdicts. Only override a track we actually saw content for — an
// undecrypted/unread track keeps its vendor-derived flag.
for s in &mut title.streams {
if let Stream::Subtitle(sub) = s {
if sub.codec == Codec::Pgs {
if let Some(t) = trackers.get(&sub.pid) {
if t.observed() {
sub.forced = t.is_forced();
}
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::{ContentFormat, Extent, LabelQualifier, SubtitleStream};
/// A reader that yields all-zeros (an encrypted / unreadable clip) for a
/// bounded span, then EOF.
struct ZeroReader {
served: u32,
cap: u32,
}
impl SectorSource for ZeroReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
if self.served >= self.cap {
return Ok(0);
}
self.served += count as u32;
buf.fill(0);
Ok(buf.len())
}
fn capacity_sectors(&self) -> u32 {
self.cap
}
}
fn pgs_title(pid: u16, vendor_forced: bool) -> DiscTitle {
DiscTitle {
playlist: String::new(),
playlist_id: 0,
duration_secs: 0.0,
size_bytes: 0,
clips: vec![],
streams: vec![Stream::Subtitle(SubtitleStream {
pid,
codec: Codec::Pgs,
language: "eng".into(),
forced: vendor_forced,
qualifier: LabelQualifier::None,
codec_data: None,
})],
chapters: vec![],
extents: vec![Extent {
start_lba: 0,
sector_count: 4,
}],
content_format: ContentFormat::BdTs,
codec_privates: vec![None],
}
}
#[test]
fn no_observed_content_preserves_vendor_forced() {
// An unreadable/encrypted clip yields no PGS display sets — the probe must
// leave the existing vendor-derived forced flag untouched, never assert
// "not forced" from having seen nothing.
let mut reader = ZeroReader { served: 0, cap: 4 };
let mut title = pgs_title(0x1200, true);
probe_and_set_forced(&mut reader, &mut title);
let Stream::Subtitle(s) = &title.streams[0] else {
panic!()
};
assert!(s.forced, "no content observed → vendor forced preserved");
}
#[test]
fn no_pgs_streams_is_noop() {
// A title with no PGS subtitle streams is a no-op (the reader is never
// touched — a DVD/VobSub or audio-only title).
let mut reader = ZeroReader { served: 0, cap: 0 };
let mut title = pgs_title(0x1200, false);
// Swap the PGS sub for an audio stream so there are no PGS PIDs.
title.streams.clear();
probe_and_set_forced(&mut reader, &mut title);
assert_eq!(reader.served, 0, "no PGS PIDs → no reads");
}
}
+97
View File
@@ -114,6 +114,12 @@ pub struct ReadCtx {
/// good reads after the last error in the cluster." Used to count
/// zone entries and to bound zone_reads accurately.
pub in_damage_zone: bool,
/// Count of RECOVERED ERROR (marginal) reads the drive reported this pass
/// (surfaced by the PER=1 mode-select at drive-prep). Each is distrusted and
/// marked NonTrimmed for a Pass N re-read; the count is reported in the
/// pass summary so an operator can see how much of a "clean" rip was actually
/// marginal.
pub marginal_recovered: u64,
}
/// Coarse classification of a SCSI sense key for diagnostic logging.
@@ -187,6 +193,7 @@ impl ReadCtx {
zones_entered: 0,
jumps_taken: 0,
in_damage_zone: false,
marginal_recovered: 0,
}
}
@@ -226,6 +233,7 @@ impl ReadCtx {
zones_entered: 0,
jumps_taken: 0,
in_damage_zone: false,
marginal_recovered: 0,
}
}
@@ -283,6 +291,7 @@ impl ReadCtx {
total_errors: self.total_errors,
zones_entered: self.zones_entered,
jumps_taken: self.jumps_taken,
marginal_recovered: self.marginal_recovered,
}
}
}
@@ -296,6 +305,8 @@ pub struct PassSummary {
pub total_errors: u64,
pub zones_entered: u64,
pub jumps_taken: u64,
/// RECOVERED ERROR (marginal) reads distrusted and re-queued for Pass N.
pub marginal_recovered: u64,
}
/// What the caller should do after a read failure. The caller owns the
@@ -639,6 +650,36 @@ pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
};
}
// 4b. RECOVERED ERROR — the drive returned this sector but had to fight for
// it (ECC worked hard / retried). We enable reporting of these via MODE
// SELECT PER=1 at drive-prep precisely so they surface: on marginal/dirty
// media the drive's best-effort correction can be silently WRONG (a rip
// that "passed clean" but decoded with errors — the Bourne case). We
// distrust it: mark THIS block NonTrimmed and let Pass N re-read it with
// proper recovery (FUA cache-bypass) — a clean re-read wins, a persistent
// marginal becomes an honest concealed gap.
//
// Critically this is a per-block SkipBlock, NOT the damage-jump path: a
// recovered error is a single marginal sector, not a clustered hard-damage
// zone, so it must NOT trigger the 64 MB JumpAhead (which would nuke huge
// swaths of good data on a lightly-smudged disc). It also does NOT touch
// the damage window / multiplier — a recovered read is not damage-zone
// signal. Returns before the fast-jump logic below.
if sense_key == scsi::SENSE_KEY_RECOVERED_ERROR {
ctx.marginal_recovered += 1;
tracing::warn!(
target: "freemkv::disc",
phase = "recovered_error",
marginal_recovered = ctx.marginal_recovered,
asc = err.scsi_sense().map(|s| s.asc),
ascq = err.scsi_sense().map(|s| s.ascq),
"drive reported a recovered (marginal) read — distrusting; marking NonTrimmed for Pass N re-read"
);
return ReadAction::SkipBlock {
pause_secs: FAIL_PAUSE_SECS,
};
}
// 5. Marginal media (MEDIUM_ERROR / ABORTED_COMMAND) on a multi-
// sector batch: the drive can often read the same sectors
// individually. Bisect into single-sector reads (gentler on the
@@ -803,6 +844,62 @@ mod tests {
}
}
fn recovered_err() -> Error {
Error::DiscRead {
sector: 100,
status: Some(2),
sense: Some(ScsiSense {
sense_key: scsi::SENSE_KEY_RECOVERED_ERROR,
asc: 0x17,
ascq: 0x01,
}),
}
}
#[test]
fn recovered_error_skips_block_not_jump_pass_1() {
// A recovered (marginal) read on Pass 1 must NOT trigger the 64 MB
// damage-jump — that would nuke huge good regions on a lightly-smudged
// disc. It marks just this block NonTrimmed (SkipBlock) for a Pass N
// re-read, and is counted as marginal_recovered.
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&recovered_err(), &mut ctx);
assert!(
matches!(action, ReadAction::SkipBlock { .. }),
"recovered error must SkipBlock, not JumpAhead; got {action:?}"
);
assert_eq!(ctx.marginal_recovered, 1);
// It must not have inflated the damage-jump multiplier (not damage signal).
assert_eq!(ctx.jump_multiplier, 1);
assert_eq!(ctx.jumps_taken, 0);
}
#[test]
fn recovered_error_skips_block_pass_n_too() {
// Pass N sees the same: a recovered read is distrusted → SkipBlock (the
// outer patch loop re-reads the range with FUA).
let mut ctx = ReadCtx::for_patch(1);
let action = handle_read_error(&recovered_err(), &mut ctx);
assert!(
matches!(action, ReadAction::SkipBlock { .. }),
"got {action:?}"
);
assert_eq!(ctx.marginal_recovered, 1);
}
#[test]
fn many_recovered_errors_never_jump() {
// Even a run of recovered errors must never escalate to a damage-jump —
// they're marginal reads, not a hard-damage cluster.
let mut ctx = ReadCtx::for_sweep(32);
for _ in 0..40 {
let a = handle_read_error(&recovered_err(), &mut ctx);
assert!(matches!(a, ReadAction::SkipBlock { .. }), "got {a:?}");
}
assert_eq!(ctx.jumps_taken, 0, "recovered errors never jump");
assert_eq!(ctx.marginal_recovered, 40);
}
#[test]
fn pass_n_marginal_with_batch_gt_1_bisects() {
let mut ctx = ReadCtx::for_patch(32);
+191
View File
@@ -61,8 +61,28 @@ const SPIN_UP_SETTLE_SECS: u64 = 10;
const SCSI_PREVENT_ALLOW_MEDIUM_REMOVAL: u8 = 0x1E;
const SCSI_GET_EVENT_STATUS: u8 = 0x4A;
const SCSI_MODE_SENSE: u8 = 0x5A;
const SCSI_MODE_SELECT: u8 = 0x55;
const SCSI_REPORT_KEY: u8 = 0xA4;
/// SBC/MMC Read-Write Error Recovery mode page (page code 0x01). We flip the
/// `PER` bit to make the drive REPORT a recovered read (via CHECK CONDITION +
/// sense key RECOVERED ERROR) instead of silently returning best-effort data as
/// GOOD status. On marginal/dirty media that silent-GOOD data can be
/// mis-corrected — a rip that "passed clean" but decoded with errors. With PER
/// on, freemkv sees the marginal read and re-reads it in Pass N (a loud miss,
/// never a silent commit). See `build_error_recovery_select_payload`.
const MODE_PAGE_ERROR_RECOVERY: u8 = 0x01;
/// Bit masks in the Read-Write Error Recovery flags byte (page byte 2).
const ERP_FLAG_TB: u8 = 0x20; // Transfer Block: still deliver the recovered data
const ERP_FLAG_PER: u8 = 0x04; // Post Error: report recovered errors
const ERP_FLAG_DTE: u8 = 0x02; // Data Terminate on Error: MUST be off (we want the data)
/// `Parameters Saveable` bit in a mode page's byte 0 — valid only on MODE SENSE;
/// must be cleared before echoing the page back in a MODE SELECT.
const MODE_PAGE_PS_BIT: u8 = 0x80;
/// MODE SENSE(10) parameter header length (bytes), preceding any block
/// descriptors and the mode pages.
const MODE10_HEADER_LEN: usize = 8;
/// Optical disc drive session -- open, identify, unlock, and read.
pub struct Drive {
scsi: Box<dyn ScsiTransport>,
@@ -464,6 +484,12 @@ impl Drive {
// wants max speed. Best-effort: a failure here must NOT fail the rip.
if r.is_ok() {
self.set_speed(crate::speed::DriveSpeed::Max.to_kbps());
// Ask the drive to REPORT recovered/marginal reads rather than
// silently commit best-effort data as GOOD (the dirty-disc
// "passed-clean-but-decodes-with-errors" trap). Best-effort: a drive
// that doesn't honor it just keeps its defaults — no regression, and
// on a clean disc it changes nothing.
self.enable_recovered_error_reporting();
}
tracing::info!(
target: "freemkv::drive",
@@ -599,6 +625,53 @@ impl Drive {
}
}
/// Ask the drive to REPORT recovered/marginal reads instead of silently
/// returning best-effort data as GOOD status. MODE SENSE the Read-Write
/// Error Recovery page, flip `PER` (and `TB` on / `DTE` off so we still get
/// the data), and MODE SELECT it back — preserving the drive's own retry
/// count and other bits.
///
/// Best-effort: a drive that doesn't support the page, or rejects the SELECT,
/// simply keeps its default behaviour — no regression, the rip proceeds. On a
/// clean disc this changes nothing (no recovered errors fire); it only
/// surfaces the marginal reads that a dirty disc would otherwise commit
/// silently. Returns whether the page was successfully written.
pub fn enable_recovered_error_reporting(&mut self) -> bool {
let Some(sense) = self.mode_sense_page(MODE_PAGE_ERROR_RECOVERY) else {
tracing::debug!(target: "freemkv::drive", "MODE SENSE error-recovery page unavailable; leaving drive defaults");
return false;
};
let Some(payload) = build_error_recovery_select_payload(&sense) else {
tracing::debug!(target: "freemkv::drive", "error-recovery page malformed/short; leaving drive defaults");
return false;
};
// MODE SELECT(10): PF=1 (page format), parameter list length = payload.
let len = payload.len() as u16;
let cdb = [
SCSI_MODE_SELECT,
0x10, // PF=1, SP=0 (don't persist across power cycles)
0x00,
0x00,
0x00,
0x00,
0x00,
(len >> 8) as u8,
len as u8,
0x00,
];
let mut buf = payload;
match self.checked_exec(&cdb, crate::scsi::DataDirection::ToDevice, &mut buf, 5_000) {
Ok(_) => {
tracing::info!(target: "freemkv::drive", phase = "error_recovery", "recovered-error reporting enabled (PER=1) — marginal reads will surface instead of committing silently");
true
}
Err(e) => {
tracing::debug!(target: "freemkv::drive", error = %e, "MODE SELECT error-recovery page rejected; leaving drive defaults");
false
}
}
}
/// Read vendor-specific READ BUFFER data.
pub fn read_buffer(&mut self, mode: u8, buffer_id: u8, length: u16) -> Option<Vec<u8>> {
let cdb = crate::scsi::build_read_buffer(mode, buffer_id, 0, length as u32);
@@ -1101,6 +1174,46 @@ fn select_drive_with_media(drives: impl Iterator<Item = Drive>) -> Option<Drive>
fallback
}
/// Turn a MODE SENSE(10) Read-Write Error Recovery page response into the
/// payload for a MODE SELECT(10) that enables recovered-error REPORTING —
/// preserving every other bit (notably the drive's own read-retry count).
///
/// Pure so the bit-twiddling is unit-tested without a drive. Steps:
/// - locate the page after the 8-byte header + block descriptors (bytes 6-7);
/// - verify it is page 0x01 with a flags byte present;
/// - in the flags byte: set `PER` (report) and `TB` (still deliver the data),
/// clear `DTE` (don't terminate the transfer on the recovered error);
/// - clear the page's `PS` bit (valid only on SENSE) and zero the header's
/// mode-data-length field (reserved on SELECT).
///
/// Returns `None` (caller leaves the drive at its defaults) when the response is
/// too short or isn't the error-recovery page — never panics on adversarial
/// bytes.
fn build_error_recovery_select_payload(sense: &[u8]) -> Option<Vec<u8>> {
if sense.len() < MODE10_HEADER_LEN {
return None;
}
let block_desc_len = u16::from_be_bytes([sense[6], sense[7]]) as usize;
let page_off = MODE10_HEADER_LEN.checked_add(block_desc_len)?;
// Need page byte 0 (code), byte 1 (length), byte 2 (flags).
if page_off.checked_add(3)? > sense.len() {
return None;
}
if sense[page_off] & 0x3F != MODE_PAGE_ERROR_RECOVERY {
return None;
}
let mut payload = sense.to_vec();
// Header: mode-data-length is reserved on SELECT — zero it.
payload[0] = 0;
payload[1] = 0;
// Page byte 0: clear PS (SENSE-only).
payload[page_off] &= !MODE_PAGE_PS_BIT;
// Flags byte: PER on, TB on, DTE off. Retry count (next byte) untouched.
payload[page_off + 2] |= ERP_FLAG_PER | ERP_FLAG_TB;
payload[page_off + 2] &= !ERP_FLAG_DTE;
Some(payload)
}
/// Decode a READ CAPACITY (10) response into a sector count.
///
/// A short transfer (`bytes_transferred < 4`, which would leave the high
@@ -1276,6 +1389,84 @@ mod command_tests {
use super::*;
use crate::scsi::{DataDirection, ScsiResult, ScsiTransport};
/// A minimal MODE SENSE(10) response carrying the Read-Write Error Recovery
/// page (0x01) with the given flags byte and retry count, no block
/// descriptors. `ps` sets the page's PS bit (SENSE-only), which the SELECT
/// payload must clear.
fn mode_sense_error_recovery(flags: u8, retry: u8, ps: bool) -> Vec<u8> {
let mut v = vec![0u8; MODE10_HEADER_LEN + 12];
// Header: nonzero mode-data-length (must be zeroed on SELECT); no block
// descriptors.
v[0] = 0x00;
v[1] = 0x22;
v[6] = 0x00;
v[7] = 0x00; // block descriptor length = 0
let po = MODE10_HEADER_LEN;
v[po] = MODE_PAGE_ERROR_RECOVERY | if ps { MODE_PAGE_PS_BIT } else { 0 };
v[po + 1] = 0x0A; // page length
v[po + 2] = flags; // error-recovery flags
v[po + 3] = retry; // read retry count
v
}
#[test]
fn error_recovery_payload_sets_per_tb_clears_dte_ps_preserves_retry() {
// Start with PER off, DTE on, PS on, a specific retry count. The SELECT
// payload must flip PER on, TB on, DTE off, clear PS, zero the header
// mode-data-length, and leave the retry count untouched.
let sense = mode_sense_error_recovery(ERP_FLAG_DTE, 0x2C, true);
let out = build_error_recovery_select_payload(&sense).expect("valid page");
let po = MODE10_HEADER_LEN;
assert_eq!(out[0], 0, "header mode-data-length zeroed for SELECT");
assert_eq!(out[1], 0);
assert_eq!(out[po] & MODE_PAGE_PS_BIT, 0, "PS cleared for SELECT");
assert_eq!(out[po] & 0x3F, MODE_PAGE_ERROR_RECOVERY, "still page 0x01");
assert_eq!(out[po + 2] & ERP_FLAG_PER, ERP_FLAG_PER, "PER set");
assert_eq!(
out[po + 2] & ERP_FLAG_TB,
ERP_FLAG_TB,
"TB set (still get data)"
);
assert_eq!(
out[po + 2] & ERP_FLAG_DTE,
0,
"DTE cleared (don't terminate)"
);
assert_eq!(out[po + 3], 0x2C, "read retry count preserved");
}
#[test]
fn error_recovery_payload_honors_block_descriptor_offset() {
// With an 8-byte block descriptor between header and page, the function
// must locate the page at header+desc, not a fixed offset.
let mut sense = vec![0u8; MODE10_HEADER_LEN + 8 + 12];
sense[7] = 8; // block descriptor length
let po = MODE10_HEADER_LEN + 8;
sense[po] = MODE_PAGE_ERROR_RECOVERY;
sense[po + 1] = 0x0A;
sense[po + 2] = 0x00;
let out = build_error_recovery_select_payload(&sense).expect("valid");
assert_eq!(
out[po + 2] & ERP_FLAG_PER,
ERP_FLAG_PER,
"PER set at the descriptor-offset page"
);
}
#[test]
fn error_recovery_payload_rejects_wrong_or_short_page() {
// Wrong page code → None (leave drive at defaults).
let mut wrong = mode_sense_error_recovery(0, 0, false);
wrong[MODE10_HEADER_LEN] = 0x08; // page 0x08 (caching), not 0x01
assert!(build_error_recovery_select_payload(&wrong).is_none());
// Too short to hold the header → None, no panic.
assert!(build_error_recovery_select_payload(&[0u8; 4]).is_none());
// Header claims a block descriptor that runs off the buffer → None.
let mut bad = mode_sense_error_recovery(0, 0, false);
bad[7] = 0xF0; // descriptor length way past the buffer
assert!(build_error_recovery_select_payload(&bad).is_none());
}
/// Mock transport: returns a fixed data payload (copied into the
/// caller's buffer, truncated to fit) on every `execute()`.
struct FixedTransport {
+31
View File
@@ -150,6 +150,14 @@ pub const E_NETWORK_ADDR_BLOCKED: u16 = 9022;
/// frame dropped before the first keyframe) cannot report success.
pub const E_MUX_EMPTY: u16 = 9023;
pub const E_EXTENT_NOT_UNIT_ALIGNED: u16 = 9030;
/// `mp4://` output but the title has no (primary) video track to carry.
pub const E_MP4_NO_VIDEO_TRACK: u16 = 9048;
/// `mp4://` video track uses a codec with no MP4 mapping / no config record
/// available here (e.g. VC-1, or MPEG-2 without an avcC/hvcC-style record).
pub const E_MP4_UNSUPPORTED_VIDEO_CODEC: u16 = 9049;
/// `mp4://` video track is missing its codec-configuration record
/// (`hvcC`/`avcC`), without which the sample entry can't be written.
pub const E_MP4_MISSING_CODEC_PRIVATE: u16 = 9050;
/// READ CAPACITY returned a short or overflowing transfer.
pub const E_DISC_CAPACITY_MALFORMED: u16 = 9047;
@@ -420,6 +428,12 @@ pub enum Error {
/// every frame dropped before the first keyframe — fails loudly. The
/// `m2ts://` analogue of [`Error::MkvInvalid`]'s zero-frame guard.
MuxEmpty,
/// `mp4://` target title has no primary video track to mux.
MuxNoVideoTrack,
/// `mp4://` video codec has no MP4 sample-entry mapping available here.
Mp4UnsupportedVideoCodec,
/// `mp4://` video track is missing its `hvcC`/`avcC` configuration record.
MuxMissingCodecPrivate,
PesFrameTooLarge {
size: usize,
},
@@ -599,6 +613,9 @@ impl Error {
Error::StreamUrlMissingPort { .. } => E_STREAM_URL_MISSING_PORT,
Error::NetworkAddrBlocked { .. } => E_NETWORK_ADDR_BLOCKED,
Error::MuxEmpty => E_MUX_EMPTY,
Error::MuxNoVideoTrack => E_MP4_NO_VIDEO_TRACK,
Error::Mp4UnsupportedVideoCodec => E_MP4_UNSUPPORTED_VIDEO_CODEC,
Error::MuxMissingCodecPrivate => E_MP4_MISSING_CODEC_PRIVATE,
Error::PesFrameTooLarge { .. } => E_PES_FRAME_TOO_LARGE,
Error::PesInvalidMagic => E_PES_INVALID_MAGIC,
Error::PesTrackTooLarge { .. } => E_PES_TRACK_TOO_LARGE,
@@ -833,6 +850,11 @@ impl From<Error> for std::io::Error {
// 9023 MuxEmpty: finish() reached with zero frames — the output
// would be a header-only container. Treat as invalid output.
E_MUX_EMPTY => std::io::ErrorKind::InvalidData,
// 9048-9050 mp4:// track/codec/config mismatches: the requested
// output can't hold this title's video — invalid output request.
E_MP4_NO_VIDEO_TRACK | E_MP4_UNSUPPORTED_VIDEO_CODEC | E_MP4_MISSING_CODEC_PRIVATE => {
std::io::ErrorKind::InvalidData
}
// 9030 ExtentNotUnitAligned: a malformed/non-AACS-aligned
// extent was handed to the prefetch producer.
9030 => std::io::ErrorKind::InvalidInput,
@@ -1236,6 +1258,9 @@ mod tests {
E_STREAM_URL_MISSING_PORT,
E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY,
E_MP4_NO_VIDEO_TRACK,
E_MP4_UNSUPPORTED_VIDEO_CODEC,
E_MP4_MISSING_CODEC_PRIVATE,
E_PES_FRAME_TOO_LARGE,
E_PES_INVALID_MAGIC,
E_PES_TRACK_TOO_LARGE,
@@ -1323,6 +1348,12 @@ mod tests {
(Error::PipelineConsumerGone, E_PIPELINE_CONSUMER_GONE),
(Error::DiscCapacityOverflow, E_DISC_CAPACITY_OVERFLOW),
(Error::MuxEmpty, E_MUX_EMPTY),
(Error::MuxNoVideoTrack, E_MP4_NO_VIDEO_TRACK),
(
Error::Mp4UnsupportedVideoCodec,
E_MP4_UNSUPPORTED_VIDEO_CODEC,
),
(Error::MuxMissingCodecPrivate, E_MP4_MISSING_CODEC_PRIVATE),
(Error::M2tsPacketMalformed, E_M2TS_PACKET_MALFORMED),
(Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED),
(Error::DiscCapacityMalformed, E_DISC_CAPACITY_MALFORMED),
+1
View File
@@ -241,6 +241,7 @@ pub use mux::NullStream;
pub use mux::StdioStream;
pub use mux::WriteSeek;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
pub use mux::{Mp4FitReport, Mp4SkipReason, mp4_fit_report};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
+246 -53
View File
@@ -41,6 +41,11 @@ pub struct Ac3Parser {
/// the running per-frame PTS at the point the partial tail was retained.
/// Used by `flush()` to time the final buffered frame at EOS.
flush_pts_ns: i64,
/// Keep/drop bookkeeping for the CRC decodability gate. A frame that fails
/// its native CRC is dropped rather than shipped as a decoder-choking glitch;
/// the running PTS is advanced across it (see the emit loop) so the drop is a
/// silence gap, never a shift of the following audio.
tally: super::dropgate::DropTally,
}
impl Default for Ac3Parser {
@@ -54,8 +59,99 @@ impl Ac3Parser {
Self {
buf: Vec::with_capacity(4096),
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
}
}
/// Access units dropped as undecodable so far — surfaced to the CLI/mux.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Emit the final buffered frame at EOS, through the decodability gate.
/// During streaming a final frame may sit in `buf` with no following PES to
/// complete it; without this drain the last ~32 ms of audio is lost. Only a
/// fully-sized frame at a syncword is considered; a partial/garbage tail is
/// discarded, and a corrupt (CRC-failing) final frame is dropped.
fn flush_tail(&mut self) -> Vec<Frame> {
let buf = std::mem::take(&mut self.buf);
let Some(off) = find_ac3_sync(&buf) else {
return Vec::new();
};
let frame_all = &buf[off..];
if frame_all.len() < 6 {
return Vec::new();
}
let bsid = get_bsid(frame_all);
let frame_size = if bsid >= 11 {
eac3_frame_size(frame_all)
} else {
ac3_frame_size(frame_all)
};
if !(MIN_FRAME_BYTES..=8192).contains(&frame_size) || off + frame_size > buf.len() {
return Vec::new();
}
let frame = &buf[off..off + frame_size];
let duration_ns = frame_duration_ns(frame, bsid);
if let Some(reason) = ac3_drop_reason(&self.tally, frame, bsid) {
self.tally
.record_drop(self.flush_pts_ns, duration_ns as i64, frame.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.flush_pts_ns,
keyframe: true,
data: frame.to_vec(),
duration_ns: Some(duration_ns),
}]
}
}
use super::crc::crc16_ansi;
/// Whether a fully-buffered (E-)AC-3 frame passes its native CRC. ffmpeg's
/// decoder checks exactly this — `av_crc(AV_CRC_16_ANSI, 0, &buf[2],
/// frame_size - 2) == 0` (ac3dec.c) — over the frame after the 2-byte syncword;
/// the trailing crc word makes a clean frame's residue zero. A nonzero residue
/// is a ~1-in-65536-certain sign of payload corruption, so we drop the frame
/// (silence gap) rather than ship a glitch. `frame` must be exactly the frame
/// bytes (syncword .. frame_size).
fn frame_crc_ok(frame: &[u8]) -> bool {
// Need the syncword (2) plus at least one covered byte; the caller only
// invokes this on a fully-sized frame, so this is defensive.
if frame.len() < 4 {
return true;
}
crc16_ansi(&frame[2..]) == 0
}
/// Decodability verdict for a fully-sized (E-)AC-3 frame: `Some(reason)` when it
/// must be dropped, `None` when it decodes. Drops (in order): a poisoned track
/// (mostly-undecodable → drop the rest), a bitstream id ffmpeg's parser rejects
/// (`bsid > 16` → `AC3_PARSE_ERROR_BSID`), or a failed native frame CRC.
fn ac3_drop_reason(
tally: &super::dropgate::DropTally,
frame: &[u8],
bsid: u8,
) -> Option<&'static str> {
if tally.is_poisoned() {
Some("track-poisoned")
} else if bsid > 16 {
Some("bsid")
} else if !frame_crc_ok(frame) {
Some("crc")
} else {
None
}
}
impl CodecParser for Ac3Parser {
@@ -91,10 +187,12 @@ impl CodecParser for Ac3Parser {
// in practice, so this is defense-in-depth.
let base_pts_ns = pes.pts.map(pts_to_ns).unwrap_or(self.flush_pts_ns);
// Prepend leftover from previous PES
// Prepend leftover from previous PES, then take the whole buffer into a
// local so the emit loop can call `self.tally` (the bytes are no longer
// borrowed from `self`). The unconsumed tail is written back at the end.
self.buf.extend_from_slice(&pes.data);
let data = &self.buf;
let buf = std::mem::take(&mut self.buf);
let data = &buf;
let mut frames = Vec::new();
let mut pos = 0;
// Running PTS for the next frame to emit in this call.
@@ -134,15 +232,26 @@ impl CodecParser for Ac3Parser {
}
let duration_ns = frame_duration_ns(remaining, bsid);
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: frame_pts_ns,
keyframe: true,
data: data[start..start + frame_size].to_vec(),
duration_ns: Some(duration_ns),
});
let frame = &data[start..start + frame_size];
// Decodability gate: drop a frame ffmpeg's parser rejects (bsid > 16)
// or whose native CRC fails (payload corruption). `frame_pts_ns` is
// advanced BELOW whether or not the frame survives, so a drop is a
// silence gap and the following frames keep their true PTS.
if let Some(reason) = ac3_drop_reason(&self.tally, frame, bsid) {
self.tally
.record_drop(frame_pts_ns, duration_ns as i64, frame.len(), reason);
} else {
self.tally.record_kept();
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: frame_pts_ns,
keyframe: true,
data: frame.to_vec(),
duration_ns: Some(duration_ns),
});
}
frame_pts_ns += duration_ns as i64;
pos = start + frame_size;
}
@@ -198,38 +307,10 @@ impl CodecParser for Ac3Parser {
}
fn flush(&mut self) -> Vec<Frame> {
// End of stream: emit a complete final frame still buffered. During
// streaming a final frame may sit in `buf` with no following PES to
// complete/confirm it; without this drain the last ~32 ms of audio is
// dropped at EOS (mirrors dts.rs::flush). Only a fully-sized frame at a
// syncword is emitted; a partial/garbage tail is discarded.
let buf = std::mem::take(&mut self.buf);
let Some(off) = find_ac3_sync(&buf) else {
return Vec::new();
};
let frame = &buf[off..];
if frame.len() < 6 {
return Vec::new();
}
let bsid = get_bsid(frame);
let frame_size = if bsid >= 11 {
eac3_frame_size(frame)
} else {
ac3_frame_size(frame)
};
if !(MIN_FRAME_BYTES..=8192).contains(&frame_size) || off + frame_size > buf.len() {
return Vec::new();
}
let duration_ns = frame_duration_ns(frame, bsid);
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.flush_pts_ns,
keyframe: true,
data: buf[off..off + frame_size].to_vec(),
duration_ns: Some(duration_ns),
}]
let out = self.flush_tail();
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
}
fn codec_private(&self) -> Option<Vec<u8>> {
@@ -460,9 +541,24 @@ mod tests {
frame[1] = 0x77;
frame[4] = (fscod << 6) | frmsizecod;
frame[5] = 0x08 << 3; // bsid = 8 (AC-3)
finalize_ac3_crc(&mut frame);
frame
}
/// Set the trailing CRC word so the whole-frame residue over `[2..]` is zero
/// — i.e. the frame passes the decodability gate. Relies on the CRC-16/ANSI
/// residue property: appending `crc16([2..n-2])` (big-endian) zeroes the
/// register over `[2..n]`. Leaves the crc1 field (bytes 2-3) untouched.
fn finalize_ac3_crc(frame: &mut [u8]) {
let n = frame.len();
if n < 4 {
return;
}
let c = crc16_ansi(&frame[2..n - 2]);
frame[n - 2] = (c >> 8) as u8;
frame[n - 1] = (c & 0xFF) as u8;
}
#[test]
fn parse_empty_pes() {
let mut parser = Ac3Parser::new();
@@ -1090,27 +1186,33 @@ mod tests {
// --- frame acceptance / rejection at the size boundaries ---
#[test]
fn eac3_frame_at_min_frame_bytes_is_accepted() {
// The smallest acceptable (E-)AC-3 frame is MIN_FRAME_BYTES = 6.
// Build an E-AC-3 frame whose frmsiz sizes it to exactly 6 bytes
// (frmsiz=2). bsid >= 11 selects E-AC-3 sizing. The parser must emit it.
fn eac3_frame_at_min_frame_bytes_passes_sizing_then_crc_gate() {
// The smallest frame the SIZING layer accepts is MIN_FRAME_BYTES = 6
// (frmsiz=2). A synthetic all-zero 6-byte frame passes sizing (so it
// reaches the decodability gate — proven by it being COUNTED as a drop,
// not silently size-skipped) but fails the CRC gate and is dropped; the
// following real AC-3 frame (valid CRC) is emitted.
let mut parser = Ac3Parser::new();
// 0x0B 0x77 | byte2=0 byte3=2 (frmsiz=2 → 6 bytes) | byte4=0 | byte5 bsid
let mut data = vec![0x0B, 0x77, 0x00, 0x02, 0x00, 16 << 3];
// pad to exactly 6 bytes (already 6). Then a trailing real AC-3 frame so
// the 6-byte frame isn't a tail that needs more data.
data.truncate(6);
data.extend_from_slice(&make_ac3_frame(0, 2));
let f = parser.parse(&make_eac3_pes(data));
assert_eq!(f.len(), 2, "6-byte E-AC-3 frame accepted + following AC-3");
assert_eq!(f[0].data.len(), 6);
assert_eq!(f.len(), 1, "6-byte frame dropped (CRC), real AC-3 emitted");
assert_eq!(f[0].data.len(), 160, "the surviving frame is the real AC-3");
assert_eq!(
parser.dropped_frames(),
1,
"the 6-byte frame reached the gate"
);
}
#[test]
fn eac3_max_frmsiz_frame_within_window_accepted() {
// E-AC-3 frmsiz is an 11-bit field (3 bits of byte2 + 8 bits of byte3),
// so its maximum value is 0x7FF = 2047 → (2048)*2 = 4096 bytes, which is
// inside the MIN_FRAME_BYTES..=8192 accept window and must be emitted.
// inside the MIN_FRAME_BYTES..=8192 accept window and, with a valid CRC,
// must be emitted.
let mut parser = Ac3Parser::new();
let mut frame = vec![0u8; 4096];
frame[0] = 0x0B;
@@ -1118,6 +1220,7 @@ mod tests {
frame[2] = 0x07; // frmsiz high
frame[3] = 0xFF; // frmsiz low → 0x7FF = 2047 → 4096 bytes
frame[5] = 16 << 3; // bsid 16 (E-AC-3)
finalize_ac3_crc(&mut frame); // pass the decodability gate
let f = parser.parse(&make_eac3_pes(frame));
assert_eq!(f.len(), 1, "4096-byte E-AC-3 frame within window accepted");
assert_eq!(f[0].data.len(), 4096);
@@ -1296,6 +1399,96 @@ mod tests {
assert_eq!(acmod_channels(&frame), Some(2));
}
// --- decodability (CRC) gate: keep clean frames, drop corrupt ones ---
/// A structurally-valid AC-3 frame with one payload byte corrupted so its
/// native CRC fails (header/size intact, so the framer delimits it normally).
fn make_corrupt_ac3_frame(fscod: u8, frmsizecod: u8) -> Vec<u8> {
let mut f = make_ac3_frame(fscod, frmsizecod);
f[20] ^= 0xFF; // flip a payload byte → CRC no longer zero
assert!(!frame_crc_ok(&f), "corruption must break the CRC");
f
}
#[test]
fn crc16_residue_zero_after_finalize_nonzero_after_corruption() {
// The CRC-16/ANSI residue property the gate relies on: a finalized frame
// has residue 0 over [2..]; flipping any covered byte makes it nonzero.
let good = make_ac3_frame(0, 2);
assert!(frame_crc_ok(&good));
let bad = make_corrupt_ac3_frame(0, 2);
assert!(!frame_crc_ok(&bad));
}
#[test]
fn crc_fail_frame_is_dropped_survivors_kept() {
// good / corrupt / good in one PES: the corrupt middle frame is dropped
// (CRC), the two clean frames are emitted, and the drop is counted.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2);
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2));
data.extend_from_slice(&make_ac3_frame(0, 2));
let f = parser.parse(&make_eac3_pes(data));
// Only two of three survive; flush has nothing (all closed in-call).
assert_eq!(f.len(), 2, "corrupt frame dropped, two clean survive");
assert_eq!(parser.dropped_frames(), 1);
assert_eq!(
parser.dropped_duration_ns(),
32_000_000,
"one 32ms frame of silence"
);
}
#[test]
fn crc_drop_preserves_pts_sync_no_shift() {
// THE INVARIANT: dropping a corrupt frame must not shift the audio after
// it. good / corrupt / good in one PES — the corrupt frame is dropped but
// the trailing clean frame keeps the EXACT PTS it would have had with no
// drop (base + 2 frame durations): a silence gap, not a shift.
let mut parser = Ac3Parser::new();
let mut data = make_ac3_frame(0, 2); // f0
data.extend_from_slice(&make_corrupt_ac3_frame(0, 2)); // dropped
data.extend_from_slice(&make_ac3_frame(0, 2)); // f2
let f = parser.parse(&make_eac3_pes(data));
assert_eq!(f.len(), 2);
let base = pts_to_ns(90000);
let frame_dur = 32_000_000i64; // 1536 @ 48k
assert_eq!(f[0].pts_ns, base, "f0 at PES base");
assert_eq!(
f[1].pts_ns,
base + 2 * frame_dur,
"surviving frame keeps its true timeline (base + 2 frames) — gap, not shift"
);
}
#[test]
fn bsid_over_16_is_dropped() {
// ffmpeg's parser rejects bsid > 16 (AC3_PARSE_ERROR_BSID). A frame with
// bsid = 17 that still sizes must be dropped, not emitted.
let mut frame = vec![0u8; 128];
frame[0] = 0x0B;
frame[1] = 0x77;
frame[3] = 63; // frmsiz = 63 → (63+1)*2 = 128 bytes (E-AC-3 sizing)
frame[5] = 17 << 3; // bsid = 17 (> 16)
assert_eq!(get_bsid(&frame), 17);
let tally = super::super::dropgate::DropTally::new("ac3");
assert_eq!(ac3_drop_reason(&tally, &frame, 17), Some("bsid"));
}
#[test]
fn clean_stream_drops_nothing() {
// A stream of valid frames passes untouched — zero false positives.
let mut parser = Ac3Parser::new();
let mut data = Vec::new();
for _ in 0..5 {
data.extend_from_slice(&make_ac3_frame(0, 2));
}
let mut f = parser.parse(&make_eac3_pes(data));
f.extend(parser.flush());
assert_eq!(f.len(), 5);
assert_eq!(parser.dropped_frames(), 0);
}
// helper: PES with a generic pts for E-AC-3 tests
fn make_eac3_pes(data: Vec<u8>) -> PesPacket {
PesPacket {
+220
View File
@@ -0,0 +1,220 @@
//! AAC ADTS decodability gate.
//!
//! ffmpeg's `ff_adts_header_parse` (adts_header.c) has exactly three hard
//! rejects: syncword != 0xFFF, a reserved `sampling_frequency_index`
//! (`ff_mpeg4audio_sample_rates[sr] == 0`, i.e. index ≥ 13), and
//! `aac_frame_length < 7`. It does NOT verify the optional ADTS CRC (it only
//! `skip_bits(16)` past it). So the gate mirrors those three rejects: a packet
//! that begins with the ADTS sync but is otherwise malformed is dropped; a
//! packet with no ADTS sync is raw AAC (e.g. from mp4, which carries no ADTS
//! header) or a continuation and passes through unchanged — never false-dropped.
//! Raw AAC has no per-frame integrity data, so like LPCM it cannot be gated.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// `ff_mpeg4audio_sample_rates` — 13 valid entries; indices 13/14/15 are 0
/// (reserved), which is exactly what ffmpeg rejects.
const ADTS_SAMPLE_RATE_VALID: [u32; 16] = [
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350, 0, 0,
0,
];
/// ADTS header verdict for the packet head.
enum AdtsVerdict {
/// No 12-bit ADTS sync at the head — not an ADTS frame we can validate.
NoSync,
/// Sync present and the three ffmpeg-checked fields are legal.
Valid,
/// Sync present but a reserved sample-rate index or a sub-header
/// frame-length — ffmpeg's parser rejects this.
Invalid,
}
fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// Need the full 7-byte fixed+variable header to read frame_length.
if data.len() < 7 {
return AdtsVerdict::NoSync;
}
// 12-bit syncword 0xFFF: byte0 == 0xFF and top nibble of byte1 == 0xF.
if data[0] != 0xFF || (data[1] & 0xF0) != 0xF0 {
return AdtsVerdict::NoSync;
}
// sampling_frequency_index: byte2 bits 5..2.
let sr_index = ((data[2] >> 2) & 0x0F) as usize;
if ADTS_SAMPLE_RATE_VALID[sr_index] == 0 {
return AdtsVerdict::Invalid;
}
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
if frame_length < 7 {
return AdtsVerdict::Invalid;
}
AdtsVerdict::Valid
}
pub struct AdtsParser {
tally: DropTally,
}
impl Default for AdtsParser {
fn default() -> Self {
Self::new()
}
}
impl AdtsParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("aac"),
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for AdtsParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
let drop =
self.tally.is_poisoned() || matches!(adts_verdict(&pes.data), AdtsVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with
/// aac_frame_length set to the total size.
fn adts_frame(payload: usize) -> Vec<u8> {
let total = 7 + payload;
let mut f = vec![0u8; total];
f[0] = 0xFF;
f[1] = 0xF1; // sync + MPEG-4 + no CRC (protection_absent=1)
f[2] = 0x50; // profile=AAC-LC, sr_index=4 (44.1 kHz)
f[3] = 0x80; // channel_config low + start of frame_length
// frame_length (13 bits) = total.
let fl = total as u32;
f[3] = (f[3] & 0xFC) | ((fl >> 11) & 0x03) as u8;
f[4] = ((fl >> 3) & 0xFF) as u8;
f[5] = (((fl & 0x07) << 5) as u8) | 0x1F; // low 3 bits of len + buffer-fullness bits
f
}
#[test]
fn valid_adts_is_kept() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(adts_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn reserved_sample_rate_index_is_dropped() {
// sr_index = 13 (reserved). byte2 bits5..2 = 1101 → 0x34.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[2] = (f[2] & 0xC3) | (13 << 2); // set sr_index = 13
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn subheader_frame_length_is_dropped() {
// frame_length < 7 (here 0) is a sub-header length → reject.
let mut p = AdtsParser::new();
let mut f = adts_frame(400);
f[3] &= 0xFC; // clear len high bits
f[4] = 0;
f[5] &= 0x1F; // clear len low bits → frame_length = 0
assert!(p.parse(&make_pes(f, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn raw_aac_without_sync_passes_through() {
// No ADTS sync (e.g. raw AAC from mp4) → cannot validate → keep.
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(
vec![0x21, 0x00, 0x03, 0x40, 0x00, 0x00, 0x00],
Some(0),
));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = AdtsParser::new();
let mut bad = adts_frame(400);
bad[2] = (bad[2] & 0xC3) | (14 << 2); // reserved sr_index
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(adts_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
#[test]
fn short_packet_passes_through() {
let mut p = AdtsParser::new();
let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0)));
assert_eq!(f.len(), 1, "too short to validate → kept");
}
}
+110
View File
@@ -0,0 +1,110 @@
//! Bit-exact CRC helpers shared by the audio codec decodability gates.
//!
//! Both match ffmpeg's `av_crc` tables so a frame that ffmpeg's decoder would
//! flag as a CRC mismatch is flagged identically here. All are MSB-first
//! (non-reflected), init 0, no final XOR — the ffmpeg `AV_CRC_*` (big-endian)
//! variants. Each format transmits its CRC so that the residue over
//! `data + transmitted_crc` is zero, which is exactly how these are used:
//! compute over the whole frame (including its trailing CRC) and check `== 0`.
/// CRC-16/ANSI (a.k.a. CRC-16/BUYPASS): polynomial 0x8005, init 0x0000,
/// MSB-first, no reflection, no final XOR — ffmpeg `AV_CRC_16_ANSI`.
/// Used by AC-3/E-AC-3 (frame CRC), FLAC (frame footer), MPEG-audio and
/// AAC-ADTS (header CRC).
pub(crate) fn crc16_ansi(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x8005
} else {
crc << 1
};
}
}
crc
}
/// CRC-16 with polynomial 0x002D, init 0, MSB-first — ffmpeg's `crc_2D` table
/// (`av_crc_init(crc_2D, 0, 16, 0x002D)`), used by the MLP/TrueHD major-sync
/// header checksum. NOTE: MLP's checksum is the "reversed" scheme — ffmpeg
/// computes `av_crc(...) ^ AV_RL16(trailer)` and compares against `AV_RL16` of
/// the stored word; equivalently, this standard CRC compared against the stored
/// bytes read big-endian. The caller handles that comparison
/// (see `truehd::mlp_major_sync_ok`). Verified against real ffmpeg TrueHD
/// output (225/225 major-sync AUs).
pub(crate) fn crc16_mlp(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x002D
} else {
crc << 1
};
}
}
crc
}
/// CRC-8/ATM (a.k.a. CRC-8/ITU without the final XOR): polynomial 0x07, init 0,
/// MSB-first, no reflection — ffmpeg `AV_CRC_8_ATM`. Used by the FLAC frame
/// header.
pub(crate) fn crc8_atm(data: &[u8]) -> u8 {
let mut crc: u8 = 0;
for &b in data {
crc ^= b;
for _ in 0..8 {
crc = if crc & 0x80 != 0 {
(crc << 1) ^ 0x07
} else {
crc << 1
};
}
}
crc
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn crc16_residue_property_holds() {
// Appending the big-endian CRC-16 of a message zeroes the residue over
// message+crc — the property every frame gate relies on.
let msg = [0x12u8, 0x34, 0x56, 0x78, 0x9A];
let c = crc16_ansi(&msg);
let mut framed = msg.to_vec();
framed.push((c >> 8) as u8);
framed.push((c & 0xFF) as u8);
assert_eq!(crc16_ansi(&framed), 0);
}
#[test]
fn crc16_known_vector_check_bytes() {
// CRC-16/BUYPASS check value for the ASCII string "123456789" is 0xFEE8
// (the standard catalogue check value for poly 0x8005, init 0).
assert_eq!(crc16_ansi(b"123456789"), 0xFEE8);
}
#[test]
fn crc8_residue_property_holds() {
// Appending the CRC-8 of a message zeroes the residue over message+crc —
// how FLAC's header CRC-8 is verified.
let msg = [0xDEu8, 0xAD, 0xBE, 0xEF];
let c = crc8_atm(&msg);
let mut framed = msg.to_vec();
framed.push(c);
assert_eq!(crc8_atm(&framed), 0);
}
#[test]
fn crc8_known_vector_check_byte() {
// CRC-8/SMBUS (poly 0x07, init 0, no reflection) check value for
// "123456789" is 0xF4 — the catalogue check value.
assert_eq!(crc8_atm(b"123456789"), 0xF4);
}
}
+201
View File
@@ -0,0 +1,201 @@
//! Shared "keep what decodes, drop what doesn't" bookkeeping for the audio
//! codec parsers.
//!
//! The user's rule: a clean mux keeps every frame it can and drops the ones it
//! can't — video always survives (it's inter-frame predicted; a per-frame drop
//! would cascade, so video resyncs/conceals instead), audio keeps every
//! decodable access unit, and a damaged audio AU is dropped rather than shipped
//! as a decoder-choking glitch.
//!
//! The DETECTION is inherently per-codec — each format carries its own
//! authoritative corruption check (DTS: ffmpeg's core-header parse; AC-3: the
//! header CRC; FLAC: the frame CRC-16; …). This type only carries the UNIFORM
//! response so every audio parser behaves identically:
//!
//! 1. **Count** kept vs dropped AUs and the dropped duration.
//! 2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
//! once-per-track aggregate at `warn` so it surfaces without debug logging.
//! 3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
//! a poison flag so the remainder is dropped too (a track that damaged isn't
//! worth muxing).
//!
//! **Sync preservation is the caller's responsibility**, not this type's: the
//! parser must advance its PTS clock across a dropped AU exactly as it would for
//! an emitted one, so a drop becomes a silence gap and never a shift of the
//! following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
/// Minimum access units observed before the whole-track drop verdict can fire.
/// Below this, a short damaged burst can't poison an otherwise-good track.
const TRACK_VERDICT_MIN_AUS: u64 = 200;
/// Per-track drop bookkeeping shared by the audio codec parsers.
pub(crate) struct DropTally {
/// Static codec label for log lines (e.g. `"dts"`, `"ac3"`).
codec: &'static str,
kept: u64,
dropped: u64,
/// AUs dropped because they were INDIVIDUALLY verified undecodable (a failed
/// CRC/header/parity check). Only these feed the whole-track poison verdict.
/// Distinct from `dropped`, which also counts *collateral* drops — AUs
/// discarded as a consequence of one corruption (TrueHD's resync-forward run,
/// or a poisoned track), which must NOT amplify a few real errors into a
/// false whole-track loss.
verified_dropped: u64,
dropped_dur_ns: u64,
poisoned: bool,
}
impl DropTally {
pub(crate) fn new(codec: &'static str) -> Self {
Self {
codec,
kept: 0,
dropped: 0,
verified_dropped: 0,
dropped_dur_ns: 0,
poisoned: false,
}
}
/// Whether the track has been judged too damaged to mux. Once `true`, the
/// caller should drop every remaining AU (passing them to [`record_drop`]
/// with a poison reason) rather than emit them.
pub(crate) fn is_poisoned(&self) -> bool {
self.poisoned
}
/// Access units dropped as undecodable so far — surfaced to the CLI/mux.
pub(crate) fn dropped_frames(&self) -> u64 {
self.dropped
}
/// Total decoded duration (ns) of dropped AUs — the audio silence introduced.
pub(crate) fn dropped_duration_ns(&self) -> u64 {
self.dropped_dur_ns
}
/// Record an emitted (decodable) access unit.
pub(crate) fn record_kept(&mut self) {
self.kept += 1;
}
/// Record a dropped access unit that was INDIVIDUALLY verified undecodable
/// (a failed CRC/header/parity check). Counts toward the whole-track poison
/// verdict. `reason` is a short static label for the check that failed.
pub(crate) fn record_drop(&mut self, pts_ns: i64, dur_ns: i64, bytes: usize, reason: &str) {
self.verified_dropped += 1;
self.record_drop_common(pts_ns, dur_ns, bytes, reason);
self.maybe_poison();
}
/// Record a COLLATERAL drop — an AU discarded as a consequence of another
/// corruption rather than being individually undecodable (TrueHD's
/// resync-forward run to the next major sync, or an already-poisoned track).
/// Counted and logged for the drop report, but deliberately does NOT feed the
/// poison verdict, so one corruption event can't amplify into a false
/// whole-track loss.
pub(crate) fn record_collateral_drop(
&mut self,
pts_ns: i64,
dur_ns: i64,
bytes: usize,
reason: &str,
) {
self.record_drop_common(pts_ns, dur_ns, bytes, reason);
}
fn record_drop_common(&mut self, pts_ns: i64, dur_ns: i64, bytes: usize, reason: &str) {
self.dropped += 1;
self.dropped_dur_ns += dur_ns.max(0) as u64;
tracing::debug!(
target: "mux",
"{}: dropped undecodable AU #{} pts_ns={} dur_ns={} bytes={} reason={}",
self.codec,
self.dropped,
pts_ns,
dur_ns,
bytes,
reason
);
}
/// Whole-track fallback: after enough AUs to judge, if more than half were
/// dropped the track is too damaged to be worth muxing — latch `poisoned`
/// and log it loudly once. The minimum-sample gate keeps a short damaged
/// burst from poisoning an otherwise-good track.
fn maybe_poison(&mut self) {
if self.poisoned {
return;
}
// Judge on VERIFIED drops vs all AUs seen: a track is only poisoned when
// a majority of its access units are individually undecodable — not when
// a couple of corruption events forced long collateral resync runs.
let total = self.kept + self.dropped;
if total >= TRACK_VERDICT_MIN_AUS && self.verified_dropped * 2 > total {
self.poisoned = true;
tracing::warn!(
target: "mux",
"{}: track too damaged to mux — {}/{} AUs individually undecodable (>50%); dropping the whole track",
self.codec,
self.verified_dropped,
total
);
}
}
/// End-of-stream aggregate report, logged at `warn` so a track's dropped
/// audio is never hidden even without debug logging. No-op if nothing was
/// dropped.
pub(crate) fn log_summary(&self) {
if self.dropped > 0 {
tracing::warn!(
target: "mux",
"{}: dropped {} undecodable AU(s) totaling {} ns of audio ({} kept)",
self.codec,
self.dropped,
self.dropped_dur_ns,
self.kept
);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn counts_kept_and_dropped() {
let mut t = DropTally::new("test");
t.record_kept();
t.record_drop(0, 1000, 512, "bad");
t.record_kept();
assert_eq!(t.dropped_frames(), 1);
assert_eq!(t.dropped_duration_ns(), 1000);
assert!(!t.is_poisoned());
}
#[test]
fn poisons_after_min_aus_over_half_dropped() {
let mut t = DropTally::new("test");
// 199 AUs, all dropped: below the min-AU gate, must NOT poison yet.
for _ in 0..199 {
t.record_drop(0, 1000, 512, "bad");
}
assert!(!t.is_poisoned(), "below the 200-AU minimum, no verdict");
// The 200th drop reaches the minimum with >50% dropped → poison.
t.record_drop(0, 1000, 512, "bad");
assert!(t.is_poisoned());
}
#[test]
fn does_not_poison_a_mostly_good_track() {
let mut t = DropTally::new("test");
// 400 AUs, 1 dropped: nowhere near 50%.
t.record_drop(0, 1000, 512, "bad");
for _ in 0..399 {
t.record_kept();
}
assert!(!t.is_poisoned());
}
}
+535 -41
View File
@@ -49,6 +49,12 @@ pub struct DtsParser {
/// running cursor: previous emit + its duration). Only consulted when
/// `front_pts` is unchanged from `last_front_pts`. `PTS_UNSET` = no base yet.
next_pts_ns: i64,
/// Keep/drop bookkeeping for the decodability gate: counts, per-drop and
/// aggregate logging, and the whole-track poison fallback. A dropped AU is
/// NEVER emitted, but the PTS clock is still advanced across it (see
/// [`stamp_pts`] usage) so every SURVIVING AU keeps the exact timestamp it
/// would have had — a drop becomes a silence gap, never a shift.
tally: super::dropgate::DropTally,
}
impl Default for DtsParser {
@@ -65,6 +71,50 @@ impl DtsParser {
pts_marks: std::collections::VecDeque::new(),
last_front_pts: PTS_UNSET,
next_pts_ns: PTS_UNSET,
tally: super::dropgate::DropTally::new("dts"),
}
}
/// Number of access units dropped as undecodable so far. The mux/CLI reads
/// this to surface the count ("dropped N damaged DTS frames").
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of all dropped access units — the length of
/// audio silence introduced by dropping undecodable frames.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Gate an assembled access unit through the decodability check and either
/// push it or drop it. `au_pts`/`dur_ns` are already stamped on the shared
/// PTS clock (which the caller advances whether or not the AU survives), so
/// a drop leaves the following audio on its true timeline — a gap, not a
/// shift. Every drop is logged (fail-loud, never silent).
fn emit_or_drop(&mut self, au: Vec<u8>, au_pts: i64, dur_ns: i64, out: &mut Vec<Frame>) {
let verdict = if self.tally.is_poisoned() {
Err(DropReason::TrackPoisoned)
} else {
core_header_drop_reason(&au).map_or(Ok(()), Err)
};
match verdict {
Ok(()) => {
self.tally.record_kept();
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: au_pts,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
});
}
Err(reason) => {
self.tally
.record_drop(au_pts, dur_ns, au.len(), reason.as_str());
}
}
}
@@ -299,8 +349,8 @@ impl CodecParser for DtsParser {
// flush is an extension-substream PES, carrying its own later
// timestamp) must NOT become the next unit's PTS base.
let mut forced = false;
let au_end = match next_core_boundary(&self.buf, core_size) {
NextCore::Found(end) => end,
let (au_end, ext_clean) = match next_core_boundary(&self.buf, core_size) {
NextCore::Found { end, ext_clean } => (end, ext_clean),
NextCore::NeedMore => break, // candidate sync needs more header
NextCore::None => {
// No next core sync buffered yet. The trailing extension
@@ -312,27 +362,32 @@ impl CodecParser for DtsParser {
break;
}
forced = true;
self.buf.len()
(self.buf.len(), true)
}
};
let au: Vec<u8> = self.buf[..au_end].to_vec();
// Damaged source encoding: when the extension boundary was GARBAGE
// (not any DTS sync — `ext_clean == false`), the extension bytes for
// this AU are corrupt and would make the decoder cascade "DSYNC check
// failed" / "Read past end of XLL band data". Emit the clean DTS core
// ALONE (a decodable, lossy frame) and still drain past the garbage to
// the next core — a perfect mux drops the bad frame's corrupt part
// rather than shipping it. A recognized-but-unsizeable extension
// (`ext_clean == true`) is preserved in full (lossless).
let emit_end = if ext_clean { au_end } else { core_size };
let au: Vec<u8> = self.buf[..emit_end].to_vec();
// The AU's own core PES PTS (the PES covering its first byte, even if
// that PES preceded the one(s) carrying its extensions or the next
// core), stamped monotonically: honored when it advances past the
// running clock (UHD one-AU-per-PES), but never allowed to collide
// with the previous AU when several cores share ONE PES (DVD).
let dur_ns = dts_core_duration_ns(&au) as i64;
// Advance the PTS clock for this AU BEFORE the decodability gate, so
// a dropped AU still advances the timeline exactly as an emitted one
// would: the following AU keeps its true PTS and the drop is a gap,
// never a shift. `emit_or_drop` decides whether to actually push it.
let au_pts = self.stamp_pts(self.front_pts(), dur_ns);
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: au_pts,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
});
self.emit_or_drop(au, au_pts, dur_ns, &mut frames);
self.drain_front(au_end);
// After draining, the marker covering the new front (if any) carries
// the next AU's PTS; `pending_pts` is only the fallback when no
@@ -359,10 +414,23 @@ impl CodecParser for DtsParser {
}
fn flush(&mut self) -> Vec<Frame> {
// End of stream: emit the final access unit still buffered (the last
// core + its extension substreams, which had no following core sync to
// close it during streaming). Require a complete core frame; drop a
// bare partial sync tail.
let out = self.flush_tail();
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
out
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
impl DtsParser {
/// Emit the final buffered access unit (the last core + its extension
/// substreams, which had no following core sync to close it during
/// streaming), gated through the decodability check. Require a complete core
/// frame; drop a bare partial sync tail.
fn flush_tail(&mut self) -> Vec<Frame> {
if find_sync(&self.buf, &DTS_CORE_SYNC) != Some(0) || self.buf.len() < CORE_HEADER_MIN_BYTES
{
self.buf.clear();
@@ -381,19 +449,9 @@ impl CodecParser for DtsParser {
let dur_ns = dts_core_duration_ns(&au) as i64;
let pts_ns = self.stamp_pts(self.front_pts(), dur_ns);
self.pts_marks.clear();
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: au,
duration_ns: Some(dur_ns as u64),
}]
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
let mut out = Vec::new();
self.emit_or_drop(au, pts_ns, dur_ns, &mut out);
out
}
}
@@ -407,7 +465,14 @@ fn find_sync(data: &[u8], pattern: &[u8; 4]) -> Option<usize> {
/// Result of scanning for the next valid core sync that closes an access unit.
enum NextCore {
/// A valid next core sync was found; the access unit ends at this offset.
Found(usize),
/// `ext_clean` is `false` only when the byte at the extension boundary was
/// GARBAGE — neither a core sync nor a DTS-HD extension sync — meaning the
/// extension region is corrupt (damaged source encoding). The caller then
/// emits the clean DTS core alone and drops the garbage, instead of shipping
/// a corrupt AU that makes the decoder cascade DSYNC / "Read past end of XLL".
/// It stays `true` when the region is a real (if unsizeable) extension sync —
/// that path is load-bearing for valid streams and must NOT be dropped.
Found { end: usize, ext_clean: bool },
/// A candidate core sync was found but its header isn't fully buffered yet,
/// so its validity can't be decided — wait for more data.
NeedMore,
@@ -492,8 +557,11 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
pos += sz; // skip the whole extension substream precisely
}
// Couldn't size it (truncated/garbage header) — heuristic fallback.
_ => return scan_for_next_core(buf, pos),
// A real extension sync we couldn't size (truncated header /
// unsupported sub-form) — heuristic fallback, but the region IS a
// recognized extension, so keep it (ext_clean = true). This path
// is load-bearing for valid streams.
_ => return scan_for_next_core(buf, pos, true),
}
} else if buf[pos..].starts_with(&DTS_CORE_SYNC) {
// The bytes right after the precisely-skipped extensions are the next
@@ -503,13 +571,18 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
return NextCore::Found {
end: pos,
ext_clean: true,
};
}
return scan_for_next_core(buf, pos); // implausible core here — fall back
return scan_for_next_core(buf, pos, true); // implausible core — recognized sync, keep
} else {
// Neither a known extension nor a core sync at the precise boundary
// (padding / junk) — fall back to the heuristic scan.
return scan_for_next_core(buf, pos);
// GARBAGE at the extension boundary — neither a core sync nor a
// DTS-HD extension sync. This is damaged source encoding: the
// extension region is corrupt. Mark ext_clean = false so the caller
// emits the clean core alone and drops the garbage.
return scan_for_next_core(buf, pos, false);
}
}
}
@@ -518,7 +591,7 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
/// syncword whose decoded size is plausible. Used only when precise extension
/// skipping can't proceed; a chance core syncword in extension payload usually
/// decodes to an implausible size and is skipped.
fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
fn scan_for_next_core(buf: &[u8], from: usize, ext_clean: bool) -> NextCore {
let mut from = from;
while let Some(rel) = find_sync(&buf[from..], &DTS_CORE_SYNC) {
let pos = from + rel;
@@ -527,7 +600,10 @@ fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
return NextCore::Found {
end: pos,
ext_clean,
};
}
from = pos + SYNCWORD_BYTES;
}
@@ -597,6 +673,128 @@ fn dts_core_duration_ns(data: &[u8]) -> u64 {
(samples * 1_000_000_000 + rate / 2) / rate
}
/// DCA core-header constants, mirrored from ffmpeg `libavcodec/dca_core.h`.
/// `deficit_samples` must equal this (`DCA_PCMBLOCK_SAMPLES`); `npcmblocks`
/// must be a multiple of `DCA_SUBBAND_SAMPLES`; `audio_mode` must be below
/// `DCA_AMODE_COUNT`; `lfe_present == DCA_LFE_FLAG_INVALID` is rejected.
const DTS_PCMBLOCK_SAMPLES: u32 = 32;
const DTS_SUBBAND_SAMPLES: u32 = 8;
const DTS_AMODE_COUNT: u32 = 10;
const DTS_LFE_FLAG_INVALID: u32 = 3;
/// `ff_dca_sample_rates[16]` — sample rate (Hz) per core `SFREQ` code; a `0`
/// entry marks a reserved code that ffmpeg's parser rejects
/// (`DCA_PARSE_ERROR_SAMPLE_RATE`). Valid entries are locked to the spec by
/// `dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
/// {0, 4, 5, 9, 10}.
const DTS_CORE_SR_VALID: [u32; 16] = [
0, 8_000, 16_000, 32_000, 0, 0, 11_025, 22_050, 44_100, 0, 0, 12_000, 24_000, 48_000, 96_000,
192_000,
];
/// `ff_dca_bits_per_sample[8]` — a `0` entry marks a reserved `PCMR` code that
/// ffmpeg's parser rejects (`DCA_PARSE_ERROR_PCM_RES`); reserved codes are
/// {4, 7}.
const DTS_CORE_PCMR_BITS: [u8; 8] = [16, 16, 20, 20, 0, 24, 24, 0];
/// Why an access unit was judged undecodable. Each core-header variant is the
/// exact condition under which ffmpeg's `ff_dca_parse_core_frame_header` returns
/// the matching `DCA_PARSE_ERROR_*`; `TrackPoisoned` is our whole-track drop.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum DropReason {
DeficitSamples,
PcmBlocks,
FrameSize,
Amode,
SampleRate,
ReservedBit,
LfeFlag,
PcmRes,
TrackPoisoned,
}
impl DropReason {
/// Short static label for the drop log (the shared tally logs `&str`).
fn as_str(&self) -> &'static str {
match self {
DropReason::DeficitSamples => "deficit-samples",
DropReason::PcmBlocks => "pcm-blocks",
DropReason::FrameSize => "frame-size",
DropReason::Amode => "audio-mode",
DropReason::SampleRate => "sample-rate",
DropReason::ReservedBit => "reserved-bit",
DropReason::LfeFlag => "lfe-flag",
DropReason::PcmRes => "pcm-resolution",
DropReason::TrackPoisoned => "track-poisoned",
}
}
}
/// Decodability gate: a faithful port of ffmpeg's `ff_dca_parse_core_frame_header`
/// validity checks (libavcodec/dca.c). Returns `Some(reason)` when ffmpeg's own
/// parser would reject this core frame's header — in which case the packet is
/// undecodable ("Invalid data found") and dropping it loses nothing a decoder
/// could have used. Returns `None` (keep) for a decodable header OR if the
/// header can't be fully read (never false-drop on our own buffer underrun; the
/// framer only emits AUs whose core is fully buffered and ≥ 96 bytes).
///
/// The 4-byte core sync is already validated by the framer, so this reads the
/// header fields that follow it. ffmpeg's parser does NOT verify the CPF header
/// CRC (it `skip_bits(16)` past it — dca.c) and the core decoder likewise skips
/// the audio-header/side-info CRCs (dca_core.c), so no CRC check is mirrored
/// here: doing so would drop frames ffmpeg decodes fine (false positives).
fn core_header_drop_reason(au: &[u8]) -> Option<DropReason> {
let mut r = BitReader::new(au.get(SYNCWORD_BYTES..)?);
let _ftype = r.read_bit()?;
let deficit_samples = r.read_bits(5)? + 1;
if deficit_samples != DTS_PCMBLOCK_SAMPLES {
return Some(DropReason::DeficitSamples);
}
let crc_present = r.read_bit()? == 1;
let npcmblocks = r.read_bits(7)? + 1;
if npcmblocks & (DTS_SUBBAND_SAMPLES - 1) != 0 {
return Some(DropReason::PcmBlocks);
}
let frame_size = r.read_bits(14)? + 1;
if frame_size < MIN_CORE_FRAME_BYTES as u32 {
return Some(DropReason::FrameSize);
}
let audio_mode = r.read_bits(6)?;
if audio_mode >= DTS_AMODE_COUNT {
return Some(DropReason::Amode);
}
let sr_code = r.read_bits(4)? as usize;
if DTS_CORE_SR_VALID[sr_code] == 0 {
return Some(DropReason::SampleRate);
}
let _br_code = r.read_bits(5)?;
if r.read_bit()? != 0 {
return Some(DropReason::ReservedBit);
}
// drc, ts, aux, hdcd (1 each) → ext_audio_type (3) → ext_present, aspf (1 each).
r.skip_bits(4)?;
r.skip_bits(3)?;
r.skip_bits(2)?;
let lfe_present = r.read_bits(2)?;
if lfe_present == DTS_LFE_FLAG_INVALID {
return Some(DropReason::LfeFlag);
}
let _predictor_history = r.read_bit()?;
if crc_present {
// ffmpeg only skips the 16-bit header CRC here — it is not verified.
r.skip_bits(16)?;
}
let _filter_perfect = r.read_bit()?;
let _encoder_rev = r.read_bits(4)?;
let _copy_hist = r.read_bits(2)?;
let pcmr_code = r.read_bits(3)? as usize;
if DTS_CORE_PCMR_BITS[pcmr_code] == 0 {
return Some(DropReason::PcmRes);
}
None
}
#[cfg(test)]
mod tests {
use super::*;
@@ -617,6 +815,11 @@ mod tests {
let fsize = size - 1;
let mut data = vec![0u8; size];
data[0..4].copy_from_slice(&DTS_CORE_SYNC);
// byte4: FTYPE(0) SHORT(5) CPF(0) NBLKS-high(0). SHORT = 31 makes
// deficit_samples = 32 = DCA_PCMBLOCK_SAMPLES, which ffmpeg's parser
// (and our decodability gate) require of a real core frame. NBLKS high
// bit (byte4 bit0) stays 0 for NBLKS = 15.
data[4] = 31u8 << 2;
// NBLKS = 15 → (15+1)*32 = 512 samples/frame (the DVD/UHD DTS-core norm).
// NBLKS is byte4 bit0 + byte5 bits7-2; here byte4 bit0 = 0, byte5 = 15<<2.
data[5] = (15u8 << 2) | ((fsize >> 12) & 0x03) as u8;
@@ -676,12 +879,60 @@ mod tests {
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found(end) if end == core.len() + exss.len()
NextCore::Found { end, .. } if end == core.len() + exss.len()
),
"AU must end at the REAL next core (after the full EXSS), not the false sync inside it"
);
}
#[test]
fn garbage_extension_emits_core_only_but_valid_ext_is_kept() {
// Damaged source: a valid core, then GARBAGE (no core sync, no extension
// sync) where the extension belongs, then the next core. The framer must
// mark this boundary ext_clean=false and the parser must emit the clean
// 512-byte CORE alone (dropping the garbage), draining to the next core.
let core = make_dts_core(512);
let garbage = vec![0xE4, 0x3F, 0xE3, 0x90, 0xCC, 0x6C]; // real Bourne head bytes
let mut garbage = garbage;
garbage.extend(std::iter::repeat(0xAB).take(300));
let next = make_dts_core(512);
let mut buf = core.clone();
buf.extend_from_slice(&garbage);
buf.extend_from_slice(&next);
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found { end, ext_clean: false } if end == core.len() + garbage.len()
),
"garbage boundary must be flagged unclean"
);
let mut parser = DtsParser::new();
let mut frames = parser.parse(&make_pes(buf, Some(90000)));
frames.extend(parser.flush());
assert!(!frames.is_empty());
for f in &frames {
assert_eq!(f.data.len(), 512, "garbage-extension AU emits core only");
assert_eq!(&f.data[0..4], &DTS_CORE_SYNC);
}
// Contrast: a REAL extension sync (even if unsizeable) must be KEPT in
// full — ext_clean stays true, never downgraded to core-only.
let mut buf2 = make_dts_core(512);
buf2.extend_from_slice(&make_dts_ext(256));
buf2.extend_from_slice(&make_dts_core(512));
assert!(
matches!(
next_core_boundary(&buf2, 512),
NextCore::Found {
ext_clean: true,
..
}
),
"a recognized extension sync is preserved, not dropped"
);
}
#[test]
fn parse_empty_pes() {
let mut parser = DtsParser::new();
@@ -1478,4 +1729,247 @@ mod tests {
"sub-floor sync skipped, real 512 core is AU1"
);
}
/// A structurally-framed but UNDECODABLE core: a valid `make_dts_core`
/// whose reserved header bit is set. It still sizes and syncs correctly (so
/// the framer delimits it normally), but ffmpeg's `ff_dca_parse_core_frame_header`
/// — and our port — reject it (`DCA_PARSE_ERROR_RESERVED_BIT`). The reserved
/// bit is byte9 bit4 in the core header (after SYNC..RATE).
fn make_bad_dts_core(size: usize) -> Vec<u8> {
let mut d = make_dts_core(size);
assert!(
core_header_drop_reason(&d).is_none(),
"base core is decodable"
);
d[9] |= 0x10; // set the reserved bit
assert_eq!(
core_header_drop_reason(&d),
Some(DropReason::ReservedBit),
"reserved-bit core must be judged undecodable"
);
d
}
#[test]
fn valid_stream_drops_nothing() {
// A clean stream of decodable cores must pass the gate untouched — the
// detector is an exact ffmpeg-parity port, so zero false positives.
let mut parser = DtsParser::new();
let mut stream = Vec::new();
for _ in 0..5 {
stream.extend_from_slice(&make_dts_core(512));
}
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 5, "all five cores emitted");
assert_eq!(parser.dropped_frames(), 0, "nothing dropped");
assert_eq!(parser.dropped_duration_ns(), 0);
}
#[test]
fn undecodable_core_is_dropped_and_counted() {
// A single undecodable core between good ones is dropped; the survivors
// are emitted and the drop is counted.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512);
stream.extend_from_slice(&make_bad_dts_core(512));
stream.extend_from_slice(&make_dts_core(640));
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "the bad core is dropped, two survive");
assert_eq!(frames[0].data.len(), 512);
assert_eq!(frames[1].data.len(), 640);
assert_eq!(parser.dropped_frames(), 1);
assert_eq!(
parser.dropped_duration_ns(),
DTS_CORE_DUR_NS as u64,
"one frame's worth of audio silence introduced"
);
}
#[test]
fn drop_preserves_av_sync_no_shift() {
// THE INVARIANT: dropping an undecodable AU must never shift the audio
// that follows. A good/bad/good run in ONE PES — the bad middle core is
// dropped, but the trailing good core must keep the EXACT PTS it would
// have had with no drop (base + 2 frame durations), so the drop is a
// silence gap, not a shift.
let mut parser = DtsParser::new();
let mut stream = make_dts_core(512); // c1: good
stream.extend_from_slice(&make_bad_dts_core(512)); // c2: undecodable
stream.extend_from_slice(&make_dts_core(640)); // c3: good
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "c2 dropped; c1 and c3 survive");
let base = pts_to_ns(90000);
assert_eq!(frames[0].pts_ns, base, "c1 keeps the PES base PTS");
assert_eq!(
frames[1].pts_ns,
base + 2 * DTS_CORE_DUR_NS,
"c3 keeps its TRUE timeline (base + 2 frames) — the drop is a gap, not a shift"
);
// The gap between the survivors is exactly the dropped frame's duration
// beyond the normal one-frame spacing.
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
2 * DTS_CORE_DUR_NS,
"surviving AUs are spaced by the real timeline including the dropped frame's slot"
);
assert_eq!(parser.dropped_frames(), 1);
}
#[test]
fn whole_track_poison_drops_remainder() {
// A track dominated by undecodable frames is judged too damaged to mux:
// once the >50% verdict fires (after the minimum sample count), the
// whole track — including any later good frames — is dropped.
let mut parser = DtsParser::new();
let mut stream = Vec::new();
// 300 AUs, ~2/3 undecodable → well over the 50% threshold and the
// 200-AU minimum.
for i in 0..300 {
if i % 3 == 0 {
stream.extend_from_slice(&make_dts_core(512));
} else {
stream.extend_from_slice(&make_bad_dts_core(512));
}
}
// A trailing burst of GOOD cores that must be dropped once poisoned.
for _ in 0..20 {
stream.extend_from_slice(&make_dts_core(512));
}
let mut frames = parser.parse(&make_pes(stream, Some(90000)));
frames.extend(parser.flush());
assert!(
parser.tally.is_poisoned(),
"track poisoned by >50% drop rate"
);
// Once poisoned, later good cores are dropped too, so the kept count
// (kept = emitted survivors) is far below the ~120 good cores present.
let kept = frames.len() as u64;
assert!(
kept < 120,
"post-poison good frames also dropped (kept={kept})"
);
assert!(parser.dropped_frames() > 150, "majority dropped");
}
#[test]
fn sr_validity_table_marks_reserved_codes() {
// The core-header sample-rate validity table must have ZERO (reject) at
// exactly the reserved SFREQ codes {0,4,5,9,10} and a real rate
// elsewhere — this is what mirrors ffmpeg's DCA_PARSE_ERROR_SAMPLE_RATE.
for code in 0..16usize {
let reserved = matches!(code, 0 | 4 | 5 | 9 | 10);
assert_eq!(
DTS_CORE_SR_VALID[code] == 0,
reserved,
"SFREQ code {code} reserved={reserved}"
);
}
}
#[test]
fn every_core_header_error_class_is_detected() {
// Exercise each ffmpeg-parity rejection so the port stays faithful.
// Start from a decodable core and corrupt one field at a time.
let good = make_dts_core(512);
assert_eq!(core_header_drop_reason(&good), None);
// deficit_samples != 32: clear SHORT (byte4 bits6-2) → deficit = 1.
let mut d = good.clone();
d[4] &= !0x7C;
assert_eq!(
core_header_drop_reason(&d),
Some(DropReason::DeficitSamples)
);
// npcmblocks not a multiple of 8: NBLKS low bits (byte5 bits7-2) → 14
// (npcmblocks=15, 15 & 7 = 7 ≠ 0).
let mut d = good.clone();
d[5] = (d[5] & 0x03) | (14u8 << 2);
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmBlocks));
// audio_mode >= 10: AMODE = byte7 bits3-0 (high 4) + byte8 bits7-6. Set
// AMODE high nibble to 0xF → audio_mode >= 60.
let mut d = good.clone();
d[7] |= 0x0F;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::Amode));
// sample_rate reserved: SFREQ (byte8 bits5-2) = 0.
let mut d = good.clone();
d[8] &= !0x3C;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::SampleRate));
// reserved bit set: byte9 bit4.
let mut d = good.clone();
d[9] |= 0x10;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::ReservedBit));
// lfe_present == 3: LFE is byte10 bits2-1.
let mut d = good.clone();
d[10] |= 0x06;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::LfeFlag));
// pcmr_code reserved (7): pcmr is byte11 bit0 + byte12 bits7-6 → set all
// three to 1 (code 7 → ff_dca_bits_per_sample[7] = 0).
let mut d = good.clone();
d[11] |= 0x01;
d[12] |= 0xC0;
assert_eq!(core_header_drop_reason(&d), Some(DropReason::PcmRes));
}
/// Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
/// through `DtsParser` and writes the emitted access units back out, so the
/// garbage-extension → core-only drop can be validated against an actual
/// damaged stream (e.g. the extracted Bourne DTS-HD MA track) end-to-end
/// with ffmpeg. Env: `DTS_IN` (input), `DTS_OUT` (output).
/// cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
#[test]
#[ignore]
fn reparse_real_dts_file() {
use std::io::Write;
let inp = std::env::var("DTS_IN").expect("DTS_IN");
let outp = std::env::var("DTS_OUT").expect("DTS_OUT");
let bytes = std::fs::read(&inp).expect("read DTS_IN");
let mut parser = DtsParser::new();
let mut out =
std::io::BufWriter::new(std::fs::File::create(&outp).expect("create DTS_OUT"));
let mut au_count = 0usize;
let mut out_bytes = 0usize;
// 90 kHz PTS advancing per chunk; arbitrary chunking is faithful because
// the framer resyncs on core sync and buffers across PES boundaries.
let mut pts: i64 = 90_000;
const CHUNK: usize = 64 * 1024;
for chunk in bytes.chunks(CHUNK) {
let pes = PesPacket {
source: None,
pid: 0x1100,
pts: Some(pts),
dts: None,
data: chunk.to_vec(),
discontinuity: false,
};
pts += 2_100; // ~one AU worth; value irrelevant to AU framing/bytes
for f in parser.parse(&pes) {
au_count += 1;
out_bytes += f.data.len();
out.write_all(&f.data).expect("write AU");
}
}
for f in parser.flush() {
au_count += 1;
out_bytes += f.data.len();
out.write_all(&f.data).expect("write AU");
}
out.flush().expect("flush");
eprintln!(
"REPARSE in={} bytes -> out={} bytes across {} AUs ({} bytes dropped)",
bytes.len(),
out_bytes,
au_count,
bytes.len().saturating_sub(out_bytes)
);
}
}
+218
View File
@@ -0,0 +1,218 @@
//! FLAC elementary-stream decodability gate.
//!
//! FLAC frames carry no length field, so a raw stream is delimited only by
//! sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
//! it comes from mp4/mkv, where each packet is exactly one container-delimited
//! FLAC frame (the `PARSER_FLAG_COMPLETE_FRAMES` case in ffmpeg). So this parser
//! is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
//! (poly 0x8005) computed so the residue over the whole frame is zero
//! (ffmpeg `flac_decode_frame`, `av_crc(AV_CRC_16_ANSI, 0, buf, len) == 0`). A
//! nonzero residue is definitive corruption → drop the frame (a silence gap,
//! never a shift — each packet keeps its own PTS), logged via the shared tally.
//!
//! A packet that does not begin with the FLAC frame sync is not a delimited
//! frame we can validate, so it is passed through unchanged (never false-dropped).
use super::crc::crc16_ansi;
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// FLAC frame sync: 14-bit code `0x3FFE` + a mandatory-0 reserved bit; the next
/// bit (blocking strategy) is masked off. ffmpeg tests `(AV_RB16 & 0xFFFE) ==
/// 0xFFF8` (flac_parser.c).
fn has_flac_sync(data: &[u8]) -> bool {
data.len() >= 2 && ((u16::from(data[0]) << 8 | u16::from(data[1])) & 0xFFFE) == 0xFFF8
}
/// Block-size code → samples, `ff_flac_blocksize_table` (0 = reserved/explicit).
const FLAC_BLOCKSIZE_TABLE: [u32; 16] = [
0, 192, 576, 1152, 2304, 4608, 0, 0, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768,
];
/// Sample-rate code → Hz, `ff_flac_sample_rate_table` (0 = STREAMINFO/explicit).
const FLAC_SAMPLE_RATE_TABLE: [u32; 16] = [
0, 88_200, 176_400, 192_000, 8_000, 16_000, 22_050, 24_000, 32_000, 44_100, 48_000, 96_000, 0,
0, 0, 0,
];
/// Best-effort duration (ns) of a FLAC frame from its header block-size and
/// sample-rate codes (byte 2). Only the table-coded cases are resolved; the
/// explicit-in-trailing-bytes codes (block 6/7, rate 12/13/14) and
/// STREAMINFO-derived (code 0) return `None`. Used only for the dropped-audio
/// accounting, so a `None` (→ 0) is harmless.
fn flac_frame_duration_ns(frame: &[u8]) -> Option<i64> {
if frame.len() < 3 {
return None;
}
let bs_code = (frame[2] >> 4) & 0x0F;
let sr_code = frame[2] & 0x0F;
let blocksize = FLAC_BLOCKSIZE_TABLE[bs_code as usize];
let rate = FLAC_SAMPLE_RATE_TABLE[sr_code as usize];
if blocksize == 0 || rate == 0 {
return None;
}
Some((blocksize as i64 * 1_000_000_000 + rate as i64 / 2) / rate as i64)
}
pub struct FlacParser {
tally: DropTally,
}
impl Default for FlacParser {
fn default() -> Self {
Self::new()
}
}
impl FlacParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("flac"),
}
}
/// Access units dropped as undecodable so far.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for FlacParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
// Gate: a packet that begins with a FLAC frame sync but whose whole-frame
// CRC-16 residue is nonzero is corrupt → drop. Anything else passes
// through (a non-sync packet is not a frame we can validate; a poisoned
// track drops everything).
let corrupt = has_flac_sync(&pes.data) && crc16_ansi(&pes.data) != 0;
if self.tally.is_poisoned() || corrupt {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"crc"
};
let dur = flac_frame_duration_ns(&pes.data).unwrap_or(0);
self.tally.record_drop(pts_ns, dur, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A minimal FLAC-frame-shaped buffer: sync `0xFFF8`, a plausible header
/// (block code 1 = 192 samples, rate code 9 = 44.1 kHz), some payload, and a
/// trailing CRC-16 so the whole-frame residue is zero (a valid frame).
fn make_flac_frame(payload_len: usize) -> Vec<u8> {
let mut f = vec![0u8; 6 + payload_len + 2];
f[0] = 0xFF;
f[1] = 0xF8; // sync + fixed blocksize
f[2] = (1 << 4) | 9; // bs_code=1 (192), sr_code=9 (44100)
// bytes 3..end-2 arbitrary; last two bytes carry the CRC-16.
let n = f.len();
let c = crc16_ansi(&f[..n - 2]);
f[n - 2] = (c >> 8) as u8;
f[n - 1] = (c & 0xFF) as u8;
assert_eq!(crc16_ansi(&f), 0, "finalized frame has zero residue");
f
}
#[test]
fn valid_frame_is_kept() {
let mut p = FlacParser::new();
let f = p.parse(&make_pes(make_flac_frame(100), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn corrupt_frame_is_dropped() {
let mut p = FlacParser::new();
let mut frame = make_flac_frame(100);
frame[20] ^= 0xFF; // corrupt a payload byte → CRC residue nonzero
assert!(crc16_ansi(&frame) != 0);
let f = p.parse(&make_pes(frame, Some(90000)));
assert!(f.is_empty(), "corrupt FLAC frame dropped");
assert_eq!(p.dropped_frames(), 1);
// 192 samples @ 44.1 kHz ≈ 4.354 ms of silence accounted.
assert_eq!(
p.dropped_duration_ns(),
(192u64 * 1_000_000_000 + 44_100 / 2) / 44_100
);
}
#[test]
fn corrupt_drop_preserves_sync_via_own_pts() {
// Each packet carries its own PTS, so dropping one leaves the next frame
// on its true timeline — a gap, not a shift.
let mut p = FlacParser::new();
let mut bad = make_flac_frame(100);
bad[20] ^= 0xFF;
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(make_flac_frame(100), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"surviving frame keeps its own container PTS — the drop is a gap"
);
}
#[test]
fn non_flac_packet_passes_through() {
// A packet without the FLAC sync isn't a frame we can validate — never
// false-drop it.
let mut p = FlacParser::new();
let f = p.parse(&make_pes(vec![0x00, 0x01, 0x02, 0x03], Some(0)));
assert_eq!(f.len(), 1, "unrecognized packet passed through");
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn empty_pes_emits_nothing() {
let mut p = FlacParser::new();
assert!(p.parse(&make_pes(Vec::new(), Some(0))).is_empty());
}
}
+32 -3
View File
@@ -9,12 +9,19 @@
/// AC-3 / E-AC-3 (Dolby Digital / Digital Plus) elementary-stream parser.
pub mod ac3;
pub mod adts;
/// Codec-agnostic per-picture coding carrier (`PictureInfo` + accessors).
pub mod coding;
/// DTS / DTS-HD elementary-stream parser.
pub(crate) mod crc;
pub(crate) mod dropgate;
pub mod dts;
/// DVD bitmap subtitle (VobSub) parser.
pub mod dvdsub;
pub mod flac;
/// H.264 (AVC) Annex-B elementary-stream parser.
pub mod h264;
/// HEVC (H.265) Annex-B elementary-stream parser.
@@ -23,6 +30,8 @@ pub mod hevc;
pub mod lpcm;
/// MPEG-2 Video elementary-stream parser.
pub mod mpeg2;
pub mod mpegaudio;
/// HDMV PGS (Presentation Graphics Stream) subtitle parser.
pub mod pgs;
/// Display-order PTS reconstruction for sparse-PTS program-stream video.
@@ -154,6 +163,25 @@ impl CodecParser for PassthroughParser {
}
}
/// Drop-on-undecodable policy across codecs ("clean muxes always"):
///
/// - **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
/// through a per-codec corruption check and drops the ones that fail, keeping
/// A/V sync (a drop is a silence gap, never a shift) and logging every drop
/// via the shared [`dropgate::DropTally`]. DTS uses ffmpeg's core-header parse;
/// AC-3 uses its native frame CRC.
/// - **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
/// data, so a corrupt sample is indistinguishable from a quiet one — there is
/// nothing to detect, so nothing can be honestly dropped.
/// - **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
/// predicted, so dropping one frame corrupts every frame that references it
/// until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
/// ResyncGate) and lets the decoder conceal — a fundamentally different model
/// than per-frame audio dropping.
/// - TrueHD/MLP, FLAC, MP2/MP3 and AAC-ADTS also gate undecodable frames via a
/// `DropTally` (poison/drop-forward for MLP's inter-AU restart state on a
/// major-sync boundary; CRC/sync-verdict drops for the passthrough codecs).
///
/// Create the appropriate parser for a codec, with optional codec private data.
///
/// For DvdSub, `codec_data` should be the pre-formatted VobSub .idx palette header.
@@ -177,6 +205,9 @@ pub fn parser_for_codec(
Codec::Mpeg2 => Box::new(mpeg2::Mpeg2Parser::new()),
Codec::Vc1 => Box::new(vc1::Vc1Parser::new().with_ps_reorder(is_dvd_ps)),
Codec::Ac3 | Codec::Ac3Plus => Box::new(ac3::Ac3Parser::new()),
Codec::Flac => Box::new(flac::FlacParser::new()),
Codec::Mp2 | Codec::Mp3 => Box::new(mpegaudio::MpegAudioParser::new()),
Codec::Aac => Box::new(adts::AdtsParser::new()),
Codec::DtsHdMa | Codec::DtsHdHr | Codec::Dts => Box::new(dts::DtsParser::new()),
Codec::TrueHd => Box::new(truehd::TrueHdParser::new()),
Codec::Pgs => Box::new(pgs::PgsParser::new()),
@@ -200,9 +231,7 @@ pub fn parser_for_codec(
// Remaining audio-only codecs (Aac, Mp2, Mp3, Flac, Opus) where PES =
// frame: all-keyframe passthrough is correct. Subtitle/Unknown also land
// here; keyframe flag is irrelevant for them.
Codec::Aac | Codec::Mp2 | Codec::Mp3 | Codec::Flac | Codec::Opus => {
Box::new(PassthroughParser::new(true))
}
Codec::Opus => Box::new(PassthroughParser::new(true)),
Codec::Srt | Codec::Ssa | Codec::Unknown(_) => Box::new(PassthroughParser::new(true)),
}
}
+224
View File
@@ -0,0 +1,224 @@
//! MPEG-1/2/2.5 audio (MP1/MP2/MP3) decodability gate.
//!
//! ffmpeg validates MPEG-audio frames by header sanity + framing resync, not a
//! payload CRC (`mpegaudiodecheader.c` `ff_mpa_check_header`; the optional CRC
//! covers only side-info and is off by default). Its `ff_mpa_decode_header`
//! additionally rejects free-format (`bitrate_index == 0`). So the gate mirrors
//! exactly those header rejects: a packet that begins with the 11-bit MPEG-audio
//! sync but whose version / layer / bitrate-index / sample-rate fields are the
//! reserved/invalid values is undecodable → drop it (a silence gap; each packet
//! keeps its own PTS). A packet with no leading sync is not a frame we can
//! validate (raw payload / continuation), so it passes through unchanged —
//! never false-dropped.
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// Decoded validity of a candidate MPEG-audio header.
enum MpaVerdict {
/// No 11-bit sync at the packet head — not a frame we can validate.
NoSync,
/// Sync present and every field is legal — decodable.
Valid,
/// Sync present but a field is reserved/invalid (or free-format) — ffmpeg's
/// parser rejects this exactly.
Invalid,
}
/// Mirror ffmpeg's `ff_mpa_check_header` + the `ff_mpa_decode_header`
/// free-format reject. A dropped MPEG-audio frame has a corrupt header, so no
/// duration is computed (the fields it would come from are the invalid ones).
fn mpa_verdict(data: &[u8]) -> MpaVerdict {
if data.len() < 4 {
return MpaVerdict::NoSync;
}
let h = u32::from_be_bytes([data[0], data[1], data[2], data[3]]);
// 11-bit sync (0x7FF at the top).
if (h & 0xffe0_0000) != 0xffe0_0000 {
return MpaVerdict::NoSync;
}
// ff_mpa_check_header rejects: version field 01, layer field 00,
// bitrate_index 15, sample-rate field 3.
if (h & (3 << 19)) == (1 << 19)
|| (h & (3 << 17)) == 0
|| (h & (0xf << 12)) == (0xf << 12)
|| (h & (3 << 10)) == (3 << 10)
{
return MpaVerdict::Invalid;
}
// NOTE: bitrate_index == 0 (free format) is NOT rejected. It is a legal,
// decodable MPEG-audio mode (ffmpeg's ff_mpa_check_header accepts it and the
// decoder derives the frame size from the sync spacing). Dropping it would be
// a false positive on a clean stream, so it passes the gate.
MpaVerdict::Valid
}
pub struct MpegAudioParser {
tally: DropTally,
}
impl Default for MpegAudioParser {
fn default() -> Self {
Self::new()
}
}
impl MpegAudioParser {
pub fn new() -> Self {
Self {
tally: DropTally::new("mpegaudio"),
}
}
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
}
impl CodecParser for MpegAudioParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.or(pes.dts).map(pts_to_ns).unwrap_or(0);
let drop =
self.tally.is_poisoned() || matches!(mpa_verdict(&pes.data), MpaVerdict::Invalid);
if drop {
let reason = if self.tally.is_poisoned() {
"track-poisoned"
} else {
"header"
};
self.tally.record_drop(pts_ns, 0, pes.data.len(), reason);
return Vec::new();
}
self.tally.record_kept();
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
pts_ns,
keyframe: true,
data: pes.data.clone(),
duration_ns: None,
}]
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
pid: 0x1100,
pts,
dts: None,
data,
discontinuity: false,
}
}
/// A valid MPEG-1 Layer III header: sync 0xFFF, version MPEG-1 (11), layer
/// III (01), bitrate_index 9, sample-rate 0 (44.1 kHz), no CRC. Bytes:
/// 0xFF 0xFB 0x90 0x00 — the canonical MP3 frame header.
fn mp3_frame(payload: usize) -> Vec<u8> {
let mut f = vec![0xFF, 0xFB, 0x90, 0x00];
f.extend(std::iter::repeat(0xAA).take(payload));
f
}
#[test]
fn valid_header_is_kept() {
let mut p = MpegAudioParser::new();
let f = p.parse(&make_pes(mp3_frame(400), Some(90000)));
assert_eq!(f.len(), 1);
assert_eq!(f[0].pts_ns, pts_to_ns(90000));
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn reserved_sample_rate_is_dropped() {
// Sync present but sample-rate field = 3 (reserved) → ffmpeg rejects.
// 0xFF 0xFB then byte2 with bits 11..10 = 11: 0x9C.
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0x9C; // freq field = 3
let f = p.parse(&make_pes(frame, Some(90000)));
assert!(f.is_empty(), "reserved sample rate dropped");
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn reserved_layer_is_dropped() {
// Layer field 00 (reserved). byte1 bits 2..1 = 00 → 0xF9 keeps sync
// (0xFFF needs byte1 top 3 bits set) and sets layer=00.
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[1] = 0xF9; // 1111_1001: sync ok (top 3 =111), version 11, layer 00
let f = p.parse(&make_pes(frame, Some(0)));
assert!(f.is_empty(), "reserved layer dropped");
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn bad_bitrate_index_15_is_dropped() {
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0xF0; // bitrate_index = 1111
assert!(p.parse(&make_pes(frame, Some(0))).is_empty());
assert_eq!(p.dropped_frames(), 1);
}
#[test]
fn free_format_bitrate_zero_is_kept() {
// Free format (bitrate_index == 0) is legal and decodable — it must NOT
// be dropped (that would be a false positive on a clean stream).
let mut p = MpegAudioParser::new();
let mut frame = mp3_frame(400);
frame[2] = 0x00; // bitrate_index = 0000 (free format); sync/layer/rate ok
let f = p.parse(&make_pes(frame, Some(0)));
assert_eq!(f.len(), 1, "free-format frame kept");
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn non_sync_packet_passes_through() {
// No 11-bit sync → not a validatable frame → keep (conservative).
let mut p = MpegAudioParser::new();
let f = p.parse(&make_pes(vec![0x00, 0x11, 0x22, 0x33, 0x44], Some(0)));
assert_eq!(f.len(), 1);
assert_eq!(p.dropped_frames(), 0);
}
#[test]
fn drop_preserves_sync_via_own_pts() {
let mut p = MpegAudioParser::new();
let mut bad = mp3_frame(400);
bad[2] = 0x9C; // reserved sample rate
assert!(p.parse(&make_pes(bad, Some(90000))).is_empty());
let f = p.parse(&make_pes(mp3_frame(400), Some(96000)));
assert_eq!(f.len(), 1);
assert_eq!(
f[0].pts_ns,
pts_to_ns(96000),
"next frame keeps its own PTS"
);
}
}
+119
View File
@@ -30,6 +30,90 @@ const MAX_PGS_PENDING_BYTES: usize = 4 * 1024 * 1024;
// (video_w/h, frame_rate, comp_num, comp_state, palette_update,
// palette_id_ref) = 13.
const PCS_NUM_OBJECTS_OFFSET: usize = 13;
// Offset of the first composition_object's flags byte within a PCS PES payload:
// PCS header(13) + number_of_composition_objects(1) + object_id_ref(2) +
// window_id_ref(1) = 17. `forced_on_flag` is bit 0x40 of that byte (HDMV PCS).
const PCS_FIRST_OBJECT_FLAGS_OFFSET: usize = 17;
const PCS_FORCED_ON_FLAG: u8 = 0x40;
/// Whether an emitted PGS display-set frame is a FORCED subtitle — the
/// `forced_on_flag` (0x40) on its first composition object. The frame data an
/// emitted PGS block carries begins with the display PCS (segment type 0x16), so
/// the flag is read directly from it. Returns `None` when the block is not a
/// display PCS with a composition object (nothing to classify — a clear PCS, a
/// non-PCS segment, or a truncated header).
///
/// The mux uses this to detect a *forced-narrative track* (every displayed
/// subtitle forced) without relying on the disc's vendor label metadata, so
/// forced subs are flagged `FlagForced` even on discs that carry no such blob.
pub fn display_set_is_forced(frame_data: &[u8]) -> Option<bool> {
if frame_data.first() != Some(&SEGMENT_PCS) {
return None;
}
if *frame_data.get(PCS_NUM_OBJECTS_OFFSET)? == 0 {
return None; // clear PCS — no composition to classify
}
let flags = *frame_data.get(PCS_FIRST_OBJECT_FLAGS_OFFSET)?;
Some(flags & PCS_FORCED_ON_FLAG != 0)
}
/// Accumulates the "is this PGS subtitle track a forced-narrative track?" verdict
/// from its display sets. A track is forced iff it displayed at least one subtitle
/// and EVERY display set carried the forced_on_flag — a dedicated forced track,
/// as opposed to a full track that merely has occasional forced signs.
///
/// This is the SINGLE classification used by both the MKV muxer (accumulating a
/// track's frames during a rip) and the `info`-time forced probe (feeding the
/// demuxed display sets), so both reach the identical verdict.
#[derive(Debug, Clone)]
pub struct ForcedTracker {
has_display: bool,
all_forced: bool,
}
impl Default for ForcedTracker {
fn default() -> Self {
Self {
has_display: false,
all_forced: true,
}
}
}
impl ForcedTracker {
pub fn new() -> Self {
Self::default()
}
/// Fold one emitted PGS block into the verdict. Non-display blocks (clear
/// PCS, other segments) are ignored.
pub fn observe(&mut self, frame_data: &[u8]) {
if let Some(forced) = display_set_is_forced(frame_data) {
self.has_display = true;
self.all_forced &= forced;
}
}
/// Whether the track has already shown a NON-forced subtitle — i.e. its
/// verdict is settled at "not forced" and further observation can be skipped
/// (the early-exit the probe uses to avoid reading the whole clip).
pub fn settled_not_forced(&self) -> bool {
self.has_display && !self.all_forced
}
/// Whether ANY display set was observed. When false the track's forced state
/// is unknown (no PGS content seen — e.g. an undecrypted/unread stream), so a
/// probe should leave any existing (vendor-derived) flag untouched rather
/// than assert "not forced".
pub fn observed(&self) -> bool {
self.has_display
}
/// Final verdict: forced iff it displayed subtitles and every one was forced.
pub fn is_forced(&self) -> bool {
self.has_display && self.all_forced
}
}
/// Stateful parser that collapses PGS display/clear PCS pairs into
/// duration-bearing Matroska frames. Implements [`CodecParser`].
@@ -222,6 +306,41 @@ mod tests {
use super::*;
use crate::mux::ts::PesPacket;
/// A PCS display-set block with one composition object; `forced` sets the
/// forced_on_flag (0x40) in its flags byte at offset 17.
fn pcs_display(forced: bool) -> Vec<u8> {
let mut d = vec![0u8; 18];
d[0] = SEGMENT_PCS;
d[PCS_NUM_OBJECTS_OFFSET] = 1;
d[PCS_FIRST_OBJECT_FLAGS_OFFSET] = if forced { PCS_FORCED_ON_FLAG } else { 0 };
d
}
#[test]
fn display_set_forced_flag_detection() {
assert_eq!(display_set_is_forced(&pcs_display(true)), Some(true));
assert_eq!(display_set_is_forced(&pcs_display(false)), Some(false));
// Other flag bits set but not forced_on_flag → still not forced.
let mut cropped = pcs_display(false);
cropped[PCS_FIRST_OBJECT_FLAGS_OFFSET] = 0x80; // object_cropped_flag only
assert_eq!(display_set_is_forced(&cropped), Some(false));
}
#[test]
fn display_set_forced_none_for_non_display() {
// Clear PCS (0 objects) → None.
let mut clear = pcs_display(false);
clear[PCS_NUM_OBJECTS_OFFSET] = 0;
assert_eq!(display_set_is_forced(&clear), None);
// Non-PCS segment → None.
let mut ods = pcs_display(true);
ods[0] = 0x15; // ODS
assert_eq!(display_set_is_forced(&ods), None);
// Truncated (no flags byte) → None, no panic.
assert_eq!(display_set_is_forced(&pcs_display(true)[..15]), None);
assert_eq!(display_set_is_forced(&[]), None);
}
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
PesPacket {
source: None,
+562 -20
View File
@@ -12,6 +12,8 @@
//! AC-3 frames (interleaved, same PID): start with sync word 0x0B77.
//! We skip AC-3 frames and only emit TrueHD access units.
use super::crc::crc16_mlp;
use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
use crate::mux::timeline::DISCONTINUITY_BACKSTEP_NS;
@@ -43,6 +45,18 @@ pub struct TrueHdParser {
/// yet seen (head of stream) — preserving byte-identical timing for the
/// common 48 kHz case.
au_duration_ns: i64,
/// Keep/drop bookkeeping for the decodability gate.
tally: DropTally,
/// `num_substreams` from the most recent major sync — needed to size the
/// substream directory for the per-AU parity check. `None` until the first
/// major sync is seen (before which no AU can be parity-checked).
num_substreams: Option<u8>,
/// True while dropping forward to the next clean resync point. MLP/TrueHD
/// carries filter/predictor + restart state ACROSS access units, so a corrupt
/// AU cannot be excised in place — it poisons decoding until the next major
/// sync re-initialises state. On corruption we set this and drop every AU
/// until a major sync whose header CRC validates, which we then emit.
resync_pending: bool,
}
impl Default for TrueHdParser {
@@ -57,6 +71,82 @@ impl TrueHdParser {
buf: Vec::with_capacity(32768),
next_pts_ns: 0,
au_duration_ns: AU_DURATION_NS,
tally: DropTally::new("truehd"),
num_substreams: None,
resync_pending: false,
}
}
/// Access units dropped as undecodable so far.
pub fn dropped_frames(&self) -> u64 {
self.tally.dropped_frames()
}
/// Total decoded duration (ns) of dropped access units.
pub fn dropped_duration_ns(&self) -> u64 {
self.tally.dropped_duration_ns()
}
/// Decide whether an access unit is corrupt, updating `num_substreams` from a
/// valid major sync. Mirrors ffmpeg `read_access_unit`: a major sync with a
/// bad header CRC, or any AU whose header parity fails, is undecodable.
/// Returns `false` (not corrupt) when the AU is too short to judge or no
/// major sync has established `num_substreams` yet — we never drop what we
/// cannot verify. Verified against real ffmpeg TrueHD (3600/3600 AUs).
fn au_check(&mut self, au: &[u8], is_major_sync: bool) -> AuCheck {
let mut header_size = 4;
let mut format_info = None;
if is_major_sync {
let ms = &au[4..];
let Some(mshdr) = mlp_major_sync_header_size(ms) else {
// A major sync too short to hold its header can't be CRC-validated
// — NOT a safe resync/re-init point. Treat as unverifiable, not a
// clean major sync.
return AuCheck::Unverifiable;
};
if !mlp_major_sync_crc_ok(ms, mshdr) {
// A failing major-sync checksum is only a TRUSTWORTHY corruption
// signal once we already hold a validated baseline (num_substreams
// captured from a PRIOR checksum-clean major sync). Before that — at
// stream head, when no major sync has validated yet — a checksum
// mismatch is far more likely a limitation of our own major-sync
// header-size parse than real corruption. Arming the drop-forward
// there is catastrophic: every following AU is collateral-dropped
// waiting for a "validated" major sync that (if our parse is the
// problem) NEVER comes, so the WHOLE TrueHD track is silently dropped
// — the AUs are physically emitted late (empty until the mux end),
// de-interleaving the track and sending decoders into an unbounded
// memory spiral. So until we have a baseline to protect, treat a
// checksum-failed major sync as UNVERIFIABLE (keep it) rather than
// Corrupt. Once a clean major sync HAS established the baseline, a
// later failure is a real re-sync trigger and still drops forward.
if self.num_substreams.is_some() {
return AuCheck::Corrupt; // real corruption vs a proven baseline
}
return AuCheck::Unverifiable; // no baseline yet — keep, don't nuke the track
}
self.num_substreams = mlp_num_substreams(ms);
header_size += mshdr;
// The rate nibble is only trustworthy once the major sync's CRC has
// validated (above), so capture format_info here and refine the PTS
// cadence from it ONLY on this validated path.
if au.len() >= 12 {
format_info = Some(u32::from_be_bytes([au[8], au[9], au[10], au[11]]));
}
}
let Some(nss) = self.num_substreams else {
return AuCheck::Unverifiable; // no major sync seen yet — can't check parity
};
let Some(shs) = mlp_substr_header_size(au, header_size, nss) else {
return AuCheck::Unverifiable; // directory runs off the AU — can't judge
};
if !mlp_parity_ok(au, header_size, shs) {
return AuCheck::Corrupt;
}
if is_major_sync {
AuCheck::ValidMajorSync { format_info }
} else {
AuCheck::Ok
}
}
@@ -114,6 +204,23 @@ fn ac3_boundary_corroborated(buf: &[u8], frame_bytes: usize) -> bool {
next_words != 0 && next_words * 2 <= 32768
}
/// Decodability verdict for one TrueHD/MLP access unit.
enum AuCheck {
/// Verified undecodable: a major-sync header whose CRC failed, or any AU
/// whose substream-directory parity failed. Feeds the poison verdict.
Corrupt,
/// A CRC-validated major sync — a safe re-init / resync point. `format_info`
/// (AU bytes 8..12, present when the AU is long enough) is trustworthy here,
/// so the caller refines the PTS cadence ONLY from this validated path.
ValidMajorSync { format_info: Option<u32> },
/// A valid (parity-OK) non-major-sync access unit.
Ok,
/// Cannot be judged — a major sync too short to hold/CRC its header, or a
/// stream head before any major sync established `num_substreams`. Never
/// dropped on its own, and never treated as a clean resync point.
Unverifiable,
}
/// Outcome of sizing the AC-3 frame at the TrueHD buffer head.
enum Ac3Size {
/// fscod/frmsizecod don't map to a real frame size — resync, don't wait.
@@ -124,6 +231,92 @@ enum Ac3Size {
Frame(usize),
}
// --- MLP/TrueHD access-unit integrity (mirrors ffmpeg mlpdec.c / mlp_parse.c) ---
/// Major-sync header size in bytes: base 28, plus `2 + extensions*2` when the
/// extension flag (major-sync byte 25, bit 0) is set (`extensions` = byte 26
/// high nibble). `ms` is the major-sync header, i.e. AU bytes `[4..]`. `None`
/// when the AU is too short to contain the full header.
fn mlp_major_sync_header_size(ms: &[u8]) -> Option<usize> {
if ms.len() < 28 {
return None;
}
let mut size = 28;
if ms[25] & 1 != 0 {
size += 2 + ((ms[26] >> 4) as usize) * 2;
}
if ms.len() < size {
return None;
}
Some(size)
}
/// Validate the MLP/TrueHD major-sync header checksum (ffmpeg `ff_mlp_checksum16`,
/// CRC-16 poly 0x002D). The stored trailer is the last 2 header bytes; because
/// MLP's checksum is byte-reversed relative to a standard CRC, a standard CRC of
/// the header body XOR the little-endian word before the trailer must equal the
/// trailer read big-endian.
fn mlp_major_sync_crc_ok(ms: &[u8], mshdr: usize) -> bool {
if mshdr < 4 || ms.len() < mshdr {
return false;
}
// ffmpeg `ff_mlp_checksum16(buf, buf_size)` (libavcodec/mlp.c):
// av_crc(crc_2D, 0, buf, buf_size - 2) ^ AV_RL16(buf + buf_size - 2)
// is called with `buf_size = mshdr - 2` and its result compared to
// `AV_RL16(buf + mshdr - 2)` — i.e. the 16-bit checksum over `ms[..mshdr-4]`,
// XORed with the LITTLE-ENDIAN word just before the trailer, must equal the
// LITTLE-ENDIAN trailer word. `crc16_mlp` is the same crc_2D table (poly 0x2D,
// MSB-first) but yields its two bytes in the OPPOSITE order to libavutil's
// `av_crc`, so swap them back to match. (The previous code mixed endianness —
// little-endian XOR word but big-endian compare — so the checksum could never
// validate any real extended major sync, silently dropping the whole track;
// cross-verified byte-exact against real 7.1/Atmos and 5.1 discs.)
let checksum = crc16_mlp(&ms[..mshdr - 4]).swap_bytes()
^ u16::from_le_bytes([ms[mshdr - 4], ms[mshdr - 3]]);
checksum == u16::from_le_bytes([ms[mshdr - 2], ms[mshdr - 1]])
}
/// `num_substreams` from a major-sync header: it sits at bit 128 (byte 16, top
/// nibble) for both MLP (0xbb) and TrueHD (0xba) — the fields before it total
/// the same 128 bits in either layout.
fn mlp_num_substreams(ms: &[u8]) -> Option<u8> {
ms.get(16).map(|&b| b >> 4)
}
/// Size in bytes of the substream directory that follows the AU header: each of
/// the `num_substreams` entries is 2 bytes, plus 2 more when its extraword flag
/// (entry's top bit) is set. `None` if the directory runs past the AU.
fn mlp_substr_header_size(au: &[u8], header_size: usize, num_substreams: u8) -> Option<usize> {
let mut off = header_size;
let mut shs = 0;
for _ in 0..num_substreams {
if off + 2 > au.len() {
return None;
}
let extraword = au[off] & 0x80 != 0;
shs += 2;
off += 2;
if extraword {
shs += 2;
off += 2;
}
}
Some(shs)
}
/// MLP/TrueHD AU-header parity check (ffmpeg `ff_mlp_calculate_parity`): the XOR
/// of the 4-byte AU header with the substream directory, folded, must have its
/// two nibbles XOR to 0xF.
fn mlp_parity_ok(au: &[u8], header_size: usize, substr_header_size: usize) -> bool {
let end = header_size + substr_header_size;
if end > au.len() {
return false;
}
let xor_fold = |d: &[u8]| d.iter().fold(0u8, |a, &b| a ^ b);
let p = xor_fold(&au[0..4]) ^ xor_fold(&au[header_size..end]);
((p >> 4) ^ p) & 0xF == 0xF
}
impl CodecParser for TrueHdParser {
fn parse(&mut self, pes: &PesPacket) -> Vec<Frame> {
// B1: a concealed/lost gap means the buffered TrueHD AU is TRUNCATED.
@@ -256,26 +449,93 @@ impl CodecParser for TrueHdParser {
& 0xFFFF_FFFE)
== 0xF872_6FBA;
// On a major sync the 32-bit `format_info` word (immediately after
// the 4-byte sync, i.e. AU bytes 8..12) carries the rate nibble.
// Refine the per-AU PTS increment to the actual rate family. The
// 48 kHz family resolves to the unchanged 833_333 default, so the
// common case stays byte-identical; only the 44.1 kHz family shifts.
if is_major_sync && unit_bytes >= 12 {
let format_info =
u32::from_be_bytes([self.buf[8], self.buf[9], self.buf[10], self.buf[11]]);
self.au_duration_ns = truehd_au_duration_ns(format_info);
// Decodability gate. MLP/TrueHD decode state persists across access
// units, so a corrupt AU is dropped FORWARD to the next VALIDATED
// major sync (the clean re-init point) rather than excised in place.
// The PTS clock advances across every dropped AU so a drop is a
// silence gap, never a shift.
let au = self.buf[..unit_bytes].to_vec();
let pts = self.next_pts_ns;
let mut emit_keyframe: Option<bool> = None; // Some(is_keyframe) => emit
let mut drop_reason: Option<(&'static str, bool)> = None; // (reason, verified)
if self.tally.is_poisoned() {
// Whole track already judged dead — collateral drop (does not
// re-feed the poison verdict).
drop_reason = Some(("track-poisoned", false));
} else {
match self.au_check(&au, is_major_sync) {
AuCheck::ValidMajorSync { format_info } => {
// The rate nibble is trustworthy only now that the major
// sync's CRC has validated. Refine the per-AU PTS
// increment (48 kHz family stays the 833_333 default).
if let Some(fi) = format_info {
self.au_duration_ns = truehd_au_duration_ns(fi);
}
// A validated major sync is the ONLY clean resync point.
self.resync_pending = false;
emit_keyframe = Some(true);
}
AuCheck::Corrupt => {
if self.resync_pending {
// Part of the current drop-forward run — collateral.
drop_reason = Some(("resync", false));
} else {
// The trigger: one verified corruption that starts the
// drop-forward. Only this counts toward poison.
let r = if is_major_sync {
"major-sync-crc"
} else {
"parity"
};
drop_reason = Some((r, true));
self.resync_pending = true;
}
}
AuCheck::Ok => {
if self.resync_pending {
// Decode state is invalid until the next validated
// major sync, so even a parity-OK AU is undecodable
// here — collateral drop.
drop_reason = Some(("resync", false));
} else {
emit_keyframe = Some(false);
}
}
AuCheck::Unverifiable => {
if self.resync_pending {
// Not a validated major sync — do NOT clear the resync
// on it; keep dropping forward.
drop_reason = Some(("resync", false));
} else {
// Head of stream / too-short AU: keep (never drop what
// we cannot verify).
emit_keyframe = Some(is_major_sync);
}
}
}
}
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: self.next_pts_ns,
keyframe: is_major_sync,
data: self.buf[..unit_bytes].to_vec(),
duration_ns: None,
});
if let Some(keyframe) = emit_keyframe {
self.tally.record_kept();
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: pts,
keyframe,
data: au,
duration_ns: None,
});
} else if let Some((reason, verified)) = drop_reason {
if verified {
self.tally
.record_drop(pts, self.au_duration_ns, au.len(), reason);
} else {
self.tally
.record_collateral_drop(pts, self.au_duration_ns, au.len(), reason);
}
}
self.buf.drain(..unit_bytes);
self.next_pts_ns += self.au_duration_ns;
}
@@ -289,6 +549,11 @@ impl CodecParser for TrueHdParser {
frames
}
fn flush(&mut self) -> Vec<Frame> {
self.tally.log_summary();
Vec::new()
}
fn codec_private(&self) -> Option<Vec<u8>> {
None
}
@@ -451,6 +716,275 @@ mod tests {
data
}
/// Turn a synthetic major-sync AU (sync bytes already set at offset 4, any
/// `format_info` set) into one that passes the decodability gate: 1 substream,
/// a clean substream directory, a valid major-sync CRC-16, and a valid header
/// parity nibble. Mirrors what a real encoder writes (verified against real
/// ffmpeg TrueHD). The AU must be ≥ 36 bytes (4 AU header + 28 major-sync
/// header + 2 directory + slack), which every `make_truehd_unit(≥200)` is.
fn finalize_major_sync(au: &mut [u8]) {
const MSHDR: usize = 28; // no extension (byte 25 clear)
// num_substreams = 1 → major-sync byte 16 (AU[20]) top nibble.
au[20] = (au[20] & 0x0F) | 0x10;
// Substream directory entry at AU[4+MSHDR] = AU[32]: extraword flag clear.
au[32] &= 0x7F;
// Major-sync checksum, built EXACTLY as `mlp_major_sync_crc_ok` verifies it
// (ffmpeg `ff_mlp_checksum16`): swap_bytes(crc16_mlp(body)) ^ LE word before
// the trailer, stored little-endian in the trailer.
let body_end = 4 + MSHDR - 4; // AU[4..28]
let crc = super::crc16_mlp(&au[4..body_end]).swap_bytes()
^ u16::from_le_bytes([au[body_end], au[body_end + 1]]);
au[4 + MSHDR - 2] = (crc & 0xFF) as u8;
au[4 + MSHDR - 1] = (crc >> 8) as u8;
// Parity: choose the AU check nibble (AU[0] high bits) so the header +
// directory fold to 0xF. The length low nibble (AU[0] low bits) is kept.
let hi = au[0] & 0x0F;
let p0 = (hi ^ au[1] ^ au[2] ^ au[3]) ^ (au[32] ^ au[33]);
let c = ((p0 >> 4) ^ (p0 & 0x0F) ^ 0x0F) & 0x0F;
au[0] = (c << 4) | hi;
}
/// Give a synthetic NON-major-sync AU a valid header parity nibble (1
/// substream, directory at AU[4..6]), so it passes the gate once a preceding
/// major sync has established `num_substreams`.
fn finalize_normal_parity(au: &mut [u8]) {
au[4] &= 0x7F; // no extraword
let hi = au[0] & 0x0F;
let p0 = (hi ^ au[1] ^ au[2] ^ au[3]) ^ (au[4] ^ au[5]);
let c = ((p0 >> 4) ^ (p0 & 0x0F) ^ 0x0F) & 0x0F;
au[0] = (c << 4) | hi;
}
fn valid_major_sync() -> Vec<u8> {
let mut u = make_truehd_unit(200);
u[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
finalize_major_sync(&mut u);
u
}
fn valid_normal_au() -> Vec<u8> {
let mut u = make_truehd_unit(200);
finalize_normal_parity(&mut u);
u
}
#[test]
fn corrupt_major_sync_drops_forward_to_next_valid() {
// MLP state carries across AUs, so a corrupt AU is dropped FORWARD to the
// next valid major sync (the clean re-init point). Sequence: valid MS,
// corrupt MS (bad CRC), a normal AU, then a valid MS. Only the two valid
// major syncs survive; the corrupt MS and the intervening normal AU are
// dropped (the latter because decode state is poisoned until re-init).
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF; // corrupt a CRC-covered header byte
let normal = valid_normal_au(); // clean parity, but arrives mid-resync
let ms2 = valid_major_sync();
let mut data = ms1.clone();
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&normal);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "only the two valid major syncs survive");
assert!(frames[0].keyframe && frames[1].keyframe);
assert_eq!(
parser.dropped_frames(),
2,
"corrupt MS + poisoned normal AU"
);
}
#[test]
fn crc_failed_head_major_sync_is_kept_not_track_killed() {
// REGRESSION (every TrueHD title muxed after the checksum gate landed): a
// real-world major sync whose checksum our own header-size parse can't
// validate must NOT, at stream head — before ANY major sync has validated —
// arm the drop-forward. Doing so latched `resync_pending` on the very first
// AU and collateral-dropped EVERY following AU forever (no "validated" major
// sync ever came), silently dropping the entire TrueHD track: its AUs were
// emitted only at mux end, de-interleaving the track and spiralling decoders
// into unbounded memory ("decoder ran out of memory"). With no baseline to
// protect, a checksum-failed major sync is KEPT and the audio flows.
let mut parser = TrueHdParser::new();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF; // break a checksum-covered header byte → checksum fails
let mut data = ms_bad;
for _ in 0..6 {
data.extend_from_slice(&valid_normal_au());
}
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(
frames.len(),
7,
"no baseline yet: the CRC-failed head major sync + all following AUs are \
kept, not dropped (got {})",
frames.len()
);
assert_eq!(
parser.dropped_frames(),
0,
"nothing dropped without a validated baseline to protect"
);
// And the invariant still holds ONCE a baseline exists: after a genuinely
// valid major sync, a later corrupt one IS dropped (see
// `corrupt_major_sync_drops_forward_to_next_valid`).
}
#[test]
fn parity_failure_is_dropped() {
// A normal AU whose header parity is broken (after a major sync sets
// num_substreams) is undecodable → dropped.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut bad = valid_normal_au();
// A single-nibble flip: MLP's nibble-fold parity (like ffmpeg's) is blind
// to a full-byte flip, which changes both nibbles equally and cancels.
bad[2] ^= 0x01;
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&bad);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2, "the parity-broken AU is dropped");
assert_eq!(parser.dropped_frames(), 1);
}
#[test]
fn drop_forward_preserves_av_sync_no_shift() {
// THE INVARIANT: the resumed major sync keeps the exact PTS it would have
// had with no drop — base + 3 AU durations (MS1, corrupt-MS, normal, MS2)
// — so the drop is a silence gap, never a shift.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut ms_bad = valid_major_sync();
ms_bad[10] ^= 0xFF;
let normal = valid_normal_au();
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&normal);
data.extend_from_slice(&ms2);
let mut frames = parser.parse(&make_pes(data, Some(90000)));
frames.extend(parser.flush());
assert_eq!(frames.len(), 2);
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
3 * AU_DURATION_NS,
"resumed major sync keeps its true timeline (gap, not shift)"
);
}
#[test]
fn transient_corruptions_do_not_poison_whole_track() {
// Regression (audit HIGH): TrueHD drop-forward must NOT amplify a couple
// of transient errors into a false whole-track poison. Two corruptions,
// each forcing a long collateral resync run past 200 total AUs, must
// leave the track un-poisoned and keep the good audio that follows.
let mut parser = TrueHdParser::new();
let mut data = valid_major_sync();
// Corruption #1 then a long run of normal AUs (all collateral-dropped
// while resyncing — no major sync to re-init on).
let mut bad1 = valid_normal_au();
bad1[2] ^= 0x01; // single-nibble parity break
data.extend_from_slice(&bad1);
for _ in 0..210 {
data.extend_from_slice(&valid_normal_au());
}
// A valid major sync resumes; the good AUs after it MUST be kept.
data.extend_from_slice(&valid_major_sync());
for _ in 0..5 {
data.extend_from_slice(&valid_normal_au());
}
let frames = parser.parse(&make_pes(data, Some(90000)));
assert!(
!parser.tally.is_poisoned(),
"two transient errors must not poison the track"
);
// MS1 + resumed MS2 + the 5 good AUs after it survive.
assert_eq!(
frames.len(),
7,
"post-resync good audio is kept, not poisoned away"
);
assert!(
parser.dropped_frames() > 200,
"the resync run was still counted for reporting"
);
}
#[test]
fn corrupt_major_sync_rate_nibble_does_not_shift_pts() {
// Regression (audit MED): a corrupt major sync whose rate nibble decodes
// to the 44.1 kHz family must NOT refine au_duration_ns — the rate is only
// trustworthy after the CRC validates. Otherwise the resumed 48 kHz audio
// is shifted (not gapped).
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync(); // 48 kHz
let mut ms_bad = valid_major_sync();
// Set the rate nibble (top nibble of format_info = au[8]) to 0x8 (44.1k).
// au[8] is CRC-covered, so this also breaks the major-sync CRC → corrupt.
ms_bad[8] = (ms_bad[8] & 0x0F) | 0x80;
let ms2 = valid_major_sync(); // 48 kHz
let mut data = ms1;
data.extend_from_slice(&ms_bad);
data.extend_from_slice(&ms2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2, "corrupt MS dropped; MS1 and MS2 survive");
assert_eq!(
frames[1].pts_ns - frames[0].pts_ns,
2 * AU_DURATION_NS,
"resumed audio keeps the 48 kHz cadence — the corrupt MS's 44.1k rate was ignored"
);
}
#[test]
fn too_short_major_sync_does_not_clear_resync() {
// Regression (audit LOW): while resyncing, a major sync too short to hold
// (and CRC-validate) its header must NOT be treated as a clean resync
// point — the runt is dropped and only a real validated major sync resumes.
let mut parser = TrueHdParser::new();
let ms1 = valid_major_sync();
let mut bad = valid_normal_au();
bad[2] ^= 0x01; // parity break → triggers resync
// An 8-byte "major sync": length=4 words, sync at bytes 4..8, too short
// to hold the 28-byte major-sync header.
let runt = vec![0x00, 0x04, 0x00, 0x00, 0xF8, 0x72, 0x6F, 0xBA];
let ms2 = valid_major_sync();
let mut data = ms1;
data.extend_from_slice(&bad);
data.extend_from_slice(&runt);
data.extend_from_slice(&ms2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2, "the runt major sync did not resume decode");
for f in &frames {
assert_eq!(
f.data.len(),
200,
"only the real 200-byte major syncs survive"
);
}
}
#[test]
fn clean_truehd_stream_drops_nothing() {
// A run of valid AUs passes untouched — zero false positives (the CRC and
// parity are verified against real ffmpeg TrueHD output).
let mut parser = TrueHdParser::new();
let mut data = valid_major_sync();
for _ in 0..5 {
data.extend_from_slice(&valid_normal_au());
}
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 6);
assert_eq!(parser.dropped_frames(), 0);
}
fn make_ac3_frame() -> Vec<u8> {
// Minimal AC-3 frame: sync 0x0B77, fscod=0 (48kHz), frmsizecod=0 (64 words = 128 bytes)
let mut data = vec![0u8; 128];
@@ -878,6 +1412,7 @@ mod tests {
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
finalize_major_sync(&mut unit);
let f = parser.parse(&make_pes(unit, Some(90000)));
assert_eq!(f.len(), 1);
assert!(f[0].keyframe, "major-sync AU must be flagged keyframe");
@@ -899,6 +1434,7 @@ mod tests {
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBBu32.to_be_bytes());
finalize_major_sync(&mut unit);
let f = parser.parse(&make_pes(unit, Some(90000)));
assert_eq!(f.len(), 1);
assert!(f[0].keyframe, "major-sync variant 0xFB also a keyframe");
@@ -1101,8 +1637,11 @@ mod tests {
let mut a1 = make_truehd_unit(200);
a1[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes()); // major sync
a1[8..12].copy_from_slice(&format_info_with(0x8).to_be_bytes()); // 44.1 k
finalize_major_sync(&mut a1);
let mut a2 = make_truehd_unit(200);
finalize_normal_parity(&mut a2);
let mut data = a1;
data.extend_from_slice(&make_truehd_unit(200));
data.extend_from_slice(&a2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2);
assert_eq!(
@@ -1120,8 +1659,11 @@ mod tests {
let mut a1 = make_truehd_unit(200);
a1[4..8].copy_from_slice(&0xF872_6FBAu32.to_be_bytes());
a1[8..12].copy_from_slice(&format_info_with(0x0).to_be_bytes()); // 48 k
finalize_major_sync(&mut a1);
let mut a2 = make_truehd_unit(200);
finalize_normal_parity(&mut a2);
let mut data = a1;
data.extend_from_slice(&make_truehd_unit(200));
data.extend_from_slice(&a2);
let frames = parser.parse(&make_pes(data, Some(90000)));
assert_eq!(frames.len(), 2);
assert_eq!(frames[1].pts_ns - frames[0].pts_ns, 833_333);
+62 -5
View File
@@ -88,6 +88,10 @@ pub struct DemuxOptions {
pub export_chapters: bool,
/// Selected track indices. `None` = all tracks.
pub selection: Option<Vec<usize>>,
/// Restrict output to one track class. `None` = every class (plain
/// `demux://`). `Some(Audio)` is the `audio://` sink; `Some(Subtitle)` is
/// `sub://`. Filtered tracks are skipped entirely (no file written).
pub kind_filter: Option<TrackKind>,
}
impl Default for DemuxOptions {
@@ -99,13 +103,16 @@ impl Default for DemuxOptions {
chapters_fmt: ChaptersFmt::default(),
export_chapters: true,
selection: None,
kind_filter: None,
}
}
}
/// Track class, used for delay attribution and naming.
/// Track class, used for delay attribution and naming — and, via
/// [`DemuxOptions::kind_filter`], to restrict a demux to one class (the
/// `audio://` / `sub://` sinks are a `demux://` filtered to Audio / Subtitle).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum TrackKind {
pub enum TrackKind {
Video,
Audio,
Subtitle,
@@ -143,7 +150,7 @@ fn extension_for(codec: Codec) -> &'static str {
}
/// Short codec label for friendly filenames.
fn codec_label(codec: Codec) -> &'static str {
pub(crate) fn codec_label(codec: Codec) -> &'static str {
match codec {
Codec::Hevc => "HEVC",
Codec::H264 => "AVC",
@@ -541,7 +548,7 @@ fn fmt_chapter_time_ns(time_secs: f64) -> String {
}
/// Serialize chapters as mkvmerge chapter XML.
fn chapters_xml(chapters: &[Chapter]) -> String {
pub(crate) fn chapters_xml(chapters: &[Chapter]) -> String {
let mut s = String::new();
s.push_str("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
s.push_str("<!DOCTYPE Chapters SYSTEM \"matroskachapters.dtd\">\n");
@@ -571,7 +578,7 @@ fn chapters_xml(chapters: &[Chapter]) -> String {
}
/// Serialize chapters as OGM/simple chapter text.
fn chapters_ogm(chapters: &[Chapter]) -> String {
pub(crate) fn chapters_ogm(chapters: &[Chapter]) -> String {
let mut s = String::new();
for (i, c) in chapters.iter().enumerate() {
let n = i + 1;
@@ -658,6 +665,11 @@ impl DemuxSink {
(TrackKind::Subtitle, s.codec, s.pid, s.language.clone())
}
};
// Kind filter: `audio://` / `sub://` keep only their class.
if opts.kind_filter.is_some_and(|k| k != kind) {
tracks.push(None);
continue;
}
if kind == TrackKind::Video && ref_video_track.is_none() {
ref_video_track = Some(idx);
}
@@ -874,6 +886,17 @@ mod tests {
})
}
fn subtitle_stream(codec: Codec, lang: &str) -> DiscStream {
DiscStream::Subtitle(crate::disc::SubtitleStream {
pid: 0x1200,
codec,
language: lang.to_string(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
})
}
fn title_with(streams: Vec<DiscStream>, privates: Vec<Option<Vec<u8>>>) -> DiscTitle {
let mut t = DiscTitle::empty();
t.streams = streams;
@@ -882,6 +905,40 @@ mod tests {
t
}
/// `audio://` and `sub://` are `demux://` with a kind filter: only tracks of
/// the selected class get a file; every other track is skipped entirely.
#[test]
fn kind_filter_keeps_only_the_selected_class() {
let title = title_with(
vec![
video_stream(Codec::H264),
audio_stream(Codec::Ac3, "eng"),
subtitle_stream(Codec::Pgs, "eng"),
],
vec![None, None, None],
);
let sub_opts = DemuxOptions {
kind_filter: Some(TrackKind::Subtitle),
export_chapters: false,
..Default::default()
};
let sub = DemuxSink::create(&tempdir(), &title, &sub_opts).unwrap();
assert!(
sub.tracks[0].is_none() && sub.tracks[1].is_none() && sub.tracks[2].is_some(),
"sub:// keeps only the subtitle track"
);
let audio_opts = DemuxOptions {
kind_filter: Some(TrackKind::Audio),
export_chapters: false,
..Default::default()
};
let audio = DemuxSink::create(&tempdir(), &title, &audio_opts).unwrap();
assert!(
audio.tracks[0].is_none() && audio.tracks[1].is_some() && audio.tracks[2].is_none(),
"audio:// keeps only the audio track"
);
}
// ── Annex-B reframing ────────────────────────────────────────────────────
//
// The length-prefixed → Annex-B conversion and the hvcC/avcC param-set
+138 -12
View File
@@ -209,14 +209,34 @@ impl DiscStream {
/// The caller opens the source, scans for titles/keys, and passes them in.
/// The stream handles demuxing, decryption, and codec parsing internally.
pub fn new(
reader: Box<dyn SectorSource>,
mut reader: Box<dyn SectorSource>,
title: DiscTitle,
decrypt_keys: crate::decrypt::DecryptKeys,
mut decrypt_keys: crate::decrypt::DecryptKeys,
batch_sectors: u16,
content_format: crate::disc::ContentFormat,
) -> Self {
raw: bool,
halt: Option<Halt>,
) -> std::io::Result<Self> {
let mut title = title;
let extents = title.extents.clone();
// Resolve this title's CSS key from the reader if the caller supplied
// none — the SAME shared step the file-backed mux highway
// (`build_iso_pipeline`) uses, so single-pass and multi-pass descramble a
// DVD identically. No-op for AACS / already-keyed / genuinely-clear input
// or `raw`; a scrambled-but-uncrackable DVD is a hard `CssKeyMissing`.
// `halt` is passed here (not deferred to `with_halt`) so a Stop during the
// crack scan is honored — the scan runs at construction, before the caller
// can attach a token.
crate::css::resolve_dvd_title_key(
&mut *reader,
&extents,
&mut decrypt_keys,
batch_sectors,
content_format,
raw,
halt.as_ref(),
)?;
let bytes_total_extents: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
// Debug log reader type at construction — critical for diagnosing mux
@@ -301,7 +321,7 @@ impl DiscStream {
.map(|_| super::resync::ResyncGate::new())
.collect();
Self {
Ok(Self {
reader,
title,
decrypt_keys,
@@ -315,7 +335,7 @@ impl DiscStream {
errors: 0,
lost_bytes: 0,
skip_errors: false,
halt: None,
halt,
event_fn: None,
eof: false,
dropped_nav_packets: 0,
@@ -330,7 +350,7 @@ impl DiscStream {
profiling: std::env::var_os("FREEMKV_PROFILE").is_some(),
resync,
is_video,
}
})
}
/// Set event handler for sector-level events (binary search, skip, recover).
@@ -1094,7 +1114,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
let mut src: Box<dyn Stream> = Box::new(stream);
@@ -1129,7 +1152,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
crate::disc::ContentFormat::BdTs,
false,
None,
)
.unwrap()
.with_halt(halt.clone());
assert!(!stream.is_halted());
halt.cancel();
@@ -1161,7 +1187,10 @@ mod tests {
aacs,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap()
.with_key_map(std::sync::Arc::new(map));
let total: u32 = stream.extents.iter().map(|e| e.sector_count).sum();
assert_eq!(
@@ -1320,7 +1349,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
// skip_errors=false: if the recovery read did NOT succeed, fill_extents
// would return Err — so reaching EOF cleanly proves recovery worked.
stream.skip_errors = false;
@@ -1433,7 +1465,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
stream.skip_errors = true;
// Drive fill_extents across batches: the good leading sectors mux fine,
@@ -1493,7 +1528,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
stream.skip_errors = true;
let res = stream.fill_extents();
@@ -1539,7 +1577,16 @@ mod tests {
read_data_key: None,
format: crate::disc::ContentFormat::BdTs,
};
let mut stream = DiscStream::new(Box::new(reader), title, keys, 8, ContentFormat::BdTs);
let mut stream = DiscStream::new(
Box::new(reader),
title,
keys,
8,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
stream.skip_errors = true;
assert_eq!(
stream.unit_align, ALIGN as u16,
@@ -1635,7 +1682,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
false,
None,
)
.unwrap();
stream.skip_errors = true;
assert_eq!(stream.unit_align, 1, "None keys must leave unit_align=1");
@@ -1676,7 +1726,10 @@ mod tests {
crate::decrypt::DecryptKeys::None,
8,
crate::disc::ContentFormat::BdTs,
false,
None,
)
.unwrap()
.with_halt(Halt::from_arc(arc.clone()));
assert!(!stream.is_halted());
arc.store(true, std::sync::atomic::Ordering::Relaxed);
@@ -1685,4 +1738,77 @@ mod tests {
"with_halt(Halt::from_arc) must observe Arc-side flips"
);
}
/// Every read fails CSS-locked (`05/6F/03`) — a scrambled DVD whose title key
/// can't be cracked. Drives `resolve_dvd_title_key` to `ScrambledUncracked`.
struct LockedReader;
impl crate::sector::SectorSource for LockedReader {
fn read_sectors(
&mut self,
lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(2),
sense: Some(crate::scsi::ScsiSense {
sense_key: 0x05,
asc: 0x6F,
ascq: 0x03,
}),
})
}
fn capacity_sectors(&self) -> u32 {
64
}
}
fn mpegps_title(sector_count: u32) -> DiscTitle {
let mut t = synthetic_title(sector_count);
t.content_format = ContentFormat::MpegPs;
t
}
/// PARITY with `build_iso_pipeline_dvd_none_keys_scrambled_hard_fails`: the
/// live-drive single-pass constructor must ALSO hard-fail (not build a
/// scrambled-passthrough stream) for a `None`-keyed scrambled MPEG-PS DVD —
/// the exact 328k-decode-error corruption path, on the single-pass side.
#[test]
fn disc_stream_new_dvd_none_scrambled_hard_fails() {
let res = DiscStream::new(
Box::new(LockedReader),
mpegps_title(8),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::MpegPs,
false,
None,
);
assert!(
res.is_err(),
"single-pass DiscStream must hard-fail on a scrambled, keyless CSS DVD"
);
}
/// `raw` must bypass the CSS crack at the DiscStream boundary too: the same
/// scrambled-uncrackable input that hard-fails above must CONSTRUCT in raw
/// mode (ciphertext passthrough), never hard-fail.
#[test]
fn disc_stream_new_raw_bypasses_css_crack() {
let res = DiscStream::new(
Box::new(LockedReader),
mpegps_title(8),
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::MpegPs,
true, // raw
None,
);
assert!(
res.is_ok(),
"raw single-pass must construct without cracking, even on scrambled-uncrackable input"
);
}
}
+357
View File
@@ -0,0 +1,357 @@
//! `chapters://` and `json://` metadata sinks.
//!
//! Both ignore the PES stream entirely: everything they emit is already known
//! from the [`DiscTitle`] at construction, so each writes its whole file at
//! `create()` and treats every `write()` frame as a no-op. They are wired
//! through [`super::resolve::output`] like the other write-only sinks; the
//! ISO/disc scan that builds the title is all they need.
use crate::disc::{Chapter, DiscTitle, Stream as DiscStream};
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Write};
use std::path::Path;
// ── chapters:// ──────────────────────────────────────────────────────────────
/// `HH:MM:SS.mmm` for a WebVTT cue timestamp.
fn vtt_time(secs: f64) -> String {
let total_ms = (secs.max(0.0) * 1000.0).round() as u64;
let ms = total_ms % 1000;
let total_s = total_ms / 1000;
format!(
"{:02}:{:02}:{:02}.{:03}",
total_s / 3600,
(total_s / 60) % 60,
total_s % 60,
ms
)
}
/// WebVTT chapter cues (`.vtt`). Each chapter spans until the next one starts
/// (the last runs to its own start — length is unknown without the title tail).
fn chapters_vtt(chapters: &[Chapter]) -> String {
let mut s = String::from("WEBVTT\n\n");
for (i, c) in chapters.iter().enumerate() {
let start = c.time_secs.max(0.0);
let end = chapters
.get(i + 1)
.map(|n| n.time_secs.max(0.0))
.unwrap_or(start);
s.push_str(&format!(
"{}\n{} --> {}\nChapter {}\n\n",
i + 1,
vtt_time(start),
vtt_time(end),
c.name
));
}
s
}
/// Chapter content in the format the output extension selects: `.txt`/`.ogm`
/// (OGM simple), `.vtt` (WebVTT), else Matroska XML (`.xml` / default).
pub(crate) fn chapters_content(chapters: &[Chapter], ext: Option<&str>) -> String {
match ext.map(|e| e.to_ascii_lowercase()).as_deref() {
Some("txt") | Some("ogm") => super::demux_sink::chapters_ogm(chapters),
Some("vtt") => chapters_vtt(chapters),
_ => super::demux_sink::chapters_xml(chapters),
}
}
/// `chapters://` sink: writes the title's chapter markers at construction; the
/// PES stream is ignored.
pub struct ChaptersSink {
title: DiscTitle,
}
impl ChaptersSink {
pub fn create(path: &Path, title: &DiscTitle) -> io::Result<Self> {
let ext = path.extension().and_then(|e| e.to_str());
let content = chapters_content(&title.chapters, ext);
File::create(path)?.write_all(content.as_bytes())?;
Ok(Self {
title: title.clone(),
})
}
}
impl Stream for ChaptersSink {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Ok(()) // whole file written at create()
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
// ── json:// ──────────────────────────────────────────────────────────────────
/// Serialization id for an audio stream's editorial purpose.
fn purpose_id(p: crate::labels::LabelPurpose) -> &'static str {
use crate::labels::LabelPurpose::*;
match p {
Normal => "normal",
Commentary => "commentary",
Descriptive => "descriptive",
Score => "score",
Ime => "ime",
}
}
/// Serialization id for a subtitle stream's qualifier.
fn qualifier_id(q: crate::labels::LabelQualifier) -> &'static str {
use crate::labels::LabelQualifier::*;
match q {
None => "none",
Sdh => "sdh",
DescriptiveService => "descriptive_service",
Forced => "forced",
}
}
/// One stream as JSON — every field the scan resolved, nothing dropped. This is
/// the complete per-stream model (`disc::Stream`), not a summary: consumers get
/// resolution/HDR/aspect for video, channels/sample-rate/purpose for audio, and
/// the qualifier for subtitles, all in machine-readable form.
fn stream_json(s: &DiscStream) -> serde_json::Value {
use super::demux_sink::codec_label;
use serde_json::json;
match s {
DiscStream::Video(v) => {
let (w, h) = v.resolution.pixels();
let (fps_num, fps_den) = v.frame_rate.as_fraction();
let mut o = json!({
"kind": "video",
"codec": codec_label(v.codec),
"pid": v.pid,
"resolution": v.resolution.to_string(),
"width": w,
"height": h,
"interlaced": v.resolution.is_interlaced(),
"frame_rate": v.frame_rate.to_string(),
"frame_rate_num": fps_num,
"frame_rate_den": fps_den,
"hdr": v.hdr.id(),
"color_space": v.color_space.id(),
"secondary": v.secondary,
"mvc_dependent": v.is_mvc_dependent(),
});
if let Some((num, den)) = v.display_aspect {
o["display_aspect"] = json!(format!("{num}:{den}"));
}
if let Some(c) = v.measured_cicp {
o["measured_cicp"] = json!({
"matrix": c.matrix,
"transfer": c.transfer,
"primaries": c.primaries,
"range": c.range,
});
}
if !v.label.is_empty() {
o["label"] = json!(v.label);
}
o
}
DiscStream::Audio(a) => {
let mut o = json!({
"kind": "audio",
"codec": codec_label(a.codec),
"pid": a.pid,
"language": a.language,
"channels": a.channels.to_string(),
"channel_count": a.channels.count(),
"sample_rate": a.sample_rate.to_string(),
"sample_rate_hz": a.sample_rate.hz(),
"secondary": a.secondary,
"purpose": purpose_id(a.purpose),
});
if !a.label.is_empty() {
o["label"] = json!(a.label);
}
o
}
DiscStream::Subtitle(t) => json!({
"kind": "subtitle",
"codec": codec_label(t.codec),
"pid": t.pid,
"language": t.language,
"forced": t.forced,
"qualifier": qualifier_id(t.qualifier),
}),
}
}
/// The `json://` document for one title: identity, duration/size, its clips,
/// its complete stream models, and its chapter points. A stable, machine-
/// readable view of one title — the same information the scan resolved, no loss.
pub(crate) fn title_json(title: &DiscTitle) -> serde_json::Value {
use serde_json::json;
let streams: Vec<_> = title.streams.iter().map(stream_json).collect();
let clips: Vec<_> = title
.clips
.iter()
.map(|c| {
json!({
"clip_id": c.clip_id,
"duration_secs": c.duration_secs,
"source_packets": c.source_packets,
})
})
.collect();
let chapters: Vec<_> = title
.chapters
.iter()
.enumerate()
.map(|(i, c)| json!({ "n": i + 1, "start_secs": c.time_secs, "name": c.name }))
.collect();
json!({
"playlist": title.playlist,
"playlist_id": title.playlist_id,
"duration_secs": title.duration_secs,
"size_bytes": title.size_bytes,
"format": format!("{:?}", title.content_format),
"clips": clips,
"streams": streams,
"chapters": chapters,
})
}
/// `json://` sink: writes the title's structured metadata at construction; the
/// PES stream is ignored.
pub struct JsonSink {
title: DiscTitle,
}
impl JsonSink {
pub fn create(path: &Path, title: &DiscTitle) -> io::Result<Self> {
let doc =
serde_json::to_string_pretty(&title_json(title)).unwrap_or_else(|_| "{}".to_string());
let mut f = File::create(path)?;
f.write_all(doc.as_bytes())?;
f.write_all(b"\n")?;
Ok(Self {
title: title.clone(),
})
}
}
impl Stream for JsonSink {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Ok(())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::Chapter;
fn chaps() -> Vec<Chapter> {
vec![
Chapter {
time_secs: 0.0,
name: "1".into(),
},
Chapter {
time_secs: 62.5,
name: "2".into(),
},
]
}
#[test]
fn chapters_format_selected_by_extension() {
let xml = chapters_content(&chaps(), Some("xml"));
assert!(xml.contains("<Chapters>"), "xml chosen for .xml");
let ogm = chapters_content(&chaps(), Some("txt"));
assert!(ogm.contains("CHAPTER01="), "ogm chosen for .txt");
let vtt = chapters_content(&chaps(), Some("vtt"));
assert!(
vtt.starts_with("WEBVTT") && vtt.contains("00:01:02.500"),
"vtt chosen for .vtt, with cue timing"
);
// Unknown / missing extension defaults to XML.
assert!(chapters_content(&chaps(), None).contains("<Chapters>"));
}
#[test]
fn title_json_carries_streams_and_chapters() {
use crate::disc::{AudioChannels, AudioStream, Codec, DiscTitle};
use crate::disc::{LabelPurpose, SampleRate, Stream as DiscStream};
let mut t = DiscTitle::empty();
t.playlist = "MAIN".into();
t.chapters = chaps();
t.streams = vec![DiscStream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Stereo,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
})];
let v = title_json(&t);
assert_eq!(v["playlist"], "MAIN");
let a = &v["streams"][0];
assert_eq!(a["kind"], "audio");
assert_eq!(a["codec"], "TrueHD");
assert_eq!(a["language"], "eng");
// Completeness: audio carries channels + sample rate + purpose, not just codec.
assert_eq!(a["channels"], "stereo");
assert_eq!(a["channel_count"], 2);
assert_eq!(a["sample_rate"], "48kHz");
assert_eq!(a["sample_rate_hz"], 48000.0);
assert_eq!(a["purpose"], "normal");
assert_eq!(v["chapters"][1]["n"], 2);
assert_eq!(v["chapters"][1]["start_secs"], 62.5);
assert_eq!(v["chapters"][1]["name"], "2");
}
#[test]
fn video_json_carries_resolution_and_hdr() {
use crate::disc::Codec;
use crate::disc::{
ColorSpace, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStream,
VideoStream,
};
let mut t = DiscTitle::empty();
t.streams = vec![DiscStream::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R2160p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
})];
let vid = &title_json(&t)["streams"][0];
assert_eq!(vid["kind"], "video");
assert_eq!(vid["resolution"], "2160p");
assert_eq!(vid["width"], 3840);
assert_eq!(vid["height"], 2160);
assert_eq!(vid["frame_rate"], "23.976");
assert_eq!(vid["frame_rate_num"], 24000);
assert_eq!(vid["hdr"], "hdr10");
assert_eq!(vid["color_space"], "bt2020");
}
}
+179 -5
View File
@@ -672,6 +672,14 @@ pub struct MkvMuxer<W: Write + Seek> {
/// (to patch in place) and the IFO-claimed count (to warn on disagreement);
/// `corrected` flips once patched so we only act on the first frame.
ac3_channel_fixups: std::collections::HashMap<usize, Ac3ChannelFixup>,
/// Deferred PGS forced-subtitle detection. A PGS subtitle track reserves a
/// 1-byte `FlagForced` value up-front; as its display sets are written, the
/// track is judged forced iff it displayed at least one subtitle and EVERY
/// display set carried the HDMV `forced_on_flag` (a dedicated forced/narrative
/// track). At `finish()` the reserved byte is promoted to 1 for such tracks —
/// so forced subs are flagged even on discs without vendor label metadata.
/// Only ever promotes (0→1); a scan/vendor forced flag is never demoted.
pgs_forced_fixups: std::collections::HashMap<usize, PgsForcedFixup>,
/// `--log-level 3` opening-frame capture: the first ~100 coded frames per
/// track are written (raw) to a `<output>.opening.bin` side file with a
/// per-frame summary logged, so an opening-GOP / menu / mid-GOP-open issue is
@@ -692,6 +700,15 @@ struct Ac3ChannelFixup {
corrected: bool,
}
/// Deferred PGS forced-subtitle detection state for one PGS subtitle track.
struct PgsForcedFixup {
/// Absolute file offset of the 1-byte `FlagForced` value in the Tracks element.
value_offset: u64,
/// Shared forced-narrative classifier fed the track's display sets. The same
/// type drives the `info`-time forced probe, so both classify identically.
tracker: super::codec::pgs::ForcedTracker,
}
/// TimestampScale: nanoseconds per Matroska timestamp tick. 0.1 ms (100_000 ns).
///
/// The classic 1 ms scale truncates two distinct cadences onto the same tick:
@@ -917,6 +934,8 @@ impl<W: Write + Seek> MkvMuxer<W> {
let mut track_uids: Vec<u64> = Vec::with_capacity(tracks.len());
let mut ac3_channel_fixups: std::collections::HashMap<usize, Ac3ChannelFixup> =
std::collections::HashMap::new();
let mut pgs_forced_fixups: std::collections::HashMap<usize, PgsForcedFixup> =
std::collections::HashMap::new();
// Per track: whether it emitted a conforming `mvcC` BlockAdditionMapping.
// Filled below from the SAME built record that drives the CodecPrivate
// mvcC extension, so the three MVC signals never diverge.
@@ -948,7 +967,24 @@ impl<W: Write + Seek> MkvMuxer<W> {
if !track.is_default {
ebml::write_uint(&mut writer, ebml::FLAG_DEFAULT, 0)?;
}
if track.is_forced {
if track.track_type == ebml::TRACK_TYPE_SUBTITLE && track.codec_id == ebml::CODEC_PGS {
// Reserve a 1-byte FlagForced (initial value = the scan/vendor
// flag) and record its offset, so PGS content can promote it to 1
// at finish() if the track proves to be forced narrative. Written
// explicitly (ID + size + value) so the value is a single,
// in-place-patchable byte — same idiom as the Channels fixup.
ebml::write_id(&mut writer, ebml::FLAG_FORCED)?;
ebml::write_size(&mut writer, 1)?;
let value_offset = writer.stream_position()?;
writer.write_all(&[track.is_forced as u8])?;
pgs_forced_fixups.insert(
i,
PgsForcedFixup {
value_offset,
tracker: super::codec::pgs::ForcedTracker::new(),
},
);
} else if track.is_forced {
ebml::write_uint(&mut writer, ebml::FLAG_FORCED, 1)?;
}
@@ -1203,6 +1239,7 @@ impl<W: Write + Seek> MkvMuxer<W> {
max_block_ticks: 0,
last_video_keyframe_ticks: None,
ac3_channel_fixups,
pgs_forced_fixups,
opening_capture: None,
})
}
@@ -1514,6 +1551,14 @@ impl<W: Write + Seek> MkvMuxer<W> {
}
}
// Accumulate PGS forced-subtitle state: a display set marks the track as
// having shown a subtitle, and clears `all_forced` the moment a
// non-forced set appears. `finish()` promotes FlagForced only for a track
// that displayed subtitles and had every one forced.
if let Some(fixup) = self.pgs_forced_fixups.get_mut(&track_idx) {
fixup.tracker.observe(data);
}
Ok(())
}
@@ -1546,6 +1591,26 @@ impl<W: Write + Seek> MkvMuxer<W> {
// Close final cluster
self.end_cluster()?;
// Promote FlagForced for PGS subtitle tracks that proved to be forced
// narrative (displayed subtitles, every one forced). In-place single-byte
// rewrite of the reserved value, then restore the append position for the
// Cues that follow. Only promotes (0→1); a track already forced from the
// scan/vendor flag stays forced.
let forced_offsets: Vec<u64> = self
.pgs_forced_fixups
.values()
.filter(|f| f.tracker.is_forced())
.map(|f| f.value_offset)
.collect();
if !forced_offsets.is_empty() {
let here = self.writer.stream_position()?;
for off in forced_offsets {
self.writer.seek(std::io::SeekFrom::Start(off))?;
self.writer.write_all(&[1u8])?;
}
self.writer.seek(std::io::SeekFrom::Start(here))?;
}
// Write Cues
let cues_start = self.writer.stream_position()?;
let cues_offset = cues_start - self.segment_start;
@@ -2600,13 +2665,122 @@ mod tests {
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// FlagForced should NOT be written for non-forced tracks
assert!(
find_id(&data, ebml::FLAG_FORCED).is_none(),
"FlagForced element should not be present for non-forced subtitle"
// A PGS subtitle track now RESERVES a FlagForced element (so PGS content
// can promote it later), but for a non-forced track its value is 0. The
// value byte sits after the 2-byte ID (0x55AA) + 1-byte size.
let pos =
find_id(&data, ebml::FLAG_FORCED).expect("PGS subtitle reserves a FlagForced element");
assert_eq!(
data[pos + 3],
0,
"reserved FlagForced value must be 0 for a non-forced subtitle"
);
}
/// A PGS display-set PES block (PCS) with one composition object, whose
/// forced_on_flag is set per `forced`.
#[cfg(test)]
fn pgs_display_set(forced: bool) -> Vec<u8> {
let mut pcs = vec![0u8; 18];
pcs[0] = 0x16; // segment type PCS
pcs[13] = 1; // number_of_composition_objects
pcs[17] = if forced { 0x40 } else { 0x00 }; // first object flags
pcs
}
#[test]
fn mkv_pgs_forced_promoted_when_all_display_sets_forced() {
// End-to-end: a PGS subtitle track whose every display set is forced is
// promoted to FlagForced=1 at finish(), even though the scan flag was
// false (no vendor metadata).
use crate::disc::SubtitleStream;
use std::sync::{Arc, Mutex};
let shared = Arc::new(Mutex::new(Cursor::new(Vec::new())));
let sub = MkvTrack::subtitle(&SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
});
let tracks = [make_video_track(), sub];
let mut muxer =
MkvMuxer::new(SharedWriter(shared.clone()), &tracks, None, 60.0, &[]).unwrap();
muxer
.write_frame(0, 0, true, &[0u8; 16], Some(40_000_000), None)
.unwrap(); // video keyframe opens a cluster
muxer
.write_frame(
1,
1_000_000,
true,
&pgs_display_set(true),
Some(2_000_000),
None,
)
.unwrap();
muxer.finish().unwrap();
let data = shared.lock().unwrap().clone().into_inner();
let pos = find_id(&data, ebml::FLAG_FORCED).expect("reserved FlagForced");
assert_eq!(
data[pos + 3],
1,
"all-forced PGS track promoted to FlagForced=1"
);
}
#[test]
fn mkv_pgs_not_promoted_when_a_display_set_is_not_forced() {
// A track with ANY non-forced display set is a full track, not forced
// narrative — FlagForced stays 0.
use crate::disc::SubtitleStream;
use std::sync::{Arc, Mutex};
let shared = Arc::new(Mutex::new(Cursor::new(Vec::new())));
let sub = MkvTrack::subtitle(&SubtitleStream {
pid: 0x1200,
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
});
let tracks = [make_video_track(), sub];
let mut muxer =
MkvMuxer::new(SharedWriter(shared.clone()), &tracks, None, 60.0, &[]).unwrap();
muxer
.write_frame(0, 0, true, &[0u8; 16], Some(40_000_000), None)
.unwrap();
muxer
.write_frame(
1,
1_000_000,
true,
&pgs_display_set(true),
Some(2_000_000),
None,
)
.unwrap();
muxer
.write_frame(
1,
3_000_000,
true,
&pgs_display_set(false),
Some(4_000_000),
None,
)
.unwrap(); // a normal (non-forced) subtitle → not a forced track
muxer.finish().unwrap();
let data = shared.lock().unwrap().clone().into_inner();
let pos = find_id(&data, ebml::FLAG_FORCED).expect("reserved FlagForced");
assert_eq!(data[pos + 3], 0, "mixed PGS track stays FlagForced=0");
}
// ============================================================
// Seekability tests: SeekHead, keyframe-aligned clusters, Cues
// ============================================================
+9
View File
@@ -62,6 +62,7 @@ pub(crate) mod m2ts;
/// to round-trip codec_privates that don't fit inside the underlying format).
/// Exposed for integration tests that exercise the wire format directly.
pub mod meta;
pub(crate) mod meta_sink;
// ── Sequential-sink muxers ──────────────────────────────────────────────────
//
@@ -89,6 +90,7 @@ pub(crate) mod hevc;
pub(crate) mod m2ts_mux;
pub(crate) mod mkv;
pub(crate) mod mkvstream;
pub(crate) mod mp4;
pub(crate) mod network;
pub(crate) mod null;
pub(crate) mod ps;
@@ -114,6 +116,7 @@ pub(crate) mod videomap;
pub use disc::DiscStream;
pub use m2ts::M2tsStream;
pub use mkvstream::MkvStream;
pub use mp4::{Mp4FitReport, Mp4SkipReason, fit_report as mp4_fit_report};
pub use network::NetworkStream;
pub use null::NullStream;
pub use pipelined_stream::PipelinedPesStream;
@@ -153,11 +156,17 @@ mod tests {
assert_eq!(parse_url("disc://").scheme(), "disc");
assert_eq!(parse_url("m2ts://f").scheme(), "m2ts");
assert_eq!(parse_url("mkv://f").scheme(), "mkv");
assert_eq!(parse_url("mp4://f").scheme(), "mp4");
assert_eq!(parse_url("network://h:1").scheme(), "network");
assert_eq!(parse_url("stdio://").scheme(), "stdio");
assert_eq!(parse_url("iso://f").scheme(), "iso");
assert_eq!(parse_url("null://").scheme(), "null");
assert_eq!(parse_url("demux://out/").scheme(), "demux");
assert_eq!(parse_url("video://out/").scheme(), "video");
assert_eq!(parse_url("audio://out/").scheme(), "audio");
assert_eq!(parse_url("sub://out/").scheme(), "sub");
assert_eq!(parse_url("chapters://c.xml").scheme(), "chapters");
assert_eq!(parse_url("json://t.json").scheme(), "json");
assert_eq!(parse_url("bogus://x").scheme(), "unknown");
}
+526
View File
@@ -0,0 +1,526 @@
//! MP4 audio sample entries and codec-config boxes for the `mp4://` muxer.
//!
//! Covers the codecs that map cleanly into MP4 and play widely: **AC-3**
//! (`ac-3` + `dac3`), **E-AC-3 / Dolby Digital Plus** (`ec-3` + `dec3`, incl.
//! Atmos-in-DD+ JOC), and **DTS / DTS-HD** (`dtsc`/`dtsh` + `ddts`, describing
//! the core with whole access units passed through so an HD decoder finds the
//! extension). Config boxes are derived from the first audio frame's bitstream
//! (ISO/IEC 14496-12 amendments; ETSI TS 102 366 / 102 114). Codecs with no
//! clean MP4 mapping (TrueHD, LPCM, bitmap subtitles) are excluded by the fit
//! oracle in the sink.
use super::boxes::bx;
use crate::disc::Codec;
/// AC-3 / E-AC-3 sample rates indexed by `fscod` (byte-4 bits 7-6).
const FSCOD_RATES: [u32; 3] = [48_000, 44_100, 32_000];
/// E-AC-3 reduced rates indexed by `fscod2` (byte-4 bits 5-4) when `fscod == 3`.
const EAC3_REDUCED_RATES: [u32; 4] = [24_000, 22_050, 16_000, 48_000];
/// Base channel count per `acmod` (A/52 Table 5.8), before the LFE.
const ACMOD_CHANNELS: [u8; 8] = [2, 1, 2, 3, 3, 4, 4, 5];
/// A big-endian MSB-first bit reader over a byte slice.
struct BitReader<'a> {
data: &'a [u8],
bit: usize,
}
impl<'a> BitReader<'a> {
fn new(data: &'a [u8]) -> Self {
Self { data, bit: 0 }
}
fn skip(&mut self, n: usize) {
self.bit += n;
}
/// Read `n` bits (n ≤ 32). Returns 0 past end of data (callers pre-check len).
fn read(&mut self, n: usize) -> u32 {
let mut v = 0u32;
for _ in 0..n {
let byte = self.data.get(self.bit / 8).copied().unwrap_or(0);
let shift = 7 - (self.bit % 8);
v = (v << 1) | ((byte >> shift) & 1) as u32;
self.bit += 1;
}
v
}
}
/// Decoded (E-)AC-3 stream parameters needed for the `dac3`/`dec3` config box
/// and the audio sample entry.
pub(super) struct DolbyConfig {
pub fscod: u8,
pub bsid: u8,
pub bsmod: u8,
pub acmod: u8,
pub lfeon: bool,
/// AC-3 only: `bit_rate_code` (= `frmsizecod >> 1`). Unused for E-AC-3.
pub bit_rate_code: u8,
/// E-AC-3 only: nominal data rate in kbps (for `dec3`). 0 for AC-3.
pub data_rate_kbps: u16,
pub sample_rate: u32,
pub channels: u16,
}
impl DolbyConfig {
fn channel_count(acmod: u8, lfeon: bool) -> u16 {
ACMOD_CHANNELS[acmod as usize] as u16 + lfeon as u16
}
}
/// Parse the first (E-)AC-3 frame starting at the 0x0B77 syncword. Returns
/// `None` if the frame is too short or the syncword is absent.
pub(super) fn parse_dolby(frame: &[u8]) -> Option<DolbyConfig> {
let start =
(0..frame.len().saturating_sub(1)).find(|&i| frame[i] == 0x0B && frame[i + 1] == 0x77)?;
let f = &frame[start..];
if f.len() < 6 {
return None;
}
// bsid lives in byte 5 bits 7-3 for both AC-3 and E-AC-3.
let bsid = (f[5] >> 3) & 0x1F;
if bsid >= 11 {
parse_eac3(f)
} else {
parse_ac3(f)
}
}
/// Legacy AC-3 (A/52 §5.3.2): syncword | crc(16) | fscod(2) frmsizecod(6) |
/// bsid(5) bsmod(3) | acmod(3) …optional… lfeon.
fn parse_ac3(f: &[u8]) -> Option<DolbyConfig> {
if f.len() < 8 {
return None;
}
let fscod = (f[4] >> 6) & 0x03;
let frmsizecod = f[4] & 0x3F;
let bsid = (f[5] >> 3) & 0x1F;
let bsmod = f[5] & 0x07;
// acmod + trailing optional 2-bit fields, then lfeon (byte 6 onward).
let mut r = BitReader::new(f);
r.bit = 6 * 8;
let acmod = r.read(3) as u8;
if (acmod & 0x1) != 0 && acmod != 0x1 {
r.skip(2); // cmixlev
}
if (acmod & 0x4) != 0 {
r.skip(2); // surmixlev
}
if acmod == 0x2 {
r.skip(2); // dsurmod
}
let lfeon = r.read(1) == 1;
Some(DolbyConfig {
fscod,
bsid,
bsmod,
acmod,
lfeon,
bit_rate_code: frmsizecod >> 1,
data_rate_kbps: 0,
sample_rate: FSCOD_RATES.get(fscod as usize).copied().unwrap_or(48_000),
channels: DolbyConfig::channel_count(acmod, lfeon),
})
}
/// E-AC-3 (A/52 Annex E BSI): syncword | strmtyp(2) substreamid(3) frmsiz(11) |
/// fscod(2) numblkscod(2) acmod(3) lfeon(1) | bsid(5) …
fn parse_eac3(f: &[u8]) -> Option<DolbyConfig> {
if f.len() < 6 {
return None;
}
let frmsiz = (((f[2] & 0x07) as u32) << 8) | f[3] as u32; // words minus one
let fscod = (f[4] >> 6) & 0x03;
let numblkscod = (f[4] >> 4) & 0x03;
let acmod = (f[4] >> 1) & 0x07;
let lfeon = (f[4] & 0x01) == 1;
let bsid = (f[5] >> 3) & 0x1F;
let (sample_rate, blocks) = if fscod == 0x03 {
let fscod2 = (f[4] >> 4) & 0x03; // shares bits with numblkscod when fscod==3
(EAC3_REDUCED_RATES[fscod2 as usize], 6u32)
} else {
let blocks = [1u32, 2, 3, 6][numblkscod as usize];
(FSCOD_RATES[fscod as usize], blocks)
};
// Nominal data rate (kbps): frame is (frmsiz+1) 16-bit words per (blocks·256)
// samples at sample_rate. rate = bytes·8·sr / samples / 1000.
let frame_bytes = (frmsiz as u64 + 1) * 2;
let samples = blocks as u64 * 256;
let data_rate_kbps = if samples > 0 {
((frame_bytes * 8 * sample_rate as u64) / samples / 1000) as u16
} else {
0
};
Some(DolbyConfig {
fscod,
bsid,
bsmod: 0, // not in the E-AC-3 main header; dec3 default
acmod,
lfeon,
bit_rate_code: 0,
data_rate_kbps,
sample_rate,
channels: DolbyConfig::channel_count(acmod, lfeon),
})
}
/// The `dac3` config box (ETSI TS 102 366 Annex F.4): 24 bits —
/// fscod(2) bsid(5) bsmod(3) acmod(3) lfeon(1) bit_rate_code(5) reserved(5).
pub(super) fn dac3_box(c: &DolbyConfig) -> Vec<u8> {
let mut v: u32 = 0;
let mut push = |val: u32, bits: u32| v = (v << bits) | (val & ((1 << bits) - 1));
push(c.fscod as u32, 2);
push(c.bsid as u32, 5);
push(c.bsmod as u32, 3);
push(c.acmod as u32, 3);
push(c.lfeon as u32, 1);
push(c.bit_rate_code as u32, 5);
push(0, 5); // reserved
// 24 bits → the top 3 bytes of the big-endian u32.
let b = v.to_be_bytes();
bx(b"dac3", &[b[1], b[2], b[3]])
}
/// The `dec3` config box (ETSI TS 102 366 Annex G.3) for a single independent
/// substream, no dependent substreams: data_rate(13) num_ind_sub(3) then
/// fscod(2) bsid(5) reserved(1) asvc(1) bsmod(3) acmod(3) lfeon(1) reserved(3)
/// num_dep_sub(4) reserved(1).
pub(super) fn dec3_box(c: &DolbyConfig) -> Vec<u8> {
let mut v: u64 = 0;
let mut push = |val: u64, bits: u32| v = (v << bits) | (val & ((1u64 << bits) - 1));
push(c.data_rate_kbps as u64, 13);
push(0, 3); // num_ind_sub - 1 = 0 (one substream)
push(c.fscod as u64, 2);
push(c.bsid as u64, 5);
push(0, 1); // reserved
push(0, 1); // asvc
push(c.bsmod as u64, 3);
push(c.acmod as u64, 3);
push(c.lfeon as u64, 1);
push(0, 3); // reserved
push(0, 4); // num_dep_sub = 0
push(0, 1); // reserved (chan_loc absent when num_dep_sub == 0)
// 40 bits → the low 5 bytes of the big-endian u64.
let b = v.to_be_bytes();
bx(b"dec3", &[b[3], b[4], b[5], b[6], b[7]])
}
/// Build an audio sample entry (`ac-3` / `ec-3`) with the given config box.
/// `AudioSampleEntry` per ISO/IEC 14496-12 §12.2.3.
pub(super) fn audio_sample_entry(
fourcc: &[u8; 4],
channels: u16,
sample_rate: u32,
config: &[u8],
) -> Vec<u8> {
let mut e = Vec::new();
e.extend_from_slice(&[0u8; 6]); // reserved
e.extend_from_slice(&1u16.to_be_bytes()); // data_reference_index
e.extend_from_slice(&[0u8; 8]); // reserved (version 0)
e.extend_from_slice(&channels.to_be_bytes());
e.extend_from_slice(&16u16.to_be_bytes()); // samplesize
e.extend_from_slice(&0u16.to_be_bytes()); // pre_defined
e.extend_from_slice(&0u16.to_be_bytes()); // reserved
// samplerate is 16.16 fixed point; the integer rate in the high 16 bits.
e.extend_from_slice(&(sample_rate << 16).to_be_bytes());
e.extend_from_slice(config);
bx(fourcc, &e)
}
// ── DTS (dtsc/dtsh + ddts) ───────────────────────────────────────────────────
/// DTS core `SFREQ` (4-bit) → sample rate (Hz). Reserved indices → 48 kHz.
const DTS_SFREQ: [u32; 16] = [
48_000, 8_000, 16_000, 32_000, 48_000, 48_000, 11_025, 22_050, 44_100, 48_000, 48_000, 12_000,
24_000, 48_000, 96_000, 192_000,
];
/// DTS core base channel count per `AMODE` (0..=9); higher AMODEs are rare on disc.
const DTS_AMODE_CH: [u8; 10] = [1, 2, 2, 2, 2, 3, 3, 4, 4, 5];
/// Decoded DTS core parameters needed for the `ddts` box.
struct DtsConfig {
sample_rate: u32,
channels: u16,
amode: u8,
lfe: bool,
core_size: u32,
/// Samples per frame ((NBLKS+1)·32).
frame_samples: u32,
/// Whether a DTS-HD extension substream follows the core.
has_extension: bool,
channel_layout: u16,
}
/// Parse the DTS core header (ETSI TS 102 114 §5.3.1), starting at the
/// 0x7FFE8001 big-endian core sync. Returns `None` if too short / no sync.
fn parse_dts(frame: &[u8]) -> Option<DtsConfig> {
let start = (0..frame.len().saturating_sub(3)).find(|&i| {
frame[i] == 0x7F && frame[i + 1] == 0xFE && frame[i + 2] == 0x80 && frame[i + 3] == 0x01
})?;
let f = &frame[start..];
if f.len() < 11 {
return None;
}
// Bit fields after the 32-bit sync (MSB-first):
// FTYPE1 SHORT5 CPF1 NBLKS7 FSIZE14 AMODE6 SFREQ4 RATE5 ...
let nblks = (((f[4] & 0x01) as u32) << 6) | ((f[5] >> 2) as u32 & 0x3F);
let fsize = (((f[5] & 0x03) as u32) << 12) | ((f[6] as u32) << 4) | ((f[7] >> 4) as u32 & 0x0F);
let amode = (((f[7] & 0x0F) << 2) | ((f[8] >> 6) & 0x03)) as usize;
let sfreq = ((f[8] >> 2) & 0x0F) as usize;
// LFF is 2 bits at bit offset 85 → byte10 bits 2-1.
let lff = (f[10] >> 1) & 0x03;
let lfe = lff == 1 || lff == 2;
let sample_rate = DTS_SFREQ[sfreq];
let base_ch = DTS_AMODE_CH.get(amode).copied().unwrap_or(6);
let channels = base_ch as u16 + lfe as u16;
let channel_layout = dts_channel_layout(amode, lfe);
// DTS-HD extension substream sync (0x64582025) after the core frame.
let ext_sync = [0x64, 0x58, 0x20, 0x25];
let has_extension = f
.windows(4)
.skip((fsize as usize + 1).min(f.len()).saturating_sub(4))
.any(|w| w == ext_sync)
|| f.windows(4).any(|w| w == ext_sync);
Some(DtsConfig {
sample_rate,
channels,
amode: amode as u8,
lfe,
core_size: fsize + 1,
frame_samples: (nblks + 1) * 32,
has_extension,
channel_layout,
})
}
/// `ddts` ChannelLayout (16-bit speaker mask) for the common core layouts.
/// bit0=C, bit1=L/R, bit2=Ls/Rs, bit3=LFE.
fn dts_channel_layout(amode: usize, lfe: bool) -> u16 {
let mut m = match amode {
0 => 0x0001, // C (mono)
1..=4 => 0x0002, // L/R
5 => 0x0003, // C + L/R
6 | 8 => 0x0006, // L/R + Ls/Rs (no centre)
_ => 0x0007, // C + L/R + surround (amode 7, 9, …)
};
if lfe {
m |= 0x0008;
}
m
}
/// The `ddts` config box (ETSI TS 102 114 Annex; DTS-in-ISO registration).
/// Describes the DTS core; whole access units (core + any extension) are passed
/// through as samples, so a DTS-HD-aware decoder still finds the extension.
fn ddts_box(c: &DtsConfig) -> Vec<u8> {
// avg/max bitrate: computed from the core frame size × frame rate (the core
// RATE field reads "open/variable" for lossless, so it's not usable directly).
let frames_per_sec = if c.frame_samples > 0 {
c.sample_rate as u64 / c.frame_samples as u64
} else {
0
};
let bitrate = (c.core_size as u64 * 8 * frames_per_sec) as u32;
let mut out = Vec::new();
out.extend_from_slice(&c.sample_rate.to_be_bytes()); // DTSSamplingFrequency
out.extend_from_slice(&bitrate.to_be_bytes()); // maxBitrate
out.extend_from_slice(&bitrate.to_be_bytes()); // avgBitrate
out.push(if c.has_extension { 24 } else { 16 }); // pcmSampleDepth
// Bit-packed tail (56 bits):
// FrameDuration2 StreamConstruction5 CoreLFEPresent1 CoreLayout6 CoreSize14
// StereoDownmix1 RepresentationType3 ChannelLayout16 MultiAssetFlag1
// LBRDurationMod1 ReservedBoxPresent1 Reserved5
let frame_duration = match c.frame_samples {
0..=512 => 0,
513..=1024 => 1,
1025..=2048 => 2,
_ => 3,
};
// StreamConstruction: 1 = DTS core present. Whole-AU passthrough means an
// HD decoder still parses the extension substreams from the stream itself.
let stream_construction = 1u128;
let mut v: u128 = 0;
let mut push = |val: u128, bits: u32| v = (v << bits) | (val & ((1u128 << bits) - 1));
push(frame_duration as u128, 2);
push(stream_construction, 5);
push(c.lfe as u128, 1);
push(c.amode as u128, 6);
push(c.core_size as u128, 14);
push(0, 1); // StereoDownmix
push(0, 3); // RepresentationType
push(c.channel_layout as u128, 16);
push(c.has_extension as u128, 1); // MultiAssetFlag
push(0, 1); // LBRDurationMod
push(0, 1); // ReservedBoxPresent
push(0, 5); // Reserved
// 56 bits → the low 7 bytes of the big-endian u128.
let b = v.to_be_bytes();
out.extend_from_slice(&b[9..16]);
bx(b"ddts", &out)
}
/// The MP4 fourcc + config box for an audio frame, or `None` if the codec has no
/// MP4 mapping here. Together with [`audio_fits`] this is the fit oracle for
/// audio: only what returns `Some` is muxable.
pub(super) fn dolby_sample_entry(codec: Codec, first_frame: &[u8]) -> Option<Vec<u8>> {
match codec {
Codec::Ac3 => {
let c = parse_dolby(first_frame)?;
Some(audio_sample_entry(
b"ac-3",
c.channels,
c.sample_rate,
&dac3_box(&c),
))
}
Codec::Ac3Plus => {
let c = parse_dolby(first_frame)?;
Some(audio_sample_entry(
b"ec-3",
c.channels,
c.sample_rate,
&dec3_box(&c),
))
}
Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr => {
let c = parse_dts(first_frame)?;
// `dtsc` = DTS core; `dtsh` = DTS-HD (core + extension substreams).
let fourcc: &[u8; 4] = if c.has_extension { b"dtsh" } else { b"dtsc" };
Some(audio_sample_entry(
fourcc,
c.channels,
c.sample_rate,
&ddts_box(&c),
))
}
_ => None,
}
}
/// Fit oracle for an audio codec: does `mp4://` currently carry it? Covers the
/// Dolby family (AC-3 / E-AC-3) and DTS (core / DTS-HD HRA / DTS-HD MA — the core
/// is described, whole access units pass through). TrueHD, LPCM, AAC are not yet
/// mapped and are skipped with a loud report (never silently dropped).
pub(super) fn audio_fits(codec: Codec) -> bool {
matches!(
codec,
Codec::Ac3 | Codec::Ac3Plus | Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr
)
}
#[cfg(test)]
mod tests {
use super::*;
/// A synthetic legacy AC-3 header: syncword, crc, fscod=0 (48k),
/// frmsizecod, bsid=8, bsmod=0, acmod=7 (3/2), lfeon=1 → 5.1.
fn ac3_frame_5_1() -> Vec<u8> {
let mut f = vec![0x0B, 0x77, 0x00, 0x00];
// byte4: fscod(2)=0 | frmsizecod(6)=0b010110 (22)
f.push(0b00_010110);
// byte5: bsid(5)=8 (0b01000) | bsmod(3)=0
f.push(0b01000_000);
// byte6: acmod(3)=7 (0b111) | cmixlev(2) | surmixlev(2) | lfeon(1)...
// acmod=7 has centre (needs cmixlev) and surround (needs surmixlev):
// 111 | 00 | 00 | 1(lfeon) = 0b111_00_00_1
f.push(0b111_00_00_1);
f.push(0x00);
f
}
#[test]
fn ac3_bsi_and_dac3() {
let c = parse_dolby(&ac3_frame_5_1()).expect("parsed");
assert!(c.bsid < 11, "legacy AC-3");
assert_eq!(c.fscod, 0);
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.bsid, 8);
assert_eq!(c.acmod, 7);
assert!(c.lfeon);
assert_eq!(c.channels, 6, "3/2 + LFE = 5.1");
assert_eq!(c.bit_rate_code, 22 >> 1);
let dac3 = dac3_box(&c);
// [size:4]["dac3"][3-byte payload] = 11 bytes.
assert_eq!(dac3.len(), 11);
assert_eq!(&dac3[4..8], b"dac3");
}
#[test]
fn eac3_bsi_and_dec3() {
// E-AC-3: syncword | strmtyp/substreamid/frmsiz | fscod/numblks/acmod/lfeon | bsid
let mut f = vec![0x0B, 0x77];
f.push(0x00); // strmtyp=0, substreamid=0, frmsiz high=0
f.push(0x3F); // frmsiz low = 63 → frame 128 bytes
// byte4: fscod(2)=0 | numblkscod(2)=3 (6 blocks) | acmod(3)=7 | lfeon(1)=1
f.push(0b00_11_111_1);
// byte5: bsid(5)=16 (E-AC-3) | dialnorm high(3)
f.push(0b10000_000);
f.push(0x00);
let c = parse_dolby(&f).expect("parsed");
assert!(c.bsid >= 11, "E-AC-3");
assert_eq!(c.bsid, 16);
assert_eq!(c.fscod, 0);
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.acmod, 7);
assert!(c.lfeon);
assert_eq!(c.channels, 6);
let dec3 = dec3_box(&c);
// [size:4]["dec3"][5-byte payload] = 13 bytes.
assert_eq!(dec3.len(), 13);
assert_eq!(&dec3[4..8], b"dec3");
}
#[test]
fn sample_entry_shape() {
let c = parse_dolby(&ac3_frame_5_1()).unwrap();
let e = audio_sample_entry(b"ac-3", c.channels, c.sample_rate, &dac3_box(&c));
assert_eq!(&e[4..8], b"ac-3");
// channelcount at entry-body offset 16 (after 6 reserved + 2 dri + 8 reserved).
let ch = u16::from_be_bytes([e[8 + 16], e[8 + 17]]);
assert_eq!(ch, 6);
}
#[test]
fn fit_oracle_covers_dolby_and_dts() {
assert!(audio_fits(Codec::Ac3));
assert!(audio_fits(Codec::Ac3Plus));
assert!(audio_fits(Codec::Dts));
assert!(audio_fits(Codec::DtsHdMa));
assert!(!audio_fits(Codec::TrueHd));
assert!(!audio_fits(Codec::Lpcm));
}
#[test]
fn dts_core_5_1_and_ddts() {
// Synthetic DTS core: SFREQ=13 (48k), AMODE=9 (5ch), LFF=1 (LFE) → 5.1.
let f = vec![
0x7F, 0xFE, 0x80, 0x01, 0x00, 0x3C, 0x05, 0xF2, 0x77, 0x00, 0x02, 0x00,
];
let c = parse_dts(&f).expect("dts core parsed");
assert_eq!(c.sample_rate, 48_000);
assert_eq!(c.amode, 9);
assert!(c.lfe);
assert_eq!(c.channels, 6, "5 core + LFE = 5.1");
assert_eq!(c.channel_layout, 0x000F, "C + L/R + Ls/Rs + LFE");
assert_eq!(c.core_size, 96);
assert_eq!(c.frame_samples, 512);
let ddts = ddts_box(&c);
assert_eq!(&ddts[4..8], b"ddts");
// DTSSamplingFrequency (first field) = 48000.
assert_eq!(
u32::from_be_bytes([ddts[8], ddts[9], ddts[10], ddts[11]]),
48_000
);
// Sample entry uses dtsc (no extension in this synthetic frame).
let e = dolby_sample_entry(Codec::DtsHdMa, &f).unwrap();
assert_eq!(&e[4..8], b"dtsc");
}
}
+27
View File
@@ -0,0 +1,27 @@
//! ISO-BMFF box primitives: `[size:u32-BE][type:4][body]` (ISO/IEC 14496-12
//! §4.2), and the FullBox variant that prefixes a 1-byte version + 3-byte flags.
/// Wrap a body in a plain box `[size][type][body]`. `size` counts the 8-byte
/// header. All `moov`-tree boxes are small (the large `mdat` is written directly
/// with a 64-bit size, not through here), so a `u32` size never overflows.
pub(super) fn bx(box_type: &[u8; 4], body: &[u8]) -> Vec<u8> {
let total = body.len() + 8;
debug_assert!(
total <= u32::MAX as usize,
"mp4 box {box_type:?} exceeds u32"
);
let mut out = Vec::with_capacity(total);
out.extend_from_slice(&(total as u32).to_be_bytes());
out.extend_from_slice(box_type);
out.extend_from_slice(body);
out
}
/// Wrap a body in a FullBox: `[size][type][version:1][flags:3][body]`.
pub(super) fn fullbox(box_type: &[u8; 4], version: u8, flags: u32, body: &[u8]) -> Vec<u8> {
let mut full = Vec::with_capacity(body.len() + 4);
full.push(version);
full.extend_from_slice(&flags.to_be_bytes()[1..]); // low 3 bytes
full.extend_from_slice(body);
bx(box_type, &full)
}
+1099
View File
File diff suppressed because it is too large Load Diff
+705
View File
@@ -0,0 +1,705 @@
//! Progressive MP4 (ISO-BMFF) demuxer — the read side of `mp4://`.
//!
//! The inverse of the writer in [`super`]: parse `moov`/`trak`/`stbl`, rebuild a
//! [`DiscTitle`] and a per-sample index (offset/size/timing/sync from
//! `stsc`+`stco`/`co64`+`stsz`, `stts`+`ctts`, `stss`), then stream each sample
//! out as a [`PesFrame`] in decode order. Video NALs are length-prefixed in MP4 —
//! exactly the framing the MKV muxer consumes — so `mp4://` → `mkv://` (and
//! `mp4://` → any sink) needs no reframing.
//!
//! Scope: progressive MP4 (`moov` + `mdat`). Fragmented MP4 (`moof`) is out of
//! scope for now — its samples live in `traf`/`trun`, not `stbl`.
use crate::disc::{
AudioChannels, AudioStream, Codec, DiscTitle, Resolution, SampleRate, Stream as DiscStream,
VideoStream,
};
use crate::labels::LabelPurpose;
use crate::pes::{PesFrame, Stream};
use std::fs::File;
use std::io::{self, Read, Seek, SeekFrom};
use std::path::Path;
const NS: i128 = 1_000_000_000;
/// One sample's location + timing in the emission plan.
struct SampleRef {
track: usize,
offset: u64,
size: u32,
/// Composition (presentation) time in nanoseconds.
pts_ns: i64,
/// Decode time in nanoseconds — the key the global emission order sorts on.
dts_ns: i64,
keyframe: bool,
}
/// MP4 reader: a `Stream` source that emits a file's samples as PES frames.
/// Generic over the backing reader so it works over a `File` (the `mp4://`
/// source) or an in-memory `Cursor` (round-trip tests).
pub struct Mp4Reader<R: Read + Seek> {
file: R,
title: DiscTitle,
samples: Vec<SampleRef>,
cursor: usize,
}
impl Mp4Reader<File> {
/// Open and index an MP4 file by path.
pub fn open(path: &Path) -> io::Result<Self> {
let name = path
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("mp4")
.to_string();
Self::from_reader(File::open(path)?, name)
}
}
impl<R: Read + Seek> Mp4Reader<R> {
/// Index an already-opened seekable MP4 reader.
pub fn from_reader(mut file: R, name: String) -> io::Result<Self> {
let moov = read_moov(&mut file)?;
let mut title = DiscTitle::empty();
title.playlist = name;
let mut samples: Vec<SampleRef> = Vec::new();
let mut codec_privates: Vec<Option<Vec<u8>>> = Vec::new();
let mut track_idx = 0usize;
for trak in find_boxes(&moov, b"trak") {
let Some(mdia) = find_box(trak, b"mdia") else {
continue;
};
let timescale = find_box(mdia, b"mdhd")
.and_then(mdhd_timescale)
.unwrap_or(90_000);
let language = find_box(mdia, b"mdhd").and_then(mdhd_language);
let handler = find_box(mdia, b"hdlr").and_then(hdlr_type);
let Some(minf) = find_box(mdia, b"minf") else {
continue;
};
let Some(stbl) = find_box(minf, b"stbl") else {
continue;
};
let Some(stsd) = find_box(stbl, b"stsd") else {
continue;
};
let Some(StsdInfo {
codec,
height,
config,
channels,
}) = parse_stsd(stsd)
else {
continue; // unrecognised sample entry — skip the track
};
// Build the stream model for this track.
let stream = match handler {
Some(h) if &h == b"vide" => DiscStream::Video(VideoStream {
pid: 0x1011 + track_idx as u16,
codec,
resolution: Resolution::from_height(height as u32),
frame_rate: crate::disc::FrameRate::Unknown,
hdr: crate::disc::HdrFormat::Sdr,
color_space: crate::disc::ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
}),
Some(h) if &h == b"soun" => DiscStream::Audio(AudioStream {
pid: 0x1100 + track_idx as u16,
codec,
channels: AudioChannels::from_count(channels as u8),
language: language.clone().unwrap_or_else(|| "und".into()),
sample_rate: SampleRate::from_hz(timescale),
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
}),
_ => continue, // non-A/V handler
};
// Per-sample tables.
let sizes = find_box(stbl, b"stsz").map(parse_stsz).unwrap_or_default();
let n = sizes.len();
if n == 0 {
track_idx += 1;
title.streams.push(stream);
codec_privates.push(config);
continue;
}
let chunk_offsets = find_box(stbl, b"stco")
.map(|b| parse_stco(b, false))
.or_else(|| find_box(stbl, b"co64").map(|b| parse_stco(b, true)))
.unwrap_or_default();
let stsc = find_box(stbl, b"stsc").map(parse_stsc).unwrap_or_default();
let offsets = sample_offsets(&sizes, &chunk_offsets, &stsc);
let durations = find_box(stbl, b"stts").map(parse_stts).unwrap_or_default();
let ctts = find_box(stbl, b"ctts").map(parse_ctts).unwrap_or_default();
let sync = find_box(stbl, b"stss").map(parse_stss);
let mut decode_ticks: i64 = 0;
for (i, &size) in sizes.iter().enumerate() {
let dur = durations.get(i).copied().unwrap_or(0);
let comp = ctts.get(i).copied().unwrap_or(0);
let dts_ns = (decode_ticks as i128 * NS / timescale as i128) as i64;
let pts_ticks = decode_ticks + comp as i64;
let pts_ns = (pts_ticks as i128 * NS / timescale as i128) as i64;
decode_ticks += dur as i64;
let keyframe = match &sync {
Some(set) => set.contains(&(i as u32 + 1)),
None => true, // no stss → every sample is a sync sample
};
samples.push(SampleRef {
track: track_idx,
offset: offsets.get(i).copied().unwrap_or(0),
size,
pts_ns,
dts_ns,
keyframe,
});
}
title.streams.push(stream);
codec_privates.push(config);
track_idx += 1;
}
if title.streams.is_empty() {
return Err(crate::error::Error::MkvInvalid.into());
}
title.codec_privates = codec_privates;
// Emit in global decode order so the consumer sees interleaved,
// monotonic-DTS frames (a stable sort keeps per-track order on ties).
samples.sort_by_key(|s| s.dts_ns);
Ok(Self {
file,
title,
samples,
cursor: 0,
})
}
}
impl<R: Read + Seek + Send> Stream for Mp4Reader<R> {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
let Some(s) = self.samples.get(self.cursor) else {
return Ok(None);
};
self.cursor += 1;
self.file.seek(SeekFrom::Start(s.offset))?;
let mut data = vec![0u8; s.size as usize];
self.file.read_exact(&mut data)?;
Ok(Some(PesFrame {
track: s.track,
pts: s.pts_ns,
keyframe: s.keyframe,
data,
duration_ns: None,
source: None,
coding: None,
}))
}
fn write(&mut self, _frame: &PesFrame) -> io::Result<()> {
Err(crate::error::Error::StreamReadOnly.into())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
self.title.codec_privates.get(track).and_then(|c| c.clone())
}
}
// ── box tree navigation ──────────────────────────────────────────────────────
/// Read top-level boxes until `moov`, returning its payload (after the header).
/// Skips over `ftyp`/`mdat`/etc. via seek; samples are read later by offset.
fn read_moov<R: Read + Seek>(file: &mut R) -> io::Result<Vec<u8>> {
loop {
let mut hdr = [0u8; 8];
if file.read_exact(&mut hdr).is_err() {
return Err(crate::error::Error::MkvInvalid.into());
}
let size32 = u32::from_be_bytes([hdr[0], hdr[1], hdr[2], hdr[3]]);
let btype = [hdr[4], hdr[5], hdr[6], hdr[7]];
// 64-bit largesize (size==1): the real size is the next 8 bytes; a
// 16-byte header precedes the payload. size==0 means "to EOF".
let payload_len: u64 = if size32 == 1 {
let mut ext = [0u8; 8];
file.read_exact(&mut ext)?;
u64::from_be_bytes(ext).saturating_sub(16)
} else {
(size32 as u64).saturating_sub(8)
};
if &btype == b"moov" {
let mut buf = vec![0u8; payload_len as usize];
file.read_exact(&mut buf)?;
return Ok(buf);
}
file.seek(SeekFrom::Current(payload_len as i64))?;
}
}
/// The first child box of `payload` with the given type — returns its payload
/// (bytes after the 8-byte header). One level.
fn find_box<'a>(payload: &'a [u8], want: &[u8; 4]) -> Option<&'a [u8]> {
find_boxes(payload, want).into_iter().next()
}
/// All child boxes of `payload` with the given type — each as its payload slice.
fn find_boxes<'a>(payload: &'a [u8], want: &[u8; 4]) -> Vec<&'a [u8]> {
let mut out = Vec::new();
let mut pos = 0;
while pos + 8 <= payload.len() {
let size = u32::from_be_bytes([
payload[pos],
payload[pos + 1],
payload[pos + 2],
payload[pos + 3],
]) as usize;
let bt = [
payload[pos + 4],
payload[pos + 5],
payload[pos + 6],
payload[pos + 7],
];
if size < 8 || pos + size > payload.len() {
break;
}
if &bt == want {
out.push(&payload[pos + 8..pos + size]);
}
pos += size;
}
out
}
fn be32(b: &[u8], o: usize) -> u32 {
u32::from_be_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]])
}
fn be16(b: &[u8], o: usize) -> u16 {
u16::from_be_bytes([b[o], b[o + 1]])
}
/// mdhd (version 0/1) → media timescale.
fn mdhd_timescale(b: &[u8]) -> Option<u32> {
let version = b.first().copied()?;
if version == 1 {
// version(1)+flags(3) creation(8) modification(8) timescale(4) ...
(b.len() >= 24).then(|| be32(b, 20))
} else {
// creation(4) modification(4) timescale(4) ...
(b.len() >= 16).then(|| be32(b, 12))
}
}
/// mdhd language (5-bit packed ISO 639-2) → lowercase 3-letter code.
fn mdhd_language(b: &[u8]) -> Option<String> {
let version = b.first().copied()?;
let off = if version == 1 { 28 } else { 20 };
if b.len() < off + 2 {
return None;
}
let packed = be16(b, off);
let c0 = ((packed >> 10) & 0x1F) as u8 + 0x60;
let c1 = ((packed >> 5) & 0x1F) as u8 + 0x60;
let c2 = (packed & 0x1F) as u8 + 0x60;
let s: String = [c0, c1, c2].iter().map(|&c| c as char).collect();
if s.chars().all(|c| c.is_ascii_lowercase()) {
Some(s)
} else {
None
}
}
/// hdlr → handler_type fourcc ('vide' / 'soun').
fn hdlr_type(b: &[u8]) -> Option<[u8; 4]> {
// version+flags(4) pre_defined(4) handler_type(4) ...
(b.len() >= 12).then(|| [b[8], b[9], b[10], b[11]])
}
/// Decoded first sample entry of an `stsd` box.
struct StsdInfo {
codec: Codec,
height: u16,
config: Option<Vec<u8>>,
channels: u16,
}
/// stsd → codec + dimensions + codec_private + channel count (first entry).
fn parse_stsd(b: &[u8]) -> Option<StsdInfo> {
// version+flags(4) entry_count(4) then the first sample entry box.
if b.len() < 8 {
return None;
}
let entry = &b[8..];
if entry.len() < 8 {
return None;
}
let size = be32(entry, 0) as usize;
let fourcc = [entry[4], entry[5], entry[6], entry[7]];
let body = &entry[8..size.min(entry.len())];
let codec = match &fourcc {
b"hvc1" | b"hev1" => Codec::Hevc,
b"avc1" | b"avc3" => Codec::H264,
b"ac-3" => Codec::Ac3,
b"ec-3" => Codec::Ac3Plus,
b"mp4a" => Codec::Aac,
b"dtsc" | b"dtse" | b"dtsh" | b"dtsl" => Codec::Dts,
_ => return None,
};
if matches!(codec, Codec::Hevc | Codec::H264) {
// VisualSampleEntry: 6 reserved + 2 dri + 16 pre/reserved + width(2)
// height(2) + 14 + 32 compressorname + 2 depth + 2 pre = 78 bytes, then
// child boxes (hvcC/avcC, colr, …).
if body.len() < 78 {
return None;
}
let height = be16(body, 26);
let children = &body[78..];
let config = find_box(children, b"hvcC")
.or_else(|| find_box(children, b"avcC"))
.map(|c| c.to_vec());
Some(StsdInfo {
codec,
height,
config,
channels: 0,
})
} else {
// AudioSampleEntry: 6 reserved + 2 dri + 8 reserved + channelcount(2)
// samplesize(2) + 2 pre + 2 reserved + samplerate(4) = 28 bytes.
let channels = if body.len() >= 28 { be16(body, 16) } else { 2 };
Some(StsdInfo {
codec,
height: 0,
config: None,
channels,
})
}
}
/// stsz → per-sample sizes.
fn parse_stsz(b: &[u8]) -> Vec<u32> {
if b.len() < 12 {
return Vec::new();
}
let sample_size = be32(b, 4);
let count = be32(b, 8) as usize;
if sample_size != 0 {
return vec![sample_size; count];
}
let mut out = Vec::with_capacity(count);
for i in 0..count {
let o = 12 + i * 4;
if o + 4 > b.len() {
break;
}
out.push(be32(b, o));
}
out
}
/// stco (32-bit) / co64 (64-bit) → chunk offsets.
fn parse_stco(b: &[u8], is64: bool) -> Vec<u64> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::with_capacity(count);
let stride = if is64 { 8 } else { 4 };
for i in 0..count {
let o = 8 + i * stride;
if o + stride > b.len() {
break;
}
if is64 {
out.push(u64::from_be_bytes([
b[o],
b[o + 1],
b[o + 2],
b[o + 3],
b[o + 4],
b[o + 5],
b[o + 6],
b[o + 7],
]));
} else {
out.push(be32(b, o) as u64);
}
}
out
}
/// stsc → (first_chunk, samples_per_chunk) entries (1-based first_chunk).
fn parse_stsc(b: &[u8]) -> Vec<(u32, u32)> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::with_capacity(count);
for i in 0..count {
let o = 8 + i * 12;
if o + 12 > b.len() {
break;
}
out.push((be32(b, o), be32(b, o + 4)));
}
out
}
/// Reconstruct per-sample file offsets from sizes + chunk offsets + stsc.
fn sample_offsets(sizes: &[u32], chunk_offsets: &[u64], stsc: &[(u32, u32)]) -> Vec<u64> {
let n_chunks = chunk_offsets.len();
// Expand stsc → samples_per_chunk for every chunk.
let mut spc = vec![0u32; n_chunks];
for (idx, &(first, per)) in stsc.iter().enumerate() {
let start = (first.saturating_sub(1)) as usize;
let end = stsc
.get(idx + 1)
.map(|&(nf, _)| (nf.saturating_sub(1)) as usize)
.unwrap_or(n_chunks);
let end = end.min(n_chunks);
if start < end {
spc[start..end].fill(per);
}
}
let mut offsets = Vec::with_capacity(sizes.len());
let mut sidx = 0usize;
for (ci, &choff) in chunk_offsets.iter().enumerate() {
let mut off = choff;
for _ in 0..spc[ci] {
if sidx >= sizes.len() {
break;
}
offsets.push(off);
off += sizes[sidx] as u64;
sidx += 1;
}
}
// Any trailing samples with no chunk mapping: pack after the last offset.
while offsets.len() < sizes.len() {
let last = offsets.last().copied().unwrap_or(0);
let last_sz = sizes
.get(offsets.len().saturating_sub(1))
.copied()
.unwrap_or(0);
offsets.push(last + last_sz as u64);
}
offsets
}
/// stts → per-sample decode durations (expanded from run-length entries).
fn parse_stts(b: &[u8]) -> Vec<u32> {
if b.len() < 8 {
return Vec::new();
}
let count = be32(b, 4) as usize;
let mut out = Vec::new();
for i in 0..count {
let o = 8 + i * 8;
if o + 8 > b.len() {
break;
}
let n = be32(b, o);
let delta = be32(b, o + 4);
for _ in 0..n {
out.push(delta);
}
}
out
}
/// ctts → per-sample composition offsets (version 0 unsigned / version 1 signed).
fn parse_ctts(b: &[u8]) -> Vec<i32> {
if b.len() < 8 {
return Vec::new();
}
// version 0 = unsigned, version 1 = signed offsets; the u32→i32 bit-cast
// reads both correctly (real composition offsets fit in i32 either way).
let count = be32(b, 4) as usize;
let mut out = Vec::new();
for i in 0..count {
let o = 8 + i * 8;
if o + 8 > b.len() {
break;
}
let n = be32(b, o);
let offset = be32(b, o + 4) as i32;
for _ in 0..n {
out.push(offset);
}
}
out
}
/// stss → set of 1-based sync sample numbers.
fn parse_stss(b: &[u8]) -> std::collections::HashSet<u32> {
let mut set = std::collections::HashSet::new();
if b.len() < 8 {
return set;
}
let count = be32(b, 4) as usize;
for i in 0..count {
let o = 8 + i * 4;
if o + 4 > b.len() {
break;
}
set.insert(be32(b, o));
}
set
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn stsc_offsets_one_sample_per_chunk() {
// Our writer's layout: 1 sample/chunk, co64 lists every offset.
let sizes = vec![100u32, 200, 300];
let chunks = vec![1000u64, 1100, 1300];
let stsc = vec![(1u32, 1u32)];
assert_eq!(
sample_offsets(&sizes, &chunks, &stsc),
vec![1000, 1100, 1300]
);
}
#[test]
fn stsc_offsets_multi_sample_chunks() {
// 2 samples in chunk 1, 1 in chunk 2.
let sizes = vec![10u32, 20, 30];
let chunks = vec![500u64, 900];
let stsc = vec![(1u32, 2u32), (2u32, 1u32)];
// chunk1@500: s0@500, s1@510; chunk2@900: s2@900.
assert_eq!(sample_offsets(&sizes, &chunks, &stsc), vec![500, 510, 900]);
}
#[test]
fn write_then_read_round_trip() {
// Mux a small A/V title to an in-memory MP4, then demux it back and
// check the streams, codec_private, and sample payloads survive.
use crate::disc::{
AudioChannels, AudioStream, Codec, DiscTitle, FrameRate, HdrFormat, LabelPurpose,
SampleRate, Stream as DiscStreamE, VideoStream,
};
use crate::mux::mp4::Mp4Sink;
use crate::pes::{PesFrame, Stream as _};
use std::io::Cursor;
let mut t = DiscTitle::empty();
t.streams = vec![
DiscStreamE::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R1080p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Sdr,
color_space: crate::disc::ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
}),
DiscStreamE::Audio(AudioStream {
pid: 0x1100,
codec: Codec::Ac3,
channels: AudioChannels::Surround51,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
}),
];
let hvcc = vec![0xAA, 0xBB, 0xCC, 0xDD, 0xEE];
t.codec_privates = vec![Some(hvcc.clone()), None];
let ac3 = vec![
0x0B,
0x77,
0x00,
0x00,
0b00_010110,
0b01000_000,
0b111_00_00_1,
0x00,
0xFF,
0xFF,
];
let d = 41_708_333i64;
let vid0 = vec![0x11u8; 700];
let vid1 = vec![0x22u8; 350];
let mut buf = Vec::new();
{
let mk = |track, pts, key, data: Vec<u8>| PesFrame {
track,
pts,
keyframe: key,
data,
duration_ns: None,
source: None,
coding: None,
};
let mut sink = Mp4Sink::create(Cursor::new(&mut buf), &t).unwrap();
sink.write(&mk(0, 0, true, vid0.clone())).unwrap();
sink.write(&mk(1, 0, true, ac3.clone())).unwrap();
sink.write(&mk(0, d, false, vid1.clone())).unwrap();
sink.write(&mk(1, 32_000_000, true, ac3.clone())).unwrap();
sink.finish().unwrap();
}
let mut rd = Mp4Reader::from_reader(Cursor::new(buf), "rt".into()).unwrap();
// Two streams: HEVC video + AC-3 audio, and the hvcC round-trips.
assert_eq!(rd.info().streams.len(), 2);
assert!(
matches!(rd.info().streams[0], DiscStreamE::Video(ref v) if v.codec == Codec::Hevc)
);
assert!(matches!(rd.info().streams[1], DiscStreamE::Audio(ref a) if a.codec == Codec::Ac3));
assert_eq!(rd.codec_private(0), Some(hvcc));
// Read all frames back; match them to what we wrote by (track, size).
let mut got = Vec::new();
while let Some(f) = rd.read().unwrap() {
got.push((f.track, f.data.len(), f.keyframe));
}
assert_eq!(got.len(), 4, "4 samples round-trip");
let vids: Vec<_> = got.iter().filter(|(t, _, _)| *t == 0).collect();
let auds: Vec<_> = got.iter().filter(|(t, _, _)| *t == 1).collect();
assert_eq!(vids.len(), 2);
assert_eq!(auds.len(), 2);
assert_eq!(vids[0].1, 700, "first video sample size");
assert_eq!(vids[1].1, 350, "second video sample size");
assert!(vids[0].2, "first video frame is a keyframe");
assert_eq!(auds[0].1, ac3.len());
}
#[test]
fn stts_and_ctts_expand() {
// stts: 3 samples × 1001 ticks.
let mut stts = Vec::new();
stts.extend_from_slice(&[0, 0, 0, 0]); // version+flags
stts.extend_from_slice(&1u32.to_be_bytes()); // entry_count
stts.extend_from_slice(&3u32.to_be_bytes());
stts.extend_from_slice(&1001u32.to_be_bytes());
assert_eq!(parse_stts(&stts), vec![1001, 1001, 1001]);
}
}
+220 -24
View File
@@ -45,6 +45,10 @@ pub enum StreamUrl {
M2ts { path: PathBuf },
/// Matroska container file.
Mkv { path: PathBuf },
/// Progressive MP4 (ISO-BMFF) mux output (`mp4://`). Like `mkv://` but writes
/// a single self-contained `.mp4` (ftyp+mdat+moov). Compatibility export —
/// carries only MP4-mappable codecs; see `mux::mp4`.
Mp4 { path: PathBuf },
/// Network stream (host:port).
Network { addr: String },
/// Standard I/O (stdin/stdout).
@@ -62,10 +66,31 @@ pub enum StreamUrl {
/// chapters + delay metadata). Like `dir://` it targets a directory; the
/// CLI constructs the `DemuxSink` with full options before the mux loop.
Demux { dir: PathBuf },
/// Video-only per-track output directory (`video://`) — a `demux://`
/// restricted to video tracks (native elementary streams: `.hevc`, `.h264`,
/// `.vc1`, `.m2v`, …). One file per video track; no audio/subtitles.
Video { dir: PathBuf },
/// Audio-only per-track output directory (`audio://`) — a `demux://`
/// restricted to audio tracks (native containers: `.thd`, `.dts`, `.ac3`,
/// `.eac3`, `.pcm`, …). One file per audio track; no video/subtitles.
Audio { dir: PathBuf },
/// Subtitle-only per-track output directory (`sub://`) — a `demux://`
/// restricted to subtitle tracks (PGS `.sup`, VobSub `.idx`+`.sub`, text
/// `.srt`). One file per subtitle track.
Sub { dir: PathBuf },
/// freemkv native per-picture video index (`fvi://`). A write-only PES sink
/// that emits one JSON-Lines record per coded picture of the title's primary
/// video track to a `.fvi` file (normative spec `docs/FVI_FORMAT.md`).
Fvi { path: PathBuf },
/// Chapter-marker export (`chapters://`). A write-only sink that ignores the
/// PES stream and writes the title's chapter points to a single file, format
/// chosen by the output extension: `.xml` (Matroska, default), `.txt` (OGM),
/// `.vtt` (WebVTT).
Chapters { path: PathBuf },
/// Structured title/stream/chapter metadata (`json://`). A write-only sink
/// that ignores the PES stream and writes the selected title's model as one
/// JSON document — machine-readable `info` for one title.
Json { path: PathBuf },
/// Unrecognized URL.
Unknown { raw: String },
}
@@ -77,13 +102,19 @@ impl StreamUrl {
StreamUrl::Disc { .. } => "disc",
StreamUrl::M2ts { .. } => "m2ts",
StreamUrl::Mkv { .. } => "mkv",
StreamUrl::Mp4 { .. } => "mp4",
StreamUrl::Network { .. } => "network",
StreamUrl::Stdio => "stdio",
StreamUrl::Iso { .. } => "iso",
StreamUrl::Dir { .. } => "dir",
StreamUrl::Null => "null",
StreamUrl::Demux { .. } => "demux",
StreamUrl::Video { .. } => "video",
StreamUrl::Audio { .. } => "audio",
StreamUrl::Sub { .. } => "sub",
StreamUrl::Fvi { .. } => "fvi",
StreamUrl::Chapters { .. } => "chapters",
StreamUrl::Json { .. } => "json",
StreamUrl::Unknown { .. } => "unknown",
}
}
@@ -95,10 +126,16 @@ impl StreamUrl {
StreamUrl::Disc { device: None } => "",
StreamUrl::M2ts { path }
| StreamUrl::Mkv { path }
| StreamUrl::Mp4 { path }
| StreamUrl::Iso { path }
| StreamUrl::Dir { path }
| StreamUrl::Demux { dir: path }
| StreamUrl::Fvi { path } => path.to_str().unwrap_or(""),
| StreamUrl::Video { dir: path }
| StreamUrl::Audio { dir: path }
| StreamUrl::Sub { dir: path }
| StreamUrl::Fvi { path }
| StreamUrl::Chapters { path }
| StreamUrl::Json { path } => path.to_str().unwrap_or(""),
StreamUrl::Network { addr } => addr,
StreamUrl::Stdio | StreamUrl::Null => "",
StreamUrl::Unknown { raw } => raw,
@@ -138,6 +175,11 @@ pub fn parse_url(url: &str) -> StreamUrl {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("mp4://") {
return StreamUrl::Mp4 {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("network://") {
return StreamUrl::Network {
addr: rest.to_string(),
@@ -171,6 +213,31 @@ pub fn parse_url(url: &str) -> StreamUrl {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("video://") {
return StreamUrl::Video {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("audio://") {
return StreamUrl::Audio {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("sub://") {
return StreamUrl::Sub {
dir: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("chapters://") {
return StreamUrl::Chapters {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("json://") {
return StreamUrl::Json {
path: PathBuf::from(rest),
};
}
if let Some(rest) = url.strip_prefix("fvi://") {
return StreamUrl::Fvi {
path: PathBuf::from(rest),
@@ -336,26 +403,29 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
}
.into());
}
// Per-title key resolution. For a multi-VTS CSS DVD the scan's
// single cracked key only descrambles its own VTS; re-crack from
// the chosen title's extents if it lives elsewhere. A fresh reader
// avoids disturbing the mux reader below. 64 sectors is a
// file-safe batch for an ISO. AACS / single-VTS paths are
// unchanged (decrypt_keys_for_title short-circuits to decrypt_keys).
let (keys, title_is_clear) =
match crate::io::file_sector_source::FileSectorSource::open(path) {
Ok(mut crack_reader) => disc.decrypt_keys_for_title(idx, &mut crack_reader, 64),
Err(_) => (disc.decrypt_keys(), false),
};
// Per-title decrypt gate (parallel to the disc-wide gate above): on
// a multi-VTS CSS disc, the per-title re-crack may return `None` when
// the chosen title's VTS could not be re-cracked. Muxing that would
// emit scrambled ciphertext verbatim, so fail loudly here — EXCEPT
// when the title proved genuinely clear (`title_is_clear`), an
// unencrypted stub on an otherwise-CSS disc that needs no key. That
// case must NOT raise a false E7023.
disc.ensure_title_decryptable(opts.raw, &keys, title_is_clear)
.map_err(|e| -> io::Error { e.into() })?;
// Per-title key resolution. DVD CSS is resolved at exactly ONE site —
// `build_iso_pipeline`'s per-title crack (below), which decrypts a
// crackable title, passes a genuinely-clear one through, and
// hard-fails an uncrackable one with CssKeyMissing. So for a DVD we do
// NOT pre-crack here: pass `None` and let the pipeline own it.
// Pre-cracking would re-open the ISO and re-scan every clear title
// (`decrypt_keys_for_title` → None → the pipeline re-cracks anyway).
// AACS / unencrypted resolve from `decrypt_keys()` with NO read; `--raw`
// (any format) is deliberate ciphertext passthrough — also `None`.
let is_dvd = disc.format == crate::disc::DiscFormat::Dvd;
let (keys, title_is_clear) = if opts.raw || is_dvd {
(crate::decrypt::DecryptKeys::None, false)
} else {
(disc.decrypt_keys(), false)
};
// Decrypt gate for the AACS / non-DVD path: a None key means no usable
// disc key, which would mux scrambled ciphertext verbatim — fail loudly
// (NoDiscKey). The DVD path is gated inside `build_iso_pipeline` (its
// CSS hard-fail), and `--raw` passes.
if !is_dvd {
disc.ensure_title_decryptable(opts.raw, &keys, title_is_clear)
.map_err(|e| -> io::Error { e.into() })?;
}
// FMTS (AACS 2.1) forensic segments are sourced + fail-loud-checked
// downstream by `resolve_mux_key_map`/`resolve_fmts_key_map`, which hold
// the key-fetch closure and can actually attempt resolution. (An older
@@ -418,6 +488,7 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
effective_keys,
ISO_MUX_BATCH_SECTORS,
format,
opts.raw,
None,
None,
fetch,
@@ -446,8 +517,16 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
// PES source. Mirror `null://` → write-only.
StreamUrl::Dir { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Null => Err(crate::error::Error::StreamWriteOnly.into()),
// `mp4://` as a source: demux a progressive MP4 back into PES frames, so
// `mp4://` flows to every sink (mkv://, audio://, json://, …).
StreamUrl::Mp4 { ref path } => Ok(Box::new(super::mp4::Mp4Reader::open(path)?)),
// `demux://` is an output-only sink (per-track ES files); never a source.
StreamUrl::Demux { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Demux { .. }
| StreamUrl::Video { .. }
| StreamUrl::Audio { .. }
| StreamUrl::Sub { .. }
| StreamUrl::Chapters { .. }
| StreamUrl::Json { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
// `fvi://` is an output-only sink (per-picture video index); never a source.
StreamUrl::Fvi { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Unknown { ref raw } => {
@@ -479,6 +558,18 @@ pub fn output(
));
Ok(Box::new(MkvStream::create(writer, title, Some(path))?))
}
StreamUrl::Mp4 { ref path } => {
validate_file_path(path, "mp4")?;
// Bounded-cache writeback (like mkv://) so a UHD-scale mux to slow /
// network-attached staging doesn't hit the dirty-page burst
// pathology; the mdat backpatch is an ordinary seek WritebackFile
// handles. BufWriter coalesces the many small moov box-header writes.
let writer = std::io::BufWriter::with_capacity(
IO_BUF_SIZE,
crate::io::WritebackFile::create_with_size_hint(path, title.size_bytes)?,
);
Ok(Box::new(super::mp4::Mp4Sink::create(writer, title)?))
}
StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?;
let writer = std::io::BufWriter::with_capacity(
@@ -526,6 +617,31 @@ pub fn output(
dir, title, &opts,
)?))
}
// `video://`, `audio://`, and `sub://` are `demux://` restricted to one
// track class — video as native elementary streams, audio in native
// containers, or subtitles as `.sup`/`.idx+.sub`/`.srt`. No chapters
// sidecar (that's a `demux://` / `chapters://` concern).
StreamUrl::Video { ref dir }
| StreamUrl::Audio { ref dir }
| StreamUrl::Sub { ref dir } => {
let (scheme, kind) = match parsed {
StreamUrl::Video { .. } => ("video", super::demux_sink::TrackKind::Video),
StreamUrl::Audio { .. } => ("audio", super::demux_sink::TrackKind::Audio),
_ => ("sub", super::demux_sink::TrackKind::Subtitle),
};
validate_file_path(dir, scheme)?;
let mut opts = super::demux_sink::DemuxOptions {
kind_filter: Some(kind),
export_chapters: false,
..Default::default()
};
if !title.playlist.is_empty() {
opts.base = title.playlist.clone();
}
Ok(Box::new(super::demux_sink::DemuxSink::create(
dir, title, &opts,
)?))
}
// `fvi://` writes the per-picture video index (`docs/FVI_FORMAT.md`).
// The bare `output()` arm records the resolver path as the provenance
// `source.path` and defaults the title index to 0 (the resolver carries
@@ -539,6 +655,18 @@ pub fn output(
0,
)?))
}
// `chapters://` and `json://` write the title metadata at construction and
// ignore the PES stream (see `meta_sink`).
StreamUrl::Chapters { ref path } => {
validate_file_path(path, "chapters")?;
Ok(Box::new(super::meta_sink::ChaptersSink::create(
path, title,
)?))
}
StreamUrl::Json { ref path } => {
validate_file_path(path, "json")?;
Ok(Box::new(super::meta_sink::JsonSink::create(path, title)?))
}
StreamUrl::Unknown { ref raw } => {
Err(crate::error::Error::StreamUrlInvalid { url: raw.clone() }.into())
}
@@ -996,13 +1124,17 @@ pub fn resolve_mux_key_map(
/// - `batch_sectors`: read batch size in logical (2048-byte) sectors — a
/// throughput/latency tuning knob, not a correctness parameter.
/// - `format`: container format (`BdTs` → TS demuxer, `MpegPs` → PS demuxer).
/// - `raw`: ciphertext passthrough. When `true`, the per-title CSS crack
/// (`resolve_dvd_title_key`) is skipped entirely — no key is resolved and a
/// scrambled title is neither descrambled nor hard-failed.
/// - `halt`: cooperative cancel token (not a timeout); when cancelled the
/// pipeline stops at the next boundary. `None` disables cancellation.
/// pipeline stops at the next boundary (and the CSS crack surfaces `Halted`).
/// `None` disables cancellation.
/// - `event_fn`: optional progress/event callback invoked by the prefetcher.
/// - `fetch`: optional key source used UP FRONT by [`resolve_mux_key_map`] to
/// secure any CPS-unit key the pool is missing. Not a per-unit mux-time
/// callback: the map decides the key for every LBA before the read loop starts.
// Eight reader/title/keys/tuning/callback params is inherent to the mux entry
// Nine reader/title/keys/tuning/callback params is inherent to the mux entry
// point; grouping them into a struct would only move the same fields around.
#[allow(clippy::too_many_arguments)]
pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
@@ -1011,11 +1143,27 @@ pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
mut keys: crate::decrypt::DecryptKeys,
batch_sectors: u16,
format: ContentFormat,
raw: bool,
halt: Option<crate::halt::Halt>,
event_fn: Option<crate::sector::prefetched::EventFn>,
fetch: Option<crate::sector::KeyFetch>,
) -> io::Result<PipelinedPesStream> {
let extents = title.extents.clone();
// CSS (DVD) key resolution — the shared per-title step (also used by the
// live-drive single-pass `DiscStream`). A `None`/MPEG-PS title cracks its own
// key from the reader in playback order; AACS `.evo` (also MPEG-PS) arrives as
// `Aacs` and is untouched; a clear DVD stays `None`; `raw` skips it entirely.
// Without this a detection-miss CSS DVD would mux scrambled sectors as corrupt
// video. `halt` lets /api/stop interrupt the crack scan.
crate::css::resolve_dvd_title_key(
&mut reader,
&extents,
&mut keys,
batch_sectors,
format,
raw,
halt.as_ref(),
)?;
// Unit alignment is an AACS concept: AACS decrypts whole 6144-byte (3-sector)
// units, so the producer must hand the decrypt step 3-sector-aligned batches.
// CSS (DVD) and unencrypted content decrypt per 2048-byte sector — forcing
@@ -1578,6 +1726,7 @@ mod tests {
DecryptKeys::None,
8192,
ContentFormat::BdTs,
false,
None,
None,
None,
@@ -1619,6 +1768,7 @@ mod tests {
DecryptKeys::None,
8192,
ContentFormat::BdTs,
false,
None,
None,
None,
@@ -1671,10 +1821,56 @@ mod tests {
DecryptKeys::None,
0,
ContentFormat::BdTs,
false,
None,
None,
None,
);
assert!(res.is_err(), "zero batch_sectors must be rejected");
}
/// REGRESSION (autorip production corruption): `build_iso_pipeline` for a DVD
/// (MPEG-PS) with `None` keys — what autorip's mux passes on a detection-miss
/// DVD (`disc.decrypt_keys()` == None) — must resolve the CSS key from the
/// reader itself. A scrambled-but-uncrackable title must HARD-FAIL, never
/// build a passthrough pipeline that muxes the scrambled sectors as corrupt
/// video. Before this fix, autorip handed None straight through and the mux
/// wrote garbage at exit 0.
#[test]
fn build_iso_pipeline_dvd_none_keys_scrambled_hard_fails() {
// One CSS-scrambled, crib-less (uncrackable) MPEG-PS sector.
let key = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let mut sec = vec![0u8; 2048];
sec[0..4].copy_from_slice(&crate::css::PACK_START);
for (i, b) in sec.iter_mut().enumerate().take(0x80).skip(4) {
*b = (i as u8).wrapping_mul(7).wrapping_add(1); // non-repeating → no crib
}
sec[0x14] = 0x10; // scramble flag
for (i, b) in sec.iter_mut().enumerate().skip(0x80) {
*b = (i as u8) ^ 0x3C;
}
crate::css::lfsr::scramble_sector(&key, &mut sec);
let mut title = aac_audio_title(0x1100);
title.extents = vec![Extent {
start_lba: 0,
sector_count: 1,
}];
let res = build_iso_pipeline(
MemSource { data: sec },
title,
DecryptKeys::None,
8192,
ContentFormat::MpegPs,
false,
None,
None,
None,
);
assert!(
res.is_err(),
"a scrambled DVD title with no key must hard-fail, not build a scrambled-passthrough pipeline"
);
}
}
+1
View File
@@ -175,6 +175,7 @@ fn run_to_fvi(image: Vec<u8>, title: DiscTitle, path: &std::path::Path) {
DecryptKeys::None,
3, // 3-sector (one AACS unit) batches → one source stamp per GOP region
ContentFormat::MpegPs,
false,
None,
None,
None,
+20 -2
View File
@@ -136,7 +136,16 @@ fn test_bytes_read_emitted_during_disc_copy() {
let title = synthetic_title(64);
let keys = libfreemkv::DecryptKeys::None;
let mut stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
let mut stream = DiscStream::new(
Box::new(reader),
title,
keys,
60,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
let count = Arc::new(AtomicU64::new(0));
let count_cb = count.clone();
@@ -286,7 +295,16 @@ fn test_drop_impls_do_not_panic_or_block() {
let reader = ZeroSectorReader::new(64);
let title = synthetic_title(64);
let keys = libfreemkv::DecryptKeys::None;
let stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
let stream = DiscStream::new(
Box::new(reader),
title,
keys,
60,
ContentFormat::BdTs,
false,
None,
)
.unwrap();
// Drop on a worker thread; main thread enforces the timeout.
let handle = std::thread::spawn(move || {